# TakagiTaupin - Calculate the Takagi-Taupin curves

This file demonstrates the usage of TakagiTaupin that is the main class of the package. TakagiTaupin solves the TT-equations for deformed crystal using the information given to it in TTcrystal and TTscan. For detailed instruction on their use, please refer to the respective documentation and example Jupyter notebooks.

```python
import sys
import os.path

import numpy as np
import matplotlib.pyplot as plt

sys.path.insert(1, '..')
from pyTTE import TakagiTaupin, TTcrystal, TTscan, Quantity
```

### First example: rocking curve of a perfect Si crystal in the Bragg case

Let us start with a simple example of calculating the rocking curve of the (111) reflection of a 1 mm thick Si perfect, single crystal. First we define the crystal using TTcrystal class:

```python
xtal = TTcrystal(crystal='Si', hkl=[1,1,1], thickness=Quantity(1,'mm'))
```

The given keywords `crystal`, `hkl`, and `thickness` are the required minimum amount of parameters to define a crystal. To avoid conversion errors, all the quantities with physical units are handled in _pyTTE_ using the Quantity class which supports most units commonly used in the context of X-ray diffraction.

Full information of a TTcrystal instance can be seen by passing it to the `print` function:

```python
print(xtal)
```

    Crystal: Si
    Crystallographic parameters:
        a = 0.543069 nm,  b = 0.543069 nm,  c = 0.543069 nm
        alpha = 90.0 deg,  beta = 90.0 nm,  gamma = 90.0 deg
    Direct primitive vectors (before rotations, in nm):
        a1 = [0.5431 0.     0.    ]
        a2 = [0.     0.5431 0.    ]
        a3 = [0.     0.     0.5431]
    Reciprocal primitive vectors (before rotations, in 1/nm):
        b1 = [11.5698 -0.     -0.    ]
        b2 = [ 0.     11.5698 -0.    ]
        b3 = [ 0.      0.     11.5698]

    Reflection: [1, 1, 1]
    Asymmetry angle: 0 deg
    In-plane rotation angle: 0 deg
    Crystal directions parallel to the Cartesian axes (after rotations):
        x || [ 1.     -0.2679 -0.7321]
        y || [-0.2679  1.     -0.7321]
        z || [1. 1. 1.]

    Crystal thickness: 1 mm
    Debye-Waller factor: 1.0

    Deformation model: anisotropic toroidal, fixed shape (built-in)
    Meridional bending radius: inf m
    Sagittal bending radius: inf m
    Material elastic isotropy: anisotropic
    Compliance matrix S (with rotations applied):
    [[ 0.0059 -0.0015 -0.001  -0.0012  0.0012  0.    ]
     [-0.0015  0.0059 -0.001   0.0012 -0.0012  0.    ]
     [-0.001  -0.001   0.0053  0.      0.     -0.    ]
     [-0.0012  0.0012  0.      0.0173 -0.     -0.0024]
     [ 0.0012 -0.0012  0.      0.      0.0173 -0.0024]
     [ 0.      0.     -0.     -0.0024 -0.0024  0.0149]] GPa^-1

In addition to the crystal parameters, we need to define the scan using the TTscan class. Let's fix the photon energy of the incident beam to 5 keV, use $\sigma$-polarization and scan the rocking angle from -50 to 150 µrad in 150 steps. In _pyTTE_, all scans are performed relative to the values fulfilling the kinematical Bragg condition $\lambda = 2 d \sin \theta$.

```python
scan = TTscan(constant = Quantity(5,'keV'), scan = Quantity(np.linspace(-50,150,150),'urad'), polarization = 'sigma')
print(scan)
```

    Scan type     : angle
    Scan constant : 5 keV
    Polarization  : sigma
    Scan points   : 150
    Scan range    : manual from -50.0 to 150.0 urad

    Output type                : photon flux
    Integrator                 : zvode_bdf
    (Minimum) integration step : 1e-10 um
    Alternative starting depth : None

The rocking curve is now calculated by initializing the TakagiTaupin instance with the crystal and scan parameter instances, and calling `.run()`.

```python
tt = TakagiTaupin(xtal, scan)

scan_vector, reflectivity, transmission = tt.run()
```

    Solving the 1D Takagi-Taupin equation for Si(1,1,1) reflection
    ---------------------------------------------------------------

    The direction of diffraction out of the crystal -> Bragg case
    Solving for sigma-polarization
    Asymmetry angle : 0.0 deg
    Wavelength      : 0.247968386 nm
    Energy          : 5.0 keV
    Bragg angle     : 23.292879649001986 deg
    Incidence angle : 23.292879649001986 deg
    Exit angle      : 23.292879649001986 deg

    Structure factors
    F0   : (115.0489207774186+6.498559984990174j)
    Fh   : (46.915823878813626-40.417263893823474j)
    Fb   : (40.417263893823446+46.91582387881365j)

    Susceptibilities
    chi0 : (-3.961773705462428e-05+2.237815348282492e-06j)
    chih : (-1.615572542160448e-05-1.3917910073321994e-05j)
    chib : (-1.3917910073321984e-05+1.6155725421604487e-05j)

    (Mean F and chi values for energy scan)

    Transmission in the Bragg case not implemented!

    Calculating the TT-curve using 4 cores.
    Solving...0%
    Done.

(Note that Jupyter does not update the solving process in the notebook but in the terminal/command prompt.)

`.run()` returns the solution but also stores it in `.solution` with all the relevant information and settings pertaining to it. The stored solution can be quickly visualized by calling `.plot()` and saved as an ASCII file with `.save(filename)`

```python
tt.plot()
```

For reference, the same curve was calculated using _XCRYSTAL_ in _XOP_ 2.4 (http://dx.doi.org/10.1117/12.560903) with the EPDL97 anomalous scattering cross-sections, that are also used by _xraylib_.

```python
ref = np.loadtxt('reference_spectra/xop_Si_111_bragg.dat')

plt.plot(scan_vector,reflectivity)
plt.plot(ref[:,0],ref[:,1])
plt.legend(['pyTTE','XCRYSTAL'])
plt.ylabel('Reflectivity')
plt.xlabel('Angle (urad)')
plt.show()
```

The result is practically identical with the one calculated with _pyTTE_. Nearly insignificant deviations are probably explained by slight differences in structure factors:

_PyTTE_ output:
```
F0   :  (115.0489207774186+6.498559984990174j)
Fh   :  (46.915823878813626-40.417263893823474j)
Fb   :  (40.417263893823446+46.91582387881365j)
```

_Crystal Parameters_ of `XCRYSTAL`:
```
 Structure factor F(0,0,0) =  (  115.01669312000000     ,  6.5506672799999999     )
 Structure factor FH =  (  47.595073721453730     , -41.044406441453731     )
 Structure factor FH_BAR =  (  41.044406441453731     ,  47.595073721453730     )
```

### Second example: The Laue case and automatic scan limits

_PyTTE_ does not require explicit differentiation between the Bragg case (reflection geometry) and the Laue case (transmission geometry) since the diffraction geometry is determined automatically from the propagation direction of the diffracted beam. This can be controlled via the asymmetry angle. For a symmetric Laue case, asymmetry angle is set to 90 degrees:

```python
xtal = TTcrystal(crystal='Ge', hkl=[4,0,0], thickness=Quantity(100,'um'), asymmetry = Quantity(90,'deg'))
```

In the previous example we defined the scan points manually but it can be inconvenient to iterate good values for the scan limits, especially with bent crystals. To make the task easier, _pyTTE_ can try to determine the limits automatically by giving `TTscan` the number of scan points instead of a scan vector.

```python
scan = TTscan(constant = Quantity(7,'keV'), scan = 150, polarization = 'sigma')
print(scan)
```

    Scan type     : angle
    Scan constant : 7 keV
    Polarization  : sigma
    Scan points   : 150
    Scan range    : automatic

    Output type                : photon flux
    Integrator                 : zvode_bdf
    (Minimum) integration step : 1e-10 um
    Alternative starting depth : None

```python
tt = TakagiTaupin(xtal, scan)

scan_vector, diffracted, transmission = tt.run()
tt.plot()
```

Also in this case, the correspondence with _XCRYSTAL_ is practically perfect.

```python
ref = np.loadtxt('reference_spectra/xop_Ge_400_laue_transmission.dat')
ref2 = np.loadtxt('reference_spectra/xop_Ge_400_laue_diffraction.dat')

plt.plot(scan_vector,transmission)
plt.plot(scan_vector,diffracted)
plt.plot(ref[:,0],ref[:,1])
plt.plot(ref2[:,0],ref2[:,1])
plt.legend(['pyTTE transmission', 'pyTTE diffraction', 'XCRYSTAL transmission', 'XCRYSTAL diffraction'])
plt.ylabel('Reflectivity')
plt.xlabel('Angle (urad)')
plt.show()
```

### Third example: bent crystals and energy scans

Deformation can be introduced by passing the meridional and sagittal bending radii (`Rx` and `Ry`, respectively) to TTcrystal. In the case of spherical bending, a single keyword `R` can be used.

```python
ttx = TTcrystal(crystal='Si', hkl=[6,6,0], thickness=Quantity(150,'um'), R = Quantity(100,'cm'), asymmetry = Quantity(5, 'deg'))
tts = TTscan(scan = 200, constant = Quantity(75, 'deg'), polarization = 'pi')

tt = TakagiTaupin(ttx,tts)

tt.run()
tt.plot()
```

By default _pyTTE_ attempts to calculate the toroidal/spherical/cylindrical bending using elastic constants from a built-in compliance matrix $S$ (Source: CRC Handbook of Chemistry and Physics, 82nd edition).

For anisotropic crystals, _PyTTE_ has two different deformation models, selected with the TTcrystal keyword `fix_to_axis`. The default value is `'shape'` which assumes that the main axes of curvature are along the coordinate axes $x$ and $y$, and are given by `Rx` and `Ry`. This corresponds to the situation where the crystal is forced to a specific shape _e.g._ by the substrate. The other option is `torques` which assumes that the crystal is bent by two orthogonal torques acting about the aforemention coordinate axes. This applies, for example, when a free standing crystal slab is bent by its ends. In the general case these models are not equal due to non-diagonal elements of $S$. See TTcrystal.ipynb, help(TTcrystal) or the technical docs for more information.

Built-in $S$:s are available only for a handful of crystals commonly encountered in X-ray optics. In other cases user needs to define $S$ manually with the keyword `S`. Alternatively, an isotropic deformation model can be used by giving Poisson's ratio $\nu$ with `nu`. Also Young's modulus $E$ is required but its value is not important here as it is not used in by deformation model. `fix_to_axes` is neglected as the two anisotropic models reduce to the same isotropic model.

```python
ttx = TTcrystal(crystal='Si', hkl=[6,6,0], thickness=Quantity(150,'um'), R = Quantity(100,'cm'), asymmetry = Quantity(5, 'deg'), nu = 0.22, E = Quantity(150, 'GPa'))
tts = TTscan(scan = 200, constant = Quantity(75, 'deg'), polarization = 'pi')

tt = TakagiTaupin(ttx,tts)

tt.run()
tt.plot()
```

Also custom deformation fields can be defined by providing the Jacobian of the displacement vector $\mathbf{u}$ as a function of $x$ and $z$:

$\begin{equation}
J(x,z) = \left[\begin{matrix}
\frac{\partial u_x}{\partial x}(x,z) & \frac{\partial u_x}{\partial z}(x,z) \\
\frac{\partial u_z}{\partial x}(x,z) & \frac{\partial u_z}{\partial z}(x,z)
\end{matrix}\right]
\end{equation}$

For reference, we do this for the isotropic bending below. Note that the lengths are exceptionally required to be in micrometers.

```python
def jacobian(x,z):
    nu = 0.22
    R_bend_in_um = 1e6
    thickness_in_um = 150

    ux_x = -(z+0.5*thickness_in_um)/R_bend_in_um
    ux_z = -x/R_bend_in_um

    uz_x = x/R_bend_in_um
    uz_z = 2*nu/(1-nu)/R_bend_in_um*(z+0.5*thickness_in_um)

    return [[ux_x,ux_z],[uz_x,uz_z]]

ttx = TTcrystal(crystal='Si', hkl=[6,6,0], thickness=Quantity(150,'um'), asymmetry = Quantity(5, 'deg'))
tts = TTscan(scan = 200, constant = Quantity(75, 'deg'), polarization = 'pi')

#The custom Jacobian is given to TTcrystal after the initialization
ttx.set_deformation(jacobian = jacobian)

tt = TakagiTaupin(ttx,tts)

tt.run()
tt.plot()
```

### Fourth example: Initialization from file

Parameters can also be defined in two input files for `TTcrystal` and `TTscan` which can be passed directly to `TakagiTaupin`.

```python
'''
Contents of TTcrystal_init.inp:

crystal LiF
hkl 2 0 0
thickness 200 um
asymmetry 2.5 deg
Rx 1 m

Contents of TTscan_init.inp:

constant 8 keV
scan -100 25 150 arcsec
polarization sigma
solver zvode_bdf
integration_step 1 nm
'''

tt = TakagiTaupin('TTcrystal_init.inp','TTscan_init.inp')

print(tt)

tt.run()
tt.plot()
plt.show()
```


### This notebook computes the joint likelihood in 5 dimensions for the parameters of interest (see below). See inference_demo for more details about basic usage

To run this notebook, download the pre-computed likelihoods for each lens from dropbox, put the .zip file into lenslikelihood/precomputed_likelihoods/ and unpack it: https://www.dropbox.com/s/gbj4d58kg6cax4e/logprior_likelihoods.zip?dl=0

```python
from lenslikelihood.measurements import flux_measurements, flux_measurement_uncertainties, all_lens_names, all_param_ranges_version2
import numpy as np

# Note that the syntax for the uncertainties is
# {'lens_name': (1-sigma-uncertainty, reference_index, uncertainty_in_ratio)}
# where reference_index is the reference image with which to compute the flux ratio, and uncertainty_in_ratio specifies
# whether the measurement uncertainty refers to the flux or the flux ratio
param_names = ['LOS_normalization', 'beta', 'log10c0', 'delta_power_law_index', 'sigma_sub']
param_ranges = [all_param_ranges_version2[name] for name in param_names]
for name in all_lens_names:
    print(name)
    print('fluxes/flux ratios measured: ', flux_measurements[name])
    print('uncertainties: ', flux_measurement_uncertainties[name])
    print('\n')
```

    WGDJ0405
    fluxes/flux ratios measured:  [0.8  0.52 1.   0.94]
    uncertainties:  ([0.04, 0.061538461538461535, 0.024, 0.03418803418803419], 0, False)

    HE0435
    fluxes/flux ratios measured:  [0.96  0.976 1.    0.65 ]
    uncertainties:  ([0.05, 0.049, 0.048, 0.056], 0, False)

    WGD2038
    fluxes/flux ratios measured:  [0.86 1.   0.79 0.4 ]
    uncertainties:  ([0.01, 0.01724137931034483, 0.021739130434782608, 0.021739130434782608], 0, False)

    B1422
    fluxes/flux ratios measured:  [0.88  1.    0.474 0.025]
    uncertainties:  ([0.011363636363636364, 0.01, 0.012765957446808512, None], 0, False)

    WFI2033
    fluxes/flux ratios measured:  [1.   0.65 0.5  0.53]
    uncertainties:  ([0.03, 0.046875, 0.04, 0.03773584905660377], 0, False)

    PSJ1606
    fluxes/flux ratios measured:  [1.   1.   0.59 0.79]
    uncertainties:  ([0.03, 0.03, 0.03333333333333333, 0.02564102564102564], 0, False)

    WFI2026
    fluxes/flux ratios measured:  [1.   0.75 0.31 0.28]
    uncertainties:  ([0.02, 0.02666666666666667, 0.06451612903225806, 0.03571428571428571], 0, False)

    RXJ0911
    fluxes/flux ratios measured:  [0.56 1.   0.53 0.24]
    uncertainties:  ([0.07142857142857142, 0.05, 0.07547169811320754, 0.16666666666666669], 0, False)

    RXJ1131
    fluxes/flux ratios measured:  [1.   0.61 0.73 0.12]
    uncertainties:  ([[0.012345679012345678, 0.024691358024691357], [0.10084033613445378, 0.025210084033613446], None], 1, True)

    MG0414
    fluxes/flux ratios measured:  [1.   0.83 0.36 0.16]
    uncertainties:  ([0.06024096385542169, 0.11111111111111112, 0.11764705882352941], 0, True)

    PG1115
    fluxes/flux ratios measured:  [1.   0.93 0.16 0.21]
    uncertainties:  ([0.06451612903225806, 0.43750000000000006, 0.1904761904761905], 0, True)

### Models implemented for the halo mass function and concentration-mass relation

The full set of hyper-parameters we're interested in constraining are defined by the parameterizations of the halo mass function and concentration-mass relation. They are $\Sigma_{\rm{sub}}$, $\delta_{\rm{LOS}}$, $\Delta \alpha$, $q$, $c_8$, and $\beta$. The first four define to the subhalo and field halo mass functions, and the last two define the concentration-mass relation.

The field halo mass function is parameterized as
\begin{equation}
\frac{dN_{\rm{LOS}}}{dm dV} = \delta_{\rm{LOS}} \left(1+\xi_{\rm{2halo}}\right) \left(\frac{m}{10^8}\right)^{\Delta \alpha} \ \frac{dN_{\rm{ShethTormen}}}{dm dV}
\end{equation}
where $\delta_{\rm{LOS}}$ scales the overall normalization, and $\Delta \alpha$ parameterizes deviations from the logarithmic slope predicted by CDM around $10^8 M_{\odot}$.

The subhalo mass function is parameterized as
\begin{equation}
\frac{dN_{\rm{sub}}}{dm dA} \sim \Sigma_{\rm{sub}} \ \left(\frac{m}{10^8}\right)^{\alpha + q \Delta \alpha}
\end{equation}
where $\Sigma_{\rm{sub}}$ is the normalization, $\alpha$ is the logarithmic slope predicted by CDM, $\Delta \alpha$ parameterizes deviations from the value predicted by CDM, and $q$ controls the coupling between the line of sight halo mass function slope and the subhalo mass function slope. When $q=1$ the slopes change in the same way, and when $q=0$ the slopes of the subhalo and field halo mass functions are completely decoupled.

The concentration-mass relation is parameterized as

\begin{equation}
c\left(M, z\right) = c_8 \left(1+z\right)^{\zeta} \left(\frac{\nu\left(M, z\right)}{\nu\left(10^8, z\right)}\right)^{-\beta}
\end{equation}
i.e. it is a power-law in the peak height $\nu$ with normalization $c_8$ at $10^8$ and a logarithmic slope $\beta$. The parameter $\zeta$ modifies the redshift evolution and is marginalized over in the sampling.

The parameter names used in the python code have the following correspondence:

1) sigma_sub = $\Sigma_{\rm{sub}}$

2) delta_power_law_index = $\Delta \alpha$

3) c0 = $c_8$

4) beta = $\beta$

5) delta_power_law_index_coupling = $q$

6) LOS_normalization = $\delta_{\rm{LOS}}$

### Example inference on three parameters with a subset of lenses

First load the model samples, define what parameters we want to look at

```python
from trikde.pdfs import DensitySamples, IndepdendentLikelihoods
import os
import pickle

nbins = 20
likelihoods = []

filename_extension = '_joint_logprior'
base_path = './../lenslikelihood/precomputed_likelihoods/'
print(all_lens_names)

lens_list = ['B1422',
            'HE0435',
            'WGD2038',
            'WGDJ0405',
            'WFI2033',
            'PSJ1606',
            'WFI2026',
            'RXJ0911',
            'RXJ1131',
            'MG0414',
            'PG1115']

for lens in lens_list:

    fname = base_path + lens + filename_extension
    print('loading joint likelihoods for lens '+lens+' ...')
    f = open(fname, 'rb')
    single_lens_likelihood = pickle.load(f)
    f.close()
    likelihoods.append(single_lens_likelihood)

likelihood = IndepdendentLikelihoods(likelihoods)
```

    ['WGDJ0405', 'HE0435', 'WGD2038', 'B1422', 'WFI2033', 'PSJ1606', 'WFI2026', 'RXJ0911', 'RXJ1131', 'MG0414', 'PG1115']
    loading joint likelihoods for lens B1422 ...
    loading joint likelihoods for lens HE0435 ...
    loading joint likelihoods for lens WGD2038 ...
    loading joint likelihoods for lens WGDJ0405 ...
    loading joint likelihoods for lens WFI2033 ...
    loading joint likelihoods for lens PSJ1606 ...
    loading joint likelihoods for lens WFI2026 ...
    loading joint likelihoods for lens RXJ0911 ...
    loading joint likelihoods for lens RXJ1131 ...
    loading joint likelihoods for lens MG0414 ...
    loading joint likelihoods for lens PG1115 ...

```python
from trikde.triangleplot import TrianglePlot
triangle_plot = TrianglePlot([likelihood])
triangle_plot.set_cmap('magma')
axes = triangle_plot.make_triplot(filled_contours=False, show_intervals=False, show_contours=False
                                 )
beta = r'$\beta$'
beta_ticks = [0, 3, 6, 9, 12, 15]
c0 = r'$\log_{10}\left(c_8\right)$'
c0_ticks = [0, 1, 2 ,3, 4]
delta_power_law_index = r'$\Delta \alpha$'
dpli_ticks = [-0.6, -0.3, 0., 0.3, 0.6, 0.9]
sigma_sub = r'$\Sigma_{\rm{sub}} \ \left[\rm{kpc^{-2}}\right]$'
sigma_sub_ticks = [0., 0.025, 0.05, 0.075, 0.1]
delta_LOS = r'$\delta_{\rm{LOS}}$'
dlos_ticks = [0., 0.5, 1., 1.5, 2.]
ticksize = 14
labelsize = 18
rotation = 40

axes[5].set_ylabel(beta, fontsize=labelsize)
axes[5].set_yticks(beta_ticks)
axes[5].set_yticklabels(beta_ticks, fontsize=ticksize)

axes[10].set_ylabel(c0, fontsize=labelsize)
axes[10].set_yticks(c0_ticks)
axes[10].set_yticklabels(c0_ticks, fontsize=ticksize)

axes[15].set_ylabel(delta_power_law_index, fontsize=labelsize)
axes[15].set_yticks(dpli_ticks)
axes[15].set_yticklabels(dpli_ticks, fontsize=ticksize)

axes[20].set_ylabel(sigma_sub, fontsize=labelsize)
axes[20].set_yticks(sigma_sub_ticks)
axes[20].set_yticklabels(sigma_sub_ticks, fontsize=ticksize)

axes[20].set_xlabel(delta_LOS, fontsize=labelsize)
axes[20].set_xticks(dlos_ticks)
axes[20].set_xticklabels(dlos_ticks, fontsize=ticksize, rotation=rotation)

axes[21].set_xlabel(beta, fontsize=labelsize)
axes[21].set_xticks(beta_ticks)
axes[21].set_xticklabels(beta_ticks, fontsize=ticksize, rotation=rotation)

axes[22].set_xlabel(c0, fontsize=labelsize)
axes[22].set_xticks(c0_ticks)
axes[22].set_xticklabels(c0_ticks, fontsize=ticksize, rotation=rotation)

axes[23].set_xlabel(delta_power_law_index, fontsize=labelsize)
axes[23].set_xticks(dpli_ticks)
axes[23].set_xticklabels(dpli_ticks, fontsize=ticksize, rotation=rotation)

axes[24].set_xlabel(sigma_sub, fontsize=labelsize)
axes[24].set_xticks(sigma_sub_ticks)
axes[24].set_xticklabels(sigma_sub_ticks, fontsize=ticksize, rotation=rotation)
# can change axis labels
```

```python
from trikde.pdfs import CustomPriorHyperCube

def couple_mass_functions(samples, sigma_sub_theory=0.05, coupling_strength=0.05):

    delta_los_samples = samples[:, 0]
    sigma_sub_samples = samples[:, -1]
    delta_sigma_sub = sigma_sub_samples/sigma_sub_theory
    chi2 = (delta_sigma_sub - delta_los_samples)**2/coupling_strength**2
    return chi2

kwargs_1 = {'sigma_sub_theory': 0.05}
kwargs_2 = {'sigma_sub_theory': 0.025}
prior_on_mass_functions_1 = CustomPriorHyperCube(couple_mass_functions, param_names, param_ranges, nbins, kwargs_1)
prior_on_mass_functions_2 = CustomPriorHyperCube(couple_mass_functions, param_names, param_ranges, nbins, kwargs_2)

likelihoods_coupled_with_prior_mass_functions = likelihoods + [prior_on_mass_functions_1]
likelihood_coupled_with_prior_mass_functions_1 = IndepdendentLikelihoods(likelihoods_coupled_with_prior_mass_functions)
likelihoods_coupled_with_prior_mass_functions = likelihoods + [prior_on_mass_functions_2]
likelihood_coupled_with_prior_mass_functions_2 = IndepdendentLikelihoods(likelihoods_coupled_with_prior_mass_functions)
```

```python
triangle_plot = TrianglePlot([likelihood_coupled_with_prior_mass_functions_1])
triangle_plot.set_cmap('magma')
axes = triangle_plot.make_triplot(filled_contours=False, show_intervals=False, show_contours=False
                                 )
```

```python

```


# Лабораторая работа 4.02
# Определение расстояния между двумя щелями интерфереционным методом

## Выполнил: Коняхин Всеволод Владимирович, M32051

## Краткие теоретические сведения
При наложении когерентных волн происходит перераспределение светового
потока в пространстве, в результате чего в одних местах возникают максимумы, а
в других — минимумы интенсивности. Это явление называется интерференцией
волн.
Существуют два основных типа интерференционных схем: схема,
построенная на основе деления волнового фронта, и схема, построенная на методе
деления амплитуды.
В данной работе рассматривается опыт Юнга, основанный на делении волнового фронта. Снизу приведена схема опыта Юнга.

## Цель работы
Определить расстояния между двумя щелями по полученной от них интерференционной картине.

## Рабочие формулы и исходные данные

### Формулы
$\Delta x = \frac{\lambda}{d}L$
$L = x_{Э} - x_{О}$

### Исходные данные
$\lambda = (632.82 \pm 0.01) \space нм$

### Схема установки

## Результаты измерений и расчеты

```python
import sympy
import scipy
import numpy as np
import pandas as pd
from scipy.signal import argrelextrema
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"] = (10,5)
%matplotlib inline
```

### Координаты последовательных минимумов интенсивности при разных положениях объекта 33

```python
df33 = pd.DataFrame({'X объекта, мм': [166, 266, 366, 466, 566],
                     '$x_1$, мм' : [-25, -25.5, -25.5, -26, -26.5],
                     '$x_2$, мм' : [-22.5, -23, -22.5, -24, -24.5],
                     '$x_3$, мм' : [-18.5, -19.5, -19.5, -21.5, -22],
                     '$x_4$, мм' : [-15, -16, -17, -19, -20],
                     '$x_5$, мм' : [-11.5, -13, -14, -17, -18],
                     '$x_6$, мм' : [-8.5, -10.5, -11.5, -14, -16],
                     '$x_7$, мм' : [-4.5, -6.5, -9, -11.5, -14],
                     '$x_8$, мм' : [-0.5, -3, -5.5, -8.5, -12],
                     '$x_9$, мм' : [3.5, 0, -3.5, -4, -10],
                     '$x_{10}$, мм' : [7, 3.5, 0.5, -2, -8],
                     '$x_{11}$, мм' : [11, 6.5, 3, 1, -6],
                     '$x_{12}$, мм' : [14.5, 10, 6, 3.5, -3.5],
                     '$x_{13}$, мм' : [18, 13.5, 9, 6, -1.5],
                     '$x_{14}$, мм' : [21.5, 16.5, 12, 8.5, 1],
                     '$x_{15}$, мм' : [25.5, 19.5, 15, 11, 3]})

x_screen = 1090

alpha = 632.82 * 10 ** (-9)
delta_alpha = 0.01 * 10 ** (-9)

df33
```

      X объекта, мм
      $x_1$, мм
      $x_2$, мм
      $x_3$, мм
      $x_4$, мм
      $x_5$, мм
      $x_6$, мм
      $x_7$, мм
      $x_8$, мм
      $x_9$, мм
      $x_{10}$, мм
      $x_{11}$, мм
      $x_{12}$, мм
      $x_{13}$, мм
      $x_{14}$, мм
      $x_{15}$, мм

      0
      166
      -25.0
      -22.5
      -18.5
      -15
      -11.5
      -8.5
      -4.5
      -0.5
      3.5
      7.0
      11.0
      14.5
      18.0
      21.5
      25.5

      1
      266
      -25.5
      -23.0
      -19.5
      -16
      -13.0
      -10.5
      -6.5
      -3.0
      0.0
      3.5
      6.5
      10.0
      13.5
      16.5
      19.5

      2
      366
      -25.5
      -22.5
      -19.5
      -17
      -14.0
      -11.5
      -9.0
      -5.5
      -3.5
      0.5
      3.0
      6.0
      9.0
      12.0
      15.0

      3
      466
      -26.0
      -24.0
      -21.5
      -19
      -17.0
      -14.0
      -11.5
      -8.5
      -4.0
      -2.0
      1.0
      3.5
      6.0
      8.5
      11.0

      4
      566
      -26.5
      -24.5
      -22.0
      -20
      -18.0
      -16.0
      -14.0
      -12.0
      -10.0
      -8.0
      -6.0
      -3.5
      -1.5
      1.0
      3.0

```python
print('Координата экрана, {} мм'.format(x_screen))
```

    Координата экрана, 1090 мм

Посчитаем расстояние от экрана до объекта $L$, ширину интерференционной полосы $\Delta x$:

$L = x_{Э} - x_{О}$

$\Delta x = \frac{x_{min\space max} - x_{min\space min}}{number\space of\space minimums}$

```python
df33_calc = pd.DataFrame({'X объекта, мм': [166, 266, 366, 466, 566]})
df33_calc['L, мм'] = x_screen - df33_calc['X объекта, мм']
df33_calc['$\Delta x$, мм'] = (df33['$x_{15}$, мм'] - df33['$x_1$, мм']) / 15
df33_calc
```

      X объекта, мм
      L, мм
      $\Delta x$, мм

      0
      166
      924
      3.366667

      1
      266
      824
      3.000000

      2
      366
      724
      2.700000

      3
      466
      624
      2.466667

      4
      566
      524
      1.966667

### Координаты последовательных минимумов интенсивности при разных положениях объекта 32

```python
df32 = pd.DataFrame({'X объекта, мм': [166, 266, 366, 466, 566],
                     '$x_1$, мм' : [-25, -27, -25.5, -25, -25.5],
                     '$x_2$, мм' : [-22, -23, -22, -23, -23],
                     '$x_3$, мм' : [-17, -19, -19, -19.5, -20.5],
                     '$x_4$, мм' : [-13, -15, -16, -16.5, -18.5],
                     '$x_5$, мм' : [-9, -11.5, -12.5, -14, -16],
                     '$x_6$, мм' : [-5, -7.5, -9.5, -11, -13.5],
                     '$x_7$, мм' : [-1, -4, -6, -8, -11],
                     '$x_8$, мм' : [3, -0.5, -3, -5, -9],
                     '$x_9$, мм' : [7, 3.5, 0.5, -2, -6.5],
                     '$x_{10}$, мм' : [11, 7, 4, 0.5, -4],
                     '$x_{11}$, мм' : [15, 11, 7, 4, -1.5],
                     '$x_{12}$, мм' : [19, 14.5, 11.5, 6.5, 0.5]})

df32_calc = pd.DataFrame({'X объекта, мм': [166, 266, 366, 466, 566]})
df32
```

      X объекта, мм
      $x_1$, мм
      $x_2$, мм
      $x_3$, мм
      $x_4$, мм
      $x_5$, мм
      $x_6$, мм
      $x_7$, мм
      $x_8$, мм
      $x_9$, мм
      $x_{10}$, мм
      $x_{11}$, мм
      $x_{12}$, мм

      0
      166
      -25.0
      -22
      -17.0
      -13.0
      -9.0
      -5.0
      -1
      3.0
      7.0
      11.0
      15.0
      19.0

      1
      266
      -27.0
      -23
      -19.0
      -15.0
      -11.5
      -7.5
      -4
      -0.5
      3.5
      7.0
      11.0
      14.5

      2
      366
      -25.5
      -22
      -19.0
      -16.0
      -12.5
      -9.5
      -6
      -3.0
      0.5
      4.0
      7.0
      11.5

      3
      466
      -25.0
      -23
      -19.5
      -16.5
      -14.0
      -11.0
      -8
      -5.0
      -2.0
      0.5
      4.0
      6.5

      4
      566
      -25.5
      -23
      -20.5
      -18.5
      -16.0
      -13.5
      -11
      -9.0
      -6.5
      -4.0
      -1.5
      0.5

```python
df32_calc = pd.DataFrame({'X объекта, мм': [166, 266, 366, 466, 566]})
df32_calc['L, мм'] = x_screen - df32_calc['X объекта, мм']
df32_calc['$\Delta x$, мм'] = (df32['$x_{12}$, мм'] - df32['$x_1$, мм']) / 12
df32_calc
```

      X объекта, мм
      L, мм
      $\Delta x$, мм

      0
      166
      924
      3.666667

      1
      266
      824
      3.458333

      2
      366
      724
      3.083333

      3
      466
      624
      2.625000

      4
      566
      524
      2.166667

### Построения аппроксимирующей прямой
Для этого обучим линейную регрессию на $\Delta x$ от $L$:

```python
from sklearn import linear_model

regr_33 = linear_model.LinearRegression()
X_33 = np.array(df33_calc['L, мм']).reshape(-1, 1)
y_33 = np.array(df33_calc['$\Delta x$, мм'])
regr_33.fit(X_33, y_33)

regr_32 = linear_model.LinearRegression()
X_32 = np.array(df32_calc['L, мм']).reshape(-1, 1)
y_32 = np.array(df32_calc['$\Delta x$, мм'])
regr_32.fit(X_32, y_32)

print('Для объекта 33, K = {:.3f}, b = {:.3f}'.format(regr_33.coef_[0], regr_33.intercept_))
print('Для объекта 32, K = {:.3f}, b = {:.3f}'.format(regr_32.coef_[0], regr_32.intercept_))
```

    Для объекта 33, K = 0.003, b = 0.287
    Для объекта 32, K = 0.004, b = 0.225

```python
fig, ax = plt.subplots(figsize=(20, 5))
ax.set_title('Зависимость ширины интерференционной полосы от расстояния между экраном и объектом для двух объектов')
X_33 = X_33.reshape(5)
ax.scatter(X_33, y_33, c='r')
k, b = regr_33.coef_[0], regr_33.intercept_
x = np.linspace(500, 1000, 50)
ax.plot(x, np.polyval([k, b], x), 'r--')

X_32 = X_32.reshape(5)
ax.scatter(X_32, y_32, c='b')
k, b = regr_32.coef_[0], regr_32.intercept_
x = np.linspace(500, 1000, 50)
ax.plot(x, np.polyval([k, b], x), 'b--')

plt.show()
```

Найдем $d_{33}$ и $d_{32}$, используя $d = \frac{\lambda}{K}$

```python
d_32 = alpha / regr_32.coef_[0]
print('Расстояние между щелями для 32-го объекта: {:.3f} мкм'.format(d_32 * 10 ** 6))

d_33 = alpha / regr_33.coef_[0]
print('Расстояние между щелями для 33-го объекта: {:.3f} мкм'.format(d_33 * 10 ** 6))
```

    Расстояние между щелями для 32-го объекта: 165.083 мкм
    Расстояние между щелями для 33-го объекта: 189.846 мкм

Найдем $\overline d$ для расстояния щели по двум точкам

```python
print('Среднее расстояние между щелями: {:.3f} мкм'.format((d_33 + d_32) / 2 * 10 ** 6))
```

    Среднее расстояние между щелями: 177.465 мкм

Найдем погрешность для среднего расстояния между щелями:
$\Delta K = \sqrt{\frac{\sum(y_i-Kx_i-b)^2}{(n-2)\sum(x_i-\overline{x})^2}}$

$\Delta d = \sqrt{(\frac{\delta d}{\delta \lambda}\cdot \Delta \lambda)^2 +  (\frac{\delta d}{\delta K}\cdot \Delta K)^2 } = \sqrt{(\frac{1}{K}\Delta \lambda)^2 + (\frac{\lambda}{K^2}\Delta K)^2}$

```python
from sklearn.metrics import mean_squared_error

final_k = np.array([(regr_33.coef_[0] + regr_32.coef_[0]) / 2]).reshape(1, 1)
final_b = np.array([(regr_33.intercept_ + regr_32.intercept_) / 2])

Xs = np.concatenate((X_32, X_33))
ys = np.concatenate((y_32, y_33))

mse = mean_squared_error(ys, (final_k * Xs + final_b).reshape(10))

delta_k = np.sqrt(mse / np.sum((Xs - np.mean(Xs)) ** 2))
delta_d = np.sqrt((delta_alpha / final_k[0][0]) ** 2 + (alpha * delta_k / final_k[0][0] ** 2) ** 2)

print('Погрешность для среднего расстояния между щелями: {} мкм'.format(delta_d * 10 ** 6))
```

    Погрешность для среднего расстояния между щелями: 18.444950130275327 мкм

Тогда $d = \overline d + \Delta d = (177 \pm 18) \space мкм$

### Выводы

В ходе данной лабораторной работы был рассмотрен опыт Юнга для вычисления расстояния между щелями с помощью длины волны $\lambda$, расстояния между объектом и экраном $L$, а также ширины интерференционной полосы $\Delta x$. Был найден коэффицент $K=\frac{\lambda}{d}$ с помощью линейной регрессии $\Delta x(L) = \frac{\lambda}{d}\cdot L$, откуда было взято расстояние $d = \frac{\lambda}{K}$. Это расстояние между щелями получилось равным $(177 \pm 18)$ мкм


Inter-areal balanced amplification enhances signal propagation in a large-scale circuit model of the primate cortex
======

Joglekar MR, Mejias JF, Yang GR, Wang XJ.. Neuron. 2018 Apr 4;98(1):222-234.e8. doi: 10.1016/j.neuron.2018.02.031. Epub 2018 Mar 22. PMID: 29576389.

Journal Club Tuesday 10th April 2018
------

#### Abstract
This notebook implements an interative version of the models in the Joglekar et al 2018 paper.

## Introduction
The paper discusses signal propagation from
This notebook follows the stucture of the Joglekar et al. 2018 paper [1]:
* Firstly describing the rate model;
* Secondly, describing the laminar model;
* Finally, describig the spiking network model.

```python
# LIBRARY
# vector manipulation
import numpy as np
# math functions
import math
import sys

# THIS IS FOR PLOTTING
%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import warnings
warnings.filterwarnings("ignore")
```

## The Rate Model
The rate model is motivated by work in a previous wang paper [2]. The equations describe excitatory (E) and inhibitory (I) firing rates of a neuronal population:
\begin{equation}  \tau_E\frac{dv_E}{dt} =-v_e+\beta_E[I_E]_{+}, \end{equation}
\begin{equation}  \tau_I\frac{dv_I}{dt} =-v_I+\beta_I[I_I]_{+}, \end{equation}
where $[I]_+=\max(I,0)$ is the input to the system, $\tau_E$ and $\tau_I$ are time constants. $\beta_E$ and $\beta_I$ are the slopes of the f-l??? curves.

### Parameters
The parameters for the model are:

| Parameter | Excitatory (E)  | Inhibitory (I) |
|------|------|------|
|   $\tau$ (ms)| 20|10|
|   $\beta$ | 0.066|0.351|

### Model Input
The incoming current is given by
\begin{equation}
I^i_E=(1+\eta h_i)(w_{EE} v^i_E+I_{lr,E^i})-w_{EI}v_I^i+I_{ext,E^i}
\end{equation}

\begin{equation}
I^i_I=(1+\eta h_i)(w_{IE} v^i_E+I_{lr,I^i})-w_{II}v_I^i+I_{ext,I^i}
\end{equation}
where $I^i_E$ and $I^i_I$ denote the input currents to the excitatory and inhibitory populations from area i.

$w$ denotes the local-circuit excitatory and inhibitory connections.

$h$ is the hierarchical position, it is normalised between $0$ and $1$.

$\eta$ scales the exciatory firing rate, it is set as $\eta=0.68$.

The local connectivity parameters for the input are:

| Parameter | Value  |
|------|------|
|   $\mu_{IE}$ | 25.3|
|   $w_{EE}$ | 24.3|
|   $w_{IE}$ | 12.2|
|   $w_{II}$ | 12.5|

```python
#### Model Parameters

mu_IE=25.3
mu_EE=33.7

def Input(input_amp,time):
    I=np.zeros((len(time)))     # CURRENT (INPUT)

    for k in range (0,len(time)):
        if time[k] >0.1 and time[k]<0.35:
            I[k]=input_amp # Input change
    return I

# Numerical Solution to the Izhikevich model
def DiscreteMillerRateModel(u_E,u_I,I,kI,w):
    tau_E=1000/20
    tau_I=1000/10
    w_EE=w
    w_IE=kI*w
    w_II=w
    w_EI=kI*w
    u_E = u_E + 0.0001*(-(1+w_EE)*u_E+w_EI*u_I+I)/tau_E # Discrete form of membrane potential of the neuron
    u_I = u_I + 0.0001*(-(w_IE)*u_E+(w_II-1)*u_I+I)/tau_I            # Discrete form of membrane recovery variable
   # u_E = u_E + 0.0001*(-1*u_E+I)/tau_E # Discrete form of membrane potential of the neuron
    #u_I = u_I + 0.0001*(-1*u_I+I)/tau_I            # Discrete form of membrane recovery variable
    return u_E,u_I

time=np.arange(0,2,0.0001)  # time period 1000ms (1s)
input_amp=0
I=Input(input_amp,time)
fig, ax1 = plt.subplots(figsize=(12,3))
ax1.plot(time, Input(input_amp,time), 'b-') # y label
ax1.set_xlabel('time (s)') # x label
# Make the y-axis label, ticks and tick labels match the line color.
ax1.set_ylabel('Input mV', color='b')
ax1.set_ylim(0,3*input_amp) # setting the plotting range
plt.title('Figure 2: Input to the system')
plt.show()
```

```python
vE=1*np.ones(len(time))
vI=1*np.ones(len(time))

for k in range (0,len(time)-1):
   vE[k+1], vI[k+1]= DiscreteMillerRateModel(vE[k],vI[k],I[k],1.1,4)

fig, ax1 = plt.subplots(figsize=(12,3))
ax1.plot( time,vE, 'b-', label = 'Output')
ax1.set_xlabel('time (ms)')
# Make the y-axis label, ticks and tick labels match the line color.
ax1.set_ylabel('Output mV', color='b')
ax1.tick_params('y', colors='b')
ax2 = ax1.twinx()
ax2.plot(time, I, 'r', label = 'Input')
ax2.set_ylim(0,input_amp*20)
ax2.set_ylabel('Input (mV)', color='r')
ax2.tick_params('y', colors='r')
fig.tight_layout()
ax1.legend(loc=1)
ax2.legend(loc=3)
plt.show()
```

The background firing rate are 10Hz for excitatory rate and 35Hz for inhibitory, which has been subtracted for the figures.

The long-range input currents are given by
\begin{equation}
I_{lr,E}^i=\mu_{EE}\sum_{j=1}^{N} FLN_{ij}v_{E}^j
\end{equation}
\begin{equation}
I_{lr,I}^i=\mu_{IE}\sum_{j=1}^{N} FLN_{ij}v_{E}^j
\end{equation}

To model global

The parameters for the model are:

| Parameter | weak GBA | strong GBA |
|------|------|------|
|   $w_{EI}$ | 19.7|25.2|
|   $\mu_{EE}$ | 33.7|51.5|

```python
# Numerical Solution to the Izhikevich model
def DiscreteRateModel(u_E,u_I,I_E,I_I):
    tau_E=20
    tau_I=10
    beta_E=0.066
    beta_I=0.351
    u_E = u_E + 0.0001*(-u_E+beta_E*I_E)/tau_E # Discrete form of membrane potential of the neuron
    u_I = u_I + 0.0001*(-u_I+beta_I*I_I)/tau_I            # Discrete form of membrane recovery variable
    return u_E,u_I
```

```python

FLN=np.random.rand(3,3)

time=np.arange(0,2,0.0001)  # time period 1000ms (1s)
vE=np.zeros((3,len(time)))
vI=np.zeros((3,len(time)))
INTERNAL= np.dot(FLN,vE[:,0])
INTERNAL= np.dot(FLN,vI[:,0])
u_E=np.zeros(3)
u_I=np.zeros(3)

I_E=np.zeros(3)
I_I=np.zeros(3)

for k in range (0,len(time)):
    for i in range(0,3):
         i+k
         u_E, v_E= DiscreteRateModel(uE,u_I,I_E,I_I)

def I_E(input_duration,input_amp,time):
    eta=0.68
    w_EE=24.3
    mu_IE=25.3
    w_IE=12.2
    I_E=(1+eta)*(w_EE+v_e+)
    I=np.zeros((len(time)))     # CURRENT (INPUT)

    for k in range (0,len(time)):
        if time[k] >0.1 & time[k]<0.35:
            I[k]=input_amp # Input change
    return I

```

#### References

1 Joglekar MR, Mejias JF, Yang GR, Wang XJ. Inter-areal Balanced Amplification Enhances Signal Propagation in a Large-Scale Circuit Model of the Primate Cortex. Neuron. 2018 Apr 4;98(1):222-234.e8. doi: 10.1016/j.neuron.2018.02.031. Epub 2018 Mar 22. PMID: 29576389.

```python

```

Joglekar et al 2018/Joglekar et al  2018 Neuron - Rate Model.ipynb


```python
%matplotlib inline
```

```python
import os
os.chdir("../")

from markowitz.structs import DBCache
from markowitz.layout_parser import from_file
from markowitz import consumme_window
from matplotlib.pyplot import show

def main(db, layout_file):
    layout = from_file(layout_file)

    for window in layout:
        consumme_window(window, db)

    show()

db = DBCache("cac40.db")
```

# Représentation graphique de la Théorie moderne du portefeuille

## Léo Duret & Adrien Chaillout-Morlot

## Introduction

La Théorie Moderne du Portefeuille De Markowitz  est un modèle incontournable des théories de gestion des actifs.
Elle a été proposé en 1952 par l'économiste américain Harry Markowitz, le rendant lauréat d'un prix Nobel en Sciences Economiques. On connait bien les représentations graphiques classiques des portefeuilles à deux titres qui ne font pas parler les nuances des bases de données. Ainsi dans ce rapport nous essayerons d'apporter de nouveaux éléments sur ces représentations graphiques en construisant une application et des outils permettant cette visualisation.

## 1. Modèle

Afin de comprendre l'origine les graphiques qui vont suivre nous proposons au lecteur un rappel sur la théorie moderne  du portefeuille.

### 1.1 Hypothèses

#### 1.1.1 Rationalité des agents

La théorie moderne du portefeuille ne propose pas de portefeuille optimal, seulement un ensemble paires risque/rendement optimaux (la frontière efficiente). L'hypotèse de rationnalité implique qu'il existe un seul portefeuille qui maximise l'utilité de l'agent, celui avec le meilleur rendement au risque le plus faible (selon l'aversion au risque de l'agent).

#### 1.1.2 Aversion aux risques

Dans le modèle de markowitz le rendement se paie au prix du risque. En effet, l'investisseur n'augmentera son risque si et seulement si son ésperance de gains augmente.

### 1.2 Rendement-Risque
Les hypotèses en têtes nous pouvons ainsi definir les variables de mesure du rendement et du risque dans le modèle de markowitz.
Tout d'abord on procède à une analyse individuelle des titres en étudiant les variations historiques. Dans notre cas nous avons étudié les titres du CAC40 entre le 20/10/2015 et les 19/10/2015.

Pour un titre possedant $t$ valeurs on calcule la variation entre les valeurs de fermeture du marché:

\begin{align}
    V_t = \ln(X_t) - \ln(X_{t-1})
\end{align}

ainsi que le rendement, soit l'ésperance de variation:

\begin{align}
    \overline{R_{titre}} = E(R_{titre} )=\frac{1}{N}\sum\nolimits_{i=0}^{N}R_i
\end{align}

Enfin afin de determiner le risque d'un actif calcule l'écart type des variations, que l'on peut interpreter comme étant la volatilité du titre durant la période étudiée.

\begin{align}
    \sigma_{titre}=\sqrt{\frac{\mathrm{\sum\nolimits_{i=0}^{N}(R_i-\overline{R_{titre}})^2} }{\mathrm{(N-1)} }   }
\end{align}

### 1.3 Portefeuille
On considère un portefeuille comme étant une collection d'actifs, ainsi par croisements des données on peut observer les matrices de covariance et par extension de corrélation.

On rappelle le calcul de la covariance:

\begin{align}
    Cov(R_A,R_B)= E(R_A \cdot R_B ) - E(R_A)\cdot E(R_B)
\end{align}

Et du coefficient de corrélation $\varphi$:

\begin{align}
    \varphi_{AB}= \frac{\mathrm{Cov( R_A \cdot R_B )} }{\mathrm{  \sigma_A \sigma_B } }
\end{align}

Exemple de matrice de covariance pour 3 titres

\begin{pmatrix}
    Var(R_A)&Cov(R_A, R_B)&Cov(R_A, R_C)\\
    Cov(R_B, R_A)&Var(R_B)&Cov(R_B, R_C)\\
    Cov(R_C, R_A)&Cov(R_C, R_B)&Var(R_C)
\end{pmatrix}

Idem pour la matrice de corrélation

\begin{pmatrix}
    1&\varphi_{AB}&\varphi_{AC}\\
    \varphi_{BA}&1&\varphi_{BC}\\
    \varphi_{CA}&\varphi_{CB}&1
\end{pmatrix}

```python
# Nous avons ajouté la possibilité d'afficher ces variables dans un récapitulatif du portefeuille

p = db.load("renault","peugeot","lvmh")

p.recap()

```

      renault
      peugeot
      lvmh

      len
      515
      515
      515

      avg
      0.00035596
      0.000786689
      0.000911761

      stdv
      0.0197384
      0.0212004
      0.0143329

      COVAR

      renault
      0.000389606
      0.00033382
      0.000152542

      peugeot
      0.00033382
      0.000449456
      0.000163033

      lvmh
      0.000152542
      0.000163033
      0.000205432

      CORR

      renault
      1
      0.797731
      0.539191

      peugeot
      0.797731
      1
      0.536535

      lvmh
      0.539191
      0.536535
      1

### 1.4 Base de données

#### 1.4.1 Aperçu
Nous avions à notre disposition des fichiers Excel d'actifs du CAC 40.

L'interval est 20/10/2015 - 19/10/2017 pour 515 entrées par tableau, entrées ayant toutes leurs valeurs.

##### Ex: Airliquide

| ISIN         | JOUR       | OUVR     | PHAUT    | PBAS     | CLOT     | VOLUME  |
|--------------|------------|----------|----------|----------|----------|---------|
| FR0000120073 | 20/10/2015 | 100      | 100.1818 | 98.4546  | 99.0909  | 833851  |
| FR0000120073 | 21/10/2015 | 99.6364  | 100.0455 | 98.3636  | 99.6818  | 714208  |
| FR0000120073 | 22/10/2015 | 99.5     | 103.4546 | 99.1818  | 103.0455 | 1264656 |
| FR0000120073 | 23/10/2015 | 104.0454 | 105.9546 | 103.6818 | 105.3636 | 1224405 |

#### 1.4.2 Transformation

Afin de pouvoir construire un application permettant d'afficher les graphiques il était important de pouvoir recuperer les données au même endroit. Dans cette optique nous avons créé un script `docs/script/excel_to_db.py` permettant de transferer un ensemble de fichiers excel sur une base de donnée (tel que un fichier soit un tableau). On peut observer ci dessous l'instruction de création des tableaux sur une base de données [SQLite](https://sqlite.org/index.html).

## 2. Construction de l'application

### 2.1 Avant de commencer

#### 2.1.1 Objectifs

Afin de pouvoir automatiser la production de graphiques de la théorie moderne du portefeuille il fallait determiner un certain nombre d'objectifs à respecter, nous fournissant ainsi une direction pour la construction de l'application.

- Tout d'abord il est certain qu'il fallait faire en sorte que l'application soit facile d'utilisation. La construction de graphiques simples se doit d'être simple, et donc ne doit pas demander à l'utilisateur de connaitre les fonctionnalités obscures d'un tableur ou la complexité d'un language de programmation.

- En suite une interface modulable serait de mise car on ne peut pas se contenter de montrer qu'une vérité. Laisser libre cours à l'utilisateur de tordre la donnée dans le sens qu'il souhaite était donc un élément indispensable.

- Enfin construire un application a part entière est déjà en soit une belle aventure, cependant le format pousse une utilisation pré-maché du logiciel. Ainsi une part du projet est consacré a l'élaboration d'utilitaires pour faciliter la programme d'outils connexe à la théorie moderne du portefeuille.

#### 2.1.2 Les outils

Pour ce projet nous avons décidé d'utiliser python, un language de programmation général crée par Guido van Rossum et publié en 1991. Cependant, avec maturité du language, nous n'avions pas à construire tous les outils permettant l'affichage de graphique. Ainsi, nous avons utilisé plusieurs librairie permettant de construire les courbes ainsi que de structurer les informations pendant l'execution du programme.

Les outils utilisés sont les suivants:
- [Numpy](https://www.numpy.org/) La librairie de référence en matière de calculs mathématiques et de travail avec les listes. les autres librairies dépendent de celle-ci.
- [Pandas](http://pandas.pydata.org/) une librairie permettant l'utilisation de DataFrames (sorte de tableau) semblables à celles en R.
- [Matplotlib](https://matplotlib.org/) la librairie pour l'affichage des graphiques.

### 2.2 L'implémentation

#### 2.2.1 Les produits selon markowitz et accès à la base de donnée

##### 2.2.1.1 Titres
Rappelons nous des méthodes de calculs vu précédements pour imaginer un concept de "titre financier de markowitz". Nous avons décris un calcul se déroulant à partir des valeurs passés du titres ainsi la première chose à faire est acceder la base de donnée pour récuperer cet historique. Heureusement pour nous la librairie pandas nous facilite la tache en nous permettant de créer une DataFrame directement depuis une requete de la base de donnée.

```python
# Nous travaillerons sur les valeurs de clotures ainsi la requete se matérialise de cette manière:
import pandas as pd
import sqlite3 as sql

conn = sql.connect("cac40.db")
df = pd.read_sql("SELECT clot FROM AXA", conn)

print(df.head(n=5))  # 5 premières valeurs
```

         clot
    0  22.915
    1  23.095
    2  23.820
    3  24.050
    4  24.155

```python
# On calcule la variation:

print(df["clot"].pct_change().head(n=5))  # 5 premières valeurs
```

    0         NaN
    1    0.007855
    2    0.031392
    3    0.009656
    4    0.004366
    Name: clot, dtype: float64

Dans notre cas les instances de clotures ayant toutes des valeurs cohérentes, il n'est pas necessaire de nettoyer le tableau des valeurs manquantes ou inéxactes. Pour d'optimiser les opérations de lecture de la structure "Titre" nous stockerons indépendamment les valeurs constante fréquemments utilisé calculés à partir du tableau des variation, soit l'éspérance et l'équart type. Nous gardons tout de meme le tableau des variations pour le calcul de la corrélation des portefeuilles.

##### 2.2.1.2 Portefeuille

Comme définit précédemment, un portefeuille est un ensemble de titres il est donc important que nous utilisions les équations générales et que nous puissions passer autant de titres que nous voulons tout en gardant la possibilité de calculer leur ésperance. Python nous facilite la tache avec une notation permettant de passer autant d'arguments à une fonction/méthode `def __init__(self, *args):`. Il ne reste plus qu'à implementer les fonctions de calculs de l'éspérance et de la variance (ci-dessous).

###### L'ésperance
Soit $\{R_1, ..., R_n\}$ titre dans le portefeuille

\begin{align}
    E(R_p) = \sum_{i=0}^{n}\alpha_i \cdot E(R_i)
\end{align}

```python
# Esperance
def avg(self, dot: m_types.FLOAT_COLLECTION) -> m_types.M_FLOAT:  # dot: liste de nombre à virgule
    mus = [el.avg for el in self.assets.values()]  # Liste des éspérances des titres
    assert(len(dot) == len(mus))  # Assure le nombre de dotations rapporté à la taille du portefeuille

    # Somme du produit des éspérances (mus) et des dotations (dot)
    result: float = 0
    for i in range(len(dot)):
        result += dot[i] * mus[i]

    return m_types.M_FLOAT(result)
```

###### L'écart type

\begin{align}
    Var(R_P) = (\sum_{i=0}^{n}\alpha_i \cdot \sigma_i)^2
\end{align}

C'est une forme quadratique d'ordre N donc il existe une matrice symétrique $ N \cdot N $ associée
Pour 3 titres on trouve

\begin{pmatrix}
     \alpha^2_{1}& \frac{\mathrm{\alpha_1\alpha_2} }{\mathrm{2}} &\frac{\mathrm{\alpha_1\alpha_3} }{\mathrm{2}}\\
    \frac{\mathrm{\alpha_1\alpha_2} }{\mathrm{2}}&\alpha^2_2&\frac{\mathrm{\alpha_2\alpha_3} }{\mathrm{2}}\\
    \frac{\mathrm{\alpha_1\alpha_3} }{\mathrm{2}}&\frac{\mathrm{\alpha_2\alpha_3} }{\mathrm{2}}&\alpha^2_{3}
\end{pmatrix}

Ainsi

\begin{align}
    A = M_{quadratique} \cdot M_{covar}\\
\end{align}

La trace de A est la somme des variances pondérés par les $\alpha_i$, soit la variance de $R_p$

\begin{align}
    trace(A)= Var(R_P)
\end{align}

\begin{align}
    \sigma_{R_P} = \sqrt{Var(R_P)} = \sqrt{trace(A)}
\end{align}

```python
# Ecart type

def stdv(self, dot: m_types.FLOAT_COLLECTION) -> m_types.M_FLOAT:
    assert(len(dot) == len(self.assets))  # Assure le nombre de dotations rapportés à la taille du portefeuille

    product = m_maths.dot(m_maths.symmetric_matrix(dot), self.covar_matrix)  # Produit matriciel de la matrice symmétrique et de covariance
    trace = m_maths.sqrt(m_maths.trace(product))  # Racine carré de la trace du produit matriciel

    return m_types.M_FLOAT(trace)
```

##### 2.2.1.3 L'interface de la base de donnée

Les produits ci-dessus sont construits à partir d'une base de donnée, cependant on remarque que si l'on souhaite afficher plusieurs fois le même titres avec l'application, les memes informations seront demandé plusieurs fois à la base de donnée. C'est comme si à chaque fois que vous rejoignez votre hotel on vous demandait de repasser par la reception pour prendre une chambre.

Pour remedier à ce problème et optimiser l'application nous avons mis en place un schema très simple mais éfficace. Tout d'abord on va distinguer deux elements dans la base de donnée. Ces éléments vont se comporter comme un gardien et un réceptioniste. En effet si le gardien vous reconnait il vous laisse acceder à votre chambre sinon c'est la case reception. Traduisons cela et vous avez une instance d'object servant de gardien dont le travaille est de determiner si le titre à déjà été créé sinon demander à l'autre objet (la récéption) de s'en charger et de vous le remettre.

De cette manière nous sommes assuré que chaque titre et portfeuille est chargé au maximum une fois. Ceci conclut notre module structure de donnée dont l'interface est basique: les services ayant besoin d'un titre n'ont qu'a effectuer une demande à l'objet servant de gardien.

#### 2.2.2 L'architecture

Concernant l'architecture du programme elle est plutot simple:
- D'abord la déconstruction des fichiers de layout en classes connexes contenant les informations des differents graphiques
- En suite lecture des objects de graphique, chargement de la base de donnée et des objets graphiques (prochaine section)
- Enfin assemblage des titres et des objets de graphiques
- Affichage des graphiques

### 2.3 Les graphiques

Pour afficher les graphiques nous avons créé une syntaxe nous permettant de lire les graphiques directemment depuis un fichier contenant seulement les informations sur les titres et le type de graphique à utiliser. Ces type de graphique sont en  fait des référence à des fichiers contenant les objects du meme noms. Ainsi les graphiques sont généré dynamiquement lorsque leur nom est reconnu.

La syntaxe des fichiers est plutôt simple
- Pour les commentaires: `! ...`
- Nouvelle fenetre: `& nom_fenetre (configuration...) { matrice... }`
- Pour la configuration: `(nom=valeur,...)`
- Pour la matrice:
    - ligne: `[...]`
    - separateur de colonne: `|`
- Les objects: `nom_objet(titre)` ou `nom_objet(titre/titre/...)` pour plusieurs titres

Le chargements de objects est effectué dynamiquement celon le nom de l'objet appelé dans le fichier. Enfin la base de donnée charge un titre lorsque que l'analyse du fichier ne trouve qu'un `titre`, sinon un portefeuille contenant les titres est généré dans le cas où plusieurs titres sont reconnus (`titre/titre`)

#### 2.3.1 Analyse des titres

##### 2.3.1.1 Distribution normale

Afin d'analyser et comparer les titres nous avons fournit un moyen de représenter graphiquement l'ésperance et l'écart type de ces derniers. En effet les titres étant composé d'une large quantité de valeurs, on peut estimer que la loi de leur variation peut s'approximer par une loi normale. Ainsi, dans un premier temps nous avons fournit un objet permettant de calculer les points d'une loi normale de paramètre $N(\mu, \sigma)$

En créant un ensemble de points entre $-10$ et $10$ leur applicant cette fonction, nous avons créé un emble de points $x$ et $y$ que nous affichons sur un graphique.

```python
# NormalGraph.const1 = np.sqrt(2*np.pi)
# args[0] = mu
# args[1] = sigma

def densite(point, *args):
    t1 = 1/(args[1]*NormalGraph.const1)
    x = (point-args[0])/args[1]
    t2 = np.exp(-0.5*x*x)
    return t1*t2
```

```python
with open("docs/layouts/normale.ly") as ly:
    print(ly.read())
```

    ! docs/layouts/layout3.ly

    & Normale (scale=100) {
    	[ NormalGraph(axa) | NormalGraph(lvmh) ]
        [ NormalGraph(peugeot) ]
    }

```python
main(db, "docs/layouts/normale.ly")
```

##### 2.3.1.1 Estimation de la densité par la méthode des noyaux

Cependant la distribution normale n'est pas un bon indicateur pour exprimer les series à evennements improbables. En effet les "pattes" de la loi normale nous montrent que les hautes variations ont très peut de chance d'arriver. Cependant sur les marchés financiers ces evennements ne sont pas a sous-estimer. Pour demontrer cette réalité il faut trouver un moyen d'afficher la "vraie" taille des pattes de la loi normale. Pour ce faire nous avons implémenté un moyen d'observer une approximation de la densité des la série en utilisant la méthode de l'éstimation de la densité par noyau, Kernel Density Estimation en anglais, d'ou le nom KDE pour les graphiques.

Pour l'estimation de la densité nous avons choisit un noyau normale car nous cherchions à estimer la taille des pattes. Et un parametre $h$ de 1.06 selon les recommendations, le resultat paraissant satisfaisant. Le principe de l'estimation de densité par noyau est de créer un ensemble de loi normale et d'en faire la somme. Ainsi on observe une agrégation plus importante autour de l'esperance de la série mais les valeurs plus extremes sont aussi prisent en compte.

```python
# Kde.const1 = np.sqrt(2*np.pi)

def _kernel(k):
    """ Kernel of the KDE """
    return (1/Kde.const1)*np.exp(-0.5*k*k)

def densite(point, df, stdv):
    """ estimation function """
    n = len(df-1)  # index 0 is NaN
    h = 1.06 * (stdv)/np.power(n, 0.2)
    pre = 1.0/(n*h)
    results = [
        Kde._kernel((point-xi)/h)
        for xi in df
        if not np.isnan(xi)
    ]
    return np.sum(results) * pre
```

```python
with open("docs/layouts/kde.ly") as ly:
    print(ly.read())
```

    ! docs/layouts/layout3.ly

    & Kde (scale=100) {
    	[ NormalGraph(axa) Kde(axa) ]
    }

```python
main(db, "docs/layouts/kde.ly")
```

#### 2.3.2 Efficacité des portefeuilles

##### 2.3.2.1 Génération du plan à deux titre

Afin d'afficher un graphique nous devons spécifier un ensemble de dotations à passer pour le calcul de l'éspérance et de la variance du portefeuille.
Dans le cas ou nous avons seulement 2 dotations il suffit de generer $x$ et donner $y = 1-x$ ainsi à partir d'une liste de $x$ nous donnons une liste de $y$, nous pouvons passer ces paires directement dans la fonction de calcul. Le plan à deux titres peut s'interpreter de cette manière: _Il n'existe pas de meilleur paires rendement/risque que celles sur la courbe_.

```python
with open("docs/layouts/layout1.ly") as ly:
    print(ly.read())
```

    ! docs/layouts/layout1.ly

    & 2PF (scale=100) {
    	[ EfficientFrontier(axa/lvmh) ]
    }

```python
main(db, "docs/layouts/layout1.ly")
```

##### 2.3.2.2 Génération du plan barycentrique (pour 3 titres)
Pour trois titres nous devons trouver une manière efficace de générer un triplet de points dont la somme équivaut à 1. Une manière de faire cela est de calculer des points sur le plan barycentrique. Heureusement, l'industrie du graphisme numérique à developpé ces algorithmes afin de pouvoir remplir des triangles et afficher des modèles coloré. L'algo rythme utilisé est plutot simple:

On fixe un point (en l'occurence z) auquel on retire à plusieur étapes un nombre 's' ( $ 1 / totaletapes$ ) puis on calcule tous les points x, y qui correspondent à
$x + y = nbetapes*s$

```python
with open("docs/layouts/layout2.ly") as ly:
    print(ly.read())
```

    ! docs/layouts/layout2.ly

    & 3PF (scale=100) {
    	[ EfficientFrontier(axa/lvmh/engie) ]
    }

```python
main(db, "docs/layouts/layout2.ly")
```

##### 2.3.2.3 Génération du plan à N titre

Pour afficher la frontière efficiente à N titres nous avons du utiliser des méthodes plus approximatives. En effet l'algorithme au dessus est déjà couteux pour un grand nombre de points généré. Ansi pour N titres nous faisons appel à la librairie *numpy* qui permet de calculer des points d'une distribution de *Dirichlet*, et cela de manière aléatoire. Ainsi nous avons un ensemble de points respectant $\sum_{i=0}^{n}\alpha_i = 1$

```python
with open("docs/layouts/layout3.ly") as ly:
    print(ly.read())
```

    ! docs/layouts/layout3.ly

    & NPF (scale=100) {
    	[ EfficientFrontier(axa/lvmh/engie/peugeot/renault) ]
    }

```python
main(db, "docs/layouts/layout3.ly")
```

## Conclusion

Ainsi à travers ces differents graphiques nous avons pu observer les différents manière d'observe la théorie moderne du portefeuille de markowitz, que se soit du point de vu des titres, ou bien encore de la fontière efficiente à toutes les dimensions. Par la théorie moderne du portefeuille markowitz nous montre que la diversification des actifs, en baissant le rendement, réduit considérablement le risque des agents.

### Critiques

On observe deux critiques majeurs de la théorie moderne du portefeuille:

- La première critique vient de Benoit Mandelbrot qui estime cette théorie trop déconnecté des marchés financiers. En effet la loi de gauss sous évalue la probabilité d'évenements improbables (notamment les krachs boursiers). Nous avons observé un example en comparant avec l'estimation par noyau.

- La deuxième critique, plus intuitive, sont que les suppositions néo-classiques sont irréalistes, les individus ne sont pas rationnels et les variations passé ne se répètent pas dans le futur.

### Ouverture

Afin de répondre à ces critiques et mieux anticiper les évenements improbables on peut utiliser la loi de pareto ou les lois de puissance qui on moins tendance à les sous evaluer.

Le code est disponibles sur : [github.com/hyyking/markowitz](https://github.com/hyyking/markowitz)

La bibliographie : [bibliographie](https://github.com/hyyking/markowitz/tree/master/docs/references)


---
title: "Order statistics — Part 3: Proof of expectation formula for uniform RVs"
date: 2021-03-08
categories: [statistics, order statistics]
tags: [statistics, order statistics, uniform distribution, expectation formula, gamma function, beta function]
preview_image: /enrightward.github.io/assets/img/order-statistics/part3/gamma.png
---

## 1. Introduction

In the [last post](https://enrightward.github.io/enrightward.github.io/posts/order-statistics-part-2/), we proved the following general formulae, after conducting some numerical experiments to gain intuition:

\begin{align}
F_{X_{(k)}}(x) &= \sum_{j=k}^{n} \binom{n}{j} F_{X}(x)^{j} (1 - F_{X}(x))^{n-j}, \newline
f_{X_{(k)}}(x) &= k \binom{n}{k} f_{X}(x) \cdot F_{X}(x)^{k-1} \, (1 - F_{X}(x))^{n-k}, \newline
\mathbb{E} \left \lbrack X_{(k)} \right \rbrack &=
k \binom{n}{k} \int_{0}^{1} x \cdot f_{X}(x) \cdot F_{X}(x)^{k-1} \, (1 - F_{X}(x))^{n-k} \, dx,
\end{align}

Note they depend only on $n, k, f_{X}$ and $F_{X}$. We will now round out the exposition of $k^{\textrm{th}}$ order statistics by specialising these general formulae to the case of the uniform distribution $U(0, 1)$. We will show that:

\begin{align}
F_{X_{(k)}}(x) &= \sum_{j=k}^{n} \binom{n}{j} x^{j} (1 - x)^{n-j}, \newline
f_{X_{(k)}}(x) &= k \binom{n}{k} x^{k-1} \, (1 - x)^{n-k}, \newline
\mathbb{E} \left \lbrack X_{(k)} \right \rbrack &= \frac{k}{n+1} \cdot
\end{align}

This post is purely theoretical, and has no code. The mathematical tools we use are the beta and gamma functions, introduced below.

## 2. Specialising the formulae to $U(0, 1)$

In the case where the $X \sim U(0, 1)$, we have $F_{X}(x) = x$ and $f_{X}(x) = 1$ on the unit interval, so the formulae computed above specialise to:

\begin{align}
F_{X_{(k)}}(x) &= \sum_{j=k}^{n} \binom{n}{j} x^{j}(1 - x)^{n - j}, \newline
f_{X_{(k)}}(x) &= k \binom{n}{k} x^{k-1} (1 - x)^{n-k}, \newline
\mathbb{E}[X_{(k)}] &=
k \binom{n}{k} \int_{0}^{1} x \cdot x^{k-1} \, (1 - x)^{n-k} \, dx =
k \binom{n}{k} \int_{0}^{1} x^{k} (1 - x)^{n-k} \, dx.
\end{align}

## 3. Proving the expectation formulae using beta and gamma functions

The integral in the above expectation formula is an example of the [_beta function_](https://en.wikipedia.org/wiki/Beta_function), defined as

\begin{align}
B(z, w) := \int_{0}^{1} x^{z-1} (1 - x)^{w-1} \, dx.
\end{align}

This formula makes sense for complex numbers $w$ and $z$ with positive real part, but we are only interested in integer-valued arguments, because our exponents for $x$ and $1-x$ are $k$ and $n-k$, respectively. In the new notation of beta functions, our expectations are written:

\begin{align}
\mathbb{E}[X_{(k)}] = k \binom{n}{k} \int_{0}^{1} x^{k} (1 - x)^{n-k} \, dx = k \binom{n}{k} B(k+1, n-k+1).
\end{align}

This is interesting to us because the beta function can be computed in terms of the [_gamma function_](https://en.wikipedia.org/wiki/Gamma_function)

\begin{align}
\Gamma(z) := \int_{0}^{\infty} x^{z-1} e^{-x} \, dx,
\end{align}

and the gamma function is a generalisation to the (positive half) complex plane of the factorial function $n \mapsto n!$, hence reduces to factorial computations for integer-valued arguments. Concretely, the relations we need are:

\begin{align}
B(z, w) &= \frac{\Gamma(z) \Gamma(w)}{\Gamma(z + w)}, \quad \textrm{and} \newline
\Gamma(n) &= (n-1)!
\end{align}

We will show these relations shortly. For now, observe they imply the formulae we conjectured for the expectations of $X_{(k)}$. Indeed, using each of these results in turn, we have

\begin{align}
B(k+1, n-k+1) = \frac{\Gamma(k+1) \Gamma(n-k+1)}{\Gamma(n+2)} =
\frac{k!(n-k)!}{(n+1)!} = \frac{1}{(n+1) \binom{n}{k}},
\end{align}

and hence

\begin{align}
\mathbb{E}[X_{(k)}] &= k \binom{n}{k} B(k+1, n-k+1)
= \frac{k \binom{n}{k}}{(n+1) \binom{n}{k}} = \frac{k}{n+1} \cdot
\end{align}

## 4. Proving the necessary properties of beta and gamma functions

Now we prove the relations for $B(z, w)$ and $\Gamma(z)$. To see that $\Gamma(n) = (n-1)!$, it suffices to prove the recursive formula

\begin{align}
\Gamma(z) = (z-1)\Gamma(z-1),
\end{align}

for any $z$ with positive real part. For then we would have:

\begin{align}
\Gamma(n) &= (n-1)\Gamma(n-1) \newline
&= (n-1)(n-2)\Gamma(n-2) \newline
\vdots \newline
&= (n-1)(n-2)(n-3) \ldots (2)(1) = (n-1)!
\end{align}

We show this using the definite integral version of the integration by parts formula:

\begin{align}
\int_{a}^{b} u \, dv = \left \lbrack uv \right \rbrack_{a}^{b} - \int_{a}^{b} v \, du,
\end{align}

with $u = x^{z-1}$ and $dv = e^{-x} dx$, implying that $du = (z-1)x^{z-2} dx$ and $v = -e^{-x}$. Note this formula holds also in the case where $a$ or $b$ is $\pm \infty$, provided we interpret the term $\left \lbrack uv \right \rbrack_{a}^{b}$ as expressing the appropriate limit(s). We have:

\begin{align}
\Gamma(z) &= \int_{0}^{\infty} x^{z-1} e^{-x} \, dx \newline
&= \lim_{b \rightarrow \infty} \left \lbrack -x^{z-1} e^{-x} \right \rbrack_{0}^{b} + (z-1) \int_{0}^{\infty} x^{z-2} e^{-x} \, dx \newline
&= 0 + (z-1) \int_{0}^{\infty} x^{z-2} e^{-x} \, dx \newline
&= (z-1)\Gamma(z-1).
\end{align}

Now we prove the formula connecting $B(z, w)$ and $\Gamma(z)$. I'm taking this proof [from Wikipedia](https://en.wikipedia.org/wiki/Beta_function#Relationship_to_the_gamma_function) and filling in details on the variable substitution using [these lecture notes](https://homepage.tudelft.nl/11r49/documents/wi4006/gammabeta.pdf) (Theorem 2, page 3). Note: Curiously, although the Wikipedia article cites Emil Artin's book [Gamma Functions](https://web.archive.org/web/20161112081854/http://www.plouffe.fr/simon/math/Artin%20E.%20The%20Gamma%20Function%20(1931)(23s).pdf), pages 18-19, Artin's proof is not the one written down there.

In any case, the argument begins like this:

\begin{align}
\Gamma(z) \, \Gamma(w) &= \int_{x=0}^{\infty} x^{z-1} e^{-x} \, dx \int_{y=0}^{\infty} y^{w-1} e^{-y} \, dy \newline
&= \int_{x=0}^{\infty} \int_{y=0}^{\infty} e^{-(x+y)} x^{z-1} y^{w-1} \, dx \, dy.
\end{align}

Now we introduce an implicit change of variables $s, t$, defined by the conditions $x = st$ and $y=s(1 - t)$. This transformation has Jacobian matrix:

\begin{align}
J(x(s, t), y(s, t)) = \left(
\begin{array}{cc}
\frac{\partial x}{\partial s} & \frac{\partial x}{\partial t} \newline
\frac{\partial y}{\partial s} & \frac{\partial y}{\partial t}
\end{array} \right) = \left(
\begin{array}{cc}
t & s \newline
1-t & -s
\end{array} \right),
\end{align}

with determinant $\det(J) = -st - s(1-t) = -s$. It follows that

\begin{align}
dx \, dy = \left\| \det{J} \right\| ds \, dt = s \, ds \, dt.
\end{align}

Observe that since $x$ and $y$ range over $[0, \infty)$ and $x + y = st + s(1-t) = s$, then $s$ must range over
$[0, \infty)$, too. On the other hand,

\begin{align}
t = \frac{x}{s} = \frac{x}{x + y},
\end{align}

so $t$ ranges only over the unit interval $[0, 1]$. Making these substitutions gives:

\begin{align}
\Gamma(z) \cdot \Gamma(w) &=
\int_{x=0}^{\infty} \int_{y=0}^{\infty} e^{-(x+y)} x^{z-1} y^{w-1} \, dx \, dy \newline
&= \int_{s=0}^{\infty} \int_{t=0}^{1} e^{-s} (st)^{z-1} (s(1 - t))^{w-1} \, s \, dt \, ds \newline
&= \int_{s=0}^{\infty} \int_{t=0}^{1} e^{-s} s^{z+w-1} t^{z-1}(1 - t)^{w-1} \, dt \, ds \newline
&= \int_{0}^{\infty} s^{z+w-1} e^{-s} \left\{ \int_{0}^{1} t^{z-1}(1 - t)^{w-1} \, dt \, \right\} ds \newline
&= B(z, w) \int_{0}^{\infty} s^{z+w-1} e^{-s} ds \newline
&= B(z, w) \cdot \Gamma(z+w).
\end{align}

Dividing both sides of this equation by $\Gamma(z+w)$ now gives the result.

## 5. Roundup

We specialised the general formulae:

\begin{align}
F_{X_{(k)}}(x) &= \sum_{j=k}^{n} \binom{n}{j} F_{X}(x)^{j} (1 - F_{X}(x))^{n-j}, \newline
f_{X_{(k)}}(x) &= k \binom{n}{k} f_{X}(x) \cdot F_{X}(x)^{k-1} \, (1 - F_{X}(x))^{n-k}, \newline
\mathbb{E} \left \lbrack X_{(k)} \right \rbrack &=
k \binom{n}{k} \int_{0}^{1} x \cdot f_{X}(x) \cdot F_{X}(x)^{k-1} \, (1 - F_{X}(x))^{n-k} \, dx,
\end{align}

which depend only on $n, k, f_{X}$ and $F_{X}$, to the case of the uniform distribution $U(0, 1)$. We showed that:

\begin{align}
F_{X_{(k)}}(x) &= \sum_{j=k}^{n} \binom{n}{j} x^{j} (1 - x)^{n-j}, \newline
f_{X_{(k)}}(x) &= k \binom{n}{k} x^{k-1} \, (1 - x)^{n-k}, \newline
\mathbb{E} \left \lbrack X_{(k)} \right \rbrack &= \frac{k}{n+1} \cdot
\end{align}

To prove the expectation formula, we introduced the beta and gamma function, and proved some identities they satisfy.


# Python Prerequirements for the Deep Learning Workshop

In this notebook we have some exercises that contain concepts largely unrelated to deep learning but are used frequently through-out the workshop. We would like to confirm understanding of these techniques by completing this notebook.

### Section 1: Tuples and Lists

We use tuples frequently as a way to store and receive various lists of arguments. Python has the capability to _destructure_ tuples and lists, and this turns out to often by quite convenient.

Exercise 1: **Constructing tuples**: Tuples are built typically using the `,` operator. For example, `a = 1,2`.

You may like to use the `type` inbuilt function to determine if you're completing these exercises correctly.

- (1.1) Construct a tuple containing two elements.

- (1.2) Construct a tuple containing 1 element.

```python
# TODO:
# Exercise 1.1:

# Exercise 1.2:
```

Exercise 2: **Deconstructing tuples**: (aka _De-tupling_ aka _Unpacking_) Tuples (and lists!) can be _deconstructed_ using standard assignment operators. Destructuring here refers to picking out individual components of the tuple. This can also be performed using standard python indexing, but sometimes destructuring may be more simple.

For the purposes of these exercises, let:

```
a = (1, 2, "Hello")
```

- (2.1) Using only the variable `a` as so-defined, define three new variables that take on each of the values in the tuple.

- (2.2) Again using only the variable `a`, define a single new variable that takes on the value `"Hello"`. (Hint: Oftentimes we will use the special variable name `_` to indicate that we do not care about a value.)

```python
a = (1, 2, "Hello")

# TODO:
# Exercise 2.1:

# Exercise 2.2:
```

Exercise 3: **Lists and Destructuring**: We typically build a list with square brackets, like `a = [1,2]`. Can we do the same de-structuring operations with a list?

- (3.1) Repeat exercises 1.1, 1.2, 2.1 and 2.2 using lists instead of tuples. Does everything just work?

```python
# TODO:

# Exercise 3.1
```

### Section 2: Slicing

Slicing is a great tool. It allows us to access ranges of elements in lists/tuples/arrays in a compact fashion. It also is supported on numpy arrays, which we typically use quite often. We won't attempt to cover _all_ the details of Python slicing here, only the parts that we use in the workshop.

In order to understand slicing, note that in Python (and most programming languages) when we index into an array, we start at `0`. So given:

```
a = [2, 3, 5, 7, 11, 13] # Prime numbers ...
```

Then `a[0] == 2` and `a[2] == 5`.

Exercise 4: **Slicing**.

- (4.1) Confirm the above statements.

- (4.2) We can also use negative numbers to pick out elements starting from the end of the list. What item does `-1` return?

```python
# TODO:

# Exercise 4.1

# Exercise 4.2
```

Exercise 5: **Selecting segments with slicing**: While we can pick out specific values with indexing, slicing lets us pick out specific resgions of our list.

The syntax for a slice is `m:n` for some integers `m` and `n`. We use it like `a[0:2]` to pick out all the elements from `a` starting with `0` up to but not including `2`.

- (5.1) Before running the above slice, can you guess how many elements will be returned? Test your guess.

- (5.2) Try other numbers in the slice operation. What happens? Can you generate an Exception? What happens when you try negative numbers?

```python
# TODO:

# Exercise 5.1

# Exercise 5.2
```

### Section 3: `numpy` Arrays

Numpy is used to manage data coming in and going out of TensorFlow. There are a few concepts in numpy and TensorFlow that will be used frequently. In particular, let us look at the "Shape".

Consider:

```python
import numpy as np

a = np.array([2, 3, 5, 7, 11, 13])
```

Then we can get the _shape_ with `.shape`:

```python
a.shape
```

Here the shape is hinting that the array can actually be multi-dimensional. And indeed, they can, and it turns out to be very useful!

Exercise 6: **Shapes**.

- (6.1) Use the function [`np.ones`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.ones.html) to construct a multi-dimensional array of size `1280x1024x3` (i.e. a standard RGB image of height 1280, width 1024 and 3 colour channels). Check the shape with `.shape`.

- (6.2) *Multi-dimensional slicing* We can perform a slice on this variable, say `b`, like `b[:, :, 1]` to pick out the second RGB dimension (Green) from this faux-image. Confirm this by doing it, and checking the shape. Try out other multi-dimensional slices!

### Section 4: TensorFlow, The Computation Graph & The Session

Here we will not use any deep-learning functionality in TensorFlow, but we will introduce two key ideas.

The first is the _Computation Graph_. This is a global object that we construct indirectly by defining various operations, variables, and placeholders via the TensorFlow API.

For example:

```python
import tensorflow as tf

a = tf.constant(2)
b = tf.constant(9)
c = tf.add(tf.square(a), b)

print(c)
```

The above establishes the computation graph to consider the expression:

$$
\begin{align}
a &= 2 \\
b &= 9 \\
c &= a^2 + b
\end{align}
$$

In diagram form, the flow of data for this expression looks like:

Clearly, TensorFlow is very verbose! And `c` doesn't even take on a concrete value, evidently (we'll get into this when we discuss the _Session_).

We can simplify it a bit (if we are only interested in the value of `c`), by utilising operator overloading:

```python
a = 2
b = 9
c = tf.square(a) + b

print(c)
```

Exercise 7: **Building graphs by using operations**: We refer to the process of definine such expressions as "building the graph". TensorFlow supports many mathematical operations (after all, we're going to do deep learning later!) but for now, stick to the ones you know to build up some interesting TensorFlow graphs. Note that the result of all of the expressions is a _Tensor_.

- (7.1) Define your own mathematical expression using the TensorFlow operations:

    - [tf.add](https://www.tensorflow.org/api_docs/python/tf/add)
    - [tf.square](https://www.tensorflow.org/api_docs/python/tf/square)
    - [tf.subtract](https://www.tensorflow.org/api_docs/python/tf/subtract)

```python
# TODO:

# Exercise 7.1
```

All of this is well-and-good, but we're interested in _evaluating_ such expression. To perform evaluation, we need to use a _Session_. This sets up a manager of all the resources we may need. This tends to be useful if we are performing computations across GPUs, where we don't want to worry about memory-management: TensorFlow takes care of it for us.

Here's an example:

```python
sess = tf.InteractiveSession()

# Note: We could also use the `resource` form here, like:
#
# with tf.Session() as sess:
#     a = 2
#     b = 9
#     c = tf.square(a) + b
#     c_value = sess.run(c)
#
# but it turns out to be more convenient to use the
# `InteractiveSession` one here, as it let's us re-use
# the session between cells.

a = 2
b = 9
c = tf.square(a) + b

c_value = sess.run(c)

print(c_value)
```

Indeed, we've computed the value finally! We often call this "evaluating" the graph. Note that TensorFlow only computes those values (which it calls nodes) that are required for the particular value you are seeking. So for example, `c` depends on `a` and `b`, so those values are required.

Exercise 8: **Sessions and running them**.

- (8.1) Run your own expressions with `sess.run`.

- (8.2) Try defining two different equations for `c` and `d`, then running them with `sess.run([c, d])`. What is the return value here? How does it relate to what we've done with tuples/lists above?

### Section 5: Tensorflow - Feeding in values with `feed_dict`

So now we have the ability to compute arbitrary math functions.

Consider now that we'd like to set up a graph to compute the following expression:

$$
c(x, y) = x^2 + y
$$

Note how this differs notationally from our earlier example. Here we are calling out that `c` depends on `x` and `y`, and that these variables need to be provided. In TensorFlow we accomplish this with the `feed_dict` argument to `sess.run( c, feed_dict=... )`, and [_placeholders_](https://www.tensorflow.org/api_docs/python/tf/placeholder).

Let's see an example of a placeholder:

```python
a = tf.placeholder(tf.int32, shape=(1,))
c = a + 2

a_value, = sess.run(c, feed_dict={ a: (9,)})

print(a_value)
```

Exercise 9: **Placeholders and feed dicts**. We can set up expressions with values that we later fill-in by using placeholders. The `feed_dict` argument to [`Session.run( ... )`](https://www.tensorflow.org/api_docs/python/tf/Session) lets us provide values for the placeholders by constructing a dictionary like `{ placholderVariable: placeholderValue }`. Note that matching up shapes is crucial, and TensorFlow doesn't let us provide a value that isn't in an array of some kind!

- (9.1) Using placeholders, define a graph where the value $c(x,y)$ can be computed by providing the value for two placeholders named `x` and `y`, and compute them for a few different values of `x` and `y` using the feed_dict.

```python
# TODO:

# Exercise 9.1
```

### That's it!

Thanks for completing the pre-requistites! See you at the workshop!

```python

```


```python
%pylab inline
numpy.random.seed(0)

import seaborn; seaborn.set_style('whitegrid')
from tqdm import tqdm_notebook as tqdm
```

    Populating the interactive namespace from numpy and matplotlib

## 1. Submodular Selection
Author: Jacob Schreiber <jmschreiber91@gmail.com>

As an increasing amount of data in the world has enabled increasingly complex machine learning models that perform tasks like translate text between languages, classify objects in videos, and enable precision medicine. However, training these models comes at an increased computational cost. While some of this cost can be mitigated through parallelized implementations and the utilization of GPUs, this hardware costs money that individuals or small organizations don't always have.

Submodular selection is one approach to decreasing the computational cost of training complex models through the selection of a representative subset of the data to instead train models on. This subset is selected in a manner that reduces redundancy in the data set and increased the diversity of the chosen examples. Correspondingly, the models that are trained using this subset almost always outperform random subsets and frequently can achieve optimal performance in only a fraction of the samples.

Briely, the way that submodular search works is that one defines a function that returns approximately how useful a set of samples are, and then greedily optimizes that function. Optimally, this function would be the performance by some machine learning model. One would then be able to identify a minimal set that yielded top performance, where adding additional examples didn't improve the performance of the model. However, because this would require retraining a model over and over again, frequently an approximation of model performance is instead used.

apricot implements two forms of submodular selection: feature based functions and facility location functions. These functions will be described in more detail below, but feature based functions force a diversity of feature values by modeling the saturation of each feature in the growing subset, whereas facility location functions force diversity in the original space by measuring the distance from all points to their nearest representative.

### Feature Based Submodularity

A problem with some submodular functions is that they do not scale well across massive data sets. For example, the facility location function first involves calculating some pairwise information across the data set, and second involves recalculating this pairwise measurement each time a sample is potentially added to the growing set. While optimizations exist that can greatly speed this up when compared to a naive implementation, it still can be prohibitively slow in some cases.

A different type of submodular function, called a "feature based" function, does not involve calculating pairwise information and is much more scalable to large data sets. These functions involve calculating how "saturated" a particular feature is and preferentially selecting samples that have large feature values that have not yet been seen. These function takes the following form:

\begin{equation}
f(X) = \sum\limits_{u \in U} w_{u} \phi_{u} \left( \sum\limits_{x \in X} m_{u}(x) \right)
\end{equation}

In this equation, $U$ refers to all features, or dimensions, of a sample, and $u$ refers to a specific feature. $X$ refers to the original data set that we are selecting from and $x$ refers to a single sample from that data set. $w$ is a vector of weights that indicate how important each feature is, with $w_{u}$ being a scalar referring to how important feature $u$ is. Frequently these weights are uniform. $\phi$ refers to a set of saturating functions, such as $sqrt(X)$ or $log(X + 1)$ that take some

Feature based functions perform best when the value of each feature corresponds roughly to some notion of how important it is rather than being an arbitrary value. For example, when considering audio data, a spike in noise indicates that something is happening in an otherwise silent background, and when considering financial portfolios the more money a person has in a stock or fund, the more important that fund is for them. Examples of features where feature based functions are not likely to work well on are those without this notion, including categorical variables such as car color or bimodally distributed variables.

The reason that feature based funtions don't perform well when thre is no notion of importance can easily be visualized by looking at two Gaussian blobs. One might expect that a good summarization function would select a similar number of points from each blob, perhaps near the boundary between the two. Let's see what a feature based function does.

```python
from apricot import FeatureBasedSelection

X = numpy.concatenate([numpy.random.normal(5, 1, size=(50, 2)), numpy.random.normal(9, 1, size=(35, 2))])
Xi = FeatureBasedSelection(10).fit_transform(X)

plt.figure(figsize=(8, 6))
plt.title("Submodular selection on Gaussian blobs", fontsize=16)
plt.scatter(X[:,0], X[:,1])
plt.scatter(Xi[:,0], Xi[:,1], label="Selected Samples")
plt.legend(fontsize=14)
plt.show()
```

The feature based selection chooses those samples with the highest absolute magnitude. This is not surprising because the function is trying to maximize the sum of the saturated values over each feature. You can maximize this value by always selecting the biggest values.

#### 20 newsgroups Data Set

The setting in which feature based functions perform the best is when the value of the feature corresponds to a quantity or a significance, where a higher value of that feature corresponds to having a large amount of that feature. A natural example of this is using word counts or tf-idf to represent sentences where each feature is a word. Let's see that in action here with an example from the 20newsgroups data set.

```python
from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import TfidfVectorizer

cats = ('sci.med', 'sci.space')
stop = ('headers', 'footers', 'quotes')

train_data = fetch_20newsgroups(subset='train', categories=cats, remove=stop)
test_data = fetch_20newsgroups(subset='test', categories=cats, remove=stop)

X_train, y_train = train_data.data, train_data.target
X_test, y_test = test_data.data, test_data.target

vectorizer = TfidfVectorizer(stop_words='english', max_features=1000)

X_train = vectorizer.fit_transform(X_train).toarray()
X_test = vectorizer.transform(X_test).toarray()

X_train.shape, X_test.shape
```

    ((1187, 1000), (790, 1000))

We've downloading 1,187 samples for our training set and 790 for our test set. This is just a toy example and so training on the entire training set is feasible, but let's see how good models trained using small subsets can perform when using random selection versus using a feature based function. We'll also perform our random selection 20 times, showing the best, worst, and average performance across the 20 times.

```python
from sklearn.linear_model import LogisticRegressionCV

submodular_accuracy, random_accuracy = [], []

selection = FeatureBasedSelection(1000)
selection.fit(X_train)

numpy.random.seed(0)
random_idxs = []
for i in range(20):
    idxs = numpy.arange(X_train.shape[0])
    numpy.random.shuffle(idxs)
    random_idxs.append(idxs)

x = [20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000]

for n in tqdm(x):
    Xi, yi = selection.transform(X_train, y_train)
    Xi, yi = Xi[:n], yi[:n]

    model = LogisticRegressionCV().fit(Xi, yi)
    submodular_accuracy.append(model.score(X_test, y_test))

    accs = []
    for i in range(20):
        idx = random_idxs[i][:n]
        Xi, yi = X_train[idx], y_train[idx]

        model = LogisticRegressionCV().fit(Xi, yi)
        accs.append(model.score(X_test, y_test))

    random_accuracy.append(accs)

random_accuracy = numpy.array(random_accuracy)
```

    A Jupyter Widget

```python
plt.title("20 newsgroups space vs medicine articles", fontsize=14)
plt.scatter(x, submodular_accuracy, label="Feature Based Selection")
plt.scatter(x, random_accuracy.mean(axis=1), color='m', label="Random Selection")

plt.plot(x, submodular_accuracy)
plt.plot(x, random_accuracy.mean(axis=1), color='m')
plt.plot(x, random_accuracy.min(axis=1), alpha=0.3, color='m')
plt.plot(x, random_accuracy.max(axis=1), alpha=0.3, color='m')

plt.fill_between(x, random_accuracy.min(axis=1), random_accuracy.max(axis=1), color='m', alpha=0.2)
plt.xlabel("Samples Used", fontsize=14)
plt.ylabel("Accuracy", fontsize=14)
plt.legend(fontsize=12)
plt.xlim(0, 1050)
plt.savefig("img/20newsgroups.png")
plt.show()
```

It looks like submodular selection through a feature based method distinctly outperforms using random selection for a small number of samples. Using only 100 samples selected using submodular selection one can achieve high accuracy.

### Facility Location

Another approach to subset selection comes in the form of facility location functions. These functions operate primarily on pairwise similarity scores between two samples rather than their feature values directly. Because of this, the feature values of the samples can be negative as long as the similarity score is strictly non-negative.

Facility location functions are named because an early use was in placing facilities for companies, such as distribution warehouses or stores. The question was, "if we have facilities at some locations, what is the next best location to put a facility?" The answer is typically an underserved region. This results in a greedy selection algorithm where the first facility is at a central location and then subsequent facilities are spaced out based on where locations exist.

```python
from apricot import FacilityLocationSelection

numpy.random.seed(0)
X = numpy.concatenate([numpy.random.normal((1, 1), 0.5, size=(15, 2)),
                       numpy.random.normal((6, 3), 0.5, size=(25, 2)),
                       numpy.random.normal((5, 7), 0.5, size=(40, 2)),
                       numpy.random.normal((1, 7), 0.5, size=(30, 2)),
                       numpy.random.normal((10, 4), 0.5, size=(15, 2)),
                       numpy.random.normal((3, 4), 0.5, size=(15, 2))])

Xi = FacilityLocationSelection(6, 'euclidean').fit_transform(X)
Xr = numpy.random.choice(numpy.arange(X.shape[0]), size=6)
Xr = X[Xr]

plt.figure(figsize=(8, 6))
plt.title("Facility location for exemplar identification", fontsize=16)
plt.scatter(X[:,0], X[:,1], s=10)
plt.scatter(Xi[:,0], Xi[:,1], color="#FF6600", label="Submodular Selection")
plt.scatter(Xr[:,0], Xr[:,1], color="#8A2BE2", label="Random Selection", alpha=0.6)
plt.legend(fontsize=14, loc=1)
plt.xlim(-1, 14)
plt.axis('off')
plt.show()
```

In this example an example of selecting representative points from six clusters using either a facility location function or random selection. You can see immediately that the points are more representative of the total space than a feature based function would be. A random selection of points is unlikely to sample each of the clusters only once, and this skew worsens with the skew of samples per cluster, whereas a facility location function is more stable towards that. An animation of the selection process can give some insight into how it works.

The first selected sample is at a central location because it is representative of the full data set. In the context where facility location is actually selection store locations, this would be the optimal location to put a single store to service everyone. The second location is in the largest cluster of points but skewed towards the second largest cluster of points. At this point something interesting happens: the next selected samples seem to be central to the cluster they're selected in. The third location is at a central location in a cluster of samples that is furthest from the selected items, and thus least served. The next three points are also at the center of their respective clusters. Essentially, once the space is sparsely covered then the next selected sample can focus on serving a local neighborhood.

Facility location can be thought of as a greedy version of k-means clustering. K-means is an iterative procedure that would've resulted in the centroids lying at the center of each of the clusters above. However, it would be significantly more time intensive to calculate these. Correspondingly, facility location functions can be a good approximation of k-means clustering, and sometimes are even used as an initialization to the k-means algorithm.

#### Digits Data Set

We can use facility location for sample selection for machine learning algorithms as well. In fact, it works better than feature based methods in cases where the features do not have the nice properties seen before. Let's load up the digits data set. A feature based function would try to select samples that had a diversity of high pixel values. That's not exactly what we want because the digit can exist in several places in the image with different sizes and thicknesses.

```python
from sklearn.datasets import load_digits

data = load_digits()
X, y = data.data, data.target

idx = numpy.arange(X.shape[0])
numpy.random.shuffle(idx)

X, y = X[idx], y[idx]
X_train, y_train = X[:1250], y[:1250]
X_test, y_test = X[1250:], y[1250:]
X_train.shape, X_test.shape
```

    ((1250, 64), (547, 64))

Let's take 1,250 samples for the training set and 547 samples for the test set. We can try using a facility location function on the data that calculates the euclidean distance between samples and compare that both to random selection and to using a feature based selection algorithm.

```python
selector = FacilityLocationSelection(X_train.shape[0], 'euclidean', verbose=True)
selector.fit(X_train)
r1 = selector.ranking

selector2 = FeatureBasedSelection(X_train.shape[0], verbose=True)
selector2.fit(X_train)
r2 = selector2.ranking
```

      0%|          | 0/1250 [00:00<?, ?it/s]
      0%|          | 0/1250 [00:00<?, ?it/s]Exception KeyError: KeyError(<weakref at 0x7f120df1dd08; to 'tqdm' at 0x7f1205790c50>,) in <bound method tqdm.__del__ of   0%|          | 1/1250 [00:00<00:40, 30.81it/s]> ignored

      1%|          | 12/1250 [00:00<00:10, 113.90it/s]
     46%|████▌     | 570/1250 [00:00<00:04, 161.30it/s]
    100%|██████████| 1250/1250 [00:00<00:00, 3771.99it/s]

Now let's try training logistic regression models like before in these settings.

```python
from sklearn.linear_model import LogisticRegression

numpy.random.seed(0)
random_idxs = []
for i in range(20):
    idxs = numpy.arange(X_train.shape[0])
    numpy.random.shuffle(idxs)
    random_idxs.append(idxs)

fl_acc, fb_acc, random_acc = [], [], []
n_samples = range(25, 251, 25) + range(300, 1251, 100)

for n in tqdm(n_samples):
    Xi, yi = selector.transform(X_train, y_train)
    model = LogisticRegression().fit(Xi[:n], yi[:n])
    fl_acc.append(model.score(X_test, y_test))

    Xi, yi = selector2.transform(X_train, y_train)
    model = LogisticRegression().fit(Xi[:n], yi[:n])
    fb_acc.append(model.score(X_test, y_test))

    accs = []
    for i in range(20):
        idx = random_idxs[i][:n]
        Xi, yi = X_train[idx], y_train[idx]

        model = LogisticRegression().fit(Xi, yi)
        accs.append(model.score(X_test, y_test))

    random_acc.append(accs)

random_acc = numpy.array(random_acc)
```

    A Jupyter Widget

```python
plt.title("MNIST Performance using Facility Location", fontsize=14)
plt.scatter(n_samples, fl_acc, c='b', label="Facility Location")
plt.scatter(n_samples, fb_acc, c='g', label="Feature Based")
plt.scatter(n_samples, random_acc.mean(axis=1), color='m', label="Random Selection")

plt.plot(n_samples, fl_acc, c='b')
plt.plot(n_samples, fb_acc, c='g')
plt.plot(n_samples, random_acc.mean(axis=1), color='m')
plt.plot(n_samples, random_acc.min(axis=1), alpha=0.3, color='m')
plt.plot(n_samples, random_acc.max(axis=1), alpha=0.3, color='m')

plt.fill_between(n_samples, random_acc.min(axis=1), random_acc.max(axis=1), color='m', alpha=0.2)
plt.xlabel("Number of Selected Points", fontsize=14)
plt.ylabel("Accuracy", fontsize=14)
plt.legend(fontsize=12)
plt.show()
```

It looks like the facility location function outperforms both the feature based function and the random selection. Similar to the case of the feature based function before the model performance appears to saturate with just a small percentage of the samples. In fact, the feature based function performs worse than the random selection. This is not surprising in the case where optimizing a diversity of features irrespective of the structure of the features may not be the best idea.

tutorials/1. Introduction to Submodular Selection.ipynb