[Theory paper](https://arxiv.org/pdf/1710.09839.pdf)
[GAIA paper](https://www.aanda.org/articles/aa/full_html/2016/11/aa29272-16/aa29272-16.html)

Note about units: The default unit for length is parsec, for mass is a solar mass, and for time is years.

```python
import sympy
sympy.init_printing()
import numpy
import pylab
import time
from sklearn.manifold import TSNE
```

Progress bar

```python
def log_progress(sequence, every=None, size=None, name='Items'):
    from ipywidgets import IntProgress, HTML, VBox
    from IPython.display import display

    is_iterator = False
    if size is None:
        try:
            size = len(sequence)
        except TypeError:
            is_iterator = True
    if size is not None:
        if every is None:
            if size <= 200:
                every = 1
            else:
                every = int(size / 200)     # every 0.5%
    else:
        assert every is not None, 'sequence is iterator, set every'

    if is_iterator:
        progress = IntProgress(min=0, max=1, value=1)
        progress.bar_style = 'info'
    else:
        progress = IntProgress(min=0, max=size, value=0)
    label = HTML()
    box = VBox(children=[label, progress])
    display(box)

    index = 0
    try:
        for index, record in enumerate(sequence, 1):
            if index == 1 or index % every == 0:
                if is_iterator:
                    label.value = '{name}: {index} / ?'.format(
                        name=name,
                        index=index
                    )
                else:
                    progress.value = index
                    label.value = u'{name}: {index} / {size}'.format(
                        name=name,
                        index=index,
                        size=size
                    )
            yield record
    except:
        progress.bar_style = 'danger'
        raise
    else:
        progress.bar_style = 'success'
        progress.value = index
        label.value = "{name}: {index}".format(
            name=name,
            index=str(index or '?')
        )
```

[Salpeter initial mass function](https://en.wikipedia.org/wiki/Initial_mass_function)

```python
M_star = sympy.Symbol('M_*', positive=True)
M_star_min = sympy.Symbol('M_{*min}', positive=True)
M_star_max = sympy.Symbol('M_{*max}', positive=True)
a_salpeter = sympy.Symbol('a_{salpeter}', positive=True)
C_salpeter = sympy.Symbol('C_{salpeter}', positive=True)
salpeter_imf = C_salpeter*M_star**(-a_salpeter)
salpeter_imf
```

```python
temp = salpeter_imf
temp = sympy.integrate(temp, M_star, conds='none')
C_salpeter_expr = sympy.solve(temp.subs(M_star, M_star_max) - temp.subs(M_star, M_star_min)-1, C_salpeter)[0]
C_salpeter_expr
```

```python
temp = salpeter_imf.subs(C_salpeter, C_salpeter_expr)
temp = sympy.integrate(temp, M_star, conds='none')
temp = temp - temp.subs(M_star, M_star_min)
cumulative_salpeter_imf = temp.simplify()
cumulative_salpeter_imf
```

What is the probability that a star is born with a mass in excess of 20 $M_{\odot}$?

```python
1-cumulative_salpeter_imf.subs({
    M_star: 20,
    M_star_min:0.08,
    M_star_max:100,
    a_salpeter:2.35
})
```

Distribution function for masses above 20 $M_{\odot}$

```python
xi = sympy.Symbol('xi', positive=True)
f = sympy.Symbol('f', positive=True)
temp = sympy.Eq(f, cumulative_salpeter_imf)
temp = sympy.solve(temp, M_star)[0].simplify()
temp = sympy.expand_power_base(temp.subs(M_star_max, xi*M_star_min), force=True).simplify()
temp = sympy.expand_power_base(temp, force=True).simplify()
temp = temp.subs(xi, M_star_max/M_star_min)
temp.subs({a_salpeter: 2.35,
           M_star_min: 20,
           M_star_max: 100
          })
```

This is a simplified version of the IMFs used in the theory paper (equation 1 and 2 in theory paper)

```python
def randomise_salpeter_bh_progenitor_mass(l):

    f = numpy.random.rand(l)

    return 20.0/(1.0-0.886*f)**0.741
```

Mass ratio (equation 3)

```python
def randomise_mass_ratio(M1_list):

    q_min = 0.08/M1_list
    f_list = numpy.random.rand(len(M1_list))
    return q_min + f_list*(1-q_min)
```

Semi major axis distribution (equation 6)

```python
def randomise_semi_major_axis(M1_list):

    f_list = numpy.random.rand(len(M1_list))
    A_min = 1e-8
    A_max = 1e-2
    #log10A_list = 8.0*(f_list - 1)
    return A_min*(A_max/A_min)**f_list
```

Position in the galaxy (equation 11)

```python
def randomise_height_above_galactic_disc(M1_list):

    f_list = numpy.random.rand(len(M1_list))
    return -250*numpy.log(1-f_list)
```

```python
def randomise_distance_from_galactic_centre(M1_list):

    f_list = numpy.random.rand(len(M1_list))
    return -3500*numpy.log(1-f_list)
```

Angle around the galactic rotation axis, relative to the line connecting the galactic centre to the sun

```python
def randomise_angle_gc_sol(M1_list):

    return numpy.pi*numpy.random.rand(len(M1_list))
```

Relation between mass and radius, using the empiric relation from [Demircan and Kahraman (1991)](http://adsabs.harvard.edu/abs/1991Ap%26SS.181..313D), page 319

```python
def mass2radius_demircan_kahraman(m):

    return 1.6*m**0.83*2.3e-8
```

Calculation of the ratio between the Roche lobe radius and the radius of the donor, according to the mass ratio $q$, using (equation 7)

```python
def roche_lobe_radius_ratio(q):

    return (0.6*q**-0.67+numpy.log(1+q**-0.33))/(0.49*q**-0.67)
```

Terminal semi major axis for binaries with comparable masses (equation 20)

```python
def calc_current_semi_major_axis_ratio_massive(q_list, k_list):

    a = 1.0
    b = 0.5
    c1 = a*(1-b)-2
    c2 = -a*(1-b)/b-2
    return (k_list+b*(1-k_list)+q_list)/(1+q_list)*k_list**c1*(b*(1-k_list)/q_list+1)**c2
```

Terminal semi major axis for binaries with large mass ratio (equation 23)

```python
def calc_current_semi_major_axis_ratio_wimpy(q_list, k_list, alphalambda=1.0):

    rl_list = 1.0/ roche_lobe_radius_ratio(q_list)

    return k_list/(2.0*(1.0-k_list)/(alphalambda*rl_list*q_list)+1.0)
```

Terminal semi major axis for both cases

```python
def calc_current_semi_major_axis(q_list, k_list, isma_list, alphalambda=1.0):

    return isma_list*numpy.where(
        q_list>0.5,
        calc_current_semi_major_axis_ratio_massive(q_list, k_list),
        calc_current_semi_major_axis_ratio_wimpy(q_list, k_list, alphalambda=alphalambda)
                               )
```

Relation between progenitor and black hole mass (equation 19)

```python
def calc_terminal_black_hole_mass(M1_list):

    return 8.0*numpy.log(M1_list - 19)/numpy.log(81.0)+2
```

Stellar classification

Based on [this table](http://www.pas.rochester.edu/~emamajek/EEM_dwarf_UBVIJHK_colors_Teff.txt)

Parse table

```python
with open('EEM_dwarf_UBVIJHK_colors_Teff.txt') as f:
    data = f.readlines()
table_lines = []
important_flag = False

for line in data:
    if '#SpT' in line:
        important_flag = not important_flag
        continue
    if important_flag:
        table_lines.append(line)

table_mass = []
table_vmag = []
for line in table_lines:
    mass_candidate_text = line.split()[19]
    mv_candidate_text = line.split()[4]
    if '...' in mass_candidate_text or '...' in mv_candidate_text:
        continue
    table_mass.append(float(mass_candidate_text))
    table_vmag.append(float(mv_candidate_text))
table_mass = numpy.array(table_mass)
table_vmag = numpy.array(table_vmag)
```

```python
def stellar_mass_to_vmag(mass):
    return numpy.interp(-mass, -table_mass, table_vmag)
%matplotlib inline
pylab.semilogx(table_mass, table_vmag,'.')
pylab.semilogx(table_mass, stellar_mass_to_vmag(table_mass));
```

```python
def absolute_to_apparent_magnitude(absolute, d_pc, extinction=False):
    if extinction:
        return absolute+5*(numpy.log10(d_pc)-1)+d_pc/1e3
    return absolute+5*(numpy.log10(d_pc)-1)
```

Gaia Sensitivity (section 8 in Gaia paper)

```python
def calc_gaia_z12p09(gmag):

    capped_gmag = numpy.clip(gmag, 12, 200)
    return numpy.clip(10**(0.4*(capped_gmag-15)),10**(0.4*(12.09-15)),1e30)

def calc_sigma_pomega(gmag):

    z = calc_gaia_z12p09(gmag)
    return numpy.sqrt(-1.63+680.8*z+32.7*z**2)

def calc_sigma_G(gmag):

    z = calc_gaia_z12p09(gmag)
    return 1.2e-3*numpy.sqrt(0.05*z**2+1.9*z+0.0002)
```

Synthetic stellar population

```python
ssp = {'primary mass':randomise_salpeter_bh_progenitor_mass(int(1e6))}
ssp['initial mass ratio'] = randomise_mass_ratio(ssp['primary mass'])
ssp['initial companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['initial semi major axis'] = randomise_semi_major_axis(ssp['primary mass'])
ssp['z'] = randomise_height_above_galactic_disc(ssp['primary mass'])
ssp['r'] = randomise_distance_from_galactic_centre(ssp['primary mass'])
ssp['angle'] = randomise_angle_gc_sol(ssp['primary mass'])
ssp['black hole mass'] = calc_terminal_black_hole_mass(ssp['primary mass'])
#ssp['black hole mass'] = 0.2*ssp['primary mass']
ssp['terminal companion mass'] = (ssp['initial companion mass'] +
                                  numpy.where(ssp['initial mass ratio']>0.5,
                                              0.5*(ssp['primary mass']-ssp['black hole mass']),
                                              numpy.zeros_like(ssp['primary mass'])))
ssp['terminal mass ratio'] = ssp['black hole mass']/ssp['terminal companion mass']
ssp['age'] = numpy.random.rand(len(ssp['primary mass'])) # Age / galactic age
ssp['lifetime'] = ssp['terminal companion mass']**-2.5 - ssp['primary mass']**-2.5 # Normalised by the age of the galaxy
ssp['companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['companion radius'] = mass2radius_demircan_kahraman(ssp['companion mass'])
ssp['initial primary radius'] = mass2radius_demircan_kahraman(ssp['primary mass'])
ssp['initial roche radius'] = roche_lobe_radius_ratio(ssp['initial mass ratio'])*ssp['initial primary radius']
ssp['terminal roche radius'] = roche_lobe_radius_ratio(1.0/ssp['terminal mass ratio'])*ssp['companion radius']
ssp['terminal semi major axis'] = calc_current_semi_major_axis(
    ssp['initial mass ratio'],
    ssp['black hole mass']/ssp['primary mass'],
    ssp['initial semi major axis'],
    alphalambda=1.0
)
ssp['terminal period'] = 9.4e7*(ssp['terminal semi major axis']**1.5*
                                (ssp['companion mass']+ssp['black hole mass'])**(-0.5))
ssp['companion absolute magnitude'] = stellar_mass_to_vmag(ssp['primary mass']*ssp['initial mass ratio'])
ssp['distance from earth'] = numpy.sqrt(ssp['z']**2+8000**2+ssp['r']**2-2*8000*ssp['r']*numpy.cos(ssp['angle']))
ssp['companion apparent magnitude'] = absolute_to_apparent_magnitude(ssp['companion absolute magnitude'],
                                                                     ssp['distance from earth'],
                                                                    extinction=True)
ssp['sigma pomega'] = calc_sigma_pomega(ssp['companion apparent magnitude'])
ssp['parallax'] = 1e6/ssp['distance from earth'] # in uas
ssp['sigma G'] = calc_sigma_G(ssp['companion apparent magnitude'])
ssp['statistical weight'] = numpy.clip(ssp['lifetime'], 0, 1)
ssp_no_kick = ssp
```

Count how many will be visible by Gaia

```python
filter_list = [ssp['terminal semi major axis']>ssp['terminal roche radius'],
               ssp['terminal period']<5,
               ssp['terminal period']>0.137,
               #ssp['initial semi major axis']<3.2e-5,
               ssp['black hole mass']>3.0/(1.0-ssp['sigma pomega']/ssp['parallax']-ssp['sigma G']-ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
               ssp['companion apparent magnitude']<20,
               ssp['terminal semi major axis']>10*(1+ssp['initial mass ratio']/0.2)*ssp['sigma pomega']*ssp['distance from earth']*5e-12
              ]
mask = numpy.all(filter_list,axis=0)
gaia_no_kick_mask = mask
ssp['statistical weight'][mask].sum()
```

Statistics of detected black holes - luminous companion binaries

```python
%matplotlib inline
import matplotlib
font = {'size':22}
matplotlib.rc('font',**font)

pylab.hist(ssp['black hole mass'][mask],20, weights=ssp['statistical weight'][mask])
pylab.xlabel('Black hole mass [$M_{\odot}$]');
pylab.ylabel('Number')
pass
```

```python
%matplotlib inline
import matplotlib
font = {'size':22}
matplotlib.rc('font',**font)

pylab.hist(ssp['initial semi major axis'][mask],20, weights=ssp['statistical weight'][mask])
#pylab.xlabel('Black hole mass [$M_{\odot}$]');
pylab.ylabel('Number')
pylab.xscale('log')
pass
```

```python
pylab.hist(ssp['terminal period'][mask],20, weights=ssp['statistical weight'][mask])
pylab.xlabel('Terminal period [years]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['distance from earth'][mask]/1000,20, weights=ssp['statistical weight'][mask])
pylab.xlabel('Distance from earth [kpc]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['companion mass'][mask],20, weights=ssp['statistical weight'][mask])
pylab.xscale('log')
pylab.xlabel(r'Companion mass [$M_{\odot}$]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['terminal semi major axis'][mask]/ssp['distance from earth'][mask]/4.8e-9,50,
          weights=ssp['statistical weight'][mask])
pylab.xscale('log')
pylab.xlabel(r'Angular separation [mas]')
pylab.ylabel('Number')
pass
```

```python
pylab.scatter(ssp['companion mass'][mask],
              ssp['terminal semi major axis'][mask]/ssp['distance from earth'][mask]/4.8e-9
             )
pylab.xscale('log')
pylab.yscale('log')
pylab.xlabel('Companion mass[$M_{\odot}$]')
pylab.ylabel('Angular separation [mas]')
pass
```

Count how many will be seen by [Hipparcos](https://arxiv.org/pdf/1202.2064.pdf)

```python
filter_list = [ssp['terminal semi major axis']>ssp['terminal roche radius'],
               ssp['terminal period']<5,
               ssp['terminal period']>0.137,
               ssp['initial semi major axis']<3.2e-5,
               ssp['black hole mass']>
               3.0/(1.0-100*ssp['sigma pomega']/ssp['parallax']-
                    100*ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
               ssp['companion apparent magnitude']<12.4,
               ssp['terminal semi major axis']>
               10*(1+ssp['initial mass ratio']/0.2)*100*ssp['sigma pomega']*ssp['distance from earth']*5e-12
              ]
mask = numpy.all(filter_list, axis=0)
ssp['statistical weight'][mask].sum()
```

# Inefficient Common Envelope

```python
ssp = {'primary mass':randomise_salpeter_bh_progenitor_mass(int(1e6))}
ssp['initial mass ratio'] = randomise_mass_ratio(ssp['primary mass'])
ssp['initial companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['initial semi major axis'] = randomise_semi_major_axis(ssp['primary mass'])
ssp['z'] = randomise_height_above_galactic_disc(ssp['primary mass'])
ssp['r'] = randomise_distance_from_galactic_centre(ssp['primary mass'])
ssp['angle'] = randomise_angle_gc_sol(ssp['primary mass'])
ssp['black hole mass'] = calc_terminal_black_hole_mass(ssp['primary mass'])
#ssp['black hole mass'] = 0.2*ssp['primary mass']
ssp['terminal companion mass'] = (ssp['initial companion mass'] +
                                  numpy.where(ssp['initial mass ratio']>0.5,
                                              0.5*(ssp['primary mass']-ssp['black hole mass']),
                                              numpy.zeros_like(ssp['primary mass'])))
ssp['terminal mass ratio'] = ssp['black hole mass']/ssp['terminal companion mass']
ssp['age'] = numpy.random.rand(len(ssp['primary mass'])) # Age / galactic age
ssp['lifetime'] = ssp['terminal companion mass']**-2.5 - ssp['primary mass']**-2.5 # Normalised by the age of the galaxy
ssp['companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['companion radius'] = mass2radius_demircan_kahraman(ssp['companion mass'])
ssp['initial primary radius'] = mass2radius_demircan_kahraman(ssp['primary mass'])
ssp['initial roche radius'] = roche_lobe_radius_ratio(ssp['initial mass ratio'])*ssp['initial primary radius']
ssp['terminal roche radius'] = roche_lobe_radius_ratio(1.0/ssp['terminal mass ratio'])*ssp['companion radius']
ssp['terminal semi major axis'] = calc_current_semi_major_axis(
    ssp['initial mass ratio'],
    ssp['black hole mass']/ssp['primary mass'],
    ssp['initial semi major axis'],
    alphalambda=0.1
)
ssp['terminal period'] = 9.4e7*(ssp['terminal semi major axis']**1.5*
                                (ssp['companion mass']+ssp['black hole mass'])**(-0.5))
ssp['companion absolute magnitude'] = stellar_mass_to_vmag(ssp['primary mass']*ssp['initial mass ratio'])
ssp['distance from earth'] = numpy.sqrt(ssp['z']**2+8000**2+ssp['r']**2-2*8000*ssp['r']*numpy.cos(ssp['angle']))
ssp['companion apparent magnitude'] = absolute_to_apparent_magnitude(ssp['companion absolute magnitude'],
                                                                     ssp['distance from earth'],
                                                                    extinction=True)
ssp['sigma pomega'] = calc_sigma_pomega(ssp['companion apparent magnitude'])
ssp['parallax'] = 1e6/ssp['distance from earth'] # in uas
ssp['sigma G'] = calc_sigma_G(ssp['companion apparent magnitude'])
ssp['statistical weight'] = numpy.clip(ssp['lifetime'], 0, 1)
ssp_inefficient = ssp
```

Count how many will be detected by gaia

```python
filter_list = [ssp['terminal semi major axis']>ssp['terminal roche radius'],
               ssp['terminal period']<5,
               ssp['terminal period']>0.137,
               ssp['black hole mass']>3.0/(1.0-ssp['sigma pomega']/ssp['parallax']-ssp['sigma G']-ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
               ssp['companion apparent magnitude']<20,
               ssp['terminal semi major axis']>10*(1+ssp['initial mass ratio']/0.2)*ssp['sigma pomega']*ssp['distance from earth']*5e-12
              ]
mask = numpy.all(filter_list,axis=0)
gaia_inefficient_mask = mask
ssp['statistical weight'][mask].sum()
```

Count how many will be detected by hipparcos

```python
filter_list = [ssp['terminal semi major axis']>ssp['terminal roche radius'],
               ssp['terminal period']<5,
               ssp['terminal period']>0.137,
               ssp['black hole mass']>
               3.0/(1.0-100*ssp['sigma pomega']/ssp['parallax']-
                    100*ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
               ssp['companion apparent magnitude']<12.4,
               ssp['terminal semi major axis']>
               10*(1+ssp['initial mass ratio']/0.2)*100*ssp['sigma pomega']*ssp['distance from earth']*5e-12
              ]
mask = numpy.all(filter_list, axis=0)
ssp['statistical weight'][mask].sum()
```

# Natal Kicks

## Same velocity as neutron stars

```python
def kick_semi_major_axis_ratio(vk_list, vi_list, chi, mu_list):

    y = vk_list/vi_list
    return 2-chi*(1+2*mu_list*y+y**2)
```

```python
def kick_eccentricity(vk_list, vi_list, chi, mu_list, phi_list):

    sma_ratio = kick_semi_major_axis_ratio(vk_list,
                                          vi_list,
                                          chi,
                                          mu_list)
    mu = mu_list
    y = vk_list/vi_list
    temp = (1+mu*y)**2-(mu**2-1)*y**2*numpy.sin(phi_list)**2

    return numpy.sqrt(1-sma_ratio*temp)
```

```python
ssp = {'primary mass':randomise_salpeter_bh_progenitor_mass(int(1e6))}
ssp['initial mass ratio'] = randomise_mass_ratio(ssp['primary mass'])
ssp['initial companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['initial semi major axis'] = randomise_semi_major_axis(ssp['primary mass'])
ssp['z'] = randomise_height_above_galactic_disc(ssp['primary mass'])
ssp['r'] = randomise_distance_from_galactic_centre(ssp['primary mass'])
ssp['angle'] = randomise_angle_gc_sol(ssp['primary mass'])
ssp['black hole mass'] = calc_terminal_black_hole_mass(ssp['primary mass'])
ssp['age'] = numpy.random.rand(len(ssp['primary mass'])) # Age / galactic age
ssp['companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['companion radius'] = mass2radius_demircan_kahraman(ssp['companion mass'])
ssp['initial primary radius'] = mass2radius_demircan_kahraman(ssp['primary mass'])
ssp['initial roche radius'] = roche_lobe_radius_ratio(ssp['initial mass ratio'])*ssp['initial primary radius']
ssp['terminal companion mass'] = (ssp['initial companion mass'] +
                                  numpy.where(ssp['initial mass ratio']>0.5,
                                              0.5*(ssp['primary mass']-ssp['black hole mass']),
                                              numpy.zeros_like(ssp['primary mass'])))
ssp['pre kick semi major axis'] = calc_current_semi_major_axis(
    ssp['initial mass ratio'],
    ssp['black hole mass']/ssp['primary mass'],
    ssp['initial semi major axis'],
    alphalambda=1.0
)
ssp['terminal mass ratio'] = ssp['black hole mass']/ssp['terminal companion mass']
ssp['lifetime'] = ssp['terminal companion mass']**-2.5 - ssp['primary mass']**-2.5 # Normalised by the age of the galaxy
ssp['terminal roche radius'] = roche_lobe_radius_ratio(1.0/ssp['terminal mass ratio'])*ssp['companion radius']

# Kicks
ssp['kick mu'] = numpy.random.rand(len(ssp['primary mass']))*2-1
ssp['kick phi'] = numpy.random.rand(len(ssp['primary mass']))*numpy.pi
ssp['kick v_k'] = numpy.absolute(numpy.random.normal(0, 265, len(ssp['primary mass']))) # km/s
ssp['kick v_i'] = 0.07*numpy.sqrt((ssp['primary mass'] + ssp['companion mass'])/ssp['pre kick semi major axis']) # km/s
ssp['kick chi'] = (ssp['primary mass']+ssp['initial companion mass'])/(ssp['black hole mass'] + ssp['terminal companion mass'])

ssp['terminal semi major axis'] = ssp['pre kick semi major axis']/(
    kick_semi_major_axis_ratio(ssp['kick v_k'],
                               ssp['kick v_i'],
                               ssp['kick chi'],
                               ssp['kick mu'])
)

ssp['terminal eccentricity'] = kick_eccentricity(ssp['kick v_k'],
                                                 ssp['kick v_i'],
                                                 ssp['kick chi'],
                                                 ssp['kick mu'],
                                                 ssp['kick phi'])

ssp['terminal period'] = 9.4e7*(ssp['terminal semi major axis']**1.5*
                                ssp['primary mass']**-0.5*
                                (0.2+ssp['initial mass ratio'])**-0.5)
ssp['companion absolute magnitude'] = stellar_mass_to_vmag(ssp['primary mass']*ssp['initial mass ratio'])
ssp['distance from earth'] = numpy.sqrt(ssp['z']**2+8000**2+ssp['r']**2-2*8000*ssp['r']*numpy.cos(ssp['angle']))
ssp['companion apparent magnitude'] = absolute_to_apparent_magnitude(ssp['companion absolute magnitude'],
                                                                     ssp['distance from earth'],
                                                                    extinction=True)
ssp['sigma pomega'] = calc_sigma_pomega(ssp['companion apparent magnitude'])
ssp['parallax'] = 1e6/ssp['distance from earth'] # in uas
ssp['sigma G'] = calc_sigma_G(ssp['companion apparent magnitude'])
ssp['statistical weight'] = numpy.clip(ssp['lifetime'],0,1)
ssp_kick_velocity=ssp
```

    /usr/local/lib/python2.7/dist-packages/ipykernel_launcher.py:49: RuntimeWarning: invalid value encountered in power

Count how many will be seen by hipparcos

```python
filter_list = [ssp['terminal semi major axis']>0,
               ssp['terminal eccentricity'] <0.9,
               ssp['lifetime'] > ssp['age'],
               ssp['terminal semi major axis']>ssp['terminal roche radius'],
               ssp['terminal period']<5,
               ssp['terminal period']>0.137,
               ssp['black hole mass']>
               3.0/(1.0-100*ssp['sigma pomega']/ssp['parallax']-
                    100*ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
               ssp['companion apparent magnitude']<12.4,
               ssp['terminal semi major axis']>
               10*(1+ssp['initial mass ratio']/0.2)*100*ssp['sigma pomega']*ssp['distance from earth']*5e-12
              ]
mask = numpy.all(filter_list,axis=0)
ssp['statistical weight'][mask].sum()
```

Count how many will be seen by Gaia

```python
filter_list = [
    ssp['terminal semi major axis'] > 0,
    ssp['terminal eccentricity'] <0.9,
    ssp['terminal semi major axis']>ssp['terminal roche radius'],
    ssp['terminal period']<5,
    ssp['terminal period']>0.137,
    ssp['black hole mass']>
    3.0/(1.0-ssp['sigma pomega']/ssp['parallax']-ssp['sigma G']-
         ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
    ssp['companion apparent magnitude']<20,
    ssp['terminal semi major axis']>
    10*(1+ssp['initial mass ratio']/0.2)*ssp['sigma pomega']*ssp['distance from earth']*5e-12
]
mask = numpy.all(filter_list, axis=0)
gaia_kick_velocity_mask = mask
ssp['statistical weight'][mask].sum()
```

```python
%matplotlib inline
import matplotlib
font = {'size':22}
matplotlib.rc('font',**font)

pylab.hist(ssp['black hole mass'][mask],10, weights=ssp['statistical weight'][mask])
pylab.xlabel('Black hole mass [$M_{\odot}$]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['distance from earth'][mask]/1000,10, weights=ssp['statistical weight'][mask])
pylab.xlabel('Distance from earth [kpc]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['companion mass'][mask],10, weights=ssp['statistical weight'][mask])
pylab.xscale('log')
pylab.xlabel(r'Companion mass [$M_{\odot}$]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['terminal semi major axis'][mask]/ssp['distance from earth'][mask]/4.8e-9,10,
          weights=ssp['statistical weight'][mask])
pylab.xscale('log')
pylab.xlabel(r'Angular separation [mas]')
pylab.ylabel('Number')
pass
```

```python
pylab.scatter(ssp['companion mass'][mask],
              ssp['terminal semi major axis'][mask]/ssp['distance from earth'][mask]/4.8e-9
             )
pylab.xscale('log')
pylab.yscale('log')
pylab.xlabel('Companion mass[$M_{\odot}$]')
pylab.ylabel('Angular separation [mas]')
pass
```

## Momentum conservatoin

```python
ssp = {'primary mass':randomise_salpeter_bh_progenitor_mass(int(1e6))}
ssp['initial mass ratio'] = randomise_mass_ratio(ssp['primary mass'])
ssp['initial companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['initial semi major axis'] = randomise_semi_major_axis(ssp['primary mass'])
ssp['z'] = randomise_height_above_galactic_disc(ssp['primary mass'])
ssp['r'] = randomise_distance_from_galactic_centre(ssp['primary mass'])
ssp['angle'] = randomise_angle_gc_sol(ssp['primary mass'])
ssp['black hole mass'] = calc_terminal_black_hole_mass(ssp['primary mass'])
ssp['age'] = numpy.random.rand(len(ssp['primary mass'])) # Age / galactic age
ssp['companion mass'] = ssp['primary mass']*ssp['initial mass ratio']
ssp['companion radius'] = mass2radius_demircan_kahraman(ssp['companion mass'])
ssp['initial primary radius'] = mass2radius_demircan_kahraman(ssp['primary mass'])
ssp['initial roche radius'] = roche_lobe_radius_ratio(ssp['initial mass ratio'])*ssp['initial primary radius']
ssp['terminal companion mass'] = (ssp['initial companion mass'] +
                                  numpy.where(ssp['initial mass ratio']>0.5,
                                              0.5*(ssp['primary mass']-ssp['black hole mass']),
                                              numpy.zeros_like(ssp['primary mass'])))
ssp['pre kick semi major axis'] = calc_current_semi_major_axis(
    ssp['initial mass ratio'],
    ssp['black hole mass']/ssp['primary mass'],
    ssp['initial semi major axis'],
    alphalambda=1.0
)
ssp['terminal mass ratio'] = ssp['black hole mass']/ssp['terminal companion mass']
ssp['lifetime'] = ssp['terminal companion mass']**-2.5 - ssp['primary mass']**-2.5 # Normalised by the age of the galaxy
ssp['terminal roche radius'] = roche_lobe_radius_ratio(1.0/ssp['terminal mass ratio'])*ssp['companion radius']

# Kicks
ssp['kick mu'] = numpy.random.rand(len(ssp['primary mass']))*2-1
ssp['kick phi'] = numpy.random.rand(len(ssp['primary mass']))*numpy.pi
ssp['kick v_k'] = numpy.absolute(numpy.random.normal(0, 265, len(ssp['primary mass'])))/ssp['black hole mass'] # km/s
ssp['kick v_i'] = 0.07*numpy.sqrt((ssp['primary mass'] + ssp['companion mass'])/ssp['pre kick semi major axis']) # km/s
ssp['kick chi'] = (ssp['primary mass']+ssp['initial companion mass'])/(ssp['black hole mass'] + ssp['terminal companion mass'])

ssp['terminal semi major axis'] = ssp['pre kick semi major axis']/(
    kick_semi_major_axis_ratio(ssp['kick v_k'],
                               ssp['kick v_i'],
                               ssp['kick chi'],
                               ssp['kick mu'])
)

ssp['terminal eccentricity'] = kick_eccentricity(ssp['kick v_k'],
                                                 ssp['kick v_i'],
                                                 ssp['kick chi'],
                                                 ssp['kick mu'],
                                                 ssp['kick phi'])

ssp['terminal period'] = 9.4e7*(ssp['terminal semi major axis']**1.5*
                                ssp['primary mass']**-0.5*
                                (0.2+ssp['initial mass ratio'])**-0.5)
ssp['companion absolute magnitude'] = stellar_mass_to_vmag(ssp['primary mass']*ssp['initial mass ratio'])
ssp['distance from earth'] = numpy.sqrt(ssp['z']**2+8000**2+ssp['r']**2-2*8000*ssp['r']*numpy.cos(ssp['angle']))
ssp['companion apparent magnitude'] = absolute_to_apparent_magnitude(ssp['companion absolute magnitude'],
                                                                     ssp['distance from earth'],
                                                                    extinction=True)
ssp['sigma pomega'] = calc_sigma_pomega(ssp['companion apparent magnitude'])
ssp['parallax'] = 1e6/ssp['distance from earth'] # in uas
ssp['sigma G'] = calc_sigma_G(ssp['companion apparent magnitude'])
ssp['statistical weight'] = numpy.clip(ssp['lifetime'],0,1)
ssp_kick_momentum=ssp
```

    /usr/local/lib/python2.7/dist-packages/ipykernel_launcher.py:49: RuntimeWarning: invalid value encountered in power

Hipparcos prediction

```python
filter_list = [
    ssp['terminal semi major axis'] > 0,
    ssp['terminal eccentricity'] <0.9,
    ssp['lifetime'] > ssp['age'],
    ssp['terminal semi major axis']>ssp['terminal roche radius'],
    ssp['terminal period']<5,
    ssp['terminal period']>0.137,
    ssp['black hole mass']>
    3.0/(1.0-100*ssp['sigma pomega']/ssp['parallax']-
         100*ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
    ssp['companion apparent magnitude']<12.4,
    ssp['terminal semi major axis']>
    10*(1+ssp['initial mass ratio']/0.2)*100*ssp['sigma pomega']*ssp['distance from earth']*5e-12
]
mask = numpy.all(filter_list, axis=0)
ssp['statistical weight'][mask].sum()
```

Gaia prediction

```python
filter_list = [
    ssp['terminal semi major axis'] > 0,
    ssp['terminal eccentricity'] <0.9,
    ssp['terminal semi major axis']>ssp['terminal roche radius'],
    ssp['terminal period']<5,
    ssp['terminal period']>0.137,
    ssp['black hole mass']>
    3.0/(1.0-ssp['sigma pomega']/ssp['parallax']-ssp['sigma G']-
         ssp['sigma pomega']*ssp['distance from earth']/2e11/ssp['terminal semi major axis']),
    ssp['companion apparent magnitude']<20,
    ssp['terminal semi major axis']>
    10*(1+ssp['initial mass ratio']/0.2)*ssp['sigma pomega']*ssp['distance from earth']*5e-12
]
mask = numpy.all(filter_list, axis=0)
gaia_kick_momentum_mask = mask
ssp['statistical weight'][mask].sum()
```

```python
%matplotlib inline
import matplotlib
font = {'size':22}
matplotlib.rc('font',**font)

pylab.hist(ssp['black hole mass'][mask],10, weights=ssp['statistical weight'][mask])
pylab.xlabel('Black hole mass [$M_{\odot}$]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['distance from earth'][mask]/1000,10, weights=ssp['statistical weight'][mask])
pylab.xlabel('Distance from earth [kpc]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['companion mass'][mask],10, weights=ssp['statistical weight'][mask])
pylab.xscale('log')
pylab.xlabel(r'Companion mass [$M_{\odot}$]');
pylab.ylabel('Number')
pass
```

```python
pylab.hist(ssp['terminal semi major axis'][mask]/ssp['distance from earth'][mask]/4.8e-9,10,
          weights=ssp['statistical weight'][mask])
pylab.xscale('log')
pylab.xlabel(r'Angular separation [mas]')
pylab.ylabel('Number')
pass
```

```python
pylab.scatter(ssp['companion mass'][mask],
              ssp['terminal semi major axis'][mask]/ssp['distance from earth'][mask]/4.8e-9
             )
pylab.xscale('log')
pylab.yscale('log')
pylab.xlabel('Companion mass[$M_{\odot}$]')
pylab.ylabel('Angular separation [mas]')
pass
```

# Figures for the paper

```python
%matplotlib inline
font = {'size':22}
matplotlib.rc('font',**font)

for ssp, mask in zip([ssp_no_kick, ssp_inefficient, ssp_kick_momentum, ssp_kick_velocity],
                    [gaia_no_kick_mask, gaia_inefficient_mask, gaia_kick_momentum_mask, gaia_kick_velocity_mask]):
    pylab.hist(ssp['terminal semi major axis'][mask]/
               ssp['distance from earth'][mask]/4.8e-9,10,
               weights=ssp['statistical weight'][mask]*100,
               histtype='step')
pylab.yscale('log')
pylab.xlabel(r'Angular separation [mas]')
pylab.ylabel('Number')
pass
```

```python
%matplotlib inline
font = {'size':22}
matplotlib.rc('font',**font)

for ssp, mask in zip([ssp_no_kick, ssp_inefficient, ssp_kick_momentum, ssp_kick_velocity],
                    [gaia_no_kick_mask, gaia_inefficient_mask, gaia_kick_momentum_mask, gaia_kick_velocity_mask]):
    pylab.hist(ssp['black hole mass'][mask],
               weights=ssp['statistical weight'][mask]*100,
               histtype='step',
              linewidth=4)
pylab.yscale('log')
pylab.xlabel(r'Black hole mass [$M_{\odot}$]')
pylab.ylabel('Number')
pylab.legend(['fid','al01','nkm','nkv'],bbox_to_anchor=(1,1))
pylab.tight_layout()
pass
```

```python
%matplotlib inline
font = {'size':22}
matplotlib.rc('font',**font)

for ssp, mask in zip([ssp_no_kick, ssp_inefficient, ssp_kick_momentum, ssp_kick_velocity],
                    [gaia_no_kick_mask, gaia_inefficient_mask, gaia_kick_momentum_mask, gaia_kick_velocity_mask]):
    pylab.hist(ssp['terminal companion mass'][mask],
               weights=ssp['statistical weight'][mask]*100,
               histtype='step',
              linewidth=4)
pylab.yscale('log')
pylab.xscale('log')
pylab.xlabel(r'Companion mass [$M_{\odot}$]')
pylab.legend(['fid','al01','nkm','nkv'],loc='best',bbox_to_anchor=(1,1))
pylab.ylabel('Number')
pass
```

```python
%matplotlib inline
font = {'size':22}
matplotlib.rc('font',**font)

for ssp, mask in zip([ssp_no_kick, ssp_inefficient, ssp_kick_momentum, ssp_kick_velocity],
                    [gaia_no_kick_mask, gaia_inefficient_mask, gaia_kick_momentum_mask, gaia_kick_velocity_mask]):
    pylab.hist(ssp['terminal period'][mask],
               weights=ssp['statistical weight'][mask]*100,
               histtype='step',
              linewidth=4)
pylab.yscale('log')
pylab.xlabel(r'Period [years]')
pylab.ylabel('Number')
pylab.legend(['fid','al01','nkm','nkv'],loc='best',bbox_to_anchor=(1,1))
pass
```

```python
%matplotlib inline
font = {'size':22}
matplotlib.rc('font',**font)

for ssp, mask in zip([ssp_no_kick, ssp_inefficient, ssp_kick_momentum, ssp_kick_velocity],
                    [gaia_no_kick_mask, gaia_inefficient_mask, gaia_kick_momentum_mask, gaia_kick_velocity_mask]):
    pylab.hist(ssp['distance from earth'][mask]/1000,
               weights=ssp['statistical weight'][mask]*100,
               histtype='step',
              linewidth=4)
pylab.yscale('log')
pylab.xlabel(r'Distance from earth [kpc]')
pylab.ylabel('Number')
pylab.legend(['fid','al01','nkm','nkv'],loc='best',bbox_to_anchor=(1,1))
pass
```

# Machine Learning Playground

```python
X = numpy.vstack((
    numpy.concatenate((ssp_no_kick['black hole mass'][gaia_no_kick_mask],
                       ssp_kick_momentum['black hole mass'][gaia_kick_momentum_mask],
                       ssp_kick_velocity['black hole mass'][gaia_kick_velocity_mask])),
        numpy.concatenate((ssp_no_kick['companion mass'][gaia_no_kick_mask],
                       ssp_kick_momentum['companion mass'][gaia_kick_momentum_mask],
                       ssp_kick_velocity['companion mass'][gaia_kick_velocity_mask])),
        numpy.concatenate((ssp_no_kick['terminal semi major axis'][gaia_no_kick_mask],
                       ssp_kick_momentum['terminal semi major axis'][gaia_kick_momentum_mask],
                       ssp_kick_velocity['terminal semi major axis'][gaia_kick_velocity_mask])),
    numpy.concatenate((ssp_no_kick['distance from earth'][gaia_no_kick_mask],
                       ssp_kick_momentum['distance from earth'][gaia_kick_momentum_mask],
                       ssp_kick_velocity['distance from earth'][gaia_kick_velocity_mask])))).T
    #ssp_no_kick['terminal semi major axis'][gaia_no_kick_mask],
    #ssp_no_kick['distance from earth'][gaia_no_kick_mask])).T
X_embedded = TSNE(n_components=2).fit_transform(X)
n1 = len(ssp_no_kick['black hole mass'][gaia_no_kick_mask])
n2 = len(ssp_kick_momentum['black hole mass'][gaia_kick_momentum_mask])
n3 = len(ssp_kick_velocity['black hole mass'][gaia_kick_velocity_mask])
%matplotlib inline
pylab.scatter(X_embedded.T[0][:n1],
              X_embedded.T[1][:n1])
pylab.scatter(X_embedded.T[0][n1:n1+n2],
              X_embedded.T[1][n1:n1+n2])
pylab.scatter(X_embedded.T[0][n1+n2:n1+n2+n3],
              X_embedded.T[1][n1+n2:n1+n2+n3])
```

All models occupy the same region in a reduced dimenion space. This mean they are indistinguishable.

```python

```


# 1. Electrical capacitance and energy of the mitochondrial inner membrane

A capacitor is an electrical circuit element that stores electrical energy in an electric field (Wikipedia REF). The mitochondrial inner membrane acts as an electrical capacitor to store energy in an electrostatic potential different between the milieux on the two sides the membrane.

Electrical capacitance is measured in units of charge per unit voltage, or in standard units, Coulombs per Volt (C/V), equivalently, a Farad. (1 F = 1 C/V)

## 1.1. What is the capacitance of mitochondrial inner membrane? Make a reasonable assumption about the size of a mitochondrion and assume that the inner membrane has 5 or 10 times more area than the outer membrane.

A typical range for the capacity density of a biological membrane is 0.5-1.0 $\mu$F cm$^{-2}$ [REF]. Let’s take a mitochondrion as a sphere with radius 1  $\mu$m. In this case the outer membrane would have a surface area of $4\pi r^2$, or approximately 12.5  $\mu$m$^2$. If the inner membrane has 10 times more area than the outer, then the area of the inner membrane is 125 $\mu$m$^2$. So, we can approximate the capacitance of this mitochondrion’s inner membrane as

$$
  c_{mito} = \left( 1 \frac{\mu{\rm F}} {{\rm cm}^2} \right)
             \left(  125 \, \mu{\rm m}^2  \right)
             \left( \frac{ 1 \, {\rm cm}^2} {10^8 \mu{\rm m}^2} \right) =
             1.25 \times 10^{-6} \mu{\rm F} .
$$

## 1.2. Express inner mitochondrial membrane capacitance in units of: (a.) farads per liter of mitochondria; (b.) moles per millivolt per unit liter of mitochondria.

Our one-micron radius spherical mitochondrion has a volume of approximately 4 $\mu$m$^3$, or $4 \times 10^{-15}$ L. Thus, the capacitance per untit volume is

$$
  \frac{1.25 \times 10^{-6} \mu{\rm F}}{4 \times 10^{-15} \, {\rm L}} =
  3.125 \times 10^{8} \frac{\mu{\rm F}}{{\rm L}} =
  3.125 \times 10^{2} \frac{{\rm F}}{{\rm L}} =
  3.125 \times 10^{2} \frac{{\rm C}}{{\rm V \, L}} =
  3.125 \times 10^{-1} \frac{{\rm C}}{{\rm mV \, L}}
$$

Using the fact that there are $1.036 \times 10^{−5}$ moles per coulomb, the capacitance can be expressed

$$
 3.24 \times 10^{-6} {\rm mol}/({\rm mV \, L}).
$$

## 1.3. If the electrostatic potential across the inner mitochondrial membrane is 180 mV, how much electrical energy per unit volume is stored in mitochondria? How much electrical energy is stored in mitochondria per unit volume of myocardium? Express your answer in joules per liter.

The total energy stored in a capacitor is $1/2\, c \Delta\Psi^2$, where $c$ is the capacitance and $\Delta\Psi$ is the voltage drop across the capacitance. (Can you derive this formula? Verify that the units are correct.) The density energy in the electrostatic potential across the mitochondrial inner membrane is computed

$$
  e_{mito} = \frac{1}{2} c_{mito} \Delta\Psi^2
  = \frac{3.125}{2} \times 10^{-1} \frac{{\rm C}}{{\rm mV \, L}} ( 180 \, {\rm mV)}) (0.180\, {\rm V})^2 \, ,
$$

which is roughly equal to 5 joule / L. The myocardium is approximately 1/3 by volume mitochondria. So the mitochondrial electrical energy per unit volume of myocardium is about 5/3 joule / L.

## 1.4. Approximately much electrical energy is stored in mitochondria in the whole human heart? How does that compare to the amount of energy supplied by a AA battery? How does it compare to the amount of mechanical work the heart does per unit time?

The heart is around 250 grams, or in terms of volume, approximately $1/4$ L. So the inner mitochondrial membrane of mitochondrial stores around $5/12$ of a joule of energy in the electrical potential.

For comparison a AA battery provides approximately 10 kJ of energy.

Another comparison: the LV does approximately 1 watt of work under resting baseline conditions. That is: 1 W = 1 joule / second. The $1/4$ joule of electrical energy stored in the inner mitochodondrial membrane is equivalent to how much mechanical work the heart does at rest in $1/4$second.

# 2. Converting electrical potential to the ATP hydrolysis chemical potential

The electrostatic energy potential across the mitochondrial inner membrane is used to drive the synthesis of ATP in the final step of oxidative ATP synthesis. The mammalian mitochondrial F1F0-ATPase synthesizes ATP in the mitochondrial matrix from ADP and inorganic phosphate, coupled to the translocation of protons (H+ ions) from outside to inside of the matrix. The chemical reaction stoichiometry can be expressed:

\begin{equation}\label{eq:ATPase1}
	{\rm MgADP}^{1-} + {\rm HPO}_4^{2-} + ({\rm H}^+)_{\rm inside} \leftrightharpoons
    {\rm MgATP}^{2-} + {\rm H}_2{\rm O},
\end{equation}

where the term $({\rm H}^+)_{\rm inside}$ indicates that a hydrogen ion from inside the matrix is covalently incorporated into the synthesized ATP. The species MgADP$^{1-}$ and MgATP$^{2-}$ are the magnesium-bound species of ADP and ATP. This chemical reaction is coupled to the transport of $n_A = 8/3$ protons across the inner membrane:

\begin{equation}\label{eq:ATPase2}
    n_A \, ({\rm H}^+)_{\rm outside} \leftrightharpoons n_A \, ({\rm H}^+)_{\rm inside}.
\end{equation}

## 2.1. Given a free magnesium concentration [Mg$^{2+}$] and hydrogen ion activity [H$^+$] = 10$^{\rm -pH}$, how can you compute the concentrations of MgADP$^{1-}$, MgATP$^{2-}$, and HPO$_4^{2-}$ in terms of the total concentrations of the reactants [$\Sigma$ADP], [$\Sigma$ATP], and [$\Sigma$Pi]? (You will need to account for binding of biochemical species to [Mg$^{2+}$] and [H$^+$].)

Let’s start with looking at the species of ATP:
\begin{eqnarray}
  [\Sigma{\rm ATP}] & = & [{\rm ATP}^{4-}] + [{\rm Mg}\cdot{\rm ATP}^{2-}] +
                          [{\rm H}\cdot{\rm ATP}^{3-}] + [{\rm K}\cdot{\rm ATP}^{3-}] \\
                    & = & [{\rm ATP}^{4-}]  + \frac{[{\rm Mg}^{2+}]}{K_{MgATP}}[{\rm ATP}^{4-}]  +
                          \frac{[{\rm H}^{+}]}{K_{HATP}}[{\rm ATP}^{4-}] +
                          \frac{[{\rm K}^{+}]}{K_{KATP}}[{\rm ATP}^{4-}]  \\
                    & = & [{\rm ATP}^{4-}]\left(1 + \frac{[{\rm Mg}^{2+}]}{K_{MgATP}} +
                          \frac{[{\rm H}^{+}]}{K_{HATP}} + \frac{[{\rm K}^{+}]}{K_{KATP}} \right)
\end{eqnarray}

(I am ignoring the two-cation-bound species.)

We can define something called the “binding polynomial” for each reactant:
\begin{eqnarray}
  P_{ATP} & = & 1 + \frac{[{\rm Mg}^{2+}]}{K_{MgATP}} +
                     \frac{[{\rm H}^{+}]}{K_{HATP}} + \frac{[{\rm K}^{+}]}{K_{KATP}} \\
  P_{ADP} & = & 1 + \frac{[{\rm Mg}^{2+}]}{K_{MgADP}} +
                     \frac{[{\rm H}^{+}]}{K_{HADP}} + \frac{[{\rm K}^{+}]}{K_{KADP}} \\
  P_{Pi}  & = & 1 + \frac{[{\rm Mg}^{2+}]}{K_{MgPi}} +
                     \frac{[{\rm H}^{+}]}{K_{HPi}} + \frac{[{\rm K}^{+}]}{K_{KPi}} \, ,
\end{eqnarray}
and
\begin{eqnarray}
  [\Sigma{\rm ATP}] & = & [{\rm ATP}^{4-}] P_{ATP} \\
  [\Sigma{\rm ADP}] & = & [{\rm ADP}^{3-}] P_{ADP} \\
  [\Sigma{\rm Pi}] & = & [{\rm HPO}_4^{2-}] P_{Pi} \, .
\end{eqnarray}

Furthermore
\begin{eqnarray}
  [{\rm Mg}\cdot{\rm ATP}^{2-}] & = & \frac{[{\rm Mg}^{2+}]/K_{MgATP}}{P_{ATP}}[\Sigma{\rm ATP}] \\
  [{\rm Mg}\cdot{\rm ADP}^{1-}] & = & \frac{[{\rm Mg}^{2+}]/K_{MgADP}}{P_{ADP}}[\Sigma{\rm ADP}] \\
  [{\rm HPO}_4^{2-}] & = & \frac{1}{P_{Pi}}[\Sigma{\rm Pi}] \\
\end{eqnarray}

## 2.2. Derive an expression for the Gibbs free energy change associated with reaction (2.1) in terms of the reference $\Delta G_r^0$, the concentrations of biochemical reactants, and the cation concentrations [Mg$^{2+}$] and [H$^+$]. What is the free energy of ATP hydrolysis in the mitochondrial matrix? Assume that pH = 7.2 and [$\Sigma$ADP] = 9.5 mM, [$\Sigma$ATP] = 0.5 mM, and [$\Sigma$Pi] = 1 mM.

Consider the ATP synthesis reaction
$$
  \rm ADP^{3-} + HPO_4^{2-} + H^+ \rightleftharpoons ATP^{4-} + H_2O \, ,
$$
with equilibrium mass action ratio
$$
  K_{eq} = e^{-\Delta G_r^0 / RT} = \left( \frac{[{\rm ATP}^{4-}]}
        {[{\rm H}^+][{\rm HPO}_4^{2-}][{\rm ADP}^{3-}]} \right)_{eq} \, .
$$
Substituting the expressions for the binding polynomials, we have
$$
  K_{eq} = e^{-\Delta G_r^0 / RT} = \left( \frac{[\Sigma{\rm ATP}]}
        {[\Sigma{\rm Pi}][\Sigma{\rm ADP}]} \right)_{eq}
        \frac{P_{ADP}P_{Pi}}{[{\rm H}^+]P_{ATP}}\, ,
$$

which yields and expression for the apparent equilibrium constant in terms of biochemical reactants
$$
  K'_{eq} = e^{-\Delta G_r^{\prime 0} / RT} = \left( \frac{[\Sigma{\rm ATP}]}
        {[\Sigma{\rm Pi}][\Sigma{\rm ADP}]} \right)_{eq} =
        K_{eq}\frac{[{\rm H}^+]P_{ATP}}{P_{ADP}P_{Pi}} \, .
$$

Using these equations, there are a couple of equivalent ways to express the Gibbs free enegy change for this reaction. One is based on the referenc chemical reaction

\begin{equation}
  \Delta G_r = \Delta G^0_r - RT \ln\left( \frac{[{\rm H}^+][{\rm HPO}_4^{2-}][{\rm ADP}^{3-}]}
                                                  {[{\rm ATP}^{4-}]}\right) \, .
\end{equation}

The other is
\begin{equation}
  \Delta G_r = \Delta G^{\prime 0}_r - RT \ln\left( \frac{[\Sigma{\rm Pi}][\Sigma{\rm ADP}]}
                                                  {[\Sigma{\rm ATP}]}\right) \, .
\end{equation}

To verify these two equations give the same value, take the following values for dissociation constants

| Reactant (L) | $K_{H-L}$                | $K_{K-L}$               | $K_{Mg-L}$                |
| ------------ | ------------------------ | ----------------------- | ------------------------- |
| ATP          | 2.757$\times 10^{-7}$ M  | 9.809$\times 10^{-2}$ M   | 8.430$\times 10^{-5}$ M   |
| ADP          | 4.106$\times 10^{-7}$ M  | 1.319$\times 10^{-1}$ M   | 7.149$\times 10^{-4}$ M   |
| PI           | 2.308$\times 10^{-7}$ M  | 3.803$\times 10^{-1}$ M  | 2.815$\times 10^{-2}$ M   |

and $\Delta_r^0 G = -4.67$ kJ/mole, which yields $K_{eq} = 6.1163$ at 37$^\circ$ C.

The Python code below verifies that both approaches yield the same value of 45.730 kJ/mole.

```python
# %%

import numpy

# Dissociation Constants
K_HATP = 2.757e-7
K_KATP = 9.809e-2
K_MATP = 8.430e-5
K_HADP = 4.106e-7
K_KADP = 1.319e-1
K_MADP = 7.149e-4
K_HPI  = 2.308e-7
K_KPI  = 3.803e-1
K_MPI  = 2.815e-2

# Other constants
Keq = 6.1163
DGr0 = -4670
RT = 310.15*8.3145
DGr0 = -RT*numpy.log(Keq)

# Concentrations
ATP = 0.5e-3
ADP = 9.5e-3
Pi  = 1e-3
Mg  = 1e-3
H   = 10**(-7.2)
K   = 100e-3

# binding polynomials:
P_ATP = 1 + H/K_HATP + K/K_KATP + Mg/K_MATP
P_ADP = 1 + H/K_HADP + K/K_KADP + Mg/K_MADP
P_PI = 1 + H/K_HPI + K/K_KPI + Mg/K_MPI

# reference species concentration
ATPs = ATP/P_ATP
ADPs = ADP/P_ADP
Pi_s = Pi/P_PI

# compute DeltaG_r in terms of reference species
DG1 = +DGr0 - RT*numpy.log(H*Pi_s*ADPs/ATPs)
print("First equation:",DG1," J/mole")

K_apparent = Keq*H*P_ATP/(P_ADP*P_PI)
DG2 = -RT*numpy.log(K_apparent) - RT*numpy.log(ADP*Pi/ATP)
print("Second equation:",DG2," J/mole")

ATPs, ATP
```

    First equation: 45729.93159766788  J/mole
    Second equation: 45729.93159766787  J/mole

    (3.543404140361549e-05, 0.0005)

```python

```

This positive value for $\Delta_r G$ means that the reaction is not favorable to proceed in the forward driection. (In other words, ATP synthesis cannot proceed without being coupled to some other driving force.)

## 2.3. What is the free energy change of Equation (2.2) at $\Delta\Psi$ = 180 mV? How does the free energy change of Equation (2.1) compare to that of Equation (2.2)? How efficient is the transduction of electrical to chemical free energy in this step in ATP synthesis? (What is the ratio of energy stored in ATP to the total energy consumed?)

The electrical potential of 0.18 J/C can be converted to J/mole using the unit conversion factor $F = 96485.33$ C/mole, which is known as Faraday's constant:

\begin{equation}
  \Delta G_{H^+} = \left(0.18\,{\rm J/C}\right)\left( 96485.33\,{\rm C/mole} \right)
                 = 17367.35 \, {\rm J/mole} \, .
\end{equation}

(This equation ignores differences in pH on either side of the inner membrane.) This is the energy required (per mole) to move positive charges against the electrostatic gradient, from the inside to the outside of the matrix. In other words, there is an electrical force driving postive charge into the matrix.

Since ATP synthesis reaction in mitochondrial is coupled to $n_A = 8/3$ H$^+$ translocate to the matrix, the overall free energy change for the coupled transport and reaction process is

\begin{equation}
  \Delta G = \Delta G^{\prime 0}_r - RT \ln\left( \frac{[\Sigma{\rm Pi}][\Sigma{\rm ADP}]}
                                                  {[\Sigma{\rm ATP}]}\right)
                                   - n_A \Delta G_{H^+} \, .
\end{equation}

Plugging the numbers from Section (2.2) we get

\begin{eqnarray}
  \Delta G &=& +45.73 - (8/3)*17.37 \\
           &=& +45.73 -46.32  \\
           &=& -0.59 \, {\rm kJ/mole} \, ,
\end{eqnarray}

which means that, given the assumed concentrations, the system is operating close to equilibrium. The ratio of chemical energy stored in the ATP hydrolysis potential to electrical energy consumed is $45.73/46.32 \approx 98.7 \%$.

## 2.4. Given the concentrations assumed in 2.2, what is the minimum value of $\Delta\Psi$ at which ATP can be synthesized in the mitochondrial matrix?

At equilibrium the electrical energy consumed is equal to the ATP hydrolysis potential:
$$
 (8/3) \Delta G_{H^+} = 45.73 \, {\rm kJ/mole} ,
$$
yielding $\Delta\Psi = 177.7$ mV.

## 2.5. Assume that reaction (2.1) proceeds by simple mass-action kinetics, with a constant reverse rate kr. How does the forward rate constant necessarily depend on $\Delta\Psi$ for the reaction kinetics to be properly thermodynamically balanced?

To recap, the free energy for the ATP synthesis reaction
\begin{equation}
	{\rm ADP}^{3-} + {\rm HPO}_4^{2-} + ({\rm H}^+)_{\rm inside} \leftrightharpoons
    {\rm ATP}^{4-} + {\rm H}_2{\rm O},
\end{equation}
is
\begin{equation}
  \Delta G_r = \Delta G_r^0 - RT \log\left( \frac{[{\rm H}^+]_{\rm inside}[{\rm HPO}_4^{2-}][{\rm ADP}^{3-}]}
      {[{\rm ATP}^{4-}]} \right)
\end{equation}
where $\Delta G_r^0 \approx -4.6$ kJ/mole is the standard Gibbs free energy change for the reference chemical reaction. In mitochondria the ATP synthesis is coupled to proton transport from outside to inside of the matrix:
\begin{equation}
   n_A \left({\rm H}^+ \right)_{\rm outside} \leftrightharpoons n_A \left({\rm H}^+ \right)_{\rm inside}
\end{equation}
which is associated with a free energy change
\begin{equation}
  \Delta G_H = -n_A F\Delta\Psi - RT \log\left( \frac{[{\rm H}^+]_{\rm outside}}
          { [{\rm H}^+]_{\rm inside}} \right)^{n_A} \, .
\end{equation}
This free energy is sometimes called the proton motive force. The last term on the right-hand side represents the contribution to the proton-transport free energy from a gradient in pH across the inner mitochondrial membrane.

The free energy of coupled ATP synthsis and proton transport is
\begin{equation}
  \Delta G_r = \Delta G_r^0 - RT \log\left( \frac{[{\rm HPO}_4^{2-}][{\rm ADP}^{3-}]}
      {[{\rm ATP}^{4-}]} \right) -n_A F\Delta\Psi - RT \log \frac{\left( [{\rm H}^+]_{\rm outside}\right)^{n_A}}
          {\left( [{\rm H}^+]_{\rm inside}\right)^{n_A-1}}  \, .
\end{equation}

Chemical equilibrium occurs when $\Delta G_r = 0$:
\begin{equation}
  \left( \frac{[{\rm ATP}^{4-}]}{[{\rm HPO}_4^{2-}][{\rm ADP}^{3-}]}
       \right)_{eq} = \exp\left[-\left( \Delta G_r^0 - N_A F\Delta\Psi \right)/RT \right]\cdot
      \frac{\left( [{\rm H}^+]_{\rm outside}\right)^{n_A}}
          {\left( [{\rm H}^+]_{\rm inside}\right)^{n_A-1}} \, ,
\end{equation}
or in terms of sums of species
\begin{equation}
  K'_{F1F0ATPase} =
  \left( \frac{[\Sigma{\rm ATP}]}{[\Sigma{\rm Pi}][\Sigma{\rm ADP}]}
       \right)_{eq} = \exp\left[-\left( \Delta G_r^0 - N_A F\Delta\Psi \right)/RT \right]\cdot
       \frac{P_{ATP}}{P_{ADP} P_{Pi}}\cdot
      \frac{\left( [{\rm H}^+]_{\rm outside}\right)^{n_A}}
          {\left( [{\rm H}^+]_{\rm inside}\right)^{n_A-1}} \, .
\end{equation}

A simple mass-action expression for the reaction flux that satisfies this thermodynamic relation is
\begin{equation}
  J_{F1F0ATPase} = k_r\left( K'_{F1F0ATPase}[\Sigma{\rm Pi}][\Sigma{\rm ADP}] - [\Sigma{\rm ATP}] \right) .
\end{equation}

## 2.6. Write a simple program that simulates the kinetics of [$\Sigma$ADP], [$\Sigma$ATP] , and [$\Sigma$Pi] in the matrix given a fixed membrane potential, pH, and magnesium concentration, and given arbitrary initial conditions. How do the predicted steady-state concentrations depend on membrane potential, pH, and magnesium concentration?

```python
import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
import math as mt
# init_conditions = ATP, ADP, P
# ATP, ADP, P = init_conditions
# Psi input in mV
def J_ATPase(psi, pH_inner, pH_outer, Mg, concentrations):
    # Define initial conditions
    ATP, ADP, Pi , ATPe , ADPe = concentrations

    # Assume k_r = 1
    k_r = 10
    N_A = 8/3
    # Dissociation Constants
    K_HATP = 2.757e-7
    K_KATP = 9.809e-2
    K_MATP = 8.430e-5
    K_HADP = 4.106e-7
    K_KADP = 1.319e-1
    K_MADP = 7.149e-4
    K_HPI  = 2.308e-7
    K_KPI  = 3.803e-1
    K_MPI  = 2.815e-2

    # H concentrations
    H_in = 10**(-pH_inner)
    H_out = 10**(-pH_outter)
    # print("H_out",H_out)
    # print("H_in", H_in)
    H = H_in

    # binding polynomials:
    P_ATP = 1 + H/K_HATP + Mg/K_MATP
    P_ADP = 1 + H/K_HADP + Mg/K_MADP
    P_PI = 1 + H/K_HPI + Mg/K_MPI

    # Other constants
    # Keq = 6.1163
    DGr0 = -4670
    RT = 310.15*8.3145
    # DGr0 = -RT*np.log(Keq)
    F = 96485.33  # C/mole
    # R = 8.3145

    # Define K'_F1F0ATPase
    k_prime = np.exp(-(DGr0 - F * psi/1000* N_A)/(RT))*P_ATP/(P_ADP*P_PI)*H_out**N_A / H_in**(N_A-1) #Psi / 1000 b/c input in mv
    return k_r*(k_prime*Pi * ADP - ATP)

```

```python
def J_ANT(psi, pH_inner, pH_outer, Mg, concentrations):
    # Define initial conditions
    ATP, ADP, Pi , ATPe, ADPe = concentrations

    del_D = 0.0167
    del_T = 0.0699
    k2_ANT = 9.54/60
    k3_ANT = 30.05/60
    K_D_o_ANT = 38.89e-6
    K_T_o_ANT = 56.05e-6
    A = +0.2829
    B = -0.2086
    C = +0.2372
    RT = 310.15*8.3145
    F = 96485.33  # C/mole
    psi = .180
    fi = F*psi/RT
    E_ANT = 0.141
    k2_ANT_fi = k2_ANT*mt.exp((A*(-3)+B*(-4)+C)*fi)
    k3_ANT_fi = k3_ANT*mt.exp((A*(-4)+B*(-3)+C)*fi)

    K_D_o_ANT_fi = K_D_o_ANT*mt.exp(3*del_D*fi)
    K_T_o_ANT_fi = K_T_o_ANT*mt.exp(4*del_T*fi)
    q = k3_ANT_fi*K_D_o_ANT_fi*mt.exp(fi)/(k2_ANT_fi*K_T_o_ANT_fi)

    term2 = k2_ANT_fi*ATP*ADPe*q/K_D_o_ANT_fi
    term3 = k3_ANT_fi*ADP*ATPe/K_T_o_ANT_fi
    num = term2 - term3
    den = (1 + ATPe/K_T_o_ANT_fi + ADPe/K_D_o_ANT_fi)*(ADP + ATP*q)

    return E_ANT*num/den

```

```python
pH_outter = 7.2
pH_inner = 7.4
ADP = 9.5 # mM
ATP = 0.5 # mM

Mg  = 1e-3
psi = 180
ATP = 0.5e-3
ADP = 9.5e-3
Pi  = 1e-3
Mg  = 1e-3
H   = 10**(-7.2)
K   = 100e-3
ATPe = 7.5e-3;
ADPe = 0.05e-3;

pH_outter = 7.2
pH_inner = 7.4
VOLe = 2  #4.54e3
VOLx = 1 #0.65
init_conditions = np.array([ATP, ADP, Pi, ATPe, ADPe])
# J_ATPase(psi, pH_inner, pH_outter, Mg, init_conditions)
def dSTATES_dt(STATES, t, psi, pH_inner, pH_outter, Mg):
    rate = J_ATPase(psi, pH_inner, pH_outter, Mg, STATES)
    dATP = -(J_ANT(psi, pH_inner, pH_outter, Mg, STATES)- rate)/VOLx
    dADP = (J_ANT(psi, pH_inner, pH_outter, Mg, STATES)- rate)/VOLx
    dPi = 0
    dATPe = J_ANT(psi, pH_inner, pH_outter, Mg, STATES) /VOLe
    dADPe = -J_ANT(psi, pH_inner, pH_outter, Mg, STATES) /VOLe
    return (dATP, dADP, dPi, dATPe, dADPe )
```

```python
pH_outter = 7.2
pH_inner = 7.4
Mg  = 1e-3
psi = 180

t = np.linspace(0,2)

dSTATES_dt(init_conditions, t, psi, pH_inner, pH_outter, Mg)

ode_results = odeint(dSTATES_dt,init_conditions, t, (psi, pH_inner, pH_outter, Mg)).T
plt.plot(t,ode_results[0], label = 'ATP')
plt.plot(t,ode_results[1], label = 'ADP')
plt.plot(t,ode_results[2], label = 'Pi')
plt.plot(t,ode_results[3], label = 'ATPe')
plt.plot(t,ode_results[4], label = 'ADPe')

plt.legend()
ode_results[:,-1][0]/ode_results[:,-1][1]
```

```python
RT*np.log(0.2)
```

    -4150.325422837771

### Mg Analysis

```python
Mg_array = np.linspace(1e-4, 1e-2)
SS_end = []
for Mg in Mg_array:
    ode_results = odeint(dSTATES_dt,init_conditions, t, (psi, pH_inner, pH_outter, Mg)).T
    SS_end.append(ode_results[:,-1])
SS_end= np.array(SS_end)

plt.figure()
plt.plot(Mg_array,SS_end[:,0])
Mg  = 1e-3
```

### Psi Analysis

```python
psi_array = np.linspace(120, 220)
SS_end = []
for psi in psi_array:
    ode_results = odeint(dSTATES_dt,init_conditions, t, (psi, pH_inner, pH_outter, Mg)).T
    SS_end.append(ode_results[:,-1])
SS_end= np.array(SS_end)

plt.figure()
plt.plot(psi_array,SS_end[:,0])
plt.plot(psi_array,SS_end[:,1])
plt.plot(psi_array,SS_end[:,2])
psi  = 180
```

### Inner pH

```python
pH_array = np.linspace(7, 8)
SS_end = []
for pH_inner in pH_array:
    ode_results = odeint(dSTATES_dt,init_conditions, t, (psi, pH_inner, pH_outter, Mg)).T
    SS_end.append(ode_results[:,-1])
SS_end= np.array(SS_end)

plt.figure()
plt.plot(pH_array,SS_end[:,0])
plt.plot(pH_array,SS_end[:,1])
plt.plot(pH_array,SS_end[:,2])
psi  = 180
```

## 2.7 Buiding on the model developed in the previous section, add an extramitochondrial cytoplasmic compartment coupled to the mitochondria with the kinetic model for the adenine nucleotide translocase (ANT) of Bazil et al. https://doi.org/10.1016/j.bpj.2015.09.036. (The model formulation and kinetic parameters are in the supplementary material for the paper.) With constant $\Delta\Psi$ and inorganic phosphate concentrations, simulate how matrix and cytoplasmic ATP and ADP concentration depend on cytoplasmic ATP hydrolysis rate.

To recap, the free energy for the ATP synthesis reaction
\begin{equation}
	{\rm ATP}_{x} + {\rm ADP}_{e} \leftrightharpoons
    {\rm ATP}_{e} + {\rm ADP}_{x},
\end{equation}

\begin{equation}
  \Delta G_r =  - E_{ANT}( \frac{[{\rm H}^+]_{\rm inside}[{\rm HPO}_4^{2-}][{\rm ADP}^{3-}]}
      {[{\rm ATP}^{4-}]} \right)
\end{equation}

```python

```

```python

```

Python Code/Quantitative Aspects of Mitochondrial FunctionJun30DAN.ipynb


# Lecture 15

## Systems of Differential Equations II:

### System Simplification

```python
import numpy as np
import sympy as sp
import scipy.integrate
sp.init_printing()
##################################################
##### Matplotlib boilerplate for consistency #####
##################################################
from ipywidgets import interact
from ipywidgets import FloatSlider
from matplotlib import pyplot as plt

import cmath

%matplotlib inline

from IPython.display import set_matplotlib_formats
set_matplotlib_formats('svg')

global_fig_width = 8
global_fig_height = global_fig_width / 1.61803399
font_size = 12

plt.rcParams['axes.axisbelow'] = True
plt.rcParams['axes.edgecolor'] = '0.8'
plt.rcParams['axes.grid'] = True
plt.rcParams['axes.labelpad'] = 8
plt.rcParams['axes.linewidth'] = 2
plt.rcParams['axes.titlepad'] = 16.0
plt.rcParams['axes.titlesize'] = font_size * 1.4
plt.rcParams['figure.figsize'] = (global_fig_width, global_fig_height)
plt.rcParams['font.sans-serif'] = ['Computer Modern Sans Serif', 'DejaVu Sans', 'sans-serif']
plt.rcParams['font.size'] = font_size
plt.rcParams['grid.color'] = '0.8'
plt.rcParams['grid.linestyle'] = 'dashed'
plt.rcParams['grid.linewidth'] = 2
plt.rcParams['lines.dash_capstyle'] = 'round'
plt.rcParams['lines.dashed_pattern'] = [1, 4]
plt.rcParams['xtick.labelsize'] = font_size
plt.rcParams['xtick.major.pad'] = 4
plt.rcParams['xtick.major.size'] = 0
plt.rcParams['ytick.labelsize'] = font_size
plt.rcParams['ytick.major.pad'] = 4
plt.rcParams['ytick.major.size'] = 0
##################################################
```

## Plan for this lecture
Recap

- So far we have looked at systems of **first order**, **linear** ODEs in **two dimensions**
    - Are you comfortable with the words in bold?

- These systems can be solved analytically
    - 3 methods of solving systems of first order ODEs
    - Diagonalisation extends to $N$-dimensional systems

Plan

- Aim to look at systems of **first order**, **nonlinear** ODEs in **more dimensions**
- How we go about modelling a problem
- Simplifying systems of ODEs
    - Reducing number of parameters
    - Reducing number of equations

- Phase plane analysis

## System simplification --- Rescaling

This predator-prey model is in dimensional terms and contains 5 parameters.

\begin{eqnarray*}
\frac{{\rm d}N}{{\rm d}T} &=& rN\left(1-\frac{N}{K}\right) - bNP,\\
\frac{{\rm d}P}{{\rm d}T} &=& ebNP - mP.
\end{eqnarray*}

- $r$: prey intrinsic growth rate
- $K$: prey carrying capacity (bound to logistic growth)
- $b$: predation rate
- $e$: conversion efficiency
- $m$: per capita predator mortality rate

We substitute

- $N=\theta{n}$
- $P=\phi{p}$
- $T=\tau{t}$

where $n,p,t$ are nondimensional, and the $\theta$ is a typical value for $N$ etc.

Note that $\frac{{\rm d}}{{\rm d}T} = \frac{{\rm d}}{{\rm d}t}\frac{{\rm d}t}{{\rm d}T} = \frac{1}{\tau}  \frac{{\rm d}}{{\rm d}t}.$

Original:

\begin{eqnarray*}
\frac{{\rm dN}}{{\rm dT}} &=& rN\left(1-\frac{N}{K}\right) - bNP,\\
\frac{{\rm dP}}{{\rm dT}} &=& ebNP - mP.
\end{eqnarray*}

Rescaled:

\begin{eqnarray*}
\frac{\theta}{\tau}\frac{{\rm d}n}{{\rm d}t} &=& {\theta}rn\left(1-\frac{{\theta}n}{K}\right) - bnp{\theta}{\phi},\\
\frac{\phi}{\tau}\frac{{\rm d}p}{{\rm d}t} &=& ebnp{\theta}{\phi} - mp{\phi}.
\end{eqnarray*}

Note that $\theta,$ $\phi,$ and $\tau$  are arbitrary values for
scaling $N,$ $P,$ and $T$.

\begin{eqnarray*}
\frac{{\rm d}n}{{\rm d}t} &=& \tau\left( rn\left(1-\frac{{\theta}n}{K}\right) - bnp{\phi}\right),\\
\frac{{\rm d}p}{{\rm d}t} &=& \tau\left( ebnp{\theta} - mp\right).
\end{eqnarray*}

After rearranging we note that choosing $\theta=K$ will remove the
parameter $K$ from the equations.  Choosing $\tau = 1/m$ (or $\tau =
1/r$) will remove a further parameter.

\begin{eqnarray*}
\dot{n} &=& \left( \frac{r}{m}n(1-n) - \frac{b{\phi}}{m}np\right),\\
\dot{p} &=& \left( \frac{ebpK}{m}n - p\right).
\end{eqnarray*}

$\phi=m/b$ gives the two parameter system:

\begin{eqnarray*}
\dot{n} &=& \left( \frac{r}{m}n(1-n) - np\right),\\
\dot{p} &=& \left( \frac{ebpK}{m}n - p\right),
\end{eqnarray*}

where we have shifted the groups of parameters to only two places.

We could make it more obvious that there are only two "tuning dials" by writing it as:

\begin{eqnarray*}
\dot{n} &=&  v_1 n(1-n) - np,\\
\dot{p} &=&  p(v_2 n - 1),
\end{eqnarray*}

## System simplification --- Conservation

Consider a chemical system where a product is formed from a substrate
via an intermediate complex involving an enzyme.
$$
S + E
\overset{k_1}{\underset{k_{-1}}\rightleftharpoons}
C
\overset{k_2}\rightarrow
P + E
$$

This system can be written as a system of 4 ODEs for $s, e, c,
p$ which describe the concentrations of $S, E, C$ and $P$.
\begin{eqnarray*}
\dot{s} &=& -k_1 se + k_{-1}c, \\
\dot{e} &=& -k_1 se + k_{-1}c + k_{2}c,\\
\dot{c} &=& k_1 se - k_{-1}c - k_{2}c, \\
\dot{p} &=& k_{2}c.
\end{eqnarray*}

However the enzyme is recycled: it is used in the complex and then
released.  This means that
$e + c = e_{tot}$ where $e_{tot}$ is constant.

Making the substitution $e =  e_{tot} - c$ to eliminate $e$ we arrive at the 3 ODE
system:

\begin{eqnarray}
\dot{s} &=& -k_1 s(e_{tot} - c) + k_{-1}c, \\
\dot{c} &=& k_1 s(e_{tot} - c) - k_{-1}c - k_{2}c,  \label{eq:cdot}\\
\dot{p} &=& k_{2}c.
\end{eqnarray}

## System simplification --- Quasi-steady state

In the previous example it may be known that the rate of conversion to
complex (from $k_1$ and $k_{-1}$) is much faster than the degradation
of complex into product (rate $k_2$).

If this is the case then we can assume that the concentration of $C$
arrives at some level $c^*$ (a quasi-steady state) and then moves very
slowly.

Solve equation on previous slide for $\dot{c}=0$:

\begin{eqnarray*}
\dot{c} &=& k_1 s(e_{tot} - c) - k_{-1}c - k_{2}c = 0, \\
\implies
c^*&=&\frac{k_1  e_{tot} s}{k_{-1} + k_2   + k_1 s},
\end{eqnarray*}
which is **not a steady-state**: it moves slowly with $s$.

Substituting this quasi-steady state into the remaining equations gives
an **approximate** system of 2 ODEs:

\begin{eqnarray*}
\dot{s} &=& -\frac{k_1 k_2 e_{tot} s}{ k_{-1} + k_2 + k_1 s}, \\
\dot{p} &=& \frac{k_1 k_2 e_{tot} s}{ k_{-1} + k_2 + k_1 s}.
\end{eqnarray*}

This means that we have used conservation and quasi-steady state to go
from a 4-dimensional system $(s,e,c,p)$ to a two-dimensional approximation
which captures some of the behaviour.

Two dimensions are good because we can plot their behaviour on a phase
plane diagram...

## Phase planes and nullclines
A system of **nonlinear** ODEs may have more than one  fixed
point (or may have none).  Finding fixed points in two-dimensional
systems is aided by **nullclines**.

A $x$-nullcline is a line where $\dot{x}=0$ and  a $y$-nullcline is a
line where $\dot{y}=0$. (These are straight lines in the linear case.)

For example

\begin{eqnarray*}
\dot{x} &=& x(1-x) -xy,\\
\dot{y} &=& 2y\left(1-\frac{y}{2}\right) - 3xy,
\end{eqnarray*}

has $x$-nullclines at $x=0$ and $1-x-y=0$; and $y$-nullclines at $y=0$ and
$2-3x-y=0$.

Nullcline intersections give us the fixed points.  Nullclines can be
annotated to give the direction (and magnitude) of the non-zero derivative.

```python
def plot_nullclines():

    plt.figure(figsize=(10,10))

    # plot nullclines
    x = np.linspace(-0.1,1.1,24)
    y = np.linspace(-0.1,2.1,24)

    plt.hlines(0,-1,15, color='#F39200', lw=4, label='y-nullcline 1')
    plt.plot(x,1 - x, color='#0072bd', lw=4, label='x-nullcline 2')
    plt.vlines(0,-1,15, color='#0072bd', lw=4, label='x-nullcline 1')
    plt.plot(x,2 - 3*x, color='#F39200', lw=4, label='y-nullcline 2')

    plt.xlim(-0.05,1.1)
    plt.ylim(-0.05,2.1)
    plt.xlabel('x')
    plt.ylabel('y')
```

```python
plot_nullclines()
```

```python
def dX_dt(X, t):
    return np.array([ X[0]*(1. - X[0]) - X[0]*X[1],
                     2.*X[1]*(1.-X[1]/2.) -3*X[0]*X[1]])

def plot_phase_plane():

    plt.figure(figsize=(10,10))

    init_x = [1.05, 0.9, 0.7, 0.5, 0.5, 0.32, 0.1]
    init_y = [1.0, 1.3, 1.6, 1.8, 0.2, 0.2, 0.2]

    plt.plot(init_x, init_y, 'g*', markersize=20)

    for v in zip(init_x,init_y):
        X0 = v                              # starting point
        X = scipy.integrate.odeint( dX_dt, X0, np.linspace(0,10,100))         # we don't need infodict here
        plt.plot( X[:,0], X[:,1], lw=3, color='green')

    # plot nullclines
    x = np.linspace(-0.1,1.1,24)
    y = np.linspace(-0.1,2.1,24)

    plt.hlines(0,-1,15, color='#F39200', lw=4, label='y-nullcline 1')
    plt.plot(x,1 - x, color='#0072bd', lw=4, label='x-nullcline 2')
    plt.vlines(0,-1,15, color='#0072bd', lw=4, label='x-nullcline 1')
    plt.plot(x,2 - 3*x, color='#F39200', lw=4, label='y-nullcline 2')

    # quiverplot - define a grid and compute direction at each point
    X , Y  = np.meshgrid(x, y)                  # create a grid
    DX = X*(1-X) - X*Y                          # evaluate dx/dt
    DY = 2*Y*(1 - Y/2.0) - 3*X*Y                # evaluate dy/dt
    M = (np.hypot(DX, DY))                      # norm growth rate
    M[ M == 0] = 1.                             # avoid zero division errors

    plt.quiver(X, Y, DX/M, DY/M, M)
    plt.xlim(-0.05,1.1)
    plt.ylim(-0.05,2.1)
    plt.xlabel('x')
    plt.ylabel('y')
```

```python
plot_phase_plane()
```

## Summary

- Simplification
    - Rescaling to dimensionless quantities
    - Conservation
    - Quasi-steady state approximation

- Nullclines for finding steady states and phase flow


## Practical Data Science - Classroom to Careers

### Session 2

Today we intend to cover the following topics. We shall look at -
 - What are the areas of application ?
 - What parameters are important when dealing with estimated related to these distributions ?

##### Poisson Distribution
##### Geometric Distribution
##### Exponential Distribution

Also we shall cover the Basics of Linear Algebra to get some hang of matrix operations and try to visualize how this is helpful in representing an equation

##### Basic Linear Algebra, Matrices  - why linear algebraic view is important??

 - Matrix Addition
 - Matrix Multiplication
 - Transpose of a Matrix
 - Diagonal Matrix
 - Identity Matrix
 - Inverse of a Matrix

##### Poisson Distribution

The Poisson distribution is a discrete probability distribution of the number of events (count of events) occurring in a given time period, given the average number of times the event occurs over that time period. For example, number of users visited a website in a given time interval can be thought of a Poisson process

**Parameters**

Poisson distribution is described in terms of the rate (mu) at which the events happen

**Note:** This is ONLY applicable for **time-independent** events

**Poisson probability equation**

If X follows a Poisson distribution, then the probability of observing k events over the time period is

**X** Discrete random variable that represents the number of events observed over a given time period

**lambda (λ)** The expected value (average) of X.

\begin{equation*}
P(X=k) = \frac{\lambda^ke^{-\lambda}}{k!}
\end{equation*}

**Applications**

 - Staffing and Scheduling
 - Finance
 - Disaster Management

```python
from scipy.stats import poisson
import seaborn as sns
import warnings

warnings.filterwarnings(action='ignore')

data_poisson = poisson.rvs(mu=2, size=1000)
ax = sns.distplot(data_poisson,
                  kde=False,
                  color='blue',
                  hist_kws={"linewidth": 15,'alpha':1})
ax.set(xlabel='Poisson prob', ylabel='Frequency / Event Counts')
```

##### Exponential and Geometric Distributions

**Geometric Distribution**

The Geometric distribution is a probability distribution (discrete values) of the number of failures (count of failures) occurring before the first success. For example, probability of k failures in an examination attempt before the success

**Parameters**

Geometric distribution is described in terms of -

**k** no. of failures

**p** the probability of success

**Geometric probability equation**

\begin{equation*}
P(X=k) = (1-p)^k*p
\end{equation*}

**Exponential Distribution**

The exponential distribution is a continuous probability distribution which describes the amount of time it takes to obtain a success in a series of continuously occurring independent trials. It is a continuous analog of the geometric distribution.

#### Bayes Theorem on Conditional Probability

Bayes Theorem or Bayes Rule describes the probability of an event, given prior knowledge of conditions that might be related to the event

It is mathematically expressed as follows

\begin{equation*}
P(A|B) = \frac{P(B|A)P(A)}{P(B)}
\end{equation*}

**Applications**

 - Association Rule Mining
 - Market Basket Analysis (follows from above)

#### Matrix operations

**Matrix Addition/ Subtraction**

We add matrices/each individual elements of matrices together, which produces another Matrix

$$A = \begin{bmatrix} a11 & a12 & a13 \\ a21 & a22 & a23 \\ a31 & a32 & a33 \end{bmatrix}$$

$$B = \begin{bmatrix} b11 & b12 & b13 \\ b21 & b22 & b23 \\ b31 & b32 & b33 \end{bmatrix}$$

$$ A + B = \begin{bmatrix} a11+b11 & a12+b12 & a13+b13 \\ a21+b21 & a22+b22 & a23+b23 \\ a31+b31 & a32+b32 & a33+b33 \end{bmatrix}$$

Matrix addition follows transitive property

A + (-)B = (-)B + A

**Matrix Multiplication**

***Scalar Product***

We multiply a matrix/each individual elements of a matrix by a scalar, which produces another matrix

If we multiply a matrix A by a scalar k, the result is going to be a new matrix (M), which is nothing but a scaled version of matrix A by a factor of k

M = k.A =  $$M = \begin{bmatrix} k*a11 & k*a12 & k*a13 \\ k*a21 & k*a22 & k*a23 \\ k*a31 & k*a32 & k*a33 \end{bmatrix}$$

***Vector Product***

Two Matrices A and B can be multiplies if the following condition is met - **Number of columns in the matrix on the left** should be same as the **Number of rows in the matrix on the right**

The resultant is going to be a Matrix of dimension **(No. of rows in Left Matrix) x (No. of columns in Right Matrix)**

***Determinant of a Square matrix***

The determinant of a square (2x2) matrix is computed as follows -

$$A =   \begin{bmatrix}
    a & b \\
    c & d
  \end{bmatrix}$$

$$D = \begin{equation}
    \det{(A)} = (ad - bc) \\
  \end{equation}$$

**Major Application in formulating model equations**

**y1 = w11*x1 + w12*x2 + w13*x3 + w14*x4**

**y2 = w21*x1 + w22*x2 + w23*x3 + w24*x4**

\begin{equation}
  \begin{bmatrix}
    w11 & w12 & w13 & w14 \\
    w21 & w22 & w23 & w24
  \end{bmatrix}
  %
  \begin{bmatrix}
    x1 \\
    x2 \\
    x3 \\
    x4 \\
  \end{bmatrix}
  %
   = \begin{bmatrix}
    w11*x1 + w12*x2 + w13*x3 +  w14*x4 \\
    w21*x1 + w22*x2 + w23*x3 +  w24*x4 \\
  \end{bmatrix}
\end{equation}

#### Eigenvalues and Eigen Vectors

An eigenvector is a vector whose direction remains unchanged when a linear transformation is applied to it.

Consider the image below in which three vectors are shown. The green square is only drawn to illustrate the linear transformation that is applied to each of these three vectors.

image and content courtesy: www.visiondummy.com

When a given **nxn** Matrix is given we will have to identify a vector **v** such that

  **A**.**v**=λ.**v**

  where -

  **v** is the Eigen Vector of Matrix A

  **λ** is the Eigen value corresponding to the eigen vector **v**

***Understanding with an Example***

Compute the Eigen-value(s) and Eigen-vector(s) corresponding to the Matrix A

$$A = \begin{bmatrix} 2 & 3 \\ 2 & 1 \end{bmatrix}$$

```python

```


Code for HW 2

```python
import numpy as np
```

Q4

```python
v0 = np.array([0.5, 0.5, 0.5, 0.5]).T
v1 = np.array([0.5, 0.5, -0.5, -0.5]).T
v2 = np.array([0.5, -0.5, 0.5, -0.5]).T
```

```python
A = 0.5*np.matrix([[1, 1, 1, 1],[1, 1, -1 ,-1],[1, -1, 1, -1]])
```

```python
y = np.matrix([-0.5, 0.5, 0.5, 1.5]).T
```

```python
from sympy import *
```

```python
a, b, c, d = symbols('a b c d')
v3 = np.array([a, b, c, d])
answer = solve([v0.dot(v3), v1.dot(v3), v2.dot(v3), v3.dot(v3)-1], (a, b, c, d))
```

```python
q1 = np.asarray(answer[0], dtype='float')
q2 = np.asarray(answer[1], dtype='float')

print(q1)
print('------------')
print(q2)
```

    [-0.5  0.5  0.5 -0.5]
    ------------
    [ 0.5 -0.5 -0.5  0.5]

```python
A1 = np.stack((v0, v1, v2,q1),axis = -1)

print(A1)
```

    [[ 0.5  0.5  0.5 -0.5]
     [ 0.5  0.5 -0.5  0.5]
     [ 0.5 -0.5  0.5  0.5]
     [ 0.5 -0.5 -0.5 -0.5]]

```python
A2 = np.stack((v0, v1, v2, q2),axis = -1)

print(A2)
```

    [[ 0.5  0.5  0.5  0.5]
     [ 0.5  0.5 -0.5 -0.5]
     [ 0.5 -0.5  0.5 -0.5]
     [ 0.5 -0.5 -0.5  0.5]]

y = Ac

c = inv(C)*y

```python
from numpy import linalg as la
```

```python
print("Shape of A matrix: ",A1.shape)

print("Shape of y: ", y.shape)
```

    Shape of A matrix:  (4, 4)
    Shape of y:  (4, 1)

Evaluating the coefficients needed to produce y give the bases matrix A1 and A2

```python
A1_prime = la.inv(A1)
print(np.dot(A1_prime,y))
```

    [[ 1.]
     [-1.]
     [-1.]
     [ 0.]]

```python
A2_prime = la.inv(A2)
print(np.dot(A2_prime,y))
```

    [[ 1.]
     [-1.]
     [-1.]
     [ 0.]]

## Q4

```python
y = np.reshape(y, (4,1))
```

```python
q1 = np.reshape(q1, (4,1))
q2 = np.reshape(q2, (4,1))
```

```python
v0 = np.reshape(v0, (4,1))
v1 = np.reshape(v1, (4,1))
v2 = np.reshape(v2, (4,1))
```

```python
Q1 = np.hstack((y,v1,v2,q1))
print(la.matrix_rank(Q1))
```

    4

```python
Q2 = np.hstack((y,v0,v2,q1))
print(la.matrix_rank(Q2))
```

    4

```python
Q3 = np.hstack((y,v0,v1,v2))
print(la.matrix_rank(Q3))
```

    3

```python
Q4 = np.hstack((y,v1,v2,q1-2*v1))
print(la.matrix_rank(Q4))
```

    4

Select option 1,2 and 4

```python

```


# MATH50003 Numerical Analysis: Problem Sheet 7

This problem sheet explores condition numbers, indefinite integration,
and Euler's method.

Questions marked with a ⋆ are meant to be completed without using a computer.
Problems are denoted A/B/C to indicate their difficulty.

```julia
using LinearAlgebra, Plots, Test
```

## 1. Two-point boundary value problems

**Problem 1.1 (C)** Construct a finite-difference approximation to the
forced Helmholtz equation
$$
\begin{align*}
u(0) &= 0 \\
u(1) &= 0 \\
u'' + k^2 u &= {\rm e}^x
\end{align*}
$$
and find an $n$ such  the error is less than $10^{-4}$ when compared
with the true solution for $k=10$:
$$
u(x) = (-\cos(k x) + {\rm e}^x \cos(k x)^2 + \cot(k) \sin(k x) - {\rm e} \cos(k) \cot(k) \sin(k x) - {\rm e} \sin(k) \sin(k x) + {\rm e}^x \sin(k x)^2)/(1 + k^2)
$$

```julia
function helm(k, n)
    x = range(0, 1; length = n)
    h = step(x)
    # TODO: Create a SymTridiagonal discretisation
    T = SymTridiagonal(ones(n-2)*(-2/h^2 + k^2),ones(n-3)*1/h^2)
    u = T \ exp.(x[2:end-1])
    [0; u; 0]
end

k = 10
u = x -> (-cos(k*x) + exp(x)cos(k*x)^2 + cot(k)sin(k*x) - ℯ*cos(k)cot(k)sin(k*x) - ℯ*sin(k)sin(k*x) + exp(x)sin(k*x)^2)/(1 + k^2)

n = 2048 # TODO: choose n to get convergence
x = range(0, 1; length=n)
@test norm(helm(k, n) - u.(x)) ≤ 1E-4
```

    Test Passed
      Expression: norm(helm(k, n) - u.(x)) ≤ 0.0001
       Evaluated: 3.5377802870397806e-5 ≤ 0.0001

**Problem 1.2 (B)** Discretisations can also be used to solve eigenvalue equations.
Consider the Schrödinger equation with quadratic oscillator:
$$
u(-L) = u(L) = 0, -u'' + x^2 u = λ u
$$
Approximate the eigenvalues using `eigvals(A)` (which returns the eigenvalues of a
matrix `A`) with $L = 10$.
Can you conjecture their exact value if $L = ∞$? (Hint: they are integers and the eigenvalues
closest to zero are most accurate.)

**SOLUTION**

```julia
L = 10
n = 1000
x = range(-L,L; length=n)
h = step(x)
eigvals(SymTridiagonal(fill(2/h^2,n-2)  + x[2:end-1].^2, fill(-1/h^2, n-3)))
```

    998-element Vector{Float64}:
         0.9999749492977316
         2.999874743976697
         4.999674327058133
         6.999373691007382
         8.998972828286286
        10.99847173136418
        12.997870392696692
        14.997168804739214
        16.9963669599553
        18.995464850797372
        20.99446246971592
        22.99335980916551
        24.992156861590303
         ⋮
     10020.084643187867
     10020.08464318788
     10026.484841478732
     10026.484841478743
     10033.606931820945
     10033.606931820945
     10041.682004542276
     10041.68200454228
     10051.156993554187
     10051.156993554216
     10063.190188090453
     10063.190188090477

On inspection of the smallest values, it seems that the positive odd integers are the eigenvalues for $L = \infty$. Increasing $L$ (and also $n$) it becomes more obvious:

```julia
L = 100
n = 10000
x = range(-L,L; length = n)
h = step(x)
A = SymTridiagonal(x[2:end-1] .^ 2 .+ 2/h^2,ones(n-3)* (-1)/h^2)
sort((eigvals(A)))[1:20]
```

    20-element Vector{Float64}:
      0.9999749943075145
      2.99987496938245
      4.999674913091877
      6.999374818127133
      8.998974676717353
     10.998474481592305
     12.997874225358162
     14.997173900093017
     16.996373498599244
     18.99547301311537
     20.994472436336384
     22.99337176079773
     24.992170978749115
     26.99087008278113
     28.98946906533
     30.987967918813577
     32.986366635754024
     34.98466520883829
     36.982863630196455
     38.9809618924304

**Problem 1.3⋆ (A)** Consider Helmholtz with Neumann conditions:
$$
u'(0) = c_0 \\
u'(1) = c_1 \\
u_{xx} + k^2 u = f(x)
$$
Write down the finite difference approximation approximating
$u(x_k) ≈ u_k$ on
 an evenly spaced grid $x_k = (k-1)/(n-1)$ for $k=1,…,n$
using the first order derivative approximation conditions:
$$
\begin{align*}
u'(0) &≈ (u_2-u_1)/h = c_0 \\
u'(1) &≈ (u_n-u_{n-1})/h  = c_1
\end{align*}
$$
Use pivoting to reduce the equation to one involving a
symmetric tridiagonal matrix.

**SOLUTION**

We have, with $u(x_k) = u_k$ (and using $\kappa$ instead of $k$ in the equation $u_{xx} + k^2u = f(x)$ so as to avoid confusion with the indices):
\begin{align}
\frac{u_2 - u_1}{h} &= c_0, \\
\frac{u_{k-1} - 2u_k + u_{k+1}}{h^2} + \kappa^2u_k &= f(x_k), \hspace{5mm} \textrm{ for } k=2:n-1\\
\frac{u_n - u_{n-1}}{h} &= c_1,
\end{align}
which we write in matrix form as:

$$
\left[\begin{matrix}
-\frac{1}{h} & \frac{1}{h} \\
\frac{1}{h^2} & \kappa^2 - \frac{2}{h^2} & \frac{1}{h^2} \\
&\ddots & \ddots & \ddots \\
&&\frac{1}{h^2} & \kappa^2 - \frac{2}{h^2} & \frac{1}{h^2} \\
&&& -\frac{1}{h} & \frac{1}{h}
\end{matrix}
\right] \mathbf{u} = \left[\begin{matrix}
c_0 \\ f(x_2)\\ \vdots \\f(x_{n-1}) \\ c_1
\end{matrix}\right],
$$
which we can make symmetric tridiagonal by multiplying the first row by $1/h$ and the final row by $-1/h$:
$$
\left[\begin{matrix}
-\frac{1}{h^2} & \frac{1}{h^2} \\
\frac{1}{h^2} & \kappa^2 - \frac{2}{h^2} & \frac{1}{h^2} \\
&\ddots & \ddots & \ddots \\
&&\frac{1}{h^2} & \kappa^2 - \frac{2}{h^2} & \frac{1}{h^2} \\
&&& \frac{1}{h^2} & -\frac{1}{h^2}
\end{matrix}
\right] \mathbf{u} = \left[\begin{matrix}
\frac{c_0}{h} \\ f(x_2)\\ \vdots \\f(x_{n-1}) \\ -\frac{c_1}{h}
\end{matrix}\right],
$$

## 2. Convergence

**Problem 2.1⋆ (B)** For the equation
$$
\begin{align*}
u(0) &= c_0 \\
u' + au &= f(x)
\end{align*}
$$
where $a ∈ ℝ$ and $0 ≤ x ≤ 1$,
prove convergence as $n → ∞$ for the method constructed in PS6 using the approximation
where we take the average of the two grid points:
$$
{u'(x_{k+1}) + u'(x_k) \over 2} ≈ {u_{k+1} - u_k \over h}.
$$

**SOLUTION**
Using the approximation from PS6 we obtain

$${(f(x_{k+1}) + f(x_k))\over 2} = {( u’(x_{k+1}) + u'(x_k) )\over 2} + {a(u(x_{k+1}) +  u(x_k)) \over 2}≈ {(u_{k+1} - u_k ) \over h} + {a u_{k+1}\over 2} + {a u_k \over 2}$$

So we get

$$\left(\frac{a}{2}-\frac{1}{h}\right)u_k + \left(\frac{a}{2}+\frac{1}{h}\right)u_{k+1} = \frac{f(x_{k+1})+f(x_k)}{2}$$

We want to prove that $\sup_{k=1,...,n-1}|u(x_k) - u_{k}|$ converges to 0 as $n\to \infty$.

Take $\hat u = [u_0,...,u_{n-1}]^T$ and rewrite the system as

$$\hat L \hat u = \begin{bmatrix} c_0 \\ \hat fᶠ \end{bmatrix}$$

where $f_k = {f(x_{k})+f(x_{k-1}) \over 2}$, $k=1,...,n-1$ and

$$\hat L =
\begin{bmatrix}
1 \\
{a\over 2} - {1 \over h} & {a\over 2} + {1 \over h} \\
& {a\over 2} - {1 \over h} & {a\over 2} + {1 \over h}\\
&& \ddots & \ddots \\
&&& {a\over 2} - {1 \over h} & {a\over 2} + {1 \over h}
\end{bmatrix}$$

Note that $\hat L$ is lower bidiagonal.

Now, similarly to Euler's methods convergence theorem, we study consistency and stability.

##### Consistency:
Our discretisation approximates the true equation.

$$\hat Lu = \begin{bmatrix} c_0 \\
{u(x_1) - u(x_0) \over h} + {a\over2}(u(x_1) + u(x_0)) \\
\vdots \\
{u(x_{n-1}) - u(x_{n-2}) \over h}  + {a\over2}(u(x_{n-1}) + u(x_{n-2}))\end{bmatrix}
= \begin{bmatrix} c_0 \\
\frac{1}{2}\left({u(x_1) - u(x_0) \over h} +{u(x_1) - u(x_0) \over h} + {a}(u(x_1) + u(x_0))\right) \\
\vdots \\
\frac{1}{2}\left({u(x_{n-1}) - u(x_{n-2}) \over h} + {u(x_1) - u(x_0) \over h} + {a}(u(x_{n-1}) + u(x_{n-2}))\right)\end{bmatrix}
= \begin{bmatrix} c_0 \\
\frac{1}{2}\left(u'(x_0) + a u(x_0) + u''(τ_0) h + u'(x_1) + a u(x_1) + u''(\sigma_1) h \right)\\
\vdots \\
\frac{1}{2}\left(u'(x_{n-2}) + a u(x_{n-2}) + u''(τ_{n-2}) h + u'(x_{n-1}) + a u(x_{n-1}) + u''(\sigma_{n-1}) h \right) \end{bmatrix} =
\begin{bmatrix} c_0 \\
{f(x_0)+f(x_1)\over 2} + {u''(τ_0)+u''(\sigma_1)\over 2} h \\
\vdots \\
{f(x_{n-2})+f(x_{n-1})\over 2} + {u''(τ_{n-2})+u''(\sigma_{n-1})\over 2} h \end{bmatrix} =
\begin{bmatrix} c_0 \\ \hat fᶠ \end{bmatrix} + \begin{bmatrix} 0 \\ δ \end{bmatrix}$$

where $x_k ≤ τ_k, \sigma_k ≤ x_{k+1}$, and uniform boundedness implies that $\|δ\|_∞ = O(h)$

##### Stability:
The inverse does not blow up the error.

$$\hat L = \underbrace{\begin{bmatrix} 1 \\ & \left({a\over 2} + {1 \over h}\right)^{-1} \\ && ⋱ \\ &&& \left({a\over 2} + {1 \over h}\right)^{-1}  \end{bmatrix}}_D \underbrace{\begin{bmatrix} 1 \\ \left({a\over 2} + {1 \over h}\right)\left({a\over 2} - {1 \over h}\right)  & 1 \\ & ⋱ & ⋱ \\ && \left({a\over 2} + {1 \over h}\right)\left({a\over 2} - {1 \over h}\right) &1 \end{bmatrix}}_{ L}$$

Thus, we have $\| L^{-1}\|_{1 → ∞} ≤ \left|{a^2\over 4} - {1 \over h^2}\right|^{-(n-1)} ≤ \left|{a^2\over 4} - {n^2}\right|^{-(n-1)} \left|{4\over a^2 + 4n^2}\right|^{n-1}
 = O(1)$

Note that $\left|{a\over 2} + {1 \over h}\right|^{-1} = \left|{h \over {ah \over 2} + 1}\right|\le 2h$, as $h\to 0$ (taking $h$ small enough)

Combining stability and consistency we have
$$\|𝐮ᶠ - 𝐮\|_∞  = \|\hat L^{-1} (\hat L𝐮ᶠ - \hat L𝐮)\|_∞  = \| L^{-1} D^{-1} \begin{bmatrix} 0 \\ δ \end{bmatrix} \|_∞ ≤ 2h \| L^{-1}\|_{1 → ∞} \|δ\|_1 = O(h)$$

**Problem 2.2⋆ (A)** Consider the matrices
$$
L = \begin{bmatrix} 1 \\
    -a_1 & 1 \\
        & -a_2 & 1\\
        && ⋱ & ⋱ \\
        &&& -a_{n-1} & 1
\end{bmatrix}, \qquad T = \begin{bmatrix} 1 \\
    -a & 1 \\
        & -a & 1\\
        && ⋱ & ⋱ \\
        &&& -a & 1
\end{bmatrix}.
$$
By writing down the inverse explicitly prove that if $|a_k| ≤ a$ then
$$
\|L^{-1}\|_{1 → ∞} ≤ \|T^{-1}\|_{1 → ∞}.
$$
Use this to prove convergence as $n → ∞$ of forward Euler for
$$
\begin{align*}
u(0) &= c_0 \\
u'(x) - a(x)u(x) &= f(x)
\end{align*}
$$

**SOLUTION**

Since

$$L^{-1}=
\begin{bmatrix}
1 & 0 & 0 & 0 & 0 &... & 0\\
a_1 & 1 & 0 & 0 & 0 &... & 0\\
a_1a_2 & a_2 & 1 & 0 & 0 &... & 0\\
a_1a_2a_3 & a_2a_3 & a_3 & 1 & 0 & ... & 0\\
\vdots & \vdots & \ddots & \ddots & \ddots & \ddots & \vdots \\
\vdots & \vdots & & \ddots & \ddots & 1 & 0 \\
\prod_{i=1}^{n-1}a_i & \prod_{i=2}^{n-1}a_i & ... & ... & a_{n-2}a_{n-1} & a_{n-1} & 1
\end{bmatrix}$$

and

$$T^{-1}=
\begin{bmatrix}
1 & 0 & 0 & 0 & 0 &... & 0\\
a & 1 & 0 & 0 & 0 &... & 0\\
a^2 & a & 1 & 0 & 0 &... & 0\\
a^3 & a^2 & a & 1 & 0 & ... & 0\\
\vdots & \vdots & \ddots & \ddots & \ddots & \ddots & \vdots \\
\vdots & \vdots & & \ddots & \ddots & 1 & 0 \\
a^{n-1} & a^{n-2} & ... & ... & a^2 & a & 1
\end{bmatrix}$$

Then, $\forall x$

$$\|T^{-1}x\|_{\infty}=\max_i|(T^{-1}x)_i|= \max_i \left|x_i +\sum_{j=1}^{i-1}a^{i-j}x_j \right| = \begin{cases}
1 & if \ a\in[0,1] \\
a^{n-1} & if \ a\ge 1
\end{cases}$$

since, given $b=\max\{1,a\}$,

$$\max_i \left|x_i +\sum_{j=1}^{i-1}a^{i-j}x_j \right|\le \max_i \left(|x_i| +\sum_{j=1}^{i-1}\left|a^{i-j}x_j \right| \right) \le b^n\sum_{j=1}^n |x_j| = b^n\|x\|_1$$

thus,

$\|T^{-1}\|_{1\to\infty} = \sup_{x\ne 0} \frac{\|T^{-1}x\|_\infty}{\|x\|_1}\le b^n$ and, in particular,
$$\|T^{-1}\|_{1\to\infty}= b^n$$

since $$\frac{|T^{-1}x|_\infty}{\|x\|_1}=b^n$$ it is obtained using
$$x=\begin{cases}e_1 & b=1 \\ e_n & b=a \end{cases}$$

Moreover, $|a_j|\le b$, $\forall j=1,...,n$, thus,

$$\|L^{-1}x\|_{\infty}=\max_i|(L^{-1}x)_i|= \max_i \left|x_i +\sum_{j=1}^{i-1}a_{j}...a_{i-1}x_j \right| \le \max_i |x_i| +\sum_{j=1}^{i-1}|a_{j}...a_{i-1}x_j | \le b^n\|x\|_1$$

Hence,

$$\|L^{-1}\|_{1\to \infty}=\sup_{x} \frac{\|L^{-1}x\|_{\infty}}{\|x\|_{1}} \le b^n = \|T^{-1}\|_{1 \to \infty}$$

Now we prove convergence for the forward Euler method as $n → ∞$ for

$$
\begin{align*}
u(0) &= c_0 \\
u'(x) &= a(x)u(x) + f(x)
\end{align*}
$$

Using equidistanced (with step $h$) points $x_0,...,x_{n-1}$, we use the approximations $u(x_k) \approx u_k$, where $u_0 = c_0$ and
$$u_{k+1} = u_k + h\left(a(x_k)u_k + f(x_k)\right)$$

In order to study convergence we consider the limit as $n\to\infty$ of
$$\sup_{i=1,...,n-1} |u_i - u(x_i)|$$

Similarly to Euler's methods convergence theorem, we study consistency and stability.

In order to apply the theorem we note that we can define $a_k=a(x_k)$, $k=1,...n-1$ and we have that for every $k$, $|a_k|\le a:= max_{i=1,n-1}|a_i|$.

##### Consistency:
Our discretisation approximates the true equation.

$$\hat Lu = \begin{bmatrix} c_0 \\
{u(x_1) - u(x_0) \over h} - a_1 u(x_0) \\
\vdots \\
{u(x_{n-1}) - u(x_{n-2}) \over h} - a_{n-1} u(x_{n-2})\end{bmatrix} =
\begin{bmatrix} c_0 \\
u'(x_0) - a_1 u(x_0) + u''(τ_0) h \\
\vdots \\
u'(x_{n-2}) - a_{n-1} u(x_{n-2}) + u''(τ_{n-2}) h\end{bmatrix} =
\begin{bmatrix} c_0 \\
f(x_0) + u''(τ_0) h \\
\vdots \\
f(x_{n-2}) + u''(τ_{n-2}) h \end{bmatrix} =
\begin{bmatrix} c_0 \\ 𝐟ᶠ \end{bmatrix} + \begin{bmatrix} 0 \\ δ \end{bmatrix}$$

where $x_k ≤ τ_k ≤ x_{k+1}$, and uniform boundedness implies that $\|δ\|_∞ = O(h)$

##### Stability:
The inverse does not blow up the error.
First write, for $l_k = 1-a_k$

$$\hat L = \underbrace{\begin{bmatrix} 1 \\ & h^{-1} \\ && ⋱ \\ &&& h^{-1} \end{bmatrix}}_D \underbrace{\begin{bmatrix} 1 \\ -l_1 & 1 \\ & ⋱ & ⋱ \\ && -l_{n-1} &1 \end{bmatrix}}_{ L}$$

Thus, we have $\| L^{-1}\|_{1 → ∞} ≤ \|T^{-1}\|_{1 → ∞} = O(1)$

Combining stability and consistency we have
$$\|𝐮ᶠ - 𝐮\|_∞  = \|\hat L^{-1} (\hat L𝐮ᶠ - \hat L𝐮)\|_∞  = \| L^{-1} D^{-1} \begin{bmatrix} 0 \\ δ \end{bmatrix} \|_∞ ≤ h \| L^{-1}\|_{1 → ∞} \|δ\|_1 = O(h)$$

## 3. Fourier series

**Problem 3.1⋆ (C)** Give explicit formulae for $f̂_k$ and $f̂_k^n$ for the following functions:
$$
\cos θ, \cos 4θ, \sin^4θ, {3 \over 3 - {\rm e}^{\rm i θ}}, {1 \over 1 - 2{\rm e}^{\rm i θ}}
$$
Hint: You may wish to try the change of variables $z = {\rm e}^{-{\rm i}θ}$.

**SOLUTION**
The explicit formulae for $f̂ₖ$ can be found by direct computation. Making use of trigonometric identities and noting that cos x and sin x are even and odd respectively on a periodic interval, hence orthogonal

  1) $$f̂ₖ = \frac{1}{2π} \int^{2π}_{0} \cos θ \exp^{-ikθ} dθ = \frac{1}{2π} \int^{2π}_{0} ( \cos θ \cos kθ ) dθ $$
  For $k=1$, yields
      $$f̂ₖ = \frac{1}{2π} \int^{2π}_{0} \cos^2 θ dθ = \frac{1}{2}$$
  and $k \neq 1$ using standard trigonometric identity
  $$f̂ₖ = \frac{1}{4π} \int^{2π}_{0} \cos{θ(1-k)} + \cos{θ(1+k)} dθ = 0$$

  2) similar to 1), replace cases $k =,\neq 1$ by $k =, \neq 4$

  3) Either use Euler's formula to expand $\sin^4θ$ into exponentials or use the identity $$ sin^4 θ = \frac{3 - 4 \cos 2θ + \cos 4 θ}{8} $$
  Noting that $\sin^4 θ$ is an even function yields $$f̂ₖ = \frac{1}{2π} \int^{2π}_{0} \sin^4 θ \exp^{-ikθ} dθ = \frac{1}{2π} \int^{2π}_{0} ( \sin^4 θ \cos kθ ) dθ $$
  Using the above identity we need to take care of the case $k \neq 2,4$, which is zero as we have seen in 1). For special cases  $k = 2, 4$ we have $f̂_2 = - \frac{1}{4}$ and $f̂_4 = \frac{1}{16}$.

  4) We will have to separate the cases $k<0$ and $k \geq 0$.

  ($k \geq 0$) Make the substition $z = {3\rm e}^{-{\rm i}θ}$ and integrating over the contour of the circle of radius 3
  $$f̂ₖ = \frac{-i}{2π}\frac{1}{3^{k+1}} \int_{|z|=3} {z^k \over 1 - z} dz $$
  then by the residue theorem yields $f̂ₖ = \frac{1}{3^{k}}$.

  ($k<0$) Make the substition $z = {\rm e}^{{\rm i}θ}$ and observe that the pole lies outside of the contour of radius $1$, hence $f̂ₖ = 0$ for all $k<0$.

  5) We will have to separate the cases $k<0$ and $k \geq 0$.

  ($k \geq 0$) Make the substition $z = \frac{{\rm e}^{-{\rm i}θ}}{2}$ and observe that the pole lies outside of the contour of radius $\frac{1}{2}$ , hence $f̂ₖ = 0$ for all $k \geq 0$.

  ($k<0$) Make the substition $z = {2 \rm e}^{{\rm i}θ}$ and integrating over the contour of the circle of radius 2
  $$f̂ₖ = \frac{i}{2π}\frac{1}{2^{k+1}} \int_{|z|=2} {z^{k-1} \over z - 1} dz $$
  then by the residue theorem yields $f̂ₖ = \frac{-1}{2^{|k|}}$.

For the student unfamiliar with complex analysis:

   4*) Substitute for $z = \frac{{\rm e}^{-{\rm i}θ}}{3}$ such that $f(θ) = \frac{1}{1-z}$
   then write the function as a geometric series $\Sigma_j z^j$ plug in

   $$f̂ₖ = \frac{1}{2π3^{j}} \int^{2π}_{0} \Sigma_j {\rm e}^{iθ(j-k)} dθ $$

   and finally use the Lemma "Discrete orthogonality" from the lecture notes to show the result above.

   5*) Similar idea to 4) however notice that this time we only have non-zero contributions for $k<0$.

**Problem 3.2⋆ (B)** Prove that if the first $λ-1$ derivatives $f(θ), f'(θ), …, f^{(λ-1)}(θ)$
are 2π-periodic and $f^{(λ)}$ is uniformly bounded  that
$$
|f̂_k| = O(|k|^{-λ})\qquad \hbox{as $|k| → ∞$}
$$
Use this to show for the Taylor case ($0 = f̂_{-1} = f̂_{-2} = ⋯$) that
$$
|f(θ) - ∑_{k=0}^{n-1} f̂_k {\rm e}^{{\rm i}kθ}| = O(n^{1-λ})
$$

**SOLUTION**
A straightforward application of integration by parts yields the result

$$f̂ₖ = \frac{1}{2π} \int^{2π}_{0} f(θ) {\rm e}^{-ikθ} dθ = \frac{(-i)^λ}{2π k^{λ}} \int^{2π}_{0} f^{(λ)}(θ) {\rm e}^{-ikθ} dθ  $$
given that $f^{(λ)}$ is uniformly bounded, the second part follows directly from this result

$$
|∑_{k=n}^{\infty} f̂_k {\rm e}^{{\rm i}kθ}| \leq ∑_{k=n}^{\infty} |f̂_k | \leq C ∑_{k=n}^{\infty} k^{-λ}
$$

for some constant $C$.

**Problem 3.3⋆ (C)**
If $f$ is a trigonometric polynomial  ($f̂_k = 0$ for $|k| > m$) show
for $n ≥ 2m+1$ we can exactly recover $f$:
$$
f(θ) = \sum_{k=-m}^m f̂_k^n {\rm e}^{{\rm i} k θ}
$$

**SOLUTION**
This proof is nearly identical to the proof of "Theorem (Taylor series converges)" in the lecture notes. Only now one has to also subtract the negative coefficients from the negative approximate coefficients in the chain of arguments.

**Problem 3.4⋆ (B)** For the general (non-Taylor) case and $n = 2m+1$, prove convergence for
$$
f_{-m:m}(θ) := ∑_{k=-m}^m f̂_k^n {\rm e}^{{\rm i} k θ}
$$
to $f(θ)$ as $n \rightarrow ∞$.
What is the rate of convergence if the first $λ-1$ derivatives $f(θ), f'(θ), …, f^{(λ-1)}(θ)$
are 2π-periodic and $f^{(λ)}$ is uniformly bounded?

**SOLUTION**

Observe that by aliasing (see corollary in lecture notes) and triangle inequality we have the following

$$ |f̂_k^n - f̂_k|  \leq \sum_{p=1}^{\infty} (|f̂_{k+pm}|+|f̂_{k-pm}|) $$

Using the result from Problem 3.2 yields

$$ |f̂_k^n - f̂_k| \leq \frac{C}{n^\lambda} \sum_{p=1}^{\infty} \frac{1}{\left(p + \frac{k}{n}\right)^\lambda} + \frac{1}{\left(p - \frac{k}{n}\right)^\lambda} $$

now we pick $|q| < \frac{1}{2}$ (such that the estimate below will hold for both summands above) and construct an integral with convex and monotonocally decreasing integrand such that

$$ \left( p + q \right)^{-\lambda} < \int_{p-\frac{1}{2}}^{p+\frac{1}{2}} (x + q)^{-\lambda} dx $$

more over summing over the left-hand side from $1$ to $\infty$ yields a bound by the integral:

$$ \int^{\infty}_{\frac{1}{2}} (x + q)^{-\lambda} dx = \frac{1}{\lambda}(\frac{1}{2} + q)^{- \lambda + 1}$$

Finally let $q = \pm \frac{k}{n}$ to achieve the rate of convergence

$$ |f̂_k^n - f̂_k| \leq \frac{C_{\lambda}}{ n^{\lambda}} \left(  \left( \frac{1}{2} + k/n \right)^{ - \lambda + 1} + \left( \left( \frac{1}{2} - k/n \right) \right)^{- \lambda +1} \right)$$

where $C_{\lambda}$ is a constant depending on $\lambda$. Note that it is indeed important to split the $n$ coefficients equally over the negative and positive coefficients as stated in the notes, due to the estatime we used above.