# Week 10 of Introduction to Biological System Design
## Compiling Chemical Reaction Network Models for Biological Systems
### Ayush Pandey

Pre-requisite: To get the best out of this notebook, make sure that you have basic understanding of chemical reaction networks and ordinary differential equations (ODE). Further, we also use Hill functions to build models of biological systems. Refer to the [E164 class material](pages.hmc.edu/pandey/) for background on any topics in this notebook.

This notebook discusses a Python package called [BioCRNpyler](https://github.com/BuildACell/BioCRNPyler) (pronounced Bio-Compiler) that can be used to compile chemical reaction network models for biological systems.

Disclaimer: The content in this notebook is taken from the BioCRNpyler Github examples.
Copyright: Build-A-Cell.
Package Authors: William Poole, Ayush Pandey, Andrey Shur, Zoltan Tuza, and Richard M. Murray

## Building Chemical Reaction Networks (CRNs) Directly with BioCRNpyler

### What is a CRN?
A CRN is a widely established model of chemistry and biochemistry.
* A set of species $S$
* A set of reactions $R$ interconvert species $I_r$ to $O_r$

\begin{align}
\\
I \xrightarrow[]{\rho(s)} O
\\
\end{align}

 * $I$ and $O$ are multisets of species $S$.
 * $\rho(s): S \to \mathbb{R}$ is a function that determines how fast the reaction occurs.

```python
# Running this notebook for the first time?
# Make sure you have biocrnpyler installed in your environment.
# To install biocrnpyler uncomment the following and run:
# !pip install biocrnpyler
```

```python
#Import everything from biocrnpyler
from biocrnpyler import *
```

    C:\Users\apand\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\compat\_optional.py:138: UserWarning: Pandas requires version '2.7.0' or newer of 'numexpr' (version '2.6.9' currently installed).
      warnings.warn(msg, UserWarning)

## Combining Species and Reactions into a CRN

The following code defines a species called 'S' made out of material 'material'. Species can also have attributes to help identify them. Note that Species with the same name, but different materials or attributes are considered different species in terms of the reactions they participate in.

    S = Species('name', material_type = 'material', attributes = [])

The collowing code produces a reaction R

    R = Reaction(Inputs, Outputs, k)

here Inputs and Outputs must both be a list of Species. the parameter k is the rate constant of the reaction. By default, propensities in BioCRNpyler are massaction:

### $\rho(S) = k \Pi_{s} s^{I_s}$

Note: for stochastic models mass action propensities are $\rho(S) = k \Pi_{s} s!/(s - I_s)!$.

Massaction reactions can be made reversible with the k_rev keyword:

    R_reversible = Reaction(Inputs, Outputs, k, k_rev = krev)

is the same as two reactions:

    R = Reaction(Inputs, Outputs, k)
    Rrev = Reaction(Outputs, Inputs, krev)

Finally, a CRN can be made by combining species and reactions:

    CRN = ChemicalReactionNetwork(species = species, reactions = reactions, initial_condition_dict = {})

Here, initial_condition_dict is an optional dictionary to store the initial values of different species.

    initial_condition_dict = {Species:value}

Species without an initial condition will default to 0.

### An example:

```python
#Example: Model the CRN consisting of: A --> 2B,
# 2B <--> B + C where C has the same name as B but a new material
A = Species("A", material_type = "m1",
            attributes = ["attribute"])
B = Species("B", material_type = "m1")
C = Species("B", material_type = "m2")
D = Species("D")

print("Species can be printed to show"\
      "their string representation:", A, B, C, D)

#Reaction Rates
k1 = 3.
k2 = 1.4
k2rev = 0.15

#Reaciton Objects
R1 = Reaction.from_massaction([A], [B, B], k_forward = k1, k_reverse = 0.9)
R2 = Reaction.from_massaction([B], [C, D], k_forward = k2)

print("\nReactions can be printed as well:\n", R1,"\n", R2)

#create an initial condition so A has a non-zero value
initial_concentration_dict = {A:10}

#Make a CRN
CRN = ChemicalReactionNetwork(species = [A, B, C, D],
                              reactions = [R1, R2],
                              initial_concentration_dict =
                              initial_concentration_dict)

#CRNs can be printed in two different ways
print("\nDirectly printing a CRN shows the string"\
        "representation of the species used in BioCRNpyler:")
print(CRN)

print("\nCRN.pretty_print(...) is a function that prints"\
      "a more customizable version of the CRN, but doesn't"\
      "show the proper string representation of species.")
print(CRN.pretty_print(show_materials = True,
                       show_rates = True, show_attributes = True))
```

    Species can be printed to showtheir string representation: m1_A_attribute m1_B m2_B D

    Reactions can be printed as well:
     m1[A(attribute)] <--> 2m1[B]
     m1[B] --> m2[B]+D

    Directly printing a CRN shows the stringrepresentation of the species used in BioCRNpyler:
    Species = m1_A_attribute, m1_B, m2_B, D
    Reactions = [
    	m1[A(attribute)] <--> 2m1[B]
    	m1[B] --> m2[B]+D
    ]

    CRN.pretty_print(...) is a function that printsa more customizable version of the CRN, but doesn'tshow the proper string representation of species.
    Species(N = 4) = {
    m1[A(attribute)] (@ 10),  D (@ 0),  m2[B] (@ 0),  m1[B] (@ 0),
    }

    Reactions (2) = [
    0. m1[A(attribute)] <--> 2m1[B]
     Kf=k_forward * m1_A_attribute
     Kr=k_reverse * m1_B^2
      k_forward=3.0
      k_reverse=0.9

    1. m1[B] --> m2[B]+D
     Kf=k_forward * m1_B
      k_forward=1.4

    ]

### CRNs can be saved as SBML and simulated

To save a CRN as SBML:

    CRN.write_sbml_file("file_name.xml")

To simulate a CRN with biosrape:

    Results, Model = CRN_expression.simulate_with_bioscrape(timepoints, initial_condition_dict = x0)

Where x0 is a dictionary: x0 = {species_name:initial_value}

```python
# To simulate the CRN, install Bioscrape, a Python-based simulator
# Uncomment the following line to install Bioscrape
# !pip install bioscrape
```

```python
#Saving and simulating a CRN
CRN.write_sbml_file("build_crns_directly.xml")

try:
    import bioscrape
    import numpy as np
    import pylab as plt
    import pandas as pd

    #Initial conditions can be set with a dictionary:
    x0 = {A:120}
    #Timepoints to simulate over
    timepoints = np.linspace(0, 1, 100)

    #This function can also take a filename keyword to
    # save the file at the same time
    R = CRN.simulate_with_bioscrape_via_sbml(timepoints = timepoints,
                                             initial_condition_dict = x0)

    #Check to ensure simulation worked
    #Results are in a Pandas Dictionary and can be accessed
    # via string-names of species
    plt.plot(R['time'], R[str(A)], label = "A")
    plt.plot(R['time'], R[str(B)], label = "B")
    plt.plot(R['time'], R[str(C)], "--", label = "C")
    plt.plot(R['time'], R[str(D)],":", label = "D")
    plt.xlabel('Time')
    plt.ylabel('Species')
    plt.legend()

except ModuleNotFoundError:
    print("Plotting Modules not installed.")
```

## Hill Functions with BioCRNpyler

### HillPositive:
$\rho(s) = k \frac{s_1^n}{K^n+s_1^n}$

Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1".

```python
#create the propensity
R = Species("R")
hill_pos  = HillPositive(k=1, s1=R, K=5, n=2)

#create the reaction
r_hill_pos = Reaction([A], [B], propensity_type = hill_pos)

#print the reaction
print(r_hill_pos.pretty_print())
```

    m1[A(attribute)] --> m1[B]
     Kf = k R^n / ( K^n + R^n )
      k=1
      K=5
      n=2

### HillNegative:
$\rho(s) = k \frac{1}{K^n+s_1^n}$

Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1".

```python
#create the propensity
R = Species("R")
hill_neg  = HillPositive(k=1, s1=R, K=5, n=2)

#create the reaction
r_hill_neg = Reaction([A], [B], propensity_type = hill_neg)

#print the reaction
print(r_hill_neg.pretty_print())
```

    m1[A(attribute)] --> m1[B]
     Kf = k R^n / ( K^n + R^n )
      k=1
      K=5
      n=2

### ProportionalHillPositive:
$\rho(s, d) = k d \frac{s_1^n}{K^n + s_1^n}$

Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1", proportional species "d"

```python
#create the propensity
R = Species("R")
D = Species("D")
prop_hill_pos  = ProportionalHillPositive(k=1, s1=R, K=5, n=2, d = D)

#create the reaction
r_prop_hill_pos = Reaction([A], [B], propensity_type = prop_hill_pos)

#print the reaction
print(r_prop_hill_pos.pretty_print())
```

    m1[A(attribute)] --> m1[B]
     Kf = k D R^n / ( K^n + R^n )
      k=1
      K=5
      n=2

### ProportionalHillNegative:
$\rho(s, d) = k d \frac{1}{K^n + s_1^n}$

Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1", proportional species "d"

```python
#create the propensity
R = Species("R")
D = Species("D")
prop_hill_neg  = ProportionalHillNegative(k=1, s1=R, K=5, n=2, d = D)

#create the reaction
r_prop_hill_neg = Reaction([A], [B], propensity_type = prop_hill_neg)

#print the reaction
print(r_prop_hill_neg.pretty_print())
```

    m1[A(attribute)] --> m1[B]
     Kf = k D / ( 1 + (R/K)^2 )
      k=1
      K=5
      n=2

### General Propensity:
$\rho(s) = $ function of your choice

For general propensities, the function must be written out as a string with all species and parameters declared.

```python
#create species
# create some parameters - note that parameters will be discussed in the next lecture
k1 = ParameterEntry("k1", 1.11)
k2 = ParameterEntry("k2", 2.22)
S = Species("S")

#type the string as a rate then declare teh species and parameters
general = GeneralPropensity(f'k1*2 - k2/{S}^2', propensity_species=[S], propensity_parameters=[k1, k2])

r_general = Reaction([A, B], [], propensity_type = general)

print(r_general.pretty_print())
```

    m1[A(attribute)]+m1[B] -->
    k1*2 - k2/S^2
      k1=1.11
      k2=2.22

## Next week:

### 1. Compiling CRNs with Enzymes Catalysis and Binding
### 2. DNA Assemblies gene expression transcription and translation
### 3. Promoters Transcriptional Regulation and Gene Regulatory Networks
### 4. Simulating and Analyzing SBML models


```python
# Método para resolver las energías y eigenfunciones de un sistema cuántico numéricamente por Teoría de Pertubaciones
# Modelado Molecular 2
# By: José Manuel Casillas Martín     22-oct-2017
import numpy as np
from sympy import *
from sympy.physics.qho_1d import E_n, psi_n
from sympy.physics.hydrogen import E_nl, R_nl
from sympy import init_printing; init_printing(use_latex = 'mathjax')
from scipy import integrate
from scipy.constants import hbar, m_e, m_p, e
from mpmath import spherharm
from numpy import inf, array
import numpy as np
import matplotlib.pyplot as plt
import traitlets
from IPython.display import display
from ipywidgets import Layout, Box, Text, Dropdown, Label, IntRangeSlider, IntSlider, RadioButtons
```

Teoría de Perturbaciones

Consiste en resolver un sistema perturbado(se conoce la solución al no perturbado), y donde el interés es conocer la contribución de la parte perturbada $H'$ al nuevo sistema total.

$$ H = H^{0}+H'$$

La resolución adecuada del problema, depende en gran parte, de una correcta elección de $H'$.

```python
form_item_layout = Layout(display='flex',flex_flow='row',justify_content='space-between')

PType=Dropdown(options=['Particle in a one-dimensional box', 'Harmonic oscilator', 'Hydrogen atom (Helium correction)'])
Pert=Text()
Rang=IntRangeSlider(min=0, max=20, step=1, disabled=False, continuous_update=False, orientation='horizontal',\
                    readout=True, readout_format='d')
M=Text()
Correc=Dropdown(options=['1', '2'])
hbarra=Dropdown(options=[1, 1.0545718e-34])

form_items = [
    Box([Label(value='Problem'),PType], layout=form_item_layout),
    Box([Label(value='Perturbation'),Pert], layout=form_item_layout),
    Box([Label(value='Correction order'),Correc], layout=form_item_layout),
    Box([Label(value='n Range'),Rang], layout=form_item_layout),
    Box([Label(value='Mass'),M], layout=form_item_layout),
    Box([Label(value='Hbar'),hbarra], layout=form_item_layout),]

form = Box(form_items, layout=Layout(display='flex',flex_flow='column',border='solid 2px',align_items='stretch',width='40%'))
form
```

En esta caja interactiva llena los datos del problema que deseas resolver.

# Nota 1:
Es recomendable usar unidades atómicas de Hartree para eficientar los cálculos. 1 u.a. (energía)= 27.211eV.

# Nota 2:
Para la partícula en una caja unidimensional es recomendable que n sea mayor o igual a 1.

## Nota 3:
Para la correción a la energía del átomo de Helio sólo es necesario seleccionar el problema, automáticamente se calcula la correción a primer orden y no se corrigen las funciones de onda.

```python
Problem=PType.value
form_item_layout = Layout(display='flex',flex_flow='row',justify_content='space-between')

L=Text()
W=Text()
atomic_number=RadioButtons(options=['1 (Show Hydrogen energies)','2 (Correct Helium first energy)'],disabled=False)

if Problem=='Particle in a one-dimensional box':
    form_items = [Box([Label(value='Large of box'),L], layout=form_item_layout)]

if Problem=='Harmonic oscilator':
    form_items = [Box([Label(value='Angular Fr'),W], layout=form_item_layout)]

if Problem=='Hydrogen atom (Helium correction)':
    form_items = [Box([Label(value='Atomic number'),atomic_number], layout=form_item_layout)]

form = Box(form_items, layout=Layout(display='flex',flex_flow='column',border='solid 2px',align_items='stretch',width='40%'))
form
```

```python
# Variables que se utilizarán
# x=variable de integracion, l=largo del pozo, m=masa del electrón, w=frecuencia angular
# n=número cuántico principal, Z=Número atómico, q=número cuántico angular(l)
var('x theta phi')
var('r1 r2', real=True)
var('l m hbar w n Z', positive=True, real=True)

# Perturbación
if Pert.value!='':
    H_p=sympify(Pert.value)
    h_p=eval(Pert.value)
else:
    H_p=0
    h_p=0

# Constantes
h=hbarra.value
a0=5.2917721067e-11
if M.value!='':
    mass=float(eval(M.value))
else:
    mass=1

# Energías y funciones que se desea corregir
n_inf=min(Rang.value)
n_sup=max(Rang.value)

if Problem=='Particle in a one-dimensional box':
    if L.value=='':
        large=1
    else:
        large=float(eval(L.value))
    omega=0
    # Energías del pozo de potencial infinito
    k=n*pi/l
    En=hbar**2*k**2/(2*m)

    # Funciones de onda del pozo de potencial infinito
    Psin=sqrt(2/l)*sin(n*pi*x/l)

    # Límites del pozo definido de 0 a l para sympy
    li_sympy=0
    ls_sympy=l
    # Mismo limites para scipy
    li_scipy=0
    ls_scipy=large

if Problem=='Harmonic oscilator':
    large=0
    if W.value=='':
        omega=1
    else:
        omega=float(eval(W.value))
    # Energías del oscilador armónico cuántico
    En=E_n(n,w)

    # Funciones de onda del oscilador armónico cuántico
    Psin=psi_n(n,x,m,w)

     # Límites del pozo definido de -oo a oo para sympy
    li_sympy=-oo
    ls_sympy=oo
    # Límites del pozo definido de -oo a oo para scipy
    li_scipy=-inf
    ls_scipy=inf

if Problem=='Hydrogen atom (Helium correction)':
    if atomic_number.value=='1 (Show Hydrogen energies)':
        z=1
    if atomic_number.value=='2 (Correct Helium first energy)':
        z=2
    large=0
    omega=0

    # Energías del átomo hidrogenoide
    En=z*E_nl(n,z)

    # Funciones de onda del átomo de hidrógeno
    # Número cuántico l=0
    q=0 # La variable l ya esta siendo utilizada para el largo de la caja por ello se sustituyo por q
    Psin=(R_nl(n,q,r1,z)*R_nl(n,q,r2,z))

     # Límites del átomo de hidrógeno de 0 a oo para sympy
    li_sympy=0
    ls_sympy=oo
    # Límites del átomo de hidrógeno de 0 a oo para scipy
    li_scipy=0
    ls_scipy=inf
```

Para sistemas no degenerados, la corrección a la energía a primer orden se calcula como

$$ E_{n}^{(1)} = \int\psi_{n}^{(0)*} H' \psi_{n}^{(0)}d\tau$$

** Tarea 1 : Programar esta ecuación si conoces $H^{0}$ y sus soluciones. **

```python
def correcion_1st_order_Energy(E_n,Psi_n,H_p,li,ls):
    E1_n=Integral(Psi_n*(H_p)*Psi_n,(x,li,ls)).doit()
    return(E_n+E1_n)
```

```python
# Correción de la energía a primer orden
E=[]
Eev=[]
Ec1=[]
if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
    for i in range(n_inf,n_sup+1):
        E.append(En.subs({n:i}))
        Eev.append(E[i-n_inf].subs({m:mass, l:large, hbar:h}).evalf())
        Ec1.append(correcion_1st_order_Energy(En.subs({n:i}),Psin.subs({n:i}),H_p,li_sympy,ls_sympy))
if Problem=='Hydrogen atom (Helium correction)':
    for i in range(n_inf,n_sup+1):
        E.append(En.subs({n:i}))
        Eev.append(E[i-n_inf])
    if z==2:
        integral_1=Integral(Integral((16*z**6*r1*r2**2*exp(-2*z*(r1+r2))),(r2,0,r1)),(r1,0,oo)).doit()
        integral_2=Integral(Integral((16*z**6*r1**2*r2*exp(-2*z*(r1+r2))),(r2,r1,oo)),(r1,0,oo)).doit()
        integral_total=(integral_1+integral_2)
        Ec1.append(E[0]+integral_total)
```

Y la corrección a la función de onda, también a primer orden, se obtiene como:

 $$ \psi_{n}^{(1)} = \sum_{m\neq n} \frac{\langle\psi_{m}^{(0)} | H' | \psi_{n}^{(0)} \rangle}{E_{n}^{(0)} - E_{m}^{(0)}} \psi_{m}^{(0)}$$

**Tarea 2: Programar esta ecuación si conoces $H^{0}$ y sus soluciones. **

```python
# Correción de las funciones a primer orden
if Pert.value!='':
    if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
        Psi_c=[]
        integrals=np.zeros((n_sup+1,n_sup+1))
        for i in range(n_inf,n_sup+1):
            a=0
            for j in range(n_inf,n_sup+1):
                if i!=j:
                    integ= lambda x: eval(str(Psin.subs({n:j})*(h_p)*Psin.subs({n:i}))).subs({m:mass,l:large,w:omega,hbar:h})
                    integrals[i,j]=integrate.quad(integ,li_scipy,ls_scipy)[0]
                    cte=integrals[i,j]/(En.subs({n:i,m:mass,l:large})-En.subs({n:j,m:mass,l:large})).evalf()
                    a=a+cte*Psin.subs({n:j})
            Psi_c.append(Psin.subs({n:i})+a)
```

**Tarea 3: Investigue las soluciones a segundo orden y también programe las soluciones. **

Y la corrección a la energía a segundo orden, se obtiene como:

 $$  E_{n}^{(2)} = \sum_{m\neq n} \frac{|\langle\psi_{m}^{(0)} | H' | \psi_{n}^{(0)} \rangle|^{2}}{E_{n}^{(0)} - E_{m}^{(0)}} $$

```python
# Correción a la energía a segundo orden
if Pert.value!='':
    if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
        if Correc.value=='2':
            Ec2=[]
            for i in range(n_inf,n_sup+1):
                a=0
                for j in range(n_inf,n_sup+1):
                    if i!=j:
                        cte=((integrals[i,j])**2)/(En.subs({n:i,m:mass,l:large,hbar:h})-En.subs({n:j,m:mass,l:large,hbar:h})).evalf()
                        a=a+cte
                Ec2.append(Ec1[i-n_inf]+a)
```

**A continuación se muestran algunos de los resultados al problema resuelto**

Las energías sin perturbación son:

```python
E
```

$$\left [ \frac{\pi^{2} \hbar^{2}}{2 l^{2} m}, \quad \frac{2 \pi^{2} \hbar^{2}}{l^{2} m}\right ]$$

La correción a primer orden de las energías son:

```python
Ec1
```

$$\left [ \frac{\pi^{2} \hbar^{2}}{2 l^{2} m} + \frac{1}{l} \left(- \frac{l^{3}}{2 \pi^{2}} + \frac{l^{3}}{3}\right), \quad \frac{2 \pi^{2} \hbar^{2}}{l^{2} m} + \frac{1}{l} \left(- \frac{l^{3}}{8 \pi^{2}} + \frac{l^{3}}{3}\right)\right ]$$

Si seleccionaste en los parámetros iniciales una correción a segundo orden entonces...

Las correciones a la energía a segundo orden son:

```python
Ec2
```

$$\left [ \frac{\pi^{2} \hbar^{2}}{2 l^{2} m} - 0.00219161594373447 + \frac{1}{l} \left(- \frac{l^{3}}{2 \pi^{2}} + \frac{l^{3}}{3}\right), \quad \frac{2 \pi^{2} \hbar^{2}}{l^{2} m} + 0.00219161594373447 + \frac{1}{l} \left(- \frac{l^{3}}{8 \pi^{2}} + \frac{l^{3}}{3}\right)\right ]$$

Ahora vamos con la función de onda $(\psi)$

```python
form_item_layout = Layout(
    display='flex',
    flex_flow='row',
    justify_content='space-between')

Graph=IntSlider(min=n_inf, max=n_sup, step=1, disabled=False, continuous_update=False, orientation='horizontal',\
                readout=True, readout_format='d')

form_items = [
    Box([Label(value='What function do you want to see?'),
         Graph], layout=form_item_layout)]

form = Box(form_items, layout=Layout(
    display='flex',
    flex_flow='column',
    border='solid 2px',
    align_items='stretch',
    width='40%'))
form
```

La función de onda original es:

```python
Psin.subs({n:Graph.value})
```

$$\frac{\sqrt{2}}{\sqrt{l}} \sin{\left (\frac{2 \pi}{l} x \right )}$$

La correción a primer orden a la función de onda (utilizando todas las funciones en el rango seleccionado) es:

```python
Psi_c[Graph.value-n_inf]
```

$$\frac{\sqrt{2}}{\sqrt{l}} \sin{\left (\frac{2 \pi}{l} x \right )} - \frac{0.0121670900796259 \sqrt{2}}{\hbar^{2} \sqrt{l}} \sin{\left (\frac{\pi x}{l} \right )}$$

Vamos a graficarlas para verlas mejor...

La función de onda original es:

```python
if Problem=='Particle in a one-dimensional box':
    plot(eval(str(Psin)).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),xlim=(li_scipy,ls_scipy),\
         title='$\psi_{%d}$'%Graph.value)
if Problem=='Harmonic oscilator':
    plot(eval(str(Psin)).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),xlim=(-10*h/(mass*omega),10*h/(mass*omega)),\
         title='$\psi_{%d}$'%Graph.value)
if Problem=='Hydrogen atom (Helium correction)':
    print('Densidad de probabilidad para un electrón')
    plot(eval(str((4*pi*x**2*R_nl(Graph.value,q,x,z)**2))),xlim=(0,10),ylim=(0,20/Graph.value), title='$\psi_{%ds}$'%Graph.value)
    print('Tome en cuenta que debido a la dificultad para seleccionar los límites de la gráfica se muestran bien los primeros\n\
3 estados. A partir de ahí visualizar la gráfica se complica.')
```

La corrección a la función de onda es:

```python
if Problem=='Particle in a one-dimensional box':
    if Pert.value!='':
        plot(eval(str(Psi_c[Graph.value-n_inf])).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),\
             xlim=(li_scipy,ls_scipy),title='$\psi_{%d}$'%Graph.value)
    if Pert.value=='':
        print('No se ingreso ninguna perturbación')
if Problem=='Harmonic oscilator':
    if Pert.value!='':
        plot(eval(str(Psi_c[Graph.value-n_inf])).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),\
             xlim=(-10*h/(mass*omega),10*h/(mass*omega)),title='$\psi_{%d}$' %Graph.value)
    if Pert.value=='':
        print('No se ingreso ninguna perturbación')
if Problem=='Hydrogen atom (Helium correction)':
    print('Este programa no corrige las fucniones de un átomo hidrogenoide')
```

**Tarea 4.  Resolver el átomo de helio aplicando los programas anteriores.**

Para el cálculo a las correciones del átomo de Helio se tomó en cuenta lo siguiente...

La función de onda del átomo de Helio puede ser representada como:

$$ \psi_{nlm} = \psi(r1)_{nlm} \psi(r2)_{nlm}$$

Donde, para el estado fundamental:

$$ \psi(r_{1}.r_{2})_{100} = \frac{Z^{3}}{\pi a_{0}^{3}} e^{\frac{-Z}{a_{0}}(r_{1}+r_{2})}$$

Y la perturbación sería el término de repulsión entre los dos electrones, es decir:

$$ H'= \frac{e^{2}}{r_{12}}=\frac{e^{2}}{|r_{1}-r_{2}|}$$

Finalmente la correción a primer orden de la energía sería:

$$ E^{(1)}= \langle\psi_{n}^{(0)} | H' | \psi_{n}^{(0)} \rangle =\frac{Z^{6}e^{2}}{\pi^{2} a_{0}^{6}} \int_{0}^{2\pi}\int_{0}^{2\pi}\int_{0}^{\pi}\int_{0}^{\pi}\int_{0}^{\infty}\int_{0}^{\infty} \frac{e^{\frac{-2Z}{a_{0}}(r_{1}+r_{2})}}{r_{12}} r_{1}^{2}r_{2}^{2}sen{\theta_{1}}sen{\theta_{2}} dr_{2} dr_{1} d\theta_{2} d\theta_{1} d\phi_{2} d\phi_{1}$$

Se utiliza una expansión del termino de repulsión con los armónicos esféricos y se integra la parte angular. Una vez hecho eso, la integral queda expresada de la siguiente manera:

$$ E^{(1)}= \frac{16Z^{6}e^{2}}{a_{0}^{6}} \left[\int_{0}^{\infty} r_{1}^{2} e^{\frac{-2Z}{a_{0}}r_{1}} \left(\int_{0}^{r_{1}} \frac{r_{2}^{2}}{r_{1}} e^{\frac{-2Z}{a_{0}}r_{2}} dr_{2}+\int_{r_{1}}^{\infty}r_{2} e^{\frac{-2Z}{a_{0}}r_{2}}dr_{2}\right) dr_{1} \right]$$

Esta fue la integral que se programó para hacer la correción a la energía del Helio

**Tarea 5: Método variacional-perturbativo. **

Este método nos permite estimar de forma precisa $E^{(2)}$ y correcciones perturbativas de la energía de órdenes más elevados para el estado fundamental del sistema, sin evaluar sumas infinitas. Ver ecuación 9.38 del libro.

$$ \langle u | H^{0} - E_{g}^{(0)} | u \rangle + \langle u | H' - E_{g}^{(1)} | \psi_{g}^{(0)} \rangle +\langle\psi_{g}^{(0)} | H' - E_{g}^{(1)} | u \rangle \geq E_{g}^{(2)} $$

Donde:

u, es cualquier función que se comporte bien y satisfaga condiciones de frontera.

Por lo general, u tiene un parámetro variacional que permite minimizar el lado izquierdo de la ecuación para estimar $E_{g}^{(2)} $. La función u resulta ser una aproximación a $\psi_{g}^{(1)}$, por lo que se puede utilizar está misma función para estimar $E_{g}^{(3)}$ y seguir haciendo iteraciones para hacer correciones de orden superior tanto a la energía como a la función de onda.

Es necesario meter parámetros de optimización a las funciones de prueba para que tenga sentido intentar programar esto. Esto nos limita a usar Sympy para resolver la integral simbólicamente. Entonces, estamos limitados a las capacidades de esta librería para resolver las integrales y a la capacidad de nuestro procesador. Ya corroboramos que un intel I-7 es cerca de 10 veces más rápido que un I-5.

A continuación se muestra un código que, aprovechando los datos introducidos anteriormente, encuentra por medio del método variacional-perturbativo la correción a segundo orden de la energía y a primer orden de la función de onda.

```python
form_item_layout = Layout(display='flex',flex_flow='row',justify_content='space-between')

F_prueba=Text()
Ran_n=IntSlider(min=n_inf, max=n_sup, step=1, disabled=False, continuous_update=False, orientation='horizontal',\
                    readout=True, readout_format='d')
Correcc=RadioButtons(options=[2],disabled=False)

form_items = [
    Box([Label(value='Test function'),F_prueba], layout=form_item_layout),
    Box([Label(value='Correction order'),Correcc], layout=form_item_layout),
    Box([Label(value='Function to correct'),Ran_n], layout=form_item_layout),]

form = Box(form_items, layout=Layout(display='flex',flex_flow='column',border='solid 2px',align_items='stretch',width='40%'))
form
```

Para la función de prueba se espera que el usuario introduzca una función que sea cuadrado-integrable y que sea compatible en las fronteras con el tipo de problema que se esta solucionando. Además, puede intruducir una constante de optimización (utilice "c")

```python
# Variables que se utilizarán
# c y d=constantes de optimización
var('c', real=True)

u=eval(F_prueba.value)
order=Correcc.value
n_n=Ran_n.value

if Problem=='Particle in a one-dimensional box':
    V=0

if Problem=='Harmonic oscilator':
    V=(m/2)*w**2*x**2

if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
    integrando_a=(u)*((-hbar**2)/(2*m)*diff(u,x,2))+(u)*V*(u)-(u)*En.subs({n:n_n})*(u)
    integrando_b=(u)*h_p*(Psin.subs({n:n_n}))-(u)*(Ec1[n_n-n_inf]-En.subs({n:n_n}))*(Psin.subs({n:n_n}))
    integral_a=Integral(eval(str(integrando_a)),(x,li_sympy,ls_sympy)).doit()
    integral_b=Integral(eval(str(integrando_b)),(x,li_sympy,ls_sympy)).doit()
    integral_T=integral_a+2*integral_b
    f_opt=diff(integral_T,c,1)
    c2=solve(f_opt,c)
    E_c2=Ec1[n_n-n_inf]+integral_T.subs({c:c2[0]})
    Psi_c1=(Psin.subs({n:n_n})+u.subs({c:c2[0]}))
```

Las correciones a la energía son:

```python
E[n_n-n_inf]
```

$$\frac{2 \pi^{2} \hbar^{2}}{l^{2} m}$$

```python
Ec1[n_n-n_inf]
```

$$\frac{2 \pi^{2} \hbar^{2}}{l^{2} m} + \frac{1}{l} \left(- \frac{l^{3}}{8 \pi^{2}} + \frac{l^{3}}{3}\right)$$

```python
E_c2
```

$$\frac{2 \pi^{2} \hbar^{2}}{l^{2} m} - \frac{l^{3}}{3 m} \left(\frac{2025 l^{3} m^{2}}{2 \pi^{6} \hbar^{2} \left(-5 + 2 \pi^{2}\right)^{2}} + \frac{2025 l^{3} m^{2}}{\pi^{4} \hbar^{2} \left(-5 + 2 \pi^{2}\right)^{2}}\right) + \frac{1}{l} \left(- \frac{l^{3}}{8 \pi^{2}} + \frac{l^{3}}{3}\right) + \frac{135 l^{6} m}{2 \pi^{8} \hbar^{2} \left(-5 + 2 \pi^{2}\right)} + \frac{2025 l^{6} m}{4 \pi^{6} \hbar^{2} \left(-5 + 2 \pi^{2}\right)^{2}} + \frac{1215 l^{6} m}{2 \pi^{4} \hbar^{2} \left(-5 + 2 \pi^{2}\right)^{2}} + \frac{90 l m \left(- \frac{3 l^{5}}{4 \pi^{5}} + \frac{3 l^{5}}{4 \pi^{3}}\right)}{\pi^{3} \hbar^{2} \left(-5 + 2 \pi^{2}\right)}$$

Las correción a la función de onda:

```python
Psin.subs({n:n_n})
```

$$\frac{\sqrt{2}}{\sqrt{l}} \sin{\left (\frac{2 \pi}{l} x \right )}$$

```python
Psi_c1
```

$$\frac{\sqrt{2}}{\sqrt{l}} \sin{\left (\frac{2 \pi}{l} x \right )} + \frac{45 \sqrt{2} l^{\frac{3}{2}} m x \left(- l + x\right)}{2 \pi^{3} \hbar^{2} \left(-5 + 2 \pi^{2}\right)}$$

Veamos ahora las gráficas de estas funciones...

La función de onda original es:

```python
if Problem=='Particle in a one-dimensional box':
    plot(eval(str(Psin)).subs({n:n_n,m:mass,l:large,w:omega,hbar:h}),xlim=(li_scipy,ls_scipy),\
         title='$\psi_{%d}$'%n_n)
if Problem=='Harmonic oscilator':
    plot(eval(str(Psin)).subs({n:n_n,m:mass,l:large,w:omega,hbar:h}),xlim=(-10*h/(mass*omega),10*h/(mass*omega)),\
         title='$\psi_{%d}$'%n_n)
```

Su correción por medio del método variacional-perturbativo es:

```python
if Problem=='Particle in a one-dimensional box':
    if Pert.value!='':
        plot(Psi_c1.subs({m:mass,l:large,w:omega,hbar:h}),\
             xlim=(li_scipy,ls_scipy),ylim=(-1.5,1.5),title='$\psi_{%d}$'%n_n)
    if Pert.value=='':
        print('No se ingreso ninguna perturbación')
if Problem=='Harmonic oscilator':
    if Pert.value!='':
        plot(Psi_c1.subs({m:mass,l:large,w:omega,hbar:h}),\
             xlim=(-10*h/(mass*omega),10*h/(mass*omega)),title='$\psi_{%d}$' %n_n)
```

**Tarea 6. Revisar sección 9.7. **

Inicialmente a mano, y sengunda instancia favor de intentar programar sección del problema, i.e. integral de Coulomb  e integral de intercambio.

Para comenzar a solucionar este problema, es necesario ser capaz de escribir el termino de repulsión que aparece en la fuerza culombiana ($\frac {1} {r_{12} }$) en términos de los armónicos esféricos de la siguiente manera:

$$ \frac {1} {r_{12} } = \sum_{l=0}^{\infty} \sum_{m=-l}^{l} \frac{4\pi}{2l+1} \frac{r_{<}^{l}}{r_{>}^{l+1}}  \left[Y_{l}^{m}(\theta_{1},\phi_{1})  \right]^{*} Y_{l}^{m}(\theta_{2},\phi_{2})  $$

Intentar programar esto se complica, puesto que los armónicos esféricos sólo han sido definido en librerias de python de forma numérica. Si intentamos programar los armónicos de manera simbólica es necesario utilizar las Polinomios asociados de Legendre, que de igual manera, sólo han sido definidos de manera numérica.

Se podría intentar programar las integral de intercambio de forma numérica pero hacerlo de forma simbólica permitiría simplificar mucho los cálculos, porque al integrar los armónicos esféricos aparecen $\delta_{l,l'}$ y $\delta_{m,m'}$, por lo que la mayoría de las integrales se hacen $0$.

Aún cuando se lograra programar todo esto, hace falta verificar que se seleccionaron correctamente los límites de integración radiales para que el término $\frac{r_{<}^{l}}{r_{>}^{l+1}}$ haga sentido.

```python

```

Perturbaciones/Chema/Ejemplo_variacional-perturbativo_(particula en una caja).ipynb


# Mach-Zehnder Interferometer (MZI)

**We use SiEPIC EBeam library in this tutorial.**

   This notebook walks through the process of setting up and simulating a mach-zehnder interferometer device using the OPICS package.

   A mach-zehnder interferometer is a basic waveguide interference device. It consists of two couplers (or Y branches) connected by two waveguides of different length (see below). The difference between the two waveguide lengths causes differential delay, which contributes to the frequency dependent interference pattern.

```python
import time
import numpy as np
import matplotlib.pyplot as plt
import opics
```

### Import component library
Import `ebeam` library from `libs` module.

```python
ebeam = opics.libraries.ebeam
```

### Define network
Create an instance of `Network` class, which is used to add, connect, and simulate circuit components.

```python
#defining custom frequency data points for a component
f = np.linspace(opics.C*1e6/1.5, opics.C*1e6/1.6, 2000)
circuit_name = "mzi"
circuit = opics.Network(network_id=circuit_name, f=f)
```

```python
ebeam.Waveguide?
```

### Add circuit components

Add grating couplers, 3dB power splitters (e.g. Y-splitter or Y-branch), and waveguides to circuit. You can define custom frequency data points for a component as well (see the example for output_GC).

```python

#define component instances
input_gc  = circuit.add_component(ebeam.GC)
y1 =   circuit.add_component(ebeam.Y)
wg1 =  circuit.add_component(ebeam.Waveguide, params=dict(length=50e-6))
wg2 =  circuit.add_component(ebeam.Waveguide, params=dict(length=150e-6))
y2 =  circuit.add_component(ebeam.Y)
output_gc = circuit.add_component(ebeam.GC)

```

### Define circuit connectivity

In this section, we define the component connections. The connections are defined using `Network.connect`, e.g.

`Network.connect(component1, component1_port, component2, component2_port)`

```python
#define circuit connectivity
circuit.connect(input_gc, 1, y1, 0)
circuit.connect(y1, 1, wg1, 0)
circuit.connect(y1, 2, wg2, 0)
circuit.connect(y2, 0, output_gc, 1)
circuit.connect(wg1, 1, y2, 1)
circuit.connect(wg2, 1, y2, 2)
```

### Simuate the circuit

```python
sim_start = time.time()

#simulate network
circuit.simulate_network()

print("simulation finished in %ss"%(str(round(time.time()-sim_start,2))))
```

### Visualize the simulation result

```python
circuit.sim_result.plot_sparameters(show_freq=False, scale="log")
```

```python
from sympy import *
```

```python
X, a, b, c, d, e, f, g, h = symbols('X a b c d e f g h')
```

```python
simplify((X * (a*b - c*d) + e*f*c - e*g*a - f*h*b + h*g*d)/(a*b - c*d))
```


```python
%matplotlib inline
```

Sequence Models and Long-Short Term Memory Networks
===================================================

At this point, we have seen various feed-forward networks. That is,
there is no state maintained by the network at all. This might not be
the behavior we want. Sequence models are central to NLP: they are
models where there is some sort of dependence through time between your
inputs. The classical example of a sequence model is the Hidden Markov
Model for part-of-speech tagging. Another example is the conditional
random field.

A recurrent neural network is a network that maintains some kind of
state. For example, its output could be used as part of the next input,
so that information can propogate along as the network passes over the
sequence. In the case of an LSTM, for each element in the sequence,
there is a corresponding *hidden state* $h_t$, which in principle
can contain information from arbitrary points earlier in the sequence.
We can use the hidden state to predict words in a language model,
part-of-speech tags, and a myriad of other things.

LSTM's in Pytorch
~~~~~~~~~~~~~~~~~

Before getting to the example, note a few things. Pytorch's LSTM expects
all of its inputs to be 3D tensors. The semantics of the axes of these
tensors is important. The first axis is the sequence itself, the second
indexes instances in the mini-batch, and the third indexes elements of
the input. We haven't discussed mini-batching, so lets just ignore that
and assume we will always have just 1 dimension on the second axis. If
we want to run the sequence model over the sentence "The cow jumped",
our input should look like

\begin{align}\begin{bmatrix}
   \overbrace{q_\text{The}}^\text{row vector} \\
   q_\text{cow} \\
   q_\text{jumped}
   \end{bmatrix}\end{align}

Except remember there is an additional 2nd dimension with size 1.

In addition, you could go through the sequence one at a time, in which
case the 1st axis will have size 1 also.

Let's see a quick example.

```python
# Author: Robert Guthrie

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)
```

    <torch._C.Generator at 0x2b624c779b0>

```python
lstm = nn.LSTM(3, 3)  # Input dim is 3, output dim is 3
inputs = [torch.randn(1, 3) for _ in range(5)]  # make a sequence of length 5

# initialize the hidden state.
init_hidden = (torch.randn(1, 1, 3), torch.randn(1, 1, 3))
hidden = init_hidden

for i in inputs:
    # Step through the sequence one element at a time.
    # after each step, hidden contains the hidden state.
    out, hidden = lstm(i.view(1, 1, -1), hidden)
    print("out: ", out)
    print("hidden: ", hidden)

# alternatively, we can do the entire sequence all at once.
# the first value returned by LSTM is all of the hidden states throughout
# the sequence. the second is just the most recent hidden state
# (compare the last slice of "out" with "hidden" below, they are the same)
# The reason for this is that:
# "out" will give you access to all hidden states in the sequence
# "hidden" will allow you to continue the sequence and backpropagate,
# by passing it as an argument  to the lstm at a later time
# Add the extra 2nd dimension
inputs = torch.cat(inputs).view(len(inputs), 1, -1)
# hidden = (torch.randn(1, 1, 3), torch.randn(1, 1, 3))  # clean out hidden state
hidden = init_hidden

out, hidden = lstm(inputs, hidden)
print("batch out: ", out)
print("final hidden: ", hidden)

```

    out:  tensor([[[-0.0270,  0.1753,  0.3409]]], grad_fn=<CatBackward>)
    hidden:  (tensor([[[-0.0270,  0.1753,  0.3409]]], grad_fn=<ViewBackward>), tensor([[[-0.0525,  0.6791,  0.5669]]], grad_fn=<ViewBackward>))
    out:  tensor([[[ 0.1600, -0.0196,  0.3136]]], grad_fn=<CatBackward>)
    hidden:  (tensor([[[ 0.1600, -0.0196,  0.3136]]], grad_fn=<ViewBackward>), tensor([[[ 0.3229, -0.0641,  0.4017]]], grad_fn=<ViewBackward>))
    out:  tensor([[[ 0.0526,  0.0861, -0.1031]]], grad_fn=<CatBackward>)
    hidden:  (tensor([[[ 0.0526,  0.0861, -0.1031]]], grad_fn=<ViewBackward>), tensor([[[ 0.1281,  0.2225, -0.2335]]], grad_fn=<ViewBackward>))
    out:  tensor([[[0.0883, 0.1529, 0.0473]]], grad_fn=<CatBackward>)
    hidden:  (tensor([[[0.0883, 0.1529, 0.0473]]], grad_fn=<ViewBackward>), tensor([[[0.1546, 0.2758, 0.0848]]], grad_fn=<ViewBackward>))
    out:  tensor([[[-0.0075, -0.0359,  0.0541]]], grad_fn=<CatBackward>)
    hidden:  (tensor([[[-0.0075, -0.0359,  0.0541]]], grad_fn=<ViewBackward>), tensor([[[-0.0090, -0.2157,  0.1439]]], grad_fn=<ViewBackward>))
    batch out:  tensor([[[-0.0270,  0.1753,  0.3409]],

            [[ 0.1600, -0.0196,  0.3136]],

            [[ 0.0526,  0.0861, -0.1031]],

            [[ 0.0883,  0.1529,  0.0473]],

            [[-0.0075, -0.0359,  0.0541]]], grad_fn=<CatBackward>)
    final hidden:  (tensor([[[-0.0075, -0.0359,  0.0541]]], grad_fn=<ViewBackward>), tensor([[[-0.0090, -0.2157,  0.1439]]], grad_fn=<ViewBackward>))

Example: An LSTM for Part-of-Speech Tagging
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In this section, we will use an LSTM to get part of speech tags. We will
not use Viterbi or Forward-Backward or anything like that, but as a
(challenging) exercise to the reader, think about how Viterbi could be
used after you have seen what is going on.

The model is as follows: let our input sentence be
$w_1, \dots, w_M$, where $w_i \in V$, our vocab. Also, let
$T$ be our tag set, and $y_i$ the tag of word $w_i$.
Denote our prediction of the tag of word $w_i$ by
$\hat{y}_i$.

This is a structure prediction, model, where our output is a sequence
$\hat{y}_1, \dots, \hat{y}_M$, where $\hat{y}_i \in T$.

To do the prediction, pass an LSTM over the sentence. Denote the hidden
state at timestep $i$ as $h_i$. Also, assign each tag a
unique index (like how we had word\_to\_ix in the word embeddings
section). Then our prediction rule for $\hat{y}_i$ is

\begin{align}\hat{y}_i = \text{argmax}_j \  (\log \text{Softmax}(Ah_i + b))_j\end{align}

That is, take the log softmax of the affine map of the hidden state,
and the predicted tag is the tag that has the maximum value in this
vector. Note this implies immediately that the dimensionality of the
target space of $A$ is $|T|$.

Prepare data:

```python
def prepare_sequence(seq, to_ix):
    idxs = [to_ix[w] for w in seq]
    return torch.tensor(idxs, dtype=torch.long)

training_data = [
    ("The dog ate the apple".split(), ["DET", "NN", "V", "DET", "NN"]),
    ("Everybody read that book".split(), ["NN", "V", "DET", "NN"])
]
word_to_ix = {}
char_to_ix = {}
for sent, tags in training_data:
    for word in sent:
        if word not in word_to_ix:
            word_to_ix[word] = len(word_to_ix)
        for char in word:
            if char not in char_to_ix:
                char_to_ix[char] = len(char_to_ix)
print("word to ix: ", word_to_ix)
print("char to ix: ", char_to_ix)
tag_to_ix = {"DET": 0, "NN": 1, "V": 2}
print("tag to ix: ", tag_to_ix)
# These will usually be more like 32 or 64 dimensional.
# We will keep them small, so we can see how the weights change as we train.
WORD_EMBEDDING_DIM = 6
FULL_HIDDEN_DIM = 9  # hidden vector state for the full word + its characters
CHAR_EMBEDDING_DIM = 3
CHAR_HIDDEN_DIM = 3
```

    word to ix:  {'The': 0, 'dog': 1, 'ate': 2, 'the': 3, 'apple': 4, 'Everybody': 5, 'read': 6, 'that': 7, 'book': 8}
    char to ix:  {'T': 0, 'h': 1, 'e': 2, 'd': 3, 'o': 4, 'g': 5, 'a': 6, 't': 7, 'p': 8, 'l': 9, 'E': 10, 'v': 11, 'r': 12, 'y': 13, 'b': 14, 'k': 15}
    tag to ix:  {'DET': 0, 'NN': 1, 'V': 2}

Create the model:

```python
class LSTMTagger(nn.Module):
    def __init__(self, word_embedding_dim, full_hidden_dim, vocab_size,
                 tagset_size,
                 char_embedding_dim, char_hidden_dim, char_dict_size,
                 batch_size=1, num_layers=1):
        super(LSTMTagger, self).__init__()
        self.full_hidden_dim = full_hidden_dim
        self.batch_size = batch_size
        self.num_layers = num_layers
        self.embedding_dim = word_embedding_dim
        self.char_embedding_dim = char_embedding_dim
        self.char_hidden_dim = char_hidden_dim
        self.char_dict_size = char_dict_size

        self.char_embeddings = nn.Embedding(char_dict_size, char_embedding_dim)

        self.char_lstm = nn.LSTM(input_size=char_embedding_dim,
                                 hidden_size=char_hidden_dim,
                                 num_layers=num_layers)

        self.char_hidden = self.init_char_hidden()

        self.word_embeddings = nn.Embedding(vocab_size, word_embedding_dim)

        # The LSTM takes word embeddings as inputs, and outputs hidden states
        # with dimensionality hidden_dim.
        self.lstm = nn.LSTM(input_size=word_embedding_dim + char_embedding_dim,
                            hidden_size=full_hidden_dim, num_layers=num_layers)

        # The linear layer that maps from cell state space to tag space
        self.hidden2tag = nn.Linear(full_hidden_dim, tagset_size)

        self.full_hidden = self.init_full_hidden()

    def init_full_hidden(self):
        # Before we've done anything, we don't have any hidden state.
        # Refer to the Pytorch documentation to see exactly
        # why they have this dimensionality.
        # The axes semantics are (num_layers, minibatch_size, hidden_dim)
        hidden_state = torch.zeros(self.num_layers, self.batch_size,
                                   self.full_hidden_dim)
        cell_state = torch.zeros(self.num_layers, self.batch_size,
                                 self.full_hidden_dim)
        return hidden_state, cell_state

    def init_char_hidden(self):
        hidden_state = torch.zeros(self.num_layers, self.batch_size,
                                   self.char_hidden_dim)
        cell_state = torch.zeros(self.num_layers, self.batch_size,
                                 self.char_hidden_dim)
        return hidden_state, cell_state

    def forward(self, sentence):
        char_embeds = []
        for word_ix in sentence:
            char_ix = prepare_sequence(wor)
            char_embed = self.char_embeddings(word)
            self.char_hidden = init_char_hidden()
            char_out, self.char_hidden = self.char_lstm(char_embed.view(
                len(word), self.batch_size, -1))
            char_embeds.append(char_out)

        word_embeds = self.word_embeddings(sentence)

        full_embeds = []
        for i, char_embed in enumerate(char_embeds):
            full_embed = torch.cat((word_embeds[i], char_embed))
            full_embeds.append(full_embed)

        full_embeds = torch.tensor(full_embeds)

        lstm_out, self.hidden = self.lstm(
            full_embeds.view(len(sentence), self.batch_size, -1), self.hidden)
        tag_space = self.hidden2tag(lstm_out.view(len(sentence), -1))
        tag_scores = F.log_softmax(tag_space, dim=1)
        return tag_scores

```

Train the model:

```python
model = LSTMTagger(word_embedding_dim=WORD_EMBEDDING_DIM, full_hidden_dim=FULL_HIDDEN_DIM,
                   vocab_size=len(word_to_ix), tagset_size=len(tag_to_ix),
                   char_embedding_dim=CHAR_EMBEDDING_DIM, char_hidden_dim=CHAR_HIDDEN_DIM,
                   char_dict_size=len(char_to_ix))
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# See what the scores are before training
# Note that element i,j of the output is the score for tag j for word i.
# Here we don't need to train, so the code is wrapped in torch.no_grad()
with torch.no_grad():
    inputs = prepare_sequence(training_data[0][0], word_to_ix)
    tag_scores = model(inputs)
    print(tag_scores)

for epoch in range(
        300):  # again, normally you would NOT do 300 epochs, it is toy data
    for sentence, tags in training_data:
        # Step 1. Remember that Pytorch accumulates gradients.
        # We need to clear them out before each instance
        model.zero_grad()

        # Also, we need to clear out the hidden state of the LSTM,
        # detaching it from its history on the last instance.
        model.hidden = model.init_full_hidden()

        # Step 2. Get our inputs ready for the network, that is, turn them into
        # Tensors of word indices.
        sentence_in = prepare_sequence(sentence, word_to_ix)
        targets = prepare_sequence(tags, tag_to_ix)

        # Step 3. Run our forward pass.
        tag_scores = model(sentence_in)

        # Step 4. Compute the loss, gradients, and update the parameters by
        #  calling optimizer.step()
        loss = loss_function(tag_scores, targets)
        loss.backward()
        optimizer.step()

# See what the scores are after training
with torch.no_grad():
    inputs = prepare_sequence(training_data[0][0], word_to_ix)
    tag_scores = model(inputs)

    # The sentence is "the dog ate the apple".  i,j corresponds to score for tag j
    # for word i. The predicted tag is the maximum scoring tag.
    # Here, we can see the predicted sequence below is 0 1 2 0 1
    # since 0 is index of the maximum value of row 1,
    # 1 is the index of maximum value of row 2, etc.
    # Which is DET NOUN VERB DET NOUN, the correct sequence!
    print(tag_scores)

```

    tensor([[-1.2231, -0.8153, -1.3349],
            [-1.3205, -0.7126, -1.4162],
            [-1.3797, -0.6769, -1.4264],
            [-1.3987, -0.6565, -1.4505],
            [-1.3729, -0.6791, -1.4289]])
    tensor([[-0.0433, -3.4213, -4.6308],
            [-5.0335, -0.0169, -4.5826],
            [-3.4704, -3.5979, -0.0603],
            [-0.1952, -3.1356, -2.0112],
            [-4.6059, -0.0257, -4.1740]])

Exercise: Augmenting the LSTM part-of-speech tagger with character-level features
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the example above, each word had an embedding, which served as the
inputs to our sequence model. Let's augment the word embeddings with a
representation derived from the characters of the word. We expect that
this should help significantly, since character-level information like
affixes have a large bearing on part-of-speech. For example, words with
the affix *-ly* are almost always tagged as adverbs in English.

To do this, let $c_w$ be the character-level representation of
word $w$. Let $x_w$ be the word embedding as before. Then
the input to our sequence model is the concatenation of $x_w$ and
$c_w$. So if $x_w$ has dimension 5, and $c_w$
dimension 3, then our LSTM should accept an input of dimension 8.

To get the character level representation, do an LSTM over the
characters of a word, and let $c_w$ be the final hidden state of
this LSTM. Hints:

* There are going to be two LSTM's in your new model.
  The original one that outputs POS tag scores, and the new one that
  outputs a character-level representation of each word.
* To do a sequence model over characters, you will have to embed characters.
  The character embeddings will be the input to the character LSTM.


**IMPORTANT NOTICE**

Before submitting, **please**, make sure that your notebook runs **without errors** in Python 3.6
and **reproduces your solution as intended**, when you **Restart the Kernel and re-run the whole
notebook**!
You will be severely penalized if you notebook does not run.

Whereever applicable your solution will be graded based on the **plots**, generated by
**your code** on **TA's** computer.

# Home Assignment 4

Please, write your implementation within the designated blocks:
```python
...
### BEGIN Solution

# >>> your solution here <<<

### END Solution
...
```

Write your theoretical derivations within such blocks:
```markdown
**BEGIN Solution**

<!-- >>> your derivation here <<< -->

**END Solution**
```

## $\LaTeX$ in Jupyter
Jupyter has constantly improving $\LaTeX$ support. Below are the basic methods to
write **neat, tidy, and well typeset** equations in your notebooks:
* to write an **inline** equation use
```markdown
$ you latex equation here $
```
* to write an equation, that is **displayed on a separate line** use
```markdown
$$ you latex equation here $$
```
* to write a **block of equations** use
```markdown
\begin{align}
    left-hand-side
        &= right-hand-side on line 1
        \\
        &= right-hand-side on line 2
        \\
        &= right-hand-side on the last line
\end{align}
```
The **ampersand** (`&`) aligns the equations horizontally and the **double backslash**
(`\\`) creates a new line.

# GLM Bayesian Models (35 pt.)

```python
import numpy as np
import pandas as pd

import torch

%matplotlib inline
import matplotlib.pyplot as plt
```

## Task 1 (5 pt.): Theoretical problem

Consider a univariate Gaussian distribution $\mathcal{N}(x; \mu, \tau^{-1})$.
Let's define Gaussian-Gamma prior for parameters $(\mu, \tau)$:

\begin{equation}
    p(\mu, \tau)
        = \mathcal{N}(\mu; \mu_0, (\beta \tau)^{-1})
            \otimes \text{Gamma}(\tau; a, b)
        \,.
\end{equation}

Find the posterior distribution of $(\mu, \tau)$ after observing $X = (x_1, \dots, x_n)$.

**BEGIN Solution**

\begin{align}
&p(\mu,\tau \vert X) \propto p(X\vert \mu,\tau) p(\mu, \tau) = \frac{\tau^{n+1}}{(\sqrt{2\pi})^n+1}\prod_{i=1}^n \left[ e^{-\frac{(x_i-\mu)^2}{2}\tau^2}\right] \frac{\beta \tau}{\sqrt{2\pi}} e^{-\frac{(\beta \tau)^2(\mu - \mu_0)^2}{2}} \frac{1}{\Gamma(a)b^a} \tau^{a-1} e^{-\frac{\tau}{b}} = \\
&= \frac{\beta \tau^{n+a}}{(\sqrt{2\pi})^{n+1}}e^{\left[ -\tau^2\sum_{i=1}^n(x_i-\mu)^2 - (\mu-\mu_0)^2\beta\tau^2 - \tau/b\right]/2}\frac{1}{\Gamma(a)b^a}
\end{align}

**END Solution**

## Task 2 (10 pt.): Practical problem

Evaluate the following integral using the Laplace approximation:
\begin{equation}
    x \mapsto \int \sigma(w^T x) \mathcal{N}(w; 0, \Sigma) dw \,,
\end{equation}
for $x = \bigl(\tfrac23, \tfrac16, \tfrac16\bigr)\in \mathbb{R}^3$ and
\begin{equation}
    \Sigma
        = \begin{pmatrix}
             1    & -0.25 & 0.75 \\
            -0.25 &  1    & 0.5  \\
             0.75 &  0.5  & 2
           \end{pmatrix}
        \,.
\end{equation}

### Part 2.1 (4 pt.)
Use the Hessian matrix computed numericaly via finite differences. (Check out [Numdifftools](https://pypi.python.org/pypi/Numdifftools))

```python
### BEGIN Solution
import scipy.stats as spstats
import numdifftools as dift
import scipy.optimize as scopt

def f(z):
    return 1/(1+np.exp(-2/3*z[0]-1/6*z[1]-1/6*z[2])) * spstats.multivariate_normal(mean=[0,0,0],cov=np.array([[1,-0.25,0.75],[-0.25,1,0.5],[0.75,0.5,2]])).pdf(z)

def log_f(z):
    return -np.log(f(z)) #minus for scopt.min to maximize

def getHess(z):
    return dift.core.Hessdiag(log_f)(z)

#print(getHess(np.array([5,8,7])))

#find optimal solution
w0=scopt.minimize(log_f, x0=[1,1,1]).x

#Exclude off-diagonal terms
approximatedIntegral = f(w0)* np.sqrt(2*np.pi)**3 / np.sqrt(np.prod(getHess(w0)))
print(approximatedIntegral)

### END Solution
```

    0.296430875553

### Part 2.2 (4 pt.)
Use the diagonal approximation of the Hessian computed by autodifferentiation
in **pytorch**.

```python
### BEGIN Solution
#It does not work

Sigma=np.array([[1,-0.25,0.75],[-0.25,1,0.5],[0.75,0.5,2]])

def MyMultivariateNormal(x):
    return np.float(1/(np.power(np.sqrt(2*np.pi*np.linalg.det(Sigma)),3))) * torch.exp(-1/2 * (x[0]*x[0]+ x[1]*x[1]+2*x[2]*x[2]+x[1]*x[2]-0.5*x[0]*x[1]+1.5*x[0]*x[2]))

def torchp(z):
    print(MyMultivariateNormal(z) )
    return torch.sigmoid(-2/3*z[0]-1/6*z[1]-1/6*z[2]) * MyMultivariateNormal(z)

def torchlog_p(z):
    return torch.log(torchp(z))

def hessian(func, point):
    theta = torch.autograd.Variable(torch.FloatTensor(point), requires_grad=True)
    y = func(theta)
    print(y)
    #grad_params =
    hessian = torch.autograd.Variable(torch.Tensor([[torch.autograd.grad(torch.autograd.grad(y, theta)[i], theta, create_graph=True)[j] for i in [0,1,2]] for j in[0,1,2]]))
    print("###",torch.autograd.grad(y, theta,create_graph=True,only_inputs=False)[0])
    return 0#hessian.data.numpy()

#hessian(torchlog_p,w0) #differentiated input is unreachable??

### END Solution
```

### Part 2.3 (2 pt.)

Compare the results

```python
### BEGIN Solution

#Use Monte-Carlo to estimate the mean
def mySigmoid(x):
    return 1/ (1+np.exp(-x))

sample=np.random.multivariate_normal(mean=np.array([0,0,0]),cov=np.array([[1,-0.25,0.75],[-0.25,1,0.5],[0.75,0.5,2]]),size=1000)

monteCarloApproximation = np.mean(mySigmoid(sample.dot(np.array([2/3,1/6,1/6]))))
print("MonteCarlo:",monteCarloApproximation)
print("FiniteDifferenceHessian:",approximatedIntegral)

### END Solution
```

    MonteCarlo: 0.498888326427
    FiniteDifferenceHessian: 0.296430875553

## Task 3 (10 pt.): Gaussian Process (theoretical)

### Part 3.1 (5 pt.)

Assuimng the matrices $A \in \mathbb{R}^{n \times n}$ and $D \in \mathbb{R}^{d \times d}$
are invertable, using **gaussian elimination** find the inverse matrix for the following
block matrix:
\begin{equation}
    \begin{pmatrix} A & B \\ C & D \end{pmatrix} \,,
\end{equation}
where $C \in \mathbb{R}^{d \times n}$ and $B \in \mathbb{R}^{n \times d}$.

**BEGIN Solution**
The matrix $F$ is the inverse of given one if

$$\begin{bmatrix}
A & B\\
C & D\\
\end{bmatrix}\begin{bmatrix}
F_1 & F_2\\
F_3 & F_4\\
\end{bmatrix} = I,$$
where $I$ is an identity matrix of proper size. Therefore the following equations should be satisfied:

\begin{align*}
&AF_1 + BF_3 = I_n,\\
&AF_2 + BF_4 = 0,\\
&CF_1 + DF_3 = 0,\\
&CF_2 + DF_4 = I_d.
\end{align*}

From the zero equalities we can obtain $F_2 = -A^{-1}BF_4, F_3=-D^{-1}CF_1$ and after substitution to the conditions the result is

$$
F_1 = (I-A^{-1}BD^{-1}C)^{-1}A^{-1}, \quad F_4 = (I-D^{-1}CA^{-1}B)^{-1}D^{-1}, \quad F_2 = -A^{-1}BF_4, \quad F_3=-D^{-1}CF_1.
$$
**END Solution**

### Part 3.2 (5 pt.)

Assume, that the function $y(x)$, $x \in \mathbb{R}^d$, is a realization of the Gaussian
Process $GP\bigl(0; K(\cdot, \cdot)\bigr)$ with $K(a, b) = \exp({- \gamma \|a - b\|_2^2}))$.

Suppose two datasets were observed: **noiseless** and **noisy**
\begin{aligned}
    & D_0 = \bigl(x_i, y(x_i) \bigr)_{i=1}^{n} \,, \\
    & D_1 = \bigl(x'_i, y(x'_i) + \varepsilon_i \bigr)_{i=1}^{m} \,,
\end{aligned}
where $\varepsilon_i \sim \text{ iid } \mathcal{N}(0, \sigma^2)$.

Derive the conditional distribution of $y(x) \big\vert_{D_0, D_1}$ at a new $x$.

**BEGIN Solution**

Consider the vector $\left[ y(x_i)_{i=1}^n, (y(x'_i)+\epsilon_i)_{i=1}^n, y(x) \right]$. Since $y(x)$ is a Gaussian process and $\epsilon_i$ are Gaussian variables, the whole vector has joint Gaussian distribution with mean $0$ and covariance matrix
$$
K=\begin{bmatrix}
K(x_i,x_j)_{i,j=1}^n & k_* & \text{Cov}(y(x),y(x_i))^T\\
k_*^{T} & K(x'_i,x'_j)_{i,j=1}^m + \sigma^2 I_m & \text{Cov}(y(x),y(x'_i))^T \\
\text{Cov}(y(x),y(x_i)) & \text{Cov}(y(x),y(x'_i)) & 1
\end{bmatrix},
$$

where $1=Var(y(x))$ by definition of the process, $k_{*ij}=\text{Cov}(y(x_i),y(x'_j)+\epsilon_j)=\text{Cov}(y(x_i),y(x'_j))$. Denote
$$
A_{11}=\begin{bmatrix}
K(x_i,x_j)_{i,j=1}^n & k_* \\
k_*^{T} & K(x'_i,x'_j)_{i,j=1}^m + \sigma^2 I_m
\end{bmatrix}, A_{12} = \begin{bmatrix}\text{Cov}(y(x),y(x_i))^T \\ \text{Cov}(y(x),y(x'_i))^T\end{bmatrix}.
$$

We could use the fact that any subvector of Gaussian vector is Gaussian with mean $\mu_*=A_{12}^T A_{11}^{-1}\left[y(x_i)_{i=1}^n ,y(x'_j)_{j=1}^m \right]^T$  and covariance matrix (in 1d-case just the variance)

$$
\sigma_*^2 = 1 - A_{12}^{T}A_{11}^{-1}A_{12}.
$$

As result,
$$
y(x)\vert_{D_0,D_1} \sim \mathcal{N}(\mu_*,\sigma_*^2).
$$

**END Solution**

## Task 4 (10 pt.): Gaussian Process (practical)

> In the late 1950’s Charles Keeling invented an accurate way to measure atmospheric $CO_2$ concentration and began taking regular measurements at the Mauna Loa observatory.

Download the data from http://scrippsco2.ucsd.edu/data/atmospheric_co2/mlo

### Part 4.1 (3 pt.): Preprocessing

Take `monthly_in_situ_co2_mlo.csv` dataset, load it and prepare the data.

**ATTENTION** The first couple of lines of the file contain the description of
the dataset. Make sure to **read it** carefully.

Here is what the dataset might look like when loaded:

      Yr
      Mn
      Date
      Date
      CO2
      seasonally
      fit
      seasonally
      CO2
      seasonally

      adjusted

      adjusted fit
      filled
      adjusted filled

      Excel

      [ppm]
      [ppm]
      [ppm]
      [ppm]
      [ppm]
      [ppm]

      45
      1961
      10
      22569
      1961.7890
      315.31
      318.31
      314.93
      317.92
      315.31
      318.31

      142
      1969
      11
      25522
      1969.8740
      322.85
      324.80
      323.15
      325.08
      322.85
      324.80

      168
      1972
      1
      26313
      1972.0410
      326.77
      326.74
      326.82
      326.78
      326.77
      326.74

      179
      1972
      12
      26648
      1972.9563
      327.55
      328.36
      327.71
      328.51
      327.55
      328.36

      249
      1978
      10
      28778
      1978.7890
      332.55
      335.74
      332.71
      335.89
      332.55
      335.74

Your task:
0. Load the `CO2 [ppm]` time series
1. Replace $-99.99$ with **NaN** and drop the missing observations
2. Construct a *decimal years time index*: **January** of the year of the first observation must correspond to $0.0$
3. Split the time series into train and test
4. Normalize the target value by fitting a transformation on the train
5. Plot the resulting target against the time index

```python
### BEGIN Solution
dF = pd.read_csv("./monthly_in_situ_co2_mlo.csv",sep=",")
dF.head()
#just test it
```

      Yr
      Mn
      Excel
      Date
      CO2
      CO2Adj
      fit
      fitAdj
      CO2Filled
      CO2FilledAdj

      0
      1958
      1
      21200
      1958.0411
      -99.99
      -99.99
      -99.99
      -99.99
      -99.99
      -99.99

      1
      1958
      2
      21231
      1958.1260
      -99.99
      -99.99
      -99.99
      -99.99
      -99.99
      -99.99

      2
      1958
      3
      21259
      1958.2027
      315.70
      314.43
      316.18
      314.90
      315.70
      314.43

      3
      1958
      4
      21290
      1958.2877
      317.46
      315.16
      317.29
      314.98
      317.46
      315.16

      4
      1958
      5
      21320
      1958.3699
      317.51
      314.73
      317.83
      315.06
      317.51
      314.73

```python
dF=dF.replace([-99.99],np.nan)
print(dF.head())
dF=dF.dropna(axis=0, how='any')
dF.head()

```

         Yr  Mn  Excel       Date     CO2  CO2Adj     fit  fitAdj  CO2Filled  \
    0  1958   1  21200  1958.0411     NaN     NaN     NaN     NaN        NaN
    1  1958   2  21231  1958.1260     NaN     NaN     NaN     NaN        NaN
    2  1958   3  21259  1958.2027  315.70  314.43  316.18  314.90     315.70
    3  1958   4  21290  1958.2877  317.46  315.16  317.29  314.98     317.46
    4  1958   5  21320  1958.3699  317.51  314.73  317.83  315.06     317.51

       CO2FilledAdj
    0           NaN
    1           NaN
    2        314.43
    3        315.16
    4        314.73

      Yr
      Mn
      Excel
      Date
      CO2
      CO2Adj
      fit
      fitAdj
      CO2Filled
      CO2FilledAdj

      2
      1958
      3
      21259
      1958.2027
      315.70
      314.43
      316.18
      314.90
      315.70
      314.43

      3
      1958
      4
      21290
      1958.2877
      317.46
      315.16
      317.29
      314.98
      317.46
      315.16

      4
      1958
      5
      21320
      1958.3699
      317.51
      314.73
      317.83
      315.06
      317.51
      314.73

      6
      1958
      7
      21381
      1958.5370
      315.86
      315.18
      315.87
      315.21
      315.86
      315.18

      7
      1958
      8
      21412
      1958.6219
      314.93
      316.17
      314.01
      315.29
      314.93
      316.17

```python
dF["Yr"]=dF["Yr"]-1958
ts= dF["Yr"]*12+dF["Mn"]-1 #-1 for indexing January with 0
X = np.array(dF["CO2"])
ts=np.array(ts)
### END Solution
```

```python
train_index = np.array(np.arange(np.round(X.shape[0]*0.7)),dtype="int")
test_index = np.array(np.arange(np.round(X.shape[0]*0.7),X.shape[0]), dtype="int")
```

### Part 4.2 (7 pt.)

Use [**GPy**](https://pypi.python.org/pypi/GPy) library for training and prediction.
Make sure you've read the documentation!. Consult with the seminar if necessary.

Your general task is:
* to fit a GP and run the predict on the test:
    * useful kernels to combine: `GPy.kern.RBF`, `GPy.kern.Poly`, `GPy.kern.StdPeriodic`, `GPy.kern.White`
* to plot residuals. Inspect them on normality by QQ-plot, try to make them as "normal" as possible
* to provide a point-estimate of $R^2$ score on the test part

```python
### BEGIN Solution
import GPy

k_trend = GPy.kern.Poly(1, order=2) #+ GPy.kern.RBF(1)
k_periodicity = GPy.kern.StdPeriodic(1) #* GPy.kern.RBF(1)
k_noise = GPy.kern.White(1) #* GPy.kern.Linear(1)
kernel = k_trend+k_periodicity#+k_noise

model = GPy.models.GPRegression(ts[train_index].reshape(-1, 1), X[train_index].reshape(-1, 1), kernel)
#model.sum.std_periodic.variance.fix(1e-3)
#model.sum.std_periodic.period.fix(12)
model.Gaussian_noise.variance.fix(0.601)
model.optimize()

print(model)
model.plot(figsize=(16, 5))

```

After some tuning of parameteres I chose to fix noise variance. As result we get normal residuals.

```python
#QQ
import pylab
import scipy.stats as stats

resids = X-model._raw_predict(ts.reshape(-1,1))[0][:,0]
stats.probplot(resids, dist="norm", plot=pylab)
pylab.show()
```

```python
#and here is R^2 estimated on test_data

print(1-np.sum(np.power(resids[test_index],2))/np.sum(np.power(X[test_index]-np.mean(X[test_index]),2)))

## END SOLUTION
```

    0.9953913496

```python

```


(sec:hyper)=
# The Hyperbolic setting

Invariant sets (see Definition {ref}`here<invariant_set>`) play a fundamental role in dynamical systems theory. In particular, they cannot be "crossed" by trajectories and therefore they play a role in determining the possibilities for how trajectories may navigate phase space, i.e. phase space transport. Invariant sets can be characterized by their stability properties. Further, these can be divided into stability characteristics normal to the invariant set and tangential to the invariant set. The normal stability properties indicate how trajectories near, but not contained in, the invariant set behave.

Normally hyperbolic invariant sets play a particularly important role in phase space transport, particularly when the invariant set has the additional property of being an invariant manifold. For our purposes the reader can think of an invariant manifold as being a generalization of a curve or surface for which  the equation defining such a mathematical object is differentiable {cite}`guillemin2010differential`.  _Normally hyperbolic invariant manifolds_ (NHIMs) have invariant stable and unstable manifolds that form the skeleton or framework,  on which phase space transport  occurs. But before we discuss normally hyperbolic invariant manifolds in general, we discuss the simplest type of non-trivial type of normally hyperbolic invariant manifold—a hyperbolic periodic orbit.

Unstable periodic orbits play an important role in our understanding of the dynamics of Hamiltonian systems. Periodic orbits have constant energy, as well as their stable and unstable manifolds, and are contained in the energy surface that is of one less dimension than the dimension of the phase space, i.e. the energy surface has codimension one. In general, the codimension of a surface contained in phase space is the dimension of phase space minus the dimension of the surface. If the  codimension is one then it may divide the ambient space, and therefore  act as a barrier to transport in the space.

In two degree-of-freedom Hamiltonian systems a one dimensional unstable periodic orbits has two dimensional stable and unstable manifolds that can _"divide"_ the energy surface and can be used to create invariant regions (_"lobes"_)  that transport throughout the phase space, while respecting the _"barrier nature"_ of invariant manifolds.

For more than two degrees-of-freedom periodic orbits and their stable and unstable manifolds do not have sufficient dimension (or, equivalently, their codimension is greater than one) to form barriers to transport or lobes.

We also note (cf. Section {ref}`3 <sec:LDs>` ) the global phase space structure can be studied by creating a mapping along trajectories from a surface transverse to the flow in the energy surface, i.e. the Poincaré map and the Poincaré section. For two degrees-of-freedom the Poincaré section is two dimensional. For $n$ degrees-of-freedom the Poincaré section is $2n-2$ dimensional. The dynamics in the Poincaré section is not easy to visualize for more than two degrees-of-freedom.

In the table below we collect together some dimensionality and codimensionality properties of periodic orbits and their associated geometrical structures within the energy surface, i.e. $E= {\rm constant}$.

```{list-table} Dimension count of phase space structures. Codimension of certain objects appears in parentheses.
:header-rows: 1
:name: tab:dimensions

* - Degrees of freedom
  - 2
  - 3
  - $n$
* - Phase space
  - 4
  - 6
  - $2n$
* - Energy surface
  - 3
  - 5
  - $2n$
* - Poincaré section
  - 2
  - 4
  - $2n-2$
* - Periodic orbit
  - 1 (2)
  - 1 (4)
  - 1 ($2n-2$)
* - Stable and unstable manifolds of a periodic orbit
  - 2 (1)
  - 2 or 3 (3 or 2)
  - 2 to $n$ ($2n-3$ to $n-1$)
* - Barriers in energy surface
  - 2 (1)
  - 4 (1)
  - $2n-2$ (1)
```

NHIMs are a generalization of unstable periodic orbits to more than two degrees-of-freedom in this sense that they _may_ have stable and unstable manifolds of codimension one in the energy surface and, hence, for the framework on which phase space transport occurs  in Hamiltonian systems with more than two degrees-of-freedom {cite}`wiggins1990geometry`.  Now we want to develop this idea of a NHIM, and its consequences, in more detail. Recent references for this topic are {cite}`Wigginsbook1994,Wiggins2001,Wigginsbook2004,Eldering2013`, but we will begin with an elementary description.

In understanding what is meant by _"normally hyperbolic invariant manifold"_ we start by explaining the last word in the phrase and working towards the first.

* A _manifold_ is the natural generalization of a differentiable curve or surface.
* A set in the constant energy level set is called _invariant_ under the flow if any trajectory in the set is contained in the set at all times (link to definition in previous chapter).
* _Normal hyperbolicity_ means that locally under the linearized dynamics, the growth and decay rates in the normal directions to the manifold dominate the growth and decay rates tangent to it. These growth rates are quantified by eigenvalues of the matrix associated with the linearization for equilibria, Floquet exponents for the monodromy matrix associated with periodic orbits, and generalized Lyapunov exponents for more general invariant manifolds {cite}`Fenichel1971`.

Now we consider a number of simple examples in order to explain in more detail these concepts. The following exposition is based on the works {cite}`Wiggins2001,Wiggins2016`.

In all of our examples in this section we will consider the classical autonomous Hamiltonian obtained as the sum of kinetic plus potential energy $(H = K + V)$.

(sec:1dofsaddle)=
## One Degree-of-Freedom  Hyperbolic Equilibrium Point

The most simple Hamiltonian function with a hyperbolic equilibrium point is

```{math}
---
label:
---
\begin{equation}
H = K_{1}(p_{1}) + V_{1}(q_{1})= \frac{\lambda}{2} (p^2_1 - q^2_1) ,\quad \lambda > 0,
\end{equation}
```

```{figure} figures/Vq1.jpg
---
name: Vq1
---
The potential energy $V(q_1)$. An unstable equilibrium point is located at $q_1=0$.
```

```{figure} figures/q1p1.jpg
---
name: q1p1
---
Plane $q_1$--$p_1$. The equilibrium point is at the origin $\mathcal{M}=(q_1=0, p_1=0)$. The red line is the stable manifold $W^s(M)$, the blue line is the unstable manifold $W^u(M)$.
```

The equations of motion are

```{math}
---
label:
---
\begin{eqnarray}
\dot{q_1} &=& \frac{\partial H}{ \partial p_1} =\lambda p_1  \nonumber \\
\dot{p_1} &=& -\frac{\partial H}{ \partial q_1} = \lambda q_1.
\end{eqnarray}
```
For energies $E_1<0$ the energy level set consists of two disconnected parts, a trajectory cannot reach $q_1=0$ due insufficient energy. For $E_1=0$ the two previously disconnected parts connect at $q_1=p_1=0$ and for energies $E_1>0$, any value of $q_1$ is accessible.

The origin, $\mathcal{M} = (q_1=0,p_1=0)$, is a equilibrium point of the system. The trajectory that starts at $\mathcal{M}$ remains always at the same point.

**Showing the invariance of candidate sets.**

Let us consider the line $q_1 = - p_1$. The Hamiltonian vector field evaluated on the line is tangent to the line.

```{math}
---
label: eq:ws
---
\begin{eqnarray}
\dot{q_1} &=& -\lambda q_1  \nonumber \\
\dot{p_1} &=& -\lambda p_1.
\label{eq:ws}
\end{eqnarray}
```
A trajectory that starts on the line $q_1 = - p_1$ remains on the line. This line is an invariant set under the flow.

The solution of the system of differential Eq. {eq}`eq:ws` decay exponentially in time.

```{math}
---
label:
---
\begin{eqnarray}
q_1(t) &=& q_1(0) e^{-\lambda t}  \nonumber\\
p_1(t) &=& p_1(0) e^{-\lambda t}.
\end{eqnarray}
```
Any trajectory that starts on $q_1 = - p_1$ converges to the equilibrium point $\mathcal{M}$ in an infinite time. These properties motivate the definition of the stable manifold of the set $\mathcal{M}$. The stable manifold $W^s(\mathcal{M})$ of the set $\mathcal{M}$ is the set of points in the energy level set such that their corresponding trajectories converge to the set $\mathcal{M}$ when the time goes to infinity.

```{math}
---
label:
---
\begin{equation}
\displaystyle{ W^s(\mathcal{M})=  \lbrace (q_1,p_1) | (q_1(t), p_1(t) ) \rightarrow \mathcal{M} , t \rightarrow + \infty \rbrace} = \lbrace  (q_1, p_1 )  |  q_1 = - p_1 \rbrace
\end{equation}
```
Now, let us consider the trajectories that start on the line $q_1 = p_1$. Like in the previous case, the Hamiltonian vector field is tangent to the line,

```{math}
---
label:
---
\begin{eqnarray}
\dot{q_1} &=& \lambda q_1  \nonumber\\
\dot{p_1} &=& \lambda p_1.
\end{eqnarray}
```
However, the vector field points away from $\mathcal{M}$. The trajectories that start on the line $q_1 = p_1$ diverge from the equilibrium point $\mathcal{M}$.

```{math}
---
label:
---
\begin{eqnarray}
q_1(t) &=& q_1(0) e^{\lambda t}  \nonumber\\
p_1(t) &=& p_1(0) e^{\lambda t}.
\end{eqnarray}
```
Analogously, these properties motivate the definition of the unstable manifold of the set $\mathcal{M}$. The unstable manifold $W^u(\mathcal{M})$ of $\mathcal{M}$ is the set of points in the phase space such that their corresponding trajectories converge to the set $\mathcal{M}$ when the time goes to minus infinity.

```{math}
---
label:
---
\begin{equation}
\displaystyle{ W^u(\mathcal{M})= \lbrace (q_1,p_1) | (q_1(t), p_1(t) ) \rightarrow \mathcal{M} , t \rightarrow - \infty  \rbrace =  \lbrace  (q_1, p_1 )  |  q_1 = p_1  \rbrace  }.
\end{equation}
```
The existence of two special directions where the trajectories converge to the $\mathcal{M}$ in some direction and diverge from $\mathcal{M}$ in other one is the essence of the concept of hyperbolicity.

(sec:2dofupo)=
## Two Degrees-of-Freedom and the Hyperbolic Periodic Orbit

In this case, the Hamiltonian function for the 2-degree of freedom system is the sum of the Hamiltonian function of the 1-degree of freedom of the previous example plus the Hamiltonian of a harmonic oscillator (Section {ref}`sec:KAM` ). The total energy of the system is split between both DoF to yield:

```{math}
---
label:
---
\begin{eqnarray}
H &=& K(p_{1},p_{2}) + V(q_{1},q_{2}) \\
  &=& K_1(p_1) + V_1(q) +  K_2(p_2) + V_2(q_2) \\
  &=& \frac{\lambda}{2} (p^2_1 - q^2_1) + \frac{\omega}{2} (p^2_2 + q^2_2),\quad \lambda, \omega >0
\end{eqnarray}
```

```{figure} figures/Vm.jpg
---
name: Vm
---
Potential energy associated to the 2-degree of freedom system $V(q_1,q_2) = V_1(q_1)+V_2(q_2)$. The potential has a saddle point at the point $(q_1, q_2)=(0,0)$
```

```{figure} figures/Vcm.jpg
---
name: Vcm
---
Contour plot of $V(q_1,q_2)$. The black curves are the equipotential curves.
```

Hamilton's equations are

```{math}
---
label:
---
\begin{eqnarray}
\dot{q_1} &=& \frac{\partial H}{ \partial p_1} = \lambda p_1  \\
\dot{p_1} &=& -\frac{\partial H}{ \partial q_1} = \lambda q_1  \\
\dot{q_2} &=& \frac{\partial H}{ \partial p_2} = \omega p_2 \\
\dot{p_2} &=& -\frac{\partial H}{ \partial q_2} = -\omega q_2
\end{eqnarray}
```

```{figure} figures/Vq1.jpg
---
name: Vq1
---
(a) Potential $V_1(q_1)$
```

```{figure} figures/Vq2.jpg
---
name: Vq2
---
(b) Potential $V_2(q_2)$
```

```{figure} figures/q1p1.jpg
---
name: q1p1
---
(c) Plane $q_1$--$p_1$
```

```{figure} figures/q2p2.jpg
---
name: q2p2
---
(d) Plane $q_2$--$p_2$
```

Again, the point in the phase space $(q_1,q_2,p_1,p_2)=(0,0,0,0)$ is a equilibrium point of the system at the energy $E=0$.  The equilibrium point has stable and unstable manifolds, similar to the 1-degree of freedom example above.

For any $E_2>0$ and $E_1=0$, i.e $E=E_2$, a periodic orbit $\gamma$ exists in the $(q_2,p_2)$ plane and this periodic orbit is unstable.

```{math}
---
label: eq:NHIM
---
\begin{equation}
\displaystyle{\gamma = \lbrace  (0,q_2, 0,p_2 )  |  \frac{\omega}{2} (p^2_2 + q^2_2) = E  > 0 \rbrace}.
\label{eq:NHIM}
\end{equation}
```
In a similar way to the 1-degree of freedom case, we can find the stable and unstable manifolds of the periodic orbit $\gamma$. To find $W^u(\gamma)$ and $W^s(\gamma)$ in this two degree of freedom system, it is necessary to consider the composition of the oscillation in $q_2$ direction and the motion along the stable and unstable manifolds associated to the 1-degree of freedom case studied before. The stable and unstable manifolds of $\gamma$ are

```{math}
---
label:
---
\begin{equation}
\displaystyle{ W^u(\gamma)= \lbrace  (q_1,q_2, p_1,p_2 )  |  \frac{\omega}{2} (p^2_2 + q^2_2) = E > 0,  q_1 = p_1 \rbrace }.
\end{equation}
```

```{math}
---
label:
---
\begin{equation}
\displaystyle{ W^s(\gamma)= \lbrace (q_1,q_2, p_1,p_2 )  |  \frac{\omega}{2} (p^2_2 + q^2_2) = E > 0, q_1 = - p_1  \rbrace }.
\end{equation}
```

(sec:3dofnhim)=
## Three and More Degrees-of-Freedom and NHIMs

Let us consider the most simple example with $n$ degrees of freedom. The basic idea is the same as in the previous example with 2 degrees of freedom, where the dynamics of each degree of freedom is independent from the other degrees of freedom. In this case, the most simple Hamiltonian function of a $n$ degree of freedom is

```{math}
---
label:
---
\begin{equation}
H = \frac{\lambda}{2} (p^2_1 - q^2_1)+ \sum^n_{i=2} \frac{\omega_i}{2} (p^2_i + q^2_i),
\end{equation}
```
where $\lambda, \omega_i >0$.

The equation of motion are :

```{math}
---
label:
---
\begin{eqnarray}
\dot{q_1} &=& \frac{\partial H}{ \partial p_1} = \lambda p_1\\
\dot{p_1} &=& -\frac{\partial H}{ \partial q_1} = \lambda q_1\\
\dot{q_i} &=& \frac{\partial H}{ \partial p_i} = \omega_i p_i\\
\dot{p_i} &=& -\frac{\partial H}{ \partial q_i} = -\omega_i q_i, \quad i = 2,...,n.
\end{eqnarray}
```

```{figure} figures/Vq1.jpg
---
name:
---
(a) Potential $V_1(q_1)$
```

```{figure} figures/Vq2.jpg
---
name:
---
(b) Potential $V_2(q_2)$
```

```{figure} figures/Vqn.jpg
---
name:
---
(c) Potential $V_n(q_n)$
```

```{figure} figures/q1p1.jpg
---
name:
---
(d) Plane $q_1$--$p_1$
```

```{figure} figures/q2p2.jpg
---
name:
---
(e) Plane $q_2$--$p_2$
```

```{figure} figures/qnpn.jpg
---
name:
---
(f) Plane $q_n$--$p_n$.
```

The NHIM and its stable and unstable manifolds can be found analogously to the  2 degree of freedom system case. If $q_1=p_1=0$ then $\dot{q_1}=0$ and $\dot{p_1}=0$ and the system consists of $n-1$ harmonic oscillators bounded by the fixed total energy. For $E=0$, $q_1,p_1=0$ is an unstable equilibrium point and for $E>0$, an invariant set that can be shown to be a sphere. The dimension of the sphere depends on the number of degrees of freedom with non-zero energy. This sphere is a manifold, it is invariant and normally hyperbolic, hence a NHIM. In general a NHIM does not have to be a sphere. If all degrees of freedom have non-zero energy, except for $q_1,p_1=0$, the NHIM is $2n-3$ dimensional and its stable and unstable manifolds are dynamical barriers on the energy level set according to {numref}`tab:dimensions` .

```{math}
---
label:
---
\begin{equation}
\displaystyle{ \Gamma = \lbrace  (0,q_2,...,q_n, 0,p_2,...,p_n ) | \sum_{i=2}^n \frac{\omega_i}{2} (p^2_i + q^2_i) = E  > 0,\rbrace }
\end{equation}
```
Note that the NHIM is a generalization of the hyperbolic equilibrium point and hyperbolic periodic orbit from previous examples.

The stable and unstable manifolds, $W^{u}(\Gamma)$ and $W^{s}(\Gamma)$, are multidimensional cylinders that intersect in the multidimensional sphere $\Gamma$.

```{math}
---
label:
---
\begin{equation}
\displaystyle{ W^u(\Gamma)= \lbrace  (q_1,q_2,...,q_n, p_1,p_2,...,p_n )  |    \sum^n_{i=2} \frac{\omega_i}{2} (p^2_i + q^2_i) = E > 0,  q_1 = p_1  \rbrace }.
\end{equation}
```

```{math}
---
label:
---
\begin{equation}
\displaystyle{ W^s(\Gamma)= \lbrace  (q_1,q_2,...,q_n, p_1,p_2,...,p_n )  |    \sum^n_{i=2} \frac{\omega_i}{2} (p^2_i + q^2_i)    = E > 0,  q_1 = - p_1  \rbrace }.
\end{equation}
```
The general study  of NHIMs, including their characterization,  the nature of their  invariant manifolds, and their properties under perturbations for more general systems are discussed in the works by Fenichel {cite}`Fenichel1971,Fenichel1974,Fenichel1977` and more recent works {cite}`Wigginsbook1994,Eldering2013`.

It is important to note that the  basic theory of NHIMs does not tell us how to find NHIMs in specific systems. It only characterizes them and describes their properties based on that characterization. Next we describe a theorem from Hamiltonian systems that provides sufficient conditions for the existence of NHIMs under certain specific circumstances.

(sec:LyapSub)=
## The Lyapunov Subcenter Theorem

The Lyapunov subcenter theorem {cite}`liapounoff1907probleme` is a fundamental theorem in Hamiltonian mechanics that describes how index one saddle points of Hamilton's equations give rise to periodic orbits, or ``NHIMs''. Basic background on the topic can be found in {cite}`moser1958generalization,kelley1967liapounov,moser1976periodic,weinstein1973normal,duistermaat1972periodic`.

We will explain the ideas in the context of a two degree-of-freedom quadratic Hamiltonian, and then describe how they extend to more general settings.

```{math}
---
label: eq:LST1
---
\begin{equation}
H = \frac{\lambda}{2} (p_1^2 - q_1^2) + \frac{\omega}{2} ( p_2^2 + q_2^2), \quad \lambda, \, \omega >0, \quad (q_1, p_1, q_2, p_2) \in \mathbb{R}^4,
\label{eq:LST1}
\end{equation}
```

with corresponding Hamilton's equations:

```{math}
---
label: eq:LST2
---
\begin{eqnarray}
\dot{q}_1 & = & \frac{\partial H}{\partial p_1} =  \lambda p_1,  \\
\dot{p}_1 & = & - \frac{\partial H}{\partial q_1} =  \lambda q_1,  \\
\dot{q}_2 & = & \frac{\partial H}{\partial p_2} =  \omega  p_2,  \\
\dot{p}_2 & = & - \frac{\partial H}{\partial q_2} =  - \omega q_2,
\label{eq:LST2}
\end{eqnarray}
```

The point $(q_1, p_1, q_2, p_2) = (0, 0, 0, 0)$ is an equilibrium point of Hamilton's equations, and the eigenvalues of the matrix associated with the linearization of Hamilton's equations about this equilibrium point are
$\pm \lambda, \, \pm i \omega$. Hence, this equilibrium point is an index one saddle point. Note that the total energy of this equilibrium point, i.e. the value of the Hamiltonian {eq}`eq:LST1`  evaluated at the equilibrium point, is zero.

```{math}
---
label: eq:LST3
---
\begin{equation}
 \frac{\lambda}{2} (p_1^2 - q_1^2) + \frac{\omega}{2} ( p_2^2 + q_2^2)=E >0,
\label{eq:LST3}
\end{equation}
```
We now want to show that for energies above the index one saddle, i.e. $E>0$, there is a two dimensional invariant manifold of unstable periodic orbits.

First, note that $q_1 = p_1 =0$ is an invariant manifold for all values of $E$.  We can compute the intersection of this invariant manifold by substituting $q_1 = p_1 =0$ into the energy equation to obtain:

```{math}
---
label: eq:LST4
---
\begin{equation}
 \frac{\omega}{2} ( p_2^2 + q_2^2)=E >0,
\label{eq:LST4}
\end{equation}
```
which is a periodic orbit for each $E > 0$. It is instructive to count the dimensions for intersecting sets. The phase space is four dimensional, the energy surface is three dimensional, and $q_1 = p_1 =0$ is two dimensional. A three dimensional surface intersects a two dimensional surface in a four dimensional space in a one dimensional set, which is the periodic orbit {eq}`eq:LST4` . This family of periodic orbits ("family" in terms of the energy $E>0$, lies on the two dimensional invariant manifold $q_1 = p_1 =0$. This is the Lyapunov subcenter manifold. Note that the Lyapunov subcenter manifold is *not* isoenergetic. Each periodic orbit has a different energy, monotonically increasing from zero, which is the energy of the index one saddle point.

The Lyapunov subcenter theorem says that this result holds, for sufficiently small $E > 0$, if higher  order terms are added to {eq}`eq:LST1` . All of these  periodic orbits are unstable since the dynamics normal to the Lyapunov subcenter manifold is exponentially growing and decaying, i.e. the Lyapunov subcenter manifold is a NHIM.

This set-up can be generalized to higher dimensions. The quadratic normal form associated to an index $n_s$ saddle has the general form (assuming  that the $\lambda_i$ are distinct and the  $\omega_i$ are distinct):

```{math}
---
label: eq:LST5
---
\begin{equation}
H =\sum_{i=1}^{n_s} \frac{\lambda_i}{2} (p_i^2 - q_i^2) + \sum_{i=1}^{n_c}\frac{\omega_{i+n_s}}{2} ( p_{i+n_s}^2 + q_{i+n_s}^2),
\label{eq:LST5}
\end{equation}
```
where  $\lambda_i, \, \omega_i >0, \quad (q_1, p_1,  \ldots, q_{n_s}, p_{n_s},  q_{n_s +1}, p_{n_s + 1}, \ldots, q_{n_s +n_c}, p_{n_s + n_c})
\in \mathbb{R}^{2(n_s + n_c)}$.

The structure of the NHIM is treated in detail for ${n_s} =1$ in {cite}`wiggins2016role`.  Results for ${n_s} >1$ are treated in {cite}`collins2011index`.

# References
```{bibliography} bibliography/chapter2.bib
```