# COBRAme Reaction Buidling

```python
import cobrame
from cobrame.util import dogma, building
import cobra
import cobra.test
from collections import defaultdict

#import warnings
#warnings.filterwarnings('ignore')
```

    /home/sbrg-cjlloyd/cobrapy/cobra/io/sbml3.py:24: UserWarning: Install lxml for faster SBML I/O
      warn("Install lxml for faster SBML I/O")
    /home/sbrg-cjlloyd/cobrapy/cobra/io/__init__.py:12: UserWarning: cobra.io.sbml requires libsbml
      warn("cobra.io.sbml requires libsbml")

## Overview

COBRAme is constructed entirely over COBRApy. This means that ME reactions will have all of the same properties, methods, and functions as a COBRApy reaction. However, one key difference is that, in order to facilliate the template nature of many gene expression reactions, reactions are constructed and their components are manipulated via the use of the ProcessData classes. These act as information vessels for holding the information assocatied with a cellular process in simple, standard datatypes such as dictionaries and strings.

-----

This tutorial will go step-by-step through the process of creating the generic enzyme catalyzed reaction or metabolic reaction:
$$ a \rightarrow b $$
that requires the formation of **complex\_ab** in order to proceed

-------

In order for this reaction to carry flux in the model we will additionally need to first add the corresponding:
1. Transcription reactions
2. Translation reactions
3. tRNA charging reactions
4. Complex formation reactions

Once these are added we will add in synthesis of key macromolecular components (ribosome, RNA polymerase, etc.) and show how they are applied to their respective reactions with their coupling constraints implemented.

## Initializing new ME-models

When applying some constraints in the ME-model, metabolite properties are required. For instance, to calculate the total biomass produced by a particular reaction, the amino acid, nucleotide, etc. molecular weights are required.To enable these calculations, all metabolites from iJO1366, along with their metabolite attributes are added to the newly created ME-model.

Further the reactions from *i*JO1366 will be added to the ME-model to demonstrate ME-model solving procedures.

```python
# create empty ME-model
me = cobrame.MEModel('test')
ijo = cobra.test.create_test_model('ecoli')

# Add all metabolites and reactions from iJO1366 to the new ME-model
for met in ijo.metabolites:
    me.add_metabolites(met)
for rxn in ijo.reactions:
    me.add_reaction(rxn)
```

The ME-model contains a "global_info" attribute which stores information used to calculate coupling constraints, along with other functions. The specifics of each of these constraints will be discussed when they are implemented.

*Note*: k_deg will initially be set to 0. We will apply RNA degradation later in the tutorial.

```python
# "Translational capacity" of organism
me.global_info['kt'] =  4.5 #(in h-1)scott 2010, RNA-to-protein curve fit
me.global_info['r0'] =  0.087 #scott 2010, RNA-to-protein curve fit
me.global_info['k_deg'] =  0. #1.0/5. * 60.0  # 1/5 1/min 60 min/h # h-1

# Molecular mass of RNA component of ribosome
me.global_info['m_rr'] = 1700. # in kDa

# Average molecular mass of an amino acid
me.global_info['m_aa'] = 109. / 1000. #109. / 1000. # in kDa

# Proportion of RNA that is rRNA
me.global_info['f_rRNA'] = .86
me.global_info['m_nt'] = 324. / 1000. # in kDa
me.global_info['f_mRNA'] = .02

# tRNA associated global information
me.global_info['m_tRNA'] = 25000. / 1000. # in kDA
me.global_info['f_tRNA'] = .12

sequence = ("ATG" + "TTT" * 12 + "TAT" * 12 +
            "ACG" * 12 + "GAT" * 12 + "AGT" * 12 + "TGA")
```

## Add Transcription

The general workflow for adding any reaction to a ME-model using COBRAme occurs in three steps:
1. Create the ProcessData(s) associated with the reaction and populate them with the necessary information
2. Create the MEReaction and link the appropriate ProcessData
3. Execute the MEReaction's update method

### Without utility functions

```python
data = cobrame.TranscriptionData('TU_a',me,RNA_products={'RNA_a'})
data.nucleotide_sequence = sequence
rxn = cobrame.TranscriptionReaction('transcription_TU_a')
rxn.transcription_data = data
me.add_reaction(rxn)
rxn.update()
print(rxn.reaction)
```

    86 atp_c + 38 ctp_c + 12 gtp_c + 186 h2o_c + 50 utp_c --> RNA_a + 86 amp_c + 38 cmp_c + 12 gmp_c + 186 h_c + 187 ppi_c + 50 ump_c

    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:513 UserWarning: RNA Polymerase () not found
    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:524 UserWarning: Transcript (RNA_a) added to model

As stated in the above warning, we have not added the RNA_a metabolite to the model. This means that the model does not know the RNA_type (mRNA, tRNA or rRNA) or the specific sequence of RNA_a. Therefore, the *biomass* constraint for this RNA cannot be included in the reaction since the sequence is required to calculate the molecular weight of RNA_a. This sequence cannot be determined from the transcription unit (TU) sequence because a single TU often contains several different RNAs. Adding this information to the RNA_a metabolite and updating the transcription reaction adds the *biomass* constraint to the reaction.

**Note**: the RNA_polymerase complex is not included in the reaction. This will be added later

```python
me.metabolites.RNA_a.RNA_type = 'mRNA'
me.metabolites.RNA_a.nucleotide_sequence = sequence
rxn.update()
print(rxn.reaction)
```

    86 atp_c + 38 ctp_c + 12 gtp_c + 186 h2o_c + 50 utp_c --> RNA_a + 186 h_c + 59.172286 mRNA_biomass + 186 ppi_c

    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:513 UserWarning: RNA Polymerase () not found

This reaction also produces a small amount of the a $biomass$ metabolite (constraint). This term has a coefficient corresponding to the molecular weight (in $kDA$) of the protein being translated. This constraint will be implemented into a $v_{biomasss\_dilution}$ reaction with the form:
$$\mu \leq v_{biomass\_dilution} \leq \mu $$

A mathematical description of the *biomass* constraint can be found in **Biomass Dilution Constraints**.

### Using utility functions
Using the utility functions to create the TranscribedGene metabolite has the advantage of forcing the assignment of sequence, strand and RNA_type.

```python
building.create_transcribed_gene(me, 'b', 1, 2, sequence, '+', 'mRNA')
building.add_transcription_reaction(me, 'TU_b', {'b'}, sequence)
print(me.reactions.transcription_TU_b.reaction)
```

    86 atp_c + 38 ctp_c + 12 gtp_c + 186 h2o_c + 50 utp_c --> RNA_b + 186 h_c + 59.172286 mRNA_biomass + 186 ppi_c

    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:513 UserWarning: RNA Polymerase () not found

**Note**: This is not a complete picture of transcriptiont because the RNA polymerase is missing.
### Associate RNA Polymerase

```python
RNAP = cobrame.RNAP('RNA_polymerase')
me.add_metabolites(RNAP)
for data in me.transcription_data:
    data.RNA_polymerase = RNAP.id
me.reactions.transcription_TU_a.update()
me.reactions.transcription_TU_b.update()

print(me.reactions.transcription_TU_a.reaction)
```

    0.000759722875816993*mu + 0.000297431505882353 RNA_polymerase + 86 atp_c + 38 ctp_c + 12 gtp_c + 186 h2o_c + 50 utp_c --> RNA_a + 186 h_c + 59.172286 mRNA_biomass + 186 ppi_c

The coefficient for RNA_polymerase is the first instance where a coupling constraint is imposed. In this case the constraint couples the formation of a RNA_polymerase metabolite to its transcription flux. This constraint is formulated as in [O'brien et. al. 2013](https://www.ncbi.nlm.nih.gov/pubmed/24084808), with assumption that $k_{rnap} = 3 \cdot k_{ribosome}$ based on data from [Proshkin et al. 2010](https://www.ncbi.nlm.nih.gov/pubmed/20413502):

$$
\begin{align}
v_{dilution,RNAP, j} =  \frac{l_{TU,j}}{3 c_{ribo}\kappa_{\tau}} v_{transcription,j} (\mu+r_0\kappa_{\tau}), & \forall j \in TU
\end{align}
$$

where:

 - $\kappa_{\tau}$ and $r_0$ are phenomenological parameters from [Scott et. al. 2010](https://www.ncbi.nlm.nih.gov/pubmed/21097934) that describe the linear relationship between the observed RNA/protein ratio of *E. coli* and its growth rate ($\mu$)

 - $c_{ribo} = \frac{m_{rr}}{f_{rRNA}\cdot m_{aa}}$ where: $m_{nt}$ is the mass of rRNA per ribosome. $f_{rRNA}$ is the fraction of total RNA that is rRNA $m_{aa}$ is the molecular weight of an average amino acid

 - $v_{transcription, j}$ is the rate of transcription for $TU_j$

 - $l_{TU, j}$ is number of nucleotides in $TU_j$

----
## Add Translation
### Without utility functions
#### Add the TranslationData for protein_a
In order to add a TranslationData object to a ME model the user must additionally specifify the mRNA id and protein id of the translation reaction that will be added. This information as well as a nucleotide sequence is the only information required to add a translation reaction.

```python
data = cobrame.TranslationData('a', me, 'RNA_a', 'protein_a')
data.nucleotide_sequence = sequence
```

#### Add the TranslationReaction for protein_a
By associating the TranslationReaction with its corresponding TranslationData object and running the update function, COBRAme will create a reaction reaction for the nucleotide sequence given based on the organisms codon table and prespecified translation machinery.

```python
rxn = cobrame.TranslationReaction('translation_a')
rxn.translation_data = data
me.add_reaction(rxn)
rxn.update()
print(rxn.reaction)
```

    0.00149519890260631*mu + 0.00058537037037037 RNA_a + 12 asp__L_c + met__L_c + 12 phe__L_c + 12 ser__L_c + 12 thr__L_c + 12 tyr__L_c --> protein_a + 7.500606 protein_biomass

    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:666 UserWarning: ribosome not found
    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:706 UserWarning: RNA_degradosome not found

In this case the constraint couples the formation of a mRNA metabolite to its translation flux. This constraint is formulated as in [O'brien et. al. 2013](https://www.ncbi.nlm.nih.gov/pubmed/24084808):

$$
\begin{align}
& v_{dilution,j} = \frac{3}{\kappa_{\tau} c_{mRNA}} \cdot (\mu + \kappa_{\tau} r_0) v_{translation,j}  , & \forall j \in mRNA \\
\end{align}
$$

where:

 - $\kappa_{\tau}$ and $r_0$ are phenomenological parameters from [Scott et. al. 2010](https://www.ncbi.nlm.nih.gov/pubmed/21097934) that describe the linear relationship between the observed RNA/protein ratio of *E. coli* and its growth rate ($\mu$)

 - $c_{mRNA} = \frac{m_{nt}}{f_{mRNA}\cdot m_{aa}}$ where: $m_{nt}$ is the molecular weight of an average mRNA nucleotide. $f_{mRNA}$ is the fraction of total RNA that is mRNA $m_{aa}$ is the molecular weight of an average amino acid

 - $v_{translation, j}$ is the rate of translation for $mRNA_j$

### Using utility functions
`add_translation_reaction` assumes that the RNA and protein have the same locus_id. It creates the appropriate `TranslationData` and `TranslationReaction` instance, links the two together and updates the `TranslationReaction`.

```python
building.add_translation_reaction(me, 'b', dna_sequence=sequence, update=True)
print(me.reactions.translation_b.reaction)
```

    0.00149519890260631*mu + 0.00058537037037037 RNA_b + 12 asp__L_c + met__L_c + 12 phe__L_c + 12 ser__L_c + 12 thr__L_c + 12 tyr__L_c --> protein_b + 7.500606 protein_biomass

    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:666 UserWarning: ribosome not found
    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:706 UserWarning: RNA_degradosome not found

### Incorporate Ribosome

```python
ribosome = cobrame.Ribosome('ribosome')
me.add_metabolites([ribosome])
me.reactions.translation_a.update()
print(me.reactions.translation_a.reaction)
```

    0.00149519890260631*mu + 0.00058537037037037 RNA_a + 12 asp__L_c + met__L_c + 12 phe__L_c + 0.000747469281045752*mu + 0.000292634223529412 ribosome + 12 ser__L_c + 12 thr__L_c + 12 tyr__L_c --> protein_a + 7.500606 protein_biomass

    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:706 UserWarning: RNA_degradosome not found

This imposes a new coupling constraint for the ribosome. In this case the constraint couples the formation of a ribosome to its translation flux. This constraint is formulated as in [O'brien et. al. 2013](https://www.ncbi.nlm.nih.gov/pubmed/24084808):

$$
\begin{align}
v_{dilution,ribo, j} = \frac{l_{p,j}}{c_{ribo}\kappa_{\tau}} v_{translation,j} (\mu+r_0\kappa_{\tau}) , & \forall j \in mRNA
\end{align}
$$

where:

 - $\kappa_{\tau}$ and $r_0$ are phenomenological parameters from [Scott et. al. 2010](https://www.ncbi.nlm.nih.gov/pubmed/21097934) that describe the linear relationship between the observed RNA/protein ratio of *E. coli* and its growth rate ($\mu$)

 - $c_{ribo} = \frac{m_{rr}}{f_{rRNA}\cdot m_{aa}}$ where: $m_{nt}$ is the mass of rRNA per ribosome. $f_{rRNA}$ is the fraction of total RNA that is rRNA $m_{aa}$ is the molecular weight of an average amino acid

 - $v_{translation, j}$ is the rate of translation for $mRNA_j$

 - $l_{p, j}$ is number of amino acids in peptide translated from $mRNA_j$

-------
**Note**: The above reactions do not provide a complete picture of translation considering that it's missing charged tRNAs to facillitate tRNA addition.

Below, we'll correct this by adding in an tRNA charging reaction.

## Add tRNA charging reaction

### Without utility functions (There are no utility functions for adding individual tRNA charging reactions)
#### Add tRNAData

```python
data = cobrame.tRNAData('tRNA_d_GUA', me, 'val__L_c', 'RNA_d', 'GUA')
rxn = cobrame.tRNAChargingReaction('charging_tRNA_d_GUA')
me.add_reaction(rxn)
rxn.tRNA_data = data
```

#### Add tRNAChargingReaction

```python
#Setting verbose=False suppresses print statements indicating that new metabolites were created
rxn.update(verbose=False)
print(rxn.reaction)
```

    0.000116266666666667*mu + 4.55184e-5 RNA_d + 0.000116266666666667*mu + 4.55184e-5 val__L_c --> generic_tRNA_GUA_val__L_c

This reaction creates one ```generic_charged_tRNA``` equivalement that can then be used in a translation reaction

The coefficient for ```RNA_d``` and ```lys__L_c``` are defined by:

$$
\begin{align}
v_{dilution,j} \geq \frac{1}{\kappa_{\tau} c_{tRNA,j}} (\mu + \kappa_{\tau} r_0)  v_{charging,j} , & \forall j \in tRNA
\end{align}
$$

where:

 - $\kappa_{\tau}$ and $r_0$ are phenomenological parameters from [Scott et. al. 2010](https://www.ncbi.nlm.nih.gov/pubmed/21097934) that describe the linear relationship between the observed RNA/protein ratio of *E. coli* and its growth rate ($\mu$)

 - $c_{tRNA, j} = \frac{m_{tRNA}}{f_{tRNA}\cdot m_{aa}}$ where: $m_{tRNA}$ is molecular weight of an average tRNA. $f_{tRNA}$ is the fraction of total RNA that is tRNA $m_{aa}$ is the molecular weight of an average amino acid

 - $v_{charging, j}$ is the rate of charging for $tRNA_j$

**Note**: This tRNA charging reaction is still missing a tRNA synthetase which catalyzes the amino acid addition to the uncharged tRNA.

### Incorporate tRNA Synthetases

```python
synthetase = cobrame.Complex('synthetase')
me.add_metabolites(synthetase)
data.synthetase = synthetase.id
rxn.update()
print(rxn.reaction)
```

    0.000116266666666667*mu + 4.55184e-5 RNA_d + 4.27350427350427e-6*mu*(0.000116266666666667*mu + 1.0000455184) synthetase + 0.000116266666666667*mu + 4.55184e-5 val__L_c --> generic_tRNA_GUA_val__L_c

The synthetase coupling was reformulated from [O'brien et. al. 2013](https://www.ncbi.nlm.nih.gov/pubmed/24084808) enable more modularity in the ME-model. A more complete mathematical description of the tRNA synthetase coupling constraints can be found in the [tRNA.ipynb](tRNA.ipynb)

---

### Add tRNAs to translation

Here we take advantage of an additional subclass of `ProcessData`, called a `SubreactionData` object. This class is used to lump together processeses that occur as a result of many individual reactions, including translation elongation, ribosome formation, tRNA modification, etc. Since each of these steps often involve an enzyme that requires its own coupling constraint, this process allows these processes to be lumped into one reaction while still enabling each subprocess to be modified.

`TranslationData` objects have an `elongation_subreactions` method that returns any subreactions that have been used added to the model and are part of translation elongation. Since no tRNA-mediated amino acid addition subreactions have been added to the model, the below call returns nothing.

```python
print(me.translation_data.a.elongation_subreactions)
```

    {}

    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:317 UserWarning: RNA_a starts with 'AUG' which is not a start codon
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction met_addition_at_AUG not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction phe_addition_at_UUU not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction tyr_addition_at_UAU not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction thr_addition_at_ACG not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction asp_addition_at_GAU not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction ser_addition_at_AGU not in model

`UserWarnings` are returned to indicate that tRNA subreactions have not been added for each codon.

Below, we add the SubreactionData (excluding enzymes) for the addition of an amino acid using information from the *E. coli* codon table.

```python
data = cobrame.SubreactionData('asp_addition_at_GAU', me)
data.stoichiometry = {'generic_tRNA_GAU_asp__L_c': -1,
                      'gtp_c': -1, 'gdp_c': 1, 'h_c': 1,
                      'pi_c': 1}
```

Now calling `elongation_subreactions` returns the number of tRNA subreactions that should be added to the `TranslationData`

```python
translation_subreactions = me.translation_data.a.elongation_subreactions
print(translation_subreactions)
```

    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:317 UserWarning: RNA_a starts with 'AUG' which is not a start codon

    {'asp_addition_at_GAU': 12}

    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction met_addition_at_AUG not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction phe_addition_at_UUU not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction tyr_addition_at_UAU not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction thr_addition_at_ACG not in model
    /home/sbrg-cjlloyd/cobrame/cobrame/core/ProcessData.py:355 UserWarning: elongation subreaction ser_addition_at_AGU not in model

Updating `TranslationData.subreactions` with the tRNA subreactions incorporates this information into the `TranslationReaction`

```python
print("Before adding tRNA subreaction")
print("------------------------------")
print(me.reactions.translation_a.reaction)
print("")
# Update tranlation_data with subreactions and update
for subreaction, value in translation_subreactions.items():
    me.translation_data.a.subreactions[subreaction] = value
me.reactions.translation_a.update(verbose=False)
print("After adding tRNA subreaction")
print("-----------------------------")
print(me.reactions.translation_a.reaction)
```

    Before adding tRNA subreaction
    ------------------------------
    0.00149519890260631*mu + 0.00058537037037037 RNA_a + 12 asp__L_c + met__L_c + 12 phe__L_c + 0.000747469281045752*mu + 0.000292634223529412 ribosome + 12 ser__L_c + 12 thr__L_c + 12 tyr__L_c --> protein_a + 7.500606 protein_biomass

    After adding tRNA subreaction
    -----------------------------
    0.00149519890260631*mu + 0.00058537037037037 RNA_a + 12 asp__L_c + 12 generic_tRNA_GAU_asp__L_c + 12 gtp_c + met__L_c + 12 phe__L_c + 0.000747469281045752*mu + 0.000292634223529412 ribosome + 12 ser__L_c + 12 thr__L_c + 12 tyr__L_c --> 12 gdp_c + 12 h_c + 12 pi_c + protein_a + 7.500606 protein_biomass

    /home/sbrg-cjlloyd/cobrame/cobrame/core/MEReactions.py:706 UserWarning: RNA_degradosome not found

## Create complexes
### Without utility functions
#### Create ComplexData and ComplexFormation
For COBRAme models, the reaction gene-protein-reaction rule (GPR) is replaced with a metabolite representing the synthesis of the enzyme(s) catalyzing a reaction. This metabolite is formed explicitly in a ME model by seperate reaction to transcribe the gene(s) and translate the protein(s) the compose the complex. These reactions will be added later.

ComplexData objects contain:
 - Subunit stoichiometry
 - Modification dictionary (discussed later)
 - Translocation dictionary (discussed later)

```python
data = cobrame.ComplexData('complex_ab', me)
data.stoichiometry = {'protein_a': 1, 'protein_b': 1}

rxn = cobrame.ComplexFormation('formation_complex_ab')
me.add_reaction(rxn)
rxn.complex_data_id = data.id
rxn._complex_id = data.id
rxn.update(verbose=False)
print(me.reactions.formation_complex_ab.reaction)
```

    protein_a + protein_b --> complex_ab

### Using utility functions
Alternatively, ComplexData has a ```create_complex_formation()``` function to create the sythesis reaction following the naming conventions. It contains an ```update()``` function which incorporates changes in the ComplexData

```python
data = cobrame.ComplexData('complex_ba', me)
data.stoichiometry = {'protein_a': 1, 'protein_b': 1}
data.create_complex_formation()
print(me.reactions.formation_complex_ba.reaction)
```

    protein_a + protein_b --> complex_ba

### Apply modification to complex formation reaction
Many enzyme complexes in an ME-model require cofactors or prosthetic groups in order to properly function. Information about such processes are stored as ModificationData.

For instance, we can add the modification of an iron-sulfur cluster, a common prosthetic group, by doing the following:

```python
# Define the stoichiometry of the modification
mod_data = cobrame.ModificationData('mod_2fe2s_c', me)
mod_data.stoichiometry = {'2fe2s_c': -1}
# this process can also be catalyzed by a chaperone
mod_data.enzyme = 'complex_ba'
mod_data.keff = 65.  # default value
```

Associate modification to complex and ```update()``` its formation

```python
complex_data = me.complex_data.complex_ab
complex_data.modifications['mod_2fe2s_c'] = 1
print(me.reactions.formation_complex_ab.reaction)
me.reactions.formation_complex_ab.update()
print(me.reactions.formation_complex_ab.reaction)
```

    protein_a + protein_b --> complex_ab
    2fe2s_c + 4.27350427350427e-6*mu complex_ba + protein_a + protein_b --> 0.17582 biomass + complex_ab

## Add Metabolic Reaction
MetabolicReactions require, at a minimum, one corresponding StoichiometricData. StoichiometricData essentially holds the information contained in an M-model reaction. This includes the metabolite stoichiometry and the upper and lower bound of the reaction. As a best practice, StoichiometricData typically uses an ID equivalent to the M-model reaction ID.

### Without utility functions
So first, we will create a StoichiometricData object to define the stoichiometry of the conversion of *a* to *b*

The code below creates a StoichiometricData object which stores all of the stoichiometric information associated with the MetabolicReaction. This includes:
 - Reaction Stoichiometry
 - Lower and upper bound

Only one StoichiometricData object should be created for both reversible and irreversible reactions

```python
# unique to COBRAme, construct a stoichiometric data object with the reaction information
data = cobrame.StoichiometricData('a_to_b', me)
stoichiometry = {'a':-1, 'b': 1}
data._stoichiometry = stoichiometry
data.lower_bound = -1000
data.upper_bound = 1000
```

The StoichiometricData for this reversible reaction is then assigned to two different MetabolicReactions (Due to the enzyme dilution constraint, all enzyme catalyzed reactions must be reverisble; more on this later). The MetabolicReactions require:
 - The associated StoichiometricData
 - The *reverse* flag set to True for reverse reactions, False for forward reactions
 - Enzyme $K_{eff}$ for reaction (discussed later, dafault=65)

These fields are then processed and the actual model reaction is created using the MetabolicReaction's update() function

```python
# Create a forward ME Metabolic Reaction and associate the stoichiometric data to it
rxn = cobrame.MetabolicReaction('a_to_b_FWD_complex_ab')
me.add_reaction(rxn)
rxn.stoichiometric_data = data
rxn.reverse = False
rxn.keff = 65.
# Update
rxn.update(verbose=False)

# Create a reverse ME Metabolic Reaction and associate the stoichiometric data to it
rxn = cobrame.MetabolicReaction('a_to_b_REV_complex_ab')
me.add_reaction(rxn)
rxn.stoichiometric_data = data
rxn.reverse = True
rxn.keff = 65.
# Update
rxn.update(verbose=False)

print(me.reactions.a_to_b_FWD_complex_ab.reaction)
print(me.reactions.a_to_b_REV_complex_ab.reaction)
```

    a --> b
    b --> a

**Note**: the $k_{eff}$ and `complex_ab` is not included in the reaction since no complex has been associated to it yet

### Associate enzyme with MetabolicReaction

The ComplexData object created in the previous cell can be incorporated into the MetabolicReaction using code below.

**Note**: the update() function is required to apply the change.

```python
data = me.complex_data.complex_ab
me.reactions.a_to_b_FWD_complex_ab.complex_data = data
print('Forward reaction (before update): %s' %
      (me.reactions.a_to_b_FWD_complex_ab.reaction))
me.reactions.a_to_b_FWD_complex_ab.update()
print('Forward reaction (after update): %s' %
      (me.reactions.a_to_b_FWD_complex_ab.reaction))
print('')

me.reactions.a_to_b_REV_complex_ab.complex_data = data
print('Reverse reaction (before update): %s' %
      (me.reactions.a_to_b_REV_complex_ab.reaction))
me.reactions.a_to_b_REV_complex_ab.update()
print('Reverse reaction (after update): %s' %
      (me.reactions.a_to_b_REV_complex_ab.reaction))
```

    Forward reaction (before update): a --> b
    Forward reaction (after update): a + 4.27350427350427e-6*mu complex_ab --> b

    Reverse reaction (before update): b --> a
    Reverse reaction (after update): b + 4.27350427350427e-6*mu complex_ab --> a

The coefficient for complex_ab is determined by the expression $$\frac{\mu}{k_{eff}}$$ which in its entirety represents the dilution of an enzyme following a cell doubling. The coupling constraint can be summarized as followed
$$
\begin{align}
&v_{dilution,j} = \mu \sum_{i} \left( \frac{1}{k_{eff,i}} v_{usage,i} \right), & \forall j \in Enzyme
\end{align}
$$

Where

 - $v_{usage,i}$ is the flux through the metabolic reaction
 - $k_{eff}$ is the turnover rate for the process and conveys the productivity of the enzyme complex. Physically, it can be thought of as the number of reactions the enzyme can catalyze per cell division.

By default the $k_{eff}$ for a MetabolicReaction is set to 65 but this can be changed using the code below.

```python
me.reactions.a_to_b_FWD_complex_ab.keff = .00001
me.reactions.a_to_b_FWD_complex_ab.update()

# The forward and reverse direction can have differing keffs
print('Forward reaction')
print('----------------')
print(me.reactions.a_to_b_FWD_complex_ab.reaction)
print('')
print('Reverse reaction')
print('----------------')
print(me.reactions.a_to_b_REV_complex_ab.reaction)
```

    Forward reaction
    ----------------
    a + 27.7777777777778*mu complex_ab --> b

    Reverse reaction
    ----------------
    b + 4.27350427350427e-6*mu complex_ab --> a

### Using utility functions

```python
stoich_data = cobrame.StoichiometricData('b_to_c', me)
stoich_data._stoichiometry = {'b': -1, 'c': 1}
stoich_data.lower_bound = 0
stoich_data.upper_bound = 1000.
building.add_metabolic_reaction_to_model(me, stoich_data.id, 'forward', complex_id='complex_ab',
                                         update=True)
print('Reaction b_to_c')
print('---------------')
print(me.reactions.b_to_c_FWD_complex_ab.reaction)
```

    Created <Metabolite c at 0x7f18a6e81240> in <MetabolicReaction b_to_c_FWD_complex_ab at 0x7f18a6e813c8>
    Reaction b_to_c
    ---------------
    b + 4.27350427350427e-6*mu complex_ab --> c


```python
%run ../common/import_all.py

from common.setup_notebook import set_css_style, setup_matplotlib, config_ipython
config_ipython()
setup_matplotlib()
set_css_style()
```

# The backpropagation algorithm

The backpropagation algorithm is the core of how artificial neural networks manage to *learn*, doing so by iteratively correcting the error between the actual value and the predicted value in a back-propagation fashion. The original paper proposing this now universally adopted mechanism is a Nature from 1986 by Rumelhart, Hinton, Williams [[1]](#paper-original).

## What is it

The backpropagation algorithm is a brilliant way to train a network by perturbing each weight iteratively with an amount proportional to the partial derivative of the cost function with respect to it, propagating these derivatives backwards in the network. This is done to aid gradient descent and eventually train the network by reducing the error between what gets predicted and what is actually.

The idea per se is simple, the implementation is hard though and it took some research to figure out an efficient mechanism for it. Mechanism that arrived with Rumelhart & co. paper in 1986.

The reason why backpropagation is the core of the learning procedure of neural networks is that by adjusting the weights though little kicks and repeatedly, the hidden layers of the network come to *learn* features. While what happens to the input and output layers is controllable, it is the hidden layer(s) that do all the painstaking work of representing the featured of the input data. If in a network there were no hidden layer, it would be easy to change the weights in such a way that the output matches the expected real output. But the network wouldn't be learning and wouldn't do anything worth of excitement. It is via backpropagation that the network can learn, in its hidden neurons, how to represent the data.

## The procedure in detail

### Prologue: gradient descent

The notes here will follow both the original paper cited above and [this very helpful paragraph on Wikipedia](https://en.wikipedia.org/wiki/Backpropagation#Finding_the_derivative_of_the_error) about the topic, and will refer to a [feedforward network](anns.ipynb#Feedforward-networks) of [sigmoid neurons](types/sigmoid-neuron.ipynb), however the backpropagation procedure applies to a generic activation function, so long as it's differentiable, but the sigmoid makes for very nice calculations.

Let's consider the transmission of information to a neuron $k$ in the $l$-th layer, we will use $i$ to indicate an input and $o$ to indicate an output, and will make use of [this note](types/sigmoid-neuron.ipynb#A-note-on-notation) to factor the bias inside the transfer function as a further weight. The neuron receives input from all the neurons in the previous, $l-1$-th layer as a weighted combination of their outputs as per activation function:

$$
i_k^l = \sum_i o_i^{l-1} w_{ik}^{l-1,l}
$$

where the apex indicates the layer we are referring to. Note that the weight $w_{ik}^{l-1,l}$ is meant to represent the weight of the connection between neuron $i$ in layer $l-1$ and our reference neuron $k$ in layer $l$.

As per output function, the output of $k$ is (using a sigmoid output function as per tradition, this will prove to be a very convenient choice later on)

$$
o_k^l = \frac{1}{1 + e^{- \sum_i o_i^{l-1} w_{ik}^{l-1,l}}}
$$

The goal of training the network is finding the weights such that the network output is near the expected one. For this goal, we have to define a cost function which measures the difference between expected and obtained result, and minimise it. Now, the cost function can be given as the mean squared error

$$
E = \frac{1}{2n} \sum_i^n [o_i - e_i]^2 \ ,
$$

where the sum goes over all training samples, $o$ is the network output for the sample and $e$ the expected output. It gets minimised via [gradient descent](../maths/algorithms/gradient-descent.ipynb), which requires calculating the partial derivatives of it with respect to all weights (its parameters).

Note that the cost function could in principle be given differently, as long as it satisfies two requirements:

1. it is differentiable
2. it can be written as a sum of constributions (or, we can say as a mean) of the single training data points

The first requirement is needed because as per gradient descent we will have to compute derivatives of the cost function; the second because we generalise the calculation to the whole training set by applying the sum over the calculations on a training data point.

Also note that in practive the version of gradient descent applied is the stochastic one.

### Calculating the derivatives of the cost function: backpropagation

Backpropagation propagates the partial derivatives of the cost function with respect to the parameter weights from the output layer back to the first one, iteratively. The derivative of the cost function with respect to a weight is computable via chain rule (for simplicity, we are not indicating the layer)

$$
\begin{equation}
    \frac{\partial E}{\partial w_{ij}} = \frac{\partial E}{\partial o_j} \frac{\partial o_j}{\partial i_j} \frac{\partial i_j}{\partial w_{ij}} \ ,
\end{equation}
$$

Now let's break down the components. For the last one, we have

$$
\frac{\partial i_j}{\partial w_{ij}} = o_i \ .
$$

For the second one, using the form of the activation function from above, we have (you can easily prove that the equality is right)

$$
\frac{\partial o_j}{\partial i_j} = o_j (1 - o_j) \ .
$$

We are left with calculating the first bit, $\frac{\partial E}{\partial o_j}$. The derivative of the cost function with respect to $o_j$ can be calculated if we think of the cost as a function of all outputs of all the neurons in layer $\bar{l}$ which receive input from neuron $j$,

$$
\frac{\partial E}{\partial o_j} = \frac{\partial E(\{o_k \forall k \in \bar l\})}{\partial o_j} \ ,
$$

which can be written as

$$
\frac{\partial E}{\partial o_j} = \sum_{k \in \bar l} \frac{\partial E}{\partial o_k} \frac{\partial o_k}{\partial o_j} = \sum_{k \in \bar l} \frac{\partial E}{\partial o_k} \frac{\partial o_k}{\partial i_k} \frac{\partial i_k}{\partial o_j} = \sum_{k \in \bar l} \frac{\partial E}{\partial o_k} \frac{\partial o_k}{\partial i_k} w_{jk}
$$

This recursive equation states that the derivative of the cost function with respect to $o_j$ can be calculated from the same derivatives with respect to the output of neurons in the further layer.

This all can be written as

$$
\begin{equation}
    \frac{\partial E}{\partial w_{ij}} = o_i \delta_j \ ,
\end{equation}
$$

with

$$
\delta_j = \sum_{k \in \bar l} \delta_k w_{jk} o_j (1-o_j) \ ,
$$

and in the case of a neuron in the output layer things are much easier to compute, leading to $\delta_j = (o_j-e_j)o_j(1-o_j)$.

These expression encapsulate the essence of backpropagation: you calculate the difference between the expected output and the obtained one and starting from the output layer you propagate the derivatives back in the network. $\delta$ is the error we are propagating. We have to start from where we can compute it (the output layer) and then iteratively go back scaling the previous layers in order to obtain the calculation in the preceding layers.

Following the gradient descent procedure, we need to update the parameters of the cost function (the weights) by perturbing them by an amount proportional to the derivative via a learning rate:

$$
\Delta w_{ji} = - \alpha o_i \delta_j
$$

## References

1.  D E Rumelhart, G E Hinton, R J Williams, [**Learning representations by back-propagating errors**](http://www.cs.toronto.edu/~hinton/absps/naturebp.pdf), *Nature*, 323.6088, 1986
2. [**Nielsen's book on backpropagation**](http://neuralnetworksanddeeplearning.com/chap2.html)

```python

```


## Machine Translation
A sequence-to-sequence model is a model that takes a sequence of items and outputs another sequence of items using two networks that are trained end-to-end. This is perfect for machine translation since input sequences are directly related to output sequences. We will looking at preparing a dataset for Machine Translation task and implementing a seq2seq model. We will be using the parallel corpus available from [here](ftp://ftp.monash.edu/pub/nihongo/examples.utf.gz)

### Concepts covered here:
- Encoder-Decoder architecture: We will be using a encoder to encode input sequences from one language in a latent space and use the encoding to generate words in the target language one token at a time. The encoder and the decoder are an RNNs.

- Attention: We will be looking at the attention mechanism described in [Bahdanau et al., 2015](https://arxiv.org/pdf/1409.0473.pdf). We will explore the theory, the intuition behind it and how to move from theory and intuition to implementation.

```python
from pathlib import Path
import re,string
import numpy as np
```

```python
#!mkdir data
```

```python
path = Path('data')
```

We will need to download the translation corpus and install a tokenizer for Japanese text.

```python
#get corpus
#!wget ftp://ftp.monash.edu/pub/nihongo/examples.utf.gz
#decompress it and move it to data folder
#gunzip examples.utf.gz
#mv examples.utf.gz data/
#install dependencies for mecab tokenizer
#!sudo apt install swig
#!sudo apt install mecab
#!sudo apt install libmecab-dev
#!sudo apt install mecab-ipadic-utf8
#sudo pip3 install mecab-python3
```

```python
corpus = (path/'examples.utf').open().readlines()
corpus[:5]
```

    ['A: ムーリエルは２０歳になりました。\tMuiriel is 20 now.#ID=1282_4707\n',
     'B: は 二十歳(はたち){２０歳} になる[01]{になりました}\n',
     'A: すぐに戻ります。\tI will be back soon.#ID=1284_4709\n',
     'B: 直ぐに{すぐに} 戻る{戻ります}\n',
     'A: すぐに諦めて昼寝をするかも知れない。\tI may give up soon and just nap instead.#ID=1300_4727\n']

```python
def make_corpus(corpus_path):
    corpus = corpus_path.open().readlines()
    en,ja = [],[]
    pat = r'#ID.+\n'
    for c in corpus:
        if 'A: ' in c:
            clean_c = c.replace('A: ','')
            res = re.search(pat,clean_c)
            clean_c = clean_c.replace(res.group(0),'').split('\t')
            ja.append(clean_c[0])
            en.append(clean_c[1])
    return en,ja
```

```python
en, ja = make_corpus(path/'examples.utf')
en[:2],ja[:2]
```

    (['Muiriel is 20 now.', 'I will be back soon.'],
     ['ムーリエルは２０歳になりました。', 'すぐに戻ります。'])

```python
import MeCab
```

```python
tagger = MeCab.Tagger('-Owakati')

def ja_tokenizer(text):
    result = tagger.parse(text)
    words = result.split()
    if len(words) ==0: return []
    if words[-1] == '\n':return words[:-1]
    return words
```

```python
ja_tokenizer(ja[0])
```

    ['ムーリエル', 'は', '２', '０', '歳', 'に', 'なり', 'まし', 'た', '。']

```python
import spacy
from spacy.symbols import ORTH
```

```python
en_tok = spacy.load('en')
```

```python
def en_tokenizer(text):
    text = text.lower()
    return [t.text for t in en_tok.tokenizer(text)]
```

```python
en_tokenizer(en[0])
```

    ['muiriel', 'is', '20', 'now', '.']

```python
en_toks = [en_tokenizer(text) for text in en]
ja_toks = [ja_tokenizer(text) for text in ja]
en_toks[:2], ja_toks[:2]
```

    ([['muiriel', 'is', '20', 'now', '.'],
      ['i', 'will', 'be', 'back', 'soon', '.']],
     [['ムーリエル', 'は', '２', '０', '歳', 'に', 'なり', 'まし', 'た', '。'],
      ['すぐ', 'に', '戻り', 'ます', '。']])

```python
len(en_toks), len(ja_toks)
```

    (149786, 149786)

```python
from collections import Counter,defaultdict
```

```python
def numericalize_tok(tokens, max_vocab=50000, min_freq=0, unk_tok="_unk_", pad_tok="_pad_", bos_tok="_bos_", eos_tok="_eos_"):
    if isinstance(tokens, str):
        raise ValueError("Expected to receive a list of tokens. Received a string instead")
    if isinstance(tokens[0], list):
        tokens = [p for o in tokens for p in o]
    freq = Counter(tokens)
    int2tok = [o for o,c in freq.most_common(max_vocab) if c>min_freq]
    unk_id = 3
    int2tok.insert(0, bos_tok)
    int2tok.insert(1, pad_tok)
    int2tok.insert(2, eos_tok)
    int2tok.insert(unk_id, unk_tok)
    tok2int = defaultdict(lambda:unk_id, {v:k for k,v in enumerate(int2tok)})
    return int2tok, tok2int
```

```python
int2j,j2int = numericalize_tok(ja_toks)
int2en,en2int = numericalize_tok(en_toks)
```

```python
len(int2j), len(int2en)
```

    (31813, 21393)

```python
import pickle
```

```python
pickle.dump(int2j,(path/'int2j.pkl').open('wb'))
pickle.dump(int2en,(path/'int2en.pkl').open('wb'))
```

```python
int2j = pickle.load((path/'int2j.pkl').open('rb'))
int2en = pickle.load((path/'int2en.pkl').open('rb'))
j2int = defaultdict(lambda:3, {v:k for k,v in enumerate(int2j)})
en2int = defaultdict(lambda:3, {v:k for k,v in enumerate(int2en)})
```

```python
len(int2j), len(int2en)
```

    (31813, 21393)

```python
j_ids = np.array([[0]+[j2int[o] for o in sent]+[2] for sent in ja_toks])
en_ids = np.array([[0]+[en2int[o] for o in sent]+[2] for sent in en_toks])
len(j_ids),len(en_ids), j_ids[10],en_ids[10]
```

    (149786,
     149786,
     [0,
      48,
      6,
      4891,
      5,
      109,
      11,
      143,
      10,
      83,
      8,
      57,
      86,
      1798,
      7,
      2146,
      232,
      255,
      47,
      36,
      4,
      2],
     [0, 114, 2251, 107, 38, 97, 85, 2649, 77, 28, 356, 4, 2])

```python
np.random.seed(42)
```

```python
trn_keep = np.random.rand(len(en_ids))>0.1
en_trn,j_trn = en_ids[trn_keep],j_ids[trn_keep]
en_val,j_val = en_ids[~trn_keep],j_ids[~trn_keep]
len(en_trn),len(en_val)
```

    (134775, 15011)

```python
import torch
import torch.nn as nn
import torch.nn.functional as F

from torch.autograd.variable import Variable
from torch.utils.data import Dataset,DataLoader
```

```python
from numpy import array as A
```

```python
class Seq2SeqDataset(Dataset):
    def __init__(self, x, y): self.x,self.y = x,y
    def __getitem__(self, idx): return A(self.x[idx]), A(self.y[idx])
    def __len__(self): return len(self.x)
```

```python
trn_ds = Seq2SeqDataset(en_trn,j_trn)
val_ds = Seq2SeqDataset(en_val,j_val)

bs = 120

trn_dl = DataLoader(trn_ds,batch_size=bs,shuffle=True)
val_dl = DataLoader(val_ds,batch_size=bs)
```

```python
x, y = next(iter(val_dl))
x.size(), y.size()
```

    (torch.Size([120, 25]), torch.Size([120, 25]))

```python
from keras.preprocessing.sequence import pad_sequences
```

    Using TensorFlow backend.

```python
max(len(o) for o in en_ids), max(len(o) for o in j_ids)
```

    (56, 70)

```python
j_ids = pad_sequences(j_ids, maxlen=25, dtype='int32', padding='post', truncating='post', value=1)
en_ids = pad_sequences(en_ids, maxlen=25, dtype='int32', padding='post', truncating='post', value=1)
```

```python
## load fasttext vectors
import io
def load_vectors(fname):
    fin = io.open(fname, 'r', encoding='utf-8', newline='\n', errors='ignore')
    header = fin.readline().split()
    n, d = int(header[0]), int(header[1])
    data = {}
    for line in fin:
        tokens = line.rstrip().split(' ')
        data[tokens[0]] = np.array(tokens[1:], dtype=float)
    return data, n, d
```

```python
# get word vectors
#!wget https://dl.fbaipublicfiles.com/fasttext/vectors-english/wiki-news-300d-1M.vec.zip
#!wget https://dl.fbaipublicfiles.com/fasttext/vectors-crawl/cc.ja.300.vec.gz
#!unzip wiki-news-300d-1M.vec.zip
#!gunzip cc.ja.300.vec.gz
#!mv wiki-news-300d-1M.vec ../data/
#!mv cc.ja.300.vec ../data/
```

```python
en_vecs,_,dim_en_vec = load_vectors('../data/wiki-news-300d-1M.vec')
j_vecs,_,dim_j_vec = load_vectors('../data/cc.ja.300.vec')
```

```python
def create_emb(vecs, itos, em_sz):
    emb = nn.Embedding(len(itos), em_sz, padding_idx=1)
    wgts = emb.weight.data
    miss = []
    for i,w in enumerate(itos):
        try: wgts[i] = torch.from_numpy(vecs[w])
        except: miss.append(w)
    print('Number of unknowns in data: {}'.format(len(miss)))
    return emb

```

```python
def V(tensor,req_grad=True):
    if torch.cuda.is_available():return Variable(tensor.cuda())
    else: return Variable(tensor)
```

### RNN Visualization

```python
from IPython.display import HTML
HTML('')
## video borrowed from http://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/
```

### Seq2Seq Architecture
> ...a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector.

[Sutskever et al., 2014](https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf)

Image:[from here](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)

```python
class Seq2Seq(nn.Module):
    def __init__(self,en_vecs,int2en,j_vecs,int2j,em_sz,nh=128,out_sl=25,dropf=1,nl=2):
        super().__init__()
        #encoder
        self.nl,self.nh,self.em_sz,self.out_sl = nl,nh,em_sz,out_sl
        self.emb_enc = create_emb(en_vecs,int2en,dim_en_vec)
        self.emb_drop = nn.Dropout(0.15*dropf)
        self.encoder = nn.GRU(dim_en_vec,nh,num_layers=nl,dropout=0.25*dropf, bidirectional=True)
        #decoder
        self.emb_dec = create_emb(j_vecs,int2j,dim_j_vec)
        self.decoder = nn.GRU(dim_en_vec,nh*2,num_layers=nl,dropout=0.25*dropf)
        self.out_drop = nn.Dropout(0.35*dropf)
        self.out = nn.Linear(nh*2,len(int2j))

    def forward(self,inp,y=None):
        sl, bs = inp.size()
        emb_in = self.emb_drop(self.emb_enc(inp))
        h_n = self.initHidden(bs)
        enc_out, h_n = self.encoder(emb_in,h_n)
        h_n = h_n.view(2,2,bs,-1).permute(0,2,1,3).contiguous().view(self.nl,bs,-1)

        dec_inp = V(torch.zeros(bs).long())
        res = []
        for i in range(self.out_sl):
            dec_emb = self.emb_dec(dec_inp)
            outp,h_n = self.decoder(dec_emb.unsqueeze(0),h_n)
            outp = F.log_softmax(self.out(self.out_drop(outp[0])),dim=-1)
            res.append(outp)
            dec_inp = outp.data.max(1)[1]
            if (dec_inp==1).all(): break
        return torch.stack(res)

    def initHidden(self,bs):
        return V(torch.zeros([self.nl*2,bs,self.nh]))
```

### Why Bidirectional Encoder
> Finally, we found that reversing the order of the words in all source sentences (butnot target sentences) improved the LSTM’s performance markedly, because doing so introduced many short term dependencies between the source and the targetsentence which made the optimization problem easier.

[Sutskever et al., 2014](https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf)

Input and output sequences may not directly map to each other so preserving information from both passes of the input sequence will help learn how tokens relate to each other. For example in a translation task, subject and object can be in opposite positions depending on the language structure.

```python
seq2seq = Seq2Seq(en_vecs,int2en,j_vecs,int2j,dim_en_vec)
seq2seq.cuda()
```

    Number of unknowns in data: 974
    Number of unknowns in data: 492

    Seq2Seq(
      (emb_enc): Embedding(21393, 300, padding_idx=1)
      (emb_drop): Dropout(p=0.15)
      (encoder): GRU(300, 128, num_layers=2, dropout=0.25, bidirectional=True)
      (emb_dec): Embedding(31813, 300, padding_idx=1)
      (decoder): GRU(300, 256, num_layers=2, dropout=0.25)
      (out_drop): Dropout(p=0.35)
      (out): Linear(in_features=256, out_features=31813, bias=True)
    )

```python
out = seq2seq(V(x.transpose(1,0).long()))
out.size()
```

    torch.Size([25, 120, 31813])

```python
out[-1].size()
```

    torch.Size([120, 31813])

```python
def seq2seq_loss(input, target):
    sl,bs = target.size()
    sl_in,bs_in,nc = input.size()
    if sl>sl_in: input = F.pad(input, (0,0,0,0,0,sl-sl_in))
    input = input[:sl]
    return F.cross_entropy(input.view(-1,nc), target.view(-1))
```

```python
seq2seq_loss(out,V(y.transpose(1,0).long()))
```

    tensor(10.3854, device='cuda:0', grad_fn=<NllLossBackward>)

```python
def step(x, y, epoch, m, crit, opt, clip=None):
    output = m(x, y)
    if isinstance(output,tuple): output = output[0]
    opt.zero_grad()
    loss = crit(output, y)
    loss.backward()
    if clip:
        nn.utils.clip_grad_norm_(m.parameters(), clip)
    opt.step()
    return loss.data.item()
```

```python
from tqdm import tqdm
```

```python
def train(trn_dl,val_dl,model,crit,opt,epochs=10,clip=None):
    for epoch in range(epochs):
        loss_val = loss_trn = 0
        with tqdm(total=len(trn_dl)) as pbar:
            model.train()
            for i, ds in enumerate(trn_dl):
                x, y = ds
                #if isinstance(x,tuple): x = x[0]
                x, y = x.transpose(1,0), y.transpose(1,0)
                loss = step(V(x.long()),V(y.long()),epoch,model,crit,opt)
                loss_trn += loss
                pbar.update()
        model.eval()
        for i, ds in enumerate(val_dl):
            with torch.no_grad():
                x, y = ds
                #if isinstance(x,tuple): x = x[0]
                x, y = x.transpose(1,0), y.transpose(1,0)
                out = model(V(x.long()))
                if isinstance(out,tuple): out = out[0]
                loss_val+= crit(out, V(y.long()))
                #loss_val +=loss
        print(f'Epoch: {epoch} trn loss: {loss_trn/len(trn_dl)} val loss: {loss_val/len(val_dl)}')
```

```python
from torch import optim
```

```python
opt = optim.Adam(seq2seq.parameters(),lr=3e-3,betas=(0.7,0.8))
```

```python
train(trn_dl,val_dl,seq2seq,seq2seq_loss,opt,epochs=10)
```

    100%|██████████| 1124/1124 [02:40<00:00,  7.02it/s]
      0%|          | 1/1124 [00:00<03:05,  6.06it/s]

    Epoch: 0 trn loss: 4.750732962983359 val loss: 4.082951068878174

    100%|██████████| 1124/1124 [02:52<00:00,  6.51it/s]
      0%|          | 1/1124 [00:00<03:11,  5.86it/s]

    Epoch: 1 trn loss: 3.5997307712073003 val loss: 3.3246185779571533

    100%|██████████| 1124/1124 [02:55<00:00,  6.39it/s]
      0%|          | 1/1124 [00:00<03:04,  6.08it/s]

    Epoch: 2 trn loss: 3.1077632744965604 val loss: 3.4751904010772705

    100%|██████████| 1124/1124 [02:57<00:00,  6.34it/s]
      0%|          | 1/1124 [00:00<02:50,  6.60it/s]

    Epoch: 3 trn loss: 2.8518904808153036 val loss: 3.94274640083313

    100%|██████████| 1124/1124 [02:58<00:00,  6.30it/s]
      0%|          | 1/1124 [00:00<03:06,  6.01it/s]

    Epoch: 4 trn loss: 2.7045018161743135 val loss: 2.766321897506714

    100%|██████████| 1124/1124 [02:58<00:00,  6.29it/s]
      0%|          | 1/1124 [00:00<03:06,  6.01it/s]

    Epoch: 5 trn loss: 2.5743112220458713 val loss: 2.752124071121216

    100%|██████████| 1124/1124 [02:58<00:00,  6.28it/s]
      0%|          | 1/1124 [00:00<03:04,  6.08it/s]

    Epoch: 6 trn loss: 2.5332621384769998 val loss: 2.935807228088379

    100%|██████████| 1124/1124 [02:59<00:00,  6.27it/s]
      0%|          | 1/1124 [00:00<03:03,  6.11it/s]

    Epoch: 7 trn loss: 2.500852593323514 val loss: 3.133931875228882

    100%|██████████| 1124/1124 [02:59<00:00,  6.99it/s]
      0%|          | 1/1124 [00:00<03:04,  6.10it/s]

    Epoch: 8 trn loss: 2.458315055141245 val loss: 2.8610451221466064

    100%|██████████| 1124/1124 [02:59<00:00,  6.26it/s]

    Epoch: 9 trn loss: 2.456163497369909 val loss: 3.016350269317627

```python
def produce_out(val_dl, model,int2en,int2j,interval=(20,30)):
    model.eval()
    x,y = next(iter(val_dl))
    x, y = x.transpose(1,0), y.transpose(1,0)
    probs = seq2seq(V(x.long()))
    if isinstance(probs,tuple): probs = probs[0]
    preds = A(probs.max(2)[1])
    for i in range(interval[0],interval[1]):
        print(' '.join([int2en[o] for o in x[:,i] if o not in [0,1,2]]))
        print(''.join([int2j[o] for o in y[:,i] if o not in [0,1,2]]))
        print(''.join([int2j[o] for o in preds[:,i] if o not in [0,1,2]]))
        print()
```

```python
produce_out(trn_dl,seq2seq,int2en,int2j)
```

    my grandfather tells us about old things .
    祖父は昔の話をしてくれます。
    私は私のののををて。

    if it had not been for your help , we might have failed .
    もしあなたの援助がなければ、私たちは失敗していたかもしれない。
    もしののがなかったら、私私失敗失敗失敗たうううう

    there was a long interval before he answered .
    彼が答えるまでに長い間があった。
    彼ははのののががた。

    i was scolded by my mother for being lazy .
    怠けていて母にしかられた。
    私は母にれれれれたた

    while she was running after the naughty boy , one of her shoes came off .
    いたずらっ子を追いかけているうちに彼女の靴が片方脱げた。
    彼女はが彼女て、、彼女彼女のににてて。。

    they contributed money to the red cross .
    彼らは赤十字にお金を寄付した。
    彼らはお金をためををたた。

    i 've just oiled the wheels .
    車輪には油を注したばかりだ。
    私はをををたた。。

    there are such shops about .
    そのような店はあちらこちらにある。
    そのににはははががて。。

    give me a knife to cut this string with .
    この紐を切るためのナイフを貸してください。
    このをををををててて。

    he is bound to pass the test .
    彼はきっと試験に合格する。
    彼は試験ににするするだろ。

```python
torch.save(seq2seq.state_dict(),open(path/'translate_seq2seq.pth','wb'))
```

```python
seq2seq.load_state_dict(torch.load(path/'translate_seq2seq.pth', map_location=lambda storage, loc: storage))
```

### Seq2Seq w/ Attention
A critical and apparent disadvantage of this fixed-length context vector design is incapability of remembering long sentences. Often it has forgotten the first part once it completes processing the whole input. The attention mechanism was born [Bahdanau et al., 2015](https://arxiv.org/pdf/1409.0473.pdf) to resolve this problem.

Given the following vectors:
\begin{align}
\boldsymbol{x} = \{x_1,x_2,x_3,\ldots,x_n\} \\
\boldsymbol{y} = \{y_1,y_2,y_3,\ldots,y_m\} \\
\end{align}
The hidden state from the Bidir encoder is given by:
\begin{align}
\boldsymbol{h}_i = [\overrightarrow{\boldsymbol{h}}_i^\top; \overleftarrow{\boldsymbol{h}}_i^\top]^\top, i=1,\dots,n \\
\end{align}

The hidden state from the decoder at time $t$ is given by:  $s_t$

The score for at time $t$:

\begin{aligned}
\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \mathbf{v}_a^\top \tanh(\mathbf{W}_a[\boldsymbol{s}_t; \boldsymbol{h}_i])
\end{aligned}

where both $v_a$ and $W_a$ are weight matrices to be learned in the alignment model.

\begin{aligned}
\mathbf{c}_t &= \sum_{i=1}^n \alpha_{t,i} \boldsymbol{h}_i & \small{\text{; Context vector for output }y_t}\\
\alpha_{t,i} &= \text{align}(y_t, x_i) & \small{\text{; How well two words }y_t\text{ and }x_i\text{ are aligned.}}\\
&= \frac{\exp(\text{score}(\boldsymbol{s}_{t-1}, \boldsymbol{h}_i))}{\sum_{i'=1}^n \exp(\text{score}(\boldsymbol{s}_{t-1}, \boldsymbol{h}_{i'}))} & \small{\text{; Softmax of some predefined alignment score.}}.
\end{aligned}

Equations borrowed from [here](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)

```python
HTML('')
## video borrowed from http://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/
```

```python
import math,random

def rand_t(*sz): return torch.randn(sz)/math.sqrt(sz[0])
def rand_p(*sz): return nn.Parameter(rand_t(*sz))
```

```python
class Seq2SeqAttention(nn.Module):
    def __init__(self,en_vecs,int2en,j_vecs,int2j,em_sz,nh=128,out_sl=25,dropf=1,nl=2):
        super().__init__()
        #encoder
        self.nl,self.nh,self.em_sz,self.out_sl = nl,nh,em_sz,out_sl
        self.emb_enc = create_emb(en_vecs,int2en,dim_en_vec)
        self.emb_drop = nn.Dropout(0.15*dropf)
        self.encoder = nn.GRU(dim_en_vec,nh,num_layers=nl,dropout=0.25*dropf, bidirectional=True)
        #decoder
        self.emb_dec = create_emb(j_vecs,int2j,dim_j_vec)
        self.decoder = nn.GRU(dim_en_vec,nh*2,num_layers=nl,dropout=0.25*dropf)
        self.out_drop = nn.Dropout(0.35*dropf)
        self.out = nn.Linear(nh*2,len(int2j))
        #attention layer
        self.W1 = rand_p(nh*2, nh*2) #parameter
        self.l2 = nn.Linear(nh*2, nh*2)
        self.l3 = nn.Linear(dim_en_vec+nh*2, dim_en_vec)
        self.V = rand_p(nh*2) #parameter

    def forward(self,inp,y=None):
        sl, bs = inp.size()
        emb_in = self.emb_drop(self.emb_enc(inp))
        h_n = self.initHidden(bs)
        enc_out, h_n = self.encoder(emb_in,h_n)
        h_n = h_n.view(2,2,bs,-1).permute(0,2,1,3).contiguous().view(self.nl,bs,-1)

        dec_inp = V(torch.zeros(bs).long())
        res,attns = [], []
        #multiply by parameter
        w1e = enc_out @ self.W1
        for i in range(self.out_sl):
            #linear layer
            w2h = self.l2(h_n[-1])
            #non-linear activation to calculate score
            u = torch.tanh(w1e + w2h)
            #softmax to make them into probs
            a = F.softmax(u @ self.V, 0)
            attns.append(a)
            #multiply each vector by scores and then add them up
            Xa = (a.unsqueeze(2) * enc_out).sum(0)
            dec_emb = self.emb_dec(dec_inp)
            #linear layer to reduce dimensions
            wgt_enc = self.l3(torch.cat([dec_emb, Xa], 1))
            outp,h_n = self.decoder(wgt_enc.unsqueeze(0),h_n)
            outp = F.log_softmax(self.out(self.out_drop(outp[0])),dim=-1)
            res.append(outp)
            dec_inp = outp.data.max(1)[1]
            if (random.random() > 0.5) and y is not None: dec_inp=y[i]
            if (dec_inp==1).all(): break
        return torch.stack(res),attns

    def initHidden(self,bs):
        return V(torch.zeros([self.nl*2,bs,self.nh]))
```

```python
seq2seq = Seq2SeqAttention(en_vecs,int2en,j_vecs,int2j,dim_en_vec)
seq2seq.cuda()
```

    Number of unknowns in data: 974
    Number of unknowns in data: 492

    Seq2SeqAttention(
      (emb_enc): Embedding(21393, 300, padding_idx=1)
      (emb_drop): Dropout(p=0.15)
      (encoder): GRU(300, 128, num_layers=2, dropout=0.25, bidirectional=True)
      (emb_dec): Embedding(31813, 300, padding_idx=1)
      (decoder): GRU(300, 256, num_layers=2, dropout=0.25)
      (out_drop): Dropout(p=0.35)
      (out): Linear(in_features=256, out_features=31813, bias=True)
      (l2): Linear(in_features=256, out_features=256, bias=True)
      (l3): Linear(in_features=556, out_features=300, bias=True)
    )

```python
opt = optim.Adam(seq2seq.parameters(),lr=3e-3,betas=(0.7,0.8))
```

```python
train(trn_dl,val_dl,seq2seq,seq2seq_loss,opt,epochs=10)
```

    100%|██████████| 1124/1124 [03:21<00:00,  6.28it/s]
      0%|          | 1/1124 [00:00<03:27,  5.42it/s]

    Epoch: 0 trn loss: 2.797933133683595 val loss: 4.138897895812988

    100%|██████████| 1124/1124 [03:26<00:00,  6.24it/s]
      0%|          | 1/1124 [00:00<03:32,  5.28it/s]

    Epoch: 1 trn loss: 2.16145394790215 val loss: 2.8873095512390137

    100%|██████████| 1124/1124 [03:26<00:00,  6.17it/s]
      0%|          | 1/1124 [00:00<03:29,  5.35it/s]

    Epoch: 2 trn loss: 2.0703822573733075 val loss: 3.0309219360351562

    100%|██████████| 1124/1124 [03:26<00:00,  6.07it/s]
      0%|          | 1/1124 [00:00<03:28,  5.38it/s]

    Epoch: 3 trn loss: 2.07098806148322 val loss: 2.844566822052002

    100%|██████████| 1124/1124 [03:26<00:00,  6.07it/s]
      0%|          | 1/1124 [00:00<03:28,  5.38it/s]

    Epoch: 4 trn loss: 2.088127380906475 val loss: 2.8696582317352295

    100%|██████████| 1124/1124 [03:26<00:00,  6.26it/s]
      0%|          | 1/1124 [00:00<03:29,  5.36it/s]

    Epoch: 5 trn loss: 2.093448414603162 val loss: 2.7895729541778564

    100%|██████████| 1124/1124 [03:26<00:00,  6.12it/s]
      0%|          | 1/1124 [00:00<03:29,  5.37it/s]

    Epoch: 6 trn loss: 2.1055155513125383 val loss: 2.9072670936584473

    100%|██████████| 1124/1124 [03:26<00:00,  5.45it/s]
      0%|          | 1/1124 [00:00<03:30,  5.34it/s]

    Epoch: 7 trn loss: 2.11192792569191 val loss: 2.7808303833007812

    100%|██████████| 1124/1124 [03:26<00:00,  6.23it/s]
      0%|          | 1/1124 [00:00<03:29,  5.37it/s]

    Epoch: 8 trn loss: 2.116793169139543 val loss: 2.8389084339141846

    100%|██████████| 1124/1124 [03:26<00:00,  6.13it/s]

    Epoch: 9 trn loss: 2.1336968614325404 val loss: 2.8806753158569336

```python
produce_out(trn_dl,seq2seq,int2en,int2j)
```

    i 've been to osaka on business .
    仕事で大阪まで行ってきた。
    私は東京に行っていた。

    she has no sense of the beautiful .
    彼女は美に対するセンスを持っていない。
    彼女はその美しいががない。

    please give him my best regards .
    彼によろしくお伝え下さい。
    私におて下さい。

    she is working night and day .
    彼女は昼も夜も働いている。
    彼女は一日中にてている。

    words fail right when you need them .
    いよいよという時に言葉が出ない。
    あなたの言葉にはている。

    she excels her class in music .
    彼女は音楽ではクラスの誰よりも優れている。
    彼女はクラスの音楽が好きだ。

    he was brilliant in the morning sun .
    彼が朝日を受けて光り輝いていた。
    彼は太陽のにににた。

    he searched the room for the lost key .
    彼はなくした鍵を捜して部屋を調べた。
    彼は部屋を部屋ををた。

    a sprain like this should heal within a week or so .
    この程度の捻挫なら、１週間程で治るでしょう。
    こののははににににはている。

    rules in connection with staff selection are set as follows .
    職員の選考に関する規則を次のように定める。
    次ののははののである。

```python
torch.save(seq2seq.state_dict(),open(path/'translate_seq2seq_attention.pth','wb'))
```

```python
seq2seq.load_state_dict(torch.load(path/'translate_seq2seq_attention.pth', map_location=lambda storage, loc: storage))
```

```python
out, atts = seq2seq(x.long().cuda())
```

```python
import matplotlib.pyplot as plt
plt.switch_backend('agg')
import matplotlib.ticker as ticker
```

```python
from matplotlib import rc
rc('font',family='TakaoPGothic')
rc('xtick', labelsize=20)
rc('ytick', labelsize=20)
```

```python
atts[0].size()
```

    torch.Size([120, 25])

```python
plt.matshow(torch.stack(atts)[:,2,:].detach().cpu().numpy())
```

```python
def get_out(inp,model):
    model.eval()
    inp_toks = en_tokenizer(inp)
    inp_ids = torch.LongTensor([0]+[en2int[w] for w in inp_toks]+[2]).cuda()
    probs, atts = model(inp_ids.unsqueeze(1))
    preds = A(probs.max(2)[1])
    out_sent = [int2j[i] for i in preds[:,0] if i not in [1]]
    return ['bos_']+inp_toks, out_sent, atts
```

```python
input_sentence, output_words, attentions = get_out('How are you?',seq2seq)
```

```python
def showAttention(input_sentence, output_words, attentions):
    # attention viz func borrowed from official pytorch seq2seq tutorial
    # Set up figure with colorbar
    fig = plt.figure()
    ax = fig.add_subplot(111)
    max_len = max(len(input_sentence), len(output_words))
    cax = ax.matshow(torch.stack(attentions)[:max_len,:,0].detach().cpu().numpy(), cmap='bone')
    fig.colorbar(cax)

    # Set up axes
    ax.set_xticklabels(['']+input_sentence+['_eos_'],rotation=90)
    ax.set_yticklabels(['']+output_words)

    #for label in ax.get_xticklabels():
    #    label.set_fontproperties(prop)

    #for label in ax.get_yticklabels():
    #    label.set_fontproperties(prop)

    # Show label at every tick
    ax.xaxis.set_major_locator(ticker.MultipleLocator(1))
    ax.yaxis.set_major_locator(ticker.MultipleLocator(1))
    plt.show()
```

```python
showAttention(*get_out('How are you?',seq2seq))
```

```python

```