# Construction of the Model of Adaptive Resistance in Melanoma (MARM1)

In this notebook we will provide a step-by-step construction of the MARM model. The model is constructed using the rule-based modeling tool PySB . The model definition files generated by this script in PySB (*MARM1.py*), BGN (*MARM1.bng*) and SBML (*MARM1.sbml*) formats will replace the current ones in the MARM1 supplement directory and can be used to simulate different versions of the MARM1 model. Please see the Jupyter Notebooks *MARM1_simulation_single_run.ipynb* and *MARM1_simulation_multiple_conditions.ipynb* to run simulations using the generated model.To start, we import all required pysb classes and instantiate the model:

```python
from pysb import Model, Monomer, Parameter, Expression,  Rule, Observable, Initial, Annotation, EnergyPattern, ANY
from pysb.bng import generate_equations
from pysb.export import export
from sympy import exp, log

Model();
```

In the following, we split the model into individual submodules of strongly interacting components, which we will describe individually. The model describes protein and drug species as intracellular concentrations in uM at a volume of `1pL = 1e-12L`. For ligands and drugs we assume that the extracellular compartment is much bigger than the intracellular compartement and that equilibration is fast such that concentrations can be assumed to be constanct over time (implemented by specifying `fixed=True` in the initialization). For EGF, the parameter that specifies the abundance, `EGF_0` is assumed to have units [ng/ml] and is accordingly transformed into [uM] using the appropriate molecular weight `m_Da_EGF`. For the drugs Vemurafenib (`RAFi`)  and Cobimetinib (`MEKi`), the respective parameters, `RAFi_0` and `MEKi_0`, are assumed to be in uM. The unit of time is hours.

```python
Expression('N_Avogadro', 6.02214076000000e+23)
Expression('volume', 1.00000000000000e-12)
Expression('m_Da_EGF', 6200.00000000000)

Monomer('RAFi', ['raf'])
Annotation(RAFi, 'http://identifiers.org/chebi/63637', 'is')
Parameter('RAFi_0', 0.0)
Expression('initRAFi', RAFi_0)
Initial(RAFi(raf=None), initRAFi, fixed=True)

Monomer('MEKi', ['mek'])
Annotation(MEKi, 'http://identifiers.org/chebi/90851', 'is')
Parameter('MEKi_0', 0.0)
Expression('initMEKi', MEKi_0)
Initial(MEKi(mek=None), initMEKi, fixed=True)

Monomer('EGF', ['rtk'])
Annotation(EGF, 'http://identifiers.org/uniprot/P01133', 'is')
Parameter('EGF_0', 0.0)
Expression('initEGF', 6.02214085854916e+23*EGF_0*(m_Da_EGF*N_Avogadro)**(-1))
Initial(EGF(rtk=None), initEGF, fixed=True);
```

## EGFR Signaling
The EGFR submodule describes the interaction between Egfr, Egf, Grb2, Sos1 and Ras (which describes the action K-Ras, N-Ras and H-Ras). Here, we instantiate
the respective monomers and provide uniprot IDs as monomer annotations.

```python
Monomer('EGFR', ['SH2', 'rtk', 'rtkf'])
Annotation(EGFR, 'http://identifiers.org/uniprot/P00533', 'is')

Monomer('GRB2', ['SH2', 'SH3'])
Annotation(GRB2, 'http://identifiers.org/uniprot/P62993', 'is')

Monomer('SOS1', ['S1134', 'SH3', 'ras'], {'S1134': ['p', 'u']})
Annotation(SOS1, 'http://identifiers.org/uniprot/Q07889', 'is')

Monomer('RAS', ['raf', 'sos1', 'state'], {'state': ['gdp', 'gtp']})
Annotation(RAS, 'http://identifiers.org/uniprot/P01111', 'is')
Annotation(RAS, 'http://identifiers.org/uniprot/P01116', 'is')
Annotation(RAS, 'http://identifiers.org/uniprot/P01112', 'is');
```

## EGF activation
To describe activation of Egfr through Egf, we implement base binding rates between Egf and Egfr. We implement these rules as energy rules, which allow us to later use the energy-based formalism in BioNetGen  to describe allosteric interactions between the two molecules. Energy rules
are parameterised using three distinct parameters: the `kD` parameter which describes the affinity between the two molecules, the `kf` parameter which describes the timescale of the interaction and the `phi` parameter which takes values between 0 and 1 and balance whether allosteric interactions modulate the affinity by changing the association rate (`kf`, only association if `phi=0`) or the dissociation rate (`kr = kf*kD`, only dissociation if `phi=1`).

We implement the ligand-receptor binding using the `rtk` binding site on the ligand and the `rtkf` binding site on the receptor. We implement the receptor-receptor binding using the `rtk` binding site on the receptor. This formalism simplifies future extensions of the model to additional ligands or receptors by ensuring that each receptor can only bind one ligand and one receptor.

```python
# binding Egf-Egfr
Parameter('bind_EGF_EGFR_kf', 10.0)
Parameter('bind_EGF_EGFR_kD', 0.01)
Parameter('bind_EGF_EGFR_phi', 1.0)
Expression('bind_EGF_EGFR_kr', bind_EGF_EGFR_kD*bind_EGF_EGFR_kf)
Expression('Gf_bind_EGF_EGFR', log(bind_EGF_EGFR_kD))
Expression('Ea0_bind_EGF_EGFR', -bind_EGF_EGFR_phi*log(bind_EGF_EGFR_kD) - log(bind_EGF_EGFR_kf))
Rule('EGF_and_EGFR_bind_and_dissociate',
     EGF(rtk=None) + EGFR(rtkf=None) | EGF(rtk=1) % EGFR(rtkf=1),
     bind_EGF_EGFR_phi, Ea0_bind_EGF_EGFR, energy=True)
EnergyPattern('ep_bind_EGF_EGFR', EGF(rtk=1) % EGFR(rtkf=1), Gf_bind_EGF_EGFR)

# binding Egfr-Egfr
Parameter('bind_EGFR_EGFR_kf', 10.0)
Parameter('bind_EGFR_EGFR_kD', 100.0)
Parameter('bind_EGFR_EGFR_phi', 1.0)
Expression('bind_EGFR_EGFR_kr', bind_EGFR_EGFR_kD*bind_EGFR_EGFR_kf)
Expression('Gf_bind_EGFR_EGFR', log(bind_EGFR_EGFR_kD))
Expression('Ea0_bind_EGFR_EGFR', -bind_EGFR_EGFR_phi*log(bind_EGFR_EGFR_kD) - log(bind_EGFR_EGFR_kf))
Rule('EGFR_and_EGFR_bind_and_dissociate',
     EGFR(rtk=None) + EGFR(rtk=None) | EGFR(rtk=1) % EGFR(rtk=1),
     bind_EGFR_EGFR_phi, Ea0_bind_EGFR_EGFR, energy=True)
EnergyPattern('ep_bind_EGFR_EGFR', EGFR(rtk=1) % EGFR(rtk=1), Gf_bind_EGFR_EGFR);
```

To implement allosteric interaction between Egf ligands and Egfr , we add two energy patterns that allow modulation of the affinities of Egfr-Egf and Egfr-Egfr interactions through binding of ligands.

```python
# Egf mediated affinity modulation
Parameter('ep_EGFR_EGFR_mod_EGF_single_deltaG', 0.001)
Parameter('ep_EGFR_EGFR_mod_EGF_double_deltaG', 0.001)
Expression('ep_EGFR_EGFR_mod_EGF_single_Gf', log(ep_EGFR_EGFR_mod_EGF_single_deltaG))
Expression('ep_EGFR_EGFR_mod_EGF_double_Gf',
           log(ep_EGFR_EGFR_mod_EGF_double_deltaG) + log(ep_EGFR_EGFR_mod_EGF_single_deltaG))

EnergyPattern('ep_EGFR_EGFR_mod_EGF_single',
              EGF(rtk=2) % EGFR(rtk=1, rtkf=2) % EGFR(rtk=1, rtkf=None),
              ep_EGFR_EGFR_mod_EGF_single_Gf)
EnergyPattern('ep_EGFR_EGFR_mod_EGF_double',
              EGF(rtk=2) % EGF(rtk=3) % EGFR(rtk=1, rtkf=2) % EGFR(rtk=1, rtkf=3),
              ep_EGFR_EGFR_mod_EGF_double_Gf)

Annotation(EGF_and_EGFR_bind_and_dissociate, 'http://identifiers.org/pubmed/16946702', 'isDescribedBy')
Annotation(EGFR_and_EGFR_bind_and_dissociate, 'http://identifiers.org/pubmed/16946702', 'isDescribedBy');
```

## GRB2 recruitement
Recruitement of Grb2 to receptor requires transactivation  of multiple Egfr phosphorylation sites that are recognized by the Grb2 Src2 homology 2 (SH2) domain . To simplify these interactions we require Egfr to be bound to an activating ligand-Egfr complex to recruit Grb2 to the plasma membrane. To simplify notation the binding site on the Egfr monomer is also called `SH2`, although Egfr does not harbor any SH2 domain.

The dissociation is implemented as unconditional rule to avoid the creation of irreversible states (e.g, dissociation of receptor dimers could protect from dissociation of Grb2). Accordingly, this rule is not implemented as energy rule, as energy rules always require the same conditions for association and dissociation rules. Nevertheless, we parameterize the rule using `kf` and `kD` parameters, although, no `phi` parameter is required here as the rule kinetics are never modulated through energy patterns.

```python
# EGFR-Grb2 binding
Parameter('bind_EGFR_GRB2_kf', 10.0)
Parameter('bind_EGFR_GRB2_kD', 0.01)
Expression('bind_EGFR_GRB2_kr', bind_EGFR_GRB2_kD*bind_EGFR_GRB2_kf)
Rule('transactivated_EGFR_dimers_bind_GRB2',
     EGFR(SH2=None, rtk=1) % EGFR(rtk=1, rtkf=ANY) + GRB2(SH2=None)
     >>
     EGFR(SH2=2, rtk=1) % EGFR(rtk=1, rtkf=ANY) % GRB2(SH2=2),
     bind_EGFR_GRB2_kf)

Rule('GRB2_dissociates_from_EGFR',
     EGFR(SH2=1) % GRB2(SH2=1) >> EGFR(SH2=None) + GRB2(SH2=None),
     bind_EGFR_GRB2_kr)

Annotation(transactivated_EGFR_dimers_bind_GRB2, 'http://identifiers.org/pubmed/16777603', 'isDescribedBy');
```

The recruitement of Grb2 to Egfr is not oly important for signal transduction, but also regulates the balance between slower, clathrin-independent and faster, clathrin-dependent (Grb2 dependent) receptor endocytosis   .

We describe the baseline Egfr expression (determined by clathrin-independent endocytosis) using basal Egfr expression and synthesis rules. Similar to binding rules, synthesis rules are parameterized equilibrium abundance parameter `[]_eq`, which regulates steady-state behaviour, and degradation rate `[]_kdeg`, which regulates the timescale of the rule. The equilibrium abundance can be modulated by the `[]_crispr` factor by up or down-regulating the synthesis rate `[]_ksyn`. To simplify specification of estimation boundaries for the `[...]_eq` parameter, we assume that the parameter is specified in [molecules/cell] and accordingly rescale the respective synthesis rate to [uM/h].

We implement clathrin-dependent endocytosis as three seperate rules for Egfr dimers with one Grb2 bound, with two Grb2 bound and Grb2 bound Egfr monomers. The `delete_molecules=True` parameter encodes that only Egfr molecules are degradede and all other molecules are remain as possibly fragmented comlexes. The formulation as three seperate rules ensures that Egfr endocytosis rate constant with respect to Grb2 stochiometry in the complex and that dissociation of Egfr homodimers does not protect from endocytosis.

```python
# basal synthesis + degradation
Parameter('EGFR_eq', 10000.0)
Parameter('synthesize_ERKphosphop_EGFR_kdeg', 10.0)
Parameter('EGFR_crispr', 1.0)
Expression('synthesize_ERKphosphop_EGFR_ksyn',
           1000000.0*EGFR_crispr*EGFR_eq*synthesize_ERKphosphop_EGFR_kdeg/(N_Avogadro*volume))

Rule('basal_synthesis_EGFR',
     None >> EGFR(SH2=None, rtk=None, rtkf=None),
     synthesize_ERKphosphop_EGFR_ksyn)
Rule('basal_degradation_EGFR',
     EGFR() >> None,
     synthesize_ERKphosphop_EGFR_kdeg, delete_molecules=True)

# Grb2 mediated degradation
Parameter('catalyze_GRB2_RTKSH2ANY_deg_kcat', 10.0)
Rule('GRB2_degrades_activeRTK_dimers',
     EGFR(SH2=ANY, rtk=1) % EGFR(SH2=None, rtk=1) >> None,
     catalyze_GRB2_RTKSH2ANY_deg_kcat, delete_molecules=True)
Rule('GRB2_degrades_doubleactiveRTK_dimers',
     EGFR(SH2=ANY, rtk=1) % EGFR(SH2=ANY, rtk=1) >> None,
     catalyze_GRB2_RTKSH2ANY_deg_kcat, delete_molecules=True)
Rule('GRB2_degrades_activeRTK_monomers',
     EGFR(SH2=ANY, rtk=None) >> None,
     catalyze_GRB2_RTKSH2ANY_deg_kcat, delete_molecules=True);
```

## SOS1 and Ras activation
Grb2 recruitement by Egfr also induces association of Sos1 to the plasma-membrane, where it can activate Ras by inducing nucleotide exchange . In the model we implement that Grb2 and Sos1 preform complexes in the cytoplasm  and that Sos1 is activated through association to the plasma-membrane, which is mediated recruitement of Grb2 by Egfr. Given that the training data does not include any data or perturbations at the Ras level, we implement a simple catalysis rule for activation of Ras, where SOS1 only binds GDP bound Ras and catalysis is implemented without homonucleotide exchange, allosteric feedback mechanisms or interactions with cdc25 . Similarly, we implement inactivation of Ras through GTPase activation proteins, such as NF1, as a simple linear reaction with constant rate.

As previous binding reactions, the rules are paramterized using `kf` and `kD` values. As for the Grb2-Egfr interaction, the binding domains on Sos1 and Grb2 are implemented as `SH3` sites on both molecules, simplifying the much more intricate binding dynamics . The translocation of Grb2 and Sos1 to the plasma membrane is not explicitely implemented in the model but implicitely encoded by requiring Grb2 and Egfr association for binding to Ras. To reduce correlations between parameter estimates, we implement the inactivation rate `[...]_gdp_kcat` of Ras as product of the activation rate `[...]_gtp_kcat` and a scaling factor `[...]_gdp_kcatr`. The requirement of `raf=None` in the inactivation rule will be explained in the Section on [Raf activation](#Raf-activation).

```python
# binding Grb2-Sos1
Parameter('bind_GRB2_SOS1_kf', 10.0)
Parameter('bind_GRB2_SOS1_kD', 0.01)
Parameter('bind_GRB2_SOS1_phi', 1.0)
Expression('bind_GRB2_SOS1_kr', bind_GRB2_SOS1_kD*bind_GRB2_SOS1_kf)
Expression('Gf_bind_GRB2_SOS1', log(bind_GRB2_SOS1_kD))
Expression('Ea0_bind_GRB2_SOS1',
           -bind_GRB2_SOS1_phi*log(bind_GRB2_SOS1_kD) - log(bind_GRB2_SOS1_kf))
EnergyPattern('ep_bind_GRB2_SOS1', GRB2(SH3=1) % SOS1(SH3=1), Gf_bind_GRB2_SOS1)
Rule('GRB2_and_SOS1_bind_and_dissociate',
     GRB2(SH3=None) + SOS1(SH3=None) | GRB2(SH3=1) % SOS1(SH3=1),
     bind_GRB2_SOS1_phi, Ea0_bind_GRB2_SOS1, energy=True)

Annotation(GRB2_and_SOS1_bind_and_dissociate, 'http://identifiers.org/pubmed/7893993', 'isDescribedBy')

# binding Sos1-Ras
Parameter('bind_SOS1_RAS_kf', 10.0)
Parameter('bind_SOS1_RAS_kD', 0.01)
Expression('bind_SOS1_RAS_kr', bind_SOS1_RAS_kD*bind_SOS1_RAS_kf)
Rule('RTK_and_GRB2_bound_SOS1_binds_RASgdp',
     GRB2(SH2=ANY, SH3=1) % SOS1(SH3=1, ras=None) + RAS(sos1=None, state='gdp')
     >>
     GRB2(SH2=ANY, SH3=1) % RAS(sos1=2, state='gdp') % SOS1(SH3=1, ras=2),
     bind_SOS1_RAS_kf)

Rule('SOS1_dissociates_from_RAS',
     RAS(sos1=1) % SOS1(ras=1) >> SOS1(ras=None) + RAS(sos1=None),
     bind_SOS1_RAS_kr)

Annotation(RTK_and_GRB2_bound_SOS1_binds_RASgdp, 'http://identifiers.org/pubmed/26565026', 'isDescribedBy')

# activation + inactivation Ras
Parameter('catalyze_SOS1_RAS_gtp_kcat', 0.01)
Parameter('catalyze_NF1_RAS_gdp_kcatr', 100.0)
Expression('catalyze_NF1_RAS_gdp_kcat', catalyze_NF1_RAS_gdp_kcatr*catalyze_SOS1_RAS_gtp_kcat)
Rule('SOS1_catalyzes_RAS_guanosine_exchange',
     RAS(sos1=1, state='gdp') % SOS1(ras=1) >> SOS1(ras=None) + RAS(sos1=None, state='gtp'),
     catalyze_SOS1_RAS_gtp_kcat)

Rule('RAS_hydrolisis_GTP',
     RAS(raf=None, state='gtp') >> RAS(raf=None, state='gdp'),
     catalyze_NF1_RAS_gdp_kcat);
```

# ERK signaling
The Erk submodule describes the interaction between the two Rafs c-Raf and B-Raf, Mek (which describes Mek1 and Mek2) and Erk (which describes Erk1 and Erk2). Again, we instantiate the respective monomers and provide uniprot IDs as monomer annotations.

```python
Monomer('BRAF', ['AA600', 'RBD', 'mek', 'raf', 'rafi'], {'AA600': ['E']})
Annotation(BRAF, 'http://identifiers.org/uniprot/P15056', 'is')

Monomer('CRAF', ['RBD', 'mek', 'raf', 'rafi'])
Annotation(CRAF, 'http://identifiers.org/uniprot/P04049', 'is')

Monomer('MEK', ['Dsite', 'meki', 'phospho', 'raf'], {'phospho': ['p', 'u']})
Annotation(MEK, 'http://identifiers.org/uniprot/Q02750', 'is')
Annotation(MEK, 'http://identifiers.org/uniprot/P36507', 'is')

Monomer('ERK', ['CD', 'phospho'], {'phospho': ['p', 'u']})
Annotation(ERK, 'http://identifiers.org/uniprot/P27361', 'is')
Annotation(ERK, 'http://identifiers.org/uniprot/P28482', 'is');
```

## Raf activation
GTP-bound Ras activates Raf by recruiting it to the membrane where it is phosphorylated and dimerizes. As understanding of the exact mechanisms of this activation process are still incomplete, we implemented a simplistic model of this process that captures essential features of the interaction. We implement a base dimerization rate between Raf monomers and only allow binding of Raf monomers to GTP-bound Ras. To implement the Ras-mediated dimerization of Raf, we implement an energy pattern for Ras-Raf tetramers modulates both binding affinities.

We implement both binding rules as energy rules, where we assume that affinities for B-Raf and C-Raf are the same for all interactions. As energy rules require symmetric forward and backward reactions, inactivation of Ras would protect Raf from unbinding from Ras. Accordingly, we add two additional inactivation reactions that dissociate Raf from Ras and required that no Raf was bound in the baseline inactivation.

```python
# Raf-Raf binding
Parameter('bind_RAF_RAF_kf', 10.0)
Parameter('bind_RAF_RAF_kD', 0.01)
Parameter('bind_RAF_RAF_phi', 1.0)
Expression('bind_BRAF_BRAF_kr', bind_RAF_RAF_kD*bind_RAF_RAF_kf)
Expression('Gf_bind_BRAF_BRAF', log(bind_RAF_RAF_kD))
Expression('Ea0_bind_BRAF_BRAF', -bind_RAF_RAF_phi*log(bind_RAF_RAF_kD) - log(bind_RAF_RAF_kf))
Expression('bind_BRAF_CRAF_kr', bind_RAF_RAF_kD*bind_RAF_RAF_kf)
Expression('Gf_bind_BRAF_CRAF', log(bind_RAF_RAF_kD))
Expression('Ea0_bind_BRAF_CRAF', -bind_RAF_RAF_phi*log(bind_RAF_RAF_kD) - log(bind_RAF_RAF_kf))
Expression('bind_CRAF_CRAF_kr', bind_RAF_RAF_kD*bind_RAF_RAF_kf)
Expression('Gf_bind_CRAF_CRAF', log(bind_RAF_RAF_kD))
Expression('Ea0_bind_CRAF_CRAF', -bind_RAF_RAF_phi*log(bind_RAF_RAF_kD) - log(bind_RAF_RAF_kf))
Rule('BRAF_and_BRAF_bind_and_dissociate',
     BRAF(raf=None) + BRAF(raf=None) | BRAF(raf=1) % BRAF(raf=1),
     bind_RAF_RAF_phi, Ea0_bind_BRAF_BRAF, energy=True)
Rule('BRAF_and_CRAF_bind_and_dissociate',
     BRAF(raf=None) + CRAF(raf=None) | BRAF(raf=1) % CRAF(raf=1),
     bind_RAF_RAF_phi, Ea0_bind_BRAF_CRAF, energy=True)
Rule('CRAF_and_CRAF_bind_and_dissociate',
     CRAF(raf=None) + CRAF(raf=None) | CRAF(raf=1) % CRAF(raf=1),
     bind_RAF_RAF_phi, Ea0_bind_CRAF_CRAF, energy=True)

EnergyPattern('ep_bind_BRAF_BRAF', BRAF(raf=1) % BRAF(raf=1), Gf_bind_BRAF_BRAF)
EnergyPattern('ep_bind_BRAF_CRAF', BRAF(raf=1) % CRAF(raf=1), Gf_bind_BRAF_CRAF)
EnergyPattern('ep_bind_CRAF_CRAF', CRAF(raf=1) % CRAF(raf=1), Gf_bind_CRAF_CRAF)

# Ras_gtp-RAF binding
Parameter('bind_RASstategtp_RAF_kf', 10.0)
Parameter('bind_RASstategtp_RAF_kD', 0.01)
Parameter('bind_RASstategtp_RAF_phi', 1.0)
Expression('bind_RASstategtp_BRAF_kr',
           bind_RASstategtp_RAF_kD*bind_RASstategtp_RAF_kf)
Expression('Gf_bind_RASstategtp_BRAF', log(bind_RASstategtp_RAF_kD))
Expression('Ea0_bind_RASstategtp_BRAF',
           -bind_RASstategtp_RAF_phi*log(bind_RASstategtp_RAF_kD) - log(bind_RASstategtp_RAF_kf))
Expression('bind_RASstategtp_CRAF_kr',
           bind_RASstategtp_RAF_kD*bind_RASstategtp_RAF_kf)
Expression('Gf_bind_RASstategtp_CRAF', log(bind_RASstategtp_RAF_kD))
Expression('Ea0_bind_RASstategtp_CRAF',
           -bind_RASstategtp_RAF_phi*log(bind_RASstategtp_RAF_kD) - log(bind_RASstategtp_RAF_kf))

Rule('RASgtp_and_BRAF_bind_and_dissociate',
     RAS(raf=None, state='gtp') + BRAF(RBD=None) | BRAF(RBD=1) % RAS(raf=1, state='gtp'),
     bind_RASstategtp_RAF_phi, Ea0_bind_RASstategtp_BRAF, energy=True)
Rule('RASgtp_and_CRAF_bind_and_dissociate',
     RAS(raf=None, state='gtp') + CRAF(RBD=None) | CRAF(RBD=1) % RAS(raf=1, state='gtp'),
     bind_RASstategtp_RAF_phi, Ea0_bind_RASstategtp_CRAF, energy=True)

EnergyPattern('ep_bind_RASstategtp_BRAF', BRAF(RBD=1) % RAS(raf=1, state='gtp'), Gf_bind_RASstategtp_BRAF)
EnergyPattern('ep_bind_RASstategtp_CRAF', CRAF(RBD=1) % RAS(raf=1, state='gtp'), Gf_bind_RASstategtp_CRAF)

Annotation(RASgtp_and_BRAF_bind_and_dissociate, 'http://identifiers.org/pubmed/7969158', 'isDescribedBy')
Annotation(RASgtp_and_CRAF_bind_and_dissociate, 'http://identifiers.org/pubmed/7969158', 'isDescribedBy')

# Ras-Raf inactivation
Rule('GTP_hydrolysis_dissociates_BRAF_from_RAS',
     BRAF(RBD=1) % RAS(raf=1, state='gtp') >> RAS(raf=None, state='gdp') + BRAF(RBD=None),
     catalyze_NF1_RAS_gdp_kcat)
Rule('GTP_hydrolysis_dissociates_CRAF_from_RAS',
     CRAF(RBD=1) % RAS(raf=1, state='gtp') >> RAS(raf=None, state='gdp') + CRAF(RBD=None),
     catalyze_NF1_RAS_gdp_kcat)

# Ras induces Raf dimerization
Parameter('ep_RAF_RAF_mod_RASstategtp_double_deltaG', 1000.0)
Expression('ep_RAF_RAF_mod_RASstategtp_double_Gf', log(ep_RAF_RAF_mod_RASstategtp_double_deltaG))
EnergyPattern('ep_BRAF_BRAF_mod_RAS_double',
              BRAF(RBD=2, raf=1) % BRAF(RBD=3, raf=1) % RAS(raf=2, state='gtp') % RAS(raf=3, state='gtp'),
              ep_RAF_RAF_mod_RASstategtp_double_Gf)
EnergyPattern('ep_BRAF_CRAF_mod_RAS_double',
              BRAF(RBD=2, raf=1) % CRAF(RBD=3, raf=1) % RAS(raf=2, state='gtp') % RAS(raf=3, state='gtp'),
              ep_RAF_RAF_mod_RASstategtp_double_Gf)
EnergyPattern('ep_CRAF_CRAF_mod_RAS_double',
              CRAF(RBD=2, raf=1) % CRAF(RBD=3, raf=1) % RAS(raf=2, state='gtp') % RAS(raf=3, state='gtp'),
              ep_RAF_RAF_mod_RASstategtp_double_Gf);
```

## Raf inhibition
Although Vemurafenib is an ATP-competitive inihibitor, it is known to exert allosteric effects. Here, we adapt a previously published model for these allosteric interactions between C-Raf, B-Raf and Vemurafenib . The model implements binding between Vemurafenib (`RAFi`) and C-Raf and B-Raf (`RAF`) as energy rule as well as two sets of energy patterns that modulate `RAF-RAF` and `RAF-RAFi` affinities in single and double drug bound RAF-RAF homo and heterodimers.

In this implementation, C-Raf and B-Raf both correspond to kinase monomers `R` of the Kholodenko Core Model (KCM). `K1` of the KCM is equivalent to the parameter `bind_RAF_RAF_kD`, which was introduced in the previous Section. `K2` in the KCM is equivalent to `bind_RAFi_RAF_kD`, `f` to `ep_RAF_RAF_mod_RAFi_single_deltaG` and `g` to `ep_RAF_RAF_mod_RAFi_double_deltaG`.

```python
# Binding RAFi-Raf
Parameter('bind_RAFi_RAF_kf', 10.0)
Parameter('bind_RAFi_RAF_kD', 0.01)
Parameter('bind_RAFi_RAF_phi', 1.0)
Expression('bind_BRAFi_RAF_kr', bind_RAFi_RAF_kD*bind_RAFi_RAF_kf)
Expression('Gf_bind_BRAFi_RAF', log(bind_RAFi_RAF_kD))
Expression('Ea0_bind_BRAFi_RAF', -bind_RAFi_RAF_phi*log(bind_RAFi_RAF_kD) - log(bind_RAFi_RAF_kf))
Expression('bind_CRAFi_RAF_kr', bind_RAFi_RAF_kD*bind_RAFi_RAF_kf)
Expression('Gf_bind_CRAFi_RAF', log(bind_RAFi_RAF_kD))
Expression('Ea0_bind_CRAFi_RAF', -bind_RAFi_RAF_phi*log(bind_RAFi_RAF_kD) - log(bind_RAFi_RAF_kf))
Rule('RAFi_and_BRAF_bind_and_dissociate',
     RAFi(raf=None) + BRAF(rafi=None) | BRAF(rafi=1) % RAFi(raf=1),
     bind_RAFi_RAF_phi, Ea0_bind_BRAFi_RAF, energy=True)
Rule('RAFi_and_CRAF_bind_and_dissociate',
     RAFi(raf=None) + CRAF(rafi=None) | CRAF(rafi=1) % RAFi(raf=1),
     bind_RAFi_RAF_phi, Ea0_bind_CRAFi_RAF, energy=True)

EnergyPattern('ep_bind_BRAFi_RAF', BRAF(rafi=1) % RAFi(raf=1), Gf_bind_BRAFi_RAF)
EnergyPattern('ep_bind_CRAFi_RAF', CRAF(rafi=1) % RAFi(raf=1), Gf_bind_CRAFi_RAF)

# RAFi mediated affinity modulation
Parameter('ep_RAF_RAF_mod_RAFi_single_deltaG', 0.001)
Parameter('ep_RAF_RAF_mod_RAFi_double_deltaG', 1000.0)
Expression('ep_RAF_RAF_mod_RAFi_single_Gf', log(ep_RAF_RAF_mod_RAFi_single_deltaG))
Expression('ep_RAF_RAF_mod_RAFi_double_Gf',
           log(ep_RAF_RAF_mod_RAFi_double_deltaG) + log(ep_RAF_RAF_mod_RAFi_single_deltaG))

## Single Vemurafenib bound Raf dimers
EnergyPattern('ep_BRAF_BRAF_mod_RAFi_single',
              BRAF(raf=1, rafi=2) % BRAF(raf=1, rafi=None) % RAFi(raf=2),
              ep_RAF_RAF_mod_RAFi_single_Gf)
EnergyPattern('ep_BRAF_CRAF_mod_RAFi_single',
              BRAF(raf=1, rafi=None) % CRAF(raf=1, rafi=2) % RAFi(raf=2),
              ep_RAF_RAF_mod_RAFi_single_Gf)
EnergyPattern('ep_CRAF_BRAF_mod_RAFi_single',
              BRAF(raf=1, rafi=2) % CRAF(raf=1, rafi=None) % RAFi(raf=2),
              ep_RAF_RAF_mod_RAFi_single_Gf)
EnergyPattern('ep_CRAF_CRAF_mod_RAFi_single',
              CRAF(raf=1, rafi=2) % CRAF(raf=1, rafi=None) % RAFi(raf=2),
              ep_RAF_RAF_mod_RAFi_single_Gf)

## Double Vemurafenib bound Raf dimers
EnergyPattern('ep_BRAF_BRAF_mod_RAFi_double',
              BRAF(raf=1, rafi=2) % BRAF(raf=1, rafi=3) % RAFi(raf=2) % RAFi(raf=3),
              ep_RAF_RAF_mod_RAFi_double_Gf)
EnergyPattern('ep_BRAF_CRAF_mod_RAFi_double',
              BRAF(raf=1, rafi=2) % CRAF(raf=1, rafi=3) % RAFi(raf=2) % RAFi(raf=3),
              ep_RAF_RAF_mod_RAFi_double_Gf)
EnergyPattern('ep_CRAF_CRAF_mod_RAFi_double',
              CRAF(raf=1, rafi=2) % CRAF(raf=1, rafi=3) % RAFi(raf=2) % RAFi(raf=3),
              ep_RAF_RAF_mod_RAFi_double_Gf);
```

## Mek activation

Raf activation of Mek is mediated through phosphorylation at to S218 and S222 . As neither mass-spectrometry nor immunofluorescence measurements resolved individual phosphorylations on these sites, we decided to combine both into a single `phospho` site in the model. We implement the respective Mek phosphorylation as two step catalytic process including binding and phosphorylation. In agreement with previous studies, we assume that binding of Raf to Mek does not require prior activation of Raf, but is specific for unphosphorylated Mek .

```python
# Raf-Mek binding
Parameter('bind_RAF_MEKphosphou_kf', 10.0)
Parameter('bind_RAF_MEKphosphou_kD', 0.01)
Parameter('bind_RAF_MEKphosphou_phi', 1.0)
Expression('bind_BRAF_MEKphosphou_kr', bind_RAF_MEKphosphou_kD*bind_RAF_MEKphosphou_kf)
Expression('Gf_bind_BRAF_MEKphosphou', log(bind_RAF_MEKphosphou_kD))
Expression('Ea0_bind_BRAF_MEKphosphou',
           -bind_RAF_MEKphosphou_phi*log(bind_RAF_MEKphosphou_kD) - log(bind_RAF_MEKphosphou_kf))
Expression('bind_CRAF_MEKphosphou_kr', bind_RAF_MEKphosphou_kD*bind_RAF_MEKphosphou_kf)
Expression('Gf_bind_CRAF_MEKphosphou', log(bind_RAF_MEKphosphou_kD))
Expression('Ea0_bind_CRAF_MEKphosphou',
           -bind_RAF_MEKphosphou_phi*log(bind_RAF_MEKphosphou_kD) - log(bind_RAF_MEKphosphou_kf))

Rule('BRAF_and_uMEK_bind_and_dissociate',
     BRAF(mek=None) + MEK(phospho='u', raf=None) | BRAF(mek=1) % MEK(phospho='u', raf=1),
     bind_RAF_MEKphosphou_phi, Ea0_bind_BRAF_MEKphosphou, energy=True)
Rule('CRAF_and_uMEK_bind_and_dissociate',
     CRAF(mek=None) + MEK(phospho='u', raf=None) | CRAF(mek=1) % MEK(phospho='u', raf=1),
     bind_RAF_MEKphosphou_phi, Ea0_bind_CRAF_MEKphosphou, energy=True)

EnergyPattern('ep_bind_BRAF_MEKphosphou', BRAF(mek=1) % MEK(phospho='u', raf=1), Gf_bind_BRAF_MEKphosphou)
EnergyPattern('ep_bind_CRAF_MEKphosphou', CRAF(mek=1) % MEK(phospho='u', raf=1), Gf_bind_CRAF_MEKphosphou)

Annotation(BRAF_and_uMEK_bind_and_dissociate, 'http://identifiers.org/pubmed/25155755', 'isDescribedBy')
Annotation(CRAF_and_uMEK_bind_and_dissociate, 'http://identifiers.org/pubmed/25155755', 'isDescribedBy');
```

For the activation step, we implement a physiological activation through Raf-Ras tetramers, which is implemented in four rules that account for all possible Raf homo- and heterodimer configurations, as well as oncogenic activation through B-Raf V600E mutation, which is implemented in four rules that account for B-Raf V600E molecules complexes that do not signal as dimers. In both cases, we require that the activating Raf molecule is not bound by an inhibitor and assume that kinetic rates are the same in both cases.

Experimental data presented in this study indicated phospho-Mek accumulation for physiological signaling but not for oncogenic signaling, which is consistent with previous studies  . we required Mek not to be bound by an inhibitor. The exact molecular mechanisms for this difference between dimer mediated and V600E mediated are unknown. The exact molecular mechanisms for this difference between dimer mediated and V600E mediated are unknown but previous studies have suggested the involvement of drug-induced effects on RAF:MEK stabiliziation, MEK dimerization or involvement of KSR scaffolding proteins  .

```python
# phosphorylation
Parameter('catalyze_RAFrafiNone_MEK_p_kcat', 10.0)
## Raf-dimer mediated
Rule('BRAF_BRAF_phosphorylates_MEK',
     BRAF(RBD=ANY, mek=1, raf=2, rafi=None) % BRAF(RBD=ANY, raf=2) % MEK(phospho='u', raf=1)
     >>
     MEK(phospho='p', raf=None) + BRAF(RBD=ANY, mek=None, raf=2, rafi=None) % BRAF(RBD=ANY, raf=2),
     catalyze_RAFrafiNone_MEK_p_kcat)
Rule('BRAF_CRAF_phosphorylates_MEK',
     BRAF(RBD=ANY, mek=1, raf=2, rafi=None) % CRAF(RBD=ANY, raf=2) % MEK(phospho='u', raf=1)
     >>
     MEK(phospho='p', raf=None) + BRAF(RBD=ANY, mek=None, raf=2, rafi=None) % CRAF(RBD=ANY, raf=2),
     catalyze_RAFrafiNone_MEK_p_kcat)
Rule('CRAF_BRAF_phosphorylates_MEK',
     BRAF(RBD=ANY, raf=2) % CRAF(RBD=ANY, mek=1, raf=2, rafi=None) % MEK(phospho='u', raf=1)
     >>
     MEK(phospho='p', raf=None) + BRAF(RBD=ANY, raf=2) % CRAF(RBD=ANY, mek=None, raf=2, rafi=None),
     catalyze_RAFrafiNone_MEK_p_kcat)
Rule('CRAF_CRAF_phosphorylates_MEK',
     CRAF(RBD=ANY, mek=1, raf=2, rafi=None) % CRAF(RBD=ANY, raf=2) % MEK(phospho='u', raf=1)
     >>
     MEK(phospho='p', raf=None) + CRAF(RBD=ANY, mek=None, raf=2, rafi=None) % CRAF(RBD=ANY, raf=2),
     catalyze_RAFrafiNone_MEK_p_kcat)

## B-Raf V600E mediated
Rule('BRAFV600E_phosphorylates_MEK_1',
     BRAF(AA600='E', mek=1, raf=None, rafi=None) % MEK(meki=None, phospho='u', raf=1)
     >>
     MEK(meki=None, phospho='p', raf=None) + BRAF(AA600='E', mek=None, raf=None, rafi=None),
     catalyze_RAFrafiNone_MEK_p_kcat)
Rule('BRAFV600E_phosphorylates_MEK_2',
     BRAF(AA600='E', RBD=None, mek=1, raf=ANY, rafi=None) % MEK(meki=None, phospho='u', raf=1)
     >>
     MEK(meki=None, phospho='p', raf=None) + BRAF(AA600='E', RBD=None, mek=None, raf=ANY, rafi=None),
     catalyze_RAFrafiNone_MEK_p_kcat)
Rule('BRAFV600E_phosphorylates_MEK_3',
     BRAF(AA600='E', RBD=ANY, mek=1, raf=2, rafi=None) % BRAF(RBD=None, raf=2) % MEK(meki=None, phospho='u', raf=1)
     >>
     MEK(meki=None, phospho='p', raf=None) + BRAF(AA600='E', RBD=ANY, mek=None, raf=2, rafi=None) % BRAF(RBD=None, raf=2),
     catalyze_RAFrafiNone_MEK_p_kcat)
Rule('BRAFV600E_phosphorylates_MEK_4',
     BRAF(AA600='E', RBD=ANY, mek=1, raf=2, rafi=None) % CRAF(RBD=None, raf=2) % MEK(meki=None, phospho='u', raf=1)
     >>
     MEK(meki=None, phospho='p', raf=None) + BRAF(AA600='E', RBD=ANY, mek=None, raf=2, rafi=None) % CRAF(RBD=None, raf=2),
     catalyze_RAFrafiNone_MEK_p_kcat);
```

PP2A and PPP1 have been suggested as putative phosphatases of Mek1, but exact regulatory mechanisms remain poorly understood. Accordingly we implement MEK dephosphorylation as conversion rule with constant rate.

```python
# dephosphorylation
Parameter('catalyze_PP2A_MEK_u_kcatr', 1.0)
Expression('catalyze_PP2A_MEK_u_kcat', catalyze_PP2A_MEK_u_kcatr*catalyze_RAFrafiNone_MEK_p_kcat)
Rule('MEK_is_dephosphorylated', MEK(phospho='p') >> MEK(phospho='u'), catalyze_PP2A_MEK_u_kcat);
```

## Mek inhibition
Cobimetinib is an allosteric (type III) inhibitor that potently and selectively inhibits Erk activation when binding Mek. Recent studies reported a distinct potency of allosteric Mek inhibitor for Ras-GTP versus B-Raf V600E driven Mek signaling. One component of this difference in potency is the accumulation of phosphorylated Mek in Ras-GTP driven signaling, but not in B-Raf V600E driven signaling that was already described in the previous Section. A second component is the differential affinity of allosteric Mek inihibitors towards phosphorylated MEK  . Accordingly, we implement binding as energy rule and add an energy pattern that allows Mek phosphorylation to modulation the respective affinity.

```python
# MEKi binding
Parameter('bind_MEKi_MEK_kf', 10.0)
Parameter('bind_MEKi_MEK_kD', 0.01)
Parameter('bind_MEKi_MEK_phi', 1.0)
Expression('bind_MEKi_MEK_kr', bind_MEKi_MEK_kD*bind_MEKi_MEK_kf)
Expression('Gf_bind_MEKi_MEK', log(bind_MEKi_MEK_kD))
Expression('Ea0_bind_MEKi_MEK', -bind_MEKi_MEK_phi*log(bind_MEKi_MEK_kD) - log(bind_MEKi_MEK_kf))
Rule('MEKi_and_MEK_bind_and_dissociate',
     MEKi(mek=None) + MEK(meki=None) | MEK(meki=1) % MEKi(mek=1),
     bind_MEKi_MEK_phi, Ea0_bind_MEKi_MEK, energy=True)

EnergyPattern('ep_bind_MEKi_MEK', MEK(meki=1) % MEKi(mek=1), Gf_bind_MEKi_MEK)

# Modulated pMEK affinity
Parameter('ep_MEKphosphop_MEKi_deltaG', 100.0)
Expression('ep_MEKphosphop_MEKi_Gf', log(ep_MEKphosphop_MEKi_deltaG))
EnergyPattern('ep_MEKphosphop_MEKi_single', MEK(meki=1, phospho='p') % MEKi(mek=1), ep_MEKphosphop_MEKi_Gf);
```

## Erk activation
Mek phosphorylates Erk1 at T202 and T185 and Erk2 at Y204 and Y187. As for Mek phosphorylation sites, neither mass-spectrometry nor immunofluorescence measurements resolved individual phosphorylations, both sites were combined into a single `phospho` site in the model.

As for the Mek phosphorylation we implement Erk phosphorylation as two step catalysis reaction. The binding between Mek and Erk is mediated through interaction of the MAPK-Docking site on the N-Terminus of MEK (`Dsite` site in the model)   with the common docking domain on Erk (`CD` site in the model)  . In combination with Mek's nuclear export sequence, this interaction allows Mek to act as a cytosolic anchor for inactive Erk , which suggests that this interaction is not dependent of Mek phosphorylation. After Mek activation, Erk releases from Mek and shuttles in and out of the nucleus, which suggests that affinity between Mek and Erk depends on Erk phosphorylation   . In the model, we implement the dependence on Erk phosphorylation as binary interaction, where Mek only binds unphosphorylated Erk. Nuclear shuttling of Mek and Erk was not implemented in the model.

```python
# Mek-Erk binding
Parameter('bind_MEK_ERKphosphou_kf', 10.0)
Parameter('bind_MEK_ERKphosphou_kD', 0.01)
Expression('bind_MEK_ERKphosphou_kr', bind_MEK_ERKphosphou_kD*bind_MEK_ERKphosphou_kf)
Rule('MEK_binds_uERK',
     MEK(Dsite=None) + ERK(CD=None, phospho='u') >> ERK(CD=1, phospho='u') % MEK(Dsite=1),
     bind_MEK_ERKphosphou_kf)
Rule('MEK_dissociates_from_ERK',
     ERK(CD=1) % MEK(Dsite=1) >> MEK(Dsite=None) + ERK(CD=None),
     bind_MEK_ERKphosphou_kr)

Annotation(MEK_dissociates_from_ERK, 'http://identifiers.org/pubmed/10567369', 'isDescribedBy')
Annotation(MEK_dissociates_from_ERK, 'http://identifiers.org/pubmed/10655591', 'isDescribedBy')
Annotation(MEK_dissociates_from_ERK, 'http://identifiers.org/pubmed/11157753', 'isDescribedBy')
Annotation(MEK_dissociates_from_ERK, 'http://identifiers.org/pubmed/15979847', 'isDescribedBy')

# Erk phosphorylation
Parameter('catalyze_MEKmekiNone_phosphop_ERK_p_kcat', 10.0)
Rule('pMEK_phosphorylates_ERK',
     ERK(CD=1, phospho='u') % MEK(Dsite=1, meki=None, phospho='p')
     >>
     ERK(CD=None, phospho='p') + MEK(Dsite=None, meki=None, phospho='p'),
     catalyze_MEKmekiNone_phosphop_ERK_p_kcat)

Annotation(pMEK_phosphorylates_ERK, 'http://identifiers.org/pubmed/19406201', 'isDescribedBy');
```

# Feedback mechanisms
Activation of Erk through phosphorylation induces a variety of different negative feedback mechanisms that inhibit signal transduction in Egfr and Erk pathways  . In this module we specifically implement four of these mechanisms that, specifically the increase in Spry, Egfr and Dusp expression levels as well as phosphorylation of Sos1. For this purpose we introduce two new monomers, namely Dusp (which describes Dusp4 and Dusp6) and Spry (which describes Spry2 and Spry4).

Some of the negative feedback mechanisms that were described in previous studies are omitted in this model, including Erk negative feedback phosphorylations on Mek, and Raf. As the timescale of phosphorylation reactions is in the order of seconds, a high inhibitory potency of these feedback mechanisms should give rise to phospho-Erk pulses on a timescale of seconds. Yet the experimentally observed timescale of phospho-Erk pulses was in the order of minutes, which suggests that it is unlikely that phosphorylations feedbacks are the primary determinant of the pulse shape. This hypothesis is backed by previous studies on phospho-turnover in Egfr mediated signaling . Accordingly we only expect a subtle effect of the negative feedback mechanisms, which would be difficult to be resolve in the model, given that no experimental data on respective phosphorylation sites or Ras-GTP levels were available.

```python
Monomer('DUSP', ['erk'])
Annotation(DUSP, 'http://identifiers.org/uniprot/Q13115', 'is')
Annotation(DUSP, 'http://identifiers.org/uniprot/Q16828', 'is')

Monomer('SPRY', ['SH3'])
Annotation(SPRY, 'http://identifiers.org/uniprot/O43597', 'is')
Annotation(SPRY, 'http://identifiers.org/uniprot/Q9C004', 'is');
```

## Spry expression
The proteomic data collected in this study confirmed previous findings that phosphorylation of Erk enhances expression of Spry. Increase in Spry expression inhibits Egfr signaling and is mediated by regulation of downstream transcription factors . Specifically, Spry competes with Sos1 for the binding of Grb2  . Accordingly higher levels of Spry can antagonize Sos1 signaling by sequestering Grb2 in Spry:Grb2 complexes.

Here, we implement basal synthesis and degradation reactions for Spry, which are parameterised with `[...]_eq` and `[...]_kdeg` parameters as done for Egfr expression in the Section on [Grb2 activation](#GRB2-Activation). On top of this basal expression, we implement an additional expression rule for which the expression rate linearly depends on the abundance of phospho-Erk. To avoid correlations of parameter estimates, we normalize this expression rate with the basal Spry degradation rate. The binding between Spry and Grb2 is parametrized using `[...]_kD` and `[...]_kf` parameters as previously described. Competitive binding between Spry and Sos1 for Grb2 is implemented by specifying the same binding site on Grb2 (`SH3`) for the binding of both reactions.

```python
# basal expression and degradation
Parameter('SPRY_eq', 10000.0)
Parameter('synthesize_ERKphosphop_SPRY_kdeg', 10.0)
Expression('synthesize_ERKphosphop_SPRY_ksyn',
           1000000.0*SPRY_eq*synthesize_ERKphosphop_SPRY_kdeg/(N_Avogadro*volume))

Rule('basal_synthesis_SPRY',
     None >> SPRY(SH3=None), synthesize_ERKphosphop_SPRY_ksyn)
Rule('basal_degradation_SPRY',
     SPRY() >> None, synthesize_ERKphosphop_SPRY_kdeg, delete_molecules=True)

# Erk mediated expression
Parameter('synthesize_ERKphosphop_SPRY_ERK_gexpslope', 1000.0)
Expression('synthesize_ERKphosphop_SPRY_kmodslope',
           synthesize_ERKphosphop_SPRY_ERK_gexpslope*synthesize_ERKphosphop_SPRY_kdeg)
Rule('ERK_synthesizes_SPRY',
     None + ERK(phospho='p') >> SPRY(SH3=None) + ERK(phospho='p'),
     synthesize_ERKphosphop_SPRY_kmodslope)

# Spry-Grb2 binding
Parameter('bind_SPRY_GRB2_kf', 10.0)
Parameter('bind_SPRY_GRB2_kD', 0.01)
Expression('bind_SPRY_GRB2_kr', bind_SPRY_GRB2_kD*bind_SPRY_GRB2_kf)
Rule('SPRY_binds_GRB2',
     SPRY(SH3=None) + GRB2(SH3=None) >> GRB2(SH3=1) % SPRY(SH3=1),
     bind_SPRY_GRB2_kf)
Rule('SPRY_dissociates_from_GRB2',
     GRB2(SH3=1) % SPRY(SH3=1) >> SPRY(SH3=None) + GRB2(SH3=None),
     bind_SPRY_GRB2_kr);
```

## Sos1 phosphorylation
Phospho-proteomic data collected in this study confirmed previous findings that phosphorylation of Erk enhances phosphorylation of Sos1 at S1134. Phosphorylation of Sos1 inhibits Egfr signaling and is mediated through upregulation of Rsk expression and phosphorylation . Specifically it creates a docking site for 14-3-3 on Sos1 and thereby sequestering it from Grb2 .

We do not include Rsk or 14-3-3 in the model as absolute abundances were measured for neither of proteins. Instead we decided to implement basal phosphorylation and dephosphorylation rules as well as an phospho-Erk dependent phosphorylation rule for which the phosphorylation linearly depends on Erk phosphorylation. The baseline rules were implemented using constant rate `[...]_kbase` and `[...]_kcat` respectively. To avoid parameter correlations, the phospho-Erk dependent phosphorylation rate was normalized by the baseline phosphorylation rate.

```python
# base phosphorylation rate
Parameter('catalyze_ERKphosphop_SOS1_pS1134_kbase', 0.0)
Parameter('catalyze_ERKphosphop_SOS1_pS1134_kcat', 10.0)
Parameter('catalyze_phosphatase_SOS1_uS1134_kcatr', 1.0)
Expression('catalyze_phosphatase_SOS1_uS1134_kcat',
           catalyze_ERKphosphop_SOS1_pS1134_kcat*catalyze_phosphatase_SOS1_uS1134_kcatr)

Rule('SOS1_is_phosphorylated',
     SOS1(S1134='u') >> SOS1(S1134='p'),
     catalyze_ERKphosphop_SOS1_pS1134_kbase)

Rule('SOS1_is_dephosphorylated',
     SOS1(S1134='p') >> SOS1(S1134='u'),
     catalyze_phosphatase_SOS1_uS1134_kcat)

Rule('pERK_phosphorylates_SOS1',
     ERK(phospho='p') + SOS1(S1134='u') >> ERK(phospho='p') + SOS1(S1134='p'),
     catalyze_ERKphosphop_SOS1_pS1134_kcat)

# SPRY modulates binding between GRB2 and SOS1
Parameter('ep_SOS1S1134p_GRB2_deltaG', 100.0)
Expression('ep_SOS1S1134p_GRB2_Gf', log(ep_SOS1S1134p_GRB2_deltaG))
EnergyPattern('ep_SOS1S1134p_GRB2_single', GRB2(SH3=1) % SOS1(S1134='p', SH3=1), ep_SOS1S1134p_GRB2_Gf);
```

As the binding between as Grb2 and Sos1 was implemented as energy rule, we use an energy pattern to implement the modulated affinity of Grb2 to Sos1.

## Egfr expression
Proteomic data collected in this study confirmed previous findings that phosphorylation of Erk increases Egfr expression. Increased Egfr expression promotes Egfr signaling and is mediated through regulation of downstream transcription factors .

As baseline Egfr expression and degradation rules are already implemented in the Section on [GRB2 activation](#GRB2-Activation), we here simply add an expression rule for which the rate linearly depends on phospho-Erk. As for Spry expression, we normalized the phospho-Erk dependent expression rate by the baseline degradation rate.

```python
Parameter('synthesize_ERKphosphop_EGFR_ERK_gexpslope', 1000.0)
Expression('synthesize_ERKphosphop_EGFR_kmodslope',
           synthesize_ERKphosphop_EGFR_ERK_gexpslope*synthesize_ERKphosphop_EGFR_kdeg)

Rule('ERK_synthesizes_EGFR',
     None + ERK(phospho='p') >> EGFR(SH2=None, rtk=None, rtkf=None) + ERK(phospho='p'),
     synthesize_ERKphosphop_EGFR_kmodslope);
```

## Dusp expression
The proteomic data collected in this study confirmed previous findings that phosphorylation of Erk enhances expression of Dusp. Increase in Dusp expression inhibits Erk signaling and is mediated by regulation of downstream transcription factors . Specifically, Dusp are phosphatases that dephosphorylated Erk on the same residues that are phosphorylated by Mek. In the proteomic data we individually measured Dusp4 and Dusp6 abundances. Dusp4 has primarily nuclear localisation , while Dusp6 has primarily cytosolic localisation . As we did not implement shuttling of Erk between the nucleus and cytosol, we lumped Dusp4 and Dusp6 species.

We implement the dephosphorylation as two-step catalytic process. As both Dusps interact with Erk via the common docking domain that also mediates interaction between Mek to Erk , the binding was implemented using the same site on the Erk (`CD`) monomer. This could lead to competitive binding between Mek and Erk, yet as Dusp, in contrast to Mek, seem to preferentially bind the phosphorylated form of Erk , we constrained binding of Dusp to Erk on Erk phosphorylation, which makes the two binding reactions independent.

```python
# expression
Parameter('DUSP_eq', 10000.0)
Parameter('synthesize_ERKphosphop_DUSP_ERK_gexpslope', 1000.0)
Parameter('synthesize_ERKphosphop_DUSP_kdeg', 10.0)
Expression('synthesize_ERKphosphop_DUSP_ksyn',
           1000000.0*DUSP_eq*synthesize_ERKphosphop_DUSP_kdeg/(N_Avogadro*volume))
Expression('synthesize_ERKphosphop_DUSP_kmodslope',
           synthesize_ERKphosphop_DUSP_ERK_gexpslope*synthesize_ERKphosphop_DUSP_kdeg)

Rule('basal_synthesis_DUSP',
     None >> DUSP(erk=None), synthesize_ERKphosphop_DUSP_ksyn)
Rule('basal_degradation_DUSP',
     DUSP() >> None, synthesize_ERKphosphop_DUSP_kdeg, delete_molecules=True)
Rule('ERK_synthesizes_DUSP',
     None + ERK(phospho='p') >> DUSP(erk=None) + ERK(phospho='p'), synthesize_ERKphosphop_DUSP_kmodslope)

# Dusp-Erk binding
Parameter('bind_DUSP_ERKphosphop_kf', 10.0)
Parameter('bind_DUSP_ERKphosphop_kD', 0.01)
Expression('bind_DUSP_ERKphosphop_kr', bind_DUSP_ERKphosphop_kD*bind_DUSP_ERKphosphop_kf)
Rule('DUSP_binds_pERK',
     DUSP(erk=None) + ERK(CD=None, phospho='p') >> DUSP(erk=1) % ERK(CD=1, phospho='p'),
     bind_DUSP_ERKphosphop_kf)
Rule('DUSP_dissociates_from_ERK',
     DUSP(erk=1) % ERK(CD=1) >> DUSP(erk=None) + ERK(CD=None),
     bind_DUSP_ERKphosphop_kr)

Annotation(DUSP_dissociates_from_ERK, 'http://identifiers.org/pubmed/10655591', 'isDescribedBy')
Annotation(DUSP_dissociates_from_ERK, 'http://identifiers.org/pubmed/11157753', 'isDescribedBy')

# Erk dephosphorylation
Parameter('catalyze_DUSP_ERK_u_kcatr', 1.0)
Expression('catalyze_DUSP_ERK_u_kcat', catalyze_DUSP_ERK_u_kcatr*catalyze_MEKmekiNone_phosphop_ERK_p_kcat)
Rule('DUSP_dephosphorylates_ERK',
     DUSP(erk=1) % ERK(CD=1, phospho='p') >> ERK(CD=None, phospho='u') + DUSP(erk=None),
     catalyze_DUSP_ERK_u_kcat);
```

# Initialization
For all monomers for which we did not implement expression and basal degradation rules we implemented initial abundances that were assumed to be constant in all experiments. To simplify specification of estimation boundaries for these parameters, these parameters were assumed to be specified in [molecules/cell] and then accordingly transformed to yield model initializations in [uM].

```python
Parameter('BRAF_0', 0.0)
Parameter('CRAF_0', 0.0)
Parameter('RAS_0', 0.0)
Parameter('MEK_0', 0.0)
Parameter('ERK_0', 0.0)
Parameter('GRB2_0', 0.0)
Parameter('SOS1_0', 0.0)

Expression('initBRAF', 1000000.0*BRAF_0/(N_Avogadro*volume))
Expression('initCRAF', 1000000.0*CRAF_0/(N_Avogadro*volume))
Expression('initRAS', 1000000.0*RAS_0/(N_Avogadro*volume))
Expression('initMEK', 1000000.0*MEK_0/(N_Avogadro*volume))
Expression('initERK', 1000000.0*ERK_0/(N_Avogadro*volume))
Expression('initGRB2', 1000000.0*GRB2_0/(N_Avogadro*volume))
Expression('initSOS1', 1000000.0*SOS1_0/(N_Avogadro*volume))

Initial(BRAF(AA600='E', RBD=None, mek=None, raf=None, rafi=None), initBRAF)
Initial(CRAF(RBD=None, mek=None, raf=None, rafi=None), initCRAF)
Initial(RAS(raf=None, sos1=None, state='gdp'), initRAS)
Initial(MEK(Dsite=None, meki=None, phospho='u', raf=None), initMEK)
Initial(ERK(CD=None, phospho='u'), initERK)
Initial(GRB2(SH2=None, SH3=None), initGRB2)
Initial(SOS1(S1134='u', SH3=None, ras=None), initSOS1);
```

# Link to Experimental Data
To calibrate the model to experimental data we create a series of expression that account for normalizations in data generation. For absolute proteomic measurements we create expressions `t[...]_obs` that convert the unit of model simulations from [uM] to [molecules/cell]. For phospho-proteomic measurements we create expressions `p[...]_obs` that are equal to phospho-abundances normalized by total abundances. For immuno-fluorescence measurements we created expressions `p[...]_IF_obs` that were first normalized by total abundances and then scaled by a mulitplicative factor `[...]_IF_scale` and an additive factor `[...]_IF_offset`. For species that describe multiple protein isoforms, absolute proteomic data was averaged over both isoforms.

```python
# proteomic measurements
Observable('tBRAF', BRAF())
Observable('tCRAF', CRAF())
Observable('tRAS', RAS())
Observable('tMEK', MEK())
Observable('tERK', ERK())
Observable('tDUSP', DUSP())
Observable('tEGFR', EGFR())
Observable('tGRB2', GRB2())
Observable('tSPRY', SPRY())
Observable('tSOS1', SOS1())
Expression('tBRAF_obs', 1.0e-6*tBRAF*N_Avogadro*volume)
Expression('tCRAF_obs', 1.0e-6*tCRAF*N_Avogadro*volume)
Expression('tRAS_obs', 1.0e-6*tRAS*N_Avogadro*volume)
Expression('tMEK_obs', 1.0e-6*tMEK*N_Avogadro*volume)
Expression('tDUSP_obs', 1.0e-6*tDUSP*N_Avogadro*volume)
Expression('tEGFR_obs', 1.0e-6*tEGFR*N_Avogadro*volume)
Expression('tGRB2_obs', 1.0e-6*tGRB2*N_Avogadro*volume)
Expression('tSPRY_obs', 1.0e-6*tSPRY*N_Avogadro*volume)
Expression('tSOS1_obs', 1.0e-6*tSOS1*N_Avogadro*volume)
Expression('tERK_obs', 1.0e-6*tERK*N_Avogadro*volume)

# phospho-proteomic measurements
Observable('pERK', ERK(phospho='p'))
Observable('pS1134SOS1', SOS1(S1134='p'))
Expression('pERK_obs', pERK/tERK)
Expression('pS1134SOS1_obs', pS1134SOS1/tSOS1)

# immunofluorescence measurements
Parameter('pMEK_IF_scale', 1.0)
Parameter('pMEK_IF_offset', 0.1)
Parameter('pERK_IF_scale', 1.0)
Parameter('pERK_IF_offset', 0.1)
Observable('pMEK', MEK(phospho='p'))
Expression('pMEK_obs', pMEK/tMEK)
Expression('pMEK_IF_obs', pMEK_obs*pMEK_IF_scale + pMEK_IF_offset)
Expression('pERK_IF_obs', pERK_obs*pERK_IF_scale + pERK_IF_offset);
```

# Observables

We add observables that are used to extract the biological activity of proteins of interest.

```python
#total EGF
Observable('tEGF', EGF())
#active SOS1 (SOS1 is bound to a GRB2 that is bound to EGFR)
Observable('SOS1_active', GRB2(SH2=ANY, SH3=1) % SOS1(SH3=1))
#signaling competent SOS1 (SOS1 is bond to GRB2)
Observable('SOS1_signaling_competent', GRB2(SH3=1) % SOS1(SH3=1))
#signaling inhibited GBR2 (bound by SPRY)
Observable('GRB2_bound_by_SPRY', GRB2(SH3=1) % SPRY(SH3=1))
#unphosphorylated SOS1
Observable('uS1134SOS1', SOS1(S1134='u'))
#Ras-GDP and -GTP
Observable('RAS_gtp', RAS(state='gtp'))
Observable('RAS_gdp', RAS(state='gdp'))
#active RAF monomers that are not bound by RAFi
Observable('active_RAF_monomers', BRAF(AA600='E', raf=None, rafi=None) +  BRAF(AA600='E', RBD=None, raf=ANY, rafi=None) + BRAF(AA600='E', RBD=ANY, raf=2, rafi=None) % BRAF(RBD=None, raf=2) + BRAF(AA600='E', RBD=ANY, raf=2, rafi=None) % CRAF(RBD=None, raf=2))
#active RAF monomers bound by RAFi
Observable('inhibited_RAF_monomers', BRAF(AA600='E', raf=None, rafi=ANY) +  BRAF(AA600='E', RBD=None, raf=ANY, rafi=ANY) + BRAF(AA600='E', RBD=ANY, raf=2, rafi=ANY) % BRAF(RBD=None, raf=2) + BRAF(AA600='E', RBD=ANY, raf=2, rafi=ANY) % CRAF(RBD=None, raf=2))
#active RAF dimers that are not bound by RAFi
Observable('active_RAF_dimers', BRAF(RBD=ANY, raf=1, rafi=None) % BRAF(RBD=ANY, raf=1) + BRAF(RBD=ANY, raf=1, rafi=None) % CRAF(RBD=ANY, raf=1) + CRAF(RBD=ANY, raf=1, rafi=None) % BRAF(RBD=ANY, raf=1) + CRAF(RBD=ANY, raf=1, rafi=None) % CRAF(RBD=ANY, raf=1))
#active RAF dimers bound by RAFi
Observable('inhibited_RAF_dimers', BRAF(RBD=ANY, raf=1, rafi=ANY) % BRAF(RBD=ANY, raf=1) + BRAF(RBD=ANY, raf=1, rafi=ANY) % CRAF(RBD=ANY, raf=1) + CRAF(RBD=ANY, raf=1, rafi=ANY) % BRAF(RBD=ANY, raf=1) + CRAF(RBD=ANY, raf=1, rafi=ANY) % CRAF(RBD=ANY, raf=1))
#active MEK that are not bound by MEKi
Observable('active_pMEK', MEK(meki=None, phospho='p'))
#active MEK bound by MEKi
Observable('inhibited_pMEK', MEK(meki=ANY, phospho='p'))
#unphosphorylate MEK
Observable('uMEK', MEK(phospho='u'))
#unophosphorylated ERK
Observable('uERK', ERK(phospho='u'));
#total RAFi
Observable('tRAFi', RAFi())
#total RAFi
Observable('tMEKi', MEKi());
```

# Save generated MARM1 in SBML, BNG and PySB formats

We save the generated MARM1 model as a SBML, BNG and PySB format. These files are identical to one provided in the Docker container.

We explicitly set the model name here so the result matches the provided files.

```python
model.name = 'MARM1'
```

We first generate the PySB and BNGL formats. We do this before the SBML generation, as we do not want to include the extra expressions induced by the Energy BNG process that get added for the SBML export.

```python
generated_model_code = export(model, 'pysb_flat')
with open('MARM1.py', 'wt') as f:
    f.write(generated_model_code);

generated_model_code = export(model, 'bngl')
with open('MARM1.bngl', 'wt') as f:
    f.write(generated_model_code);
```

Now generate the SBML format, which implicitly triggers construction of the extra energy expressions.

```python
generated_model_code = export(model, 'sbml')
with open('MARM1.sbml', 'wt') as f:
    f.write(generated_model_code);
```

# MARM1 information

We list here the attributes of the MARM1 PySB model constructed by the code above.

```python
print ('MARM1 information')
nexpress=len(model.expressions);
generate_equations(model)
print ('Species:',len(model.species))
#parameters include initial conditions that were estimated, apart the three that are perturbations (RAFi, MEKi, EGF)
print ('Parameters:',len(model.parameters)+len(model.initial_conditions)-3)
print ('Expressions:',nexpress)
print ('Observables:', len(model.observables))
ntotr=len(model.rules);
nenergy=len([r for r in model.rules if r.energy]);
print ('Total Rules:', ntotr)
print ('Energy Rules:', nenergy)
print('Non-energy Rules:', ntotr-nenergy)
print('Energy Patterns:', len(model.energypatterns))
print('Reactions:',len(model.reactions))
```

    MARM1 information
    Species: 1007
    Parameters: 82
    Expressions: 185
    Observables: 30
    Total Rules: 53
    Energy Rules: 13
    Non-energy Rules: 40
    Energy Patterns: 27
    Reactions: 13102

# Bibliography

```python

```


# Tutorial 3: Confidence intervals and bootstrapping
**Week 1, Day 3: Model Fitting**

**By Neuromatch Academy**

**Content creators**: Pierre-Étienne Fiquet, Anqi Wu, Alex Hyafil with help from Byron Galbraith

**Content reviewers**: Lina Teichmann, Saeed Salehi, Patrick Mineault, Ella Batty, Michael Waskom

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

# Tutorial Objectives

*Estimated timing of tutorial: 23 minutes*

This is Tutorial 3 of a series on fitting models to data. We start with simple linear regression, using least squares optimization (Tutorial 1) and Maximum Likelihood Estimation (Tutorial 2). We will use bootstrapping to build confidence intervals around the inferred linear model parameters (Tutorial 3). We'll finish our exploration of regression models by generalizing to multiple linear regression and polynomial regression (Tutorial 4). We end by learning how to choose between these various models. We discuss the bias-variance trade-off (Tutorial 5) and Cross Validation for model selection (Tutorial 6).

In this tutorial, we will discuss how to gauge how good our estimated model parameters are.
- Learn how to use bootstrapping to generate new sample datasets
- Estimate our model parameter on these new sample datasets
- Quantify the variance of our estimate using confidence intervals

```python
# @title Tutorial slides

# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/2mkq4/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

```python
# @title Video 1: Confidence Intervals & Bootstrapping
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1vK4y1s7py", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="hs6bVGQNSIs", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

Up to this point we have been finding ways to estimate model parameters to fit some observed data. Our approach has been to optimize some criterion, either minimize the mean squared error or maximize the likelihood while using the entire dataset. How good is our estimate really? How confident are we that it will generalize to describe new data we haven't seen yet?

One solution to this is to just collect more data and check the MSE on this new dataset with the previously estimated parameters. However this is not always feasible and still leaves open the question of how quantifiably confident we are in the accuracy of our model.

In Section 1, we will explore how to implement bootstrapping. In Section 2, we will build confidence intervals of our estimates using the bootstrapping method.

---
# Setup

```python
# Imports

import numpy as np
import matplotlib.pyplot as plt
```

```python
#@title Figure Settings
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
# @title Plotting Functions

def plot_original_and_resample(x, y, x_, y_):
  """ Plot the original sample and the resampled points from this sample.

  Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    x_ (ndarray): An array of shape (samples,) with a subset of input values from x
    y_ (ndarray): An array of shape (samples,) with a the corresponding subset
      of measurement values as x_ from y

  """
  fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))
  ax1.scatter(x, y)
  ax1.set(title='Original', xlabel='x', ylabel='y')

  ax2.scatter(x_, y_, color='c')

  ax2.set(title='Resampled', xlabel='x', ylabel='y',
          xlim=ax1.get_xlim(), ylim=ax1.get_ylim());
```

---
# Section 1: Bootstrapping

*Estimated timing to here from start of tutorial: 7 min*

[Bootstrapping](https://en.wikipedia.org/wiki/Bootstrapping_(statistics)) is a widely applicable method to assess confidence/uncertainty about estimated parameters, it was originally [proposed](https://projecteuclid.org/euclid.aos/1176344552) by [Bradley Efron](https://en.wikipedia.org/wiki/Bradley_Efron). The idea is to generate many new synthetic datasets from the initial true dataset by randomly sampling from it, then finding estimators for each one of these new datasets, and finally looking at the distribution of all these estimators to quantify our confidence.

Note that each new resampled datasets will be the same size as our original one, with the new data points sampled with replacement i.e. we can repeat the same data point multiple times. Also note that in practice we need a lot of resampled datasets, here we use 2000.

To explore this idea, we will start again with our noisy samples along the line $y_i = 1.2x_i + \epsilon_i$, but this time only use half the data points as last time (15 instead of 30).

```python
#@title

#@markdown Execute this cell to simulate some data

# setting a fixed seed to our random number generator ensures we will always
# get the same psuedorandom number sequence
np.random.seed(121)

# Let's set some parameters
theta = 1.2
n_samples = 15

# Draw x and then calculate y
x = 10 * np.random.rand(n_samples)  # sample from a uniform distribution over [0,10)
noise = np.random.randn(n_samples)  # sample from a standard normal distribution
y = theta * x + noise

fig, ax = plt.subplots()
ax.scatter(x, y)  # produces a scatter plot
ax.set(xlabel='x', ylabel='y');
```

## Coding Exercise 1: Resample Dataset with Replacement

In this exercise you will implement a method to resample a dataset with replacement. The method accepts $\mathbf{x}$ and $\mathbf{y}$ arrays. It should return a new set of $\mathbf{x}'$ and $\mathbf{y}'$ arrays that are created by randomly sampling from the originals.

We will then compare the original dataset to a resampled dataset.

TIP: The [numpy.random.choice](https://numpy.org/doc/stable/reference/random/generated/numpy.random.choice.html) method would be useful here.

```python
def resample_with_replacement(x, y):
  """Resample data points with replacement from the dataset of `x` inputs and
  `y` measurements.

  Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.

  Returns:
    ndarray, ndarray: The newly resampled `x` and `y` data points.
  """
  #######################################################
  ## TODO for students: resample dataset with replacement
  # Fill out function and remove
  # raise NotImplementedError("Student exercise: resample dataset with replacement")
  #######################################################

  # Get array of indices for resampled points
  sample_idx =  np.random.choice(len(x), size=len(x), replace=True)

  # Sample from x and y according to sample_idx
  x_ = x[sample_idx]
  y_ = y[sample_idx]

  return x_, y_

x_, y_ = resample_with_replacement(x, y)

plot_original_and_resample(x, y, x_, y_)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D3_ModelFitting/solutions/W1D3_Tutorial3_Solution_81af3bd6.py)

*Example output:*

In the resampled plot on the right, the actual number of points is the same, but some have been repeated so they only display once.

Now that we have a way to resample the data, we can use that in the full bootstrapping process.

## Coding Exercise 2: Bootstrap Estimates

In this exercise you will implement a method to run the bootstrap process of generating a set of $\hat\theta$ values from a dataset of inputs ($\mathbf{x}$) and  measurements ($\mathbf{y}$). You should use `resample_with_replacement` here, and you may also invoke helper function `solve_normal_eqn` from Tutorial 1 to produce the MSE-based estimator.

We will then use this function to look at the theta_hat from different samples.

```python
# @markdown Execute this cell for helper function `solve_normal_eqn`
def solve_normal_eqn(x, y):
  """Solve the normal equations to produce the value of theta_hat that minimizes
    MSE.

    Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    thata_hat (float): An estimate of the slope parameter.

  Returns:
    float: the value for theta_hat arrived from minimizing MSE
  """
  theta_hat = (x.T @ y) / (x.T @ x)
  return theta_hat
```

```python
def bootstrap_estimates(x, y, n=2000):
  """Generate a set of theta_hat estimates using the bootstrap method.

  Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    n (int): The number of estimates to compute

  Returns:
    ndarray: An array of estimated parameters with size (n,)
  """
  theta_hats = np.zeros(n)

  ##############################################################################
  ## TODO for students: implement bootstrap estimation
  # Fill out function and remove
  # raise NotImplementedError("Student exercise: implement bootstrap estimation")
  ##############################################################################

  # Loop over number of estimates
  for i in range(n):

    # Resample x and y
    x_, y_ = resample_with_replacement(x, y)

    # Compute theta_hat for this sample
    theta_hats[i] = solve_normal_eqn(x_, y_)

  return theta_hats

# Set random seed
np.random.seed(123)

# Get boostrap estimates
theta_hats = bootstrap_estimates(x, y, n=2000)
print(theta_hats[0:5])
```

    [1.27550888 1.17317819 1.18198819 1.25329255 1.20714664]

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D3_ModelFitting/solutions/W1D3_Tutorial3_Solution_1b78959c.py)

You should see `[1.27550888 1.17317819 1.18198819 1.25329255 1.20714664]` as the first five estimates.

Now that we have our bootstrap estimates, we can visualize all the potential models (models computed with different resampling) together to see how distributed they are.

```python
#@title
#@markdown Execute this cell to visualize all potential models

fig, ax = plt.subplots()

# For each theta_hat, plot model
theta_hats = bootstrap_estimates(x, y, n=2000)
for i, theta_hat in enumerate(theta_hats):
  y_hat = theta_hat * x
  ax.plot(x, y_hat, c='r', alpha=0.01, label='Resampled Fits' if i==0 else '')

# Plot observed data
ax.scatter(x, y, label='Observed')

# Plot true fit data
y_true = theta * x
ax.plot(x, y_true, 'g', linewidth=2, label='True Model')

ax.set(
  title='Bootstrapped Slope Estimation',
  xlabel='x',
  ylabel='y'
)

# Change legend line alpha property
handles, labels = ax.get_legend_handles_labels()
handles[0].set_alpha(1)

ax.legend();
```

This looks pretty good! The bootstrapped estimates spread around the true model, as we would have hoped. Note that here we have the luxury to know the ground truth value for $\theta$, but in applications we are trying to guess it from data. Therefore, assessing  the  quality  of  estimates  based  on  finite data is a task of fundamental importance in data analysis.

---
# Section 2: Confidence Intervals

*Estimated timing to here from start of tutorial: 17 min*

Let us now quantify how uncertain our estimated slope is. We do so by computing [confidence intervals](https://en.wikipedia.org/wiki/Confidence_interval) (CIs) from our bootstrapped estimates. The most direct approach is to compute percentiles from the empirical distribution of bootstrapped estimates. Note that this is widely applicable as we are not assuming that this empirical distribution is Gaussian.

```python
#@title

#@markdown Execute this cell to plot bootstrapped CI

theta_hats = bootstrap_estimates(x, y, n=2000)
print(f"mean = {np.mean(theta_hats):.2f}, std = {np.std(theta_hats):.2f}")

fig, ax = plt.subplots()
ax.hist(theta_hats, bins=20, facecolor='C1', alpha=0.75)
ax.axvline(theta, c='g', label=r'True $\theta$')
ax.axvline(np.percentile(theta_hats, 50), color='r', label='Median')
ax.axvline(np.percentile(theta_hats, 2.5), color='b', label='95% CI')
ax.axvline(np.percentile(theta_hats, 97.5), color='b')
ax.legend()
ax.set(
    title='Bootstrapped Confidence Interval',
    xlabel=r'$\hat{{\theta}}$',
    ylabel='count',
    xlim=[1.0, 1.5]
);
```

Looking at the distribution of bootstrapped $\hat{\theta}$ values, we see that the true $\theta$ falls well within the 95% confidence interval, wich is reinsuring. We also see that the value $\theta = 1$ does not fall within the confidence interval. From this we would reject the hypothesis that the slope was 1.

---
# Summary

*Estimated timing of tutorial: 23 minutes*

- Bootstrapping is a resampling procedure that allows to build confidence intervals around inferred parameter values
- it is a widely applicable and very practical method that relies on computational power and pseudo-random number generators (as opposed to more classical approaches than depend on analytical derivations)

---
# Notation

\begin{align}
\theta &\quad \text{parameter}\\
\hat{\theta} &\quad \text{estimated parameter}\\
x &\quad \text{input, independent variable}\\
y &\quad \text{response measurement, dependent variable}\\
\mathbf{x} &\quad \text{vector of input values}\\
\mathbf{y} &\quad \text{vector of measurements}\\
\mathbf{x}' &\quad \text{vector of resampled input values }\\
\mathbf{y}' &\quad \text{vector of resampled measurement values}\\
\end{align}

**Suggested readings**

Computer Age Statistical Inference: Algorithms, Evidence and Data Science, by Bradley Efron and Trevor Hastie


# Symbolic Differentiation vs Automatic Differentiation

Consider the function below that, at least computationally, is very simple.

```python
from math import sin, cos

def func(x):
    y = x
    for i in range(30):
        y = sin(x + y)

    return y
```

We can compute a derivative symbolically, but it is of course horrendous (see below).  Think of how much worse it would be if we chose a function with products, more dimensions, or iterated more than 20 times.

```python
from sympy import diff, Symbol, sin
from __future__ import print_function

x = Symbol('x')
dexp = diff(func(x), x)
print(dexp)
```

    (((((((((((((((((((((((((((((2*cos(2*x) + 1)*cos(x + sin(2*x)) + 1)*cos(x + sin(x + sin(2*x))) + 1)*cos(x + sin(x + sin(x + sin(2*x)))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(2*x))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x)))))))))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))))))))))))))) + 1)*cos(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(x + sin(2*x))))))))))))))))))))))))))))))

We can now evaluate the expression.

```python
xpt = 0.1

dfdx = dexp.subs(x, xpt)

print('dfdx =', dfdx)
```

    dfdx = 1.91770676038667

Let's compare with automatic differentiation using operator overloading:

```python
from algopy import UTPM, sin

x_algopy = UTPM.init_jacobian(xpt)
y_algopy = func(x_algopy)
dfdx = UTPM.extract_jacobian(y_algopy)

print('dfdx =', dfdx)
```

    dfdx = [ 1.91770676]

Let's also compare to AD using a source code transformation method (I used Tapenade in Fortran)

```python
def funcad(x):
    xd = 1.0
    yd = xd
    y = x
    for i in range(30):
        yd = (xd + yd)*cos(x + y)
        y = sin(x + y)
    return yd

dfdx = funcad(xpt)

print('dfdx =', dfdx)
```

    dfdx = 1.91770676039

For a simple expression like this, symbolic differentiation is long but actually works reasonbly well, and both will give a numerically exact answer.  But if we change the loop to 100 (go ahead and try this) or add other complications, the symbolic solver will fail.  However, automatic differentiation will continue to work without issue (see the simple source code transformation version).  Furthermore, if we add other dimensions to the problem, symbolic differentiation quickly becomes costly as lots of computations get repeated, whereas automatic differentiation is able to reuse a lot of calculations.


# 02 Data Structures and Libraries
## CLASS MATERIAL

 1. Data Structures

 2. Libraries

 3. Installing Pygame

 4. Review Exercises

# Update the new class notes.
__Navigate to the directory where your files are stored.__

__Update the course notes by downloading the changes__

##### Windows
Search for __Git Bash__ in the programs menu.

Select __Git Bash__, a terminal will open.

Use `cd` to navigate to *inside* the __ILAS_PyEv2019__ repository you downloaded.

Run the command:
>`./automerge`

##### Mac
Open a terminal.

Use `cd` to navigate to *inside* the __ILAS_PyEv2019__ repository you downloaded.

Run the command:
>`sudo ./automerge`

Enter your password when prompted.

# Primer Summary
For more information refer to the primer notebook for this class 02_DataStructures_Libraries__Primer.ipynb

###### Data Structures

- A data structure is used to assign a collection of values to a single collection name.
 - A Python list can store multiple items of data in sequentially numbered elements (numbering starts at zero)
 - Data stored in a list element can be referenced using the list name can be referenced using the list name followed by an index number in [] square brackets.
 - The `len()` function returns the length of a specified list.

###### Libraries
- Python has an extensive __standard library__ of built-in functions.
- More specialised libraries of functions and constants are available. We call these __packages__.
- Packages are imported using the keyword `import`
- The function documentation tells is what it does and how to use it.
- When calling a library function it must be prefixed with a __namespace__ is used to show from which package it should be called.

## Lesson Goal

- Build a guessing game.
- Build the game 0 and Xs or tic-tac-toe.

## Fundamental programming concepts
 - Importing existing libraries of code to use in your program
 - Storing and representing data e.g. a grid with 0 and X in each grid cell

# 1. Data Structures

In the last seminar we learnt to generate a range of numbers for use in control flow of  a program, using the function `range()`:

       for j in range(20):
           ...

Often we want to manipulate data that is more meaningful than ranges of numbers.

These collections of variables might include:
 - the results of an experiment
 - a list of names
 - the components of a vector
 - a telephone directory with names and associated numbers.

Python has different __data structures__ that can be used to store and manipulate these values.

Like variable types (`string`, `int`,`float`...) different data structures behave in different ways.

Today we will learn to use `list`s

A list is a container with compartments in which we can store data:

Example

If we want to store the names of students in a laboratory group,
rather than representing each students using an individual string variable, we could use a list of names.

```python
lab_group0 = ["Yukari", "Sajid", "Hemma", "Ayako"]
lab_group1 = ["Sara", "Mari", "Quang", "Sam", "Ryo", "Nao", "Takashi"]

print(lab_group0)
print(lab_group1)
```

    ['Yukari', 'Sajid', 'Hemma', 'Ayako']
    ['Sara', 'Mari', 'Quang', 'Sam', 'Ryo', 'Nao', 'Takashi']

This is useful because we can perform operations on lists such as:
 - checking its length (number of students in a lab group)
 - sorting the names in the list into alphabetical order
 - making a list of lists (we call this a *nested list*):

```python
lab_groups = [lab_group0, lab_group1]
print(lab_groups)
```

    [['Yukari', 'Sajid', 'Hemma', 'Ayako'], ['Sara', 'Mari', 'Quang', 'Sam', 'Ryo', 'Nao', 'Takashi']]

### Example: Change in Position: (Representing Vectors using Lists)

__Vector:__ A quantity with magnitude and direction.

The position of a point in 2D space (e.g. the position of a character in a game), can be expressed in terms of horizontal (x) and vertical (y) conrdinates.

The movement to a new position can be expressed as a change in x and y.

[Daniel Schiffman, The Nature of Code]

We can conveniently express the position $\mathbf{r}$ in matrix (or basis vector) form using the coefficients $x$ and  $y$:
$$
\mathbf{r} = [r_x, r_y]
$$

__...which looks a lot like a Python list!__

When we move a character in a game, we change it's position.

The change in position with each time-step is the __velocity__ of the character.

__Velocity__ : the magnitude and direction of a change in position per time increment.

$$
\mathbf{v} = [v_x, v_y]
$$

To get the position at the next time step we simply add the x and y component of the veclocity to the x and y component of the initial position vector:

\begin{align}
      {\displaystyle {\begin{aligned}\
      \mathbf{w}
      &=\mathbf{u} + \mathbf{u}\\
      &=[(u_x+v_x),\;\;   (u_y+v_y)] \\ \end{aligned}}}
\end{align}

For example, let's find the position at the next timestep where:
 - initial position,  $\mathbf{u} = [5, 2]$
 - velocity,  $\mathbf{v} = [3, 4]$

 [Daniel Schiffman, The Nature of Code]

```python
# Example: Change in Position

```

```python
# Example Solution: Change in Position

u = [5, 2]
v = [3, 4]

u = [u[0] + v[0],
     u[1] + v[1]]

print(u)
```

    [8, 6]

Arranging the code on seperate lines:
 - makes the code more readable
 - does not effect how the code works

Line breaks can only be used within code that is enclosed by at elast one set of brackets (), [].

__Check Your Solution:__

$ \mathbf{u} = [5, 2]$

 $ \mathbf{v} = [3, 4]$

\begin{align}
      {\displaystyle {\begin{aligned}\
      \mathbf{u} + \mathbf{v}
      &=[5, 2]+ [3, 4]\\
      &=[(5+3), \quad (2+4)] \\
      & = [8, 6] \end{aligned}}}
\end{align}

# 2. Libraries

    __2.1 The Standard Library__

    __2.2 Packages__

    __2.3 Function Documentation__

    __2.4 Using Package Functions to Optimise your Code__

One of the most important concepts in good programming is to reuse code and avoid repetitions.

Python, like other modern programming languages, has an extensive *library* of built-in functions.

These functions are designed, tested and optimised by the developers of the Python langauge.

We can use these functions to make our code shorter, faster and more reliable.

## 2.1 The Standard Library

Python has a large standard library.

e.g. `print()` takes the __input__ in the parentheses and __outputs__ a visible representation.

They are listed on the Python website:
https://docs.python.org/3/library/functions.html

We could write our own code to find the minimum of a group of numbers

```python
x0 = 1
x1 = 2
x2 = 4

x_min = x0
if x1 < x_min:
    x_min = x1
if x2 < x_min:
    x_min = x2

print(x_min)
```

    1

However, it is much faster to use the build in function:

```python
print(min(1,2,4))
```

    1

The built-in functions can be found in (.py) files called 'modules'.

The files are neatly arranged into a system of __sub-packages__ (sub-folders) and __modules__ (files).

These files are stored on the computer you are using.

A quick google search for "python function to sum all the numbers in a list"...

https://www.google.co.jp/search?q=python+function+to+sum+all+the+numbers+in+a+list&rlz=1C5CHFA_enJP751JP751&oq=python+function+to+sum+&aqs=chrome.0.0j69i57j0l4.7962j0j7&sourceid=chrome&ie=UTF-8

...returns the function `sum()`.

`sum()` finds the sum of the values in a data structure.

```python
print(sum([1,2,3,4,5]))

print(sum((1,2,3,4,5)))

a = [1,2,3,4,5]
print(sum(a))
```

    15
    15
    15

The function `max()` finds the maximum value in data structure.

## 2.2 Packages

The standard library tools are available in any Python environment.

More specialised libraries, called packages, are available for more specific tasks

e.g. solving trigonometric functions.

Packages contain functions and constants.

We install the packages to use them.

__Pygame__

For a large part of this course we will use functions from a package called `Pygame`.

`Pygame` is a set of Python modules designed for writing computer games, graphics and sound projects.

Instructions for how install pygame will be given later in today's seminar.

__math__

`math` is already installed and allows you to use convenient mathematical functions and operators.

A package is a collection of Python modules:
- a __module__ is a single Python file
- a __package__ is a directory of Python modules.
(It contains an __init__.py file, to distinguish it from folders that are not libraries).

The files that are stored on your computer when Pygame is installed:

https://github.com/pygame/pygame

## Importing a Package

To use an installed package, we  simply `import` it.

We only need to import the package once.

The `import` statement must appear before the use of the package in the code so all packages are usually imported at the start of the program.

        import math

We can then use variable and functions defined within that package by prefixing them with the name of the package:

```python
import math
```

After this, any constant, variable or function from the package can be used by prefixing it with the name of the package:

Any constant in `math` can be called as:

        `math.constant`.

Any function in `numpy` can be called as:

        `math.function()`

```python
# pi
print(math.pi)

x = 1

# Trigonometric functions e.g. cosine
y = math.cos(x)
print(y)
```

    3.141592653589793
    0.5403023058681398

## Using Package Functions.

Let's learn to use `math` functions in our programs...

```python
# Some examples math functions with their definitions (as given in the documentation)

x = 1

# Return the sine of x radians.
print(math.sin(x))

# Return the tangent of x radians.
print(math.tan(x))

# Return the inverse hyperbolic tangent of x.
print(math.atan(x))

```

    0.8414709848078965
    1.557407724654902
    0.7853981633974483

```python
x = 1

# Convert angle x from radians to degrees.
degrees = math.degrees(x)
print(degrees)

# Convert angle x from degrees to radians.
radians = math.radians(degrees)
print(radians)
```

    57.29577951308232
    1.0

## 2.3 Function Documentation

Online documentation can be used to find out:
- what to include in the () parentheses
- allowable data types to use as arguments
- the order in which arguments should be given

A google search for 'python math documentation' returns:

https://docs.python.org/3/library/math.html

(this list is not exhaustive).

### Try it yourself:

 Find a function in the Python math documentation (https://docs.python.org/3/library/math.html) that can be used to solve the following problem:

##### Return, $\left| x \right|$, the absolute value of x

Write your answer in the cell below:

```python
# Return the absolute value of x.
```

### Example : math.pow ($x^y$)
Documentation : https://docs.python.org/3/library/math.html#hyperbolic-functions

The documentation tells us the following information...

##### What arguments to input:
"math.pow(x, y)"

##### What the function returns:
"Return x raised to the power y."

##### The format of the returned argument:
math.pow() converts both its arguments to type float

## 2.4 Using Package Functions to Optimise your Code
One  purpose of using imported functions is to make your code shorter and neater.

For example, when designing a game it can be very useful to generate (pseudo) random numbers so that the challenges and problems for the user to solve are not identical every time the game is played.

Writing your own algorithm to generate random numbers is unecessarily time-consuming.

`random` is a python module that implements pseudo-random number generators for various distributions.

The documentation of the functions in this package can be found here:

https://docs.python.org/3/library/random.html#

Import random to use functions from this package:

```python
import random
```

Here are some examples of functions from `random`...

```python
# random.randint(a, b)
# Return a random integer N such that a <= N <= b.
random.randint(1, 15)

```

    14

```python
# random.sample(population, k)
# Return a k length list of unique elements chosen from the population sequence or set.
# Used for random sampling without replacement.
random.sample([1,2,3,4,5,6,7,8], 4)
```

    [4, 7, 8, 1]

Random numbers can be used to introduce an element of uncertainty to your programs.

This makes a game more intersting as the outcome may be different every time it is played.

For example, an adventure game may have a different outcome depending on a random variable.

```python
print("Inside the castle you see a giant spider.")
answer = input("Do you fight the spider?  (Yes/No) ")

if answer == 'Yes':
    print("you defeat the spider. \nYou win!")

else:
    print("The spider eats you. \nYou lose!")
```

    Inside the castle you see a giant spider.
    Do you fight the spider?  (Yes/No) Yes
    you defeat the spider.
    You win!

An adventure game where the outcome is random can be more interesting.

```python
print("Inside the castle you see a giant spider.")
answer = input("Do you fight the spider?  (Yes/No) ")

if answer == 'Yes':
    number = int(random.randint(0, 2))
    if number < 2:
        print("The spider defeats you. \nYou lose!")
    else:
        print("you defeat the spider. \nYou win!")

else:
    print("The spider eats you. \nYou lose!")
```

    Inside the castle you see a giant spider.
    Do you fight the spider?  (Yes/No)
    The spider eats you.
    You lose!

Random, computer generated values can also be used to build a game of rock, paper scissors where the user plays against the computer.

The code section below shows the case that the player choses *rock*.

```python

computer = int(random.randint(0, 2))
        if computer == 0:
            c_choice = "rock"
        elif computer == 1:
            c_choice = "paper"
        else:
            c_choice = "scissors"
```

The full version of the code is given in example program: Examples/02_RockPaperScissors.py

Try running this from the terminal by typing `python3 02_RockPaperScissors.py` from within the __PyEv2019/Examples__ directory.

## 3. Installing Pygame
Install Pygame now. We will begin using this package in next week's class.

##### Windows

1. Open the Anaconda Prompt from the terminal.

1. The window that opens will look like the command line. In the window type the following code then press 'Enter':
>`conda install -c anaconda pip`

1. When the installation completes type the following code then press 'Enter':
>`pip install pygame`

##### Mac

1. Open a terminal.

1. Type the following code then press 'Enter':
>`conda install -c anaconda pip`

1. When the installation completes type the following code then press 'Enter':
>`pip install pygame`

To check the installation has worked type:
>`import pygame`

in a Jupyter notebook cell and run the cell. If no error is generated you have installed pygame successfully.

# 4. Review Exercises

Compete the exercise below.

Save your answers as .py files and email them to:

philamore.hemma.5s@kyoto-u.ac.jp

## Review Exercise 1 : Guessing Game
__(A)__

Write a game that:
- chooses a random number between 1 and 10
- asks the user to guess what it is
- exits when the user guesses the right number

Use:
- a while loop
- a break statement (to break out of the while loop)
- a random number generator (from the package, `random`)

Example : The output from your game might look like this:

```
    I'm thinking of a random number between 1 and 10.
    Can you guess what it is?

    Guess what number I am thinking of: 5

    Guess what number I am thinking of: 2

    Guess what number I am thinking of: 1

    Guess what number I am thinking of: 9

    You win! I was thinking of 9.
```

*Hint: Remember that the function `input` returns a string even when a numerical value is entered.*

```python
# Review Exercise A: Guessing Game

```

    I'm thinking of a random number between 1 and 10.
    Can you guess what it is?
    Guess what number I am thinking of: 4
    Guess what number I am thinking of: 5
    Guess what number I am thinking of: 6
    You win! I was thinking of 6

```python
# Review Exercise : Guessing Game
# Example Solution

import random
import math

num = random.randint(1, 10)

print("I'm thinking of a random number between 1 and 10.")
print("Can you guess what it is?")

while(1):

    guess = int(input("Guess what number I am thinking of: "))

    if guess == num:
        print(f"You win! I was thinking of {num}")
        break
```

__(B)__

Use `if` and `else` to generate a __clue__ for the player gets the answer wrong

The output from your game might look like this:
```
    I'm thinking of a random number between 1 and 10.
    Can you guess what it is?

    Guess what number I am thinking of: 1
    Too low.

    Guess what number I am thinking of: 5
    Too low.

    Guess what number I am thinking of: 9

    You win! I was thinking of 9.
    ```

```python
# Review Exercise B: Guessing Game with Clues
```

```python
# Review Exercise : Guessing Game with Clues
# Example Solution

import random
import math

num = random.randint(1, 10)

print("I'm thinking of a random number between 1 and 10.")
print("Can you guess what it is?")

while(1):

    guess = int(input("Guess what number I am thinking of: "))

    if guess == num:
        print(f"You win! I was thinking of {num}")
        break

    elif guess < num:
        print("Too low")

    else:
        print("Too high")

```

    I'm thinking of a random number between 1 and 10.
    Can you guess what it is?
    Guess what number I am thinking of: 2
    Too low
    Guess what number I am thinking of: 5
    Too high
    Guess what number I am thinking of: 3
    Too low
    Guess what number I am thinking of: 4
    You win! I was thinking of 4

__(C)__

Use `break` to quit the game if the user makes three consecutive wrong guesses.

The output from your game might look something like this:

    I'm thinking of a random number between 1 and 10.
    Can you guess what it is? You have 3 guesses...

    Guess what number I am thinking of: 1
    Too low.

    Guess what number I am thinking of: 2
    Too low.

    Guess what number I am thinking of: 5

    You used all your lives! You lose! I was thinking of 9.

```python
# Review Exercise C: Guessing game with maximum number of tries

```

```python
# Review Exercise : Guessing game with maximum number of tries
# Example Solution

import random
import math

num = random.randint(1, 10)

print("I'm thinking of a random number between 1 and 10.")
print("Can you guess what it is?")

for i in range(3):
    guess = int(input("Guess what number I am thinking of: "))

    if guess == num:
        print(f"You win! I was thinking of {num}")
        break

    elif guess < num:
        print("Too low")

    else:
        print("Too high")

print(f"You used all your lives! You lose! I was thinking of {num}.")
```

    I'm thinking of a random number between 1 and 10.
    Can you guess what it is?
    Guess what number I am thinking of: 3
    Too low
    Guess what number I am thinking of: 4
    Too low
    Guess what number I am thinking of: 5
    Too low
    You used all your lives! You lose! I was thinking of 7.

## Review Exercise 2:  List with `for` loop.
In the cell below, use a `for` loop to print the first letter of each month in the list.

```python
# Print the first letter of each month in the list

months = ["January",
         "February",
         "March",
         "April",
         "May",
         "June",
         "July",
         "August",
         "September",
         "October",
         "November",
         "December"]
```

```python
# Review Exercise: List with for loop
# Example Solution
```

## Review Exercise 3 : Tic-tac-toe
A list of lists can be used to represent a discrete set of positions the a player can occupy.

We can write a program that plays the the game tic-tac-toe, using a list of list.

The program is shown in the cell below.

__(A)__

The program decides whether to quit the game based on user input after each turn taken.

Copy and paste the program into the cell provided below.

Edit the program to quit the game if either:
 - one of the players places three marks in a row.
 - all positions have been marked but noone has won.

*Hints:*
- Use:
 - `if` and `else`
 - the boolean operators `and`, `or`, `not`
- If the value of three colinear places is equal, the game has finished/been won.

```python
# tic-tac-toe
#
# Board positions:
#
# 00   01   02
# 10   11   12
# 20   21   22

# Example solution

# # use a for loop to set-up the 3x3 grid
board = []
for row in range(3):
    #board.append(['_', '_', '_'])
    board.append(['_']*3)

# print the output nicely
for row in board:
    for column in row:
        print(column, end='\t')
    # new line at end of row
    print()

# choose who goes first, X or 0
player = 'X'

# keep playing the game until told to quit
while(1):
    # use if and else to take turns at adding input from each player to the board
    if player == 'X':
        position = input('Player X, choose position: ')
        board[int(position[0])][int(position[1])] = 'X'
        player = '0'

    else:
        position = input('Player 0, choose position: ')
        board[int(position[0])][int(position[1])] = '0'
        player = 'X'

    #board[int(position[0])][int(position[1])] = player

    # print the output nicely
    for row in board:
        for column in row:
            print(column, end='\t')
        # new line at end of row
        print()

    print('\nnext turn \n')

    game_over = input('Game over?(Y/N): ')
    if game_over == 'Y':
        break
```

__(B)__

Edit the program to prevent a player from choosing a place that is already occupied.

For example, the program might ask the player to choose again.

e.g.

        position already occupied, Player X choose a different position

```python
# Review Exercise: Tic-tac-toe
```

```python
# Review Exercise: Tic-tac-toe
# Example Solution

# use a for loop to set-up the board
board = []
for row in range(3):
    #board.append(['_', '_', '_'])
    board.append(['_']*3)

# display the output nicely
for lists in board:
    for i in lists:
        print(i,end='\t')
    # leave gap before next row
    print()

# choose who goes first, X or 0
player = 'X'

# keep playing the game until told to quit
while(1):
    # use if and else to take turns at adding input from each player to the board
    if player == 'X':
        position = input('Player X, choose position: ')
        while(board[int(position[0])][int(position[1])] != '_'):
            position = input('Position taken, Player X, choose again: ')
        board[int(position[0])][int(position[1])] = 'X'
        player = '0'

    else:
        position = input('Player 0, choose position: ')
        while(board[int(position[0])][int(position[1])] != '_'):
            position = input('Position taken, Player 0, choose again: ')
        board[int(position[0])][int(position[1])] = '0'
        player = 'X'

    #board[int(position[0])][int(position[1])] = player

    # display the output nicely
    for lists in board:
        for i in lists:
            print(i,end='\t')
        print()

    # game over : one player gets 3 in a row
    if((board[0][0] == board[1][1] == board[2][2]== 'X')or
       (board[2][0] == board[1][1] == board[0][2]== 'X')or

       (board[0][0] == board[1][0] == board[2][0]== 'X')or
       (board[0][1] == board[1][1] == board[2][1]== 'X')or
       (board[0][2] == board[1][2] == board[2][2]== 'X')or

       (board[0][0] == board[0][1] == board[0][2]== 'X')or
       (board[1][0] == board[1][1] == board[1][2]== 'X')or
       (board[2][0] == board[2][1] == board[2][2]== 'X')or

       (board[0][0] == board[1][1] == board[2][2]== '0')or
       (board[2][0] == board[1][1] == board[0][2]== '0')or

       (board[0][0] == board[1][0] == board[2][0]== '0')or
       (board[0][1] == board[1][1] == board[2][1]== '0')or
       (board[0][2] == board[1][2] == board[2][2]== '0')or

       (board[0][0] == board[0][1] == board[0][2]== '0')or
       (board[1][0] == board[1][1] == board[1][2]== '0')or
       (board[2][0] == board[2][1] == board[2][2]== '0')):
        print("Game over! Player " + player + " wins!")
        break

    # game over : no empty sapces remain
    gameover = True
    for lists in board:
        for i in lists:
            if i=='_':
                gameover = False
    if gameover:
        print("Game over! No winner.")

```

    _	_	_
    _	_	_
    _	_	_
    Player X, choose position: 20
    _	_	_
    _	_	_
    X	_	_
    Player 0, choose position: 11
    _	_	_
    _	0	_
    X	_	_
    Player X, choose position: 20
    Position taken, Player X, choose again: 00
    X	_	_
    _	0	_
    X	_	_
    Player 0, choose position: 11
    Position taken, Player 0, choose again: 21
    X	_	_
    _	0	_
    X	0	_

```python
# tic-tac-toe
```