# Single Particle Model (SPM)

## Model Equations

The SPM consists of two spherically symmetric diffusion equations: one within a representative negative particle ($\text{k}=\text{n}$) and one within a representative positive particle ($\text{k}=\text{p}$). In the centre of the particle the classical no-flux condition is imposed. Since the SPM assumes that all particles in an electrode behave in exactly the same way, the flux on the surface of a particle is simply the current $I$ divided by the thickness of the electrode $L_{\text{k}}$. The concentration of lithium in electrode $\text{k}$ is denoted $c_{\text{k}}$ and the current is denoted by $I$. All parameters in the model stated here are dimensionless and are given in terms of dimensional parameters at the end of this notebook. The model equations for the SPM are then:
\begin{align}
\mathcal{C}_{\text{k}} \frac{\partial c_{\text{s,k}}}{\partial t} &= -\frac{1}{r_{\text{k}}^2} \frac{\partial}{\partial r_{\text{k}}} \left(r_{\text{k}}^2 N_{\text{s,k}}\right), \\
N_{\text{s,k}} &= -D_{\text{s,k}}(c_{\text{s,k}}) \frac{\partial c_{\text{s,k}}}{\partial r_{\text{k}}}, \quad \text{k} \in \text{n, p}, \end{align}
$$
N_{\text{s,k}}\big|_{r_{\text{k}}=0} = 0, \quad \text{k} \in \text{n, p}, \quad \ \ - \frac{a_{\text{k}}\gamma_{\text{k}}}{\mathcal{C}_{\text{k}}} N_{\text{s,k}}\big|_{r_{\text{k}}=1} =
\begin{cases}
		  \frac{I}{L_{\text{n}}}, \quad &\text{k}=\text{n}, \\
		  -\frac{I}{L_{\text{p}}}, \quad &\text{k}=\text{p},
\end{cases} \\
c_{\text{s,k}}(r_{\text{k}},0) = c_{\text{s,k,0}}, \quad \text{k} \in \text{n, p},$$
where $D_{\text{s,k}}$ is the diffusion coefficient in the solid, $N_{\text{s,k}}$ denotes the flux of lithium ions in the solid particle within the region $\text{k}$, and $r_{\text{k}} \in[0,1]$ is the radial coordinate of the particle in electrode $\text{k}$.

### Voltage Expression
The terminal voltage is obtained from the expression:
$$
V = U_{\text{p}}(c_{\text{p}})\big|_{r_{\text{p}}=1} - U_{\text{n}}(c_{\text{n}})\big|_{r_{\text{n}}=1} -2\sinh^{-1}\left(\frac{I}{j_{\text{0,p}} L_{\text{p}}}\right) - 2\sinh^{-1}\left(\frac{I}{j_{\text{0,n}} L_{\text{n}}}\right)
$$
with the exchange current densities given by
$$j_{\text{0,k}} =  \frac{\gamma_{\text{k}}}{\mathcal{C}_{\text{r,k}}}(c_{\text{k}})^{1/2}(1-c_{\text{k}})^{1/2}  $$

## Example solving SPM using PyBaMM

Below we show how to solve the SPM model, using the default geometry, mesh, paramters, discretisation and solver provided with PyBaMM.

First we need to import `pybamm`, and then change our working directory to the root of the pybamm folder.

```python
import pybamm
import numpy as np
import os
import matplotlib.pyplot as plt
os.chdir(pybamm.__path__[0]+'/..')
```

We then get the SPM model equations:

```python
model = pybamm.lithium_ion.SPM()
```

The model object is a subtype of [`pybamm.BaseModel`](https://pybamm.readthedocs.io/en/latest/source/models/base_model.html), and contains all the equations that define this particular model. For example, the `rhs` dict contained in `model` has a dictionary mapping variables such as $c_n$ to the equation representing its rate of change with time (i.e. $\partial{c_n}/\partial{t}$). We can see this explicitly by visualising this entry in the `rhs` dict:

```python
variable = next(iter(model.rhs.keys()))
equation = next(iter(model.rhs.values()))
print('rhs equation for variable \'',variable,'\' is:')
path = 'examples/notebooks/models/'
equation.visualise(path+'spm1.png')
```

    rhs equation for variable ' Discharge capacity [A.h] ' is:

We need a geometry to define our model equations over. In pybamm this is represented by the [`pybamm.Geometry`](https://pybamm.readthedocs.io/en/latest/source/geometry/geometry.html) class. In this case we use the default geometry object defined by the model

```python
geometry = model.default_geometry
```

This geometry object defines a number of geometry domains, each with its own name, spatial variables and min/max limits (the latter are represented as equations similar to the rhs equation shown above). For instance, the SPM model has the following domains:

```python
print('SPM domains:')
for i, (k, v) in enumerate(geometry.items()):
    print(str(i+1)+'.',k,'with variables:')
    for var, rng in v['primary'].items():
        if 'min' in rng:
            print('  -(',rng['min'],') <=',var,'<= (',rng['max'],')')
        else:
            print(var, '=', rng['position'])
```

    SPM domains:
    1. negative electrode with variables:
      -( 0.0 ) <= x_n <= ( Negative electrode thickness [m] / Negative electrode thickness [m] + Separator thickness [m] + Positive electrode thickness [m] )
    2. separator with variables:
      -( Negative electrode thickness [m] / Negative electrode thickness [m] + Separator thickness [m] + Positive electrode thickness [m] ) <= x_s <= ( Negative electrode thickness [m] / Negative electrode thickness [m] + Separator thickness [m] + Positive electrode thickness [m] + Separator thickness [m] / Negative electrode thickness [m] + Separator thickness [m] + Positive electrode thickness [m] )
    3. positive electrode with variables:
      -( Negative electrode thickness [m] / Negative electrode thickness [m] + Separator thickness [m] + Positive electrode thickness [m] + Separator thickness [m] / Negative electrode thickness [m] + Separator thickness [m] + Positive electrode thickness [m] ) <= x_p <= ( 1.0 )
    4. current collector with variables:
    z = 1.0
    5. negative particle with variables:
      -( 0.0 ) <= r_n <= ( 1.0 )
    6. positive particle with variables:
      -( 0.0 ) <= r_p <= ( 1.0 )

Both the model equations and the geometry are defined by a set of parameters, such as $\gamma_p$ or $L_p$. We can substitute these symbolic parameters in the model with values by using the [`pybamm.ParameterValues`](https://pybamm.readthedocs.io/en/latest/source/parameters/parameter_values.html) class, which takes either a python dictionary or CSV file with the mapping between parameter names and values. Rather than create our own instance of `pybamm.ParameterValues`, we will use the default parameter set included in the model

```python
param = model.default_parameter_values
```

We can then apply this parameter set to the model and geometry

```python
param.process_model(model)
param.process_geometry(geometry)
```

The next step is to mesh the input geometry. We can do this using the [`pybamm.Mesh`](https://pybamm.readthedocs.io/en/latest/source/meshes/meshes.html) class. This class takes in the geometry of the problem, and also two dictionaries containing the type of mesh to use within each domain of the geometry (i.e. within the positive or negative electrode domains), and the number of mesh points.

The default mesh types and the default number of points to use in each variable for the SPM model are:

```python
for k, t in model.default_submesh_types.items():
    print(k,'is of type',t.__repr__())
for var, npts in model.default_var_pts.items():
    print(var,'has',npts,'mesh points')
```

    negative electrode is of type Generator for Uniform1DSubMesh
    separator is of type Generator for Uniform1DSubMesh
    positive electrode is of type Generator for Uniform1DSubMesh
    negative particle is of type Generator for Uniform1DSubMesh
    positive particle is of type Generator for Uniform1DSubMesh
    current collector is of type Generator for SubMesh0D
    x_n has 20 mesh points
    x_s has 20 mesh points
    x_p has 20 mesh points
    r_n has 10 mesh points
    r_p has 10 mesh points
    y has 10 mesh points
    z has 10 mesh points

With these defaults, we can then create our mesh of the given geometry:

```python
mesh = pybamm.Mesh(geometry, model.default_submesh_types, model.default_var_pts)
```

The next step is to discretise the model equations over this mesh. We do this using the [`pybamm.Discretisation`](https://pybamm.readthedocs.io/en/latest/source/discretisations/discretisation.html) class, which takes both the mesh we have already created, and a dictionary of spatial methods to use for each geometry domain. For the case of the SPM model, we use the following defaults for the spatial discretisation methods:

```python
for k, method in model.default_spatial_methods.items():
    print(k,'is discretised using',method.__class__.__name__,'method')
```

    macroscale is discretised using FiniteVolume method
    negative particle is discretised using FiniteVolume method
    positive particle is discretised using FiniteVolume method
    current collector is discretised using ZeroDimensionalMethod method

We then create the `pybamm.Discretisation` object, and use this to discretise the model equations

```python
disc = pybamm.Discretisation(mesh, model.default_spatial_methods)
disc.process_model(model)
```

    <pybamm.models.full_battery_models.lithium_ion.spm.SPM at 0x10681a1d0>

After this stage, all the equations in `model` have been discretised into purely linear algebra expressions that are ready to be evaluated within a time-stepping loop of a given solver. For example, the rhs expression for $\partial{c_n}/\partial{t}$ that we visualised above is now represented by:

```python
model.concatenated_rhs.children[0].visualise(path+'spm2.png')
```

Now we are ready to run the time-stepping loop to solve the model. Once again we use the default ODE solver like so:

```python
# Solve the model at the given time points (in seconds)
solver = model.default_solver
n = 250
t_eval = np.linspace(0, 3600, n)
print('Solving using',type(solver).__name__,'solver...')
solution = solver.solve(model, t_eval)
print('Finished.')
```

    Solving using ScipySolver solver...
    Finished.

Each model in pybamm has a list of relevent variables defined in the model, for use in visualising the model solution or for comparison with other models. The SPM model defines the following variables:

```python
print('SPM model variables:')
for v in model.variables.keys():
    print('\t-',v)
```

    SPM model variables:
    	- Time
    	- Time [s]
    	- Time [min]
    	- Time [h]
    	- x
    	- x [m]
    	- x_n
    	- x_n [m]
    	- x_s
    	- x_s [m]
    	- x_p
    	- x_p [m]
    	- r_n
    	- r_n [m]
    	- r_p
    	- r_p [m]
    	- Total current density
    	- Total current density [A.m-2]
    	- Current [A]
    	- C-rate
    	- Discharge capacity [A.h]
    	- Porosity
    	- Negative electrode porosity
    	- Separator porosity
    	- Positive electrode porosity
    	- X-averaged negative electrode porosity
    	- X-averaged separator porosity
    	- X-averaged positive electrode porosity
    	- Active material volume fraction
    	- Negative electrode active material volume fraction
    	- Separator active material volume fraction
    	- Positive electrode active material volume fraction
    	- X-averaged negative electrode active material volume fraction
    	- X-averaged separator active material volume fraction
    	- X-averaged positive electrode active material volume fraction
    	- Leading-order porosity
    	- Leading-order negative electrode porosity
    	- Leading-order separator porosity
    	- Leading-order positive electrode porosity
    	- Leading-order x-averaged negative electrode porosity
    	- Leading-order x-averaged separator porosity
    	- Leading-order x-averaged positive electrode porosity
    	- Leading-order active material volume fraction
    	- Leading-order negative electrode active material volume fraction
    	- Leading-order separator active material volume fraction
    	- Leading-order positive electrode active material volume fraction
    	- Leading-order x-averaged negative electrode active material volume fraction
    	- Leading-order x-averaged separator active material volume fraction
    	- Leading-order x-averaged positive electrode active material volume fraction
    	- Porosity change
    	- Negative electrode porosity change
    	- Separator porosity change
    	- Positive electrode porosity change
    	- X-averaged porosity change
    	- X-averaged negative electrode porosity change
    	- X-averaged separator porosity change
    	- X-averaged positive electrode porosity change
    	- Leading-order x-averaged negative electrode porosity change
    	- Leading-order x-averaged separator porosity change
    	- Leading-order x-averaged positive electrode porosity change
    	- Volume-averaged velocity
    	- Volume-averaged velocity [m.s-1]
    	- Electrolyte pressure
    	- Negative particle concentration
    	- Negative particle concentration [mol.m-3]
    	- X-averaged negative particle concentration
    	- X-averaged negative particle concentration [mol.m-3]
    	- Negative particle surface concentration
    	- Negative particle surface concentration [mol.m-3]
    	- X-averaged negative particle surface concentration
    	- X-averaged negative particle surface concentration [mol.m-3]
    	- Negative electrode active volume fraction
    	- Negative electrode volume-averaged concentration
    	- Negative electrode volume-averaged concentration [mol.m-3]
    	- Negative electrode average extent of lithiation
    	- Positive particle concentration
    	- Positive particle concentration [mol.m-3]
    	- X-averaged positive particle concentration
    	- X-averaged positive particle concentration [mol.m-3]
    	- Positive particle surface concentration
    	- Positive particle surface concentration [mol.m-3]
    	- X-averaged positive particle surface concentration
    	- X-averaged positive particle surface concentration [mol.m-3]
    	- Positive electrode active volume fraction
    	- Positive electrode volume-averaged concentration
    	- Positive electrode volume-averaged concentration [mol.m-3]
    	- Positive electrode average extent of lithiation
    	- Electrolyte concentration
    	- Electrolyte concentration [mol.m-3]
    	- Electrolyte concentration [Molar]
    	- X-averaged electrolyte concentration
    	- X-averaged electrolyte concentration [mol.m-3]
    	- X-averaged electrolyte concentration [Molar]
    	- Negative electrolyte concentration
    	- Negative electrolyte concentration [mol.m-3]
    	- Negative electrolyte concentration [Molar]
    	- Separator electrolyte concentration
    	- Separator electrolyte concentration [mol.m-3]
    	- Separator electrolyte concentration [Molar]
    	- Positive electrolyte concentration
    	- Positive electrolyte concentration [mol.m-3]
    	- Positive electrolyte concentration [Molar]
    	- X-averaged negative electrolyte concentration
    	- X-averaged negative electrolyte concentration [mol.m-3]
    	- X-averaged separator electrolyte concentration
    	- X-averaged separator electrolyte concentration [mol.m-3]
    	- X-averaged positive electrolyte concentration
    	- X-averaged positive electrolyte concentration [mol.m-3]
    	- Electrolyte flux
    	- Electrolyte flux [mol.m-2.s-1]
    	- Negative current collector temperature
    	- Negative current collector temperature [K]
    	- X-averaged negative electrode temperature
    	- X-averaged negative electrode temperature [K]
    	- Negative electrode temperature
    	- Negative electrode temperature [K]
    	- X-averaged separator temperature
    	- X-averaged separator temperature [K]
    	- Separator temperature
    	- Separator temperature [K]
    	- X-averaged positive electrode temperature
    	- X-averaged positive electrode temperature [K]
    	- Positive electrode temperature
    	- Positive electrode temperature [K]
    	- Positive current collector temperature
    	- Positive current collector temperature [K]
    	- Cell temperature
    	- Cell temperature [K]
    	- X-averaged cell temperature
    	- X-averaged cell temperature [K]
    	- Volume-averaged cell temperature
    	- Volume-averaged cell temperature [K]
    	- Heat flux
    	- Heat flux [W.m-2]
    	- Electrolyte tortuosity
    	- Negative electrolyte tortuosity
    	- Positive electrolyte tortuosity
    	- X-averaged negative electrolyte tortuosity
    	- X-averaged positive electrolyte tortuosity
    	- Separator tortuosity
    	- X-averaged separator tortuosity
    	- Electrode tortuosity
    	- Negative electrode tortuosity
    	- Positive electrode tortuosity
    	- X-averaged negative electrode tortuosity
    	- X-averaged positive electrode tortuosity
    	- Negative particle flux
    	- X-averaged negative particle flux
    	- Positive particle flux
    	- X-averaged positive particle flux
    	- Ohmic heating
    	- Ohmic heating [W.m-3]
    	- Irreversible electrochemical heating
    	- Irreversible electrochemical heating [W.m-3]
    	- Reversible heating
    	- Reversible heating [W.m-3]
    	- Total heating
    	- Total heating [W.m-3]
    	- X-averaged total heating
    	- X-averaged total heating [W.m-3]
    	- Volume-averaged total heating
    	- Volume-averaged total heating [W.m-3]
    	- Negative current collector potential
    	- Negative current collector potential [V]
    	- Current collector current density
    	- Current collector current density [A.m-2]
    	- Leading-order current collector current density
    	- Negative electrode interfacial current density
    	- X-averaged negative electrode interfacial current density
    	- Negative electrode interfacial current density [A.m-2]
    	- X-averaged negative electrode interfacial current density [A.m-2]
    	- Negative electrode interfacial current density per volume [A.m-3]
    	- X-averaged negative electrode interfacial current density per volume [A.m-3]
    	- X-averaged negative electrode total interfacial current density
    	- X-averaged negative electrode total interfacial current density [A.m-2]
    	- X-averaged negative electrode total interfacial current density per volume [A.m-3]
    	- Negative electrode exchange current density
    	- X-averaged negative electrode exchange current density
    	- Negative electrode exchange current density [A.m-2]
    	- X-averaged negative electrode exchange current density [A.m-2]
    	- Negative electrode exchange current density per volume [A.m-3]
    	- X-averaged negative electrode exchange current density per volume [A.m-3]
    	- Negative electrode reaction overpotential
    	- X-averaged negative electrode reaction overpotential
    	- Negative electrode reaction overpotential [V]
    	- X-averaged negative electrode reaction overpotential [V]
    	- Negative electrode surface potential difference
    	- X-averaged negative electrode surface potential difference
    	- Negative electrode surface potential difference [V]
    	- X-averaged negative electrode surface potential difference [V]
    	- Negative electrode open circuit potential
    	- Negative electrode open circuit potential [V]
    	- X-averaged negative electrode open circuit potential
    	- X-averaged negative electrode open circuit potential [V]
    	- Negative electrode entropic change
    	- X-averaged negative electrode entropic change
    	- Positive electrode interfacial current density
    	- X-averaged positive electrode interfacial current density
    	- Positive electrode interfacial current density [A.m-2]
    	- X-averaged positive electrode interfacial current density [A.m-2]
    	- Positive electrode interfacial current density per volume [A.m-3]
    	- X-averaged positive electrode interfacial current density per volume [A.m-3]
    	- X-averaged positive electrode total interfacial current density
    	- X-averaged positive electrode total interfacial current density [A.m-2]
    	- X-averaged positive electrode total interfacial current density per volume [A.m-3]
    	- Positive electrode exchange current density
    	- X-averaged positive electrode exchange current density
    	- Positive electrode exchange current density [A.m-2]
    	- X-averaged positive electrode exchange current density [A.m-2]
    	- Positive electrode exchange current density per volume [A.m-3]
    	- X-averaged positive electrode exchange current density per volume [A.m-3]
    	- Positive electrode reaction overpotential
    	- X-averaged positive electrode reaction overpotential
    	- Positive electrode reaction overpotential [V]
    	- X-averaged positive electrode reaction overpotential [V]
    	- Positive electrode surface potential difference
    	- X-averaged positive electrode surface potential difference
    	- Positive electrode surface potential difference [V]
    	- X-averaged positive electrode surface potential difference [V]
    	- Positive electrode open circuit potential
    	- Positive electrode open circuit potential [V]
    	- X-averaged positive electrode open circuit potential
    	- X-averaged positive electrode open circuit potential [V]
    	- Positive electrode entropic change
    	- X-averaged positive electrode entropic change
    	- Interfacial current density
    	- Interfacial current density [A.m-2]
    	- Interfacial current density per volume [A.m-3]
    	- Exchange current density
    	- Exchange current density [A.m-2]
    	- Exchange current density per volume [A.m-3]
    	- Negative electrode potential
    	- Negative electrode potential [V]
    	- X-averaged negative electrode potential
    	- X-averaged negative electrode potential [V]
    	- Negative electrode ohmic losses
    	- Negative electrode ohmic losses [V]
    	- X-averaged negative electrode ohmic losses
    	- X-averaged negative electrode ohmic losses [V]
    	- Gradient of negative electrode potential
    	- Negative electrode current density
    	- Negative electrode current density [A.m-2]
    	- Negative electrolyte potential
    	- Negative electrolyte potential [V]
    	- Separator electrolyte potential
    	- Separator electrolyte potential [V]
    	- Positive electrolyte potential
    	- Positive electrolyte potential [V]
    	- Electrolyte potential
    	- Electrolyte potential [V]
    	- X-averaged electrolyte potential
    	- X-averaged electrolyte potential [V]
    	- X-averaged negative electrolyte potential
    	- X-averaged negative electrolyte potential [V]
    	- X-averaged separator electrolyte potential
    	- X-averaged separator electrolyte potential [V]
    	- X-averaged positive electrolyte potential
    	- X-averaged positive electrolyte potential [V]
    	- X-averaged electrolyte overpotential
    	- X-averaged electrolyte overpotential [V]
    	- Gradient of negative electrolyte potential
    	- Gradient of separator electrolyte potential
    	- Gradient of positive electrolyte potential
    	- Gradient of electrolyte potential
    	- Electrolyte current density
    	- Electrolyte current density [A.m-2]
    	- Negative electrolyte current density
    	- Negative electrolyte current density [A.m-2]
    	- Positive electrolyte current density
    	- Positive electrolyte current density [A.m-2]
    	- X-averaged concentration overpotential
    	- X-averaged electrolyte ohmic losses
    	- X-averaged concentration overpotential [V]
    	- X-averaged electrolyte ohmic losses [V]
    	- Positive electrode potential
    	- Positive electrode potential [V]
    	- X-averaged positive electrode potential
    	- X-averaged positive electrode potential [V]
    	- Positive electrode ohmic losses
    	- Positive electrode ohmic losses [V]
    	- X-averaged positive electrode ohmic losses
    	- X-averaged positive electrode ohmic losses [V]
    	- Gradient of positive electrode potential
    	- Positive electrode current density
    	- Positive electrode current density [A.m-2]
    	- Electrode current density
    	- Positive current collector potential
    	- Positive current collector potential [V]
    	- Local voltage
    	- Local voltage [V]
    	- Terminal voltage
    	- Terminal voltage [V]
    	- X-averaged open circuit voltage
    	- Measured open circuit voltage
    	- X-averaged open circuit voltage [V]
    	- Measured open circuit voltage [V]
    	- X-averaged reaction overpotential
    	- X-averaged reaction overpotential [V]
    	- X-averaged solid phase ohmic losses
    	- X-averaged solid phase ohmic losses [V]
    	- X-averaged battery open circuit voltage [V]
    	- Measured battery open circuit voltage [V]
    	- X-averaged battery reaction overpotential [V]
    	- X-averaged battery solid phase ohmic losses [V]
    	- X-averaged battery electrolyte ohmic losses [V]
    	- X-averaged battery concentration overpotential [V]
    	- Battery voltage [V]
    	- Terminal power [W]

To help visualise the results, pybamm provides the `pybamm.ProcessedVariable` class, which takes the output of a solver and a variable, and allows the user to evaluate the value of that variable at any given time or $x$ value. These processed variables are automatically created by the solution dictionary.

```python
voltage = solution['Terminal voltage [V]']
c_s_n_surf = solution['Negative particle surface concentration']
c_s_p_surf = solution['Positive particle surface concentration']
```

One we have these variables in hand, we can begin generating plots using a library such as Matplotlib. Below we plot the terminal voltage and surface particle concentrations versus time

```python
t = np.linspace(0,1,250)
f, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(13,4))

ax1.plot(solution.t, voltage(solution.t))
ax1.set_xlabel(r'$t$')
ax1.set_ylabel('Terminal voltage')

ax2.plot(solution.t, c_s_n_surf(t=solution.t, x=0))
ax2.set_xlabel(r'$t$')
ax2.set_ylabel('Negative particle surface concentration')

ax3.plot(solution.t, c_s_p_surf(t=solution.t, x=1))
ax3.set_xlabel(r'$t$')
ax3.set_ylabel('Positive particle surface concentration')

plt.tight_layout()
plt.show()
```

Some of the output variables are defined over space as well as time. Once option to visualise these variables is to use the `interact` slider widget. Below we plot the negative/positive particle concentration over $r$, using a slider to change the current time point

```python
c_s_n = solution['Negative particle concentration']
c_s_p = solution['Positive particle concentration']
r = np.linspace(0,1,100)

def plot_concentrations(t):
    f, (ax1, ax2) = plt.subplots(1, 2 ,figsize=(10,5))
    plot_c_n, = ax1.plot(r, c_s_n_surf(x=r,t=t))
    plot_c_p, = ax2.plot(r, c_s_p_surf(x=r,t=t))
    ax1.set_ylabel('Negative particle concentration')
    ax2.set_ylabel('Positive particle concentration')
    ax1.set_xlabel(r'$r_n$')
    ax2.set_xlabel(r'$r_p$')
    ax1.set_ylim(0, 1)
    ax2.set_ylim(0, 1)
    plt.show()

import ipywidgets as widgets
widgets.interact(plot_concentrations, t=widgets.FloatSlider(min=0,max=0.17,step=0.01,value=0));

```

    interactive(children=(FloatSlider(value=0.0, description='t', max=0.17, step=0.01), Output()), _dom_classes=('…

The QuickPlot class can be used to plot the common set of useful outputs which should give you a good initial overview of the model. The method Quickplot.plot(t) is simply a function like plot_concentrations(t) above. We can therefore either use it statically for a particularl t or employ the slider widget.

```python
quick_plot = pybamm.QuickPlot(solution)
widgets.interact(quick_plot.plot, t=widgets.FloatSlider(min=0,max=quick_plot.max_t,step=0.05,value=0));
```

    interactive(children=(FloatSlider(value=0.0, description='t', max=0.9999999999999999, step=0.05), Output()), _…

## Dimensionless Parameters

In the table below, we provide the dimensionless parameters in the SPM in terms of the dimensional parameters in LCO.csv. We use a superscript * to indicate dimensional quanitities.

| Parameter                 | Expression                              |Interpretation                             |
|:--------------------------|:----------------------------------------|:------------------------------------------|
| $L_{\text{k}}$            | $L_{\text{k}}^*/L^*$                    | Ratio of region thickness to cell thickness|
|$\mathcal{C}_{\text{k}}$   | $\tau_{\text{k}}^*/\tau_{\text{d}}^*$   | Ratio of solid diffusion and discharge timescales |
|$\mathcal{C}_{\text{r,k}}$ |$\tau_{\text{r,k}}^*/\tau_{\text{d}}^*$  |Ratio of reaction and discharge timescales|
|$a_{\text{k}}$             |$a_{\text{k}}^* R_{\text{k}}^*$          | Product of particle radius and surface area density|
|$\gamma_{\text{k}}$        |$c_{\text{k,max}}^*/c_{\text{n,max}}^*$  |Ratio of maximum lithium concentrations in solid|

```python

```


# Regularized Linear Regression and Bias vs Variance

In the first half of the example, we will implement regularized linear regression to predict the amount of water flowing out of a dam using the change of water level in a reservoir. In the next half, you will go through some diagnostics of debugging learning algorithms and examine the effects of bias v.s. variance.

NOTE: The example and sample data is being taken from the "Machine Learning course by Andrew Ng" in Coursera.

```python
# initial imports
import numpy as np
from matplotlib import pyplot as plt
%matplotlib notebook
import seaborn as sns
```

```python
# imports from my models designed for these examples
from models.linear_regression import compute_cost, gradient_descent, compute_gradient, train_linear_reg, validation_curve
from models.data_preprocessing import add_bias_unit, map_feature, feature_normalize
```

## Loading and Visualizing Data

We start the exercise by first loading and visualizing the dataset.
The following code will load the dataset into your environment and plot the data.

We will begin by visualizing the dataset containing historical records on the
change in the water level, x, and the amount of water flowing out of the dam,
y.

This dataset is divided into three parts:

A training set that your model will learn on: X, y

A cross validation set for determining the regularization parameter: Xval, yval

A test set for evaluating performance. These are “unseen” examples which your model did not see during training: Xtest, ytest.

In the following parts, we will implement linear regression and use that to fit a straight line to the data and plot learning curves. Following that, we will implement polynomial regression to find a better fit to the data.

```python
# loading dataset
print("Loading & Visualizing Data")
import scipy.io as sio  # sio for loading matlab file .mat
data = sio.loadmat('data/ex5data1.mat')  # it geneates a dictionary of arrays found in the mat file
X = data['X']  # (12, 1)
y = data['y'][:, 0]  # (12,)
X_val = data['Xval']  # (21, 1)
y_val = data['yval'][:, 0]  # (21,)
X_test = data['Xtest']  # (21, 1)
y_test = data['ytest'][:, 0]  # (21,)
print(X_test.shape)
```

    Loading & Visualizing Data
    (21, 1)

```python
m = X.shape[0]  # number of examples = 12
print("Plotting training data")
plt.rcParams['figure.dpi'] = 200  # setting figure dpi for better quality graphs
sns.set(context="notebook", style="whitegrid")  # graph styling using seaborn

# the following lines generate a figure with 1 subplot
fig, ax = plt.subplots()
ax.scatter(X, y, marker='+', color='r', s=50);
ax.set_xlabel("Change in water level");
ax.set_ylabel('Water Flowing out of dam');
```

### Regualrized linear regression cost and gradient testing

The regularized linear regression has the following cost function:

\begin{equation}J(\theta)=\frac{1}{2 m}\left(\sum_{i=1}^{m}\left(h_{\theta}\left(x^{(i)}\right)-y^{(i)}\right)^{2}\right) + \frac{\lambda}{2 m}\left(\sum_{j=1}^{n} \theta_{j}^{2}\right)\end{equation}

where λ is a regularization parameter which controls the degree of regularization (thus, help preventing overfitting). The regularization term puts a
penalty on the overal cost J. As the magnitudes of the model parameters $\theta_{j}$
increase, the penalty increases as well. Note that you should not regularize
the $\theta_{0}$ term.

Correspondingly, the partial derivative of regularized linear regression’s cost
for $\theta_{j}$ is defined as:

\begin{align*}
& \frac{\delta J(\theta)}{\delta\theta_{0}} = \frac{1}{m}\sum_{i=1}^{m}\left(h_{\theta}\left(x^{(i)}\right)-y^{(i)}\right)x^{(i)}_{j} & \text{for j}&=0 \\
& \frac{\delta J(\theta)}{\delta\theta_{j}} = \left(\frac{1}{m}\sum_{i=1}^{m}\left(h_{\theta}\left(x^{(i)}\right)-y^{(i)}\right)x^{(i)}_{j}\right) + \frac{\lambda}{m}\theta_{j} & \text{for j}&>=1
\end{align*}

Now we are testing the cost and gradient function in the models.linear_regression file

```python
# initializing testing parameters
theta_test = np.array([1,1])
J = compute_cost(add_bias_unit(X), y, theta_test, lamda=1)
grad = compute_gradient(add_bias_unit(X), y, theta_test, lamda=1)
print("Cost at theta [1,1] is {}\n(this value should be 303.993192)".format(J))
print("Gradient at theta [1,1] is {}\n(this value should be [-15.303016, 598.250744])".format(grad.flatten()))

```

    Cost at theta [1,1] is [303.99319222]
    (this value should be 303.993192)
    Gradient at theta [1,1] is [-15.30301567 598.25074417]
    (this value should be [-15.303016, 598.250744])

## Training Linear Regression

For training the linear regression model we are going to define a function which is going to use scipy.optimize.minimize function to find the optimum theta or weights according to the given data.

In this part, we set regularization parameter λ to zero. Because our current implementation of linear regression is trying to fit a 2-dimensional θ, regularization will not be incredibly helpful for a θ of such low dimension. In the later parts of the example, we will be using polynomial regression with regularization.

```python
# adding bias unit to X
bias_added_X = add_bias_unit(X)

# res will point to the result object generated by scipy minimize function.
# cst is the list of thetas after each iteration
res, cst = train_linear_reg(bias_added_X, y, 0)
```

```python
#plotting the data and curve trained by model
fig1, ax1 = plt.subplots()
ax1.scatter(X, y, marker='+', color='r', s=50);
ax1.set_xlabel("Change in water level")
ax1.set_ylabel('Water Flowing out of dam')
ax1.plot(X, bias_added_X@res.x);  # plotting best fit line
```

The best fit line tells us that the model is not a good fit to the data because the data has a non-linear pattern. While visualizing the best fit as shown is one possible way to debug your learning algorithm, it is not always easy to visualize the data and model. In the next section, you will implement a function to generate learning curves that can help you debug your learning algorithm even if it is not easy to visualize the data.

## Bias-variance

    An important concept in machine learning is the bias-variance tradeoff. Models with high bias are not complex enough for the data and tend to underfit, while models with high variance overfit to the training data.

### Learning Curves

We will now implement code to generate the learning curves that will be useful in debugging learning algorithms. **A learning curve plots training and cross validation error as a function of training set size.**

To plot the learning curve, we need a training and cross validation set error for different training set sizes. To obtain different training set sizes, we will use different subsets of the original training set X. Specifically, for a training set size of i, we will use the first i examples (i.e., X(1:i,:) and y(1:i)).
You can use the train_linear_reg function to find the θ parameters.

After learning the θ parameters, we will compute the error on the training and cross validation sets. Recall that the training error for a dataset is defined as

\begin{equation}
J_{train}(\theta) = \frac{1}{2m}\left[\sum_{i=1}^{m}\left( h_{\theta}\left( x^{(i)} \right) - y^{(i)} \right)^{2} \right]
\end{equation}

In particular, note that the training error does not include the regularization term. One way to compute the training error is to use your existing cost function and set λ to 0 only when using it to compute the training error and cross validation error. When you are computing the training set error, make sure you compute it on the training subset (i.e., X(1:n,:) and y(1:n)) (instead of the entire training set). However, for the cross validation error, you should compute it over the entire cross validation set. You should store the computed errors in the vectors error train and error val.

```python
print("Plotting Learning Curves")

lamda = 0
error_train = []
error_val = []
bias_added_X_val = add_bias_unit(X_val)

# computing training and cross validation error using training examples
for i in range(1,m+1):
    print()
    print("processing dataset from 0 to {}".format(i))

    # training the model
    temp_result, _ = train_linear_reg(bias_added_X[0:i, :], y[0:i], lamda)
    temp_theta = temp_result.x

    print("temp_theta is {}".format(temp_theta))

    # storing errors in list
    error_train.append(compute_cost(bias_added_X[0:i, :], y[0:i], temp_theta, lamda))
    error_val.append(compute_cost(bias_added_X_val, y_val, temp_theta, lamda))

    print("processing complete!!")

```

    Plotting Learning Curves

    processing dataset from 0 to 1
    temp_theta is [ 0.00837049 -0.13339852]
    processing complete!!

    processing dataset from 0 to 2
    temp_theta is [3.29303644 0.07271423]
    processing complete!!

    processing dataset from 0 to 3
    temp_theta is [14.15499007  0.53912266]
    processing complete!!

    processing dataset from 0 to 4
    temp_theta is [14.62429855  0.55985727]
    processing complete!!

    processing dataset from 0 to 5
    temp_theta is [17.21375974  0.44974975]
    processing complete!!

    processing dataset from 0 to 6
    temp_theta is [15.40485112  0.45741711]
    processing complete!!

    processing dataset from 0 to 7
    temp_theta is [14.26678005  0.4368173 ]
    processing complete!!

    processing dataset from 0 to 8
    temp_theta is [14.62343987  0.42312838]
    processing complete!!

    processing dataset from 0 to 9
    temp_theta is [13.29660996  0.41299409]
    processing complete!!

    processing dataset from 0 to 10
    temp_theta is [13.85366299  0.37780979]
    processing complete!!

    processing dataset from 0 to 11
    temp_theta is [12.93724155  0.36342988]
    processing complete!!

    processing dataset from 0 to 12
    temp_theta is [13.08790351  0.36777923]
    processing complete!!

```python
print("Generating Learning curve")
plt.plot(np.arange(m), error_train, label="Training Curve")
plt.plot(np.arange(m), error_val, label="Cross Validation Curve")
plt.xlabel("Number of training examples")
plt.ylabel('Error')
plt.legend()
```

In above Figure, you can observe that both the train error and cross validation
error are high when the number of training examples is increased. This reflects a high bias problem in the model – the linear regression model is too simple and is unable to fit our dataset well. In the next section, we will implement polynomial regression to fit a better model for this dataset.
## Polynomial Regression
The problem with our linear model was that it was too simple for the data
and resulted in underfitting (high bias). In this part of the exercise, you will
address this problem by adding more features.
    For use polynomial regression, our hypothesis has the form:

\begin{align*} h_{\theta}(x) & = \theta_{0} + \theta_{1} * (waterLevel) + \theta_{2} *(waterLevel)^{2} + ... + \theta_{p} * (waterLevel)^{p} \\
& = \theta_{0} + \theta_{1}x_{2} + \dots + \theta_{p}x_{p}
\end{align*}

Notice that by defining $x_{1} = (waterLevel), x_{2} = (waterLevel)^{2}, \dots , x_{p} = (waterLevel)^{p}$, we obtain a linear regression model where the features are the
various powers of the original value $(\text{waterLevel})$.
Now, you will add more features using the higher powers of the existing feature x in the dataset. NOw we are going to define a map_feature function, so that the function maps the original training set X of size $m × 1$ into its higher powers. Specifically, when a training set X of size $m × 1$ is passed into the function, the function should return a $m×p$ matrix $X poly$,where column 1 holds the original values of $X$, column 2 holds the values of $X^{2}$, column 3 holds the values of $X^{3}$, and so on. Note that you don’t have to account for the zero-eth power in this function.

```python
max_degree = 8  # maximum degree upto which the polynomial terms are to be formed
# mapping X to polynomial features with degree upto 8
X_poly = map_feature(X, degree=max_degree, bias_unit=False)

# since the values of X_poly vary too much we are normalizing them
# mu is the mean values of individual features
# sigma is the standard daviation of individual features
X_poly, mu, sigma = feature_normalize(X_poly)

# adding bias unit to the polymerised data
X_poly = add_bias_unit(X_poly)

# mapping and normalizing the test dataset with the mu and sigma
X_poly_test = map_feature(X_test, degree=max_degree, bias_unit=False)
X_poly_test, _, __ = feature_normalize(X_poly_test, mean=mu, sigma=sigma)
X_poly_test = add_bias_unit(X_poly_test)

# mapping and normalizing the cross validation dataset with the mu and sigma
X_poly_val = map_feature(X_val, degree=max_degree, bias_unit=False)
X_poly_val, _, __ = feature_normalize(X_poly_val, mean=mu, sigma=sigma)
X_poly_val = add_bias_unit(X_poly_val)

print("First 5 Normalized training examples: ")
with np.printoptions(precision=8, suppress=True, linewidth=150):
    print(X_poly[:5, :])
```

    First 5 Normalized training examples:
    [[ 1.         -0.3782437  -0.78866232  0.19032872 -0.7375913   0.32025197 -0.6171516   0.35983501 -0.53109126]
     [ 1.         -0.8389201   0.0013142  -0.25896174 -0.34156482  0.09754927 -0.45519664  0.26677343 -0.46887381]
     [ 1.          1.43871736  0.61083158  1.30534069  0.25622     1.02186338 -0.01269621  0.79021001 -0.17792698]
     [ 1.          1.4841233   0.73806846  1.4203124   0.41312183  1.1553483   0.13122371  0.91070022 -0.06228954]
     [ 1.         -1.49791929  1.93643966 -2.12774745  2.43510061 -2.51876748  2.71792174 -2.7633169   2.88908182]]

## Training using Polynomial Regression

In this section we are going to train the model using the new features(i.e. X_poly) generated by map_features function.

Keep in mind that even though we have polynomial terms in our feature vector, we are still solving a linear regression optimization problem. The polynomial terms have simply turned into features that we can use for linear regression. We are using the same cost function and gradient that we used for the earlier part of this exercise.

For this part of the exercise, we will be using a polynomial of degree 8. It turns out that if we run the training directly on the projected data, will not work well as the features would be badly scaled (e.g., an example with x = 40 will now have a feature x8 = 408 = 6.5 × 1012). Therefore, we used feature normalization in above section.

```python
lamda = 0

result_poly, cost_lst_poly = train_linear_reg(X_poly, y, lamda)
print("Training of polynomial regressor completed")
print("Training Cost: {}".format(result_poly.fun))
```

    Training of polynomial regressor completed
    Training Cost: [0.03193852]

```python
plt.scatter(X, y)

# generating an array containing X_intercepts
X_intercepts = np.arange(X.min()-15, X.max()+15, 0.05).reshape([-1,1])

# mapping the X-intercepts to max 8 degree
X_intercepts_poly = map_feature(X_intercepts, 8, bias_unit=False)
X_intercepts_poly, _, __ = feature_normalize(X_intercepts_poly, mean=mu, sigma=sigma)
X_intercepts_poly = add_bias_unit(X_intercepts_poly)

# plotting the curve using the thetas found by polynomial data
plt.plot(X_intercepts, X_intercepts_poly@result_poly.x)
plt.ylabel("Water flowing out of Dam")
plt.xlabel("Change in Water level")
plt.title("Polynomial Fit, λ = 0")
```

After learning the parameters $\theta_{polynomial}$, from above Figure, you could see that the polynomial fit is able to follow the datapoints very well - thus, obtaining a low training error. However, the polynomial fit is very complex and even drops off at the extremes. This is
an indicator that the polynomial regression model is overfitting the training
data and will not generalize well.

## Learning Curve for Polynomial Regression
    To better understand the problems with the unregularized (λ = 0) model, we are going to plot learning curve for the new model.

```python
print("Computing Training and Cross Validation Error\n")
lamda = 0
error_train_poly = []
error_val_poly = []

print("Training Data size\tTrain cost\tCross Validation Cost")
# computing training and cross validation error using training examples
for i in range(1,m+1):
    # training the model
    temp_result_poly, _ = train_linear_reg(X_poly[0:i, :], y[0:i], lamda)
    temp_theta_poly = temp_result_poly.x

    # storing errors in list
    error_train_poly.append(compute_cost(X_poly[0:i, :], y[0:i], temp_theta_poly, lamda))
    error_val_poly.append(compute_cost(X_poly_val, y_val, temp_theta_poly, lamda))

    print("\t{}\t\t{:0.6f}\t{:0.6f}".format(i, error_train_poly[-1][0], error_val_poly[-1][0]))

print("processing complete!!")
```

    Computing Training and Cross Validation Error

    Training Data size	Train cost	Cross Validation Cost
    	1		0.000000	161.442168
    	2		0.000000	160.792292
    	3		0.000000	68.190945
    	4		0.000000	84.863965
    	5		0.000000	6.486683
    	6		0.000000	10.782185
    	7		0.000000	27.972688
    	8		0.000004	20.371517
    	9		0.009334	29.825986
    	10		0.039082	20.788307
    	11		0.041092	16.836138
    	12		0.031939	45.501432
    processing complete!!

```python
print("Plotting Learning curve with above generated data")

plt.plot(np.arange(1,m+1), error_train_poly)
plt.plot(np.arange(1,m+1), error_val_poly)
```

    You can see that the learning curve (Figure 5) shows the same effect where the low training error is low, but the cross validation error is high. There is a gap between the training and cross validation errors, indicating a high variance problem.

## Adjusting the regularization parameter

In this section, we will observe how the regularization parameter affects the bias-variance of regularized polynomial regression.

We are now going to modify the lambda paramter(λ) and try λ = 1, 100. For
each of these values, the script should generate a polynomial fit to the data
and also a learning curve.

For λ = 1, you should see a polynomial fit that follows the data trend
well and a learning curve showing that both the cross
validation and training error converge to a relatively low value. This shows
the λ = 1 regularized polynomial regression model does not have the highbias or high-variance problems. In effect, it achieves a good trade-off between
bias and variance.
For λ = 100, you should see a polynomial fit that does not
follow the data well. In this case, there is too much regularization and the model is unable to fit the training data.

### Lambda λ = 1

```python
lamda = 1

result_poly_1, cost_lst_poly_1 = train_linear_reg(X_poly, y, lamda)
print("Training of polynomial regressor completed (λ = 1)")
print("Training Cost: {}".format(result_poly_1.fun))
```

    Training of polynomial regressor completed (λ = 1)
    Training Cost: [6.83046332]

```python
plt.scatter(X, y)

# generating an array containing X_intercepts
X_intercepts = np.arange(X.min()-15, X.max()+15, 0.05).reshape([-1,1])

# mapping the X-intercepts to max 8 degree
X_intercepts_poly = map_feature(X_intercepts, 8, bias_unit=False)
X_intercepts_poly, _, __ = feature_normalize(X_intercepts_poly, mean=mu, sigma=sigma)
X_intercepts_poly = add_bias_unit(X_intercepts_poly)

# plotting the curve using the thetas found by polynomial data
plt.plot(X_intercepts, X_intercepts_poly@result_poly_1.x)
plt.ylabel("Water flowing out of Dam")
plt.xlabel("Change in Water level")
plt.title("Polynomial Fit, λ = 1")
```

```python
print("Computing Training and Cross Validation Error (λ = 1)\n")
lamda = 1
error_train_poly = []
error_val_poly = []

print("Training Data size\tTrain cost\tCross Validation Cost")
# computing training and cross validation error using training examples
for i in range(1,m+1):
    # training the model
    temp_result_poly, _ = train_linear_reg(X_poly[0:i, :], y[0:i], lamda)
    temp_theta_poly = temp_result_poly.x

    # storing errors in list
    error_train_poly.append(compute_cost(X_poly[0:i, :], y[0:i], temp_theta_poly, lamda))
    error_val_poly.append(compute_cost(X_poly_val, y_val, temp_theta_poly, lamda))

    print("\t{}\t\t{:0.6f}\t{:0.6f}".format(i, error_train_poly[-1][0], error_val_poly[-1][0]))

print("processing complete!!")
```

    Computing Training and Cross Validation Error (λ = 1)

    Training Data size	Train cost	Cross Validation Cost
    	1		0.000000	138.846776
    	2		0.070327	143.505063
    	3		17.182727	7.845035
    	4		13.476899	9.212261
    	5		10.781524	9.222846
    	6		9.347014	11.024707
    	7		8.754387	8.290400
    	8		7.682189	8.001082
    	9		7.277291	8.793858
    	10		6.592031	8.528102
    	11		5.994056	8.585953
    	12		6.830463	7.047218
    processing complete!!

```python
print("Plotting Learning curve with above generated data")

plt.plot(np.arange(1,m+1), error_train_poly)
plt.plot(np.arange(1,m+1), error_val_poly)
plt.title("Learning Curve (λ = 1)")
```

### Lambda λ = 100

```python
lamda = 100

result_poly_100, cost_lst_poly_100 = train_linear_reg(X_poly, y, lamda)
print("Training of polynomial regressor completed (λ = 100)")
print("Training Cost: {}".format(result_poly_100.fun))
```

    Training of polynomial regressor completed (λ = 100)
    Training Cost: [65.95095506]

```python
plt.scatter(X, y)

# generating an array containing X_intercepts
X_intercepts = np.arange(X.min()-15, X.max()+15, 0.05).reshape([-1,1])

# mapping the X-intercepts to max 8 degree
X_intercepts_poly = map_feature(X_intercepts, 8, bias_unit=False)
X_intercepts_poly, _, __ = feature_normalize(X_intercepts_poly, mean=mu, sigma=sigma)
X_intercepts_poly = add_bias_unit(X_intercepts_poly)

# plotting the curve using the thetas found by polynomial data
plt.plot(X_intercepts, X_intercepts_poly@result_poly_100.x)
plt.ylabel("Water flowing out of Dam")
plt.xlabel("Change in Water level")
plt.title("Polynomial Fit, λ = 100")
```

```python
print("Computing Training and Cross Validation Error (λ = 1)\n")
lamda = 100
error_train_poly = []
error_val_poly = []

print("Training Data size\tTrain cost\tCross Validation Cost")
# computing training and cross validation error using training examples
for i in range(1,m+1):
    # training the model
    temp_result_poly, _ = train_linear_reg(X_poly[0:i, :], y[0:i], lamda)
    temp_theta_poly = temp_result_poly.x

    # storing errors in list
    error_train_poly.append(compute_cost(X_poly[0:i, :], y[0:i], temp_theta_poly, lamda))
    error_val_poly.append(compute_cost(X_poly_val, y_val, temp_theta_poly, lamda))

    print("\t{}\t\t{:0.6f}\t{:0.6f}".format(i, error_train_poly[-1][0], error_val_poly[-1][0]))

print("processing complete!!")
```

    Computing Training and Cross Validation Error (λ = 1)

    Training Data size	Train cost	Cross Validation Cost
    	1		0.000000	138.846777
    	2		0.114717	144.124099
    	3		112.255156	70.935873
    	4		131.638608	79.137897
    	5		116.062526	66.126942
    	6		108.778507	61.957759
    	7		93.297101	62.018723
    	8		86.349367	61.581277
    	9		80.206754	62.481408
    	10		73.060272	62.979548
    	11		66.883255	63.626911
    	12		65.950955	61.559272
    processing complete!!

```python
print("Plotting Learning curve with above generated data")

plt.plot(np.arange(1,m+1), error_train_poly)
plt.plot(np.arange(1,m+1), error_val_poly)
```

## Selecting λ using cross validation curve
From the previous parts of the exercise, we observed that the value of λ can significantly affect the results of regularized polynomial regression on the training and cross validation set.

In particular, a model without regularization (λ = 0) fits the training set well, but does not generalize. Conversely,a model with too much regularization (λ = 100) does not fit the training set and testing set well. A good choice of λ (e.g., λ = 1) can provide a good fit to the data.

In this section, we will implement training with different lambda values to select the λ parameter. Concretely, we will use a cross validation set to evaluate how good each λ value is. After selecting the best λ value using the cross validation set, we can then evaluate the model on the test set to estimate
how well the model will perform on actual unseen data.

Specifically, we will use the train_linear_reg function to train the model using different values of λ and compute the training error and cross validation error.

**You should try λ in the following range: {0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10}**

```python
lambda_list = [0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]

_, train_cost_lst, cv_cost_lst = validation_curve(X_poly, y, X_poly_val, y_val, lambda_list)

# train_cost_lst - training cost corresponding to lamda values
# cv_cost_lst - cross validation cost corresponding to lamda values

# plotting training cost and cross validation cost
plt.plot(np.array(lambda_list), train_cost_lst, label="Train")
plt.plot(np.array(lambda_list), cv_cost_lst, label="Cross Validation")
plt.xlabel("Lambda")
plt.ylabel("Error")
plt.legend()
#plt.xticks(np.arange(len(lambda_list)),lambda_list)
```

In this
figure, we can see that the best value of λ is around 3. Due to randomness
in the training and validation splits of the dataset, the cross validation error
can sometimes be lower than the training error.

Model Performance Evaluation/Regularized Linear Regression and Bias vs Variance.ipynb