###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, C.D. Cooper, G.F. Forsyth.

# Riding the wave

## Convection problems

Welcome to *Riding the wave: Convection problems*, the third module of  ["Practical Numerical Methods with Python"](https://openedx.seas.gwu.edu/courses/course-v1:MAE+MAE6286+2017/about).

In the [first module](https://github.com/numerical-mooc/numerical-mooc/tree/master/lessons/01_phugoid), we learned about numerical integration methods for the solution of ordinary differential equations (ODEs). The [second module](https://github.com/numerical-mooc/numerical-mooc/tree/master/lessons/02_spacetime) introduced the finite difference method for numerical solution of partial differential equations (PDEs), where we need to discretize both *space* and *time*.

This module explores the convection equation in more depth, applied to a traffic-flow problem.  We already introduced convection in [Lesson 1 of Module 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_01_1DConvection.ipynb). This hyperbolic equation is very interesting because the solution can develop *shocks*, or regions with very high gradient, which are difficult to resolve well with numerical methods.

We will start by introducing the concept of a conservation law, closely related to the convection equation. Then we'll explore different numerical schemes and how they perform when shocks are present.

## Conservation laws

You know from (non relativistic) physics that mass is _conserved_. This is one example of a conserved quantity, but there are others (like momentum and energy) and they all obey a _conservation law_. Let's start with the more intuitive case of conservation of mass.

### Conservation of mass

In any closed system, we know that the mass $M$ in the system does not change, which we can write: $\frac{D\,M}{Dt} =0$. When we change the point of view from a closed system to what engineers call a _control volume_, mass can move in and out of the volume and conservation of mass is now expressed by:

Let's imagine the control volume as a tiny cylinder of cross-section dA and length dx, like in the sketch below.

#### Figure 1. Tiny control volume in the shape of a cylinder.

If we represent the mass density by $\rho$, then mass is equal to $\rho\times$ volume. For simplicity, let's assume that mass flows in or out of the control volume only in one direction, say, the $x$-direction. Express the 1D velocity component by $u$, and conservation of mass for the control volume is translated to a mathematical expression as follows:

$$
\begin{equation}
\frac{\partial}{\partial t}\int_{\text{cv}}\rho \, dV + \int_{\text{cs}}\rho \, u\, dA =0
\end{equation}
$$

where "cv" stands for control volume and "cs" stands for control surface. The first term represents the rate of change of mass in the control volume, and the second term is the rate of flow of mass, with velocity $u$, across the control surface.

Since the control volume is very small, we can take, to leading order, $\rho$ as a uniform quantity inside it, and the first term in equation (1) can be simplified to the time derivative of density multiplied by the volume of the tiny cylinder, $dAdx$:

$$
\frac{\partial}{\partial t}\int_{\text{cv}}\rho \, dV \rightarrow \frac{\partial \rho}{\partial t} dA dx
$$

Now, for the second term in equation (1), we have to do a little more work. The quantity inside the integral is now $\rho u$ and, to leading order, we have to take into consideration that this quantity can change in the distance $dx$. Take $\rho u$ to be the value in the center of the cylinder. Then the flow of mass on each side is illustrated in the figure below, where we use a Taylor expansion of the quantity $\rho u$ around the center of the control volume (to first order).

#### Figure 2. Flux terms on the control surfaces.

Subtracting the negative flux on the left to the positive flux on the right, we arrive at the total flux of mass across the control surfaces, the second term in equation (1):

$$
\int_{\text{cs}}\rho \, u\, dA \rightarrow \frac{\partial}{\partial x}(\rho u) dA dx
$$

We can now put together the equation of conservation of mass for the tiny cylindrical control volume, which after diving by $dA dx$ is:

$$
\begin{equation}
\frac{\partial \rho}{\partial t} + \frac{\partial}{\partial x}(\rho u)=0
\end{equation}
$$

This is the 1D mass conservation equation in differential form. If we take $u$ to be a constant and take it out of the spatial derivative this equation looks the same as the first PDE we studied: the linear convection equation in [Lesson 1 of Module 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_01_1DConvection.ipynb).
But in the form shown above, it is a typical _conservation law_. The term under the spatial derivative is called the _flux_, for reasons that should be clear from our discussion above: it represents amounts of the conserved quantity flowing across the boundary of the control volume.

##### Dig deeper

You can follow the derivation of the full three-dimensional equation of conservation of mass for a flow on this screencast by Prof. Barba (duration 12:47).

```python
from IPython.display import YouTubeVideo
YouTubeVideo('35unQgSaT88')
```

### General conservation laws

All conservation laws express the same idea: the variation of a conserved quantity inside a control volume is due to the total flux of that quantity crossing the boundary surface (plus possibly the effect of any sources inside the volume, but let's ignore those for now).

The _flux_ is a fundamental concept in conservation laws: it represents the amount of the quantity that crosses a surface per unit time. Our discussion above was limited to flow in one dimension, but in general the flux has any direction and is a vector quantity. Think about this: if the direction of flow is parallel to the surface, then no quantity comes in or out. We really only care about the component of flux perpendicular to the surface. Mathematically, for a vector flux $\vec{F}$, the amount of the conserved quantity crossing a small surface element is:

$$
\vec{F}\cdot d\vec{A}
$$

where $d\vec{A}$ points in the direction of the outward normal to the surface. A general conservation law for a quantity $e$ is thus (still ignoring possible sources):

$$
\begin{equation}
\frac{\partial}{\partial t}\int_{\text{cv}}e \, dV + \oint_{\text{cs}}\vec{F}\cdot d\vec{A} =0
\end{equation}
$$

To obtain a differential form of this conservation equation, we can apply the theorem of Gauss to the second integral, which brings the gradient of $\vec{F}$ into play. One way to recognize a conservation law in differential form is that the _fluxes appear only under the gradient operator_.

Recall the non-linear convection equation from [Lesson 1 of Module 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_01_1DConvection.ipynb). It was:

$$
\begin{equation}
\frac{\partial u}{\partial t} + u \frac{\partial u}{\partial x} = 0
\end{equation}
$$

If we look closely at the spatial derivative, we can rewrite this equation as

$$
\begin{equation}
\frac{\partial u}{\partial t} + \frac{\partial}{\partial x} \left(\frac{u^2}{2} \right) = 0
\end{equation}
$$

which is the *conservation form* of the non-linear convection equation, with flux $F=\frac{u^2}{2}$.

## Traffic flow model

We've all experienced it: as rush hour approaches certain roads in or out of city centers start getting full of cars, and the speed of travel can reduce to a crawl. Sometimes, the cars stop altogether. If you're a driver, you know that the more cars on the road, the slower your trip will flow.

Traffic flow models seek to describe these everyday experiences with mathematics, to help engineers design better road systems.

Let's review the [Lighthill-Whitham-Richards](http://en.wikipedia.org/wiki/Macroscopic_traffic_flow_model) traffic model that was offered as an exercise at the end of Module 2. This model considers cars with a continuous *traffic density* (average number of cars per unit length of road) rather than keeping track of them individually. If $\rho(x)=0$, there are no cars at that point $x$ of the road.  If $\rho(x) = \rho_{\rm max}$, traffic is literally bumper to bumper.

If the number of cars on a bounded stretch of road changes, it means that cars are entering or leaving the road somehow. _Traffic density obeys a conservation law_ (!) where the flux is the number of cars leaving the road per unit time. It is given by $F=\rho u$—as with mass conservation, flux equals density times velocity. But don't forget your experience on the road: the speed of travel depends on the car density. Here, $u$ refers not to the speed of each individual car, but to the _traffic speed_ at a given point of the road.

You know from experience that with more cars on the road, the speed of travel decreases. It is also true that if you are traveling at fast speed, you are advised to leave a larger gap with cars ahead. These two considerations lead us to propose a monotonically decreasing $u=u(\rho)$ function. As a first approximation, we may consider the linear function:

$$
\begin{equation}
u(\rho) = u_{\rm max} \left(1-\frac{\rho}{\rho_{\rm max}}\right)
\end{equation}
$$

#### Figure 3. Traffic speed vs. traffic density.

The linear model of the behavior of drivers satisfies these experimental observations:
1.  All drivers will approach a maximum velocity $u_{max}$ when the road is empty.
2.  If the road is completely jampacked ($\rho \rightarrow \rho_{max}$), velocity goes to zero.

That seems like a reasonable approximation of reality!

Applying a conservation law to the vehicle traffic, the traffic density will obey the following transport equation:

$$
\begin{equation}
\frac{\partial \rho}{\partial t} + \frac{\partial F}{\partial x} = 0
\end{equation}
$$

where $F$ is the *traffic flux*, which in the linear traffic-speed model is given by:

$$
\begin{equation}
F = \rho u_{\rm max} \left(1-\frac{\rho}{\rho_{\rm max}}\right)
\end{equation}
$$

We can now use our numerical kung-fu to solve some interesting traffic situations, and check if our simple model gives realistic results!

### Green light!

Let's say that we are examining a road of length $4$ where the speed limit is $u_{\rm max}=1$, fitting $10$ cars per unit length $(\rho_{\rm max}=10)$. Now, imagine we have an intersection with a red light at $x=2$.  At the stoplight, traffic is bumper-to-bumper, and the traffic density decreases linearly to zero as we approach the beginning of our road. Ahead of the stoplight, the road is clear.

Mathematically, we can represent this situation with the following initial condition:

$$
\begin{equation}
\rho(x,0) = \left\{
\begin{array}{cc}
\rho_{\rm max}\frac{x}{2} & 0 \leq x < 2  \\
0 & 2 \leq x \leq 4 \\
\end{array}
\right.
\end{equation}
$$

Let's see what a plot of that looks like.

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
```

```python
# Set the font family and size to use for Matplotlib figures.
pyplot.rcParams['font.family'] = 'serif'
pyplot.rcParams['font.size'] = 16
```

```python
def rho_green_light(x, rho_light):
    """
    Computes the "green light" initial condition.
    It consists of a shock with a linear distribution behind it.

    Parameters
    ----------
    x : numpy.ndaray
        Locations on the road as a 1D array of floats.
    rho_light : float
        Car density at the stoplight.

    Returns
    -------
    rho : numpy.ndarray
        The initial car density along the road as a 1D array of floats.
    """
    rho = numpy.zeros_like(x)
    mask = numpy.where(x < 2.0)
    rho[mask] = rho_light * x[mask] / 2.0
    return rho
```

```python
# Set parameters.
nx = 81  # number of locations on the road
L = 4.0  # length of the road
dx = L / (nx - 1)  # distance between two consecutive locations
nt = 30  # number of time step to compute
u_max = 1.0  # maximum speed allowed on the road
rho_max = 10.0  # maximum car density allowed on the road
rho_light = 10.0  # car density at the stoplight

# Discretize the road.
x = numpy.linspace(0.0, L, num=nx)

# Compute the initial traffic density.
rho0 = rho_green_light(x, rho_light)
```

```python
# Plot the initial car density on the road.
pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel(r'$x$')
pyplot.ylabel(r'$\rho$')
pyplot.grid()
pyplot.plot(x, rho0, color='C0', linestyle='-', linewidth=2)
pyplot.xlim(0.0, L)
pyplot.ylim(-0.5, 11.0);
```

**How does the traffic behave once the light turns green?** Cars should slowly start moving forward: the density profile should move to the right. Let's see if the numerical solution agrees with that!

Before we start, let's define a function to calculate the traffic flux. We'll use it in each time step of our numerical solution.

```python
def flux(rho, u_max, rho_max):
    """
    Computes the traffic flux F = V * rho.

    Parameters
    ----------
    rho : numpy.ndarray
       Traffic density along the road as a 1D array of floats.
    u_max : float
        Maximum speed allowed on the road.
    rho_max : float
        Maximum car density allowed on the road.

    Returns
    -------
    F : numpy.ndarray
        The traffic flux along the road as a 1D array of floats.
    """
    F = rho * u_max * (1.0 - rho / rho_max)
    return F
```

### Forward-time/backward-space

Start by using a forward-time, backward-space scheme, like you used in Module 2. The discretized form of our traffic model is:

$$
\begin{equation}
\frac{\rho^{n+1}_i- \rho^n_{i}}{\Delta t}+ \frac{F^n_{i}-F^n_{i-1}}{\Delta x}=0
\end{equation}
$$

Like before, we'll step in time via a for-loop, and we'll operate on all spatial points simultaneously via array operations. In each time step, we also need to call the function that computes the flux. Here is a function that implements in code the forward-time/backward-space difference scheme:

```python
def ftbs(rho0, nt, dt, dx, bc_value, *args):
    """
    Computes the history of the traffic density on the road
    at a certain time given the initial traffic density.

    Parameters
    ----------
    rho0 : numpy.ndarray
        The initial car density along the road
        as a 1D array of floats.
    nt : integer
        The number of time steps to compute.
    dt : float
        The time-step size to integrate.
    dx : float
        The distance between two consecutive locations.
    bc_value : float
        The constant density at the first station.
    args : list or tuple
        Positional arguments to be passed to the flux function.

    Returns
    -------
    rho_hist : list of numpy.ndarray objects
        The history of the car density along the road.
    """
    rho_hist = [rho0.copy()]
    rho = rho0.copy()
    for n in range(nt):
        # Compute the flux.
        F = flux(rho, *args)
        # Advance in time.
        rho[1:] = rho[1:] - dt / dx * (F[1:] - F[:-1])
        # Set the left boundary condition.
        rho[0] = bc_value
        # Record the time-step solution.
        rho_hist.append(rho.copy())
    return rho_hist
```

We're all good to go!

**Note:** The code above saves the complete traffic density at each time step—we'll use that in a second to create animations with our results.

Running the numerical solution is easy now: we just need to call the function for evolving the initial condition with the forward-time/backward-space scheme.

Let's see how that looks.

```python
# Set the time-step size based on CFL limit.
sigma = 1.0
dt = sigma * dx / u_max  # time-step size

# Compute the traffic density at all time steps.
rho_hist = ftbs(rho0, nt, dt, dx, rho0[0], u_max, rho_max)
```

Let's now create an animation of the traffic flow density using the `animation` module of Matplotlib.

```python
from matplotlib import animation
from IPython.display import HTML
```

```python
# Plot the initial traffic density.
fig = pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel(r'$x$')
pyplot.ylabel(r'$\rho$')
pyplot.grid()
line = pyplot.plot(x, rho0,
                    color='C0', linestyle='-', linewidth=2)[0]
pyplot.xlim(0.0, L)
pyplot.ylim(-0.5, 11.0)
fig.tight_layout()
```

```python
def update_plot(n, rho_hist):
    """
    Update the line y-data of the Matplotlib figure.

    Parameters
    ----------
    n : integer
        The time-step index.
    rho_hist : list of numpy.ndarray objects
        The history of the numerical solution.
    """
    fig.suptitle('Time step {:0>2}'.format(n))
    line.set_ydata(rho_hist[n])
```

```python
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
                               frames=nt, fargs=(rho_hist,),
                               interval=100)
# Display the video.
HTML(anim.to_html5_video())
```

**Yikes! The solution is blowing up.** This didn't happen in your traffic-flow exercise (coding assignment) for Module 2! (Thankfully.) What is going on? Is there a bug in the code?

No need to panic. Let's take a closer look at the equation we are solving:

$$
\begin{equation}
\frac{\partial \rho}{\partial t} + \frac{\partial F}{\partial x} = 0
\end{equation}
$$

Using the chain rule of calculus, rewrite is as follows:

$$
\begin{equation}
\frac{\partial \rho}{\partial t} + \frac{\partial F}{\partial \rho} \frac{\partial \rho}{\partial x} = 0
\end{equation}
$$

This form of the equation looks like the nonlinear convection equation from [Lesson 1 of Module 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_01_1DConvection.ipynb), right? This is a wave equation where the wave speed is $u_{\rm wave} = \frac{\partial F}{\partial\rho}$. That term is:

$$
\begin{equation}
u_{\rm wave} = \frac{\partial F}{\partial \rho} = u_{\rm max} \left( 1-2\frac{\rho}{\rho_{\rm max}} \right)
\end{equation}
$$

See how the wave speed changes sign at $\rho = \rho_{\rm max}/2$? That means that for the initial conditions given for the green-light problem, the part of the wave under $\rho = \rho_{\rm max}/2$ will want to move right, whereas the part of the wave over this mark, will move left!

There is no real problem with that in terms of the model, but a scheme that is backward in space is *unstable* for negative values of the wave speed.

## Upwind schemes

Maybe you noticed that the backward-space discretization is spatially biased: we include the points $i$ and $i-1$ in the formula. Look again at the stencil and you'll see what we mean.

#### Figure 4. Stencil of forward-time/backward-space.

In fact, the spatial bias was meant to be in the direction of propagation of the wave—this was true when we solved the convection equation (with positive wave speed $c$), but now we have some problems. Discretization schemes that are biased in the direction that information propagates are called _upwind schemes_.

Remember when we discussed the characteristic lines for the linear convection equation in [lesson 1 of the previous module](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_01_1DConvection.ipynb)? Compare the sketch of the characteristic lines with the stencil above. The point is that there is an inherent directionality in the physics, and we want the numerical scheme to have the same directionality. This is one example of _choosing an appropriate scheme_ for the physical problem.

If we wanted to solve the convection equation with negative wave speed, $c<0$, we would need a spatial bias "slanting left," which we would obtain by using the points $i$ and $i+1$ in the formula.

But if we have waves traveling in both directions, we are in a bit of a bind. One way to avoid this problem with our traffic flow model is to simply use an initial condition that doesn't produce negative speed. This should work. But later we will learn about other numerical schemes that are able to handle waves in both directions.

Just for a sanity check, let's try the forward-time/backward-space scheme with the initial conditions

\begin{equation}\rho(x,0) = \left\{ \begin{array}{cc}
2.5 x & 0 \leq x < 2  \\
0 & 2 \leq x \leq 4 \\ \end{array} \right.\end{equation}

If all values of $\rho \leq \rho_{\rm max}/2$, then $\frac{\partial F}{\partial \rho}$ is positive everywhere. For these conditions, our forward-time/backward-space scheme shouldn't have any trouble, as all wave speeds are positive.

```python
# Modify some parameters.
nt = 40  # number of time step to compute
rho_light = 5.0  # car density at the stoplight

# Compute the initial traffic density.
rho0 = rho_green_light(x, rho_light)

# Plot the initial traffic density.
fig = pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel(r'$x$')
pyplot.ylabel(r'$\rho$')
pyplot.grid()
line = pyplot.plot(x, rho0,
                   color='C0', linestyle='-', linewidth=2)[0]
pyplot.hlines(rho_max / 2.0, 0.0, L,
              label=r'$\rho_{max} / 2$',
              color='black', linestyle='--', linewidth=2)
pyplot.legend()
pyplot.xlim(0.0, L)
pyplot.ylim(-0.5, 11.0)
fig.tight_layout()
```

```python
# Compute the traffic density at all time steps.
rho_hist = ftbs(rho0, nt, dt, dx, rho0[0], u_max, rho_max)
```

```python
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
                               frames=nt, fargs=(rho_hist,),
                               interval=100)
# Display the animation.
HTML(anim.to_html5_video())
```

Phew!  It works! Try this out yourself with different initial conditions.

Also, you can easily create a new function `ftfs` to do a forward-time/forward-space scheme, which is stable for negative wave speeds. Unfortunately, forward in space is unstable for positive wave speeds.  If you don't want it blowing up, make sure the wave speed is negative everywhere: $u_{\rm wave} = \frac{\partial F}{\partial \rho} < 0 \ \forall \ x$.

Look at that solution again, and you'll get some nice insights of the real physical problem. See how on the trailing edge, a shock is developing? In the context of the traffic flow problem, a shock is a sign of a traffic jam: a region where traffic is heavy and slow next to a region that is free of cars. In the initial condition, the cars in the rear end of the triangle see a mostly empty road (traffic density is low!).  They see an empty road and speed up, accordingly.  The cars in the peak of the triangle are moving pretty slowly because traffic density is higher there.  Eventually the cars that started in the rear will catch up with the rest and form a traffic jam.

## Beware the CFL!

[Lesson 2 of Module 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_02_CFLCondition.ipynb) discusses the CFL condition for the linear convection equation. To refresh your memory, for a constant wave speed $u_{\rm wave} = c$:

$$
\begin{equation}
\sigma = c\frac{\Delta t}{\Delta x} < 1
\end{equation}
$$

What happens for non-linear equations? The wave speed is space- and time-dependent, $u_{\rm wave} = u_{\rm wave}(x,t)$, and the CFL condition needs to apply for every point in space, at every instant of time. We just need $\sigma>1$ in one spot, for the whole solution to blow up!

Let's generalize the CFL condition to

$$
\begin{equation}
\sigma = \max\left[ \left| u_{\rm wave} \right| \frac{\Delta t}{\Delta x} \right] < 1
\end{equation}
$$

which in our case is

$$
\begin{equation}
\sigma = \max\left[ u_{\rm max} \left| 1-\frac{2 \rho}{\rho_{\rm max}} \right| \frac{\Delta t}{\Delta x} \right] < 1
\end{equation}
$$

Here, the closer $\rho$ is to zero, the more likely it is to be unstable.

### Green light and CFL

We know that the green-light problem with density at the stop light $\rho = \rho_{\rm light} = 4$ is stable using a forward-time/backward -space scheme. Earlier, we used $u_{\rm max} = 1$, and $\Delta t/\Delta x=1$, which gives a CFL $= 1$, when $\rho = 0$.

What if we change the conditions slightly, say $u_{\rm max} = 1.1$?

```python
# Set parameters.
rho_light = 4.0
u_max = 1.1

# Compute the initial traffic density.
rho0 = rho_green_light(x, rho_light)

# Compute the traffic density at all time steps.
rho_hist = ftbs(rho0, nt, dt, dx, rho0[0], u_max, rho_max)

# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
                               frames=nt, fargs=(rho_hist,),
                               interval=100)
# Display the video.
HTML(anim.to_html5_video())
```

That failed miserably! Only by changing $u_{\rm max}$ to $1.1$, even an algorithm that we know is stable for this problem, fails. Since we kept $\Delta t/\Delta x=1$, the CFL number for $\rho=0$ is $1.1$. See where the instability begins? Beware the CFL!

## References

* Neville D. Fowkes and John J. Mahony, *"An Introduction to Mathematical Modelling,"* Wiley & Sons, 1994. Chapter 14: Traffic Flow.

* M. J. Lighthill and G. B. Whitham (1955), On kinematic waves. II. Theory of traffic flow and long crowded roads, _Proc. Roy. Soc. A_, Vol. 229, pp. 317–345. [PDF from amath.colorado.edu](https://amath.colorado.edu/sites/default/files/2013/09/1710796241/PRSA_Lighthill_1955.pdf), checked Oct. 14, 2014. [Original source](http://rspa.royalsocietypublishing.org/content/229/1178/317.short) on the Royal Society site.

---

###### The cell below loads the style of the notebook.

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<link href='https://fonts.googleapis.com/css?family=Source+Code+Pro' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
    font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
    font-family: "Source Code Pro";
    font-size: 90%;
}

.alert-box {
    padding:10px 10px 10px 36px;
    margin:5px;
}

.success {
    color:#666600;
    background:rgb(240,242,229);
}
</style>


2019年度
線形代数　I　講座資料

資料番号　No.3

## [3]　平面と直線の方程式

####  直線の方程式  (2次元)　平行なベクトルを使うとき

直線を通る１つの固定点をA$（A_x,A_y）$とし、直線に平行な１つの方向ベクトルを$ \boldsymbol{a}=(a_x,a_y) $とする

直線上を移動する任意の点を$P(x,y)$とすると

$\vec{AP}=(x , y)-(A_x , A_y)= (x - A_x,y - A_y)$

t: 実数とする　ベクトル$\vec{AP}と\boldsymbol{a}$が平行な条件は
$\vec{AP}=t \boldsymbol{a}$である

$ (x,y) -(A_x , A_y) = t (a_x,a_y )$　または　$ (x,y) =(A_x , A_y) + t (a_x,a_y )$

$(x ,y)=(A_x+ta_x,A_y+ta_y ) $より

$x =A_x+ta_x,  y= A_y+ta_y  $  ,または  $x -A_x = ta_x,  y- A_y =ta_y  $

$t$で共通にしてみると

$$t= \frac{ x - A_x}{a_x}=  \frac{y - A_y}{a_y}$$

上式より　$$ y= \frac{a_y}{a_x}(x-A_x)+A_y $$ とも表される

fig-3.png

###### 垂直なベクトルを使うとき   直線の方程式  (2次元)

直線を通る１つの固定点をA$（A_x,A_y）$とし、直線に垂直な１つの方向ベクトルを$ \boldsymbol{n}=(n_x,n_y) $とする

直線上を移動する任意の点を$P(x,y)$とすると

$\vec{AP}=(x , y)-(A_x , A_y)= (x - A_x,y - A_y)$

直交条件より

$\vec{AP}\cdot \boldsymbol{n} =0 $である

$(x - A_x,y - A_y)\cdot(n_x,n_y)=0$

$$ n_x(x - A_x)+n_y(y - A_y) =0 $$

または、$ n_y \neq 0 $のとき　$$ y = \left(-\frac{n_x}{n_y}\right)x + \left(\frac{n_x}{n_y}A_x + A_y\right) $$

fig-3-1.png

##  直線の方程式  (3次元　空間)

直線を通る１つの固定点をA$（A_x,A_y,A_z）$とし、直線に平行な１つの方向ベクトルを（$ \boldsymbol{a}=(a_x,a_y,a_z) $　）とする

直線上を移動する任意の点を$P(x,y,z)$とすると

$\vec{AP}=(x,y,z)-(A_x , A_y,A_z)= (x - A_x,y - A_y.z-A_z)$

t: 実数とする　ベクトル$\vec{AP}と\boldsymbol{a}$が平行な条件は
$\vec{AP}=t \boldsymbol{a}$である

$ (x,y,z)-(A_x , A_y,A_z) = t (a_x,a_y,a_z)$　または　$ (x,y,z)=(A_x , A_y,A_z) + t (a_x,a_y,a_z)$

$(x ,y)=(A_x+ta_x,A_y+ta_y ,A_z+ta_z) $より

$x =A_x+ta_x,  y= A_y+ta_y , z= A_z+ta_z  $  ,または  $x -A_x = ta_x,  y- A_y =ta_y ,  z- A_z =ta_z  $

$t$で共通にしてみると

$$t= \frac{x - A_x}{a_x}=  \frac{y - A_y}{a_y} =  \frac{z - A_z}{a_z}$$

と表される

### 直線の方程式

### $$ (x,y,z)=(A_x , A_y,A_z) + t (a_x,a_y,a_z) \tag{1.8}$$

### $$ \frac{x - A_x}{a_x}=  \frac{y - A_y}{a_y} =  \frac{z - A_z}{a_z} \tag{1.9}$$

直線の方程式  (2次元)  上にあるものと同じ

##  平面の方程式 （3次元　空間）

平面に固定する１つの点A$（A_x,A_y,A_z）$とし、平面に垂直な１つの方向ベクトルを$ \boldsymbol{n}=(n_x,n_y,n_z) $とする

平面上を移動する任意の点を$P(x,y,z)$とすると

$\vec{AP}=(x,y,z)-(A_x , A_y,A_z)= (x - A_x,y - A_y.z-A_z)$

内積の直交条件より

$ \vec{AP}\cdot \boldsymbol{n} = 0 $ (実数でのゼロ)　より

$ (x - A_x,y - A_y.z-A_z)\cdot (n_x,n_y,n_z) =0 $

### $$ n_x(x - A_x)+n_y(y - A_y)+n_z(z-A_z)=0  \tag{1.7}$$

原点(0,0,0)を通る場合には、
$ ax+by+cz =0$　の形になり

一般的な平面の方程式は、
$ ax+by+cz =d$　の形になる

fig-3-2.png

```python
from sympy import *
init_printing()
# init_session()
```

```python
import numpy as np
from matplotlib import pyplot
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
```

```python
# 問１．２０　（２）
# 点(1, 1, 1),　点(5, 7, 9)　2点を通る直線の方程式

p1, p2 = Point3D(1, 1, 1), Point3D(5, 7, 9)
l1 = Line3D(p1, p2)
l1.equation()
```

```python
#  直線の方程式の例点（1,2,0）平行なベクトル（2,3,4)のとき

from sympy import *
init_printing()

%matplotlib notebook
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import cm

x = np.arange(-5, 5, 0.25)
y = np.arange(-5, 5, 0.25)
X, Y = np.meshgrid(x, y)

# ２つの平面の式より
#Z1 = -X +2*Y-3
#Z2 = (X +2*Y-5)/2

#  交線を求めた結果より，3次元の直線ｚ＝ｆ（ｘ，ｙ２（ｘ））として求めた結果を利用している
y2=(3/2)*x +(1/2)
z =2*(x-1)

fig = plt.figure()
ax = Axes3D(fig)
ax.plot(x, y2, z, "-o", color="#00aa00", ms=4, mew=0.5)

x=1
y=2
z =0
u =2
v =3
w= 4

ax.quiver(x, y, z, u, v, w, length=6, normalize=True, color='red',linewidth = 6.0)

x = [1]
y = [2]
z = [0]
ax.scatter(x, y, z,s=200, c='b', marker='o', label='test')

plt.show()

```

    <IPython.core.display.Javascript object>

```python
# 平面の方程式　点A(１，１，１，)　法線ベクトル n=（１，ー２，３）

from sympy import *
init_printing()

%matplotlib notebook
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import cm

x = np.arange(-5, 5, 0.25)
y = np.arange(-5, 5, 0.25)
X, Y = np.meshgrid(x, y)

#平面
Z = -(1/3)*(X -2*Y-2)

fig = plt.figure()
ax = Axes3D(fig)

#fig = plt.figure()
#ax = fig.add_subplot(111,projection='3d')
ax.set_aspect('equal')

x=0
y=0
z =0
u =1
v=-2
w=3

#ax.scatter(x, y, z, c='r', marker='^', label='test')
# plt.plot(ax,ay,az)

ax.plot_wireframe(X, Y, Z, rstride=4, cstride=4,  color='blue')
ax.quiver(x, y, z, u, v, w, length=5, normalize=True, color='green',linewidth = 4.0)

x = [1]
y = [1]
z = [1]
ax.scatter(x, y, z,s=200, c='r', marker='o', label='test')

#ax.legend()
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
plt.show()
```

    <IPython.core.display.Javascript object>

```python

```

```python

```


Calculator

Using *Python* as an Advanced Calculator.

- [Overview](#Overview)
- [Arithmetic operations](#Arithmetic-operations)
- [Order of operations](#Order-of-operations)
- [Assignment statements](#Assignment-statements)
- [Variable names](#Variable-names)
- [Expressions and statements](#Expressions-and-statements)
- [Code comments](#Code-comments)
- [Flow of execution](#Flow-of-execution)
- [Nbtutor](#Nbtutor)
- [Notebook namespace](#Notebook-namespace)
- [Debugging](#Debugging)
- [Glossary](#Glossary)
- [Exercises](#Exercises)

## Overview

For each *Code Cell* there is a special prompt (`In [#]:`), where `#` is a command or execution reference number.
*Python* instructions (code) can be typed into these *Code Cell*s and executed by selecting the *Code Cell* and pressing *Shift + Enter*.
The output from the *Code Cell* will then be displayed (if the code requires it).

```python
2 + 3
```

When the *Code Cell* above is executed (by pressing *Shift + Enter*) the answer `5` is calculated and displayed as the output.
The special prompt (`Out [#]:`) and the output of `5` is displayed to indicate that this *Python* instruction was computed successfully.

 More Information

Line numbers, between the input prompt (`In [#]`) and the *Python* code, can be toggled (turned on or off) by selecting any *Code Cell* and pressing the *L* key or by selecting `View` $\to$ `Toggle Line Numbers` in the menu bar above. Line numbers will be used through out this book for explanation purposes.

## Arithmetic operations

*Python* provides **operators**, which are special symbols that represent computations like addition and multiplication and *Python* can be used (or abused) as a simple calculator to compute numerical values.
The symbols used for these computations are:

  | Computation    | Math Symbol | *Code Cell*           |
  | --------------:|:-----------:|:----------------------|
  | Addition       | `+`         | `In [#]: 5 + 2`       |
  | Subtraction    | `-`         | `In [#]: 5 - 2`       |
  | Multiplication | `*`         | `In [#]: 5 * 2`       |
  | Division       | `/`         | `In [#]: 5 / 2`       |
  | Power          | $x^y$       | `In [#]: 5 ** 2`      |
  | Grouping       | `()`        | `In [#]: 1 / (2 + 5)` |

 Take Note

The power of $(x^y)$ is computed as `x**y` in *Python* and not traditionally as `x^y`

## Order of operations

When an expression contains more than one operator, the order of evaluation depends on the order of operations. For mathematical operators, *Python* follows mathematical convention. The acronym **BODMAS** or **PEDMAS** is a useful way to remember the rules:

- **B**rackets (or **P**arentheses) have the highest precedence and can be used to force an expression to evaluate in the order you want. Since expressions in parentheses are evaluated first, `2 * (3 - 1)` is `4`, and `(1 + 1)**(5 - 2)` is `8`. You can also use parentheses to make an expression easier to read, as in `(minute * 100) / 60`, even if it doesn’t change the result.

- **O**rders (or **E**xponentiation) has the next highest precedence, so `1 + 2**3` is `9`, not `27`, and `2 * 3**2` is `18`, not `36`.

- **D**ivision and **M**ultiplication have higher precedence than

- **A**ddition and **S**ubtraction. So `2 * 3 - 1` is `5`, not `4`, and `6 + 4 / 2` is `8`, not `5`.

- Operators with the same precedence are evaluated from left to right (except exponentiation). So in the expression `degrees / 2 * pi`, the division happens first and the result is multiplied by `pi`. To divide by $2\pi$, you can use parentheses or write `degrees / 2 / pi`.

I don’t work very hard to remember the precedence of operators. If I can’t tell by looking at the expression, I use parentheses to make it obvious.

 More Information

More information regarding operator priority can be displayed by executing `help('+')`.

```python
help('+')
```

### Example - Falling Object

Consider the example of an object falling from a height above the ground.
The equations describing the motion of this object, assuming constant acceleration, are:

$$
\begin{align}
    a(t) &= g \: \text{(constant)} \\
    v(t) &= \int a(t) \: \mathrm{d}t  =  v_0 + gt \\
    s(t) &= \int v(t) \: \mathrm{d}t = s_0 + v_0t + 0.5 gt^2
\end{align}
$$

How long does an object fall from a $30 \: m$ height, given the following initial values?

- $s_0 = 30 \: m$ (initial height)
- $v_0 = 0 \: m/s$ (initial velocity)
- $g = -9.81 \: m/s^2$ (gravitational acceleration)

The solution to this problem is to solve for $t$ such that:

$$ s(t) = s_0 + v_0t + 0.5 gt^2 = 0 $$

Solving for $t$ gives:

$$ t = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} $$

where
$$
\begin{align}
    a &= 0.5g = 0.5 (-9.81) \\
    b &= v_0 = 0 \\
    c &= s_0 = 30
\end{align}
$$

Substituting `a`, `b`, and `c` and solving for `t` gives the positive root as:

```python
-0 + (0**2 - 4 * 0.5 * -9.81 * 30)**0.5 / (2 * 0.5 * -9.81)
```

and the negative root as:

```python
-0 - (0**2 - 4 * 0.5 * -9.81 * 30)**0.5 / (2 * 0.5 * -9.81)
```

Play around with this example above and try to solve it for different initial values.

## Assignment statements

When using *Python* as a simple calculator each number, executed in a *Code Cell*, was actually an object created in the computers’ memory.

It is important to note that everything in *Python* is an object in memory.
So executing `1 + 2` in a *Code Cell* actually creates three objects in memory:
- the object `1`
- the object `2` and
- the object `3` (the result of `1 + 2`).

Names are used in *Python* to reference these objects in memory. Engineers and scientists usually work with numbers or arrays of numbers, so most of the names dealt with in this textbook will reference numerical objects.

A **variable** is a name that refers to a object and one of the most powerful features of a programming language is the ability to manipulate variables. In *Python*, a name is assigned to an object by typing a name followed by an equal sign followed by the object, for example:

```python
pi = 3.141592653589793
```

Executing the *Code Cell* above does not give any output. If does however create the object `3.141592653589793` in memory and then binds the name `pi` to that object. Thus the name `pi` now references the object (number) `3.141592653589793`.

Keep the following in mind when assigning names to objects (consider the image above):

- The `RHS` of the `=` has to be known objects or values that can be evaluated (e.g. `7 + 3.3`)
- The `RHS` of the `=` is calculated first according to the mathematical priority discussed previously
- The `LHS` of the `=` can only be the chosen name (e.g. `foo`)
- The `LHS` name is then bound to the result of the `RHS`

 Take Note

Mathematically `x = 10.3` may be equivalent to `10.3 = x`, however in *Python* `x = 10.3` is not the same as `10.3 = x`. Executing `10.3 = x` in a *Code Cell* give an error, for example:

```python
10.3 = x
```

### Example - Falling Object

Consider the example of an object falling from a height above the ground.
The equations describing the motion of this object, assuming constant acceleration, are:

$$
\begin{align}
    a(t) &= g \: \text{(constant)} \\
    v(t) &= \int a(t) \: \mathrm{d}t  =  v_0 + gt \\
    s(t) &= \int v(t) \: \mathrm{d}t = s_0 + v_0t + 0.5 gt^2
\end{align}
$$

Calculate $v(t)$ and $s(t)$ for the following values:

- $s_0 = 30 \: m$ (initial height)
- $v_0 = 0 \: m/s$ (initial velocity)
- $g = -9.81 \: m/s^2$ (gravitational acceleration)
- $t = 1.5 \: s$ (time)

```python
s0 = 30
v0 = 0
g = -9.81
t = 1.5

vt = v0 + g*t
st = s0 + v0*t + 0.5*g*t**2

print(vt)
print(st)
```

For the example above it should be evident that names make it easier to read and understand a program / piece of code. Names also allow you to break up a complex calculations into smaller pieces and reference the answers of a calculations for later use.

Play around with this example above and try to solve it for different initial values.

 Take Note

`v(t)` cannot be used directly as a variable name in *Python* as this would be interpreted as a function call, I.e. call (use) the function `v` with the input `t`. Function calls will be discussed in more detail in the following section.

 More Information

The `print` function can be used to print information after the *Code Cell*. For example `print(st)` will print the value that the name `st` references.

## Variable names

Programmers generally choose names for their variables that are meaningful - they document what the variable is used for. Variable names can be as long as you like. They can contain both letters and numbers, but they can’t begin with a number.

Names are case sensitive therefore the name `Value1`, `VALUE1` and `VaLuE1` are distinct and each can be bound to the same or to different objects.

```python
Value1 = 10
VALUE1 = 20
VaLuE1 = 30
print(Value1, VALUE1, VaLuE1)
```

It is legal to use uppercase letters, but it is conventional to use only lower case for variables names. The underscore character, `_`, can also appear in a name and it is often used in names with multiple words, such as `your_name` or `airspeed_of_unladen_swallow`.

Names should be at least 3 characters in length and descriptive and should not be any of the *Python* keywords - execute `help(’keywords’)` in a *Code Cell* to see the list:

```python
help('keywords')
```

If you give a variable an illegal name, you get a syntax error, for example:

```python
76trombones = 50.235
```

```python
more@ = 1000000
```

```python
class = 68
```

`76trombones` is illegal because it begins with a number. `more@` is illegal because it contains an illegal character, `@` and `class` is illegal because it is one of *Python*'s keywords.

## Expressions and statements

A **statement** is a unit of code that has an effect, like creating a variable or displaying an object.

```python
n = 17
print(n)
```

The first line is an assignment statement that gives a value to `n`. The second line is a print statement that displays the value of `n`.

When you type a statement, the interpreter executes it, which means that it does whatever the statement says. In general, statements don’t have objects.

An **expression** is a combination of objects, variables, and operators. An object all by itself is considered an expression, and so is a variable, so the following are all legal expressions:

```python
42
```

```python
n
```

```python
n + 25
```

When you type an expression at the prompt, the interpreter **evaluates** it, which means that it finds the value of the expression. In this example, `n` has the value `17` and `n + 25` has the value `42`.

## Code comments

As programs get bigger and more complicated, they get more difficult to read. Formal languages are dense, and it is often difficult to look at a piece of code and figure out what it is doing, or why.

For this reason, it is a good idea to add notes to your programs to explain in natural language what the program is doing. These notes are called comments, and they start with the `#` symbol:

```python
# compute the percentage of the hour that has elapsed
minute = 12
percentage = (minute * 100) / 60
print(percentage)
```

In this case, the comment appears on a line by itself. You can also put comments at the end of a line:

```python
minute = 12
percentage = (minute * 100) / 60  # percentage of an hour
print(percentage)
```

Everything from the `#` to the end of the line is ignored - it has no effect on the execution of the program.

Comments are most useful when they document non-obvious features of the code. It is reasonable to assume that the reader can figure out *what* the code does; it is more useful to explain *why*.

This comment is redundant with the code and useless:

```python
v = 5  # assign 5 to v
```

This comment contains useful information that is not in the code:

```python
v = 5  # velocity in meters/second.
```

Good variable names can reduce the need for comments, but long names can make complex expressions hard to read, so there is a tradeoff.

## Flow of execution

The order statements run in is called the flow of execution. Execution always begins at the first statement of the *Code Cell*. Statements are run one at a time, in order from top to bottom. Consider the following example:

```python
foo = 2.2
bar = foo + 3.2
print(bar)

foo = 4.2
bar = foo + 3.2
print(bar)
```

Executing the *Code Cell* above does the following:
- Line 1: Creates the object `2.2` in memory and binds the name `foo` to this object
- Line 2:
    - `RHS`: Looks up the value of the name `foo` (which is `2.2`)
    - `RHS`: Creates the object `3.2` in memory
    - `RHS`: Adds the `2.2` object to the `3.2` object to get the `5.4` object (which is also created in memory)
    - `LHS`: The name `bar` is then bound to this `5.4` object
- Line 3: Displays the value of the name `bar` below the *Code Cell*
- Line 5:
    - Creates the object `4.2` in memory and
    - Re-binds the name `foo` to this object (thus removing the previous binding to the `2.2` object)
- Line 6:
    - Evaluates the result of `foo + 3.2` again (similar to line 2) to get the `7.4` object (which is also created in memory)
    - The name `bar` is then re-bound to this `7.4` object
- Line 7: Displays the value of the name `bar` below the *Code Cell*

 Take Note

*Python* executes a *Code Cell* systematically (one line at a time) from the first line in the *Code Cell* to the last line in the *Code Cell*. *Python* does not jump (or look) ahead, and only has information about objects and names created in previous lines of code.

Also keep in mind that the `RHS` of the `=` must be known, I.e. all the quantities (numbers, other names and objects) that appear to the right of the equal sign must be known or can be evalutated. Therefore, make sure that a object has been bound to a name before you use that name to the `RHS` of the `=` sign.

The following example gives an error because not all objects to the `RHS` of the `=` are known when line 1 is executed:

```python
foo = x + 100
x = 0.2
```

## Nbtutor

To expand on the name assignment and flow of execution behaviour in *Python* we will make use of the Notebook extension [nbtutor](https://github.com/lgpage/nbtutor) which can be installed following the instructions given on GitHub (https://github.com/lgpage/nbtutor). Nbtutor will allow us to execute the code inside a *Code Cell* one line at a time to see what *Python* is doing.

To use nbtutor we first need to tell the Notebook what nbtutor is. This is done by executing `%load_ext nbtutor` is a *Code Cell*:

```python
%load_ext nbtutor
```

 More Information

Loading the nbtutor extension only needs to be done once (for each Notebook opened, that uses nbtutor).

Lines starting with a single or double `%` symbol are called magic functions. These lines are not strictly *Python* code statements, but are specific to the *IPython* (**I**nteractive **Python**) interpreter that is used in *Jupyter Notebook*

Execute `%magic` in a *Code Cell* for more information about *IPython* magic functions

```python
%magic
```

To visualize the code (in **any** *Code Cell*) add `%%nbtutor -r -f` as line 1 of the *Code Cell*, for example:

```python
%%nbtutor -r -f
foo = 1
bar = 1
bar = 2
eggs = foo + bar
print(eggs)
```

Execute the *Code Cell* and step through the code execution line-by-line. Executing the *Code Cell* above does the following:
- Line 2: The object `1` is created in memory and the name `foo` is bound to that object. In other words the name `foo` points to the object `1`.
- Line 3:
    - Another name `bar` is also bound to the same object `1` (The object `1` was already created in memory in line 1)
    - So at this point there are two names (`foo` and `bar`) bound to the same object `1`
- Line 4: The object `2` is created in memory and the name `bar` is re-bound to this new object `2` (thus removing the previous binding to the object `1`)
- Line 5:
    - The objects `1` and `2` are added together and the resulting object `3` is created in memory
    - The name `eggs` is then bound to this resulting object `3`.

 Take Note

Objects in memory can have multiply names bound to them. In other words multiple names can point to the same object in memory.

 More Information

If you wish to execute the *Code Cell* without visualizing the code, either comment out or delete the `%%nbtutor` line

The `-r` option, following `%%nbtutor`, resets (clears) the Notebook namespace prior to executing the *Code Cell* and the `-f` option forces the reset without asking for confirmation. Notebook namespace is discussed in more detail in the next section.

## Notebook namespace

A very important concept that needs to be discussed (especially when using *Jupyter Notebooks*) is namespace. In *Python* namespace refers to the space in which a name is created when you execute a *Code Cell*. More formally a namespace is a mapping from names to objects.

You can consider this namespace as a blank piece of paper (at least it is blank when you first open a *Notebook*). Every time you execute a *Code Cell* that defines a name, that name is put onto this piece of paper. You can view all names (and the objects these names point to) on this piece of paper by executing `%whos` in a *Code Cell*, for example:

```python
%whos
```

Each *Notebook* has its own namespace that it puts names onto (when these names are defined by executing *Code Cell*s), this is termed the *global namespace* or *global frame* for the *Notebook*.

 Take Note

It is the action of executing a *Code Cell* with an assignment statement that adds a mapping from the name to the object in *Python*'s namespace and not the action of typing an assignment statement in a *Code Cell*.

To manually clear the namespace you can either restart the kernel by selecting `Kernel` $\to$ `Restart` in the menu bar above; or you can execute `%reset` in a *Code Cell*. This is usually only done for debugging purposes or *nbtutor* visualizations.

```python
%reset
```

 More Information

A `-f` option can be added following `%reset` to force the reset without asking for confirmation

```python
%whos
```

```python
foo = 100
```

```python
%whos
```

Consider the following 3 *Code Cell*s. The *Code Cell*s are designated `One`, `Two`, and `Three` by a code comment at the top of each *Code Cell*.

```python
# Code Cell One
this = 100
```

```python
# Code Cell Two
other = this + that
print(other)
```

```python
# Code Cell Three
that = 1.5
```

If these *Code Cell*s are executed in the order `One`, `Two`, and then `Three` you will get a `NameError` when executing *Code Cell* `Two` because the name `that` has not been defined (has not been added to the *namespace*) after executing *Code Cell* `One`. In more detail the following happens:

- *Code Cell* `One`:
    - Creates the object `100` in memory and binds the name `this` to the `100` object
    - The name `this` is defined in the *global namespace*

- *Code Cell* `Two`:
    - `RHS`: Looks for the name `this` in the *global namespace* and finds it
    - `RHS`: Looks up the value of the name `this` (which is `100`)
    - `RHS`: Looks for the name `that` in the *global namespace*, but does not find it
    - `RHS`: Gives an error because the name `that` was not found (not defined) in the *global namespace*

However, if these *Code Cell*s are executed in the order `One`, `Three`, and then `Two` it works, you will get the output of `101.5`. In more detail the following happens:

- *Code Cell* `One`:
    - Creates the object `100` in memory and binds the name `this` to the `100` object
    - The name `this` is defined in the *global namespace*

- *Code Cell* `Three`:
    - Creates the object `1.5` in memory and binds the name `that` to the `1.5` object
    - The name `that` is also defined in the *global namespace*

- *Code Cell* `Two`:
    - `RHS`: Looks for the name `this` in the *global namespace* and finds it
    - `RHS`: Looks up the value of the name `this` (which is `100`)
    - `RHS`: Looks for the name `that` in the *global namespace* and now finds it
    - `RHS`: Looks up the value of the name `that` (which is `1.5`)
    - `RHS`: Adds `100` to `1.5` to get the `101.5` object (which is created in memory)
    - `LHS`: The name `other` is bound to the resulting `101.5` object
    - The name `other` is also defined in the *global namespace*

 Take Note

This can very be dangerous at times and may lead to unexpected behavior if you are not careful. Consider the following hypothetical case: You have created a program in a *Code Cell* lower down in a *Notebook*; This program relies on several constants in a certain calculation; You have defined all of these constants but one; Your program executes without any errors but does not give the correct output; You spend several hours going over and over and over your code to try find the problem, only to realize the constant you forgot to define was defined in a *Code Cell* earlier on in the *Notebook*. This is a form of semantic (or logic) error.

Because of this reason this textbook will always present examples and code in "self contained" *Code Cell*s. In other words all *Code Cell*s in this textbook will be indepedant of other *Code Cell*s. It is also highly recommended you create independant programs or code in *Code Cell*s that can be executed independantly of other *Code Cell*s, the exception being for functions - which will be discussed at a later stage.

 More Information

The term semantic (or logic) error refers to a program or code that executes without giving any errors, but does not give the correct result or output. The first step in trying to fix a logic error should be to clear the *global namespace* and then only execute the *Code Cell* with your program or code in it.

### Assignment Operators

The same name on the `LHS` of the `=` sign can also be used on the `RHS` of the `=` sign, as long as it is known (can be evaluated).
Remember that everything on the `RHS` of the `=` has to be known or defined whereas there is only one name on the `LHS` of the `=` sign.

```python
number = 5
number = number + 3

print(number)
```

Executing the *Code Cell* above gives the ouput `8`. Looking at each statement more closely, the *Code Cell* above does the following:
- Line 1:
    - Creates the object `5` in memory and binds the the name `number` to the `5` object
    - Because the name `number` is known (defined), it can be used on the `RHS` of the `=` sign.
- Line 2:
    - The `RHS` of the `=` sign gets computed first:
        - The value of the name `number` is looked up (which is `5`)
        - This `5` object is added to the `3` object to get `8` (which is created in memory)
    - The name `number` is then bound to the resulting `8` object

```python
number = 5
number = number + 3
number = number * 2

print(number)
```

Similarly, executing the *Code Cell* above gives the output `16` and the name `number` is now bound to the object `16`.

There is also a short-hand syntax (know as *Assignment Operators*) for modifying variables in this fashion, for example:

```python
number = 5
number += 3
number *= 2

print(number)
```

The following table shows a summary of the assignment operators:

  | Computation    | Example          | Operator    |
  | --------------:|:---------------- |:----------- |
  |       Addition | `foo = foo + 10` | `foo += 10` |
  |    Subtraction | `foo = foo - 10` | `foo -= 10` |
  | Multiplication | `foo = foo * 2`  | `foo *= 2`  |
  |       Division | `foo = foo / 2`  | `foo /= 2`  |
  |          Power | `foo = foo ** 2` | `foo **= 2` |

 More Information

This is a powerful technique which comes in handy in many situations. Using names in this fashion allows you to remember what you have computed so far using the same name, and reduces the number of names needed in your programs. Play around with these operators and make sure you are comfortable with both methods of updating / changing the same variable.

## Debugging

Programmers make mistakes. For whimsical reasons, programming errors are called bugs and the process of tracking them down is called debugging.

Programming, and especially debugging, sometimes brings out strong emotions. If you are struggling with a difficult bug, you might feel angry, despondent, or embarrassed.

There is evidence that people naturally respond to computers as if they were people. When they work well, we think of them as teammates, and when they are obstinate or rude, we respond to them the same way we respond to rude, obstinate people (Reeves and Nass, *The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places*).

Preparing for these reactions might help you deal with them. One approach is to think of the computer as an employee with certain strengths, like speed and precision, and particular weaknesses, like lack of empathy and inability to grasp the big picture.

Your job is to be a good manager: find ways to take advantage of the strengths and mitigate the weaknesses and find ways to use your emotions to engage with the problem, without letting your reactions interfere with your ability to work effectively.

Learning to debug can be frustrating, but it is a valuable skill that is useful for many activities beyond programming. At the end of each chapter there is a section, like this one, with my suggestions for debugging. I hope they help!

Three kinds of errors can occur in a program: syntax errors, runtime errors, and semantic errors. It is useful to distinguish between them in order to track them down more quickly.

**Syntax error**: “Syntax” refers to the structure of a program and the rules about that structure. For example, parentheses have to come in matching pairs, so `(1 + 2)` is legal, but `8)` is a syntax error. If there is a syntax error anywhere in your program, *Python* displays an error message and quits, and you will not be able to run the program. During the first few weeks of your programming career, you might spend a lot of time tracking down syntax errors. As you gain experience, you will make fewer errors and find them faster.

**Runtime error**: The second type of error is a runtime error, so called because the error does not appear until after the program has started running. These errors are also called exceptions because they usually indicate that something exceptional (and bad) has happened. Runtime errors are rare in the simple programs you will see in the first few chapters, so it might be a while before you encounter one.

**Semantic error**: The third type of error is “semantic”, which means related to meaning. If there is a semantic error in your program, it will run without generating error messages, but it will not do the right thing. It will do something else. Specifically, it will do what you told it to do. Identifying semantic errors can be tricky because it requires you to work backward by looking at the output of the program and trying to figure out what it is doing.

## Glossary

**assignment**: A statement that assigns a value to a variable.

**bug**: An error in a program.

**comment**: Information in a program that is meant for other programmers (or anyone reading the source code) and has no effect on the execution of the program.

**debugging**: The process of finding and correcting bugs.

**evaluate**: To simplify an expression by performing the operations in order to yield a single value.

**exception**: An error that is detected while the program is running.

**execute**: To run a statement and do what it says.

**expression**: A combination of variables, operators, and values that represents a single result.

**float**: A type that represents numbers with fractional parts.

**flow of execution**: The order statements run in.

**integer**: A type that represents whole numbers.

**keyword**: A reserved word that is used to parse a program; you cannot use keywords like `if`, `def`, and `while` as variable names.

**namespace**: A mapping from names to objects.

**object**: One of the basic units of data, like a number or string, that a program manipulates.

**operand**: One of the values on which an operator operates.

**operator**: A special symbol that represents a simple computation like addition, multiplication, or string concatenation.

**order of operations**: Rules governing the order in which expressions involving multiple operators and operands are evaluated.

**print statement**: An instruction that causes the Python interpreter to display a value on the screen.

**program**: A set of instructions that specifies a computation.

**prompt**: Characters displayed by the interpreter to indicate that it is ready to take input from the user.

**semantics**: The meaning of a program.

**semantic error**: An error in a program that makes it do something other than what the programmer intended.

**statement**: A section of code that represents a command or action. So far, the statements we have seen are assignments and print statements.

**syntax**: The rules that govern the structure of a program.

**syntax error**: An error in a program that makes it impossible to parse (and therefore impossible to interpret).

**variable**: A name that refers to a value.

## Exercises

1) Whenever you are experimenting with a new feature, you should try to make mistakes. This kind of experiment helps you remember what you read; it also helps when you are programming, because you get to know what the error messages mean. It is better to make mistakes now and on purpose than later and accidentally.

- In a print statement, what happens if you leave out one of the parentheses, or both?
- You can use a minus sign to make a negative number like `-2`. What happens if you put a plus sign before a number? What about `2++2`?
- In math notation, leading zeros are ok, as in `02`. What happens if you try this in *Python*?
- What happens if you have two values with no operator between them?

2) How many seconds are there in 42 minutes 42 seconds?

3) How many kilometers are there in 10 miles?

*Hint:* there are 1.61 kilometers in a mile.

4) If you run a 10 mile race in 42 minutes 42 seconds, what is your average pace (time per kilometers in minutes and seconds)? What is your average speed in kilometers per hour?

5) For each term in the expression below, which arithmetic operation (terms) does *Python* compute first?

$$
    (300-154)/(11/2+16\times6)
$$

6) For each term in the expression below, which arithmetic operation (terms) does *Python* compute first?

$$
    300-154/11/2+16\times6
$$

7) What is the value of $154/11/2$?
- Does `154` first get divided by `11` to give `14` which is then divided by `2`?
- Or does `11` get divided by `2` to give `5.5` such that `154` can be divided by `5.5`?

8) Identify the order in which the operations will execute by placing round brackets `()` around the operations that execute together (for example $12 + 3\times4 = (12 + (3\times4))$):

$$
    250-350/35/5+125\times3^2/2
$$

9). Compute the answer of

$$
    \frac{154-17}{29+54\times3}
$$

10). Compute the answer of

$$
    154 - \frac{17}{29}+54\times3
$$

11) We’ve seen that `n = 42` is legal. How about `x = y = 1`?

12) Which operators (terms) are executed first when executing `calc = 5*5*5 + 4*4`

13) In math notation you can multiply `x` and `y` like this: `xy`. What happens if you try that in *Python*?

14) In math notation you can multiply `x` and `(y + z)` like this: `x(y + z)`. What happens if you try that in *Python*?

15) The volume of a sphere with radius `r` is $\frac{4}{3} \pi r^3$. What is the volume of a sphere with radius 5?

16) Redo the computation `calc = 5*5*5 + 4*4` using the `**` operator instead of the `*` operator

17) Suppose the cover price of a book is R149.95, but bookstores get a 40% discount. Shipping costs R25.00 for the first copy and R1.50 for each additional copy. What is the total wholesale cost for 60 copies?

18). Compute the terminal velocity of a body in a non-dense medium by using the following:

$$
    v_t = \sqrt{\frac{2mg}{\rho A C_d}}
$$

with:

- $C_d = 0.2$ − coefficient of drag (dimensionless)
- $m = 80$ − mass($kg$)
- $\rho = 1.225$ − density of medium in which body is falling ($kg/m^3$)
- $A = 0.1$ − cross sectional area of body perpendicular to direction of flow ($m^2$)

```python

```


```python
# Data processing
import numpy as np
import pandas as pd

# Stats
seed = np.random.SeedSequence(42)
rng = np.random.default_rng(seed=seed)
import pystan
import scipy.stats as st
from statsmodels.distributions.empirical_distribution import ECDF

# Data viz
import bokeh.io
import bokeh.plotting
bokeh.io.output_notebook()

# Helper functions
from stan_workup import summarize, predictive_regression
```

# Overview
This notebook contains code used to generate estimates for enzymatic parameters for Gordon Rix _et al._, 2020.

## Workflow
### Initial velocity measurements
The goal is to estimate $k_\text{cat}$ for each enzyme and $K_\text{M}$ for indole. Reaction mixtures of indole were prepared via serial dilution (where possible) to obtain the following concentration values:
```
concs = [500, 400, 300, 200, 100, 50, 25, 12.5, 6.25, 3.125, 1.5625, 0] # µM
```
with a constant concentration of 40 mM serine in 50 mM potassium phosphate buffer, pH 8.0, at 30 °C.

Enzyme was added (50 nM) and the absorbance at 290 nm was immediately recorded to observe the initial velocity of the reaction. For very low indole concentrations the velocities could be seen to level off over time (expecially when >50 nM enzyme was used) as all of the indole is consumed, so only the first minute (really 10–70 seconds) was kept to determine the intitial velocity. Although the values can be noisy, this is accounted for during the Bayesian inference process by propogating the error in the measurements during the whole process.

The absorbance difference between indole and tryptophan is known at 290 nm, allowing units of AU/time to be converted to mM/time, providing a specific rate. Since the enzyme concentration is also known, we can convert each velocity measurement to an enzyme-normalized rate.

### Bayesian modeling
#### Generative distributions in Stan
Stan uses a generative modeling strategy to model the system and data and provide inferences. In other words, if one knows a reasonable model of the data-generating process, Stan will use a probabilitic approach to providing estimates of each parameter in the system. In this case, the data-generating process is as follows: at each indole concentration, a rate is sampled from the theoretical Michaelis-Menten curve, with some error (assumed to be Normal). If the Michaelis-Menten equation is of the form:

\begin{align}
k = \frac{k_\text{cat}\text{[indole]}}{K_\text{M} + \text{[indole]}},
\end{align}

then its generative distribution can be written as:

\begin{align}
k \sim \text{Norm}(\frac{k_\text{cat}\text{[indole]}}{K_\text{M} + \text{[indole]}}, σ_k),
\end{align}

where $σ_k$ is the standard deviation for the Michaelis-Menten generative distribution.

Rates are not measured perfectly, however, and so each rate that is measured has its own data-generating process, depending on hardware limitations (i.e., the lamp and detector on the UV-Vis spectrophotometer) and other factors. The rates are measured as the conversion of indole to tryptophan over time (which is measured via absorbance) and should be linear, and so are simply linear regressions of the absorbance values over time, with deviations from the theoretical slope according to some normal distribution. (An alternative solution would be to model this as a stochastic differential equation.) As an additional factor, the noise of a spectrophotometer is typically heteroscedastic for large changes in absorbance, so each rate measurement is assigned its own standard deviation (as the noise should be greater for higher concentration samples than lower). For measuring absorbance over time (which can be converted to true rates as mentioned above), this data-generating process is a follows:

\begin{align}
a_t = a \times t + a_0,
\end{align}

where $a$ is the rate of absorbance change, $t$ is the time (in seconds), and $a_0$ is absorbance at $t=0$, and each rate is modeled as

\begin{align}
a \sim \text{Norm}(m, σ_a),
\end{align}

where $m$ is the estimated mean slope and $σ_a$ is the sample-specific standard deviation from the mean slope.

#### Priors
Bayesian inference require "priors" for the estimated parameters ($k_\text{cat}$ and $K_\text{M}$), which should be large-and non-specific, but reasonably limit the starting search space for the Markov chain Monte Carlo (MCMC) sampler (more on this below). In other words, it places initial probability density over reasonable values for these parameters, according to your previous knowledge. As long as your posterior distributions for each parameter are much more constrained than your priors, it is unlikely that your prior choices are strongly influencing  your results. For $k_\text{cat}$ and $K_\text{M}$ we choose large lognormal priors, which are useful as they constrain the values to be >0 (which they will be) and, in practice, these parameters vary in log-space rather than linear-space. That is, rates and rate constants vary multiplicatively as the energy of the system varies additively, so a lognormal prior better approximates what we know about these parameters. Finally, as Stan's sampler is _very_ efficient, as long as our model is not exceptionally complex or pathological, a reasonable posterior distribution will be found as long as the prior probability is not identically zero. (Note that is somewhat of an oversimplification.)

#### Sampling
The sampling procedure will pick initial parameter guesses from your prior distributions and test how well your data agree with those parameter guesses in the scope of your model. This is stored as a sample, assuming we are out of a warm-up stage. Then new parameter guesses will be made and the sampler will see how well those agree. If they are better, then they will be accepted and stored. In they are worse, they will be accepted and stored _with some probability_ which depends on how reasonable the guess it. This is the basis of Markov chain Monte Carlo (MCMC) sampling, and it can provide an estimation of your posterior distribution, which is a quantification of your uncertainty in your parameter estimations. These samples are ideally drawn independently and so are not correlated (which can be checked in the summary and diagnostics of the samples).

With the samples, you can compute summary statistics about your parameters, such as mean, median, and credible regions, and see how predictive your parameters are for the data. This is shown in the examples below.

# A toy problem
To confirm that the model returns values that we expect, we can provide generated data (according to our data-generating process) and send it through Stan. The values we output should be reasonably close to the input values.

```python
# Define initial parameters
k_cat = 1 # per sec
K_M = 10 # µM

sigma_k = 0.1

# Concentrations in duplicate
# Needs to be sorted and have the zeros as the first entries
concs = np.array(sorted([500, 400, 300, 200, 100, 50, 25, 12.5, 6.25, 3.125, 1.5625, 0]*2)) # µM

# Set up function to generate data
def MM(k_cat, K_M, conc, sigma=0):
    """Theoretical Michaelis-Menten equation, using
    absolute parameters with units of µM, seconds.
    """
    k = (k_cat*conc)/(K_M + conc)

    noisy_k = rng.normal(k, sigma)

    return noisy_k

# Generate data
ks = MM(k_cat, K_M, concs, sigma_k)

# Check
ks
```

Looks okay, let's plot it and see.

```python
theor_x = np.linspace(0, 500, 1000)
theor_k = MM(k_cat, K_M, theor_x, sigma=0)

p = bokeh.plotting.figure(plot_width=500, plot_height=400)
p.xaxis.axis_label = '[indole] (µM)'
p.yaxis.axis_label = 'k (per sec)'

p.line(theor_x, theor_k, color='black', line_width=2)
p.circle(concs, ks, size=8, fill_alpha=0.5)

bokeh.io.show(p)
```

Looks good. We'll now take these rates and convert them to the values we actually get: change in absorbance over time.

```python
# Unit conversions
c_Enz = 50 # nM
epsilon = 1.89 # mM/sec

# Convert rate to specific rate
vs = ks*(c_Enz/1000) # µM per sec

# Convert µM per sec to mAU per sec
a_rates = vs*epsilon # mAU per sec

# Check
a_rates
```

With these we can now generate noisy absorbance values over time. We'll demonstrate for one of them first, and then do it with all of them.

```python
# Take a 500 µM case
working_rate = a_rates[-1]

# Generate some times; 0.2 second increments
times = np.arange(0, 60, 0.2)

# Noise; use as a function of a_rate
sigma_a_factor = 5

def a_t(a_rate, time, a_0=0, sigma_factor=0):
    """Creates a noisy timecourse based on a slope and
    times, and an optional y-intercept and noise factor.
    """
    a_t = a_rate*time + a_0

    noisy_a_t = rng.normal(a_t, abs(sigma_factor*a_rate)+0.05)

    return noisy_a_t

# Get a timecourse
a_ts = a_t(working_rate, times, sigma_factor=sigma_a_factor)

# Plot
p = bokeh.plotting.figure(plot_width=500, plot_height=400)
p.xaxis.axis_label = 'time (sec)'
p.yaxis.axis_label = 'mAU'

p.line(times, a_ts, line_width=1)

bokeh.io.show(p)
```

Very reasonable. Now let's plot each.

```python
# Set up full milliabsorbance matrix, ordered by concs
mabs = np.array([
    a_t(working_rate, times, sigma_factor=sigma_a_factor)
    for working_rate in a_rates
])

p = bokeh.plotting.figure(plot_width=800, plot_height=400)
p.xaxis.axis_label = 'time (s)'
p.yaxis.axis_label = 'mAU'

# Set up color dict
colors = bokeh.palettes.magma(len(concs))[::2]
color_dict = {
    conc: color for conc, color in zip(np.unique(concs), colors)
}

for i, conc in enumerate(concs):
    mab = mabs[i]

    p.line(times, mab, color=color_dict[conc], legend_label=str(conc))

bokeh.io.show(p)
```

Now we can send these values through the Stan sampler and see if we get reasonable estimates of our original parameters ($k_\text{cat}$ = 1, $K_\text{M}$ = 10).

Let's first compile and display the model, which is written according to what was specified in 'Generative Distributions in Stan'.

```python
# Create the model and display it
model = pystan.StanModel('MM_model.stan')

print(model)
```

Now we can prepare the data and do some sampling.

```python
# Multiply absorbances by 100 to give rates of ~1
# This speeds up modeling, and is later factored in
scaling_factor = 100
scaled_absorbances = mabs*scaling_factor

# Store as dictionary
data = dict(
    N=len(times),
    M=len(concs),
    M0=sum(concs == 0),
    t=times,
    conc=concs,
    a=scaled_absorbances,
    scaling_factor=scaling_factor,
    epsilon=epsilon,
    c_Enz=c_Enz,
    max_conc=int(max(concs)),
    conc_ppc=range(0, int(max(concs)+1)),
)

# Sample
samples = model.sampling(data)

# Check out a summary
samples
```

```python
# Convert to a dataframe
df_stan = samples.to_dataframe()

# Check the results
summarize(df_stan['k_cat'], units='per sec')
```

```python
summarize(df_stan['K_M'], units='µM')
```

The analysis looks good. We notice a few things:

The priors were lognormal (and absolutely enormous) to start. The posterior for $k_\text{cat}$ now looks normal, which means we have enough data to overwhelm the prior and have our samples be distributed more akin to the model (which assumes a normal distribution for $k_\text{cat}$). On the other hand, $K_\text{M}$ is still a bit lognormal, as evidenced by the tail toward higher values. This results in a larger upper bound for $K_\text{M}$ (50% to 97.5%), but the true value still lies within the 95% credible region. The posterior for $K_\text{M}$ has still been _greatly_ informed by the data, however, and hardly resembles the original prior at all. (See below.)

We can see how our samples for the rates at each indole concentration compare to the original data we created.

```python
# Get each rate sample
rate_samples = [col for col in df_stan.columns if 'rate' in col]
df_rates = df_stan[rate_samples]

# Tidy the data
df_rates = df_rates.melt(var_name='sample', value_name='estimated scaled rate')

# Add in concentration info for each sample
df_rates['[indole] (µM)'] = df_rates['sample'].map({sample: conc for sample, conc in zip(rate_samples, concs)})

# Convert rate to k
df_rates['estimated rate (per sec)'] = (((df_rates['estimated scaled rate']/scaling_factor)
                                                                           /epsilon)
                                                                           /(c_Enz/1000))

# Set easier indexing
df_rates.set_index('sample', inplace=True)

# Check
df_rates
```

```python
theor_x = np.linspace(0, 500, 1000)
theor_k = MM(k_cat, K_M, theor_x, sigma=0)

p = bokeh.plotting.figure(plot_width=500, plot_height=400)
p.xaxis.axis_label = '[indole] (µM)'
p.yaxis.axis_label = 'k (per sec)'

p.line(theor_x, theor_k, color='black', line_width=2, legend_label='theoretical MM curve')
p.circle(concs, ks, size=8, fill_alpha=0.5, legend_label='actual sampled rates')

# Plot the 4000 actual sample values (effectively error bars)
p.square(
    df_rates['[indole] (µM)'],
    df_rates['estimated rate (per sec)'],
    size=2,
    fill_color='black',
    line_color=None,
    fill_alpha=1,
)

# Plot the median values
for sample in df_rates.index.unique():
    _df = df_rates.loc[sample]
    p.circle(
        _df['[indole] (µM)'].median(),
        _df['estimated rate (per sec)'].median(),
        size=6,
        fill_color='orange',
        line_color='black',
        fill_alpha=0.8,
        legend_label='median rate estimates',
    )

p.legend.location = 'bottom_right'

bokeh.io.show(p)
```

Here we see that the points are often a little bit off. This comes a part of the data-generating process in the Stan model that was not used to generate the data, which is background correction. In the full model, it is assumed that backround rate of change for the samples _without_ any indole are the same in all cases (e.g., some sort of background drift). So this value is subtracted from all rate estimates. We see that this isn't the case for the generated data (as we know how we generated it and that the estimates don't align), but that's only because we have the _real_ data. This is impossible to know in a real experiment, thus, we model what we expect to happen in our real experiment as best we can.

Finally, we can use our median estimates for $k_\text{cat}$ and $K_\text{M}$ to plot our median Michaelis-Menten curve estimation (what our theoretical Michaelis-Menten behavior would look like assuming the median estimates were _exactly correct_).

```python
# Add median estimation
estimated_ks = MM(df_stan['k_cat'].median(), df_stan['K_M'].median(), theor_x, sigma=0)
p.line(theor_x, estimated_ks, color='#004D00', line_width=3, legend_label='median estimated MM curve')

bokeh.io.show(p)
```

#### Posterior predictive checks
Part of the model was to use the samples to generate _posterior predictive checks_, which are shown to give an idea of how predictive the estimated parameters for the given data. This is best shown for the theoretical Michaelis-Menten curve in the form of shaded credible regions, which we'll have as 95% (widest region), 75%, 50%, 25% (narrowest region), and the median line (which should align well with the median estimated MM curve above). We'll overlay all the previous data as well, except the median estimate curve.

```python
p = predictive_regression(df_stan, 'k_ppc')

p.line(theor_x, theor_k, color='black', line_width=2, legend_label='theoretical MM curve')
p.circle(concs, ks, size=8, fill_alpha=0.5, legend_label='actual sampled rates')

# Plot the 4000 actual sample values (effectively error bars)
p.square(
    df_rates['[indole] (µM)'],
    df_rates['estimated rate (per sec)'],
    size=2,
    fill_color='black',
    line_color=None,
    fill_alpha=1,
)

# Plot the median values
for sample in df_rates.index.unique():
    _df = df_rates.loc[sample]
    p.circle(
        _df['[indole] (µM)'].median(),
        _df['estimated rate (per sec)'].median(),
        size=6,
        fill_color='orange',
        line_color='black',
        fill_alpha=0.8,
        legend_label='median rate estimates',
    )

p.legend.location = 'bottom_right'

bokeh.io.show(p)
```

(As a note, the jaggedness of the credible regions can be smoothed out by taking more than 4000 samples, but this will take longer and take up much or space in memory or physical memory if the samples are saved.)

#### Visualizing what we learned.
We'll quickly look at how much our data informed the priors.

```python
xs = np.linspace(0, 2, 1000)

test = st.lognorm(s=2.5, loc=0, scale=np.log(150)).cdf(xs)

p = bokeh.plotting.figure(plot_width=500, plot_height=400)
p.xaxis.axis_label = 'k_cat'
p.yaxis.axis_label = '(E)CDF'

ecdf = ECDF(df_stan['k_cat'].values)
p.circle(ecdf.x, ecdf.y, legend_label='posterior ECDF')
p.line(xs, test, color='black', line_width=3, legend_label='prior CDF')

bokeh.io.show(p)
```

```python
xs = np.linspace(0, 50, 1000)

test = st.lognorm(s=1.5, loc=0, scale=np.log(500)).cdf(xs)

p = bokeh.plotting.figure(plot_width=500, plot_height=400)
p.xaxis.axis_label = 'K_M'
p.yaxis.axis_label = '(E)CDF'

ecdf = ECDF(df_stan['K_M'].values)
p.circle(ecdf.x, ecdf.y, legend_label='posterior ECDF')
p.line(xs, test, color='black', line_width=3, legend_label='prior CDF')

bokeh.io.show(p)
```

# Real data

Now that we're convinced that we can recover parameters with this model (and assuming we trust the model as a reasonable and useful description of our system), we'll use some actual enzyme kinetics data, as was done to estimate parameters for the paper.

### Data workup
First we'll deal with cleaning up the raw absorbance data.

```python
df = pd.read_excel('example_kinetics_data.xlsx')

df.head()
```

Clean up the column names.

(Note: multiple columns with the same name in a pandas DataFrame is not recommended; only momentarily using it here to prepare numpy arrays.)

```python
# Instantiate list
new_cols = [None for _ in df.columns]

for i, column in enumerate(df.columns):

    # Standardize time label
    if column == 'Time ( Second )':
        new_cols[i] = 'Time (s)'

    # Clean up experimental names
    else:
        working_string = column.replace(' - RawData', '')
        split_string = working_string.split('_')

        if len(split_string) == 1:
            working_string = split_string[0]
        else:
            working_string = split_string[2].replace('uM', '')

        new_cols[i] = working_string

# Rename the columns
df.columns = new_cols

# Check
df.head()
```

```python
# t0 subtract and transpose to simplify operations
t0_sub_values = (df.values - df.values[0]).T

# Ignore Time (column 0) and blank (column -1) columns
full_absorbances = t0_sub_values[1:-1]

# Convert AU to mAU
absorbances = full_absorbances*1000

# Check
absorbances
```

```python
# Plot to check
concs = df.columns[1:-1].astype(float)
times = df['Time (s)'].values

p = bokeh.plotting.figure(plot_width=800, plot_height=400)
p.xaxis.axis_label = 'Time (s)'
p.yaxis.axis_label = 'mAU'

for i, absorbance in enumerate(absorbances):
    color = bokeh.palettes.magma(27)
    p.line(times, absorbance, color=color[i], legend_label=str(concs[i]))

bokeh.io.show(p)
```

Here we see that the real data looks remarkably like generated data (with some more real-world inconsistencies), suggesting that our data-generating process (and our overall model) is reasonable. We can proceed.

#### Shorten the timecourse to 1 minute, from 10 seconds to 70 seconds.

```python
df_min = df[(df['Time (s)'] >= 10) & (df['Time (s)'] <= 70)].copy()

df_min.head()
```

```python
# Prepare values
absorbances = (df_min.values - df_min.values[0]).T[1:-1]*1000
concs = df_min.columns[1:-1].astype(float)
times = df_min['Time (s)'].values

# Plot
p = bokeh.plotting.figure(plot_width=800, plot_height=400)
p.xaxis.axis_label = 'Time (s)'
p.yaxis.axis_label = 'mAU'

for i, absorbance in enumerate(absorbances):
    color = bokeh.palettes.magma(len(concs)+2)
    p.line(times, absorbance, color=color[i], legend_label=str(concs[i]))

bokeh.io.show(p)
```

### Model the system

```python
# Multiply absorbances by 100 to give rates of ~1
# This speeds up modeling, and is later factored in
scaling_factor = 100
scaled_absorbances = absorbances*scaling_factor

# Enzyme concentration, in nM
c_Enz = 50

# Store as dictionary
data = dict(
    N=len(times),
    M=len(concs),
    M0=sum(concs == 0),
    t=times,
    conc=concs,
    a=scaled_absorbances,
    scaling_factor=scaling_factor,
    epsilon=1.89,
    c_Enz=c_Enz,
    max_conc=int(max(concs)),
    conc_ppc=range(0, int(max(concs)+1)),
)

# Sample; use more iterations and warm-up for better results
# This should take ~1 minute on a CPU
samples = model.sampling(data)

# Check out a summary
samples
```

```python
# Convert to a dataframe
df_stan = samples.to_dataframe()

# Check
df_stan
```

Efficient storage, if needed.

```python
# # Write out a ~20 MB file
# df_stan.to_parquet('df_stan.parquet', compression='gzip')

# # Read in the file
# df_stan = pd.read_parquet('df_stan.parquet')

# # Check again
# df_stan
```

### Data analysis

```python
# Get summary stats
summarize(df_stan['k_cat'], units='per sec')
```

```python
summarize(df_stan['K_M'], units='µM')
```

```python
# Get each rate sample
rate_samples = [col for col in df_stan.columns if 'rate' in col]
df_rates = df_stan[rate_samples]

# Tidy the data
df_rates = df_rates.melt(var_name='sample', value_name='estimated scaled rate')

# Add in concentration info for each sample
df_rates['[indole] (µM)'] = df_rates['sample'].map({sample: conc for sample, conc in zip(rate_samples, concs)})

# Convert rate to k
df_rates['estimated rate (per sec)'] = (((df_rates['estimated scaled rate']/scaling_factor)
                                                                           /epsilon)
                                                                           /(c_Enz/1000))

# Set easier indexing
df_rates.set_index('sample', inplace=True)

# Generate the predictive regression plot
p = predictive_regression(df_stan, 'k_ppc')

# Plot the 4000 actual sample values (effectively error bars)
p.square(
    df_rates['[indole] (µM)'],
    df_rates['estimated rate (per sec)'],
    size=2,
    fill_color='black',
    line_color=None,
    fill_alpha=1,
)

# Plot the median values
for sample in df_rates.index.unique():
    _df = df_rates.loc[sample]
    p.circle(
        _df['[indole] (µM)'].median(),
        _df['estimated rate (per sec)'].median(),
        size=6,
        fill_color='orange',
        line_color='black',
        fill_alpha=0.8,
        legend_label='median rate estimates',
    )

p.legend.location = 'bottom_right'

bokeh.io.show(p)
```

```python

```


# Partial Differential Equation and Gaussian Process

# 1 The Spatial Temporal Model

A model of post-transcriptional processing is formulated to describe the spatio-temporal Drosophila
protein expression data(Becker 2012). Protein production is considered to be linearly dependent on the
concentration of mRNA at an earlier time point. The model also allows for diffusion of protein between
nuclei and linear protein decay. These processes are dependent on the diffusion parameter and the degradation
rate of protein respectively.

\begin{equation}
 a \frac{\partial ^2 y_{x,t}}{\partial x^2} + b \frac{\partial y_{x,t}}{\partial t} + c y_{x,t}=  f_{x,t}
\end{equation}

The coefficients a, b and c are unknown. In this study, we use Gaussian process with RBF kernel as a prior over $y_{x,t}$. The multi-output Gaussian process are developed by applying the partial differential operator on the spatial-temporal RBF kernel.

```python
# Copyright (c) 2012, GPy authors (see AUTHORS.txt).
# Licensed under the BSD 3-clause license (see LICENSE.txt)
# model : a*d^2y/dx^2  + b * dy/dt + c * y = f
#  lengthscale for Yt :
#  lengthscale for Yx :
# variance for Yt  :
# variance for Yx  :

%pylab inline
%config InlineBackend.figure_format = 'svg'
import numpy as np
import pylab as pb
import GPy
```

    Populating the interactive namespace from numpy and matplotlib

The spatiol-temporal multi-output partial differential equation kernel are developed with kernel name "GPy.kern.ODE_st" in GPy. The inputs are one dimension spatial data, one dimension temporal data and one dimension index which is used to indicate f and Y.

```python
KY = GPy.kern.ODE_st(input_dim=3,a=1.,b=1,c=1,variance_Yx=1.,variance_Yt=1.,
    lengthscale_Yx=1.,lengthscale_Yt=1.)

nd =10
t1 = 10*np.random.rand(nd)[:,None]
x1 = 10*np.random.rand(nd)[:,None]
inx = np.zeros(nd**2)[:,None]

T = np.kron(t1,np.ones(nd)[:,None])
S = np.kron(np.ones(nd)[:,None],x1)

inx[nd**2/2:nd**2]=1
X = np.hstack([T,S,inx])

#Y=np.sin(X[:,0:1])
#Y=np.sin(X[:,0:1])
Y = np.sin(X[:,0:1]) + np.cos(X[:,1:2])#*(1-X[:,1:2]) #+ np.random.randn(16,1)*0.1

Y[nd**2/2:nd**2] =  np.sin(X[nd**2/2:nd**2,0:1]) + np.cos(X[nd**2/2:nd**2,0:1]) + 2*np.cos(X[nd**2/2:nd**2,1:2])
```

```python
m = GPy.models.GPRegression(X,Y,KY)
```

# 2 Model Output

In order to test the kernel, we defined

$y_{x,t} = cos(x) + sin(t)$

$f_{x,t} = sin(t) + cos(t) + 2cos(x)$

and a, b and c are equal to 1. With some arbitray choices of kernel parameters we plot the random field of f and y separatly.
The spatial variable can be fixed to certain value. It become easier to compare the true curve and our estimation.

```python
fixX = 5.
xplot = np.linspace(-2,12,200)
m.plot(fixed_inputs=[(2,0)], which_data_rows = slice(0,nd**2/2))
m.plot(fixed_inputs=[(1,fixX),(2,0)], which_data_rows = slice(0,nd**2/2))
pb.plot(xplot,np.cos(fixX) + np.sin(xplot))

m.plot(fixed_inputs=[(2,1)], which_data_rows = slice(nd**2/2,nd**2))
m.plot(fixed_inputs=[(1,fixX),(2,1)], which_data_rows = slice(nd**2/2,nd**2))
pb.plot(xplot,np.cos(xplot) + np.sin(xplot) + 2*np.cos(fixX))
```

     /home/nbuser/anaconda2_501/lib/python2.7/site-packages/matplotlib/contour.py:967: UserWarning:The following kwargs were not used by contour: 'linewidth', 'label'

    [<matplotlib.lines.Line2D at 0x7feaae4ae2d0>]

     /home/nbuser/anaconda2_501/lib/python2.7/site-packages/matplotlib/figure.py:1999: UserWarning:This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.

```python
m.optimize()
```

    <paramz.optimization.optimization.opt_lbfgsb at 0x7feab2164050>

```python
print m
```

    Name : GP regression
    Objective : -316.891724445
    Number of Parameters : 8
    Number of Optimization Parameters : 8
    Updates : True
    Parameters:
      GP_regression.           |              value  |  constraints  |  priors
      ode_st.a                 |     0.927831276506  |      +ve      |
      ode_st.b                 |     0.878872529752  |      +ve      |
      ode_st.c                 |      1.07222979737  |      +ve      |
      ode_st.variance_Yt       |      5.87569800896  |      +ve      |
      ode_st.variance_Yx       |      5.87569800896  |      +ve      |
      ode_st.lengthscale_Yt    |       12.464230291  |      +ve      |
      ode_st.lengthscale_Yx    |       12.713259674  |      +ve      |
      Gaussian_noise.variance  |  1.74044285035e-16  |      +ve      |

After optimization, the estimated value of a, b and c are printed. It is quite close to the true parameters we used to generate the testing data. The plot of the random fields are plotted below.

```python
fixX = 5.
xplot = np.linspace(-2,12,200)
m.plot(fixed_inputs=[(2,0)], which_data_rows = slice(0,nd**2/2))
m.plot(fixed_inputs=[(1,fixX),(2,0)], which_data_rows = slice(0,nd**2/2))
pb.plot(xplot,np.cos(fixX) + np.sin(xplot))

m.plot(fixed_inputs=[(2,1)], which_data_rows = slice(nd**2/2,nd**2))
m.plot(fixed_inputs=[(1,fixX),(2,1)], which_data_rows = slice(nd**2/2,nd**2))
pb.plot(xplot,np.cos(xplot) + np.sin(xplot) + 2*np.cos(fixX))
```

    [<matplotlib.lines.Line2D at 0x7feaadcfeb10>]

The true random field is plotted below

```python
xplot = np.linspace(-1,10,200)
#pb.plot(xplot,np.sin(xplot))
PP = np.zeros((xplot.shape[0], xplot.shape[0]))
QQ = np.zeros((xplot.shape[0], xplot.shape[0]))

for i in range(0,200):
    for j in range(0,200):
        PP[i,j] = np.sin(xplot[i]) + np.cos(xplot[j])
        QQ[i,j] = np.sin(xplot[i]) + np.cos(xplot[i]) + 2*np.cos(xplot[j])

pb.figure()
pb.imshow(PP)
```

    <matplotlib.image.AxesImage at 0x7feaad8b8090>

```python

```