### GeostatsPy: Confidence Intervals and Hypothesis Testing for Subsurface Data Analytics in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

##### Reporting Uncertainty and Significance

With confidence intervals and hypothesis testing we have the opportunity to report uncertainty and to report significance in our statistics. These methods are based on standard methods with their associated limitations and assumptions.  See the lecture and workflow on Bootstrap https://git.io/fhgUW for a general, empirical approach to assess uncertianty in statistics.

This is a tutorial / demonstration of **Confidence Intervals and Hypothesis Testing in Python** for Subsurface Modeling.  In Python, the SciPy package, specifically the Stats functions (https://docs.scipy.org/doc/scipy/reference/stats.html) provide excellent tools for efficient use of statistics.

I have previously provided these examples worked out by-hand in Excel (https://github.com/GeostatsGuy/LectureExercises/blob/master/Lecture7_CI_Hypoth_eg_R.xlsx) and also in R (https://github.com/GeostatsGuy/LectureExercises/blob/master/Lecture7_CI_Hypoth_eg.R).  In all cases, I use the same dataset available as a comma delimited file (https://git.io/fxLAt).

This tutorial includes basic, typical confidence interval and hypothesis testing methods that would commonly be required for Engineers and Geoscientists including:

1. Student-t confidence interval for the mean
2. Student-t hypothesis test for difference in means (pooled variance)
3. Student-t hypothesis test for difference in means (difference variances), Welch's t Test
3. F-distribution hypothesis test for difference in variances

##### Caveats

I have not included all the details, specifically the test assumptions in this document.  These are included in the accompanying course notes, Lec08_hypothesis.pdf.

#### Workflow Goal

0. Introduction to Python in Jupyter including setting a working directory, loading data into a Pandas DataFrame.
1. Learn the basics for working with confidence intervals and hypothesis testing in Python.
2. Demonstrate the efficiency of using Python and SciPy package for statistical analysis.
3. Learn how to quantify uncertainty and significance in samples.

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - PorositySample2Units.csv at https://git.io/fhrM8

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Load the required libraries

The following code loads the required libraries.

```python
import geostatspy.GSLIB as GSLIB                       # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats                 # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import os                                              # to set current working directory
import numpy as np                                     # arrays and matrix math
import scipy.stats as st                               # statistical methods
import pandas as pd                                    # DataFrames
import matplotlib.pyplot as plt                        # plotting
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).  Also, in this case make sure to place the required (see below) data file in this directory.  When we are done with this tutorial we will write our new dataset back to this directory.

```python
os.chdir("C:\PGE383")                                  # set the working directory
```

#### Loading Data

Let's load the provided dataset. 'PorositySamples2Units.csv' is available at https://github.com/GeostatsGuy/GeoDataSets. It is a comma delimited file with 20 porosity measures from 2 rock units from the subsurface, porosity (as a fraction). We load it with the pandas 'read_csv' function into a data frame we called 'df' and then preview it by printing a slice and by utilizing the 'head' DataFrame member function (with a nice and clean format, see below).

```python
df = pd.read_csv("PorositySample2Units.csv")           # read a .csv file in as a DataFrame
#print(df.iloc[0:5,:])                                 # display first 4 samples in the table as a preview
df.head()                                              # we could also use this command for a table preview
```

      X1
      X2

      0
      0.21
      0.20

      1
      0.17
      0.26

      2
      0.15
      0.20

      3
      0.20
      0.19

      4
      0.19
      0.13

It is useful to review the summary statistics of our loaded DataFrame.  That can be accomplished with the 'describe' DataFrame member function.  We transpose to switch the axes for ease of visualization.

```python
df.describe().transpose()                              # visualize summary statistics
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      X1
      20.0
      0.1645
      0.027810
      0.11
      0.1500
      0.17
      0.19
      0.21

      X2
      20.0
      0.2000
      0.045422
      0.11
      0.1675
      0.20
      0.23
      0.30

Here we extract the X1 and X2 unit porosity samples from the DataFrame into separate arrays called 'Por1' and 'Por2' for convenience.

```python
por1 = df['X1'].values                                 # extract well 1 porosity to a ndarray
por2 = df['X2'].values                                 # extract well 2 porosity to a ndarray
```

We should first visualize the distributions.  It is convenient to plot the distributions on top of each other.

```python
plt.subplot(121)
plt.hist(por1, facecolor='red',bins=np.linspace(0.0,0.4,20),alpha=0.2,density=False,edgecolor='black',label='Well 1')
plt.hist(por2, facecolor='blue',bins=np.linspace(0.0,0.4,20),alpha=0.2,density=False,edgecolor='black',label = 'Well 2')
plt.xlim([0.0,0.4]); plt.ylim([0,8.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Porosity Well 1 and 2')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(122)
plt.hist(por1, facecolor='red',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black',label='Well 1')
plt.hist(por2, facecolor='blue',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black',label='Well 2')
plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Well 1 and 2')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

The histogram and cumulative distribution functions indicate the porosity distributions in both wells 1 and 2. Ocular inspection does indicate:

1. not a lot of samples are available for each (n=20)
2. the distributions look different

Let's use statistics to learn more about the subsurface from these two wells.

#### Confidence Intervals

Let's first demonstrate the calculation of the confidence interval for the sample mean at a 95% confidence level.  This could be interpreted as the interval over which there is a 95% confidence that it contains the true population mean.  We use the student's t distribution as we assume we do not know the variance and the sample size is small.

\begin{equation}
x̅ \pm t_{\frac{\alpha}{2},n-1} \times \frac {s}{\sqrt{n}}
\end{equation}

```python
ci_95_por1 = st.t.interval(0.95, len(df)-1, loc=np.mean(por1), scale=st.sem(por1))
print('The confidence interval for the Well 1 mean porosity is ' + str(round(np.mean(por1),3)) + ' +/- ' + str(round(ci_95_por1[1]-np.mean(por1),3))
      + ', with a range of ' + str(round(ci_95_por1[0],3)) + ', ' + str(round(ci_95_por1[1],3)))

ci_95_por2 = st.t.interval(0.95, len(df)-1, loc=np.mean(por2), scale=st.sem(por2))
print('The confidence interval for the Well 2 mean porosity is ' + str(round(np.mean(por2),3)) + ' +/- ' + str(round(ci_95_por2[1]-np.mean(por2),3))
      + ', with a range of ' + str(round(ci_95_por2[0],3)) + ', ' + str(round(ci_95_por2[1],3)))
```

    The confidence interval for the Well 1 mean porosity is 0.164 +/- 0.013, with a range of 0.151, 0.178
    The confidence interval for the Well 2 mean porosity is 0.2 +/- 0.021, with a range of 0.179, 0.221

#### Using Confidence Intervals to Build an Uncertainty Model

The confidence interval for the mean could be applied directly as an uncertainty model! The uncertainty in mean porosity at each well could be applied to calculate the uncertainty in the mean porosity applied to model porosity around the two wells. This method would account for the limited number of porosity samples available. If more porosity samples we available then this uncertainty would decrease (note the $\sqrt{n}$ in the denominator of the confidence interval in the mean calculation above.

One possible work flow would be to apply the affine correction from the *GeostatsPy* package to formulate a the P5 and P95 porosity distributions for porosity in each well. Note, these distribution scenarios are based only on uncertainty in the mean porosity and assumes the distribution tails are not anchored (are free to shift).

```python
stdev1 = np.std(por1); stdev2 = np.std(por2)
por1r05 = GSLIB.affine(por1,tmean=ci_95_por1[0],tstdev=stdev1)
por1r95 = GSLIB.affine(por1,tmean=ci_95_por1[1],tstdev=stdev1)
por2r05 = GSLIB.affine(por2,tmean=ci_95_por2[0],tstdev=stdev2)
por2r95 = GSLIB.affine(por2,tmean=ci_95_por2[1],tstdev=stdev2)

plt.subplot(121)
plt.hist(por1r05, facecolor='yellow',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',label='P05')
plt.hist(por1, facecolor='orange',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',label='Expected')
plt.hist(por1r95, facecolor='red',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',label='P95')

plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Distribution Uncertainty - Well 1')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(122)
plt.hist(por2r05, facecolor='yellow',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',label='P05')
plt.hist(por2, facecolor='orange',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',label='Expected')
plt.hist(por2r95, facecolor='red',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',label='P95')

plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Distribution Uncertainty - Well 2')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

We just calculated a scenario-based uncertainty model for the porosity distribution around wells 1 and 2. Of course, we could actually sample porosity means continuously from our confidence calculation as we have access to the complete distribution for uncertainty in the mean porosity of both wells.

Communicating uncertainty is powerful, but always remember to state the assumptions. For example, here we assumed:

1. The population distribution of porosity for each well is Gaussian distributed
2. That the samples are independent.

One can check the Excel file linked above with the confidence interval calculated by hand and confirm that this result is correct.

#### Hypothesis Testing

The confidence intervals help with uncertainty in the distributions of porosity.  Now let's try to figure out if:

1. wells 1 and 2 drilled into the same type of rock?
2. did something change between the 2 wells?
3. different units are being compared between the 2 wells (issues with stratal correlation)?

Now, let's try the t test, hypothesis test for difference in means.  This test assumes that the variances are similar along with the data being Gaussian distributed (see the course notes for more on this).  This is our test:

\begin{equation}
H_0: \mu_{X1} = \mu_{X2}
\end{equation}

\begin{equation}
H_1: \mu_{X1} \ne \mu_{X2}
\end{equation}

For the resulting t-statistic and p-value we run this command.

```python
t_pooled, p_pooled = st.ttest_ind(por1,por2)
print('The t statistic is ' + str(round(t_pooled,3)) + ' and the p-value is ' + str(round(p_pooled,3)))
```

    The t statistic is -2.981 and the p-value is 0.005

The p-value, $p$, is the symmetric interval probaiblity our outside.  In other words the $p$ reported is 2 x cumulative probaiblity of the t statistic applied to the sampling t distribution.  Another way to look at it, if one used the $\pm t_{t_{statistic},.d.f}$ statistic as thresholds, $p$ is the probability being outside this symmetric interval. So we will reject the null hypothesis if $p \lt \alpha$.  From the p-value alone it is clear that we would reject the null hypothesis and accept the alternative hypothesis that the means are not equal.

In case you want to compare the t-statistic to t-critical, we can apply the inverse of the student's t distribution at $\frac{\alpha}{2}$ and $1-\frac{\alpha}{2}$ to get the upper and lower critcal values.

```python
t_critical = st.t.ppf([0.025,0.975], df=len(por1)+len(por2)-2)
print('The t crical lower and upper values are ' + str(np.round(t_critical,2)))
```

    The t crical lower and upper values are [-2.02  2.02]

We can observe that, as expected, the t-statistic is outside the t-critcal interval.  These results are exactly what we got when we worked out the problem by hand in Excel, but so much more efficient!

Now let's try the t-test, hypothesis test for difference in means allowing for unequal variances, this is also known as the Welch's t test.  All we have to do is set the parameter 'equal_var' to false, note it defaults to true (e.g. the command above).

```python
st.ttest_ind(por1, por2, equal_var = False)
```

    Ttest_indResult(statistic=-2.9808897468855644, pvalue=0.005502572350112333)

Once again we can see by $p$ that we will clearly reject the null hypothesis.

Let's now compare the variances with the F-test for difference in variances.

\begin{equation}
H_0: \sigma^{2}_{X2} \le \sigma^{2}_{X1}
\end{equation}

\begin{equation}
H_1: \sigma^{2}_{X2} \gt \sigma^{2}_{X1}
\end{equation}

Note, by ordering the variances we eliminate the case of $\sigma^{2}_{X2} \lt \sigma^{2}_{X1}$.

Details about the test are available in the course notes (along with assumptions such as Gaussian distributed) and this example is also worked out by hand in the linked Excel workbook.  We can accomplish the F-test in with SciPy.Stats the function with one line of code if we calculate the ratio of the sample variances ensuring that the larger variance is in the numerator and get the degrees of freedom using the len() command, ensuring that we are consistent with the numerator degrees of freedom set as 'dfn' and the denominator degrees of freedom set as 'dfd'.  We take a p-value of $1-p$ since the test is configured to be a single, right tailed test.

```python
p_value = 1 - st.f.cdf(np.var(por2)/np.var(por1), dfn=len(por2)-1, dfd=len(por1)-1)
print('The p-value for the F-test for difference is variances is ' + str(round(p_value,3)))
```

    The p-value for the F-test for difference is variances is 0.019

Once again we would clearly reject the null hypothesis since $p \lt alpha$ and assume that the variances are not equal.  Note this is a single tail test so our p-critical is 0.05 for a 95% confidence level.

#### Reporting the Hypothesis Test Results

The difference in the means and variances were tested for statistical significance for porosity measured in wells 1 and 2.  In all cases we rejected the null hypotheses and adopted the alternative hypotheses that the porosity distributions' means and variances for well 1 and 2 are statistically significant in their differences. It is possible that Well 1 and 2 have drilled into different rock units or have encountered nonstationary behavoirs in a single rock unit. This result may be applied to update subsurface maps and consider a difference in the subsurface at each well for development decision making.

#### Comments

We are just scratching the surface for the application of confidence intervals and hypothesis tests to subsurface modeling.  Once again, there are a lot of details left out of the problem formulation and assumptions, see the course notes for more coverage.  By running the same confidence interval and hypothesis tests: 1) by hand in Excel and with 2) R and now here in 3) Python code, I hope this demonstration will enable and encourage more engineers and scientists to make these R and Python tools part of their common practice. I'm always happy to discuss,

I hope this was helpful,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)


# ¿Cómo funciona la suspensión de un auto?

> Una primer aproximación al modelo de la suspensión de un automovil es considerar el *oscilador armónico amortiguado*.

Referencia:
- https://es.wikipedia.org/wiki/Oscilador_arm%C3%B3nico#Oscilador_arm.C3.B3nico_amortiguado

Un **modelo** que describe el comportamiento del sistema mecánico anterior es

\begin{equation}
m\frac{d^2 x}{dt^2}=-c\frac{dx}{dt}-kx
\end{equation}

donde $c$ es la constante de amortiguamiento y $k$ es la constante de elasticidad.  Revisar modelado

Documentación de los paquetes que utilizaremos hoy.
- https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html
- https://docs.scipy.org/doc/scipy/reference/index.html

___
En `python` existe una función llamada _odeint_ del paquete _integrate_ de la libreria _scipy_, que permite integrar sistemas vectoriales de primer orden, del tipo

\begin{equation}
\frac{d\boldsymbol{y}}{dt} = \boldsymbol{f}(t,\boldsymbol{y}); \qquad \text{ con }\quad \boldsymbol{y}\in\mathbb{R}^n,\quad \boldsymbol{f}:\mathbb{R}_{+}\times\mathbb{R}^n\to\mathbb{R}^n
\end{equation}

con condiciones iniciales $\boldsymbol{y}(0) = \boldsymbol{y}_{0}$. Notar que  $\boldsymbol{y}$ representa un vector de $n$ componentes.

Ahora, si nos fijamos bien, el modelo del *oscilador armónico amortiguado* que obtuvimos es una ecuación diferencial ordinaria (EDO) de segundo orden. No hay problema. La podemos convertir en un sistema de ecuaciones de primer orden de la siguiente manera:

1. Seleccionamos el vector $\boldsymbol{y}=\left[y_1\quad y_2\right]^T$, con $y_1=x$ y $y_2=\frac{dx}{dt}$.
2. Notamos que $\frac{dy_1}{dt}=\frac{dx}{dt}=y_2$ y $\frac{dy_2}{dt}=\frac{d^2x}{dt^2}=-\frac{c}{m}\frac{dx}{dt}-\frac{k}{m}x=-\frac{c}{m}y_2-\frac{k}{m}y_1$.
3. Entonces, el modelo de segundo orden lo podemos representar como el siguiente sistema vectorial de primer orden:

\begin{equation}
\frac{d\boldsymbol{y}}{dt}=\left[\begin{array}{c}\frac{dy_1}{dt} \\ \frac{dy_2}{dt}\end{array}\right]=\left[\begin{array}{c}y_2 \\ -\frac{k}{m}y_1-\frac{c}{m}y_2\end{array}\right]=\left[\begin{array}{cc}0 & 1 \\-\frac{k}{m} & -\frac{c}{m}\end{array}\right]\boldsymbol{y}.
\end{equation}

```python
# Primero importamos todas las librerias, paquetes y/o funciones que vamos a utlizar
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import odeint
```

```python
# Definimos los parámetros k, m y c
k, m, c = 3, 1, 0.5

# Función f(y,t) que vamos a integrar
def amortiguado(y,t,k,m,c):
    y1 = y[0]
    y2 = y[1]
    return np.array([y2, -(k/m)*y1-(c/m)*y2])

# Condiciones iniciales
y0 = np.array([1,1])

# Especificamos los puntos de tiempo donde queremos la solución
t = np.linspace(0, 30, 300)

# Solución numérica
y = odeint(amortiguado, y0, t, args=(k,m,c))
```

¿Cómo entrega odeint las soluciones?

```python
# Averiguar la forma de solución
y.shape
```

    (300, 2)

```python
# Mostrar la solución
y
```

    array([[ 1.00000000e+00,  1.00000000e+00],
           [ 1.08255354e+00,  6.44400045e-01],
           [ 1.12926028e+00,  2.87265734e-01],
           [ 1.14050185e+00, -6.08345277e-02],
           [ 1.11768327e+00, -3.90114960e-01],
           [ 1.06314152e+00, -6.91821494e-01],
           [ 9.80031347e-01, -9.58438604e-01],
           [ 8.72192532e-01, -1.18384948e+00],
           [ 7.44003548e-01, -1.36344733e+00],
           [ 6.00226559e-01, -1.49419684e+00],
           [ 4.45848722e-01, -1.57464647e+00],
           [ 2.85924657e-01, -1.60489362e+00],
           [ 1.25424603e-01, -1.58650595e+00],
           [-3.09074594e-02, -1.52240341e+00],
           [-1.78682133e-01, -1.41670627e+00],
           [-3.13976227e-01, -1.27455532e+00],
           [-4.33425216e-01, -1.10191089e+00],
           [-5.34294753e-01, -9.05337619e-01],
           [-6.14530179e-01, -6.91782064e-01],
           [-6.72783643e-01, -4.68349828e-01],
           [-7.08419119e-01, -2.42089037e-01],
           [-7.21496257e-01, -1.97860630e-02],
           [-7.12734597e-01,  1.92221059e-01],
           [-6.83460139e-01,  3.88206930e-01],
           [-6.35536705e-01,  5.63197023e-01],
           [-5.71284864e-01,  7.13077112e-01],
           [-4.93391403e-01,  8.34672057e-01],
           [-4.04812458e-01,  9.25793147e-01],
           [-3.08673486e-01,  9.85254145e-01],
           [-2.08169142e-01,  1.01285709e+00],
           [-1.06466032e-01,  1.00934976e+00],
           [-6.61104632e-03,  9.76357430e-01],
           [ 8.85523259e-02,  9.16292147e-01],
           [ 1.76457556e-01,  8.32243416e-01],
           [ 2.54876953e-01,  7.27854260e-01],
           [ 3.21970545e-01,  6.07187080e-01],
           [ 3.76321163e-01,  4.74583783e-01],
           [ 4.16955391e-01,  3.34524538e-01],
           [ 4.43350454e-01,  1.91489371e-01],
           [ 4.55427534e-01,  4.98265382e-02],
           [ 4.53532357e-01, -8.63687837e-02],
           [ 4.38404262e-01, -2.13362439e-01],
           [ 4.11135226e-01, -3.27873347e-01],
           [ 3.73120515e-01, -4.27147733e-01],
           [ 3.26002856e-01, -5.09014244e-01],
           [ 2.71612061e-01, -5.71919246e-01],
           [ 2.11902105e-01, -6.14942256e-01],
           [ 1.48887646e-01, -6.37791984e-01],
           [ 8.45818567e-02, -6.40784133e-01],
           [ 2.09373508e-02, -6.24802410e-01],
           [-4.02082314e-02, -5.91244778e-01],
           [-9.71805876e-02, -5.41957213e-01],
           [-1.48509956e-01, -4.79157531e-01],
           [-1.92964291e-01, -4.05352053e-01],
           [-2.29573832e-01, -3.23247831e-01],
           [-2.57646763e-01, -2.35663301e-01],
           [-2.76775942e-01, -1.45440011e-01],
           [-2.86836931e-01, -5.53580104e-02],
           [-2.87977823e-01,  3.19428361e-02],
           [-2.80601553e-01,  1.14033541e-01],
           [-2.65341593e-01,  1.88757055e-01],
           [-2.43032054e-01,  2.54278333e-01],
           [-2.14673367e-01,  3.09122495e-01],
           [-1.81394746e-01,  3.52200544e-01],
           [-1.44414712e-01,  3.82822476e-01],
           [-1.05000916e-01,  4.00698037e-01],
           [-6.44304938e-02,  4.05925681e-01],
           [-2.39520746e-02,  3.98970667e-01],
           [ 1.52495098e-02,  3.80633436e-01],
           [ 5.20849165e-02,  3.52009701e-01],
           [ 8.55876270e-02,  3.14443793e-01],
           [ 1.14936335e-01,  2.69477011e-01],
           [ 1.39471973e-01,  2.18792684e-01],
           [ 1.58709138e-01,  1.64159774e-01],
           [ 1.72341858e-01,  1.07376701e-01],
           [ 1.80243801e-01,  5.02170689e-02],
           [ 1.82463197e-01, -5.62123894e-03],
           [ 1.79212881e-01, -5.85622380e-02],
           [ 1.70855985e-01, -1.07192185e-01],
           [ 1.57887912e-01, -1.50293175e-01],
           [ 1.40915311e-01, -1.86869344e-01],
           [ 1.20632813e-01, -2.16165278e-01],
           [ 9.77983079e-02, -2.37676424e-01],
           [ 7.32075851e-02, -2.51151649e-01],
           [ 4.76690987e-02, -2.56588223e-01],
           [ 2.19795918e-02, -2.54219734e-01],
           [-3.09873955e-03, -2.44497674e-01],
           [-2.68589505e-02, -2.28067501e-01],
           [-4.86674160e-02, -2.05740192e-01],
           [-6.79788598e-02, -1.78460311e-01],
           [-8.43480530e-02, -1.47271718e-01],
           [-9.74380008e-02, -1.13282048e-01],
           [-1.07024548e-01, -7.76270369e-02],
           [-1.12997443e-01, -4.14357813e-02],
           [-1.15358006e-01, -5.79788809e-03],
           [-1.14213629e-01,  2.82666093e-02],
           [-1.09769431e-01,  5.98338166e-02],
           [-1.02317451e-01,  8.80984204e-02],
           [-9.22238117e-02,  1.12391550e-01],
           [-7.99143379e-02,  1.32193748e-01],
           [-6.58591253e-02,  1.47142925e-01],
           [-5.05565614e-02,  1.57037300e-01],
           [-3.45172979e-02,  1.61833489e-01],
           [-1.82486470e-02,  1.61640028e-01],
           [-2.23983493e-03,  1.56706740e-01],
           [ 1.30514820e-02,  1.47410493e-01],
           [ 2.72111227e-02,  1.34237906e-01],
           [ 3.98784229e-02,  1.17765682e-01],
           [ 5.07542116e-02,  9.86392579e-02],
           [ 5.96065967e-02,  7.75504737e-02],
           [ 6.62744819e-02,  5.52149751e-02],
           [ 7.06688393e-02,  3.23500227e-02],
           [ 7.27717970e-02,  9.65333856e-03],
           [ 7.26336834e-02, -1.22164388e-02],
           [ 7.03682113e-02, -3.26571938e-02],
           [ 6.61460356e-02, -5.11382596e-02],
           [ 6.01869530e-02, -6.72125140e-02],
           [ 5.27510391e-02, -8.05254290e-02],
           [ 4.41290354e-02, -9.08209578e-02],
           [ 3.46323078e-02, -9.79442377e-02],
           [ 2.45826886e-02, -1.01841190e-01],
           [ 1.43025071e-02, -1.02555178e-01],
           [ 4.10509572e-03, -1.00220965e-01],
           [-5.71397588e-03, -9.50562807e-02],
           [-1.48847166e-02, -8.73513683e-02],
           [-2.31693954e-02, -7.74569047e-02],
           [-3.03679467e-02, -6.57707453e-02],
           [-3.63220085e-02, -5.27239153e-02],
           [-4.09175312e-02, -3.87663328e-02],
           [-4.40859626e-02, -2.43526600e-02],
           [-4.58040350e-02, -9.92871470e-03],
           [-4.60922337e-02,  4.08119800e-03],
           [-4.50120538e-02,  1.72856986e-02],
           [-4.26621872e-02,  2.93362230e-02],
           [-3.91737998e-02,  3.99351709e-02],
           [-3.47050959e-02,  4.88421445e-02],
           [-2.94353420e-02,  5.58782266e-02],
           [-2.35585689e-02,  6.09282254e-02],
           [-1.72771482e-02,  6.39409421e-02],
           [-1.07954393e-02,  6.49275402e-02],
           [-4.31368716e-03,  6.39581692e-02],
           [ 1.97765943e-03,  6.11570151e-02],
           [ 7.90306528e-03,  5.66960110e-02],
           [ 1.33063359e-02,  5.07874454e-02],
           [ 1.80542610e-02,  4.36757518e-02],
           [ 2.20394035e-02,  3.56287536e-02],
           [ 2.51819948e-02,  2.69286518e-02],
           [ 2.74309256e-02,  1.78630283e-02],
           [ 2.87638445e-02,  8.71613240e-03],
           [ 2.91864079e-02, -2.39306389e-04],
           [ 2.87307434e-02, -8.74952811e-03],
           [ 2.74532105e-02, -1.65862707e-02],
           [ 2.54315592e-02, -2.35522285e-02],
           [ 2.27615988e-02, -2.94853359e-02],
           [ 1.95534995e-02, -3.42618035e-02],
           [ 1.59278522e-02, -3.77978836e-02],
           [ 1.20116157e-02, -4.00503712e-02],
           [ 7.93407486e-03, -4.10158893e-02],
           [ 3.82292821e-03, -4.07290402e-02],
           [-1.99386584e-04, -3.92595299e-02],
           [-4.01903279e-03, -3.67084026e-02],
           [-7.53369787e-03, -3.32035400e-02],
           [-1.06550374e-02, -2.88945878e-02],
           [-1.33105865e-02, -2.39474939e-02],
           [-1.54451110e-02, -1.85388367e-02],
           [-1.70213884e-02, -1.28501164e-02],
           [-1.80204191e-02, -7.06218181e-03],
           [-1.84410945e-02, -1.34994748e-03],
           [-1.82993527e-02,  4.12245437e-03],
           [-1.76268770e-02,  9.20596955e-03],
           [-1.64693946e-02,  1.37702627e-02],
           [-1.48846455e-02,  1.77066315e-02],
           [-1.29400987e-02,  2.09301420e-02],
           [-1.07104931e-02,  2.33809612e-02],
           [-8.27528669e-03,  2.50248869e-02],
           [-5.71609021e-03,  2.58530983e-02],
           [-3.11416444e-03,  2.58811736e-02],
           [-5.48049501e-04,  2.51474380e-02],
           [ 1.90860958e-03,  2.37107251e-02],
           [ 4.18898848e-03,  2.16476415e-02],
           [ 6.23470843e-03,  1.90494555e-02],
           [ 7.99714491e-03,  1.60186882e-02],
           [ 9.43837476e-03,  1.26655558e-02],
           [ 1.05317686e-02,  9.10435258e-03],
           [ 1.12622197e-02,  5.44989437e-03],
           [ 1.16260257e-02,  1.81412182e-03],
           [ 1.16304384e-02, -1.69705203e-03],
           [ 1.12929114e-02, -4.98657608e-03],
           [ 1.06400909e-02, -7.96862948e-03],
           [ 9.70658499e-03, -1.05706460e-02],
           [ 8.53355543e-03, -1.27347635e-02],
           [ 7.16718837e-03, -1.44188132e-02],
           [ 5.65709128e-03, -1.55968093e-02],
           [ 4.05466582e-03, -1.62589055e-02],
           [ 2.41151165e-03, -1.64108831e-02],
           [ 7.77900648e-04, -1.60732421e-02],
           [-7.98635319e-04, -1.52798610e-02],
           [-2.27456622e-03, -1.40763419e-02],
           [-3.61144889e-03, -1.25181215e-02],
           [-4.77680726e-03, -1.06683391e-02],
           [-5.74479375e-03, -8.59560656e-03],
           [-6.49662364e-03, -6.37173957e-03],
           [-7.02078378e-03, -4.06946867e-03],
           [-7.31301267e-03, -1.76027952e-03],
           [-7.37607266e-03,  4.87642282e-04],
           [-7.21932418e-03,  2.61124781e-03],
           [-6.85812738e-03,  4.55422110e-03],
           [-6.31309922e-03,  6.26831530e-03],
           [-5.60924821e-03,  7.71436370e-03],
           [-4.77502635e-03,  8.86299514e-03],
           [-3.84132387e-03,  9.69501328e-03],
           [-2.84044197e-03,  1.02014636e-02],
           [-1.80507533e-03,  1.03833963e-02],
           [-7.67331710e-04,  1.02513437e-02],
           [ 2.42181483e-04,  9.82454909e-03],
           [ 1.19518410e-03,  9.12997398e-03],
           [ 2.06644020e-03,  8.20112874e-03],
           [ 2.83435051e-03,  7.07676734e-03],
           [ 3.48140918e-03,  5.79949187e-03],
           [ 3.99451765e-03,  4.41431307e-03],
           [ 4.36515405e-03,  2.96720883e-03],
           [ 4.58940006e-03,  1.50372425e-03],
           [ 4.66782792e-03,  6.76642050e-05],
           [ 4.60526827e-03, -1.30011880e-03],
           [ 4.41045781e-03, -2.56277695e-03],
           [ 4.09559101e-03, -3.68835291e-03],
           [ 3.67579419e-03, -4.65048450e-03],
           [ 3.16853576e-03, -5.42889919e-03],
           [ 2.59299920e-03, -6.00971481e-03],
           [ 1.96943622e-03, -6.38553482e-03],
           [ 1.31851777e-03, -6.55534650e-03],
           [ 6.60707493e-04, -6.52424133e-03],
           [ 1.56706421e-05, -6.30296331e-03],
           [-5.98265584e-04, -5.90731860e-03],
           [-1.16458455e-03, -5.35746407e-03],
           [-1.66897588e-03, -4.67710064e-03],
           [-2.09965002e-03, -3.89260437e-03],
           [-2.44756023e-03, -3.03212050e-03],
           [-2.70653293e-03, -2.12464979e-03],
           [-2.87331116e-03, -1.19915451e-03],
           [-2.94750598e-03, -2.83709131e-04],
           [-2.93146652e-03,  5.95279962e-04],
           [-2.83007955e-03,  1.41377000e-03],
           [-2.65049987e-03,  2.15067188e-03],
           [-2.40183020e-03,  2.78832655e-03],
           [-2.09475750e-03,  3.31285466e-03],
           [-1.74116178e-03,  3.71437984e-03],
           [-1.35370872e-03,  3.98712298e-03],
           [-9.45439348e-04,  4.12937155e-03],
           [-5.29367996e-04,  4.14332975e-03],
           [-1.18101593e-04,  4.03486555e-03],
           [ 2.76511191e-04,  3.81315686e-03],
           [ 6.43690822e-04,  3.49025933e-03],
           [ 9.73991278e-04,  3.08061156e-03],
           [ 1.25951079e-03,  2.60049377e-03],
           [ 1.49405297e-03,  2.06746256e-03],
           [ 1.67322061e-03,  1.49976805e-03],
           [ 1.79445903e-03,  9.15786421e-04],
           [ 1.85704068e-03,  3.33473308e-04],
           [ 1.86199702e-03, -2.30143606e-04],
           [ 1.81200277e-03, -7.59419712e-04],
           [ 1.71121630e-03, -1.24048634e-03],
           [ 1.56508289e-03, -1.66156660e-03],
           [ 1.38011504e-03, -2.01322342e-03],
           [ 1.16364754e-03, -2.28852126e-03],
           [ 9.23581944e-04, -2.48310780e-03],
           [ 6.68126447e-04, -2.59521496e-03],
           [ 4.05540031e-04, -2.62558383e-03],
           [ 1.43887408e-04, -2.57732093e-03],
           [-1.09190231e-04, -2.45570220e-03],
           [-3.46677662e-04, -2.26787852e-03],
           [-5.62359431e-04, -2.02261209e-03],
           [-7.50965013e-04, -1.72991853e-03],
           [-9.08277890e-04, -1.40071545e-03],
           [-1.03120674e-03, -1.04646479e-03],
           [-1.11782219e-03, -6.78801085e-04],
           [-1.16735638e-03, -3.09181726e-04],
           [-1.18016783e-03,  5.14368857e-05],
           [-1.15767707e-03,  3.92899575e-04],
           [-1.10227236e-03,  7.06110899e-04],
           [-1.01719178e-03,  9.83250186e-04],
           [-9.06387198e-04,  1.21793999e-03],
           [-7.74373028e-04,  1.40536142e-03],
           [-6.26065857e-04,  1.54232050e-03],
           [-4.66620193e-04,  1.62726244e-03],
           [-3.01265866e-04,  1.66023638e-03],
           [-1.35150539e-04,  1.64281632e-03],
           [ 2.68080687e-05,  1.57797952e-03],
           [ 1.80054177e-04,  1.46995057e-03],
           [ 3.20511446e-04,  1.32401743e-03],
           [ 4.44678792e-04,  1.14632296e-03],
           [ 5.49705086e-04,  9.43642342e-04],
           [ 6.33441051e-04,  7.23153279e-04],
           [ 6.94468231e-04,  4.92205466e-04],
           [ 7.32104638e-04,  2.58096254e-04],
           [ 7.46388941e-04,  2.78588802e-05],
           [ 7.38044078e-04, -1.91931321e-04],
           [ 7.08422500e-04, -3.95327716e-04],
           [ 6.59435631e-04, -5.77156322e-04],
           [ 5.93470317e-04, -7.33129892e-04]])

- $y$ es una matriz de n filas y 2 columnas.
- La primer columna de $y$ corresponde a $y_1$.
- La segunda columna de $y$ corresponde a $y_2$.

¿Cómo extraemos los resultados $y_1$ y $y_2$ independientemente?

```python
# Extraer y1 y y2
y1 = y[:,0]
y2 = y[:,1]
```

### Para hacer participativamente...
- Graficar en una misma ventana $y_1$ vs. $t$ y $y_2$ vs. $t$... ¿qué pueden observar?

```python
# Gráfica
plt.figure(figsize=(8,6))
plt.plot(t, y1, 'b', lw=3, label='Posición [m]: $y_1(t)$')
plt.plot(t, y2, 'r', lw=3, label='Velocidad [m/s]: $y_2(t)$')
plt.xlabel("Tiempo [s] $t$")
plt.legend(loc="best")
plt.grid()
plt.show()
```

- Graficar $y_2/\omega_0$ vs. $y_1$... ¿cómo se complementan estos gráficos? ¿conclusiones?

```python
# Gráfica
omega0=(k/m)**0.5

plt.figure(figsize=(8,6))
plt.plot(y1, y2/omega0, 'b', lw=3)
plt.xlabel("Posición $y_1$")
plt.ylabel("Velocidad normalizada $y_2/\omega_0$")
plt.grid()
plt.show()
```

## Dependiendo de los parámetros, 3 tipos de soluciones

Teníamos
\begin{equation}
m\frac{d^2 x}{dt^2} + c\frac{dx}{dt} + kx = 0
\end{equation}

si recordamos que $\omega_0 ^2 = \frac{k}{m}$ y definimos $\frac{c}{m}\equiv 2\Gamma$, tendremos

\begin{equation}
\frac{d^2 x}{dt^2} + 2\Gamma \frac{dx}{dt}+ \omega_0^2 x = 0
\end{equation}

El comportamiento viene determinado por las raices de la ecuación característica. Ver en el tablero...

### Subamortiguado

Si $\omega_0^2 > \Gamma^2$ se tiene movimiento oscilatorio *subamortiguado*.

```python
omega0 = (k/m)**0.5
Gamma = c/(2*m)
```

```python
omega0**2, Gamma**2
```

    (2.9999999999999996, 0.0625)

```python
omega0**2 > Gamma**2
```

    True

Entonces, el primer caso que ya habíamos presentado corresponde a movimiento amortiguado.

```python
# Gráfica, de nuevo
plt.figure(figsize=(8,6))
plt.plot(t, y1, 'b', lw=3, label='Posición [m]: $y_1(t)$')
plt.plot(t, y2, 'r', lw=3, label='Velocidad [m/s]: $y_2(t)$')
plt.xlabel("Tiempo [s] $t$")
plt.legend(loc="best")
plt.grid()
plt.show()
```

### Sobreamortiguado

Si $\omega_0^2 < \Gamma^2$ se tiene movimiento oscilatorio *sobreamortiguado*.

```python
# Nuevas constantes
k = .1 # Constante del muelle
m = 1.0   # Masa
c = 1 # Constante de amortiguación
```

Simular y graficar...

```python
omega0 = np.sqrt(k/m)
Gamma = c/(2*m)
```

```python
omega0**2, Gamma**2
```

    (0.1, 0.25)

```python
omega0**2<Gamma**2
```

    True

```python
# Simular
y = odeint(amortiguado, y0, t, args=(k,m,c))
y1s = y[:,0]
y2s = y[:,1]
```

```python
# Graficar
plt.figure(figsize=(8,6))
plt.plot(t, y1s, 'b', lw=3, label='Posición [m]: $y_1(t)$')
plt.plot(t, y2s, 'r', lw=3, label='Velocidad [m/s]: $y_2(t)$')
plt.xlabel("Tiempo [s] $t$")
plt.legend(loc="best")
plt.grid()
plt.show()
```

### Amortiguamiento crítico

Si $\omega_0^2 = \Gamma^2$ se tiene movimiento *críticamente amortiguado*.

```python
# Nuevas constantes
k = .0625 # Constante del muelle
m = 1.0   # Masa
c = .5 # Constante de amortiguación
```

Simular y graficar...

```python
omega0 = np.sqrt(k/m)
Gamma = c/(2*m)
```

```python
omega0**2, Gamma**2
```

    (0.0625, 0.0625)

```python
omega0**2 == Gamma**2
```

    True

```python
# Simular
y = odeint(amortiguado, y0, t, args=(k,m,c))
y1c = y[:,0]
y2c = y[:,1]
```

```python
# Graficar
plt.figure(figsize=(8,6))
plt.plot(t, y1c, 'b', lw=3, label='Posición [m]: $y_1(t)$')
plt.plot(t, y2c, 'r', lw=3, label='Velocidad [m/s]: $y_2(t)$')
plt.xlabel("Tiempo [s] $t$")
plt.legend(loc="best")
plt.grid()
plt.show()
```

En resumen, se tiene entonces:

```python
tt = t
fig, ((ax1, ax2, ax3), (ax4, ax5, ax6)) = plt.subplots(2, 3, sharex='col',
                                                       sharey='row',figsize =(10,6))
ax1.plot(tt, y1, c = 'k')
ax1.set_title('Amortiguado', fontsize = 14)
ax1.set_ylabel('Posición', fontsize = 14)
ax2.plot(tt, y1s, c = 'b')
ax2.set_title('Sobreamortiguado', fontsize = 14)
ax3.plot(tt, y1c, c = 'r')
ax3.set_title('Crítico', fontsize = 16)
ax4.plot(tt, y2, c = 'k')
ax4.set_ylabel('Velocidad', fontsize = 14)
ax4.set_xlabel('tiempo', fontsize = 14)
ax5.plot(tt, y2s, c = 'b')
ax5.set_xlabel('tiempo', fontsize = 14)
ax6.plot(tt, y2c, c = 'r')
ax6.set_xlabel('tiempo', fontsize = 14)
plt.show()
```

> **Tarea**. ¿Cómo se ve el espacio fase para los diferentes casos así como para diferentes condiciones iniciales?

> En un gráfico como el anterior, realizar gráficas del plano fase para los distintos movimientos y para cuatro conjuntos de condiciones iniciales distintas
- y0 = [1, 1]
- y0 = [1, -1]
- y0 = [-1, 1]
- y0 = [-1, -1]

Hacer lo anterior en un nuevo notebook de jupyter llamado Tarea7_ApellidoNombre.ipynb y subir en el espacio habilitado.

Created with Jupyter by Lázaro Alonso. Modified by Esteban Jiménez Rodríguez.


Hasta el momento hemos trabajado independientemente con caminantes aleatorios y distribuciones de probabilidad. Ahora trataremos de juntar los dos tópicos.

Si $P_{t}(i)$ es la probabilidad de que un caminante se encuentra en el sitio $i$ al tiempo $t$, entonces la distribución de probabilidad está dada por el conjunto $\{ P_{t}(i): i \in \mathbb{Z} \}$. Abstrayendo un poco, este objeto puede ser visto también como un vector con el número de entradas igual al número de sitios en el cual puede estar nuestro caminante.

A este vector lo llamaremos $\mathbf{P}_{t}$.

**Nota 1**: Este es el primer ejemplo en cual podemos decir que el tiempo es discreto, por lo que $t \in \mathbb{N}$.

**Nota 2**: En principio, el *lugar* donde nuestros caminantes marchen puede ser *infinito* (por ahora en una dimensión), por lo que el vector $\mathbf{P}_{t}$ tendría una infinidad de entradas.

##  Ecuación maestra

Coloquemonos en nuestro paso $0$, i.e. $t=0$. El caminante estará en su condición inicial esperando a la bandera de salida. Supongamos que su condición inicial es en $i = 0$ y que el caminante tiene una probabilidad $p$ de dar un paso a la derecha y $q:= 1-p$ de darlo a la izquierda.

Para lo siguiente supondremos que $p = q = \frac{1}{2}$

No es dificil llegar entonces a que $P_{0}(i) = 0,   \ \forall i \neq 0$ y que $P_{0}(0) = 1 $

En el primer paso, tendremos que:
$$
\begin{matrix}
P_{1}(-1) = \frac{1}{2} & P_{1}(0) = 0 & P_{1}(1) = \frac{1}{2}
\end{matrix}
$$

¿Qué pasa en el siguiente paso?

El espacio que el caminante puede abarcar se hace más grande, yendo de $i =-2,\dotsc, 2$. Ahora veamos como queda la distribución de probabilidad.

$$
\begin{matrix}
P_{2}(-2) = \frac{1}{4} & P_{2}(-1) = 0 & P_{2}(0) = \frac{1}{2} & P_{2}(1) = 0 & P_{2}(2) = \frac{1}{4}
\end{matrix}
$$

Las cosas ya se han puesto un poco más interesantes. Para entender un poco más como se han calculado un poco más rigurosamente estas probabilidades tomemos el caso de $P_{2}(0)$.

en el paso $t = 1$, el caminante tenía $\frac{1}{2}$ de probabilidad de estar en $i= -1, 1$. Supongamos que estaba en la celda $i=-1$, entonces el caminante tiene $\frac{1}{2}$ de probabilidad de dar un paso a la derecha en el paso $t = 2$; y de la misma forma, si el caminante estuviera en la celda $i = 1$, también habría $\frac{1}{2}$ de probabilidad de que al siguiente paso estuviera de nuevo en la celda $i = 0$.

De esta manera llegamos a la **ecuación maestra** de nuestro ejemplo en una dimensión y con probabilidades $p  =q = \frac{1}{2}$

$$P_{t+1}(i) = \frac{1}{2}P_t(i-1) + \frac{1}{2}P_t(i+1) $$

La generalización de la **ecuación maestra** es la siguiente

$$
\begin{equation}
P_{t+1}(i) = pP_t(i-1) + (1-p)P_t(i+1) \ \ \ \ \ (1)
\end{equation}
$$

**[1a]** Pongamos manos a la obra. El objetivo general será graficar cómo evoluciona temporalmente la distribución de probabilidad con ayuda de nuestra ecuación maestra.

Para esto necesitaremos un `kernel` un poco más sofisticado de los que ya hemos hecho. El gran cambio es que aquí sólo jugaremos con *un solo* caminante aleatorio, y no con varios como lo habíamos hecho hasta entonces. La `malla` de `bloques` de `núcleos` pasa a ser el espacio en que se mueve el caminante. Cambio sutil pero de grandes consecuencias.

En primer lugar necesitamos el arreglo en el cual el caminante pueda moverse de un lado a otro. A este le llamaremos $X$. Recuerda hacerlo lo suficientemente grande para que el caminante no choque con los extremos.

Para calcular la distribución de probabilidades del paso $t+1$ necesitamos la distribución del paso $t$. Sin embargo al sobreescribir nuestro arreglo $X$ perderemos información, y por lo tanto nuestros cálculos serán incorrectos. Es por esto que necesitaremos declarar otro `arreglo` en el cual podamos copiar nuestra información del tiempo $t$ para calcular la distribución deseada.

Ahora, aquí es donde viene lo interesante. *La manera de copiar los datos*. Para esto nos basaremos en el ejemplo de *tiled programmation* hecho en el Notebook 6 de la primera parte de este curso el cual se basaba en declarar un `arreglo` tipo `__shared__` en el cual copiar los datos.

Así que veamos un poco como se vería el kernel. Supongamos que, tal cual dicho anteriormente, nuestros datos estuviesen en un arreglo $X$ y el lugar en el cual nos apoyaremos sea un arreglo en la memoria *compartida* llamado $X_{copia}$.

La idea general es entonces es de ir calculando los estados de X en el tiempo $t+1$ tesela por tesela. Supongamos entonces que X consiste de un arreglo de 200 celdas y buscamos calcular estas 200 celdas en el tiempo $t+1$ en grupos de 5. Entonces en $X_{copia}$ habremos de tener todos aquellas celdas con las cuales podamos realizar dichos cálculos.

En este caso en específico, puesto que el estado de una celda en el tiempo $t+1$ está determinada por ella misma **y sus vecinos**, entonces habremos de copiar cada uno de estos en $X_{copia}$. Esto no causa problemas a no ser por las celdas extremas de cada bloque. Para resolver esto tendremos que copiar también los vecinos que no aparecen en nuestro bloque de 5 celdas pero que también son necesarios para los cálculos.

De esta manera, para un arreglo de 200 celdas cuyos estados quieren ser calculados en bloques de 5, entonces necesitaremos en la memoria compartida teselas de 7 celdas.

A continuación mostramos la manera en la cual se copian los datos a la memoria compartida. En primer lugar mostraremos un programa en Python para darnos una idea más clara de que es lo que estamos buscando. Sólo después pasaremos al kernel.

```python
import numpy as np
```

Supongamos un arreglo A con 17 estados iniciales. Nuestra intención es calcular el estado de cada celda en el tiempo siguiente. Usaremos el método con teselas para copiar los datos. Cada tesela se ocupará de 4 datos en A, por lo que según la ecuación maestra (1) necesitaremos que la dimensión teselar sea de 6.

```python
A = np.array([1,2,3,4,5,6,7,8,9,10,11,12, 13, 14, 15, 16, 17])

tesela_A = np.ones(6)
```

```python
# blockDim es la número de celdas que serán copiadas a la tesela
# gridDim es el número de bloques que tendremos
# ANCHO_TESELA es el número de datos que necesitaremos para calcular blockDim celdas en el siguiente estado

blockDim = 4
gridDim = len(A)/blockDim+1
ANCHO_TESELA = blockDim+2

# Regresamos a tener dos bucles...

# El primer bucle va de bloque en bloque por A
for blockIdx in xrange(gridDim):

    # el segundo bucle busca los elementos de cada bloque
    for tx in xrange(ANCHO_TESELA-1):

        # los elementos necesarios son copiados a la tesela
        if blockDim*blockIdx + tx-1 >= 0 and blockDim*blockIdx + tx-1 < len(A) :
            tesela_A[tx] = A[blockDim*blockIdx + tx-1]

        # y si hemos llegado a los extremos de A, entonces se coloca un 0.
        else:
            tesela_A[tx] = 0.0

    # Este if else se dedica a colocar aquellos datos en la frontera derecha de la tesela
    if 4*(blockIdx+1) < len(A):
        tesela_A[ANCHO_TESELA-1] = A[blockDim*(blockIdx+1)]
    else:
        tesela_A[ANCHO_TESELA-1] = 0.

    print B
```

    [ 0.  1.  2.  3.  4.  5.]
    [ 4.  5.  6.  7.  8.  9.]
    [  8.   9.  10.  11.  12.  13.]
    [ 12.  13.  14.  15.  16.  17.]
    [ 16.  17.   0.   0.   0.   0.]

Nota como en cada tesela los 4 valores centrales corresponden a aquellas celdas cuyo estado será calculado al tiempo $t+1$. En el caso de la última tesela, puesto que los valores de A ya fueron cubiertos, entonces la tesela es llenada con $0$'s para que no haya cálculos erróneos.

Ahora sí podemos pasar al kernel en CUDA C. Algunos nombres fueron cambiados debido al modo de escribir los programas, sin embargo la idea es la misma. Entre estos cambios notamos que `blockDim` fue cambiado a `TAMANIO_BLOQUE`y la introducción de `tesela_idx` que es en realidad `blockDim*blockIdx + tx-1` con el que trabajamos en Python.

Este índice es usado puesto que para cada dato en A, necesitamos también su vecino a la izquierda (idx-1). Con `tesela_idx` cubrimos cada uno de estos. Ahora sólo falta el vecino derecho de la última celda. Este estará cubierto por otro pequeño programa condicional con un `if else`.

```C++

__shared__ float tesela_X[TAMANIO_BLOQUE+2] ;

int tx = threadIdx.x ;
int idx = blockIdx.x*TAMANIO_BLOQUE + tx ;
int tesela_idx = idx - 1 ;

if tx < TAMANIO_BLOQUE {
    if ((tesela_idx >= 0) && (tesela_idx < Dim_Camino) ) {

        tesela_X[tx] = X[tesela_idx] ;

        } else {

        tesela_X[tx] = 0.0f ;

        }
    __syncthreads() ;
}

if blockDim.x*(blockIdx.x+1) < Dim_Camino {

    tesela_X[TAMANIO_BLOQUE+1] = X[blockDim.x*(blockIdx.x +1)] ;

} else {

    tesela_X[TEMANIO_BLOQUE+1] = 0.0f ;

}

__syncthreads() ;

```

Una vez hecha la copia de los datos de `X` en `tesela_X` sólo falta reescribir `X` con los nuevos valores. Eso quedará de ustedes.

También es importante fijar el tamaño de los bloques `TAMANIO_BLOQUE`. Así que es hora de que el lector se ponga a trabajar y complete el `kernel` para luego graficar la evolución temporal de la distribución de probabilidad del caminante aleatorio. Supón en un primer tiempo que $p = q = \frac{1}{2}$.

Recomendamos graficar con la función `imshow()` de `matplotlib`.

A este método de resolver la ecuación maestra numéricamente se le llama **enumeración exacta** y es sumamente importante y utilizado para resolver ecuaciones diferenciales parciales.

**[1b]** Una vez que hayas obtenido las imágenes, cambia el valor de $p$ para ver como varía la distribución de probabilidad.

##  Dos dimensiones

**[2]** Escribe la ecuación maestra del caminante aleatorio en dos dimensiones.

**[3]** Modifica tu código para obtener una seria de imágenes con las que puedas observar la evolución temporal de la distribución de probabilidad en 2 dimensiones.

**Hint**: En este caso tendrás que hacer una matriz tipo `__shared__` y no un arreglo unidimensional. Te recomendamos revisar los notebooks sobre multiplicación de matrices para que recuerdes la manera de indexar.

Ahora tendremos cuatro indices:

```C++

	int Fila = blockIdx.y*BLOCK_SIZE + ty ;
	int Columna = blockIdx.x*BLOCK_SIZE + tx ;
	int Fila_copia = Fila - 1 ; int Columna_copia = Columna - 1 ;

    if( (Fila_copia >= 0) && (Fila_copia < DimY) && (Columna_copia >= 0) && (Columna_copia < DimX) ) {
        ds_copiaPlano[ty][tx] = Plano[Fila_copia][Columna_copia];
    } else {
        ds_copiaPlano[ty][tx] = 0.0f ;
    }

```

En caso de que te pierdas con los índices y copias, haz un código en Python semejante al de arriba que te sirva como guía.

## Fronteras

Hasta ahora no nos hemos enfrentado con el problema de las fronteras, pero no podíamos escapar de él. Supongamos que son paredes *reflejantes* y no *absorbentes*, lo cual hará que el caminante "rebote" en las fronteras.

**[4]** Escribe la regla que tienen que seguir las probabilidades cuando un caminante llega a cualquiera de las cuatro fronteras.

**[5]** Implementa esta regla en tu código y observa qué pasa.

##  Una primera aproximación a las EDP

Supongamos ahora que los cambios en el espacio y en el tiempo del caminante aleatorio se dan por diferenciales $\delta x$ y $\delta t$. La ecuación (1) se vuelve entonces

$$ P(x, t+\delta t) = pP(x-\delta x, t) + qP(x+\delta x, t) $$

Si ahora expandemos cada termino en series de Taylor (hasta 2º orden), llegamos a que:

$$ \frac{\partial P}{\partial t}(x, t) = (q-p)\frac{\delta x}{\delta t}\frac{\partial P}{\partial x}(x, t)+ \frac{\delta x^2}{2\delta t}\frac{\partial^2 P}{\partial x^2}(x, t) $$

Si ahora volvemos de nuevo al caso $p = q = \frac{1}{2}$, y haciendo $D = \frac{\delta x^2}{2\delta t}$ obtenemos entonces la ya conocida ecuación de difusión.

$$\frac{\partial P}{\partial t}(x, t) = D\frac{\partial^2 P}{\partial x^2}(x, t)$$

**[6]** Las soluciones analíticas de esta EDP son bien conocidas. Compáralas con tu solución numérica.

Así, vemos que el método de enumeración exacta para un caminante aleatorio provee un método numérico para resolver esta EDP de evolución. [El método se llama de diferencias finitas.]

## Referencias

+ [Ecuación maestra](https://en.wikipedia.org/wiki/Master_equation)
+ Método de [diferencias finitas](https://en.wikipedia.org/wiki/Finite_difference_method)
+ [Ecuación de difusión](https://en.wikipedia.org/wiki/Diffusion_equation)

```python

```

Parte 2 - PyCUDA y aplicaciones/04 - Ecuacion maestra.ipynb


(Evaluate block to execute LaTeX definitions)
$ \newcommand{\ybar}{\overline{y}} $
$ \renewcommand{\d}[2]{\frac{d #1}{d #2}} $
$ \newcommand{\dd}[2]{\frac{d^2 #1}{d #2^2}} $
$ \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} $
$ \newcommand{\pdd}[2]{\frac{\partial^2 #1}{\partial #2^2}} $
$ \renewcommand{\b}{\beta} $
$ \newcommand{\m}{\mu} $
$ \renewcommand{\v}[1]{\mathbf{#1}} $
$ \newcommand{\N}{\mathcal{N}} $
$ \renewcommand{\l}{\lambda} $
$ \newcommand{\ol}[1]{\overline{#1}} $

# Calculating the Dispersion Relation for Zonostrophic Instability

This is the supplementary material for the paper [Dynamics of zonal flows: failure of wave-kinetic theory, and new geometrical optics approximations](https://www.cambridge.org/core/journals/journal-of-plasma-physics/article/dynamics-of-zonal-flows-failure-of-wave-kinetic-theory-and-new-geometrical-optics-approximations/5B51BB7D026E19E8F21801568ED6EA75).  The article may also be found on arXiv [here](https://arxiv.org/abs/1604.06904).

The dispersion relation for zonostrophic instability is shown in Figure 2 in the paper.  We show how to derive the general form of the dispersion relation for zonostrophic instability, and then how to compute it numerically for a given set of parameters.  Python code is included.

In each case, we reduce the general form of the dispersion relation to a simpler form by using a fluctuation spectrum that consists of a thin ring in wavevector space $\sim \delta(k - k_f)$ centered at $k_f$.  This reduction was first considered for CE2 by Srinivasan and Young (2012); more details can be found in Parker (Ph.D. Thesis, 2014).

If this document is being read as a PDF or webpage, an interactive ipython notebook can be found [here](https://github.com/jeffbparker/ZonalFlowWaveKinetic), or at a [direct link](https://github.com/jeffbparker/ZonalFlowWaveKinetic/raw/master/Supplemental_Material.ipynb) (right click, save as).  In the ipython notebook one can follow along the mathematics and run the Python code inside the notebook using the [Jupyter](http://jupyter.org/) software.  To view it non-interactively within a browser without the Jupyter software, see  [here](http://nbviewer.jupyter.org/github/jeffbparker/ZonalFlowWaveKinetic/blob/master/Supplemental_Material.ipynb).

## Coordinate Convention
This document uses the geophysical convention for coordinates.  To use the standard plasma coordinates, let $(x,y,\ybar,\b,U) \mapsto (-y, x, \ol{x}, \kappa, -U)$.

## General Form for Zonostrophic Instability
### Asymptotic WKE
We start from Eq. (3.2).  There is a homogeneous equilibrium, independent of $\ybar$, at $\N_H(k_x,k_y) = F(k_x,k_y) / 2\b\m$, $U=0$.  Using the form
  \begin{equation}
    \N = \N_H + e^{iq\ybar} e^{\l t} \N_1(k_x, k_y),
  \end{equation}
  \begin{equation}
    U = e^{iq\ybar} e^{\l t} U_1,
  \end{equation}
we linearize Eq. (3.2) for perturbations about the equilibrium.  We obtain
  \begin{equation}
    \l \N_1 = iqk_x U_1 \pd{\N_H}{k_y} - \frac{2i\b q k_x k_y}{\ol{k}^4} \N_1 - 2\m \N_1
  \end{equation}
  \begin{equation}
    (\l + \m) U_1 = iq \int \frac{d\v{k}}{(2\pi)^2} \frac{k_x k_y}{\ol{k}^4} \b \N_1.
  \end{equation}

The first equation is solved for $\N_1$ in terms of $U_1$ and then substituted into the second equation.  This procedure yields a single nonlinear equation for the eigenvalues $\l$ corresponding to zonostrophic instability.  One finds
  \begin{equation}
    \l + \m = -q^2 \int \frac{d\v{k}}{(2\pi)^2} \frac{k_x^2 k_y}{(\l + 2\m)\ol{k}^4 + 2i\b q k_x k_y} \b \pd{\N_H}{k_y}
  \end{equation}

When $q$ is large, $\l \sim q$.

### Zonostrophic Instability in CE2-GO
We follow the same procedure, starting from Eq. (4.1).  After linearizing about the homogeneous equilibrium $W_H(k_x,k_y) = F(k_x,k_y) / 2\m$, we find
  \begin{equation}
    \l W_1 = iqk_x U_1 \pd{}{k_y} \left[ \left( 1 - \frac{q^2}{\ol{k}^2} \right) W_H \right] - \frac{2i\b q k_x k_y}{\ol{k}^4} W_1 - 2\m W_1,
  \end{equation}
  \begin{equation}
    (\l + \m)U_1 = iq \int \frac{d\v{k}}{(2\pi)^2} \frac{k_x k_y}{\ol{k}^4} W_1.
  \end{equation}

Solving for $W_1$ and substituting into the second equation yields
  \begin{equation}
    \l + \m = -q^2 \int \frac{d\v{k}}{(2\pi)^2} \frac{k_x^2 k_y}{(\l + 2\m)\ol{k}^4 + 2i\b q k_x k_y} \pd{}{k_y} \left[ \left( 1 - \frac{q^2}{\ol{k}^2} \right) W_H \right]
  \end{equation}

### Zonostrophic Instability in the WKE
We follow the same procedure, starting from Eq. (4.3).  We linearize about the homogeneous equilibrium $\N_H(k_x,k_y) = F(k_x,k_y) / 2\m \b$, and find

  \begin{equation}
    \l \N_1 = iqk_x U_1 \left( 1 - \frac{q^2}{\ol{k}^2} \right) \pd{\N_H}{k_y} - \frac{2i\b q k_x k_y}{\ol{k}^4} \N_1 - \frac{q^2 F}{\b^2} U_1 - 2 \m \N_1,
  \end{equation}
  \begin{equation}
    (\l + \m) U_1 = iq \int \frac{d\v{k}}{(2\pi)^2} \frac{k_x k_y}{\ol{k}^4} (\b \N_1 - U_1'' \N_H).
  \end{equation}

Under the assumption that $F$ and $\N_H$ are even in $k_x$ and in $k_y$, the integral over $U_1'' \N_H$ vanishes.  Solving for $\N_1$ gives
  \begin{equation}
    \N_1 = \frac{iqk_x U_1 (1 - q^2/\ol{k}^2)}{\l + 2\m + 2i\b q k_x k_y / \ol{k}^4} \pd{\N_H}{k_y} - \frac{q^2 F U_1}{\b^2 (\l + 2\m + 2i\b q k_x k_y / \ol{k}^4)}
  \end{equation}
Plugging this into the zonal flow equation yields the dispersion relation,
  \begin{equation}
    \l + \m = -q^2 \int \frac{d\v{k}}{(2\pi)^2} \left[ \frac{\b k_x^2 k_y (1 - q^2/\ol{k}^2)}{(\l+2\m) \ol{k}^4 + 2i\b q k_x k_y} \pd{\N_H}{k_y} + \frac{qF}{\b} \frac{i k_x k_y}{(\l + 2\m) \ol{k}^4 + 2i\b q k_x k_y} \right]
  \end{equation}

The last term here has no analog in the other models and its presence is a consequence of the invalidity of this model --- wave action is not conserved during this instability.

### Zonostrophic Instability in CE2
The same procedure can be applied to Eq. (2.3) with a little more algebraic complexity.  See Srinivasan and Young (2012) or Parker (PhD Thesis, Section 3.2) for the derivation.  Using Eqs. (3.25) and (3.26) of Parker (PhD Thesis), and taking $\ol{q}^2 = q^2$ (corresponding to $L_d^{-2} = 0$ for the zonal flows) and the limit of zero viscosity $\nu=0$, the dispersion relation is
  \begin{equation}
    \l + \m = -q \int \frac{d\v{k}}{(2\pi)^2} \frac{k_x^2 k_y}{(\l + 2\m) \ol{h}^2_1 \ol{h}^2_2 + 2i\b qk_xk_y } \left[ \left( 1 - \frac{q^2}{\ol{h}_1^2} \right) W_H(k_x, k_y + \tfrac{1}{2}q) - \left(1 - \frac{q^2}{\ol{h}_2^2} \right) W_H(k_x, k_y - \tfrac{1}{2}q) \right],
  \end{equation}

where $\ol{h}_{1,2}^2 = k_x^2 + (k_y \pm \tfrac{1}{2} q)^2 + L_d^{-2}$.  After a Taylor expansion for small $q$, this equation reduces to the CE2-GO dispersion relation.

## Reduction for Thin-ring, isotropic forcing
A thin-ring forcing isotropic in wavevector space is a convenient simplification.  We use
  \begin{equation}
    F(k) = 4\pi \varepsilon k_f \delta(k-k_f).
  \end{equation}
If $L_d^{-2} = 0$, then $\varepsilon$ is equal to the energy input into the system by the forcing.  With this forcing, the dispersion relations above can be simplified by converting the integrals to polar coordinates.  One is left with only one integral, the angle integral, that must be computed numerically.

For some of these computations we will integrate by parts, and we need the relation

  \begin{equation}
    \pd{}{k_y} \frac{k_y}{(\l + 2\m) \ol{k}^4 + 2i\b qk_x k_y} = \frac{(\l + 2\m) \ol{k}^2 (\ol{k}^2 - 4k_y^2)}{[(\l + 2\m) \ol{k}^4 + 2i\b qk_x k_y]^2}
  \end{equation}

### Asymptotic WKE
First we apply integration by parts in $k_y$ for the dispersion relation, then substitute $\N_H = F/2\m \b$.  This yields

  \begin{equation}
    \l + \m = q^2 \int \frac{d\v{k}}{(2\pi)^2} \frac{F}{2\m} \frac{(\l + 2\m) \ol{k}^2 k_x^2 (\ol{k}^2 - 4k_y^2)} {[(\l + 2\m) \ol{k}^4 + 2i\b qk_x k_y]^2}
  \end{equation}

For isotropic $F$, we use polar coordinates with $k_x = k\sin \phi$, $k_y = -k \cos \phi$.  The dispersion relation becomes

  \begin{equation}
    \l + \m = q^2 \int_0^\infty dk\, \frac{F(k)}{4\pi \m} k^2 (\l + 2\m)(1+m) \times \int_0^{2\pi} \frac{d\phi}{2\pi} \frac{\sin^2 \phi (1 + m - 4\cos^2 \phi)}{ [(\l + 2\m)k^2 (1+m)^2 - 2i\b q \cos\phi \sin\phi]^2}
  \end{equation}

where $m = (k L_d)^{-2}$.  Substituting the thin-ring forcing $F = 4\pi \varepsilon k_f \delta(k-k_f)$, we obtain the final form of the dispersion relation,

  \begin{equation}
    \l + \m = q^2 \frac{\varepsilon}{\m} k_f^4 (\l + 2\m) (1 + m_f) \int_0^{2\pi} \frac{d\phi}{2\pi} \frac{(1 + m_f - 4\cos^2\phi) \sin^2 \phi}{[(\l + 2\m)k_f^2 (1 + m_f)^2 - 2i\b q \cos\phi \sin \phi]^2}
  \end{equation}

with $m_f = (k_f L_d)^{-2}$.

Now, this is a nonlinear equation that we can solve for $\l$.  For most choices of the spectrum W_H, an unstable perturbation with $\text{Re } \l > 0$, has an eigenvalue $\l$ that is purely real.  It can be seen that if $\l$ is real, the imaginary part of the $\phi$ integral vanishes, so that one can work only with the real part.  Working only with real $\l$ rather than complex $\l$ has the advantage that robust one-dimensional root finders can be used.  For the integrand above, we have
    \begin{equation}
    \frac{A}{(B + iC)^2} = \frac{A(B^2 - C^2)}{(B^2 + C^2)^2} + \text{imag. part}
    \end{equation}
where $A$, $B$, and $C$ are assumed real.  If $\l$ is real, the imaginary part vanishes after integration, so we drop this term.  We do indeed find real solutions for $\l$.

Below is Python code to solve the dispersion for zonostrophic instability within the Asymptotic WKE.

```python
# Some python setup
from __future__ import division
import numpy as np
import scipy.optimize
import scipy.integrate
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

matplotlib.style.use('ggplot')
plt.rcParams['figure.figsize'] = (8.0, 8.0)
font = {'family' : 'normal',
        'weight' : 'bold',
        'size'   : 14}
plt.rc('font', **font)
```

```python
def lambda_of_params(beta, mu, epsilon, Ldinvsq, kf, q):
    """Given parameters, calculate the eigenvalue lambda for the asymptotic WKE."""

    funres = lambda lamb: disp_relation_residual(beta, mu, epsilon, Ldinvsq, kf, q, lamb)
    domainmin = -2*mu
    domainmax = 100
    lamb = scipy.optimize.brentq(funres, domainmin, domainmax) #1D root finder
    return lamb

def disp_relation_residual(beta, mu, epsilon, Ldinvsq, kf, q, lamb):
    """Compute the residual for the dispersion relation.  lamb must be real."""
    mf = Ldinvsq / (kf*kf)
    lhs = lamb + mu
    rhs = q*q*epsilon/mu * kf**4 * (lamb + 2*mu) * (1+mf) * polarint(beta, mu, epsilon, Ldinvsq, kf, q, lamb)
    res = rhs - lhs
    return res

def polarint(beta, mu, epsilon, Ldinvsq, kf, q, lamb):
    """Carry out the polar integral"""
    fun = lambda phi: polarint_integrand(beta, mu, epsilon, Ldinvsq, kf, q, lamb, phi)
    out, err = scipy.integrate.quad(fun, 0, 2*np.pi, epsabs=2e-6, epsrel=2e-6)
    out = out / (2*np.pi)
    return out

def polarint_integrand(beta, mu, epsilon, Ldinvsq, kf, q, lamb, phi):
    """ As explained above, integrand is A/(B+iC)^2.  Keep only the real part.
    So use A(B^2 - C^2) / (B^2 + C^2)^2."""
    c = np.cos(phi)
    s = np.sin(phi)
    mf = Ldinvsq / (kf*kf)
    A = (1 + mf - 4*c*c) * s * s
    B = (lamb + 2*mu) * kf**2 * (1+mf)**2
    C = -2*beta*q*c*s

    out = A*(B*B - C*C)  / (B*B + C*C)**2
    return out

# Now, let's go and calculate the dispersion relation lambda(q)
beta = 1
mu = 0.02
epsilon = 1
Ldinvsq = 1
kf = 1

q1 = np.logspace(-4, 0, 150)
q2 = np.linspace(1.02, 2.2, 150)
q = np.concatenate([q1, q2])
lamb = np.zeros_like(q)

for j in range(len(q)):
    lamb[j] = lambda_of_params(beta, mu, epsilon, Ldinvsq, kf, q[j])

fig = plt.figure()
plt.plot(q, lamb)
plt.xlabel(r'$q$')
plt.ylabel(r'$\lambda$ (Asymptotic WKE)')
```

### CE2-GO
The only difference for the general dispersion relation in CE2-GO as compared to the asymptotic WKE is the presence of a $(1 - q^2 / \ol{k}^2)$ term.  For the same isotropic thin-ring forcing, the dispersion relation reduces to the same result except that in CE2-GO there is an additional factor of
  \begin{equation}
    1 - \frac{q^2}{k_f^2 (1+m_f)}
  \end{equation}
in front of the polar integral.

```python
def CE2GO_lambda_of_params(beta, mu, epsilon, Ldinvsq, kf, q):
    """Given parameters, calculate the eigenvalue lambda."""

    funres = lambda lamb: CE2GO_disp_relation_residual(beta, mu, epsilon, Ldinvsq, kf, q, lamb)
    domainmin = -2*mu
    domainmax = 100
    lamb = scipy.optimize.brentq(funres, domainmin, domainmax)  # 1D root finder
    return lamb

def CE2GO_disp_relation_residual(beta, mu, epsilon, Ldinvsq, kf, q, lamb):
    """Compute the residual for the dispersion relation.  lamb must be real."""
    mf = Ldinvsq / (kf*kf)
    lhs = lamb + mu
    rhs = q*q*epsilon/mu * kf**4 * (lamb + 2*mu) * (1+mf) * (1-q*q/(kf*kf*(1+mf))) * CE2GO_polarint(beta, mu, epsilon, Ldinvsq, kf, q, lamb)
    res = rhs - lhs
    return res

def CE2GO_polarint(beta, mu, epsilon, Ldinvsq, kf, q, lamb):
    """Carry out the polar integral.  Same as for asymptotic WKE"""
    fun = lambda phi: CE2GO_polarint_integrand(beta, mu, epsilon, Ldinvsq, kf, q, lamb, phi)
    out, err = scipy.integrate.quad(fun, 0, 2*np.pi, epsabs=2e-6, epsrel=2e-6)
    out = out / (2*np.pi)
    return out

def CE2GO_polarint_integrand(beta, mu, epsilon, Ldinvsq, kf, q, lamb, phi):
    """ As explained above, integrand is A/(B+iC)^2.  Keep only the real part.
    So use A(B^2 - C^2) / (B^2 + C^2)^2."""
    c = np.cos(phi)
    s = np.sin(phi)
    mf = Ldinvsq / (kf*kf)
    A = (1 + mf - 4*c*c) * s * s
    B = (lamb + 2*mu) * kf**2 * (1+mf)**2
    C = -2*beta*q*c*s

    out = A*(B*B - C*C)  / (B*B + C*C)**2
    return out

# Now, let's go and calculate the dispersion relation lambda(q)
beta = 1
mu = 0.02
epsilon = 1
Ldinvsq = 1
kf = 1

q1 = np.logspace(-4, 0, 150)
q2 = np.linspace(1.02, 2.2, 150)
q = np.concatenate([q1, q2])
lamb = np.zeros_like(q)

for j in range(len(q)):
    lamb[j] = CE2GO_lambda_of_params(beta, mu, epsilon, Ldinvsq, kf, q[j])

fig = plt.figure()
plt.plot(q, lamb)
plt.xlabel(r'$q$')
plt.ylabel(r'$\lambda$ (CE2-GO)')
```

### WKE
For the WKE, the dispersion relation is calculated similarly (details and code are omitted here).  However, a brief note is warranted on how the dispersion relation in Figure 2 is obtained.  Because the WKE is invalid for calculating the dispersion relation, it is not wholly surprising to see strange behavior.  When one calculates the dispersion relation about an equilibrium that balances forcing and dissipation, one finds that for some values of $q$, the eigenvalue $\l$ becomes complex, even though the correct answer is that $\l$ is real.  The source of this can be traced to a linearization of the $F/(\b - U'')$ term, which gives $F U_1'' / \b^2$.

The WKE dispersion relation in Figure 2 is obtained by neglecting the $F U_1'' / \b^2$ term, which is justifiable through an alternate procedure.  There are two ways of obtaining the equilibrium incoherent spectrum that we linearize about.  The first way to realize the equilibrium is the route we have been using, which is a balance between external forcing and linear dissipation.  An alternate route is to remove forcing and dissipation, in which case *any* homogeneous spectrum is an equilibrium.  This alternate procedure yields effectively the same dispersion relation, although within the WKE linearization it has the effect of removing the $FU_1''/\b^2$ term in the linearization.  This procedure yields a real $\l$ and is the one shown in Figure 2.

The main point here is solely to demonstrate quantitatively that the WKE is not correct, which is to be expected because one had to assume that $U$ varied slowly in time in order to derive the WKE.

### CE2
The CE2 dispersion relation is obtained in a similar way (details and code omitted).  Since there is no $k_y$ derivative, one does not use an integration by parts but rather a shift of the integration variable, after which the isotropic form for $W_H$ can be inserted.  For details, see Srinivsan and Young (2012) or Parker (Ph.D. thesis, section 3.2.3).


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, C.D. Cooper, G.F. Forsyth.

# Riding the wave

We've reached the third lesson of the module _Riding the wave: Convection problems_, where we explore the numerical solution of conservation laws. We learned in the [first lesson](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/03_wave/03_01_conservationLaw.ipynb) how to think about conservation laws, starting with the most intuitive case: that of conservation of mass. But there are many physical quantities that can be conserved: energy, for example. Or cars on a road.

Developing a conservation law for traffic in a one-lane roadway is fun, because we can relate to the insights it gives. We've all experienced traffic slowing down to a crawl when the density of cars gets very high, and stepping on the accelerator pedal with glee when there are no cars on the road!

In the previous two lessons, we developed a traffic-flow model and explored [different choices of numerical scheme](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/03_wave/03_02_convectionSchemes.ipynb).  Not everything worked as you expected, and by now you might be realizing how some restrictions come about from the numerical methods themselves, while others still are imposed by the models we use. This lesson will develop a better traffic model, and also show you some impressive SymPy kung fu.

## Traffic flow, revisited

### A better flux model

Like you saw in the first lesson, cars obey a typical conservation law,

\begin{equation}
 \frac{\partial \rho}{\partial t} + \frac{\partial F}{\partial x} = 0, \end{equation}

where $F$ is the flux, $F=\rho u$—flux equals density times velocity. From our experience on the road, we know that the traffic speed is a function of traffic density, and we proposed as a first approximation a linear relation between the two. Thus,

\begin{equation}
F(\rho) = \rho \, u_{max}\left(1 - \frac{\rho}{\rho_{max}} \right)
\end{equation}

This flux model meets the two requirements, based on a qualitative view of traffic flow, that:
1.  $u \rightarrow u_{max}$ and $F\rightarrow 0$ when $\rho \rightarrow 0$.
2.  $u \rightarrow 0$ as $\rho \rightarrow \rho_{max}$

However, it leads to some unrealistic or at least improbable results.  For example, note that if the traffic speed is a linear function of density, the flux function will be quadratic (see Figure 1). In this case, the maximum flux will occur when $\rho^{\star} = \rho_{max}/2$, corresponding to a traffic speed $u_{max}/2$.

#### Figure 1. Traffic speed (left) and flux (right) vs. density.

A good question to ask here is:  should the maximum flux on a given stretch of road have a strict dependence on the maximum speed that the roadway allows, be it by physical restrictions or posted speed limits? In other words, do we really expect the maximum flux to increase if we allow arbitrarily high speeds?

Probably not.  But there *should* be some ideal traffic speed, $u^{\star}$, corresponding to an ideal traffic density, $\rho^{\star}$, resulting in the maximum traffic flux:

\begin{equation}F_{\rm max} = \rho^{\star}u^{\star}\end{equation}

Let us improve the initial flux model by taking this observation into account. One thing that we can try is to introduce a flux model that is cubic in $\rho$ instead of quadratic:

\begin{equation}F(\rho) = u_{\rm max}\rho (1 - A\rho - B \rho^2)\end{equation}

This new model still meets the first criterion listed above: $F\rightarrow 0$ when $\rho \rightarrow 0$. Moreover, we impose the following conditions:

*  When $\rho = \rho_{\rm max}$ traffic flux goes to zero:

\begin{equation}F(\rho_{\rm max}) = 0 = u_{\rm max}\, \rho_{\rm max}(1 - A \rho_{\rm max} - B \rho_{\rm max}^2)\end{equation}

*  Based on eq. (3), maximum flux occurs when $\rho = \rho^{\star}$ and $F'(\rho^{\star}) = 0$:

\begin{equation}
F'(\rho^{\star}) = 0  = u_{\rm max}(1 - 2A\rho^{\star} - 3B(\rho^{\star})^2)\end{equation}

*  $u^{\star}$ is obtained when $\rho = \rho^{\star}$:

\begin{equation}
u^{\star} = u_{\rm max}(1 - A \rho^{\star} - B(\rho^{\star})^2)\end{equation}

We have three equations and four unknowns $A,B,\rho^{\star}, u^{\star}$. However, in practice, the ideal traffic speed could be obtained for a given road by observations. Similarly to $u_{max}$ and $\rho_{max}$ it will therefore be taken as a parameter.

### Solving the new flux equation

Equations $(5)$, $(6)$, and $(7)$ are not incredibly difficult to solve with pen and paper. Instead of following that route, we can use [SymPy](http://sympy.org/en/index.html), the symbolic mathematics library of Python.  You used SymPy already in the [Burgers' equation lesson](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_04_1DBurgers.ipynb) of Module 2, _"Space & Time"_, and you will learn some new functionalities here.

We begin by importing SymPy, initializing $\LaTeX$ printing and defining a set of symbolic variables that we'll use in the calculations. Remember: variables are not defined automatically, you have to tell SymPy that a name will correspond to a symbolic variable by using the keyword `symbols`. This behavior is different from many other symbolic math systems that implicitly construct symbols for you, so you may be perplexed if you've used these other sytems before. The reason for this behavior in SymPy is that the system is fully built on Python, a general-purpose language that needs you to define all objects before using them.

```python
import sympy
sympy.init_printing()

(u_max, u_star, rho_max,
 rho_star, A, B) = sympy.symbols('u_max u_star rho_max rho_star A B')
```

Notice that we used the special character `_` (the under-dash) to create a symbol with a subscript. Here, for example, we've created the symbols $u_{max}$ and $u_{star}$, representing $u_{\rm max}$ and $u^{\star}$ ($u$-star) in the equations, and assigned them to the variable names `u_max` and `u_star`. SymPy also allows you to create symbols with a superscript by means of the special character `^`. Be careful, though: SymPy is built on Python, so you denote an exponent in an expression by `**` (this may also be different from other symbolic math systems that you have used in the past).

Next, use `sympy.Eq()` to define the three equations—corresponding to Equations $(5)$, $(6)$ and $(7)$, above—in terms of symbolic variables. The function `sympy.Eq()` creates an equality between two SymPy objects, passed as arguments separated by a comma. We need to remember that the equal sign in Python is used for variable assignment, and for that reason it cannot be used to build a symbolic equation with SymPy. That is why we need `sympy.Eq()` to create symbolic equalities. But the equal sign _is_ used to _assign_ an equation to a name: here, `eq1`, `eq2`, `eq3` are names for the symbolic equalities we create with `sympy.Eq()`.

```python
eq1 = sympy.Eq( 0, u_max*rho_max*(1 - A*rho_max-B*rho_max**2) )
eq2 = sympy.Eq( 0, u_max*(1 - 2*A*rho_star-3*B*rho_star**2) )
eq3 = sympy.Eq( u_star, u_max*(1 - A*rho_star - B*rho_star**2) )
```

Check this out: you can display these equations in pretty typeset mathematics just by executing their name in a code cell:

```python
eq1
```

```python
eq2
```

```python
eq3
```

As stressed above, we have three equations with three unknowns (assuming we have some observed value of $u_{star}$, corresponding to the ideal traffic speed)—there must be a solution!

To eliminate the term with $B$ in `eq2`, leaving it only in terms of $A$ and $\rho^{\star}$, we might subtract `3*eq3`.  But this will not work in SymPy if you attempt equation subtraction using the equation names. Just like we couldn't use the equal sign (which in Python means assignment) to create a symbolic equality (needing `sympy.Eq` instead), we can't use mathematical operators directly on the SymPy equations, because they are really _equalities_. What does it mean to subtract two _equalities_? It doesn't make sense. Try it ...

```python
eq2 - 3*eq3
```

See?  SymPy just printed out what you suggested but it did not manipulate algebraically the left and right sides of `eq2` like we were aiming for. What we _can_ do is create a _new_ equation, perform the left-hand side (LHS) and right-hand side (RHS) operations separately and then recombine them into our desired result. For this, it is helpful to know that there are built-in properties of a SymPy equality for the left-hand side and the right-hand side of the equality. Check it out:

```python
eq4 = sympy.Eq(eq2.lhs - 3*eq3.lhs, eq2.rhs - 3*eq3.rhs)
eq4
```

That still needs a little work. SymPy offers several methods to help reduce expressions. You can use [`simplify()`](http://docs.sympy.org/latest/modules/simplify/simplify.html?highlight=simplify#sympy.simplify.simplify.simplify) to attempt to make the expression "simpler," but you can imagine that the quality of an expression being simple is not well defined. One may expect the simpler expression to be shorter, maybe; but not always. The SymPy `simplify()` function applies several strategies, heuristically, to give you an equivalent reduced expression. But some expressions are uncooperative and `simplify()` gives up, returning the argument unchanged. Let's see what it can do with our expression `eq4`.

```python
eq4.simplify()
```

That is actually a useful result. We see that `eq4` allows solving for $\rho^{\star}$ in terms of $A$, for example (remember that $u_{max}$ is known: the road's maximum velocity). Then we could try to manipulate `eq1` to solve for $B$ in terms of $A$, as well, so that we can substitute back in `eq2` leaving everything in terms of $A$. We can do all that without the help of `simplify()`, but using it interactively to reason about the long expression helped us to decide on a course of action.

Another SymPy function that can help us examine complicated expressions is [`expand()`](http://docs.sympy.org/latest/modules/core.html?highlight=expand#sympy.core.function.expand). Its purpose is to expand bracketed factors in expressions and group powers of symbols. Let's see what that does here. Notice first that `eq4` hasn't changed; we just printed the result of applying `simplify()` to it.

```python
eq4
```

And now we print the result of applying `expand()` to it.

```python
eq4.expand()
```

That's very similar to our previous result, except without the bracketed factor. Simplifying an expression can be accomplished by expanding or factoring terms, or a combination of the two. Whether to `simplify()` or `expand()` depends on the situation and you just have to experiment.

We now have an idea of what to do with our three equations. We'll get expressions in terms of $A$ from `eq4` and `eq1` and substitute them back into `eq2`. For that, we can use the SymPy functions [`solve()`](http://docs.sympy.org/dev/modules/solvers/solvers.html#sympy.solvers.solvers.solve) and `subs()`, respectively. The arguments to `solve()` are the equality that needs to be solved, and the symbol to solve for.

**Note**: `sympy.solve()` always returns results in a *list*, since you often end up with multiple solutions for a given variable.  These linear equations will only return one solution, so we can skip a step and ask right away for the `[0]`-th element of the list.

```python
rho_sol = sympy.solve(eq4,rho_star)[0]
rho_sol
```

```python
B_sol = sympy.solve(eq1,B)[0]
B_sol
```

The arguments to the SymPy function [`subs()`](http://docs.sympy.org/dev/modules/core.html#sympy.core.basic.Basic.subs) are given as (old, new) pairs, where the "new" expression substitutes the "old" one. So here, in `eq2`, we substitute `rho_sol` in place of `rho_s` and we substitute `B_sol` in place of `B`.

```python
quadA = eq2.subs([(rho_star, rho_sol), (B,B_sol)])
quadA
```

```python
quadA.simplify()
```

It's a little bit ugly, but that is quadratic in $A$, and that means we can solve for the roots of the equation. SymPy's `solve()` function in this case will return a list with the two roots. They are long expressions, so let's print each root separately.

```python
A_sol = sympy.solve(quadA, A)
```

```python
A_sol[0]
```

```python
A_sol[1]
```

### Evaluating the new flux equation

Quadratic equations, of course, have two solutions. Here we have to select the positive root, otherwise our model would have an inconsistency. Are you able to see why?  To determine its value, we need to fill in some actual numbers into the model.

Let's start with the same numerical values that we used in [lesson 1](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/03_wave/03_01_conservationLaw.ipynb) for $\rho_{\rm max}$ and $u_{\rm max}$. But we also have to supply a value for $u^{\star}$, a quantity that should be experimentally observed in a given road. We propose $u^{\star} = 0.7\, u_{\rm max}$ for this exercise.  This would correspond to 84 km/h for a highway with a 120-km/h speed limit, for example.

Let's numerically evaluate the solutions for $A$ using the following values:

\begin{align}
\rho_{\rm max} &=10.0 \nonumber\\ u_{\rm max} &=1.0 \nonumber\\ u^{\star} &=0.7 \nonumber
\end{align}

Now evaluate the numeric result for each root of $A$ using the `evalf()` function, where you pass the numeric substitution as an argument. It's very cool.  Let's evaluate both roots and we will pick the positive one.

Let's try the `[0]`-th solution for $A$, first:

```python
aval_a = A_sol[0].evalf(subs={u_star: 0.7, u_max:1.0, rho_max:10.0} )
aval_a
```

Now let's try the `[1]`-th solution for $A$:

```python
aval_b = A_sol[1].evalf(subs={u_star: 0.7, u_max:1.0, rho_max:10.0} )
aval_b
```

Because Sympy seems not to return the roots in a deterministic order and  we want to use the positive root, we will automatically pick this one by using an `if` statement.

```python
if aval_a > aval_b:
    aval = aval_a
else:
    aval = aval_b
```

```python
aval
```

Now, numerically evaluate $B$ in the same way using the positive root `aval`:

```python
bval = B_sol.evalf(subs={rho_max:10.0, A:aval} )
bval
```

### Turn off $\LaTeX$

$\LaTeX$ output is great when we're looking at algebraic expressions, but it's a little too much for simple numeric output.  Let's turn it off for the rest of the exercise:

```python
sympy.init_printing(use_latex=False)
```

## Green light: take 2

Let's re-examine the green-light problem from [lesson 1](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/03_wave/03_01_conservationLaw.ipynb) but using our newly computed traffic-flux equation.  We shouldn't have to change much—in fact, we can simply create a new `computeF` function and leave the rest of the code unchanged.

There's one last bit of housekeeping to do so we don't run into trouble -- we used the variables `rho_max` and `u_max` in our SymPy code and we defined them as SymPy `symbols`.  You can check on the status of a variable using `type()`.

```python
print(type(rho_max), type(u_max))
```

    <class 'sympy.core.symbol.Symbol'> <class 'sympy.core.symbol.Symbol'>

If we try to use SymPy variables with NumPy arrays, Python is going to be very unhappy.  We can re-define them as floats and avoid that messy situation.

```python
rho_max = 10.
u_max = 1.
```

```python
def computeF(u_max, rho, aval, bval):
    return u_max*rho*(1 - aval*rho-bval*rho**2)
```

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

```python
def rho_green_light(x, rho_light):
    """Computes "green light" initial condition with shock, and linear distribution behind

    Parameters
    ----------
    x         : array of floats
        Locations on the road
    rho_light : float
        Density of cars at stoplight

    Returns
    -------
    rho: array of floats
        Array with initial values of density
    """
    rho = numpy.zeros_like(x)
    mask = numpy.where(x < 2.0)
    rho[mask] = rho_light * x[mask] / 2.0

    return rho
```

```python
#Basic initial condition parameters
#defining grid size, time steps
nx = 81
nt = 30
dx = 4.0/(nx-1)

x = numpy.linspace(0,4,nx)

rho_light = 5.5

```

```python
rho_initial = rho_green_light(x, rho_light)
```

```python
pyplot.plot(x, rho_initial, color='#003366', ls='-', lw=3)
pyplot.ylim(-0.5,11.);
```

```python
def ftbs(rho, nt, dt, dx, rho_max, u_max):
    """ Computes the solution with forward in time, backward in space

    Parameters
    ----------
    rho    : array of floats
            Density at current time-step
    nt     : int
            Number of time steps
    dt     : float
            Time-step size
    dx     : float
            Mesh spacing
    rho_max: float
            Maximum allowed car density
    u_max  : float
            Speed limit

    Returns
    -------
    rho_n : array of floats
            Density after nt time steps at every point x
    """

    #initialize our results array with dimensions nt by nx
    rho_n = numpy.zeros((nt,len(rho)))

    #copy the initial u array into each row of our new array
    rho_n[0,:] = rho.copy()

    for t in range(1,nt):
        F = computeF(u_max, rho, aval, bval)
        rho_n[t,1:] = rho[1:] - dt/dx*(F[1:]-F[:-1])
        rho_n[t,0] = rho[0]
        rho_n[t,-1] = rho[-1]
        rho = rho_n[t].copy()

    return rho_n
```

```python
sigma = 1.
dt = sigma*dx/u_max

rho_n = ftbs(rho_initial, nt, dt, dx, rho_max, u_max)
```

```python
from matplotlib import animation
from IPython.display import HTML
```

```python
fig = pyplot.figure();
ax = pyplot.axes(xlim=(0,4),ylim=(-1,8),xlabel=('Distance'),ylabel=('Traffic density'));
line, = ax.plot([],[],color='#003366', lw=2);

def animate(data):
    x = numpy.linspace(0,4,nx)
    y = data
    line.set_data(x,y)
    return line,

anim = animation.FuncAnimation(fig, animate, frames=rho_n, interval=50)
```

```python
HTML(anim.to_html5_video())
```

That definitely looks different!  Do you think this is more or less accurate than our previous model?

##### Dig Deeper

The new traffic-flux model that we developed here changes the way that traffic patterns evolve. In this lesson, we only experimented with the most basic scheme: forward-time, backward-space.  Try to implement the green-light problem using one of the second-order schemes from [Lesson 2](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/03_wave/03_02_convectionSchemes.ipynb).

## References

* Neville D. Fowkes and John J. Mahony, *"An Introduction to Mathematical Modelling,"* Wiley & Sons, 1994. Chapter 14: Traffic Flow.

---

###### The cell below loads the style of the notebook.

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<link href='https://fonts.googleapis.com/css?family=Source+Code+Pro' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
    font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
    font-family: "Source Code Pro";
    font-size: 90%;
}

.alert-box {
    padding:10px 10px 10px 36px;
    margin:5px;
}

.success {
    color:#666600;
    background:rgb(240,242,229);
}
</style>