#  CS109B Data Science 2: Advanced Topics in Data Science

## Lab 03 - Hierarchical Models in PyMC3

**Harvard University**

**Spring 2022**

**Instructors:** Mark Glickman and Pavlos Protopapas

**Lab instructor and content:** Eleni Angelaki Kaxiras

---

```python
import warnings

import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pymc3 as pm
import theano.tensor as tt

from pymc3 import summary
from pymc3 import Model, Normal, HalfNormal, model_to_graphviz, HalfCauchy
from pymc3 import NUTS, sample, find_MAP
from scipy import optimize

warnings.simplefilter(action="ignore", category=FutureWarning)
warnings.filterwarnings('ignore')

import numpy as np
import matplotlib.pyplot as plt
from matplotlib import gridspec
import scipy.stats as stats
import pandas as pd
import seaborn as sns
%matplotlib inline

import warnings
print('Running on PyMC3 v{}'.format(pm.__version__))
```

    Running on PyMC3 v3.11.4

```javascript
%%javascript
IPython.OutputArea.auto_scroll_threshold = 20000;
```

    <IPython.core.display.Javascript object>

### Learning Objectives

By the end of this lab, you should know how to create Bayesian hierarchical models in PyMC3

**This lab maps to Homework 2.**

## Hierarchical Models

**Gelman et al.** famous radon dataset is a classic for hierarchical modeling. In this dataset the amount of the radioactive gas radon has been measured among different households in all county's of several states. Radon gas is known to be the highest cause of lung cancer in non-smokers. It is believed to be more strongly present in households containing a basement and to differ in amount present among types of soil. Here we'll investigate this differences and try to make predictions of radon levels in different county's based on the county itself and the presence of a basement.

```python
df = pd.read_csv('radon.csv', index_col=[0])
df['log_radon'] = df['log_radon'].astype('float')
county_names = df.county.unique()
county_idx = df.county_code.values

n_counties = len(county_names)
df.head()
```

      idnum
      state
      state2
      stfips
      zip
      region
      typebldg
      floor
      room
      basement
      ...
      pcterr
      adjwt
      dupflag
      zipflag
      cntyfips
      county
      fips
      Uppm
      county_code
      log_radon

      0
      5081.0
      MN
      MN
      27.0
      55735
      5.0
      1.0
      1.0
      3.0
      N
      ...
      9.7
      1146.499190
      1.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.832909

      1
      5082.0
      MN
      MN
      27.0
      55748
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      14.5
      471.366223
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.832909

      2
      5083.0
      MN
      MN
      27.0
      55748
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      9.6
      433.316718
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      1.098612

      3
      5084.0
      MN
      MN
      27.0
      56469
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      24.3
      461.623670
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.095310

      4
      5085.0
      MN
      MN
      27.0
      55011
      3.0
      1.0
      0.0
      4.0
      Y
      ...
      13.8
      433.316718
      0.0
      0.0
      3.0
      ANOKA
      27003.0
      0.428565
      1
      1.163151

5 rows × 29 columns

Each **row** in the dataframe represents the radon measurements for **one house** in a specific county including whether the house has a basement (floor = 0) or not (floor = 1). We are interested in whether having a basement increases the radon measured in the house.

To keep things simple let's keep only the following three variables: `county`, `log_radon`, and `floor`

```python
# keep only these variables
data = df[['county', 'log_radon', 'floor']]
data.head()
```

      county
      log_radon
      floor

      0
      AITKIN
      0.832909
      1.0

      1
      AITKIN
      0.832909
      0.0

      2
      AITKIN
      1.098612
      0.0

      3
      AITKIN
      0.095310
      0.0

      4
      ANOKA
      1.163151
      0.0

Let's check how many different counties we have. We also notice that they have a different number of houses. Some have a large number of houses measured, some only 1.

```python
data['county'].value_counts().head(5)
```

    ST LOUIS      116
    HENNEPIN      105
    DAKOTA         63
    ANOKA          52
    WASHINGTON     46
    Name: county, dtype: int64

```python
data['county'].value_counts()[-5:]
```

    MILLE LACS         2
    YELLOW MEDICINE    2
    MAHNOMEN           1
    WILKIN             1
    MURRAY             1
    Name: county, dtype: int64

```python
# let's add a column that numbers the counties from 0 to n
raw_ids = np.unique(data['county'])
raw2newid = {x:np.where(raw_ids == x)[0][0] for x in raw_ids}

data['county_id'] = data['county'].map(raw2newid)
data
```

      county
      log_radon
      floor
      county_id

      0
      AITKIN
      0.832909
      1.0
      0

      1
      AITKIN
      0.832909
      0.0
      0

      2
      AITKIN
      1.098612
      0.0
      0

      3
      AITKIN
      0.095310
      0.0
      0

      4
      ANOKA
      1.163151
      0.0
      1

      ...
      ...
      ...
      ...
      ...

      914
      WRIGHT
      1.871802
      0.0
      83

      915
      WRIGHT
      1.526056
      0.0
      83

      916
      WRIGHT
      1.629241
      0.0
      83

      917
      YELLOW MEDICINE
      1.335001
      0.0
      84

      918
      YELLOW MEDICINE
      1.098612
      0.0
      84

919 rows × 4 columns

### 1 - Pooling: Same Linear Regression for all

We can just pool all the data and estimate one big regression to asses the influence of having a basement on radon levels across all counties. Our model would be:

\begin{equation}
y_{i} = \alpha + \beta*floor_{i}
\end{equation}

Where $i$ represents the measurement (house), and floor contains a 0 or 1 if the house has a basement or not. By ignoring the county feature, we do not differenciate on counties.

```python
with pm.Model() as pooled_model:
    # common priors for all
    a = pm.Normal('a', mu=0, sigma=100)
    b = pm.Normal('b', mu=0, sigma=100)

    # radon estimate
    radon_est = a + b*data['floor'].values

    # likelihood after radon observations
    radon_obs = pm.Normal('radon_obs', mu=radon_est,
                           observed=data['log_radon']) # note here we enter the whole dataset
```

```python
from pymc3 import model_to_graphviz
model_to_graphviz(pooled_model)
```

```python
with pooled_model:

    pooled_trace = sample(2000, tune=1000, target_accept=0.9)
    print(f'DONE')
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (4 chains in 4 jobs)
    NUTS: [b, a]

  <progress value='12000' class='' max='12000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [12000/12000 00:04<00:00 Sampling 4 chains, 0 divergences]

    Sampling 4 chains for 1_000 tune and 2_000 draw iterations (4_000 + 8_000 draws total) took 27 seconds.

    DONE

```python
pm.traceplot(pooled_trace);
```

Remember, with the pooled model we have only one intercept, $\alpha$, and only one slope, $\beta$ for all the counties. Let's plot the regression lines.

```python
# plot just a subset of the countries
counties = ['HENNEPIN','AITKIN','WASHINGTON', 'MURRAY', 'YELLOW MEDICINE', 'MAHNOMEN']

plt.figure(figsize=(10,5))
rows = 2
gs = gridspec.GridSpec(rows, len(counties)//rows)

for i, county in enumerate(counties):
    county_data = data.loc[data['county']==county]
    x = np.linspace(-0.2, 1.2)
    radon_est = pooled_trace['a'].mean() + pooled_trace['b'].mean()*x
    subplt = plt.subplot(gs[i])

    subplt.set_ylim(0.,4.)
    subplt.scatter(county_data['floor'], county_data['log_radon'])
    subplt.plot(x, radon_est, c='r', label='pooled line');
    subplt.set_xlabel('floor', fontsize=10)
    subplt.set_ylabel('radon level', fontsize=10)
    subplt.set_title(str(county) + ' County')
    subplt.legend()
    plt.tight_layout()
```

### 2 - Unpooling: Separate Linear Regression for each county - not Hierarchical Models yet

We believe that different counties have different relationships of radon and basements. Our model would be:

\begin{equation}
radon_{i,c} = \alpha_c + \beta_c*floor_{i,c}
\end{equation}

Where $i$ represents the measurement, $c$ the county, and floor contains a 0 or 1 if the house has a basement or not.

Notice we have separate coefficients for each county in $a_c$ and $b_c$. They are totally different, they do not even come from the same distribution.

We will do this for only one county, as an example. We pick HENNEPIN county.

```python
# chose a county
county = 'MEEKER'
county_data = data.loc[data['county']==county]
county_data.head()
```

      county
      log_radon
      floor
      county_id

      479
      MEEKER
      0.875469
      0.0
      45

      480
      MEEKER
      1.386294
      0.0
      45

      481
      MEEKER
      1.987874
      0.0
      45

      482
      MEEKER
      0.788457
      0.0
      45

      483
      MEEKER
      1.193922
      0.0
      45

```python
#help(pm.Normal)
```

```python
with pm.Model() as unpooled_model:
    mu_a = pm.Normal('mu_a', mu=0., sigma=100)
    sigma_a = pm.HalfNormal('sigma_a', 5.)
    mu_b = pm.Normal('mu_b', mu=0., sigma=100)
    sigma_b = pm.HalfNormal('sigma_b', 5.)

    a = pm.Normal('a', mu=mu_a, sigma=sigma_a)
    b = pm.Normal('b', mu=mu_b, sigma=sigma_b)

    radon_est = a + b*county_data['floor'].values

    radon_obs = pm.Normal('radon_like', mu=radon_est,
                           observed=county_data['log_radon'])
```

```python
model_to_graphviz(unpooled_model)
```

```python
with unpooled_model:

    unpooled_trace = sample(2000, tune=1000, target_accept=0.9)
    print(f'DONE')
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (4 chains in 4 jobs)
    NUTS: [b, a, sigma_b, mu_b, sigma_a, mu_a]

  <progress value='12000' class='' max='12000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [12000/12000 00:28<00:00 Sampling 4 chains, 565 divergences]

    Sampling 4 chains for 1_000 tune and 2_000 draw iterations (4_000 + 8_000 draws total) took 49 seconds.
    There were 126 divergences after tuning. Increase `target_accept` or reparameterize.
    There were 200 divergences after tuning. Increase `target_accept` or reparameterize.
    The acceptance probability does not match the target. It is 0.8300042540663897, but should be close to 0.9. Try to increase the number of tuning steps.
    There were 144 divergences after tuning. Increase `target_accept` or reparameterize.
    There were 95 divergences after tuning. Increase `target_accept` or reparameterize.
    The estimated number of effective samples is smaller than 200 for some parameters.

    DONE

```python
pm.traceplot(unpooled_trace);
plt.tight_layout()
```

Print the regression line for our chosen county alone.

```python
county = 'MEEKER'
county_data = data.loc[data['county']==county]

x = np.arange(len(county_data['floor'].values))
radon_est_unpooled = unpooled_trace['a'].mean() + unpooled_trace['b'].mean()*county_data['floor'].values

xx = np.linspace(-0.2, 1.2)
radon_est_pooled = pooled_trace['a'].mean() + pooled_trace['b'].mean()*xx

plt.scatter(county_data['floor'], county_data['log_radon'])
plt.xlim(-0.1,1.1)
plt.xlabel('floor', fontsize=10)
plt.ylabel('radon level', fontsize=10)
plt.title(f'{str(county)} county Radon levels')
plt.plot(x, radon_est_unpooled, c='g', label='unpooled line');
plt.plot(xx, radon_est_pooled, c='r', label='pooled line');
plt.legend();
```

### 3 - Partial pooling: Hierarchical Regression (Varying-Coefficients Model)

Merely by the fact that all counties are counties, they share similarities, so there is a middle ground to both of these extremes. Specifically, we may assume that while $\alpha_c$ and $\beta_c$are different for each county as in the unpooled case, the coefficients are all drawn from the same distribution:

\begin{equation}
radon_{i,c} = \alpha_c + \beta_c*floor_{i,c}
\end{equation}

\begin{equation}
a_c \sim  \mathcal{N}(\mu_a,\,\sigma_a^{2})
\end{equation}

\begin{equation}
b_c \sim  \mathcal{N}(\mu_b,\,\sigma_b^{2})
\end{equation}

where the common parameters are:
\begin{eqnarray}
\mu_a \sim  \mathcal{N}(0,\,10) \\
\sigma_a^2 \sim  |\mathcal{N}(0,\,10)| \\
\mu_b \sim  \mathcal{N}(0,\,10) \\
\sigma_b^2 \sim  |\mathcal{N}(0,\,10)|
\end{eqnarray}

Add math for mu and sigma

The different counties are effectively sharing information through the commin priors. We are thus observing what is known as shrinkage; modeling the groups not as independent from each other, neither as a single group but rather as related.

NOTES

We saw that some counties had only one sample, so if that house is a really old with old lead pipes, our prediction will be that all houses in this county have radon. On the other extreme, if we have a newer house with no radon then again we will have missleading results. In one case, you will overestimate the bad quality and in the other underestimate it. Under a hierarchical model, the miss-estimation of one group will be offset by the information provided by the other groups. As always gathering more data helps if this is an option.

```python

```

### Defining the Model for the Hierarchical Model

```python
with pm.Model() as hierarchical_model:
    # Hyperpriors for group nodes
    mu_a = pm.Normal('mu_a', mu=0., sigma=100)
    sigma_a = pm.HalfNormal('sigma_a', 5.)
    mu_b = pm.Normal('mu_b', mu=0., sigma=100)
    sigma_b = pm.HalfNormal('sigma_b', 5.)

    # Above we just set mu and sd to a fixed value while here we
    # plug in a common group distribution for all a and b (which are
    # vectors of length n_counties).

    # Intercept for each county, distributed around group mean mu_a
    a = pm.Normal('a', mu=mu_a, sigma=sigma_a, shape=n_counties)

    # beta for each county, distributed around group mean mu_b
    b = pm.Normal('b', mu=mu_b, sigma=sigma_b, shape=n_counties)

    # Model error
    #eps = pm.HalfCauchy('eps', 5.)

    radon_est = a[county_idx] + b[county_idx]*data['floor'].values

    # Data likelihood with sigma for random error
#     radon_like = pm.Normal('radon_like', mu=radon_est,
#                            sigma=eps, observed=data['log_radon'])

    # Data likelihood with sigma without random error
    radon_like = pm.Normal('radon_like', mu=radon_est, #sigma=eps,
                           observed=data['log_radon'])
```

```python
model_to_graphviz(hierarchical_model)
```

#### Inference

```python
with hierarchical_model:
    hierarchical_trace = pm.sample(2000, tune=2000, target_accept=.9)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (4 chains in 4 jobs)
    NUTS: [b, a, sigma_b, mu_b, sigma_a, mu_a]

  <progress value='16000' class='' max='16000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [16000/16000 01:22<00:00 Sampling 4 chains, 2,625 divergences]

    Sampling 4 chains for 2_000 tune and 2_000 draw iterations (8_000 + 8_000 draws total) took 104 seconds.
    There were 399 divergences after tuning. Increase `target_accept` or reparameterize.
    The acceptance probability does not match the target. It is 0.7009339264679122, but should be close to 0.9. Try to increase the number of tuning steps.
    The chain contains only diverging samples. The model is probably misspecified.
    The acceptance probability does not match the target. It is 0.01894821085104743, but should be close to 0.9. Try to increase the number of tuning steps.
    The acceptance probability does not match the target. It is 0.987759829141593, but should be close to 0.9. Try to increase the number of tuning steps.
    There were 226 divergences after tuning. Increase `target_accept` or reparameterize.
    The acceptance probability does not match the target. It is 0.8128804269724217, but should be close to 0.9. Try to increase the number of tuning steps.
    The rhat statistic is larger than 1.4 for some parameters. The sampler did not converge.
    The estimated number of effective samples is smaller than 200 for some parameters.

```python
pm.traceplot(hierarchical_trace, var_names=['mu_a', 'mu_b',
                                            'sigma_a', 'sigma_b']);
```

```python
results = pm.summary(hierarchical_trace)
results
```

    Got error No model on context stack. trying to find log_likelihood in translation.

      mean
      sd
      hdi_3%
      hdi_97%
      mcse_mean
      mcse_sd
      ess_bulk
      ess_tail
      r_hat

      mu_a
      1.438
      0.064
      1.353
      1.548
      0.022
      0.016
      9.0
      84.0
      1.35

      mu_b
      -0.627
      0.082
      -0.795
      -0.472
      0.005
      0.004
      189.0
      472.0
      1.52

      a[0]
      1.362
      0.208
      0.927
      1.730
      0.021
      0.015
      103.0
      1208.0
      1.34

      a[1]
      1.075
      0.118
      0.838
      1.306
      0.008
      0.006
      163.0
      67.0
      1.06

      a[2]
      1.498
      0.206
      1.084
      1.866
      0.029
      0.020
      72.0
      1845.0
      1.16

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      b[82]
      -0.663
      0.152
      -0.978
      -0.399
      0.011
      0.009
      190.0
      352.0
      1.40

      b[83]
      -0.630
      0.144
      -0.936
      -0.359
      0.005
      0.005
      316.0
      469.0
      1.34

      b[84]
      -0.628
      0.156
      -0.948
      -0.339
      0.005
      0.005
      446.0
      299.0
      1.52

      sigma_a
      0.260
      0.047
      0.165
      0.333
      0.010
      0.007
      23.0
      614.0
      1.20

      sigma_b
      0.093
      0.086
      0.007
      0.258
      0.027
      0.020
      6.0
      10.0
      1.66

174 rows × 9 columns

```python
# plot just a subset of the countries
counties = ['HENNEPIN','AITKIN','WASHINGTON', 'LAKE OF THE WOODS', 'YELLOW MEDICINE', 'ANOKA']

plt.figure(figsize=(10,5))
rows = 2
gs = gridspec.GridSpec(rows, len(counties)//rows)

for i, county in enumerate(counties):
    county_data = data.loc[data['county']==county]
    subplt = plt.subplot(gs[i])

    # pooled line (single values coeff for all)
    xx = np.linspace(-0.2, 1.2)
    radon_est = pooled_trace['a'].mean() + pooled_trace['b'].mean()*xx

    radon_est_hier = np.mean(hierarchical_trace['a'][i]) + \
                      np.mean(hierarchical_trace['b'][i])*xx

    # un-pooled (single subject)
    sns.regplot(x='floor', y='log_radon', ci=None, label='unpooled',
                data=county_data).set_title('County '+str(county))

    # hierarchical line
    subplt.set_ylim(0.,4.)
    subplt.scatter(county_data['floor'], county_data['log_radon'])
    subplt.plot(xx, radon_est, c='r', label='pooled');

    # plot the hierarchical, varying coefficient model
    subplt.plot(xx, radon_est_hier, c='g', label='hierarchical');

    subplt.set_xlabel('floor', fontsize=10)
    subplt.set_ylabel('radon level', fontsize=10)
    subplt.set_title(str(county) + ' County')
    subplt.legend()
    plt.tight_layout()
```

This tutorial is modified from PyMC3 docs: https://docs.pymc.io/notebooks/GLM-hierarchical.html


# Plotting with Matplotlib

## Prepare for action

```
import numpy as np
import scipy as sp
import sympy

# Pylab combines the pyplot functionality (for plotting) with the numpy
# functionality (for mathematics and for working with arrays) in a single namespace
# aims to provide a closer MATLAB feel (the easy way). Note that his approach
# should only be used when doing some interactive quick and dirty data inspection.
# DO NOT USE THIS FOR SCRIPTS
#from pylab import *

# the convienient Matplotib plotting interface pyplot (the tidy/right way)
# use this for building scripts. The examples here will all use pyplot.
import matplotlib.pyplot as plt

# for using the matplotlib API directly (the hard and verbose way)
# use this when building applications, and/or backends
import matplotlib as mpl
```

How would you like the IPython notebook show your plots? In order to use the
matplotlib IPython magic youre IPython notebook should be launched as

    ipython notebook --matplotlib=inline

Make plots appear as a pop up window, chose the backend: 'gtk', 'inline', 'osx', 'qt', 'qt4', 'tk', 'wx'

    %matplotlib qt

or inline the notebook (no panning, zooming through the plot). Not working in IPython 0.x

    %matplotib inline

```
# activate pop up plots
#%matplotlib qt
# or change to inline plots
#%matplotlib inline
%matplotlib inline
```

### Matplotlib documentation

Finding your own way (aka RTFM). Hint: there is search box available!

* http://matplotlib.org/contents.html

The Matplotlib API docs:

* http://matplotlib.org/api/index.html

Pyplot, object oriented plotting:

* http://matplotlib.org/api/pyplot_api.html
* http://matplotlib.org/api/pyplot_summary.html

Extensive gallery with examples:

* http://matplotlib.org/gallery.html

### Tutorials for those who want to start playing

If reading manuals is too much for you, there is a very good tutorial available here:

* http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-4-Matplotlib.ipynb

Note that this tutorial uses

    from pylab import *

which is usually not adviced in more advanced script environments. When using

    import matplotlib.pyplot as plt

you need to preceed all plotting commands as used in the above tutorial with

    plt.

Give me more!

[EuroScipy 2012 Matlotlib tutorial](http://www.loria.fr/~rougier/teaching/matplotlib/). Note that here the author uses ```from pylab import * ```. When using ```import matplotliblib.pyplot as plt``` the plotting commands need to be proceeded with ```plt.```

## Plotting template starting point

```
# some sample data
x = np.arange(-10,10,0.1)
```

To change the default plot configuration values.

```
page_width_cm = 13
dpi = 200
inch = 2.54 # inch in cm
# setting global plot configuration using the RC configuration style
plt.rc('font', family='serif')
plt.rc('xtick', labelsize=12) # tick labels
plt.rc('ytick', labelsize=20) # tick labels
plt.rc('axes', labelsize=20)  # axes labels
# If you don’t need LaTeX, don’t use it. It is slower to plot, and text
# looks just fine without. If you need it, e.g. for symbols, then use it.
plt.rc('text', usetex=True) #<- P-E: Doesn't work on my Mac
```

```
# create a figure instance, note that figure size is given in inches!
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8,6))
# set the big title (note aligment relative to figure)
fig.suptitle("suptitle 16, figure alignment", fontsize=16)

# actual plotting
ax.plot(x, x**2, label="label 12")

# set axes title (note aligment relative to axes)
ax.set_title("title 14, axes alignment", fontsize=14)

# axes labels
ax.set_xlabel('xlabel 12')
ax.set_ylabel(r'$y_{\alpha}$ 12')

# legend
ax.legend(fontsize=12, loc="best")

# saving the figure in different formats
fig.savefig('figure-%03i.png' % dpi, dpi=dpi)
fig.savefig('figure.svg')
fig.savefig('figure.eps')
```

```
ax.grid(True)
fig.canvas.draw()
```

```
# following steps are only relevant when using figures as pop up windows (with %matplotlib qt)
# to update a figure with has been modified
fig.canvas.draw()
# show a figure
fig.show()
```

## Exercise

The current section is about you trying to figure out how to do several plotting features. You should use the previously mentioned resources to find how to do that. In many cases, google is your friend!

* add a grid to the plot

```
plt.plot(x,x**2)
#Write code to show grid in plot here
```

* change the location of the legend to different places

```
plt.plot(x,x**2, label="label 12")
plt.legend(fontsize=12, loc="best")
```

* find a way to control the line type and color, marker type and color, control the frequency of the marks (`markevery`). See plot options at: http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.plot

```
plt.plot(x,x**2, 'o-')
```

* add different sub-plots

```
fig, ax = plt.subplots(nrows=1, ncols=1)
ax.plot(x,x**2)

```

* size the figure such that when included on an A4 page the fonts are given in their true size

```

```

* make a contour plot

```
X, Y = np.meshgrid(x,x)

```

* use twinx() to create a second axis on the right for the second plot

```
plt.plot(x,x**2)
plt.plot(x,x**4, 'r')
```

* add horizontal and vertical lines using axvline(), axhline()

```
plt.plot(x,x**2)
```

* autoformat dates for nice printing on the x-axis using fig.autofmt_xdate()

```
import datetime
dates = np.array([datetime.datetime.now() + datetime.timedelta(days=i) for i in xrange(24)])
fig, ax = plt.subplots(nrows=1, ncols=1)

```

## Advanced exercises

We are going to play a bit with regression

* Create a vector x of equally spaced number between $x \in [0, 5\pi]$ of 1000 points (keyword: linspace)

```

```

* create a vector y, so that y=sin(x) with some random noise

```

```

* plot it like this:

```

```

Try to do a polynomial fit on y(x) with different polynomial degree (Use numpy.polyfit to obtain coefficients)

Plot it like this (use np.poly1d(coef)(x) to plot polynomials)

```

```

```

```