# Time_Series

```python
from __future__ import print_function
%matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
import seaborn as sns
import statsmodels.api as sm
import random
import itertools
from scipy.stats import norm
from datetime import datetime
import requests
from io import BytesIO
plt.style.use('fivethirtyeight')
import warnings
warnings.filterwarnings('ignore')
```

## Time Series

A [time series](https://en.wikipedia.org/wiki/Time_series) is a sequence of data points, typically consisting of successive measurements made over a time interval. This is a very common type of data as we frequently measure how something varies over time.  usually a time series is an ordered sequence of values of a variable at equally spaced time intervals; but methods exist to deal with irregular sampling.

Examples of time series include:

* Stock Market
* The change of weather
* An electrocardiogram (EKG or ECG), that is, the electrical activity of your heart.
* The popularity of a celebirty or politician
* Economic Forecasting

- from https://commons.wikimedia.org/wiki/File:Gold_Spot_Price_per_Gram_from_Jan_1971_to_Jan_2012.svg

Univariate (bivariate, multivariate) time series: collection of observations of one (two, several) state variables, that are made in sequential momentsin time.

The [sampling](https://en.wikipedia.org/wiki/Sampling_(signal_processing)) frequency (or sample rate) is the number of samples per unit of time. (e.g once per minute).

- from https://commons.wikimedia.org/wiki/File:Signal_Sampling.png

### Nyquist criterion

Nyquist criterion, or [Nyquist frequency](https://en.wikipedia.org/wiki/Nyquist_frequency), named after electronic engineer [Harry Nyquist](https://en.wikipedia.org/wiki/Harry_Nyquist), is half of the sampling rate of a discrete signal processing system. In order to correctly determine the frequency spectrum of a signal, the signal must be measured at least twice per period.

### Reconstruction

Reconstruction is the opposite of sampling, the process by which a sampled signal is converted into a continuous signal.

Under-Sampling

## Components of Time Series

The pattern or behavior of the data in a time series can be made up of several components. Theoretically, any time series can be decomposed into:

*Trend*

The trend component accounts for the gradual shifting of the time series to relatively higher or lower values over a long period of time.

*Cyclical*

A regular pattern of sequences of values that go above and below the trend line is a cyclical component.

*Seasonal*

The seasonal component accounts for regular patterns of variability within certain time periods, such as a year. While seasons could be modeled using cyclical components. Often within-year, within-month, within-week or within-day cycles are treated as “seasonal” behavior.

*Irregular*

The irregular components represent erratic, unsystematic, ‘residual’ fluctuations. That is, noise.

## Assumptions of time series

* Observations are roughly equally spaced
* Indices are units of time

## Time series trends

* Observations are roughly equally spaced
* Indices are units of time
*

## Stationary versus Non-stationary time series

Non-stationary time series or have means, variances and covariances that change over time. Whereas while stationary time series have up and down flucutions they have means, variances and covariances.  That is, a [stationary process](https://en.wikipedia.org/wiki/Stationary_process) is a stochastic process whose joint probability distribution does not change when shifted in time.

For example, we expect financial time series data, to be non-stationary since we would expect a deterministic upward trend in price.

A grown adults weight would more likely be stationary time series unless he or she fundenemtally changes their "state".

*Two simulated time series processes, one stationary the other non-stationary.*
- from https://commons.wikimedia.org/wiki/File:Stationarycomparison.png

### How to Check Stationarity of a Time Series?

The series is assumed to be stationary if it has constant statistical properties over time:

* constant mean.
* constant variance.
* an autocovariance that does not depend on time.

## Smoothing Methods of Time Series

Smoothing a time series: to eliminate some of short-term fluctuations. Sometimes smoothing is done to remove cyclical or  seasonal fluctuations. (i.e. "deseasonalize"" a time series). Usually smoothing refers to methods for reducing of canceling the effect due to random variation. The simplest smoothing method is just the "simple" average of all past data.  Of course, the "simple" average or mean of all past observations is only a useful estimate for forecasting when there are no trends.

## Time Series Decomposition

Time Series Decomposition is a procedure to identify the component factors of a time series. That is, to create a model that expresses the time series variable Y in terms of the components T (trend), C (cycle), S (seasonal) and I (iregular).

## Time Series Analysis

Time series analysis generates a model that accounts an internal structure (such as autocorrelation, trend or seasonal variation) of a set of data points taken over time. This is very frequently used for forecasting. That is, predicting and future event in time based on a sequential historical sample.

## Autocorrelation

[Autocorrelation](https://en.wikipedia.org/wiki/Autocorrelation), also known as serial correlation or cross-autocorrelation, is the cross-correlation of a signal with itself at different points in time. It is the correlation between values of the process at different times. Informally, it is the similarity between observations as a function of the time lag between them. It is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise.

Let X be some repeatable process, and i be some point in time after the start of that process. (i may be an integer for a discrete-time process or a real number for a continuous-time process.) Then $X_i$ is the value (or realization) produced by a given run of the process at time i. Suppose that the process is further known to have defined values for mean μi and variance σi2 for all times i. Then the definition of the autocorrelation between times s and t is

$$
R(s,t) = \frac{\operatorname{E}[(X_t - \mu_t)(X_s - \mu_s)]}{\sigma_t\sigma_s}\, ,
$$

where "E" is the expected value operator. Note that this expression is not well-defined for all time series or processes, because the variance may be zero (for a constant process) or infinite (for processes with distribution lacking well-behaved moments, such as certain types of power law).

### Correlogram

In the analysis of data, a [correlogram](https://en.wikipedia.org/wiki/Correlogram) is an image of correlation statistics. For example, in time series analysis, a correlogram, also known as an autocorrelation plot, is a plot of the sample autocorrelations $r_{h}, r_{h}$, versus $h$ (the time lags).

If cross-correlation is used, the result is called a cross-correlogram. The correlogram is a commonly used tool for checking randomness in a data set. This randomness is ascertained by computing autocorrelations for data values at varying time lags. If random, such autocorrelations should be near zero for any and all time-lag separations. If non-random, then one or more of the autocorrelations will be significantly non-zero.

[By M. W. Toews - Own work, CC BY 4.0](https://commons.wikimedia.org/w/index.php?curid=2379384)

### Partial autocorrelation function

The [partial autocorrelation function (PACF)](https://en.wikipedia.org/wiki/Partial_autocorrelation_function) gives the partial correlation of a time series with its own lagged values, controlling for the values of the time series at all shorter lags. It contrasts with the autocorrelation function, which does not control for other lags.

This function plays an important role in data analyses aimed at identifying the extent of the lag in an autoregressive model. The use of this function was introduced as part of the Box–Jenkins approach to time series modelling, where by plotting the partial autocorrelative functions one could determine the appropriate lags p in an AR (p) model or in an extended ARIMA (p,d,q) model.

Given a time series $z_t$, the partial autocorrelation of lag $k$, denoted $\alpha(k)$, is the autocorrelation between $z_t$ and $z_{t+k}$ with the linear dependence of $z_t$ on $z_{t+1}$ through $z_{t+k-1}$ removed; equivalently, it is the autocorrelation between $z_t$ and $z_{t+k}$ that is not accounted for by lags 1 to $k$ − 1, inclusive.

 $\alpha(1) = \operatorname{Cor}(z_{t+1}, z_t),$

 $\alpha(k) = \operatorname{Cor}(z_{t+k} - P_{t,k}(z_{t+k}),\, z_t - P_{t,k}(z_t)),\text{ for }k\geq 2,$

where $P_{t,k}(x)$ denotes the projection of $x$ onto the space spanned by $ x_{t+1}, \dots, x_{t+k-1}$.

There are algorithms for estimating the partial autocorrelation based on the sample autocorrelations. These algorithms derive from the exact theoretical relation between the partial autocorrelation function and the autocorrelation function.

Partial autocorrelation plots are a commonly used tool for identifying the order of an autoregressive model.

## Naive & Simple Averaging

Naive or "simple" averaging is just the mean (or median) of all past data.  Of course, the "simple" average or mean of all past observations is only a useful estimate for forecasting when there are no trends.

For example, if we have prices are $p_M, p_{M-1},\dots,p_{M-(n-1)}$ then the formula is

$$
\textit{SMA} = { p_M + p_{M-1} + \cdots + p_{M-(n-1)} \over n }
$$

## Moving Averages

A moving average (rolling average or running average) is an average that is updated for a window or history of n events. This is sometimes called the arithmetic moving average the most recent n data values. For an an equally weighted average of the sequence of n values $x_1. \ldots, x_n$ up to the current time:
$$
\textit{MA}_n = {{x_1 + \cdots + x_n} \over n}\,.
$$

### Weighted Moving Averages

A weighted moving average places more weight on recent observations.  Sum of the weights needs to equal 1. For example. in technical analysis of financial data, a weighted moving average (WMA) often uses weights that decrease in arithmetical progression. In an n-day WMA the latest day has weight n, the second latest n − 1, etc., down to one.

$$
\text{WMA}_{M} = { n p_{M} + (n-1) p_{M-1} + \cdots + 2 p_{(M-n+2)} + p_{(M-n+1)} \over n + (n-1) + \cdots + 2 + 1}
$$

That is, the denominator is a triangle number equal to $\frac{n(n+1)}{2}$.

There are many other methods for weighting the importance of more recent observations.

"Weighted moving average weights N=15". Licensed under CC BY-SA 3.0 via Commons

## Exponential moving average

Exponential moving average (also called Single Exponential Smoothing) continually revises a forecast in light of more recent experiences.  Averaging (smoothing) past values of a series in a decreasing (exponential) manner.

The EMA for a series Y may be calculated recursively:

$$
S_1   Y_1 \quad for  \quad t > 1, S_{t} = \alpha \cdot Y_{t} + (1-\alpha) \cdot S_{t-1}
$$

Where:

* The coefficient $\alpha$ represents the degree of weighting decrease, a constant smoothing factor between 0 and 1. A higher  $\alpha$  discounts older observations faster.
* $Y_t$ is the value at a time period t.
* $S_t$ is the value of the EMA at any time period t.

Since the base values of the recursive intial call is undefined. The priming of values such as $S_1$ may be initialized in a number of different ways, most commonly by setting $S_1$ to $Y_1$, or by setting  $S_1$to an average of the first few observations.

"Exponential moving average weights N=15". Licensed under CC BY-SA 3.0 via Commons

### Double exponential smoothing

Holt-Winters method for exponential smoothing extended Holt’s method to capture seasonality. This is sometimes called "Double exponential smoothing." The Holt-Winters seasonal method comprises the forecast equation and two smoothing equations. e use {$s_t$} to represent the smoothed value for time t, and {$b_t$} is our best estimate of the trend at time t. The output of the algorithm is now written as Ft+m, an estimate of the value of x at time t+m, m>0 based on the raw data up to time t. Double exponential smoothing is given by the formulas

$$
\begin{align}
s_1& = x_1\\
b_1& = x_1 - x_0\\
\end{align}
$$

And for t > 1 by

$$
\begin{align}
s_{t}& = \alpha x_{t} + (1-\alpha)(s_{t-1} + b_{t-1})\\
b_{t}& = \beta (s_t - s_{t-1}) + (1-\beta)b_{t-1}\\
\end{align}
$$

where α is the data smoothing factor, 0 < $\alpha$ < 1, and β is the trend smoothing factor, 0 < $\beta$ < 1.

### Triple exponential smoothing

Triple exponential smoothing account seasonal changes as well as trends using three smoothing equations — one for the level, one for trend, and one for the seasonal component denoted, with smoothing parameters $\alpha, \beta,$ and $\gamma$. We use a variable $m$ to denote the period of the seasonality, i.e., the number of seasons in a year. For example, for quarterly data m=4, and for monthly data m=12.

There are two variations to this method that differ in the nature of the seasonal component. The additive method is preferred when the seasonal variations are roughly constant through the series, while the multiplicative method is preferred when the seasonal variations are changing proportional to the level of the series.

## Linear Time-Series Model

Using regression we can model and forecast the trend in time series data by including the time as a predictor variable. Time series processes are often described by multiple linear regression (MLR) models of the form:

$$y_t = X_t \beta + e_t,$$

Note that the above equation differs from our usual regression in that the predictor variables are a function of time, $t$. As oppossed to the more familair form below:

$$ Y = \beta X + e $$

where  $y_t$ is an observed response and  $X_t$ includes columns for contemporaneous values of observable predictors. The partial regression coefficients in  $\beta$ represent the marginal contributions of individual predictors to the variation in  $y_t$ when all of the other predictors are held fixed. A common feature of time series data is a trend. Regressing non-stationary time series can lead to spurious regressions.

The error term $\varepsilon_i$ still has the following assumptions:

* have mean zero; otherwise the forecasts will be systematically biased.
* statistical independence of the errors (in particular, no correlation between consecutive errors in the case of time series data).
* homoscedasticity (constant variance) of the errors.
* normality of the error distribution.

## Autoregressive integrated moving average (ARIMA)

An autoregressive integrated moving average (ARIMA or ARMA) model combines an autoregressive component with a moving average component in to a single model.

An [autoregressive integrated moving average (ARIMA or ARMA)](https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average) model is a generalization of an autoregressive moving average (ARMA) model. These models are fitted to time series data either to better understand the data or to predict future points in the series (forecasting). They are applied in some cases where data show evidence of non-stationarity, where an initial differencing step (corresponding to the "integrated" part of the model) can be applied to reduce the non-stationarity.

Non-seasonal ARIMA models are generally denoted $ARIMA(p, d, q)$ where parameters $p, d,  \quad and \quad q$ are non-negative integers, $p$ is the order of the Autoregressive model, $d$ is the degree of differencing, and $q$ is the order of the Moving-average model.  The he number of differences $d$ is determined using repeated statistical tests. The values of $p$ and $q$ are then chosen by minimizing the AIC after differencing the data $d$ times.

* AR: Autoregression. A dependent relationship between an observation and some number of lagged observations.
* I: Integrated. The use of differencing of raw observations in order to make the time series stationary.
* MA: Moving Average. A model that uses the dependency between an observation and a residual error from a moving average model applied to lagged observations.

A non-seasonal ARIMA model can be (almost) completely summarized by three numbers:

* $p$ = the number of autoregressive terms.  he number of lag observations included in the model.
* $d$ = the number of nonseasonal differences. The number of times that the raw observations are differenced, also called the degree of differencing.
* $q$ = the number of moving-average terms. The size of the moving average window, also called the order of moving average.

When a value of 0 is used as a parameter, it means that parameter is not used the model.

The ARIMA model uses an iterative three-stage modeling approach:

Model identification and model selection: making sure that the variables are stationary, identifying seasonality in the dependent series (seasonally differencing it if necessary), and using plots of the autocorrelation and partial autocorrelation functions of the dependent time series to decide which (if any) autoregressive or moving average component should be used in the model.

Parameter estimation using computation algorithms to arrive at coefficients that best fit the selected ARIMA model. The most common methods use maximum likelihood estimation or non-linear least-squares estimation.

Model checking by testing whether the estimated model conforms to the specifications of a stationary univariate process. In particular, the residuals should be independent of each other and constant in mean and variance over time. (Plotting the mean and variance of residuals over time and performing a Ljung-Box test or plotting autocorrelation and partial autocorrelation of the residuals are helpful to identify misspecification.) If the estimation is inadequate, we have to return to step one and attempt to build a better model.

## Moving average (MA) versus autoregressive integrated moving average (AR)

AR(p,d,q)

* $p$ = the number of autoregressive terms
* $d$ = the number of nonseasonal differences
* $q$ = the number of moving-average terms

### AR(0,0,1) = MA(1) = first-order moving average model

ARIMA(0,0,1) is MA(1)

Image from [An introduction to Moving Average Order One processes](https://youtu.be/lUhtcP2SUsg)

### AR(1,0,0) = first-order autoregressive model

If the series is stationary and autocorrelated, perhaps it can be predicted as a multiple of its own previous value, plus a constant.

### AR(0,1,0) = random walk

If the series is not stationary, the simplest possible model for it is a random walk model.

### AR(1,1,0) = differenced first-order autoregressive model

If the errors of a random walk model are autocorrelated, perhaps the problem can be fixed by adding one lag of the dependent variable to the prediction equation.

### AR(0,1,1) without constant = simple exponential smoothing

If the errors of a random walk model are autocorrelated, perhaps the problem can be fixed by the simple exponential smoothing model.

### AR(0,2,1) or (0,2,2) without constant = linear exponential smoothing

Linear exponential smoothing models are ARIMA models which use two nonseasonal differences in conjunction with MA terms.

## Trend Analysis

Trend Analysis

Trend Analysis is the practice of collecting information and attempting to spot a pattern, or trend, in the information. Typically this involves analyzing the variance for a change over time. The null hypothesis: $H_0$ is that there is no trend. Many techniques can be used to identify trends, we'll use an ARMA model again.

##  Dickey-Fuller Test

The Dickey-Fuller Test is a test for the stationarity of a time series.

The [Dickey-Fuller test](https://en.wikipedia.org/wiki/Dickey%E2%80%93Fuller_test) tests whether a unit root is present in an autoregressive model.  simple AR(1) model is
$$
y_{t}=\rho y_{t-1}+u_{t}\,
$$

where $y_{t}$ is the variable of interest, $t$ is the time index, $\rho$ is a coefficient, and $u_{t}$ is the error term. A unit root is present if $\rho = 1$. The model would be non-stationary in this case.

The regression model can be written as
$$
 \nabla y_{t}=(\rho-1)y_{t-1}+u_{t}=\delta y_{t-1}+ u_{t}\,
$$
where $\nabla$ is the first difference operator.

The Dickey-Fuller Test uses a specific distribution simply known as the Dickey-Fuller table to assess whether $\nabla y_{t}$ is signficant.

## Seasonal Models

Seasonal Models

When there are patterns that repeat over known, fixed periods of time (i.e. day, week, month, quarter, year, etc.) within the data set it is considered to be seasonal variation. One has a model for the periodic fluctuations based on knowledge of the domain.

The seasonal ARIMA model incorporates both non-seasonal and seasonal factors in a multiplicative model.  In a seasonal ARIMA model, seasonal parameters predict xt using data values and errors at times with lags that are multiples of S (the span of the seasonality).  Before we model for a given data set, one must have an initial guess about the data generation process, that is the span of the seasonality (i.e.day, week, month, quarter, year, etc.)

The seasonal part of an ARIMA model has the same structure as the non-seasonal part: an AR factor, an MA factor, and/or an order of differencing. In the seasonal part of the model, all of these factors operate across multiples of lag s (the number of periods in a season).
A seasonal ARIMA model is classified as an ARIMA(p,d,q)x(P,D,Q) model, where P=number of seasonal autoregressive (SAR) terms, D=number of seasonal differences, Q=number of seasonal moving average (SMA) terms.

## Spectral Analysis

Spectral analysis or Spectrum analysis is based on the analysis of frequencies rather than fluctuations of numbers. In statistical signal processing, the goal of spectral density estimation (SDE) is to estimate the spectral density (also known as the power spectral density) of a random signal from a sequence of time samples of the signal. Intuitively speaking, the spectral density characterizes the frequency content of the signal. One purpose of estimating the spectral density is to detect any periodicities in the data, by observing peaks at the frequencies corresponding to these periodicities.

Statistics and signal processing, an algorithm that estimates the strength of different frequency components (the power spectrum) of a time-domain signal. This may also be called frequency domain analysis. There are many approaches to spectral density. Below is a partial list of parametric and non-parametric spectral density estimation techniques.

Non-parametric spectral density estimation techniques:

* Periodogram, the basic modulus-squared of the discrete Fourier transform
* Bartlett's method is the average of the periodograms taken of multiple segments of the signal to reduce variance of the spectral density estimate
Welch's method a windowed version of Bartlett's method that uses overlapping segments
* Multitaper is a periodogram-based method that uses multiple tapers, or windows, to form independent estimates of the spectral density to reduce variance of the spectral density estimate
* Least-squares spectral analysis, based on least squares fitting to known frequencies
* Non-uniform discrete Fourier transform is used when the signal samples are unevenly spaced in time
* Singular spectrum analysis is a nonparametric method that uses a singular value decomposition of the covariance matrix to estimate the spectral density
Short-time Fourier transform

Parametric spectral density estimation techniques:

* Autoregressive model (AR) estimation, which assumes that the nth sample is correlated with the previous p samples.
* Moving-average model (MA) estimation, which assumes that the nth sample is correlated with noise terms in the previous p samples.
* Autoregressive moving average (ARMA) estimation, which generalizes the AR and MA models.
* Maximum entropy spectral estimation is an all-poles method useful for SDE when singular spectral features, such as sharp peaks, are expected.

## Periodogram

Given a signal that is sampled at N different times, with the samples uniformly spaced by $\Delta t$, giving values $x_n$. Since the power spectral density of a continuous function defined on the entire real line is the modulus squared of its Fourier transform, the simplest technique to estimate the spectrum is the periodogram, given by the modulus squared of the discrete Fourier transform,

$$
S(f)=\frac{\Delta t}{N} \left| \sum_{n=0}^{N-1} x_n e^{-i2\pi n f} \right|^2, \qquad -\frac{1}{2\Delta t} < f \le \frac{1}{2\Delta t}
$$

where $1/(2\Delta t)$ is the Nyquist frequency. The name "periodogram" was coined by Arthur Schuster in 1898. Despite the simplicity of the periodogram, the method suffers from severe deficiencies. It is an inconsistent estimator because it does not converge to the true spectral density as $N\rightarrow\infty$. It exhibits very high spectral leakage although this can be reduced by multiplying $x_n$ by a window function. In the presence of additive noise, the estimate has a positive bias.

## Fourier analysis

[Fourier analysis](https://en.wikipedia.org/wiki/Fourier_analysis) is the study of the way general functions may be represented or approximated by sums of simpler trigonometric functions. Fourier analysis grew from the study of Fourier series. A [Fourier series](https://en.wikipedia.org/wiki/Fourier_series) is a way to represent a (wave-like) function as the sum of simple sine waves

If, s(x) denotes a function of the real variable x, and s is integrable on an interval [x0, x0 + P], for real numbers x0 and P. We will attempt to represent  s  in that interval as an infinite sum, or series, of harmonically related sinusoidal functions. Outside the interval, the series is periodic with period P (frequency 1/P). It follows that if s also has that property, the approximation is valid on the entire real line. We can begin with a finite summation (or partial sum):

$$
s_N(x) = \frac{A_0}{2} + \sum_{n=1}^N A_n\cdot \sin(\tfrac{2\pi nx}{P}+\phi_n), \quad \scriptstyle \text{for integer}\ N\ \ge\ 1.
$$

$s_N(x)$  is a periodic function with period P.  Using the identities:

$$
\sin(\tfrac{2\pi nx}{P}+\phi_n) \equiv \sin(\phi_n) \cos(\tfrac{2\pi nx}{P}) + \cos(\phi_n) \sin(\tfrac{2\pi nx}{P})
$$
$$
\sin(\tfrac{2\pi nx}{P}+\phi_n) \equiv \text{Re}\left\{\frac{1}{i}\cdot e^{i \left(\tfrac{2\pi nx}{P}+\phi_n\right)}\right\} = \frac{1}{2i}\cdot e^{i \left(\tfrac{2\pi nx}{P}+\phi_n\right)} +\left(\frac{1}{2i}\cdot e^{i \left(\tfrac{2\pi nx}{P}+\phi_n\right)}\right)^*,
$$

Function s(x) (in red) is a sum of six sine functions of different amplitudes and harmonically related frequencies. Their summation is called a Fourier series. The Fourier transform, S(f) (in blue), which depicts amplitude vs frequency, reveals the 6 frequencies and their amplitudes.

## AIC, BIC

Statistical tests are used to assess the fit of a regression model. The most common being the *Akaike information criterion (AIC)*, the *Bayesian information criterion (BIC)*

_Akaike information criterion_

The Akaike information criterion (AIC) is a measure of the relative quality of statistical models for a given set of data. Given a collection of models for the data, AIC estimates the quality of each model, relative to each of the other models. Hence, AIC provides a means for model selection.

AIC is founded on information theory: it offers a relative estimate of the information lost when a given model is used to represent the process that generates the data. In doing so, it deals with the trade-off between the goodness of fit of the model and the complexity of the model.

- from [Akaike information criterion - Wikipedia](https://en.wikipedia.org/wiki/Akaike_information_criterion)

_Bayesian information criterion (BIC)_

In statistics, the Bayesian information criterion (BIC) or Schwarz criterion (also SBC, SBIC) is a criterion for model selection among a finite set of models; the model with the lowest BIC is preferred. It is based, in part, on the likelihood function and it is closely related to the Akaike information criterion (AIC).

When fitting models, it is possible to increase the likelihood by adding parameters, but doing so may result in overfitting. Both BIC and AIC resolve this problem by introducing a penalty term for the number of parameters in the model; the penalty term is larger in BIC than in AIC. The BIC was developed by Gideon E. Schwarz and published in a 1978 paper, where he gave a Bayesian argument for adopting it.

- from [Bayesian information criterion (BIC) - Wikipedia](https://en.wikipedia.org/wiki/Bayesian_information_criterion)

Test          | "Good"
------------- | -------------
$R^2$         | High
AIC           | Low
BIC           | Low

## Time Series in python

_statsmodels_ has a dataset called "Atmospheric CO2 from Continuous Air Samples at Mauna Loa Observatory, Hawaii, U.S.A.," which collected CO2 samples from March 1958 to December 2001.

The following code is adapted from [Time Series Forecasting with ARIMA in Python 3](https://www.digitalocean.com/community/tutorials/a-guide-to-time-series-forecasting-with-arima-in-python-3)

The AIC measures how well a model fits the data while taking into account the overall complexity of the model. A model that fits the data very well while using lots of features will be assigned a larger AIC score than a model that uses fewer features to achieve the same goodness-of-fit. Therefore, we are interested in finding the model that yields the lowest AIC value.

```python
co2_data = sm.datasets.co2.load_pandas().data
co2_data.head()
```

<div>
<style>
    .dataframe thead tr:only-child th {
        text-align: right;
    }

    .dataframe thead th {
        text-align: left;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>co2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>1958-03-29</th>
      <td>316.1</td>
    </tr>
    <tr>
      <th>1958-04-05</th>
      <td>317.3</td>
    </tr>
    <tr>
      <th>1958-04-12</th>
      <td>317.6</td>
    </tr>
    <tr>
      <th>1958-04-19</th>
      <td>317.5</td>
    </tr>
    <tr>
      <th>1958-04-26</th>
      <td>316.4</td>
    </tr>
  </tbody>
</table>
</div>

```python
co2_data = co2_data.fillna(co2_data.bfill())
co2_data.head()
```

<div>
<style>
    .dataframe thead tr:only-child th {
        text-align: right;
    }

    .dataframe thead th {
        text-align: left;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>co2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>1958-03-29</th>
      <td>316.1</td>
    </tr>
    <tr>
      <th>1958-04-05</th>
      <td>317.3</td>
    </tr>
    <tr>
      <th>1958-04-12</th>
      <td>317.6</td>
    </tr>
    <tr>
      <th>1958-04-19</th>
      <td>317.5</td>
    </tr>
    <tr>
      <th>1958-04-26</th>
      <td>316.4</td>
    </tr>
  </tbody>
</table>
</div>

```python
co2_data.plot()
```

```python
from pandas.tools.plotting import autocorrelation_plot

autocorrelation_plot(co2_data['co2'])
```

```python
from statsmodels.graphics.tsaplots import plot_acf
plot_acf(co2_data['co2'], lags=52)
```

```python
plot_acf(co2_data['co2'], lags=7)
```

```python
co2_data_month = co2_data['co2'].resample('M').mean()
# The 'M' indicates we want to resample by month. At the bottom of this post is a summary of different time frames.
# mean() is used to indicate we want to use the mean for the resample.
# The level on which to roll-up to, 'D' for day, 'W' for week, 'M' for month, 'A' for year
# What aggregation to perform: 'mean', 'median', 'sum', etc.
co2_data_month.head()
```

    1958-03-31    316.100
    1958-04-30    317.200
    1958-05-31    317.120
    1958-06-30    315.800
    1958-07-31    315.625
    Freq: M, Name: co2, dtype: float64

```python
co2_data_month.plot()
```

```python
co2_data_week = co2_data['co2'].resample('W').mean()
# The 'W' indicates we want to resample by week. At the bottom of this post is a summary of different time frames.
# mean() is used to indicate we want to use the mean for the resample.
co2_data_week.head()
```

    1958-03-30    316.1
    1958-04-06    317.3
    1958-04-13    317.6
    1958-04-20    317.5
    1958-04-27    316.4
    Freq: W-SUN, Name: co2, dtype: float64

```python
co2_data_week.tail()
```

    2001-12-02    370.3
    2001-12-09    370.8
    2001-12-16    371.2
    2001-12-23    371.3
    2001-12-30    371.5
    Freq: W-SUN, Name: co2, dtype: float64

```python
co2_data_week.plot()
```

```python
# Define the p, d and q parameters to take any value between 0 and 2
p = d = q = range(0, 2)

# Generate all different combinations of p, q and q triplets
pdq = list(itertools.product(p, d, q))

# Generate all different combinations of seasonal p, q and q triplets
seasonal_pdq = [(x[0], x[1], x[2], 12) for x in list(itertools.product(p, d, q))]

print('Examples of parameter combinations for Seasonal ARIMA...')
print('SARIMAX: {} x {}'.format(pdq[1], seasonal_pdq[1]))
print('SARIMAX: {} x {}'.format(pdq[1], seasonal_pdq[2]))
print('SARIMAX: {} x {}'.format(pdq[2], seasonal_pdq[3]))
print('SARIMAX: {} x {}'.format(pdq[2], seasonal_pdq[4]))
```

    Examples of parameter combinations for Seasonal ARIMA...
    SARIMAX: (0, 0, 1) x (0, 0, 1, 12)
    SARIMAX: (0, 0, 1) x (0, 1, 0, 12)
    SARIMAX: (0, 1, 0) x (0, 1, 1, 12)
    SARIMAX: (0, 1, 0) x (1, 0, 0, 12)

```python
mod = sm.tsa.ARIMA(co2_data_week, order=pdq[0])
results = mod.fit()
print('ARIMA{} - AIC:{}'.format(pdq[0], results.aic))
```

    ARIMA(0, 0, 0) - AIC:19453.86166651757

```python
for param in pdq:
    print('Trying ARIMA{}'.format(param))
    try:
        mod = sm.tsa.ARIMA(co2_data_week,
                                            order=param)

        results = mod.fit()
        print('ARIMA{} - AIC:{}'.format(param, results.aic))
    except:
        continue
```

    Trying ARIMA(0, 0, 0)
    ARIMA(0, 0, 0) - AIC:19453.86166651757
    Trying ARIMA(0, 0, 1)
    ARIMA(0, 0, 1) - AIC:16376.291798833521
    Trying ARIMA(0, 1, 0)
    ARIMA(0, 1, 0) - AIC:3279.9105252889694
    Trying ARIMA(0, 1, 1)
    ARIMA(0, 1, 1) - AIC:3270.363670525847
    Trying ARIMA(1, 0, 0)
    ARIMA(1, 0, 0) - AIC:3296.5351535841355
    Trying ARIMA(1, 0, 1)
    ARIMA(1, 0, 1) - AIC:3286.347980023006
    Trying ARIMA(1, 1, 0)
    ARIMA(1, 1, 0) - AIC:3267.028227686567
    Trying ARIMA(1, 1, 1)
    ARIMA(1, 1, 1) - AIC:3080.476465251886

### http://www.statsmodels.org/ notebooks

The examples below are adapted to the [statsmodels.org](http://www.statsmodels.org) notebooks

```python
# Dataset
wpi1 = requests.get('http://www.stata-press.com/data/r12/wpi1.dta').content
data = pd.read_stata(BytesIO(wpi1))
data.index = data.t

# Fit the model
mod = sm.tsa.statespace.SARIMAX(data['wpi'], trend='c', order=(1,1,1))
res = mod.fit(disp=False)
res.summary()
```

<table class="simpletable">
<caption>Statespace Model Results</caption>
<tr>
  <th>Dep. Variable:</th>          <td>wpi</td>       <th>  No. Observations:  </th>    <td>124</td>
</tr>
<tr>
  <th>Model:</th>           <td>SARIMAX(1, 1, 1)</td> <th>  Log Likelihood     </th> <td>-134.983</td>
</tr>
<tr>
  <th>Date:</th>            <td>Sun, 24 Sep 2017</td> <th>  AIC                </th>  <td>277.965</td>
</tr>
<tr>
  <th>Time:</th>                <td>17:47:00</td>     <th>  BIC                </th>  <td>289.246</td>
</tr>
<tr>
  <th>Sample:</th>             <td>01-01-1960</td>    <th>  HQIC               </th>  <td>282.548</td>
</tr>
<tr>
  <th></th>                   <td>- 10-01-1990</td>   <th>                     </th>     <td> </td>
</tr>
<tr>
  <th>Covariance Type:</th>        <td>opg</td>       <th>                     </th>     <td> </td>
</tr>
</table>
<table class="simpletable">
<tr>
      <td></td>         <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th>  <th>[0.025</th>    <th>0.975]</th>
</tr>
<tr>
  <th>intercept</th> <td>    0.1050</td> <td>    0.068</td> <td>    1.546</td> <td> 0.122</td> <td>   -0.028</td> <td>    0.238</td>
</tr>
<tr>
  <th>ar.L1</th>     <td>    0.8740</td> <td>    0.054</td> <td>   16.178</td> <td> 0.000</td> <td>    0.768</td> <td>    0.980</td>
</tr>
<tr>
  <th>ma.L1</th>     <td>   -0.4206</td> <td>    0.100</td> <td>   -4.191</td> <td> 0.000</td> <td>   -0.617</td> <td>   -0.224</td>
</tr>
<tr>
  <th>sigma2</th>    <td>    0.5226</td> <td>    0.053</td> <td>    9.926</td> <td> 0.000</td> <td>    0.419</td> <td>    0.626</td>
</tr>
</table>
<table class="simpletable">
<tr>
  <th>Ljung-Box (Q):</th>          <td>36.96</td> <th>  Jarque-Bera (JB):  </th> <td>10.15</td>
</tr>
<tr>
  <th>Prob(Q):</th>                <td>0.61</td>  <th>  Prob(JB):          </th> <td>0.01</td>
</tr>
<tr>
  <th>Heteroskedasticity (H):</th> <td>17.00</td> <th>  Skew:              </th> <td>0.27</td>
</tr>
<tr>
  <th>Prob(H) (two-sided):</th>    <td>0.00</td>  <th>  Kurtosis:          </th> <td>4.30</td>
</tr>
</table>

```python
# Dataset
data = pd.read_stata(BytesIO(wpi1))
data.index = data.t
data['ln_wpi'] = np.log(data['wpi'])
data['D.ln_wpi'] = data['ln_wpi'].diff()
```

```python
# Graph data
fig, axes = plt.subplots(1, 2, figsize=(15,4))

# Levels
axes[0].plot(data.index._mpl_repr(), data['wpi'], '-')
axes[0].set(title='US Wholesale Price Index')

# Log difference
axes[1].plot(data.index._mpl_repr(), data['D.ln_wpi'], '-')
axes[1].hlines(0, data.index[0], data.index[-1], 'r')
axes[1].set(title='US Wholesale Price Index - difference of logs');
```

```python
# Graph data
fig, axes = plt.subplots(1, 2, figsize=(15,4))

fig = sm.graphics.tsa.plot_acf(data.ix[1:, 'D.ln_wpi'], lags=40, ax=axes[0])
fig = sm.graphics.tsa.plot_pacf(data.ix[1:, 'D.ln_wpi'], lags=40, ax=axes[1])
```

```python
# Fit the model
mod = sm.tsa.statespace.SARIMAX(data['ln_wpi'], trend='c', order=(1,1,1))
res = mod.fit(disp=False)
res.summary()
```

<table class="simpletable">
<caption>Statespace Model Results</caption>
<tr>
  <th>Dep. Variable:</th>        <td>ln_wpi</td>      <th>  No. Observations:  </th>    <td>124</td>
</tr>
<tr>
  <th>Model:</th>           <td>SARIMAX(1, 1, 1)</td> <th>  Log Likelihood     </th>  <td>382.365</td>
</tr>
<tr>
  <th>Date:</th>            <td>Sun, 24 Sep 2017</td> <th>  AIC                </th> <td>-756.730</td>
</tr>
<tr>
  <th>Time:</th>                <td>17:47:01</td>     <th>  BIC                </th> <td>-745.448</td>
</tr>
<tr>
  <th>Sample:</th>             <td>01-01-1960</td>    <th>  HQIC               </th> <td>-752.147</td>
</tr>
<tr>
  <th></th>                   <td>- 10-01-1990</td>   <th>                     </th>     <td> </td>
</tr>
<tr>
  <th>Covariance Type:</th>        <td>opg</td>       <th>                     </th>     <td> </td>
</tr>
</table>
<table class="simpletable">
<tr>
      <td></td>         <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th>  <th>[0.025</th>    <th>0.975]</th>
</tr>
<tr>
  <th>intercept</th> <td>    0.0019</td> <td>    0.001</td> <td>    1.669</td> <td> 0.095</td> <td>   -0.000</td> <td>    0.004</td>
</tr>
<tr>
  <th>ar.L1</th>     <td>    0.8351</td> <td>    0.049</td> <td>   16.975</td> <td> 0.000</td> <td>    0.739</td> <td>    0.932</td>
</tr>
<tr>
  <th>ma.L1</th>     <td>   -0.3777</td> <td>    0.094</td> <td>   -4.004</td> <td> 0.000</td> <td>   -0.563</td> <td>   -0.193</td>
</tr>
<tr>
  <th>sigma2</th>    <td>    0.0001</td> <td> 9.94e-06</td> <td>   11.679</td> <td> 0.000</td> <td> 9.66e-05</td> <td>    0.000</td>
</tr>
</table>
<table class="simpletable">
<tr>
  <th>Ljung-Box (Q):</th>          <td>39.62</td> <th>  Jarque-Bera (JB):  </th> <td>84.28</td>
</tr>
<tr>
  <th>Prob(Q):</th>                <td>0.49</td>  <th>  Prob(JB):          </th> <td>0.00</td>
</tr>
<tr>
  <th>Heteroskedasticity (H):</th> <td>2.47</td>  <th>  Skew:              </th> <td>0.84</td>
</tr>
<tr>
  <th>Prob(H) (two-sided):</th>    <td>0.00</td>  <th>  Kurtosis:          </th> <td>6.69</td>
</tr>
</table>

```python
# Dataset
air2 = requests.get('http://www.stata-press.com/data/r12/air2.dta').content
data = pd.read_stata(BytesIO(air2))
data.index = pd.date_range(start=datetime(data.time[0], 1, 1), periods=len(data), freq='MS')
data['lnair'] = np.log(data['air'])

# Fit the model
mod = sm.tsa.statespace.SARIMAX(data['lnair'], order=(2,1,0), seasonal_order=(1,1,0,12), simple_differencing=True)
res = mod.fit(disp=False)
res.summary()
```

<table class="simpletable">
<caption>Statespace Model Results</caption>
<tr>
  <th>Dep. Variable:</th>            <td>D.DS12.lnair</td>          <th>  No. Observations:  </th>    <td>131</td>
</tr>
<tr>
  <th>Model:</th>           <td>SARIMAX(2, 0, 0)x(1, 0, 0, 12)</td> <th>  Log Likelihood     </th>  <td>240.821</td>
</tr>
<tr>
  <th>Date:</th>                   <td>Sun, 24 Sep 2017</td>        <th>  AIC                </th> <td>-473.643</td>
</tr>
<tr>
  <th>Time:</th>                       <td>17:47:01</td>            <th>  BIC                </th> <td>-462.142</td>
</tr>
<tr>
  <th>Sample:</th>                    <td>02-01-1950</td>           <th>  HQIC               </th> <td>-468.970</td>
</tr>
<tr>
  <th></th>                          <td>- 12-01-1960</td>          <th>                     </th>     <td> </td>
</tr>
<tr>
  <th>Covariance Type:</th>               <td>opg</td>              <th>                     </th>     <td> </td>
</tr>
</table>
<table class="simpletable">
<tr>
      <td></td>        <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th>  <th>[0.025</th>    <th>0.975]</th>
</tr>
<tr>
  <th>ar.L1</th>    <td>   -0.4057</td> <td>    0.080</td> <td>   -5.045</td> <td> 0.000</td> <td>   -0.563</td> <td>   -0.248</td>
</tr>
<tr>
  <th>ar.L2</th>    <td>   -0.0799</td> <td>    0.099</td> <td>   -0.809</td> <td> 0.419</td> <td>   -0.274</td> <td>    0.114</td>
</tr>
<tr>
  <th>ar.S.L12</th> <td>   -0.4723</td> <td>    0.072</td> <td>   -6.592</td> <td> 0.000</td> <td>   -0.613</td> <td>   -0.332</td>
</tr>
<tr>
  <th>sigma2</th>   <td>    0.0014</td> <td>    0.000</td> <td>    8.403</td> <td> 0.000</td> <td>    0.001</td> <td>    0.002</td>
</tr>
</table>
<table class="simpletable">
<tr>
  <th>Ljung-Box (Q):</th>          <td>49.89</td> <th>  Jarque-Bera (JB):  </th> <td>0.72</td>
</tr>
<tr>
  <th>Prob(Q):</th>                <td>0.14</td>  <th>  Prob(JB):          </th> <td>0.70</td>
</tr>
<tr>
  <th>Heteroskedasticity (H):</th> <td>0.54</td>  <th>  Skew:              </th> <td>0.14</td>
</tr>
<tr>
  <th>Prob(H) (two-sided):</th>    <td>0.04</td>  <th>  Kurtosis:          </th> <td>3.23</td>
</tr>
</table>

```python
# Dataset
friedman2 = requests.get('http://www.stata-press.com/data/r12/friedman2.dta').content
data = pd.read_stata(BytesIO(friedman2))
data.index = data.time

# Variables
endog = data.ix['1959':'1981', 'consump']
exog = sm.add_constant(data.ix['1959':'1981', 'm2'])

# Fit the model
mod = sm.tsa.statespace.SARIMAX(endog, exog, order=(1,0,1))
res = mod.fit(disp=False)
res.summary()
```

<table class="simpletable">
<caption>Statespace Model Results</caption>
<tr>
  <th>Dep. Variable:</th>        <td>consump</td>     <th>  No. Observations:  </th>    <td>92</td>
</tr>
<tr>
  <th>Model:</th>           <td>SARIMAX(1, 0, 1)</td> <th>  Log Likelihood     </th> <td>-340.508</td>
</tr>
<tr>
  <th>Date:</th>            <td>Sun, 24 Sep 2017</td> <th>  AIC                </th>  <td>691.015</td>
</tr>
<tr>
  <th>Time:</th>                <td>17:47:02</td>     <th>  BIC                </th>  <td>703.624</td>
</tr>
<tr>
  <th>Sample:</th>             <td>01-01-1959</td>    <th>  HQIC               </th>  <td>696.105</td>
</tr>
<tr>
  <th></th>                   <td>- 10-01-1981</td>   <th>                     </th>     <td> </td>
</tr>
<tr>
  <th>Covariance Type:</th>        <td>opg</td>       <th>                     </th>     <td> </td>
</tr>
</table>
<table class="simpletable">
<tr>
     <td></td>       <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th>  <th>[0.025</th>    <th>0.975]</th>
</tr>
<tr>
  <th>const</th>  <td>  -36.0651</td> <td>   56.632</td> <td>   -0.637</td> <td> 0.524</td> <td> -147.062</td> <td>   74.932</td>
</tr>
<tr>
  <th>m2</th>     <td>    1.1220</td> <td>    0.036</td> <td>   30.830</td> <td> 0.000</td> <td>    1.051</td> <td>    1.193</td>
</tr>
<tr>
  <th>ar.L1</th>  <td>    0.9348</td> <td>    0.041</td> <td>   22.717</td> <td> 0.000</td> <td>    0.854</td> <td>    1.015</td>
</tr>
<tr>
  <th>ma.L1</th>  <td>    0.3091</td> <td>    0.089</td> <td>    3.488</td> <td> 0.000</td> <td>    0.135</td> <td>    0.483</td>
</tr>
<tr>
  <th>sigma2</th> <td>   93.2547</td> <td>   10.888</td> <td>    8.565</td> <td> 0.000</td> <td>   71.914</td> <td>  114.596</td>
</tr>
</table>
<table class="simpletable">
<tr>
  <th>Ljung-Box (Q):</th>          <td>38.72</td> <th>  Jarque-Bera (JB):  </th> <td>23.49</td>
</tr>
<tr>
  <th>Prob(Q):</th>                <td>0.53</td>  <th>  Prob(JB):          </th> <td>0.00</td>
</tr>
<tr>
  <th>Heteroskedasticity (H):</th> <td>22.51</td> <th>  Skew:              </th> <td>0.17</td>
</tr>
<tr>
  <th>Prob(H) (two-sided):</th>    <td>0.00</td>  <th>  Kurtosis:          </th> <td>5.45</td>
</tr>
</table>

```python
# Dataset
raw = pd.read_stata(BytesIO(friedman2))
raw.index = raw.time
data = raw.ix[:'1981']

# Variables
endog = data.ix['1959':, 'consump']
exog = sm.add_constant(data.ix['1959':, 'm2'])
nobs = endog.shape[0]

# Fit the model
mod = sm.tsa.statespace.SARIMAX(endog.ix[:'1978-01-01'], exog=exog.ix[:'1978-01-01'], order=(1,0,1))
fit_res = mod.fit(disp=False)
fit_res.summary()
```

<table class="simpletable">
<caption>Statespace Model Results</caption>
<tr>
  <th>Dep. Variable:</th>        <td>consump</td>     <th>  No. Observations:  </th>    <td>77</td>
</tr>
<tr>
  <th>Model:</th>           <td>SARIMAX(1, 0, 1)</td> <th>  Log Likelihood     </th> <td>-243.316</td>
</tr>
<tr>
  <th>Date:</th>            <td>Sun, 24 Sep 2017</td> <th>  AIC                </th>  <td>496.633</td>
</tr>
<tr>
  <th>Time:</th>                <td>17:47:02</td>     <th>  BIC                </th>  <td>508.352</td>
</tr>
<tr>
  <th>Sample:</th>             <td>01-01-1959</td>    <th>  HQIC               </th>  <td>501.320</td>
</tr>
<tr>
  <th></th>                   <td>- 01-01-1978</td>   <th>                     </th>     <td> </td>
</tr>
<tr>
  <th>Covariance Type:</th>        <td>opg</td>       <th>                     </th>     <td> </td>
</tr>
</table>
<table class="simpletable">
<tr>
     <td></td>       <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th>  <th>[0.025</th>    <th>0.975]</th>
</tr>
<tr>
  <th>const</th>  <td>    0.6786</td> <td>   18.492</td> <td>    0.037</td> <td> 0.971</td> <td>  -35.565</td> <td>   36.922</td>
</tr>
<tr>
  <th>m2</th>     <td>    1.0379</td> <td>    0.021</td> <td>   50.329</td> <td> 0.000</td> <td>    0.997</td> <td>    1.078</td>
</tr>
<tr>
  <th>ar.L1</th>  <td>    0.8775</td> <td>    0.059</td> <td>   14.859</td> <td> 0.000</td> <td>    0.762</td> <td>    0.993</td>
</tr>
<tr>
  <th>ma.L1</th>  <td>    0.2771</td> <td>    0.108</td> <td>    2.572</td> <td> 0.010</td> <td>    0.066</td> <td>    0.488</td>
</tr>
<tr>
  <th>sigma2</th> <td>   31.6989</td> <td>    4.683</td> <td>    6.769</td> <td> 0.000</td> <td>   22.520</td> <td>   40.878</td>
</tr>
</table>
<table class="simpletable">
<tr>
  <th>Ljung-Box (Q):</th>          <td>46.77</td> <th>  Jarque-Bera (JB):  </th> <td>6.05</td>
</tr>
<tr>
  <th>Prob(Q):</th>                <td>0.21</td>  <th>  Prob(JB):          </th> <td>0.05</td>
</tr>
<tr>
  <th>Heteroskedasticity (H):</th> <td>6.09</td>  <th>  Skew:              </th> <td>0.57</td>
</tr>
<tr>
  <th>Prob(H) (two-sided):</th>    <td>0.00</td>  <th>  Kurtosis:          </th> <td>3.76</td>
</tr>
</table>

```python
mod = sm.tsa.statespace.SARIMAX(endog, exog=exog, order=(1,0,1))
res = mod.filter(fit_res.params)
# In-sample one-step-ahead predictions
predict = res.get_prediction()
predict_ci = predict.conf_int()
# Dynamic predictions
predict_dy = res.get_prediction(dynamic='1978-01-01')
predict_dy_ci = predict_dy.conf_int()
# Graph
fig, ax = plt.subplots(figsize=(9,4))
npre = 4
ax.set(title='Personal consumption', xlabel='Date', ylabel='Billions of dollars')

# Plot data points
data.ix['1977-07-01':, 'consump'].plot(ax=ax, style='o', label='Observed')

# Plot predictions
predict.predicted_mean.ix['1977-07-01':].plot(ax=ax, style='r--', label='One-step-ahead forecast')
ci = predict_ci.ix['1977-07-01':]
ax.fill_between(ci.index, ci.ix[:,0], ci.ix[:,1], color='r', alpha=0.1)
predict_dy.predicted_mean.ix['1977-07-01':].plot(ax=ax, style='g', label='Dynamic forecast (1978)')
ci = predict_dy_ci.ix['1977-07-01':]
ax.fill_between(ci.index, ci.ix[:,0], ci.ix[:,1], color='g', alpha=0.1)

legend = ax.legend(loc='lower right')
```

```python
# Prediction error

# Graph
fig, ax = plt.subplots(figsize=(9,4))
npre = 4
ax.set(title='Forecast error', xlabel='Date', ylabel='Forecast - Actual')

# In-sample one-step-ahead predictions and 95% confidence intervals
predict_error = predict.predicted_mean - endog
predict_error.ix['1977-10-01':].plot(ax=ax, label='One-step-ahead forecast')
ci = predict_ci.ix['1977-10-01':].copy()
ci.iloc[:,0] -= endog.loc['1977-10-01':]
ci.iloc[:,1] -= endog.loc['1977-10-01':]
ax.fill_between(ci.index, ci.ix[:,0], ci.ix[:,1], alpha=0.1)

# Dynamic predictions and 95% confidence intervals
predict_dy_error = predict_dy.predicted_mean - endog
predict_dy_error.ix['1977-10-01':].plot(ax=ax, style='r', label='Dynamic forecast (1978)')
ci = predict_dy_ci.ix['1977-10-01':].copy()
ci.iloc[:,0] -= endog.loc['1977-10-01':]
ci.iloc[:,1] -= endog.loc['1977-10-01':]
ax.fill_between(ci.index, ci.ix[:,0], ci.ix[:,1], color='r', alpha=0.1)

legend = ax.legend(loc='lower left');
legend.get_frame().set_facecolor('w')
```


#  Neuromatch Academy: Week 1, Day 1, Tutorial 2
# Model Types: "How" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

___
# Tutorial Objectives
This is tutorial 2 of a 3-part series on different flavors of models used to understand neural data. In this tutorial we will explore models that can potentially explain *how* the spiking data we have observed is produced

To understand the mechanisms that give rise to the neural data we save in Tutorial 1,  we will build simple neuronal models and compare their spiking response to real data. We will:
- Write code to simulate a  simple "leaky integrate-and-fire" neuron model
- Make the model more complicated — but also more realistic — by adding more physiologically-inspired details

```python
#@title Video 1: "How" models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='PpnagITsb3E', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=PpnagITsb3E

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")

```

```python
#@title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
  """Plot a step histogram given counts over bins."""
  if ax is None:
    _, ax = plt.subplots()

  # duplicate the first element of `counts` to match bin edges
  counts = np.insert(counts, 0, counts[0])

  ax.fill_between(bins, counts, step="pre", alpha=0.4, **kwargs)  # area shading
  ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

  for x in vlines:
    ax.axvline(x, color='r', linestyle='dotted')  # vertical line

  if ax_args is None:
    ax_args = {}

  # heuristically set max y to leave a bit of room
  ymin, ymax = ax_args.get('ylim', [None, None])
  if ymax is None:
    ymax = np.max(counts)
    if ax_args.get('yscale', 'linear') == 'log':
      ymax *= 1.5
    else:
      ymax *= 1.1
      if ymin is None:
        ymin = 0

  if ymax == ymin:
    ymax = None

  ax_args['ylim'] = [ymin, ymax]

  ax.set(**ax_args)
  ax.autoscale(enable=False, axis='x', tight=True)

def plot_neuron_stats(v, spike_times):
  fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))

  # membrane voltage trace
  ax1.plot(v[0:100])
  ax1.set(xlabel='Time', ylabel='Voltage')
  # plot spike events
  for x in spike_times:
    if x >= 100:
      break
    ax1.axvline(x, color='red')

  # ISI distribution
  isi = np.diff(spike_times)
  n_bins = np.arange(isi.min(), isi.max() + 2) - .5
  counts, bins = np.histogram(isi, n_bins)
  vlines = []
  if len(isi) > 0:
    vlines = [np.mean(isi)]
  xmax = max(20, int(bins[-1])+5)
  histogram(counts, bins, vlines=vlines, ax=ax2, ax_args={
    'xlabel': 'Inter-spike interval',
    'ylabel': 'Number of intervals',
    'xlim': [0, xmax]
  })
  plt.show()
```

# Section 1: The Linear Integrate-and-Fire Neuron

How does a neuron spike?

A neuron charges and discharges an electric field across its cell membrane. The state of this electric field can be described by the _membrane potential_. The membrane potential rises due to excitation of the neuron, and when it reaches a threshold a spike occurs. The potential resets, and must rise to a threshold again before the next spike occurs.

One of the simplest models of spiking neuron behavior is the linear integrate-and-fire model neuron. In this model, the neuron increases its membrane potential $V_m$ over time in response to excitatory input currents $I$ scaled by some factor $\alpha$:

\begin{align}
  dV_m = {\alpha}I
\end{align}

Once $V_m$ reaches a threshold value a spike is produced, $V_m$ is reset to a starting value, and the process continues.

Here, we will take the starting and threshold potentials as $0$ and $1$, respectively. So, for example, if $\alpha I=0.1$ is constant---that is, the input current is constant---then $dV_m=0.1$, and at each timestep the membrane potential $V_m$ increases by $0.1$ until after $(1-0)/0.1 = 10$ timesteps it reaches the threshold and resets to $V_m=0$, and so on.

Note that we define the membrane potential $V_m$ as a scalar: a single real (or floating point) number. However, a biological neuron's membrane potential will not be exactly constant at all points on its cell membrane at a given time. We could capture this variation with a more complex model (e.g. with more numbers). Do we need to?

The proposed model is a 1D simplification. There are many details we could add to it, to preserve different parts of the complex structure and dynamics of a real neuron. If we were interested in small or local changes in the membrane potential, our 1D simplification could be a problem. However, we'll assume an idealized "point" neuron model for our current purpose.

#### Spiking Inputs

Given our simplified model for the neuron dynamics, we still need to consider what form the input $I$ will take. How should we specify the firing behavior of the presynaptic neuron(s) providing the inputs to our model neuron?

Unlike in the simple example above, where $\alpha I=0.1$, the input current is generally not constant. Physical inputs tend to vary with time. We can describe this variation with a distribution.

We'll assume the input current $I$ over a timestep is due to equal contributions from a non-negative ($\ge 0$) integer number of input spikes arriving in that timestep. Our model neuron might integrate currents from 3 input spikes in one timestep, and 7 spikes in the next timestep. We should see similar behavior when sampling from our distribution.

Given no other information about the input neurons, we will also assume that the distribution has a mean (i.e. mean rate, or number of spikes received per timestep), and that the spiking events of the input neuron(s) are independent in time. Are these reasonable assumptions in the context of real neurons?

A suitable distribution given these assumptions is the Poisson distribution, which we'll use to model $I$:

\begin{align}
  I \sim \mathrm{Poisson}(\lambda)
\end{align}

where $\lambda$ is the mean of the distribution: the average rate of spikes received per timestep.

### Exercise 1: Compute $dV_m$

For your first exercise, you will write the code to compute the change in voltage $dV_m$ (per timestep) of the linear integrate-and-fire model neuron. The rest of the code to handle numerical integration is provided for you, so you just need to fill in a definition for `dv` in the `lif_neuron` function below. The value of $\lambda$ for the Poisson random variable is given by the function argument `rate`.

The [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) package is a great resource for working with and sampling from various probability distributions. We will use the `scipy.stats.poisson` class and its method `rvs` to produce Poisson-distributed random samples. In this tutorial, we have imported this package with the alias `stats`, so you should refer to it in your code as `stats.poisson`.

```python
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes

  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  for i in range(1, n_steps):

    dv = alpha*exc[i]

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines after completing the lif_neuron function
v, spike_times = lif_neuron()
plot_neuron_stats(v, spike_times)

t = np.arange(20)
p = [1-stats.poisson(10*time).cdf(np.floor(1/0.01)) for time in t]
plt.plot(t[:-1],np.diff(p))
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial2_Solution_f8960ca1.py)

*Example output:*

## Interactive Demo: Linear-IF neuron
Like last time, you can now explore how various parametes of the LIF model influence the ISI distribution.

```python
#@title

#@markdown You don't need to worry about how the code works – but you do need to **run the cell** to enable the sliders.

def _lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  exc = stats.poisson(rate).rvs(n_steps)
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = alpha * exc[i]
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0
  return v, spike_times

@widgets.interact(
  n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
  alpha=widgets.FloatLogSlider(0.01, min=-2, max=-1),
  rate=widgets.IntSlider(10, min=5, max=20)
)
def plot_lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  v, spike_times = _lif_neuron(int(n_steps), alpha, rate)
  plot_neuron_stats(v, spike_times)
```

    interactive(children=(FloatLogSlider(value=1000.0, description='n_steps', min=2.0), FloatLogSlider(value=0.01,…

```python
#@title Video 2: Linear-IF models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='QBD7kulhg4U', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=QBD7kulhg4U

# Section 2: Inhibitory signals

Our linear integrate-and-fire neuron from the previous section was indeed able to produce spikes. However, our ISI histogram doesn't look much like empirical ISI histograms seen in Tutorial 1, which had an exponential-like shape. What is our model neuron missing, given that it doesn't behave like a real neuron?

In the previous model we only considered excitatory behavior -- the only way the membrane potential could decrease was upon a spike event. We know, however, that there are other factors that can drive $V_m$ down. First is the natural tendency of the neuron to return to some steady state or resting potential. We can update our previous model as follows:

\begin{align}
  dV_m = -{\beta}V_m + {\alpha}I
\end{align}

where $V_m$ is the current membrane potential and $\beta$ is some leakage factor. This is a basic form of the popular Leaky Integrate-and-Fire model neuron (for a more detailed discussion of the LIF Neuron, see the Appendix).

We also know that in addition to excitatory presynaptic neurons, we can have inhibitory presynaptic neurons as well. We can model these inhibitory neurons with another Poisson random variable:

\begin{align}
I = I_{exc} - I_{inh} \\
I_{exc} \sim \mathrm{Poisson}(\lambda_{exc}) \\
I_{inh} \sim \mathrm{Poisson}(\lambda_{inh})
\end{align}

where $\lambda_{exc}$ and $\lambda_{inh}$ are the average spike rates (per timestep) of the excitatory and inhibitory presynaptic neurons, respectively.

### Exercise 2: Compute $dV_m$ with inhibitory signals

For your second exercise, you will again write the code to compute the change in voltage $dV_m$, though now of the LIF model neuron described above. Like last time, the rest of the code needed to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` below.

```python
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """

  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  for i in range(1, n_steps):

    dv = -beta*v[i-1] + alpha*(exc[i]-inh[i])

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines do make the plot once you've completed the function
v, spike_times = lif_neuron_inh()
plot_neuron_stats(v, spike_times)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial2_Solution_4d9a2677.py)

*Example output:*

## Interactive Demo: LIF + inhibition neuron

```python
#@title
#@markdown **Run the cell** to enable the sliders.
def _lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2.5, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-1, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-1, max=0),
                  exc_rate=widgets.IntSlider(12, min=10, max=20),
                  inh_rate=widgets.IntSlider(12, min=10, max=20))
def plot_lif_neuron(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  v, spike_times = _lif_neuron_inh(int(n_steps), alpha, beta, exc_rate, inh_rate)
  plot_neuron_stats(v, spike_times)
  counts = np.histogram(np.diff(spike_times),bins=10)[0]
  pmf = counts/np.sum(counts)
  pmf = pmf[pmf>0]
  print(-np.sum(pmf*np.log2(pmf)))
```

```python
#@title Video 3: LIF + inhibition
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='Aq7JrxRkn2w', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=Aq7JrxRkn2w

#Summary

In this tutorial we gained some intuition for the mechanisms that produce the observed behavior in our real neural data. First, we built a simple neuron model with excitatory input and saw that it's behavior, measured using the ISI distribution, did not match our real neurons. We then improved our model by adding leakiness and inhibitory input. The behavior of this balanced model was much closer to the real neural data.

# Bonus

### Why do neurons spike?

A neuron stores energy in an electric field across its cell membrane, by controlling the distribution of charges (ions) on either side of the membrane. This energy is rapidly discharged to generate a spike when the field potential (or membrane potential) crosses a threshold. The membrane potential may be driven toward or away from this threshold, depending on inputs from other neurons: excitatory or inhibitory, respectively. The membrane potential tends to revert to a resting potential, for example due to the leakage of ions across the membrane, so that reaching the spiking threshold depends not only on the amount of input ever received following the last spike, but also the timing of the inputs.

The storage of energy by maintaining a field potential across an insulating membrane can be modeled by a capacitor. The leakage of charge across the membrane can be modeled by a resistor. This is the basis for the leaky integrate-and-fire neuron model.

### The LIF Model Neuron

The full equation for the LIF neuron is

\begin{align}
C_{m}\frac{dV_m}{dt} = -(V_m - V_{rest})/R_{m} + I
\end{align}

where $C_m$ is the membrane capacitance, $R_M$ is the membrane resistance, $V_{rest}$ is the resting potential, and $I$ is some input current (from other neurons, an electrode, ...).

In our above examples we set many of these parameters to convenient values ($C_m = R_m = dt = 1$, $V_{rest} = 0$) to focus more on the general behavior of the model. However, these too can be manipulated to achieve different dynamics, or to ensure the dimensions of the problem are preserved between simulation units and experimental units (e.g. with $V_m$ given in millivolts, $R_m$ in megaohms, $t$ in milliseconds).


Simulación matemática 2018

Lázaro Alonso

Email:   `alonsosilva@iteso.mx, lazarus.alon@gmail.com`

< [Mapa Logístico](Clase9_MapaLogistico.ipynb) | [Guía](Clase0_GuiaSimulacionM.ipynb) | [Oscilador amortiguado](Clase10_OsciladorAmortiguado.ipynb) >
___

#  ¿Cómo se mueve un péndulo?

> Se dice que un sistema cualquiera, mecánico, eléctrico, neumático, etc., es un oscilador armónico si, cuando se deja en libertad fuera de su posición de equilibrio, vuelve hacia ella describiendo oscilaciones sinusoidales, o sinusoidales amortiguadas en torno a dicha posición estable.
- https://es.wikipedia.org/wiki/Oscilador_armónico

Referencias:
   - http://matplotlib.org
   - https://seaborn.pydata.org
   - http://www.numpy.org
   - http://ipywidgets.readthedocs.io/en/latest/index.html

**En realidad esto es el estudio de oscilaciones. **

```python
from IPython.display import YouTubeVideo
YouTubeVideo('k5yTVHr6V14')
```

Los sistemas mas sencillos a estudiar en oscilaciones son el sistema ` masa-resorte` y el `péndulo simple`.

\begin{align}
\frac{d^2 x}{dt^2} +  \omega_{0}^2 x &= 0, \quad \omega_{0} = \sqrt{\frac{k}{m}}\notag\\
\frac{d^2 \theta}{dt^2} + \omega_{0}^{2}\, \theta &= 0, \quad\mbox{donde}\quad \omega_{0}^2 = \frac{g}{l}
\end{align}

___
## Sistema `masa-resorte`

La solución a este sistema `masa-resorte` se explica en términos de la segunda ley de Newton. Para este caso, si la masa permanece constante y solo consideramos la dirección en $x$. Entonces,
\begin{equation}
F = m \frac{d^2x}{dt^2}.
\end{equation}

¿Cuál es la fuerza? ** Ley de Hooke! **
\begin{equation}
F = -k x, \quad k > 0.
\end{equation}

Vemos que la fuerza se opone al desplazamiento y su intensidad es proporcional al mismo.  Y $k$ es la constante elástica o recuperadora del resorte.

Entonces, un modelo del sistema `masa-resorte` está descrito por la siguiente **ecuación diferencial**:

\begin{equation}
\frac{d^2x}{dt^2} + \frac{k}{m}x = 0,
\end{equation}

cuya solución se escribe como
\begin{equation}
x(t) = A \cos(\omega_{o} t) + B \sin(\omega_{o} t)
\end{equation}
Y su primera derivada (velocidad) sería
\begin{equation}
\frac{dx(t)}{dt} = \omega_{0}[- A \sin(\omega_{0} t) + B\cos(\omega_{0}t)]
\end{equation}

### **¿Cómo se ven las gráficas de $x$ vs $t$ y $\frac{dx}{dt}$ vs $t$?**

_Esta instrucción es para que las gráficas aparezcan dentro de este entorno._

```python
%matplotlib inline
```

_Esta es la librería con todas las instrucciones para realizar gráficos. _

```python
import matplotlib.pyplot as plt
```

```python
import matplotlib as mpl
label_size = 14
mpl.rcParams['xtick.labelsize'] = label_size
mpl.rcParams['ytick.labelsize'] = label_size
```

 _Y esta es la librería con todas las funciones matemáticas necesarias._

```python
import numpy as np
```

```python
# Definición de funciones a graficar
A, B, w0 = .5, .1, .5                            # Parámetros
t = np.linspace(0, 50, 100)                      # Creamos vector de tiempo de 0 a 50 con 100 puntos
x = A*np.cos(w0*t)+B*np.sin(w0*t)                # Función de posición
dx = w0*(-A*np.sin(w0*t)+B*np.cos(w0*t))         # Función de velocidad

# Gráfico
plt.figure(figsize = (7, 4))                     # Ventana de gráfica con tamaño
plt.plot(t, x, '-', lw = 1, ms = 1,
         label = '$x(t)$')                       # Explicación
plt.plot(t, dx, 'ro-', lw = 1, ms = 4,
         label = r'$\dot{x}(t)$')
plt.legend(loc='best')
plt.xlabel('$t$', fontsize = 20)                 # Etiqueta eje x
plt.show()
```

```python
# Colores, etiquetas y otros formatos

plt.figure(figsize = (7, 4))
plt.scatter(t, x, lw = 0, c = 'red',
            label = '$x(t)$')           # Gráfica con puntos
plt.plot(t, x, 'r-', lw = 1)            # Grafica normal
plt.scatter(t, dx, lw = 0, c = 'b',
            label = r'$\frac{dx}{dt}$') # Con la r, los backslash se tratan como un literal, no como un escape
plt.plot(t, dx, 'b-', lw = 1)
plt.xlabel('$t$', fontsize = 20)
plt.legend(loc = 'best')                # Leyenda con las etiquetas de las gráficas
plt.show()
```

Y si consideramos un conjunto de frecuencias de oscilación, entonces

```python
frecuencias = np.array([.1, .2 , .5, .6])   # Vector de diferentes frecuencias
plt.figure(figsize = (7, 4))                # Ventana de gráfica con tamaño

# Graficamos para cada frecuencia
for w0 in frecuencias:
    x = A*np.cos(w0*t)+B*np.sin(w0*t)
    plt.plot(t, x, 'D-')
plt.xlabel('$t$', fontsize = 16)            # Etiqueta eje x
plt.ylabel('$x(t)$', fontsize = 16)         # Etiqueta eje y
plt.title('Oscilaciones', fontsize = 16)    # Título de la gráfica
plt.show()
```

Estos colores, son el default de `matplotlib`, sin embargo existe otra librería dedicada, entre otras cosas, a la presentación de gráficos.

```python
import seaborn as sns
sns.set(style='ticks', palette='Set2')
```

```python
frecuencias = np.array([.1, .2 , .5, .6])
plt.figure(figsize = (7, 4))
for w0 in frecuencias:
    x = A*np.cos(w0*t)+B*np.sin(w0*t)
    plt.plot(t, x, 'o-',
             label = '$\omega_0 = %s$'%w0)   # Etiqueta cada gráfica con frecuencia correspondiente (conversion float a string)
plt.xlabel('$t$', fontsize = 16)
plt.ylabel('$x(t)$', fontsize = 16)
plt.title('Oscilaciones', fontsize = 16)
plt.legend(loc='center left', bbox_to_anchor=(1.05, 0.5), prop={'size': 10})
plt.show()
```

Si queremos tener manipular un poco mas las cosas, hacemos uso de lo siguiente:

```python
from ipywidgets import *
```

```python
def masa_resorte(t = 0):
    A, B, w0 = .5, .1, .5                            # Parámetros
    x = A*np.cos(w0*t)+B*np.sin(w0*t)                # Función de posición

    fig = plt.figure()
    ax = fig.add_subplot(1, 1, 1)
    ax.plot(x, [0],  'ko', ms = 10)
    ax.set_xlim(xmin = -0.6, xmax = .6)
    ax.axvline(x=0, color = 'r')
    ax.axhline(y=0, color = 'grey', lw = 1)
    fig.canvas.draw()

interact(masa_resorte, t = (0, 50,.01));
```

Failed to display Jupyter Widget of type interactive.

  If you're reading this message in the Jupyter Notebook or JupyterLab Notebook, it may mean
  that the widgets JavaScript is still loading. If this message persists, it
  likely means that the widgets JavaScript library is either not installed or
  not enabled. See the Jupyter
  Widgets Documentation for setup instructions.

  If you're reading this message in another frontend (for example, a static
  rendering on GitHub or NBViewer),
  it may mean that your frontend doesn't currently support widgets.

La opción de arriba generalmente será lenta, así que lo recomendable es usar `interact_manual`.

```python
def masa_resorte(t = 0):
    A, B, w0 = .5, .1, .5                            # Parámetros
    x = A*np.cos(w0*t)+B*np.sin(w0*t)                # Función de posición

    fig = plt.figure()
    ax = fig.add_subplot(1, 1, 1)
    ax.plot(x, [0],  'ko', ms = 10)
    ax.set_xlim(xmin = -0.6, xmax = .6)
    ax.axvline(x=0, color = 'r')
    ax.axhline(y=0, color = 'grey', lw = 1)
    fig.canvas.draw()

interact_manual(masa_resorte, t = (0, 50,.01));
```

Failed to display Jupyter Widget of type interactive.

  If you're reading this message in the Jupyter Notebook or JupyterLab Notebook, it may mean
  that the widgets JavaScript is still loading. If this message persists, it
  likely means that the widgets JavaScript library is either not installed or
  not enabled. See the Jupyter
  Widgets Documentation for setup instructions.

  If you're reading this message in another frontend (for example, a static
  rendering on GitHub or NBViewer),
  it may mean that your frontend doesn't currently support widgets.

___
## Péndulo simple
Ahora, si fijamos nuestra atención al movimiento de un péndulo simple _(oscilaciones pequeñas)_, la ecuación diferencial a resolver tiene la misma forma:

\begin{equation}
\frac{d^2 \theta}{dt^2} + \omega_{0}^{2}\, \theta = 0, \quad\mbox{donde}\quad \omega_{0}^2 = \frac{g}{l}.
\end{equation}

La diferencia más evidente es como hemos definido a $\omega_{0}$. Esto quiere decir que,

\begin{equation}
\theta(t) = A\cos(\omega_{0} t) + B\sin(\omega_{0}t)
\end{equation}

Si graficamos la ecuación de arriba vamos a encontrar un comportamiento muy similar al ya discutido anteriormente. Es por ello que ahora veremos el movimiento en el plano $xy$. Es decir,

\begin{align}
x &= l \sin(\theta), \quad
y = l \cos(\theta)
\end{align}

```python
# Podemos definir una función que nos entregue theta dados los parámetros y el tiempo
def theta_t(a, b, g, l, t):
    omega_0 = np.sqrt(g/l)
    return a * np.cos(omega_0 * t) + b * np.sin(omega_0 * t)
```

```python
# Hacemos un gráfico interactivo del péndulo
def pendulo_simple(t = 0):
    fig = plt.figure(figsize = (5,5))
    ax = fig.add_subplot(1, 1, 1)
    x = 2 * np.sin(theta_t(.4, .6, 9.8, 2, t))
    y =  - 2 * np.cos(theta_t(.4, .6, 9.8, 2, t))
    ax.plot(x, y, 'ko', ms = 10)
    ax.plot([0], [0], 'rD')
    ax.plot([0, x ], [0, y], 'k-', lw = 1)
    ax.set_xlim(xmin = -2.2, xmax = 2.2)
    ax.set_ylim(ymin = -2.2, ymax = .2)
    fig.canvas.draw()

interact_manual(pendulo_simple, t = (0, 10,.01));
```

Failed to display Jupyter Widget of type interactive.

  If you're reading this message in the Jupyter Notebook or JupyterLab Notebook, it may mean
  that the widgets JavaScript is still loading. If this message persists, it
  likely means that the widgets JavaScript library is either not installed or
  not enabled. See the Jupyter
  Widgets Documentation for setup instructions.

  If you're reading this message in another frontend (for example, a static
  rendering on GitHub or NBViewer),
  it may mean that your frontend doesn't currently support widgets.

### Condiciones iniciales

Realmente lo que se tiene que resolver es,

\begin{equation}
\theta(t) = \theta(0) \cos(\omega_{0} t) + \frac{\dot{\theta}(0)}{\omega_{0}} \sin(\omega_{0} t)
\end{equation}

> **Actividad.** Modificar el programa anterior para incorporar las condiciones iniciales.

```python
# Solución:
def theta_t(theta_0,dtheta_0,g,l,t):
    omega_0 = np.sqrt(g/l)
    return theta_0 * np.cos(omega_0 * t) + dtheta_0/omega_0 * np.sin(omega_0 * t)
```

```python
def pendulo_simple(t = 0):
    fig = plt.figure(figsize = (5,5))
    ax = fig.add_subplot(1, 1, 1)
    x = 2 * np.sin(theta_t(0.1, 0, 9.8, 2, t))
    y =  - 2 * np.cos(theta_t(0.1, 0, 9.8, 2, t))
    ax.plot(x, y, 'ko', ms = 10)
    ax.plot([0], [0], 'rD')
    ax.plot([0, x ], [0, y], 'k-', lw = 1)
    ax.set_xlim(xmin = -2.2, xmax = 2.2)
    ax.set_ylim(ymin = -2.2, ymax = .2)
    fig.canvas.draw()
interact_manual(pendulo_simple, t = (0, 10,.01));
```

Failed to display Jupyter Widget of type interactive.

  If you're reading this message in the Jupyter Notebook or JupyterLab Notebook, it may mean
  that the widgets JavaScript is still loading. If this message persists, it
  likely means that the widgets JavaScript library is either not installed or
  not enabled. See the Jupyter
  Widgets Documentation for setup instructions.

  If you're reading this message in another frontend (for example, a static
  rendering on GitHub or NBViewer),
  it may mean that your frontend doesn't currently support widgets.

### Espacio fase  $(x, \frac{dx}{dt})$

La posición y velocidad para el sistema `masa-resorte` se escriben como:

\begin{align}
x(t) &= x(0) \cos(\omega_{o} t) + \frac{\dot{x}(0)}{\omega_{0}} \sin(\omega_{o} t)\\
\dot{x}(t) &= -\omega_{0}x(0) \sin(\omega_{0} t) + \dot{x}(0)\cos(\omega_{0}t)]
\end{align}

```python
k = 3  #constante elástica [N]/[m]
m = 1 # [kg]
omega_0 = np.sqrt(k/m)
x_0 = .5
dx_0 = .1
```

```python
t = np.linspace(0, 15, 300)
```

```python
x_t = x_0 *np.cos(omega_0 *t) + (dx_0/omega_0) * np.sin(omega_0 *t)
dx_t = -omega_0 * x_0 * np.sin(omega_0 * t) + dx_0 * np.cos(omega_0 * t)
```

```python
plt.figure(figsize = (7, 4))
plt.plot(t, x_t, label = '$x(t)$', lw = 1)
#plt.plot(t, dx_t, label = '$\dot{x}(t)$', lw = 1)
plt.plot(t, dx_t/omega_0, label = '$\dot{x}(t)$', lw = 1)  # Mostrar que al escalar, la amplitud queda igual
plt.legend(loc='center left', bbox_to_anchor=(1.01, 0.5), prop={'size': 14})
plt.xlabel('$t$', fontsize = 18)
plt.show()
```

```python
plt.figure(figsize = (5, 5))
plt.plot(x_t, dx_t/omega_0, 'ro', ms = 2)
plt.xlabel('$x(t)$', fontsize = 18)
plt.ylabel('$\dot{x}(t)/\omega_0$', fontsize = 18)
plt.show()
```

```python
plt.figure(figsize = (5, 5))
plt.scatter(x_t, dx_t/omega_0, cmap = 'viridis', c = dx_t, s = 8, lw = 0)
plt.xlabel('$x(t)$', fontsize = 18)
plt.ylabel('$\dot{x}(t)/\omega_0$', fontsize = 18)
plt.show()
```

#### Multiples condiciones iniciales

```python
k = 3  #constante elástica [N]/[m]
m = 1 # [kg]
omega_0 = np.sqrt(k/m)
```

```python
t = np.linspace(0, 50, 50)
```

```python
x_0s = np.array([.7, .5, .25, .1])
dx_0s = np.array([.2, .1, .05, .01])
cmaps = np.array(['viridis', 'inferno', 'magma', 'plasma'])
```

```python
plt.figure(figsize = (6, 6))
for indx, x_0 in enumerate(x_0s):
    x_t = x_0 *np.cos(omega_0 *t) + (dx_0s[indx]/omega_0) * np.sin(omega_0 *t)
    dx_t = -omega_0 * x_0 * np.sin(omega_0 * t) + dx_0s[indx] * np.cos(omega_0 * t)
    plt.scatter(x_t,  dx_t/omega_0, cmap = cmaps[indx],
                c = dx_t, s = 10,
                lw = 0)
    plt.xlabel('$x(t)$', fontsize = 18)
    plt.ylabel('$\dot{x}(t)/\omega_0$', fontsize = 18)
    #plt.legend(loc='center left', bbox_to_anchor=(1.05, 0.5))
```

Trayectorias del oscilador armónico simple en el espacio fase $(x,\, \dot{x}\,/\omega_0)$ para diferentes valores de la energía.

> **Actividad... **
considerar un pendulo simple en diferentes planetas, y graficar su espacio fase.

< [Mapa Logístico](Clase9_MapaLogistico.ipynb) | [Guía](Clase0_GuiaSimulacionM.ipynb) | [Oscilador amortiguado](Clase10_OsciladorAmortiguado.ipynb) >
___

Created with Jupyter by Lázaro Alonso.
  Copyright:  Public Domain como en [CC](https://creativecommons.org/licenses/by/2.0/) (Exepto donde se indique lo contrario)