# Time-series Regression (Deep Learning to Detect Change Points)

> CNNs are used after converting time-series to images
- toc: true
- branch: master
- badges: false
- comments: true
- hide: false
- search_exclude: true
- metadata_key1: metadata_value1
- metadata_key2: metadata_value2
- image: images/tsr_fig4.png
- categories: [Deep Learning,Time series,Image Regression,CNN,RNN,   Python,fastai]
- show_tags: true

## Purpose

This project investigates whether it is feasible to use Convolutional Neural Networks (CNNs) to perform time-series regression, rather than more traditional methods. Before a CNN can be used, the time-series has to be converted to a spatial signal point, i.e. an image.

## Problem

The problem for our purpose is the following: Given a time-series, detect the points in time where changes occur in the value of the time-series. The formal name for this is *change point detection*.

### Change Point Detection

In Python, a promising package for the detection of change points, is the [ruptures](https://github.com/deepcharles/ruptures) package by Truong, Oudre and Vayatis (2020). This package includes a number of implemented algorithms which are covered in the associated publication called ["Selective review of offline change point detection methods"](https://www.sciencedirect.com/science/article/pii/S0165168419303494?via%3Dihub).

The authors provide the following three highlights:
>- A structured and didactic review of more than 140 articles related to offline change point detection. Thanks to the methodological framework proposed in this survey, all methods are presented as the combination of three functional blocks, which facilitates comparison between the different approaches.
- The survey provides details on mathematical as well as algorithmic aspects such as complexity, asymptotic consistency, estimation of the number of changes, calibration, etc.
- The review is linked to a Python package that includes most of the pre- sented methods, and allows the user to perform experiments and bench- marks.

... and this abstract:

> This article presents a selective survey of algorithms for the offline detection of multiple change points in multivariate time series. A general yet structuring methodological strategy is adopted to organize this vast body of work. More precisely, detection algorithms considered in this review are characterized by three elements: a cost function, a search method and a constraint on the number of changes. Each of those elements is described, reviewed and discussed separately. Implementations of the main algorithms described in this article are provided within a Python package called ruptures.

This summary comes from the GitHub site:

> **ruptures** is a Python library for off-line change point detection. This package provides methods for the analysis and segmentation of non-stationary signals. Implemented algorithms include exact and approximate detection for various parametric and non-parametric models. ruptures focuses on ease of use by providing a well-documented and consistent interface. In addition, thanks to its modular structure, different algorithms and models can be connected and extended within this package.

Here is an example of how the ruptures package might be used. The following code generates a noisy piecewise constant signal. Then it performs a penalized kernel change point detection and displays the results. Alternating colors outline the true regimes and vertical dashed lines mark the detected change points.

~~~ python
import matplotlib.pyplot as plt
import ruptures as rpt

# generate signal
n_samples, dim, sigma = 1000, 3, 4
n_bkps = 4  # number of breakpoints
signal, bkps = rpt.pw_constant(n_samples, dim, n_bkps, noise_std=sigma)

# detection
algo = rpt.Pelt(model="rbf").fit(signal)
result = algo.predict(pen=10)

# display
rpt.display(signal, bkps, result)
plt.show()
~~~

### Novel approach to Change Point Detection

In the approach of this project, we do not have access to first principles models (i.e. white-box models). Instead we rely on a series of convolution neural networks (CNNs) to learn an empirical model (i.e. black-box model).

The *predictor points* for our problem are not structured vectors as is common in the case of structured data analysis. Here we have to use a time-series or sequence of scalar-valued predictor points and have the model learn the associated *target point* which is a vector of scalars that identify the times of the change points in each case.

In the deep learning subfield, it is common to use a *Recurrent Neural Network* (RNN) for this kind of problem. Examples of this, but for the case of time-series classification, rather than regression, are Hüsken and Stagge (2003), and also Sun, Di, and Fang (2019). However, the training of an RNN can be challenging due to high demands on computing resources including processing power, processing time, and memory. There is also the vanishing/exploding gradients problem, addressed in various ways, but is often still lurking in the background.

Consider how easy it is for the human visual system to handle this problem, and in a fraction of a second. In fact, this is exactly how analysts sometimes do their change detections manually. This suggests that we might benefit from tapping into the biological mechanisms for analyzing visual data (i.e. images). Recently, some researchers started adopting this insight. See, for example, Wang and Oates (2015a, 2015b, 2015c) and Wang, Zan and Oates (2017). The essence of this line of thought is the following: Instead of analyzing a sequence of 1-D or scalar-valued *temporal* data points, we transform them into a single 2-D or matrix-valued *spatial* data point. The spatial data point is simply an image which means the time-series signal has been transformed into an image. This allows for the application of a large body of relatively well-developed computer vision techniques to the above-stated problem. Most of these techniques center around the *Convolutional Neural Network* (CNN). In summary, the *time-series regression* problem has been converted to an *image regression* problem.

We will use a simple transformation technique to transform the time-series into an image. After transformation of the synthetic training dataset of time-series, a CNN will be trained which will serve as a regressor. *Transfer learning* will be used to speed up the training of the CNNs.

## Value Proposition

This project seeks to provide value in a number of ways:
* Demonstrates how synthetic time-series can be turned into images for more effective regression
* Demonstrates how transfer learning greatly speedup the time to train a CNN neural network for the regression of time-series
* Satisfies my personal interest to think "outside the box" and apply existing technology in novel ways

## Data Source

This project makes use of synthetic data. Each time-series is generated randomly and has the following properties:

- Consists of two changes, called actions
- Starts at a time (beg_t=0) at a random level (beg_l)
- The first action starts at a later random time (actn1_t) at a random level (actn1_l=beg_l)
- The first action ends at a later random time (comp1_t) at a random level (comp1_l)
- The second action starts at a later random time (actn2_t) at a random level (actn1_l)
- The second action ends at a later random time (comp1_t) at a random level (comp2_l)
- Ends at a time (end_t=0) at a random level (comp2_l=end_l)
- Is captured in a .png file
    - file naming uses the same names as those used in the [Time Series Classification Project](https://closedloopai.github.io/portfolio/2019/09/15/TimeSeriesClassification.html)
    - this is just a convenience and no data from the other project is used

Figure 2 shows examples of randomly generated synthetic time-series.

## Modeling

In this section, we will look at the important concept of time-series regression and how it relates to two of the most important deep learning architectures: Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs).

There seem to be advantages to the use of deep learning to perform regression on time-series. One specific advantage is the ability to detect time invariant characteristics. This is similar to how spatially invariant filters detect patterns in images.

### Recurrent Neural Networks (RNNs)

Recurrent layers in RNNs are described by the equations:

$$ \Large
\begin{align}
\mathbf{a}^{<t>} &= g(\mathbf{W}_{aa} \mathbf{a}^{<t-1>} + \mathbf{W}_{ax} \mathbf{x}^{<t>} + \mathbf{b}_a) \\
\hat{\mathbf{y}}^{<t>}  &= g(\mathbf W_{ya}\mathbf{a}^{<t>} + \mathbf{b_y})
\end{align}
$$

The parameters or weights that undergo training are captured in a number of *filters* or *kernels*. The *feedback* filter is $\mathbf{W}_{aa}$, the *input* filter $\mathbf{W}_{ax}$, and the *output* filter $\mathbf{W}_{ya}$. The *signal* is the data that are used as examples during training. The symbols $\mathbf{x}^{<t>}$ and $\mathbf{\hat{y}}^{<t>}$ represent the input and output signals respectively. The hidden state, or internal signal, is given by $\mathbf{a}^{<t>}$. The filters are matrices while the signals are vector-valued. There is often a single layer in an RNN. Note, however, that this architecture is recursive. This means that each time-step could be considered a separate layer in time.

In the context of our regression problem, the input is a sequence of scalar-valued continuous predictor points. The output (target point) is a vector of scalar-valued continuous values (after being processed by a sigmoid function). The values of a target point are the two time values where changes occur in the level of the time-series. This type of RNN is also known as a *many-to-one* RNN because a series of input data points leads to a single output datapoint.

### Disadvantages of RNNs

In 2015 RNNs made a dramatic come-back (Karpathy, 2015). A year or two after this the *ResNet* (He, Zhang, Ren & Sun, 2016) and the *attention* mechanism (Xu et al., 2015) were invented. This provided an expanded context for the evaluation of RNNs and the Long Short Term Memory (LSTM). A further two years later, arguably, saw the beginning of a measure of decline of the popularity of the RNN and the LSTM in some disciplines.

Culurciell (2018) points out some shortcomings of RNNs in his article “The fall of RNN / LSTM.” In this regard, he mentions the problem of vanishing gradients and that RNNs are not hardware friendly. RNNs are also harder to train and parallelize.

### Time-series transformation to images

Before we look at Convolutional Neural Networks (CNNs), we will mention how a time-series can be transformed into an image. The purpose of this transformation is to enable computers to “visually” recognize and perform regression on the time-series signal. By doing this transformation we can take advantage of the impressive successes of deep learning architectures (using CNNs) in computer vision. This allows us to identify the structure of a time-series, leading to, hopefully, more effective regression.

To create this spatial (or image) representation of the time-series, we will use a dark pen on a light background. No graphical annotations will be included, i.e. graphic frame, tick marks, tick labels, axis labels, and heading. Annotations will make the learning process unnecessarily complex.

### Convolutional Neural Networks (CNNs)

Convolutional Neural Networks are structured in a way that enables them to exploit translational invariance. They extract features through receptive fields. Significant weight sharing drastically reduces the number of parameters involved in training them. They are the state-of-the-art architecture in the handling of computer vision tasks.

A CNN consists of different types of layers. The most important ones are *convolution*, *pooling*, and *dense* layers. Convolution and pooling layers often alternate during the initial layers. Near the end, a number of dense layers usually occurs which often ends with a sigmoid or softmax layer in the case of a classification CNN.

#### Convolution layers

Convolution layers are described by the equations:

$$ \Large
\begin{align}
\mathbf{Z}^{[l]} &= \mathbf{W}^{[l]} \ast \mathbf{A}^{[l-1]} + \mathbf{b}^{[l]} \qquad(1) \\
\mathbf{A}^{[l]} &= g^{[l]}(\mathbf{Z}^{[l]}) \qquad\qquad\qquad\qquad\,(2)
\end{align}
$$

The *filter* (also called *kernel*) of a convolution layer is indicated by $\mathbf{W}^{[l]}$ where $l$ is the index of the layer. $\mathbf{W}^{[l]}$ is tensor-valued with each element $w_{ijk}^{[l]} \in \mathbb{R}$. The values of $w_{ijk}^{[l]}$ are learned by the training process. The *dimensions* (or *shape*) of $\mathbf{W}^{[l]}$ are $n_C^{[l-1]} \times f^{[l]} \times f^{[l]}$ where $n_C^{[l-1]}$ is the *number of filters* (also the *number of channels*) in the previous layer. The filter size is indicated by $f^{[l]}$. If we have multiple filters in layer $l$, the dimensions of $\mathbf{W}^{[l]}$ expand to $n_C^{[l]} \times n_C^{[l-1]} \times f^{[l]} \times f^{[l]}$ making $\mathbf{W}^{[l]}$ a vector of tensors.

The *activations* in layer $l$ are represented by another tensor $\mathbf{A}^{[l]}$. For each layer, we distinguish between the *input* activations, $\mathbf{A}^{[l-1]}$, and *output* activations, $\mathbf{A}^{[l]}$. The dimensions of $\mathbf{A}^{[l-1]}$ are $n_C^{[l-1]} \times n_H^{[l-1]} \times n_W^{[l-1]}$ where $n_C^{[l-1]}$ is the number of channels in the previous layer, $n_H^{[l-1]}$ the *height* of the image in the previous layer, and $n_W^{[l-1]}$ the *width* of the image in the previous layer.

The dimensions of $\mathbf{A}^{[l]}$ are $n_C^{[l]} \times n_H^{[l]} \times n_W^{[l]}$  where

$$ \Large
n_H^{[l]}=\Bigg \lfloor \frac{n_H^{[l-1]}+2p^{[l]}-f^{[l]}}{s^{[l]}} + 1 \Bigg \rfloor\qquad\qquad\qquad\qquad(3)
$$

$$ \Large
n_W^{[l]}=\Bigg \lfloor \frac{n_W^{[l-1]}+2p^{[l]}-f^{[l]}}{s^{[l]}} + 1 \Bigg \rfloor\qquad\qquad\qquad\qquad(4)
$$

The *padding* size in layer $l$ is indicated by $p^{[l]}$. The *stride* is represented by $s^{[l]}$. If we make use of mini-batch training, the dimensions of $\mathbf{A}^{[l]}$ will expand to $n_C^{[l]} \times n_H^{[l]} \times n_W^{[l]}$ where $m$ is the mini-batch size. In this case $\mathbf{A}^{[l]}$ becomes a vector of tensors.

The *bias* vector in layer $l$ is indicated by $\mathbf{b}^{[l]}$. The *convolution operation* is indicated by the symbol $\ast$. Equation 1 describes the *linear* part of the convolution. The *activation* part is captured by Equation 2. The *activation function*, $g^{[l]}$, is often a rectified linear unit (ReLU).

Equations 1 and 2 can be combined into a single equation:

$$ \Large
\mathbf{A}^{[l]} = g^{[l]}(\mathbf{W}^{[l]} \ast \mathbf{A}^{[l-1]} + \mathbf{b}^{[l]}) \qquad\qquad\qquad\qquad\,(5)
$$

[Piotr  Skalski](https://towardsdatascience.com/gentle-dive-into-math-behind-convolutional-neural-networks-79a07dd44cf9)
 (2019) has a blog that makes the operation of CNNs easy to understand. It includes a number of animations that provide valuable insight.

#### Pooling layers

The filter of a pooling layer has no weights that need to be trained. It has a filter size $f^{[l]}$, and a stride $s^{[l]}$. The value for padding is almost always zero, $p^{[l]}=0$. The filter performs an aggregation operation as it slides over the activation signal. This operation is performed on each of the input channels independently. Types of aggregation operations are *maximum* and *average*. The most common type is the *max-pool* layer. As the $f \times f$ filter slides across the image, it picks the maximum activation for each position and sends that to the output image, thereby reducing the size of the input image according to equations (3) and (4). There is also no activation function. This means pooling layers are described by the following equation:

$$ \Large
\mathbf{A}^{[l]}=\text{max}(\mathbf{A}^{[l-1]}) \qquad\qquad\qquad(6)
$$

#### Dense layers

*Dense layers* always form the last few layers of a CNN that performs classification. These layers are also called *fully connected* layers. Dense layers are described by Equations 7 and 8:

$$ \Large
\begin{align}
\mathbf{z}^{[l]} &= \mathbf{W}^{[l]}\mathbf{a}^{[l-1]}+\mathbf{b}^{[l]}\qquad\qquad (7)\\
\mathbf{a}^{[l]} &= g^{[l]}(\mathbf{z}^{[l]})\qquad\qquad\qquad\qquad\quad(8)
\end{align}
$$

Notice the absence of the convolution operator, which have been replaced by matrix multiplication.

The *filter* of a dense layer is indicated by $\mathbf{W}^{[l]}$ where $l$ is the index of the layer. $\mathbf{W}^{[l]}$ is matrix-valued with each element $w_{ij}^{[l]}\in\mathbb{R}$. The values of $w_{ij}^{[l]}$ are learned by the training process. The *dimensions* of $\mathbf{W}^{[l]}$ are $n^{[l]} \times n^{[l-1]}$ where $n^{[l-1]}$ is the *number of input features* (also the number of output features in the previous layer), and $n^{[l]}$ is the *number of output features*.

The *activations* in layer $l$ are represented by a vector $\mathbf{a}^{[l]}$. For each layer, we distinguish between the *input* activations, $\mathbf{a}^{[l-1]}$, and *output* activations, $\mathbf{a}^{[l]}$. The dimension of $\mathbf{a}^{[l-1]}$ is $n^{[l-1]}$ where $n^{[l-1]}$ is the number of neurons or hidden units in the previous layer. The dimension of $\mathbf{a}^{[l]}$ is $n^{[l]}$ where $n^{[l]}$ is the number of units in the current layer.

There is no concept of padding nor of stride. The *bias* vector in layer $l$ is indicated by $\mathbf{b}^{[l]}$. Equation 7 describes the *linear* part of the filter. The *activation* part is captured by Equation 8. The *activation function*, $g^{[l]}$, is often a ReLU.

As before, we can combine Equations 7 and 8 into a single equation (see Equation 9).

$$ \Large
\mathbf{a}^{[l]} = g^{[l]}(\mathbf{W}^{[l]} \mathbf{a}^{[l-1]} + \mathbf{b}^{[l]}) \qquad\qquad\qquad\qquad\,(9)
$$

If we make use of mini-batch training, the dimensions of $\mathbf{a}^{[l]}$ will expand to $n^{[l]} \times m$ where $m$ is the mini-batch size. In this case $\mathbf{a}^{[l]}$ becomes a matrix $\mathbf{A}^{[l]}$ so that we have

$$ \Large
\mathbf{A}^{[l]} = g^{[l]}(\mathbf{W}^{[l]} \mathbf{A}^{[l-1]} + \mathbf{b}^{[l]}) \qquad\qquad\qquad\qquad(10)
$$

#### Residual Networks

He et al. (2016) provides an impressive and state-of-the-art architecture to improve the performance of CNNs even more. This architecture is called a *residual network*, or a *ResNet*. [Sik-Ho Tsang](https://towardsdatascience.com/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8) (2018) has a blog that presents some of the details in a more digestible form.

A ResNet layer or block is developed as follows. As before, for a convolution layer, we have:

$$ \Large
\begin{align}
\mathbf{Z}^{[l]} &= \mathbf{W}^{[l]} \ast \mathbf{A}^{[l-1]} + \mathbf{b}^{[l]} \\
\mathbf{A}^{[l]} &= g^{[l]}(\mathbf{Z}^{[l]})
\end{align}
$$

Let’s add 1 to the index values for the purpose of deriving:

$$ \Large
\begin{align}
\mathbf{Z}^{[l+1]} &= \mathbf{W}^{[l+1]} \ast \mathbf{A}^{[l]} + \mathbf{b}^{[l+1]} \qquad(11) \\
\mathbf{A}^{[l+1]} &= g^{[l+1]}(\mathbf{Z}^{[l+1]}) \qquad\qquad\qquad\qquad\,(12)
\end{align}
$$

The next layer will then be:

$$ \Large
\begin{align}
\mathbf{Z}^{[l+2]} &= \mathbf{W}^{[l+2]} \ast \mathbf{A}^{[l+1]} + \mathbf{b}^{[l+2]} \qquad(13) \\
\mathbf{A}^{[l+2]} &= g^{[l+2]}(\mathbf{Z}^{[l+2]}) \qquad\qquad\qquad\qquad\quad(14)
\end{align}
$$

Now comes the crucial step. We feed the activations $\mathbf{A}^{[l]}$ in Equation 11 *forward* by means of a *skip-connection* (also called a *short-circuit-connection*) and add them to $\mathbf{Z}^{[l+2]}$ in Equation 13. This means Equation 14 now becomes:

$$ \Large
\mathbf{A}^{[l+2]} = g^{[l+2]}(\mathbf{Z}^{[l+2]} + \mathbf{A}^{[l]})
$$

The ResNet block is therefore described by the equations:

$$ \Large
\begin{align}
\mathbf{Z}^{[l+2]} &= \mathbf{W}^{[l+2]} \ast \mathbf{A}^{[l+1]} + \mathbf{b}^{[l+2]} \qquad(15) \\
\mathbf{A}^{[l+2]} &= g^{[l+2]}(\mathbf{Z}^{[l+2]} + \mathbf{A}^{[l]}) \qquad\qquad\quad(16)
\end{align}
$$

Expressed as a single equation we have:

$$ \Large
\mathbf{A}^{[l+2]} = g^{[l+2]}(\mathbf{W}^{[l+2]} \ast \mathbf{A}^{[l+1]} + \mathbf{b}^{[l+2]} + \mathbf{A}^{[l]})\quad(17)
$$

Adjusting the indexes again, we have

$$ \Large
\mathbf{A}^{[l]} = g^{[l]}(\mathbf{W}^{[l]} \ast \mathbf{A}^{[l-1]} + \mathbf{b}^{[l]} + \mathbf{A}^{[l-2]})\qquad(18)
$$

The ResNet architecture allows for much deeper networks than before by stacking ResNet blocks as deep as needed. In theory, as the number of layers in a traditional deep neural network increases, the training error should keep on decreasing. The reality is that when the network gets too deep, the training error actually starts to increase again. ResNets rescue this situation, allowing the training error to keep on falling even with the number of layers approaching one thousand.

Next we will discuss all the classification models used in this paper. All of them make use of a 50-layer ResNet architecture.

## Regression Model

The input datapoints for this model are all the synthetic time-series images. In the software, this model is called *mod9*. The purpose of the model is to predict the two time points at which changes occur.

The labeled training data appears in the file synthetic.csv:

* dashlink_regression
    * Tail_687_1
        * png9
            * synthetic.csv

Figure 3 shows the first few records of synthetic.csv.

The code for the model is present in Python notebook:

[30_mod9_canon.ipynb](https://nbviewer.jupyter.org/github/kobus78/dashlink_regression/blob/master/30_mod9_canon.ipynb)

The model’s notebook performs the following steps:

* Ingest data
    * Form item list
    * Form train and validation item lists
        * Train list is 80% of data
        * Validation list is 20% of data
    * Form label lists
        * The labels come from synthetic.csv
    * Normalize data using the ImageNet statistics (for transfer learning)
* Train model
    * Create a learner with the data and a ResNet-18 architecture
    * Plot the Loss vs the Learning Rate that allows selection of the frozen learning rate by inspection
    * Fit the learner’s model to the data using the frozen learning rate
    * Plot the Train and Validation Loss vs the number of Batches
    * Save the frozen model and iterate if necessary
    * Unfreeze the model
    * Plot the Loss vs the Learning Rate that allows selection of the unfrozen learning rate by inspection
    * Fit the learner’s model to the data using the unfrozen learning rate and 10% of the frozen learning rate
    * Plot the Train and Validation Loss vs the number of Batches
    * Save the unfrozen model and iterate if necessary
* Test Inference/Production (on train data)
    * Export the trained, unfrozen model
    * Loop through all the data and submit each image to the trained model to predict the locations of the changes

Figure 4 shows an example of predicted location of changes (green dotted lines).

This brings us to the end of the modeling section. We looked at the concept of time-series regression and two deep learning architectures that can be used for this.

## Inference

We have finally come to the application of the models developed above. During inference (also called testing in this paper) the trained models are presented with previously unseen data points. This is the situation in production when the developed models are called upon to provide value for users.

## Test Dataset

The dataset for testing/inference will be prepared in the same way as the dataset for training. Note that the datapoints are not labeled for inference. It is a simple matter to generate additional random data and submit it to the trained model and assess the performance. This work is still outstanding.

## Conclusions & Recommendations

We have demonstrated how randomly generated time-series can be turned into images for a novel way of performing time-series regression. Using transfer learning, we showed how quickly a deep learning model could be trained.

We recommend that interested analysts take this work as a starting point and adapt it to suit their needs. This may even involve changing the technology stack, for example, making use of other deep learning libraries and a different programming language. Here we used the fastai Python library (built on top of PyTorch) and the Python language. There are a number of other useful technology environments, e.g. Java, TensorFlow, Julia, and MATLAB.

We suggest that analysts need not shy away from the use of deep learning for analyses that conventionally make use of traditional approaches. The use of transfer learning makes the training of deep learning models very tractable. In our case, transfer learning was based on the ImageNet model which was trained on over 14 million images to classify them into more than 20,000 categories. There are many cloud providers offering the use of GPUs (Graphical Processing Units), ideal for the training process. GPUs are not necessary for inference. Even without access to a GPU, the training process is still tractable on an ordinary laptop. This is the beauty of transfer learning.

## Further Experimentation

There are a good number of hyper-parameters that may be adjusted leading to further experiments, for example:
* Fraction of train data dedicated for validation (20% here)
* The batch size during training
* A technique that holds promise is to first down-sample images drastically, say to 32 x 32. Then, after training, transfer learning is used while progressively up-sampling again.
* Learning rates. This is arguably the most influential hyper-parameter during training. It may be worthwhile to adjust the used learning rates, (both for the frozen learning rate, *lrf*, as well as the unfrozen learning rate, *lru*.

We used data that always contains exactly two changes. Future experiments can make the occurrance of change points more flexible.

Our synthetic data do not include noise. Future experiments can investigate the performance of the model in the presence of noise.

The CNN architecture used was ResNet-18. We believe ResNet is the current state-of-the-art but there are other promising architectures, i.e. the Inception network.

## REFERENCES

Culurciell, E. (2018). The fall of RNN / LSTM. [Weblog]. Retrieved from
https://towardsdatascience.com/the-fall-of-rnn-lstm-2d1594c74ce0

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 770-778. doi: 10.1109/CVPR.2016.90

Hüsken, M., & Stagge, P. (2003). Recurrent neural networks for time series classification. Neurocomputing, 50, 223-235. Retrieved from https://www.sciencedirect.com/science/article/pii/S0925231201007068?via=ihub

Karpathy, A. (2015). The Unreasonable Effectiveness of Recurrent Neural Networks. [Weblog]. Retrieved from http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Sun, Z., Di, L. & Fang, H. (2019). Using long short-term memory recurrent neural network in land cover classification on Landsat and Cropland data layer time series. International Journal of Remote Sensing, 40(2), 593-614. DOI: 10.1080/01431161.2018.1516313. Retrieved from https://www.tandfonline.com/doi/abs/10.1080/01431161.2018.1516313

Truong, C., Oudre, L., & Vayatis, N. (2020). Selective review of offline change point detection methods. Signal Processing, 167:107299, 2020.

Wang, Z., Yan, W., & Oates, T. (2017). Time series classification from scratch with deep neural networks: A strong baseline. 2017 International Joint Conference on Neural Networks (IJCNN), 1578-1585.

Wang, Z., & Oates, T. (2015a). Encoding Time Series as Images for Visual Inspection and Classification Using Tiled Convolutional Neural Networks. Trajectory-Based Behavior Analytics: Papers from the 2015 AAAI Workshop. Retrieved from https://aaai.org/ocs/index.php/WS/AAAIW15/paper/viewFile/10179/10251

Wang, Z., & Oates, T. (2015b). Imaging time-series to improve classification and imputation. International Conference on Artificial Intelligence, pp 3939-3945.

Wang, Z., & Oates, T. (2015c). Spatially Encoding Temporal Correlations to Classify Temporal Data Using Convolutional Neural Networks. ArXiv, abs/1509.07481.

Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., … Bengio, Y. (2015). Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. Proceedings of the 32nd International Conference on Machine Learning, in PMLR, 37, 2048-2057.


Universidade Federal do Rio Grande do Sul (UFRGS)
Programa de Pós-Graduação em Engenharia Civil (PPGEC)

# Pré-Introdução à teoria das vibrações

## Aula 4 - equilíbrio dinâmico para vibração livre não amortecida

### *Daniel Barbosa Mapurunga Matos (Aluno PPGEC/UFRGS)*

```python
import numpy as np
import matplotlib.pyplot as plt
```

## 1. Frequência natural de vibração
A frequência natural de vibração pode ser definida a partir do teorema da conservacao da energia.

\begin{align}
\frac{KA^2}{2} = \frac{mv^2}{2}
\end{align}

Com base na cinemática angular, a velocidade pode ser escrita em função da sua frequência natural e a amplitude do movimento $v= \omega A$, sendo assim:
\begin{align}
\frac{KA^2}{2} = \frac{m(\omega_n A)^2}{2}
\end{align}

Simplificando a equação, obtém-se a seguinte expressão para o cálculo da frequência natural:

\begin{align}
\omega_n = \sqrt{\frac{K}{m}}
\end{align}

Percebe-se que, num sistema linear elástico, a frequência natural de vibraçâo depende apenas de **caracteristicas do sistema (K e m)**

## 2. Funções harmônicas

Uma função harmônica é aquela  em que a sua derivada segunda é proporcional a função inicial. Por exemplo, as funções seno e cosseno, como mostrado abaixo.

\begin{align}
Y &= A\sin{ \omega t}\\
Y' &= \omega A\cos{ \omega t}\\
Y'' &= -\omega^2A\sin{ \omega t} = -\omega^2 Y
\end{align}

## 3. Equilibrio dinâmico

Numa vibração livre não amortecida, há duas forças que atuam no corpo: **inercial** e **restitutiva**, como ilustra a figura abaixo:

Pode-se, portanto escrever a equacao de equilibrio da seguinte forma:

\begin{align}
m\ddot{x} + Kx = 0
\end{align}

Dividindo ambos os termos pela massa e lembrando que $\frac{K}{m} = \omega^2$, reescreve-se a expressão:

\begin{align*}
\ddot{x} + \omega_n^2x &= 0 \\
\ddot{x} &=- \omega_n^2x
\end{align*}

Portanto, é notável que se trata de uma função harmônica, sendo possível representar x como uma função trigonométrica:

\begin{align}
x(t) = a\cos{\omega t} + b\sin{\omega t}
\end{align}

Para definir o valor das constantes a e b, utiliza-se os valores de deslocamento inicial $x_0$ e velocidade inicial $v_0$.

\begin{align}
x_0 &= a\cos{\omega_n 0} + b\sin{\omega_n 0}\\
a &= x_0\\
v_0 &= -\omega_n a\sin{\omega_n 0} + \omega_n b\cos{\omega_n 0}\\
b &= \frac{v_0}{\omega_n}
\end{align}

Sendo assim, a equação final é dada por:

\begin{align}
x(t) = x_0\cos{\omega_n t} + \frac{v_0}{\omega_n}\sin{\omega_n t}
\end{align}

## Exemplo

```python
x0 = 2                                  # deslocamento inicial [m]
v0 = 0                                  # velocidade inicial[m/s]
m  = 90                                 # massa[kg]
K  = 150                                # rigidez [N/m]
w = np.sqrt(K/m)                        # frequencia natural
t = np.linspace(0,100,1000)             # tempo discretizado
x = x0*np.cos(w*t)+v0/w*np.sin(w*t)     # equacao do movimento

plt.figure(1,figsize=(12,4))
plt.plot(t,x,'black')
plt.xlabel("tempo"); plt.ylabel("deslocamento")
plt.grid(True)
```

## 4. Resolução da equação de equilíbrio a partir da transformada de Laplace

A transformada de Laplace permite a resolução da equação de equiíbrio dinâmico de forma simples, sem que seja necessário trabalhar com equações diferenciais. Nesta aula, não entraremos em detalhes sobre a transformada, sendo necessário apenas apresentar algumas das transformadas necessárias.

Portanto, para a resolução da equação de vibração livre não amortecida, serão necessárias as seguintes relações das funções trigonométricas:

\begin{align}
\mathscr{L} \left\{ \sin (\omega t) \right\} &= \frac{\omega}{s^2 + \omega^2} \\
\mathscr{L} \left\{ \cos (\omega t) \right\} &=  \frac{s}{s^2 + \omega^2}
\end{align}
E as transformadas das derivadas primeira e segunda:
\begin{align}
\mathscr{L} \left\{ \dot{f}(t)  \right\}     &= -f(0) + s \bar{f}(s)\\
\mathscr{L} \left\{ \ddot{f}(t) \right\}     &= -s f(0) - \dot{f}(0) + s^2 \bar{f}(s)
\end{align}

Agora, vamos resolver a equação de equilíbrio dinâmico a partir das transformadas.

\begin{align}
m\ddot{x} + Kx = 0
\end{align}

Dividindo ambos os termos pela massa, obtém-se:

\begin{align}
\ddot{x} + \omega_n^2x &= 0 \\
\end{align}

Aplicando a transformada de Laplace, frisando que a transformada é um operador linear, ficamos com a seguinte relação:

\begin{align}
\mathscr{L} \left\{ \ddot{x} \right\} + \omega_n^2\mathscr{L}\left\{x\right\} &= 0 \\
\end{align}

Aplicando a transformada da derivada segunda:

\begin{align}
-s u(0) - \dot{x}(0) + s^2 \bar{x}(s) + \omega_n^2 \bar{x}(s) = 0
\end{align}

Definindo $ x(0) = x_0$ e $ \dot{x}(0) = v_0$, a equação é reescrita da seguinte forma:

\begin{align}
-s x_0 - v_0 + s^2 \bar{x}(s) + \omega_n^2 \bar{x}(s) = 0
\end{align}

Isolando agora o termo $\bar{x}(s)$:

\begin{align}
\bar{x}(s) = x_0\frac{s}{s^2 +\omega_n^2} + \frac{v_0}{\omega_n}\frac{\omega_n}{s^2+\omega_n^2}
\end{align}

Voltando agora ao domínio do tempo, utilizando as transformadas do seno e do cosseno, chegamos a equação da vibração livre não-amortecida:

\begin{align}
x(t) = x_0\cos{\omega_n t} + \frac{v_0}{\omega_n}\sin{\omega_n t}
\end{align}

```python

```


# Demo 1: Flow

Flow is a fundamental driver of processes in the surface and subsurface. In this set of demonstrations we will introduce

  * Richards Model for variably saturated subsurface flow
  * Diffusion Wave model for overland flow (surface flow)
  * A fully coupled (integrated hydrology) model

For each model we will provide the mathematical equations, highlight information about the discretization and provide tips on setting boundary conditions, initial conditions, source terms and key parameters.

## Subsurface Flow (Richards Equation)

Richards (1931) model for variably saturated flow:

$\newcommand{porosity}{\phi}$
$\newcommand{dens}{n_l}$
$\newcommand{sat}{s}$
$\newcommand{flux}{\mathbf{q}}$
$\newcommand{krel}{k_r}$
$\newcommand{kabs}{\kappa}$
$\newcommand{viscosity}{\mu}$

$$
\begin{align}
  \frac{\partial}{\partial t} (\porosity \dens \sat) &+ \nabla \cdot \flux = 0 \\
  \flux &= - \dens \frac{\krel \kabs}{\viscosity} (\nabla p + \rho \mathbf{g})
\end{align}
$$

### Model Setup: Infiltration into a soil column (1D)
These tests demonstrate solution of Richards equation on a single column. A single column of cells, oriented vertically.

#### Initial conditions

- hydrostatic profile with the water table at 3m.

#### Boundary condition

- Infilitration is turned on and water fills the domain. Inward flux is set to 0.05 [mol s].

```python

```

### Simulation Results
After running the simulation we can plot the results using some of the python utilities provided by ATS. In this case we will plot the saturation as a function of depth, at several times.

```python
%matplotlib inline
```

```python
import sys,os
sys.path.append(os.path.join(os.environ['ATS_SRC_DIR'],'tools', 'utils'))
import ats_xdmf
import plot_column_data
import numpy as np
from matplotlib import pyplot as plt
```

```python
def plot(dirname, title):
    """A generic plotting function that plots saturation as a function of depth."""
    fig = plt.figure()

    vis = ats_xdmf.VisFile(dirname, time_unit='d')
    vis.loadMesh(columnar=True)

    # plot the solution
    ax = fig.subplots(1,1)
    plot_column_data.plot_subsurface(vis, ['saturation_liquid',], ax, None, cmap='jet')
    ax.set_title(title)
    ax.set_xlabel('saturation [-]')
    ax.set_ylabel('z [m]')
```

```python
plot("./richards/run_results/", "infiltration example")
```

```python
import numpy as np
from matplotlib import pyplot as plt

flux_data = np.loadtxt("./richards/run_results/observation_flux.dat",skiprows=11)
flux_data = np.transpose(flux_data)
fig = plt.figure()
ax = fig.subplots(1,1)
plt.plot(flux_data[0]/86400, flux_data[1])
ax.set_xlabel("days")
ax.set_ylabel(r"$mol\cdot m^{-2}\cdot s^{-1}$")

```

## Overland Flow (Diffusion Wave Equation)

Diffusion wave model for overland flow

$\newcommand{pondeddepth}{h_w}$
$\newcommand{watersource}{q_w}$
$\newcommand{flux}{{\bf U_w}}$
$\newcommand{manning}{n_{man}}$

$$
\begin{align}
  \frac{\partial \pondeddepth}{\partial t}  &+ \nabla \cdot \pondeddepth \flux = \watersource \\
  \flux &= -\displaystyle\frac{\pondeddepth^{2/3}}{\manning\left(||\nabla Z_s|| + \varepsilon\right)^{1/2}}
  \nabla\left(\nabla Z_s + \pondeddepth\right)
\end{align}
$$

$\pondeddepth$ - ponded depth
$\watersource$ - water source
$Z_s$ - surface elevation
$\manning$ - Manning’s coefficient

### Model Setup: Rainfall on a Hillslope

This demonstration simulates a rainfall event over a hillslope domain with different boundary conditions.

#### Boundary conditions:

A complete description of supported boundary conditions is provided in the documentation of the input specification online, e.g., for flow BCs see [ATS input specification v1.2 - flow-specific-boundary-conditions](https://github.com/amanzi/ats/blob/master/docs/documentation/source/input_spec/ATSNativeSpec_1_2.rst#flow-specific-boundary-conditions).  Here we focus on four important boundary conditions for surface flow.

**No-Flow**:

The default boundary condition is no-flow, and is set with an empty list,

```xml
<ParameterList name="boundary conditions" type="ParameterList">
</ParameterList>
```

**Critical Depth**:

This is an _outlet_ boundary condition which looks to set an outward flux to take away runoff. This condition is given by the following condition on outward flux

$$q = \sqrt{g\hat z} n_{w} h^{3/2}$$

It is described in the input spec as

```xml
<ParameterList name="boundary conditions" type="ParameterList">
  <ParameterList name="critical depth" type="ParameterList">
    <ParameterList name="outlet" type="ParameterList">
      <Parameter name="regions" type="Array(string)" value="{right side}" />
    </ParameterList>
  </ParameterList>
</ParameterList>
```

**Zero Head Gradient:**

This is an _outlet_ boundary condition which looks to enforce the condition that

$$ \nabla h \cdot n = 0 $$

but should only be used on boundaries where

$$ \nabla Z \cdot n  < 0 $$

This is expressed in the input file as

```xml
<ParameterList name="boundary conditions" type="ParameterList">
  <ParameterList name="zero gradient" type="ParameterList">
    <ParameterList name="outlet" type="ParameterList">
      <Parameter name="regions" type="Array(string)" value="{right side}" />
    </ParameterList>
  </ParameterList>
</ParameterList>
```

**Max Head or Surface Seepage:**

This allows water to flow out once the ponded depth exceeds the specified head, but does not allow flow back into the domain:

if ${\bf U}_w\cdot n < 0$ then we set ${\bf U}_w\cdot n = 0$

if $ h > h_{max}$ then we set $h = h_{max}$

And is written in the input file as

```xml
<ParameterList name="boundary conditions" type="ParameterList">
  <ParameterList name="seepage face head" type="ParameterList">
    <ParameterList name="outlet" type="ParameterList">
      <Parameter name="regions" type="Array(string)" value="{right side}" />
      <ParameterList name="boundary head" type="ParameterList">
        <ParameterList name="function-constant" type="ParameterList">
          <Parameter name="value" type="double" value="0.025" />
        </ParameterList>
      </ParameterList>
    </ParameterList>
  </ParameterList>
</ParameterList>
```

#### Simulation Results

```python
%matplotlib inline
import sys,os
sys.path.append(os.path.join(os.environ['ATS_SRC_DIR'],'tools', 'utils'))
import ats_xdmf
import colors
import numpy as np
from matplotlib import pyplot as plt
import pandas
```

```python
def plot_surface(directory):
    """A generic plotting script plotting ponded depth and discharge/runoff"""
    vis = ats_xdmf.VisFile(directory, domain='surface', time_unit='s')
    vis.loadMesh(order=['x','y'])

    cm = colors.cm_mapper(vis.times[0], vis.times[-1], cmap='jet')
    x = vis.centroids[:,0]

    fig, ax = plt.subplots(1,2, figsize=(14,3))

    # This simply gets a colorbar axis
    C = [[0,0],[0,0]]
    C3 = ax[0].contourf(C, vis.times, cmap='jet')
    ax[0].cla()

    pd = vis.getArray('surface-ponded_depth')
    elev = vis.get('surface-elevation', '0')

    #ax[0].plot(x, elev, 'k')
    for i,time in enumerate(vis.times):
        #ax[0].plot(x, elev + pd[i,:], color=cm(i))
        #print(cm(i))
        ax[0].plot(x, pd[i,:], color=cm(time))

    ax[0].set_xlabel('x-coordinate [m]')
    ax[0].set_ylabel('ponded depth [m]')

    hydro = np.loadtxt(os.path.join(directory, 'surface_outlet_flux.dat'), skiprows=11)
    hydro = np.transpose(hydro)
    ax[1].plot(hydro[0][:]/60.0, hydro[1][:], 'b')
    ax[1].set_xlabel('time [min]')
    ax[1].set_ylabel('discharge flow rate [mol/s]')

    cbar = plt.colorbar(C3, ax=ax[0])
    cbar.ax.set_ylabel("time [s]")

    plt.tight_layout()
    plt.show()
```

```python
plot_surface("overland_flow/run_no_flow_results")
```

```python
plot_surface("overland_flow/run_critical_depth_results")
```

```python
plot_surface("overland_flow/run_zero_gradient_results")
```

```python
plot_surface("overland_flow/run_max_head_results")
```

### Model Setup: Rainfall on a Small Watershed

It's much more fun to run simulations with real data.  One popular well studied small watershed is the Borden site in Ontario, Canada, which was the subject of early field studies.  It was also used in recent model intercomparions study, and so it is a relevant example for you to play with.

From the model intercomparison work we have the 0.5[m] and 1[m] DEMs, and we have used these with TINerator to make a variety of meshes.  Here we'll use the following coarse mesh to ensure simulation times are short:

The jupyter notebook we used to generate this mesh is included in the repository for you to use to generate finer and higher quality meshes.

#### Initial and Boundary Conditions

**Initial Condition**

* Zero ponded depth (dry soil, $h=0$).

**Boundary Conditions**

* Critical depth on the stream outlet
* Default no flow conditions elsewhere.

**Simulation Scenario**

* Heavy Rain for 18 hours
  * ParameterList for _cycle driver_ sets the start and stop times of the simulation
  * ParameterList for _surface-water_source_ prescribes the rainfall

#### Simulation Results

We plotted the results using VisIt, with custom colormaps, so you could see what is possible.  We acknowledge at least an hour is needed to get you started with VisIt, but if you're interested ask about a quick demo during the collaboration time.

The heavy rainfall fills the channel fairly quickly (left image is t=1[h]), the rain continues for 18 hours (middle figure) and the depth of the ponded water increases some.  However, there is flow out at the stream outlet, so the increase in the ponded depth is limited.  Finally, when the rain stops at t=18[h], the stream drains quickly, with lighter colors (lower ponded depth) visible at 19 hours.

This coarse mesh has some issues as well. Specifically, the channel is quite rough, with water appearing to be trapped and unable to drain.  Yes, this could be physically correct in some parts of the domain, but a finer mesh (and some preprocessing of the DEM, Ethan and Pin will cover this for Watershed Workflow), would help separate real from anomolous ponding and could improve the surface flow, particularly along the stream bed.

## Integrated Hydrology

### Model Setup: 2D Transect of Open-Book Watershed

Consider a two-dimensional transect in (x,z) of an open-book watershed (V-watershed), with each cell 1[m] in y. So the cells are three-dimensional volumes, and the surface faces are two-dimensional squares (1[m]x1[m]).

Describe how facesets are set geometrically or by labelled sets.

#### Initial and Boundary Conditions

**Initial Condition**

* Subsurface Flow
  * Water table set to 2[m] (bottom of the mesh is 0[m]).
  * Hydrostatic pressure based on this water table.
* Surface Flow
  * Set based on continuity of pressure and normal flux.

**Boundary Conditions**

* Subsurface Flow:
  * no flow boundary conditions on west, east, north, south, and bottom.
  * no boundary condition on top surface - continuity conditions are used in coupling
* Overland Flow (Surface):
  * no flow on all sides
  * usually would have an outflow condition at the stream outlet (next example).

**Simulation Scenario**

* Heavy Rain for 10 days
  * ParameterList for _cycle driver_ sets the start and stop times of the simulation
  * ParameterList for _surface-water_source_ prescribes the rainfall

#### Simulation Results

```python
# Plot the saturation at a few times

import colors
from matplotlib import colorbar

vis = ats_xdmf.VisFile(directory="integrated_hydro/run_2D_results",
                       filename="ih_subsurface_data.h5",
                       mesh_filename="ih_subsurface_mesh.h5",
                       time_unit='d')
#select output every 2 days
vis.filterIndices([0,2,4,6,8,10])
vis.loadMeshPolygons()

vis_surf = ats_xdmf.VisFile(directory="integrated_hydro/run_2D_results",
                            domain="surface",
                            filename="ih_surface_data.h5" ,
                            mesh_filename="ih_surface_mesh.h5",
                            time_unit='d')
#select same output as subsurface
vis_surf.filterIndices([0,2,4,6,8,10])
vis_surf.loadMesh(order=['x','z'])

fig, axs = plt.subplots(2,3,figsize=(12,8))

def plot(i, ax):

    sat = vis.get("saturation_liquid", vis.cycles[i])
    poly = vis.getMeshPolygons(cmap='jet_r', linewidth=1)
    poly.set_array(sat)
    poly.set_clim(0.4,1)
    ax.add_collection(poly)

    elev = vis_surf.get('surface-elevation', vis.cycles[i])
    pd = vis_surf.get('surface-ponded_depth', vis.cycles[i])

    ax.plot(vis_surf.centroids[:,0], elev+pd, 'b', linewidth=2)
    ax.set_title(f"{2*i} days")

plot(0, axs[0,0])
plot(1, axs[0,1])
plot(2, axs[0,2])
plot(3, axs[1,0])
plot(4, axs[1,1])
plot(5, axs[1,2])
```

The domain is filling as the boundary conditions on the sides and bottom are _no flow_.  This is clearly visible in the saturation, although at this point the water table is only starting to rise in the at the end of the 10 days. The ponded water is also rising, but in this simulation this rise is primarily due the rainfall.  You can modify the input file to start with a higher water table, and possibly increase the permeability of the subsurface to increase infiltration - as this would ultimately cause the water table to reach the stream bed sooner, leading to groundwater contributions to the ponded water.

### Model Setup: 3D Open-Book Watershed

#### Parameters of 3D simulation:

**Simulation Scenario**

  * simulation start: 0.0 day
  * simulation end:  0.8 day

**Initial Condition**

* Subsurface Flow:
  * Hydrostatic pressure with water table at 6.0[m]
* Surface Flow
  * Set based on continuity of pressure and normal flux.

**Boundary Conditions**

* Subsurface Flow:
  * no flow boundary conditions
* Surface Flow
  * Zero-gradient conditions at the stream outlet

**Rainfall Rate**

#### Simulation Results

```python
def plot_transect(directory):
    vis = ats_xdmf.VisFile(directory, domain='surface', filename='ih_surface_data.h5', mesh_filename='ih_surface_mesh.h5', time_unit='s')
    vis.loadMesh(order=['x','y'])
    y = vis.centroids[:,1]
    pd = vis.getArray('surface-ponded_depth')
    cm = colors.cm_mapper(vis.times[0], vis.times[-1], cmap='jet')

    fig, ax = plt.subplots(1,2, figsize=(14,3))

    # This simply gets a colorbar axis
    C = [[0,0],[0,0]]
    C3 = ax[0].contourf(C, vis.times, cmap='jet')
    ax[0].cla()

    for i,time in enumerate(vis.times):
        #ax[0].plot(x, elev + pd[i,:], color=cm(i))
        #print(cm(i))
        ax[0].plot(y[100:119], pd[i,100:119], color=cm(time))

    ax[0].set_xlabel('y-coordinate [m]')
    ax[0].set_ylabel('ponded depth [m]')

    cbar = plt.colorbar(C3, ax=ax[0])
    cbar.ax.set_ylabel("time [s]")

    #ax[0].plot(y[120:139], pd[10,120:139], 'b-')
    #ax[0].plot(y[100:119], pd[10,100:119], 'r-')
    #ax[0].plot(y[80:99], pd[10,80:99], 'g-')

    out_flux = np.loadtxt(os.path.join(directory,'surface_outlet_flux.dat'), skiprows=11)
    out_flux = np.transpose(out_flux)
    ax[1].plot(out_flux[0][:]/3600.0, out_flux[1][:], 'b')
    ax[1].set_xlabel('time [h]')
    ax[1].set_ylabel('discharge flow rate [mol/s]')

```

```python
plot_transect('integrated_hydro/run_3D_results')
```

### Model Setup: Integrated Hydrology on Small Watershed

Building on the overland flow example where we simulated a rainfall event on the Borden Watershed, here we consider integrated hydrology at the same site. We consider a simulaton scenario similar to the model incomparison to provide a good starting point for you to modify the file and experiment.

Here are sample results, showing the subsurface is filling (saturation is increasing) as the stream depth is rising from the rainfall.

The input file is *integrated_hydro-borden_rainfall.xml*, and the VisIt session file is *integrated_hydro-borden_rainfall.session*

```python

```


```python
%load_ext autoreload
%autoreload 2

import modelproject.chxmodelproject as chx
```

```python
import numpy as np
import sympy as sm
#%matplotlib inline
import matplotlib.pyplot as plt
from sympy import *
from scipy import linalg
```

#### Introduction:
This model project examines the simple model of the Keynesian Cross as seen in Macroeconomics by N. Gregory Mankiw, International 8th edition.

### Model description: The Keynesian Cross
#### In a small open economy with fixed exchange rate in the short run

$Y$ is production and is equal the planned total spending $AD$:
$$ Y = AD $$

$PD$ consists of consumption $C$, investments $I$, public consumption $G$ and the netto export $NX$:
$$ AD = C + I + \bar{G} + NX $$

Private consumption is a linear function of the production left after taxes. $b$ is the marginal consumption rate.
$$ C = a + b (Y-\bar{T}), \quad a > 0, \quad 0<b<1 $$

Investments goes down if the interest rate $r$ rises:
$$ I = c - dr, \quad c,d>0 $$
Because of free capital movement the real domestic interest rate equals the real foreign interest rate, $r^*$
$$ r = \bar{r}^* $$

Net export goes down if the real exchange rate goes up:
$$ NX = f - g \epsilon, \quad f,g, >0 $$

The real exchange rate is determined by the nominal exchange rate and the relative price ratio between the domestic and foreign price level.
$$ \epsilon = \frac{\bar{e} \bar{P}}{\bar{P}^*} $$

As we are in the short run. Prices, public expenditure, taxes, the foreign interest rate are all exogenous. Making all variables but the consumption decided by external parameters.

$$ AD = a + b ( Y - \bar{T}) + I + G + NX$$

### Equalibrium analysis

```python
# To solve the analytically we use sympy:
Y, AD, a, b, T, I, G, NX = sm.symbols('Y, AD, a, b, T, I, G, NX')
```

```python
# We set AD=Y and solve wrt. Y:
eq_AD = sm.Eq(Y, a + b*(Y-T) + I + G + NX)
eq = sm.solve(eq_AD, Y)[0]
yeq = sm.factor(eq)

print('Y =')
yeq
```

    Y =

$\displaystyle - \frac{G + I + NX - T b + a}{b - 1}$

```python
# I how a change in net export changes the production by differentiating:
diff_Y = sm.diff(yeq, NX)
diff_Y
```

$\displaystyle - \frac{1}{b - 1}$

```python
# The yeq is turned in to a python function:
eq_func = sm.lambdify((T, I, G, NX, a, b), yeq)
```

```python
# We plot the 45-degree line of AD=Y and the AD=C+I+G+NX with the following parameters:
T = 40
I = 20
G = 30
NX = 10
a = 30
b = 0.3

# The plot of the cross given the parameters above:
cross = chx.keynesian_cross(T, I, G, NX, a, b)
equalibrium = chx.cross_equalibrium(T, I, G, NX, a, b)
```

```python
print(f'The equalibrium for the Keynesian cross is: Y = {equalibrium:.1f}')
print(f'The equalibrium for the Keynesian cross is: Y = {eq_func(T, I, G, NX, a, b):.1f}')
```

    The equalibrium for the Keynesian cross is: Y = 111.4
    The equalibrium for the Keynesian cross is: Y = 111.4

For the practice I also try to solve with linear algebra:

We have 3 equations of 3 unknown, all other variables are decided exogenously:

$$ AD = C + I + \bar{G} + NX $$
$$ C = a + b (Y-\bar{T})$$
$$ Y = AD $$

We rename $I + G + NX$ = $EE$, exogenous expenditure.

We rewrite the equations:

$$ - C + AD    = EE $$
$$ C - bY      = a -bT $$
$$ AD - Y      = 0 $$

This gives us the :

\\[
\begin{eqnarray*}
Ax = k \Leftrightarrow
\begin{bmatrix}
-1 & 1 & 0 \\
1 & 0 & -b \\
0 & 1 & -1
\end{bmatrix} \cdot
\begin{bmatrix}
C \\
AD \\
Y
\end{bmatrix} \,=\,
\begin{bmatrix}
EE \\
a - bT \\
0
\end{bmatrix}
\end{eqnarray*}
\\]

```python
# The result optained by linear algebra:
EE = I + G + NX
print(f'EE = {EE}')

# The matrix A
A = np.array([[-1, 1, 0], [1, 0, -b], [0, 1, -1]])
print(f'A = {A}')
# The vector k:
k = np.array([EE, a-b*T, 0])
print(f'k = {k}')

# Lastly I can solve A equals k, to optain the values of x:
x1 = linalg.solve(A, k)
print(f'[ C AD Y ] ={x1}')
```

    EE = 60
    A = [[-1.   1.   0. ]
     [ 1.   0.  -0.3]
     [ 0.   1.  -1. ]]
    k = [60. 18.  0.]
    [ C AD Y ] =[ 51.42857143 111.42857143 111.42857143]

We get the same result as above.

#### Theory of the Keynesian cross:

The models equalibrium is found where the two models cross, $Y^*$. They will always cross since $0<b<1$. If the production is higher than the aggregate expenditure and demand, the production stock will start to increase. This will make the producers correct their production level. This lower production leads to lower consumption, which again lowers demand. The production will continue to be lowered untill total production equals demand.

I examine what happens when the net export increases:

```python
# The increase is set:
delta_NX = 20
new_NX = NX + delta_NX

# A plot showing the new and the old AD-curve.
NX_shift = chx.keynesian_cross_NXshift(T, I, G, NX, a, b, delta_NX)

# The change in Y due to the change in NX:
delta_Y = eq_func(T, I, G, new_NX, a, b) - eq_func(T, I, G, NX, a, b)

print(f'The new equalibrium for the Keynesian cross is: Y = {eq_func(T, I, G, new_NX, a, b):.1f}')
print(f'Change in production when delta_NX=20: delta_Y = {delta_Y:.1f}')
```

Because of the parameter b, the new equalibrium production is even greater than the increase in NX. 28.6 > 20.

### Numerical optimization

I examine how much the net export will have to rise in order to get an equilibrium in the short run with $Y=150$:

```python
# Production goal:
Y_goal = 150

result = chx.num_opt(Y_goal,T,I,G,a,b).x
print(f'Size of NX optained by numerical optimization when production is set to 150: NX={result[0]:.3f}')

# Test
NX_goal = result[0]
equalibrium = chx.cross_equalibrium(T, I, G, NX_goal, a, b)
print(f'Y in equilibrium when NX is equal to 37: Y*={equalibrium}')
```

    Size of NX optained by numerical optimization when production is set to 150: NX=37.000
    Y in equilibrium when NX is equal to 37: Y*=150.00000114393617

### Extension

Instead of a lump sum tax we make taxes depend on the production size:

$$ T = k + t Y $$

```python
# We repeat the steps from the first part of the analysis:
Y, AD, a, b, k, t, I, G, NX = sm.symbols('Y, AD, a, b, k, t, I, G, NX')

eq_AD_t = sm.Eq(Y, a + b*(Y-(k + t*Y)) + I + G + NX)
eq_t = sm.solve(eq_AD_t, Y)[0]
yeq_t = sm.factor(eq_t)

print('Y=')
yeq_t
```

    Y=

$\displaystyle \frac{G + I + NX + a - b k}{b t - b + 1}$

```python
# We differentiate:
diff_Y_t = sm.diff(yeq_t, NX)
diff_Y_t
```

$\displaystyle \frac{1}{b t - b + 1}$

We can see that the new slope is less steep. The multiplicator effect is smaller.

```python
print(f'{diff_Y_t} < {diff_Y}')
```

    1/(b*t - b + 1) < -1/(b - 1)

```python
# We make it into a python function:
eq_func_t = sm.lambdify((k, t, I, G, NX, a, b), yeq_t)
```

```python
# We parameterize and plot the graphs:
k = 20
t = 0.2
I = 20
G = 30
NX = 10
a = 30
b = 0.3

delta_NX = 20
new_NX = NX + delta_NX

print(f'The equalibrium for the Keynesian cross is: Y = {eq_func_t(k, t, I, G, NX, a, b):.1f}')

NX_shift_t = chx.keynesian_cross_NXshift_t(k, t, I, G, NX, a, b, delta_NX)

delta_Y_t = eq_func_t(k, t, I, G, new_NX, a, b) - eq_func_t(k, t, I, G, NX, a, b)

print(f'The new equalibrium for the Keynesian cross is: Y = {eq_func_t(k, t, I, G, new_NX, a, b):.1f}')
print(f'Change in production when delta_NX = 20: delta_Y = {delta_Y_t:.1f}')
```

Because of the dependent tax the shift in the total production is smaller. 26.3 < 28.6. By including a production dependent tax the economy gets more stable and shifts in exogenous variables have a smaller effect on the economy.


# Data Science Unit 1 Sprint Challenge 3

## Exploring Data, Testing Hypotheses

In this sprint challenge you will look at a dataset of people being approved or rejected for credit.

https://archive.ics.uci.edu/ml/datasets/Credit+Approval

Data Set Information: This file concerns credit card applications. All attribute names and values have been changed to meaningless symbols to protect confidentiality of the data. This dataset is interesting because there is a good mix of attributes -- continuous, nominal with small numbers of values, and nominal with larger numbers of values. There are also a few missing values.

Attribute Information:
- A1: b, a.
- A2: continuous.
- A3: continuous.
- A4: u, y, l, t.
- A5: g, p, gg.
- A6: c, d, cc, i, j, k, m, r, q, w, x, e, aa, ff.
- A7: v, h, bb, j, n, z, dd, ff, o.
- A8: continuous.
- A9: t, f.
- A10: t, f.
- A11: continuous.
- A12: t, f.
- A13: g, p, s.
- A14: continuous.
- A15: continuous.
- A16: +,- (class attribute)

Yes, most of that doesn't mean anything. A16 is a variable that indicates whether or not a person's request for credit has been approved or denied. This is a good candidate for a y variable since we might want to use the other features to predict this one. The remaining variables have been obfuscated for privacy - a challenge you may have to deal with in your data science career.

Sprint challenges are evaluated based on satisfactory completion of each part. It is suggested you work through it in order, getting each aspect reasonably working, before trying to deeply explore, iterate, or refine any given step. Once you get to the end, if you want to go back and improve things, go for it!

## Part 1 - Load and validate the data

- Load the data as a `pandas` data frame.
- Validate that it has the appropriate number of observations (you can check the raw file, and also read the dataset description from UCI).
- UCI says there should be missing data - check, and if necessary change the data so pandas recognizes it as na
- Make sure that the loaded features are of the types described above (continuous values should be treated as float), and correct as necessary

This is review, but skills that you'll use at the start of any data exploration. Further, you may have to do some investigation to figure out which file to load from - that is part of the puzzle.

Hint: If a column has the datatype of "object" even though it's made up of float or integer values, you can coerce it to act as a numeric column by using the `pd.to_numeric()` function.

My hypothosis for this M & F and Credit Approval Column,

I was under the impression that F would have more credit approval.

But the statics, mostly the graphics proved otherwise.

```
import pandas as pd
import numpy as np
import seaborn as sns
import scipy.stats as stats
```

```

column_headers = ['A1','12','A3','A4','A5','A6','A7','A8','A9','A10','A11','A12','A13','A14','A15','A16']

df=pd.read_csv('crx.csv', names=column_headers,na_values='?')

```

```
df.shape
```

    (690, 16)

```
df.head()
```

      A1
      12
      A3
      A4
      A5
      A6
      A7
      A8
      A9
      A10
      A11
      A12
      A13
      A14
      A15
      A16

      0
      b
      30.83
      0.000
      u
      g
      w
      v
      1.25
      t
      t
      1
      f
      g
      202.0
      0
      +

      1
      a
      58.67
      4.460
      u
      g
      q
      h
      3.04
      t
      t
      6
      f
      g
      43.0
      560
      +

      2
      a
      24.50
      0.500
      u
      g
      q
      h
      1.50
      t
      f
      0
      f
      g
      280.0
      824
      +

      3
      b
      27.83
      1.540
      u
      g
      w
      v
      3.75
      t
      t
      5
      t
      g
      100.0
      3
      +

      4
      b
      20.17
      5.625
      u
      g
      w
      v
      1.71
      t
      f
      0
      f
      s
      120.0
      0
      +

```
df.shape

```

    (690, 16)

```
df.isnull().sum()
```

    A1     12
    12     12
    A3      0
    A4      6
    A5      6
    A6      9
    A7      9
    A8      0
    A9      0
    A10     0
    A11     0
    A12     0
    A13     0
    A14    13
    A15     0
    A16     0
    dtype: int64

```
df.dtypes
```

    A1      object
    12     float64
    A3     float64
    A4      object
    A5      object
    A6      object
    A7      object
    A8     float64
    A9      object
    A10     object
    A11      int64
    A12     object
    A13     object
    A14    float64
    A15      int64
    A16     object
    dtype: object

```
df.A15 = df.A15.astype(float)
df.head()
```

      A1
      12
      A3
      A4
      A5
      A6
      A7
      A8
      A9
      A10
      A11
      A12
      A13
      A14
      A15
      A16

      0
      b
      30.83
      0.000
      u
      g
      w
      v
      1.25
      t
      t
      1
      f
      g
      202.0
      0.0
      +

      1
      a
      58.67
      4.460
      u
      g
      q
      h
      3.04
      t
      t
      6
      f
      g
      43.0
      560.0
      +

      2
      a
      24.50
      0.500
      u
      g
      q
      h
      1.50
      t
      f
      0
      f
      g
      280.0
      824.0
      +

      3
      b
      27.83
      1.540
      u
      g
      w
      v
      3.75
      t
      t
      5
      t
      g
      100.0
      3.0
      +

      4
      b
      20.17
      5.625
      u
      g
      w
      v
      1.71
      t
      f
      0
      f
      s
      120.0
      0.0
      +

```
# df=df['A16'].replace({'+': 1,'-': 0})
# df['A16']np.replace('+'': 1)
# df1=df.A16.map(dict('+'+:1, 0))
# df1=df['A16'].replace('+', 1)(inplace=True)
df['A16'].replace('+','1',inplace=True)
```

```
df.head(1)
```

      A1
      12
      A3
      A4
      A5
      A6
      A7
      A8
      A9
      A10
      A11
      A12
      A13
      A14
      A15
      A16

      0
      b
      30.83
      0.0
      u
      g
      w
      v
      1.25
      t
      t
      1
      f
      g
      202.0
      0.0
      1

```
df.shape
```

    (690, 16)

```
df['A16'].replace('+','0',inplace=True)
```

```
df['A16'].replace('-','0',inplace=True)
```

```
df['A1'].replace('a','1',inplace=True)
```

```
df['A1'].replace('b','0',inplace=True)
```

```
df.head(1)
```

      A1
      12
      A3
      A4
      A5
      A6
      A7
      A8
      A9
      A10
      A11
      A12
      A13
      A14
      A15
      A16

      0
      0
      30.83
      0.0
      u
      g
      w
      v
      1.25
      t
      t
      1
      f
      g
      202.0
      0.0
      1

```
df.shape
```

    (690, 16)

## Part 2 - Exploring data, Testing hypotheses

The only thing we really know about this data is that A16 is the class label. Besides that, we have 6 continuous (float) features and 9 categorical features.

Explore the data: you can use whatever approach (tables, utility functions, visualizations) to get an impression of the distributions and relationships of the variables. In general, your goal is to understand how the features are different when grouped by the two class labels (`+` and `-`).

For the 6 continuous features, how are they different when split between the two class labels? Choose two features to run t-tests (again split by class label) - specifically, select one feature that is *extremely* different between the classes, and another feature that is notably less different (though perhaps still "statistically significantly" different). You may have to explore more than two features to do this.

For the categorical features, explore by creating "cross tabs" (aka [contingency tables](https://en.wikipedia.org/wiki/Contingency_table)) between them and the class label, and apply the Chi-squared test to them. [pandas.crosstab](http://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.crosstab.html) can create contingency tables, and [scipy.stats.chi2_contingency](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.chi2_contingency.html) can calculate the Chi-squared statistic for them.

There are 9 categorical features - as with the t-test, try to find one where the Chi-squared test returns an extreme result (rejecting the null that the data are independent), and one where it is less extreme.

**NOTE** - "less extreme" just means smaller test statistic/larger p-value. Even the least extreme differences may be strongly statistically significant.

Your *main* goal is the hypothesis tests, so don't spend too much time on the exploration/visualization piece. That is just a means to an end - use simple visualizations, such as boxplots or a scatter matrix (both built in to pandas), to get a feel for the overall distribution of the variables.

This is challenging, so manage your time and aim for a baseline of at least running two t-tests and two Chi-squared tests before polishing. And don't forget to answer the questions in part 3, even if your results in this part aren't what you want them to be.

###Select columns of interest:
- A1 and A16
- A1 = M or F
- A16 = Approved or Disapproved.

```
df2=df[['A1','A16']]
```

```
df2.head(500)
```

      A1
      A16

      0
      0
      1

      1
      1
      1

      2
      1
      1

      3
      0
      1

      4
      0
      1

      5
      0
      1

      6
      0
      1

      7
      1
      1

      8
      0
      1

      9
      0
      1

      10
      0
      1

      11
      0
      1

      12
      1
      1

      13
      0
      1

      14
      1
      1

      15
      0
      1

      16
      0
      1

      17
      1
      1

      18
      0
      1

      19
      1
      1

      20
      0
      1

      21
      0
      1

      22
      1
      1

      23
      1
      1

      24
      1
      1

      25
      1
      1

      26
      1
      1

      27
      0
      1

      28
      0
      1

      29
      0
      1

      ...
      ...
      ...

      470
      1
      0

      471
      0
      0

      472
      0
      0

      473
      0
      0

      474
      0
      0

      475
      0
      0

      476
      0
      0

      477
      0
      0

      478
      0
      0

      479
      NaN
      0

      480
      1
      0

      481
      0
      0

      482
      1
      0

      483
      0
      0

      484
      0
      0

      485
      0
      0

      486
      0
      0

      487
      0
      0

      488
      0
      0

      489
      NaN
      0

      490
      1
      1

      491
      0
      1

      492
      0
      1

      493
      1
      1

      494
      1
      1

      495
      0
      1

      496
      1
      1

      497
      0
      1

      498
      0
      1

      499
      0
      1

500 rows × 2 columns

```
df.shape
```

    (690, 16)

Remove the Credit Disapparval Rows

```
df2 = df2[df.A16 != '0']
```

```
df2.head(200)
```

      A1
      A16

      0
      0
      1

      1
      1
      1

      2
      1
      1

      3
      0
      1

      4
      0
      1

      5
      0
      1

      6
      0
      1

      7
      1
      1

      8
      0
      1

      9
      0
      1

      10
      0
      1

      11
      0
      1

      12
      1
      1

      13
      0
      1

      14
      1
      1

      15
      0
      1

      16
      0
      1

      17
      1
      1

      18
      0
      1

      19
      1
      1

      20
      0
      1

      21
      0
      1

      22
      1
      1

      23
      1
      1

      24
      1
      1

      25
      1
      1

      26
      1
      1

      27
      0
      1

      28
      0
      1

      29
      0
      1

      ...
      ...
      ...

      217
      0
      1

      218
      0
      1

      219
      1
      1

      220
      1
      1

      221
      0
      1

      222
      1
      1

      223
      1
      1

      224
      1
      1

      225
      0
      1

      226
      0
      1

      227
      1
      1

      228
      0
      1

      229
      0
      1

      230
      0
      1

      231
      1
      1

      232
      0
      1

      233
      0
      1

      234
      1
      1

      235
      1
      1

      236
      0
      1

      237
      0
      1

      238
      0
      1

      239
      0
      1

      240
      0
      1

      241
      0
      1

      242
      0
      1

      243
      1
      1

      244
      0
      1

      245
      0
      1

      246
      0
      1

200 rows × 2 columns

```
df.shape
```

    (690, 16)

Dropped the Credit Approval Column

now we have the M and F credit approve column

```
df2=df2.drop(columns=['A16'])
```

```
df2.head()
```

      A1

      0
      0

      1
      1

      2
      1

      3
      0

      4
      0

###Selected Appoved M or F

```
df2.sum()
```

    Series([], dtype: float64)

```
df2.shape

```

    (307, 1)

```
500-272
```

    228

```
import numpy as np
df2= np.random.binomial(n=1, p=.5, size=500)
print(np.std(df2, ddof=1))
print(df2)
```

    0.4981891054303969
    [0 1 1 0 0 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1
     1 1 1 0 1 1 1 1 1 0 1 0 1 0 1 1 1 0 0 1 1 1 0 1 1 0 0 1 1 0 0 0 0 0 0 1 0
     0 0 1 1 1 0 1 0 1 1 1 1 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 1 0 1 1 1 1 0 0 1 1
     1 1 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 0 0 0 1
     1 1 1 0 0 0 1 1 0 0 0 1 0 1 1 0 0 1 1 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1
     1 0 0 1 1 1 0 0 1 1 1 0 1 0 1 1 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 1 0
     1 1 1 1 1 0 1 1 0 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 1 1 0 0 1 0 0 0 0 1 0 0
     0 0 0 1 0 0 1 1 0 1 0 0 0 1 0 1 1 1 0 0 0 0 0 1 0 0 1 1 1 1 1 1 1 1 0 1 0
     1 0 0 1 1 0 0 1 0 0 0 1 0 0 1 1 1 1 0 0 1 1 1 0 1 0 1 0 1 1 0 1 0 1 1 0 0
     0 0 0 1 1 0 0 1 1 1 0 1 0 1 1 0 0 0 1 0 0 1 1 0 0 1 1 1 0 1 0 1 0 0 1 1 0
     0 0 1 0 1 1 0 1 0 1 1 0 1 1 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 0 1 0 1 1 0 0 0
     0 1 1 0 0 0 1 0 1 0 1 1 1 1 0 1 0 0 0 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 0 0
     0 0 0 1 1 0 1 1 0 0 0 0 1 1 1 0 0 0 1 1 0 0 0 1 0 0 1 0 0 0 1 0 1 1 1 0 0
     0 0 1 0 0 1 0 0 1 0 1 0 1 0 1 0 0 0 1]

```
import scipy.stats as stats
def confidence_interval(data, confidence=0.95):
  """
  Calculate a confidence interval around a sample mean for given data.
  Using t-distribution and two-tailed test, default 95% confidence.

  Arguments:
    data - iterable (list or numpy array) of sample observations
    confidence - level of confidence for the interval

  Returns:
    tuple of (mean, lower bound, upper bound)
  """
  data = np.array(data)
  mean = np.mean(data)
  n = len(data)
  #stderr = stats.sem(data)
  stderr = np.std(data, ddof=1) / np.sqrt(n)
  margin_of_error = stderr * stats.t.ppf((1 + confidence) / 2.0, n - 1)
  print(margin_of_error)
  return (mean, mean - margin_of_error, mean + margin_of_error)
```

```
df2 = np.random.binomial(n=1, p=.5, size=500)
confidence_interval(df2)
```

    0.04397668345020961

    (0.5, 0.4560233165497904, 0.5439766834502096)

```
df2 = np.random.binomial(n=1, p=.5, size=500)
confidence_interval(df2)
```

    0.04395944121015544

    (0.486, 0.44204055878984455, 0.5299594412101554)

```
df2 = np.random.binomial(n=1, p=.5, size=500)
confidence_interval(df2)
```

    0.04397105407442487

    (0.492, 0.44802894592557513, 0.5359710540744249)

Looking at stats.t.ppf

```
confidence_level = .95
dof =42-1
stats.t.ppf((1 + confidence_level ) / 2, dof)
```

    2.0195409639828936

```
import seaborn as sns
import matplotlib.pyplot as plt

df2_100 = np.random.binomial(n=1, p=.5, size=42 )
sns.kdeplot(df2_100)
CI = confidence_interval(df2_100)
plt.axvline(x=CI[1], color='red')
plt.axvline(x=CI[2], color='red')
plt.axvline(x=CI[0], color='k');
```

Relationship between Confidence Intervals and T-tests Confidence Interval == Bounds of statistical significance for our t-test

A sample mean that falls inside of our confidence interval will "FAIL TO REJECT" our null hypothesis

A sample mean that falls outside of our confidence interval will "REJECT" our null hypothesis

```
from scipy.stats import t, ttest_1samp
```

```
import numpy as np

df2_means = []
for x in range(0,100):
  df2 = np.random.binomial(n=1, p=.5, size=30)
  df2_means.append(df2.mean())

print(df2_means)
```

    [0.4666666666666667, 0.4, 0.6, 0.5, 0.43333333333333335, 0.5666666666666667, 0.43333333333333335, 0.4666666666666667, 0.5, 0.3, 0.43333333333333335, 0.4, 0.6, 0.6, 0.5666666666666667, 0.6333333333333333, 0.36666666666666664, 0.43333333333333335, 0.5333333333333333, 0.6, 0.4666666666666667, 0.4, 0.4, 0.5333333333333333, 0.43333333333333335, 0.4, 0.4, 0.5666666666666667, 0.43333333333333335, 0.3333333333333333, 0.6, 0.5666666666666667, 0.6, 0.4666666666666667, 0.36666666666666664, 0.43333333333333335, 0.6666666666666666, 0.6666666666666666, 0.4, 0.5, 0.5666666666666667, 0.6333333333333333, 0.6, 0.43333333333333335, 0.5333333333333333, 0.4666666666666667, 0.4, 0.5333333333333333, 0.4, 0.43333333333333335, 0.5666666666666667, 0.6333333333333333, 0.6, 0.5666666666666667, 0.4666666666666667, 0.3333333333333333, 0.43333333333333335, 0.43333333333333335, 0.36666666666666664, 0.5333333333333333, 0.5666666666666667, 0.5666666666666667, 0.6, 0.4, 0.6, 0.43333333333333335, 0.5333333333333333, 0.4666666666666667, 0.5666666666666667, 0.6333333333333333, 0.5, 0.4, 0.5333333333333333, 0.4, 0.5333333333333333, 0.4, 0.6, 0.43333333333333335, 0.4666666666666667, 0.6333333333333333, 0.43333333333333335, 0.4666666666666667, 0.6, 0.3333333333333333, 0.5333333333333333, 0.6333333333333333, 0.5333333333333333, 0.4, 0.5333333333333333, 0.5333333333333333, 0.4, 0.6333333333333333, 0.6333333333333333, 0.6, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.4, 0.43333333333333335]

```
# Sample Size
n = len(df2_means)
# Degrees of Freedom
dof = n-1
# The Mean of Means:
mean = np.mean(df2_means)
# Sample Standard Deviation
sample_std = np.std(df2_means, ddof=1)
# Standard Error
std_err = sample_std/n**.5

CI = t.interval(.95, dof, loc=mean, scale=std_err)
print("95% Confidence Interval: ", CI)
```

    95% Confidence Interval:  (0.4791113912578343, 0.5148886087421655)

```
'''You can roll your own CI calculation pretty easily.
The only thing that's a little bit challenging
is understanding the t stat lookup'''

# 95% confidence interval
t_stat = t.ppf(.975, dof)
print("t Statistic:", t_stat)

CI = (mean-(t_stat*std_err), mean+(t_stat*std_err))
print("Confidence Interval", CI)
```

    t Statistic: 1.9842169515086827
    Confidence Interval (0.4791113912578343, 0.5148886087421655)

A null hypothesis that's just inside of our confidence interval == fail to reject

```
ttest_1samp(df2_means, .481453)
```

    Ttest_1sampResult(statistic=1.72448407753429, pvalue=0.08774143616296348)

A null hypothesis that just ouside of our confidence interval == reject*Your words here!*

```
ttest_1samp(df2_means, .481453)
```

    Ttest_1sampResult(statistic=1.72448407753429, pvalue=0.08774143616296348)

```
ttest_1samp(df2_means, .481453)
```

    Ttest_1sampResult(statistic=1.72448407753429, pvalue=0.08774143616296348)

```

```

##Run a chi2 Test 'by hand' (Using Numpy)
χ2=∑(observedij−expetedi)2(expectedi)

##Run a $chi^ {2}$ Test 'by hand' (Using Numpy)

\begin{align}
\chi^2 = \sum \frac{(observed_{ij}-expeted_{i})^2}{(expected_{i})}
\end{align}

```
import pandas as pd

df= pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/adult.csv', na_values=' ?')
print(df.shape)
#      df.)
# column_headers = ['age','workclass','fnlwgt','education','education-num',
#                   'martial-status','occupation','relationship','race','sex',
#                   'capital-gain','capital-loss','hours_per_week',
#                   'native-country','income']

# df = pd.read_csv(adult_data, names=column_headers, na_values=' ?', sep=', ')
df.head(1)
```

    (32561, 15)

      age
      workclass
      fnlwgt
      education
      education-num
      marital-status
      occupation
      relationship
      race
      sex
      capital-gain
      capital-loss
      hours-per-week
      country
      salary

      0
      39
      State-gov
      77516
      Bachelors
      13
      Never-married
      Adm-clerical
      Not-in-family
      White
      Male
      2174
      0
      40
      United-States
      <=50K

```
df.describe(exclude='number')
```

      workclass
      education
      marital-status
      occupation
      relationship
      race
      sex
      country
      salary

      count
      30725
      32561
      32561
      30718
      32561
      32561
      32561
      31978
      32561

      unique
      8
      16
      7
      14
      6
      5
      2
      41
      2

      top
      Private
      HS-grad
      Married-civ-spouse
      Prof-specialty
      Husband
      White
      Male
      United-States
      <=50K

      freq
      22696
      10501
      14976
      4140
      13193
      27816
      21790
      29170
      24720

```
cut_points = [0,9,19,29,39,49,1000]
label_names = ['0-9','10-19','20-29','30-39','40-49','50+']
df['hours_per_week_categories'] = pd.cut(df['hours-per-week'], cut_points, labels=label_names)
df.head()
```

      age
      workclass
      fnlwgt
      education
      education-num
      marital-status
      occupation
      relationship
      race
      sex
      capital-gain
      capital-loss
      hours-per-week
      country
      salary
      hours_per_week_categories

      0
      39
      State-gov
      77516
      Bachelors
      13
      Never-married
      Adm-clerical
      Not-in-family
      White
      Male
      2174
      0
      40
      United-States
      <=50K
      40-49

      1
      50
      Self-emp-not-inc
      83311
      Bachelors
      13
      Married-civ-spouse
      Exec-managerial
      Husband
      White
      Male
      0
      0
      13
      United-States
      <=50K
      10-19

      2
      38
      Private
      215646
      HS-grad
      9
      Divorced
      Handlers-cleaners
      Not-in-family
      White
      Male
      0
      0
      40
      United-States
      <=50K
      40-49

      3
      53
      Private
      234721
      11th
      7
      Married-civ-spouse
      Handlers-cleaners
      Husband
      Black
      Male
      0
      0
      40
      United-States
      <=50K
      40-49

      4
      28
      Private
      338409
      Bachelors
      13
      Married-civ-spouse
      Prof-specialty
      Wife
      Black
      Female
      0
      0
      40
      Cuba
      <=50K
      40-49

```
df['sex'].value_counts()
```

     Male      21790
     Female    10771
    Name: sex, dtype: int64

```
df['hours_per_week_categories'].value_counts()

```

    40-49    18336
    50+       6462
    30-39     3667
    20-29     2392
    10-19     1246
    0-9        458
    Name: hours_per_week_categories, dtype: int64

```
contingency_table = pd.crosstab(df['sex'], df['hours_per_week_categories'], margins=True)
contingency_table
```

      hours_per_week_categories
      0-9
      10-19
      20-29
      30-39
      40-49
      50+
      All

      sex

      Female
      235
      671
      1287
      1914
      5636
      1028
      10771

      Male
      223
      575
      1105
      1753
      12700
      5434
      21790

      All
      6462
      1246
      18336
      3667
      458
      2392
      32561

```
femalecount = contingency_table.iloc[0][0:6].values
femalecount
```

    array([ 235,  671, 1287, 1914, 5636, 1028])

```
malecount = contingency_table.iloc[1][0:6].values
malecount
```

    array([  223,   575,  1105,  1753, 12700,  5434])

```
import seaborn
import matplotlib
#Plots the bar chart
fig = plt.figure(figsize=(10,5))
sns.set(font_scale=1.0)
matplotlib.style.use('ggplot')
categories = ['0-9','10-19','20-29','30-39','40-49','50+']
p1 = plt.bar(categories, malecount, 0.55, color='palegreen')
p2 = plt.bar(categories, femalecount, 0.55, color='#d62728', bottom=malecount)
plt.legend((p2[0],p1[0]),('Female', 'Male'))
plt.xlabel('Hours per Week Worked')
plt.ylabel('Count')
plt.show()
```

##Expected Value Calculation
\begin{align}
expected_{i,j} =\frac{{row_{i} \text{total}}(column_{j} \text{total}) } {\text{total observations})}
\end{align}

```
row_sums = contingency_table.iloc[0:2, 6].values
col_sums = contingency_table.iloc[2, 0:6].values
```

```
print(row_sums)
print(col_sums)
```

    [10771 21790]
    [ 6462  1246 18336  3667   458  2392]

```
total = contingency_table.loc['All','All']
total
```

    32561

```
df.shape
```

    (32561, 16)

```
import numpy as np
expected = []
for i in range(len(row_sums)):
  expected_row = []
  for column in col_sums:
    expected_val = column*row_sums[i]/total
    expected_row.append(expected_val)
    expected.append(expected_row)

expected = np.array(expected)
print(expected.shape)
print(expected)

```

    (12, 6)
    [[ 2137.59411566   412.16995793  6065.44811277  1213.02346365
        151.50388502   791.26046497]
     [ 2137.59411566   412.16995793  6065.44811277  1213.02346365
        151.50388502   791.26046497]
     [ 2137.59411566   412.16995793  6065.44811277  1213.02346365
        151.50388502   791.26046497]
     [ 2137.59411566   412.16995793  6065.44811277  1213.02346365
        151.50388502   791.26046497]
     [ 2137.59411566   412.16995793  6065.44811277  1213.02346365
        151.50388502   791.26046497]
     [ 2137.59411566   412.16995793  6065.44811277  1213.02346365
        151.50388502   791.26046497]
     [ 4324.40588434   833.83004207 12270.55188723  2453.97653635
        306.49611498  1600.73953503]
     [ 4324.40588434   833.83004207 12270.55188723  2453.97653635
        306.49611498  1600.73953503]
     [ 4324.40588434   833.83004207 12270.55188723  2453.97653635
        306.49611498  1600.73953503]
     [ 4324.40588434   833.83004207 12270.55188723  2453.97653635
        306.49611498  1600.73953503]
     [ 4324.40588434   833.83004207 12270.55188723  2453.97653635
        306.49611498  1600.73953503]
     [ 4324.40588434   833.83004207 12270.55188723  2453.97653635
        306.49611498  1600.73953503]]

```
## Chi-Squared Statistic with Numpy

# \begin{align}
# \chi^2 = \sum \frac{(obseved_{i}-expected_{i})^2}{(expected_{i})}
# \end{align
```

## Part 3 - Analysis and Interpretation

Now that you've looked at the data, answer the following questions:

- Interpret and explain the two t-tests you ran - what do they tell you about the relationships between the continuous features you selected and the class labels?

###The class labels allowed me to see the the difference between
   the men and women in both the table and the graphic.

###The ttest break everything down and surprises one to find out that what one thought was evident, was disguised by appearance.

- Interpret and explain the two Chi-squared tests you ran - what do they tell you about the relationships between the categorical features you selected and the class labels?

###The class labels allowed me to see the the difference between
   the men and women in both the table and the graphic.

- What was the most challenging part of this sprint challenge?

### I found that looking up the code for each move I made very challenging.  But, all in all, I enjoy working with data.

Answer with text, but feel free to intersperse example code/results or refer to it from earlier.

## Stretch Goals

Please do not work on anything listed in this section until you have adequately answered everything in the above three sections, all of these stretch goals are **OPTIONAL** but completing them will ensure you a score of 3 on the corresponding sections.

### Section 1 Stretch Goal: (do the following)
- Go above and beyond in conducting thorough data exploration including high quality comments and/or markdown text cells to explain your process/discoveries.

### Section 2 Stretch Goals: (do one of the following)
 - Write a function(s) to automate the execution of t-tests on the continuous variables treating the different class labels as the two samples.

 - Write a funciton(s) to automate the execution of chi^2 tests on all of the different combinations of categorical variables in the dataset.

### Section 3 Stretch Goals: (do one of the following)
- Construct a confidence interval around the mean of one of the continuous variables, communicate the results/interpretation of that confidence interval in the most consumable/easy-to-understand way that you can. (You may choose to include a graph here, but you don't necessarily have to) Communicate the precision of your estimate as clearly as possible.

- Explain the relationship between confidence intervals and T-tests, if possible, use code to demonstrate some important aspects of te


Solution to: [Day 7: Spearman's Rank Correlation Coefficient](https://www.hackerrank.com/challenges/s10-spearman-rank-correlation-coefficient/problem)

Table of Contents

- Table of Contents
- Notes
    - Spearman's Rank Correlation Coefficient
    - Example
    - Spearman - Special Case
- Solution
    - Imports
    - Input
    - Print
    - Spearman's Rank - standard implementation
    - Spearman correlation solution for unique values
    -  Main

```javascript
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')
```

    <IPython.core.display.Javascript object>

# Notes
## Spearman's Rank Correlation Coefficient

We have two random variables, X and Y:

	X = (x1, x2, x3, ... xn)
	Y = (y1, y2, y2, ... yn)

If Rank_x and Rank_y denote the respective ranks of each data point, then the **Spearman's rank correlation coefficient**, Rs, is the Pearson correlation coefficient of *X_Rank* and *Y_Rank*.

## Example
	X = {0.2, 1.3, 0.2, 1.1, 1.4, 1.5}
    X_Rank = [1, 3, 1, 2, 4, 5]

	Y = {1.9, 2.2, 3.1, 1.2, 2.2, 2.2}
    Y_Rank = [2, 3, 4, 1, 3, 3]

Rs = the Pearson correlation coefficient of rank_x and rank_y, or 0.158114

## Spearman - Special Case
When there are no duplicates in X and Y (i.e., every value has its own rank), then the Spearman Rank Correlation may bec omputed with the following equation.

\begin{equation}
\large
rs = 1 - \frac
{6 \sum d_{i}^{2}}
{n * (n^{2} - 1)}
\end{equation}

[Proof](https://www.hackerrank.com/challenges/s10-spearman-rank-correlation-coefficient/tutorial)

# Solution

## Imports

```python
from typing import Tuple
import math
import statistics
```

## Input

```python
def get_input() -> Tuple[int, list, list]:
	"""Returns input for Spearman's rank correlation coefficient.

	Returns:
		Tuple[int, float, float]: number of items, x, y
	"""
	return 10, [10, 9.8, 8, 7.8, 7.7, 1.7, 6, 5, 1.4, 2, ], [200, 44, 32, 24, 22, 17, 15, 12, 8, 4,]

	num_items = int(input())
	x = [float(val) for val in input().split()]
	y = [float(val) for val in input().split()]
	return num_items, x, y
```

## Print

```python
def print_to_scale(num: int) -> None:
	"""Prints number formatted to 3 decimal scale.

	Args:
		num (int): Number to print to scale
	"""
	print(f"{num :.3f}")
```

## Spearman's Rank - standard implementation

```python
def get_ranks(x: list) -> list:
	"""Returns corresponding ranks of x list.

	Args:
		x (list): List to rank

	Returns:
		list: Rankings of x
	"""
	x_sorted = sorted(x)

	## Get rank of each val
	val_ranks = dict()
	for i in range(len(x_sorted)):
		val_ranks[x_sorted[i]] = (i + 1)

	## Create ranking
	x_ranks = []
	for val in x:
		x_ranks.append( val_ranks[val])
	return x_ranks
```

```python
def calc_mean(x: list) -> float:
	"""Returns mean of list

	Args:
		x (list): list to mean

	Returns:
		float: mean of x
	"""
	# mean = sum(x) / len(x)
	# print(mean)
	# print(statistics.mean(x))
	return sum(x) / len(x)
```

```python
def calc_sd(x: list, population: bool = True) -> float:
	"""Returns standard deviation of list, x.

	Args:
		x (list): List to get sd

	Returns:
		float: standard deviation of x
	"""
	x_mean = calc_mean(x)
	numerator = 0
	for val in x:
		numerator += ((val - x_mean) ** 2)

	n = len(x) if population else len(x) - 1
	sd = (numerator / n) ** (1/2)
	return sd
```

```python
def calc_cov(num_items: int, x: list, y: list) -> float:
	"""Calculates covariance for (x, y).

	Args:
		num_items (int): length of x, y
		x (list): series 1
		y (list): series 2

	Returns:
		float: covariance between x and y.
	"""
	mean_x, mean_y = calc_mean(x), calc_mean(y)

	numerator = 0
	for i in range(num_items):
		numerator += (x[i] - mean_x) * (y[i] - mean_y)

	return numerator / num_items
```

```python
def calc_spearmans_coef(num_items: int, x: list, y: list) -> float:
	"""Returns spearman's coefficient for x,y

	Args:
		num_items (int): length of both lists
		x (list): series 1
		y (list): series 2

	Returns:
		float: Spearman's coefficient for (x, y)
	"""
	assert len(x) == len(y)
	rank_x, rank_y = get_ranks(x), get_ranks(y)

	cov = calc_cov(num_items, rank_x, rank_y)
	sd_x, sd_y = calc_sd(rank_x), calc_sd(rank_y)

	return cov / (sd_x * sd_y)
```

```python
def spearman_rank_norm_sol(num_items: int, x: list, y: list) -> None:
	"""Prints solution using standard formula for spearman rank.

	Args:
		num_items (int): number of items
		x (list): Series 1
		y (list): Series 2
	"""
	print_to_scale( calc_spearmans_coef(num_items, x, y))
```

## Spearman correlation solution for unique values

```python
def calc_d(x: list, y: list) -> float:
	"""Returns difference between respective values of x and y.

	Args:
		x_ranks (list): Series 1
		y_ranks (list): Series 2

	Returns:
		float: statistic representing difference between x and y ranks
	"""
	x_ranks, y_ranks = get_ranks(x), get_ranks(y)

	sum_dif = 0
	for i in range(len(x_ranks)):
		sum_dif += (x_ranks[i] - y_ranks[i]) ** 2
	return sum_dif
```

```python
def spearman_rank_unique_vals(num_items: int, x: list, y: list) -> None:
	"""Prints solution using formula for unique values.

	Args:
		num_items (int): number of items
		x (list): Series 1
		y (list): Series 2
	"""
	d = calc_d(x, y)
	r = 1 - ((6 * d) / (num_items * (num_items**2 - 1)))
	print_to_scale(r)
```

## Main

```python
def main():
	num_items, x, y = get_input()

	spearman_rank_norm_sol(num_items, x, y)
	spearman_rank_unique_vals(num_items, x, y)
```

```python
if __name__ == "__main__":
	main()
```

    0.903
    0.903


# Lecture 31
## Markov chains, Transition Matrix, Stationary Distribution
----

## What are Markov chains?

Markov Chains are an example of a stochastic process, or sequences of random variables evolving over the dimension of time or space. They were invented or discovered by Markov as way to answer the question _Does free will exist?_ However, their application is very wide.

Say we have a sequence of random variables $X_0, X_1, X_2, \cdots$. In our studies up until now, we were assuming that the distributions of these random variables were _independent and individually distributed_. If the index represents time, then this is like starting with a fresh, new independent random variable at each step.

A more interesting case is when the random variables are in some way related. But in assuming such a case, the relations can get very complex.

Markov Chains are a compromise: they are one level of complexity beyond _i.i.d._, and they come with some very useful properties. For example, think of $X_n$ as the state of a system at a particular and discrete time, like that for a wandering partical jumping from state to state.

The indexes or states can be with regards to:

* discrete time
* continuous time
* discrete space
* continuous space

For this simple introduction, we will limit ourselves to the case of _discrete time, with a finite number of states_ (discrete space), where $n \in \mathbb{Z}_{\geq 0}$.

## What is the Markov Property?

Keeping to our limitations of discrete time and space, assume that $n$ means "now" (whatever that might be). We assume $n+1$ to be the "future". Then consider that the "future" might be characterized by $P(X_{n+1} = j)$, where

\begin{align}
  P(X_{n+1} = j | X_{n} = i, X_{n} = i_{n-1}, \cdots , X_{0} = i_{0})
\end{align}

This means that the future depends upon all of the former states in existence. Very complex.

But what if we explicitly assumed a simpler model? What if we said that the future is conditionally independent of the past, _given the present_? Then we could say

\begin{align}
  P(X_{n+1} = j | X_{n} = i, X_{n} = i_{n-1}, \cdots , X_{0} = i_{0}) &= P(X_{n+1} = j | X_{n} = i)
\end{align}

That is, if this property holds, if we know the current value $X_n$, everything else in the past is irrelevant. This is the **Markov assumption**; this is the **Markov Property**.

### Definition: transition probability

The transition probability in a Markov Chain is given by $P(X_{n+1} = j | X_n = i) = q_{ij}$. If the probability in transitioning from states does not depend on time, then we say the Markov Chain is _homogeneous_.

For this simple introduction, again, we will assume the Markov Chains to be homogeneous.

Here is a graphical example of a simple 4-state Markov Chain, listing all of its transition probabilities:

### Definition: transition matrix

The _transition matrix_ is then just the matrix representation of a Markov chain's transition probabilities in the form $Q = \left[ q_{ij} \right]$in the columns, with the rows representing each possible state. Since our introductory example is for a  discrete, finite space, elements of a row are the complete listing of the probability mass function for the state in question.

For the example above, that'd be

\begin{align}
  Q &= \begin{pmatrix}
    \frac{1}{3} & \frac{2}{3} & 0 & 0 \\
    \frac{1}{2} & 0 & \frac{1}{2} & 0 \\
    0 & 0 & 0 & 1 \\
    \frac{1}{2} & 0 & \frac{1}{4} & \frac{1}{4} \\
    \end{pmatrix}
\end{align}

### Usage

We can use a Markov chain as a model to describe a system evolving over time. The case of $P(X_{n+1} = j | X_n = i) = q_{ij}$ only looks at the present $X_n$, ignoring events of the past. Using such a *first order* model might be too strong a starting assumption of conditional independence. We could, however, extend this model such that $P(X_{n+1} = j | X_n = i, X_{n-1} = k)$, to consider the last 2 states (second order), or even more past states.

There are also such things called Markov Chain Monte Carlo, where you synthesize your own Markov Chain to let you do large-scale simulations (computer-assisted, of course).

But originally, Markov came up with this idea to show there was a way to reconcile the Law of Large Numbers but relax the condition of _independent, individually distributed_, and instead of completely ignoring the past states of system, condition on the present state (first-order) to predict the future. Of course, this could be extended to condition on states prior to the present as well. He wanted to go a step beyond _i.i.d._. His  experiment was with the Russian langage and its vowels and consonants, a chain of two states.

## Using the Transition Matrix

Suppose time $n, X_n$ has distribution $\vec{s}$ ($1 \times m$ row vector). Again, in our discrete, finite state example, $\vec{s}$ is simply the PMF listed out, so its entries are non-negative and sum up to 1.0.

_What is the PMF for $X_{n+1}$?_

\begin{align}
  P(X_{n+1} = j) &=  \sum_{i} P(X_{n+1} = j | X_n = i) \, P(X_n = i) &\quad \text{ by Law of Total Probability}\\
  &= \sum_{i}^{j} q_{ij} \, s_i &\quad \text{ definition of transition matrix; and definition of transition matrix row} \\
  &= \sum_{i}^{j} s_i \, q_{ij} &\quad \text{ but this is the } j^{th} \text{ entry of } \vec{s} \, Q
\end{align}

* So row vector $\vec{s} \, Q$ is the distribution at time $n+1$.
* So row vector $\vec{s} \, Q^{2}$ is the distribution at time $n+2$ ($\vec{s}\,Q$ plays the role of $\vec{s}$).
* So row vector $\vec{s} \, Q^{3}$ is the distribution at time $n+3$ ($\vec{s}\,Q^2$ plays the role of $\vec{s}$).
* and so on, and so forth...

Similarly, we can think about $m$ steps into the future.

\begin{align}
  P(X_{n+1} = j | X_n = i) &= q_{ij} &\text{ by definition; 1 step into the future } \\
  \\\\
  P(X_{n+2} = j | X_n = i) &= \sum_k P(X_{n+2} = j | X_{n+1} = k, X_n = i) \, P(X_{n+1} = k | X_n = i) &\text{ 2 steps into the future } \\
  &= \sum_k P(X_{n+2} = j | X_{n+1} = k \require{enclose} \, \enclose{horizontalstrike}{, X_n = i}) \, P(X_{n+1} = k | X_n = i) &\text{ only the present counts } \\
  &= \sum_k q_{kj} \, q_{ik} \\
  &= \sum_k q_{ik} \, q_{kj} &\text{ but this is the } (i,j) \text{ entry of } Q^2 \\
  \\\\
  P(X_{n+m} = j | X_n = i) &= \left[ q_{i,j} \in Q^m  \right]
\end{align}

So you see that powers of the transition matrix give us a lot of information!

## Stationary Distribution

#### Definition

$\vec{s}$ is stationary for the Markov chain we are considering if

\begin{align}
  \vec{s} \, Q &= \vec{s}
\end{align}

You should immediately recognize that $\vec{s}$ is an eigenvector of transition matrix $Q$. This means that if the chain starts off in $\vec{s}$, after one step into the future it remains at $\vec{s}$. Another step into the future, and we _still_ remain at $\vec{s}$, ad nauseum. If the chain starts off with the stationary distribution, it will have that distribution forever.

But that raises some questions:

1. Does a stationary distribution exist? (all elements of $\vec{s} \in \mathbb{Z}_{\ge 0}$)
1. Is it unique?
1. Does the chain converge to $\vec{s}$?
1. How can we compute it?

It turns out, the answers to questions 1 through 3 is _yes_.

Can we compute the stationary distribution _efficiently_? In certain examples of Markov chains, we can do so very easily and without using matrices!

But that is the topic of the next lecture.