# Programming Exercise 5:
# Regularized Linear Regression and Bias vs Variance

## Introduction

In this exercise, you will implement regularized linear regression and use it to study models with different bias-variance properties. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.

All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).

Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments).

```python
# used for manipulating directory paths
import os

# Scientific and vector computation for python
import numpy as np

# Plotting library
from matplotlib import pyplot

# Optimization module in scipy
from scipy import optimize

# will be used to load MATLAB mat datafile format
from scipy.io import loadmat

# library written for this exercise providing additional functions for assignment submission, and others
import utils

# define the submission/grader object for this exercise
grader = utils.Grader()

# tells matplotlib to embed plots within the notebook
%matplotlib inline
```

## Submission and Grading

After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.

| Section | Part                                             | Submitted Function                | Points |
| :-      |:-                                                |:-                                 | :-:    |
| 1       | [Regularized Linear Regression Cost Function](#section1)      | [`linearRegCostFunction`](#linearRegCostFunction) |  25    |
| 2       | [Regularized Linear Regression Gradient](#section2)           | [`linearRegCostFunction`](#linearRegCostFunction) |25      |
| 3       | [Learning Curve](#section3)                                   | [`learningCurve`](#func2)         | 20     |
| 4       | [Polynomial Feature Mapping](#section4)                       | [`polyFeatures`](#polyFeatures)          | 10     |
| 5       | [Cross Validation Curve](#section5)                           | [`validationCurve`](#validationCurve)       | 20     |
|         | Total Points                                     |                                   |100     |

You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.

At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.

## 1 Regularized Linear Regression

In the first half of the exercise, you will implement regularized linear regression to predict the amount of water flowing out of a dam using the change of water level in a reservoir. In the next half, you will go through some diagnostics of debugging learning algorithms and examine the effects of bias v.s.
variance.

### 1.1 Visualizing the dataset

We will begin by visualizing the dataset containing historical records on the change in the water level, $x$, and the amount of water flowing out of the dam, $y$. This dataset is divided into three parts:

- A **training** set that your model will learn on: `X`, `y`
- A **cross validation** set for determining the regularization parameter: `Xval`, `yval`
- A **test** set for evaluating performance. These are “unseen” examples which your model did not see during training: `Xtest`, `ytest`

Run the next cell to plot the training data. In the following parts, you will implement linear regression and use that to fit a straight line to the data and plot learning curves. Following that, you will implement polynomial regression to find a better fit to the data.

```python
# Load from ex5data1.mat, where all variables will be store in a dictionary
data = loadmat(os.path.join('Data', 'ex5data1.mat'))

# Extract train, test, validation data from dictionary
# and also convert y's form 2-D matrix (MATLAB format) to a numpy vector
X, y = data['X'], data['y'][:, 0]
Xtest, ytest = data['Xtest'], data['ytest'][:, 0]
Xval, yval = data['Xval'], data['yval'][:, 0]

# m = Number of examples
m = y.size

# Plot training data
pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1)
pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)');
```

### 1.2 Regularized linear regression cost function

Recall that regularized linear regression has the following cost function:

$$ J(\theta) = \frac{1}{2m} \left( \sum_{i=1}^m \left( h_\theta\left( x^{(i)} \right) - y^{(i)} \right)^2 \right) + \frac{\lambda}{2m} \left( \sum_{j=1}^n \theta_j^2 \right)$$

where $\lambda$ is a regularization parameter which controls the degree of regularization (thus, help preventing overfitting). The regularization term puts a penalty on the overall cost J. As the magnitudes of the model parameters $\theta_j$ increase, the penalty increases as well. Note that you should not regularize
the $\theta_0$ term.

You should now complete the code in the function `linearRegCostFunction` in the next cell. Your task is to calculate the regularized linear regression cost function. If possible, try to vectorize your code and avoid writing loops.

```python
def linearRegCostFunction(X, y, theta, lambda_=0.0):
    """
    Compute cost and gradient for regularized linear regression
    with multiple variables. Computes the cost of using theta as
    the parameter for linear regression to fit the data points in X and y.

    Parameters
    ----------
    X : array_like
        The dataset. Matrix with shape (m x n + 1) where m is the
        total number of examples, and n is the number of features
        before adding the bias term.

    y : array_like
        The functions values at each datapoint. A vector of
        shape (m, ).

    theta : array_like
        The parameters for linear regression. A vector of shape (n+1,).

    lambda_ : float, optional
        The regularization parameter.

    Returns
    -------
    J : float
        The computed cost function.

    grad : array_like
        The value of the cost function gradient w.r.t theta.
        A vector of shape (n+1, ).

    Instructions
    ------------
    Compute the cost and gradient of regularized linear regression for
    a particular choice of theta.
    You should set J to the cost and grad to the gradient.
    """
    # Initialize some useful values
    m = y.size # number of training examples

    # You need to return the following variables correctly
    J = 0
    grad = np.zeros(theta.shape)

    # ====================== YOUR CODE HERE ======================

    hyp = X @ theta

    theta_reg = np.concatenate([[0], theta[1:]]) # removes theta_0 term

    J = (1 / (2 * m)) * ((hyp - y).T @ (hyp - y)) + (lambda_ / (2 * m)) * (np.sum(theta_reg ** 2))

    grad = (1 / m) * (X.T @ (hyp - y)) + (lambda_ / m) * (theta_reg)

    # ============================================================
    return J, grad
```

When you are finished, the next cell will run your cost function using `theta` initialized at `[1, 1]`. You should expect to see an output of 303.993.

```python
theta = np.array([1, 1])
J, _ = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)

print('Cost at theta = [1, 1]:\t   %f ' % J)
print('This value should be about 303.993192)\n' % J)
```

    Cost at theta = [1, 1]:	   303.993192
    This value should be about 303.993192)

After completing a part of the exercise, you can submit your solutions for grading by first adding the function you modified to the submission object, and then sending your function to Coursera for grading.

The submission script will prompt you for your login e-mail and submission token. You can obtain a submission token from the web page for the assignment. You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.

*Execute the following cell to grade your solution to the first part of this exercise.*

```python
grader[1] = linearRegCostFunction
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

    Use token from last successful submission (rmolinamir@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |   0 /  20 |
                     Polynomial Feature Mapping |   0 /  10 |
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  50 / 100 |

### 1.3 Regularized linear regression gradient

Correspondingly, the partial derivative of the cost function for regularized linear regression is defined as:

$$
\begin{align}
& \frac{\partial J(\theta)}{\partial \theta_0} = \frac{1}{m} \sum_{i=1}^m \left( h_\theta \left(x^{(i)} \right) - y^{(i)} \right) x_j^{(i)} & \qquad \text{for } j = 0 \\
& \frac{\partial J(\theta)}{\partial \theta_j} = \left( \frac{1}{m} \sum_{i=1}^m \left( h_\theta \left( x^{(i)} \right) - y^{(i)} \right) x_j^{(i)} \right) + \frac{\lambda}{m} \theta_j & \qquad \text{for } j \ge 1
\end{align}
$$

In the function [`linearRegCostFunction`](#linearRegCostFunction) above, add code to calculate the gradient, returning it in the variable `grad`. Do not forget to re-execute the cell containing this function to update the function's definition.

When you are finished, use the next cell to  run your gradient function using theta initialized at `[1, 1]`. You should expect to see a gradient of `[-15.30, 598.250]`.

```python
theta = np.array([1, 1])
J, grad = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)

print('Gradient at theta = [1, 1]:  [{:.6f}, {:.6f}] '.format(*grad))
print(' (this value should be about [-15.303016, 598.250744])\n')
```

    Gradient at theta = [1, 1]:  [-15.303016, 598.250744]
     (this value should be about [-15.303016, 598.250744])

*You should now submit your solutions.*

```python
grader[2] = linearRegCostFunction
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

    Use token from last successful submission (rmolinamir@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |   0 /  20 |
                     Polynomial Feature Mapping |   0 /  10 |
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  50 / 100 |

### Fitting linear regression

Once your cost function and gradient are working correctly, the next cell will run the code in `trainLinearReg` (found in the module `utils.py`) to compute the optimal values of $\theta$. This training function uses `scipy`'s optimization module to minimize the cost function.

In this part, we set regularization parameter $\lambda$ to zero. Because our current implementation of linear regression is trying to fit a 2-dimensional $\theta$, regularization will not be incredibly helpful for a $\theta$ of such low dimension. In the later parts of the exercise, you will be using polynomial regression with regularization.

Finally, the code in the next cell should also plot the best fit line, which should look like the figure below.

The best fit line tells us that the model is not a good fit to the data because the data has a non-linear pattern. While visualizing the best fit as shown is one possible way to debug your learning algorithm, it is not always easy to visualize the data and model. In the next section, you will implement a function to generate learning curves that can help you debug your learning algorithm even if it is not easy to visualize the
data.

```python
# add a columns of ones for the y-intercept
X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)
theta = utils.trainLinearReg(linearRegCostFunction, X_aug, y, lambda_=0)

#  Plot fit over the data
pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1.5)
pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)')
pyplot.plot(X, np.dot(X_aug, theta), '--', lw=2);
```

## 2 Bias-variance

An important concept in machine learning is the bias-variance tradeoff. Models with high bias are not complex enough for the data and tend to underfit, while models with high variance overfit to the training data.

In this part of the exercise, you will plot training and test errors on a learning curve to diagnose bias-variance problems.

### 2.1 Learning Curves

You will now implement code to generate the learning curves that will be useful in debugging learning algorithms. Recall that a learning curve plots training and cross validation error as a function of training set size. Your job is to fill in the function `learningCurve` in the next cell, so that it returns a vector of errors for the training set and cross validation set.

To plot the learning curve, we need a training and cross validation set error for different training set sizes. To obtain different training set sizes, you should use different subsets of the original training set `X`. Specifically, for a training set size of $i$, you should use the first $i$ examples (i.e., `X[:i, :]`
and `y[:i]`).

You can use the `trainLinearReg` function (by calling `utils.trainLinearReg(...)`) to find the $\theta$ parameters. Note that the `lambda_` is passed as a parameter to the `learningCurve` function.
After learning the $\theta$ parameters, you should compute the error on the training and cross validation sets. Recall that the training error for a dataset is defined as

$$ J_{\text{train}} = \frac{1}{2m} \left[ \sum_{i=1}^m \left(h_\theta \left( x^{(i)} \right) - y^{(i)} \right)^2 \right] $$

In particular, note that the training error does not include the regularization term. One way to compute the training error is to use your existing cost function and set $\lambda$ to 0 only when using it to compute the training error and cross validation error. When you are computing the training set error, make sure you compute it on the training subset (i.e., `X[:n,:]` and `y[:n]`) instead of the entire training set. However, for the cross validation error, you should compute it over the entire cross validation set. You should store
the computed errors in the vectors error train and error val.

```python
def learningCurve(X, y, Xval, yval, lambda_=0):
    """
    Generates the train and cross validation set errors needed to plot a learning curve
    returns the train and cross validation set errors for a learning curve.

    In this function, you will compute the train and test errors for
    dataset sizes from 1 up to m. In practice, when working with larger
    datasets, you might want to do this in larger intervals.

    Parameters
    ----------
    X : array_like
        The training dataset. Matrix with shape (m x n + 1) where m is the
        total number of examples, and n is the number of features
        before adding the bias term.

    y : array_like
        The functions values at each training datapoint. A vector of
        shape (m, ).

    Xval : array_like
        The validation dataset. Matrix with shape (m_val x n + 1) where m is the
        total number of examples, and n is the number of features
        before adding the bias term.

    yval : array_like
        The functions values at each validation datapoint. A vector of
        shape (m_val, ).

    lambda_ : float, optional
        The regularization parameter.

    Returns
    -------
    error_train : array_like
        A vector of shape m. error_train[i] contains the training error for
        i examples.
    error_val : array_like
        A vecotr of shape m. error_val[i] contains the validation error for
        i training examples.

    Instructions
    ------------
    Fill in this function to return training errors in error_train and the
    cross validation errors in error_val. i.e., error_train[i] and
    error_val[i] should give you the errors obtained after training on i examples.

    Notes
    -----
    - You should evaluate the training error on the first i training
      examples (i.e., X[:i, :] and y[:i]).

      For the cross-validation error, you should instead evaluate on
      the _entire_ cross validation set (Xval and yval).

    - If you are using your cost function (linearRegCostFunction) to compute
      the training and cross validation error, you should call the function with
      the lambda argument set to 0. Do note that you will still need to use
      lambda when running the training to obtain the theta parameters.

    Hint
    ----
    You can loop over the examples with the following:

           for i in range(1, m+1):
               # Compute train/cross validation errors using training examples
               # X[:i, :] and y[:i], storing the result in
               # error_train[i-1] and error_val[i-1]
               ....
    """
    # Number of training examples
    m = y.size

    # You need to return these values correctly
    error_train = np.zeros(m)
    error_val   = np.zeros(m)

    # ====================== YOUR CODE HERE ======================

    for i in range(1, m+1):
        # Compute train/cross validation errors using training examples
        # X[:i, :] and y[:i], storing the result in
        # error_train[i-1] and error_val[i-1]
        theta_train = utils.trainLinearReg(linearRegCostFunction, X[:i, :], y[:i], lambda_ = lambda_)

        error_train[i-1], _ = linearRegCostFunction(X[:i, :], y[:i], theta_train, lambda_ = 0)

        error_val[i-1], _ = linearRegCostFunction(Xval, yval, theta_train, lambda_ = 0)

    # =============================================================
    return error_train, error_val
```

When you are finished implementing the function `learningCurve`, executing the next cell prints the learning curves and produce a plot similar to the figure below.

In the learning curve figure, you can observe that both the train error and cross validation error are high when the number of training examples is increased. This reflects a high bias problem in the model - the linear regression model is too simple and is unable to fit our dataset well. In the next section, you will implement polynomial regression to fit a better model for this dataset.

```python
X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)
Xval_aug = np.concatenate([np.ones((yval.size, 1)), Xval], axis=1)
error_train, error_val = learningCurve(X_aug, y, Xval_aug, yval, lambda_=0)

pyplot.plot(np.arange(1, m+1), error_train, np.arange(1, m+1), error_val, lw=2)
pyplot.title('Learning curve for linear regression')
pyplot.legend(['Train', 'Cross Validation'])
pyplot.xlabel('Number of training examples')
pyplot.ylabel('Error')
pyplot.axis([0, 13, 0, 150])

print('# Training Examples\tTrain Error\tCross Validation Error')
for i in range(m):
    print('  \t%d\t\t%f\t%f' % (i+1, error_train[i], error_val[i]))
```

*You should now submit your solutions.*

```python
grader[3] = learningCurve
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

    Use token from last successful submission (rmolinamir@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |  20 /  20 | Nice work!
                     Polynomial Feature Mapping |  10 /  10 | Nice work!
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  80 / 100 |

## 3 Polynomial regression

The problem with our linear model was that it was too simple for the data
and resulted in underfitting (high bias). In this part of the exercise, you will address this problem by adding more features. For polynomial regression, our hypothesis has the form:

$$
\begin{align}
h_\theta(x)  &= \theta_0 + \theta_1 \times (\text{waterLevel}) + \theta_2 \times (\text{waterLevel})^2 + \cdots + \theta_p \times (\text{waterLevel})^p \\
& = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \cdots + \theta_p x_p
\end{align}
$$

Notice that by defining $x_1 = (\text{waterLevel})$, $x_2 = (\text{waterLevel})^2$ , $\cdots$, $x_p =
(\text{waterLevel})^p$, we obtain a linear regression model where the features are the various powers of the original value (waterLevel).

Now, you will add more features using the higher powers of the existing feature $x$ in the dataset. Your task in this part is to complete the code in the function `polyFeatures` in the next cell. The function should map the original training set $X$ of size $m \times 1$ into its higher powers. Specifically, when a training set $X$ of size $m \times 1$ is passed into the function, the function should return a $m \times p$ matrix `X_poly`, where column 1 holds the original values of X, column 2 holds the values of $X^2$, column 3 holds the values of $X^3$, and so on. Note that you don’t have to account for the zero-eth power in this function.

```python
def polyFeatures(X, p):
    """
    Maps X (1D vector) into the p-th power.

    Parameters
    ----------
    X : array_like
        A data vector of size m, where m is the number of examples.

    p : int
        The polynomial power to map the features.

    Returns
    -------
    X_poly : array_like
        A matrix of shape (m x p) where p is the polynomial
        power and m is the number of examples. That is:

        X_poly[i, :] = [X[i], X[i]**2, X[i]**3 ...  X[i]**p]

    Instructions
    ------------
    Given a vector X, return a matrix X_poly where the p-th column of
    X contains the values of X to the p-th power.
    """
    # You need to return the following variables correctly.
    X_poly = np.zeros((X.shape[0], p))

    # ====================== YOUR CODE HERE ======================

    for i in range(p):
        X_poly[:, i] = X[:, 0] ** (i + 1)

    # ============================================================
    return X_poly
```

Now you have a function that will map features to a higher dimension. The next cell will apply it to the training set, the test set, and the cross validation set.

```python
p = 8

# Map X onto Polynomial Features and Normalize
X_poly = polyFeatures(X, p)
X_poly, mu, sigma = utils.featureNormalize(X_poly)
X_poly = np.concatenate([np.ones((m, 1)), X_poly], axis=1)

# Map X_poly_test and normalize (using mu and sigma)
X_poly_test = polyFeatures(Xtest, p)
X_poly_test -= mu
X_poly_test /= sigma
X_poly_test = np.concatenate([np.ones((ytest.size, 1)), X_poly_test], axis=1)

# Map X_poly_val and normalize (using mu and sigma)
X_poly_val = polyFeatures(Xval, p)
X_poly_val -= mu
X_poly_val /= sigma
X_poly_val = np.concatenate([np.ones((yval.size, 1)), X_poly_val], axis=1)

print('Normalized Training Example 1:')
X_poly[0, :]
```

    Normalized Training Example 1:

    array([ 1.        , -0.36214078, -0.75508669,  0.18222588, -0.70618991,
            0.30661792, -0.59087767,  0.3445158 , -0.50848117])

*You should now submit your solutions.*

```python
grader[4] = polyFeatures
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

    Use token from last successful submission (rmolinamir@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |  20 /  20 | Nice work!
                     Polynomial Feature Mapping |  10 /  10 | Nice work!
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  80 / 100 |

## 3.1 Learning Polynomial Regression

After you have completed the function `polyFeatures`, we will proceed to train polynomial regression using your linear regression cost function.

Keep in mind that even though we have polynomial terms in our feature vector, we are still solving a linear regression optimization problem. The polynomial terms have simply turned into features that we can use for linear regression. We are using the same cost function and gradient that you wrote for the earlier part of this exercise.

For this part of the exercise, you will be using a polynomial of degree 8. It turns out that if we run the training directly on the projected data, will not work well as the features would be badly scaled (e.g., an example with $x = 40$ will now have a feature $x_8 = 40^8 = 6.5 \times 10^{12}$). Therefore, you will
need to use feature normalization.

Before learning the parameters $\theta$ for the polynomial regression, we first call `featureNormalize` and normalize the features of the training set, storing the mu, sigma parameters separately. We have already implemented this function for you (in `utils.py` module) and it is the same function from the first exercise.

After learning the parameters $\theta$, you should see two plots generated for polynomial regression with $\lambda = 0$, which should be similar to the ones here:

You should see that the polynomial fit is able to follow the datapoints very well, thus, obtaining a low training error. The figure on the right shows that the training error essentially stays zero for all numbers of training samples. However, the polynomial fit is very complex and even drops off at the extremes. This is an indicator that the polynomial regression model is overfitting the training data and will not generalize well.

To better understand the problems with the unregularized ($\lambda = 0$) model, you can see that the learning curve  shows the same effect where the training error is low, but the cross validation error is high. There is a gap between the training and cross validation errors, indicating a high variance problem.

```python
lambda_ = 1
theta = utils.trainLinearReg(linearRegCostFunction, X_poly, y,
                             lambda_=lambda_, maxiter=55)

# Plot training data and fit
pyplot.plot(X, y, 'ro', ms=10, mew=1.5, mec='k')

utils.plotFit(polyFeatures, np.min(X), np.max(X), mu, sigma, theta, p)

pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)')
pyplot.title('Polynomial Regression Fit (lambda = %f)' % lambda_)
pyplot.ylim([-20, 50])

pyplot.figure()
error_train, error_val = learningCurve(X_poly, y, X_poly_val, yval, lambda_)
pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_val)

pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)
pyplot.xlabel('Number of training examples')
pyplot.ylabel('Error')
pyplot.axis([0, 13, 0, 100])
pyplot.legend(['Train', 'Cross Validation'])

print('Polynomial Regression (lambda = %f)\n' % lambda_)
print('# Training Examples\tTrain Error\tCross Validation Error')
for i in range(m):
    print('  \t%d\t\t%f\t%f' % (i+1, error_train[i], error_val[i]))
```

One way to combat the overfitting (high-variance) problem is to add regularization to the model. In the next section, you will get to  try different $\lambda$ parameters to see how regularization can lead to a better model.

### 3.2 Optional (ungraded) exercise: Adjusting the regularization parameter

In this section, you will get to observe how the regularization parameter affects the bias-variance of regularized polynomial regression. You should now modify the the lambda parameter and try $\lambda = 1, 100$. For each of these values, the script should generate a polynomial fit to the data and also a learning curve.

For $\lambda = 1$, the generated plots should look like the the figure below. You should see a polynomial fit that follows the data trend well (left) and a learning curve (right) showing that both the cross validation and training error converge to a relatively low value. This shows the $\lambda = 1$ regularized polynomial regression model does not have the high-bias or high-variance problems. In effect, it achieves a good trade-off between bias and variance.

For $\lambda = 100$, you should see a polynomial fit (figure below) that does not follow the data well. In this case, there is too much regularization and the model is unable to fit the training data.

*You do not need to submit any solutions for this optional (ungraded) exercise.*

### 3.3 Selecting $\lambda$ using a cross validation set

From the previous parts of the exercise, you observed that the value of $\lambda$ can significantly affect the results of regularized polynomial regression on the training and cross validation set. In particular, a model without regularization ($\lambda = 0$) fits the training set well, but does not generalize. Conversely, a model with too much regularization ($\lambda = 100$) does not fit the training set and testing set well. A good choice of $\lambda$ (e.g., $\lambda = 1$) can provide a good fit to the data.

In this section, you will implement an automated method to select the $\lambda$ parameter. Concretely, you will use a cross validation set to evaluate how good each $\lambda$ value is. After selecting the best $\lambda$ value using the cross validation set, we can then evaluate the model on the test set to estimate
how well the model will perform on actual unseen data.

Your task is to complete the code in the function `validationCurve`. Specifically, you should should use the `utils.trainLinearReg` function to train the model using different values of $\lambda$ and compute the training error and cross validation error. You should try $\lambda$ in the following range: {0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10}.

```python
def validationCurve(X, y, Xval, yval):
    """
    Generate the train and validation errors needed to plot a validation
    curve that we can use to select lambda_.

    Parameters
    ----------
    X : array_like
        The training dataset. Matrix with shape (m x n) where m is the
        total number of training examples, and n is the number of features
        including any polynomial features.

    y : array_like
        The functions values at each training datapoint. A vector of
        shape (m, ).

    Xval : array_like
        The validation dataset. Matrix with shape (m_val x n) where m is the
        total number of validation examples, and n is the number of features
        including any polynomial features.

    yval : array_like
        The functions values at each validation datapoint. A vector of
        shape (m_val, ).

    Returns
    -------
    lambda_vec : list
        The values of the regularization parameters which were used in
        cross validation.

    error_train : list
        The training error computed at each value for the regularization
        parameter.

    error_val : list
        The validation error computed at each value for the regularization
        parameter.

    Instructions
    ------------
    Fill in this function to return training errors in `error_train` and
    the validation errors in `error_val`. The vector `lambda_vec` contains
    the different lambda parameters to use for each calculation of the
    errors, i.e, `error_train[i]`, and `error_val[i]` should give you the
    errors obtained after training with `lambda_ = lambda_vec[i]`.

    Note
    ----
    You can loop over lambda_vec with the following:

          for i in range(len(lambda_vec))
              lambda = lambda_vec[i]
              # Compute train / val errors when training linear
              # regression with regularization parameter lambda_
              # You should store the result in error_train[i]
              # and error_val[i]
              ....
    """
    # Selected values of lambda (you should not change this)
    lambda_vec = [0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]

    # You need to return these variables correctly.
    error_train = np.zeros(len(lambda_vec))
    error_val = np.zeros(len(lambda_vec))

    # ====================== YOUR CODE HERE ======================

    for i in range(len(lambda_vec)):
        lambda_ = lambda_vec[i]

        # Compute train / val errors when training linear
        # regression with regularization parameter lambda_
        # You should store the result in error_train[i]
        # and error_val[i]
        theta_train = utils.trainLinearReg(linearRegCostFunction, X, y, lambda_ = lambda_)

        error_train[i], _ = linearRegCostFunction(X, y, theta_train, lambda_ = 0)

        error_val[i], _ = linearRegCostFunction(Xval, yval, theta_train, lambda_ = 0)

    # ============================================================
    return lambda_vec, error_train, error_val
```

After you have completed the code, the next cell will run your function and plot a cross validation curve of error v.s. $\lambda$ that allows you select which $\lambda$ parameter to use. You should see a plot similar to the figure below.

In this figure, we can see that the best value of $\lambda$ is around 3. Due to randomness
in the training and validation splits of the dataset, the cross validation error can sometimes be lower than the training error.

```python
lambda_vec, error_train, error_val = validationCurve(X_poly, y, X_poly_val, yval)

pyplot.plot(lambda_vec, error_train, '-o', lambda_vec, error_val, '-o', lw=2)
pyplot.legend(['Train', 'Cross Validation'])
pyplot.xlabel('lambda')
pyplot.ylabel('Error')

print('lambda\t\tTrain Error\tValidation Error')
for i in range(len(lambda_vec)):
    print(' %f\t%f\t%f' % (lambda_vec[i], error_train[i], error_val[i]))
```

*You should now submit your solutions.*

```python
grader[5] = validationCurve
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

    Use token from last successful submission (rmolinamir@gmail.com)? (Y/n): y
                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |  20 /  20 | Nice work!
                     Polynomial Feature Mapping |  10 /  10 | Nice work!
                               Validation Curve |  20 /  20 | Nice work!
                                      --------------------------------
                                                | 100 / 100 |

### 3.4  Optional (ungraded) exercise: Computing test set error

In the previous part of the exercise, you implemented code to compute the cross validation error for various values of the regularization parameter $\lambda$. However, to get a better indication of the model’s performance in the real world, it is important to evaluate the “final” model on a test set that was not used in any part of training (that is, it was neither used to select the $\lambda$ parameters, nor to learn the model parameters $\theta$). For this optional (ungraded) exercise, you should compute the test error using the best value of $\lambda$ you found. In our cross validation, we obtained a test error of 3.8599 for $\lambda = 3$.

*You do not need to submit any solutions for this optional (ungraded) exercise.*

```python
lambda_ = 3
theta = utils.trainLinearReg(linearRegCostFunction, X_poly, y,
                             lambda_=lambda_, maxiter=55)

# Plot training data and fit
pyplot.plot(X, y, 'ro', ms=10, mew=1.5, mec='k')

utils.plotFit(polyFeatures, np.min(X), np.max(X), mu, sigma, theta, p)

pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)')
pyplot.title('Polynomial Regression Fit (lambda = %f)' % lambda_)
pyplot.ylim([-20, 50])

pyplot.figure()
error_train, error_test = learningCurve(X_poly, y, X_poly_test, ytest, lambda_)
pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_test)

pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)
pyplot.xlabel('Number of training examples')
pyplot.ylabel('Error')
pyplot.axis([0, 13, 0, 100])
pyplot.legend(['Train', 'Test'])

print('Polynomial Regression (lambda = %f)\n' % lambda_)
print('# Training Examples\tTrain Error\tTest Error')
for i in range(m):
    print('  \t%d\t\t%f\t%f' % (i+1, error_train[i], error_test[i]))
```

### 3.5 Optional (ungraded) exercise: Plotting learning curves with randomly selected examples

In practice, especially for small training sets, when you plot learning curves to debug your algorithms, it is often helpful to average across multiple sets of randomly selected examples to determine the training error and cross validation error.

Concretely, to determine the training error and cross validation error for $i$ examples, you should first randomly select $i$ examples from the training set and $i$ examples from the cross validation set. You will then learn the parameters $\theta$ using the randomly chosen training set and evaluate the parameters $\theta$ on the randomly chosen training set and cross validation set. The above steps should then be repeated multiple times (say 50) and the averaged error should be used to determine the training error and cross validation error for $i$ examples.

For this optional (ungraded) exercise, you should implement the above strategy for computing the learning curves. For reference, the figure below  shows the learning curve we obtained for polynomial regression with $\lambda = 0.01$. Your figure may differ slightly due to the random selection of examples.

*You do not need to submit any solutions for this optional (ungraded) exercise.*

```python
# TODO
```


# Positronium Part I.
> "Weber potential, positronium, two-body problem. "

- toc: false
- branch: master
- badges: false
- comments: true
- author: JHM
- categories: [weber, positronium, two-body]

Today we begin the study of Weber's potential in the isolated two-body system consisting of an electron and positron pair $e^-$ and $e^+$.
We assume the particles $e^\pm$ have equal mass $m=m_{e^{\pm}}$. The reduced mass is concentrated at the centre-of-mass $\mu=m/2$.

Weber's force is attractive between the pair $e^\pm$ at all distances.

The particles $e^\pm$ do not indefinitely spiral inwards. Simulations indicate that the radial distance between $e^\pm$ stays strictly bounded between two upper and lower limits $$0 < r_{lower} \leq r \leq r_{upper} < + \infty .$$ This is rigorously proved in [Weber-Clemente, 1990](https://www.ifi.unicamp.br/~assis/Int-J-Theor-Phys-V30-p537-545(1991).pdf).

If the electron is indivisible particle, then the above two-body problem models a pair $e^-$ and $e^+$ of isolated electron and positron.

*But do the particles $e^\pm$ ever 'collide' and annihalate?*

In the standard physics textbooks, it seems well known that annihalation between $e^\pm$ occurs and two gamma rays are ejected in opposite directions when $e^\pm$. conserving momentum, etc., and converting *all* their mass into energy. Thus it's determined that two gamma rays of energy $0.511 keV$ are released, where Einstein's formula $E=m_ec^2$ is applied, where $m_e$ is the reduced mass. [ref] The annihalation of $e^\pm$ is apparently an experimental test of the validity of Einstein's "mass-energy" hypothesis.

But what does Weber's potential say about the annihalation of $e^+$ and $e^-$ ?

If we know the centre of mass has zero net force, then we can replace the positions $r_1$, $r_2$ of the particles by their relative distance $r_{12}$ from the centre of mass. This yields $$r_1=R +\frac{m_2}{m_1+m_2} r_{12}$$ and $$r_2=R -\frac{m_1}{m_1+m_2} r_{12}.$$

Applying Newton's Second Law that $F_{21}=-F_{12}$ yields the following equation for $r_{12}''$: $$ \mu . r_{12}'' = F_{21},$$ where $\mu$ is the reduced mass of the system, namely $\mu=\frac{m_1 m_2}{m_1+m_2}=0.5$.

In the following equations we use numpy.odeint to solve Weber equations of motion of the relative distance $r_{12}$. Therefore we have reduced the two-body problem to a one-body problem. This is a standard reduction.

Given the solution for $r_{12}$, how do we reconstruct the paths/positions of the particles $r_1$, $r_2$ ? Answer: via the relation $r_1=R+\frac{m_2}{m_1+m_2} r_{12}$ and $r_2=R-\frac{m_1}{m_1 + m_2}r_{12}$.

Now the relative distance $r_{12}$ is a type of radial distance, and if $r, \omega$ is spherical coordinates, then we have $$r'^2=|v|^2= x'^2+y'^2+z'^2=(r')^2+r^2 (\theta')^2. $$  The above formula is the usual $|v|^2=v_r^2+v_t^2$, and the tangent velocity $v_t$ satisfies $v_t=r\theta'$, where $\theta'$ is the angular velocity.

The conservation of angular momentum says that the angular moment $L=\mu r\times v$, where $v$ is the linear velocity of $r$, is constant along the motion. Moreover one has $$|L|=\mu r^2 \theta'.$$ Thus we find the formula $$\theta'=\frac{|L|}{\mu r^2}.$$ This implies $$T=\frac{\mu}{2}v^2= \frac{\mu}{2}[(r')^2+\frac{|r\times v|^2}{r^{2}}]$$ represents the kinetic energy of the system.

The conservation of energy says $T+U$ is *constant* along trajectories.

```python
#collapse

# Here we define basic functions.

def cross(v1, v2):
    x1, y1, z1 = v1
    x2, y2, z2 = v2
    return [y1*z2 - z1*y2, -(x1*z2 - z1*x2), x1*y2 - y1*x2 ]

def rho(rel_position):
    x,y,z = rel_position
    return (x*x+y*y+z*z)**0.5

def dot(vector1, vector2):
    x1, y1, z1 = vector1
    x2, y2, z2 = vector2
    return x1*x2+y1*y2+z1*z2

def rdot(position, vector):
    return dot(position, vector)/rho(position)

def norm(rel_velocity):
    return rho(rel_velocity)

mu=0.5 ## reduced mass of the system. We assume m1 and m2 are equal, hence mu=1/2.
c=1.0 ## speed of light constant in Weber's potential

# Define the angular momentum
def AngMom(rel_position, rel_velocity):
    return cross(rel_position, rel_velocity)

def L(rel_position, rel_velocity):
    return norm(cross(rel_position, rel_velocity))

# Linear Kinetic Energy
def T(rel_position, rel_velocity):
    vt = norm(cross(rel_position, rel_velocity))
# next formula decomposes v^2=(vr)^2+(vt)^2, where vt=r*θ'=|L|/(mu*r)
    return (mu/2)*(rdot(rel_position, rel_velocity)**2) + (mu/2)*(rho(rel_position)**-2)*(vt**2)

## Weber Potential Energy
## Negative sign given -1=q1*q2
def U(rel_position, rel_velocity):
    x,y,z = rel_position
    vx,vy,vz = rel_velocity
    rdot=dot(rel_position, rel_velocity)/rho(rel_position)
    return -(1/rho(rel_position))*(1-(rdot*rdot)/2)

```

```python
## import the basic packages
import numpy as np
from scipy.integrate import odeint, solve_ivp
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

## Integrating two-body isolated system of oppositely charged particles
## i.e. positron+electron pair.

## The product of the charges q1*q2 is factor in Weber's force law, and appears twice
## in the formula of Newton's F=ma.

def weber(t, rel_state):
    x, y, z, vx, vy, vz = rel_state

    r=(x*x + y*y + z*z)**0.5
    rdot=(x*vx+y*vy+z*vz)/r

    A=(-1)*r**-2 ## minus sign from q1*q2
    B=1-(rdot*rdot)/2
    C=(mu+((c*c*r)**-1))**-1 ## +plus instead of -minus.

    dxdt = vx
    dydt = vy
    dzdt = vz
    dvxdt = (x/r)*A*B*C
    dvydt = (y/r)*A*B*C
    dvzdt = (z/r)*A*B*C

    return [dxdt, dydt, dzdt, dvxdt, dvydt, dvzdt]

t_span = (0.0, 100.0)
t = np.arange(0.0, 100.0, 0.1)

y1=[2.0,0,0,] # initial relative position
v1=[-0.4, 0.4, 0] # initial relative velocity

result = odeint(weber, y1+v1, t, tfirst=True) #here odeint solves the weber equations of motion relative y1+v1 for t.

Energy=T(y1,v1) + U(y1,v1)

print('The initial total energy T+U is equal to:', Energy)
print('The initial angular momentum is equal to', norm(AngMom(y1,v1)))

fig = plt.figure()
ax = fig.add_subplot(1, 2, 1, projection='3d')
ax.plot(result[:, 0],
        result[:, 1],
        result[:, 2])
ax.set_title("position")

ax = fig.add_subplot(1, 2, 2, projection='3d')
ax.plot(result[:, 3],
        result[:, 4],
        result[:, 5])
ax.set_title("velocity")

```

What does the above plot demonstrate?

It reveals a precession motion around the centre of mass. This is not predicted by Coulomb's force, which bounds the trajectories to elliptical orbits like Newton's Law of Gravitation.

The force is central, therefore we have conservation of angular momentum, and implies the system is constrained to a plane, namely orthogonal to the angular moment of the system.

Moreover the system satisfies a conservation of linear momentum, namely the sum $T+U$ is constant.

* Problem: Verify Assis-Clemente's 1990 formula for the lower and upper limits of the relative distance along the orbits.

```python
#collapse-output

import matplotlib.pyplot
import pylab

r_list=[]
for j in range(1000):
  sample_position=[result[j,0], result[j,1], result[j,2]]
  sample_velocity=[result[j,3], result[j,4], result[j,5]]
  r_list.append(
      (
          int(j), rho(sample_position) )
      )

prelistr = list(zip(*r_list))
pylab.scatter(list(prelistr[0]),list(prelistr[1]))
pylab.xlabel('time')
pylab.ylabel('rho')
pylab.title('Solutions have Upper and Lower Limits')
pylab.show()

#TU_list=[]
#for j in range(1000):
#  sample_position=[result[j,0], result[j,1], result[j,2]]
#  sample_velocity=[result[j,3], result[j,4], result[j,5]]
#  TU_list.append(
#      (rho(sample_position),T(sample_position,sample_velocity)+U(sample_position, sample_velocity)))

#prelist1 = list(zip(*TU_list))
#pylab.scatter(list(prelist1[0]),list(prelist1[1]))
#pylab.xlabel('distance r')
#pylab.ylabel('T+U')
#pylab.title('')
#pylab.show()

# The plot below demonstrates the conservation of angular momentum.
# Note that rdot is directly equal to the sample_velocity. I.e. there is
# no need to define rdot=v.hatr/r. This was error.

#A_list=[]
#for j in range(1000):
#  sample_position=[result[j,0], result[j,1], result[j,2]]
#  sample_velocity=[result[j,3], result[j,4], result[j,5]]
#  A_list.append(
#      (rho(sample_position), norm(cross(sample_position, sample_velocity)) ) )

#prelist2 = list(zip(*A_list))
#pylab.scatter(list(prelist2[0]),list(prelist2[1]))
#pylab.xlabel('rho')
#pylab.ylabel('Angular Momentum')
#pylab.title('Conservation of Angular Momentum')
#pylab.show()

#rho_list=[]
#for j in range(180):
#    rho_list.append(
#        (int(j), rho([result[j,0], result[j,1], result[j,2]]),
#         )
#    )

```

```python
#collapse

from sympy import *
t=symbols('t')
m=symbols('m')
c=symbols('c')
r=Function('r')(t)
P=Function('P')(r,t)
F=Function('F')(r,t)

U=-(r**-1)*(1-((r.diff(t))**2)*(2*c*c)**-1)
F=(-1)*(U.diff(t))*((r.diff(t))**-1)

pprint(simplify(U))
print()
pprint(simplify(F))  ## symbolic computation of the Force law.
```

                     2
           ⎛d       ⎞
           ⎜──(r(t))⎟
       2   ⎝dt      ⎠
    - c  + ───────────
                2
    ──────────────────
          2
         c ⋅r(t)

                                      2
                            ⎛d       ⎞
                  2         ⎜──(r(t))⎟
       2         d          ⎝dt      ⎠
    - c  - r(t)⋅───(r(t)) + ───────────
                  2              2
                dt
    ───────────────────────────────────
                   2  2
                  c ⋅r (t)


# hw 6: estimators
learning objectives:
* solidify "what is an estimator"
* evaluate the effectiveness of an estimator computationally (through simulation)
* understand the notion of unbiasedness, consistency, and efficiency of an estimator and evaluate these qualities computationally (through simulation)

```julia
using StatsBase
using Random
using Statistics
using PyPlot
```

### what is the most important problem in your field?

(0) It is important to effectively communicate with people from different fields. Introduce yourself to someone in the class from a different field of study (someone you haven't met). Ask them, "what is the most important problem in your field of study?". Then, argue/explain what you think is the most important problem in *your* field of study.

Write the name of the student you spoke with and their field of study.

I'd also be interested to read about what you think is the most important problem in your field (a few sentences), but this is not required.

("field of study" can be as general as "chemistry" or as specific as "self-assembly of nanoparticles".)

### counting
(1) A biologist takes a random sample of six fish from a lake. The lake has ten distinct species of fish. It is possible for the biologist to select more than one fish of any given species. e.g., one outcome is: one fish of species $X$, two fish of species $Y$, and four fish of species $Z$. How many different samples could the biologist draw from the lake?
* the sampling is done without replacement, but this is irrelevant since the lake contains more than six fish of each species
* the order in which the fish are sampled does not matter
* fish of a given species are indistinguishable

### capture, mark, release, recapture

In ecology, one wishes to estimate the size of a population (e.g. turtles). It is too costly and impractical to count *every* member of a population. One strategy is to:
1. capture a random sample from the population, without replacement
2. mark/tag each member of this random sample
3. release the marked sample back into the population
4. after sufficient time has passed, recapture another sample
5. count the number of marked members from the recaptured sample

We assume that:
* when the captured and marked sample is released back into the wild, they randomly (homogenously) mix with the rest of the (unmarked) population before we recapture
* marking a member of the population does not change its likelihood of being recaptured
* the time between capture/mark/release and recapture/count is short enough to neglect deaths, births, and migration out of the population

define the variables:
* $n$: the total number of turtles in the population (unknown)
* $k$: number of turtles captured, marked, and released in the first phase
* $k_r$: number of turtles recaptured
* $m$: the number of recaptured turtles that had marks on them

##### Lincoln-Petersen estimator
A very intuitive estimator is found by imposing that the proportion found marked in the recaptured sample is equal to the proportion of the population that was captured/marked/released in the first phase.

\begin{equation}
\frac{k}{n}=\frac{m}{k_r}
\end{equation}

giving the Lincoln-Petersen estimator for the population size $n$:
\begin{equation}
\hat{n} = \frac{k k_r}{m}
\end{equation}

##### Chapmen estimator
Chapmen derived a different estimator that we will compare to the Lincoln-Petersen estimator below.
\begin{equation}
\hat{n} = \frac{(k + 1) (k_r +1)}{m+1} - 1
\end{equation}

#### estimating the population of turtles on a small island
We wish to estimate the population of turtles on a small island by a capture, mark, release, and recapture strategy.

(2) create a mutable data structure `Turtle` that represents a turtle in the population. It should have a single attribute, `marked`, that indicates whether it has been marked or not.

```julia

```

(3) write a function `create_population(nb_turtles::Int)` that creates a population of `nb_turtles` unmarked `Turtle`s. return the turtles as an `Array{Turtle}`.

```julia

```

(4) write a function `count_marked(turtles::Array{Turtle})` that takes in a population of turtles and returns the number of these turtles that are marked.

```julia
turtles = create_population(500)
count_marked(turtles) # should return zero
```

```julia

```

(5) write a function `capture_mark_release!` that takes in two arguments:
* `turtles::Array{Turtle}` the population of turtles
* `nb_capture_mark_release::Int` the number of turtles to randomly capture (select without replacement) and mark

and modifies the `marked` attribute of `nb_capture_mark_release` randomly selected turtles in `turtles` to denote that they have been marked. think about why the function has an `!`.

```julia
turtles = create_population(500)
capture_mark_release!(turtles, 45)
count_marked(turtles) # should return 45
```

```julia

```

(6) write a function `recapture(turtles::Array{Turtle}, nb_recapture::Int)` that returns a random sample (without replacement) of `nb_recapture` turtles from `turtles` in the form of an `Array{Turtle}`.

```julia
turtles = create_population(500)
capture_mark_release!(turtles, 45)
recaptured_turtles = recapture(turtles, 50) # should return Array{Turtle} with 50 elements
```

```julia

```

(7) write two functions, one for each estimator:

* `chapman_estimator(nb_capture_mark_release::Int, nb_recapture::Int, nb_marked_in_recaptured::Int)`
* `lincoln_petersen_estimator(nb_capture_mark_release::Int, nb_recapture::Int, nb_marked_in_recaptured::Int)`

that each take in the entire population of turtles (marked and unmarked), `turtles`, and the array of recaptured turtles, `recaptured_turtles`, and returns the respective estimate $\hat{n}$ of the number of turtles. here, `nb_capture_mark_release` is $k$, `nb_recapture` is $k_r$, and `nb_marked_in_recaptured` is $m$.

```julia

```

(8) write a function `sim_capture_mark_release_recapture` that takes in:

* `nb_turtles::Int` $=n$
* `nb_capture_mark_release::Int` $=k$
* `nb_recapture::Int` $=k_r$
* `estimator::Function` either `chapman_estimator` or `lincoln_peterson_estimator` that you wrote above

and returns $\hat{n}$, the estimate of the number of turtles in this simulation.

use all of the functions you wrote above.

```julia
nb_turtles = 200
nb_capture_mark_release = 50
nb_recapture = 42
estimator = lincoln_petersen_estimator
n̂ = sim_capture_mark_release_recapture(nb_turtles, nb_capture_mark_release,
                                            nb_recapture, estimator)
```

```julia
function sim_capture_mark_release_recapture(nb_turtles::Int, nb_capture_mark_release::Int,
                                            nb_recapture::Int, estimator::Function)
end
```

    sim_capture_mark_release_recapture (generic function with 1 method)

(9) evaluate/compare the biasedness of the Chapman and Lincoln-Peterson estimators by plotting the distribution of $\hat{n}$ over many capture, mark, release, recapture simulations. Label which histogram corresponds to which estimator (in a legend if in the same histogram panel or in a title if in two subplots). plot as a vertical line the true number of turtles for comparison. print off the average and standard deviation of $\hat{n}$ over the simulations.

```julia
nb_turtles = 200
nb_capture_mark_release = 50
nb_recapture = 42
estimators = [chapman_estimator, lincoln_petersen_estimator]
estimator_names = ["Chapman", "Lincoln-Petersen"]

nb_sims = 5000 # number of simulations
```

```julia
figure()
xlabel("estimated # turtles")
ylabel("# simulations")
```

(10) comment on which estimator, Chapman or Lincoln-Petersen, that appears to be most unbiased.

(11) evaluate the consistency of the Lincoln-Petersen estimator by plotting the simulated distribution of $\hat{n}$ for several different values of `nb_recapture`.

```julia
color_norm = PyPlot.matplotlib.colors.Normalize(vmin=50, vmax=150)
kr_to_color = PyPlot.cm.ScalarMappable(norm=color_norm, cmap=get_cmap("viridis")).to_rgba

figure()
for nb_recapture in 50:25:150

    # of course, replace rand(100)
    hist(rand(100), label="# recapture = $nb_recapture", alpha=0.4,
        density=true, color=kr_to_color(nb_recapture))
end
xlabel("estimated # turtles")
ylabel("# simulations")
legend()
```


#### Nguyễn Tiến Dũng

*CTTN Toán Tin - K62*

*20170062*

***Đại học Bách khoa Hà Nội***

---

## Phân phối dừng

Cho ma trận chuyển trạng thái $P$

Giả sử tại thời điểm $t$, $X$ có thể nhận các trạng thái $1, 2, 3,...,N$ với xác suất tương ứng là $\pi_1, \pi_2,..,\pi_N$.

Khi đó $\pi = \{\pi_1, \pi_2,...,\pi_N\}$ là vector phân phối tại thời điểm $t$. Khi đó nếu $\pi \times (I - P) = 0$ thì ta gọi vector phân phối xác suất $\pi$ là `phân phối dừng`.

----

## Phân phối giới hạn

Vector trạng thái $\pi_0 = \{\pi_1, \pi_2,...,\pi_N\}$ được gọi là có `phân phối giới hạn` nếu thỏa mãn:

\begin{aligned}
    \left\{\begin{matrix}
        \pi_1 + \pi_2 + ... + \pi_N & = 1\\
        \underset{n \to \infty}{lim}P_{ij}^{(n)} & = \pi_j, \forall i
    \end{matrix}\right.
\end{aligned}

---

### Ví dụ

Trong một thành phố có $1500000$ dân có 3 siêu thị lớn cạnh tranh nhau là `BigC`, `VinMart` và `Intimex`. Tại thời điểm ban đầu, người ta thấy có $400000$ khách vào `BigC`, $600000$ khách vào `VinMart` và $500000$ khách vào `Intimex`. Qua một thời gian người ta nhận ra rằng:
- Nếu một khách hàng vào `BigC` thì có $80\%$ họ sẽ quay lại siêu thị này, $10\%$ sang `ViMart` và $10\%$ sang `Intimex`
- Mỗi khách hàng vào `VinMart` có $90\%$ sẽ quay trở lại siêu thị này, $7\%$ chuyển sang `BigC` và $3\%$ chuyển sang `Intimex`
- Mỗi một khách hàng vào `Intimex` sẽ có $85\%$ ở lại, chuyển sang `BigC` là $8.3\%$ và chuyển sang `VinMart` là $6.7\%$

Tính số lượng khách hàng ổn định của mỗi siêu thị

---

Ta có vector phân phối trạng thái

\begin{equation}
\pi_0 = \left[\frac{4}{15}, \frac{2}{5}, \frac{1}{3}\right]
\end{equation}

Ma trận chuyển trạng thái:

\begin{bmatrix}
0.8 & 0.1 & 0.1 \\
0.07 & 0.9 & 0.03 \\
0.083 & 0.067 & 0.85
\end{bmatrix}

Ta có $\pi(P - I) = 0 \leftrightarrow \pi = \pi P \rightarrow \pi = [0.273, 0.454, 0.273]$

---

Dưới đây là đồ thị mô tả thị phần của mỗi siêu thị

```python
import processviz as pvz
```

```python
g = pvz.MarkovChain()
g.from_file('./ass1/input.csv')
```

```python
g.generate_state_graph(10)
g.state_vector
```

```python
g.generate_graph(2)
g.data
```

```python

```


# Cocco, Gomes, & Maenhout (2005)

 # "[Consumption and Portfolio Choice Over the Life Cycle](https://academic.oup.com/rfs/article-abstract/18/2/491/1599892)"

 - Notebook created by Mateo Velásquez-Giraldo and Matthew Zahn.

 This notebook uses the [Econ-ARK/HARK](https://github.com/econ-ark/hark) toolkit to describe the main results and reproduce the figures in the linked paper. The main HARK tool used here is the $\texttt{PortfolioConsumerType}$ class. For an introduction to this module, see the [ConsPortfolioModelDoc.ipynb](https://github.com/econ-ark/DemARK/blob/master/notebooks/ConsPortfolioModelDoc.ipynb) notebook.

 We thank Chris Carroll and Sylvain Catherine for comments and guidance

```python
# This cell does some preliminary set up

# Packages
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# Import relevenat HARK tools
import HARK.ConsumptionSaving.ConsPortfolioModel as cpm

# This is a jupytext paired notebook that autogenerates BufferStockTheory.py
# which can be executed from a terminal command line via "ipython BufferStockTheory.py"
# But a terminal does not permit inline figures, so we need to test jupyter vs terminal
# Google "how can I check if code is executed in the ipython notebook"
from IPython import get_ipython # In case it was run from python instead of ipython

# If the ipython process contains 'terminal' assume not in a notebook
def in_ipynb():
    try:
        if 'terminal' in str(type(get_ipython())):
            return False
        else:
            return True
    except NameError:
        return False

# Determine whether to make the figures inline (for spyder or jupyter)
# vs whatever is the automatic setting that will apply if run from the terminal
if in_ipynb():
    # %matplotlib inline generates a syntax error when run from the shell
    # so do this instead
    get_ipython().run_line_magic('matplotlib', 'inline')
else:
    get_ipython().run_line_magic('matplotlib', 'auto')
```

### The base model

The authors' aim is to represent the life cycle of a consumer that is exposed to uninsurable labor income risk and how he chooses to allocate his savings between a risky and a safe asset, without the possibility to borrow or short-sell.

##### Dynamic problem

The problem of an agent $i$ of age $t$ in the base model is recursively represented as

\begin{split}
V_{i,t} =& \max_{0\leq C_{i,t} \leq X_{i,t}, \alpha_{i,t}\in[0,1]} U(C_{i,t}) + \delta p_t E_t\{ V_{i,t+1} (X_{i,t+1}) \}\\
&\text{s.t}\\
&X_{i,t+1} = Y_{i,t+1} + (X_{i,t} - C_{i,t})(\alpha_{i,t} R_{t+1} + (1-\alpha_{i,t})\bar{R}_f)
\end{split}

where $C_{i,t}$ is consumption, $\alpha_{i,t}$ is the share of savings allocated to the risky asset, $Y_{i,t}$ is labor income, and $X_{i,t}$ represents wealth. The utility function $U(\cdot)$ is assumed to be CRRA in the base model. Extensions beyond the baseline model include an additively separable bequest motive in the utility function. The discount factor is $\delta$ and $p_t$ is the probability of survival from $t$ to $t+1$. Death is certain at a maximum period $T$.

Note that the consumer cannot borrow or short-sell.

The control variables in the problem are $\{C_{it}, \alpha_{it}\}^T_{t=1}$ and the state variables are $\{t, X_{it}, v_{it} \}^T_{t=1}$. The agent solves for  policy rules as a function of the state variables—$C_{it}(X_{it}, v_{it})$ and $\alpha_{it}(X_{it}, v_{it})$.

#### Labor income

An important driver of the paper's results is the labor income process. It is specified as follows:

\begin{equation}
\log Y_{i,t} = f(t,Z_{i,t}) + v_{i,t} + \epsilon_{i,t}, \quad \text{for } t\leq K.
\end{equation}

where $K$ is the (exogenous) age of retirement, $Z_{i,t}$ are demographic characteristics, $\epsilon_{i,t}\sim \mathcal{N}(0,\sigma^2_\epsilon)$ is a transitory shock, and  $v_{i,t}$ is a permanent component following a random walk

\begin{equation}
v_{i,t} = v_{i,t-1} + u_{i,t} = v_{i,t-1} + \xi_t + \omega_{i,t}
\end{equation}

in which the innovation is decomposed into an aggregate ($\xi_t$) and an idiosyncratic component ($\omega_{i,t}$), both following mean-0 normal distributions.

Post-retirement income is a constant fraction $\lambda$ of permanent income in the last working year $K$.

A crucial aspect of the labor income process is that $f(\cdot,\cdot)$ is calibrated to match income profiles in the PSID, capturing the usual humped shape of income across lifetime.

#### Matching labor income in HARK

In HARK's consumption-saving models, the income process takes the form
\begin{equation}
    \ln Y_t = \ln P_t + \ln \theta_t
\end{equation}
where $P_t$ represents permanent income and $\ln \theta_t \sim N(0,\sigma_\theta)$ transitory shocks to income. Permanent income evolves according to
\begin{equation}
    \ln P_{t+1} = \ln \Gamma_{t+1} +  \ln \psi_{t+1} + \ln P_t
\end{equation}
where $\Gamma_{t+1}$ is a deterministic growth factor, and $\ln \psi_{t+1} \sim N(0,\sigma_\psi)$ a permanent income shock.

To represent the author's assumptions in HARK, we express both income processes as sums of deterministic components and i.i.d shocks
\begin{align}
\text{Cocco et. al} &\quad& \ln Y_{i,t} &=& f(t,Z_{i,t}) + v_{i,0} &+&\sum_{k=1}^t u_{i,k} &+&\qquad \varepsilon_{i,t} \\
\text{HARK}        &\quad& \ln Y_{i,t} &=& \ln P_{i,0} + \sum_{k=1}^t \ln \Gamma_k &+&\sum_{k=1}^t \ln \psi_{i,k} &+& \qquad \ln \theta_{i,t}.
\end{align}

These representations make evident the mapping that we use

|HARK | Cocco et. al |
| :---: | :-----------: |
| $\ln$ $P_{i,0}$ | $f(0,Z_{i,0})$ + $v_{i,0}$ |
| $\ln$ $\Gamma_{t+1}$| $f(t+1$, $Z_{i,t+1})$ - $f(t,Z_{i,t})$|
|$\ln$ $\psi_{i,k}$| $u_{i,k}$|
|$\ln$ $\theta_{i,t}$| $\varepsilon_{i,t}$|

and to achieve a retirement income that is equal to a fraction $\lambda$ of permanent income in the last working period $K$, we simply make $\Gamma_{K+1} = \lambda$ and $\Gamma_{t} = 1$ $\forall t>K+1$.

#### Assets and their returns

There are two assets available for consumers to allocate their savings.

- Bonds: paying a risk-free return $\bar{R}_f$.

- Stocks: paying a stochastic return $R_t = \bar{R}_f + \mu + \eta_t$, where the stochastic component $\eta_t \sim \mathcal{N}(0, \sigma^2_\eta)$ is allowed to be correlated with the aggregate labor income innovation $\xi_t$.

The borrowing and short-selling constraints ensure that agents cannot allocate negative dollars to either of these assets or borrow against future labor income or retirement wealth. Recall $\alpha_{i,t}$ is the proportion of the investor's savings that are invested in the risky asset. The model's constraints imply that $\alpha_{i,t}\in[0,1]$ and wealth is non-negative.

### Calibration—Summary

The paper defines and calibrates several parameters which can be broken down into the following categories:

__1. Preferences and life cycle__

 | Parameter | Description | Code | Value |
 |:---:| ---         | ---  | :---: |
 | $\delta$ | Time Preference Factor | $\texttt{DiscFac}$ | 0.96 |
 | $\gamma$ | Coefficient of Relative Risk Aversion| $\texttt{CRRA}$ | 10 |
 | $p_t$ | Survival Probability | $\texttt{LivPrb}$ | [0.6809,0.99845] |
 | $t_0$ | Starting Age | $\texttt{t_start}$ | 20 |
 | $t_r$ | Retirement Age | $\texttt{t_ret}$ | 65 |
 | $T$ | Maximum Age | $\texttt{t_end}$ | 100 |

__2. Income process and the finanical assets__

 | Parameter | Description | Code | Value|
 |:---:| ---         | ---  | :---: |
 | $f(t,Z_{i,t})$| Average income at each stage of life | $\texttt{det_income}$ | $ \exp($0.530339 + 0.16818 $t$+ (0.0323371/10) $t^2$ + (0.0019704/100) $t^3)$ |
 | $\lambda$ | Last Period Labor Income Share for Retirement | $\texttt{repl_fac}$ | 0.68212 |
 | $\log \Gamma$ | Permanent Income Growth Factor | $\texttt{PermGroFac}$ | $\{\log f_{t+1} - \log f_t\}^{t_r+1}_{t=20}$ |
 | $\mathsf{R}$ | Interest Factor | $\texttt{Rfree}$ | 1.02 |
 | $\mu$ | Average Stock Return | $\texttt{RiskyDstnFunc}$ \& $\texttt{RiskyDrawFunc}$ | 1.06 |
 | $\sigma_\eta$ | Std Dev of Stock Returns | $\texttt{RiskyDstnFunc}$ \& $\texttt{RiskyDrawFunc}$ | 0.157 |

__3. Shocks__

 | Parameter | Description | Code | Value |
 |:---:| ---         | ---  | :---: |
 | $\sigma_v$ | Std Dev of Log Permanent Shock| $\texttt{PermShkStd}$ | 0.102956 |
 | $\sigma_\epsilon$ | Std Dev of Log Transitory Shock| $\texttt{TranShkStd}$ | 0.27166 |

For reference, the authors' source Fortran code that includes these paramerization details is available on [Gomes' personal page](http://faculty.london.edu/fgomes/research.html). Code that solves the model is also available in [Julia](https://github.com/econ-ark/HARK/issues/114#issuecomment-371891418).

### Calibration in our implementation.

We replicate the previously defined parameter values and transition processes for state variables and shocks, and format them into the structures required by HARK.

This is done in the external file [Calibration/params.py](https://github.com/matthew-zahn/CGMPort/blob/master/CGMPort-Shared/Code/Python/Calibration/params.py), which we now import.

```python
# Calibrate the model in line with the information above
import sys,os
# The following code navigates to another directory where a python script with the parameters for the model is saved.
sys.path.append(os.path.realpath('Calibration/'))
# Loading the parameters from the Calibration/params.py script
from params import dict_portfolio, time_params, det_income
```

All of the model's parameters are contained in the structure <tt>dict_portfolio<tt>, which can now be passed to HARK's <tt>PortfolioConsumerType<tt> to build and solve a representation of our problem.

```python
# Solve the model with the given parameters
agent = cpm.PortfolioConsumerType(**dict_portfolio)
agent.solve()
```

### A note on normalization

The problem as specified above makes the value function homogeneous with respect to permanent labor income. This is convenient as it allows for a re-statement of the problem in variables that are normalized by permanent income or its random components, eliminating a state variable.

The authors report (page 497) taking the normalization $v_{i,t} = 1$. This amounts to defining normalized variables $\tilde{\cdot}$ as the original variable divided by $e^{v_{i,t}-1}$. For instance:

\begin{equation}
\tilde{Y_{i,t}} = \frac{Y_{i,t}}{\exp(v_{i,t}-1)} = \frac{\exp (f(t,Z_{i,t}) + v_{i,t} + \varepsilon_{i,t})}{\exp(v_{i,t}-1)} = \exp( f(t,Z_{i,t}) + 1 + \varepsilon_{i,t} )
\end{equation}

These normalized variables have the convenient interpretation of the state that things would be in if, it weren't for permanent shocks. The author's depictions of policy functions are presented in terms of these normalized variables.

On the other hand, HARK normalizes the problem by total permanent income $P_t = \exp (f(t,Z_{i,t}) + v_{i,t})$ and its solution objects are therefore in terms of normalized variables $\hat{\cdot}$, defined as
\begin{equation}
\hat{X_{i,t}} = \frac{X_{i,t}}{P_{i,t}} = \frac{X_{i,t}}{\exp (f(t,Z_{i,t}) + v_{i,t})}.
\end{equation}

Therefore, to present our results in a way consistent with that of the original authors, we would use the following relationship
\begin{equation}
\tilde{X_{i,t}} = \hat{X_{i,t}} \times \exp (f(t,Z_{i,t})+1)
\end{equation}

However, our results are much more consistent with those of the original authors when we take the normalization $v_{i,t} = 0$, which also make sense since it makes the random-walk multiplicative part of permanent income $\exp{v_{i,t}}=1$. We therefore assume this is a typo, take $v_{i,t} = 0$, document this issue in the \textbf{Puzzles} section below, and use the relationship
\begin{equation}
\tilde{X_{i,t}} = \hat{X_{i,t}} \times \exp (f(t,Z_{i,t})).
\end{equation}

The next cell defines our normalization factor

```python
# Define a normalization factor
norm_factor = det_income*np.exp(0)
```

### Key Results

#### The optimal risky asset share

The plot below shows the policy function for the risky portfolio share as a function of wealth at different ages.

The optimal risky share is decreasing in wealth. The authors argue this is due to the fact that, at low levels of wealth, relatively safe human wealth represents a higher fraction of the consumer's wealth, so he shifts his tradeable wealth towards riskier alternatives.

Analyzing the policy rule by age also shows that the risky share increases from young to middle age, and decreases from middle to old age. This is consistent with the previous interpretation: shares trace the humped shape of labor earnings.

These estimates are different from what is produced in the original paper, which are also reproduced below. Generally, the policy functions do not share the same curvature, which leads to greater reductions in the optimal portfolio share at lower levels of wealth.

```python
# Plot portfolio rule
eevalgrid = np.linspace(0,300,100)
plt.figure()
# In the last period of life you consume everything
# so portfolio choice is irrelevant

# Ages
ages = [20,30,55,75]
age_born = time_params['Age_born']
for a in ages:
    plt.plot(eevalgrid,
             agent.solution[a-age_born].RiskyShareFunc[0][0](eevalgrid/norm_factor[a-age_born]),
             label = 'Age = %i' %(a))
plt.xlabel('Wealth')
plt.ylabel('Risky portfolio share')
plt.title('Risky Portfolio Policy Function')
plt.legend()
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

We present the author's originally reported policy functions for comparison

#### Consumption behavior

The plot below shows the policy function for consumption as a function of wealth at different ages.

At all age levels consumption increases with wealth. The consumption function also appears to shift upwards as life progresses.

Our consumption policy functions again do not match those of the original paper, which are also reproduced below. Consumption also appears to increase with age in our policy functions that does not come through in the results presented in the paper.

```python
# Plot consumption function
plt.figure()
ages = [20,35,65,85]
for a in ages:
    plt.plot(eevalgrid,
             agent.solution[a-age_born].cFunc[0][0](eevalgrid/norm_factor[a-age_born])*norm_factor[a-age_born],
             label = 'Age = %i' %(a))
plt.xlabel('Wealth')
plt.ylabel('Consumption')
plt.title('Consumption Policy Function')
plt.legend()
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

We again present the author's originally reported policy functions for comparison

### Simulations

Using the policy functions obtained from solving the model we present a series of simulations to highlight features of the model.

We first run a few simulations to verify the quality of our calibration.

The figures below show simulated levels of permanent income and risky portfolio shares for 5 agents over their life spans. We can see the model generates a heterogeneous permanent income distribution. Interestingly, all of these agents tend to follow the same general pattern for investing in the risky asset. Early in life, all of their portfolios are invested in the risky asset. This declines as the agent ages and converges to approximately 35% once they reach retirement.

```python
# Set up simulation parameters

# Number of agents and periods in the simulation.
agent.AgentCount = 5 # Number of instances of the class to be simulated.
# Since agents can die, they are replaced by a new agent whenever they do.

# Number of periods to be simulated
agent.T_sim = 80

# Set up the variables we want to keep track of.
agent.track_vars = ['aNrmNow','cNrmNow', 'pLvlNow', 't_age', 'RiskyShareNow','mNrmNow']

# Run the simulations
agent.initializeSim()
agent.simulate()

# Present diagnostic plots.
plt.figure()
plt.plot(agent.t_age_hist+time_params['Age_born'], agent.pLvlNow_hist,'.')
plt.xlabel('Age')
plt.ylabel('Permanent income')
plt.title('Simulated Income Paths')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)

plt.figure()
plt.plot(agent.t_age_hist+time_params['Age_born'], agent.RiskyShareNow_hist,'.')
plt.xlabel('Age')
plt.ylabel('Risky share')
plt.title('Simulated Risky Portfolio Shares')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

#### The average life cycle patterns

We now increase the number of simulations to examine and compare the behavior of the mean values of variables of interest at different ages, conditional on survival. In each case we present the original plots from the paper for reference.

The plot below illustrates the average dynamics of permanent income, consumption, and market resources across all of the simulated agents. The plot follows the general pattern observed in the original paper. However, our results show that the agents are accumulating significantly more market resources.

```python
# Number of agents and periods in the simulation.
agent.AgentCount = 50 # Number of instances of the class to be simulated.
# Since agents can die, they are replaced by a new agent whenever they do.

# Number of periods to be simulated
agent.T_sim = 80*50

# Run the simulations
agent.initializeSim()
agent.simulate()

raw_data = {'Age': agent.t_age_hist.flatten()+time_params['Age_born'],
            'pIncome': agent.pLvlNow_hist.flatten(),
            'rShare': agent.RiskyShareNow_hist.flatten(),
            'nrmM': agent.mNrmNow_hist.flatten(),
            'nrmC': agent.cNrmNow_hist.flatten()}

Data = pd.DataFrame(raw_data)
Data['Cons'] = Data.nrmC * Data.pIncome
Data['M'] = Data.nrmM * Data.pIncome

# Find the mean of each variable at every age
AgeMeans = Data.groupby(['Age']).mean().reset_index()
```

```python
plt.figure()
plt.plot(AgeMeans.Age, AgeMeans.pIncome,
         label = 'Income')
plt.plot(AgeMeans.Age, AgeMeans.M,
         label = 'Market resources')
plt.plot(AgeMeans.Age, AgeMeans.Cons,
         label = 'Consumption')
plt.legend()
plt.xlabel('Age')
plt.title('Variable Means Conditional on Survival')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

```python
# Find age percentiles
AgePC5 = Data.groupby(['Age']).quantile(0.05).reset_index()
AgePC95 = Data.groupby(['Age']).quantile(0.95).reset_index()

plt.figure()
plt.plot(AgeMeans.Age, AgeMeans.rShare, label = 'Mean')
plt.plot(AgePC5.Age, AgePC5.rShare, '--k')
plt.plot(AgePC95.Age, AgePC95.rShare, '--k', label = 'Perc. 5 and 95')
plt.legend()

plt.xlabel('Age')
plt.ylabel('Risky Share')
plt.title('Risky Portfolio Share Mean Conditional on Survival')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

### Other results in the original paper

#### The welfare implications of different allocation rules

The authors next conduct a welfare analysis of different allocation rules, including popular heuristics. The rules are presented in the next figure.

The utility cost of each policy in terms of constant consumption streams with respect to the authors calculated optimal policy function is reported in the next table.

Interestingly, the "no-income" column corresponds to the usual portfolio choice result of the optimal share being the quotient of excess returns and risk times relative risk aversion, disregarding labor income. The experiment shows this allocation produces substantial welfare losses.

#### Heterogeneity and sensitivity analysis

The authors also considered a number of extensions to the baseline model. These are summarized below along with their main conclusions.

- Labor income risk: Income risk may vary across employment sectors relative to the baseline model. The authors examine extreme cases for industries that have a large standard deviation and temporary income shocks. While some differences appear across sectors, the results are generally in line with the baseline model.
- Disastrous labor income shocks: The authors find that even a small probability of zero labor income lowers the optimal portfolio allocation in stocks, while the qualitative features of the baseline model are preserved.
- Uncertain retirement income: The authors consider two types of uncertainty for retirement income; it is stochastic and correlated with current stock market performance and allowing for disastrous labor income draws before retirement. The first extension has results essentially the same as the baseline case. The second leads to more conservative portfolio allocations but is broadly consistent with the baseline model.
- Endogenous borrowing constraints: The authors add borrowing to their model by building on credit-market imperfections. They find that the average investor borrows about \$5,000 and are in debt for most of their working life. The agents eventually pay off this debt and save for retirement. Relative to the benchmark model, the investor has put less of their money in their portfolio and arrive at retirement with substantially less wealth. These results are particularly pronounced at the lower end of the income distribution relative to the higher end. Additional details are available in the text.
- Bequest motive: The authors introduce a bequest motive into the agent's utility function (i.e., $b>0$). Young investors are more impatient and tend to save less for bequests. As the agent ages, savings increases and is strongest once the agent retires. This leads to effects on the agent's portfolio allocation. Taking a step-back however, these effects are not very large unless $b$ is large.
- Educational attainment: The authors generally find that savings are consistent across education groups. They note that for a given age, the importance of future income is increasing with education level. This implies that riskless asset holdings are larger for these households.
- Risk aversion and intertemporal substitution: Lowering the level of risk aversion in the model leads to changes in the optimal portfolio allocation and wealth accumulation. Less risk-averse investors accumulate less precautionary savings and invest more in risky assets.

### Conclusion

This article provides a dynamic model with accurate lifetime income profiles in which labor income increases risky asset holdings, as it is seen as a closer substitute of risk-free assets. It finds an optimal risky asset share that decreases in wealth and with age, after middle age. The model is also used to show that ignoring labor income for portfolio allocation can generate substantial welfare losses.

### Puzzles and Questions

- Table 4 says stock returns are $0.06$. They might mean that the equity premium $\mu$ is $0.06$.
- The authors report taking the normalization $v_{i,t} = 1$. However the ranges of their results seem more consistent with $v_{i,t} = 0$ so that $\exp (v_{i,t}) = 1$, which also makes more sense for interpretation.

### Robustness Analyses

Given the differences between our results and the original paper, we did a number of checks to ensure our model was behaving consistently with well-established theoretical results. Specifically we checked:
- For an infinitely lived agent with log normal returns, that their optimal portfolio allocation converges to the Campbell-Viceira (2002) approximation to the optimal portfolio share in Merton-Samuelson (1969) model.
- For an infinitely lived agent with no labor income that can only invest in a single risky asset, that their marginal propensity to consumer converges to the theoretical MPC of Merton-Samuelson (1969).
- For an agent facing no labor income risk, that their consumption patterns precisely match the results from a perfect foresight solution.

In all three cases, we verified that our HARK model holds up to these results. More details and specific results are available upon request.

As the HARK toolkit continues to develop, there are additional sensitivities that we can perform to further check the credibility of our results. Specifically, once human wealth is available in the $\texttt{PortfolioConsumerType}$ class, we can perform the following additional checks, which were kindly suggested by Professor Sylvain Catherine:
- Shut down the income risk and remove retirement income. The solution to this new problem are provided by Merton 1971. Basically, you capitalize future earnings as an endowment of risk free asset. Then the equity share should be such that Equity/(Wealth+NPV of Human capital) is the same as the equity share in Merton 1969.
- Adding back the permanent income risk and check if the equity share is consistent with Viceira 2001. Viceira tells you something like this: $\pi = \frac{\mu - r}{\gamma \sigma^2_s} + \left(\frac{\mu - r}{\gamma \sigma^2_s} - \beta_{HC} \right) \frac{HC}{W}$, where $\beta_{HC} = \frac{\text{Cov}(r_{HC},r_s)}{\text{Var}(r_s)}$. In the CGM problem it is easy to compute $\beta_{HC}$ because earnings follow a simple random walk. HC is the NPV of human capital, which you can approximate very well by discounting expected earnings by $r+\beta_{HC}*(rm-r)$.

### Bibliographic entry of the original article

```python
### Bibtex entry
'''
@article{10.1093/rfs/hhi017,
    author = {Cocco, João F. and Gomes, Francisco J. and Maenhout, Pascal J.},
    title = "{Consumption and Portfolio Choice over the Life Cycle}",
    journal = {The Review of Financial Studies},
    volume = {18},
    number = {2},
    pages = {491-533},
    year = {2005},
    month = {02},
    abstract = "{This article solves a realistically calibrated life cycle model of consumption and portfolio choice with non-tradable labor income and borrowing constraints. Since labor income substitutes for riskless asset holdings, the optimal share invested in equities is roughly decreasing over life. We compute a measure of the importance of human capital for investment behavior. We find that ignoring labor income generates large utility costs, while the cost of ignoring only its risk is an order of magnitude smaller, except when we allow for a disastrous labor income shock. Moreover, we study the implications of introducing endogenous borrowing constraints in this incomplete-markets setting.}",
    issn = {0893-9454},
    doi = {10.1093/rfs/hhi017},
    url = {https://doi.org/10.1093/rfs/hhi017},
    eprint = {http://oup.prod.sis.lan/rfs/article-pdf/18/2/491/24421441/hhi017.pdf},
}
'''
```

    '\n@article{10.1093/rfs/hhi017,\n    author = {Cocco, João F. and Gomes, Francisco J. and Maenhout, Pascal J.},\n    title = "{Consumption and Portfolio Choice over the Life Cycle}",\n    journal = {The Review of Financial Studies},\n    volume = {18},\n    number = {2},\n    pages = {491-533},\n    year = {2005},\n    month = {02},\n    abstract = "{This article solves a realistically calibrated life cycle model of consumption and portfolio choice with non-tradable labor income and borrowing constraints. Since labor income substitutes for riskless asset holdings, the optimal share invested in equities is roughly decreasing over life. We compute a measure of the importance of human capital for investment behavior. We find that ignoring labor income generates large utility costs, while the cost of ignoring only its risk is an order of magnitude smaller, except when we allow for a disastrous labor income shock. Moreover, we study the implications of introducing endogenous borrowing constraints in this incomplete-markets setting.}",\n    issn = {0893-9454},\n    doi = {10.1093/rfs/hhi017},\n    url = {https://doi.org/10.1093/rfs/hhi017},\n    eprint = {http://oup.prod.sis.lan/rfs/article-pdf/18/2/491/24421441/hhi017.pdf},\n}\n'


```python
import matplotlib.pyplot as plt
import numpy as np
import scipy as sp
import scipy.linalg
import sympy as sy
sy.init_printing()
```

```python
np.set_printoptions(precision=3)
np.set_printoptions(suppress=True)
```

```python
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
```

```python
def round_expr(expr, num_digits):
    return expr.xreplace({n : round(n, num_digits) for n in expr.atoms(sy.Number)})
```

#  Calculation of Determinant

For $2\times 2$ matrix $\pmb A$, the algorithm of determinant is
$$
\pmb A=\left[\begin{matrix} a & b\cr c & d \end{matrix}\right] \qquad\text{then}\qquad \text{det}\pmb A=ad-bc
$$

```python
a, b, c, d, e, f, g, h, i = sy.symbols('a, b, c, d, e, f, g, h, i', real = True)
```

With defined symbols, the algorithms of $2\times 2$ and $3\times 3$ determinants are

```python
A = sy.Matrix([[a, b], [c, d]])
A.det()
```

```python
B = sy.Matrix([[a, b, c], [d, e, f], [g, h, i]])
B.det()
```

##  Cofactor Expansion

 The $(i,j)$-cofactor of $\pmb A$ is denoted as $C_{ij}$ given by
 $$C_{i j}=(-1)^{i+j} \operatorname{det} A_{i j} = (-1)^{i+j} M_{ij}$$
 where $M_{ij}$ is the minor determinant which excludes $i$th row and $j$th column.

Consider a matrix $\pmb A$
$$A=\left[\begin{array}{rrr}
1 & 5 & 0 \\
2 & 4 & -1 \\
0 & -2 & 0
\end{array}\right]$$
The cofactor expansion looks like this
$$\begin{aligned}
\operatorname{det} A &=1 \cdot \operatorname{det}\left[\begin{array}{rr}
4 & -1 \\
-2 & 0
\end{array}\right]-5 \cdot \operatorname{det}\left[\begin{array}{cc}
2 & -1 \\
0 & 0
\end{array}\right]+0 \cdot \operatorname{det}\left[\begin{array}{rr}
2 & 4 \\
0 & -2
\end{array}\right] \\
&=1(0-2)-5(0-0)+0(-4-0)=-2
\end{aligned}$$

The scalers in front of each smaller determinant is the first row of $\pmb A$ and the smaller determinants are the cofactors of $\pmb A$.

In general, the expansions across $i$th row or $j$th column are
$$
\operatorname{det} A=a_{i 1} C_{i 1}+a_{i 2} C_{i 2}+\cdots+a_{i n} C_{i n}
$$
$$
\operatorname{det} A=a_{1 j} C_{1 j}+a_{2 j} C_{2 j}+\cdots+a_{n j} C_{n j}
$$

###  A SymPy Example

Consider the matrix below and perform a cofactor expansion

```python
A = sy.Matrix([[49, 0, 61], [73, 22, 96], [2, 0, 32]]);A
```

Cofactor exapansion with the column which has two zero(s):

$$\operatorname{det} A = a_{12}(-1)^{1+2}C_{12}+a_{22}(-1)^{2+2}C_{22}+a_{32}(-1)^{3+2}C_{32}$$

We can use SymPy function for calculationg minors: ```sy.matrices.MatrixDeterminant.minor(A, i, 1)```. We can define a function for cofactor row expansion:

```python
def cof_exp(matrix, c): # c is the col that we want to expand
    detA = 0
    for i in range(matrix.shape[1]): # matrix.shape[1] is the number of columns
        detA += matrix[i, c]*(-1)**(i+c)*sy.matrices.MatrixDeterminant.minor(matrix, i, c)
    return detA
```

```python
cof_exp(A,1)
```

Or we can try a random matrix with the user-defined function.

```python
B = sy.randMatrix(r = 7, min=10, max=50, percent=70);B
```

```python
cof_exp(B,1)
```

Then verify the result of using determinant method  ```.det()```. We can see indeed cofactor expansion works!

```python
B.det()
```

Minor matrices can also be extracted by using code ```sy.matrices.MatrixDeterminant.minor_submatrix()```, for instance, the $M_{23}$ matrix of $\pmb B$ is

```python
sy.matrices.MatrixDeterminant.minor_submatrix(B, 2, 3)
```

Cofactor matrix is the matrix contain all cofactors of original matrix, and function ```.cofactor_matrix()``` can do the job.

$$A=\left[\begin{array}{rrr}
C_{11} & C_{12} & C_{13} \\
C_{21} & C_{22} & C_{23} \\
C_{31} & C_{32} & C_{33}
\end{array}\right]=
\left[\begin{array}{rrr}
(-1)^{1+1}M_{11} & (-1)^{1+2}M_{12} & (-1)^{1+3}M_{13} \\
(-1)^{2+1}M_{21} & (-1)^{2+2}M_{22} & (-1)^{2+3}M_{23} \\
(-1)^{3+1}M_{31} & (-1)^{3+2}M_{32} & (-1)^{3+3}M_{33}
\end{array}\right]
$$

```python
sy.matrices.MatrixDeterminant.cofactor_matrix(A)
```

##  Triangular Matrix

If $\pmb A$ is triangular matrix, we can use cofactor expansion repetitively, the determinant of the matrix will be the product of the elements on the principal diagonal.

$$
\operatorname{det}A_{n\times n} = \prod_{i=1}^n A_{ii}
$$

where $A_{ii}$ is the diagonal element.

Let's generate a random upper triangluar matrix.

```python
A = np.round(np.random.rand(5,5)*100)
A_triu = np.triu(A); A_triu
```

    array([[23., 74., 71., 67., 21.],
           [ 0., 44., 54., 43., 74.],
           [ 0.,  0., 64., 82., 91.],
           [ 0.,  0.,  0., 30., 19.],
           [ 0.,  0.,  0.,  0., 46.]])

```python
np.linalg.det(A_triu)
```

Extract the diagonal by using ```np.diag()```, then calculate the product. It works as well!

```python
A_diag = np.diag(A_triu)
np.prod(A_diag)
```

##  Properties of Determinants

There are many properties of determinants, however they are mostly derived facts from cofactor expansion.

1. Let $A$ be an $n×n$ square matrix. If one row of $A$ is multiplied by $k$ to produce the matrix $B$, then: $\text{det} B = k\, \text{det} A$.
2. Let $A$ be an $n×n$ square matrix. If two rows of $A$ are interchanged to produce a matrix $B$ , then $\text{det}B=-\text{det}A$.
3. Let $A$ be an $n×n$ square matrix. If a multiple of one row of $A$ is added to another row to produce the matrix $B$, then $\text{det}A=\text{det}B$.
4. If $A$ is an $n×n$ matrix, then $\text{det}A^T = \text{det}A$
5. A square matrix $A$ is invertible if and only if $\text{det}A\neq 0$
6. If $A$ and $B$ are $n×n$ matrices, then $\text{det}AB = (\text{det}{A})(\text{det}{B})$
7. If $A$ is an $n×n$ matrix and $k$ is a scalar, then $\text{det}\,kA=k^n\,\text{det}A$.
8. If $A$ is an invertible square matrix, then $\text{det}A^{-1}=\frac{1}{\det A}$.

All of them are straightforward, the key is to understand them in terms of cofactor expansion.

Proof of property 6:
$$\begin{aligned}
|A B| &=\left|E_{p} \cdots E_{1} B\right|=\left|E_{p}\right|\left|E_{p-1} \cdots E_{1} B\right|=\cdots \\
&=\left|E_{p}\right| \cdots\left|E_{1}\right||B|=\cdots=\left|E_{p} \cdots E_{1}\right||B| \\
&=|A||B|
\end{aligned}$$

Proof of property 7:

Because $\text{det} B = k\, \text{det} A$, one row of $A$ is multiplied by $k$ to produce $B$.Then multiply all the rows of $A$ by $k$, there will be $n$ $k$'s in front of $\text{det} A$, which is $k^n \text{det} A$

Proof of property 8:
$$\begin{aligned}
&\begin{aligned}
A A^{-1} &=I \\
\left|A A^{-1}\right| &=|I|
\end{aligned}\\
&|A|\left|A^{-1}\right|=1\\
&\left|A^{-1}\right|=\frac{1}{|A|}
\end{aligned}$$

These properties are very useful in analytical derivation of some other theorems, however not useful in numerical sense.

#  Cramer's Rule

For any $n\times n$ matrix $A$ and any $b$, denote $A_i(b)$ the matrix obtained from $A$ by replacing column $i$ by the vector $b$.

$$A_{i}(\mathbf{b})=\left[\begin{array}{lllll}
\mathbf{a}_{1} & \cdots & \mathbf{b} & \cdots & \mathbf{a}_{n}
\end{array}\right]$$

The Cramer's Rule is
$$x_{i}=\frac{\operatorname{det} A_{i}(\mathbf{b})}{\operatorname{det} A}, \quad i=1,2, \ldots, n$$

Proof：

$$\begin{aligned}
A \cdot I_{i}(\mathbf{x}) &=A\left[\mathbf{e}_{1} \quad \cdots \quad \mathbf{x} \quad \cdots \quad \mathbf{e}_{n}\right]=\left[\begin{array}{llllll}
A \mathbf{e}_{1} & \cdots & A \mathbf{x} & \cdots & A \mathbf{e}_{n}
\end{array}\right] \\
&=\left[\begin{array}{llllll}
\mathbf{a}_{1} & \cdots & \mathbf{b} & \cdots & \mathbf{a}_{n}
\end{array}\right]=A_{i}(\mathbf{b})
\end{aligned}$$

Use determinant's property,
$$(\operatorname{det} A)\left(\operatorname{det} I_{i}(\mathbf{x})\right)=\operatorname{det} A_{i}(\mathbf{b})$$

$\text{det}I_{i}(\mathbf{x})$ is $x_i$, by using cofactor expansion.Then we proved the Cramer's Rule.

##  A NumPy Example

Solve system
$$\begin{aligned}
&2 x-y+3 z=-3\\
&3 x+3 y-z=10\\
&-x-y+z=-4
\end{aligned}$$

Now we have many ways of solving the system, start from Cramer's Rule.

```python
A = np.array([[2, -1, 3], [3, 3, -1], [-1, -1, 1]])
b = np.array([-3, 10, -4])
```

```python
A_1b = np.copy(A)
A_1b[:,0]=b

A_2b = np.copy(A)
A_2b[:,1]=b

A_3b = np.copy(A)
A_3b[:,2]=b
```

```python
x1 = np.linalg.det(A_1b)/np.linalg.det(A)
x2 = np.linalg.det(A_2b)/np.linalg.det(A)
x3 = np.linalg.det(A_3b)/np.linalg.det(A)
(x1, x2, x3)
```

Next we try built-in ```np.linalg.solve```.

```python
np.linalg.solve(A, b)
```

    array([ 1.,  2., -1.])

Or we could use $A^{-1}b$

```python
np.linalg.inv(A)@b
```

    array([ 1.,  2., -1.])

However, remember that Cramer's rule is rarely carried out in practice when solving system of equation, since computational cost (measured by the number of floating operations,  flops) are much higher than Guassian-Jordon elimination.

#  A Formula For $A^{-1}$

An alternative algorithm for $A^{-1}$ is
$$A^{-1}=\frac{1}{\operatorname{det} A}\left[\begin{array}{cccc}
C_{11} & C_{21} & \cdots & C_{n 1} \\
C_{12} & C_{22} & \cdots & C_{n 2} \\
\vdots & \vdots & & \vdots \\
C_{1 n} & C_{2 n} & \cdots & C_{n n}
\end{array}\right]$$

where the matrix of cofactors on RHS is the adjugate or classical adjoint matrix, SymPy function is ```sy.matrices.MatrixDeterminant.adjugate```. And this is the transpose of the cofactor matrix which we computed using ```sy.matrices.MatrixDeterminant.cofactor_matrix()```

##  A SymPy Example

```python
A = sy.randMatrix(5, min=-5, max = 5, percent = 80); A
```

```python
A_adjugate = sy.matrices.MatrixDeterminant.adjugate(A); A_adjugate
```

Let's verify if this is really the adjugate of $A$, we pick element of $(1, 3), (2, 4), (5, 4)$

```python
(-1)**(1+3) * sy.matrices.MatrixDeterminant.minor(A, 0, 2) # transpose the index
(-1)**(2+4) * sy.matrices.MatrixDeterminant.minor(A, 3, 1)
(-1)**(5+4) * sy.matrices.MatrixDeterminant.minor(A, 3, 4)
```

Adjugate is the transpose of cofactor matrix, thus we reverse the row and column index above when referring to the elements in mind.

The ```sy.N()``` is for converting to float approximation, i.e. we don't want to see fractions.

```python
A_det = A.det()
A_inv = (1/A_det)*A_adjugate; round_expr(sy.N(A_inv), 4)
```

```python
round_expr(sy.N(A.inv()), 4)
```

We have shown that are the same.

```python
A_inv-A.inv()
```

##  The Short Proof

We define $x$ the $j$th column of $A^{-1}$ which satifies
$$
Ax= e_j
$$

and $e_j$ is the $j$th column of identity matrix, and $j$th entry of $x$ is the $(i,j)$-entry of $A^{-1}$.By Cramer's rule,

$$\left\{(i, j) \text { -entry of } A^{-1}\right\}=x_{i}=\frac{\operatorname{det} A_{i}\left(\mathbf{e}_{j}\right)}{\operatorname{det} A}$$

The cofactor expansion down column $i$ of $A_i(e_j)$,
$$\operatorname{det} A_{i}\left(\mathbf{e}_{j}\right)=(-1)^{i+j} \operatorname{det} A_{j i}=C_{j i}$$


```python
import numpy as np
import sympy as sp
import pandas as pd
import math
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split

data=pd.read_csv('C:\\Users\\Utsav\\Desktop\\pulsar_prediction\\pulsar_stars.csv', sep=',',header=0)

pulsarData=data.values
pulsarData=np.array(pulsarData)
```

```python
#split the data

train, test = train_test_split(data, test_size=0.2)
pulsar=np.array(train)
train=np.array(train)
test=np.array(test)

print(len(test))
print(len(train))
#print(train)
```

    3580
    14318

```python
#copy labels of test & train data elsewhere
#replace all test and train labels by 1 (for intercept term)

y_train=np.zeros(len(train))
y_test=np.zeros(len(test))
for i in range(len(test)):
    y_test[i]=test[i][8]
    test[i][8]=1

for i in range(len(train)):
    y_train[i]=train[i][8]
    train[i][8]=1

```

```python
%%time
#logistic regression using sklearn
from sklearn.linear_model import LogisticRegression
# from sklearn import metrics

logreg = LogisticRegression()
logreg.fit(train, y_train)

y_pred_prob = logreg.predict_proba(test)

# y_pred = logreg.predict(test)
y_pred = np.zeros(len(y_pred_prob))

print(len(y_pred_prob))
print(y_pred_prob[2500][1])
print("################")

```

    3580
    0.001196852554975546
    ################
    Wall time: 207 ms

```python
threshold = [0.1, 0.2, 0.3, 0.4, 0.5]
threshold = np.array(threshold)

scr = np.zeros(5)
```

```python
for i in threshold:
    for j in range(len(y_pred_prob)):
        if(y_pred_prob[j][1] >= i):
            y_pred[j] = 1
        else:
            y_pred[j] = 0

    score=0
    for k in range(len(test)):
        if y_pred[k]==y_test[k]:
            score=score+1
    score=score*100/len(test)
    scr[int(10*i)-1] = score

```

```python
import matplotlib.pyplot as plt
plt.plot(threshold, scr)
plt.ylabel('accuracy')
plt.xlabel('threshold')
plt.show()
```

We get maximum accuarcy if we set threshold to 0.3 instead of 0.5

```python
from sklearn.metrics import roc_auc_score

roc_auc_score(y_test, y_pred_prob[:,1])
```

    0.9823780636269884

```python

```