# Programming Exercise 5:
# Regularized Linear Regression and Bias vs Variance

## Introduction

In this exercise, you will implement regularized linear regression and use it to study models with different bias-variance properties. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.

All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).

Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments).

```python
# used for manipulating directory paths
import os

# Scientific and vector computation for python
import numpy as np

# Plotting library
from matplotlib import pyplot

# Optimization module in scipy
from scipy import optimize

# will be used to load MATLAB mat datafile format
from scipy.io import loadmat

# library written for this exercise providing additional functions for assignment submission, and others
import utils

# define the submission/grader object for this exercise
grader = utils.Grader()

# tells matplotlib to embed plots within the notebook
%matplotlib inline
```

## Submission and Grading

After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.

| Section | Part                                             | Submitted Function                | Points |
| :-      |:-                                                |:-                                 | :-:    |
| 1       | [Regularized Linear Regression Cost Function](#section1)      | [`linearRegCostFunction`](#linearRegCostFunction) |  25    |
| 2       | [Regularized Linear Regression Gradient](#section2)           | [`linearRegCostFunction`](#linearRegCostFunction) |25      |
| 3       | [Learning Curve](#section3)                                   | [`learningCurve`](#func2)         | 20     |
| 4       | [Polynomial Feature Mapping](#section4)                       | [`polyFeatures`](#polyFeatures)          | 10     |
| 5       | [Cross Validation Curve](#section5)                           | [`validationCurve`](#validationCurve)       | 20     |
|         | Total Points                                     |                                   |100     |

You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.

At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.

## 1 Regularized Linear Regression

In the first half of the exercise, you will implement regularized linear regression to predict the amount of water flowing out of a dam using the change of water level in a reservoir. In the next half, you will go through some diagnostics of debugging learning algorithms and examine the effects of bias v.s.
variance.

### 1.1 Visualizing the dataset

We will begin by visualizing the dataset containing historical records on the change in the water level, $x$, and the amount of water flowing out of the dam, $y$. This dataset is divided into three parts:

- A **training** set that your model will learn on: `X`, `y`
- A **cross validation** set for determining the regularization parameter: `Xval`, `yval`
- A **test** set for evaluating performance. These are “unseen” examples which your model did not see during training: `Xtest`, `ytest`

Run the next cell to plot the training data. In the following parts, you will implement linear regression and use that to fit a straight line to the data and plot learning curves. Following that, you will implement polynomial regression to find a better fit to the data.

```python
# Load from ex5data1.mat, where all variables will be store in a dictionary
data = loadmat(os.path.join('Data', 'ex5data1.mat'))

# Extract train, test, validation data from dictionary
# and also convert y's form 2-D matrix (MATLAB format) to a numpy vector
X, y = data['X'], data['y'][:, 0]
Xtest, ytest = data['Xtest'], data['ytest'][:, 0]
Xval, yval = data['Xval'], data['yval'][:, 0]

# m = Number of examples
m = y.size

# Plot training data
pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1)
pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)');
```

### 1.2 Regularized linear regression cost function

Recall that regularized linear regression has the following cost function:

$$ J(\theta) = \frac{1}{2m} \left( \sum_{i=1}^m \left( h_\theta\left( x^{(i)} \right) - y^{(i)} \right)^2 \right) + \frac{\lambda}{2m} \left( \sum_{j=1}^n \theta_j^2 \right)$$

where $\lambda$ is a regularization parameter which controls the degree of regularization (thus, help preventing overfitting). The regularization term puts a penalty on the overall cost J. As the magnitudes of the model parameters $\theta_j$ increase, the penalty increases as well. Note that you should not regularize
the $\theta_0$ term.

You should now complete the code in the function `linearRegCostFunction` in the next cell. Your task is to calculate the regularized linear regression cost function. If possible, try to vectorize your code and avoid writing loops.

```python
def linearRegCostFunction(X, y, theta, lambda_=0.0):
    """
    Compute cost and gradient for regularized linear regression
    with multiple variables. Computes the cost of using theta as
    the parameter for linear regression to fit the data points in X and y.

    Parameters
    ----------
    X : array_like
        The dataset. Matrix with shape (m x n + 1) where m is the
        total number of examples, and n is the number of features
        before adding the bias term.

    y : array_like
        The functions values at each datapoint. A vector of
        shape (m, ).

    theta : array_like
        The parameters for linear regression. A vector of shape (n+1,).

    lambda_ : float, optional
        The regularization parameter.

    Returns
    -------
    J : float
        The computed cost function.

    grad : array_like
        The value of the cost function gradient w.r.t theta.
        A vector of shape (n+1, ).

    Instructions
    ------------
    Compute the cost and gradient of regularized linear regression for
    a particular choice of theta.
    You should set J to the cost and grad to the gradient.
    """
    # Initialize some useful values
    m = y.size # number of training examples

    # You need to return the following variables correctly
    J = 0
    grad = np.zeros(theta.shape)

    # ====================== YOUR CODE HERE ======================
    # hyphotesis, row vector shape (m, )
    h = theta@X.T

    # cost function
    J = (1/(2*m))*np.sum((h-y)**2) + (lambda_/(2*m))*np.sum(theta[1:]**2)

    # gradients
    grad = (1/m)*(h-y)@X
    grad[1:] += (lambda_/m)*theta[1:]

    # ============================================================
    return J, grad
```

When you are finished, the next cell will run your cost function using `theta` initialized at `[1, 1]`. You should expect to see an output of 303.993.

```python
theta = np.array([1, 1])
J, _ = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)

print('Cost at theta = [1, 1]:\t   %f ' % J)
print('This value should be about 303.993192)\n' % J)
```

    Cost at theta = [1, 1]:	   303.993192
    This value should be about 303.993192)

After completing a part of the exercise, you can submit your solutions for grading by first adding the function you modified to the submission object, and then sending your function to Coursera for grading.

The submission script will prompt you for your login e-mail and submission token. You can obtain a submission token from the web page for the assignment. You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.

*Execute the following cell to grade your solution to the first part of this exercise.*

```python
grader[1] = linearRegCostFunction
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |   0 /  25 |
                                 Learning Curve |   0 /  20 |
                     Polynomial Feature Mapping |   0 /  10 |
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  25 / 100 |

### 1.3 Regularized linear regression gradient

Correspondingly, the partial derivative of the cost function for regularized linear regression is defined as:

$$
\begin{align}
& \frac{\partial J(\theta)}{\partial \theta_0} = \frac{1}{m} \sum_{i=1}^m \left( h_\theta \left(x^{(i)} \right) - y^{(i)} \right) x_j^{(i)} & \qquad \text{for } j = 0 \\
& \frac{\partial J(\theta)}{\partial \theta_j} = \left( \frac{1}{m} \sum_{i=1}^m \left( h_\theta \left( x^{(i)} \right) - y^{(i)} \right) x_j^{(i)} \right) + \frac{\lambda}{m} \theta_j & \qquad \text{for } j \ge 1
\end{align}
$$

In the function [`linearRegCostFunction`](#linearRegCostFunction) above, add code to calculate the gradient, returning it in the variable `grad`. Do not forget to re-execute the cell containing this function to update the function's definition.

When you are finished, use the next cell to  run your gradient function using theta initialized at `[1, 1]`. You should expect to see a gradient of `[-15.30, 598.250]`.

```python
theta = np.array([1, 1])
J, grad = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)

print('Gradient at theta = [1, 1]:  [{:.6f}, {:.6f}] '.format(*grad))
print(' (this value should be about [-15.303016, 598.250744])\n')
```

    Gradient at theta = [1, 1]:  [-15.303016, 598.250744]
     (this value should be about [-15.303016, 598.250744])

*You should now submit your solutions.*

```python
grader[2] = linearRegCostFunction
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |   0 /  20 |
                     Polynomial Feature Mapping |   0 /  10 |
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  50 / 100 |

### Fitting linear regression

Once your cost function and gradient are working correctly, the next cell will run the code in `trainLinearReg` (found in the module `utils.py`) to compute the optimal values of $\theta$. This training function uses `scipy`'s optimization module to minimize the cost function.

In this part, we set regularization parameter $\lambda$ to zero. Because our current implementation of linear regression is trying to fit a 2-dimensional $\theta$, regularization will not be incredibly helpful for a $\theta$ of such low dimension. In the later parts of the exercise, you will be using polynomial regression with regularization.

Finally, the code in the next cell should also plot the best fit line, which should look like the figure below.

The best fit line tells us that the model is not a good fit to the data because the data has a non-linear pattern. While visualizing the best fit as shown is one possible way to debug your learning algorithm, it is not always easy to visualize the data and model. In the next section, you will implement a function to generate learning curves that can help you debug your learning algorithm even if it is not easy to visualize the
data.

```python
# add a columns of ones for the y-intercept
X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)
theta = utils.trainLinearReg(linearRegCostFunction, X_aug, y, lambda_=0)

#  Plot fit over the data
pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1.5)
pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)')
pyplot.plot(X, np.dot(X_aug, theta), '--', lw=2);
```

## 2 Bias-variance

An important concept in machine learning is the bias-variance tradeoff. Models with high bias are not complex enough for the data and tend to underfit, while models with high variance overfit to the training data.

In this part of the exercise, you will plot training and test errors on a learning curve to diagnose bias-variance problems.

### 2.1 Learning Curves

You will now implement code to generate the learning curves that will be useful in debugging learning algorithms. Recall that a learning curve plots training and cross validation error as a function of training set size. Your job is to fill in the function `learningCurve` in the next cell, so that it returns a vector of errors for the training set and cross validation set.

To plot the learning curve, we need a training and cross validation set error for different training set sizes. To obtain different training set sizes, you should use different subsets of the original training set `X`. Specifically, for a training set size of $i$, you should use the first $i$ examples (i.e., `X[:i, :]`
and `y[:i]`).

You can use the `trainLinearReg` function (by calling `utils.trainLinearReg(...)`) to find the $\theta$ parameters. Note that the `lambda_` is passed as a parameter to the `learningCurve` function.
After learning the $\theta$ parameters, you should compute the error on the training and cross validation sets. Recall that the training error for a dataset is defined as

$$ J_{\text{train}} = \frac{1}{2m} \left[ \sum_{i=1}^m \left(h_\theta \left( x^{(i)} \right) - y^{(i)} \right)^2 \right] $$

In particular, note that the training error does not include the regularization term. One way to compute the training error is to use your existing cost function and set $\lambda$ to 0 only when using it to compute the training error and cross validation error. When you are computing the training set error, make sure you compute it on the training subset (i.e., `X[:n,:]` and `y[:n]`) instead of the entire training set. However, for the cross validation error, you should compute it over the entire cross validation set. You should store
the computed errors in the vectors error train and error val.

```python
def learningCurve(X, y, Xval, yval, lambda_=0):
    """
    Generates the train and cross validation set errors needed to plot a learning curve
    returns the train and cross validation set errors for a learning curve.

    In this function, you will compute the train and test errors for
    dataset sizes from 1 up to m. In practice, when working with larger
    datasets, you might want to do this in larger intervals.

    Parameters
    ----------
    X : array_like
        The training dataset. Matrix with shape (m x n + 1) where m is the
        total number of examples, and n is the number of features
        before adding the bias term.

    y : array_like
        The functions values at each training datapoint. A vector of
        shape (m, ).

    Xval : array_like
        The validation dataset. Matrix with shape (m_val x n + 1) where m is the
        total number of examples, and n is the number of features
        before adding the bias term.

    yval : array_like
        The functions values at each validation datapoint. A vector of
        shape (m_val, ).

    lambda_ : float, optional
        The regularization parameter.

    Returns
    -------
    error_train : array_like
        A vector of shape m. error_train[i] contains the training error for
        i examples.
    error_val : array_like
        A vecotr of shape m. error_val[i] contains the validation error for
        i training examples.

    Instructions
    ------------
    Fill in this function to return training errors in error_train and the
    cross validation errors in error_val. i.e., error_train[i] and
    error_val[i] should give you the errors obtained after training on i examples.

    Notes
    -----
    - You should evaluate the training error on the first i training
      examples (i.e., X[:i, :] and y[:i]).

      For the cross-validation error, you should instead evaluate on
      the _entire_ cross validation set (Xval and yval).

    - If you are using your cost function (linearRegCostFunction) to compute
      the training and cross validation error, you should call the function with
      the lambda argument set to 0. Do note that you will still need to use
      lambda when running the training to obtain the theta parameters.

    Hint
    ----
    You can loop over the examples with the following:

           for i in range(1, m+1):
               # Compute train/cross validation errors using training examples
               # X[:i, :] and y[:i], storing the result in
               # error_train[i-1] and error_val[i-1]
               ....
    """
    # Number of training examples
    m = y.size

    # You need to return these values correctly
    error_train = np.zeros(m)
    error_val   = np.zeros(m)

    # ====================== YOUR CODE HERE ======================

    for i in range(1, m+1):
        theta = utils.trainLinearReg(linearRegCostFunction, X[:i,:], y[:i], lambda_)
        error_train[i-1],_ = linearRegCostFunction(X[:i,:], y[:i], theta, lambda_=0.0)
        error_val[i-1],_ = linearRegCostFunction(Xval, yval, theta, lambda_=0.0)

    # =============================================================
    return error_train, error_val
```

When you are finished implementing the function `learningCurve`, executing the next cell prints the learning curves and produce a plot similar to the figure below.

In the learning curve figure, you can observe that both the train error and cross validation error are high when the number of training examples is increased. This reflects a high bias problem in the model - the linear regression model is too simple and is unable to fit our dataset well. In the next section, you will implement polynomial regression to fit a better model for this dataset.

```python
X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)
Xval_aug = np.concatenate([np.ones((yval.size, 1)), Xval], axis=1)
error_train, error_val = learningCurve(X_aug, y, Xval_aug, yval, lambda_=0)

pyplot.plot(np.arange(1, m+1), error_train, np.arange(1, m+1), error_val, lw=2)
pyplot.title('Learning curve for linear regression')
pyplot.legend(['Train', 'Cross Validation'])
pyplot.xlabel('Number of training examples')
pyplot.ylabel('Error')
pyplot.axis([0, 13, 0, 150])

print('# Training Examples\tTrain Error\tCross Validation Error')
for i in range(m):
    print('  \t%d\t\t%f\t%f' % (i+1, error_train[i], error_val[i]))
```

*You should now submit your solutions.*

```python
grader[3] = learningCurve
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |  20 /  20 | Nice work!
                     Polynomial Feature Mapping |   0 /  10 |
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  70 / 100 |

## 3 Polynomial regression

The problem with our linear model was that it was too simple for the data
and resulted in underfitting (high bias). In this part of the exercise, you will address this problem by adding more features. For polynomial regression, our hypothesis has the form:

$$
\begin{align}
h_\theta(x)  &= \theta_0 + \theta_1 \times (\text{waterLevel}) + \theta_2 \times (\text{waterLevel})^2 + \cdots + \theta_p \times (\text{waterLevel})^p \\
& = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \cdots + \theta_p x_p
\end{align}
$$

Notice that by defining $x_1 = (\text{waterLevel})$, $x_2 = (\text{waterLevel})^2$ , $\cdots$, $x_p =
(\text{waterLevel})^p$, we obtain a linear regression model where the features are the various powers of the original value (waterLevel).

Now, you will add more features using the higher powers of the existing feature $x$ in the dataset. Your task in this part is to complete the code in the function `polyFeatures` in the next cell. The function should map the original training set $X$ of size $m \times 1$ into its higher powers. Specifically, when a training set $X$ of size $m \times 1$ is passed into the function, the function should return a $m \times p$ matrix `X_poly`, where column 1 holds the original values of X, column 2 holds the values of $X^2$, column 3 holds the values of $X^3$, and so on. Note that you don’t have to account for the zero-eth power in this function.

```python
def polyFeatures(X, p):
    """
    Maps X (1D vector) into the p-th power.

    Parameters
    ----------
    X : array_like
        A data vector of size m, where m is the number of examples.

    p : int
        The polynomial power to map the features.

    Returns
    -------
    X_poly : array_like
        A matrix of shape (m x p) where p is the polynomial
        power and m is the number of examples. That is:

        X_poly[i, :] = [X[i], X[i]**2, X[i]**3 ...  X[i]**p]

    Instructions
    ------------
    Given a vector X, return a matrix X_poly where the p-th column of
    X contains the values of X to the p-th power.
    """
    # You need to return the following variables correctly.
    X_poly = np.zeros((X.shape[0], p))

    # ====================== YOUR CODE HERE ======================

    for i in range (p):

       X_poly[:,i] = X.T**(i+1)

    # ============================================================
    return X_poly
```

Now you have a function that will map features to a higher dimension. The next cell will apply it to the training set, the test set, and the cross validation set.

```python
p = 8

# Map X onto Polynomial Features and Normalize
X_poly = polyFeatures(X, p)
X_poly, mu, sigma = utils.featureNormalize(X_poly)
X_poly = np.concatenate([np.ones((m, 1)), X_poly], axis=1)

# Map X_poly_test and normalize (using mu and sigma)
X_poly_test = polyFeatures(Xtest, p)
X_poly_test -= mu
X_poly_test /= sigma
X_poly_test = np.concatenate([np.ones((ytest.size, 1)), X_poly_test], axis=1)

# Map X_poly_val and normalize (using mu and sigma)
X_poly_val = polyFeatures(Xval, p)
X_poly_val -= mu
X_poly_val /= sigma
X_poly_val = np.concatenate([np.ones((yval.size, 1)), X_poly_val], axis=1)

print('Normalized Training Example 1:')
X_poly[0, :]
```

    Normalized Training Example 1:

    array([ 1.        , -0.36214078, -0.75508669,  0.18222588, -0.70618991,
            0.30661792, -0.59087767,  0.3445158 , -0.50848117])

*You should now submit your solutions.*

```python
grader[4] = polyFeatures
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |  20 /  20 | Nice work!
                     Polynomial Feature Mapping |  10 /  10 | Nice work!
                               Validation Curve |   0 /  20 |
                                      --------------------------------
                                                |  80 / 100 |

## 3.1 Learning Polynomial Regression

After you have completed the function `polyFeatures`, we will proceed to train polynomial regression using your linear regression cost function.

Keep in mind that even though we have polynomial terms in our feature vector, we are still solving a linear regression optimization problem. The polynomial terms have simply turned into features that we can use for linear regression. We are using the same cost function and gradient that you wrote for the earlier part of this exercise.

For this part of the exercise, you will be using a polynomial of degree 8. It turns out that if we run the training directly on the projected data, will not work well as the features would be badly scaled (e.g., an example with $x = 40$ will now have a feature $x_8 = 40^8 = 6.5 \times 10^{12}$). Therefore, you will
need to use feature normalization.

Before learning the parameters $\theta$ for the polynomial regression, we first call `featureNormalize` and normalize the features of the training set, storing the mu, sigma parameters separately. We have already implemented this function for you (in `utils.py` module) and it is the same function from the first exercise.

After learning the parameters $\theta$, you should see two plots generated for polynomial regression with $\lambda = 0$, which should be similar to the ones here:

You should see that the polynomial fit is able to follow the datapoints very well, thus, obtaining a low training error. The figure on the right shows that the training error essentially stays zero for all numbers of training samples. However, the polynomial fit is very complex and even drops off at the extremes. This is an indicator that the polynomial regression model is overfitting the training data and will not generalize well.

To better understand the problems with the unregularized ($\lambda = 0$) model, you can see that the learning curve  shows the same effect where the training error is low, but the cross validation error is high. There is a gap between the training and cross validation errors, indicating a high variance problem.

```python
lambda_ = 0
theta = utils.trainLinearReg(linearRegCostFunction, X_poly, y,
                             lambda_=lambda_, maxiter=55)

# Plot training data and fit
pyplot.plot(X, y, 'ro', ms=10, mew=1.5, mec='k')

utils.plotFit(polyFeatures, np.min(X), np.max(X), mu, sigma, theta, p)

pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)')
pyplot.title('Polynomial Regression Fit (lambda = %f)' % lambda_)
pyplot.ylim([-20, 50])

pyplot.figure()
error_train, error_val = learningCurve(X_poly, y, X_poly_val, yval, lambda_)
pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_val)

pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)
pyplot.xlabel('Number of training examples')
pyplot.ylabel('Error')
pyplot.axis([0, 13, 0, 100])
pyplot.legend(['Train', 'Cross Validation'])

print('Polynomial Regression (lambda = %f)\n' % lambda_)
print('# Training Examples\tTrain Error\tCross Validation Error')
for i in range(m):
    print('  \t%d\t\t%f\t%f' % (i+1, error_train[i], error_val[i]))
```

One way to combat the overfitting (high-variance) problem is to add regularization to the model. In the next section, you will get to  try different $\lambda$ parameters to see how regularization can lead to a better model.

### 3.2 Optional (ungraded) exercise: Adjusting the regularization parameter

In this section, you will get to observe how the regularization parameter affects the bias-variance of regularized polynomial regression. You should now modify the the lambda parameter and try $\lambda = 1, 100$. For each of these values, the script should generate a polynomial fit to the data and also a learning curve.

For $\lambda = 1$, the generated plots should look like the the figure below. You should see a polynomial fit that follows the data trend well (left) and a learning curve (right) showing that both the cross validation and training error converge to a relatively low value. This shows the $\lambda = 1$ regularized polynomial regression model does not have the high-bias or high-variance problems. In effect, it achieves a good trade-off between bias and variance.

For $\lambda = 100$, you should see a polynomial fit (figure below) that does not follow the data well. In this case, there is too much regularization and the model is unable to fit the training data.

*You do not need to submit any solutions for this optional (ungraded) exercise.*

```python
lambda_ = 1
theta = utils.trainLinearReg(linearRegCostFunction, X_poly, y,
                             lambda_=lambda_, maxiter=55)

# Plot training data and fit
pyplot.plot(X, y, 'ro', ms=10, mew=1.5, mec='k')

utils.plotFit(polyFeatures, np.min(X), np.max(X), mu, sigma, theta, p)

pyplot.xlabel('Change in water level (x)')
pyplot.ylabel('Water flowing out of the dam (y)')
pyplot.title('Polynomial Regression Fit (lambda = %f)' % lambda_)
pyplot.ylim([-20, 50])

pyplot.figure()
error_train, error_val = learningCurve(X_poly, y, X_poly_val, yval, lambda_)
pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_val)

pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)
pyplot.xlabel('Number of training examples')
pyplot.ylabel('Error')
pyplot.axis([0, 13, 0, 100])
pyplot.legend(['Train', 'Cross Validation'])

print('Polynomial Regression (lambda = %f)\n' % lambda_)
print('# Training Examples\tTrain Error\tCross Validation Error')
for i in range(m):
    print('  \t%d\t\t%f\t%f' % (i+1, error_train[i], error_val[i]))
```

### 3.3 Selecting $\lambda$ using a cross validation set

From the previous parts of the exercise, you observed that the value of $\lambda$ can significantly affect the results of regularized polynomial regression on the training and cross validation set. In particular, a model without regularization ($\lambda = 0$) fits the training set well, but does not generalize. Conversely, a model with too much regularization ($\lambda = 100$) does not fit the training set and testing set well. A good choice of $\lambda$ (e.g., $\lambda = 1$) can provide a good fit to the data.

In this section, you will implement an automated method to select the $\lambda$ parameter. Concretely, you will use a cross validation set to evaluate how good each $\lambda$ value is. After selecting the best $\lambda$ value using the cross validation set, we can then evaluate the model on the test set to estimate
how well the model will perform on actual unseen data.

Your task is to complete the code in the function `validationCurve`. Specifically, you should use the `utils.trainLinearReg` function to train the model using different values of $\lambda$ and compute the training error and cross validation error. You should try $\lambda$ in the following range: {0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10}.

```python
def validationCurve(X, y, Xval, yval):
    """
    Generate the train and validation errors needed to plot a validation
    curve that we can use to select lambda_.

    Parameters
    ----------
    X : array_like
        The training dataset. Matrix with shape (m x n) where m is the
        total number of training examples, and n is the number of features
        including any polynomial features.

    y : array_like
        The functions values at each training datapoint. A vector of
        shape (m, ).

    Xval : array_like
        The validation dataset. Matrix with shape (m_val x n) where m is the
        total number of validation examples, and n is the number of features
        including any polynomial features.

    yval : array_like
        The functions values at each validation datapoint. A vector of
        shape (m_val, ).

    Returns
    -------
    lambda_vec : list
        The values of the regularization parameters which were used in
        cross validation.

    error_train : list
        The training error computed at each value for the regularization
        parameter.

    error_val : list
        The validation error computed at each value for the regularization
        parameter.

    Instructions
    ------------
    Fill in this function to return training errors in `error_train` and
    the validation errors in `error_val`. The vector `lambda_vec` contains
    the different lambda parameters to use for each calculation of the
    errors, i.e, `error_train[i]`, and `error_val[i]` should give you the
    errors obtained after training with `lambda_ = lambda_vec[i]`.

    Note
    ----
    You can loop over lambda_vec with the following:

          for i in range(len(lambda_vec))
              lambda = lambda_vec[i]
              # Compute train / val errors when training linear
              # regression with regularization parameter lambda_
              # You should store the result in error_train[i]
              # and error_val[i]
              ....
    """
    # Selected values of lambda (you should not change this)
    lambda_vec = [0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]

    # You need to return these variables correctly.
    error_train = np.zeros(len(lambda_vec))
    error_val = np.zeros(len(lambda_vec))

    # ====================== YOUR CODE HERE ======================

    for i in range (len(lambda_vec)):
        lambda_ = lambda_vec[i]
        theta = utils.trainLinearReg(linearRegCostFunction, X, y, lambda_)
        error_train[i],_ = linearRegCostFunction(X, y, theta, lambda_=0.0)
        error_val[i],_ = linearRegCostFunction(Xval, yval, theta, lambda_=0.0)

        # Compute train / val errors when training linear
        # regression with regularization parameter lambda_
        # You should store the result in error_train[i]
        # and error_val[i]

    # ============================================================
    return lambda_vec, error_train, error_val
```

After you have completed the code, the next cell will run your function and plot a cross validation curve of error v.s. $\lambda$ that allows you select which $\lambda$ parameter to use. You should see a plot similar to the figure below.

In this figure, we can see that the best value of $\lambda$ is around 3. Due to randomness
in the training and validation splits of the dataset, the cross validation error can sometimes be lower than the training error.

```python
lambda_vec, error_train, error_val = validationCurve(X_poly, y, X_poly_val, yval)

pyplot.plot(lambda_vec, error_train, '-o', lambda_vec, error_val, '-o', lw=2)
pyplot.legend(['Train', 'Cross Validation'])
pyplot.xlabel('lambda')
pyplot.ylabel('Error')

print('lambda\t\tTrain Error\tValidation Error')
for i in range(len(lambda_vec)):
    print(' %f\t%f\t%f' % (lambda_vec[i], error_train[i], error_val[i]))
```

*You should now submit your solutions.*

```python
grader[5] = validationCurve
grader.grade()
```

    Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance

                                      Part Name |     Score | Feedback
                                      --------- |     ----- | --------
    Regularized Linear Regression Cost Function |  25 /  25 | Nice work!
         Regularized Linear Regression Gradient |  25 /  25 | Nice work!
                                 Learning Curve |  20 /  20 | Nice work!
                     Polynomial Feature Mapping |  10 /  10 | Nice work!
                               Validation Curve |  20 /  20 | Nice work!
                                      --------------------------------
                                                | 100 / 100 |

### 3.4  Optional (ungraded) exercise: Computing test set error

In the previous part of the exercise, you implemented code to compute the cross validation error for various values of the regularization parameter $\lambda$. However, to get a better indication of the model’s performance in the real world, it is important to evaluate the “final” model on a test set that was not used in any part of training (that is, it was neither used to select the $\lambda$ parameters, nor to learn the model parameters $\theta$). For this optional (ungraded) exercise, you should compute the test error using the best value of $\lambda$ you found. In our cross validation, we obtained a test error of 3.8599 for $\lambda = 3$.

*You do not need to submit any solutions for this optional (ungraded) exercise.*

```python
X.shape
```

    (12, 1)

### 3.5 Optional (ungraded) exercise: Plotting learning curves with randomly selected examples

In practice, especially for small training sets, when you plot learning curves to debug your algorithms, it is often helpful to average across multiple sets of randomly selected examples to determine the training error and cross validation error.

Concretely, to determine the training error and cross validation error for $i$ examples, you should first randomly select $i$ examples from the training set and $i$ examples from the cross validation set. You will then learn the parameters $\theta$ using the randomly chosen training set and evaluate the parameters $\theta$ on the randomly chosen training set and cross validation set. The above steps should then be repeated multiple times (say 50) and the averaged error should be used to determine the training error and cross validation error for $i$ examples.

For this optional (ungraded) exercise, you should implement the above strategy for computing the learning curves. For reference, the figure below  shows the learning curve we obtained for polynomial regression with $\lambda = 0.01$. Your figure may differ slightly due to the random selection of examples.

*You do not need to submit any solutions for this optional (ungraded) exercise.*

```python

```


```python
import matplotlib.pyplot as plt
import scipy.stats as st
import seaborn as sns
import pandas as pd
from scipy.stats import norm, uniform, expon
from scipy.integrate import quad
from sympy.solvers import solve
from sympy import Symbol
import numpy as np
from pandas import Series, DataFrame
```

```python
import matplotlib.pyplot as plt
from scipy.stats import norm
import numpy as np
n = 5
m = 500
ran = np.array(norm.rvs(size=n*m))
sim = ran.reshape((n,m))
```

```python
plt.plot(sim)
plt.show()
```

```python
means = sim.mean(axis=0)
len(means)
```

    500

```python
for n in np.array([2, 5, 10, 100]):
    m = 500
    ran = np.array(norm.rvs(size=n*m))
    sim = ran.reshape((n,m))
    means = sim.mean(axis=0)
    plt.hist(sim.T, bins=20, density=True, edgecolor="black",
    facecolor="white")
    x = np.linspace(-4, 4, num=100)
    y = norm.pdf(x)
    plt.plot(x,y)
    plt.title("Histogramm sim")
    plt.show()
    sim_mean = sim.mean(axis=0)
    plt.hist(sim_mean, density=True, edgecolor="black",
    facecolor="white")
    x = np.linspace(-4, 4, num=100)
    y = norm.pdf(x, loc=0, scale=1/np.sqrt(n))
    plt.plot(x,y)
    plt.title("Histogramm sim_mean")
    plt.show()
```

```python
norm.cdf(x=95, loc=100, scale=10/np.sqrt(25))
```

    0.006209665325776132

```python
norm.cdf(x=2, loc=1, scale=2) - norm.cdf(x=0, loc=1, scale=2)
```

    0.38292492254802624

```python
norm.cdf(x=51, loc=50, scale=np.sqrt(200)) - norm.cdf(x=49, loc=50, scale=np.sqrt(200))
```

    0.05637197779701664

```python
norm.cdf(x=2, loc=1, scale=np.sqrt(4/50)) - norm.cdf(x=0, loc=1, scale=np.sqrt(4/50))
```

    0.999593047982555

```python
seriesa = Series([79.98, 80.04, 80.02, 80.04, 80.03, 80.03, 80.04, 79.97, 80.05, 80.03, 80.02, 80.00, 80.02])
seriesb = Series([80.02, 79.94, 79.98, 79.97, 79.97, 80.03, 79.95, 79.97])
```

```python
print(seriesa.mean())
print(seriesa.std()/np.sqrt(seriesa.size))
print(seriesb.mean())
print(seriesb.std()/np.sqrt(seriesb.size))
```

    80.02076923076923
    0.00664691353682878
    79.97875
    0.011090133968017208

```python
print((seriesa.std()/np.sqrt(seriesa.size))/seriesa.mean())
print((seriesb.std()/np.sqrt(seriesb.size))/seriesb.mean())
```

    8.30648542962636e-05
    0.00013866350709428702

```python
help(uniform.rvs)
```

    Help on method rvs in module scipy.stats._distn_infrastructure:

    rvs(*args, **kwds) method of scipy.stats._continuous_distns.uniform_gen instance
        Random variates of given type.

        Parameters
        ----------
        arg1, arg2, arg3,... : array_like
            The shape parameter(s) for the distribution (see docstring of the
            instance object for more information).
        loc : array_like, optional
            Location parameter (default=0).
        scale : array_like, optional
            Scale parameter (default=1).
        size : int or tuple of ints, optional
            Defining number of random variates (default is 1).
        random_state : None or int or ``np.random.RandomState`` instance, optional
            If int or RandomState, use it for drawing the random variates.
            If None, rely on ``self.random_state``.
            Default is None.

        Returns
        -------
        rvs : ndarray or scalar
            Random variates of given `size`.

```python
random = uniform.rvs(size=1000, loc=0, scale=1)
```

```python
integral = norm.cdf(1)-norm.cdf(0)
integrand = np.exp(-np.square(x)/2)
print(integral)
print(integrand)
```

    0.3413447460685429
    [3.35462628e-04 4.61959816e-04 6.32016381e-04 8.59046258e-04
     1.16002900e-03 1.55627099e-03 2.07427188e-03 2.74669361e-03
     3.61342326e-03 4.72271279e-03 6.13236949e-03 7.91095973e-03
     1.01389764e-02 1.29099076e-02 1.63311300e-02 2.05245393e-02
     2.56268178e-02 3.17892311e-02 3.91768440e-02 4.79670435e-02
     5.83472693e-02 7.05118648e-02 8.46579886e-02 1.00980559e-01
     1.19666249e-01 1.40886593e-01 1.64790334e-01 1.91495195e-01
     2.21079315e-01 2.53572656e-01 2.88948742e-01 3.27117124e-01
     3.67916978e-01 4.11112291e-01 4.56389004e-01 5.03354510e-01
     5.51539774e-01 6.00404295e-01 6.49343988e-01 6.97701953e-01
     7.44781934e-01 7.89864161e-01 8.32223097e-01 8.71146510e-01
     9.05955191e-01 9.36022558e-01 9.60793360e-01 9.79800714e-01
     9.92680728e-01 9.99184090e-01 9.99184090e-01 9.92680728e-01
     9.79800714e-01 9.60793360e-01 9.36022558e-01 9.05955191e-01
     8.71146510e-01 8.32223097e-01 7.89864161e-01 7.44781934e-01
     6.97701953e-01 6.49343988e-01 6.00404295e-01 5.51539774e-01
     5.03354510e-01 4.56389004e-01 4.11112291e-01 3.67916978e-01
     3.27117124e-01 2.88948742e-01 2.53572656e-01 2.21079315e-01
     1.91495195e-01 1.64790334e-01 1.40886593e-01 1.19666249e-01
     1.00980559e-01 8.46579886e-02 7.05118648e-02 5.83472693e-02
     4.79670435e-02 3.91768440e-02 3.17892311e-02 2.56268178e-02
     2.05245393e-02 1.63311300e-02 1.29099076e-02 1.01389764e-02
     7.91095973e-03 6.13236949e-03 4.72271279e-03 3.61342326e-03
     2.74669361e-03 2.07427188e-03 1.55627099e-03 1.16002900e-03
     8.59046258e-04 6.32016381e-04 4.61959816e-04 3.35462628e-04]

```python

```


# 3. 신경망 (Neural Network)

* Perceptron은 복잡한 함수도 표현이 가능

> ex) 컴퓨터가 수행하는 복잡한 처리도 표현 가능, 하지만 가중치(weight)를 설정하는 작업
 (원하는 결과를 출력하도록 가중치 값을 적절히 정하는 작업)은 여전히 사람이 수동으로 조정.
 이전에는 AND, OR 게이트의 logic table을 보면서 적절한 가중치 값을 정함

*  신경망(neural network)는 위와 같은 문제를 해결해줌.
 (**가중치의 매개변수의 적절한 값을 데이터로부터 자동으로 학습하는 능력이 신경망의 중요한 성질**)

-----

* 신경망을 그림으로 나타내면 가장 왼쪽(left_side)부터 입력층(input_layer), 은닉층(hidden_layer), 맨 오른쪽은 출력층(output_layer)

은닉층의 neuron은 (input,output layer와 달리) 사람의 눈에 보이지 않음, 그래서 은닉이라 불림.

 입력층 ~ 출력층 방향을 차례로 0층, 1층, 2층이라 불림.

(층 번호를 0부터 시작하는 이유는 파이썬 배열의 index도 0부터 시작하기 때문에)

# 3.0 퍼셉트론 복습

* 위의 그림과 같이 x1, x2라는 2개의 신호를 입력받아 y를 출력하는 perceptron이다.

----

\begin{align}
y = 0 \space (b + w_1x_1 + w_2x_2 \leq 0)
\end{align}
\begin{align}
y= 1 \space (b + w_1x_1 + w_2x_2 > 0)
\end{align}

* 위의 식에서 b는 **편향(bias)**을 나타내는 매개변수로 뉴런이 얼마나 쉽게 활성화되는냐를 제어한다.
 또한 w1, w2는 각 신호의 **가중치(weight)**를 나타내는 매개변수로, 각 신호의 영향력을 제어한다.

위의 식과 같이 퍼셉트론에 편향을 표시하면 위의 그림과 같다.가중치가 b이고 입력이 1인 뉴런이 추가되었다.

이 퍼셉트론의 동작은 x1, x2, 1이라는 3개의 신호가 뉴런에 입력되어, 각 신호에 가중치를 곱한 후, 다음 뉴런에 전달된다.
 **다음 뉴런에서는 이들 신호의 값을 더하여, 그 합이 0을 넘으면 1을 출력하고 그렇지 않으면 0을 출력한다.**

편향의 입력신호는 항상 1이기 때문에 그림에서는 해당 뉴런을 회색으로 채워 다른 뉴런과 구별했다.

# 3.1 활성화 함수 (Activation function)

 * **정의**: 입력 신호의 총합을 출력 신호로 변환하는 함수 / 입력 신호의 총합이 활성화를 일으키는 정함

\begin{align}
a =  b + w1 x1 + w2 x2
\end{align}
\begin{align}
y = h(a)
\end{align}

* 위의 식은 가중치가 달린 입력 신호와 편향(bias)의 총합을 계산하고 이를 a라고 한다.

* 아래의 식은 a를 함수 h()에 넣어 y를 출력하는 흐름

  위의 식들을 그림으로 표현

가중치 신호를 조합한 결과가 a라는 노드가 되고, 활성화 함수 h()를 통과하여 y라는 노드로 변환되는 과정

위에 왼쪽 그림은 일반적인 뉴런을 나타내고, 오른쪽은 활성화 처리 과정을
 명시한 뉴런(a는 입력 신호의 총합, h()는 활성화 함수, y는 출력을 의미한다.

**단순 퍼셉트론**은 단층 네트워크에서 계단 함수(임계값을 경계로 출력이 바뀌는 함수)를
 활성화 함수라 사용한 모델을 가리키고 **다층 퍼셉트론**은 신경망 (여러층으로 구성되고
 시그모이드 함수 등의 매끈한 활성화 함수를 사용하는 네트워크)을 가리킨다.

# 3.12 계단 함수 (step function)

```
def step_function(x):                          # 여기서 인수 x는 실수(부동소수점)만 받아들인다. 즉 step_function(3.0)은 되지만 step_function(np.array([1.0, 2.0]))은 안된다.
  y = x > 0                                    # 계단 함수는 입력이 0을 넘으면 1출력, 그 외에는 0을 출력
  return y.astype(np.int)
```

```
import numpy as np                             # Numpy 배열을 부등호 연산에 수행하면 배열의 원소 각각에 부등호 연산을 수행한 bool배열이 생성
import matplotlib.pylab as plt

x = np.array([-1.0,1.0,2.0])
x
```

    array([-1.,  1.,  2.])

```
y = x > 0                                      # 배열 x의 원소 각각이 0 보다 크면 True로, 0 이하면 False로 변환한 새로운 배열 y가 생성
y                                              # y는 bool 배열이다. 하지만 우리가 원하는 계단 함수는 0이나 1의 int형을 출력하는 함수. 그래서 배열y의 원소를 bool에서 int형으로 바꾼다.
```

    array([False,  True,  True])

```
y = y.astype(np.int)                          # Numpy의 배열 형 자료를 변환할 때는 astype() 메서드를 이용
y                                             # 파이썬에서는 bool을 int로 변환하면 True는 1로, False는 0으로 변환
```

    array([0, 1, 1])

```
def step_function(x):
  return np.array(x > 0, dtype=np.int)       # 계단 함수는 입력(x)가 0보다 크거나, 데이터 타입이 int로 선언했을 때 리턴값을 준다.

np.arange                                    # np.arange() 함수는 인자로 받는 값 만큼 1씩 증가하는 1차원 array를 만든다.

x = np.arange(-5.0, 5.0, 0.1)                # (-5.0 ~ 5.0까지) 0.1 간격으로 배열 생성 / numpy.arange([start, ] stop, [step, ] dtype=None)
y = step_function(x)                         # step_function()은 인수로 받은 Numpy 배열의 원소 각각을 인수로 계단 함수를 실행해, 그 결과를 다시 배열로 만듦

plt.plot(x, y)                               # plotting x & y axis
plt.ylim(-0.1, 1.1)                          # y 축의 범위 지정
plt.show()                                   # plot을 show 함수를 이용해서 시각화
                                             # 계단 함수는 0을 경계로 출력이 0에서 1(또는 1에서 0)로 바뀜 / vise versa
```

* 위의 그래프와 같이 계단 함수는 0을 경계로 출력이 0에서 1(또는 1에서 0)로 바뀐다.

# 3.12 시그모이드 함수 (Sigmoid function)

```
def sigmoid(x):
  return 1 / (1 + np.exp(-x))                     # np.exp(-x)는 exp(-x) 수식에 해당

x = np.array([-1.0, 1.0, 2.0])                    # Broadcast 기능을 이용,  Numpy 배열과 Scalar값 연산을 Numpy 배열의 원소 각각과 스칼라값의 연산으로 바꿔 수행
sigmoid(x)
```

    array([0.26894142, 0.73105858, 0.88079708])

```
t = np.array([1.0, 2.0, 3.0])                     # t에 배열(벡터)를 대입
1.0 + t                                           # Scalar값 1.0과 Numpy 배열 사이에서 수치 연산(+ / -)를 실행
```

    array([2., 3., 4.])

```
1.0 / t                                           # 결과적으로 scalar값과 Numpy 배열의 각 원소 사이에 연산이 이루어지고, 연산 결과 Numpy 배열로 출력
```

    array([1.        , 0.5       , 0.33333333])

```
x = np.arange(-5.0, 5.0, 0.1)
y = sigmoid(x)                                     # y를 출력하는 함수를 sigmoid 함수로 변경
plt.plot(x,y)                                      # x축 y축을 그려서 시각화
plt.ylim(-0.1, 1,1)                                # y축(y-axis)의 범위 지정
plt.show
```

#3.13 ReLU 함수 (ReLu function)

* ReLU (Rectified Linear Unit)는 입력이 0이 넘으면 그 입력을 그대로 출력하고, 0 이하면 0을 출력하는 함수

-----

\begin{align}
h(x) =  x \space(x>0)
\end{align}

\begin{align}
h(x) =  x \space(x\leq0)
\end{align}

* ReLU 함수를 수식으로 나타내면 위와 같다.

* ReLU에서 'Rectified' 란 정류된 이란 의미를 가지고 있음.

```
def relu(x):
    return np.maximum(0, x)                          # Numpy의 maximum 함수를 사용하여 두 입력 중 큰 값을 선택해 반환하는 함수
```

# 3.2 다차원 배열의 계산
* 다차원 배열의 기본은 '숫자의 집합'이다. 숫자가 한 줄로 늘어선 것이나 직사각형으로 늘어 놓은 것, 3차원으로 늘어놓은 것이나, N차원(4,5...)으로 나열하는 것을 통틀어 다차원 배열이라고 함

```
import numpy as np
A = np.array([1,2,3,4])                     # 1차원으로 배열로 생성
print(A)

np.ndim(A)                                  # 배열의 차원 수는 np.ndim()으로 확인이 가능
```

    [1 2 3 4]

    1

```
A.shape                                     # 배열의 형상은 인스턴스 변수인 shape로 알 수 있음
```

    (4,)

```
A.shape[0]
```

    4

```
B = np.array([[1.2],[3,4],[5,6]])
print(B)

np.ndim(B)
```

    [list([1.2]) list([3, 4]) list([5, 6])]

    1

```
B.shape
```

    (3,)

# 3.21 행렬의 내적

```
A = np.array([[1,2],[3,4]])
A.shape
```

    (2, 2)

```
B = np.array([[5,6],[7,8]])
B.shape
```

    (2, 2)

```
np.dot(A,B)                      # np.dot()으로 행렬의 곱 구현
```

    array([[19, 22],
           [43, 50]])

```
A = np.array([[1,3,5],[2,4,6]])
A.shape
```

    (2, 3)

```
B = np.array([[1,3],[2,5],[4,6,]])
B.shape
```

    (3, 2)

```
np.dot(A,B)
```

    array([[27, 48],
           [34, 62]])

```
A = np.array([[1,2],[3,4],[5,6]])
A.shape
```

    (3, 2)

```
B = np.array([7,8])
B.shape
```

    (2,)

```
np.dot(A,B)
```

    array([23, 53, 83])

# 3.22 신경망의 내적

```
X = np.array([1,2])
X.shape
```

    (2,)

행렬과 곱으로 신경망의 계산을 수행하는 법.

아래 코드의 구현엣도 X, W, Y의 형상을 봤을 때, 특히 X와 W의 대응하는 차원의 원소 수가 같아야 한다.

```
W = np.array([[1,3,5],[2,4,6]])                           # W = weight(가중치을 의미)
print(W)

W.shape
```

    [[1 3 5]
     [2 4 6]]

    (2, 3)

```
Y = np.dot(X, W)                                          # 다차원 배열의 내적을 구해주는 np.dot함수를 사용하면 단번에 결과 Y를 계산할 수 있다.
print(Y)                                                  # 만약 np.dot을 사용하지 않으면 Y의 원소를 하나 씩 따져봐야 한다(for문을 사용해서 계산)
                                                          # 그래서 행렬의 내적을 한꺼번에 계산해주는 기능은 신경망을 구현할 때 매우 중요하다.
```

    [ 5 11 17]

# 3.3 (삼층 / 3-layer) Neural Network 구현

* 입력층(0층) 2개, 1번째 은닉층(1층) 3개, 2번째 은닉층(2층) 2개, 출력층(3층) 2개로 뉴런이 구성

* 신경망에서의 계산을 행렬 계산으로 정리할 수 있다는 것을 의미. 신경망 각층의 계산은 행렬의 내적으로 처리할 수 있다.

# 각 층의 신호 전달

* 입력층에서 1층으로 신호 전달

```
X = np.array([1.0, 0.5])
W1 = np.array([[0.1, 0.3, 0.5], [0.2, 0.4, 0.6]])
B1 = np.array([0.1, 0.2, 0.3])

print(W1.shape)                  # (2, 3)
print(X.shape)                   # (2,)
print(B1.shape)                  # (3,)

A1 = np.dot(X,W1) + B1

```

    (2, 3)
    (2,)
    (3,)

* 입력층에서 1층으로 신호 전달

위의 그림과 같이 은닉층에서 가중치 합(가중 신호 편향의 총합)을 a로 표기하고 활성화 함수 h()로 변환된 신호를 z로 표기.
 여기에서 활성화 함수로 시그모이드 함수를 사용

```
Z1 = sigmoid(A1)                 # 이 시그모이드 함수는 Numpy배열을 받아 같은 수의 원소로 구성된 Numpy 배열을 반환
print(A1)                        # [0.3, 0.7, 1.1]
print(Z1)                        # [0.57444232, 0.66818777, 0.75026011]
```

    [0.3 0.7 1.1]
    [0.57444252 0.66818777 0.75026011]

* 1층에서 2층으로 신호 전달이 가는 과정.

이 구현은 1층의 출력 z1이 2층의 입력이 된다는 점을 제외하면 조금 전 구형과 똑같다.이처럼 Numpy배열을 사용하면서 층 사이 신호전달을 쉽게 구현

```
W2 = np.array([[0.1, 0.4],[0.2, 0.5], [0.3, 0.6]])
B2 = np.array([0.1, 0.2])

print(Z1.shape)                  # (3,)
print(W2.shape)                  # (3,2)
print(B2.shape)                  # (2,)

A2 = np.dot(Z1, W2) + B2
Z2 = sigmoid(A2)
```

    (3,)
    (3, 2)
    (2,)

* 2층에서 출력층으로 신호 전달

 여기에서는 항등 함수인 identity_function()을 정의하고, 이를 출력층의 활성화 함수로 이용.

 항등함수는 입력을 그대로 출력하는 함수.

```
def identity_function(x):
  return x

W3 = np.array([[0.1, 0.3],[0.2, 0.4]])
B3 = np.array([0.1, 0.2])

A3 = np.dot(Z2, W3) + B3
Y = identity_function(A3)                   # Y = A3
```

-  출력층의 활성화 함수는 풀고자 하는 문제의 성질에 맞게 정한다.

예를 들어 회귀에는 항등함수를 , 이진 클래스 분류에는 시그모이드 함수를, 다중 클래스 소프트맥스 함수를 사용

* **구현 정리**

```
# 3층 신경망 (3-layer neural network)

def init_network():
    network = {}
    network['W1'] = np.array([[0.1, 0.3, 0.5], [0.2, 0.4, 0.6]])
    network['b1'] = np.array([0.1, 0.2, 0.3])
    network['W2'] = np.array([[0.1, 0.4], [0.2, 0.5], [0.3, 0.6]])
    network['b2'] = np.array([0.1, 0.2])
    network['W3'] = np.array([[0.1, 0.3], [0.2, 0.4]])
    network['b3'] = np.array([0.1, 0.2])

    return network

def forward(network, x):
    W1, W2, W3 = network['W1'], network['W2'], network['W3']
    b1, b2, b3 = network['b1'], network['b2'], network['b3']

    a1 = np.dot(x, W1) + b1
    z1 = sigmoid(a1)
    a2 = np.dot(z1, W2) + b2
    z2 = sigmoid(a2)
    a3 = np.dot(z2, W3) + b3
    y = identity_function(a3)

    return y

network = init_network()
x = np.array([1.0, 0.5])
y = forward(network, x)
print(y)

```

    [0.31682708 0.69627909]

# 3.4 출력층 설계

```
a= np.array([0.3, 2.9, 4.0])

exp_a = np.exp(a)                 # exponential 지수함수
print(exp_a)
```

    [ 1.34985881 18.17414537 54.59815003]

```
sum_exp_a = np.sum(exp_a)         # 지수 함수의 합
print(sum_exp_a)
```

    74.1221542101633

```
y = exp_a / sum_exp_a
print(y)
```

    [0.01821127 0.24519181 0.73659691]

```
def softmax(a):                  # Softmax 함수로 구현
  exp_a = np.exp(a)
  sum_exp_a = np.sum(exp_a)
  y = exp_a / sum_exp_a

  return y
```

기계학습 문제는 분류(classification)와 회귀(regression)로 나뉜다. 분류는 데이터가 어느 클래스에 속하느냐의 문제다.
 회귀문제는 입력데이터에서 (연속적)인 수치를 예측하는 문제다.

Softmax function

\begin{align}
\text{Softmax}(y_{k}) = \frac{\exp(a_k)}{\sum_j \exp(a_i)}
\end{align}

exp(x)는 e^x을 뜻하는 지수함수(exponential function)을 뜻함. n은 출력층의 neuron 수, yk는 그 중 k번째 출력을 뜻함. 소프트맥스 함수의 분자는 입력 신호 ak의 지수 함수, 분모는 모든 입력 신호의 지수 함수의 합임.

```
a = np.array([0.3, 2.9, 4.0])

exp_a = np.exp(a)                                     # 지수 함수
print(exp_a)
```

    [ 1.34985881 18.17414537 54.59815003]

```
sum_exp_a = np.sum(exp_a)                            # 지수 함수의 합
print(sum_exp_a)
```

    74.1221542101633

```
y = exp_a / sum_exp_a
print(y)
```

    [0.01821127 0.24519181 0.73659691]

```
def softmax(a):
  exp_a = np.exp(a)
  sum_exp_a = np.sum(exp_a)
  y = exp_a / sum_exp_a

  return y
```


---
## 03. Ecuaciones Diferenciales Parciales IV. Ecuaciones Hiperbólicas No-Lineales

Eduard Larrañaga (ealarranaga@unal.edu.co)

---

### Resumen

Se presentarán algunos de los efectos de la no linealidad en las ecuaciones diferenciales parciales hiperbólicas.

---

##  La Ecuación de Advección Lineal 1-D

Muchos sistemas de PDEs hiperbólicos (como por ejemplo las ecuaciones de la hidrodinámica) pueden ser escritas en la forma de un sistema (lineal o no-lineal) de ecuaciones de advección, por lo que el estudio de este tipo de ecuaciones provee información imortante para sistemas mas complejos.

La ecuación lineal de advección unidimensional es

\begin{equation}
\partial_t \psi + v \partial_x \psi = 0
\end{equation}

donde $\psi(t,x)$ es una función escalar y $v$ es la velocidad constante con la que se advecta el perfil inicial (si $v > 0$ la advección se da de izquierda a derecha). Un ejemplo de esta ecuación corresponde a la **Ecuación de Continuidad**.

Como ya se ha mostrado, la solución de esta ecuación simplemente toma el perfil inicial, $\psi(t=0,x)$, y lo desplaza con rapidez $v$.  La forma del perfil inicial se preserva durante la advección.

Una sustitución directa muestra que $\psi(x - vt)$ es una solución de la ecuación de advección para cualquier perfil $\psi$. Esto significa que la solución es constante a lo largo de las lineas $x = v t$ (las curvas a lo largo de las cuales la solución es constante se denominan **curvas características**).

---

## PDEs Hiperbólicas No-Lineales. Ecuación de Burguer.

Considere ahora la ecuación de movimiento para un fluido en ausencia de viscosidad, campos magnéticos y otras fuerzas externas,

\begin{equation}
\rho \left( \frac{\partial \vec{v}}{\partial t} +  \vec{v} \cdot \vec{\nabla} \vec{v}\right) = - \vec{\nabla} P .
\end{equation}

Si además se considera que el fluido no tiene presión interna (gas de partículas no interactuantes o polvo), la ecuación se convierte en

\begin{equation}
\frac{\partial \vec{v}}{\partial t} +  \vec{v} \cdot \vec{\nabla} \vec{v}=  0.
\end{equation}

Esta relación se denomina **Ecuación de Burguer** y es la más simple de las **PDEs hiperbólicas no-lineales**. En términos de una función general $\psi = \psi(t,x)$ en una dimensión, se convierte en

\begin{equation}
\partial_t \psi + \psi \partial_x \psi = 0.
\end{equation}

Se puede apreciar que esta es casi identica a la ecuación de advección tratada anteriormente, excepto porque la velocidad **NO es constante** sino que corresponde al mismo campo $\psi$ que se quiere encontrar.

**Es decir, $\psi$ es al mismo tiempo la función que se advecta y la rapidez con la cual se mueve**.

---

### 1. Choques

Como se mostró arriba, en el caso de la ecuación de advección lineal, la solución es constante a lo largo de las líneas $x = vt + x_0$, las cuales son paralelas (porque $v$ es constante).

En el caso de la ecuación de Burger la velocidad no es constante y por ello las curvas características estarán dadas ahora por la condición $\frac{dx}{dt} = \psi$, con $x(0) = x_0$.
Debido a que $\psi = \psi(t,x)$, esta ecuación no se puede integrar inmediatamente.

Si se toma $\psi_0 = \psi(t=0)$, entonces se puede analizar como se comportan las curvas características en un pequeño intervalo temporal (antes de que $\psi(x,t)$ cambie significativamente).

De este comportamiento se nota que después de un corto intervalo temporal, las curvas características se intersecan. A partir de este punto de intersección,
$(t_s, x_s)$, no es posible recuperar la información hacia atrás en el tiempo para encontrar un único estado inicial (es decir, se perdió la causalidad del sistema).

Esta unión de las curvas características en el plano $t$-$x$  se denomina un **choque**, y es una de las formas en las que los problemas no-lineales difieren de los lineales.

---
### 2. Rarefacción

Otro tipo de efecto típico en los sistemas no-lineales es el de la **rarefacción**.  La siguiente figura muestra un conjunto de condiciones iniciales que implica una velocidad de advección pequeña al lado izquierdo y una velocidad grande al lado derecho.

En este caso las curvas características no convergen sino que divergen con la evolución temporal.

10. PDE I. Generalidades. Diferencias Finitas/04. Ecuaciones Hiperbolicas No-Lineales.ipynb


# Exponential cutoff

```python
from functools import partial

import pysme.hierarchy as hier
import numpy as np
from scipy.special import lambertw, factorial
```

```python
hard_cutoff_factories = {n_max: hier.HierarchyIntegratorFactory(2, n_max)
                         for n_max in range(1, 10+1)}
```

Want to pick `n_max` based of the decay constant $\lambda$ in $e^{-n/\lambda}$ used in
\begin{align}
    A_\text{exp}
    &=
    \sum_n e^{-n/\lambda}\sqrt{n+1}\left|n\middle\rangle\middle\langle n+1\right|
\end{align}

Basically want to pick it so that the elements we're throwing away are all smaller than some threshold $\varepsilon$, say $\sim1/2$.

\begin{align}
    e^{-n_\text{max}/\lambda}\sqrt{n_\text{max}+1}
    &=
    \varepsilon
\end{align}

Squaring gets me

\begin{align}
    e^{-2n_\text{max}/\lambda}(n_\text{max}+1)
    &=
    \varepsilon^2
\end{align}

Multiplying both sides by $-e^{-2/\lambda}(2/\lambda)$ gets me

\begin{align}
    -e^{-(2/\lambda)(n_\text{max}+1)}(2/\lambda)(n_\text{max}+1)
    &=
    -e^{-2/\lambda}(2/\lambda)\varepsilon^2
\end{align}

This is of the form $we^w=z$, which we can solve for using the Lambert w function: $W_k(z)=w$

```python
def calc_n_max(decay_const, threshold=0.5):
    z = -np.exp(-2/decay_const)*(2/decay_const)*threshold**2
    w = lambertw(z, k=-1).real
    n_max = -decay_const*w/2 - 1
    return n_max
```

```python
ns = np.arange(20)
decay_consts = np.arange(1, 6)
threshold = 0.5
plt.axhline(0)
for decay_const in decay_consts:
    n_max = calc_n_max(decay_const)
    plt.plot(ns, np.exp(-ns/decay_const)*np.sqrt(ns + 1) - threshold, label=decay_const)
    plt.axvline(n_max)
plt.legend()
plt.tight_layout()
```

```python
exp_cutoff_factories = {decay_const: hier.HierarchyIntegratorFactoryExpCutoff(2, int(np.floor(calc_n_max(decay_const))), decay_const)
                        for decay_const in range(1, 10+1)}
```

```python
[(decay_const, factory.n_max) for decay_const, factory in exp_cutoff_factories.items()]
```

```python
def xi_gau(t, Omega, ta):
    return np.power(Omega**2/(2*np.pi), 1/4) * np.exp(-Omega**2*(t - ta)**2/4)
```

```python
# Define Lowering operator and Sigma Z

Id = np.eye(2, dtype=complex)

zero = np.zeros((2,2), dtype=complex)

sigz = np.array([[1, 0], [0, -1]], dtype=complex)

sigm = np.array([[0, 0], [1, 0]], dtype=complex)
```

```python
S = Id
L = sigm
H = zero
r = np.log(2)
mu = 0
hard_cutoff_integrators = {n_max: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                           for n_max, factory in hard_cutoff_factories.items()}
exp_cutoff_integrators = {decay_const: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                          for decay_const, factory in exp_cutoff_factories.items()}
```

```python
def make_squeezed_state_vec(r, phi, N, normalized=True):
    r'''Make a truncated squeezed-state vector.

    The squeezed-state vector is :math:`S(r,\phi)|0\rangle`. The truncated
    vector is renormalized by default.

    Parameters
    ----------
    N: positive integer
        The dimension of the truncated Hilbert space, basis {0, ..., N-1}
    r: real number
        Squeezing amplitude
    phi: real number
        Squeezing phase
    normalized: boolean
        Whether or not the truncated vector is renormalized

    Returns
    -------
    numpy.array
        Squeezed-state vector in the truncated Hilbert space, represented in the
        number basis

    '''
    ket = np.zeros(N, dtype=np.complex)
    for n in range(N//2):
        ket[2*n] = (1 / np.sqrt(np.cosh(r))) * ((-0.5 * np.exp(2.j * phi) * np.tanh(r))**n /
                                                factorial(n)) * np.sqrt(factorial(2 * n))
    return ket / np.linalg.norm(ket) if normalized else ket
```

```python
fock_factory = hier.HierarchyIntegratorFactory(2, 20)
fock_integrator = fock_factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, 0, 0)
```

```python
fock_soln = fock_integrator.integrate(rho0, times)
```

```python
fock_field_ket = make_squeezed_state_vec(np.log(2), mu, 20 + 1)
fock_field_ket_er_3 = make_squeezed_state_vec(np.log(3), mu, 20 + 1)
fock_field_ket_er_4 = make_squeezed_state_vec(np.log(4), mu, 20 + 1)
fock_field_rho = np.outer(fock_field_ket, fock_field_ket)
fock_field_rho_er_3 = np.outer(fock_field_ket_er_3, fock_field_ket_er_3)
fock_field_rho_er_4 = np.outer(fock_field_ket_er_4, fock_field_ket_er_4)
```

```python
rho0 = (Id - sigz)/2
times = np.linspace(-3, 3, 2**8)
hard_cutoff_solns = {n_max: integrator.integrate(rho0, times) for n_max, integrator in hard_cutoff_integrators.items()}
exp_cutoff_solns = {decay_const: integrator.integrate(rho0, times) for decay_const, integrator in exp_cutoff_integrators.items()}
```

```python
for n_max, soln in hard_cutoff_solns.items():
    field_ket = np.array([1] + n_max*[0])
    field_rho = np.outer(field_ket, field_ket)
    plt.plot(times, soln.get_expectations(sigz, field_rho), label=r'$n_\mathrm{{max}}={}$'.format(n_max))
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=2$')
plt.legend()
plt.tight_layout()
```

```python
for decay_const, soln in exp_cutoff_solns.items():
    field_ket = np.array([1] + exp_cutoff_factories[decay_const].n_max*[0])
    field_rho = np.outer(field_ket, field_ket)
    plt.plot(times, soln.get_expectations(sigz, field_rho), label=r'$\lambda={}$'.format(decay_const))
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=2$')
plt.legend()
plt.tight_layout()
```

```python
n_max = 10
hard_field_ket = np.array([1] + n_max*[0])
hard_field_rho = np.outer(hard_field_ket, hard_field_ket)
plt.plot(times, hard_cutoff_solns[n_max].get_expectations(sigz, hard_field_rho), label='hard')
decay_const = 10
exp_field_ket = np.array([1] + exp_cutoff_factories[decay_const].n_max*[0])
exp_field_rho = np.outer(exp_field_ket, exp_field_ket)
plt.plot(times, exp_cutoff_solns[n_max].get_expectations(sigz, exp_field_rho), label='exponential')
plt.plot(times, fock_soln.get_expectations(sigz, fock_field_rho), linestyle=':', linewidth=4, label='fock $e^r=2$')
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=2$')
plt.legend()
plt.tight_layout()
```

```python
r = np.log(4)
hard_cutoff_integrators_er_4 = {n_max: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                                for n_max, factory in hard_cutoff_factories.items()}
exp_cutoff_integrators_er_4 = {decay_const: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                               for decay_const, factory in exp_cutoff_factories.items()}
```

```python
r = np.log(4)
hard_cutoff_integrators_er_4 = {n_max: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                                for n_max, factory in hard_cutoff_factories.items()}
exp_cutoff_integrators_er_4 = {decay_const: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                               for decay_const, factory in exp_cutoff_factories.items()}
```

```python
for n_max, soln in hard_cutoff_solns_er_4.items():
    field_ket = np.array([1] + n_max*[0])
    field_rho = np.outer(field_ket, field_ket)
    plt.plot(times, soln.get_expectations(sigz, field_rho), label=r'$n_\mathrm{{max}}={}$'.format(n_max))
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=4$')
plt.axvline(-.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.tight_layout()
```

```python
for decay_const, soln in exp_cutoff_solns_er_4.items():
    field_ket = np.array([1] + exp_cutoff_factories[decay_const].n_max*[0])
    field_rho = np.outer(field_ket, field_ket)
    plt.plot(times, soln.get_expectations(sigz, field_rho), label=r'$\lambda={}$'.format(decay_const))
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=4$')
plt.axvline(-.5)
plt.ylim(-1.1, 1.1)
plt.legend()
plt.tight_layout()
```

```python
n_max = 10
hard_field_ket = np.array([1] + n_max*[0])
hard_field_rho = np.outer(hard_field_ket, hard_field_ket)
plt.plot(times, hard_cutoff_solns_er_4[n_max].get_expectations(sigz, hard_field_rho), label='hard')
decay_const = 10
exp_field_ket = np.array([1] + exp_cutoff_factories[decay_const].n_max*[0])
exp_field_rho = np.outer(exp_field_ket, exp_field_ket)
plt.plot(times, exp_cutoff_solns_er_4[n_max].get_expectations(sigz, exp_field_rho), label='exponential')
plt.plot(times, fock_soln.get_expectations(sigz, fock_field_rho_er_4), linestyle=':', linewidth=4, label='fock $e^r=4$')
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=4$')
plt.ylim(-1.1, 1.1)
plt.legend()
plt.tight_layout()
```

```python
r = np.log(3)
hard_cutoff_integrators_er_3 = {n_max: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                                for n_max, factory in hard_cutoff_factories.items()}
exp_cutoff_integrators_er_3 = {decay_const: factory.make_uncond_integrator(partial(xi_gau, Omega=1, ta=0), S, L, H, r, mu)
                               for decay_const, factory in exp_cutoff_factories.items()}
```

```python
hard_cutoff_solns_er_3 = {n_max: integrator.integrate(rho0, times) for n_max, integrator in hard_cutoff_integrators_er_3.items()}
exp_cutoff_solns_er_3 = {decay_const: integrator.integrate(rho0, times) for decay_const, integrator in exp_cutoff_integrators_er_3.items()}
```

```python
for n_max, soln in hard_cutoff_solns_er_3.items():
    field_ket = np.array([1] + n_max*[0])
    field_rho = np.outer(field_ket, field_ket)
    plt.plot(times, soln.get_expectations(sigz, field_rho), label=r'$n_\mathrm{{max}}={}$'.format(n_max))
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=3$')
plt.ylim(-1.1, 1.1)
plt.legend()
plt.tight_layout()
```

```python
for decay_const, soln in exp_cutoff_solns_er_3.items():
    field_ket = np.array([1] + exp_cutoff_factories[decay_const].n_max*[0])
    field_rho = np.outer(field_ket, field_ket)
    plt.plot(times, soln.get_expectations(sigz, field_rho), label=r'$\lambda={}$'.format(decay_const))
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=3$')
plt.axhline(-1)
plt.legend()
plt.tight_layout()
```

```python
n_max = 10
hard_field_ket = np.array([1] + n_max*[0])
hard_field_rho = np.outer(hard_field_ket, hard_field_ket)
plt.plot(times, hard_cutoff_solns_er_3[n_max].get_expectations(sigz, hard_field_rho), label='hard')
decay_const = 10
exp_field_ket = np.array([1] + exp_cutoff_factories[decay_const].n_max*[0])
exp_field_rho = np.outer(exp_field_ket, exp_field_ket)
plt.plot(times, exp_cutoff_solns_er_3[n_max].get_expectations(sigz, exp_field_rho), label='exponential')
plt.plot(times, fock_soln.get_expectations(sigz, fock_field_rho_er_3), linestyle=':', linewidth=4, label='fock $e^r=3$')
plt.fill_between(times, xi_gau(times, Omega=1, ta=0) - 1, -1, color='k', alpha=0.125, zorder=-1, label=r'$\xi_t$')
plt.xlabel('$t$')
plt.ylabel(r'$\langle\sigma_z\rangle$')
plt.title('$e^r=3$')
plt.ylim(-1.1, 1.1)
plt.legend()
plt.tight_layout()
```

```python

```