# Batch Normalization
One way to make deep networks easier to train is to use more sophisticated optimization procedures such as SGD+momentum, RMSProp, or Adam. Another strategy is to change the architecture of the network to make it easier to train.
One idea along these lines is batch normalization which was proposed by [1] in 2015.

The idea is relatively straightforward. Machine learning methods tend to work better when their input data consists of uncorrelated features with zero mean and unit variance. When training a neural network, we can preprocess the data before feeding it to the network to explicitly decorrelate its features; this will ensure that the first layer of the network sees data that follows a nice distribution. However, even if we preprocess the input data, the activations at deeper layers of the network will likely no longer be decorrelated and will no longer have zero mean or unit variance since they are output from earlier layers in the network. Even worse, during the training process the distribution of features at each layer of the network will shift as the weights of each layer are updated.

The authors of [1] hypothesize that the shifting distribution of features inside deep neural networks may make training deep networks more difficult. To overcome this problem, [1] proposes to insert batch normalization layers into the network. At training time, a batch normalization layer uses a minibatch of data to estimate the mean and standard deviation of each feature. These estimated means and standard deviations are then used to center and normalize the features of the minibatch. A running average of these means and standard deviations is kept during training, and at test time these running averages are used to center and normalize features.

It is possible that this normalization strategy could reduce the representational power of the network, since it may sometimes be optimal for certain layers to have features that are not zero-mean or unit variance. To this end, the batch normalization layer includes learnable shift and scale parameters for each feature dimension.

[1] [Sergey Ioffe and Christian Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing
Internal Covariate Shift", ICML 2015.](https://arxiv.org/abs/1502.03167)

```python
# As usual, a bit of setup
import time
import numpy as np
import matplotlib.pyplot as plt
from cs231n.classifiers.fc_net import *
from cs231n.data_utils import get_CIFAR10_data
from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array
from cs231n.solver import Solver

%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# for auto-reloading external modules
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

def rel_error(x, y):
    """ returns relative error """
    return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))

def print_mean_std(x,axis=0):
    print('  means: ', x.mean(axis=axis))
    print('  stds:  ', x.std(axis=axis))
    print()
```

    =========== You can safely ignore the message below if you are NOT working on ConvolutionalNetworks.ipynb ===========
    	You will need to compile a Cython extension for a portion of this assignment.
    	The instructions to do this will be given in a section of the notebook below.
    	There will be an option for Colab users and another for Jupyter (local) users.

```python
# Load the (preprocessed) CIFAR10 data.
data = get_CIFAR10_data()
for k, v in data.items():
  print('%s: ' % k, v.shape)
```

    X_train:  (49000, 3, 32, 32)
    y_train:  (49000,)
    X_val:  (1000, 3, 32, 32)
    y_val:  (1000,)
    X_test:  (1000, 3, 32, 32)
    y_test:  (1000,)

## Batch normalization: forward
In the file `cs231n/layers.py`, implement the batch normalization forward pass in the function `batchnorm_forward`. Once you have done so, run the following to test your implementation.

Referencing the paper linked to above in [1] may be helpful!

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after batch normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before batch normalization:')
print_mean_std(a,axis=0)

gamma = np.ones((D3,))
beta = np.zeros((D3,))
# Means should be close to zero and stds close to one
print('After batch normalization (gamma=1, beta=0)')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)

gamma = np.asarray([1.0, 2.0, 3.0])
beta = np.asarray([11.0, 12.0, 13.0])
# Now means should be close to beta and stds close to gamma
print('After batch normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)
```

    Before batch normalization:
      means:  [ -2.3814598  -13.18038246   1.91780462]
      stds:   [27.18502186 34.21455511 37.68611762]

    After batch normalization (gamma=1, beta=0)
      means:  [5.32907052e-17 7.04991621e-17 4.11476409e-17]
      stds:   [0.99999999 1.         1.        ]

    After batch normalization (gamma= [1. 2. 3.] , beta= [11. 12. 13.] )
      means:  [11. 12. 13.]
      stds:   [0.99999999 1.99999999 2.99999999]

```python
# Check the test-time forward pass by running the training-time
# forward pass many times to warm up the running averages, and then
# checking the means and variances of activations after a test-time
# forward pass.

np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)

bn_param = {'mode': 'train'}
gamma = np.ones(D3)
beta = np.zeros(D3)

for t in range(50):
  X = np.random.randn(N, D1)
  a = np.maximum(0, X.dot(W1)).dot(W2)
  batchnorm_forward(a, gamma, beta, bn_param)

bn_param['mode'] = 'test'
X = np.random.randn(N, D1)
a = np.maximum(0, X.dot(W1)).dot(W2)
a_norm, _ = batchnorm_forward(a, gamma, beta, bn_param)

# Means should be close to zero and stds close to one, but will be
# noisier than training-time forward passes.
print('After batch normalization (test-time):')
print_mean_std(a_norm,axis=0)
```

    After batch normalization (test-time):
      means:  [-0.03927354 -0.04349152 -0.10452688]
      stds:   [1.01531428 1.01238373 0.97819988]

## Batch normalization: backward
Now implement the backward pass for batch normalization in the function `batchnorm_backward`.

To derive the backward pass you should write out the computation graph for batch normalization and backprop through each of the intermediate nodes. Some intermediates may have multiple outgoing branches; make sure to sum gradients across these branches in the backward pass.

Once you have finished, run the following to numerically check your backward pass.

```python
# Gradient check batchnorm backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
fx = lambda x: batchnorm_forward(x, gamma, beta, bn_param)[0]
fg = lambda a: batchnorm_forward(x, a, beta, bn_param)[0]
fb = lambda b: batchnorm_forward(x, gamma, b, bn_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = batchnorm_forward(x, gamma, beta, bn_param)
dx, dgamma, dbeta = batchnorm_backward(dout, cache)
#You should expect to see relative errors between 1e-13 and 1e-8
print('dx error: ', rel_error(dx_num, dx))
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
```

    dx error:  1.7029258328157158e-09
    dgamma error:  7.417225040694815e-13
    dbeta error:  2.8795057655839487e-12

## Batch normalization: alternative backward
In class we talked about two different implementations for the sigmoid backward pass. One strategy is to write out a computation graph composed of simple operations and backprop through all intermediate values. Another strategy is to work out the derivatives on paper. For example, you can derive a very simple formula for the sigmoid function's backward pass by simplifying gradients on paper.

Surprisingly, it turns out that you can do a similar simplification for the batch normalization backward pass too!

In the forward pass, given a set of inputs $X=\begin{bmatrix}x_1\\x_2\\...\\x_N\end{bmatrix}$,

we first calculate the mean $\mu$ and variance $v$.
With $\mu$ and $v$ calculated, we can calculate the standard deviation $\sigma$  and normalized data $Y$.
The equations and graph illustration below describe the computation ($y_i$ is the i-th element of the vector $Y$).

\begin{align}
& \mu=\frac{1}{N}\sum_{k=1}^N x_k  &  v=\frac{1}{N}\sum_{k=1}^N (x_k-\mu)^2 \\
& \sigma=\sqrt{v+\epsilon}         &  y_i=\frac{x_i-\mu}{\sigma}
\end{align}

The meat of our problem during backpropagation is to compute $\frac{\partial L}{\partial X}$, given the upstream gradient we receive, $\frac{\partial L}{\partial Y}.$ To do this, recall the chain rule in calculus gives us $\frac{\partial L}{\partial X} = \frac{\partial L}{\partial Y} \cdot \frac{\partial Y}{\partial X}$.

The unknown/hart part is $\frac{\partial Y}{\partial X}$. We can find this by first deriving step-by-step our local gradients at
$\frac{\partial v}{\partial X}$, $\frac{\partial \mu}{\partial X}$,
$\frac{\partial \sigma}{\partial v}$,
$\frac{\partial Y}{\partial \sigma}$, and $\frac{\partial Y}{\partial \mu}$,
and then use the chain rule to compose these gradients (which appear in the form of vectors!) appropriately to compute $\frac{\partial Y}{\partial X}$.

If it's challenging to directly reason about the gradients over $X$ and $Y$ which require matrix multiplication, try reasoning about the gradients in terms of individual elements $x_i$ and $y_i$ first: in that case, you will need to come up with the derivations for $\frac{\partial L}{\partial x_i}$, by relying on the Chain Rule to first calculate the intermediate $\frac{\partial \mu}{\partial x_i}, \frac{\partial v}{\partial x_i}, \frac{\partial \sigma}{\partial x_i},$ then assemble these pieces to calculate $\frac{\partial y_i}{\partial x_i}$.

You should make sure each of the intermediary gradient derivations are all as simplified as possible, for ease of implementation.

After doing so, implement the simplified batch normalization backward pass in the function `batchnorm_backward_alt` and compare the two implementations by running the following. Your two implementations should compute nearly identical results, but the alternative implementation should be a bit faster.

```python
np.random.seed(231)
N, D = 100, 500
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
out, cache = batchnorm_forward(x, gamma, beta, bn_param)

t1 = time.time()
dx1, dgamma1, dbeta1 = batchnorm_backward(dout, cache)
t2 = time.time()
dx2, dgamma2, dbeta2 = batchnorm_backward_alt(dout, cache)
t3 = time.time()

print('dx difference: ', rel_error(dx1, dx2))
print('dgamma difference: ', rel_error(dgamma1, dgamma2))
print('dbeta difference: ', rel_error(dbeta1, dbeta2))
print('speedup: %.2fx' % ((t2 - t1) / (t3 - t2)))
```

    dx difference:  1.9314251924176756e-12
    dgamma difference:  0.0
    dbeta difference:  0.0
    speedup: 1.76x

## Fully Connected Nets with Batch Normalization
Now that you have a working implementation for batch normalization, go back to your `FullyConnectedNet` in the file `cs231n/classifiers/fc_net.py`. Modify your implementation to add batch normalization.

Concretely, when the `normalization` flag is set to `"batchnorm"` in the constructor, you should insert a batch normalization layer before each ReLU nonlinearity. The outputs from the last layer of the network should not be normalized. Once you are done, run the following to gradient-check your implementation.

HINT: You might find it useful to define an additional helper layer similar to those in the file `cs231n/layer_utils.py`. If you decide to do so, do it in the file `cs231n/classifiers/fc_net.py`.

```python
np.random.seed(231)
N, D, H1, H2, C = 2, 15, 20, 30, 10
X = np.random.randn(N, D)
y = np.random.randint(C, size=(N,))

# You should expect losses between 1e-4~1e-10 for W,
# losses between 1e-08~1e-10 for b,
# and losses between 1e-08~1e-09 for beta and gammas.
for reg in [0, 3.14]:
  print('Running check with reg = ', reg)
  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,
                            reg=reg, weight_scale=5e-2, dtype=np.float64,
                            normalization='batchnorm')

  loss, grads = model.loss(X, y)
  print('Initial loss: ', loss)

  for name in sorted(grads):
    f = lambda _: model.loss(X, y)[0]
    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)
    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))
  if reg == 0: print()
```

    Running check with reg =  0
    Initial loss:  2.2611955101340957
    W1 relative error: 1.10e-04
    W2 relative error: 2.85e-06
    W3 relative error: 3.92e-10
    b1 relative error: 8.33e-09
    b2 relative error: 2.00e-07
    b3 relative error: 4.78e-11
    beta1 relative error: 7.33e-09
    beta2 relative error: 1.89e-09
    gamma1 relative error: 7.57e-09
    gamma2 relative error: 1.96e-09

    Running check with reg =  3.14
    Initial loss:  6.996533220108303
    W1 relative error: 1.98e-06
    W2 relative error: 2.28e-06
    W3 relative error: 1.11e-08
    b1 relative error: 1.39e-08
    b2 relative error: 2.36e-08
    b3 relative error: 2.23e-10
    beta1 relative error: 6.65e-09
    beta2 relative error: 5.69e-09
    gamma1 relative error: 8.80e-09
    gamma2 relative error: 4.14e-09

# Batchnorm for deep networks
Run the following to train a six-layer network on a subset of 1000 training examples both with and without batch normalization.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [100, 100, 100, 100, 100]

num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

weight_scale = 2e-2
bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

print('Solver with batch norm:')
bn_solver = Solver(bn_model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True,print_every=20)
bn_solver.train()

print('\nSolver without batch norm:')
solver = Solver(model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True, print_every=20)
solver.train()
```

    Solver with batch norm:
    (Iteration 1 / 200) loss: 2.340974
    (Epoch 0 / 10) train acc: 0.107000; val_acc: 0.115000
    (Epoch 1 / 10) train acc: 0.314000; val_acc: 0.266000
    (Iteration 21 / 200) loss: 2.039365
    (Epoch 2 / 10) train acc: 0.385000; val_acc: 0.279000
    (Iteration 41 / 200) loss: 2.041103
    (Epoch 3 / 10) train acc: 0.493000; val_acc: 0.308000
    (Iteration 61 / 200) loss: 1.753902
    (Epoch 4 / 10) train acc: 0.531000; val_acc: 0.307000
    (Iteration 81 / 200) loss: 1.246584
    (Epoch 5 / 10) train acc: 0.574000; val_acc: 0.314000
    (Iteration 101 / 200) loss: 1.323435
    (Epoch 6 / 10) train acc: 0.649000; val_acc: 0.327000
    (Iteration 121 / 200) loss: 1.134313
    (Epoch 7 / 10) train acc: 0.693000; val_acc: 0.316000
    (Iteration 141 / 200) loss: 1.216312
    (Epoch 8 / 10) train acc: 0.740000; val_acc: 0.305000
    (Iteration 161 / 200) loss: 0.735565
    (Epoch 9 / 10) train acc: 0.823000; val_acc: 0.335000
    (Iteration 181 / 200) loss: 0.927183
    (Epoch 10 / 10) train acc: 0.805000; val_acc: 0.301000

    Solver without batch norm:
    (Iteration 1 / 200) loss: 2.302332
    (Epoch 0 / 10) train acc: 0.129000; val_acc: 0.131000
    (Epoch 1 / 10) train acc: 0.283000; val_acc: 0.250000
    (Iteration 21 / 200) loss: 2.041970
    (Epoch 2 / 10) train acc: 0.316000; val_acc: 0.277000
    (Iteration 41 / 200) loss: 1.900473
    (Epoch 3 / 10) train acc: 0.373000; val_acc: 0.282000
    (Iteration 61 / 200) loss: 1.713156
    (Epoch 4 / 10) train acc: 0.390000; val_acc: 0.310000
    (Iteration 81 / 200) loss: 1.662209
    (Epoch 5 / 10) train acc: 0.433000; val_acc: 0.300000
    (Iteration 101 / 200) loss: 1.696236
    (Epoch 6 / 10) train acc: 0.529000; val_acc: 0.345000
    (Iteration 121 / 200) loss: 1.555476
    (Epoch 7 / 10) train acc: 0.536000; val_acc: 0.301000
    (Iteration 141 / 200) loss: 1.434704
    (Epoch 8 / 10) train acc: 0.615000; val_acc: 0.330000
    (Iteration 161 / 200) loss: 1.050906
    (Epoch 9 / 10) train acc: 0.663000; val_acc: 0.343000
    (Iteration 181 / 200) loss: 0.814525
    (Epoch 10 / 10) train acc: 0.747000; val_acc: 0.340000

Run the following to visualize the results from two networks trained above. You should find that using batch normalization helps the network to converge much faster.

```python
def plot_training_history(title, label, baseline, bn_solvers, plot_fn, bl_marker='.', bn_marker='.', labels=None):
    """utility function for plotting training history"""
    plt.title(title)
    plt.xlabel(label)
    bn_plots = [plot_fn(bn_solver) for bn_solver in bn_solvers]
    bl_plot = plot_fn(baseline)
    num_bn = len(bn_plots)
    for i in range(num_bn):
        label='with_norm'
        if labels is not None:
            label += str(labels[i])
        plt.plot(bn_plots[i], bn_marker, label=label)
    label='baseline'
    if labels is not None:
        label += str(labels[0])
    plt.plot(bl_plot, bl_marker, label=label)
    plt.legend(loc='lower center', ncol=num_bn+1)

plt.subplot(3, 1, 1)
plot_training_history('Training loss','Iteration', solver, [bn_solver], \
                      lambda x: x.loss_history, bl_marker='o', bn_marker='o')
plt.subplot(3, 1, 2)
plot_training_history('Training accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.train_acc_history, bl_marker='-o', bn_marker='-o')
plt.subplot(3, 1, 3)
plot_training_history('Validation accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.val_acc_history, bl_marker='-o', bn_marker='-o')

plt.gcf().set_size_inches(15, 15)
plt.show()
```

# Batch normalization and initialization
We will now run a small experiment to study the interaction of batch normalization and weight initialization.

The first cell will train 8-layer networks both with and without batch normalization using different scales for weight initialization. The second layer will plot training accuracy, validation set accuracy, and training loss as a function of the weight initialization scale.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [50, 50, 50, 50, 50, 50, 50]
num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

bn_solvers_ws = {}
solvers_ws = {}
weight_scales = np.logspace(-4, 0, num=20)
for i, weight_scale in enumerate(weight_scales):
    print('Running weight scale %d / %d' % (i + 1, len(weight_scales)))
    bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
    model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

    bn_solver = Solver(bn_model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
    bn_solver.train()
    bn_solvers_ws[weight_scale] = bn_solver

    solver = Solver(model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
    solver.train()
    solvers_ws[weight_scale] = solver
```

    Running weight scale 1 / 20
    Running weight scale 2 / 20
    Running weight scale 3 / 20
    Running weight scale 4 / 20
    Running weight scale 5 / 20
    Running weight scale 6 / 20
    Running weight scale 7 / 20
    Running weight scale 8 / 20
    Running weight scale 9 / 20
    Running weight scale 10 / 20
    Running weight scale 11 / 20
    Running weight scale 12 / 20
    Running weight scale 13 / 20
    Running weight scale 14 / 20
    Running weight scale 15 / 20
    Running weight scale 16 / 20
    Running weight scale 17 / 20
    Running weight scale 18 / 20
    Running weight scale 19 / 20
    Running weight scale 20 / 20

```python
# Plot results of weight scale experiment
best_train_accs, bn_best_train_accs = [], []
best_val_accs, bn_best_val_accs = [], []
final_train_loss, bn_final_train_loss = [], []

for ws in weight_scales:
  best_train_accs.append(max(solvers_ws[ws].train_acc_history))
  bn_best_train_accs.append(max(bn_solvers_ws[ws].train_acc_history))

  best_val_accs.append(max(solvers_ws[ws].val_acc_history))
  bn_best_val_accs.append(max(bn_solvers_ws[ws].val_acc_history))

  final_train_loss.append(np.mean(solvers_ws[ws].loss_history[-100:]))
  bn_final_train_loss.append(np.mean(bn_solvers_ws[ws].loss_history[-100:]))

plt.subplot(3, 1, 1)
plt.title('Best val accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best val accuracy')
plt.semilogx(weight_scales, best_val_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_val_accs, '-o', label='batchnorm')
plt.legend(ncol=2, loc='lower right')

plt.subplot(3, 1, 2)
plt.title('Best train accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best training accuracy')
plt.semilogx(weight_scales, best_train_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_train_accs, '-o', label='batchnorm')
plt.legend()

plt.subplot(3, 1, 3)
plt.title('Final training loss vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Final training loss')
plt.semilogx(weight_scales, final_train_loss, '-o', label='baseline')
plt.semilogx(weight_scales, bn_final_train_loss, '-o', label='batchnorm')
plt.legend()
plt.gca().set_ylim(1.0, 3.5)

plt.gcf().set_size_inches(15, 15)
plt.show()
```

## Inline Question 1:
Describe the results of this experiment. How does the scale of weight initialization affect models with/without batch normalization differently, and why?

## Answer:
[FILL THIS IN]

# Batch normalization and batch size
We will now run a small experiment to study the interaction of batch normalization and batch size.

The first cell will train 6-layer networks both with and without batch normalization using different batch sizes. The second layer will plot training accuracy and validation set accuracy over time.

```python
def run_batchsize_experiments(normalization_mode):
    np.random.seed(231)
    # Try training a very deep net with batchnorm
    hidden_dims = [100, 100, 100, 100, 100]
    num_train = 1000
    small_data = {
      'X_train': data['X_train'][:num_train],
      'y_train': data['y_train'][:num_train],
      'X_val': data['X_val'],
      'y_val': data['y_val'],
    }
    n_epochs=10
    weight_scale = 2e-2
    batch_sizes = [5,10,50]
    lr = 10**(-3.5)
    solver_bsize = batch_sizes[0]

    print('No normalization: batch size = ',solver_bsize)
    model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)
    solver = Solver(model, small_data,
                    num_epochs=n_epochs, batch_size=solver_bsize,
                    update_rule='adam',
                    optim_config={
                      'learning_rate': lr,
                    },
                    verbose=False)
    solver.train()

    bn_solvers = []
    for i in range(len(batch_sizes)):
        b_size=batch_sizes[i]
        print('Normalization: batch size = ',b_size)
        bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=normalization_mode)
        bn_solver = Solver(bn_model, small_data,
                        num_epochs=n_epochs, batch_size=b_size,
                        update_rule='adam',
                        optim_config={
                          'learning_rate': lr,
                        },
                        verbose=False)
        bn_solver.train()
        bn_solvers.append(bn_solver)

    return bn_solvers, solver, batch_sizes

batch_sizes = [5,10,50]
bn_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('batchnorm')
```

    No normalization: batch size =  5
    Normalization: batch size =  5
    Normalization: batch size =  10
    Normalization: batch size =  50

```python
plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 2:
Describe the results of this experiment. What does this imply about the relationship between batch normalization and batch size? Why is this relationship observed?

## Answer:
[FILL THIS IN]

# Layer Normalization
Batch normalization has proved to be effective in making networks easier to train, but the dependency on batch size makes it less useful in complex networks which have a cap on the input batch size due to hardware limitations.

Several alternatives to batch normalization have been proposed to mitigate this problem; one such technique is Layer Normalization [2]. Instead of normalizing over the batch, we normalize over the features. In other words, when using Layer Normalization, each feature vector corresponding to a single datapoint is normalized based on the sum of all terms within that feature vector.

[2] [Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer Normalization." stat 1050 (2016): 21.](https://arxiv.org/pdf/1607.06450.pdf)

## Inline Question 3:
Which of these data preprocessing steps is analogous to batch normalization, and which is analogous to layer normalization?

1. Scaling each image in the dataset, so that the RGB channels for each row of pixels within an image sums up to 1.
2. Scaling each image in the dataset, so that the RGB channels for all pixels within an image sums up to 1.
3. Subtracting the mean image of the dataset from each image in the dataset.
4. Setting all RGB values to either 0 or 1 depending on a given threshold.

## Answer:
[FILL THIS IN]

# Layer Normalization: Implementation

Now you'll implement layer normalization. This step should be relatively straightforward, as conceptually the implementation is almost identical to that of batch normalization. One significant difference though is that for layer normalization, we do not keep track of the moving moments, and the testing phase is identical to the training phase, where the mean and variance are directly calculated per datapoint.

Here's what you need to do:

* In `cs231n/layers.py`, implement the forward pass for layer normalization in the function `layernorm_forward`.

Run the cell below to check your results.
* In `cs231n/layers.py`, implement the backward pass for layer normalization in the function `layernorm_backward`.

Run the second cell below to check your results.
* Modify `cs231n/classifiers/fc_net.py` to add layer normalization to the `FullyConnectedNet`. When the `normalization` flag is set to `"layernorm"` in the constructor, you should insert a layer normalization layer before each ReLU nonlinearity.

Run the third cell below to run the batch size experiment on layer normalization.

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after layer normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 =4, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before layer normalization:')
print_mean_std(a,axis=1)

gamma = np.ones(D3)
beta = np.zeros(D3)
# Means should be close to zero and stds close to one
print('After layer normalization (gamma=1, beta=0)')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)

gamma = np.asarray([3.0,3.0,3.0])
beta = np.asarray([5.0,5.0,5.0])
# Now means should be close to beta and stds close to gamma
print('After layer normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)
```

    Before layer normalization:
      means:  [-59.06673243 -47.60782686 -43.31137368 -26.40991744]
      stds:   [10.07429373 28.39478981 35.28360729  4.01831507]

    After layer normalization (gamma=1, beta=0)
      means:  [-4.81096644e-16  0.00000000e+00  7.40148683e-17 -5.55111512e-16]
      stds:   [0.99999995 0.99999999 1.         0.99999969]

    After layer normalization (gamma= [3. 3. 3.] , beta= [5. 5. 5.] )
      means:  [5. 5. 5. 5.]
      stds:   [2.99999985 2.99999998 2.99999999 2.99999907]

```python
# Gradient check batchnorm backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

ln_param = {}
fx = lambda x: layernorm_forward(x, gamma, beta, ln_param)[0]
fg = lambda a: layernorm_forward(x, a, beta, ln_param)[0]
fb = lambda b: layernorm_forward(x, gamma, b, ln_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = layernorm_forward(x, gamma, beta, ln_param)
dx, dgamma, dbeta = layernorm_backward(dout, cache)

#You should expect to see relative errors between 1e-12 and 1e-8
print('dx error: ', rel_error(dx_num, dx))
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
```

    dx error:  1.4336160411201157e-09
    dgamma error:  4.519489546032799e-12
    dbeta error:  2.276445013433725e-12

# Layer Normalization and batch size

We will now run the previous batch size experiment with layer normalization instead of batch normalization. Compared to the previous experiment, you should see a markedly smaller influence of batch size on the training history!

```python
ln_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('layernorm')

plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 4:
When is layer normalization likely to not work well, and why?

1. Using it in a very deep network
2. Having a very small dimension of features
3. Having a high regularization term

## Answer:
[FILL THIS IN]


# Métodos iterativos para sistemas lineales

Aunque la eliminación Gaussiana es un método que da la solución **exacta** de un sistema lineal $\mathbf{Ax} = \mathbf{b}$, es un método difícil de implementar y no es simple de comprender o deducir para todos.

Así, existen métodos **iterativos**, mucho más sencillos de entender e implementar, que generan una sucesión de vectores $\{ \mathbf{x}^{(k)} \}_{k=1}^{\infty}$ que, bajo ciertas condiciones, convergen a la solución deseada.

A diferencia de los métodos anteriores donde hemos construido sucesiones de números reales, tener ahora una sucesión **vectorial** puede parecer complicado pero es básicamente lo mismo.

## Preludio Computacional I: Arreglos de arreglos vs Arreglos n-dimensionales

Como vamos a trabjar con sucesiones de vectores, una pregunta relevante resulta ser cómo representar una sucesión de vectores en la computadora.

Hay dos opciones principales para representar una sucesión de vectores:

### Opción 1: arreglo de arreglos

Cuando aprendimos sobre los arreglos, comentamos que los arreglos solo podían contener objetos del mismo tipo y, en principio, solo objetos `Int`, `Float`, `String` o `Char`. Sin embargo, también es posible que un arreglo contenga **arreglos** o Listas o cualquier otro objeto iterable como elemento dentro de sí.

Nuevamente, ya que un arreglo no puede tener elementos de tipo distinto, los elementos de un arreglo no pueden serarreglos y listas o arreglos y rangos: debe de tener objetos del mismo tipo.

```julia
# arreglo de arreglos
arr = [[1,2,3],[4,5,6],[7,8,9],[10,11,12]]
display(arr)
# veo cuál es su tipo
println(typeof(arr))
```

    4-element Array{Array{Int64,1},1}:
     [1, 2, 3]
     [4, 5, 6]
     [7, 8, 9]
     [10, 11, 12]

    Array{Array{Int64,1},1}

Si accedemos al i-ésimo elemento del arreglo, obtendremos otro arreglo

```julia
display(arr[2])
```

    3-element Array{Int64,1}:
     4
     5
     6

Podemos acceder ahora al j-ésimo elemento del i-ésimo elemento

```julia
# los primero corches indexan sobre `arr` y los segundos indexan sobre `arr[2]`
println(arr[2][3])
```

    6

Podemos utilizar esto para construir arreglos paso a paso con la función `push!`

```julia
function arregloDeArreglosPush(n)
    # Inicializo el arreglo con un elemento
    res = [[1.0,1.0,2.0]]
    # construyo n-1 términos adicionales
    for i in 2:n
        # el nuevo término es el vector anterior más [-1,2,3]
        push!(res,res[i-1]+[-1.0,2.0,3.0])
    end
    return res
end
```

    arregloDeArreglosPush (generic function with 1 method)

```julia
res = arregloDeArreglosPush(10)
display(res)
```

    10-element Array{Array{Float64,1},1}:
     [1.0, 1.0, 2.0]
     [0.0, 3.0, 5.0]
     [-1.0, 5.0, 8.0]
     [-2.0, 7.0, 11.0]
     [-3.0, 9.0, 14.0]
     [-4.0, 11.0, 17.0]
     [-5.0, 13.0, 20.0]
     [-6.0, 15.0, 23.0]
     [-7.0, 17.0, 26.0]
     [-8.0, 19.0, 29.0]

### Opción 2: usar un arreglo 2D (una matriz)

Si todos los elementos de la sucesión vectorial son del mismo tamaño (que generalmente así es), entonces los puedo almacenar en un arreglo 2D

```julia
# notación para matrices. Columnas se separan por espacios y renglones por ';'
nuevoarr = [[1 2 3] ; [4 5 6] ; [7 8 9] ; [10 11 12]]
```

    4×3 Array{Int64,2}:
      1   2   3
      4   5   6
      7   8   9
     10  11  12

Ahora, cada renglón representa un vector de la sucesión

```julia
# accedo al tercer renglón
display(nuevoarr[3,:])
```

    3-element Array{Int64,1}:
     7
     8
     9

Puedo añadirle un nuevo renglón representado por un vector mediante la función `vcat`, aunque debo transponer el vector antes de añadirlo usando la función `transpose`

```julia
# `vcat` concatena verticalmente (i.e. añade renglones) a un arreglo 2D
vcat(nuevoarr,transpose([13,14,15]))
```

    5×3 Array{Int64,2}:
      1   2   3
      4   5   6
      7   8   9
     10  11  12
     13  14  15

Notemos que, a diferencia de `push!`, `vcat` no cambia a la variable `nuevoarr`

```julia
# `nuevoarr` no tiene el nuevo renglón
display(nuevoarr)
```

    4×3 Array{Int64,2}:
      1   2   3
      4   5   6
      7   8   9
     10  11  12

Esto es diferente a la mecánica de la función `push!`, la cuál si cambia el arreglo al que añadimos algo de manera automática.

Para añadirle el vector `[13,14,15]` de manera permanente a `nuevoarr`, debo redefinir la variable:

```julia
nuevoarr = vcat(nuevoarr,transpose([13,14,15]))
display(nuevoarr)
```

    5×3 Array{Int64,2}:
      1   2   3
      4   5   6
      7   8   9
     10  11  12
     13  14  15

Podemos usar lo anterior para construir una matriz paso a paso

```julia
function matrizVcat(n)
    # Inicializo la matriz conteniendo solo un vector
    res = [1.0 1.0 2.0]
    # construyo n términos
    for i in 2:n
        # el nuevo término es el vector anterior más [-1,2,3]
        res = vcat(res,transpose(res[i-1,:]+[-1.0,2.0,3.0]))
    end
    return res
end
```

    matrizVcat (generic function with 1 method)

```julia
res = matrizVcat(10)
display(res)
```

    10×3 Array{Float64,2}:
      1.0   1.0   2.0
      0.0   3.0   5.0
     -1.0   5.0   8.0
     -2.0   7.0  11.0
     -3.0   9.0  14.0
     -4.0  11.0  17.0
     -5.0  13.0  20.0
     -6.0  15.0  23.0
     -7.0  17.0  26.0
     -8.0  19.0  29.0

## Preludio Computacional II: la librería `LinearAlgebra` de Julia

Cuando trabajamos con vectores, hay operaciones que muchas veces resultan necesarias como calcular su norma euclideana o realizar un producto punto.

Julia tiene, dentro de sí misma, una librería llamada `LinearAlgebra` con código útil para realizar operaciones de Álgebra Lineal. Su [Documentación](https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/) indica como usarla a profundidad, pero aquí mostraremos un par de ejemplos sencillos

```julia
# importamos la libreria. No hace falta instalarla pues ya está incluída con Julia
using LinearAlgebra
```

```julia
vec1 = [1.0,1.0,0.0]
vec2 = [1.0,0.0,1.0]
# `norm` nos regresa la norma euclideana de un vector
println(norm(vec1))
println(norm(vec2))
# `dot` nos permite calcular el producto punto de dos vectores
println(dot(vec1,vec2))
# `norm(vec1-vec2)` nos regresa la distancia euclideana entre vec1 y vec2
println(norm(vec1-vec2))
```

    1.4142135623730951
    1.4142135623730951
    1.0
    1.4142135623730951

Por el momento, solo utilizaremos estas dos funciones pero en el **futuro (siguiente clase)** esta librería será importante.

## Método de Jacobi

Supongamos que tenemos un sistema **con solución única** $\mathbf{Ax} = \mathbf{b}$. Explícitamente, sabemos que el sistema se muestra de la siguiente forma

$$
\begin{align}
a_{11} x_1 + a_{12} x_2 + \ldots + a_{1n} x_n &= b_1 \\
a_{21} x_1 + a_{22} x_2 + \ldots + a_{2n} x_n &= b_2 \\
\vdots \quad \quad \quad \vdots \quad \quad \quad \vdots \\
a_{n1} x_1 + a_{n2} x_2 + \ldots + a_{nn} x_n &= b_n \\
\end{align}
$$

Por simplicidad, supongamos también que todos los elementos de la diagonal son distintos de cero ($a_{ii} \neq 0$). Para un sistema con solución única siempre podemos cambiar el orden de las ecuaciones (intercambiar renglones de $\mathbf{A}$) para que esto se cumpla.

Así, podemos "despejar" cualquier $x_i$ de la i-ésima ecuación, poniéndolo en términos de los otros $x_{j}$ con $i \neq j$ de la siguiente forma:

$$
\begin{align}
 x_1 &= \frac{1}{a_{11}} \left(b_1 - ( a_{12} x_2 + a_{13} x_3 + \ldots + a_{1n} x_n) \right)  \\
 x_2 &= \frac{1}{a_{22}} \left(b_2 - ( a_{21} x_1 + a_{23} x_3  + \ldots + a_{2n} x_n) \right)  \\
 \vdots &\quad \quad \quad \vdots \quad \quad \quad \vdots \quad \quad \quad \vdots \quad \quad \quad \vdots \\
 x_n &= \frac{1}{a_{nn}} \left(b_n - ( a_{n1} x_1 + a_{n2} x_2  + \ldots + a_{n n-1} x_{n-1}) \right)  \\
\end{align}
$$

Podemos generalizar el despeje anterior para cualquier $x_i$ mediante la siguiente fórmula

$$
x_{i} = \frac{1}{a_{ii}}\left( b_i - \sum_{j=1 \\ j\neq i}^{n} a_{ij}x_j \right)
$$

Así, utilizaremos esta expresión para generar una **sucesión recursiva de vectores** $\{\mathbf{x}^{(t)}\}_{t=1}^{\infty}$ Definida, entrada a entrada, mediante:

$$
x_i^{(t)} = \begin{cases}
x^{(\text{inicial})}_i & \text{si } t=1 \\
& \\
\frac{1}{a_{ii}}\left( b_i - \sum_{j=1 \\ j\neq i}^{n} a_{ij}x^{(t-1)}_j \right) & \text{si } t>1 \\
\end{cases}
$$

Antes de implementar esta sucesión directamente, la reduciremos a una forma más compacta.

### Ejercicio 1

Escribe las ecuaciones

$$
\begin{align}
    x_{1} &= \frac{1}{a_{11}}\left( b_1 - \sum_{j=1 \\ j\neq 1}^{n} a_{1j} x_j \right) \\
    x_{2} &= \frac{1}{a_{22}}\left( b_2 - \sum_{j=1 \\ j\neq 2}^{n} a_{2j} x_j \right) \\
    &\vdots \quad \quad \quad \vdots \quad \quad \quad \vdots \\
 x_{n} &= \frac{1}{a_{nn}}\left( b_n - \sum_{j=1 \\ j\neq n}^{n} a_{nj} x_j \right) \\
\end{align}
$$

De una forma matricial:

$$
\mathbf{x} = \mathbf{Cx} +  \mathbf{d}
$$

Con $\mathbf{C}$ una matriz de $n \times n$ y $\mathbf{d}$ un vector de $n$ entradas. Debes mostrar explícitamente los elementos de $\mathbf{C}$ y de $\mathbf{d}$

**Sugerencia:** La matriz $C$ debe de tener $0$s en la diagonal

Después de encontrar la matriz $\mathbf{C}$, la sucesión recursiva de vectores puede escribirse de manera compacta como:

$$
\mathbf{x}^{(t)} = \begin{cases}
\mathbf{x}^{(\text{inicial})} & \text{si } t=1 \\
& \\
\mathbf{C}\mathbf{x}^{(t-1)} +  \mathbf{d} & \text{si } t>1 \\
\end{cases}
$$

Dicha sucesión puede converger a la solución del sistema de ecuaciones bajo ciertas condiciones. A este método para paroximar una solución del sistema lineal se le conoce como **Método de Jacobi** o **iteraciones de Jacobi**.

Si todavía te sientes confundido, puede consultar un ejemplo en el [artículo de Wikipedia](https://en.wikipedia.org/wiki/Jacobi_method#Example_1)

### Ejercicio 2

Implementa una función `jacobi(A,b,T,x_inicial)` donde $A$ es una matriz de $n \times n$, $b$ es un vector de longitud $n$, $T$ es un número natural y $x_{\text{inicial}}$ un vector de $n$ entradas. La función debe de generar los primeros $T$ términos de la sucesión vectorial $\mathbf{x}^{(t)}$ correspondiente al método de Jacobi.

La función debe de regresarte o bien un arreglo 1D `res` tal que `res[t]` es un vector de longitud $n$ correspondiente a $\mathbf{x}^{(t)}$ (un arreglo de arreglos) o bien un arreglo 2D `res` de $ T \times n $, en el que `res[t,:]` corresponde a $\mathbf{x}^{(t)}$ (una matriz)

**Sugerencia:** primero construye la matriz $\mathbf{C}$ y el vector $\mathbf{d}$ y utiliza la multiplicación de matrices por vectores y la suma de vectores para hacer tu código más compacto. Recuerda utilizar la función `copy` para hacer una copia de un arreglo a la hora de definir tanto $\mathbf{C}$ como $\mathbf{d}$

### Ejercicio 3

Implementa una función `jacobiTol(A,b,epsilon,x_inicial)` donde $A$ es una matriz de $n \times n$, $b$ es un vector de longitud $n$, $epsilon$ es un número real pequeño y $x_{\text{inicial}}$ un vector de $n$ entradas. La función debe de generar los términos de la sucesión vectorial $\mathbf{x}^{(t)}$ correspondiente al método de Jacobi hasta el término $m$ que cumpla que

$$
\Vert\mathbf{A} \mathbf{x}^{(m)} - \mathbf{b} \Vert < epsilon
$$

La función debe de regresarte o bien un arreglo 1D `res` tal que `res[t]` es un vector de longitud $n$ correspondiente a $\mathbf{x}^{(t)}$ o bien un arreglo 2D `res` de $ m \times n $, en el que `res[t,:]` corresponde a $\mathbf{x}^{(t)}$

### Ejercicio 4

Utiliza cualquiera de las dos funciones anteriores para resolver el siguiente sistema

$$
\begin{align}
4x - y + z &= 7 \\
4x - 8y + z &= -21 \\
-2x +y + 5z &= 15
\end{align}
$$

Con solución exacta $x=2$, $y=4$, $z=3$. ¿La sucesión generada si converge a la solución analítica?

Grafica el error absoluto ($\Vert \mathbf{x}^{(t)} - \mathbf{x}_{\text{exacta}} \Vert$) como función del paso $t$ de la iteración y di algo sobre su tendencia.

## Método de Gauss-Siedel

Aunque parece útil y sencillo, el método de Jacobi puede resultar **inestable**.

### Ejercicio 5

Reordena el sistema el ejercicio 4 de la siguiente forma:

$$
\begin{align}
-2x +y + 5z &= 15 \\
4x - y + z &= 7 \\
4x - 8y + z &= -21
\end{align}
$$

E intenta resolverlo usando la función `jacobi(A,b,T,x_inicial)` para un numero pequeño de iteraciones (alrededor de $10^2$). ¿El método converge?

Existe una manera de mejorarlo mucho haciendo un cambio teórico muy sencillo. Recordando la sucesión del método Jacobi entrada a entrada:

$$
x_i^{(t)} = \begin{cases}
x^{(\text{inicial})}_i & \text{si } t=1 \\
& \\
\frac{1}{a_{ii}}\left( b_i - \sum_{j=1 \\ j\neq i}^{n} a_{ij}x^{(t-1)}_j \right) & \text{si } t>1 \\
\end{cases}
$$

Notemos que para calcular $x_i^{(t)}$, utilizamos los $x_j^{(t-1)}$ para todos los $j\neq i $. Esto nos permite escribir el paso de la sucesión en términos de la matriz $\mathbf{C}$ y del vector $\mathbf{d}$ y hacer todo mucho más compacto. Además, nos permite actualizar todos los $x_i^{(t)}$ **simultáneamente**, es decir, con una misma operación para todos.

Sin embargo, qué sucede si en lugar de actualizar todo el vector $\mathbf{x}^{(t)}$ simultáneamente, vamos actualizando cada entrada $x_i^{(t)}$ en el orden que tienen en su índice. En ese caso, primero actualizaríamos

$$
x_1^{(t)} = \frac{1}{a_{11}}\left( b_1 - \sum_{j=1 \\ j\neq 1}^{n} a_{1j}x^{(t-1)}_j \right)
$$

Después

$$
x_2^{(t)} = \frac{1}{a_{22}}\left( b_2 - \sum_{j=1 \\ j\neq 1}^{n} a_{2j}x^{(t-1)}_j \right)
$$

En tercer lugar

$$
x_3^{(t)} = \frac{1}{a_{33}}\left( b_3 - \sum_{j=1 \\ j\neq 1}^{n} a_{3j}x^{(t-1)}_j \right)
$$

Y así hasta llegar al $n$.

Sin embargo, en la fórmula para $x_{2}^{(t)}$, sabemos que aparece $x_{1}^{(t-1)}$. Si realizamos las operaciones una por una, a la hora de calcular $x_{2}^{(t)}$ ya habríamos calculado $x_{1}^{(t)}$, por lo que **podríamos utilizar ese valor** en lugar de $x_{1}^{(t-1)}$ en la fórmula de $x_{2}^{(t)}$. Así, tendríamos que

$$
x_2^{(t)} = \frac{1}{a_{22}}\left( b_2 - a_{21} x_{1}^{(t)} - \sum_{j=2 \\ j\neq 2}^{n} a_{2j}x^{(t-1)}_j \right)
$$

Para actualizar $x_{3}^{(t)}$, podemos repetir el mismo argumento y reemplazar ahora tanto $x_{1}^{(t-1)}$ como $x_{2}^{(t-1)}$ por $x_{1}^{(t)}$ y $x_{2}^{(t)}$ y obtener:

$$
x_3^{(t)} = \frac{1}{a_{33}}\left( b_3 - a_{31} x_{1}^{(t)} - a_{32} x_{2}^{(t)}  - \sum_{j=3 \\ j\neq 3}^{n} a_{3j}x^{(t-1)}_j \right)
$$

Este proceso se puede generalizar para la actualización de $x_i{(t)}$  mediante la siguiente fórmula

$$
x_{i}^{(t)}  = \frac{1}{a_{ii}} \left( b_i - \sum_{j=1}^{i-1} a_{ij} x^{(t)}_j  - \sum_{j=i+1}^{n} a_{ij} x^{(t-1)}_j \right)
$$

Utilizando la ecuación anterior, la sucesión recursiva término a término se reescribe:

$$
x_i^{(t)} = \begin{cases}
x^{(\text{inicial})}_i & \text{si } t=1 \\
& \\
\frac{1}{a_{ii}} \left( b_i - \sum_{j=1}^{i-1} a_{ij} x^{(t)}_j  - \sum_{j=i+1}^{n} a_{ij} x^{(t-1)}_j \right) & \text{si } t>1 \\
\end{cases}
$$

A este método para aproximar una solución del sistema lineal se le conoce como **Método de Gauss-Siedel**. En general, es mucho más estable que el método de Jacobi.

Nuevamente, hay ejemplos disponibles en el [artículo de Wikipedia](https://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method#An_example_for_the_matrix_version)

### Ejercicio 6

Implementa una función `gaussSiedel(A,b,T,x_inicial)` donde $A$ es una matriz de $n \times n$, $b$ es un vector de longitud $n$, $T$ es un número natural y $x_{\text{inicial}}$ un vector de $n$ entradas. La función debe de generar los primeros $T$ términos de la sucesión vectorial $\mathbf{x}^{(t)}$ correspondiente al método de Gauss-Siedel

La función debe de regresarte o bien un arreglo 1D `res` tal que `res[t]` es un vector de longitud $n$ correspondiente a $\mathbf{x}^{(t)}$ o bien un arreglo 2D `res` de $ T \times n $, en el que `res[t,:]` corresponde a $\mathbf{x}^{(t)}$

**Sugerencia:** tendrás que usar un ciclo `for` sobre las entradas de los vectores dentro de otro ciclo `for` sobre el número total de iteraciones para ir construyendo, entrada por entrada, un vector correspondiente a las actualizaciones.

### Ejercicio 7

Resuelve el sistema del ejercicio 4 utilizando la función `gaussSiedel`. Grafica el error absoluto. ¿El algoritmo converge más rápido que el método de Jacobi? ¿Qué sucede con el sistema del ejercicio 5? ¿El método de Gauss-Siedel si lo puede resolver?

## Operador `\` de Julia

Debido a que resolver sistemas lineales en general es muy importante en el cómputo científico, Julia tiene un operador integrado que resuelve sistemas lineales de manera muy rápida usando métodos más sofisiticados de los que les enseñé

```julia
# resolviendo el sistema del ejercicio 4
A1 = [[4.0 -1.0 1.0] ; [4.0 -8.0 1.0] ; [-2.0 1.0 5.0] ]
b1 = [7.0,-21.0,15.0]
# Asigna a `resul` la solución del sistema A1 * x = b1
resul = A1 \ b1
display(resul)
```

    3-element Array{Float64,1}:
     2.0
     4.0
     3.0

### Ejercicio 8

Resuelve los sistemas del ejercicio 5 de esta clase y del ejercicio 7 de la clase 11 utilizando el operador `\` de Julia. ¿Las soluciones se obtienen más rápido (i.e. el tiempo de cómputo es menor) que con los otros métodos empleados?

## La visión general

Los sistemas lineales surgen en muchísimas áreas de las matemáticas y la física, por lo que es necesario conocer varios métodos para resolverlos.

De los métodos vistos aquí, ninguno es utilizado ampliamente en el día a día pues son muy lentos para resolver sistemas de tamaño grande ($n \geq 1000$). El de Gauss-Siedel es el que más se utiliza.

Hacia el final del curso, a la hora de ver Ecuaciones Diferenciales Parciales, nos empezarán a surgir muchos problemas cuya solución se reduce a resolver un conjunto de ecuaciones lineales.


# Aim of this notebook

* To construct the singular curve of universal type to finalize the solution of the optimal control problem

# Preamble

```python
from sympy import *
init_printing(use_latex='mathjax')

# Plotting
%matplotlib inline
## Make inline plots raster graphics
from IPython.display import set_matplotlib_formats
## Import modules for plotting and data analysis
import matplotlib.pyplot as plt
from matplotlib import gridspec,rc,colors
import matplotlib.ticker as plticker
## Parameters for seaborn plots
import seaborn as sns
sns.set(style='white',font_scale=1.25,
        rc={"xtick.major.size": 6, "ytick.major.size": 6,
            'text.usetex': False, 'font.family': 'serif', 'font.serif': ['Times']})

import pandas as pd
pd.set_option('mode.chained_assignment',None)

import numpy as np
from scipy.optimize import fsolve, root
from scipy.integrate import ode
backend = 'dopri5'
import warnings

# Timer
import time

from copy import deepcopy

from itertools import cycle
palette_size = 10;
clrs = sns.color_palette("Reds",palette_size)
iclrs = cycle(clrs) # iterated colors

# Suppress warnings
import warnings
warnings.filterwarnings("ignore")
```

# Parameter values

* Birth rate and const of downregulation are defined below in order to fit some experim. data

```python
d = .13 # death rate
α = .3 # low equilibrium point at expression of the main pathway (high equilibrium is at one)
θ = .45 # threshold value for the expression of the main pathway
κ = 40 # robustness parameter
```

* Symbolic variables - the list insludes μ & μbar, because they will be varied later

```python
σ, φ0, φ, x, μ, μbar = symbols('sigma, phi0, phi, x, mu, mubar')
```

* Main functions

```python
A = 1-σ*(1-θ)
Eminus = (α*A-θ)**2/2
ΔE = A*(1-α)*((1+α)*A/2-θ)
ΔEf = lambdify(σ,ΔE)
```

* Birth rate and cost of downregulation

```python
b = (0.1*(exp(κ*(ΔEf(1)))+1)-0.14*(exp(κ*ΔEf(0))+1))/(exp(κ*ΔEf(1))-exp(κ*ΔEf(0))) # birth rate
χ = 1-(0.14*(exp(κ*ΔEf(0))+1)-b*exp(κ*ΔEf(0)))/b
b, χ
```

$$\left ( 0.140168330860362, \quad 0.325961223954473\right )$$

```python
c_relative = 0.4
c = c_relative*(b-d)/b+(1-c_relative)*χ/(exp(κ*ΔEf(0))+1) # cost of resistance
c
```

$$0.0297380359370489$$

* Hamiltonian *H* and a part of it ρ that includes the control variable σ

```python
h = b*(χ/(exp(κ*ΔE)+1)*(1-x)+c*x)
H = -φ0 + φ*(b*(χ/(exp(κ*ΔE)+1)-c)*x*(1-x)+μ*(1-x)/(exp(κ*ΔE)+1)-μbar*exp(-κ*Eminus)*x) + h
ρ = (φ*(b*χ*x+μ)+b*χ)/(exp(κ*ΔE)+1)*(1-x)-φ*μbar*exp(-κ*Eminus)*x
H, ρ
```

$$\left ( \phi \left(\frac{\mu \left(- x + 1\right)}{e^{40 \left(- 0.385 \sigma + 0.7\right) \left(- 0.3575 \sigma + 0.2\right)} + 1} - \bar{\mu} x e^{- 20 \left(- 0.165 \sigma - 0.15\right)^{2}} + x \left(-0.00416833086036159 + \frac{0.045689440686899}{e^{40 \left(- 0.385 \sigma + 0.7\right) \left(- 0.3575 \sigma + 0.2\right)} + 1}\right) \left(- x + 1\right)\right) - \phi_{0} + 0.00416833086036159 x + \frac{0.045689440686899 \left(- x + 1\right)}{e^{40 \left(- 0.385 \sigma + 0.7\right) \left(- 0.3575 \sigma + 0.2\right)} + 1}, \quad - \bar{\mu} \phi x e^{- 20 \left(- 0.165 \sigma - 0.15\right)^{2}} + \frac{\left(- x + 1\right) \left(\phi \left(\mu + 0.045689440686899 x\right) + 0.045689440686899\right)}{e^{40 \left(- 0.385 \sigma + 0.7\right) \left(- 0.3575 \sigma + 0.2\right)} + 1}\right )$$

* Same but for no treatment (σ = 0)

```python
h0 = h.subs(σ,0)
H0 = H.subs(σ,0)
ρ0 = ρ.subs(σ,0)
H0, ρ0
```

$$\left ( \phi \left(0.00368423989943599 \mu \left(- x + 1\right) - 0.637628151621773 \bar{\mu} x - 0.004 x \left(- x + 1\right)\right) - \phi_{0} + 0.004 x + 0.000168330860361587, \quad - 0.637628151621773 \bar{\mu} \phi x + 0.00368423989943599 \left(- x + 1\right) \left(\phi \left(\mu + 0.045689440686899 x\right) + 0.045689440686899\right)\right )$$

* Machinery: definition of the Poisson brackets

```python
PoissonBrackets = lambda H1, H2: diff(H1,x)*diff(H2,φ)-diff(H1,φ)*diff(H2,x)
```

* Necessary functions and defining the right hand side of dynamical equations

```python
ρf = lambdify((x,φ,σ,μ,μbar),ρ)
ρ0f = lambdify((x,φ,μ,μbar),ρ0)
dxdτ = lambdify((x,φ,σ,μ,μbar),-diff(H,φ))
dφdτ = lambdify((x,φ,σ,μ,μbar),diff(H,x))
dVdτ = lambdify((x,σ),h)
dρdσ = lambdify((σ,x,φ,μ,μbar),diff(ρ,σ))
dδρdτ = lambdify((x,φ,σ,μ,μbar),-PoissonBrackets(ρ0-ρ,H))
def ode_rhs(t,state,μ,μbar):
    x, φ, V, δρ = state
    σs = [0,1]
    if (dρdσ(1.,x,φ,μ,μbar)<0) and (dρdσ(θ,x,φ,μ,μbar)>0):
        σstar = fsolve(dρdσ,.8,args=(x,φ,μ,μbar,))[0]
    else:
        σstar = 1.;
    if ρf(x,φ,σstar,μ,μbar) < ρ0f(x,φ,μ,μbar):
        sgm = 0
    else:
        sgm = σstar
    return [dxdτ(x,φ,sgm,μ,μbar),dφdτ(x,φ,sgm,μ,μbar),dVdτ(x,sgm),dδρdτ(x,φ,σstar,μ,μbar)]
```

```python
def get_primary_field(name, experiment,μ,μbar):
    solutions = {}
    solver = ode(ode_rhs).set_integrator(backend)
    τ0 = experiment['τ0']
    tms = np.linspace(τ0,experiment['T_end'],1e3+1)
    for x0 in experiment['x0']:
        δρ0 = ρ0.subs(x,x0).subs(φ,0)-ρ.subs(x,x0).subs(φ,0).subs(σ,1.)
        solver.set_initial_value([x0,0,0,δρ0],0.).set_f_params(μ,μbar)
        sol = []; k = 0;
        while (solver.t < experiment['T_end']) and (solver.y[0]<=1.) and (solver.y[0]>=0.):
            solver.integrate(tms[k])
            sol.append([solver.t]+list(solver.y))
            k += 1
        solutions[x0] = {'solution': sol}
    for x0, entry in solutions.items():
        entry['τ'] = [entry['solution'][j][0] for j in range(len(entry['solution']))]
        entry['x'] = [entry['solution'][j][1] for j in range(len(entry['solution']))]
        entry['φ'] = [entry['solution'][j][2] for j in range(len(entry['solution']))]
        entry['V'] = [entry['solution'][j][3] for j in range(len(entry['solution']))]
        entry['δρ'] = [entry['solution'][j][4] for j in range(len(entry['solution']))]
    return solutions
def get_δρ_value(tme,x0,μ,μbar):
    solver = ode(ode_rhs).set_integrator(backend)
    δρ0 = ρ0.subs(x,x0).subs(φ,0)-ρ.subs(x,x0).subs(φ,0).subs(σ,1.)
    solver.set_initial_value([x0,0,0,δρ0],0.).set_f_params(μ,μbar)
    while (solver.t < tme) and (solver.y[0]<=1.) and (solver.y[0]>=0.):
        solver.integrate(tme)
        sol = [solver.t]+list(solver.y)
    return solver.y[3]
def get_δρ_ending(params,μ,μbar):
    tme, x0 = params
    solver = ode(ode_rhs).set_integrator(backend)
    δρ0 = ρ0.subs(x,x0).subs(φ,0)-ρ.subs(x,x0).subs(φ,0).subs(σ,1.)
    solver.set_initial_value([x0,0,0,δρ0],0.).set_f_params(μ,μbar)
    δτ = 1.0e-8; tms = [tme,tme+δτ]
    _k = 0; sol = []
    while (_k<len(tms)):# and (solver.y[0]<=1.) and (solver.y[0]>=0.):
        solver.integrate(tms[_k])
        sol.append(solver.y)
        _k += 1
    #print(sol)
    return(sol[0][3],(sol[1][3]-sol[0][3])/δτ)
def get_state(tme,x0,μ,μbar):
    solver = ode(ode_rhs).set_integrator(backend)
    δρ0 = ρ0.subs(x,x0).subs(φ,0)-ρ.subs(x,x0).subs(φ,0).subs(σ,1.)
    solver.set_initial_value([x0,0,0,δρ0],0.).set_f_params(μ,μbar)
    δτ = 1.0e-8; tms = [tme,tme+δτ]
    _k = 0; sol = []
    while (solver.t < tms[-1]) and (solver.y[0]<=1.) and (solver.y[0]>=0.):
        solver.integrate(tms[_k])
        sol.append(solver.y)
        _k += 1
    return(list(sol[0])+[(sol[1][3]-sol[0][3])/δτ])
```

# Machinery for the universal line

* To find the universal singular curve we need to define two parameters

```python
γ0 = PoissonBrackets(PoissonBrackets(H,H0),H)
γ1 = PoissonBrackets(PoissonBrackets(H0,H),H0)
```

* The dynamics

```python
dxdτSingExpr = -(γ0*diff(H0,φ)+γ1*diff(H,φ))/(γ0+γ1)
dφdτSingExpr = (γ0*diff(H0,x)+γ1*diff(H,x))/(γ0+γ1)
dVdτSingExpr = (γ0*h0+γ1*h)/(γ0+γ1)
σSingExpr = γ1*σ/(γ0+γ1)
```

* Machinery for Python: lambdify the functions above

```python
dxdτSing = lambdify((x,φ,σ,μ,μbar),dxdτSingExpr)
dφdτSing = lambdify((x,φ,σ,μ,μbar),dφdτSingExpr)
dVdτSing = lambdify((x,φ,σ,μ,μbar),dVdτSingExpr)
σSing = lambdify((x,φ,σ,μ,μbar),σSingExpr)
```

```python
def ode_rhs_Sing(t,state,μ,μbar):
    x, φ, V = state
    if (dρdσ(1.,x,φ,μ,μbar)<0) and (dρdσ(θ,x,φ,μ,μbar)>0):
        σstar = fsolve(dρdσ,.8,args=(x,φ,μ,μbar,))[0]
    else:
        σstar = 1.;
    #print([σstar,σSing(x,φ,σstar,μ,μbar)])
    return [dxdτSing(x,φ,σstar,μ,μbar),dφdτSing(x,φ,σstar,μ,μbar),dVdτSing(x,φ,σstar,μ,μbar)]
# def ode_rhs_Sing(t,state,μ,μbar):
#     x, φ, V = state
#     if (dρdσ(1.,x,φ,μ,μbar)<0) and (dρdσ(θ,x,φ,μ,μbar)>0):
#         σstar = fsolve(dρdσ,.8,args=(x,φ,μ,μbar,))[0]
#     else:
#         σstar = 1.;
#     σTrav = fsolve(lambda σ: dxdτ(x,φ,σ,μ,μbar)-dxdτSing(x,φ,σstar,μ,μbar),.6)[0]
#     print([σstar,σTrav])
#     return [dxdτSing(x,φ,σstar,μ,μbar),dφdτSing(x,φ,σstar,μ,μbar),dVdτ(x,σTrav)]
def get_universal_curve(end_point,tmax,Nsteps,μ,μbar):
    tms =  np.linspace(end_point[0],tmax,Nsteps);
    solver = ode(ode_rhs_Sing).set_integrator(backend)
    solver.set_initial_value(end_point[1:4],end_point[0]).set_f_params(μ,μbar)
    _k = 0; sol = []
    while (solver.t < tms[-1]):
        solver.integrate(tms[_k])
        sol.append([solver.t]+list(solver.y))
        _k += 1
    return sol
def get_σ_universal(tme,end_point,μ,μbar):
    δτ = 1.0e-8; tms = [tme,tme+δτ]
    solver = ode(ode_rhs_Sing).set_integrator(backend)
    solver.set_initial_value(end_point[1:4],end_point[0]).set_f_params(μ,μbar)
    _k = 0; sol = []
    while (solver.t < tme+δτ):
        solver.integrate(tms[_k])
        sol.append([solver.t]+list(solver.y))
        _k += 1
    x, φ = sol[0][:2]
    sgm = fsolve(lambda σ: dxdτ(x,φ,σ,μ,μbar)-(sol[1][0]-sol[0][0])/δτ,θ/2)[0]
    return sgm
def get_state_universal(tme,end_point,μ,μbar):
    solver = ode(ode_rhs_Sing).set_integrator(backend)
    solver.set_initial_value(end_point[1:4],end_point[0]).set_f_params(μ,μbar)
    solver.integrate(tme)
    return [solver.t]+list(solver.y)
```

```python
def ode_rhs_with_σstar(t,state,μ,μbar):
    x, φ, V = state
    if (dρdσ(1.,x,φ,μ,μbar)<0) and (dρdσ(θ,x,φ,μ,μbar)>0):
        σ = fsolve(dρdσ,.8,args=(x,φ,μ,μbar,))[0]
    else:
        σ = 1.;
    return [dxdτ(x,φ,σ,μ,μbar),dφdτ(x,φ,σ,μ,μbar),dVdτ(x,σ)]
def ode_rhs_with_given_σ(t,state,σ,μ,μbar):
    x, φ, V = state
    return [dxdτ(x,φ,σ,μ,μbar),dφdτ(x,φ,σ,μ,μbar),dVdτ(x,σ)]
def get_trajectory_with_σstar(starting_point,tmax,Nsteps,μ,μbar):
    tms = np.linspace(starting_point[0],tmax,Nsteps)
    solver = ode(ode_rhs_with_σstar).set_integrator(backend)
    solver.set_initial_value(starting_point[1:],starting_point[0]).set_f_params(μ,μbar)
    sol = []; _k = 0;
    while solver.t < max(tms) and (solver.y[0]<=1.) and (solver.y[0]>=0.):
        solver.integrate(tms[_k])
        sol.append([solver.t]+list(solver.y))
        _k += 1
    return sol
def get_trajectory_with_given_σ(starting_point,tmax,Nsteps,σ,μ,μbar):
    tms = np.linspace(starting_point[0],tmax,100)
    solver = ode(ode_rhs_with_given_σ).set_integrator(backend)
    solver.set_initial_value(starting_point[1:],starting_point[0]).set_f_params(σ,μ,μbar)
    sol = []; _k = 0;
    while solver.t < max(tms) and (solver.y[0]<=1.) and (solver.y[0]>=0.):
        solver.integrate(tms[_k])
        sol.append([solver.t]+list(solver.y))
        _k += 1
    return sol
def get_state_with_σstar(tme,starting_point,μ,μbar):
    solver = ode(ode_rhs_with_σstar).set_integrator(backend)
    solver.set_initial_value(starting_point[1:4],starting_point[0]).set_f_params(μ,μbar)
    solver.integrate(tme)
    return [solver.t]+list(solver.y)
def get_finalizing_point_from_universal_curve(tme,tmx,end_point,μ,μbar):
    unv_point = get_state_universal(tme,end_point,μ,μbar)
    return get_state_with_σstar(tmx,unv_point,μ,μbar)[1]
```

# Field of optimal trajectories as the solution of the Bellman equation

* μ & μbar are varied by *T* and *T*bar ($\mu=1/T$ and $\bar\mu=1/\bar{T}$)

```python
tmx = 180.
end_switching_curve = {'t': 24., 'x': .9/.8}
# for Τ, Τbar in zip([28]*5,[14,21,28,35,60]):
for Τ, Τbar in zip([28],[60]):
    μ = 1./Τ; μbar = 1./Τbar
    print("Parameters: μ = %.5f, μbar = %.5f"%(μ,μbar))
    end_switching_curve['t'], end_switching_curve['x'] = root(get_δρ_ending,(end_switching_curve['t'],end_switching_curve['x']),args=(μ,μbar)).x
    end_point = [end_switching_curve['t']]+get_state(end_switching_curve['t'],end_switching_curve['x'],μ,μbar)
    print("Ending point for the switching line: τ = %.1f days, x = %.1f%%" % (end_point[0], end_point[1]*100))
    print("Checking the solution - should give zero values: ")
    print(get_δρ_ending([end_switching_curve['t'],end_switching_curve['x']],μ,μbar))
    print("* Constructing the primary field")
    experiments = {
        'sol1': { 'T_end': tmx, 'τ0': 0., 'x0': list(np.linspace(0,end_switching_curve['x']-(1e-3),10))+list(np.linspace(end_switching_curve['x']+(1e-6),1.,10)) } }
    primary_field = []
    for name, values in experiments.items():
        primary_field.append(get_primary_field(name,values,μ,μbar))
    print("* Constructing the switching curve")
    switching_curve = []
    x0s = np.linspace(end_switching_curve['x'],1,21); _y = end_switching_curve['t']

    for x0 in x0s:
        tme = fsolve(get_δρ_value,_y,args=(x0,μ,μbar))[0]
        if (tme>0):
            switching_curve = switching_curve+[[tme,get_state(tme,x0,μ,μbar)[0]]]
            _y = tme
    print("* Constructing the universal curve")
    universal_curve = get_universal_curve(end_point,tmx,25,μ,μbar)
    print("* Finding the last characteristic")
    #time0 = time.time()
#     tuniv = fsolve(get_finalizing_point_from_universal_curve,tmx-40.,args=(tmx,end_point,μ,μbar,))[0]
    tuniv = root(get_finalizing_point_from_universal_curve,tmx-40,args=(tmx,end_point,μ,μbar)).x
    print(tuniv)
    #print("The proccess to find the last characteristic took %0.1f minutes" % ((time.time()-time0)/60.))
    univ_point = get_state_universal(tuniv,end_point,μ,μbar)
    print("The last point on the universal line:")
    print(univ_point)
    last_trajectory = get_trajectory_with_σstar(univ_point,tmx,50,μ,μbar)
    print("Final state:")
    final_state = get_state_with_σstar(tmx,univ_point,μ,μbar)
    print(final_state)
    print("Fold-change in tumor size: %.2f"%(exp((b-d)*tmx-final_state[-1])))
    # Plotting
    plt.rcParams['figure.figsize'] = (6.75, 4)
    _k = 0
    for solutions in primary_field:
        for x0, entry in solutions.items():
            plt.plot(entry['τ'], entry['x'], 'k-', linewidth=.9, color=clrs[_k%palette_size])
            _k += 1
    plt.plot([x[0] for x in switching_curve],[x[1] for x in switching_curve],linewidth=2,color="red")
    plt.plot([end_point[0]],[end_point[1]],marker='o',color="red")
    plt.plot([x[0] for x in universal_curve],[x[1] for x in universal_curve],linewidth=2,color="red")
    plt.plot([x[0] for x in last_trajectory],[x[1] for x in last_trajectory],linewidth=.9,color="black")
    plt.xlim([0,tmx]); plt.ylim([0,1]);
    plt.xlabel("time, days"); plt.ylabel("fraction of resistant cells")
    plt.show()
    print()
```

```python
import csv
from numpy.linalg import norm
File = open("../figures/draft/sensitivity_mu-much_higher_cost.csv", 'w')
File.write("T,Tbar,mu,mubar,sw_start_x,sw_end_t,sw_end_x,univ_point_t,univ_point_x,outcome,err_sw_t,err_sw_x\n")
writer = csv.writer(File,lineterminator='\n')

end_switching_curve0 = {'t': 40.36, 'x': .92}
end_switching_curve_prev_t = end_switching_curve0['t']
tuniv = tmx-30.
Ts = np.arange(40,3,-1) #Τbars;
Τbars = np.arange(40,3,-1) #np.arange(120,1,-1) #need to change here if more
for Τ in Ts:
    μ = 1./Τ
    end_switching_curve = deepcopy(end_switching_curve0)
    for Τbar in Τbars:
        μbar = 1./Τbar
        print("* Parameters: T = %.1f, Tbar = %.1f (μ = %.5f, μbar = %.5f)"%(Τ,Τbar,μ,μbar))
        success = False; err = 1.
        while (not success)|(norm(err)>1e-6):
            end_switching_curve = {'t': 2*end_switching_curve['t']-end_switching_curve_prev_t-.001,
                                   'x': end_switching_curve['x']-0.002}
            sol = root(get_δρ_ending,(end_switching_curve['t'],end_switching_curve['x']),args=(μ,μbar))
            end_switching_curve_prev_t = end_switching_curve['t']
            end_switching_curve_prev_x = end_switching_curve['x']
            end_switching_curve['t'], end_switching_curve['x'] = sol.x
            success = sol.success
            err = get_δρ_ending([end_switching_curve['t'],end_switching_curve['x']],μ,μbar)
            if (not success):
                print("! Trying again...", sol.message)
            elif (norm(err)>1e-6):
                print("! Trying again... Convergence is not sufficient")
        else:
            end_point = [end_switching_curve['t']]+get_state(end_switching_curve['t'],end_switching_curve['x'],μ,μbar)
            print("Ending point: t = %.2f, x = %.2f%%"%(end_switching_curve['t'],100*end_switching_curve['x'])," Checking the solution:",err)
            universal_curve = get_universal_curve(end_point,tmx,25,μ,μbar)
            tuniv = root(get_finalizing_point_from_universal_curve,tuniv,args=(tmx,end_point,μ,μbar)).x
            err_tuniv = get_finalizing_point_from_universal_curve(tuniv,tmx,end_point,μ,μbar)
            univ_point = get_state_universal(tuniv,end_point,μ,μbar)
            print("tuniv = %.2f"%tuniv,"xuniv = %.2f%%"%(100*univ_point[1])," Checking the solution: ",err_tuniv)
            final_state = get_state_with_σstar(tmx,univ_point,μ,μbar)
            outcome = exp((b-d)*tmx-final_state[-1])
            print("Fold-change in tumor size: %.2f"%(outcome))
            output = [Τ,Τbar,μ,μbar,end_switching_curve['x'],end_point[0],end_point[1]]+list(univ_point[0:2])+[outcome]+list(err)+[err_tuniv]
            writer.writerow(output)
        if (Τbar==Τ):
            end_switching_curve0 = deepcopy(end_switching_curve)

File.close()
```

```python

```

scripts/B4. (much higher cost of resistance) Solution of the control problem including the universal line and Sensitivity analysis [Python].ipynb


# Coaddition and Subtraction

Version 0.1

By Yusra AlSayyad (Princeton University)

This notebook provides some problems for ground-based coaddition and subtraction.

We add and subtract images for different purposes: addition to get the benefits of longer exposures and subtraction to reveal what has changed.  However, in order to perform either operation, the following  image characteristics need to be normalized:
* Astrometric calibration (i.e. WCS)
* Photometric calibration (i.e. Zeropoint)
* Background level
* Point Spread Function (PSF) (optional for coaddition)

For these problems, we'll assume that we have images that have already been normalized for WCS, zeropoint, and background variations.

### Problem 1) Expected depth of a coadd

In order of magnitude calculations, you may hear people throw around the statement that the depth of a coadd increases by a factor of $\sqrt{N}$  for N single-epoch images. Where does this come from? Under what conditions is this a good approximation?

We are going to use our same star + noise 1-D simulation from  `IntroductionToBasicStellarPhotometry.ipynb` and `FindingSources.ipynb`:

```python
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import MultipleLocator
from scipy.stats import norm
```

```python
def pixel_plot(pix, counts, fig=None, ax=None):
    '''Make a pixelated 1D plot'''
    if fig is None and ax is None:
        fig, ax = plt.subplots()

    ax.step(pix, counts,
            where='post')

    ax.set_xlabel('pixel number')
    ax.set_ylabel('relative counts')
    ax.xaxis.set_minor_locator(MultipleLocator(1))
    ax.xaxis.set_major_locator(MultipleLocator(5))
    fig.tight_layout()
    return fig, ax

# It is sufficient to copy and paste from
# your introductionToBasicStellarPhotometry noteboook

def phi(x, mu, fwhm):
    """Evalute the 1d PSF N(mu, sigma^2) along x

    Parameters
    ----------
    x : array-like of shape (n_pixels,)
        detector pixel number
    mu : float
        mean position of the 1D star
    fwhm : float
        Full-width half-maximum of the stellar profile on the detector

    Returns
    -------
    flux : array-like of shape (n_pixels,)
        Flux in each pixel of the input array
    """
    # complete

    return flux

# Define your image simulation function to
# It is sufficient to copy and paste from
# your introductionToBasicStellarPhotometry noteboook
# Note that the background S should now be supplied as
# an array of length (x) or a constant.

def simulate(x, mu, fwhm, S, F):
    """simulate a noisy stellar signal

    Parameters
    ----------
    x : array-like
        detector pixel number
    mu : float
        mean position of the 1D star
    fwhm : float
        Full-width half-maximum of the stellar profile on the detector
    S : float or array-like of len(x)
        Sky background for each pixel
    F : float
        Total stellar flux

    Returns
    -------
    noisy_counts : array-like (same shape as x)
        the (noisy) number of counts in each pixel
    """
    # complete

    return noisy_counts
```

### Problem 1.1) Make a simple mean coadd

Simulate N observations of a star, and coadd them by taking the mean of the N observations. (We can only do this because they are already astrometrically and photometrically aligned and have the same background value.)

```python
MU = 35
S = 100
F = 100
FWHM = 5

x = np.arange(100)

# simulate a single observation of the star and plot:
y = # complete
pixel_plot(x, y)
```

```python
# Write a simulateN function that returns an array of size (N, x)
# representing N realizations of your simulated star
# This will stand in as a stack of multiple observations of one star

def simulateN(x, mu, fwhm, S, F, N):
    """simulate a noisy stellar signal

    Parameters
    ----------
    x : array-like
        detector pixel number
    mu : float
        mean position of the 1D star
    fwhm : float
        Full-width half-maximum of the stellar profile on the detector
    S : float or array-like of len(x)
        Sky background for each pixel
    F : float
        Total stellar flux
    N: int
        Number of images to simulate

    Returns
    -------
    noisy_counts : array-like of shape (N, x)
        the (noisy) number of counts in each pixel
    """
    # complete
    return noisy_counts
```

```python
# simulate N=50 images with the same star
x = np.arange(100)
stack = # complete
# where stack is an array of size (50, 100) representing a pile of 50 images with 100 pixels

# coadd by taking the mean and plot the result
coadd = # complete
pixel_plot(x, coadd)

# Try a few different N to see how it affects the S/N of your result

```

```python

# Plot the coadds of N=[1, 10, and 100] on the same plot:
# complete

```

### Problem 1.2)  SNR vs N

Now compute the observed SNR of the simulated star on each coadd and compare to the expected SNR in the idealized case. The often repeated mnemonic for SNR inscrease as a function of number of images, $N$, is that noise decreses like $\sqrt{N}$. This is of course idealized case where the noise in each observation is identical.

Using your `simulateN` function, simulate a series of mean coadds with increasing N.

* First, plot the empirical noise/uncertainty/stdev as a function of N. and overplot the expected uncertainty given the input sky level. You can use an area you know isn't touched by the star.
* Next, plot the empirical SNR of the star (measured flux/fluxErr) as a function of N. Overplot the expected SNR. You can assume you know the sky level.

Your expected scaling with N should roughly track your empirical estimate.

```python
# complete

# hint. One way to start this
# std = []
# flux = []
# Ns = np.arange(1, 1000, 5)
# for N in Ns:
    # y = simulateN(...)
    # complete

# plt.plot(Ns, ...,  label="coadd")
# plt.plot(Ns, ..., label="expected")
# plt.xlabel('N')
# plt.ylabel('pixel noise')
# plt.legend()
```

```python
# complete

# plt.plot(Ns, ...,  label="coadd")
# plt.plot(Ns, ..., label="expected")
# plt.xlabel('N')
# plt.ylabel('PSF Flux SNR')
# plt.legend()
```

```python

```

## Problem 2) PSFs and Image weights in coadds

Problem (1) pretends that the input images are identical in quality, however this is never the case in practice. In practice, adding another image does not necessarily increase the SNR. For example, imagine you have two exposures, but in one the dome light was accidentally left on. A coadd with these two images weighted equally will have a worse SNR than the first image alone. Therefore the images should be aggregated with a weighted mean, so that images of poor quality don't degrade the quality of the coadd. What weights to we pick?

Weights can be chosen to either minimize the variance on the coadd or maximize the SNR of point sources on the coadd.

#### Some background:

Assuming that all noise sources are independent, the SNR of the measurement of flux from a star is:
\begin{equation}
SNR \propto {{N_{\rm photons}}\over{ \sigma_{\rm sky}} \sqrt{A} },
\end{equation}
where $N_{\rm photons}$ is the number of photons detected from the star,
$A$ is the area in pixels covered by the star.
The per-pixel sky noise  $\sigma_{sky}$ includes all sources of noise: dark current, read noise and sky-background, and it coded in the variance plane of the image. For the epoch $i$,  $\sigma^2_{i, {\rm sky}}$ is the average of the variance plane.

The $N_{\rm photons}$ is proportional to transparency $T$, and the area that the stellar photons cover is determined by the seeing: $A \propto {\rm FWHM}^2$.

Therefore, a **coadd optimized for point-source detection** would, to weight each image by the SNR$^2$, use the following as weights which prefers good-seeing epochs taken when the sky is transparent and dark.
\begin{align}
w_i & = {\rm SNR}^2 \propto {{T_i^2}\over{{\rm FWHM_i}^2 \sigma_i^2}}.
\end{align}

The usual inverse-variance weighting which produces the **minimum-variance co-add**, is given by,
\begin{align}
w_i & =T_i^2/\sigma_i^2
\end{align}

In practice, the factor of $T_i^2$ is incorporated into the variance when flux-scaling the single-epoch images to a common zeropoint. This step multiplies the image by a scale-factor, which increases the variance of the image by the square of the scale factor. The scale factor is inversely proportional to the transparency, so that
$\sigma_{scaled}= \sigma/T$.

For this problem assume the images are all on the same zeropoint (like problem 1) i.e. T=1.

### Problem 2.1  Weighting images in Variable Sky

Now simulate 50 observations of stars with Sky `S` ranging from 100 to 1000. Remember to subtract this background off before stacking this time! Plot the plain (unweighted) mean coadd vs. the minimum variance coadd. Weights should add up to 1. What's the empirical noise estimate of the coadd?

```python
# complete
```

### Problem 2.2  Weighting images in Variable Seeing

Simulate 50 observations with FWHM's ranging from 2-10 pixels. Keep the flux amplitude `F`, and sky noise `S` both fixed.
Generate two coadds, (1) with the weights that minimize variance and (2) with the weights that maximize point source SNR. Weights should add up to 1. Plot both coadds.

```python
# complete
```

### Problem 2.3 Image variance vs per pixel variance (Challenge Problem)

Why do we use per image variances instead of per pixel variances? Let's see! Start tracking the per-pixel variance when you simulate the star. Make a coadd of 200 observations with FWHM's ranging from 2 to 20 pixels.  Make a coadd weighted by the per-pixel inverse variance. How does the profile of the star look in this coadd compared to an unweighted coadd and compared to the coadd with the $w_i  = \frac{1}{{\rm FWHM_i}^2 \sigma_i^2}$? (You may have to plot the difference to see the change).

```python
# complete
```

## Problem 3) Dipoles in Image Subtraction

In the lesson, we said that just because you see a dipole in a difference image, does not mean that the astrometric registration is terrible. For this problem, we'll forgo the pixelated simulated star from the previous problems and operate with Gaussian profiles.

```python
# Create two Gaussian 1-D profiles with

ASTROM_OFFSET = 0.1  # units of e.g. pixels
FLUX_SCALE = 1. # units of e.g nanojansky
PSF = 1  # rms pixels

x = np.linspace(-5, 5)
y1 = FLUX_SCALE * norm.pdf(x, ASTROM_OFFSET, PSF)
y2 = FLUX_SCALE * norm.pdf(x, -ASTROM_OFFSET, PSF)
plt.plot(x, y1, label='profile 1')
plt.plot(x, y2, label='profile 2')
plt.xlabel('x (pixel)')
plt.ylabel('y (flux)')
plt.legend()

```

### Problem 3.1) Plot the difference of these two profiles:

```python
# complete
```

### Problem 3.2)  What if we have amazing astrometric registration
and shrink the astrometric offset by a factor of a thousand. Is there a star sufficiently bright to produce the same dipole? What is its `FLUX SCALE`?

```python
ASTROM_OFFSET = 0.0001
PSF = 1.

# complete

# Plot both dipoles (for the offset=0.1 and the offset=0.0001 in the same figure.
# Same or different subplots up to you.
```

### Problem 3.3) Distance between peaks.

Does the distance between the dipole's positive and negative peaks depend on the astrometric offset? If not, what *does* it depend on?  You can answer this by vizualizing the dipoles vs offsets. But for a **challenge, measure the distance between peaks** and plot them as a function of astrometric offset or another factor.

```python
# complete
```

### Problem  3.4)
In the problem setup we assumed that the astrometric error was < the PSF width. Is this assumption likely to hold in an image you'd get from a wide-field imager like Rubin's?


Report 6

Jiachen Tian

## Objectives achieved this week
    - Explore the optical flow algorithm, precisely Lucas Kanade optical flow.

    - Implement Lucas Kanade optical flow from scratch and got it work on sample cases.

## Objectives for next week
    - Keep adding more functions to Lucas Kanade optical flow.

    - Adding a pyramid to the optical flow to ensure precision.

    - Parallelize optical flow as much as possible because its a slow algorithm.

## Results Demo
### Lucas Kanade optical flow is exactly what I am looking for.
   #### pros
       -Could be fast after proper parallelization.

       -Could be Robust by implementing extra algorithm including Gaussian and Laplacian Pyramids.

       -Completely automated.

   #### cons

       -Hard to implement.

   #### Theory explanation

       -Every image has gradient. After converting it to single color, Gradient could be represented by change of             numbers on the image.

       -Optical flow assunes image gradient remains constant after it moves to a new location.(as shown below, x ->           location x, y -> location y, t -> time, u and v -> moving factor, I() -> gradient factor).

       -After step 4, since we have a whole image of vectors but only two variables, we could use overconstrained             linear system to solve for the two unknowns as shown at step 5.

       -Therefore, U and V(direction vector for each pixel displacement) could be solved.

```latex
%%latex
\begin{align}
(1)I(x, y, t) = I(x + u, y + v, t + 1)//Consistency \\
(2)I(x+n, y+v,t+1) - I(x,y,t) = t_x*u+I_y*v+I_t \\
(3)I_x*u+I_y*v+I_t = 0 \\
(4)\nabla I[u v]^T + I_t = 0 \\
(5)\begin{bmatrix}
    \sum{I_xI_x} & \sum{I_xI_y}\\
    \sum{I_xI_y} & \sum{I_yI_y}
    \end{bmatrix}
    \begin{bmatrix}
    u \\
    v
    \end{bmatrix}
    =-\begin{bmatrix}
    \sum{I_xI_t}\\
    \sum{I_yI_t}
    \end{bmatrix}
\end{align}
```

\begin{align}
(1)I(x, y, t) = I(x + u, y + v, t + 1)//Consistency \\
(2)I(x+n, y+v,t+1) - I(x,y,t) = t_x*u+I_y*v+I_t \\
(3)I_x*u+I_y*v+I_t = 0 \\
(4)\nabla I[u v]^T + I_t = 0 \\
(5)\begin{bmatrix}
    \sum{I_xI_x} & \sum{I_xI_y}\\
    \sum{I_xI_y} & \sum{I_yI_y}
    \end{bmatrix}
    \begin{bmatrix}
    u \\
    v
    \end{bmatrix}
    =-\begin{bmatrix}
    \sum{I_xI_t}\\
    \sum{I_yI_t}
    \end{bmatrix}
\end{align}

### Picture Demo(Pay attention to the shift between upper picture and lower picture)

## Conclusion
    -Now we are officially getting into the fun stuffs. Successful implementation would result in best outcomes possible among all the other algoritums. Failure, on the other hand would be devastating due to its hard-to-debug nature. One last thing to look into if successfully implemented would be hidden markov models, which could further improve precision.