#  Neuromatch Academy: Week 1, Day 1, Tutorial 3
# Model Types: "Why" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

We would like to acknowledge [Steinmetz _et al._ (2019)](https://www.nature.com/articles/s41586-019-1787-x) for sharing their data, a subset of which is used here.

___
# Tutorial Objectives
This is tutorial 3 of a 3-part series on different flavors of models used to understand neural data. In parts 1 and 2 we explored mechanisms that would produce the data. In this tutorial we will explore models and techniques that can potentially explain *why* the spiking data we have observed is produced the way it is.

To understand why different spiking behaviors may be beneficial, we will learn about the concept of entropy. Specifically, we will:

- Write code to compute formula for entropy, a measure of information
- Compute the entropy of a number of toy distributions
- Compute the entropy of spiking activity from the Steinmetz dataset

```python
#@title Video 1: “Why” models
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV16t4y1Q7DR', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video

```

    Video available at https://www.bilibili.com/video/BV16t4y1Q7DR

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("/share/dataset/COMMON/nma.mplstyle.txt")

```

```python
#@title Helper Functions

def plot_pmf(pmf,isi_range):
  """Plot the probability mass function."""
  ymax = max(0.2, 1.05 * np.max(pmf))
  pmf_ = np.insert(pmf, 0, pmf[0])
  plt.plot(bins, pmf_, drawstyle="steps")
  plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
  plt.title(f"Neuron {neuron_idx}")
  plt.xlabel("Inter-spike interval (s)")
  plt.ylabel("Probability mass")
  plt.xlim(isi_range);
  plt.ylim([0, ymax])
```

```python
#@title Download Data
import io
with open('/share/dataset/W1D1/nma_steinmetz_spiketimes_cori_20161214_f32.npz', 'rb') as f:
    r = f.read()
steinmetz_spikes = np.load(io.BytesIO(r), allow_pickle=True)['spike_times']
```

# Section 1: Optimization and Information

Neurons can only fire so often in a fixed period of time, as the act of emitting a spike consumes energy that is depleted and must eventually be replenished. To communicate effectively for downstream computation, the neuron would need to make good use of its limited spiking capability. This becomes an optimization problem:

What is the optimal way for a neuron to fire in order to maximize its ability to communicate information?

In order to explore this question, we first need to have a quantifiable measure for information. Shannon introduced the concept of entropy to do just that, and defined it as

\begin{align}
  H_b(X) &= -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

where $H$ is entropy measured in units of base $b$ and $p(x)$ is the probability of observing the event $x$ from the set of all possible events in $X$. See the Appendix for a more detailed look at how this equation was derived.

The most common base of measuring entropy is $b=2$, so we often talk about *bits* of information, though other bases are used as well e.g. when $b=e$ we call the units *nats*.

First, let's explore how entropy changes between some simple discrete probability distributions. In the rest of this tutorial we will refer to these as probability mass functions (PMF), where $p(x_i)$ equals the $i^{th}$ value in an array, and mass refers to how much of the distribution is contained at that value.

For our first PMF, we will choose one where all of the probability mass is located in the middle of the distribution.

```python
n_bins = 50  # number of points supporting the distribution
x_range = (0, 1)  # will be subdivided evenly into bins corresponding to points

bins = np.linspace(*x_range, n_bins + 1)  # bin edges

pmf = np.zeros(n_bins)
pmf[len(pmf) // 2] = 1.0  # middle point has all the mass

# Since we already have a PMF, rather than un-binned samples, `plt.hist` is not
# suitable. Instead, we directly plot the PMF as a step function to visualize
# the histogram:
pmf_ = np.insert(pmf, 0, pmf[0])  # this is necessary to align plot steps with bin edges
plt.plot(bins, pmf_, drawstyle="steps")
# `fill_between` provides area shading
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
```

If we were to draw a sample from this distribution, we know exactly what we would get every time. Distributions where all the mass is concentrated on a single event are known as *deterministic*.

How much entropy is contained in a deterministic distribution?

## Exercise 1: Computing Entropy

Your first exercise is to implement a method that computes the entropy of a discrete probability distribution, given its mass function. Remember that we are interested in entropy in units of _bits_, so be sure to use the correct log function.

Recall that $\log(0)$ is undefined. When evaluated at $0$, NumPy log functions (such as `np.log2`) return `np.nan` ("Not a Number"). By convention, these undefined terms— which correspond to points in the distribution with zero mass—are excluded from the sum that computes the entropy.

```python
def entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits.

  This is a measure of information in the distribution. For a totally
  deterministic distribution, where samples are always found in the same bin,
  then samples from the distribution give no more information and the entropy
  is 0.

  For now this assumes `pmf` arrives as a well-formed distribution (that is,
  `np.sum(pmf)==1` and `not np.any(pmf < 0)`)

  Args:
    pmf (np.ndarray): The probability mass function for a discrete distribution
      represented as an array of probabilities.
  Returns:
    h (number): The entropy of the distribution in `pmf`.

  """
  ############################################################################
  # Exercise for students: compute the entropy of the provided PMF
  #   1. Exclude the points in the distribution with no mass (where `pmf==0`).
  #      Hint: this is equivalent to including only the points with `pmf>0`.
  #   2. Implement the equation for Shannon entropy (in bits).
  #  When ready to test, comment or remove the next line
  raise NotImplementedError("Excercise: implement the equation for entropy")
  ############################################################################

  # reduce to non-zero entries to avoid an error from log2(0)
  pmf = ...

  # implement the equation for Shannon entropy (in bits)
  h = ...

  # return the absolute value (avoids getting a -0 result)
  return np.abs(h)

# Uncomment to test your entropy function
# print(f"{entropy(pmf):.2f} bits")
```

[*Click for solution*](https://github.com/erlichlab/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial3_Solution_55c07dc8.py)

We expect zero surprise from a deterministic distribution. If we had done this calculation by hand, it would simply be

$-1\log_2 1 = -0=0$

Note that changing the location of the peak (i.e. the point and bin on which all the mass rests) doesn't alter the entropy. The entropy is about how predictable a sample is with respect to a distribution. A single peak is deterministic regardless of which point it sits on.

```python
pmf = np.zeros(n_bins)
pmf[2] = 1.0  # arbitrary point has all the mass

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
```

What about a distribution with mass split equally between two points?

```python
pmf = np.zeros(n_bins)
pmf[len(pmf) // 3] = 0.5
pmf[2 * len(pmf) // 3] = 0.5

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
```

Here, the entropy calculation is

$-(0.5 \log_2 0.5 + 0.5\log_2 0.5)=1$

There is 1 bit of entropy. This means that before we take a random sample, there is 1 bit of uncertainty about which point in the distribution the sample will fall on: it will either be the first peak or the second one.

Likewise, if we make one of the peaks taller (i.e. its point holds more of the probability mass) and the other one shorter, the entropy will decrease because of the increased certainty that the sample will fall on one point and not the other:

$-(0.2 \log_2 0.2 + 0.8\log_2 0.8)\approx 0.72$

Try changing the definition of the number and weighting of peaks, and see how the entropy varies.

If we split the probability mass among even more points, the entropy continues to increase. Let's derive the general form for $N$ points of equal mass, where $p_i=p=1/N$:

\begin{align}
 -\sum_i p_i \log_b p_i&= -\sum_i^N \frac{1}{N} \log_b \frac{1}{N}\\
                       &= -\log_b \frac{1}{N} \\
                       &= \log_b N
\end{align}
$$$$

If we have $N$ discrete points, the _uniform distribution_ (where all points have equal mass) is the distribution with the highest entropy: $\log_b N$. This upper bound on entropy is useful when considering binning strategies, as any estimate of entropy over $N$ discrete points (or bins) must be in the interval $[0, \log_b N]$.

```python
pmf = np.ones(n_bins) / n_bins  # [1/N] * N

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range);
plt.ylim(0, 1);
```

Here, there are 50 points and the entropy of the uniform distribution is $\log_2 50\approx 5.64$. If we construct _any_ discrete distribution $X$ over 50 points (or bins) and calculate an entropy of $H_2(X)>\log_2 50$, something must be wrong with our implementation of the discrete entropy computation.

# Section 2: Information, neurons, and spikes

```python
#@title Video 2: Entropy of different distributions
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1df4y1976g', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1df4y1976g

Recall the discussion of spike times and inter-spike intervals (ISIs) from Tutorial 1. What does the information content (or distributional entropy) of these measures say about our theory of nervous systems?

We'll consider three hypothetical neurons that all have the same mean ISI, but with different distributions:

1. Deterministic
2. Uniform
3. Exponential

Fixing the mean of the ISI distribution is equivalent to fixing its inverse: the neuron's mean firing rate. If a neuron has a fixed energy budget and each of its spikes has the same energy cost, then by fixing the mean firing rate, we are normalizing for energy expenditure. This provides a basis for comparing the entropy of different ISI distributions. In other words: if our neuron has a fixed budget, what ISI distribution should it express (all else being equal) to maximize the information content of its outputs?

Let's construct our three distributions and see how their entropies differ.

```python
n_bins = 50
mean_isi = 0.025
isi_range = (0, 0.25)

bins = np.linspace(*isi_range, n_bins + 1)
mean_idx = np.searchsorted(bins, mean_isi)

# 1. all mass concentrated on the ISI mean
pmf_single = np.zeros(n_bins)
pmf_single[mean_idx] = 1.0

# 2. mass uniformly distributed about the ISI mean
pmf_uniform = np.zeros(n_bins)
pmf_uniform[0:2*mean_idx] = 1 / (2 * mean_idx)

# 3. mass exponentially distributed about the ISI mean
pmf_exp = stats.expon.pdf(bins[1:], scale=mean_isi)
pmf_exp /= np.sum(pmf_exp)
```

```python
#@title
#@markdown Run this cell to plot the three PMFs
fig, axes = plt.subplots(ncols=3, figsize=(18, 5))

dists =  [# (subplot title, pmf, ylim)
          ("(1) Deterministic", pmf_single, (0, 1.05)),
          ("(1) Uniform", pmf_uniform, (0, 1.05)),
          ("(1) Exponential", pmf_exp, (0, 1.05))]

for ax, (label, pmf_, ylim) in zip(axes, dists):
  pmf_ = np.insert(pmf_, 0, pmf_[0])
  ax.plot(bins, pmf_, drawstyle="steps")
  ax.fill_between(bins, pmf_, step="pre", alpha=0.4)
  ax.set_title(label)
  ax.set_xlabel("Inter-spike interval (s)")
  ax.set_ylabel("Probability mass")
  ax.set_xlim(isi_range);
  ax.set_ylim(ylim);
```

```python
print(
  f"Deterministic: {entropy(pmf_single):.2f} bits",
  f"Uniform: {entropy(pmf_uniform):.2f} bits",
  f"Exponential: {entropy(pmf_exp):.2f} bits",
  sep="\n",
)
```

    Deterministic: 0.00 bits
    Uniform: 3.32 bits
    Exponential: 3.77 bits

```python
#@title Video 3: Probabilities from histogram
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1Jk4y1B7cz', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1Jk4y1B7cz

In the previous example we created the PMFs by hand to illustrate idealized scenarios. How would we compute them from data recorded from actual neurons?

One way is to convert the ISI histograms we've previously computed into discrete probability distributions using the following equation:

\begin{align}
p_i = \frac{n_i}{\sum\nolimits_{i}n_i}
\end{align}

where $p_i$ is the probability of an ISI falling within a particular interval $i$ and $n_i$ is the count of how many ISIs were observed in that interval.

### Exercise 2: Probabilty Mass Function

Your second exercise is to implement a method that will produce a probability mass function from an array of ISI bin counts.

To verify your solution, we will compute the probability distribution of ISIs from real neural data taken from the Steinmetz dataset.

```python
neuron_idx = 283

isi = np.diff(steinmetz_spikes[neuron_idx])
bins = np.linspace(*isi_range, n_bins + 1)
counts, _ = np.histogram(isi, bins)
```

```python
def pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  ###########################################################################
  # Exercise: Compute the PMF. Remove the next line to test your function
  raise NotImplementedError("Student excercise: compute the PMF from ISI counts")
  ###########################################################################

  pmf = ...

  return pmf

# Uncomment when ready to test your function
# pmf = pmf_from_counts(counts)
# plot_pmf(pmf,isi_range)
```

[*Click for solution*](https://github.com/erlichlab/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial3_Solution_49231923.py)

*Example output:*

# Section 3: Calculate entropy from a PMF

```python
#@title Video 4: Calculating entropy from pmf
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1vA411e7Cd', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1vA411e7Cd

Now that we have the probability distribution for the actual neuron spiking activity, we can calculate its entropy.

```python
print(f"Entropy for Neuron {neuron_idx}: {entropy(pmf):.2f} bits")
```

    Entropy for Neuron 283: 3.36 bits

## Interactive Demo: Entropy of neurons

We can combine the above distribution plot and entropy calculation with an interactive widget to explore how the different neurons in the dataset vary in spiking activity and relative information. Note that the mean firing rate across neurons is not fixed, so some neurons with a uniform ISI distribution may have higher entropy than neurons with a more exponential distribution.

```python
#@title
#@markdown **Run the cell** to enable the sliders.

def _pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  pmf = counts / np.sum(counts)
  return pmf

def _entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits."""
  # remove non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf > 0]
  h = -np.sum(pmf * np.log2(pmf))
  # absolute value applied to avoid getting a -0 result
  return np.abs(h)

@widgets.interact(neuron=widgets.IntSlider(0, min=0, max=(len(steinmetz_spikes)-1)))
def steinmetz_pmf(neuron):
  """ Given a neuron from the Steinmetz data, compute its PMF and entropy """
  isi = np.diff(steinmetz_spikes[neuron])
  bins = np.linspace(*isi_range, n_bins + 1)
  counts, _ = np.histogram(isi, bins)
  pmf = _pmf_from_counts(counts)

  plot_pmf(pmf,isi_range)
  plt.title(f"Neuron {neuron}: H = {_entropy(pmf):.2f} bits")

```

---
# Summary

```python
#@title Video 5: Summary of model types
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1F5411e7ww', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1F5411e7ww

Congratulations! You've finished your first NMA tutorial. In this 3 part tutorial series, we used different types of models to understand the spiking behavior of neurons recorded in the Steinmetz data set.

 - We used "what" models to discover that the ISI distribution of real neurons is closest to an exponential distribution
 - We used "how" models to discover that balanced excitatory and inbhitiory inputs, coupled with a leaky membrane, can give rise to neuronal spiking with exhibiting such an exponential ISI distribution
 - We used "why" models to discover that exponential ISI distributions contain the most information when the mean spiking is constrained

# Bonus

### The foundations for Entropy

In his foundational [1948 paper](https://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Communication) on information theory, Claude Shannon began with three criteria for a function $H$ defining the entropy of a discrete distribution of probability masses $p_i\in p(X)$ over the points $x_i\in X$:
1. $H$ should be continuous in the $p_i$.
  - That is, $H$ should change smoothly in response to smooth changes to the mass $p_i$ on each point $x_i$.
2. If all the points have equal shares of the probability mass, $p_i=1/N$, $H$ should be a non-decreasing function of $N$.
  - That is, if $X_N$ is the support with $N$ discrete points and $p(x\in X_N)$ assigns constant mass to each point, then $H(X_1) < H(X_2) < H(X_3) < \dots$
3. $H$ should be preserved by (invariant to) the equivalent (de)composition of distributions.
  - For example (from Shannon's paper) if we have a discrete distribution over three points with masses $(\frac{1}{2},\frac{1}{3},\frac{1}{6})$, then their entropy can be represented in terms of a direct choice between the three and calculated $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})$. However, it could also be represented in terms of a series of two choices:
    1. either we sample the point with mass $1/2$ or not (_not_ is the other $1/2$, whose subdivisions are not given in the first choice),
    2. if (with probability $1/2$) we _don't_ sample the first point, we sample one of the two remaining points, masses $1/3$ and $1/6$.

    Thus in this case we require that $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})=H(\frac{1}{2},\frac{1}{2}) + \frac{1}{2}H(\frac{1}{3}, \frac{1}{6})$

There is a unique function (up to a linear scaling factor) which satisfies these 3 requirements:

\begin{align}
  H_b(X) &= -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

Where the base of the logarithm $b>1$ controls the units of entropy. The two most common cases are $b=2$ for units of _bits_, and $b=e$ for _nats_.

We can view this function as the expectation of the self-information over a distribution:

$$H_b(X) = \mathbb{E}_{x\in X} \left[I_b(x)\right]$$

$$I_b(x)=-\log_b p(x)$$

Self-information is just the negative logarithm of probability, and is a measure of how surprising an event sampled from the distribution would be. Events with $p(x)=1$ are certain to occur, and their self-information is zero (as is the entropy of the distribution they compose) meaning they are totally unsurprising. The smaller the probability of an event, the higher its self-information, and the more surprising the event would be to observe.


# PHYS 2211 - Introductory Physics Laboratory I

# Measurement andError Propagation

### Name: Tatiana Krivosheev

### Partners: Oleg Krivosheev

#### Annex A

```python
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
import sympy
%matplotlib inline
```

#### Annex A - Data and Calculations

#### 1. Rectangular Block

```python
class ListTable(list):
    """ Overridden list class which takes a 2-dimensional list of
        the form [[1,2,3],[4,5,6]], and renders an HTML Table in
        IPython Notebook. """

    def _repr_html_(self):
        html = ["<table>"]
        for row in self:
            html.append("<tr>")

            for col in row:
                html.append("<td>{0}</td>".format(col))

            html.append("</tr>")
        html.append("</table>")
        return ''.join(html)
```

```python
# plain text
plt.title('alpha > beta')
```

```python
# math text
plt.title(r'$\alpha > \beta$')
```

```python
from sympy import symbols, init_printing
init_printing(use_latex=True)
delta = symbols('delta')
delta**2/3
```

```python
from sympy import symbols, init_printing
init_printing(use_latex=True)
delta = symbols('delta')
table = ListTable()
table.append(['measuring device', '  ', 'delta', 'w', 'delta w', 'h', 'delta h'])
table.append(['  ', '(cm)', '(cm)', '(cm)','(cm)', '(cm)', '(cm)'])
lr=4.9
wr=2.5
hr=1.2
lc=4.90
wc=2.54
hc=1.27
deltar=0.1
deltac=0.01
table.append(['ruler',lr, deltar, wr, deltar, hr, deltar])
table.append(['vernier caliper', lc, deltac, wc, deltac, hc, deltac])
table
```

measuring device	  	delta	w	delta w	h	delta h

  	(cm)	(cm)	(cm)	(cm)	(cm)	(cm)

ruler	4.9	0.1	2.5	0.1	1.2	0.1

vernier caliper	4.9	0.01	2.54	0.01	1.27	0.01

```python
s(t) = \mathcal{A}\/\sin(2 \omega t)
```

```python
table = ListTable()
table.append(['l', 'deltal', 'w', 'deltaw', 'h', 'deltah'])
table.append(['(cm)', '(cm)', '(cm)','(cm)', '(cm)', '(cm)'])
lr=4.9
wr=2.5
hr=1.2
lc=4.90
wc=2.54
hc=1.27
deltar=0.1
deltac=0.01
for i in range(0,len(x)):
    xx = x[i]
    yy = y[i]
    ttable.append([lr, deltar, wr, deltar, hr, deltar])able.append([lr, deltar, wr, deltar, hr, deltar])

table
```

l	deltal	w	deltaw	h	deltah

(cm)	(cm)	(cm)	(cm)	(cm)	(cm)

4.9	4.9

0.1	0.01

2.5	2.54

0.1	0.01

1.2	1.27

0.1	0.01

```python
# code below demonstrates...

import numpy as np
x = [7,10,15,20,25,30,35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95]
y= [0.228,0.298,0.441,0.568,0.697,0.826,0.956, 1.084, 1.211, 1.339,1.468, 1.599, 1.728, 1.851, 1.982, 2.115, 2.244, 2.375, 2.502]
plt.scatter(x, y)
plt.title('Linearity test')
plt. xlabel('Length (cm)')
plt. ylabel('Voltage (V)')
fit = np.polyfit(x,y,1)
fit_fn = np.poly1d(fit)
plt.plot(x,y, 'yo', x, fit_fn(x), '--k')
m,b = np.polyfit(x, y, 1)
print ('m={0}'.format(m))
print ('b={0}'.format(b))
plt.show()
```

    m=0.0258164673413
    b=0.0491959521619

#### 2. Wheatstone bridge measurements

```python
Rk = 3.5 # kOhms
table = ListTable()
table.append(['Ru', 'Ru, acc', 'L1', 'L2', 'Ru, wheatstone', 'Disc'])
table.append(['(kOhms)', '(kOhms)', '(cm)', '(cm)', '(kOhms)', ' % '])
x = [0.470,0.680,1.000, 1.500]
y= [0.512,0.712,1.131,1.590]
z= [88.65, 84.50, 76.90, 69.80]
for i in range(0,len(x)):
    xx = x[i]
    yy = y[i]
    zz = z[i]
    Rw = (100.0 - zz)/zz*Rk
    Disc = (Rw-yy)/yy*100.0
    table.append([xx, yy, zz, 100.0-zz,Rw, Disc])
table
```

Ru	Ru, acc	L1	L2	Ru, wheatstone	Disc

(kOhms)	(kOhms)	(cm)	(cm)	(kOhms)	 %

0.47	0.512	88.65	11.35	0.448110547095	-12.4784087704

0.68	0.712	84.5	15.5	0.64201183432	-9.82979855063

1.0	1.131	76.9	23.1	1.05136540962	-7.04107784059

1.5	1.59	69.8	30.2	1.51432664756	-4.75933034186

```python
x = [0.470,0.680,1.000, 1.500]
y= [0.512,0.712,1.131,1.590]
z= [88.65, 84.50, 76.90, 69.80]
for i in range(0,len(x)):
    xx = x[i]
    yy = y[i]
    zz = z[i]
    Rw = (100.0 - zz)/zz*Rk
    Disc = (Rw-yy)/yy*100.0

plt.scatter(yy, Disc)
plt.title('Discrepancy vs Resistance')
plt. xlabel('Resistance (kOhms)')
plt. ylabel('Discrepancy (%)')
plt.show()
```

```python

```


今回のレポートでは、①オートエンコーダの作成、②再帰型ニューラルネットワークの作成を試みた。

①コブダクラス型生産関数を再現できるオートエンコーダの作成が目標である。

```python
%matplotlib inline

import numpy as np
import pylab as pl
import math
from sympy import *
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from mpl_toolkits.mplot3d import Axes3D
```

```python
from NN import NN
```

定義域は0≤x≤1である。

コブ・ダグラス型生産関数は以下の通りである。

z = x_1**0.5*x_2*0.5

```python
def example1(x_1, x_2):
    z = x_1**0.5*x_2*0.5
    return z

fig = pl.figure()
ax = Axes3D(fig)

X = np.arange(0, 1, 0.1)
Y = np.arange(0, 1, 0.1)

X, Y = np.meshgrid(X, Y)
Z = example1(X, Y)

ax.plot_surface(X, Y, Z, rstride=1, cstride=1)

pl.show()
```

NNのクラスはすでにNN.pyからimportしてある。

```python
nn = NN()
```

以下に使い方を説明する。

初めに、このコブ・ダグラス型生産関数を用いる。

```python
x_1 = Symbol('x_1')
x_2 = Symbol('x_2')
f = x_1**0.5*x_2*0.5
```

入力層、中間層、出力層を作る関数を実行する。引数には層の数を用いる。

```python
nn.set_input_layer(2)
```

    array([ 0.,  0.])

```python
nn.set_hidden_layer(2)
```

    (array([ 0.,  0.]), array([ 0.,  0.]))

```python
nn.set_output_layer(2)
```

    (array([ 0.,  0.]), array([ 0.,  0.]), array([ 0.,  0.]))

nn.set_hidden_layer()は同時にシグモイド関数で変換する前の中間層も作る。

set_output_layer()は同時にシグモイド関数で変換する前の出力層、さらに教師データを入れる配列も作る。

nn.setup()で入力層ー中間層、中間層ー出力層間の重みを入れる配列を作成する。

nn.initialize()で重みを初期化する。重みは-1/√d ≤ w ≤ 1/√d (dは入力層及び中間層の数)の範囲で一様分布から決定される。

```python
nn.setup()
nn.initialize()
```

nn.supervised_function(f, idata)は教師データを作成する。引数は関数とサンプルデータをとる。

```python
idata = [1, 2]
```

```python
nn.supervised_function(f, idata)
```

nn.simulate(N, eta)は引数に更新回数と学習率をとる。普通はN=1で行うべきかもしれないが、工夫として作成してみた。N回学習した後に出力層を返す。

```python
nn.simulate(1, 0.1)
```

    array([ 0.35573033,  0.46501136])

nn.calculation()は学習せずに入力層から出力層の計算を行う。nn.simulate()内にも用いられている。

次に実際に学習を行う。サンプルデータは、

```python
X = np.arange(0, 1, 0.2)
Y = np.arange(0, 1, 0.2)
print X, Y
```

    [ 0.   0.2  0.4  0.6  0.8] [ 0.   0.2  0.4  0.6  0.8]

の組み合わせである。

```python
X = np.arange(0, 1, 0.2)
Y = np.arange(0, 1, 0.2)

a = np.array([])
b = np.array([])
c = np.array([])

nn = NN()
nn.set_network()

for x in X:
    for y in Y:
        a = np.append(a, x)
        b = np.append(b, y)

for i in range(100):
        l = np.random.choice([i for i in range(len(a))])
        m = nn.main2(1, f, [a[l], b[l]], 0.5)

for x in X:
    for y in Y:
        idata = [x, y]
        c = np.append(c, nn.realize(f, idata))
```

```python
a
```

    array([ 0. ,  0. ,  0. ,  0. ,  0. ,  0.2,  0.2,  0.2,  0.2,  0.2,  0.4,
            0.4,  0.4,  0.4,  0.4,  0.6,  0.6,  0.6,  0.6,  0.6,  0.8,  0.8,
            0.8,  0.8,  0.8])

```python
b
```

    array([ 0. ,  0.2,  0.4,  0.6,  0.8,  0. ,  0.2,  0.4,  0.6,  0.8,  0. ,
            0.2,  0.4,  0.6,  0.8,  0. ,  0.2,  0.4,  0.6,  0.8,  0. ,  0.2,
            0.4,  0.6,  0.8])

```python
c
```

    array([ 0.33477702,  0.33324062,  0.33171718,  0.33021575,  0.32874465,
            0.32133959,  0.3198509 ,  0.31838581,  0.3169523 ,  0.3155574 ,
            0.3083959 ,  0.3069744 ,  0.30558559,  0.30423612,  0.30293148,
            0.29613312,  0.29479445,  0.29349569,  0.29224199,  0.29103729,
            0.28469555,  0.28345097,  0.28225153,  0.28110086,  0.28000137])

例えば(0, 0)を入力すると0.52328635を返している（つまりa[0]とb[0]を入力して、c[0]の値を返している）。

ここでは交差検定は用いていない。

```python
fig = pl.figure()
ax = Axes3D(fig)

ax.scatter(a, b, c)

pl.show()
```

確率的勾配降下法を100回繰り返したが見た感じから近づいている。回数を10000回に増やしてみる。

```python
X = np.arange(0, 1, 0.2)
Y = np.arange(0, 1, 0.2)

a = np.array([])
b = np.array([])
c = np.array([])

nn = NN()
nn.set_network()

for x in X:
    for y in Y:
        a = np.append(a, x)
        b = np.append(b, y)

for i in range(10000):
        l = np.random.choice([i for i in range(len(a))])
        m = nn.main2(1, f, [a[l], b[l]], 0.5)

for x in X:
    for y in Y:
        idata = [x, y]
        c = np.append(c, nn.realize(f, idata))
```

```python
fig = pl.figure()
ax = Axes3D(fig)

ax.scatter(a, b, c)

pl.show()
```

見た感じ随分近づいているように見える。

最後に交差検定を行う。

初めに学習回数が極めて少ないNNである。

```python
X = np.arange(0, 1, 0.2)
Y = np.arange(0, 1, 0.2)

a = np.array([])
b = np.array([])
c = np.array([])

for x in X:
    for y in Y:
        a = np.append(a, x)
        b = np.append(b, y)

evl = np.array([])

for i in range(len(a)):
    nn = NN()
    nn.set_network()
    for j in range(1):
        l = np.random.choice([i for i in range(len(a))])
        if l != i:
            nn.main2(1, f, [a[l], b[l]], 0.5)
    idata = [a[i], b[i]]
    est = nn.realize(f, idata)
    evl = np.append(evl, math.fabs(est - nn.supervised_data))
```

```python
np.average(evl)
```

    0.17409222324854595

次に十分大きく(100回に)してみる。

```python
X = np.arange(0, 1, 0.2)
Y = np.arange(0, 1, 0.2)

a = np.array([])
b = np.array([])
c = np.array([])

nn = NN()
nn.set_network(h=7)

for x in X:
    for y in Y:
        a = np.append(a, x)
        b = np.append(b, y)

evl = np.array([])

for i in range(len(a)):
    nn = NN()
    nn.set_network()
    for j in range(100):
        l = np.random.choice([i for i in range(len(a))])
        if l != i:
            nn.main2(1, f, [a[l], b[l]], 0.5)
    idata = [a[i], b[i]]
    evl = np.append(evl, math.fabs(nn.realize(f, idata) - nn.supervised_data))
```

```python
np.average(evl)
```

    0.14482670064026229

誤差の平均であるので小さい方よい。
学習回数を増やした結果、精度が上がった。

最後にオートエンコーダを作成する。回数を増やした方がよいことが分かったため、10000回学習させてみる。

```python
nn = NN()
nn.set_network()
```

```python
X = np.arange(0, 1, 0.05)
Y = np.arange(0, 1, 0.05)

a = np.array([])
b = np.array([])
c = np.array([])

for x in X:
    for y in Y:
        a = np.append(a, x)
        b = np.append(b, y)

evl = np.array([])

s = [i for i in range(len(a))]

for j in range(1000):
    l = np.random.choice(s)
    nn.main2(1, f, [a[l], b[l]], 0.5)
```

```python
c = np.array([])

for i in range(len(a)):
    idata = [a[i], b[i]]
    c = np.append(c, nn.realize(f, idata))
```

```python
fig = pl.figure()
ax = Axes3D(fig)

ax.scatter(a, b, c)

pl.show()
```

十分再現できていることが分かる。

②ゲーム理論で用いられるTit for Tatを再現してみる。二人のプレーヤーが互いにRNNで相手の行動を予測し、相手の行動に対してTit for Tatに基づいた行動を選択する。

```python
from NN import RNN
```

最初の行動はRNNで指定できないので、所与となる。この初期値と裏切りに対する感応度で収束の仕方が決まる。

協調を1、裏切りを0としている。RNNの予測値は整数値でないが、p=（RNNの出力値）で次回に協調を行う。

例１：1期目に、プレーヤー１が協力、プレーヤー２が裏切り。

```python
nn1 = RNN()
nn1.set_network()
nn2 = RNN()
nn2.set_network()

idata1 = [[1, 0]]
idata2 = [[0, 1]]
sdata1 = [[0]]
sdata2 = [[1]]

for t in range(20):

    for i in range(10):
        nn1.main2(idata1, sdata2, 0.9)
        nn2.main2(idata2, sdata1, 0.9)

    idata1.append([sdata1[-1][0], sdata2[-1][0]])
    idata2.append([idata1[-1][1], idata1[-1][0]])

    n1r = nn1.realize(idata1)
    n2r = nn2.realize(idata1)
    sdata1.append([np.random.choice([1, 0], p=[n1r, 1-n1r])])

    sdata2.append([np.random.choice([1, 0], p=[n2r, 1-n2r])])

idata.append([sdata1[-1][0], sdata2[-1][0]])
print nn1.realize(idata1), nn2.realize(idata), idata1
```

下の図より、最初は交互に相手にしっぺ返しをしているが、やがて両者が裏切り合うこと状態に収束する。

```python
p1 = []
p2 = []
for i in range(len(idata1)):
    p1.append(idata1[i][0])
for i in range(len(idata2)):
    p2.append(idata2[i][0])
plt.plot(p1, label='player1')
plt.plot(p2, label='player2')
```

例２：1期目に、プレーヤー１が協力、プレーヤー２が協力。ただし、プレーヤー２は相手の裏切りをかなり警戒している。

警戒を表すためにp=（RNNの出力値 - 0.2）とする。p<0の場合はp=0に直す。

```python
nn1 = RNN()
nn1.set_network()
nn2 = RNN()
nn2.set_network()

idata1 = [[1, 1]]
idata2 = [[1, 1]]
sdata1 = [[1]]
sdata2 = [[1]]

for t in range(20):

    for i in range(10):
        nn1.main2(idata1, sdata2, 0.9)
        nn2.main2(idata2, sdata1, 0.9)

    idata1.append([sdata1[-1][0], sdata2[-1][0]])
    idata2.append([idata1[-1][1], idata1[-1][0]])

    n1r = nn1.realize(idata1)
    n2r = nn2.realize(idata1)

    prob1 = n1r
    prob2 = n2r - 0.3

    if prob2 < 0:
        prob2 = 0

    sdata1.append([np.random.choice([1, 0], p=[prob1, 1-prob1])])

    sdata2.append([np.random.choice([1, 0], p=[prob2, 1-prob2])])

idata.append([sdata1[-1][0], sdata2[-1][0]])
print nn1.realize(idata1), nn2.realize(idata), idata1
```

```python
p1 = []
p2 = []
for i in range(len(idata1)):
    p1.append(idata1[i][0])
for i in range(len(idata2)):
    p2.append(idata2[i][0])
plt.plot(p1, label='player1')
plt.plot(p2, label='player2')
```

例３：次に相手の行動を完全には観測できない場合を考える。t期の相手の行動をt+1期にノイズが加わって知る。例えば、1期目に相手が協調したことを、確率90%で2期目に正しく知れるが、10%で裏切りと誤って伝わる場合である。

ノイズは20%の確率で加わるものとする。その他の条件は例1と同じにした。

```python
nn1 = RNN()
nn1.set_network()
nn2 = RNN()
nn2.set_network()

idata1 = [[1, 0]]
idata2 = [[0, 1]]
sdata1 = [[0]]
sdata2 = [[1]]

for t in range(20):

    for i in range(10):
        nn1.main2(idata1, sdata2, 0.9)
        nn2.main2(idata2, sdata1, 0.9)

    idata1.append([sdata1[-1][0], np.random.choice([sdata2[-1][0], 1-sdata2[-1][0]], p=[0.8, 0.2])])
    idata2.append([sdata2[-1][0], np.random.choice([sdata1[-1][0], 1-sdata1[-1][0]], p=[0.8, 0.2])])

    n1r = nn1.realize(idata1)
    n2r = nn2.realize(idata1)

    prob1 = n1r
    prob2 = n2r

    sdata1.append([np.random.choice([1, 0], p=[prob1, 1-prob1])])

    sdata2.append([np.random.choice([1, 0], p=[prob2, 1-prob2])])

idata.append([sdata1[-1][0], sdata2[-1][0]])
print nn1.realize(idata1), nn2.realize(idata), idata1
```

```python
p1 = []
p2 = []
for i in range(len(idata1)):
    p1.append(idata1[i][0])
for i in range(len(idata2)):
    p2.append(idata2[i][0])
plt.plot(p1, label='player1')
plt.plot(p2, label='player2')
```


##                           Tarea 5 ecuaciones del documento mecanica celeste
### Integrante del grupo
#### Jose David Ramos Rios
#### incluimos ecuaciones que posterior seran utilizadas en la solucion del problema sistema binario supernova.

## Tabla de parametros
| excentricidad | e = 0.617155 ± 0.000007 |
|------|------|
|   periodo orbital  | T = 27,906,98172  ± 0.00005 s|
|Proyeccion de semimajor|a1 sin i = 2.3424  ± 0.0007 lights|
|axis of pulsar  = |  (7.0223  ± 0.002)x10 exp 10 cm |
|tasa de avance de periastron|w = 4.226 ± 0.002 yr -1|
|seno de angulo de inclinacion|sin i = 0.81 ± 0.16|
|masa funcion |f = 0.13126 +- 0.00002 M|

### centro de masa de la explosion
\begin{equation}
\frac{(qm1)*r1+(m2r2)}{q(m1) + m2 }
\end{equation}
### energia cinetica de la explosion binaria
\begin{equation}
E = \ 0.5(qm1)v1^{2}+0.5 m2^{2} -0.5(qm1+m2)^R2-G(qm1)m2 /a0
\end{equation}
### velocidad de la preexplosion binaria
\begin{equation}
v^2 = \ G(m1+m2)/a0
\end{equation}
### energia de la preexplosion
\begin{equation}
E = \ 0.5 mv^2[q(m1-m2-2qm1)/8qm1+m2)]
\end{equation}

```python
from IPython.display import Image
from IPython.core.display import HTML
Image(url= "https://i.stack.imgur.com/jcFfG.png", width=550, height=550)
```

```python

```


```python
import numpy as np
import numba
import matplotlib.pyplot as plt
import sympy as sym
plt.style.use('presentation.mplstyle')
%matplotlib notebook
```

### Fortescue

```python
alpha = np.exp(2.0/3*np.pi*1j)
A_0a =  np.array([[1, 1, 1],
                  [1, alpha**2, alpha],
                  [1, alpha, alpha**2]])

A_a0 = 1/3* np.array([[1, 1, 1],
                      [1, alpha, alpha**2],
                      [1, alpha**2, alpha]])
```

## System and prameters

```python
theta = np.deg2rad(0.0)
delta = np.deg2rad(0.0)
V_z =   0.0*np.exp(1j*0.0)
V_p = 400.0/np.sqrt(3)*np.exp(1j*theta)
V_n =  20.0*np.exp(1j*0.0)

E_z =  0.0*np.exp(1j*0.0)
E_p = 420.0/np.sqrt(3)*np.exp(1j*(theta+delta))
E_n =   0.0*np.exp(1j*0.0)

V_012 = np.array([[V_z],[V_p],[V_n]])
E_012 = np.array([[E_z],[E_p],[E_n]])

V_abc = A_0a @ V_012
E_abc = A_0a @ E_012

L = 200.0e-6
R = 0.001
Omega = 2*np.pi*50.0
```

## Current with dq

```python
# Park:
theta_pll = 0.0

v_dq_z = V_z*np.exp( 1j*theta_pll)*np.sqrt(2)
v_dq_p = V_p*np.exp( 1j*theta_pll)*np.sqrt(2)
v_dq_n = V_n*np.exp( 1j*theta_pll)*np.sqrt(2)

v_d_z = v_dq_z.real
v_q_z = v_dq_z.imag
v_d_p = v_dq_p.real
v_q_p = v_dq_p.imag
v_d_n = v_dq_n.real
v_q_n = v_dq_n.imag

e_dq_z = E_z*np.exp( 1j*theta_pll)*np.sqrt(2)
e_dq_p = E_p*np.exp( 1j*theta_pll)*np.sqrt(2)
e_dq_n = E_n*np.exp( 1j*theta_pll)*np.sqrt(2)

e_d_z = e_dq_z.real
e_q_z = e_dq_z.imag
e_d_p = e_dq_p.real
e_q_p = e_dq_p.imag
e_d_n = e_dq_n.real
e_q_n = e_dq_n.imag
```

### Currents in dq frame
For symbolic obtention see [dq steady state](#dq_steady_state)

```python
i_d_p = (L*Omega*e_q_p - L*Omega*v_q_p + R*e_d_p - R*v_d_p)/(L**2*Omega**2 + R**2)
i_q_p = (-L*Omega*e_d_p + L*Omega*v_d_p + R*e_q_p - R*v_q_p)/(L**2*Omega**2 + R**2)
i_d_n = (L*Omega*e_q_n - L*Omega*v_q_n + R*e_d_n - R*v_d_n)/(L**2*Omega**2 + R**2)
i_q_n = (-L*Omega*e_d_n + L*Omega*v_d_n + R*e_q_n - R*v_q_n)/(L**2*Omega**2 + R**2)

i_z = 0.0
i_p = i_d_p + 1j*i_q_p
i_n = i_d_n + 1j*i_q_n

I_abc_dq = A_0a @ np.array([i_z,i_p,i_n]).reshape(3,1)/np.sqrt(2)
I_abc_dq
```

    array([[  -2.14062632+134.49951858j],
           [-433.63896450 -74.16837233j],
           [ 435.77959082 -60.33114626j]])

## Current with abc and primitive

```python
Z_1 = R + 1j*Omega*L
Z_0 = Z_1
Z_2 = Z_1

Z_012 = np.diag([Z_0,Z_1,Z_2])
Z_abc = A_0a @ Z_012 @ A_a0

I_abc_abc = np.linalg.solve(Z_abc,E_abc-V_abc)
I_abc_abc
```

    array([[  -2.14062632+134.49951858j],
           [-433.63896450 -74.16837233j],
           [ 435.77959082 -60.33114626j]])

## Current with 012 and sequence impedances

```python
I_z = (E_z - V_z)/Z_0
I_p = (E_p - V_p)/Z_1
I_n = (E_n - V_n)/Z_2

I_abc_012 = A_0a @ np.array([I_z,I_p,I_n]).reshape(3,1)
I_abc_012
```

    array([[  -2.14062632+134.49951858j],
           [-433.63896450 -74.16837233j],
           [ 435.77959082 -60.33114626j]])

```python
I_n
```

    (-5.0647762593031977+318.22927776605837j)

```python
np.sum(abs(I_abc_dq - I_abc_abc)) and np.sum(abs(I_abc_abc - I_abc_012)) <1.0e-8
```

    True

## Powers

```python
I_abc = I_abc_abc
V_abc.T @ np.conj(I_abc)
```

    array([[ 1722.02392816+108197.95444046j]])

```python
t = np.linspace(0.0,0.04,200)

e_abc = np.real(E_abc*np.exp(1j*Omega*t))*np.sqrt(2)
v_abc = np.real(V_abc*np.exp(1j*Omega*t))*np.sqrt(2)
i_abc = np.real(I_abc*np.exp(1j*Omega*t))*np.sqrt(2)
v_abc_p = np.real(V_abc*np.exp(1j*Omega*t-np.pi/2))*np.sqrt(2)
p = np.sum(v_abc*i_abc,axis=0)
#q = -((v_abc[1]- v_abc[2]) * i_abc[0] + (v_abc[2]- v_abc[0]) * i_abc[1]  + (v_abc[0]- v_abc[1]) * i_abc[2] )/np.sqrt(3)
q = -((v_abc[2]- v_abc[1]) * i_abc[0] + (v_abc[0]- v_abc[2]) * i_abc[1]  + (v_abc[1]- v_abc[0]) * i_abc[2] )/np.sqrt(3)

p_e = np.sum(e_abc*i_abc,axis=0)
#q = -((v_abc[1]- v_abc[2]) * i_abc[0] + (v_abc[2]- v_abc[0]) * i_abc[1]  + (v_abc[0]- v_abc[1]) * i_abc[2] )/np.sqrt(3)
q_e = -((e_abc[2]- e_abc[1]) * i_abc[0] + (e_abc[0]- e_abc[2]) * i_abc[1]  + (e_abc[1]- e_abc[0]) * i_abc[2] )/np.sqrt(3)
q_lipo = np.sum(v_abc_p*i_abc,axis=0)

#q = (i_a*(v_c-v_b) + i_b*(v_a-v_c) + i_c*(v_b-v_a))/np.sqrt(3)
P = np.average(p)
Q = np.average(q)

print('S = ',P+1j*Q)
```

    S =  (1705.35627352+147542.965255j)

```python
fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(8, 6), sharex = True)

axes[0].plot(t, v_abc[0,:])
axes[0].plot(t, v_abc[1,:])
axes[0].plot(t, v_abc[2,:])
axes[0].plot(t, e_abc[0,:])
axes[0].plot(t, e_abc[1,:])
axes[0].plot(t, e_abc[2,:])

axes[1].plot(t, i_abc[0,:])
axes[1].plot(t, i_abc[1,:])
axes[1].plot(t, i_abc[2,:])

axes[2].plot(t, p/1000)
axes[2].plot(t, q/1000)
axes[2].plot(t, p_e/1000)
axes[2].plot(t, q_e/1000)
axes[2].plot(t, q_lipo/1000)

```

    <IPython.core.display.Javascript object>

    [<matplotlib.lines.Line2D at 0x7fd7849f9be0>]

A_0a @ V_012 = Z_abc @ A_0a @ I_012
A_0a @ V_012 = Z_abc @ A_0a @ I_012
V_012 = A_a0 @ Z_abc @ A_0a @ I_012
Z_012 = A_a0 @ Z_abc @ A_0a

```python
p_cte_ref = 1.5*i_d_n*v_d_n + 1.5*i_d_p*v_d_p + 1.5*i_q_n*v_q_n + 1.5*i_q_p*v_q_p
p_cos_ref = 1.5*i_d_n*v_d_p + 1.5*i_d_p*v_d_n + 1.5*i_q_n*v_q_p + 1.5*i_q_p*v_q_n
p_sin_ref = -1.5*i_d_n*v_q_p + 1.5*i_d_p*v_q_n + 1.5*i_q_n*v_d_p - 1.5*i_q_p*v_d_n
q_cte_ref = 1.5*i_d_n*v_q_n + 1.5*i_d_p*v_q_p - 1.5*i_q_n*v_d_n - 1.5*i_q_p*v_d_p
q_cos_ref = 1.5*i_d_n*v_q_p + 1.5*i_d_p*v_q_n - 1.5*i_q_n*v_d_p - 1.5*i_q_p*v_d_n
q_sin_ref = 1.5*i_d_n*v_d_p - 1.5*i_d_p*v_d_n + 1.5*i_q_n*v_q_p - 1.5*i_q_p*v_q_n
p_cte_ref
```

    1722.0239281630923

```python
p_cos_ref
```

    -3333.5309278310765

```python
p_sin_ref
```

    231499.49656964012

```python
q_cte_ref
```

    108197.95444046022

```python
def fun(x):
    p_cte_ref = 1000.0
    q_cte_ref =  200.0
    eq_p = - p_cte_ref + 1.5*i_d_n*v_d_n + 1.5*i_d_p*v_d_p + 1.5*i_q_n*v_q_n + 1.5*i_q_p*v_q_p
    p_cos_ref = 1.5*i_d_n*v_d_p + 1.5*i_d_p*v_d_n + 1.5*i_q_n*v_q_p + 1.5*i_q_p*v_q_n
    p_sin_ref = -1.5*i_d_n*v_q_p + 1.5*i_d_p*v_q_n + 1.5*i_q_n*v_d_p - 1.5*i_q_p*v_d_n
    eq_q = - q_cte_ref + 1.5*i_d_n*v_q_n + 1.5*i_d_p*v_q_p - 1.5*i_q_n*v_d_n - 1.5*i_q_p*v_d_p
    q_cos_ref = 1.5*i_d_n*v_q_p + 1.5*i_d_p*v_q_n - 1.5*i_q_n*v_d_p - 1.5*i_q_p*v_d_n
    q_sin_ref = 1.5*i_d_n*v_d_p - 1.5*i_d_p*v_d_n + 1.5*i_q_n*v_q_p - 1.5*i_q_p*v_q_n

    return [eq_p,p_cos_ref,p_sin_ref,eq_q,q_cos_ref,q_sin_ref]
```

## Symbolic: dq steady state

```python
from sympy.functions import re,im

v_d_p,v_q_p,v_d_n,v_q_n = sym.symbols('v_d_p,v_q_p,v_d_n,v_q_n',real=True)
i_d_p,i_q_p,i_d_n,i_q_n = sym.symbols('i_d_p,i_q_p,i_d_n,i_q_n',real=True)
e_d_p,e_q_p,e_d_n,e_q_n = sym.symbols('e_d_p,e_q_p,e_d_n,e_q_n',real=True)
R,L,Omega= sym.symbols('R,L,Omega',real=True)

di_d_p = 1.0/L*(e_d_p - R*i_d_p + L*Omega*i_q_p - v_d_p)
di_q_p = 1.0/L*(e_q_p - R*i_q_p - L*Omega*i_d_p - v_q_p)
di_d_n = 1.0/L*(e_d_n - R*i_d_n + L*Omega*i_q_n - v_d_n)
di_q_n = 1.0/L*(e_q_n - R*i_q_n - L*Omega*i_d_n - v_q_n)

sol = sym.solve([di_d_p,di_q_p,di_d_n,di_q_n],[i_d_p,i_q_p,i_d_n,i_q_n])

for item in [i_d_p,i_q_p,i_d_n,i_q_n]:
    print('{:s} = {:s}'.format(str(item) ,str(sym.simplify(sol[item]))))
```

    i_d_p = (L*Omega*e_q_p - L*Omega*v_q_p + R*e_d_p - R*v_d_p)/(L**2*Omega**2 + R**2)
    i_q_p = (-L*Omega*e_d_p + L*Omega*v_d_p + R*e_q_p - R*v_q_p)/(L**2*Omega**2 + R**2)
    i_d_n = (L*Omega*e_q_n - L*Omega*v_q_n + R*e_d_n - R*v_d_n)/(L**2*Omega**2 + R**2)
    i_q_n = (-L*Omega*e_d_n + L*Omega*v_d_n + R*e_q_n - R*v_q_n)/(L**2*Omega**2 + R**2)

```python

```

S = np.ones((1,3)) @ V_abc @ I_abc.conj = np.ones((1,3)) @ (A_0a @ V_012) @ (A_0a @ I_012).conj

S_1 = 3*V_1*I_1.conj
S_2 = 3*V_2*I_2.conj
## Q and P constants

```python
theta = np.deg2rad(0.0)
delta = np.deg2rad(0.0)
V_z =  0.0*np.exp(1j*0.0)
V_p = 400.0/np.sqrt(3)*np.exp(1j*theta)
V_n =  10.0*np.exp(1j*0.0)

V_012 = np.array([[V_z],[V_p],[V_n]])

V_abc = A_0a @ V_012
E_abc = A_0a @ E_012

L = 400.0e-6
R = 0.01
Omega = 2*np.pi*50.0

S_1_ref = 100.0e3 +1j*50e3
V_1 = V_p
V_2 = V_n
I_1 = np.conj(S_1_ref/(3*V_1))
I_2 = 0.0
E_1 = V_1 + I_1*Z_1
E_2 = V_2 + I_2*Z_2
E_012 = np.array([0.0,E_1,E_2]).reshape(3,1)
E_abc = A_0a @ E_012
```

```python
Z_1 = R + 1j*Omega*L
Z_0 = Z_1
Z_2 = Z_1

Z_012 = np.diag([Z_0,Z_1,Z_2])
Z_abc = A_0a @ Z_012 @ A_a0

I_abc_abc = np.linalg.solve(Z_abc,E_abc-V_abc)
I_abc_abc
```

    array([[ 144.33756730 -72.16878365j],
           [-134.66878365 -88.91560818j],
           [  -9.66878365+161.08439182j]])

## Powers

```python
I_012 = np.array([0.0,I_1,0.0]).reshape(3,1)
I_abc = A_0a @ I_012
#I_abc = I_abc_abc
V_abc.T @ np.conj(I_abc)
```

    array([[ 100000.+50000.j]])

```python
t = np.linspace(0.0,0.04,200)

e_abc = np.real(E_abc*np.exp(1j*Omega*t))*np.sqrt(2)
v_abc = np.real(V_abc*np.exp(1j*Omega*t))*np.sqrt(2)
i_abc = np.real(I_abc*np.exp(1j*Omega*t))*np.sqrt(2)

p = np.sum(v_abc*i_abc,axis=0)
#q = -((v_abc[1]- v_abc[2]) * i_abc[0] + (v_abc[2]- v_abc[0]) * i_abc[1]  + (v_abc[0]- v_abc[1]) * i_abc[2] )/np.sqrt(3)
q = -((v_abc[2]- v_abc[1]) * i_abc[0] + (v_abc[0]- v_abc[2]) * i_abc[1]  + (v_abc[1]- v_abc[0]) * i_abc[2] )/np.sqrt(3)

p_e = np.sum(e_abc*i_abc,axis=0)
#q = -((v_abc[1]- v_abc[2]) * i_abc[0] + (v_abc[2]- v_abc[0]) * i_abc[1]  + (v_abc[0]- v_abc[1]) * i_abc[2] )/np.sqrt(3)
q_e = -((e_abc[2]- e_abc[1]) * i_abc[0] + (e_abc[0]- e_abc[2]) * i_abc[1]  + (e_abc[1]- e_abc[0]) * i_abc[2] )/np.sqrt(3)

#q = (i_a*(v_c-v_b) + i_b*(v_a-v_c) + i_c*(v_b-v_a))/np.sqrt(3)
P = np.average(p)
Q = np.average(q)

print('S = ',P+1j*Q)
```

    S =  (100021.650635+50010.8253175j)

```python
fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(8, 6), sharex = True)

axes[0].plot(t, e_abc[0,:])
axes[0].plot(t, e_abc[1,:])
axes[0].plot(t, e_abc[2,:])
axes[0].plot(t, v_abc[0,:])
axes[0].plot(t, v_abc[1,:])
axes[0].plot(t, v_abc[2,:])

axes[1].plot(t, i_abc[0,:])
axes[1].plot(t, i_abc[1,:])
axes[1].plot(t, i_abc[2,:])

axes[2].plot(t, p/1000)
axes[2].plot(t, q/1000)
axes[2].plot(t, p_e/1000)
axes[2].plot(t, q_e/1000)
```

    <IPython.core.display.Javascript object>

    [<matplotlib.lines.Line2D at 0x7f974e29feb8>]

```python
A_a0@I_abc_abc
```

    array([[ -3.55271368e-14 +1.42108547e-14j],
           [  1.43520118e+02 +1.02723663e+01j],
           [  3.55271368e-14 +1.42108547e-14j]])

```python
V_abc.T @ I_abc.conj()
```

    array([[100000. -2.53840976e-12j]])

```python
I_012
```

    array([[   0.0000000+0.j],
           [ 144.3375673-0.j],
           [   0.0000000+0.j]])

```python

```

```python

```

```python

```

```python

```


# Batch Normalization
One way to make deep networks easier to train is to use more sophisticated optimization procedures such as SGD+momentum, RMSProp, or Adam. Another strategy is to change the architecture of the network to make it easier to train.
One idea along these lines is batch normalization which was proposed by [1] in 2015.

The idea is relatively straightforward. Machine learning methods tend to work better when their input data consists of uncorrelated features with zero mean and unit variance. When training a neural network, we can preprocess the data before feeding it to the network to explicitly decorrelate its features; this will ensure that the first layer of the network sees data that follows a nice distribution. However, even if we preprocess the input data, the activations at deeper layers of the network will likely no longer be decorrelated and will no longer have zero mean or unit variance since they are output from earlier layers in the network. Even worse, during the training process the distribution of features at each layer of the network will shift as the weights of each layer are updated.

The authors of [1] hypothesize that the shifting distribution of features inside deep neural networks may make training deep networks more difficult. To overcome this problem, [1] proposes to insert batch normalization layers into the network. At training time, a batch normalization layer uses a minibatch of data to estimate the mean and standard deviation of each feature. These estimated means and standard deviations are then used to center and normalize the features of the minibatch. A running average of these means and standard deviations is kept during training, and at test time these running averages are used to center and normalize features.

It is possible that this normalization strategy could reduce the representational power of the network, since it may sometimes be optimal for certain layers to have features that are not zero-mean or unit variance. To this end, the batch normalization layer includes learnable shift and scale parameters for each feature dimension.

[1] [Sergey Ioffe and Christian Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing
Internal Covariate Shift", ICML 2015.](https://arxiv.org/abs/1502.03167)

```python
# As usual, a bit of setup
import time
import numpy as np
import matplotlib.pyplot as plt
from cs231n.classifiers.fc_net import *
from cs231n.data_utils import get_CIFAR10_data
from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array
from cs231n.solver import Solver

%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# for auto-reloading external modules
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

def rel_error(x, y):
    """ returns relative error """
    return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))

def print_mean_std(x,axis=0):
    print('  means: ', x.mean(axis=axis))
    print('  stds:  ', x.std(axis=axis))
    print()
```

    run the following from the cs231n directory and try again:
    python setup.py build_ext --inplace
    You may also need to restart your iPython kernel

```python
# Load the (preprocessed) CIFAR10 data.
data = get_CIFAR10_data()
for k, v in data.items():
  print('%s: ' % k, v.shape)
```

    X_train:  (49000, 3, 32, 32)
    y_train:  (49000,)
    X_val:  (1000, 3, 32, 32)
    y_val:  (1000,)
    X_test:  (1000, 3, 32, 32)
    y_test:  (1000,)

## Batch normalization: forward
In the file `cs231n/layers.py`, implement the batch normalization forward pass in the function `batchnorm_forward`. Once you have done so, run the following to test your implementation.

Referencing the paper linked to above in [1] may be helpful!

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after batch normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before batch normalization:')
print_mean_std(a,axis=0)

gamma = np.ones((D3,))
beta = np.zeros((D3,))
# Means should be close to zero and stds close to one
print('After batch normalization (gamma=1, beta=0)')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)

gamma = np.asarray([1.0, 2.0, 3.0])
beta = np.asarray([11.0, 12.0, 13.0])
# Now means should be close to beta and stds close to gamma
print('After batch normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)
```

    Before batch normalization:
      means:  [ -2.3814598  -13.18038246   1.91780462]
      stds:   [27.18502186 34.21455511 37.68611762]

    After batch normalization (gamma=1, beta=0)
      means:  [5.32907052e-17 7.04991621e-17 4.22578639e-17]
      stds:   [0.99999999 1.         1.        ]

    After batch normalization (gamma= [1. 2. 3.] , beta= [11. 12. 13.] )
      means:  [11. 12. 13.]
      stds:   [0.99999999 1.99999999 2.99999999]

```python
# Check the test-time forward pass by running the training-time
# forward pass many times to warm up the running averages, and then
# checking the means and variances of activations after a test-time
# forward pass.

np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)

bn_param = {'mode': 'train'}
gamma = np.ones(D3)
beta = np.zeros(D3)

for t in range(50):
  X = np.random.randn(N, D1)
  a = np.maximum(0, X.dot(W1)).dot(W2)
  batchnorm_forward(a, gamma, beta, bn_param)

bn_param['mode'] = 'test'
X = np.random.randn(N, D1)
a = np.maximum(0, X.dot(W1)).dot(W2)
a_norm, _ = batchnorm_forward(a, gamma, beta, bn_param)

# Means should be close to zero and stds close to one, but will be
# noisier than training-time forward passes.
print('After batch normalization (test-time):')
print_mean_std(a_norm,axis=0)
```

    After batch normalization (test-time):
      means:  [-0.03927354 -0.04349152 -0.10452688]
      stds:   [1.01531428 1.01238373 0.97819988]

## Batch normalization: backward
Now implement the backward pass for batch normalization in the function `batchnorm_backward`.

To derive the backward pass you should write out the computation graph for batch normalization and backprop through each of the intermediate nodes. Some intermediates may have multiple outgoing branches; make sure to sum gradients across these branches in the backward pass.

Once you have finished, run the following to numerically check your backward pass.

```python
# Gradient check batchnorm backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
fx = lambda x: batchnorm_forward(x, gamma, beta, bn_param)[0]
fg = lambda a: batchnorm_forward(x, a, beta, bn_param)[0]
fb = lambda b: batchnorm_forward(x, gamma, b, bn_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = batchnorm_forward(x, gamma, beta, bn_param)
dx, dgamma, dbeta = batchnorm_backward(dout, cache)
#You should expect to see relative errors between 1e-13 and 1e-8
print('dx error: ', rel_error(dx_num, dx))
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
```

    dx error:  1.7029235612572515e-09
    dgamma error:  7.420414216247087e-13
    dbeta error:  2.8795057655839487e-12

## Batch normalization: alternative backward
In class we talked about two different implementations for the sigmoid backward pass. One strategy is to write out a computation graph composed of simple operations and backprop through all intermediate values. Another strategy is to work out the derivatives on paper. For example, you can derive a very simple formula for the sigmoid function's backward pass by simplifying gradients on paper.

Surprisingly, it turns out that you can do a similar simplification for the batch normalization backward pass too!

In the forward pass, given a set of inputs $X=\begin{bmatrix}x_1\\x_2\\...\\x_N\end{bmatrix}$,

we first calculate the mean $\mu$ and variance $v$.
With $\mu$ and $v$ calculated, we can calculate the standard deviation $\sigma$  and normalized data $Y$.
The equations and graph illustration below describe the computation ($y_i$ is the i-th element of the vector $Y$).

\begin{align}
& \mu=\frac{1}{N}\sum_{k=1}^N x_k  &  v=\frac{1}{N}\sum_{k=1}^N (x_k-\mu)^2 \\
& \sigma=\sqrt{v+\epsilon}         &  y_i=\frac{x_i-\mu}{\sigma}
\end{align}

The meat of our problem during backpropagation is to compute $\frac{\partial L}{\partial X}$, given the upstream gradient we receive, $\frac{\partial L}{\partial Y}.$ To do this, recall the chain rule in calculus gives us $\frac{\partial L}{\partial X} = \frac{\partial L}{\partial Y} \cdot \frac{\partial Y}{\partial X}$.

The unknown/hart part is $\frac{\partial Y}{\partial X}$. We can find this by first deriving step-by-step our local gradients at
$\frac{\partial v}{\partial X}$, $\frac{\partial \mu}{\partial X}$,
$\frac{\partial \sigma}{\partial v}$,
$\frac{\partial Y}{\partial \sigma}$, and $\frac{\partial Y}{\partial \mu}$,
and then use the chain rule to compose these gradients (which appear in the form of vectors!) appropriately to compute $\frac{\partial Y}{\partial X}$.

If it's challenging to directly reason about the gradients over $X$ and $Y$ which require matrix multiplication, try reasoning about the gradients in terms of individual elements $x_i$ and $y_i$ first: in that case, you will need to come up with the derivations for $\frac{\partial L}{\partial x_i}$, by relying on the Chain Rule to first calculate the intermediate $\frac{\partial \mu}{\partial x_i}, \frac{\partial v}{\partial x_i}, \frac{\partial \sigma}{\partial x_i},$ then assemble these pieces to calculate $\frac{\partial y_i}{\partial x_i}$.

You should make sure each of the intermediary gradient derivations are all as simplified as possible, for ease of implementation.

After doing so, implement the simplified batch normalization backward pass in the function `batchnorm_backward_alt` and compare the two implementations by running the following. Your two implementations should compute nearly identical results, but the alternative implementation should be a bit faster.

```python
np.random.seed(231)
N, D = 100, 500
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
out, cache = batchnorm_forward(x, gamma, beta, bn_param)

t1 = time.time()
dx1, dgamma1, dbeta1 = batchnorm_backward(dout, cache)
t2 = time.time()
dx2, dgamma2, dbeta2 = batchnorm_backward_alt(dout, cache)
t3 = time.time()

print('dx difference: ', rel_error(dx1, dx2))
print('dgamma difference: ', rel_error(dgamma1, dgamma2))
print('dbeta difference: ', rel_error(dbeta1, dbeta2))
print('speedup: %.2fx' % ((t2 - t1) / (t3 - t2)))
```

    dx difference:  1.6697540272642257e-12
    dgamma difference:  0.0
    dbeta difference:  0.0
    speedup: 1.86x

## Fully Connected Nets with Batch Normalization
Now that you have a working implementation for batch normalization, go back to your `FullyConnectedNet` in the file `cs231n/classifiers/fc_net.py`. Modify your implementation to add batch normalization.

Concretely, when the `normalization` flag is set to `"batchnorm"` in the constructor, you should insert a batch normalization layer before each ReLU nonlinearity. The outputs from the last layer of the network should not be normalized. Once you are done, run the following to gradient-check your implementation.

HINT: You might find it useful to define an additional helper layer similar to those in the file `cs231n/layer_utils.py`. If you decide to do so, do it in the file `cs231n/classifiers/fc_net.py`.

```python
np.random.seed(231)
N, D, H1, H2, C = 2, 15, 20, 30, 10
X = np.random.randn(N, D)
y = np.random.randint(C, size=(N,))

# You should expect losses between 1e-4~1e-10 for W,
# losses between 1e-08~1e-10 for b,
# and losses between 1e-08~1e-09 for beta and gammas.
for reg in [0, 3.14]:
  print('Running check with reg = ', reg)
  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,
                            reg=reg, weight_scale=5e-2, dtype=np.float64,
                            normalization='batchnorm')

  loss, grads = model.loss(X, y)
  print('Initial loss: ', loss)

  for name in sorted(grads):
    f = lambda _: model.loss(X, y)[0]
    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)
    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))
  if reg == 0: print()
```

    Running check with reg =  0
    Initial loss:  2.2611955101340957
    W1 relative error: 1.10e-04
    W2 relative error: 2.85e-06
    W3 relative error: 3.92e-10
    b1 relative error: 4.44e-08
    b2 relative error: 2.22e-08
    b3 relative error: 4.78e-11
    beta1 relative error: 7.33e-09
    beta2 relative error: 1.89e-09
    gamma1 relative error: 7.57e-09
    gamma2 relative error: 1.96e-09

    Running check with reg =  3.14
    Initial loss:  6.996533220108303
    W1 relative error: 1.98e-06
    W2 relative error: 2.28e-06
    W3 relative error: 1.11e-08
    b1 relative error: 1.73e-10
    b2 relative error: 5.55e-09
    b3 relative error: 2.23e-10
    beta1 relative error: 6.65e-09
    beta2 relative error: 3.48e-09
    gamma1 relative error: 5.94e-09
    gamma2 relative error: 4.14e-09

# Batchnorm for deep networks
Run the following to train a six-layer network on a subset of 1000 training examples both with and without batch normalization.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [100, 100, 100, 100, 100]

num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

weight_scale = 2e-2
bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

print('Solver with batch norm:')
bn_solver = Solver(bn_model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True,print_every=20)
bn_solver.train()

print('\nSolver without batch norm:')
solver = Solver(model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True, print_every=20)
solver.train()
```

    Solver with batch norm:
    (Iteration 1 / 200) loss: 2.340974
    (Epoch 0 / 10) train acc: 0.107000; val_acc: 0.115000
    (Epoch 1 / 10) train acc: 0.314000; val_acc: 0.266000
    (Iteration 21 / 200) loss: 2.039365
    (Epoch 2 / 10) train acc: 0.390000; val_acc: 0.279000
    (Iteration 41 / 200) loss: 2.036710
    (Epoch 3 / 10) train acc: 0.496000; val_acc: 0.316000
    (Iteration 61 / 200) loss: 1.769763
    (Epoch 4 / 10) train acc: 0.533000; val_acc: 0.309000
    (Iteration 81 / 200) loss: 1.268883
    (Epoch 5 / 10) train acc: 0.600000; val_acc: 0.325000
    (Iteration 101 / 200) loss: 1.263426
    (Epoch 6 / 10) train acc: 0.635000; val_acc: 0.317000
    (Iteration 121 / 200) loss: 1.108584
    (Epoch 7 / 10) train acc: 0.690000; val_acc: 0.341000
    (Iteration 141 / 200) loss: 1.176695
    (Epoch 8 / 10) train acc: 0.724000; val_acc: 0.318000
    (Iteration 161 / 200) loss: 0.682345
    (Epoch 9 / 10) train acc: 0.767000; val_acc: 0.319000
    (Iteration 181 / 200) loss: 0.891488
    (Epoch 10 / 10) train acc: 0.818000; val_acc: 0.328000

    Solver without batch norm:
    (Iteration 1 / 200) loss: 2.302332
    (Epoch 0 / 10) train acc: 0.129000; val_acc: 0.131000
    (Epoch 1 / 10) train acc: 0.283000; val_acc: 0.250000
    (Iteration 21 / 200) loss: 2.041970
    (Epoch 2 / 10) train acc: 0.316000; val_acc: 0.277000
    (Iteration 41 / 200) loss: 1.900473
    (Epoch 3 / 10) train acc: 0.373000; val_acc: 0.282000
    (Iteration 61 / 200) loss: 1.713156
    (Epoch 4 / 10) train acc: 0.390000; val_acc: 0.310000
    (Iteration 81 / 200) loss: 1.662208
    (Epoch 5 / 10) train acc: 0.434000; val_acc: 0.300000
    (Iteration 101 / 200) loss: 1.696063
    (Epoch 6 / 10) train acc: 0.536000; val_acc: 0.346000
    (Iteration 121 / 200) loss: 1.550786
    (Epoch 7 / 10) train acc: 0.530000; val_acc: 0.310000
    (Iteration 141 / 200) loss: 1.436340
    (Epoch 8 / 10) train acc: 0.624000; val_acc: 0.348000
    (Iteration 161 / 200) loss: 0.998066
    (Epoch 9 / 10) train acc: 0.660000; val_acc: 0.332000
    (Iteration 181 / 200) loss: 0.943247
    (Epoch 10 / 10) train acc: 0.740000; val_acc: 0.345000

Run the following to visualize the results from two networks trained above. You should find that using batch normalization helps the network to converge much faster.

```python
def plot_training_history(title, label, baseline, bn_solvers, plot_fn, bl_marker='.', bn_marker='.', labels=None):
    """utility function for plotting training history"""
    plt.title(title)
    plt.xlabel(label)
    bn_plots = [plot_fn(bn_solver) for bn_solver in bn_solvers]
    bl_plot = plot_fn(baseline)
    num_bn = len(bn_plots)
    for i in range(num_bn):
        label='with_norm'
        if labels is not None:
            label += str(labels[i])
        plt.plot(bn_plots[i], bn_marker, label=label)
    label='baseline'
    if labels is not None:
        label += str(labels[0])
    plt.plot(bl_plot, bl_marker, label=label)
    plt.legend(loc='lower center', ncol=num_bn+1)

plt.subplot(3, 1, 1)
plot_training_history('Training loss','Iteration', solver, [bn_solver], \
                      lambda x: x.loss_history, bl_marker='o', bn_marker='o')
plt.subplot(3, 1, 2)
plot_training_history('Training accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.train_acc_history, bl_marker='-o', bn_marker='-o')
plt.subplot(3, 1, 3)
plot_training_history('Validation accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.val_acc_history, bl_marker='-o', bn_marker='-o')

plt.gcf().set_size_inches(15, 15)
plt.show()
```

# Batch normalization and initialization
We will now run a small experiment to study the interaction of batch normalization and weight initialization.

The first cell will train 8-layer networks both with and without batch normalization using different scales for weight initialization. The second layer will plot training accuracy, validation set accuracy, and training loss as a function of the weight initialization scale.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [50, 50, 50, 50, 50, 50, 50]
num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

bn_solvers_ws = {}
solvers_ws = {}
weight_scales = np.logspace(-4, 0, num=20)
for i, weight_scale in enumerate(weight_scales):
  print('Running weight scale %d / %d' % (i + 1, len(weight_scales)))
  bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
  model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

  bn_solver = Solver(bn_model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
  bn_solver.train()
  bn_solvers_ws[weight_scale] = bn_solver

  solver = Solver(model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
  solver.train()
  solvers_ws[weight_scale] = solver
```

    Running weight scale 1 / 20
    Running weight scale 2 / 20
    Running weight scale 3 / 20
    Running weight scale 4 / 20
    Running weight scale 5 / 20
    Running weight scale 6 / 20
    Running weight scale 7 / 20
    Running weight scale 8 / 20
    Running weight scale 9 / 20
    Running weight scale 10 / 20
    Running weight scale 11 / 20
    Running weight scale 12 / 20
    Running weight scale 13 / 20
    Running weight scale 14 / 20
    Running weight scale 15 / 20
    Running weight scale 16 / 20
    Running weight scale 17 / 20
    Running weight scale 18 / 20
    Running weight scale 19 / 20
    Running weight scale 20 / 20

```python
# Plot results of weight scale experiment
best_train_accs, bn_best_train_accs = [], []
best_val_accs, bn_best_val_accs = [], []
final_train_loss, bn_final_train_loss = [], []

for ws in weight_scales:
  best_train_accs.append(max(solvers_ws[ws].train_acc_history))
  bn_best_train_accs.append(max(bn_solvers_ws[ws].train_acc_history))

  best_val_accs.append(max(solvers_ws[ws].val_acc_history))
  bn_best_val_accs.append(max(bn_solvers_ws[ws].val_acc_history))

  final_train_loss.append(np.mean(solvers_ws[ws].loss_history[-100:]))
  bn_final_train_loss.append(np.mean(bn_solvers_ws[ws].loss_history[-100:]))

plt.subplot(3, 1, 1)
plt.title('Best val accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best val accuracy')
plt.semilogx(weight_scales, best_val_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_val_accs, '-o', label='batchnorm')
plt.legend(ncol=2, loc='lower right')

plt.subplot(3, 1, 2)
plt.title('Best train accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best training accuracy')
plt.semilogx(weight_scales, best_train_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_train_accs, '-o', label='batchnorm')
plt.legend()

plt.subplot(3, 1, 3)
plt.title('Final training loss vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Final training loss')
plt.semilogx(weight_scales, final_train_loss, '-o', label='baseline')
plt.semilogx(weight_scales, bn_final_train_loss, '-o', label='batchnorm')
plt.legend()
plt.gca().set_ylim(1.0, 3.5)

plt.gcf().set_size_inches(15, 15)
plt.show()
```

## Inline Question 1:
Describe the results of this experiment. How does the scale of weight initialization affect models with/without batch normalization differently, and why?

## Answer:
batch normalization을 할 때가 안할 때보다 scale of weight initialization에 덜 영향을 받는다. <br/>
위 그래프에서 볼 수 있듯이 batch normalization을 진행한 경우 smooth하게 accuracy를 계산하는 것을 볼 수 있다.<br/>
또한 batch normalization을 한 모델이 아닌 것보다 accuracy가 높게 측정되며 training loss 도 더 낮게 나온다. 무엇보다 bn을 안한 모델은 loss가 발산하는 경우도 보인다.<br/><br/>

이는 batch normalization이 입력 데이터들의 분산과 분포를 어느정도 균일하게 잡아주기 때문이다.

# Batch normalization and batch size
We will now run a small experiment to study the interaction of batch normalization and batch size.

The first cell will train 6-layer networks both with and without batch normalization using different batch sizes. The second layer will plot training accuracy and validation set accuracy over time.

```python
def run_batchsize_experiments(normalization_mode):
    np.random.seed(231)
    # Try training a very deep net with batchnorm
    hidden_dims = [100, 100, 100, 100, 100]
    num_train = 1000
    small_data = {
      'X_train': data['X_train'][:num_train],
      'y_train': data['y_train'][:num_train],
      'X_val': data['X_val'],
      'y_val': data['y_val'],
    }
    n_epochs=10
    weight_scale = 2e-2
    batch_sizes = [5,10,50]
    lr = 10**(-3.5)
    solver_bsize = batch_sizes[0]

    print('No normalization: batch size = ',solver_bsize)
    model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)
    solver = Solver(model, small_data,
                    num_epochs=n_epochs, batch_size=solver_bsize,
                    update_rule='adam',
                    optim_config={
                      'learning_rate': lr,
                    },
                    verbose=False)
    solver.train()

    bn_solvers = []
    for i in range(len(batch_sizes)):
        b_size=batch_sizes[i]
        print('Normalization: batch size = ',b_size)
        bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=normalization_mode)
        bn_solver = Solver(bn_model, small_data,
                        num_epochs=n_epochs, batch_size=b_size,
                        update_rule='adam',
                        optim_config={
                          'learning_rate': lr,
                        },
                        verbose=False)
        bn_solver.train()
        bn_solvers.append(bn_solver)

    return bn_solvers, solver, batch_sizes

batch_sizes = [5,10,50]
bn_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('batchnorm')
```

    No normalization: batch size =  5
    Normalization: batch size =  5
    Normalization: batch size =  10
    Normalization: batch size =  50

```python
plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 2:
Describe the results of this experiment. What does this imply about the relationship between batch normalization and batch size? Why is this relationship observed?

## Answer:
batch normalization은 batch size가 클수록 더 좋은 성능을 내는 것으로 보인다. <br/>
batch size가 작을 때는(파란색 선) batch normalization에서 계산되는 표본 평균과 분산이 모집단과 많이 차이가 날 수 있고, 이에따라 정확도가 떨어져 baseline 보다 성능이 안좋아지는 결과가 나온 것이다.

# Layer Normalization
Batch normalization has proved to be effective in making networks easier to train, but the dependency on batch size makes it less useful in complex networks which have a cap on the input batch size due to hardware limitations.

Several alternatives to batch normalization have been proposed to mitigate this problem; one such technique is Layer Normalization [2]. Instead of normalizing over the batch, we normalize over the features. In other words, when using Layer Normalization, each feature vector corresponding to a single datapoint is normalized based on the sum of all terms within that feature vector.

[2] [Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer Normalization." stat 1050 (2016): 21.](https://arxiv.org/pdf/1607.06450.pdf)

## Inline Question 3:
Which of these data preprocessing steps is analogous to batch normalization, and which is analogous to layer normalization?

1. Scaling each image in the dataset, so that the RGB channels for each row of pixels within an image sums up to 1.
2. Scaling each image in the dataset, so that the RGB channels for all pixels within an image sums up to 1.
3. Subtracting the mean image of the dataset from each image in the dataset.
4. Setting all RGB values to either 0 or 1 depending on a given threshold.

## Answer:
1 is analogous to layer normalization and 3 is analogous to batch normalization.<br/>
참고로 수업에서 말했듯이 이미지는 크기(scale)가 정해져 있기 때문에 normalizing 할 때 분산을 나눠주지 않는다.

# Layer Normalization: Implementation

Now you'll implement layer normalization. This step should be relatively straightforward, as conceptually the implementation is almost identical to that of batch normalization. One significant difference though is that for layer normalization, we do not keep track of the moving moments, and the testing phase is identical to the training phase, where the mean and variance are directly calculated per datapoint.

Here's what you need to do:

* In `cs231n/layers.py`, implement the forward pass for layer normalization in the function `layernorm_backward`.

Run the cell below to check your results.
* In `cs231n/layers.py`, implement the backward pass for layer normalization in the function `layernorm_backward`.

Run the second cell below to check your results.
* Modify `cs231n/classifiers/fc_net.py` to add layer normalization to the `FullyConnectedNet`. When the `normalization` flag is set to `"layernorm"` in the constructor, you should insert a layer normalization layer before each ReLU nonlinearity.

Run the third cell below to run the batch size experiment on layer normalization.

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after layer normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 =4, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before layer normalization:')
print_mean_std(a,axis=1)

gamma = np.ones(D3)
beta = np.zeros(D3)
# Means should be close to zero and stds close to one
print('After layer normalization (gamma=1, beta=0)')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)

gamma = np.asarray([3.0,3.0,3.0])
beta = np.asarray([5.0,5.0,5.0])
# Now means should be close to beta and stds close to gamma
print('After layer normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)
```

    Before layer normalization:
      means:  [-59.06673243 -47.60782686 -43.31137368 -26.40991744]
      stds:   [10.07429373 28.39478981 35.28360729  4.01831507]

    After layer normalization (gamma=1, beta=0)
      means:  [-4.81096644e-16  0.00000000e+00  7.40148683e-17 -5.92118946e-16]
      stds:   [0.99999995 0.99999999 1.         0.99999969]

    After layer normalization (gamma= [3. 3. 3.] , beta= [5. 5. 5.] )
      means:  [5. 5. 5. 5.]
      stds:   [2.99999985 2.99999998 2.99999999 2.99999907]

```python
# Gradient check batchnorm backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

ln_param = {}
fx = lambda x: layernorm_forward(x, gamma, beta, ln_param)[0]
fg = lambda a: layernorm_forward(x, a, beta, ln_param)[0]
fb = lambda b: layernorm_forward(x, gamma, b, ln_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = layernorm_forward(x, gamma, beta, ln_param)
dx, dgamma, dbeta = layernorm_backward(dout, cache)

#You should expect to see relative errors between 1e-12 and 1e-8
print('dx error: ', rel_error(dx_num, dx))
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
```

    dx error:  2.107279147162234e-09
    dgamma error:  4.519489546032799e-12
    dbeta error:  2.5842537629899423e-12

# Layer Normalization and batch size

We will now run the previous batch size experiment with layer normalization instead of batch normalization. Compared to the previous experiment, you should see a markedly smaller influence of batch size on the training history!

```python
ln_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('layernorm')

plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 4:
When is layer normalization likely to not work well, and why?

1. Using it in a very deep network
2. Having a very small dimension of features
3. Having a high regularization term

## Answer: **2**
layer normalization은 feature를 normalizing하는 것이기 때문에 small dimension of features에서 별로 좋은 성능을 내지 못할 것이다.

```python

```


# Applied Stochastic Processes HW_05

**11510691 程远星$\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\Exp}{\mathrm{E}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\AcA}{\mathscr{A}}
\newcommand{\FcF}{\mathscr{F}}
\newcommand{\Var}[2][\,\!]{\mathrm{Var}_{#1}\left[#2\right]}
\newcommand{\Cov}[2][\,\!]{\mathrm{Cov}_{#1}\left(#2\right)}
\newcommand{\Corr}[2][\,\!]{\mathrm{Corr}_{#1}\left(#2\right)}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathrm{N} \left( #1 \right)}
\newcommand{\ow}{\text{otherwise}}\void^\dagger$**

## Question 1

$$\begin{align}
&  P\CB{Y_{n+1} = \P{i_{n+1},i_{n+2}} \mid Y_n = \P{i_n,i_{n+1}}, Y_{n-1} = \P{i_{n-1},i_{n-2}}, \dots, Y_{0} = \P{ i_{0},i_{1}}} \\
=& P\CB{Y_{n+1} = \P{i_{n+1},i_{n+2}} \mid X_{n+1} = i_{n+1},X_{n} = i_n,\dots,X_0 = i_0} \\
=& P\CB{Y_{n+1} = \P{i_{n+1},i_{n+2}} \mid X_{n+1} = i_{n+1},X_{n} = i_n} \\
=& P\CB{Y_{n+1} = \P{i_{n+1},i_{n+2}} \mid Y_{n} = \P{i_{n},i_{n+1}}} \\
\end{align}$$

## Question 2

$\bspace$First we write $\mathbf{P}^\P{r}$ as

$$\mathbf{P}^\P{r} = \begin{Vmatrix}
q_{11} & \cdots & q_{1n} \\
\vdots & \ddots & \vdots \\
q_{n1} & \cdots & q_{nn}
\end{Vmatrix},\bspace q_{ij} > 0, 1 \leq i,j \leq n$$

$\bspace$for each column, we find the minimum, denoted as $q^{\text{c}}_{j}>0$ for $1 \leq j \leq n$. Then for the next transition matrix $\mathbf{P}^{\P{r+1}}$, all its entries has the form:

$$\sum_{j=1}^{n} p_{ij}\cdot q_{jk} \geq \sum_{j=1}^{n} p_{ij}\cdot q^{\text{c}}_{k} = q^{\text{c}}_{k} > 0$$

$\bspace$and thus by induction, we have for $n\geq r$, $\mathbf{P}^{n}$ has all positive entries.

## Question 3

$\bspace$First we write the one-step transition matrix for states $0,1,2,\geq3$:

$$\mathbf{P} = \begin{Vmatrix}
0.7 & 0.3 & 0 & 0 \\
0.7 & 0 & 0.3 & 0 \\
0.7 & 0 & 0 & 0.3 \\
0 & 0 & 0 & 1
\end{Vmatrix}$$

$\bspace$and $P_{0,3}^{6} = 0.0837$

>However, I came up with an alternative solution. Why it's wrong?
>
>Let $H = 0.3$ and $T=0.7$. Then the probability can be calculated as:
>
>$$\ffrac{4} {\d{\binom{6} {3}}}\times H^3T^3 + \ffrac{3+6}{\d{\binom{6}{2}}} \times H^4 T^2 + 1 \times H^5T^1 + 1 \times H^6 = 0.0066636$$

## Question 4

$$\mathbf{P} = \begin{Vmatrix}
0.7 & 0.3 \\
0.6 & 0.4
\end{Vmatrix},\mathbf{P}^{\P{3}} = \begin{Vmatrix}
0.667 & 0.333 \\
0.666 & 0.334
\end{Vmatrix},\mathbf{P}^{\P{4}} = \begin{Vmatrix}
0.6667 & 0.3333 \\
0.6666 & 0.3334
\end{Vmatrix}$$

$\P{1}$

$\bspace$The desired probability is $ 0.5 P^{\P{3}}_{1,1} + 0.5P^{\P{3}}_{2,1} = 0.6665$.

$\P{2}$

$\bspace$The desired probability is $ P^{\P{4}}_{1,1} = 0.6667$.

## Question 5

$\bspace$When $n \geq N$, there's only one possible state, $B$. and when $n < N$, by the definition of $N$ we can say that $X_n$ has never been in any state belonging to $\AcA$, such that $\CB{W_n, n<N} \subset S\backslash \AcA$. Then the overall state space for $\CB{W_n, N \geq 0}$ is $\P{S\backslash \AcA} \cup \CB{B}$.

$\bspace$When $n<N$, $W_n = X_n$, obviously it's a markov chain, and when $n\geq N$, we have

$$\begin{align}
&P\CB{W_{n+1}=i_{n+1} \mid W_{n} i_n, W_{n-1}=i_{n-1},\dots,W_0 = i_0} \\[0.7em]
=& P\CB{W_{n+1}=i_{n+1} \mid W_{n} =B, W_{n-1}=i_{n-1},\dots,W_0 = i_0} \\[0.7em]
=& P\CB{W_{n+1}=i_{n+1} \mid W_{n} =B} = \begin{cases}
1, &\text{if } i_{n+1} = B\\
0, &\text{if } i_{n+1} \neq B
\end{cases}
\end{align}$$

$\bspace$So $\CB{W_n,n\geq 0}$ is a Markov chain. As for it's transition probability, the result is obvious and also presented in the textbook.

## Question 6

$\bspace$Its classes are $\CB{0,1}\cup\CB{2}\cup\CB{3}\cup\CB{4}$. $\CB{0,1}$ and $\CB{2}$ are recurrent classes; $\CB{3}$ and $\CB{4}$ are transient classes.

Probability and Statistics/Applied Random Process/HW/HW_05.ipynb


```python
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Class 14: Prescott's Real Business Cycle Model I

In this notebook, we'll consider a centralized version of the model from pages 11-17 in Edward Prescott's article "Theory Ahead of Business Cycle Measurement in the Fall 1986 of the Federal Reserve Bank of Minneapolis' *Quarterly Review* (link to article: https://www.minneapolisfed.org/research/qr/qr1042.pdf). The model is just like the RBC model that we studying in the previous lecture, except that now we include an endogenous labor supply.

## Prescott's RBC Model with Labor

The equilibrium conditions for Prescott's RBC model with labor are:

\begin{align}
\frac{1}{C_t} & = \beta E_t \left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1}L_{t+1}^{1-\alpha} +1-\delta }{C_{t+1}}\right]\\
\frac{\varphi}{1-L_t} & = \frac{(1-\alpha)A_tK_t^{\alpha}L_t^{-\alpha}}{C_t} \\
Y_t & = A_t K_t^{\alpha}L_t^{1-\alpha}\\
K_{t+1} & = I_t + (1-\delta) K_t\\
Y_t & = C_t + I_t\\
\log A_{t+1} & = \rho \log A_t + \epsilon_{t+1}
\end{align}

where $\epsilon_{t+1} \sim \mathcal{N}(0,\sigma^2)$.

The objective is use `linearsolve` to simulate impulse responses to a TFP shock using the following parameter values for the simulation:

| $$\rho$$ | $$\sigma$$ | $$\beta$$ | $$\varphi$$ | $$\alpha$$ | $$\delta $$ |
|----------|------------|-------------|-----------|------------|-------------|
| 0.75     | 0.006      | 0.99      | 1.7317      | 0.35       |  0.025      |

The value for $\beta$ implies a steady state (annualized) real interest rate of about 4 percent:

\begin{align}
4 \cdot \left(\beta^{-1} - 1\right)  & \approx 0.04040
\end{align}

$\rho = 0.75$ and $\sigma = 0.006$ are consistent with the statistical properties of the cyclical component of TFP in the US. $\alpha$ is set so that, consistent with the long-run average of the US, the labor share of income is about 65 percent of GDP. The deprecation rate of capital is calibrated to be about 10 percent annually. Finally, $\varphi$ was chosen last to ensure that in the steady state households allocate about 33 percent of their available time to labor.

## Model Preparation

Before proceding, let's recast the model in the form required for `linearsolve`. Write the model with all variables moved to the left-hand side of the equations and dropping the expecations operator $E_t$ and the exogenous shock $\epsilon_{t+1}$:

\begin{align}
0 & = \beta\left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1}L_{t+1}^{1-\alpha} +1-\delta }{C_{t+1}}\right] - \frac{1}{C_t}\\
0 & = \frac{(1-\alpha)A_tK_t^{\alpha}L_t^{-\alpha}}{C_t} - \frac{\varphi}{1-L_t}\\
0 & = A_t K_t^{\alpha}L_t^{1-\alpha} - Y_t\\
0 & = I_t + (1-\delta) K_t - K_{t+1}\\
0 & = C_t + I_t - Y_t\\
0 & = \rho \log A_t - \log A_{t+1}
\end{align}

Remember, capital and TFP are called *state variables* because they're $t+1$ values are predetermined. Output, consumption, and investment are called a *costate* or *control* variables. Note that the model as 5 equations in 5 endogenous variables.

## Initialization, Approximation, and Solution

The next several cells initialize the model in `linearsolve` and then approximate and solve it.

```python
# Create a variable called 'parameters' that stores the model parameter values in a Pandas Series
parameters = pd.Series(dtype=float)
parameters['rho'] = .75
parameters['beta'] = 0.99
parameters['phi'] = 1.7317
parameters['alpha'] = 0.35
parameters['delta'] = 0.025

# Print the model's parameters
print(parameters)
```

    rho      0.7500
    beta     0.9900
    phi      1.7317
    alpha    0.3500
    delta    0.0250
    dtype: float64

```python
# Create a variable called 'sigma' that stores the value of sigma
sigma = 0.006
```

```python
# Create variable called 'var_names' that stores the variable names in a list with state variables ordered first
var_names = ['a','k','y','c','i','l']

# Create variable called 'shock_names' that stores an exogenous shock name for each state variable.
shock_names = ['e_a','e_k']
```

```python
# Define a function that evaluates the equilibrium conditions of the model solved for zero. PROVIDED
def equilibrium_equations(variables_forward,variables_current,parameters):

    # Parameters. PROVIDED
    p = parameters

    # Current variables. PROVIDED
    cur = variables_current

    # Forward variables. PROVIDED
    fwd = variables_forward

    # Define variable to store MPK. Will make things easier later.
    mpk = p.alpha*fwd.a*fwd.k**(p.alpha-1)*fwd.l**(1-p.alpha)

    # Define variable to store MPL. Will make things easier later.
    mpl = (1-p.alpha)*fwd.a*fwd.k**p.alpha*fwd.l**-p.alpha

    # Euler equation
    euler_equation = p.beta*(mpk+1-p.delta)/fwd.c - 1/cur.c

    # Labor-labor choice
    labor_leisure = mpl/cur.c - p.phi/(1-cur.l)

    # Production function
    production_function = cur.a*cur.k**p.alpha*cur.l**(1-p.alpha) - cur.y

    # Capital evolution. PROVIDED
    capital_evolution = cur.i + (1 - p.delta)*cur.k - fwd.k

    # Market clearing. PROVIDED
    market_clearing = cur.c+cur.i - cur.y

    # Exogenous tfp. PROVIDED
    tfp_process = p.rho*np.log(cur.a) - np.log(fwd.a)

    # Stack equilibrium conditions into a numpy array
    return np.array([
        euler_equation,
        labor_leisure,
        production_function,
        capital_evolution,
        market_clearing,
        tfp_process
        ])
```

Next, initialize the model using `ls.model` which takes the following required arguments:

* `equations`
* `n_states`
* `var_names`
* `shock_names`
* `parameters`

```python
# Initialize the model into a variable named 'rbc_model'
rbc_model = ls.model(equations = equilibrium_equations,
                 n_states=2,
                 var_names=var_names,
                 shock_names=shock_names,
                 parameters=parameters)
```

```python
# Compute the steady state numerically using .compute_ss() method of rbc_model
guess = [1,4,1,1,1,0.5]
rbc_model.compute_ss(guess)

# Print the computed steady state
print(rbc_model.ss)
```

    a     1.000000
    k    11.465953
    y     1.149904
    c     0.863256
    i     0.286649
    l     0.333330
    dtype: float64

```python
# Find the log-linear approximation around the non-stochastic steady state and solve using .approximate_and_solve() method of rbc_model
rbc_model.approximate_and_solve()
```

## Impulse Responses

Compute a 26 period impulse responses of the model's variables to a 0.01 unit shock to TFP in period 5.

```python
# Compute impulse responses
rbc_model.impulse(T=26,t0=5,shocks=[0.01,0])

# Print the first 10 rows of the computed impulse responses to the TFP shock
print(rbc_model.irs['e_a'].head(10))
```

        e_a         a         k         y         c         i         l
    0  0.00  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000
    1  0.00  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000
    2  0.00  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000
    3  0.00  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000
    4  0.00  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000
    5  0.01  0.010000  0.000000  0.015475  0.001708  0.056935  0.008423
    6  0.00  0.007500  0.001423  0.011858  0.002071  0.041333  0.005938
    7  0.00  0.005625  0.002421  0.009133  0.002304  0.029698  0.004093
    8  0.00  0.004219  0.003103  0.007077  0.002442  0.021036  0.002727
    9  0.00  0.003164  0.003551  0.005524  0.002511  0.014600  0.001719

Construct a $2\times3$ grid of plots of simulated TFP, output, labor, consumption, investment, and capital. Be sure to multiply simulated values by 100 so that vertical axis units are in "percent deviation from steady state."

```python
# Create figure. PROVIDED
fig = plt.figure(figsize=(18,8))

# Create upper-left axis. PROVIDED
ax = fig.add_subplot(2,3,1)
ax.plot(rbc_model.irs['e_a']['a']*100,'b',lw=5,alpha=0.75)
ax.set_title('TFP')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.5,2])
ax.grid()

# Create upper-center axis. PROVIDED
ax = fig.add_subplot(2,3,2)
ax.plot(rbc_model.irs['e_a']['y']*100,'b',lw=5,alpha=0.75)
ax.set_title('Output')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.5,2])
ax.grid()

# Create upper-right axis. PROVIDED
ax = fig.add_subplot(2,3,3)
ax.plot(rbc_model.irs['e_a']['l']*100,'b',lw=5,alpha=0.75)
ax.set_title('Labor')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.5,2])
ax.grid()

# Create lower-left axis. PROVIDED
ax = fig.add_subplot(2,3,4)
ax.plot(rbc_model.irs['e_a']['c']*100,'b',lw=5,alpha=0.75)
ax.set_title('Consumption')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.1,0.4])
ax.grid()

# Create lower-center axis. PROVIDED
ax = fig.add_subplot(2,3,5)
ax.plot(rbc_model.irs['e_a']['i']*100,'b',lw=5,alpha=0.75)
ax.set_title('Investment')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-2,8])
ax.grid()

# Create lower-right axis. PROVIDED
ax = fig.add_subplot(2,3,6)
ax.plot(rbc_model.irs['e_a']['k']*100,'b',lw=5,alpha=0.75)
ax.set_title('Capital')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.2,0.8])
ax.grid()

fig.tight_layout()
```