# Fun with Hidden Markov Models
*by Loren Lugosch*

This notebook introduces the Hidden Markov Model (HMM), a simple model for sequential data.

We will see:
- what an HMM is and when you might want to use it;
- the so-called "three problems" of an HMM; and
- how to implement an HMM in PyTorch.

(The code in this notebook can also be found at https://github.com/lorenlugosch/pytorch_HMM.)

A hypothetical scenario
------

To motivate the use of HMMs, imagine that you have a friend who gets to do a lot of travelling. Every day, this jet-setting friend sends you a selfie from the city they’re in, to make you envious.

How would you go about guessing which city the friend is in each day, just by looking at the selfies?

If the selfie contains a really obvious landmark, like the Eiffel Tower, it will be easy to figure out where the photo was taken. If not, it will be a lot harder to infer the city.

But we have a clue to help us: the city the friend is in each day is not totally random. For example, the friend will probably remain in the same city for a few days to sightsee before flying to a new city.

## The HMM setup

The hypothetical scenario of the friend travelling between cities and sending you selfies can be modeled using an HMM.

An HMM models a system that is in a particular state at any given time and produces an output that depends on that state.

At each timestep or clock tick, the system randomly decides on a new state and jumps into that state. The system then randomly generates an observation. The states are "hidden": we can't observe them. (In the cities/selfies analogy, the unknown cities would be the hidden states, and the selfies would be the observations.)

Let's denote the sequence of states as $\mathbf{z} = \{z_1, z_2, \dots, z_T \}$, where each state is one of a finite set of $N$ states, and the sequence of observations as $\mathbf{x} = \{x_1, x_2, \dots, x_T\}$. The observations could be discrete, like letters, or real-valued, like audio frames.

An HMM makes two key assumptions:
- **Assumption 1:** The state at time $t$ depends *only* on the state at the previous time $t-1$.
- **Assumption 2:** The output at time $t$ depends *only* on the state at time $t$.

These two assumptions make it possible to efficiently compute certain quantities that we may be interested in.

## Components of an HMM
An HMM has three sets of trainable parameters.

- The **transition model** is a square matrix $A$, where $A_{s, s'}$ represents $p(z_t = s|z_{t-1} = s')$, the probability of jumping from state $s'$ to state $s$.

- The **emission model** $b_s(x_t)$ tells us $p(x_t|z_t = s)$, the probability of generating $x_t$ when the system is in state $s$. For discrete observations, which we will use in this notebook, the emission model is just a lookup table, with one row for each state, and one column for each observation. For real-valued observations, it is common to use a Gaussian mixture model or neural network to implement the emission model.

- The **state priors** tell us $p(z_1 = s)$, the probability of starting in state $s$. We use $\pi$ to denote the vector of state priors, so $\pi_s$ is the state prior for state $s$.

Let's program an HMM class in PyTorch.

```
import torch
import numpy as np

class HMM(torch.nn.Module):
  """
  Hidden Markov Model with discrete observations.
  """
  def __init__(self, M, N):
    super(HMM, self).__init__()
    self.M = M # number of possible observations
    self.N = N # number of states

    # A
    self.transition_model = TransitionModel(self.N)

    # b(x_t)
    self.emission_model = EmissionModel(self.N,self.M)

    # pi
    self.unnormalized_state_priors = torch.nn.Parameter(torch.randn(self.N))

    # use the GPU
    self.is_cuda = torch.cuda.is_available()
    if self.is_cuda: self.cuda()

class TransitionModel(torch.nn.Module):
  def __init__(self, N):
    super(TransitionModel, self).__init__()
    self.N = N
    self.unnormalized_transition_matrix = torch.nn.Parameter(torch.randn(N,N))

class EmissionModel(torch.nn.Module):
  def __init__(self, N, M):
    super(EmissionModel, self).__init__()
    self.N = N
    self.M = M
    self.unnormalized_emission_matrix = torch.nn.Parameter(torch.randn(N,M))
```

To sample from the HMM, we start by picking a random initial state from the state prior distribution.

Then, we sample an output from the emission distribution, sample a transition from the transition distribution, and repeat.

(Notice that we pass the unnormalized model parameters through a softmax function to make them into probabilities.)

```
def sample(self, T=10):
  state_priors = torch.nn.functional.softmax(self.unnormalized_state_priors, dim=0)
  transition_matrix = torch.nn.functional.softmax(self.transition_model.unnormalized_transition_matrix, dim=1)
  emission_matrix = torch.nn.functional.softmax(self.emission_model.unnormalized_emission_matrix, dim=1)

  # sample initial state
  z_t = torch.distributions.categorical.Categorical(state_priors).sample().item()
  z = []; x = []
  z.append(z_t)
  for t in range(0,T):
    # sample emission
    x_t = torch.distributions.categorical.Categorical(emission_matrix[z_t]).sample().item()
    x.append(x_t)

    # sample transition
    z_t = torch.distributions.categorical.Categorical(transition_matrix[:,z_t]).sample().item()
    if t < T-1: z.append(z_t)

  return x, z

# Add the sampling method to our HMM class
HMM.sample = sample
```

Let's try hard-coding an HMM for generating fake words. (We'll also add some helper functions for encoding and decoding strings.)

We will assume that the system has one state for generating vowels and one state for generating consonants, and the transition matrix has 0s on the diagonal---in other words, the system cannot stay in the vowel state or the consonant state for one than one timestep; it has to switch.

Since we pass the transition matrix through a softmax, to get 0s we set the unnormalized parameter values to $-\infty$.

```
import string
alphabet = string.ascii_lowercase

def encode(s):
  """
  Convert a string into a list of integers
  """
  x = [alphabet.index(ss) for ss in s]
  return x

def decode(x):
  """
  Convert list of ints to string
  """
  s = "".join([alphabet[xx] for xx in x])
  return s

# Initialize the model
model = HMM(M=len(alphabet), N=2)

# Hard-wiring the parameters!
# Let state 0 = consonant, state 1 = vowel
model.unnormalized_state_priors[0] = 0.    # Let's start with a consonant more frequently
model.unnormalized_state_priors[1] = -0.5
print("State priors:", torch.nn.functional.softmax(model.unnormalized_state_priors, dim=0))

# In state 0, only allow consonants; in state 1, only allow vowels
vowel_indices = torch.tensor([alphabet.index(letter) for letter in "aeiou"])
consonant_indices = torch.tensor([alphabet.index(letter) for letter in "bcdfghjklmnpqrstvwxyz"])
model.emission_model.unnormalized_emission_matrix[0, vowel_indices] = -np.inf
model.emission_model.unnormalized_emission_matrix[1, consonant_indices] = -np.inf
print("Emission matrix:", torch.nn.functional.softmax(model.emission_model.unnormalized_emission_matrix, dim=1))

# Only allow vowel -> consonant and consonant -> vowel
model.transition_model.unnormalized_transition_matrix[0,0] = -np.inf  # consonant -> consonant
model.transition_model.unnormalized_transition_matrix[0,1] = 0.       # vowel -> consonant
model.transition_model.unnormalized_transition_matrix[1,0] = 0.       # consonant -> vowel
model.transition_model.unnormalized_transition_matrix[1,1] = -np.inf  # vowel -> vowel
print("Transition matrix:", torch.nn.functional.softmax(model.transition_model.unnormalized_transition_matrix, dim=1))

```

    State priors: tensor([0.6225, 0.3775], device='cuda:0', grad_fn=<SoftmaxBackward>)
    Emission matrix: tensor([[0.0000, 0.1565, 0.0253, 0.0934, 0.0000, 0.0126, 0.0238, 0.0468, 0.0000,
             0.0162, 0.1639, 0.1366, 0.0172, 0.0301, 0.0000, 0.0342, 0.0071, 0.0065,
             0.0322, 0.0313, 0.0000, 0.0080, 0.0368, 0.0281, 0.0539, 0.0396],
            [0.0508, 0.0000, 0.0000, 0.0000, 0.1702, 0.0000, 0.0000, 0.0000, 0.2490,
             0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0348, 0.0000, 0.0000, 0.0000,
             0.0000, 0.0000, 0.4951, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]],
           device='cuda:0', grad_fn=<SoftmaxBackward>)
    Transition matrix: tensor([[0., 1.],
            [1., 0.]], device='cuda:0', grad_fn=<SoftmaxBackward>)

Try sampling from our hard-coded model:

```
# Sample some outputs
for _ in range(4):
  sampled_x, sampled_z = model.sample(T=5)
  print("x:", decode(sampled_x))
  print("z:", sampled_z)
```

    x: buyup
    z: [0, 1, 0, 1, 0]
    x: enabe
    z: [1, 0, 1, 0, 1]
    x: zehuk
    z: [0, 1, 0, 1, 0]
    x: xuliy
    z: [0, 1, 0, 1, 0]

## The Three Problems

In a [classic tutorial](https://www.cs.cmu.edu/~cga/behavior/rabiner1.pdf) on HMMs, Lawrence Rabiner describes "three problems" that need to be solved before you can effectively use an HMM. They are:
- Problem 1: How do we efficiently compute $p(\mathbf{x})$?
- Problem 2: How do we find the most likely state sequence $\mathbf{z}$ that could have generated the data?
- Problem 3: How do we train the model?

In the rest of the notebook, we will see how to solve each problem and implement the solutions in PyTorch.

### Problem 1: How do we compute $p(\mathbf{x})$?

#### *Why?*
Why might we care about computing $p(\mathbf{x})$? Here's two reasons.
* Given two HMMs, $\theta_1$ and $\theta_2$, we can compute the likelihood of some data $\mathbf{x}$ under each model, $p_{\theta_1}(\mathbf{x})$ and $p_{\theta_2}(\mathbf{x})$, to decide which model is a better fit to the data.

  (For example, given an HMM for English speech and an HMM for French speech, we could compute the likelihood given each model, and pick the model with the higher likelihood to infer whether the person is speaking English or French.)
* Being able to compute $p(\mathbf{x})$ gives us a way to train the model, as we will see later.

#### *How?*
Given that we want $p(\mathbf{x})$, how do we compute it?

We've assumed that the data is generated by visiting some sequence of states $\mathbf{z}$ and picking an output $x_t$ for each $z_t$ from the emission distribution $p(x_t|z_t)$. So if we knew $\mathbf{z}$, then the probability of $\mathbf{x}$ could be computed as follows:

$$p(\mathbf{x}|\mathbf{z}) = \prod_{t} p(x_t|z_t) p(z_t|z_{t-1})$$

However, we don't know $\mathbf{z}$; it's hidden. But we do know the probability of any given $\mathbf{z}$, independent of what we observe. So we could get the probability of $\mathbf{x}$ by summing over the different possibilities for $\mathbf{z}$, like this:

$$p(\mathbf{x}) = \sum_{\mathbf{z}} p(\mathbf{x}|\mathbf{z}) p(\mathbf{z}) = \sum_{\mathbf{z}} \prod_{t} p(x_t|z_t) p(z_t|z_{t-1})$$

The problem is: if you try to take that sum directly, you will need to compute $N^T$ terms. This is impossible to do for anything but very short sequences. For example, let's say the sequence is of length $T=100$ and there are $N=2$ possible states. Then we would need to check $N^T = 2^{100} \approx 10^{30}$ different possible state sequences.

We need a way to compute $p(\mathbf{x})$ that doesn't require us to explicitly calculate all $N^T$ terms. For this, we use the forward algorithm.

________

The Forward Algorithm

> for $s=1 \rightarrow N$:\
>       $\alpha_{s,1} := b_s(x_1) \cdot \pi_s$
>
> for $t = 2 \rightarrow T$:\
>       for $s = 1 \rightarrow N$:\
>
> $\alpha_{s,t} := b_s(x_t) \cdot \underset{s'}{\sum} A_{s, s'} \cdot \alpha_{s',t-1} $
>
> $p(\mathbf{x}) := \underset{s}{\sum} \alpha_{s,T}$\
> return $p(\mathbf{x})$
________

The forward algorithm is much faster than enumerating all $N^T$ possible state sequences: it requires only $O(N^2T)$ operations to run, since each step is mostly multiplying the vector of forward variables by the transition matrix. (And very often we can reduce that complexity even further, if the transition matrix is sparse.)

There is one practical problem with the forward algorithm as presented above: it is prone to underflow due to multiplying a long chain of small numbers, since probabilities are always between 0 and 1. Instead, let's do everything in the log domain. In the log domain, a multiplication becomes a sum, and a sum becomes a [logsumexp](https://en.wikipedia.org/wiki/LogSumExp).

________

The Forward Algorithm (Log Domain)

> for $s=1 \rightarrow N$:\
>       $\text{log }\alpha_{s,1} := \text{log }b_s(x_1) + \text{log }\pi_s$
>
> for $t = 2 \rightarrow T$:\
>       for $s = 1 \rightarrow N$:\
>
> $\text{log }\alpha_{s,t} := \text{log }b_s(x_t) +  \underset{s'}{\text{logsumexp}} \left( \text{log }A_{s, s'} + \text{log }\alpha_{s',t-1} \right)$
>
> $\text{log }p(\mathbf{x}) := \underset{s}{\text{logsumexp}} \left( \text{log }\alpha_{s,T} \right)$\
> return $\text{log }p(\mathbf{x})$
________

Now that we have a numerically stable version of the forward algorithm, let's implement it in PyTorch.

```
def HMM_forward(self, x, T):
  """
  x : IntTensor of shape (batch size, T_max)
  T : IntTensor of shape (batch size)

  Compute log p(x) for each example in the batch.
  T = length of each example
  """
  if self.is_cuda:
  	x = x.cuda()
  	T = T.cuda()

  batch_size = x.shape[0]; T_max = x.shape[1]
  log_state_priors = torch.nn.functional.log_softmax(self.unnormalized_state_priors, dim=0)
  log_alpha = torch.zeros(batch_size, T_max, self.N)
  if self.is_cuda: log_alpha = log_alpha.cuda()

  log_alpha[:, 0, :] = self.emission_model(x[:,0]) + log_state_priors
  for t in range(1, T_max):
    log_alpha[:, t, :] = self.emission_model(x[:,t]) + self.transition_model(log_alpha[:, t-1, :])

  # Select the sum for the final timestep (each x may have different length).
  log_sums = log_alpha.logsumexp(dim=2)
  log_probs = torch.gather(log_sums, 1, T.view(-1,1) - 1)
  return log_probs

def emission_model_forward(self, x_t):
  log_emission_matrix = torch.nn.functional.log_softmax(self.unnormalized_emission_matrix, dim=1)
  out = log_emission_matrix[:, x_t].transpose(0,1)
  return out

def transition_model_forward(self, log_alpha):
  """
  log_alpha : Tensor of shape (batch size, N)
  Multiply previous timestep's alphas by transition matrix (in log domain)
  """
  log_transition_matrix = torch.nn.functional.log_softmax(self.unnormalized_transition_matrix, dim=1)

  # Matrix multiplication in the log domain
  out = log_domain_matmul(log_transition_matrix, log_alpha.transpose(0,1)).transpose(0,1)
  return out

def log_domain_matmul(log_A, log_B):
	"""
	log_A : m x n
	log_B : n x p
	output : m x p matrix

	Normally, a matrix multiplication
	computes out_{i,j} = sum_k A_{i,k} x B_{k,j}

	A log domain matrix multiplication
	computes out_{i,j} = logsumexp_k log_A_{i,k} + log_B_{k,j}
	"""
	m = log_A.shape[0]
	n = log_A.shape[1]
	p = log_B.shape[1]

	log_A_expanded = torch.stack([log_A] * p, dim=2)
	log_B_expanded = torch.stack([log_B] * m, dim=0)

	elementwise_sum = log_A_expanded + log_B_expanded
	out = torch.logsumexp(elementwise_sum, dim=1)

	return out

TransitionModel.forward = transition_model_forward
EmissionModel.forward = emission_model_forward
HMM.forward = HMM_forward
```

Try running the forward algorithm on our vowels/consonants model from before:

```
x = torch.stack( [torch.tensor(encode("cat"))] )
T = torch.tensor([3])
print(model.forward(x, T))

x = torch.stack( [torch.tensor(encode("aba")), torch.tensor(encode("abb"))] )
T = torch.tensor([3,3])
print(model.forward(x, T))
```

    tensor([[-10.5923]], device='cuda:0', grad_fn=<GatherBackward>)
    tensor([[-8.7869],
            [   -inf]], device='cuda:0', grad_fn=<GatherBackward>)

When using the vowel <-> consonant HMM from above, notice that the forward algorithm returns $-\infty$ for $\mathbf{x} = \text{"abb"}$. That's because our transition matrix says the probability of vowel -> vowel and consonant -> consonant is 0, so the probability of $\text{"abb"}$ happening is 0, and thus the log probability is $-\infty$.

#### *Side note: deriving the forward algorithm*

If you're interested in understanding how the forward algorithm actually computes $p(\mathbf{x})$, read this section; if not, skip to the next part on "Problem 2" (finding the most likely state sequence).

To derive the forward algorithm, start by deriving the forward variable:

$$\begin{align}
    \alpha_{s,t} &= p(x_1, x_2, \dots, x_t, z_t=s) \\
     &= p(x_t | x_1, x_2, \dots, x_{t-1}, z_t = s) \cdot p(x_1, x_2, \dots, x_{t-1}, z_t = s)  \\
    &= p(x_t | z_t = s) \cdot p(x_1, x_2, \dots, x_{t-1}, z_t = s) \\
    &= p(x_t | z_t = s) \cdot \left( \sum_{s'} p(x_1, x_2, \dots, x_{t-1}, z_{t-1}=s', z_t = s) \right)\\
    &= p(x_t | z_t = s) \cdot \left( \sum_{s'} p(z_t = s | x_1, x_2, \dots, x_{t-1}, z_{t-1}=s') \cdot p(x_1, x_2, \dots, x_{t-1}, z_{t-1}=s') \right)\\
    &= \underbrace{p(x_t | z_t = s)}_{\text{emission model}} \cdot \left( \sum_{s'} \underbrace{p(z_t = s | z_{t-1}=s')}_{\text{transition model}} \cdot \underbrace{p(x_1, x_2, \dots, x_{t-1}, z_{t-1}=s')}_{\text{forward variable for previous timestep}} \right)\\
    &= b_s(x_t) \cdot \left( \sum_{s'} A_{s, s'} \cdot \alpha_{s',t-1} \right)
    \end{align}$$

I'll explain how to get to each line of this equation from the previous line.

Line 1 is the definition of the forward variable $\alpha_{s,t}$.

Line 2 is the chain rule ($p(A,B) = p(A|B) \cdot p(B)$, where $A$ is $x_t$ and $B$ is all the other variables).

In Line 3, we apply Assumption 2: the probability of observation $x_t$ depends only on the current state $z_t$.

In Line 4, we marginalize over all the possible states in the previous timestep $t-1$.

In Line 5, we apply the chain rule again.

In Line 6, we apply Assumption 1: the current state depends only on the previous state.

In Line 7, we substitute in the emission probability, the transition probability, and the forward variable for the previous timestep, to get the complete recursion.

The formula above can be used for $t = 2 \rightarrow T$. At $t=1$, there is no previous state, so instead of the transition matrix $A$, we use the state priors $\pi$, which tell us the probability of starting in each state. Thus for $t=1$, the forward variables are computed as follows:

$$\begin{align}
\alpha_{s,1} &= p(x_1, z_1=s) \\
  &= p(x_1 | z_1 = s) \cdot p(z_1 = s)  \\
&= b_s(x_1) \cdot \pi_s
\end{align}$$

Finally, to compute $p(\mathbf{x}) = p(x_1, x_2, \dots, x_T)$, we marginalize over $\alpha_{s,T}$, the forward variables computed in the last timestep:

$$\begin{align*}
p(\mathbf{x}) &= \sum_{s} p(x_1, x_2, \dots, x_T, z_T = s) \\
&= \sum_{s} \alpha_{s,T}
\end{align*}$$

You can get from this formulation to the log domain formulation by taking the log of the forward variable, and using these identities:
- $\text{log }(a \cdot b) = \text{log }a + \text{log }b$
- $\text{log }(a + b) = \text{log }(e^{\text{log }a} + e^{\text{log }b}) = \text{logsumexp}(\text{log }a, \text{log }b)$

### Problem 2: How do we compute $\underset{\mathbf{z}}{\text{argmax }} p(\mathbf{z}|\mathbf{x})$?

Given an observation sequence $\mathbf{x}$, we may want to find the most likely sequence of states that could have generated $\mathbf{x}$. (Given the sequence of selfies, we want to infer what cities the friend visited.) In other words, we want $\underset{\mathbf{z}}{\text{argmax }} p(\mathbf{z}|\mathbf{x})$.

We can use Bayes' rule to rewrite this expression:
    $$\begin{align*}
    \underset{\mathbf{z}}{\text{argmax }} p(\mathbf{z}|\mathbf{x}) &= \underset{\mathbf{z}}{\text{argmax }} \frac{p(\mathbf{x}|\mathbf{z}) p(\mathbf{z})}{p(\mathbf{x})} \\
    &= \underset{\mathbf{z}}{\text{argmax }} p(\mathbf{x}|\mathbf{z}) p(\mathbf{z})
    \end{align*}$$

Hmm! That last expression, $\underset{\mathbf{z}}{\text{argmax }} p(\mathbf{x}|\mathbf{z}) p(\mathbf{z})$, looks suspiciously similar to the intractable expression we encountered before introducing the forward algorithm, $\underset{\mathbf{z}}{\sum} p(\mathbf{x}|\mathbf{z}) p(\mathbf{z})$.

And indeed, just as the intractable *sum* over all $\mathbf{z}$ can be implemented efficiently using the forward algorithm, so too this intractable *argmax* can be implemented efficiently using a similar divide-and-conquer algorithm: the legendary Viterbi algorithm!

________

The Viterbi Algorithm

> for $s=1 \rightarrow N$:\
>       $\delta_{s,1} := b_s(x_1) \cdot \pi_s$\
>       $\psi_{s,1} := 0$
>
> for $t = 2 \rightarrow T$:\
>       for $s = 1 \rightarrow N$:\
>             $\delta_{s,t} := b_s(x_t) \cdot \left( \underset{s'}{\text{max }} A_{s, s'} \cdot \delta_{s',t-1} \right)$\
            $\psi_{s,t} := \underset{s'}{\text{argmax }} A_{s, s'} \cdot \delta_{s',t-1}$
>
> $z_T^* := \underset{s}{\text{argmax }} \delta_{s,T}$\
> for $t = T-1 \rightarrow 1$:\
      $z_{t}^* := \psi_{z_{t+1}^*,t+1}$
>
> $\mathbf{z}^* := \{z_{1}^*, \dots, z_{T}^* \}$\
return $\mathbf{z}^*$
________

The Viterbi algorithm looks somewhat gnarlier than the forward algorithm, but it is essentially the same algorithm, with two tweaks: 1) instead of taking the sum over previous states, we take the max; and 2) we record the argmax of the previous states in a table, and loop back over this table at the end to get $\mathbf{z}^*$, the most likely state sequence. (And like the forward algorithm, we should run the Viterbi algorithm in the log domain for better numerical stability.)

Let's add the Viterbi algorithm to our PyTorch model:

```
def viterbi(self, x, T):
  """
  x : IntTensor of shape (batch size, T_max)
  T : IntTensor of shape (batch size)
  Find argmax_z log p(x|z) for each (x) in the batch.
  """
  if self.is_cuda:
    x = x.cuda()
    T = T.cuda()

  batch_size = x.shape[0]; T_max = x.shape[1]
  log_state_priors = torch.nn.functional.log_softmax(self.unnormalized_state_priors, dim=0)
  log_delta = torch.zeros(batch_size, T_max, self.N).float()
  psi = torch.zeros(batch_size, T_max, self.N).long()
  if self.is_cuda:
    log_delta = log_delta.cuda()
    psi = psi.cuda()

  log_delta[:, 0, :] = self.emission_model(x[:,0]) + log_state_priors
  for t in range(1, T_max):
    max_val, argmax_val = self.transition_model.maxmul(log_delta[:, t-1, :])
    log_delta[:, t, :] = self.emission_model(x[:,t]) + max_val
    psi[:, t, :] = argmax_val

  # Get the log probability of the best path
  log_max = log_delta.max(dim=2)[0]
  best_path_scores = torch.gather(log_max, 1, T.view(-1,1) - 1)

  # This next part is a bit tricky to parallelize across the batch,
  # so we will do it separately for each example.
  z_star = []
  for i in range(0, batch_size):
    z_star_i = [ log_delta[i, T[i] - 1, :].max(dim=0)[1].item() ]
    for t in range(T[i] - 1, 0, -1):
      z_t = psi[i, t, z_star_i[0]].item()
      z_star_i.insert(0, z_t)

    z_star.append(z_star_i)

  return z_star, best_path_scores # return both the best path and its log probability

def transition_model_maxmul(self, log_alpha):
  log_transition_matrix = torch.nn.functional.log_softmax(self.unnormalized_transition_matrix, dim=1)

  out1, out2 = maxmul(log_transition_matrix, log_alpha.transpose(0,1))
  return out1.transpose(0,1), out2.transpose(0,1)

def maxmul(log_A, log_B):
	"""
	log_A : m x n
	log_B : n x p
	output : m x p matrix

	Similar to the log domain matrix multiplication,
	this computes out_{i,j} = max_k log_A_{i,k} + log_B_{k,j}
	"""
	m = log_A.shape[0]
	n = log_A.shape[1]
	p = log_B.shape[1]

	log_A_expanded = torch.stack([log_A] * p, dim=2)
	log_B_expanded = torch.stack([log_B] * m, dim=0)

	elementwise_sum = log_A_expanded + log_B_expanded
	out1,out2 = torch.max(elementwise_sum, dim=1)

	return out1,out2

TransitionModel.maxmul = transition_model_maxmul
HMM.viterbi = viterbi
```

Try running Viterbi on an input sequence, given the vowel/consonant HMM:

```
x = torch.stack( [torch.tensor(encode("aba")), torch.tensor(encode("abb"))] )
T = torch.tensor([3,3])
print(model.viterbi(x, T))
```

    ([[1, 0, 1], [1, 0, 0]], tensor([[-8.7869],
            [   -inf]], device='cuda:0', grad_fn=<GatherBackward>))

For $\mathbf{x} = \text{"aba"}$, the Viterbi algorithm returns $\mathbf{z}^* = \{1,0,1\}$. This corresponds to "vowel, consonant, vowel" according to the way we defined the states above, which is correct for this input sequence. Yay!

For $\mathbf{x} = \text{"abb"}$, the Viterbi algorithm still returns a $\mathbf{z}^*$, but we know this is gibberish because "vowel, consonant, consonant" is impossible under this HMM, and indeed the log probability of this path is $-\infty$.

Let's compare the "forward score" (the log probability of all possible paths, returned by the forward algorithm) with the "Viterbi score" (the log probability of the maximum likelihood path, returned by the Viterbi algorithm):

```
print(model.forward(x, T))
print(model.viterbi(x, T)[1])
```

    tensor([[-8.7869],
            [   -inf]], device='cuda:0', grad_fn=<GatherBackward>)
    tensor([[-8.7869],
            [   -inf]], device='cuda:0', grad_fn=<GatherBackward>)

The two scores are the same! That's because in this instance there is only one possible path through the HMM, so the probability of the most likely path is the same as the sum of the probabilities of all possible paths.

In general, though, the forward score and Viterbi score will always be somewhat close. This is because of a property of the $\text{logsumexp}$ function: $\text{logsumexp}(\mathbf{x}) \approx \max (\mathbf{x})$. ($\text{logsumexp}$ is sometimes referred to as the "smooth maximum" function.)

```
x = torch.tensor([1., 2., 3.])
print(x.max(dim=0)[0])
print(x.logsumexp(dim=0))
```

    tensor(3.)
    tensor(3.4076)

### Problem 3: How do we train the model?

Earlier, we hard-coded an HMM to have certain behavior. What we would like to do instead is have the HMM learn to model the data on its own. And while it is possible to use supervised learning with an HMM (by hard-coding the emission model or the transition model) so that the states have a particular interpretation, the really cool thing about HMMs is that they are naturally unsupervised learners, so they can learn to use their different states to represent different patterns in the data, without the programmer needing to indicate what each state means.

Like many machine learning models, an HMM can be trained using maximum likelihood estimation, i.e.:

$$\theta^* = \underset{\theta}{\text{argmin }} -\sum_{\mathbf{x}^i}\text{log }p_{\theta}(\mathbf{x}^i)$$

where $\mathbf{x}^1, \mathbf{x}^2, \dots$ are training examples.

The standard method for doing this is the Expectation-Maximization (EM) algorithm, which for HMMs is also called the "Baum-Welch" algorithm. In EM training, we alternate between an "E-step", where we estimate the values of the latent variables, and an "M-step", where the model parameters are updated given the estimated latent variables. (Think $k$-means, where you guess which cluster each data point belongs to, then reestimate where the clusters are, and repeat.) The EM algorithm has some nice properties: it is guaranteed at each step to decrease the loss function, and the E-step and M-step may have an exact closed form solution, in which case no pesky learning rates are required.

But because the HMM forward algorithm is differentiable with respect to all the model parameters, we can also just take advantage of automatic differentiation methods in libraries like PyTorch and try to minimize $-\text{log }p_{\theta}(\mathbf{x})$ directly, by backpropagating through the forward algorithm and running stochastic gradient descent. That means we don't need to write any additional HMM code to implement training: `loss.backward()` is all you need.

Here we will implement SGD training for an HMM in PyTorch. First, some helper classes:

```
import torch.utils.data
from collections import Counter
from sklearn.model_selection import train_test_split

class TextDataset(torch.utils.data.Dataset):
  def __init__(self, lines):
    self.lines = lines # list of strings
    collate = Collate() # function for generating a minibatch from strings
    self.loader = torch.utils.data.DataLoader(self, batch_size=1024, num_workers=1, shuffle=True, collate_fn=collate)

  def __len__(self):
    return len(self.lines)

  def __getitem__(self, idx):
    line = self.lines[idx].lstrip(" ").rstrip("\n").rstrip(" ").rstrip("\n")
    return line

class Collate:
  def __init__(self):
    pass

  def __call__(self, batch):
    """
    Returns a minibatch of strings, padded to have the same length.
    """
    x = []
    batch_size = len(batch)
    for index in range(batch_size):
      x_ = batch[index]

      # convert letters to integers
      x.append(encode(x_))

    # pad all sequences with 0 to have same length
    x_lengths = [len(x_) for x_ in x]
    T = max(x_lengths)
    for index in range(batch_size):
      x[index] += [0] * (T - len(x[index]))
      x[index] = torch.tensor(x[index])

    # stack into single tensor
    x = torch.stack(x)
    x_lengths = torch.tensor(x_lengths)
    return (x,x_lengths)
```

Let's load some training/testing data. By default, this will use the unix "words" file, but you could also use your own text file.

```
!wget https://raw.githubusercontent.com/lorenlugosch/pytorch_HMM/master/data/train/training.txt

filename = "training.txt"

with open(filename, "r") as f:
  lines = f.readlines() # each line of lines will have one word

alphabet = list(Counter(("".join(lines))).keys())
train_lines, valid_lines = train_test_split(lines, test_size=0.1, random_state=42)
train_dataset = TextDataset(train_lines)
valid_dataset = TextDataset(valid_lines)

M = len(alphabet)
```

    --2020-01-28 19:13:21--  https://raw.githubusercontent.com/lorenlugosch/pytorch_HMM/master/data/train/training.txt
    Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
    Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 2493109 (2.4M) [text/plain]
    Saving to: ‘training.txt’

    training.txt        100%[===================>]   2.38M  --.-KB/s    in 0.1s

    2020-01-28 19:13:26 (22.5 MB/s) - ‘training.txt’ saved [2493109/2493109]

We will use a Trainer class for training and testing the model:

```
from tqdm import tqdm # for displaying progress bar

class Trainer:
  def __init__(self, model, lr):
    self.model = model
    self.lr = lr
    self.optimizer = torch.optim.Adam(model.parameters(), lr=self.lr, weight_decay=0.00001)

  def train(self, dataset):
    train_loss = 0
    num_samples = 0
    self.model.train()
    print_interval = 50
    for idx, batch in enumerate(tqdm(dataset.loader)):
      x,T = batch
      batch_size = len(x)
      num_samples += batch_size
      log_probs = self.model(x,T)
      loss = -log_probs.mean()
      self.optimizer.zero_grad()
      loss.backward()
      self.optimizer.step()
      train_loss += loss.cpu().data.numpy().item() * batch_size
      if idx % print_interval == 0:
        print("loss:", loss.item())
        for _ in range(5):
          sampled_x, sampled_z = self.model.sample()
          print(decode(sampled_x))
          print(sampled_z)
    train_loss /= num_samples
    return train_loss

  def test(self, dataset):
    test_loss = 0
    num_samples = 0
    self.model.eval()
    print_interval = 50
    for idx, batch in enumerate(dataset.loader):
      x,T = batch
      batch_size = len(x)
      num_samples += batch_size
      log_probs = self.model(x,T)
      loss = -log_probs.mean()
      test_loss += loss.cpu().data.numpy().item() * batch_size
      if idx % print_interval == 0:
        print("loss:", loss.item())
        sampled_x, sampled_z = self.model.sample()
        print(decode(sampled_x))
        print(sampled_z)
    test_loss /= num_samples
    return test_loss
```

Finally, initialize the model and run the main training loop. Every 50 batches, the code will produce a few samples from the model. Over time, these samples should look more and more realistic.

```
# Initialize model
model = HMM(N=64, M=M)

# Train the model
num_epochs = 10
trainer = Trainer(model, lr=0.01)

for epoch in range(num_epochs):
        print("========= Epoch %d of %d =========" % (epoch+1, num_epochs))
        train_loss = trainer.train(train_dataset)
        valid_loss = trainer.test(valid_dataset)

        print("========= Results: epoch %d of %d =========" % (epoch+1, num_epochs))
        print("train loss: %.2f| valid loss: %.2f\n" % (train_loss, valid_loss) )
```

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Epoch 1 of 10 =========

      0%|          | 1/208 [00:00<01:25,  2.42it/s]

    loss: 37.582481384277344
    VxKGP

    OCJ
    [31, 37, 40, 4, 3, 29, 37, 45, 11, 29]
    tlrZYbAzdd
    [55, 31, 47, 55, 44, 40, 43, 49, 36, 20]
    bdNS
    jakwe
    [55, 42, 5, 16, 26, 37, 49, 57, 55, 8]
    guTrnXmdUh
    [8, 54, 40, 7, 28, 35, 35, 6, 37, 25]
    CasaRIOZpa
    [40, 4, 5, 55, 5, 58, 53, 46, 32, 48]

     25%|██▍       | 51/208 [00:16<00:53,  2.96it/s]

    loss: 33.11034393310547
    oZteXuAqdo
    [12, 46, 5, 13, 22, 57, 20, 23, 63, 10]
    oFgcusjdDW
    [47, 47, 38, 31, 9, 16, 34, 20, 59, 9]
    LxSHijFVtz
    [59, 28, 3, 1, 22, 9, 32, 53, 42, 4]
    -YgiishZtX
    [43, 1, 31, 7, 53, 52, 54, 35, 62, 22]
    BHMITaHpLs
    [26, 52, 39, 58, 56, 18, 36, 45, 4, 44]

     49%|████▊     | 101/208 [00:32<00:33,  3.17it/s]

    loss: 29.39785385131836
    pgvlabflda
    [45, 39, 11, 51, 19, 31, 51, 25, 38, 12]
    rZih
    gjYdo
    [57, 52, 61, 39, 37, 39, 1, 39, 51, 3]
    alistcnloe
    [55, 12, 7, 8, 33, 24, 25, 28, 3, 49]
    Tndodjmtpq
    [26, 61, 44, 24, 49, 10, 26, 8, 59, 9]
    indTtlraVb
    [10, 44, 18, 23, 18, 28, 53, 35, 31, 31]

     73%|███████▎  | 151/208 [00:48<00:18,  3.10it/s]

    loss: 26.592235565185547

    Ocappmuml
    [63, 62, 21, 50, 21, 3, 41, 30, 60, 37]
    rnpWelsfaT
    [59, 11, 53, 56, 43, 7, 60, 28, 14, 0]
    aHeRysKVTy
    [62, 42, 13, 19, 49, 29, 36, 43, 0, 27]
    stgngcukIs
    [55, 3, 0, 56, 33, 54, 16, 50, 54, 58]
    heruudcaqc
    [38, 8, 5, 54, 24, 30, 51, 55, 50, 37]

     97%|█████████▋| 201/208 [01:03<00:02,  3.18it/s]

    loss: 24.756166458129883
    nrSgirgjgg
    [41, 47, 49, 34, 0, 2, 39, 2, 39, 39]
    SteaPhpnyg
    [16, 30, 61, 35, 7, 21, 45, 29, 37, 24]
    aededlmlhL
    [6, 43, 51, 15, 42, 55, 60, 41, 21, 46]
    hlndySefCe
    [55, 41, 0, 20, 50, 20, 23, 58, 37, 26]
    teitiQpqng
    [55, 8, 61, 5, 4, 24, 63, 45, 11, 32]

    100%|██████████| 208/208 [01:05<00:00,  3.94it/s]

    loss: 24.631101608276367
    hybcycbnYn
    [38, 27, 24, 12, 15, 61, 31, 51, 46, 46]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 1 of 10 =========
    train loss: 30.02| valid loss: 24.83

    ========= Epoch 2 of 10 =========

      0%|          | 1/208 [00:00<01:20,  2.58it/s]

    loss: 24.459197998046875
    laorHXsnld
    [38, 6, 57, 26, 62, 22, 7, 29, 5, 3]
    dIardPbgNy
    [16, 30, 49, 7, 45, 53, 56, 43, 15, 1]
    thznbGrjsI
    [55, 3, 13, 49, 24, 54, 54, 26, 18, 18]
    soIiedlwcd
    [55, 10, 9, 61, 58, 58, 3, 60, 41, 49]
    nytogaxaed
    [59, 9, 5, 12, 46, 12, 60, 48, 3, 41]

     25%|██▍       | 51/208 [00:16<00:48,  3.21it/s]

    loss: 23.961589813232422
    cBoadrxahm
    [55, 46, 45, 45, 24, 56, 36, 12, 0, 53]
    ryoxsyuyts
    [38, 15, 60, 9, 9, 63, 22, 35, 3, 29]
    dlendsnenl
    [55, 61, 32, 41, 20, 26, 54, 13, 49, 0]
    sieddWsavy
    [55, 41, 11, 20, 42, 21, 61, 63, 61, 45]
    furdBtNeoh
    [55, 59, 32, 20, 33, 23, 3, 8, 26, 8]

     49%|████▊     | 101/208 [00:32<00:35,  2.97it/s]

    loss: 23.018342971801758
    pelydtulEc
    [55, 8, 25, 27, 24, 51, 22, 35, 34, 26]
    nglmngkced
    [44, 30, 0, 2, 29, 39, 28, 44, 50, 20]
    tdikoKmgds
    [11, 49, 60, 43, 20, 20, 46, 37, 49, 36]
    tessTitAns
    [55, 4, 18, 18, 49, 62, 32, 53, 11, 18]
    siedrmrwxe
    [55, 6, 13, 49, 43, 7, 42, 33, 60, 4]

     73%|███████▎  | 151/208 [00:47<00:17,  3.22it/s]

    loss: 22.27695655822754
    bevatWetey
    [55, 8, 5, 56, 7, 31, 14, 63, 34, 15]
    dredWpyhdl
    [55, 38, 4, 42, 25, 28, 1, 17, 3, 0]
    mhyyssWdra
    [55, 38, 27, 21, 18, 18, 2, 41, 34, 48]
    iysglnddee
    [6, 19, 2, 30, 63, 44, 40, 49, 13, 19]
    doynsserrn
    [55, 57, 21, 44, 11, 60, 4, 47, 40, 56]

     97%|█████████▋| 201/208 [01:03<00:02,  3.15it/s]

    loss: 22.124229431152344
    tRndssyaNe
    [55, 17, 49, 31, 60, 41, 27, 19, 60, 23]
    lulmKmljdl
    [55, 59, 9, 2, 61, 0, 40, 4, 42, 25]
    naoranidon
    [55, 17, 57, 49, 19, 44, 48, 31, 57, 54]
    acNnlsunIk
    [6, 51, 48, 46, 25, 28, 19, 45, 58, 33]
    AuydJypanr
    [55, 6, 34, 41, 31, 21, 19, 49, 46, 45]

    100%|██████████| 208/208 [01:05<00:00,  3.76it/s]

    loss: 22.349966049194336
    mecskynmmn
    [55, 8, 62, 34, 25, 28, 9, 3, 3, 56]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 2 of 10 =========
    train loss: 23.12| valid loss: 22.21

    ========= Epoch 3 of 10 =========

      0%|          | 1/208 [00:00<01:37,  2.11it/s]

    loss: 22.131309509277344
    yahoconeRi
    [23, 62, 8, 32, 2, 60, 33, 43, 9, 28]
    toyaYtrltj
    [55, 14, 53, 26, 2, 62, 61, 50, 31, 53]
    RacAynanas
    [55, 17, 52, 34, 19, 46, 48, 53, 56, 18]
    nheyoenved
    [55, 8, 13, 19, 46, 60, 29, 5, 4, 42]
    aonmdNywnG
    [55, 57, 21, 40, 42, 25, 27, 37, 54, 8]

     25%|██▍       | 51/208 [00:16<00:53,  2.94it/s]

    loss: 21.87742805480957
    bofsZedgus
    [55, 57, 54, 60, 9, 9, 20, 43, 22, 60]
    ockegNlydl
    [6, 26, 40, 43, 39, 25, 25, 28, 41, 44]
    ackstatarA
    [6, 12, 9, 53, 2, 19, 31, 48, 25, 27]
    cefnaeaded
    [55, 8, 57, 54, 17, 4, 24, 31, 13, 49]
    aaanewrgRU
    [6, 51, 48, 50, 56, 29, 39, 25, 15, 31]

     49%|████▊     | 101/208 [00:32<00:33,  3.15it/s]

    loss: 21.491273880004883
    tencbalycl
    [55, 6, 29, 2, 37, 48, 53, 19, 12, 25]
    hhledlyraE
    [55, 8, 38, 14, 20, 25, 28, 55, 17, 36]
    ceeonqnene
    [55, 17, 8, 57, 54, 17, 29, 13, 44, 11]
    icmssanakn
    [6, 45, 3, 58, 60, 5, 1, 48, 50, 25]
    qenesslyec
    [55, 59, 44, 11, 18, 58, 25, 15, 61, 62]

     73%|███████▎  | 151/208 [00:48<00:18,  3.13it/s]

    loss: 21.693378448486328
    grknedtnme
    [55, 59, 14, 44, 11, 20, 32, 21, 38, 14]
    ronandeed

    [55, 57, 54, 48, 50, 20, 56, 53, 40, 54]
    lowcaZcedn
    [55, 6, 36, 45, 17, 3, 45, 4, 42, 44]
    shyedidlal
    [55, 38, 27, 4, 42, 61, 41, 44, 48, 37]
    bealaesusm
    [55, 59, 48, 53, 56, 43, 60, 11, 18, 60]

     97%|█████████▋| 201/208 [01:03<00:02,  3.24it/s]

    loss: 21.936307907104492
    andaZlygly
    [6, 29, 31, 48, 37, 25, 28, 39, 25, 1]
    arnalonane
    [6, 57, 21, 48, 50, 57, 54, 48, 50, 17]
    oanclresUs
    [6, 10, 40, 45, 63, 44, 11, 18, 35, 60]
    chfussnedp
    [55, 59, 57, 22, 35, 35, 44, 11, 20, 19]
    sisheablyt
    [55, 6, 55, 38, 61, 48, 37, 25, 27, 56]

    100%|██████████| 208/208 [01:05<00:00,  3.81it/s]

    loss: 21.37592315673828
    idyntksdid
    [6, 36, 19, 56, 31, 33, 58, 31, 19, 24]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 3 of 10 =========
    train loss: 21.76| valid loss: 21.52

    ========= Epoch 4 of 10 =========

      0%|          | 1/208 [00:00<01:31,  2.25it/s]

    loss: 21.31800079345703
    tfuslliOdn
    [55, 57, 22, 7, 1, 52, 22, 60, 30, 44]
    ghufnnesss
    [55, 8, 36, 57, 21, 44, 11, 18, 60, 35]
    t-ljdnessn
    [55, 19, 56, 46, 20, 44, 11, 18, 60, 44]
    cytleymdne
    [55, 19, 39, 25, 11, 17, 0, 20, 44, 11]
    nremdlessd
    [55, 38, 14, 0, 42, 25, 11, 18, 58, 49]

     25%|██▍       | 51/208 [00:16<00:49,  3.18it/s]

    loss: 21.071056365966797
    Qydmklynes
    [55, 19, 20, 40, 33, 25, 15, 44, 11, 18]
    thessaalXk
    [55, 8, 13, 35, 35, 44, 48, 50, 23, 25]
    mletlyyyhy
    [55, 38, 13, 32, 25, 28, 9, 28, 8, 27]
    maladypesd
    [55, 17, 25, 56, 31, 19, 55, 59, 2, 49]
    lylytdvtal
    [55, 17, 2, 17, 9, 20, 49, 32, 48, 53]

     49%|████▊     | 101/208 [00:31<00:33,  3.20it/s]

    loss: 21.18721580505371
    icteusmenn
    [6, 36, 5, 43, 24, 35, 34, 17, 46, 44]
    driVeredle
    [55, 38, 61, 41, 56, 47, 4, 42, 25, 11]
    enhsyeysne
    [6, 26, 8, 2, 17, 3, 43, 58, 44, 11]
    ilydvyrhzl
    [6, 12, 15, 20, 13, 19, 55, 8, 32, 25]
    lhnynessus
    [55, 59, 46, 30, 44, 11, 18, 60, 22, 7]

     73%|███████▎  | 151/208 [00:47<00:17,  3.22it/s]

    loss: 21.164073944091797
    ibtleyylli
    [6, 16, 32, 25, 13, 19, 17, 12, 53, 61]
    cenesseyet
    [55, 59, 44, 11, 18, 60, 10, 40, 8, 32]
    edalessnes
    [6, 36, 48, 53, 56, 18, 35, 44, 11, 18]
    cyheablaft
    [55, 19, 55, 59, 48, 37, 25, 56, 32, 49]
    cyhyyckydQ
    [55, 19, 8, 19, 3, 51, 33, 27, 20, 45]

     97%|█████████▋| 201/208 [01:02<00:02,  3.29it/s]

    loss: 21.0625
    damlhhenes
    [55, 17, 3, 49, 8, 38, 14, 44, 11, 18]
    cledeylusl
    [55, 38, 14, 20, 9, 27, 9, 22, 52, 25]
    ctaaidenss
    [55, 55, 48, 53, 61, 41, 56, 56, 2, 60]
    synsezglec
    [55, 19, 40, 52, 56, 29, 39, 25, 56, 36]
    anennnckor
    [6, 16, 23, 56, 54, 46, 51, 33, 4, 47]

    100%|██████████| 208/208 [01:04<00:00,  4.03it/s]

    loss: 21.235027313232422
    nocuslethm
    [55, 6, 26, 22, 52, 25, 15, 32, 38, 14]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 4 of 10 =========
    train loss: 21.22| valid loss: 21.03

    ========= Epoch 5 of 10 =========

      0%|          | 1/208 [00:00<01:25,  2.41it/s]

    loss: 21.163387298583984
    nerydeydus
    [55, 4, 47, 19, 55, 8, 19, 36, 22, 7]
    lhyyt-dere
    [55, 8, 19, 17, 30, 17, 31, 4, 24, 56]
    vharlessal
    [55, 8, 4, 24, 25, 11, 18, 58, 48, 53]
    phylynessT
    [55, 8, 19, 25, 15, 44, 11, 18, 60, 34]
    yndnessyrd
    [6, 16, 40, 44, 11, 18, 58, 17, 47, 31]

     25%|██▍       | 51/208 [00:16<00:46,  3.34it/s]

    loss: 20.970998764038086
    Aalfnlyyne
    [55, 48, 53, 57, 21, 25, 28, 27, 21, 11]
    enodenesse
    [6, 29, 6, 55, 59, 44, 11, 18, 60, 23]
    seablyhetd
    [55, 4, 48, 37, 25, 27, 3, 45, 9, 20]
    vonrnTedce
    [55, 57, 21, 10, 46, 40, 4, 42, 25, 56]
    cedelylmes
    [55, 17, 31, 4, 25, 28, 53, 34, 17, 2]

     49%|████▊     | 101/208 [00:31<00:32,  3.31it/s]

    loss: 20.771623611450195
    aOdandlasm
    [6, 10, 31, 48, 50, 42, 25, 27, 7, 31]
    vcitkausly
    [55, 38, 61, 51, 33, 57, 22, 7, 25, 27]
    tymterseeh
    [55, 17, 2, 5, 4, 24, 2, 43, 17, 3]
    tyblykhrdy
    [55, 19, 37, 25, 15, 33, 38, 14, 20, 27]
    gyymmlyytl
    [55, 8, 19, 9, 0, 25, 28, 27, 32, 25]

     73%|███████▎  | 151/208 [00:47<00:18,  3.03it/s]

    loss: 20.630603790283203
    saledlyuse
    [55, 48, 53, 14, 20, 25, 27, 22, 52, 56]
    gyllylesss
    [55, 19, 37, 25, 15, 44, 11, 18, 58, 2]
    breakescke
    [55, 38, 14, 48, 5, 43, 2, 51, 33, 4]
    bintncnemt
    [55, 6, 16, 23, 9, 62, 54, 6, 16, 63]
    cyphledlye
    [55, 19, 55, 59, 44, 11, 20, 25, 27, 13]

     97%|█████████▋| 201/208 [01:02<00:02,  3.22it/s]

    loss: 20.628002166748047
    drybredide
    [55, 38, 19, 55, 38, 14, 20, 61, 41, 56]
    brinedness
    [55, 38, 61, 44, 11, 20, 44, 11, 18, 60]
    pidaltidAs
    [55, 61, 41, 48, 50, 32, 61, 41, 56, 2]
    fedessaesm
    [55, 4, 20, 11, 18, 60, 62, 56, 2, 0]
    sertedeyrt
    [55, 4, 24, 32, 4, 20, 14, 27, 26, 5]

    100%|██████████| 208/208 [01:04<00:00,  4.03it/s]

    loss: 21.04818344116211
    pasedsealf
    [55, 17, 2, 14, 20, 2, 59, 48, 53, 49]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 5 of 10 =========
    train loss: 20.83| valid loss: 20.78

    ========= Epoch 6 of 10 =========

      0%|          | 1/208 [00:00<01:24,  2.44it/s]

    loss: 20.581947326660156
    chyfnablic
    [55, 8, 19, 57, 54, 48, 37, 25, 61, 45]
    unymantess
    [6, 16, 30, 0, 48, 50, 44, 11, 18, 58]
    gyaltlyhtd
    [55, 19, 62, 46, 32, 25, 27, 8, 32, 42]
    chwvsnessu
    [55, 8, 38, 27, 52, 44, 11, 18, 58, 22]
    rhypalcten
    [55, 8, 19, 55, 48, 53, 36, 5, 43, 44]

     25%|██▍       | 51/208 [00:16<00:48,  3.23it/s]

    loss: 20.88625144958496
    phyessness
    [55, 8, 19, 11, 18, 60, 44, 11, 18, 60]
    lhymtalics
    [55, 8, 19, 55, 38, 48, 53, 61, 45, 2]
    onideryhys
    [6, 16, 61, 41, 56, 24, 27, 3, 30, 2]
    mhyetlyyne
    [55, 8, 19, 14, 32, 38, 27, 9, 44, 11]
    treydsyssm
    [55, 38, 14, 14, 20, 2, 30, 2, 58, 0]

     49%|████▊     | 101/208 [00:31<00:32,  3.31it/s]

    loss: 20.23219871520996
    bhedexonid
    [55, 8, 4, 20, 15, 26, 57, 46, 61, 41]
    elidenesss
    [6, 12, 61, 41, 56, 44, 11, 18, 60, 2]
    crenmabled
    [55, 38, 14, 40, 0, 48, 37, 25, 56, 41]
    blicaeonak
    [55, 38, 61, 51, 48, 13, 57, 54, 48, 50]
    bhticnenes
    [55, 8, 32, 61, 62, 44, 11, 44, 11, 18]

     73%|███████▎  | 151/208 [00:46<00:17,  3.24it/s]

    loss: 20.1969051361084
    trynsashea
    [55, 38, 27, 46, 2, 17, 2, 59, 14, 48]
    phednessym
    [55, 8, 4, 42, 44, 11, 18, 58, 30, 0]
    soHknesGly
    [55, 6, 36, 33, 44, 11, 18, 58, 25, 28]
    phrnalness
    [55, 8, 57, 54, 48, 50, 44, 11, 18, 58]
    lythiondee
    [55, 19, 55, 8, 61, 57, 46, 20, 14, 9]

     97%|█████████▋| 201/208 [01:02<00:02,  3.29it/s]

    loss: 20.51156234741211
    sasnessede
    [55, 17, 60, 44, 11, 18, 60, 14, 20, 13]
    tytinablyd
    [55, 14, 32, 61, 29, 48, 37, 25, 28, 20]
    andlyeonal
    [6, 16, 31, 25, 27, 59, 57, 46, 17, 25]
    lrytismned
    [55, 38, 14, 32, 61, 35, 3, 44, 11, 20]
    Faninglene
    [55, 57, 54, 61, 29, 39, 25, 56, 44, 11]

    100%|██████████| 208/208 [01:04<00:00,  4.07it/s]

    loss: 20.681076049804688
    toussyness
    [55, 57, 22, 2, 60, 30, 44, 11, 18, 60]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 6 of 10 =========
    train loss: 20.65| valid loss: 20.64

    ========= Epoch 7 of 10 =========

      0%|          | 1/208 [00:00<01:27,  2.36it/s]

    loss: 20.322513580322266
    gysthyatyn
    [55, 19, 36, 32, 38, 14, 48, 5, 15, 44]
    ullessness
    [6, 12, 25, 11, 18, 60, 44, 11, 18, 58]
    cletreedng
    [55, 38, 14, 32, 38, 14, 4, 20, 29, 39]
    bhyceriden
    [55, 8, 19, 55, 4, 47, 61, 41, 56, 44]
    elyleneEsi
    [6, 12, 19, 25, 15, 44, 11, 18, 58, 61]

     25%|██▍       | 51/208 [00:16<00:46,  3.38it/s]

    loss: 20.572463989257812
    petiacless
    [55, 6, 26, 61, 62, 45, 25, 11, 18, 60]
    chednedyli
    [55, 38, 14, 20, 44, 4, 20, 30, 12, 61]
    sitaliactl
    [55, 61, 5, 48, 53, 61, 62, 36, 32, 25]
    fyenessete
    [55, 14, 56, 44, 11, 18, 58, 17, 32, 4]
    ceeelaynls
    [55, 59, 56, 43, 25, 15, 14, 44, 11, 18]

     49%|████▊     | 101/208 [00:31<00:31,  3.38it/s]

    loss: 20.399456024169922
    deaneryhti
    [55, 59, 57, 54, 4, 47, 27, 8, 32, 61]
    gausnodmon
    [55, 57, 22, 52, 44, 11, 20, 0, 57, 54]
    mealingles
    [55, 59, 48, 53, 61, 29, 39, 25, 11, 18]
    nlessLcsel
    [55, 38, 11, 18, 58, 25, 51, 2, 9, 25]
    rhhytyldOa
    [55, 8, 38, 19, 55, 19, 12, 31, 61, 57]

     73%|███████▎  | 151/208 [00:46<00:18,  3.11it/s]

    loss: 20.67449951171875
    bueymaling
    [55, 59, 14, 30, 0, 48, 50, 61, 29, 39]
    ceeummonal
    [55, 59, 9, 22, 7, 0, 57, 54, 48, 53]
    BattyGedne
    [55, 6, 9, 5, 15, 44, 11, 20, 44, 11]
    colonnglen
    [55, 6, 12, 57, 54, 29, 39, 25, 56, 44]
    ilyldnessn
    [6, 12, 19, 12, 31, 44, 11, 18, 60, 44]

     97%|█████████▋| 201/208 [01:02<00:02,  2.99it/s]

    loss: 20.53281021118164
    mulusleals
    [55, 6, 12, 22, 2, 12, 9, 48, 50, 2]
    wytellyIes
    [55, 19, 55, 59, 12, 25, 28, 5, 13, 35]
    pordnessne
    [55, 6, 10, 40, 44, 11, 18, 60, 44, 11]
    dulyeetsee
    [55, 59, 25, 28, 13, 9, 9, 2, 13, 43]
    lhideEessn
    [55, 8, 61, 41, 56, 44, 11, 18, 60, 44]

    100%|██████████| 208/208 [01:04<00:00,  3.95it/s]

    loss: 19.893932342529297
    onthyenedl
    [6, 16, 63, 8, 19, 23, 44, 11, 20, 25]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 7 of 10 =========
    train loss: 20.54| valid loss: 20.55

    ========= Epoch 8 of 10 =========

      0%|          | 1/208 [00:00<01:39,  2.09it/s]

    loss: 20.745155334472656
    athyolymon
    [6, 36, 8, 19, 6, 25, 30, 0, 57, 54]
    dylRtymsla
    [55, 19, 12, 14, 32, 30, 0, 2, 38, 48]
    ometumglyn
    [6, 34, 17, 32, 22, 7, 39, 25, 27, 21]
    deermnglyn
    [55, 14, 4, 24, 0, 29, 39, 25, 28, 44]
    araeeumdre
    [6, 12, 48, 50, 13, 22, 7, 20, 44, 11]

     25%|██▍       | 51/208 [00:15<00:48,  3.22it/s]

    loss: 20.89170265197754
    zfalessdne
    [55, 55, 48, 53, 11, 18, 58, 42, 44, 11]
    neneumedly
    [55, 6, 16, 9, 22, 7, 23, 20, 25, 28]
    piausgedne
    [55, 61, 45, 22, 7, 39, 4, 20, 44, 11]
    kedmymarip
    [55, 59, 31, 0, 30, 0, 48, 47, 61, 45]
    hysmonllon
    [55, 19, 35, 0, 57, 21, 12, 12, 57, 54]

     49%|████▊     | 101/208 [00:31<00:34,  3.14it/s]

    loss: 20.404035568237305
    needynatri
    [55, 59, 14, 20, 19, 55, 48, 32, 38, 61]
    berdudicsy
    [55, 4, 24, 20, 10, 49, 61, 45, 2, 17]
    falnmonono
    [55, 48, 53, 21, 0, 57, 54, 57, 21, 57]
    epkymallys
    [6, 36, 33, 30, 0, 48, 37, 25, 27, 7]
    xelennssse
    [55, 59, 25, 56, 44, 11, 18, 58, 2, 59]

     73%|███████▎  | 151/208 [00:46<00:17,  3.17it/s]

    loss: 20.2503662109375
    ristfyryge
    [55, 61, 35, 3, 5, 1, 47, 27, 39, 59]
    dhyletikly
    [55, 8, 19, 12, 17, 32, 61, 62, 25, 28]
    ayyenessne
    [6, 26, 19, 1, 44, 11, 18, 58, 44, 14]
    tupnessnes
    [55, 59, 17, 44, 11, 18, 60, 44, 11, 18]
    loneelyedn
    [55, 57, 46, 59, 9, 25, 28, 14, 20, 44]

     97%|█████████▋| 201/208 [01:02<00:02,  3.17it/s]

    loss: 20.75147247314453
    beidlymsde
    [55, 6, 10, 49, 25, 30, 0, 7, 31, 4]
    acystnesst
    [6, 55, 19, 35, 3, 44, 11, 18, 60, 3]
    dhtylymgny
    [55, 8, 32, 30, 25, 27, 0, 57, 46, 14]
    slylydalre
    [55, 38, 30, 12, 19, 41, 48, 50, 44, 11]
    cukymdylye
    [55, 59, 33, 30, 0, 49, 30, 25, 27, 9]

    100%|██████████| 208/208 [01:04<00:00,  3.99it/s]

    loss: 20.438739776611328
    eydstgeabl
    [6, 16, 31, 35, 3, 39, 59, 48, 37, 25]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 8 of 10 =========
    train loss: 20.47| valid loss: 20.50

    ========= Epoch 9 of 10 =========

      0%|          | 1/208 [00:00<01:20,  2.58it/s]

    loss: 19.996936798095703
    thedleness
    [55, 59, 9, 20, 25, 56, 44, 11, 18, 58]
    aednedmada
    [6, 4, 42, 44, 11, 20, 7, 10, 41, 48]
    delidallym
    [55, 6, 12, 61, 41, 48, 37, 25, 28, 0]
    reeymeness
    [55, 59, 9, 19, 34, 17, 44, 11, 18, 58]
    hermonelys
    [55, 4, 24, 0, 57, 46, 9, 25, 28, 60]

     25%|██▍       | 51/208 [00:15<00:47,  3.29it/s]

    loss: 20.890419006347656
    Briallenes
    [55, 38, 14, 48, 37, 25, 56, 44, 11, 18]
    thiteednes
    [55, 38, 61, 5, 1, 11, 20, 44, 11, 18]
    pupksylymn
    [55, 6, 36, 33, 35, 30, 25, 28, 0, 44]
    ciofalsnes
    [55, 6, 10, 40, 48, 50, 52, 44, 11, 18]
    hoafednesf
    [55, 6, 10, 40, 4, 42, 44, 11, 18, 60]

     49%|████▊     | 101/208 [00:31<00:34,  3.11it/s]

    loss: 20.577880859375
    anlyggness
    [6, 26, 25, 28, 29, 39, 44, 11, 18, 60]
    yuynedness
    [6, 16, 30, 44, 11, 20, 44, 11, 18, 60]
    imalnessly
    [6, 34, 48, 53, 44, 11, 18, 58, 25, 28]
    sednedmene
    [55, 4, 42, 44, 11, 20, 34, 17, 44, 11]
    deefsheson
    [55, 59, 9, 40, 35, 3, 9, 31, 57, 21]

     73%|███████▎  | 151/208 [00:46<00:17,  3.21it/s]

    loss: 20.995319366455078
    ubketerony
    [6, 16, 33, 14, 32, 4, 24, 57, 21, 30]
    feabessnes
    [55, 14, 48, 37, 11, 18, 58, 44, 11, 18]
    qettermabl
    [55, 6, 36, 32, 4, 24, 0, 48, 37, 25]
    suananlyln
    [55, 59, 57, 46, 57, 21, 25, 30, 50, 44]
    Shyrlyacai
    [55, 38, 27, 37, 25, 27, 62, 51, 48, 50]

     97%|█████████▋| 201/208 [01:02<00:02,  2.98it/s]

    loss: 20.560609817504883
    rnyerynerm
    [55, 55, 14, 4, 47, 27, 21, 4, 24, 0]
    nhonlyxabl
    [55, 38, 57, 46, 25, 30, 26, 48, 37, 25]
    to
    hylleal
    [55, 6, 36, 8, 19, 12, 12, 9, 48, 37]
    ovusdledne
    [6, 55, 22, 2, 41, 25, 56, 20, 44, 11]
    cecnaleAti
    [55, 6, 34, 54, 48, 50, 14, 23, 32, 61]

    100%|██████████| 208/208 [01:04<00:00,  3.81it/s]

    loss: 20.599403381347656
    aitylydled
    [6, 10, 40, 30, 25, 30, 20, 25, 56, 20]

      0%|          | 0/208 [00:00<?, ?it/s]

    ========= Results: epoch 9 of 10 =========
    train loss: 20.43| valid loss: 20.47

    ========= Epoch 10 of 10 =========

      0%|          | 1/208 [00:00<01:28,  2.34it/s]

    loss: 20.2786865234375
    rphyenessn
    [55, 55, 38, 30, 17, 44, 11, 18, 58, 44]
    onglymntry
    [6, 29, 39, 25, 27, 0, 16, 63, 38, 19]
    celednledm
    [55, 14, 25, 56, 49, 44, 25, 56, 20, 7]
    ceeeonumea
    [55, 59, 9, 14, 57, 54, 22, 7, 56, 48]
    ghalulneto
    [55, 38, 48, 50, 22, 52, 44, 11, 32, 4]

     25%|██▍       | 51/208 [00:15<00:48,  3.27it/s]

    loss: 20.243629455566406
    adedstness
    [6, 41, 56, 20, 35, 3, 44, 11, 18, 58]
    teleyctonl
    [55, 6, 16, 23, 19, 36, 32, 57, 21, 25]
    miackmonle
    [55, 61, 62, 51, 33, 0, 57, 46, 25, 11]
    raldlynall
    [55, 6, 10, 49, 25, 27, 54, 48, 37, 25]
    grgumsnems
    [55, 38, 57, 22, 7, 60, 44, 11, 18, 60]

     49%|████▊     | 101/208 [00:31<00:34,  3.08it/s]

    loss: 20.28844451904297
    srymseondi
    [55, 38, 30, 0, 2, 23, 57, 54, 41, 61]
    genessnedu
    [55, 14, 44, 11, 18, 60, 44, 11, 20, 22]
    gyalnaeyck
    [55, 14, 48, 50, 54, 48, 50, 19, 36, 33]
    wylalustor
    [55, 19, 12, 48, 53, 22, 2, 32, 4, 20]
    nueylyetid
    [55, 59, 9, 30, 25, 28, 14, 32, 61, 41]

     73%|███████▎  | 151/208 [00:46<00:17,  3.20it/s]

    loss: 20.66337013244629
    realnesses
    [55, 59, 48, 50, 44, 11, 18, 58, 11, 18]
    vlogmiackl
    [55, 38, 6, 34, 0, 61, 62, 51, 33, 25]
    faeonsCess
    [55, 48, 13, 57, 54, 7, 44, 11, 18, 58]
    leonsssraF
    [55, 59, 57, 21, 2, 2, 7, 59, 48, 53]
    tanessxess
    [55, 6, 44, 11, 18, 58, 44, 11, 18, 60]

     97%|█████████▋| 201/208 [01:02<00:02,  3.23it/s]

    loss: 20.529830932617188
    orslyedeel
    [6, 26, 52, 25, 28, 59, 41, 59, 14, 25]
    piymonalsb
    [55, 38, 30, 0, 57, 54, 48, 53, 2, 37]
    jeolkarity
    [55, 14, 6, 12, 33, 48, 47, 61, 5, 15]
    tismalingl
    [55, 61, 35, 0, 48, 53, 61, 29, 39, 25]
    Zipkymusye
    [55, 6, 36, 33, 30, 0, 22, 2, 27, 9]

    100%|██████████| 208/208 [01:04<00:00,  4.05it/s]

    loss: 20.503211975097656
    tianatenes
    [55, 61, 57, 54, 48, 5, 43, 44, 11, 18]
    ========= Results: epoch 10 of 10 =========
    train loss: 20.41| valid loss: 20.45

You may wish to try different values of $N$ and see what the impact on sample quality is.

It's also interesting to look at the visited states. For instance, I noticed the following interesting behavior:
- If weight decay is used (`weight_decay=0.00001` in the initialization of the optimizer), the model almost always picks one of two states as the initial state, and uses one of these states to emit vowels and the other to emit consonants (without us programming that behavior or telling the model which letters are vowels!).
- But if no weight decay is used, then the model always starts from one particular state. I suspect that this is because weight decay encourages the state priors to be not too big, and if one state prior is very big, the model will always start from that state.

## Conclusion

HMMs used to be very popular in natural language processing, but they have largely been overshadowed by neural network models like RNNs and Transformers. Still, it is fun and instructive to study the HMM; some commonly used machine learning techniques like [Connectionist Temporal Classification](https://www.cs.toronto.edu/~graves/icml_2006.pdf) are inspired by HMM methods. HMMs are [still used in conjunction with neural networks in speech recognition](https://arxiv.org/abs/1811.07453), where the assumption of a one-hot state makes sense for modelling phonemes, which are spoken one at a time.

## Acknowledgments

This notebook is based partly on Lawrence Rabiner's excellent article "[A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition](https://www.cs.cmu.edu/~cga/behavior/rabiner1.pdf)", which you may also like to check out. Thanks also to Dima Serdyuk and Kyle Gorman for their feedback on the draft.


# Introduction to the Harmonic Oscillator

*Note:* Much of this is adapted/copied from https://flothesof.github.io/harmonic-oscillator-three-methods-solution.html

This week week we are going to begin studying molecular dynamics, which uses classical mechanics to study molecular systems. Our "hydrogen atom" in this section will be the 1D harmomic oscillator.

The harmonic oscillator is a system that, when displaced from its equilibrium position, experiences a restoring force F proportional to the displacement x:

$$F=-kx$$

The potential energy of this system is

$$V = {1 \over 2}k{x^2}$$

These are sometime rewritten as

$$ F=- \omega_0^2 m x, \text{ }  V(x) = {1 \over 2}  \omega_0^2 m {x^2}$$

Where $\omega_0  = \sqrt {{k \over m}} $

In classical mechanics, our goal is to determine the equations of motion, $x(t),y(t)$, that describe our system.

In this notebook we will use sympy to solve an second order, ordinary differential equation.

## 1. Solving differential equations with sympy

Soliving differential equations can be tough, and there is not always a set plan on how to proceed. Luckily for us, the harmonic osscillator is the classic second order diffferential eqations.

Consider the following second order differential equation

$$ay(t)''+by(t)'=c$$

where $y(t)'' = {{{d^2}y} \over {dt^2}}$, and $y(t)' = {{{d}y} \over {dt}}$

We can rewrite this as a homogeneous linear differential equations

$$ay(t)''+by(t)'-c=0$$

The goal here is to find $y(t)$, similar to our classical mechanics problems. Lets use sympy to solve this equation

### Second order ordinary differential equation

First we import the sympy library

```python
import sympy as sym
```

Next we initialize pretty printing

```python
sym.init_printing()
```

Next we will set our symbols

```python
t,a,b,c=sym.symbols("t,a,b,c")
```

Now for somehting new. We can define functions using `sym.Function("f")`

```python
y=sym.Function("y")
y(t)
```

Now, If I want to define a first or second derivative, I can use `sym.diff`

```python
sym.diff(y(t),(t,1)),sym.diff(y(t),(t,2))
```

My differential equation can be written as follows

```python
dfeq= a*sym.diff(y(t),(t,2))+b*sym.diff(y(t),(t,1))-c
dfeq
```

```python
sol = sym.dsolve(dfeq)
sol
```

The two constants $C_1$ and $C_2$ can be determined by setting boundry conditions.
First, we can set the condition $y(t=0)=y_0$

The next intial condition we will set is $y'(t=0)=v_0$

To setup the equality we want to solve, we are using `sym.Eq`. This function sets up an equaility between a lhs aand rhs of an equation

```python
# sym.Eq example
alpha,beta=sym.symbols("alpha,beta")
sym.Eq(alpha+2,beta)
```

Back to the actual problem

```python
y0,v0=sym.symbols("y_0,v_0")
ics=[sym.Eq(sol.args[1].subs(t, 0), y0),
     sym.Eq(sol.args[1].diff(t).subs(t, 0), v0)]
ics
```

We can use this result to first solve for $C_2$ and then solve for $C_1$.
Or we can use sympy to solve this for us.

```python
solved_ics=sym.solve(ics)
solved_ics
```

Substitute the result back into $y(t)$

```python
full_sol = sol.subs(solved_ics[0])
full_sol
```

We can plot this result too. Assume that $a,b,c=1$ and that the starting conditions are $y_0=0,v_0=0$

We will use two sample problems:

* case 1 : initial position is nonzero and initial velocity is zero
* case 2 : initial position is zero and initialvelocity is nonzero

```python
# Print plots
%matplotlib inline
```

#### Initial velocity set to zero

```python
case1 = sym.simplify(full_sol.subs({y0:0, v0:0, a:1, b:1, c:1}))
case1
```

```python
sym.plot(case1.rhs)
sym.plot(case1.rhs,(t,-2,2))
```

#### Initial velocity set to one

```python
case2 = sym.simplify(full_sol.subs({y0:0, v0:1, a:1, b:1, c:1}))
case2
```

```python
sym.plot(case2.lhs,(t,-2,2))
```

## Calculate the phase space

As we will see in lecture, the state of our classical systems are defined as points in phase space, a hyperspace defined by ${{\bf{r}}^N},{{\bf{p}}^N}$. We will convert our sympy expression into a numerical function so that we can plot the path of $y(t)$ in phase space $y,y'$.

```python
case1
```

```python
# Import numpy library
import numpy as np

# Make numerical functions out of symbolic expressions
yfunc=sym.lambdify(t,case1.rhs,'numpy')
vfunc=sym.lambdify(t,case1.rhs.diff(t),'numpy')

# Make list of numbers
tlst=np.linspace(-2,2,100)

# Import pyplot
import matplotlib
import matplotlib.pyplot as plt
# Make plot
plt.plot(yfunc(tlst),vfunc(tlst))
plt.xlabel('$y$')
plt.ylabel("$y'$")
plt.show()
```

### Exercise 1.1

Change the initial starting conditions and see how that changes the plots. Make three different plots with different starting conditions

```python
#case 3 : initial position is zero and initialvelocity is nonzero
case3 = sym.simplify(full_sol.subs({y0:0, v0:5, a:1, b:1, c:1}))
tlst=np.linspace(-2,2,100)
sym.plot(case3.rhs,(t,-2,2))

#case 4 : initial position is nonzero and initialvelocity is nonzero
case4 = sym.simplify(full_sol.subs({y0:-1, v0:5, a:1, b:1, c:1}))
tlst=np.linspace(-2,2,100)
sym.plot(case4.rhs,(t,-2,2))

#case 5 : initial position is nonzero and initialvelocity is zero
case5 = sym.simplify(full_sol.subs({y0:3, v0:0, a:1, b:1, c:1}))
tlst=np.linspace(-2,2,100)
sym.plot(case5.rhs,(t,-2,2))
```

## 2. Harmonic oscillator

Applying the harmonic oscillator force to Newton's second law leads to the following second order differential equation

$$ F = m a $$

$$ F= - \omega_0^2 m x $$

$$ a = - \omega_0^2 x $$

$$ x(t)'' = - \omega_0^2 x $$

The final expression can be rearranged into a second order homogenous differential equation, and can be solved using the methods we used above

Your goal is determine and plot the equations of motion of a 1D harmomnic oscillator

### Exercise 2.1

1. Use the methodology above to determine the equations of motion $x(t), v(t)$ for a harmonic ocillator
1. Solve for any constants by using the following initial conditions: $x(0)=x_0, v(0)=v_0$
1. Show expressions for and plot the equations of motion for the following cases:
    1. $x(0)=0, v(0)=0$
    1. $x(0)=0, v(0)>0$
    1. $x(0)>0, v(0)=0$
    1. $x(0)<0, v(0)=0$
1. Plot the phasespace diagram for the harmonic oscillator

```python
import sympy as sym
t,w0=sym.symbols("t,w0")
x = sym.Function("x")
x(t)
sym.diff(x(t),(t,1)),sym.diff(x(t),(t,2))
```

```python
dfeq= sym.diff(x(t),(t,2))+((w0)**2)*x(t)
dfeq
```

```python
sol = sym.dsolve(dfeq)
sol
```

```python
#First, we can set the condition  𝑥(t=0)=𝑥0
#The next intial condition we will set is  x′(𝑡=0)=𝑣0 or 𝑣(t=0)=𝑣0
x0,v0=sym.symbols("x_0,v_0")
ics=[sym.Eq(sol.args[1].subs(t, 0), x0),
     sym.Eq(sol.args[1].diff(t).subs(t, 0), v0)]
ics
```

```python
solved_ics=sym.solve(ics)
solved_ics
```

```python
full_sol = sol.subs(solved_ics[0])
full_sol
```

```python
#We can plot this result too. Assume that  𝑎,𝑏,𝑐=1  and that the starting conditions are  x0=0,𝑣0=0
#We will have four cases:

#caseA: 𝑥(0)=0,𝑣(0)=0
#caseB: 𝑥(0)=0,𝑣(0)>0
#caseC: 𝑥(0)>0,𝑣(0)=0
#caseD: 𝑥(0)<0,𝑣(0)=0
```

```python
caseA = sym.simplify(full_sol.subs({x0:0, v0:0, w0:1}))
caseA
```

```python
sym.plot(caseA.rhs)
sym.plot(caseA.rhs,(t,-10,10))
```

```python
caseB = sym.simplify(full_sol.subs({x0:0, v0:2, w0:1}))
caseB
```

```python
sym.plot(caseB.rhs)
sym.plot(caseB.rhs,(t,-10,10))
```

```python
caseC = sym.simplify(full_sol.subs({x0:2, v0:0, w0:1}))
caseC
```

```python
sym.plot(caseC.rhs)
sym.plot(caseC.rhs,(t,-10,10))
```

```python
caseD = sym.simplify(full_sol.subs({x0:-2, v0:0, w0:1}))
caseD
```

```python
sym.plot(caseD.rhs)
sym.plot(caseD.rhs,(t,-10,10))
```

```python
#phasespace
# Import numpy library
import numpy as np

# Make numerical functions out of symbolic expressions
xfunc=sym.lambdify(t,caseB.rhs,'numpy')
vfunc=sym.lambdify(t,caseB.rhs.diff(t),'numpy')

# Make list of numbers
tlst=np.linspace(-10,10,100)

# Import pyplot
import matplotlib
import matplotlib.pyplot as plt
# Make plot
plt.plot(xfunc(tlst),vfunc(tlst))
plt.xlabel('$x$')
plt.ylabel("$x'$")
plt.show()
```

```python

```


## BSA - $\epsilon(\lambda)$ according to [Pahn. et.al 2013](https://aip.scitation.org/doi/abs/10.1063/1.4826514)

They use Lorentz oscillators to get the dielectric constant of Bovine Serum Albumina as a function of the wavelngth.

\begin{equation}
    \epsilon_{protein}(\lambda) = 1 + \sum_j{\frac{1/{\Lambda_j}^2}
    {1/{\lambda_j}^2 -i/(\lambda\gamma_j) - 1/{\lambda}^2}}
\end{equation}

Where:

* $\Lambda_1 = 10853.54 $ nm
* $\Lambda_2 = 878.5 $ nm
* $\Lambda_3 = 92.6 $ nm
* $\Lambda_4 = 82.81 $ nm
-------------------------
* $\gamma_1 = \infty $ nm
* $\gamma_2 = 2484.52 $ nm
* $\gamma_3 = 155.28 $ nm
* $\gamma_4 = 65.38 $ nm
--------------------------
* $\lambda_1 = 6059.8 $ nm
* $\lambda_2 = 194.1 $ nm
* $\lambda_3 = 99.38 $ nm
* $\lambda_4 = 57.78 $ nm

Let's avoid the problems that $\gamma_1= \infty$ might cause when computing the function by separating that term and setting the $1/\gamma_1=0$.

Then our equation will look like:

\begin{equation}
    \epsilon_{protein}(\lambda) = 1 + \frac{1/{\Lambda_1}^2}
                                {1/{\lambda_1}^2 - 1/{\lambda}^2} +
                                \sum_{j=2}^{4}{\frac{1/{\Lambda_j}^2}
                                {1/{\lambda_j}^2 -i/(\lambda\gamma_j) - 1/{\lambda}^2}}
\end{equation}

```python
import numpy
```

```python
#define _1 variables
Lambda_1 = 10853.54
lambda_1 = 6059.8
```

```python
#define the rest (2-4) as an array

Lambda_array = numpy.array([878.5, 92.6, 82.81])
gamma_array = numpy.array([2484.52, 155.28, 65.38])
lambda_array = numpy.array([194.1, 99.38, 57.78])

#Let's use the version 1/ thing so it's less confusing

sigma_array = 1/Lambda_array
alpha_array = 1/gamma_array
f_array = 1/lambda_array
```

\begin{equation}
    \epsilon_{protein}(\lambda) = 1 + \frac{{\sigma_1}^2}
                                {{f_1}^2 - 1/{\lambda}^2} +
                                \sum_{j=2}^{4}{\frac{{\sigma_j}^2}
                                {{f_j}^2 -i\alpha_j/\lambda - 1/{\lambda}^2}}
\end{equation}

```python
def protein_dielectric(lamb, Lambda_1, lambda_1, Lambda_array, lambda_array, gamma_array):
    '''Computes the value of dielectric constant of a protein for a certain
       wavelength. It uses Lorentz oscillators, obtained from Pahn, etal. 2013

    Arguments:
    ----------
    lamb        : float, wavelength in [nm] where we want to
                         know the dielectric constant.
    Lambda_1    : float, Lorentz oscillator upper lambda 1.
    lambda_1    : float, Lorentz oscillator lower lambda 1.
    Lambda_array: array of floats, Lorentz oscillator upper lambda 2,3,4.
    lambda_array: array of floats, Lorentz oscillator lower lambda 2,3,4.
    gamma_array : array of floats, Lorentz oscillator gamma 2,3,4.

    Returns:
    --------
    epsilon: complex, dielectric constant.
    '''
    #Let's use the version 1/ thing so it's less confusing
    sigma_1 = 1/Lambda_1
    f_1 = 1/lambda_1

    sigma_array = 1/Lambda_array
    alpha_array = 1/gamma_array
    f_array = 1/lambda_array

    epsilon = 1 + sigma_1**2/(f_1**2 - (1/lamb**2)) + numpy.sum(
                            sigma_array**2/(f_array**2 -
                                            1j*alpha_array/lamb
                                          - (1/lamb**2)))

    return epsilon
```

```python
lamb = numpy.linspace(0.5,600,201)
```

```python
epsilon = numpy.zeros(len(lamb), dtype=numpy.complex128)
```

```python
for i in range(len(lamb)):
    epsilon[i] = protein_dielectric(lamb[i], Lambda_1, lambda_1, Lambda_array, lambda_array, gamma_array)

```

```python
from matplotlib import pyplot
%matplotlib inline
```

```python
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 14
rcParams['xtick.top'] = True
rcParams['ytick.right'] = True
rcParams['axes.linewidth'] = 2
```

```python
pyplot.figure(figsize=(12,4))

pyplot.subplot(121)

pyplot.plot(lamb,epsilon.real,ls='', marker='o', color='#2929a3', ms=5)

pyplot.xlabel('Wavelength [nm]')
pyplot.ylabel('Dielectric constant')
pyplot.xlim(min(lamb), max(lamb))
pyplot.ylim(0, 3.5)

pyplot.xticks(numpy.linspace(0, max(lamb), 13), rotation=35)
pyplot.tick_params(axis='both', length=8, width=1, direction='in')
pyplot.title('Real')
pyplot.grid(linestyle=':')

pyplot.subplot(122)

pyplot.plot(lamb,epsilon.imag, ls='', marker='o', color='#ff5733', ms=5)

pyplot.xlabel('Wavelength [nm]')
#pyplot.ylabel('Refractive index')
pyplot.xlim(min(lamb), max(lamb))
pyplot.ylim(0, 3.5)
pyplot.xticks(numpy.linspace(0, max(lamb), 13), rotation=35)
pyplot.tick_params(axis='both', length=8, width=1, direction='in')
pyplot.title('Imaginary')
pyplot.grid(linestyle=':')
```

To run the example lspr_7spheres we want to know the value of epsilon when the wavelength is 380 nm.

```python
eps_380 = protein_dielectric(380, Lambda_1, lambda_1, Lambda_array, lambda_array, gamma_array)
print('lambda = {}, epsilon_protein = {}'.format(380, eps_380))
```

    lambda = 380, epsilon_protein = (2.751434851700645+0.2860230708531996j)


#### 다음의 내용은 패스트캠퍼스 김도형 박사님의 강의자료를 정리한 것이다

### 벡터 a의 길이: norm ‖𝑎‖

```python
a = np.array([1, 2])
np.linalg.norm(a)
```

    2.23606797749979

### 단위벡터
- 길이가 1인 벡터

$$
\begin{align}
\dfrac{x}{\| x \|}
\end{align}
$$

```python
a = np.array([1, 0])
b = np.array([0, 1])
c = np.array([1/np.sqrt(2), 1/np.sqrt(2)])
np.linalg.norm(a), np.linalg.norm(b), np.linalg.norm(c)
```

    (1.0, 1.0, 0.9999999999999999)

### 유클리드 거리
- 두 벡터가 가리키는 점 사이의 거리
$$
\begin{align}
\| a - b \|^2 = \| a \|^2 + \| b \|^2 - 2 a^T b
\end{align}
$$

### 벡터의 내적 코사인 함수로 나타내기
$$
\begin{align}
a^Tb = \|a\|\|b\| \cos\theta
\end{align}
$$

### 직교
- 두 벡터가 이루는 각이 90도인 경우
- 코사인이 0이 된다
$$
\begin{align}
a^T b = b^T a = 0   \;\;\;\; \leftrightarrow \;\;\;\; a \perp b
\end{align}
$$

### 정규직교
만약 $N$개의 단위벡터 $v_1, v_2, \cdots, v_N$가 서로 직교하면 **정규직교(orthonormal)**라고 한다.

$$
\begin{align}
\|v_i\| = 1 \;\;\leftrightarrow \;\; v_i^Tv_i = 1
\end{align}
$$

$$
\begin{align}
v_i^T v_j = 0 \;\;(i \neq j)
\end{align}
$$

### 코사인 거리
- 1에서 코사인 유사도만큼 뺀 것
$$
\begin{align}
\text{cosine distance} = 1 - \text{cosine similarity} = 1 - \dfrac{x^Ty}{\|x\|\|y\|}
\end{align}
$$

```python
a=np.array([4,5,2,2])
b=np.array([4,0,2,0])
c=np.array([2,2,0,1])

```

```python
#유클리드 거리
np.linalg.norm(a-b)
```

    5.385164807134504

```python
#코사인 거리
1- a@b/(np.linalg.norm(a)*np.linalg.norm(b))
```

    0.36112343500006017

### 투영성분
벡터 $a$를 다른 벡터 $b$에 직교하는 성분과 벡터 $b$에 평행한 성분으로로 분해할 수 있는데, 평행한 성분을 벡터 $b$에 대한 **투영성분(projection)**, 벡터 $b$에 수직인 성분을 벡터 $b$에 대한 **직교성분(rejection)**이라고 하며 각각 다음과 같이 표기한다.

$$
\begin{align}
a^{\Vert b}
\end{align}
$$

$$
\begin{align}
a^{\perp b}
\end{align}
$$

### 투영성분의 길이
$$
\begin{align}
\| a^{\Vert b} \|
= \|a\|\cos\theta
= \dfrac{\|a\|\|b\|\cos\theta}{\|b\|}
= \dfrac{a^Tb}{\|b\|}
= \dfrac{b^Ta}{\|b\|}
= a^T\dfrac{b}{\|b\|}
\end{align}
$$

### 투영성분의 벡터
- 길이 * 방향

$$
\begin{align}
a^{\Vert b} = \dfrac{a^Tb}{\|b\|} \dfrac{b}{\|b\|}= \dfrac{a^Tb}{\|b\|^2}b
\end{align}
$$

### 직교성분
$$
\begin{align}
a^{\perp b} = a - a^{\Vert b}
\end{align}
$$

### 직선의 방정식
- 벡터w가 가르키는 방향과 만나면서, w와 직교하는 직선의 방정식

$$
\begin{align}
w^T(x - w) = 0
\end{align}
$$

정리하면 다음과 같아진다.

$$
\begin{align}
w^T(x - w) = w^Tx - w^Tw = w^Tx - \| w \|^2
\end{align}
$$

$$
\begin{align}
w^Tx - \| w \|^2 = 0
\end{align}
$$

이 직선과 원점 사이의 거리는 벡터 $w$의 놈 $\|w\|$이다.

$$
\begin{align}
\|w\|
\end{align}
$$

### w에 수직인 직선의 방정식
이번에는 벡터 $w$가 가리키는 점을 지나야 한다는 조건을 없애고 단순히

* 벡터 $w$에 수직인

직선 $x$의 방정식을 구해보자.

이때는 직선이 $w$가 아니라 $w$와 방향이 같고 길이가 다른 벡터 $w'=cw$을 지날 것이다. $c$는 양의 실수이다.

위에서 했던 방법으로 다시 직선의 방정식을 구하면 다음과 같다.

$$
\begin{align}
w'^Tx - \| w' \|^2 =  cw^Tx - c^2 \| w \|^2 = 0
\tag{3.1.53}
\end{align}
$$

$$
\begin{align}
w^Tx - c \| w \|^2 = 0
\tag{3.1.54}
\end{align}
$$

여기에서 $c \| w \|^2$는 임의의 수가 될 수 있으므로 단순히 벡터 $w$에 수직인 직선의 방정식은 다음과 같이 나타낼 수 있다.

$$
\begin{align}
w^Tx - w_0 = 0
\tag{3.1.55}
\end{align}
$$

이 직선과 원점 사이의 거리는 다음과 같다.

$$
\begin{align}
c \| w \| = \dfrac{w_0}{\|w\|}
\tag{3.1.56}
\end{align}
$$

### 직선과 점의 거리

이번에는 직선 $w^Tx - \|w\|^2 = 0$ 과 이 직선 위에 있지 않은 점 $x'$ 사이의 거리를 구해보자.

벡터 $w$에 대한 벡터 $x'$의 투영성분 $x'^{\Vert w}$의 길이

$$
\begin{align}
\|x'^{\Vert w}\| = \dfrac{w^Tx'}{\|w\|}
\tag{3.1.59}
\end{align}
$$

직선과 점 $x'$ 사이의 거리는 이 길이에서 원점에서 직선까지의 거리 $\|w\|$를 뺀 값의 절댓값이다.

$$
\begin{align}
\left|  \|x'^{\Vert w}\| - \|w\| \right| =
\left| \dfrac{w^Tx'}{\|w\|} - \|w\| \right| =
\dfrac{\left|w^Tx' - \|w\|^2 \right|}{\|w\|}
\tag{3.1.60}
\end{align}
$$

직선의 방정식이 $w^Tx - w_0 = 0$이면 직선과 점의 거리는 다음과 같다.

$$
\begin{align}
\dfrac{\left|w^Tx' - w_0 \right|}{\|w\|}
\tag{3.1.61}
\end{align}
$$

- 분자값이 직선의 방정식과 유사해보이지만, 직선 위의 점이 아니라 다른 점을 x에 넣기 때문에 0이 아닌 값이 나온다

### 선형독립
벡터 집합 $x_1, x_2, \ldots, x_N$을 이루는 벡터의 선형조합이 영벡터가 되도록 하는 스칼라 계수 $c_1, c_2, \ldots, c_N$이 존재하면 이 벡터들이 **선형종속(linearly dependent)**이라고 한다. 단 $c_1 = c_2 = \cdots = c_N = 0$ 으로 계수가 모두 0인 경우는 제외한다.

$$
\begin{align}
c_1 x_1 + c_2 x_2 + \cdots + c_N x_N = 0
\tag{3.2.1}
\end{align}
$$

반대로 벡터들의 선형조합이 0이 되면서 모두 0은 아닌 계수들이 존재하지 않으면 그 벡터들은 **선형독립(linearly independent)**이라고 한다.

### 선형종속이 되는 경우 3가지
- 다중공선성: 특징행렬 X의 열벡터가 선형종속에 가까운 경우

#### 경우 1: 벡터의 개수가 벡터의 차원보다 크면 선형종속이다.
#### 경우 2: 값이 같은 벡터가 있으면 반드시 선형종속이다.
#### 경우 3: 어떤 벡터가 다른 벡터의 선형조합이면 반드시 선형종속이다.

### 랭크
- 서로 독립인 행/열의 최대 개수
- 정리) 행랭크와 열랭크는 항상 같다

### 풀랭크
- 랭크가 행의 개수와 열의 개수 중 작은 값과 같으면 **풀랭크(full rank)**라고 한다.

### 벡터공간과 기저벡터

여러 벡터를 선형조합을 하면 다른 벡터를 만들 수 있다. 벡터 N개가 서로 선형독립이면 이 벡터들을 선형조합하여 만들어지는 모든 벡터의 집합을 **벡터공간(vector space)** $V$라 하고 이 벡터공간의 차원을 $N$이라고 한다. 그리고 그 벡터들을 벡터공간의 **기저벡터(basis vector)**라고 한다.

> **[정리] $N$개의 $N$차원 벡터 $x_1, x_2, \cdots, x_N$이 선형독립이면 이를 선형조합하여 모든 $N$차원 벡터를 만들 수 있다.**

### 정방행렬의 랭크와 역행렬
> **[정리] 정방행렬이 풀랭크면 역행렬이 존재한다. 역도 성립한다. 즉, 정방행렬의 역행렬이 존재하면 풀랭크**다.

### 벡터공간의 투영
만약 기저벡터 $v_1, v_2, \cdots, v_M$가 정규직교(orthonormal)이면 투영벡터 $x^{\Vert v}$는 각 기저벡터에 대한 내적값으로 표현된다.

$$
\begin{align}
x^{\Vert V} = (x^Tv_1)v_1 + (x^Tv_2)v_2 + \cdots + (x^Tv_M)v_M
\tag{3.2.47}
\end{align}
$$

그리고 투영벡터의 길이의 제곱은 각 기저벡터와의 내적의 제곱합이다.

$$
\begin{align}
\Vert x^{\Vert V} \Vert^2 = \sum_{i=1}^M (x^Tv_i)^2
\tag{3.2.48}
\end{align}
$$

**직교벡터 $x^{\perp V}$는 기저벡터 $v_1, v_2, \cdots, v_M$으로 이루어진 벡터공간의 모든 벡터에 대해 직교한다**.

**벡터 $x$의 투영벡터 $x^{\Vert V}$은 기저벡터 $v_1, v_2, \cdots, v_M$으로 이루어진 벡터공간의 모든 벡터 중에서 가장 벡터 $x$와 가까운 벡터**

- 표준기저벡터: 기저벡터 중 하나만 1이고 나머지 0인 것

### 좌표변환
- 새로운 기저벡터에 대해 좌표를 표시한 것

2차원의 경우를 예로 들어보자. 벡터 $x$의 기저벡터 $\{e_1, e_2\}$에 대한 좌표 $x_e$를 새로운 기저벡터 $\{g_1, g_2\}$에 대한 좌표 $x_g$로 변환하고자 한다.

새로운 기저벡터에 대한 좌푯값이 가리키는 실제 위치는 원래의 벡터가 가리키는 실제 위치와 같아야 되므로

$$
\begin{align}
x = x_{e1}e_1 + x_{e2}e_2 = x_{g1} g_1 + x_{g2} g_2
\tag{3.2.63}
\end{align}
$$

$$
\begin{align}
x
= \begin{bmatrix} e_1 & e_2 \end{bmatrix} x_e
= \begin{bmatrix} g_1 & g_2 \end{bmatrix} x_g
\tag{3.2.64}
\end{align}
$$

이 식에

$$
\begin{align}
\begin{bmatrix} g_1 & g_2 \end{bmatrix}
= \begin{bmatrix} e_1 & e_2 \end{bmatrix}
A
\tag{3.2.65}
\end{align}
$$

를 대입하면

$$
\begin{align}
x
= \begin{bmatrix} e_1 & e_2 \end{bmatrix} x_e
= \begin{bmatrix} e_1 & e_2 \end{bmatrix} A x_g
\tag{3.2.66}
\end{align}
$$

이 된다. 이 식으로부터 다음 식이 성립한다.

$$
\begin{align}
x_e = A x_g
\tag{3.2.67}
\end{align}
$$

$$
\begin{align}
x_g = A^{-1}x_e = Tx_e
\tag{3.2.68}
\end{align}
$$

이 때 $A$의 역행렬 $T=A^{-1}$을 **변환행렬(transform matrix)**이라고 한다.

```python
e1 = np.array([1, 0])
e2 = np.array([0, 1])
g1 = np.array([1, 1]) / np.sqrt(2)
g2 = np.array([-1, 1]) / np.sqrt(2)
x = np.array([2, 2])
A = np.vstack([g1, g2]).T
A
```

    array([[ 0.70710678, -0.70710678],
           [ 0.70710678,  0.70710678]])

```python
# 변환행렬
Ainv = np.linalg.inv(A)
Ainv
```

    array([[ 0.70710678,  0.70710678],
           [-0.70710678,  0.70710678]])

```python
#g1 g2를 기저벡터로 가지는 새로운 좌표
Ainv.dot(x)
```

    array([2.82842712, 0.        ])

### 고윳값과 고유분해
정방 행렬 $A$에 대해 다음 식을 만족하는 영벡터가 아닌 벡터 $v$, 실수 $\lambda$를 찾을 수 있다고 가정하자.

$$
\begin{align}
Av = \lambda v
\tag{3.3.1}
\end{align}
$$

위 식을 만족하는 실수 $\lambda$를 **고윳값**(eigenvalue), 벡터 $v$를 **고유벡터**(eigenvector)라고 한다.

$$
\begin{align}
Av - \lambda v = (A - \lambda I) v = 0
\tag{3.3.2}
\end{align}
$$

- $v$와 방향이 같은 벡터는 모두 고유벡터가 되므로 보통 고유벡터를 표시할 때 길이가 1인 단위벡터가 되도록 정규화 한다

$$
\begin{align}
\dfrac{v}{\|v\|}
\tag{3.3.8}
\end{align}
$$

### 고윳값 찾기 : 특성방정식
아래의 식을 만족하는 해를 찾으면 된다
$$
\begin{align}
\det \left( A - \lambda I \right) = 0
\tag{3.3.16}
\end{align}
$$
단,  $A - \lambda I$는 역행렬이 존재하지 않는다(만약 역행렬이 존재하면 $v$는 영행렬이 되어야 하기 때문이다

## 정방행렬 특성

### 정방행렬) 고윳값 개수 찾기
- 중복된 고윳값을 각각 별개로 생각하고 복소수인 고윳값도 고려한다면 $N$차원 정방행렬의 고윳값은 항상 $N$개다.

### 고윳값과 행렬식 및 대각합과의 관계
$$
\begin{align}
\det(A)=\prod_{i=1}^N \lambda_i
\tag{3.3.28}
\end{align}
$$

$$
\begin{align}
\text{tr}(A) =\sum_{i=1}^N \lambda_i
\tag{3.3.29}
\end{align}
$$

### 고유벡터 구하기
- 특성방정식으로 고윳값을 알아내면 고윳값분해 식의 해를 찾아 고유벡터 $v$를 구할 수 있다

$$
\begin{align}
(A - \lambda I)v = 0
\tag{3.3.36}
\end{align}
$$

- 고윳값이 중복되었다고 고유벡터도 중복되는것은 아니다

- 중복된 고윳값 $\lambda$에 대해 서로 다른 고유벡터 $v_1$, $v_2$가 존재하면 이 두 벡터의 선형조합$
\begin{align}
c_1v_1 + c_2v_2
\tag{3.3.48}
\end{align}
$

도 고윳값 $\lambda$에 대한 고유벡터이다

### 넘파이로 고유분해하기
- eig()
- 결과로 나오는 고유벡터의 행렬은 열을 고유벡터로 가진다

```python
A= np.array([[1,-2],[2,-3]])
w1, V1= np.linalg.eig(A)
print(w1)
print(V1)
```

    [-0.99999998 -1.00000002]
    [[0.70710678 0.70710678]
     [0.70710678 0.70710678]]

```python
B=np.array([[2,1],[1,2]])
w2, V2= np.linalg.eig(B)
print(w2)
print(V2)
```

    [3. 1.]
    [[ 0.70710678 -0.70710678]
     [ 0.70710678  0.70710678]]

### 대각화 :N차원의 정방행렬인 경우
**고유벡터행렬** $V$은 고유벡터를 열벡터로 옆으로 쌓아서 만든 행렬이다.

$$
\begin{align}
V = \left[ v_1 \cdots v_N \right]
\tag{3.3.50}
\end{align}
$$

**고윳값행렬** $\Lambda$은 고윳값을 대각성분으로 가지는 대각행렬이다.

$$
\begin{align}
\Lambda =
\begin{bmatrix}
\lambda_{1} & 0 & \cdots & 0 \\
0 & \lambda_{2} & \cdots & 0 \\
\vdots & \vdots & \ddots & \vdots \\
0 & 0 & \cdots & \lambda_{N} \\
\end{bmatrix}
\tag{3.3.52}
\end{align}
$$

$$
\begin{align}
\Lambda \in \mathbf{R}^{N \times N}
\tag{3.3.53}
\end{align}
$$

따라서

$$
\begin{align}
AV = V\Lambda
\tag{3.3.55}
\end{align}
$$

#### 더 나아가기
만약 **고유벡터행렬 $V$의 역행렬이 존재한다면** 행렬을 다음처럼 고유벡터행렬과 고윳값행렬의 곱으로 표현할 수 있다. 이를 행렬의 **대각화(diagonalization)**라고 한다.

$$
\begin{align}
A = V \Lambda V^{-1}
\tag{3.3.56}
\end{align}
$$

이때 A의 역행렬은
$$
\begin{align}
A^{-1} = V \Lambda^{-1} V^{-1}
\end{align}
$$

- 고유벡터행렬 $V$의 역행렬이 존재한다는 것은 고유벡터행렬의 각각의 열행렬이 서로 독립이므로 풀랭크라는 의미이다
- 행렬이 대각화 가능하려면 고유벡터가 선형독립이어야 한다
- 대각행렬의 역행렬은 각 대각성분의 역수로 이루어진 대각행렬이므로 0인 고유값만 없으면 항상 역행렬이 존재한다

```python
B= np.array([[2,3],[2,1]])
B

```

    array([[2, 3],
           [2, 1]])

```python
w,V=np.linalg.eig(B)
print(w)
print(V)
```

    [ 4. -1.]
    [[ 0.83205029 -0.70710678]
     [ 0.5547002   0.70710678]]

```python
V_inv=np.linalg.inv(V)
V_inv
```

    array([[ 0.72111026,  0.72111026],
           [-0.56568542,  0.84852814]])

```python
#대각화 중간에 들어가는 람다행렬
np.diag(w)
```

    array([[ 4.,  0.],
           [ 0., -1.]])

```python
#대각화 하기
V@np.diag(w)@V_inv
#결과가 B와 동일하다
```

    array([[2., 3.],
           [2., 1.]])

```python
C= np.array([[1,1],[0,1]])
C
```

    array([[1, 1],
           [0, 1]])

```python
w,V=np.linalg.eig(C)
print(w)
print(V)
```

    [1. 1.]
    [[ 1.00000000e+00 -1.00000000e+00]
     [ 0.00000000e+00  2.22044605e-16]]

```python
np.linalg.matrix_rank(V)
#랭크가 1이므로 역행렬이 존재하지 않는다(풀랭크가 아니다)
```

    1

```python
#A 역행렬 구하기
A = np.array([[2, 3], [2, 1]])

w, V = np.linalg.eig(A)
V_inv= np.linalg.inv(V)
```

```python
#중간 대각행렬의 인버스
np.linalg.inv(np.diag(w))
```

    array([[ 0.25,  0.  ],
           [-0.  , -1.  ]])

```python
np.diag(1/w)
```

    array([[ 0.25,  0.  ],
           [ 0.  , -1.  ]])

```python
V@np.diag(1/w)@V_inv
```

    array([[-0.25,  0.75],
           [ 0.5 , -0.5 ]])

```python
np.linalg.inv(A)
```

    array([[-0.25,  0.75],
           [ 0.5 , -0.5 ]])

## 대칭행렬

### 대칭행렬을 랭크1 행렬의 합으로 분해하기

$N$차원 대칭행렬 $A$는 다음처럼 $N$개의 랭크-1 행렬 $A_i = v_i v_i^T$ 의 합으로 표시할 수 있다.

$$
\begin{align}
\begin{aligned}
A
&= V\Lambda V^T \\
&=
\begin{bmatrix}
v_1 & v_2 & \cdots & v_N
\end{bmatrix}
\begin{bmatrix}
\lambda_{1} & 0 & \cdots & 0 \\
0 & \lambda_{2} & \cdots & 0 \\
\vdots & \vdots & \ddots & \vdots \\
0 & 0 & \cdots & \lambda_{N} \\
\end{bmatrix}
\begin{bmatrix}
v_1^T \\ v_2^T \\ \vdots \\ v_N^T
\end{bmatrix} \\
&=
\begin{bmatrix}
\lambda_{1}v_1 & \lambda_{2}v_2 & \cdots & \lambda_{N}v_N
\end{bmatrix}
\begin{bmatrix}
v_1^T \\ v_2^T \\ \vdots \\ v_N^T
\end{bmatrix} \\
\end{aligned}
\tag{3.3.71}
\end{align}
$$

따라서 $N$차원 대칭행렬 $A$는

$$
\begin{align}
A = \sum_{i=1}^{N} {\lambda_i} v_i v_i^T  = \sum_{i=1}^{N} {\lambda_i} A_i = \lambda_1 A_1 + \cdots + \lambda_N A_N
\tag{3.3.72}
\end{align}
$$

```python
A = np.array([[60., 30., 20.],
              [30., 20., 15.],
              [20., 15., 12.]])

w, V = np.linalg.eig(A)

# v1 = V[:, 0:1]
# v2 = V[:, 1:2]
# v3 = V[:, 2:3]
# A1 = v1 @ v1.T
# A2 = v2 @ v2.T
# A3 = v3 @ v3.T
```

```python
w
```

    array([84.49913563,  7.33962395,  0.16124042])

```python
w1, w2, w3 = w
```

```python
V
```

    array([[ 0.82704493,  0.54744843,  0.12765933],
           [ 0.4598639 , -0.52829024, -0.71374689],
           [ 0.32329844, -0.64900666,  0.68867153]])

```python
v1 = V[:, 0:1]
v1
```

    array([[0.82704493],
           [0.4598639 ],
           [0.32329844]])

```python
v2 = V[:, 1:2]
v2
```

    array([[ 0.54744843],
           [-0.52829024],
           [-0.64900666]])

```python
v3 = V[:, 2:3]
v3
```

    array([[ 0.12765933],
           [-0.71374689],
           [ 0.68867153]])

```python
A1=w1* v1@v1.T
A1
```

    array([[57.79768857, 32.13739648, 22.59357583],
           [32.13739648, 17.8694387 , 12.56276371],
           [22.59357583, 12.56276371,  8.83200836]])

```python
A2=w2* v2@v2.T
A2
```

    array([[ 2.19968372, -2.12270483, -2.60775134],
           [-2.12270483,  2.04841985,  2.51649195],
           [-2.60775134,  2.51649195,  3.09152039]])

```python
A3=w3* v3@v3.T
A3
```

    array([[ 0.00262772, -0.01469165,  0.01417551],
           [-0.01469165,  0.08214145, -0.07925566],
           [ 0.01417551, -0.07925566,  0.07647125]])

```python
A1+A2+A3
```

    array([[60., 30., 20.],
           [30., 20., 15.],
           [20., 15., 12.]])

**만약 0인 고윳값이 없다면** 역행렬도 다음처럼  $N$개의 랭크-1 행렬 $A_i = v_i v_i^T$ 의 합으로 표시할 수 있다.

$$
\begin{align}
A^{-1} =
V \Lambda^{-1} V^T = \sum_{i=1}^{N} \dfrac{1}{\lambda_i} v_i v_i^T
= \dfrac{1}{\lambda_1} A_1 + \cdots + \dfrac{1}{\lambda_N} A_N
\tag{3.3.74}
\end{align}
$$

- 대칭행렬의 역행렬은 중간식이 1/w로 이루어진 행렬이다
- np.diag(1/w)

```python
np.linalg.inv(A)
```

    array([[ 0.15, -0.6 ,  0.5 ],
           [-0.6 ,  3.2 , -3.  ],
           [ 0.5 , -3.  ,  3.  ]])

```python
A1_inv=1/w1* v1@v1.T
A1_inv
```

    array([[0.0080948 , 0.00450097, 0.00316432],
           [0.00450097, 0.00250269, 0.00175947],
           [0.00316432, 0.00175947, 0.00123696]])

```python
A2_inv=1/w2* v2@v2.T
A2_inv
```

    array([[ 0.04083313, -0.03940415, -0.04840816],
           [-0.03940415,  0.03802519,  0.04671409],
           [-0.04840816,  0.04671409,  0.05738845]])

```python
A3_inv=1/w3* v3@v3.T
A3_inv
```

    array([[ 0.10107208, -0.56509682,  0.54524384],
           [-0.56509682,  3.15947213, -3.04847356],
           [ 0.54524384, -3.04847356,  2.94137459]])

```python
A1_inv+A2_inv+A3_inv
```

    array([[ 0.15, -0.6 ,  0.5 ],
           [-0.6 ,  3.2 , -3.  ],
           [ 0.5 , -3.  ,  3.  ]])

### 대칭행렬의 고윳값 부호
> **[정리] 대칭행렬이 양의 정부호(positive definite)이면 고윳값은 모두 양수다. 역도 성립한다.**

> **[정리] 대칭행렬이 양의 준정부호(positive semidefinite)이면 고윳값은 모두 0이거나 양수다. 역도 성립한다.**

- 추가)대칭행렬: PD<=> 고유값은 모두 양수

## 분산행렬

- 임의의 실수 행렬  𝑋 에 대해   $X^TX$ 인 정방행렬을 분산행렬(scatter matrix)이라고 한다

> **[정리] 분산행렬은 양의 준정부호(positive semidefinite)이고 고윳값은 0보다 같거나 크다.**

연습 문제 3.3.9

(1) 붓꽃(Iris) 특징데이터 행렬  𝑋 의 분산행렬을 구하고 이 분산행렬의 고윳값들을 구하라.

(2) 보스턴 집값(Boston House Price) 특징데이터 행렬  𝑋 의 분산행렬을 구하고 이 분산행렬의 고윳값들을 구하라.

```python
#(1)
from sklearn.datasets import load_iris
iris=load_iris()
X=iris.data

```

```python
X[:3]
```

    array([[5.1, 3.5, 1.4, 0.2],
           [4.9, 3. , 1.4, 0.2],
           [4.7, 3.2, 1.3, 0.2]])

```python
iris_sca= X.T@X
iris_sca
```

    array([[5223.85, 2673.43, 3483.76, 1128.14],
           [2673.43, 1430.4 , 1674.3 ,  531.89],
           [3483.76, 1674.3 , 2582.71,  869.11],
           [1128.14,  531.89,  869.11,  302.33]])

```python
w,V= np.linalg.eig(iris_sca)
print(w)
```

    [9.20830507e+03 3.15454317e+02 1.19780429e+01 3.55257020e+00]

```python
#2
from sklearn.datasets import load_boston
boston= load_boston()
X= boston.data

boston_sca= X.T@X
boston_sca[:3]
```

    array([[4.39703436e+04, 4.68702660e+02, 3.24790952e+04, 6.48084600e+01,
            1.22612317e+03, 1.08219511e+04, 1.68514980e+05, 3.46627456e+03,
            4.11186651e+04, 1.17307321e+06, 3.64715518e+04, 4.99455291e+05,
            3.72684187e+04],
           [4.68702660e+02, 3.40029000e+05, 2.09030900e+04, 2.70000000e+02,
            2.48444175e+03, 3.87184670e+04, 2.05485400e+05, 3.82992940e+04,
            2.29180000e+04, 1.72295450e+06, 9.61321500e+04, 2.23960499e+06,
            3.80196800e+04],
           [3.24790952e+04, 2.09030900e+04, 8.65256299e+04, 4.45170000e+02,
            3.43239536e+03, 3.44618165e+04, 4.49313490e+05, 1.62206733e+04,
            7.17656500e+04, 2.72134904e+06, 1.06875320e+05, 1.89702529e+06,
            8.62407050e+04]])

```python
w,V= np.linalg.eig(boston_sca)
print(w)
```

    [1.58386796e+08 1.18747372e+07 4.17002244e+05 1.61644573e+05
     2.52697480e+04 1.47629635e+04 8.18396001e+03 6.07326738e+03
     4.23577535e+03 6.06399504e+02 3.27412564e+02 3.04157837e+01
     2.19326965e+00]

### 분산행렬의 역행렬
> **[정리] 행렬 $X$가 풀랭크이면 이 행렬의 분산행렬 $X^TX$의 역행렬이 존재한다.**

- 양의 정부호인 대칭행렬은 항상 역행렬이 존재한다
- 하지만 역행렬이 존재하는 대칭행렬은 항상 양의 정부호라고 할 수 없다
- 음의 정부호인 대칭행렬도 역행렬이 존재하는 경우가 있기 때문이다

$N$차원 정방행렬 $A$에 대해

1. 행렬 $A$는 $N$개의 고윳값-고유벡터를 가진다(복소수인 경우와 중복인 경우를 포함).
2. 행렬의 **대각합은 모든 고윳값의 합**과 같다.
3. 행렬의 **행렬식은 모든 고윳값의 곱**과 같다.
4. 행렬 $A$가 **대칭행렬**이면 $N$개의 **실수 고윳값**을 가지며 고유벡터들이 서로 **직교(orthogonal)**이다.
5. 행렬 $A$가 **대칭행렬**이고 고윳값이 모두 **양수**이면 **양의 정부호(positive-definite)이고 역행렬이 존재한다**. 역도 성립한다.
6. 행렬 $A$가 어떤 행렬 $X$의 **분산행렬** $X^TX$이면 **0 또는 양의 고윳값을 가진다**.
7. 행렬 $X$가 **풀랭크이면 분산행렬 $X^TX$은 역행렬이 존재**한다.

## 특잇값 분해
- 정방행렬은 고유분해로 고윳값과 고유벡터를 찾을 수 있었다
- 정방행렬이 아닌 행렬은 고유분해가 불가능
- 하지만 고유분해와 비슷한 특잇값 분해를 할 수 있다.

$N \times M$ 크기의 행렬 $A$에 대해
$$
\begin{align}
A = U\Sigma V^T
\tag{3.4.1}
\end{align}
$$

여기에서 $U, \Sigma, V$는 다음 조건을 만족해야 한다.

* 대각성분이 양수인 대각행렬이어야 한다. 큰 수부터 작은 수 순서로 배열한다.

$$
\begin{align}
\Sigma \in \mathbf{R}^{N \times M}
\tag{3.4.2}
\end{align}
$$

* $U$는 $N$차원 정방행렬로 모든 열벡터가 단위벡터이고 서로 직교해야 한다.

$$
\begin{align}
U \in \mathbf{R}^{N \times N}
\tag{3.4.3}
\end{align}
$$

* $V$는 $M$차원 정방행렬로 모든 열벡터가 단위벡터이고 서로 직교해야 한다.

$$
\begin{align}
V \in \mathbf{R}^{M \times M}
\tag{3.4.4}
\end{align}
$$

위 조건을 만족하는 행렬 $\Sigma$의 대각성분들을 **특잇값(singular value)**, 행렬 $U$의 열벡터들을 **왼쪽 특이벡터**(left singular vector), 행렬 $V$의 행벡터들을 **오른쪽 특이벡터**(right singular vector)라고 부른다.

> **[정리] 특이분해는 모든 행렬에 대해 가능하다. 즉 어떤 행렬이 주어지더라도 위와 같이 특이분해할 수 있다.**

### 1차원 근사
2차원 평면 위에 3개의 2차원 벡터 $a_1, a_2, a_3$가 있다. 원점을 지나면서 모든 점들과 가능한 한 가까이 있는 직선을 만들고 싶다면 직선의 방향을 어떻게 해야 할까? 직선의 방향을 나타내는 단위 벡터를 $w$라고 하자.

```python

```