# Particle Filtering for Sequential Bayesian Inference

## Introduction
In this notebook, we produce an implementation of the particle filtering methods discussed in the accompanying article. In particular, we introduce the *bearings-only tracking* problem, a nonlinear non-Gaussian sequential inference problem. Next, we implement sequential importance sampling (SIS) to solve bearings-only tracking and observe the method's limitations. Finally, we achieve a respectable solution to our problem using the bootstrap filter.

## The Problem
### Statement
The bearings-only tracking problem is as follows. Suppose a plane begins at $x_0 = (0.01, 0.95)$ with a velocity $v_0=(0.02, -0.013)$ and for $50$ time steps moves according to the following dynamical system.
\begin{align}
v_{t+1} &\sim \mathcal{N}(v_t, \Sigma_v)\\
x_{t+1} & = x_t + v_t
\end{align}

where $\Sigma_v = 10^{-6}I_2$. At each time $t$, we receive a noisy measurement of the plane's bearing, that is

\begin{equation} y_t = \mathrm{angle}(x_t) + w_t \end{equation}

where $w_t \sim \mathcal{N}(0, \sigma=0.005)$.

Our goal is to infer $p(x_{0:50}, v_{0:50}| y_{1:50})$.

### Code
First, we do our imports and set a random seed.

```python
import numpy as np
import scipy.stats as sp_stats
import scipy.special as sp_special
import matplotlib.pyplot as plt
import matplotlib.patheffects as pe
import math

np.random.seed(3)
```

Next, we parametrize our prior.

```python
# number of time steps
num_samples = 50
# number of particles
num_particles = 100
# Parameters of the prior
x0 = [0.01, 0.95]
v0 = [0.02, -0.013]
vx_var, vy_var = (1.0/1e6, 1.0/1e6)
v_cov = [[vx_var, 0], [0, vy_var]]
measurement_noise = 0.005
```

The `BearingsData` class performs the following functions
1. Simulates a ground-truth state $(x_{0:50}, v_{0:50})$ and accompanying measurements $y_{1:50}$.
2. Provides plotting functions.
3. Gives us a `__getitem__` function, which allows us to sequentially retrieve datapoints.

```python
class BearingsData(object):
    def __init__(self, num_samples):
        # Simulates velocities
        v_matrix = np.zeros((num_samples, 2))
        v_matrix[0, :] = np.random.multivariate_normal(v0, v_cov)
        for i in range(1, num_samples):
            v_matrix[i, :] = np.random.multivariate_normal(v_matrix[i-1, :], v_cov)
        self.v = v_matrix

        # Calculates positions
        x = np.cumsum(v_matrix, axis=0) + x0 - v_matrix # subtract v_matrix because x_t depends on v_{t-1}
        self.x = x

        # Calculates bearings
        noise_vector = np.random.multivariate_normal(np.zeros(num_samples),
                                                    (measurement_noise**2) * np.identity(num_samples))
        y = np.arctan(x[:, 1]/x[:, 0]) + noise_vector
        self.y = y

    """Plots trajectory and bearings"""
    def plot(self):
        plt.scatter(self.x[:, 0], self.x[:, 1])

        y = np.arctan(self.x[:, 1]/self.x[:, 0])

        for yt in self.y:
            dx = math.cos(yt) * .5
            dy = math.sin(yt) * .5
            plt.plot([0., dx], [0., dy], color="red", alpha=0.3)

        plt.show()

    """Plots inferences overlayed on ground-truth trajectory and measurements"""
    def plot_inferences(self, inferences):
        plt.plot(self.x[:, 0], self.x[:, 1], color="black", lw=2,
                path_effects=[pe.Stroke(linewidth=5, foreground='white'), pe.Normal()])

        for yt in self.y:
            dx = math.cos(yt) * .5
            dy = math.sin(yt) * .5
            plt.plot([0., dx], [0., dy], color="red", alpha=0.3)

        for x in inferences:
            plt.scatter(x[:, 0], x[:, 1])
        plt.show()

    """Retrieves position, velocity, and measurement at time t"""
    def __getitem__(self, idx):
        if idx < self.x.shape[0]:
            xt = self.x[idx, :]
            vt = self.v[idx, :]
            yt = self.y[idx]
            return xt, vt, yt
        else:
            raise IndexError('high enough')
```

Here, we simulate one run of the data-generating process.

```python
data = BearingsData(num_samples)
data.plot()
```

## SIS

In this section, we implement SIS in order to infer the hidden-state given the measurements of the data we generated in the last section.
Throughout our calculations, we use log probabilities for numerical stability.

The following defines the Markov transition kernel from state $(x_t, v_t)$ to $(x_{t+1}, v_{t+1})$. Note that the kernel only takes velocities as input: we assume that the deterministic relation $x_{t+1} = x_t + v_t$ is satisfied.

```python
"""Markov transition kernel"""
def transition_kernel(v1, v2):
    v2_dist = sp_stats.multivariate_normal(mean=v1, cov=v_cov)
    return v2_dist.logpdf(v2)
```

Next, we define the marginal distribution $p(y_t| x_t, v_t)$. Note that $p(y_t|x_t, v_t)$ does not depend on $v_t$.

```python
"""Probability of measurement y given state xt, vt"""
def measurement_prob(yt, xt):
    error = np.arctan(xt[1]/xt[0]) - yt
    return sp_stats.norm.logpdf(error, scale=measurement_noise)
```

We represent our particles as a dictionary with keys {"locations", "velocities", "weight", "prior_prob"}. We store all the particles in a list.

The following function computes a particle filter step, outputting particles with unnormalized weights. It takes as input particles $\{(x_{0:t}^{(i)}, v_{0:t}^{(i)})\}$, draws each $(x_{t+1}^{(i)}, v_{t+1}^{(i)})$, and updates weights and prior probabilities accordingly. Since we reuse this function when implementing the bootstrap filter, we add the `selection` argument that modifies how the weights are updated.

```python
def particle_filter_step(particles, yt, selection=False):
    new_particles = []
    for i in range(len(particles)):

        particle = particles[i]
        prev_x = np.squeeze(particle["locations"][-1:, :])
        prev_v = np.squeeze(particle["velocities"][-1:, :])

        vt = np.random.multivariate_normal(prev_v, v_cov)
        xt = prev_x + prev_v
        new_particle = {}
        new_particle["prior_prob"] = particle["prior_prob"] + transition_kernel(prev_v, vt)
        if selection:
            new_particle["weight"] = measurement_prob(yt, xt)
        else:
            new_particle["weight"] = particle["weight"] + measurement_prob(yt, xt)
        new_particle["locations"] = np.append(particle["locations"], np.expand_dims(xt, axis=0), axis=0)
        new_particle["velocities"] = np.append(particle["velocities"], np.expand_dims(vt, axis=0), axis=0)

        new_particles.append(new_particle)

    return new_particles
```

The following function normalizes particle weights. Note that we smooth the probability distribution induced by the weights by a factor of $10^{-10}$. If we do not do so, our algorithm attempts to compute $\log(0)$ .

```python
def normalize_weights(particles):
    log_weights = np.array([particle["weight"] for particle in particles])

    # For numerical stability, we smooth the output of softmax by 1e-10
    smoothed_log_weights = np.log(sp_special.softmax(log_weights) + 1e-10)
    normalized_log_weights = np.log(sp_special.softmax(smoothed_log_weights))

    for i in range(len(particles)):
        particles[i]['weight'] = normalized_log_weights[i]

    return particles
```

The following step combines the functions we have defined so far in order to perform the SIS algorithm.

```python
particles = [{
    "locations": np.expand_dims(x0, axis=0),
    "velocities": np.expand_dims(v0, axis=0),
    "weight": np.log(1.0/num_particles),
    "prior_prob": 0
} for _ in range(num_particles)]

for _, _, yt in data:
    particles = particle_filter_step(particles, yt)
    particles = normalize_weights(particles)
```

Let's visualize first all 100 particle and then the highest likelihood particle. The ground-truth trajectory is the black line and each colored trajectory is a particle.

```python
data.plot_inferences([particle['locations'] for particle in particles])
```

```python
best_idx = np.argmax([particle['weight'] * particle['prior_prob'] for particle in particles])
print('particle weight: ', np.exp(particles[best_idx]['weight']))
data.plot_inferences([particles[best_idx]['locations']])
```

We can see that SIS correctly identifies a particle that is quite close to the ground-truth trajectory. However, we can also see the two limitations of SIS.
1. Since we are sampling trajectories from the prior, we end up with many particles that are obviously inconsistent with our measurements. These are a waste of computation.
2. All but one of the importance weights converge to 0. This means that we effectively only have drawn one sample from $p(x_{0:t}|y_{1:t})$, meaning we only have a very low-resolution picture of the distribution.
In the next section, we see that the bootstrap filter addresses these issues.

## The Boostrap Filter
The following function defines the selection step from the boostrap filter algorithm.

```python
def selection(particles):
    new_particles = []
    probs = [np.exp(particle["weight"]) for particle in particles]

    for _ in range(len(particles)):
        idx = np.argmax(np.random.multinomial(1, probs))
        new_particles.append(particles[idx])
    return new_particles
```

This following code snippet performs the boostrap filter algorithm on our data. We have made one modification from the bootstrap filter algorithm described in the accompanying article: instead of applying the selection step at every time step, we only apply it every 10 times steps. The rationale for this is that in our particular case, doing selection at every time step still results in all but one of the importance weights becoming $0$. However, doing selection only every $\Delta t$ steps scales the variance of the log-importance weights at selection time by $1/{\Delta t}$. Setting $t=10$ is sufficient to preserve the diversity of particles.

```python
better_particles = [{
    "locations": np.expand_dims(x0, axis=0),
    "velocities": np.expand_dims(v0, axis=0),
    "weight": np.log(1.0/num_particles),
    "prior_prob": 0
} for _ in range(num_particles)]

for t, (_, _, yt) in enumerate(data):
    if t%10==0:
        better_particles = selection(better_particles)
        better_particles = particle_filter_step(better_particles, yt, selection=True)
        better_particles = normalize_weights(better_particles)
    else:
        better_particles = particle_filter_step(better_particles, yt, selection=False)
        better_particles = normalize_weights(better_particles)
```

```python
data.plot_inferences([particle['locations'] for particle in better_particles])
```

As we can see, the bootstrap filter resolves the problems associate with SIS. Namely,
1. All particles have equal weight.
2. Since all our particles are informative, we have rich information about $p(x_{0:t}|y_{1:t})$.


# The two-layer grey radiation model (aka the leaky greenhouse)

### I think this notebook needs updating to account for the reversal of the vertical pressure axis that happened awhile back in `climlab`.

- Surface temperature is $T_s$
- Atm. temperatures are $T_1, T_2$ where $T_1$ is closest to the surface.
- absorptivity of atm layers is $\epsilon_1, \epsilon_2$
- Surface emission is $\sigma T_s^4$
- Atm emission is $\epsilon_1 \sigma T_1^4, \epsilon_2 \sigma T_2^4$ (up and down)
- Abs = emissivity for atm layers
- Transmissivity for atm layers is $\tau_1, \tau_2$ where $\tau_i = (1-\epsilon_i)$

### Emission
$$ E_s = \sigma T_s^4 $$

$$ E_1 = \epsilon_1 \sigma T_1^4 $$

$$ E_2 = \epsilon_2 \sigma T_2^4 $$

### Incident radiation

$$ F_s = \tau_1 E_2 + E_1 $$

$$ F_1 = E_s + E_2 $$

$$ F_2 = \tau_1 E_s + E_1 $$

### Net radiation
(absorptivity) * incident - emission

$$ R_s = F_s - E_s $$

$$ R_1 = \epsilon_1 F_1 - 2 E_1 $$

$$ R_2 = \epsilon_2 F_2 - 2 E_2 $$

### OLR

$$ OLR = \tau_2 F_2 + E_2 $$

$$ = \tau_1 \tau_2 E_s + \tau_2 E_1 + E_2 $$
$$ = \tau_1 \tau_2 \sigma T_s^4 + \tau_2 \epsilon_1 \sigma T_1^4 + \epsilon_2 \sigma T_2^4 $$

### Net radiation in terms of emissions

$$ R_s = \tau_1 E_2 + E_1 - E_s $$

$$ R_1 = \epsilon_1 (E_s + E_2) - 2 E_1 $$

$$ R_2 = \epsilon_2 (\tau_1 E_s + E_1) - 2 E_2 $$

### Net radiation in terms of temperatures

$$ R_s = \tau_1 \epsilon_2 \sigma T_2^4 + \epsilon_1 \sigma T_1^4 - \sigma T_s^4 $$

$$ R_1 = \epsilon_1 (\sigma T_s^4 + \epsilon_2 \sigma T_2^4) - 2 \epsilon_1 \sigma T_1^4 $$

$$ R_2 = \epsilon_2 (\tau_1 \sigma T_s^4 + \epsilon_1 \sigma T_1^4) - 2 \epsilon_2 \sigma T_2^4 $$

### Net radiation in terms of temperatures and absorptivities

$$ R_s = (1-\epsilon_1) \epsilon_2 \sigma T_2^4 + \epsilon_1 \sigma T_1^4 - \sigma T_s^4 $$

$$ R_1 = \epsilon_1 (\sigma T_s^4 + \epsilon_2 \sigma T_2^4) - 2 \epsilon_1 \sigma T_1^4 $$

$$ R_2 = \epsilon_2 ((1-\epsilon_1) \sigma T_s^4 + \epsilon_1 \sigma T_1^4) - 2 \epsilon_2 \sigma T_2^4 $$

## Solve for radiative equilibrium
Need to add the solar energy source. We assume atm is transparent, solar is all absorbed at the surface.

$$ R_1 = R_2 = 0$$

$$ R_s = - (1-\alpha) Q $$

Introduce useful notation shorthand:

$$ (1-\alpha) Q = \sigma T_e^4 $$

This gives a 3x3 system which is **linear in $T^4$** (divide through by $\sigma$)

$$ - T_s^4 + \epsilon_1 T_1^4 + (1-\epsilon_1) \epsilon_2  T_2^4 + T_e^4 = 0 $$

$$ \epsilon_1 T_s^4 - 2 \epsilon_1 T_1^4 + \epsilon_1 \epsilon_2  T_2^4 = 0$$

$$ \epsilon_2 (1-\epsilon_1) T_s^4 + \epsilon_1 \epsilon_2 T_1^4 - 2 \epsilon_2 T_2^4 = 0$$

Here we use the `sympy` module to solve the algebraic system symbolically.

```python
import sympy
sympy.init_printing()
T_s, T_1, T_2, T_e, e_1, e_2 = sympy.symbols('T_s, T_1, T_2, T_e, e_1, e_2', positive=True )
system = [-T_s**4 + e_1*T_1**4 + e_2*(1-e_1)*T_2**4 + T_e**4,
              e_1*T_s**4 - 2*e_1*T_1**4 + e_1*e_2*T_2**4,
              e_2*(1-e_1)*T_s**4 + e_1*e_2*T_1**4 - 2*e_2*T_2**4]
out1 = sympy.solve( system, [T_s**4, T_1**4, T_2**4])
out1
```

```python
quarter = sympy.Rational(1,4)
out2 = {}
for var4, formula in out1.items():
    var = (var4)**quarter
    out2[var] = sympy.simplify(formula**quarter)
out2
```

```python
#  The special case of equal absorptivities
e = sympy.symbols('e')
out3 = {}
for var4, formula in out1.items():
    var = (var4)**quarter
    simple_formula = sympy.cancel(formula.subs([(e_2, e),(e_1, e)]))
    out3[var] = sympy.simplify( simple_formula**quarter )
out3
```

The solution is

\begin{align}
T_s^4 &= T_e^4 \frac{4 - \epsilon_1 \epsilon_2}{4 + \epsilon_1 \epsilon_2 - 2 \epsilon_1 - 2 \epsilon_2}  \\
T_1^4 &= T_e^4 \frac{2 -\epsilon_1 \epsilon_2 + \epsilon_2}{4 + \epsilon_1 \epsilon_2 - 2 \epsilon_1 - 2 \epsilon_2} \\
T_2^4 &= T_e^4 \frac{ 1}{2 - \epsilon_2}
\end{align}

In the special case $\epsilon_1 = \epsilon_2$ this reduces to

\begin{align}
T_s^4 &= T_e^4 \frac{2+\epsilon}{2-\epsilon}  \\
T_1^4 &= T_e^4 \frac{1+\epsilon}{2-\epsilon} \\
T_2^4 &= T_e^4 \frac{ 1}{2 - \epsilon}
\end{align}

```python
out2[T_s].subs([(T_e, 255), (e_1, 0.4), (e_2, 0.4)])
```

```python
for var, formula in out2.items():
    print(formula.subs([(T_e, 255), (e_1, 0.4), (e_2, 0.4)]))
```

    246.627893584128
    282.203889523582
    226.730624779963

```python
# Coding up the analytical solutions for radiative equilibrium
#  These use the analytical results returned by sympy and wrap them in callable functions

def Ts(Te, e1, e2):
    #return Te*((4-e1*e2)/(4+e1*e2-2*(e1+e2)))**0.25
    return out2[T_s].subs([(T_e, Te), (e_1, e1), (e_2, e2)])
def T1(Te, e1, e2):
    #return Te*((2+e2-e1*e2)/(4+e1*e2-2*(e1+e2)))**0.25
    return out2[T_1].subs([(T_e, Te), (e_1, e1), (e_2, e2)])
def T2(Te, e1, e2):
    #return Te*(1/(2-e2))**0.25
    return out2[T_2].subs([(T_e, Te), (e_1, e1), (e_2, e2)])
```

```python
%matplotlib inline
import numpy as np
from climlab import constants as const
import climlab
```

```python
mycolumn = climlab.GreyRadiationModel( num_lev=2 )
```

```python
print(mycolumn)
```

    climlab Process of type <class 'climlab.model.column.GreyRadiationModel'>.
    State variables and domain shapes:
      Tatm: (2,)
      Ts: (1,)
    The subprocess tree:
    top: <class 'climlab.model.column.GreyRadiationModel'>
       LW: <class 'climlab.radiation.greygas.GreyGas'>
       SW: <class 'climlab.radiation.greygas.GreyGasSW'>
       insolation: <class 'climlab.radiation.insolation.FixedInsolation'>

```python
mycolumn.integrate_years(10.)
```

    Integrating for 3652 steps, 3652.422 days, or 10.0 years.
    Total elapsed time is 9.99884460229 years.

```python
print(mycolumn.Ts)
print(mycolumn.Tatm)
```

    [ 287.84605967]
    [ 229.43643402  252.95019305]

```python
(e1, e2)= mycolumn.subprocess['LW'].absorptivity
print(e1, e2)
```

    0.477374247427 0.477374247427

```python
ASR = (1-mycolumn.param['albedo_sfc'])*mycolumn.param['Q']
Te = (ASR/const.sigma)**0.25
print(Te)
```

    254.865280431

## Check numerical versus analytical results

Use a tolerance value to test if the results are the same.

```python
tol = 0.01

def test_2level(col):
    (e1, e2)= col.subprocess['LW'].absorptivity
    ASR = (1-col.param['albedo_sfc'])*col.param['Q']
    Te = (ASR/const.sigma)**0.25
    print('Surface:')
    num = col.Ts
    anal = Ts(Te,e1,e2)
    print('  Numerical: %.2f   Analytical: %.2f    Same:' %(num, anal) , abs(num - anal)<tol)
    print('Level 1')
    num = col.Tatm[0]
    anal = T1(Te,e1,e2)
    print('  Numerical: %.2f   Analytical: %.2f    Same:' %(num, anal) , abs(num - anal)<tol)
    print('Level 2')
    num = col.Tatm[1]
    anal = T2(Te,e1,e2)
    print('  Numerical: %.2f   Analytical: %.2f    Same:' %(num, anal) , abs(num - anal)<tol)
```

```python
test_2level(mycolumn)
```

    Surface:
      Numerical: 287.85   Analytical: 287.85    Same: [ True]
    Level 1
      Numerical: 229.44   Analytical: 252.95    Same: False
    Level 2
      Numerical: 252.95   Analytical: 229.44    Same: False

```python
e1 = 0.3
e2 = 0.4
mycolumn.subprocess['LW'].absorptivity = np.array([e1,e2])
mycolumn.integrate_years(10.)
```

    Integrating for 3652 steps, 3652.422 days, or 10.0 years.
    Total elapsed time is 19.9976892046 years.

```python
test_2level(mycolumn)
```

    Surface:
      Numerical: 278.53   Analytical: 278.53    Same: [ True]
    Level 1
      Numerical: 223.20   Analytical: 243.87    Same: False
    Level 2
      Numerical: 241.15   Analytical: 226.61    Same: False

```python
e1 = 0.6
e2 = 0.6
mycolumn.subprocess['LW'].absorptivity = np.array([e1,e2])
mycolumn.integrate_years(10.)
test_2level(mycolumn)
```

    Integrating for 3652 steps, 3652.422 days, or 10.0 years.
    Total elapsed time is 29.9965338069 years.
    Surface:
      Numerical: 297.52   Analytical: 297.52    Same: [ True]
    Level 1
      Numerical: 234.30   Analytical: 263.52    Same: False
    Level 2
      Numerical: 263.52   Analytical: 234.30    Same: False

```python
col1 = climlab.GreyRadiationModel(num_lev=2, abs_coeff=1.9E-4)
(e1, e2) = col1.subprocess['LW'].absorptivity
print(e1, e2)
col1.integrate_years(10.)
test_2level(col1)
```

    0.652920962199 0.652920962199
    Integrating for 3652 steps, 3652.422 days, or 10.0 years.
    Total elapsed time is 9.99884460229 years.
    Surface:
      Numerical: 301.92   Analytical: 301.92    Same: [ True]
    Level 1
      Numerical: 236.57   Analytical: 268.24    Same: False
    Level 2
      Numerical: 268.24   Analytical: 236.57    Same: False

```python
col1 = climlab.GreyRadiationModel(num_lev=2, abs_coeff=1.9E-4)
lw = col1.subprocess['LW']
(e1, e2) = lw.absorptivity
e1 *= 1.2
lw.absorptivity = np.array([e1, e2])
col1.integrate_years(10.)
test_2level(col1)
```

    Integrating for 3652 steps, 3652.422 days, or 10.0 years.
    Total elapsed time is 9.99884460229 years.
    Surface:
      Numerical: 307.85   Analytical: 307.85    Same: [ True]
    Level 1
      Numerical: 242.68   Analytical: 272.49    Same: False
    Level 2
      Numerical: 276.56   Analytical: 236.57    Same: False

### Conclusion: The Two-level model works

## Three-layer model

Extend the analysis to three layers.
Start numbering the layers from 0 to be consistent with array indexing.

```python
T_s, T_0, T_1, T_2, T_e, epsilon_0, epsilon_1, epsilon_2, sigma = \
    sympy.symbols('T_s, T_0, T_1, T_2, T_e, epsilon_0, epsilon_1, epsilon_2, sigma', positive=True )
```

Define the transmissivities $\tau_i$ for layers $i=0, 1, \dots, N-1$

```python
tau_0 = (1-epsilon_0)
tau_1 = (1-epsilon_1)
tau_2 = (1-epsilon_2)
tau_0, tau_1, tau_2
```

Note that if the atmosphere has $N$ layers then $\epsilon_N = 0$

Define the emissions for each layer:

```python
E_s = sigma*T_s**4
E_0 = epsilon_0*sigma*T_0**4
E_1 = epsilon_1*sigma*T_1**4
E_2 = epsilon_2*sigma*T_2**4
E_s, E_0, E_1, E_2
```

Define the longwave fluxes incident on each layer, $F_{i}$

Note that if the atmosphere has $N$ layers then $F_{N}$ is the OLR (emission to space)

```python
F_s = E_0 + tau_0*E_1 + tau_0*tau_1*E_2
F_0 = E_s + E_1 + tau_1*E_2
F_1 = tau_0*E_s + E_0 + E_2
F_2 = tau_1*tau_0*E_s + tau_1*E_0 + E_1
F_3 = tau_2*tau_1*tau_0*E_s + tau_2*tau_1*E_0 + tau_2*E_1 + E_2

F_s, F_0, F_1, F_2, F_3
```

Now define the net absorbed longwave radiation (flux divergence) in each layer.

```python
R_s = F_s - E_s
R_0 = epsilon_0*F_0 - 2*E_0
R_1 = epsilon_1*F_1 - 2*E_1
R_2 = epsilon_2*F_2 - 2*E_2

R_s, R_0, R_1, R_2
```

### Solve for radiative equilibrium

Use `sympy.solve` to automatically solve the algebraic system.

We will solve for the **radiative equilibrium temperatures** in two steps:

- First solve for $T_i^4$, which is a purely linear problem.
- Then take the fourth roots to solve for the temperatures.

```python
out1 = sympy.solve([R_s + sigma*T_e**4, R_0, R_1, R_2],
            [T_s**4, T_0**4, T_1**4, T_2**4])
out1
```

```python
quarter = sympy.Rational(1,4)
out2 = {}
for var4, formula in out1.items():
    var = (var4)**quarter
    out2[var] = sympy.simplify(formula**quarter)
out2
```

Now wrap these analytical radiative equilibrium solutions in callable functions:

```python
def Ts(Te, e0, e1, e2):
    return out2[T_s].subs([(T_e, Te), (epsilon_0, e0), (epsilon_1, e1), (epsilon_2, e2)])
def T0(Te, e0, e1, e2):
    return out2[T_0].subs([(T_e, Te), (epsilon_0, e0), (epsilon_1, e1), (epsilon_2, e2)])
def T1(Te, e0, e1, e2):
    return out2[T_1].subs([(T_e, Te), (epsilon_0, e0), (epsilon_1, e1), (epsilon_2, e2)])
def T2(Te, e0, e1, e2):
    return out2[T_2].subs([(T_e, Te), (epsilon_0, e0), (epsilon_1, e1), (epsilon_2, e2)])
```

### Compare numerical and analytical solutions for radiative equilibrium

Define a function that takes a `climlab.GreyRadiationModel` object (which should be first integrated out to equilibrium), and compares the numerical solution to our analytical solution.

```python
tol = 0.01

def test_3level(col):
    (e0, e1, e2)= col.subprocess['LW'].absorptivity
    ASR = (1-col.param['albedo_sfc'])*col.param['Q']
    Te = (ASR/const.sigma)**0.25
    print('Surface:')
    num = col.Ts
    anal = Ts(Te,e0,e1,e2)
    print('  Numerical: %.2f   Analytical: %.2f    Same:' %(num, anal) , abs(num - anal)<tol)
    print('Level 0')
    num = col.Tatm[0]
    anal = T0(Te,e0,e1,e2)
    print('  Numerical: %.2f   Analytical: %.2f    Same:' %(num, anal) , abs(num - anal)<tol)
    print('Level 1')
    num = col.Tatm[1]
    anal = T1(Te,e0,e1,e2)
    print('  Numerical: %.2f   Analytical: %.2f    Same:' %(num, anal) , abs(num - anal)<tol)
    print('Level 2')
    num = col.Tatm[2]
    anal = T2(Te,e0,e1,e2)
    print('  Numerical: %.2f   Analytical: %.2f    Same:' %(num, anal) , abs(num - anal)<tol)
```

```python
col = climlab.GreyRadiationModel( num_lev=3 )
col.integrate_years(10.)
test_3level(col)
```

    Integrating for 3652 steps, 3652.422 days, or 10.0 years.
    Total elapsed time is 9.99884460229 years.
    Surface:
      Numerical: 287.85   Analytical: 287.85    Same: [ True]
    Level 0
      Numerical: 224.73   Analytical: 256.29    Same: False
    Level 1
      Numerical: 242.05   Analytical: 242.05    Same: True
    Level 2
      Numerical: 256.29   Analytical: 224.73    Same: False

```python
e0 = 0.3
e1 = 0.6
e2 = 0.2
col.subprocess['LW'].absorptivity = np.array([e0,e1,e2])
col.integrate_years(10.)
test_3level(col)
```

    Integrating for 3652 steps, 3652.422 days, or 10.0 years.
    Total elapsed time is 19.9976892046 years.
    Surface:
      Numerical: 291.71   Analytical: 291.71    Same: [ True]
    Level 0
      Numerical: 223.20   Analytical: 262.65    Same: False
    Level 1
      Numerical: 247.60   Analytical: 242.93    Same: False
    Level 2
      Numerical: 264.53   Analytical: 220.04    Same: False

### Conclusion: The three-level model works

```python
# The 3-layer solution reduces to two layer solution if we set e_2 = 0
out3 = {}
for var, formula in out2.items():
    if var is not T_2:
        out3[var] = sympy.simplify(formula.subs(epsilon_2,0))
out3
```

## Generalizing to N layers

\begin{align}
E_i &= \epsilon_i \sigma T_i^4  \\
F_i &= ... + \tau_{i+2} \tau_{i+1} E_{i+3} + \tau_{i+1} E_{i+2} + E_{i+1} + E_{i-1} + \tau_{i-1} E_{i-2} + \tau_{i-1} \tau_{i-2} E_{i-3} + ... \\
F_i &= \sum_{n=2}^{N-i} \bigg( \prod_{j=1}^{n-1} \tau_{i+j} \bigg) E_{i+n} + E_{i+1} + E_{i-1} + \sum_{n=2}^{i-1} \bigg( \prod_{j=1}^{n-1} \tau_{i-j} \bigg) E_{i-n}  \\
 &= \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{n-1} \tau_{i+j} \bigg) E_{i+n} / \tau_{i}  + \sum_{n=1}^{i-1} \bigg( \prod_{j=0}^{n-1} \tau_{i-j} \bigg) E_{i-n} / \tau_i \\
 &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{n-1} \tau_{i+j} \bigg) E_{i+n} + \sum_{n=1}^{i-1} \bigg( \prod_{j=0}^{n-1} \tau_{i-j} \bigg) E_{i-n}  \right\}
\end{align}

Now substitute $n \rightarrow -n$ in the second summation.

\begin{align}
F_i &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{n-1} \tau_{i+j} \bigg) E_{i+n} + \sum_{-n=1}^{i-1} \bigg( \prod_{j=0}^{-n-1} \tau_{i-j} \bigg) E_{i+n}  \right\}  \\
 &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{n-1} \tau_{i+j} \bigg) E_{i+n} + \sum_{n=1-i}^{-1} \bigg( \prod_{j=0}^{-n-1} \tau_{i-j} \bigg) E_{i+n}  \right\}
\end{align}

And substitude $j \rightarrow -j$ in the second product.

\begin{align}
F_i &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{n-1} \tau_{i+j} \bigg) E_{i+n} + \sum_{n=1-i}^{-1} \bigg( \prod_{-j=0}^{-n-1} \tau_{i+j} \bigg) E_{i+n}  \right\} \\
  &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{n-1} \tau_{i+j} \bigg) E_{i+n} + \sum_{n=1-i}^{-1} \bigg( \prod_{j=0}^{1+n} \tau_{i+j} \bigg) E_{i+n}  \right\} \\
  &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{|n|-1} \tau_{i+j} \bigg) E_{i+n} + \sum_{n=1-i}^{-1} \bigg( \prod_{j=0}^{1- |n|} \tau_{i+j} \bigg) E_{i+n}  \right\} \\
  &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{|n|-1} \tau_{i+j} \bigg) E_{i+n} + \sum_{n=1-i}^{-1} \bigg( \prod_{j=0}^{sign(n) (|n|-1)} \tau_{i+j} \bigg) E_{i+n}  \right\}
\end{align}

Now combine both sums together

\begin{align}
F_i &= \frac{1}{\tau_i} \left\{ \sum_{n=1}^{N-i} \bigg( \prod_{j=0}^{|n|-1} \tau_{i+j} \bigg) E_{i+n} + E_i + \sum_{n=1-i}^{-1} \bigg( \prod_{j=0}^{sign(n) (|n|-1)} \tau_{i+j} \bigg) E_{i+n}  \right\} - \frac{E_i}{\tau_i} \\
   &= \frac{1}{\tau_i} \left\{ \sum_{n=1-i}^{N-i} \bigg( \prod_{j=0}^{sign(n) \big(|n|-1\big)} \tau_{i+j} \bigg) E_{i+n}  - E_i \right\}
\end{align}

with the convention that $\prod_{j=0}^{-1} = 1$.

Alternatively if we set $\prod_{j=0}^{-1} = 0$ then

$$ F_i = \frac{1}{\tau_i} \left\{ \sum_{n=1-i}^{N-i} \bigg( \prod_{j=0}^{sign(n) \big(|n|-1\big)} \tau_{i+j} \bigg) E_{i+n} \right\} $$

Let's use this notation since it simplifies our expressions.

### Net radiation
(absorptivity) * incident - emission

\begin{align}
R_i &= \epsilon_i F_i - 2 E_i  \\
  &= \frac{\epsilon_i}{\tau_i}  \left\{ \sum_{n=1-i}^{N-i} \bigg( \prod_{j=0}^{sign(n) \big(|n|-1\big)} \tau_{i+j} \bigg) E_{i+n} \right\}  - 2 E_i
\end{align}

### Alternative...

We will define the **transmissivity between layer i and layer i+n **

$$ T_{in} = \left\{ \begin{array}{cc}
 \prod_{j=1}^{n-1} \tau_{i+j} &  n > 1 \\
 1 &  n = 1 \\
 0  &  n = 0 \\
 1 & n = -1 \\
 \prod_{j=1}^{-n-1} \tau_{i-j} &  n < -1
 \end{array} \right\}
 $$

Then the incident flux follows directly

$$ F_i = \sum_{n=1-i}^{N-i} T_{in} E_{i+n} $$

and the net radiation is

\begin{align}
R_i &= \epsilon_i F_i - 2 E_i  \\
 &= \epsilon_i \sum_{n=1-i}^{N-i} T_{in} E_{i+n} - 2 E_i
\end{align}

Now make the substitution $i+n \rightarrow m$

\begin{align}
F_i &= \sum_{m=1}^{N} T_{im} E_{m}  \\
T_{im} &= \left\{ \begin{array}{cc}
 \prod_{j=1}^{m-i-1} \tau_{i+j} &  m > 1+i \\
 1 &  m = i+1 \\
 0  &  m = i \\
 1 & m = i-1 \\
 \prod_{j=1}^{i-m-1} \tau_{i-j} &  m < i-1
 \end{array} \right\}
\end{align}

Or using the Einstein summation notation, since $m$ is a repeated index, we can just write

$$ F_i = T_{im} E_{m} $$

and the net radiation is

$$ R_i = \epsilon_i F_i - 2 E_i $$

## Reformulating in terms of flux between layers

Let the upwelling flux be a vector ${\bf{U}} = [U_0, U_1, ..., U_{N-1}, U_N]$.

If there are $N$ levels then $\bf{U}$ has $N+1$ elements. We will number the layers starting from 0 following `numpy` index conventions.

- $U_0$ is the upwelling flux from surface to layer 0.
- $U_1$ is the upwelling flux layer 0 to layer 1, etc.
- $U_N$ is the upwelling flux from layer N-1 (the top level) to space.

Same for the downwelling flux ${\bf{D}} = [D_0, D_1, ..., D_N]$. So $D_N$ is the flux down from space and $D_0$ is the backradiation to the surface.

The absorptivity vector is ${\bf{\epsilon}} = [\epsilon_0, \epsilon_1, ..., \epsilon_{N-1}]$ ($N$ elements)

```python
epsilon, epsilon_i, N = sympy.symbols('epsilon, epsilon_i, N', nonnegative=True )
```

##  Will do the 3 layer version first

```python
# vector of emissions
E = sympy.Matrix([E_0, E_1, E_2])
E
```

```python
# upwelling flux
fromsurface = E_s
U = sympy.Matrix([fromsurface, tau_0*fromsurface + E_0, tau_1*(tau_0*fromsurface + E_0) + E_1,
                  tau_2*(tau_1*(tau_0*fromsurface + E_0) + E_1) + E_2])
U
```

```python
# downwelling flux...
fromspace = 0
D = sympy.Matrix([ tau_0*(tau_1*(tau_2*fromspace + E_2) + E_1) + E_0,
                 tau_1*(tau_2*fromspace + E_2) + E_1, tau_2*fromspace + E_2, fromspace])
D
```

```python
# Net flux, positive up
F = U - D
F
```

```python
# The absorption is then simply the flux convergence in each layer

# define a vector of absorbed radiation -- same size as emissions
A = E.copy()

# Get the convergence
for n in range(3):
    A[n] = -(F[n+1]-F[n])

A
```

```python
# this should reduce to zero if I did it right
sympy.simplify(A - sympy.Matrix([R_0, R_1, R_2]))
```

## So that works. I can formulate the tests against numerical code this way

```python

```


```python
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import sympy as sy
sy.init_printing()
```

#  Null Space

The  null space , denoted $\text{Nul}A$ is the solution set of a homogenous linear system $Ax=0$.

Null space is a subspace of $\mathbb{R}^n$, why? Consider a linear system.

$$
3x_1-x_2+x_3 = 0\\
x_1+2x_2+3x_3= 0
$$

The augmented matrix is

$$
\left[
\begin{matrix}
2 & -1 & 1 & 0\\
1 & 2 & 3 & 0
\end{matrix}
\right]
$$

Before solving the system, we have already known there is no unique solution since a free variable exists.

```python
Aug = sy.Matrix([[2,-1,1,0],[1,2,3,0]])
Aug.rref()
```

$x_3$ is a free variable, the solution set is

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3
\end{matrix}
\right]=
\left[
\begin{matrix}
-x_3 \\ -x_3 \\ x_3
\end{matrix}
\right]=
x_3\left[
\begin{matrix}
-1 \\ -1 \\ 1
\end{matrix}
\right]
$$

which is a line passing through $(0, 0, 0)$ and $(-1, -1, 1)$, a subspace of $\mathbb{R}^3$.

Consider another example, suppose we have an augmented matrix

```python
Aug = sy.Matrix([[-3,6,-1,1,-7,0],[1,-2,2,3,-1,0],[2,-4,5,8,-4,0]]);Aug
```

```python
Aug.rref()
```

The solution can be written as:

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3 \\x_4 \\ x_5
\end{matrix}
\right]=
\left[
\begin{matrix}
2x_2+x_4-3x_5 \\ x_2 \\ -2x_4+2x_5 \\x_4 \\ x_5
\end{matrix}
\right]=
x_2\left[
\begin{matrix}
2 \\ 1 \\ 0 \\0 \\ 0
\end{matrix}
\right]
+
x_4\left[
\begin{matrix}
1 \\ 0 \\ -2 \\1 \\ 0
\end{matrix}
\right]
+x_5\left[
\begin{matrix}
-3 \\ 0 \\ 2 \\0 \\ 1
\end{matrix}
\right]
$$

The $\text{Nul}A$ is a subspace in $\mathbb{R}^5$ with dimension of 3.

#  Null Space vs Col Space

Let

```python
A = sy.Matrix([[2,4,-2,1],[-2,-5,7,3],[3,7,-8,6]]);A
```

Column space is a subspace in $\mathbb{R}^n$, what is $n$? It is the number of rows, $n=3$.

Null space is a subspace in $\mathbb{R}^m$, what is $m$? It is the number of columns, $m=4$.

Find any nonzero vector in $\text{Col}A$ and in $\text{Nul}A$.

Any column in the matrix can be a nonzero column in $\text{Col}A$, for instance first column: $(2, -2, 3)^T$.

But to find a nonzero vector in null space requires some effort, construct the augmented matrix then turn it into rref.

```python
Aug = sy.Matrix([[2,4,-2,1,0],[-2,-5,7,3,0],[3,7,-8,6,0]]);Aug.rref()
```

The solution set with a free variable $x_3$ (because column 3 has no pivot) is

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3\\x_4
\end{matrix}
\right]=
\left[
\begin{matrix}
-9x_3 \\ 5x_3 \\ x_3\\0
\end{matrix}
\right]
$$

If we pick $x_3 =1$, a nonzero vector in $\text{Nul}A$ is $(-9, 5, 1, 0)^T$

Now consider two vectors

$$
u = \left[
\begin{matrix}
3 \\ -2 \\ -1\\ 0
\end{matrix}
\right],\qquad
v = \left[
\begin{matrix}
3 \\ -1\\3
\end{matrix}
\right]\\
$$

Is $u$ in $\text{Nul}A$? It can be verified easily

```python
u = sy.Matrix([[3],[-2],[-1],[0]])
A*u
```

$Au\neq \mathbf{0}$, therefore $u$ is not in $\text{Nul}A$.

Is $v$ in $\text{Col}A$?

```python
v = sy.Matrix([[3],[-1],[3]])
A.row_join(v).rref()
```

The augmented matrix show there are solutions, i.e. $v$ is a linear combination of its column space basis, so $v$ is in $\text{Col}A$.

#  Row Space

 Row space, denoted as $\text{Row}A$, is all linear combination of row vectors and subspace in $\mathbb{R}^n$.

If we perform row operations on $A$ to get $B$, both matrices have the same row space, because $B$'s rows are linear combinations of $A$'s. However, row operation will change the row dependence.

##  An Example

Find the row, column and null space of

```python
A = sy.Matrix([[-2, -5, 8, 0, -17],
               [1, 3, -5, 1, 5],
               [3, 11, -19, 7, 1],
               [1, 7, -13, 5, -3]]);A
```

```python
B = A.rref();B
```

The basis of row space of $B$ is its first 3 rows: $(1,0,1,0,1), (0, 1, -2, 0, 3), (0, 0, 0, 1, -5)$ which are also the basis of row space of $A$. However it does not necessarily mean that first 3 rows of $A$ forms the basis for row space, because the dependence among rows changed by row operation.

In constrast, the basis of col space of $A$ is $(-2, 1, 3, 1)^T, (-5, 3, 11, 7)^T, (0, 1, 7, 5)^T$.

```python
Aug = A.row_join(sy.zeros(4,1));Aug.rref()
```

The null space is

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3\\x_4 \\x_5
\end{matrix}
\right]=
\left[
\begin{matrix}
-x_3-x_5 \\ 2x_3-3x_5 \\ x_3\\5x_5 \\x_5
\end{matrix}
\right]=
x_3\left[
\begin{matrix}
-1 \\ 2 \\ 1\\0 \\0
\end{matrix}
\right]+
x_5
\left[
\begin{matrix}
-1 \\ -3 \\ 0\\5 \\1
\end{matrix}
\right]
$$

#  Rank

Definition of rank:
The  rank  is the dimension of the column space of $A$. The  nullity  of $A$ is the dimension of the null space.

##  The Rank Theorem

The dimensions of the column space and the row space of an $m \times n$ matrix $A$ are equal that is why we only need to say rank is the dimension of the column space.

This common dimension, the rank of $A$, also equals the number of pivot positions in $A$ and satisfies the equation
$$
\operatorname{rank} A+\operatorname{dim} \mathrm{Nul} A=n
$$

The intuition is that when a matrix $A$ is converted into rref $B$, we can indirectly(matching the same column from $B$ to $A$) see the basis of column space, those columns in corresponding rref have pivots.

And in rref, we can also see the basis of row space directly, every row in the basis of row space have a pivot as well. And those rows which does not have pivots are for free variables, which is the dimension of null space.

##  Example 1

If $A$ is $45 \times 50$ matrix with a $10$-dimension nullity, what is the rank of $A$?

$10D$ nullity means 10 free variables, so the pivots are $50-10=40$, which is also the rank of $A$.

##  Example 2

The matrices below are row equivalent.
$$
A=\left[\begin{array}{rrrrr}
2 & -1 & 1 & -6 & 8 \\
1 & -2 & -4 & 3 & -2 \\
-7 & 8 & 10 & 3 & -10 \\
4 & -5 & -7 & 0 & 4
\end{array}\right], \quad B=\left[\begin{array}{rrrrr}
1 & -2 & -4 & 3 & -2 \\
0 & 3 & 9 & -12 & 12 \\
0 & 0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 0
\end{array}\right]
$$
1. Find rank $A$ and $\operatorname{dim}$ Nul $A$
2. Find bases for Col $A$ and Row $A$.
3. What is the next step to perform to find a basis for Nul $A$ ?
4. How many pivot columns are in a row echelon form of $A^{T} ?$

1. $rank(A)=2$, because $B$ has two pivots. And nullity is the number of free variables, there are 3, so $\text{dim Nul}A = 3$.

2. Bases for $\text{Col}A$ is $(2,1,-7,4)^T, (-1,-2,8,-5)^T$, and for $\text{Row}A$ is $(1,-2,-4,3,-2),(0,3,9,-12,12)$.

3. Perform rref on augmented $A$

```python
A = sy.Matrix([[2,-1,1,-6,8,0],
               [1,-2,-4,3,-2,0],
               [-7,8,10,3,-10,0],
               [4,-5,-7,0,4,0]])
A.rref()
```

The $\text{Nul}A$ and basis is

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3\\x_4 \\x_5
\end{matrix}
\right]=
\left[
\begin{matrix}
-2x_3+5x_4-6x_5 \\ -3x_3+4x_4-4x_5 \\ x_3\\x_4 \\x_5
\end{matrix}
\right]=
x_3
\left[
\begin{matrix}
-2 \\ -3 \\ 1\\0 \\0
\end{matrix}
\right]+
x_4
\left[
\begin{matrix}
5 \\ 4 \\ 0\\1 \\0
\end{matrix}
\right]+
x_5
\left[
\begin{matrix}
-6 \\ -4 \\ 0\\0 \\1
\end{matrix}
\right]
$$

4. Transpose $A$ then do rref.

```python
A.T.rref()
```

There are 2 pivot columns.

Actually, we don't need any calculation to know the rank of $A^T$, because

$$
rank(A)=rank(A^T)
$$

#  Orthogonality of $\text{Nul}A$ and $\text{Row}A$

##   $\text{Nul}A \perp \text{Row}A$

Here is the intersting connections of these subspaces we have discussed. Consider

```python
A = sy.Matrix([[5, 8, 2], [10, 16, 4], [3, 4, 1]]);A
```

```python
A.rref()
```

The basis of row space of $A$ is $(1, 0, 0)$ and $(0, 1, .25)$.And the $\text{Row}A$ is

$$
\text{Row}A=
s\left[
\begin{matrix}
1 \\ 0\\ 0
\end{matrix}
\right]+
t\left[
\begin{matrix}
0 \\ 1\\ 0.25
\end{matrix}
\right]
$$

The $\text{Nul}A$ is
$$
\left[
\begin{matrix}
x_1 \\ x_2\\ x_3
\end{matrix}
\right]=
x_3
\left[
\begin{matrix}
0 \\ -.25\\ 1
\end{matrix}
\right]
$$

Now we can visualize their relations geometrically.Again keep in mind that Matplotlib does not render 3D properly, so you need some imagination as well.

Here is what we observe.

The $\text{Row}A$ is a plane and $\text{Nul}A$ is a line which is perpendicular to the plane. It is easy to grasp the idea if you notice that in a homogeneous system $Ab = \mathbf{0}$, it breaks down into many dot products

$$
Ab =\left[
\begin{matrix}
A_{1i}\cdot b \\ A_{2i}\cdot b\\ A_{3i}\cdot b
\end{matrix}
\right]
$$

where $A_{1i}$ and etc are the rows of $A$. In later chapters we will prove when the dot product of two vectors equals zero, then they are perpendicular.

```python
%matplotlib notebook
s = np.linspace(-1, 1, 10)
t = np.linspace(-1, 1, 10)
S, T = np.meshgrid(s, t)

X = S
Y = T
Z = T*.25

fig = plt.figure(figsize = (8,8))

ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(X, Y, Z, alpha = .9, cmap=plt.cm.coolwarm)

x3 = np.linspace(-1, 1, 10)
x1 = 0*x3
x2 = -.25*x3
ax.plot(x1,x2,x3, lw = 5)

ax.set_xlabel('x-axis', size = 18)
ax.set_ylabel('y-axis', size = 18)
ax.set_zlabel('z-axis', size = 18)

ax.axis([-1,1,-1,1])

ax.text(x = 1, y = -1, z = -.25, s = r'$Row\ A$', size = 17)
ax.text(0, -.25, 1, s = r'$Nul\ A$', size = 17)

ax.view_init(7, 20)
```

    <IPython.core.display.Javascript object>

##   $\text{Nul}A^T \perp \text{Col}A$

The nullity of $A^T$ is

```python
A = sy.Matrix([[5, 8, 2], [10, 16, 4], [3, 4, 1]]);A.T.rref()
```

The $\text{Nul}A^T$ is

$$
\left[
\begin{matrix}
x_1 \\ x_2\\ x_3
\end{matrix}
\right]=
x_2
\left[
\begin{matrix}
-2 \\ 1\\ 0
\end{matrix}
\right]
$$

The  $\text{Col}A$ is

```python
A.rref()
```

$$
\text{Col}A=
s\left[
\begin{matrix}
5 \\ 10\\ 3
\end{matrix}
\right]+
t\left[
\begin{matrix}
8 \\ 16\\ 4
\end{matrix}
\right]
$$

$\text{Col}A$ is a plane and $\text{Nul}A^T$ is a line perpendicular to the plane. The intuition is similar to $\text{Nul}A \perp \text{Row}A$, here you can think of a system look like $b^TA = \mathbf{0}^T$.

```python
%matplotlib notebook
s = np.linspace(-1, 1, 10)
t = np.linspace(-1, 1, 10)
S, T = np.meshgrid(s, t)

X = 5*S+8*T
Y = 10*S+16*T
Z = 3*S+4*T

fig = plt.figure(figsize = (8,8))

ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(X, Y, Z, cmap=plt.cm.coolwarm)

x2 = np.linspace(-1, 1, 10)
x3 = x2*0
x1 = -2*x2

ax.plot(x1,x2,x3, lw = 3)

ax.set_xlabel('x-axis', size = 18)
ax.set_ylabel('y-axis', size = 18)
ax.set_zlabel('z-axis', size = 18)

ax.axis([-1,1,-1,1])

ax.view_init(-67, 35)
```

    <IPython.core.display.Javascript object>

#  Rank Decomposition

Consider a matrix $A$, the purpose is to decompose it into the multiplication of $C$, $R$, which are the  bases of column space and row space respectively.

$$
A = CR
$$

```python
A = sy.Matrix([[2, 4, 1, -1], [4, 2, -4, 2], [2, -2, -5, 3], [1, 9, -3, 2]]);A
```

```python
Arref = A.rref();Arref
```

Get the basis of $\text{Col}A$.

```python
ColA_basis = A[:,:3];ColA_basis
```

Then get the $\text{Row}A$.

```python
RowA_basis = Arref[0][0:3,:];RowA_basis
```

Multiply $CR$, we are getting back $A$.

```python
ColA_basis*RowA_basis
```

Verify if $CR$ equals $A$.

```python
ColA_basis*RowA_basis == A
```

    True

Chapter 10 -Null Space vs Col Space, Row Space and Rank.ipynb


# PageRank

#### Librerías utilizadas

```python
import numpy as np
import math
import pandas as pd
from scipy.sparse import csr_matrix

# Para las pruebas
from pytest import approx
import pprint

# Para los grafos
import networkx as nx
import matplotlib.pyplot as plt
```

## Definimos una set para practicar (por ahora 4 páginas)

Donde podemos observar en la columnas:
- `pagina`: la página a la que hace referencia.
- `outlinks`: los links a los que cada `pagina` hace referencia.

```python
pages_links = pd.DataFrame(columns=['pagina','outlinks'])
```

```python
pages_links['pagina'] = ['A', 'B', 'C', 'D']
```

```python
pages_links['outlinks'][0] = ['C', 'B']
```

```python
pages_links['outlinks'][1] = ['D']
```

```python
pages_links['outlinks'][2] = ['A', 'B', 'D']
```

```python
pages_links['outlinks'][3] = ['C']
```

```python
pages_links
```

      pagina
      outlinks

      0
      A
      [C, B]

      1
      B
      [D]

      2
      C
      [A, B, D]

      3
      D
      [C]

## Definimos un grafo pequeño de acuerdo a los datos anteriores

```python
nodos = pages_links['pagina']

plt.figure(figsize=(8,8))
G = nx.DiGraph()

for nodo in nodos:
    G.add_node(nodo)

for page, outlinks in zip(pages_links['pagina'],pages_links['outlinks']):
    for outlink in outlinks:
        G.add_edge(page, outlink)

pos = nx.spring_layout(G)

nx.draw_networkx_nodes(G, pos, node_color='#a0a88a', node_size=400)
nx.draw_networkx_labels(G, pos)
nx.draw_networkx_edges(G, pos, edge_color='#305029', width=1.8, arrows = True, style='dashed')

plt.show()
```

### Definimos nuestra matriz estocástica conforme a lo visto en el grafo:

```python
sparseMatrix = csr_matrix((len(pages_links), len(pages_links)),
                          dtype = np.float).toarray()
```

```python
newdf = pd.DataFrame(sparseMatrix, columns = pages_links['pagina'], index=pages_links['pagina'])
```

```python
count = 0
for page, outlinks in zip(pages_links['pagina'],pages_links['outlinks']):
    score = 1/len(outlinks)
    dict_page = {page: np.zeros(len(pages_links))}
    for outlink in outlinks:
        newdf[page][outlink] = score
```

```python
newdf
```

      pagina
      A
      B
      C
      D

      pagina

      A
      0.0
      0.0
      0.333333
      0.0

      B
      0.5
      0.0
      0.333333
      0.0

      C
      0.5
      0.0
      0.000000
      1.0

      D
      0.0
      1.0
      0.333333
      0.0

```python
H = newdf.to_numpy()
```

```python
pd.DataFrame(newdf)
```

      pagina
      A
      B
      C
      D

      pagina

      A
      0.0
      0.0
      0.333333
      0.0

      B
      0.5
      0.0
      0.333333
      0.0

      C
      0.5
      0.0
      0.000000
      1.0

      D
      0.0
      1.0
      0.333333
      0.0

## Probamos la obtención de Eigenvectores con Factorización QR (algoritmo desarrollado en la práctica 1.2)

```python
def sign(x):
    """
    Helper function for computing sign of real number x.
    """
    if x >= 0:
        return 1
    else:
        return -1
```

```python
def inverse_matrix(A):
    """
    Helper function for computing the inverse of matrix A nxn.
    """
    inv_temp = []

    for i in range (0,A.shape[0]):
        e1 = np.zeros((A.shape[0],1))
        e1[i] = 1
        A_inv_i = np.linalg.solve(A.T, e1)
        inv_temp.append(A_inv_i)

    for j in range (0,len(inv_temp)-1):
        if j == 0:
            r_inv = np.row_stack((inv_temp[j].T, inv_temp[j+1].T))
        else:
            r_inv = np.row_stack((r_inv, inv_temp[j+1].T))

    return(r_inv)
```

```python
def off(T):
    """
    Helper function for computing off(T).
    """
    mask = np.ones(T.shape, dtype = bool)
    np.fill_diagonal(mask, 0)
    T_sin_dig = T[mask]
    sum = 0
    for i in range (0,len(T_sin_dig)):
        # print (T_sin_dig[i])
        sum += T_sin_dig[i]**2
    off_T = np.sqrt(sum)

    return(off_T)
```

```python
def QR_Householder(A):
    """
    Decompose a real square matrix A where: A = QR usign Householder reflections.
    Args:
        A (numpy ndarray): Matrix in which QR algorithm will be performed.
    Returns:
        Q (numpy ndarray): orthogonal matrix (its columns are orthogonal unit vectors meaning
                            Q.TQ = QQ.T = I)
        R (numpy ndarray): upper triangular matrix.
    """
    a_aux = []

    for j in range (0,A.shape[0]-1):          #Asumimos que A SIEMPRE será cuadrada
        e = np.zeros((A.shape[0]-j, 1))
        e[0] = 1
        if j == 0:
            v = A[:,0] + sign(A[0,0])*np.linalg.norm(A[:,0])*e.T[0]
            beta = 2/v.dot(v)
            aux = A[:,0:] - beta*np.outer(v,v.dot(A[:,0:]))
            a_aux.append(aux)
        else:
            v = aux[1:,1] + sign(aux[1,1])*np.linalg.norm(aux[1:,1])*e.T[0]
            beta = 2/v.dot(v)
            aux = aux[1:,1:] - beta*np.outer(v,v.dot(aux[1:,1:]))
            a_aux.append(aux)

    n,n = A.shape
    count = 0

    if n == 2:
        R = a_aux[0]
    else:
        for h in range(n-2,0,-1):
            if count == 0:
                temp_col = np.column_stack((np.zeros(2 + count), a_aux[h]))
            else:
                temp_col = np.column_stack((np.zeros(2 + count), temp_row))
            temp_row = np.row_stack((a_aux[h-1][0, 0:], temp_col))
            count += 1
        R = temp_row
    R_inv = inverse_matrix(R)

    Q = A@R_inv

    return (Q,R)
```

```python
def Eig_Householder(A):
    """
    Compute QR algorithm usign Householder reflections multiple times to approximate eigenvalues and
    eigenvectors of A.
    Args:
        A (numpy ndarray): Symmetric matrix
    Returns:
        evalues (numpy ndarray): Array with eigenvalues of matrix A.
        evectors (numpy ndarray): Matrix with eigenvectors of matrix A on its columns.
    """
    tol = 10e-8
    max_iters = 2000
    tk_fro_norm = np.linalg.norm(A, 'fro')
    iterations = 0
    off_Tk = off(A)

    q_aux = []

    while off_Tk > tol*tk_fro_norm and iterations < max_iters:
        iterations += 1
        if iterations == 1:
            Tk_minus1 = A
        else:
            Tk_minus1 = Tk

        q,r = QR_Householder(Tk_minus1)
        q_aux.append(q)

        Tk = r@q

        off_Tk = off(Tk)
        tk_fro_norm = np.linalg.norm(Tk, 'fro')

    #Multiplicación Q0*Q1...Qn para cálculo de eigenvectores
    for g in range(0,len(q_aux)-1,1):
        if g == 0:
            q_temp = q_aux[g]@q_aux[g+1]
        else:
            q_temp = q_temp@q_aux[g+1]

    if iterations == 0:
        evalues = "error"
        evectors = "error"
    else:
        evectors = q_temp
        evalues = np.diag(Tk)

    return (evalues,evectors)
```

```python
eigval, eigvec = Eig_Householder(H)
```

```python
pd.DataFrame(eigval)
```

      0

      0
      1.000000

      1
      -0.626538

      2
      -0.357205

      3
      -0.016257

```python
pd.DataFrame(eigvec)
```

      0
      1
      2
      3

      0
      0.232495
      0.529635
      0.237150
      -0.780508

      1
      0.348743
      0.192872
      0.785518
      0.473434

      2
      0.697486
      -0.672574
      0.004505
      -0.247260

      3
      0.581238
      0.479512
      -0.571577
      0.324855

#### No olvidemos comprobar que los eigenvalores y eigenvectores sean los esperados

```python
eigval_eig , eigvec_eig = np.linalg.eig(H)
```

```python
pd.DataFrame(eigval_eig)
```

      0

      0
      1.000000+0.000000j

      1
      -0.626538+0.000000j

      2
      -0.186731+0.480774j

      3
      -0.186731-0.480774j

```python
pd.DataFrame(eigvec_eig)
```

      0
      1
      2
      3

      0
      -0.232495+0.000000j
      0.445747+0.000000j
      0.146497+0.377184j
      0.146497-0.377184j

      1
      -0.348743+0.000000j
      0.090025+0.000000j
      0.435929+0.112414j
      0.435929-0.112414j

      2
      -0.697486+0.000000j
      -0.837833+0.000000j
      -0.626088+0.000000j
      -0.626088-0.000000j

      3
      -0.581238+0.000000j
      0.302061+0.000000j
      0.043661-0.489599j
      0.043661+0.489599j

## Probamos con un Método de la Potencia adaptado a PageRank (obtenido de [Global Software Support](https://www.youtube.com/watch?v=kSmQbVxqOJc&feature=emb_title))

Este método de la potencia se define como:

$$ PR_{t+1} = H PR_t $$

Donde:
- $H$: la matriz estocástica.
- $PR$: PageRank de la página indicada.

#### Definimos un vector inicial

El vector será de tamaño $n$ (número de nodos) y la inicialización de cada valor será $1/n$.

En este caso:

\begin{align}
    v &= \begin{bmatrix}
           1/4 \\
           1/4 \\
           1/4 \\
           1/4
         \end{bmatrix}
  \end{align}

```python
v = np.ones(len(pages_links))
```

```python
v = v*1/len(v)
```

```python
pprint.pprint(v)
```

    array([0.25, 0.25, 0.25, 0.25])

Para obtener el segundo vector se debe hacer:

$$v_2 = Hv$$

```python
v_2 = H@v

pprint.pprint(v_2)
```

    array([0.08333333, 0.20833333, 0.375     , 0.33333333])

Para obtener el tercer vector:

$$ v_3 = Hv_2 = H(Hv) = H^2v$$

```python
v_3  = H@v_2

pprint.pprint(v_3)
```

    array([0.125     , 0.16666667, 0.375     , 0.33333333])

```python
v_3_r = (H@H)@v

pprint.pprint(v_3_r)
```

    array([0.125     , 0.16666667, 0.375     , 0.33333333])

Entonces, observamos que se puede representar de la forma:
$$v_n = H^{n}v$$

Si iteramos varias veces tiende a un valor equilibrado:

```python
n = 20
H = np.linalg.matrix_power(H, n)
v_n = H@v
```

```python
v_n
```

    array([0.12500368, 0.18750089, 0.37499297, 0.31250246])

```python
df = pd.DataFrame(v_n, columns=['pagerank'])
```

```python
df['nodo'] = nodos
```

```python
df = df.set_index('nodo')
```

```python
power_method = df.pagerank.to_dict()
```

```python
power_method
```

    {'A': 0.1250036838777434,
     'B': 0.18750089306127105,
     'C': 0.37499296714248953,
     'D': 0.3125024559184955}

```python
power_method_round = {}

for node in power_method:
    if node in power_method:
        power_method_round[node] = round(power_method[node],2)
```

```python
power_method_round
```

    {'A': 0.13, 'B': 0.19, 'C': 0.37, 'D': 0.31}

## Probamos con el Método de la Potencia visto en clase

```python
n = 4
q_k = v
max_iter = 20
q_k_iter = np.zeros((n, max_iter))
lambda_k_iter = np.zeros(max_iter)

for k in range(max_iter):
    z_k = H@q_k
    q_k = z_k/np.linalg.norm(z_k)
    q_k_iter[:,k] = q_k
    lambda_k = q_k.T@H@q_k
    lambda_k_iter[k] = lambda_k
```

```python
pd.DataFrame(q_k)
```

      0

      0
      0.232495

      1
      0.348743

      2
      0.697486

      3
      0.581238

## Ahora probamos con PageRank de `Networkx`

El cálculo del eigenvector se realiza mediante el método iterativo de la potencia y no tiene garantía de convergencia. Las iteraciones se detendrán después de que se alcance una tolerancia de error de `len(G)*tol`. Si el número de iteraciones excede el número máximo de iteraciones permitido, se genera una excepción `networkx.exception.PowerIterationFailedConvergence`.
El algoritmo PageRank fue diseñado para grafos dirigidos, pero este algoritmo no verifica si el grafo de entrada es dirigido o no, y se ejecutará en grafos no dirigidos, convirtiendo cada borde en el grafo dirigido en dos bordes.

```python
from networkx.algorithms.link_analysis.pagerank_alg import pagerank
```

```python
pr = pagerank(G, max_iter=20)
```

```python
pr
```

    {'A': 0.13867229558179425,
     'B': 0.1976082981211456,
     'C': 0.3570799577952133,
     'D': 0.30663944850184677}

```python
pagerank_round = {}

for node in pr:
    if node in pr:
        pagerank_round[node] = round(pr[node],2)
```

```python
pagerank_round
```

    {'A': 0.14, 'B': 0.2, 'C': 0.36, 'D': 0.31}

## Ahora comparamos todos los resultados obtenidos

```python
pagerank_round_pd = pd.DataFrame.from_dict(pagerank_round, orient='index')
```

```python
eigvector_comparison = pd.DataFrame({"NX PageRank": pagerank_round_pd[0],
                                    "Potencia PR": v_n,
                                    "Potencia OPT-2020": q_k,
                                    "LinAlg": eigvec_eig[:,0].real,
                                    "QR_Eq4": eigvec[:,0]}
                                    ).reset_index().iloc[:,1:]
eigvector_comparison
```

      NX PageRank
      Potencia PR
      Potencia OPT-2020
      LinAlg
      QR_Eq4

      0
      0.14
      0.125004
      0.232495
      -0.232495
      0.232495

      1
      0.20
      0.187501
      0.348743
      -0.348743
      0.348743

      2
      0.36
      0.374993
      0.697486
      -0.697486
      0.697486

      3
      0.31
      0.312502
      0.581238
      -0.581238
      0.581238

En la tabla anterior podemos apreciar que tanto el eigenvector obtenido por `Networkx` como el obtenido por el método de la potencia adaptado son muy parecidos. De hecho, éstos son factores de los vectores obtenidos con los otros 3 métodos. Adicionalmente, se aprecia que el orden se mantiene, por lo que se confirma que los PageRanks obtenidos son los correctos.

#### Comparamos método de la potencia vs `pagerank_alg.pagerank`

```python
assert pagerank_round == approx(power_method_round, rel=2e-1)
```

## Mismo grafo con el  PageRank obtenido por el método de la potencia

```python
plt.figure(figsize=(8,8))
G = nx.DiGraph()

for nodo in power_method:
    G.add_node(nodo)

for page, outlinks in zip(pages_links['pagina'],pages_links['outlinks']):
    for outlink in outlinks:
        G.add_edge(page, outlink)

pos = nx.spring_layout(G)

labels = {}
for node in G.nodes():
    if node in power_method:
        #set the node name as the key and the label as its value
        labels[node] = "{}:{}".format(node,round(power_method[node],2))

# Es lo mismo que power_method pero redondeado
print(labels)

nx.draw_networkx_nodes(G, pos, node_color='#a0a88a', node_size=400)
nx.draw_networkx_labels(G, pos, labels=labels, font_weight="bold")
nx.draw_networkx_edges(G, pos, edge_color='#305029', width=1.8, arrows = True, style='dashed')

plt.show()
```

Entonces podemos observar que la página con mayor puntuación es la `C`.

## Ahora intentaremos con otro set de datos (37 páginas)

[set de datos de Stanford](http://snap.stanford.edu/data/web-Stanford.html)

```python
path = "../data/web-Stanford.txt"
```

```python
def read_csv_Stanford(path, nrow=103):
    """
    Read the txt given, only works for web-Stanford
    """
    df = pd.read_csv(path, nrows=nrow)#, usecols=['FromNode', 'ToNode'])
    # Remove the index
    df = df[3:]
    df = df.rename(columns = {'# Directed graph (each unordered pair of nodes is saved once): web-Stanford.txt ':
                         'FromTo'})
    df[['FromNode','ToNode']] = df.FromTo.str.split('\t',expand=True)
    df = df.drop('FromTo', axis=1)
    return df
```

```python
df_stanford = read_csv_Stanford(path,103)
```

```python
df_stanford
```

      FromNode
      ToNode

      3
      1
      6548

      4
      1
      15409

      5
      6548
      57031

      6
      15409
      13102

      7
      2
      17794

      ...
      ...
      ...

      98
      74361
      226411

      99
      74361
      239874

      100
      74361
      255694

      101
      74361
      264216

      102
      74361
      266083

100 rows × 2 columns

```python
def transform_data():
    """
    Transform the nodes that are not in the FromNode and ToNode match
    This only works for 103 nrows
    """
    missing_from = list(set(df_stanford['FromNode']) - set(df_stanford['ToNode']) )
    missing_to = list(set(df_stanford['ToNode'])  - set(df_stanford['FromNode']) )
    mitad = int(len(missing_to)/2)
    for val in missing_to[:mitad]:
        df_stanford['ToNode'][df_stanford['ToNode'] == val] = missing_from[0]
    for val in missing_to[mitad:]:
        df_stanford['ToNode'][df_stanford['ToNode'] == val] = missing_from[1]
```

```python
transform_data()
```

```python
# Hacemos una agrupación y la lista de los outlinks
paginas_stanford = df_stanford.groupby('FromNode').aggregate(lambda tdf: tdf.unique().tolist()).reset_index()
```

```python
paginas_stanford = paginas_stanford.rename(columns={'FromNode': 'pagina',
                                                    'ToNode': 'outlinks'})
```

```python
paginas_stanford.head(10)
```

      pagina
      outlinks

      0
      1
      [6548, 15409]

      1
      100193
      [2]

      2
      102355
      [2]

      3
      105318
      [2]

      4
      105730
      [2, 3]

      5
      115926
      [2]

      6
      140864
      [2]

      7
      15409
      [1]

      8
      163550
      [2]

      9
      164599
      [2]

```python
paginas_stanford.shape
```

    (37, 2)

```python
def plot_node_graph(dataframe_pages, power_method=None, with_labels=False):
    """
    Plotea el grafo
    :param[in] dataframe_pages: dataframe de nx2
                El dataframe debe contener una columna llamada página y otra llamada outlinks;
                donde outlinks contenga una lista de los outlinks por página
    """
    nodos = dataframe_pages['pagina']

    plt.figure(figsize=(10,10))
    G = nx.DiGraph()

    for nodo in nodos:
        G.add_node(nodo)

    for page, outlinks in zip(dataframe_pages['pagina'], dataframe_pages['outlinks']):
        for outlink in outlinks:
            G.add_edge(page, outlink)

    pos = nx.spring_layout(G)

    nx.draw_networkx_nodes(G, pos, node_color='#a0a88a', node_size=400)
    if with_labels:
        labels = {}
        for node in G.nodes():
            if node in power_method:
                #set the node name as the key and the label as its value
                labels[node] = "{}:{}".format(node,round(power_method[node],2))
        nx.draw_networkx_labels(G, pos, labels=labels, font_weight="bold")
    else:
        nx.draw_networkx_labels(G, pos)
    nx.draw_networkx_edges(G, pos, edge_color='#305029', width=1.8, arrows = True, style='dashed')

    plt.show()
    return G
```

```python
G37 = plot_node_graph(paginas_stanford)
```

#### Definimos nuestra matriz estocástica

```python
def get_h_matrix(dataframe_pages):
    """
    Obtiene la matriz H, la matriz estocástica
    :param[in] dataframe_pages: dataframe de nx2
                El dataframe debe contener una columna llamada página y otra llamada outlinks;
                donde outlinks contenga una lista de los outlinks por página
    """
    sparseMatrix = csr_matrix((len(dataframe_pages), len(dataframe_pages)),
                          dtype = np.float).toarray()
    newdf = pd.DataFrame(sparseMatrix, columns = dataframe_pages['pagina'], index=dataframe_pages['pagina'])
    count = 0
    for page, outlinks in zip(dataframe_pages['pagina'],dataframe_pages['outlinks']):
        score = 1/len(outlinks)
        dict_page = {page: np.zeros(len(dataframe_pages))}
        for outlink in outlinks:
            newdf[page][outlink] = score
    H = newdf.to_numpy()
    return H
```

```python
H37 = get_h_matrix(paginas_stanford)
```

```python
H37.shape
```

    (37, 37)

```python
from networkx.algorithms.link_analysis.pagerank_alg import google_matrix
```

```python
matrix_37 = google_matrix(G37, nodelist=paginas_stanford.pagina)
```

```python
A_37 = matrix_37.T
```

### Obtenemos PageRank por el Método de la Potencia adaptado a PR

```python
def power_method_pagerank(dataframe_pages, H, iteraciones):
    """
    Obtiene el pagerank
    :param[in] dataframe_pages: dataframe de nx2
                El dataframe debe contener una columna llamada página y otra llamada outlinks;
                donde outlinks contenga una lista de los outlinks por página
    :param[in] H: Matriz estocástica
    :param[in] iteraciones: número de iteraciones a realizar

    :return power_method: Un diccionario con la página como llave y el pagerank como valor
    """
    # Obtenemos nuestro vector inicial
    v = np.ones(len(dataframe_pages))
    v = v*1/len(v)

    # Iteramos
    H = np.linalg.matrix_power(H, iteraciones)
    v_n = H@v

    # Convertimos a dataframe
    print(v_n.shape)
    df = pd.DataFrame(v_n.T, columns=['pagerank'])

    df['nodo'] = dataframe_pages['pagina']

    df = df.set_index('nodo')

    power_method = df.pagerank.to_dict()

    return power_method
```

```python
power_method_dic = power_method_pagerank(paginas_stanford, A_37, iteraciones=100)
```

    (1, 37)

```python
power_method_dic
```

    {'1': 0.10093628803480034,
     '100193': 0.013330403674657317,
     '102355': 0.013330403674657317,
     '105318': 0.013330403674657317,
     '105730': 0.013330403674657317,
     '115926': 0.013330403674657317,
     '140864': 0.013330403674657317,
     '15409': 0.04695200491257369,
     '163550': 0.013330403674657317,
     '164599': 0.013330403674657317,
     '175799': 0.013330403674657317,
     '17794': 0.013330403674657317,
     '178642': 0.013330403674657317,
     '181714': 0.013330403674657317,
     '190453': 0.013330403674657317,
     '2': 0.33831392139907474,
     '204189': 0.013330403674657317,
     '204604': 0.013330403674657317,
     '210870': 0.013330403674657317,
     '213966': 0.013330403674657317,
     '225119': 0.013330403674657317,
     '241596': 0.013330403674657317,
     '243294': 0.013330403674657317,
     '246897': 0.013330403674657317,
     '251658': 0.013330403674657317,
     '25202': 0.013330403674657317,
     '252915': 0.013330403674657317,
     '280935': 0.013330403674657317,
     '3': 0.026783358060477675,
     '53625': 0.013330403674657317,
     '54582': 0.013330403674657317,
     '64930': 0.013330403674657317,
     '6548': 0.04695200491257369,
     '73764': 0.013330403674657317,
     '74361': 0.026819908766137674,
     '84477': 0.013330403674657317,
     '98628': 0.013330403674657317}

```python
# Redondeamos a 2 dígitos
power_method_round = {}

for node in power_method_dic:
    if node in power_method_dic:
        #set the node name as the key and the label as its value
        power_method_round[node] = round(power_method_dic[node],2)
```

```python
G37_pot = plot_node_graph(paginas_stanford, power_method=power_method_round, with_labels=True)
```

No se puede apreciar pero:

```python
MaxKey = max(power_method_round, key=power_method_round.get)
```

```python
print("la página con más rank es: {} con un rank de {}".format(MaxKey, power_method_round[MaxKey]))
```

    la página con más rank es: 2 con un rank de 0.34

### Obtenemos PageRank con el paquete `Networkx`

```python
PR_stanford = pagerank(G37, max_iter=1000)
```

```python
PR_round_stanford = {}

for node in PR_stanford:
    if node in PR_stanford:
        PR_round_stanford[node] = round(PR_stanford[node],2)
```

```python
MaxKeyPR = max(PR_round_stanford, key=PR_round_stanford.get)
```

```python
print("La página con más rank es: {} con un rank de {}".format(MaxKeyPR, PR_round_stanford[MaxKeyPR]))
```

    La página con más rank es: 2 con un rank de 0.34

## Obtenemos con el Método de la Potencia visto en clase

```python
def power_method_opt_pagerank(dataframe_pages, H, iteraciones):
    """
    Obtiene el pagerank
    :param[in] dataframe_pages: dataframe de nx2
                El dataframe debe contener una columna llamada página y otra llamada outlinks;
                donde outlinks contenga una lista de los outlinks por página
    :param[in] H: Matriz estocástica
    :param[in] iteraciones: número de iteraciones a realizar

    :return power_method: Un diccionario con la página como llave y el pagerank como valor
    """
    # Obtenemos nuestro vector inicial
    v = np.ones(len(dataframe_pages))
    v = v*1/len(v)

    n = len(v)
    q_k = v
    q_k = q_k.reshape(n,1)
    max_iter = iteraciones

    for k in range(max_iter):
        z_k = H@q_k
        q_k = z_k/np.linalg.norm(z_k)
        lambda_k = q_k.T @ H @ q_k

    # Convertimos a dataframe
    df = pd.DataFrame(q_k, columns=['pagerank'])

    df['nodo'] = dataframe_pages['pagina']

    df = df.set_index('nodo')

    power_method_opt = df.pagerank.to_dict()

    return power_method_opt
```

```python
power_method_opt_dictionary = power_method_opt_pagerank(paginas_stanford, A_37, iteraciones=100)
```

```python
MaxKey_pot_opt = max(power_method_opt_dictionary, key=power_method_opt_dictionary.get)
```

```python
print("La página con más rank es: {} con un rank de {}".format(MaxKey_pot_opt, power_method_opt_dictionary[MaxKey_pot_opt]))
```

    La página con más rank es: 2 con un rank de 0.9173865700385999

## Comparando resultados

```python
eigvector_comparison_37 = pd.DataFrame({"Página": PR_round_stanford.keys(),
                                    "NX PageRank": PR_round_stanford.values(),
                                    "Potencia PR": power_method_round.values(),
                                    "Potencia OPT-2020": power_method_opt_dictionary.values()}
                                    #"LinAlg": eigvec_eig[:,0].real}
                                    ).reset_index().iloc[:,1:]
eigvector_comparison_37
```

      Página
      NX PageRank
      Potencia PR
      Potencia OPT-2020

      0
      1
      0.10
      0.10
      0.273703

      1
      100193
      0.01
      0.01
      0.036147

      2
      102355
      0.01
      0.01
      0.036147

      3
      105318
      0.01
      0.01
      0.036147

      4
      105730
      0.01
      0.01
      0.036147

      5
      115926
      0.01
      0.01
      0.036147

      6
      140864
      0.01
      0.01
      0.036147

      7
      15409
      0.05
      0.05
      0.127317

      8
      163550
      0.01
      0.01
      0.036147

      9
      164599
      0.01
      0.01
      0.036147

      10
      175799
      0.01
      0.01
      0.036147

      11
      17794
      0.01
      0.01
      0.036147

      12
      178642
      0.01
      0.01
      0.036147

      13
      181714
      0.01
      0.01
      0.036147

      14
      190453
      0.01
      0.01
      0.036147

      15
      2
      0.34
      0.34
      0.917387

      16
      204189
      0.01
      0.01
      0.036147

      17
      204604
      0.01
      0.01
      0.036147

      18
      210870
      0.01
      0.01
      0.036147

      19
      213966
      0.01
      0.01
      0.036147

      20
      225119
      0.01
      0.01
      0.036147

      21
      241596
      0.01
      0.01
      0.036147

      22
      243294
      0.01
      0.01
      0.036147

      23
      246897
      0.01
      0.01
      0.036147

      24
      251658
      0.01
      0.01
      0.036147

      25
      25202
      0.01
      0.01
      0.036147

      26
      252915
      0.01
      0.01
      0.036147

      27
      280935
      0.01
      0.01
      0.036147

      28
      3
      0.03
      0.03
      0.072627

      29
      53625
      0.01
      0.01
      0.036147

      30
      54582
      0.01
      0.01
      0.036147

      31
      64930
      0.01
      0.01
      0.036147

      32
      6548
      0.05
      0.05
      0.127317

      33
      73764
      0.01
      0.01
      0.036147

      34
      74361
      0.03
      0.03
      0.072726

      35
      84477
      0.01
      0.01
      0.036147

      36
      98628
      0.01
      0.01
      0.036147

## Nota:

Para obtener los mismos resultados con tres algoritmos (NetworkX, método de la potencia adaptado, y método de la potencia visto en clase), fue necesario generar la matriz de Google a partir de la matriz de adyacencias. Los algoritmos del método de la potencia se aplicaron sobre la matriz de Google para obtener su eigenvector, de esta manera, los 3 algoritmos identificaron a la página "2" como la de mayor rank.

La matriz de Google considera el factor de amortiguación a los nodos que no tienen outlinks, lo que ayuda a que no se vean beneficiados por lo mismo. Sin aplicar la matriz de Google, los métodos de la potencia identificaban al nodo "74361" como el de mayor rank; se puede apreciar en el grafo que ese nodo no cuenta con outlinks, y sí cuenta con varios inlinks. Sin embargo, intuitivamente a partir del grafo, se puede apreciar que el nodo "2" debería ser el que tiene mayor Rank. La función de `pagerank` de NetworkX ya considera un factor de amortiguación por default, por lo que arrojaba un resultado distinto al de los métodos de la potencia sin ser aplicados a la matriz de Google.

Desgraciadamente, no fue posible utilizar el algoritmo desarrollado por el equipo durante la práctia 1.2 para obtener los eigenvalores a partir de la factorización QR para la matriz de adyacencias de 37 páginas.

## Obtenemos con el algoritmo QR implementado en la Práctica 1.2

```python
eigval37, eigvec37 = Eig_Householder(A_37)
```

## Referencias

- [Google Matrix](https://networkx.org/documentation/stable//reference/algorithms/generated/networkx.algorithms.link_analysis.pagerank_alg.google_matrix.html#networkx.algorithms.link_analysis.pagerank_alg.google_matrix)
- [Código QR](https://github.com/optimizacion-2020-2-gh-classroom/practica-1-segunda-parte-graph-caroacostatovany)
- [PageRank](https://networkx.org/documentation/stable//reference/algorithms/generated/networkx.algorithms.link_analysis.pagerank_alg.pagerank.html)
- [Etiquetas para los nodos](https://stackoverflow.com/questions/14665767/networkx-specific-nodes-labeling)
- [set de datos de Stanford](http://snap.stanford.edu/data/web-Stanford.html)


```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from ipywidgets import interact, fixed
from matplotlib import cm
plt.rcParams['font.size'] = 14
plt.rcParams['axes.spines.right'] = False
plt.rcParams['ytick.right'] = False
plt.rcParams['axes.spines.top'] = False
plt.rcParams['xtick.top'] = False

# Get n grayscale colors
def getGrayColors(n):
    colors = [];
    for i in range(1, n+1):
        colors.append(i/(n+1.)*np.ones(3))
    return colors
```

### Example case

Suppose we have observed spike responses $y_i$ to some one-dimensional stimulus $x_i$, $i \in 1,\dots, N$. Furthermore, suppose that the cell exhibited some thresholding mechanism: hardly any spikes were observed for stimulus values below a certain threshold, whereas the response grew roughly linearly above the threshold. We would like to describe the cell's stimulus response mapping using a simple model, and one alternative is a soft-rectifying function:
\begin{equation}
 \hat{y_i} = f(z_i) =
 \begin{cases}
  \exp(z_i),& \text{if  } z_i \leq 0\\
  z_i + 1,& \text{if  } z_i > 0\\
 \end{cases}
\end{equation}
where $\hat{y_i}$ is the predicted output, $z_i=w_0+w_1x_i$, $w_0$ the threshold, and $w_1$ the slope. However, it is often simpler to work with vector notations, in such cases, we define $z_i$ as a dot product between a parameter vector and a stimulus vector $\mathbf{w}^T\mathbf{x}_i$, where $\mathbf{w}=[w_0,\; w_1]$ and $\mathbf{x}_i=[1; x_i]$.

Below, we generate simulated data from the proposed model, and we illustrate what the proposed model looks like for various choices of $w_0$ and $w_1$.

```python
# The soft-rectifying function (f), also known as the inverse of the link function for generalized linear models
def invLinkFun(X, w):
    z = np.dot(X, w)
    mu = np.zeros(z.shape)
    mu[z<0] = np.exp(z[z<0])
    mu[z>=0] = z[z>=0] + 1
    return mu

# Parameters
n = 200;           # Number of data points
wTrue = [-1, 0.5]  # w0 and w1 for the simulated data

# Generate example data
x = np.random.rand(n)*30 - 15
x = np.sort(x)
X = np.vstack([-np.ones(n), x]).T
mu = invLinkFun(X, wTrue)
y = np.random.poisson(mu)

# Callback plotting function for interactive plots
def plot_fun(x, y, w0, w1):
    fig = plt.figure(figsize=(7.5, 5))
    ax = fig.add_subplot(1, 1, 1)
    ax.plot(x, y, 'k.', label='data')
    wTmp = np.array([w0, w1])
    muTmp = invLinkFun(X, wTmp)
    ax.plot(x, muTmp, '-', color=0.5*np.ones(3))
    ax.set_ylabel('Spike count')
    ax.set_xlabel('x')
    ax.set_xticks([-10, 0, 10]);
    ax.set_ylim([-1, 11]);

interact(plot_fun, x=fixed(x), y=fixed(y), w0=(-5., 5.), w1=(-5., 5.));
```

    interactive(children=(FloatSlider(value=0.0, description='w0', max=5.0, min=-5.0), FloatSlider(value=0.0, desc…

### Objective functions

A manual search for good parameter values can be quite tricky and cumbersome, especially if there are more than just two model parameters. An automatic way would be a lot simpler, but for that we need some measure of how "good" a specific selection of model parameters are. One natural way of quantifying the "goodness" of parameters is through an objective function which scores each parameter combination. It might seen tricky to come up with such a function, but one common approach is to define it to be a likelihood function. The idea being that the likelihood function tells you how likely you are to observe the current data given some model parameters. For example, if we assume that the observed spike counts come from a Poisson distribution whose mean is given by our soft-rectifying function, then the probability $p$ of observing a specific data point ($x_i, y_i$) is given by:
\begin{equation}
 p(y_i|x_i, w_0, w_1) = \frac{f(z_i)^{y_i} \exp(-f(z_i))}{y_i!}.
\end{equation}
The likelihood function $l$ is then defined as the probability of observing all data points, where independence is essentially always assumed to simplify the expression into a product:
\begin{equation}
 l(w_0, w_1) = \prod_i^N \frac{f(z_i)^{y_i} \exp(-f(z_i))}{y_i!}.
\end{equation}
The likelihood function is, however, often cumbersome to work with, and people tend to use the log of the likelihood function instead. Logarithms turns products into sums, and the log-likelihood function $ll$ thus has the form:
\begin{equation}
 ll(w_0, w_1) = \sum_i^N y_i\log(f(z_i)) -f(z_i) -\log(y_i!).
\end{equation}
Finally, we note that the last term can be ignored (it does not depend on our model parameters), whereupon we are left with:
\begin{equation}
 ll(w_0, w_1) = \sum_i^N y_i\log(f(z_i)) -f(z_i),
\end{equation}
which also corresponds to the objective function for **Poisson regression**.

Equipped with a function that quantifies the "goodness" of parameter choices, we can now select parameter values that maximizes the log-likelihood function.

```python
# Poisson regression, log-likelihood
logLikFun = lambda X, w, y: np.sum(y*np.log(invLinkFun(X, w))-invLinkFun(X, w))
```

##### Exampel 1, find w0 given w1

```python
# Example parameters
w0Ex = np.array([wTrue[0]-4, wTrue[0], wTrue[0]+4])
w0Vals = np.linspace(wTrue[0]-5, wTrue[0]+5, 101)

# Log-likelihood function over the whole interval
ll = np.empty_like(w0Vals)
for i, w0 in enumerate(w0Vals):
    wTmp = np.array([w0, wTrue[1]])
    ll[i] = logLikFun(X, wTmp, y)

# Log-likelihood values for our example parameters
llEx = np.empty_like(w0Ex)
for i, w0 in enumerate(w0Ex):
    wTmp = np.array([w0, wTrue[1]])
    llEx[i] = logLikFun(X, wTmp, y)

# Plotting
grayColors = getGrayColors(3)
fig = plt.figure(figsize=(15, 5))
# Log-likelihood
ax = fig.add_subplot(1, 2, 1)
ax.plot(w0Vals, ll, 'k-')
for i in range(w0Ex.size):
    ax.plot(w0Ex[i], llEx[i], 'o', ms=10, color=grayColors[i])
ax.set_xlabel('$w_0$')
ax.set_ylabel('Log-likelihood')
# Data and example response function mappings
ax = fig.add_subplot(1, 2, 2)
ax.plot(x, y, 'k.', label='data')
for i in range(w0Ex.size):
    wTmp = np.array([w0Ex[i], wTrue[1]])
    pTmp = invLinkFun(X, wTmp)
    ax.plot(x, pTmp, '-', color=grayColors[i])
ax.set_ylabel('Spike count')
ax.set_xlabel('x')
ax.set_xticks([-10, 0, 10]);
```

##### Example 2, find both w0 and w1

```python
# Example parameters
wEx = np.array([[wTrue[0]-0.9, wTrue[0], wTrue[0]+0.9], [wTrue[1]+0.3, wTrue[1], wTrue[1]-0.3]])

# Get w0 and w1 combinations over a grid
nGrid = 51
W0, W1 = np.meshgrid(np.linspace(wTrue[0]-1, wTrue[0]+1, nGrid), np.linspace(wTrue[1]-0.5, wTrue[1]+0.5, nGrid))

# Get the log-likelihood for each parameter combination
llVals = np.zeros([nGrid, nGrid])
for i in range(nGrid):
    for j in range(nGrid):
        wTmp = np.array([W0[i, j], W1[i, j]])
        llVals[i, j] = logLikFun(X, wTmp, y)

# Plotting
grayColors = getGrayColors(3)
fig = plt.figure(figsize=(15, 5))
# Log-lieklihood and example parameters
ax = plt.subplot(1, 2, 1)
contourHandle = ax.contourf(W0, W1, llVals, 50, cmap=cm.coolwarm)
for i in range(wEx.shape[1]):
    ax.plot(wEx[0, i], wEx[1, i], 'o', ms=10, color=grayColors[i])
ax.set_xlabel('$w_0$')
ax.set_ylabel('$w_1$');
cBarHandle = plt.colorbar(contourHandle)
cBarHandle.set_label('Log-likelihood')
# Data and example response function mappings
ax = fig.add_subplot(1, 2, 2)
ax.plot(x, y, 'k.', label='data')
for i in range(wEx.shape[1]):
    wTmp = wEx[:, i]
    pTmp = invLinkFun(X, wTmp)
    ax.plot(x, pTmp, '-', color=grayColors[i])
ax.set_ylabel('Spike count')
ax.set_xlabel('x')
ax.set_xticks([-10, 0, 10]);
```

### Optimization

At first sight, it might look like the objective function only slightly simplifies our problem of finding good parameter values: finding the maxima is easy when you see the objective function, but visualizing the function for more than three parameters is extremely difficult. Luckily, however, there is a whole optimization field devoted to finding the maxima or minima of functions using methods that don't require you to visualize the objective function. However, before continuing, we will modify our objective functions once more by multiplying it with -1. This is just to turn the maximization problem into a minimization problem, which is usually the default notation. Finding the best parameters values thus corresponds to minimizing the negative log-likelihood $nll$:

\begin{equation}
 nll(w_0, w_1) = \sum_i^N f(z_i) - y_i\log(f(z_i)).
\end{equation}

```python
# Poisson regression, negative log-likelihood
negLogLikFun = lambda X, w, y: np.sum(invLinkFun(X, w) - y*np.log(invLinkFun(X, w)))
```

### Gradient

Several commonly used optimization methods utilize gradients. The gradient of a function is by definition a vector consisting of the partial derivatives with respect to each parameter. For our example from above, the gradient thus becomes:
\begin{equation}
 \nabla nll = \frac{\partial nll}{\partial \mathbf{w}} = \left[ \frac{\partial nll}{\partial w_0}, \frac{\partial nll}{\partial w_1} \right],
\end{equation}
where $\nabla$ denotes the linear nabla operator. The fact that $\nabla$ is linear means that we can obtain the gradient of the negative log-likelihood function by just summing up the gradients of each term in the sum:
\begin{align}
 \nabla nll &= \sum_i^N \frac{\partial f(z_i) - y_i\log(f(z_i))}{\partial \mathbf{w}}, \\
 \frac{\partial f(z_i) - y_i\log(f(z_i))}{\partial \mathbf{w}} &=
 \begin{cases}
  (\exp(z_i) - y_i)\mathbf{x}_i,& \text{if  } z_i \leq 0\\
  \left( 1-\frac{y_i}{z_i+1} \right) \mathbf{x}_i,& \text{if  } z_i > 0\\
 \end{cases}.
\end{align}
We can thus compute the gradient for any parameter values of our choice, but the real benefit of this possibility only becomes apparent when considering what the gradient really denotes. The partial derivative of any parameter tells us how quickly the function increases when that parameter is changed slightly, and consequently the gradient, which is the vector containing all partial derivatives, indicates in which direction in parameter space that the function value increases the most.

```python
# Poisson regression, gradient of the negative log-likelihood
def negLogLikDerFun(X, w, y):
    z = np.dot(X, w)
    der = np.dot(np.exp(z[z<0])-y[z<0], X[z<0, :])
    der += np.dot(1-y[z>=0]/(z[z>=0]+1), X[z>=0, :])
    return der
```

##### Example 3, the gradient of the negative log-likelihood function

```python
# Use a lower resolution for gradient evaluations
nGrid = 21
W0q, W1q = np.meshgrid(np.linspace(wTrue[0]-1, wTrue[0]+1, nGrid), np.linspace(wTrue[1]-0.5, wTrue[1]+0.5, nGrid))

# Get the gradient for each combination
nllVals = np.zeros([nGrid, nGrid])
W0der = np.zeros([nGrid, nGrid])
W1der = np.zeros([nGrid, nGrid])
for i in range(nGrid):
    for j in range(nGrid):
        wTmp = np.array([W0q[i, j], W1q[i, j]])
        derTmp =  negLogLikDerFun(X, wTmp, y)
        W0der[i, j] = derTmp[0]
        W1der[i, j] = derTmp[1]

# Normalize to unit length
W0derNorm = W0der / np.sqrt(W0der**2. + W1der**2.)
W1derNorm = W1der / np.sqrt(W0der**2. + W1der**2.)

# Plotting
grayColors = getGrayColors(3)
fig = plt.figure(figsize=(25, 7))
ax = plt.subplot(1, 1, 1)
ax.set_aspect('equal')
# Negative log-likelihood
contourHandle = ax.contourf(W0, W1, -llVals, 50, cmap=cm.coolwarm)
# Gradient
ax.quiver(W0q, W1q, W0derNorm, W1derNorm, angles='xy', scale_units='xy', scale=30, width=2e-3)
# Axes settings
ax.set_xlabel('$w_0$')
ax.set_ylabel('$w_1$');
cBarHandle = plt.colorbar(contourHandle)
cBarHandle.set_label('negative log-likelihood')
```

### Gradient descent

A stupidly simple idea for finding the minima of the negative log-likelihood function would then be to start from some parameters $\mathbf{w}_\mathrm{init}$ and then just iteratively move in the opposite direction to the gradient. That is, we evaluate the gradient for our current parameter guess and then we update our guess by subtracting the gradient times a constant $\eta$. More specifically, at each step $k$ of our iterative algorithm, we update our parameters as:
\begin{equation}
 \mathbf{w}_{k+1} = \mathbf{w}_k - \eta \nabla nll(\mathbf{w}_k).
\end{equation}
For sufficiently small $\eta$, this will guarantee that we backtrack along the gradients all the way to the minima of the negative log-likelihood function, as illustrated below.

```python
# Gradient descent parameters
eta = 2e-4
nIterations = 25
wInit = np.array([-1.8, 0.85])
wUpdates = np.zeros([nIterations, 2])
wUpdates[0, :] = wInit
nllVals = np.zeros(nIterations)
nllVals[0] = negLogLikFun(X, wUpdates[0, :], y)

# Gradient descent loop
for i in range(1, nIterations):
    wUpdates[i, :] = wUpdates[i-1, :] - eta*negLogLikDerFun(X, wUpdates[i-1, :], y)
    nllVals[i] = negLogLikFun(X, wUpdates[i, :], y)

# Plotting
grayColors = getGrayColors(3)
fig = plt.figure(figsize=(25, 7))
ax = plt.subplot(1, 1, 1)
ax.set_aspect('equal')
# Negative log-likelihood
contourHandle = ax.contourf(W0, W1, -llVals, 50, cmap=cm.coolwarm)
# Gradient
ax.quiver(W0q, W1q, W0derNorm, W1derNorm, angles='xy', scale_units='xy', scale=30, width=2e-3)
# Gradient descent progress
ax.plot(wUpdates[:, 0], wUpdates[:, 1], '-o', ms=6, color='white', lw=3)
# Axes settings
ax.set_xlabel('$w_0$')
ax.set_ylabel('$w_1$');
cBarHandle = plt.colorbar(contourHandle)
cBarHandle.set_label('negative log-likelihood')
```

### Backtracking line search

Finding a suitable $\eta$ value for the gradient descent approach might not be trivial: a too small value of $\eta$ will hamper progress, whereas a too large value will cause the algorithm to diverge. It would therefore be very convenient to have an adaptive method that automatically adjusted $\eta$ to take large but not too large steps on each iteration of gradient descent. One common method for convex problems is to use something called backtracking line search. The idea being that we start of with a large $\eta$ value and then successively decrease it by a factor $\beta$. This continues until the observed decrease of the objective function is within a factor $\alpha$ of what would be expected for a linear approximation of the objective function. This is illustrated below for a one-dimensional case (find $w_0$ given $w_1$) by showing the real negative log-likelihood and the linear approximation around our current guess of $w_0$.

```python
w0Init = w0Vals.min()
nllInit = negLogLikFun(X, np.array([w0Init, wTrue[1]]), y)
w0InitDer = negLogLikDerFun(X, np.array([w0Init, wTrue[1]]), y)[0]
offset = nllInit - w0Init*w0InitDer;

# Calculate real and expected decreases
realNllDecrease =  -ll - nllInit
expectedNllDecrease = offset  + w0InitDer*w0Vals - nllInit

# Plotting
fig = plt.figure(figsize=(15, 5))
# Log-likelihood
ax = fig.add_subplot(1, 2, 1)
ax.plot(w0Vals, -ll, 'k-', label='Real nll')
ax.plot(w0Vals, offset  + w0InitDer*w0Vals, 'k:', label='linear approx.')
ax.plot(w0Init, nllInit, 'o', ms=10, color='k')
ax.set_xlabel('$w_0$')
ax.set_ylabel('Negative log-likelihood')
ax.legend()
# Log-likelihood decrease
ax = fig.add_subplot(1, 2, 2)
ax.plot(w0Vals[1:], realNllDecrease[1:] / expectedNllDecrease[1:], 'k-')
ax.set_xlabel('$w_0$')
ax.set_ylabel('nll / linear approx.');
```

The linear approximation will always lie below the real negative log-likelihood for convex objective functions, but it provides a good approximation close to $w_0$. The idea of backtracking line search is thus to take the largest step possible while still ensuring that the linear approximation is good enough. Goodness is measured by how close the real objective function is to the linear approximation, and the acceptance criteria is set by $\alpha$. $\alpha=0.5$ thus means that we want the largest step size for which the decrease observed for the real objective function is at least half of what would be predicted by the linear approximation. However, we don't want to spend time solving this line search problem exactly, rather, we start of with a high value of $\eta$ and decrease it successively by a factor $\beta$ until the goodness criteria is met. That is, we decrease $\eta$ to $\eta\beta$ until
\begin{equation}
 nll(\mathbf{w}_{k}-\eta \nabla nll(\mathbf{w}_k)) > nll(\mathbf{w}_k) - \alpha \eta \nabla nll(\mathbf{w}_k)^T\nabla nll(\mathbf{w}_k),
\end{equation}
where $\eta \nabla nll(\mathbf{w}_k)^T\nabla nll(\mathbf{w}_k)$ is the predicted decrease for our linear approximation. Standard values for $\alpha$ and $\beta$ are 0.5 and 0.8, respectively. The illustration below use these standard values to show how gradient descent progresses for our example problem of finding both $w_0$ and $w_1$ using a backtracking line search implementation.

```python
# Gradient descent parameters
eta = 0.1
alpha = 0.5
beta = 0.8
wUpdates = np.zeros([nIterations, 2])
wUpdates[0, :] = wInit
nllVals = np.zeros(nIterations)
nllVals[0] = negLogLikFun(X, wUpdates[0, :], y)

# Gradient descent with backtracking
for i in range(1, nIterations):
    wUpdates[i, :] = wUpdates[i-1, :]
    delta = negLogLikDerFun(X, wUpdates[i, :], y)
    tooLarge = True
    # Backtracking loop
    while tooLarge:
        wUpdates[i, :] -= eta*delta
        nllTmp = negLogLikFun(X, wUpdates[i, :], y)
        if nllTmp > nllVals[i-1] - alpha*eta*np.dot(delta, delta):
            wUpdates[i, :] += eta*delta
            eta *= beta
        else:
            tooLarge = False
            nllVals[i] = nllTmp

# Plotting
grayColors = getGrayColors(3)
fig = plt.figure(figsize=(25, 7))
ax = plt.subplot(1, 1, 1)
ax.set_aspect('equal')
# Negative log-likelihood
contourHandle = ax.contourf(W0, W1, -llVals, 50, cmap=cm.coolwarm)
# Gradient
ax.quiver(W0q, W1q, W0derNorm, W1derNorm, angles='xy', scale_units='xy', scale=30, width=2e-3)
# Gradient descent progress
ax.plot(wUpdates[:, 0], wUpdates[:, 1], '-o', ms=6, color='white', lw=3)
# Axes settings
ax.set_xlabel('$w_0$')
ax.set_ylabel('$w_1$');
cBarHandle = plt.colorbar(contourHandle)
cBarHandle.set_label('negative log-likelihood')
```

### Acceleration

Gradient descent can be made to converge faster using a method called Nesterov accelerated gradient, named after Yurii Nesterov that introduced it. The basic idea behind Nesterov's accelerated gradient method is to first make a move in the direction of the previous parameter change, and then evaluate the gradient at this intermediate position $v$ in order to let the standard gradient descent rule make a correction step. That is, at each iteration, we first determine intermediate parameter values $v$, and then take a step in the direction of the negative gradient evaluated at $v$:
\begin{align}
 \mathbf{v}_{k+1} &= \mathbf{w}_k + \frac{k-2}{k+1}(\mathbf{w}_k - \mathbf{w}_{k-1}), \\
 \mathbf{w}_{k+1} &= \mathbf{v}_{k+1} - \eta \nabla nll(\mathbf{v}_{k+1}).
\end{align}
We thus carry with us some momentum of previous weight updates, and the coefficient $\frac{k-2}{k+1}$ tells us to use more momentum later on (it converged to 1 for large $k$). However, the momentum term gives rise to oscillations in the objective function value, and the Nesterov's accelerated gradient is therefore not a strict descent method. Acceleration can nonetheless be combined with backtracking line search, as for the standard gradient descent method, but now from the intermediate parameter values $v$.

```python
# Gradient descent parameters
eta = 0.1
wUpdates = np.zeros([nIterations, 2])
wUpdates[0:2, :] = [-1.8, 0.85]
nllVals = np.zeros(nIterations)
nllVals[0:2] = negLogLikFun(X, wUpdates[0, :], y)

# Nesterov accelerated gradient with backtracking
for i in range(2, nIterations):
    v = wUpdates[i-1, :] + (i-2.)/(i+1)*(wUpdates[i-1, :]-wUpdates[i-2, :])
    delta = negLogLikDerFun(X, v, y)
    nllTmp = negLogLikFun(X, v , y)
    tooLarge = True
    # Backtracking loop
    while tooLarge:
        wUpdates[i, :] = v - eta*delta
        nllNew = negLogLikFun(X, wUpdates[i, :] , y)
        if nllNew > nllTmp - eta*np.dot(delta, delta)/2.:
            wUpdates[i, :] = wUpdates[i-1, :]
            eta *= beta
        else:
            tooLarge = False
            nllVals[i] = nllNew

# Plotting
grayColors = getGrayColors(3)
fig = plt.figure(figsize=(25, 7))
ax = plt.subplot(1, 1, 1)
ax.set_aspect('equal')
# Negative log-likelihood
contourHandle = ax.contourf(W0, W1, -llVals, 50, cmap=cm.coolwarm)
# Gradient
ax.quiver(W0q, W1q, W0derNorm, W1derNorm, angles='xy', scale_units='xy', scale=30, width=2e-3)
# Gradient descent progress
ax.plot(wUpdates[:, 0], wUpdates[:, 1], '-o', ms=6, color='white', lw=3)
# Axes settings
ax.set_xlabel('$w_0$')
ax.set_ylabel('$w_1$');
cBarHandle = plt.colorbar(contourHandle)
cBarHandle.set_label('negative log-likelihood')
```

ConvexOptimization/Part 1, basic ideas and gradient descent.ipynb


# QUBO Generator

----

###### Note:
If you viewing this notebook directly at github.com, the LaTeX formulas may not be rendered correctly.
Thus, we highly recommend to use [nbviewer](https://nbviewer.jupyter.org/) for this purpose.

You can view this notebook directly at: https://nbviewer.jupyter.org/github/quantum-circle/qc_qubogen/blob/master/examples/example.ipynb

---

## A Minimal Example
Let's assume that we are given the following combinatorial optimization problem:

$$\min _{x} f(x) = 2x + 3 \; , $$

where $x \in \{0, 1\}$, i.e. a binary variable. Our goal is now to reformulate the above objective function as a QUBO model, i.e. a symmetric (or at least upper diagonal) matrix $\mathbf{Q} $ and a constant offset term $c$, that can be described as:

$$\min _{\mathbf{x}} f(\mathbf{x}) = \mathbf{x}^T \mathbf{Q} \mathbf{x} + c$$

Note that we used $\mathbf{x}$ here to denote a vector, yet in our example $\mathbf{x}$ consists of only one element $x_i$.

To accomplish this task, we are going to use `qc_qubogen.QUBOModel` built-in functionality:
1.   Define the objective function using sympy's symbolic variables to represent the binary variables $x_i$. This is be done by creating an instance of `sympy.Symbol`, which allows to use an arbitrary variable name (*str*), e.g. "x_0".
2. We can then create an instance of `qc_qubogen.QUBOModel` and pass the previoulsy created objective function as first argument.
3. Once the QUBO model is created, we can convert it to matrix form using the `qc_qubogen.QUBOModel().model_to_qubo()` method. Note that this method will not return the constant offset term $c$ by default, since it will not affect the optimization procedure. However, we can set the argument `return_offset=True`, to return both,  the QUBO matrix $\mathbf{Q} $ as $2-$dim `numpdy.ndarray` **and** the constant $c$ as `float`.

```python
from sympy import Symbol
from qc_qubogen import QUBOModel

# 1. Create a symbolic binary variable with sympy.Symbol
x_0 = Symbol("x_0")

# 2. Define the function to be minimized
objective = 2*x_0 + 3

# 3. Initialize the `QUBOModel` using the created objective function
model = QUBOModel(objective)

# 4. Convert the problem and return the QUBO matrix and offset constant
qubo_matrix, qubo_offset = model.model_to_qubo(return_offset=True)

print(f"QUBO matrix: Q = {qubo_matrix}")
print(f"QUBO offset: c = {qubo_offset}")
```

    QUBO matrix: Q = [[2.]]
    QUBO offset: c = 3.0

## Another Illustrative Example

To illustrate a slightly more advanced example, we choose to solve the optimization problem given by Glover et. al.$^{[1]}$:

$$ f(\mathbf{x}) = -5x_0 -3x_1 -8x_2 -6x_3 + 4x_0x_1 + 8x_0x_2 + 2x_1x_2 + 10x_2x_3 \; ,$$

which will be converted to the QUBO problem:

$$ f(\mathbf{x})  = \left(\begin{matrix} x_0 & x_1 & x_3 & x_4 \end{matrix}\right) \left(\begin{matrix} -5 & 2 & 4 & 0 \\ 2 & -3 & 1 & 0 \\ 4 & 1 & -8 & 5 \\ 0 & 0 & 5 & -6 \end{matrix} \right) \left(\begin{matrix} x_0\\ x_1 \\ x_2 \\ x_3 \end{matrix}\right) $$

Note that we omit the constant offset $c$ here, since $c = 0$.

Here, we're going to use the helper function `qc_qubogen.get_symbolic_binary_variables(n=4)` , which generates a list of $4$ `sympy.Symbol` symbolic binary variables.

---

[1] Glover, F., Kochenberger, G. and Du, Yu. Quantum Bridge Analytics I: a tutorial on formulating and using QUBO models. *Available at:* https://arxiv.org/pdf/1811.11538.pdf

```python
from qc_qubogen import QUBOModel, get_symbolic_binary_variables

x = get_symbolic_binary_variables(n=4)

objective = - 5*x[0] - 3*x[1] - 8*x[2] - 6*x[3] + 4*x[0]*x[1] + 8*x[0]*x[2] + 2*x[1]*x[2] + 10*x[2]*x[3]

model = QUBOModel(objective)
qubo_matrix, qubo_offset = model.model_to_qubo(return_offset=True)

print(f"QUBO matrix: Q = \n{qubo_matrix}")
```

    QUBO matrix: Q =
    [[-5.  2.  4.  0.]
     [ 2. -3.  1.  0.]
     [ 4.  1. -8.  5.]
     [ 0.  0.  5. -6.]]