# Evolutionary Game Theory

In the previous chapter, we considered the case of fitness being independant of the distribution of the whole population (the rates of increase of 1 type just depended on the quantity of that type). That was a specific case of Evolutionary game theory which considers **frequency dependent selection**.

---

## Frequency dependent selection

[Video](https://youtu.be/PFtwwrcouXY?list=PLnC5h3PY-znxMsG0TRYGOyrnEO-QhVwLb)

Consider. Let $x=(x_1, x_2)$ correspond to the population sizes of both types. The fitness functions are given by:

$$f_1(x)\qquad f_2(x)$$

As before we ensure a constant population size: $x_1 + x_2 = 1$. We have:

$$
\frac{dx_1}{dt}=x_1(f_1(x)-\phi) \qquad \frac{dx_2}{dt}=x_2(f_2(x)-\phi)
$$

we again have:

$$
\frac{dx_1}{dt} + \frac{dx_2}{dt}=x_1(f_1(x)-\phi) + x_2(f_2(x)-\phi)=0
$$

So $\phi=x_1f_1(x)+x_2f_2(x)$ (the average fitness).

We can substitute: $x_2=1-x_1$ to obtain:

$$
\frac{dx_1}{dt}=x_1(f_1(x)-x_1f_1(x)-x_2f_2(x))=x_1((1-x_1)f_1(x)-(1-x_1)f_2(x))
$$

$$
\frac{dx_1}{dt}=x_1(1-x_1)(f_1(x)-f_2(x))
$$

We see that we have 3 equilibria:

- $x_1=0$
- $x_1=1$
- Whatever distribution of $x$ that ensures: $f_1(x)=f_2(x)$

---

## Evolutionary Game Theory

Now we will consider potential differences of these equilibria. First we will return to considering Normal form games:

$$
A =
\begin{pmatrix}
a & b\\
c & d
\end{pmatrix}
$$

Evolutionary Game theory assigns strategies as types in a population, and indivividuals randomly encounter other individuals and play their corresponding strategy. The matrix $A$ correspods to the utility of a row player in a game where the row player is a given individual and the column player is the population.

This gives:

$$f_1=ax_1+bx_2\qquad f_2=cx_1+dx_2$$

or equivalently:

$$f=Ax\qquad \phi=fx$$

thus we have the same equation as before but in matrix notation:

$$\frac{dx}{dt}=x(f-\phi)$$

---

In this case, the 3 stable distributions correspond to:

- An entire population playing the first strategy;
- An entire population playing the second strategy;
- A population playing a mixture of first and second (such that there is indifference between the fitness).

---

We now consider the utility of a stable population in a **mutated** population.

---

## Mutated population

Given a strategy vector $x=(x_1, x_2)$, some $\epsilon>0$ and another strategy $y=(y_1, y_2)$, the post entry population $x_{\epsilon}$ is given by:

$$
x_{\epsilon} = (x_1 + \epsilon(y_1 - x_1), x_2 + \epsilon(y_2 - x_2))
$$

---

## Evolutionary Stable Strategies

[Video](https://youtu.be/lbzcToUM9ic?list=PLnC5h3PY-znxMsG0TRYGOyrnEO-QhVwLb)

Given a stable population distribution, $x$ it represents an **Evolutionary Stable Strategy** (ESS) if and only if there exists $\bar\epsilon>0$:

$$u(x, x_{\epsilon})>u(y, x_{\epsilon})\text{ for all }0<\epsilon<\bar\epsilon, y$$

where $u(x, y)$ corresponds to the fitness of strategy $x$ in population $y$ which is given by:

$$xAy^T$$

---

For the first type to be an ESS this corresponds to:

$$a(1-\epsilon)+b\epsilon > c(1-\epsilon) + d\epsilon$$

For small values of $\epsilon$ this corresponds to:

$$a>c$$

However if $a=c$, this corresponds to:

$$b>d$$

Thus the first strategy is an ESS (ie resists invasion) iff one of the two hold:

1. $a > c$
2. $a=c$ and $b > d$

```python
import numpy as np
import nashpy as nash
import matplotlib.pyplot as plt
```

The case of $a>c$:

```python
A = np.array([[4, 3], [2, 1]])
game = nash.Game(A)
timepoints = np.linspace(0, 10, 1000)
epsilon = 10 ** -1
xs = game.replicator_dynamics(
    y0=[1 - epsilon, epsilon],
    timepoints=timepoints,
)
plt.plot(xs);
```

The case of $a=c$ and $b>d$:

```python
A = np.array([[4, 3], [4, 1]])
game = nash.Game(A)
xs = game.replicator_dynamics(
    y0=[1 - epsilon, epsilon],
    timepoints=timepoints,
)
plt.plot(xs);
```

$a=c$ and $b < d$:

```python
A = np.array([[4, 3], [4, 5]])
game = nash.Game(A)
xs = game.replicator_dynamics(
    y0=[1 - epsilon, epsilon],
    timepoints=timepoints,
)
plt.plot(xs);
```

$a < c$:

```python
A = np.array([[1, 3], [4, 1]])
game = nash.Game(A)
xs = game.replicator_dynamics(
    y0=[1 - epsilon, epsilon],
    timepoints=timepoints,
)
plt.plot(xs);
```

We see in the above case that the population seems to stabilise at a mixed strategy. This leads to the general definition of the fitness of a mixed strategy: $x=(x_1, x_2)$:

$$u(x,x) = x_1f_1(x)+x_2f_2(x)$$

---

## General condition for ESS

[Video](https://youtu.be/zkhInay5xQc?list=PLnC5h3PY-znxMsG0TRYGOyrnEO-QhVwLb)

If $x$ is an ESS, then for all $y\ne x$, either:

1. $u(x,x)>u(y,x)$
2. $u(x,x)=u(y,x)$ and $u(x,y)>u(y,y)$
---

Conversely, if either (1) or (2) holds for all $y\ne x$ then $x$ is an ESS.

---

### Proof

---

If $x$ is an ESS, then by definition:

$$u(x,x_{\epsilon})>u(y,x_{\epsilon})$$

which corresponds to:

$$(1-\epsilon)u(x,x)+\epsilon u(x,y)>(1-\epsilon)u(y,x)+\epsilon u(y,y)$$

- If condition 1 of the theorem holds then the above inequality can be satisfied for \\(\epsilon\\) sufficiently small. If condition 2 holds then the inequality is satisfied.
- Conversely:

    - If $u(x,x) < u(y,x)$ then we can find $\epsilon$ sufficiently small such that the inequality is violated.

    - If $u(x, x) = u(y,x)$ and $u(x,y) \leq u(y,y)$ then the inequality is violated.

This result gives us an efficient way of computing ESS. The first condition is in fact almost a condition for Nash Equilibrium (with a strict inequality), the second is thus a stronger condition that removes certain Nash equilibria from consideration. This becomes particularly relevant when considering Nash equilibrium in mixed strategies.

To find ESS in a pairwise context population game we:

1. Write down the associated two-player game $(A, A^T)\in{\mathbb{R}^{m\times n}}^2$;
2. Identify all symmetric Nash equilibria of the game;
3. Test the Nash equilibrium against the two conditions of the above Theorem.

Let us apply it to the one example that seemed to stabilise at a mixed strategy:

$$
A =\begin{pmatrix}
1 & 3\\
4 & 1
\end{pmatrix}
$$

```python
import nashpy as nash
game = nash.Game(A, A.transpose())
list(game.support_enumeration())
```

    [(array([1., 0.]), array([0., 1.])),
     (array([0., 1.]), array([1., 0.])),
     (array([0.4, 0.6]), array([0.4, 0.6]))]

Looking at $x=(.4, .6)$ (which is the only symmetric nash equilibrium), we have

$$u(x, x)=u(y, x)$$

and (recall $y_1 + y_2 = 1$):

$$
u(x, y)=2.8y_1 + 1.8y_2=2.8y_1 + 1.8(1-y_1)=y_1+1.8
$$

\begin{align}
u(y, y)&=y_1^2+3y_1y_2+4y_1y_2+y_2^2\\
       &=y_1^2+7y_1-7y_1^2+1 - 2y_1 + y_1^2\\
       &=5y_1-5y_1^2+1
\end{align}

Thus:

$$u(x, y) - u(y, y) = -4y_1+5y_1^2+.8 = 5(y_1 - .4)^2$$

however $y_1\ne.4$ thus $x=(.4, .6)$ is an ESS.

Here is some code to verify the above calculations:

```python
import sympy as sym
sym.init_printing()
A = sym.Matrix(A)
y_1, y_2 = sym.symbols("y_1, y_2")
y = sym.Matrix([y_1, y_2])
A, y
```

```python
rhs = sym.expand((y.transpose() * A * y)[0].subs({y_2: 1 - y_1}))
rhs
```

```python
lhs = sym.expand((sym.Matrix([[.4, .6]]) * A * y)[0].subs({y_2: 1-y_1}))
lhs
```

```python
sym.factor(lhs - rhs)
```


# Unconstrained optimization algorithms

Stepsize and directions

-------------------------------

## Steepest descent

```python
import jax.numpy as jnp
from jax import grad, jit, vmap
from jax import jacfwd, jacrev
from jax import random
import numpy as np
import sympy as sym

```

```python
## test jax: evaluate cost function
def cost(x):
    cost = 3*pow(x[0], 2) + pow(x[1], 4)
    return cost
x0 = jnp.array([1.0, -2.0])
```

### Test helper functions

```python
## test gradient direction

def get_gradient_direction(cost, x_val, direction_type):

    # get derivative of the cost function
    derivative = grad(cost)(x_val)

    if direction_type is 'steepest_descent':
        # D in case of steepest descent is identity matrix
        D = jnp.identity(len(x_val))
        direction = - jnp.matmul(D, derivative)

    elif direction_type is 'newton_method':
        hessian = jacfwd(jacrev(cost))(x_val)
        D = jnp.linalg.inv(hessian)
        direction = - jnp.matmul(D, derivative)

    else:
        raise Exception('Unknown gradient based method, please input one of the following: steepst_descent, newton_method')

    return direction

direction = get_gradient_direction(cost, x0, 'newton_method')
direction
```

    DeviceArray([-1.       ,  0.6666667], dtype=float32)

```python
## test step size
import sympy as sym

def get_step_size(cost, x_current, direction, step_size_type, alpha_cons=None):
    # create symbolic step size: alpha
    alpha = sym.Symbol('alpha')
    # convert jax device array to numpy list
    x_current = x_current.tolist()
    direction = direction.tolist()

    if step_size_type is 'minimization_rule':
        sym_vec = []
        for l in range(len(direction)):
            mm = x_current[l] + alpha*direction[l]
            sym_vec.append(mm)
        sym_cost = cost(sym_vec)
        sym_derivative = sym.diff(sym_cost, alpha)
        alpha_sym = sym.solveset(sym_derivative, alpha)
        alpha_sym = list(alpha_sym)

        # only keep the real number by checking data type
        i = 0
        while True:
            condition = type(alpha_sym[i]**2) is sym.numbers.Float
            if condition is True:
                val = alpha_sym[i]
                break
            else:
                i += 1

        alpha = float(val)

    elif step_size_type is 'constant':
        alpha = alpha_cons

    elif step_size_type is 'armijo rule':

    else:
        raise Exception('Unknown step size type, please input one of the following: minimization_tule, constant, armijo rule')

    return alpha

x0 = jnp.array([1.0, -2.0])
direction = get_gradient_direction(cost, x0, 'steepest_descent')
alpha = get_step_size(cost, x0, direction, 'minimization_rule')
print(alpha)
```

```python
# Test Armijo's rule

def armijo_condition(alpha, sigma, cost, x_current, x_next, direction):
    LHS = cost(x_current) - cost(x_next)
    RHS = - sigma * alpha * jnp.matmul(grad(cost)(x_current), direction)
    print('LHS:{}, RHS:{}'.format(LHS, RHS))
    condition = True if (LHS >= RHS) else False
    return condition

# hyperparameters
beta = 0.5
s = 1
sigma = 0.1
m = 0
condition = False
x0 = jnp.array([1.0, -2.0])
x_current = x0

while True:
    print('m = {}'.format(m))
    alpha = pow(beta, m) * s
    direction = get_gradient_direction(cost, x_current, 'steepest_descent')
    x_next = x_current + alpha * direction
    condition = armijo_condition(alpha, sigma, cost, x_current, x_next, direction)
    print('m: {}, alpha: {}, x_current: {}, x_next: {}, condition: {}'.format(
        m, alpha, x_current, x_next, condition))
    print('---------------')

    if condition is True:
        break
    m += 1

# hyperparameters
beta = 0.5
s = 1
sigma = 0.1
m = 0
condition = False
x0 = jnp.array([1.0, -2.0])
x_current = x0

LHS = cost(x_current) - cost(x_next)
RHS = -sigma * pow(beta, m) * s * jnp.matmul(grad(cost)(x_current), direction)
print(LHS)
print(RHS)
print(direction)
```

    m = 0
    LHS:-810056.0, RHS:106.0
    m: 0, alpha: 1.0, x_current: [ 1. -2.], x_next: [-5. 30.], condition: False
    ---------------
    m = 1
    LHS:-38409.0, RHS:53.0
    m: 1, alpha: 0.5, x_current: [ 1. -2.], x_next: [-2. 14.], condition: False
    ---------------
    m = 2
    LHS:-1277.75, RHS:26.5
    m: 2, alpha: 0.25, x_current: [ 1. -2.], x_next: [-0.5  6. ], condition: False
    ---------------
    m = 3
    LHS:2.8125, RHS:13.25
    m: 3, alpha: 0.125, x_current: [ 1. -2.], x_next: [0.25 2.  ], condition: False
    ---------------
    m = 4
    LHS:17.828125, RHS:6.625
    m: 4, alpha: 0.0625, x_current: [ 1. -2.], x_next: [0.625 0.   ], condition: True
    ---------------
    17.828125
    106.0
    [-6. 32.]

```python

```

--------------------------------------------------------

## Helper functions

```python

def cost(x):
    cost = 3*pow(x[0], 2) + pow(x[1], 4)
    return cost

def armijo_condition(alpha, sigma, cost, x_current, x_next, direction):
    LHS = cost(x_current) - cost(x_next)
    RHS = - sigma * alpha * jnp.matmul(grad(cost)(x_current), direction)
    condition = True if (LHS >= RHS) else False
    return condition

def get_step_size(cost, x_current, direction, step_size_type,
                 beta=None, s=None, sigma=None, alpha_cons=None):

    if step_size_type is 'minimization_rule':
        # create symbolic step size: alpha
        alpha = sym.Symbol('alpha')
        # convert jax device array to numpy list
        x_current = x_current.tolist()
        direction = direction.tolist()
        sym_vec = []
        for l in range(len(direction)):
            mm = x_current[l] + alpha*direction[l]
            sym_vec.append(mm)
        sym_cost = cost(sym_vec)
        sym_derivative = sym.diff(sym_cost, alpha)
        alpha_sym = sym.solveset(sym_derivative, alpha)
        alpha_sym = list(alpha_sym)

        # only keep the real number by checking data type
        i = 0
        while True:
            condition = type(alpha_sym[i]**2) is sym.numbers.Float
            if condition is True:
                val = alpha_sym[i]
                break
            else:
                i += 1

        alpha = float(val)

    elif step_size_type is 'constant':
        alpha = alpha_cons

    elif step_size_type is 'armijo_rule':
        m = 0
        condition = False

        while True:
            alpha = pow(beta, m) * s
            x_next = x_current + alpha * direction
            condition = armijo_condition(alpha, sigma, cost, x_current, x_next, direction)

            if condition is True:
                break
            m += 1

    else:
        raise Exception('Unknown step size type, please input one of the following: minimization_tule, constant, armijo rule')

    return alpha

def run_algorithm(x0, direction_type, step_size_type, beta=None, s=None, sigma=None):
    x0 = jnp.array(x0)
    x_current = x0
    X_cache = []

    for i in range(T):
        X_cache.append(np.asarray(x_current, dtype=float))
        direction = get_gradient_direction(cost, x_current, direction_type)
        step_size = get_step_size(cost, x_current, direction, step_size_type, beta, s, sigma)
        x_next = x_current + step_size * direction
        x_current = x_next

    # save all co-ordinates into X_cache for visualization
    X_cache.append(np.asarray(x_next, dtype=float))

    return X_cache
```

------------------------------------------------------

## Run algorithm

```python

# hyperparameters for armijo rule
beta = 0.5
s = 1
sigma = 0.1

# initial condition
x0 = [1.0, -2.0]

# number of iterations
T = 10

X_cache = run_algorithm(x0, 'steepest_descent', 'armijo_rule', beta, s, sigma)
X_cache_newton_minimization = run_algorithm(x0, 'newton_method', 'minimization_rule', beta, s, sigma)

```

## Visualization

```python
import matplotlib.pyplot as plt

def cost_2d(X, Y):
    cost_2d = 3*pow(X, 2) + pow(Y, 4)
    return cost_2d

f = plt.figure(figsize=(12, 8))
plt.rc('axes', linewidth=2)

x, y = np.arange(-1.5, 1.5, 0.2), np.arange(-2.0, 2.0, 0.2)
X, Y = np.meshgrid(x, y)
F = cost_2d(X, Y)
plt.plot([X_cache[i][0] for i in range(len(X_cache))],
            [X_cache[i][1] for i in range(len(X_cache))], '-*', color='red', markersize=15, lw=2.0)
plt.scatter(1, -2, 50)
plt.contour(X, Y, F)
plt.xlabel('$x_1$', size=25)
plt.ylabel('$x_2$', size=25)
dist = []
for i in range(len(X_cache)):
    d = np.sqrt(X_cache[i][0]**2 + X_cache[i][1]**2)
    dist.append(d)
    f.savefig("steepest_descent_armijo.pdf", bbox_inches='tight')
```

```python
dist1 = []
dist2 = []

for i in range(len(X_cache)):
    d1 = np.sqrt(X_cache_steepest_minimization[i][0]**2 + X_cache[i][1]**2)
    d2 = np.sqrt(X_cache_newton_minimization[i][0]**2 + X_cache[i][1]**2)

    dist1.append(d1)
    dist2.append(d2)

f = plt.figure(figsize=(12, 8))
plt.rc('axes', linewidth=2)

plt.plot([k for k in range(len(dist1))], dist1, '-*', color='red', markersize=15, lw=2.0, label='steepest descent')
plt.plot([k for k in range(len(dist2))], dist2, '-o', color='blue', markersize=15, lw=2.0, label='newton method', alpha=0.1)

plt.ylabel('$||x-x^*||$', size=25)
plt.xlabel('$k$', size=25)
plt.legend(fontsize=18)
f.savefig("comparison_g.pdf", bbox_inches='tight')
```

```python

```

```python

```


```python
from sympy import symbols, exp, ln
```

```python
s1, s2, y = symbols('s1 s2 y')
v1, v2 = symbols('v1, v2')

n1 = (s1 - 1500) / v1
n2 = (s2 - 1500) / v2

f = 1 / (1 + exp(- (n1 - n2)))
```

```python

```

```python
f
```

$\displaystyle \frac{1}{e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}} + 1}$

```python
l = y * ln(f) + (1 - y) * ln(1 - f)
```

```python
l
```

$\displaystyle y \log{\left(\frac{1}{e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}} + 1} \right)} + \left(1 - y\right) \log{\left(1 - \frac{1}{e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}} + 1} \right)}$

```python
from sympy import diff

s1_update = diff(l, s1)
v1_update = diff(l, v1)

s2_update = diff(l, s2)
s1_update
```

$\displaystyle \frac{y e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}}}{v_{1} \left(e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}} + 1\right)} - \frac{\left(1 - y\right) e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}}}{v_{1} \left(1 - \frac{1}{e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}} + 1}\right) \left(e^{\frac{s_{2} - 1500}{v_{2}} - \frac{s_{1} - 1500}{v_{1}}} + 1\right)^{2}}$

```python
update
```

$\displaystyle \frac{y e^{- s_{1} + s_{2}}}{e^{- s_{1} + s_{2}} + 1} - \frac{\left(1 - y\right) e^{- s_{1} + s_{2}}}{\left(1 - \frac{1}{e^{- s_{1} + s_{2}} + 1}\right) \left(e^{- s_{1} + s_{2}} + 1\right)^{2}}$

```python
# S1 is higher than s2 so it is expected to win and he does
# S1
conf = dict(s1=1500, s2=1400, y=1, v1=300, v2=35)
print(conf)
print('Probability of Winning:', f.evalf(subs=conf))
print('S1 Update: ', s1_update.evalf(subs=conf))
print('V1 Update: ', v1_update.evalf(subs=conf))
```

    {'s1': 1500, 's2': 1400, 'y': 1, 'v1': 300, 'v2': 35}
    Probability of Winning: 0.945686733867359
    S1 Update:  0.000181044220442135
    V1 Update:  -0.e-263

```python
# S1 is higher than s2 so it is expected to win but losses
# S1 is less consistent
conf = dict(s1=1500, s2=1400, y=0, v1=200, v2=400)
print(conf)
print('Probability of Winning:', f.evalf(subs=conf))
print('S1 Update: ', s1_update.evalf(subs=conf))
print('V1 Update: ', v1_update.evalf(subs=conf))
```

    {'s1': 1500, 's2': 1400, 'y': 0, 'v1': 200, 'v2': 400}
    Probability of Winning: 0.562176500885798
    S1 Update:  -0.00281088250442899
    V1 Update:  0

```python
# S1 is lower than s2 so it is expected to lose but win
conf = dict(s1=1400 / 1500, s2=1500 /1500, y=1)
print(conf)
print('Probability of Winning:', f.evalf(subs=conf))
print('Update: ', s1_update.evalf(subs=conf))
```

    {'s1': 0.9333333333333333, 's2': 1.0, 'y': 1}
    Probability of Winning: 0.483339503430589
    Update:  0.516660496569411

```python
# S1 is lower than s2 so it is expected to lose and he does
conf = dict(s1=1400 / 1500, s2=1500 /1500, y=0)
print(conf)
print('Probability of Winning:', f.evalf(subs=conf))
print('Update: ', s1_update.evalf(subs=conf))
```

    {'s1': 0.9333333333333333, 's2': 1.0, 'y': 0}
    Probability of Winning: 0.483339503430589
    Update:  -0.483339503430589

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```


```python
from sympy import plot,exp,pi,sin
from sympy.abc import t
plot(exp(sin(t)),exp(-sin(t)),exp(t),exp(t)**-1,yscale='log',xlim = (-2*pi,2*pi),legend=True)
```


# Fourier series

This workbook explores the Fourier series representation of signals.  It is shown how symbolic methods can be used to calculate series coefficients, and the accuracy of the time-domain reconstruction for different numbers of coefficients is also considered.

## Background

Any signal $x(t)$ that is periodic with a period $T$ can be written in a Fourier series form
$$x(t) = \sum_{k=-\infty}^\infty c_k e^{j k \omega_0 t}$$
with $\omega_0 = 2\pi/T = \pi$ radians per second.  The coefficients satisfy
$$c_k = \frac{1}{T} \int_{-T/2}^{T/2} x(t) e^{-j k \omega_0 t} dt.$$

For real signals with $x(t) = x^\ast(t)$ one can show that $c_{-k} = c_k^{\ast}$.  Writing in polar form $c_{k} = |c_{k}| e^{j \angle c_{k}}$ the series becomes
$$
x(t) = c_0 + \sum_{k=1}^\infty ( c_k e^{j k \omega_0 t} + c_{-k} e^{-j k \omega_0 t} )
= c_0 + \sum_{k=1}^\infty |c_{k}| ( e^{j k \omega_0 t} e^{j \angle c_{k}} + e^{-j k \omega_0 t} e^{-j \angle c_{k}})
= c_0 + \sum_{k=1}^\infty 2 |c_{k}| \cos(\omega_0 t + \angle c_{k})
$$

The coefficient $c_k$ corresponds to a complex exponential with frequency $k \omega_0$.  We call the component of the signal with frequency $k \omega_0$ the $k$th *harmonic*.  The first harmonic is also called the *fundamental*.

## Signal definition and analysis

The following periodic signal is considered throughout this workbook:

The signal is has period $T=8$, and $\omega_0 = 2 \pi/8 = \pi/4$.  We can find the corresponding Fourier series coefficients:
$$c_k = \frac{1}{T} \int_{-T/2}^{T/2} x(t) e^{-j k \omega_0 t} dt
= \frac{1}{T} \int_{-2}^{2} e^{-j k \omega_0 t} dt
= \frac{1}{j k \omega_0 T} [-e^{-j k \omega_0 t}]_{t=-2}^{2}
= \frac{2}{k \omega_0 T} \sin(2 k \omega_0).
$$
Additionally, the DC coefficient $c_0 = 4/8$.

## Fourier series reconstruction

For a given set of coefficients $c_k$ we want to be able to plot the corresponding $x(t)$.  The function defined in the cell below takes a set of Fourier series coefficients for a real signal `ckv` and a fundamental frequency `omega0`, and calculates reconstructed values `xv` at time instants `tv`.

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib notebook

def fsrrec(ckv,omega0,tv):
    """Generate samples from real Fourier series representation
    ckv - 0 to N Fourier series coefficients
    omega0 - fundamental frequency
    tv - input time points
    returns xv - output values
    """

    xv = ckv[0]*np.ones(tv.shape);
    for k in range(1,len(ckv)):
        xv = xv + 2*np.abs(ckv[k])*np.cos(k*omega0*tv + np.angle(ckv[k]));  # update with kth harmonic component

    return(np.real(xv));
# end def
```

```python
def fsrrec_plots(ckv,omega0,tv):
    """The same as fssrec, but also outputs a plot of the individual harmonic components
    """

    xv = ckv[0]*np.ones(tv.shape);
    #tv.shape returns the dimensions of the matrix (or, in this case, array) tv.
    #np.ones creates an array of the input size populated with 1's
    plt.figure(1)
    plt.plot(tv,xv)
    for k in range(1,len(ckv)):
        kh = 2*np.abs(ckv[k])*np.cos(k*omega0*tv + np.angle(ckv[k]));  #create kth harmonic
        plt.plot(tv,kh);

        xv = xv + kh; #add kth harmonic to x
    plt.show()
    return(np.real(xv));
# end def
```

The cell below uses the derived expression for the coefficients of the signal and stores them in the vector `ckv`.  The $k$th element of `ckv` contains the coefficient $c_k$.

```python
# Fourier series coefficients for rectangular pulse train
T = 8;  # period
N = 10;  # maximum number of terms
omega0 = 2*np.pi/T;
ckv = np.zeros(N+1, dtype=np.complex64);
#np.zeros is like np.ones but with 0's

for k in range(1,N+1):  ckv[k] = 2/(k*omega0*T)*np.sin(2*k*omega0);
ckv[0] = 4/8;
```

We can use the `fsrrec` function to find the time-domain representation of the signal $x(t)$ using a finite number of terms in the reconstruction.  The code below does this and plots the result.

```python
# Reconstruct from series representation and plot
tv = np.linspace(-10,10,10000);
xv = fsrrec(ckv,omega0,tv);
#xv = fsrrec_plots(ckv,omega0,tv);
plt.figure(2)
plt.plot(tv,xv,'r');
plt.xlabel('t');  plt.ylabel('x(t) reconstruction from ' + str(N) + ' terms');
```

## Finding the coefficients using symbolic math

This section will show how we can calculate the Fourier series coefficients of a signal using symbolic manipulation.  The first thing to do is symbolically define the signal.  The `Piecewise` function lets you define a signal over different pieces.

```python
import sympy as sp
#sp.init_printing();  # pretty printing

t = sp.symbols('t')
x = sp.Piecewise( (0, t<=-2), (1, t<2), (0, True));
# sp.plot(x, (t,-4,4));  # worked in Python 2.8!
sp.plot(x.subs(t,sp.re(t)), (t,-4,4));  # plot (forcing t real)

#?sp.Piecewise() #uncomment if you want more information about this function
```

The following cell defines the symbolic integral for computing the coefficients.

```python
Ts, k, w0 = sp.symbols('Ts k w0');
w0 = 2*sp.pi/Ts;
expt = sp.exp(-1j*k*w0*t);
cke = 1/Ts*sp.integrate(x*expt, (t, -Ts/2, Ts/2));
#cke = sp.integrate(x*expt, (t, -sp.oo, T/2)) - sp.integrate(x*expt, (t, -sp.oo, -T/2));  # alternative
ck = cke.subs(Ts,T).doit();  # set value for period and evaluate
ck
```

$\displaystyle \frac{\begin{cases} - \frac{4.0 i e^{0.5 i \pi k}}{\pi k} + \frac{4.0 i e^{- 0.5 i \pi k}}{\pi k} & \text{for}\: k > -\infty \wedge k < \infty \wedge k \neq 0 \\4 & \text{otherwise} \end{cases}}{8}$

We now define a vector `kv` of coefficients of interest, and populate corresponding elements of `ckv` with the coefficient values.

```python
kv = np.arange(-10,11);  # coefficients to calculate
#by default, np.arange returns the integers between the given start and end points

ckvs = np.zeros(kv.shape, dtype=np.complex64);  # corresponding coefficient values
for i in range(len(kv)):
    ki = kv[i];
    ckvs[i] = ck.subs({k:ki}).evalf();
ckvs
```

    array([ 0.        +0.j,  0.03536776+0.j, -0.        +0.j, -0.04547284+0.j,
            0.        +0.j,  0.06366198+0.j, -0.        +0.j, -0.10610329+0.j,
            0.        +0.j,  0.31830987+0.j,  0.5       +0.j,  0.31830987+0.j,
            0.        +0.j, -0.10610329+0.j, -0.        +0.j,  0.06366198+0.j,
            0.        +0.j, -0.04547284+0.j, -0.        +0.j,  0.03536776+0.j,
            0.        +0.j], dtype=complex64)

Now we can plot the frequency-domain representation of the signal $x(t)$ by displaying the value of $c_k$ for each value $k$ of interest.  Since $c_k$ can in general be complex we need two plots:  one for magnitude and one for phase.

```python
fh, ax = plt.subplots(2);
ax[0].stem(kv, np.abs(ckvs), c='g');  ax[0].set_ylabel(r'$|c_k|$');
ax[1].stem(kv, np.angle(ckvs), c='g');  ax[1].set_ylabel(r'$\angle c_k$');
plt.xlabel('$k$');
```

We could also have created a lambda function from the symbolic expression.  This function takes an array of values for $k$ and calculates $c_k$ directly.  Note though that at least on my version of Python `lam_ck(0)` generates a divide-by-zero error.

```python
lam_ck = sp.lambdify(k,ck,modules=['numpy']);
lam_ck(np.array((1,2,3)))
```

With numerical values for the coefficients, obtained via symbolic computation, we can plot the partial sum for the time-domain reconstruction as before.  Recall that our `fsrrec` function only takes the coefficients for nonnegative index values.

```python
kzi = np.where(kv==0)[0][0];  # index for zero element
ckvsp = ckvs[kzi:];
tv = np.linspace(-10,10,10000);
xv = fsrrec(ckvsp,2*np.pi/T,tv);

fh = plt.figure();
plt.plot(tv,xv,'r');
plt.xlabel('t');  plt.ylabel('x(t) reconstruction from ' + str(len(ckvsp)-1) + ' terms');
```

# Tasks

These tasks involve writing code, or modifying existing code, to meet the objectives described.

1.  Find and plot the Fourier series frequency-domain representation for the signal $x_1(t)$ below over the range $k=-8, \ldots, 8$:

Do this using both symbolic processing and by evaluating the integral for the coefficients by hand.  Compare the result with that displayed earlier for $x(t)$.  Note that the signals are related in time by $x_1(t) = x(t-2)$.

You should observe that shifting a signal only changes the phase in the frequency domain, while the magnitude remains unchanged.

* Use symbolic processing to find and plot the frequency-domain representation of $x_2(t)$ below over the range $k=-8, \ldots, 8$:

Also plot the reconstruction over the range $t=-4$ to $t=4$ using only components up to and including the $5$th harmonic.

You should find that as $k$ increases the magnitude of the coefficients in this case falls off much faster than those of $x(t)$.  This is because $x_2(t)$ is smoother (it is at least continuous, while $x(t)$ is discontinuous).  The reconstruction is therefore also more accurate with a smaller number of terms.

* Find and plot the frequency-domain representation of $x_3(t)$ below over the range $k=-8, \ldots, 8$.

```python

```


```python
import numpy as np
%matplotlib inline
from mpl_toolkits import mplot3d
import matplotlib.pyplot as plt
import pickle
```

# 8-1. Optimizers

## Functions

$$
f(x, y) = \frac{1}{20} x^2 + y^2
$$

```python
def f(variables):
    return ((variables['x'] ** 2) * 0.05 + variables['y'] ** 2)

x = np.linspace(-10, 10, 30)
y = np.linspace(-10, 10, 30)

X, Y = np.meshgrid(x, y)
Z = f({'x': X, 'y': Y})
```

```python
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.plot_wireframe(X, Y, Z, color='black', linewidth=0.5)
```

```python
def gradient(variables, variable_name):
    h = 1e-4
    input2 = variables.copy()
    input2[variable_name] += h

    input1 = variables.copy()
    input1[variable_name] -= h

    z2 = f(input2)
    z1 = f(input1)

    delta_z = z2 - z1
    delta_x = h * 2
    return delta_z / delta_x
```

```python
gradient({'x': 4.5, 'y': -3.0}, 'x')
```

    0.4499999999918458

## Vanilla SGD

$$
\theta \leftarrow \theta - \eta \frac{\partial L}{\partial w}
$$

```python
current_x = 4.5
current_y = 4.5

for iteration in range(1000):
    ETA = 0.01
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    function_output = f(current)
    current_x = current_x - ETA * grad['x']
    current_y = current_y - ETA * grad['y']
    #print("iteration: {}, x: {}, y: {}, output: {}".format(iteration, current_x, current_y, function_output))

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:1.6546294114674343, y:7.57335322232322e-09

```python
current_x = 4.5
current_y = 4.5

for iteration in range(5000):
    ETA = 0.01
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    function_output = f(current)
    current_x = current_x - ETA * grad['x']
    current_y = current_y - ETA * grad['y']
    #print("iteration: {}, x: {}, y: {}, output: {}".format(iteration, current_x, current_y, function_output))

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:0.030245003819367854, y:7.968885967768458e-18

## Momentum

\begin{align}
v &\leftarrow \beta v + (1 - \beta) \frac{\partial L}{\partial w} \\
\theta &\leftarrow \theta - \eta v
\end{align}

where $\beta$ acts as friction. Usually around `0.9`.

```python
current_x = 4.5
current_y = 4.5

BETA = 0.9

vx = 0.0
vy = 0.0

for iteration in range(1000):
    ETA = 0.01
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    vx = BETA * vx + grad['x']
    vy = BETA * vy + grad['y']

    current_x = current_x - ETA * vx
    current_y = current_y - ETA * vy

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:7.020463528798055e-05, y:3.579225378800207e-21

```python
current_x = 4.5
current_y = 4.5

BETA = 0.9

vx = 0.0
vy = 0.0

for iteration in range(5000):
    ETA = 0.01
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    vx = BETA * vx + grad['x']
    vy = BETA * vy + grad['y']

    current_x = current_x - ETA * vx
    current_y = current_y - ETA * vy

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:5.310462302733221e-21, y:3.579942939503076e-21

## RMSProp

\begin{align}
s &\leftarrow \beta s + (1 - \beta) \left(\frac{\partial L}{\partial w} \right)^2 \\
\theta &\leftarrow \theta - \eta \frac{\partial L}{\partial \theta} \frac{1}{\sqrt{s + \epsilon}}
\end{align}

where $\beta$ acts as friction around `0.9` and $\epsilon$ is a small number like `1e-8` for numerical stability.

```python
current_x = 4.5
current_y = 4.5

BETA = 0.9
EPSILON = 1e-8

sx = 0.0
sy = 0.0

for iteration in range(1000):
    ETA = 0.01
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    sx = BETA * sx + (1 - BETA) * (grad['x'] ** 2)
    sy = BETA * sy + (1 - BETA) * (grad['y'] ** 2)

    current_x = current_x - ETA * grad['x'] / np.sqrt(sx + EPSILON)
    current_y = current_y - ETA * grad['y'] / np.sqrt(sy + EPSILON)

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:-0.004898979483990539, y:3.309816109390423e-21

```python
current_x = 4.5
current_y = 4.5

BETA = 0.9
EPSILON = 1e-8

sx = 0.0
sy = 0.0

for iteration in range(5000):
    ETA = 0.001
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    sx = BETA * sx + (1 - BETA) * (grad['x'] ** 2)
    sy = BETA * sy + (1 - BETA) * (grad['y'] ** 2)

    current_x = current_x - ETA * grad['x'] / np.sqrt(sx + EPSILON)
    current_y = current_y - ETA * grad['y'] / np.sqrt(sy + EPSILON)

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:4.666870927179143e-21, y:2.645159853119714e-21

## Adam

\begin{equation}
v \leftarrow \beta_1 v + (1 - \beta_1) \frac{\partial L}{\partial w} \\
s \leftarrow \beta_2 s + (1 - \beta_2) \left(\frac{\partial L}{\partial w} \right)^2 \\
v_{corrected} \leftarrow \frac{v}{1-\beta_1^t} \\
s_{corrected} \leftarrow \frac{s}{1-\beta_2^t} \\
\theta \leftarrow \theta - \eta \frac{v_{corrected}}{\sqrt{s_{corrected} + \epsilon}}
\end{equation}

where $\beta_1$ is around `0.9`, $\beta_2$ is around `0.999` and $\epsilon$ is a small number like `1e-8` for numerical stability.

```python
current_x = 4.5
current_y = 4.5

BETA1 = 0.9
BETA2 = 0.999
EPSILON = 1e-8

vx = 0.0
sx = 0.0
vy = 0.0
sy = 0.0

for iteration in range(1000):
    ETA = 0.01
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    vx = BETA1 * vx + (1 - BETA1) * grad['x']
    vy = BETA1 * vy + (1 - BETA1) * grad['y']

    sx = BETA2 * sx + (1 - BETA2) * (grad['x'] ** 2)
    sy = BETA2 * sy + (1 - BETA2) * (grad['y'] ** 2)

    c_beta1 = BETA1 ** (iteration+1)
    vcx = vx / (1 - c_beta1)
    vcy = vy / (1 - c_beta1)

    c_beta2 = BETA2 ** (iteration+1)
    scx = sx / (1 - c_beta2)
    scy = sy / (1 - c_beta2)

    current_x = current_x - ETA * vcx / np.sqrt(scx + EPSILON)
    current_y = current_y - ETA * vcy / np.sqrt(scy + EPSILON)

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:0.06483412117195497, y:0.06483408592640541

```python
current_x = 4.5
current_y = 4.5

BETA1 = 0.9
BETA2 = 0.999
EPSILON = 1e-8

vx = 0.0
sx = 0.0
vy = 0.0
sy = 0.0

for iteration in range(5000):
    ETA = 0.01
    current = {
        'x': current_x,
        'y': current_y
    }
    grad = {
        'x': gradient(current, 'x'),
        'y': gradient(current, 'y'),
    }
    vx = BETA1 * vx + (1 - BETA1) * grad['x']
    vy = BETA1 * vy + (1 - BETA1) * grad['y']

    sx = BETA2 * sx + (1 - BETA2) * (grad['x'] ** 2)
    sy = BETA2 * sy + (1 - BETA2) * (grad['y'] ** 2)

    c_beta1 = BETA1 ** (iteration+1)
    vcx = vx / (1 - c_beta1)
    vcy = vy / (1 - c_beta1)

    c_beta2 = BETA2 ** (iteration+1)
    scx = sx / (1 - c_beta2)
    scy = sy / (1 - c_beta2)

    current_x = current_x - ETA * vcx / np.sqrt(scx + EPSILON)
    current_y = current_y - ETA * vcy / np.sqrt(scy + EPSILON)

print("minimum x:{}, y:{}".format(current_x, current_y))
```

    minimum x:2.6827201657106435e-21, y:3.520151103317083e-21

```python

```


# Regresión Lineal Desde una Perspectiva Probabilística

> Al día de hoy todos debemos haber ajustado una línea recta (plano, o hiperplano) $f(x) = x^T w$ a un conjunto de puntos $\{(x_i, y_i)\}_{i=1}^{N}$, con $x_i\in\mathbb{R}^{d}$ y $y_i\in\mathbb{R}$, y parámetros $w\in\mathbb{R}^{d}$. Esto normalmente se hace bajo el enfoque de mínimos cuadrados, es decir:
>
> $$
  \hat{w} = \arg \min_{w} ||y - Xw||^2,
  $$

> donde
>
> $$
  X = \left[\begin{array}{ccc}
  - & x_1^T  & - \\
  - & x_2^T  & - \\
    & \vdots &   \\
  - & x_N^T  & - \\
  \end{array}\right] \in \mathbb{R}^{N \times d}, \qquad y = \left[\begin{array}{ccc}
  y_1 \\
  y_2 \\
  \vdots \\
  y_N \\
  \end{array}\right] \in \mathbb{R}^{N}.
  $$

> Adicionalmente, para evitar el fenómeno de overfitting, una práctica común es incluir un término de regularización penalizando el tamaño de los parámetros $w$:
>
> $$
  \hat{w} = \arg \min_{w} ||y - Xw||^2 + \lambda ||w||^2
  $$

> Pues bien, hoy veremos cómo se explica todo esto desde una perspectiva probabilística.

> **Objetivos:**
> - Explicar la regresión lineal por mínimos cuadrados regularizada desde una perspectiva probabilística.

> **Referencias:**
>
> - Bayesian Methods for Machine Learning course, HSE University, Coursera.

```python
# Importamos librerías
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import Pipeline
from matplotlib import pyplot as plt
import numpy as np
```

```python
# Generamos datos ficticios
N = 100
x = np.linspace(0, 1, N)
y = np.sin(2 * np.pi * x) + 0.3 * np.random.normal(size=(N,))
```

```python
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
```

```python
# Gráfica de los datos
plt.figure(figsize=(6, 4))
plt.plot(x_train, y_train, 'or', ms=5, label="Datos de entrenamiento")
plt.plot(x_test, y_test, 'ob', ms=5, label="Datos de prueba")
plt.legend(loc="upper left", bbox_to_anchor=(1.05, 1))
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.grid()
```

```python
# Definición del modelo
model_pipeline = Pipeline([
    ("poly", PolynomialFeatures(degree=3)),
    ("reg", LinearRegression())
])
```

```python
# Ajuste del modelo
model_pipeline.fit(x_train.reshape((-1, 1)), y_train)
```

    Pipeline(memory=None,
             steps=[('poly',
                     PolynomialFeatures(degree=3, include_bias=True,
                                        interaction_only=False, order='C')),
                    ('reg',
                     LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
                                      normalize=False))],
             verbose=False)

```python
# Visualización del modelo
plt.figure(figsize=(6, 4))
plt.plot(x_train, y_train, 'or', ms=5, label="Datos de entrenamiento")
plt.plot(x_train, model_pipeline.predict(x_train.reshape((-1, 1))), '*g', lw=2, label="Polinomio de grado 3 ajustado")
plt.legend(loc="upper left", bbox_to_anchor=(1.05, 1))
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.grid()
```

```python
# Score sobre datos de entrenamiento
model_pipeline.score(x_train.reshape((-1, 1)), y_train)
```

    0.8299202521309362

```python
plt.figure(figsize=(6, 4))
plt.plot(x_test, y_test, 'or', ms=5, label="Datos de prueba")
plt.plot(x_test, model_pipeline.predict(x_test.reshape((-1, 1))), '*g', lw=2, label="Polinomio de grado 3 ajustado")
plt.legend(loc="upper left", bbox_to_anchor=(1.05, 1))
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.grid()
```

```python
# Score sobre datos de prueba
model_pipeline.score(x_test.reshape((-1, 1)), y_test)
```

    0.8027169204325145

```python
# Definición del modelo
model_pipeline_ = Pipeline([
    ("poly", PolynomialFeatures(degree=8)),
    ("reg", LinearRegression())
])
```

```python
model_pipeline_.fit(x_train.reshape((-1, 1)), y_train)
```

    Pipeline(memory=None,
             steps=[('poly',
                     PolynomialFeatures(degree=8, include_bias=True,
                                        interaction_only=False, order='C')),
                    ('reg',
                     LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
                                      normalize=False))],
             verbose=False)

```python
# Visualización del modelo
plt.figure(figsize=(6, 4))
plt.plot(x_train, y_train, 'or', ms=5, label="Datos de entrenamiento")
plt.plot(x_train, model_pipeline_.predict(x_train.reshape((-1, 1))), '*g', lw=2, label="Polinomio de grado 8 ajustado")
plt.legend(loc="upper left", bbox_to_anchor=(1.05, 1))
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.grid()
```

```python
# Score sobre datos de entrenaimiento
model_pipeline_.score(x_train.reshape((-1, 1)), y_train)
```

    0.8358048987005741

```python
plt.figure(figsize=(6, 4))
plt.plot(x_test, y_test, 'or', ms=5, label="Datos de prueba")
plt.plot(x_test, model_pipeline_.predict(x_test.reshape((-1, 1))), '*g', lw=2, label="Polinomio de grado 8 ajustado")
plt.legend(loc="upper left", bbox_to_anchor=(1.05, 1))
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.grid()
```

```python
# Score sobre datos de prueba
model_pipeline_.score(x_test.reshape((-1, 1)), y_test)
```

    0.8137585467475051

### Overfitting

```python
model_pipeline.named_steps["reg"].coef_
```

    array([  0.        ,  13.71177813, -39.06580598,  25.87020616])

```python
model_pipeline_.named_steps["reg"].coef_
```

    array([    0.        ,     5.60686017,    68.77894649,  -523.40718161,
            1347.72005326, -1681.42816826,   958.15242705,  -100.12905775,
             -75.10849758])

### Regularización

```python
model_pipeline_reg = Pipeline([
    ("poly", PolynomialFeatures(degree=8)),
    ("reg", Ridge(alpha=0.01))
])
```

```python
model_pipeline_reg.fit(x_train.reshape((-1, 1)), y_train)
```

    Pipeline(memory=None,
             steps=[('poly',
                     PolynomialFeatures(degree=8, include_bias=True,
                                        interaction_only=False, order='C')),
                    ('reg',
                     Ridge(alpha=0.01, copy_X=True, fit_intercept=True,
                           max_iter=None, normalize=False, random_state=None,
                           solver='auto', tol=0.001))],
             verbose=False)

```python
# Visualización del modelo
plt.figure(figsize=(6, 4))
plt.plot(x_train, y_train, 'or', ms=5, label="Datos de entrenamiento")
plt.plot(x_train, model_pipeline_reg.predict(x_train.reshape((-1, 1))), '*g', lw=2, label="Polinomio de grado 8 ajustado")
plt.legend(loc="upper left", bbox_to_anchor=(1.05, 1))
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.grid()
```

```python
model_pipeline_reg.score(x_train.reshape((-1, 1)), y_train)
```

    0.7863977898634911

```python
plt.figure(figsize=(6, 4))
plt.plot(x_test, y_test, 'or', ms=5, label="Datos de prueba")
plt.plot(x_test, model_pipeline_reg.predict(x_test.reshape((-1, 1))), '*g', lw=2, label="Polinomio de grado 8 ajustado")
plt.legend(loc="upper left", bbox_to_anchor=(1.05, 1))
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.grid()
```

```python
model_pipeline_reg.score(x_test.reshape((-1, 1)), y_test)
```

    0.7731855600449242

```python
# Sin regularizacion
model_pipeline_.named_steps["reg"].coef_
```

    array([    0.        ,     5.60686017,    68.77894649,  -523.40718161,
            1347.72005326, -1681.42816826,   958.15242705,  -100.12905775,
             -75.10849758])

```python
# Regularizado
model_pipeline_reg.named_steps["reg"].coef_
```

    array([  0.        ,   5.72317488, -11.24124265,  -3.93324143,
             2.39265945,   4.53547432,   3.66532282,   1.12546011,
            -2.18374113])

## 1. Distribución normal

Antes de entrar en detalles repasemos un poco la distribución normal.

### 1.1. Distribución normal univariada

Se dice que una VA distribuye normal si su función de densidad de probabilidad es:

$$
\mathcal{N}(x|\mu, \sigma^2) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp \left\{-\frac{(x - \mu)^2}{2 \sigma^2}\right\}
$$

con parámetros $\mu \in \mathbb{R}$: media de la VA X, y $\sigma^2 \in \mathbb{R}_{\geq0}$: varianza de la VA X.

**Ejercicio.** Demostrar que:

1.
$$
\int_{-\infty}^{\infty} \mathcal{N}(x|\mu, \sigma^2) = 1.
$$

2.
$$
\mu = \arg \max_{x} \mathcal{N}(x|\mu, \sigma^2)
$$

¿Cómo luce esta densidad?

```python
# Importar scipy.stats
from scipy import stats
# Importar numpy
import numpy as np
# Importar matplitlib.pyplot
from matplotlib import pyplot as plt
```

```python
# Tres VA normales
X = stats.norm(loc=0, scale=1)
Y = stats.norm(loc=1, scale=1)
Z = stats.norm(loc=0, scale=2)
```

```python
# Vector x para graficar
x = np.linspace(-6, 6, 100)
```

```python
# Graficar
fig, ax = plt.subplots(1, 1)
ax.plot(x, X.pdf(x), label=r"$\mu=0, \sigma=1$")
ax.plot(x, Y.pdf(x), label=r"$\mu=1, \sigma=1$")
ax.axvline(x=0, color='k')
ax.axvline(x=1, color='k')
ax.legend()
plt.show()
```

```python
fig, ax = plt.subplots(1, 1)
ax.plot(x, X.pdf(x), label=r"$\mu=0, \sigma=1$")
ax.plot(x, Z.pdf(x), label=r"$\mu=0, \sigma=2$")
ax.axvline(x=0, color='k')
ax.legend()
plt.show()
```

### 1.2. Distribución normal multivariada

Haciendo una extensión natural de lo anterior a vectores, obtenemos la función de densidad de probabilidad para distribución normal multivariada:

$$
\mathcal{N}(x|\mu, \Sigma) = \frac{1}{\sqrt{\det(2 \pi \Sigma)}} \exp \left\{-\frac{1}{2}(x - \mu)^T \Sigma^{-1} (x - \mu)\right\},
$$

con parámetros $\mu \in \mathbb{R}^n$: vector de medias de la VA X y $\Sigma \in \mathbb{R}^{n \times n}$: matriz de covarianzas de la VA X (simétrica y definida positiva).

¿Cómo luce esta densidad?

```python
# VA normales
X = stats.multivariate_normal(mean=np.zeros(2),
                              cov=np.eye(2))
Y = stats.multivariate_normal(mean=np.array([2, 3]),
                              cov=np.array([[1, 0.8],
                                            [0.8, 2]]))
```

```python
from mpl_toolkits import mplot3d
```

```python
# Gráfico
x = np.linspace(-3, 3, 100)
y = np.linspace(-3, 3, 100)
x, y = np.meshgrid(x, y)
z = X.pdf(np.dstack([x, y]))
fig = plt.figure(figsize=(8, 4))
ax1 = fig.add_subplot(121, projection='3d')
ax1.plot_surface(x, y, z)
ax2 = fig.add_subplot(122)
ax2.contour(x, y, z)
plt.show()
```

```python
# Gráfico
x = np.linspace(-2, 6, 100)
y = np.linspace(-3, 9, 100)
x, y = np.meshgrid(x, y)
z = Y.pdf(np.dstack([x, y]))
fig = plt.figure(figsize=(8, 4))
ax1 = fig.add_subplot(121, projection='3d')
ax1.plot_surface(x, y, z)
ax2 = fig.add_subplot(122)
ax2.contour(x, y, z)
plt.show()
```

## 2. Regresión lineal desde una perspectiva probabilística

Para ver algunos fundamentos de regresión lineal ver la introducción de este cuaderno.

Desde una perspectiva Bayesiana, podemos representar las VA objetivo $y$ y parámetros $w$, datos $X$ de un modelo lineal $y=Xw$ como:

¿Cuál es la factorización que induce este modelo?

$$
P(y, w, X) = P(w) P(X) P(y | w, X)
$$

Como no nos interesa modelar los datos, sino los parámetros, estudiamos la distribución posterior $P(w | X, y)$, la cual se puede reescribir como:

$$
P(w | X, y) = \frac{P(y, w | X)}{P(y | X)}.
$$

Por otra parte, usando la definición de probabilidad condicional y la factorización inducida, tenemos que:

$$
P(y, w | X) = P(y | X, w) P(w),
$$

de donde

$$
P(w | X, y) = \frac{P(y, w | X)}{P(y | X)} = \frac{P(y | X, w) P(w)}{P(y | X)}.
$$

Las distribuciones posteriores son en general difíciles de encontrar exactamente, dada la complejidad de modelar la distribución de evidencia $P(y | X)$. Durante el curso estaremos viendo técnicas para encontrarlas tanto de manera exacta (cuando sea posible analíticamente) como de manera aproximada.

Para este ejercicio, una suposición plausible es que la función de verosimilitud $P(y | X, w)$ y la distribución previa $P(w)$ son distribuciones normales de la forma:

$$
P(y | X, w) = \mathcal{N}(y | X w, \sigma^2 I),
$$

$$
P(w) = \mathcal{N}(w | 0, \gamma^2 I),
$$

mientras que la evidencia la podemos pensar como un factor de normalización (para que la distribución posterior integre a 1). En general nos interesa encontrar la **distribución posterior**, sin embargo, para este ejemplo, encontraremos sólo **la moda** de la misma.

De modo que para encontrar el valor más probable de los parámetros según la evidencia de los datos y el objetivo, hacemos:

$$
\arg \max_{w} P(w | X, y) = \arg \max_{w} \frac{P(y, w | X)}{P(y | X)} = \arg \max_{w} P(y, w | X) = \arg \max_{w} P(y | X, w) P(w)
$$

___
**Propiedad de los logaritmos:** dado que el logaritmo es una función creciente,

$$
\arg \max_x f(x) = \arg \max_x \log f(x) = \arg \min_x - \log f(x)
$$
___

Así que el anterior problema lo podemos reescribir como:

$$
\arg \min_w -\log P(y | X, w) - \log P(w)
$$

Desarrollando los logaritmos en el pizarrón:

\begin{align}
-\log P(y | X, w) - \log P(w) & = \dots \\
                              & = \dots \\
                              & = \dots \\
                              & = \log C + \frac{1}{2\sigma^2} (y - X w)^T (y - X w) + \frac{1}{2\gamma^2} w^T w \\
                              & = \log C + \frac{1}{2\sigma^2} ||y - X w||^2 + \frac{1}{2\gamma^2} ||w||^2
\end{align}

Finalmente, podemos resolver el problema

- Ridge:

$$
\arg \min_w ||y - X w||^2 + \lambda ||w||_2^2,
$$

- Lasso (previa de Laplace para los parámetros):

$$
\arg \min_w ||y - X w||^2 + \lambda ||w||_1,
$$

con $\lambda = \frac{\sigma^2}{\gamma^2}$.  Interpretación de $\lambda$.

Se puede demostrar que la solución es:

$$
w^{*}_{reg} = (X^T X + \lambda I)^{-1} X^T y
$$

**Ejemplo:** Usar mínimos cuadrados regularizados para ajustar una recta (plano) a los datos de precios de casas en Portland, OR.

```python
# Importar pandas
import pandas as pd
```

```python
# Leer datos
data = pd.read_csv("housing_prices.csv")
X = data.drop(columns=["price"])
y = data["price"]
X, y
```

    (    size  n_bedrooms
     0   2104           3
     1   1600           3
     2   2400           3
     3   1416           2
     4   3000           4
     5   1985           4
     6   1534           3
     7   1427           3
     8   1380           3
     9   1494           3
     10  1940           4
     11  2000           3
     12  1890           3
     13  4478           5
     14  1268           3
     15  2300           4
     16  1320           2
     17  1236           3
     18  2609           4
     19  3031           4
     20  1767           3
     21  1888           2
     22  1604           3
     23  1962           4
     24  3890           3
     25  1100           3
     26  1458           3
     27  2526           3
     28  2200           3
     29  2637           3
     30  1839           2
     31  1000           1
     32  2040           4
     33  3137           3
     34  1811           4
     35  1437           3
     36  1239           3
     37  2132           4
     38  4215           4
     39  2162           4
     40  1664           2
     41  2238           3
     42  2567           4
     43  1200           3
     44   852           2
     45  1852           4
     46  1203           3,
     0     399900
     1     329900
     2     369000
     3     232000
     4     539900
     5     299900
     6     314900
     7     198999
     8     212000
     9     242500
     10    239999
     11    347000
     12    329999
     13    699900
     14    259900
     15    449900
     16    299900
     17    199900
     18    499998
     19    599000
     20    252900
     21    255000
     22    242900
     23    259900
     24    573900
     25    249900
     26    464500
     27    469000
     28    475000
     29    299900
     30    349900
     31    169900
     32    314900
     33    579900
     34    285900
     35    249900
     36    229900
     37    345000
     38    549000
     39    287000
     40    368500
     41    329900
     42    314000
     43    299000
     44    179900
     45    299900
     46    239500
     Name: price, dtype: int64)

```python
# Importar sklearn.linear_model.Ridge
from sklearn.linear_model import Ridge, Lasso
# Importar sklearn.pipeline.Pipeline
from sklearn.pipeline import Pipeline
# Importart sklearn.preprocessing.StandardScaler
from sklearn.preprocessing import StandardScaler
```

```python
# Scaler-regressor pipeline
ridge_pipeline = Pipeline([
    # ("scaler", StandardScaler()),
    ("reg", Ridge(alpha=1))
])
lasso_pipeline = Pipeline([
    # ("scaler", StandardScaler()),
    ("reg", Lasso(alpha=1))
])
```

```python
# Aprendizaje
ridge_pipeline.fit(X, y)
```

    Pipeline(memory=None,
             steps=[('reg',
                     Ridge(alpha=1, copy_X=True, fit_intercept=True, max_iter=None,
                           normalize=False, random_state=None, solver='auto',
                           tol=0.001))],
             verbose=False)

```python
ridge_pipeline.named_steps["reg"].coef_, ridge_pipeline.named_steps["reg"].intercept_
```

    (array([  138.96771858, -8284.92788471]), 88647.59023379174)

```python
lasso_pipeline.fit(X, y)
```

    Pipeline(memory=None,
             steps=[('reg',
                     Lasso(alpha=1, copy_X=True, fit_intercept=True, max_iter=1000,
                           normalize=False, positive=False, precompute=False,
                           random_state=None, selection='cyclic', tol=0.0001,
                           warm_start=False))],
             verbose=False)

```python
lasso_pipeline.named_steps["reg"].coef_, lasso_pipeline.named_steps["reg"].intercept_
```

    (array([  139.20929338, -8735.4473684 ]), 89592.51877296084)

```python
# Gráfico
plt3d = plt.figure().gca(projection='3d')

xx1, xx2 = np.meshgrid(np.linspace(500, 4500), np.linspace(1, 5, 5))
yp_ridge = ridge_pipeline.predict(np.array([xx1.ravel(), xx2.ravel()]).T).reshape(xx1.shape)
yp_lasso = lasso_pipeline.predict(np.array([xx1.ravel(), xx2.ravel()]).T).reshape(xx1.shape)
plt3d.plot_surface(xx1, xx2, yp_ridge, alpha=0.2)
plt3d.plot_surface(xx1, xx2, yp_lasso, alpha=0.2)
plt3d.scatter(X['size'], X['n_bedrooms'], y);
plt3d.view_init(elev=10, azim=110)
```

### Distribución de Laplace

```python
X = stats.laplace(loc=0, scale=1)
Y = stats.laplace(loc=1, scale=1)
Z = stats.laplace(loc=0, scale=2)
```

```python
x = np.linspace(-5, 5, 100)
```

```python
# Graficar
fig, ax = plt.subplots(1, 1)
ax.plot(x, X.pdf(x), label=r"$\mu=0, b=1$")
ax.plot(x, Y.pdf(x), label=r"$\mu=1, b=1$")
ax.axvline(x=0, color='k')
ax.axvline(x=1, color='k')
ax.legend()
plt.show()
```

```python
# Graficar
fig, ax = plt.subplots(1, 1)
ax.plot(x, X.pdf(x), label=r"$\mu=0, b=1$")
ax.plot(x, Z.pdf(x), label=r"$\mu=0, b=2$")
ax.axvline(x=0, color='k')
ax.legend()
plt.show()
```

# Anuncios parroquiales:

## 1. Quiz de tema 3 para el miércoles 22 de Septiembre.

Created with Jupyter by Esteban Jiménez Rodríguez.


# The quantum harmonic oscillator

\begin{align}
H \psi_n = E_n \psi_n
\end{align}

where the Hamiltontial $H = T + V$ is defined as

\begin{align}
H &= \frac{-\hbar^2}{2m}\frac{\partial^2}{\partial x^2} + \frac{1}{2} k x^2
\end{align}

The wavefunction $\psi_n$ can be shown to be:

\begin{align}
\psi_n &= N_n H_n \left(\frac{x}{\alpha}\right) e^{- \frac{x^2}{2 \alpha^2}}
\end{align}
with

\begin{align}
\alpha &= \left( \frac{\hbar^2}{m k} \right) \\
N_n &= \frac{1}{\sqrt{\alpha \pi^{1/2} 2^n  n!}}
\end{align}

and the energy $E_n = \hbar \omega \left(n + \frac{1}{2}\right)$ where $\omega = \sqrt{\frac{k}{m}}$ and $n$ is a non-negative integer number.

$H_n$ is a Hermite polynomail, you'll learn more on them in the following exercise

## Exercise 1

The first two hermite polynomials are given by

\begin{align}
H_0(x) &= 1 \\
H_1(x) &= 2x
\end{align}

The rest of the Hermite polynomials can be found using the recursion relation
\begin{align}
H_{n+1}(x) = 2xH_n(x) - 2 n H_{n-1}(x)
\end{align}

Eg. to find $H_{2}$ insert $n=1$ in the formula above and you get

\begin{align}
    H_2(x) = 2xH_1(x) - 2 H_{0}(x) = 4x^2 - 2
\end{align}

Make a function that generates the Hermite Polynomials. Using the recursion relation shown above.

To do this exercise do you need to make a recursive function, a recursive function is a function that calls itself untill a certain stop condition has been met. To read more about them check out https://www.programiz.com/python-programming/recursion or the internet.

To help you get started on this exercise use the following as a blue print.

```Python
# To get started use the following as a blueprint
def hermite_polynomial(n, x):
    if n == 0:
        return 1
    if n == 1:
        return 2*x
    else:
        return # Fill this in with the recursion relation

# Using sympy you can print out the result and compare it to the analytical solution
# that can be found on ExplainEverything WEXLRXLP

import sympy as sp

x = sp.symbols('x')
hermite_polynomial(2, x)
```

 ## Exercise 2

 Using the hermite polynomials function you made above, define the wavefunction for the Harmonic oscillator.

 Check if the wavefunction is ortonormal, i.e.,

 \begin{align}
 \int_{-\infty}^\infty |\psi_n(x)|^2 dx &= 1 \\
 \int_{-\infty}^\infty \psi_n^*(x)\psi_m(x) dx &= 0
 \end{align}

 Hint: I recommend that sympy is used to complete the rest of the exercises

 Hint 2: To integrate from $\int_{-\infty}^\infty f(x) dx $ with sympy
           `sp.integrate(f(x), (x, -sp.oo, sp.oo))`

```Python
def wavefunction(n, m, k, x):
    """ n : the wave number
        m : Mass of the partice
        k : Spring costant
        x : position of the particle
    """
    return
```

## Exercise 3

Step 1:

Plot the potential $V(x) = \frac{1}{2} k x^2$

Step 2:

Plot the potential $V(x)$ together with some energy levels of $E_n = \hbar \omega \left(n + \frac{1}{2}\right)$

Step 3:

Plot the potential $V(x)$ together with $|\psi_n|^2$ for a few values of $n$. What happens when $n$ becomes large?

## Exercise 4

In the lecture on the [virial theorem] expectation values

\begin{align}
\langle x \rangle &= 0 \\
\langle x^2 \rangle &= \hbar \omega (n + 0.5)/k \\
\langle p \rangle &= 0 \\
\langle p^2 \rangle &= m \hbar \omega (n + 0.5)
\end{align}

using physical intuition and the Virial theorem. Here I want you to make
functions that calculates these by brute force.

That is,
\begin{align}
\langle x \rangle &= \langle \psi_n|x|\psi_n\rangle \\
\langle x^2 \rangle &= \langle \psi_n|x^2|\psi_n\rangle \\
\langle p \rangle &= \langle \psi_n|p|\psi_n\rangle \\
\langle p^2 \rangle &= \langle \psi_n|p^2|\psi_n\rangle \\
\end{align}

Hint: $\langle \psi_n |a|\psi_n\rangle = \int_{-\infty}^{\infty} \psi_n^*(x)a\psi_n(x) dx$

Hint 2: To differentiate twice in sympy use
```Python
sp.diff(f(x), x, 2)```

[virial theorem]: https://expl.ai/ZLHXGRL

```Python
def expectation_x(n, m, k):

    return

def expectation_x_squared(n, m, k):

    return

def expectation_p(n, m, k):

    return

def expecation_p_squared(n, m, k):

    return
```

```python

```


# 声明符号和内置符号类型

sympy是符号运算,与一般情况下用python做数值计算的最大区别在于,在构建算式前需要先声明符号,这些符号当然可以是有多重含义的,sympy内置了几种数据类型来区别.

```python
import sympy as sp
```

## 符号类型

symbols定义了浮点数,整数,有理数,复数4种合法的数据类型,以及一些特殊的数值类型

+ 浮点数

```python
1/2
```

    0.5

+ 整数

```python
1
```

    1

+ 有理数Rational()

```python
sp.Rational(1,3)
```

$\displaystyle \frac{1}{3}$

+ 无理数 sqrt()

例如开根号,一个数如果是一个完全平方数,开根号后就是有理数；反之,是无理数.

```python
sp.sqrt(2)
```

$\displaystyle \sqrt{2}$

同时像一些常见的无理数常数比如$$e,\pi$$也都支持

也可以简单的用幂来表示

```python
2**(sp.Rational(1,2))
```

$\displaystyle \sqrt{2}$

```python
3**(sp.Rational(1,3))
```

$\displaystyle \sqrt[3]{3}$

+ 复数

除了复数单元I是虚数，符号可以被用属性创建(例如 real,positive,complex,等等)这将影响它们的表现：

```python
1+1j
```

    (1+1j)

## 内置符号声明

符号可以分为变量和常量,sympy中的常量可以直接引入,我们看一个典型的常数符号的等式,欧拉公式

$$e^{i\pi}+1=0$$

*ps:*欧拉恒等式是复分析中的欧拉公式的特例,欧拉公式如下:
$$e^{ix}=\cos x + i\sin x \\
 \text x为任意实数$$

```python
from sympy import E,I,pi#从包中导入所需常数符号
```

```python
E**(I*pi)+1
```

$\displaystyle 0$

## 自定义符号声明

声明符号使用函数`symbols(names, **args)`函数

```python
x = sp.symbols("x")
```

### 同时声明多个变量

```python
x,y,z = sp.symbols("x,y,z")
```

### 为符号指定类型

```python
x,y,z = sp.symbols("x,y,z",integer=True)
```

### 声明大量符号变量

如果用到大量的符号可以使用一些简便的方法定义有规律的符号

```python
sp.symbols('x:10')
```

    (x0, x1, x2, x3, x4, x5, x6, x7, x8, x9)

```python
sp.symbols('y5:10')
```

    (y5, y6, y7, y8, y9)

```python
sp.symbols('x(:c)')
```

    (xa, xb, xc)

```python
sp.symbols('a:z')
```

    (a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z)

```python
sp.symbols('x:2(1:3)')
```

    (x01, x02, x11, x12)

```python
sp.symbols('f((a:b))')
```

    (f(a), f(b))

```python
sp.symbols('x(:1\,:2)')  # or 'x((:1)\,(:2))'
```

    (x(0,0), x(0,1))

### 声明一个复数

复数的声明比较复杂,事实上复数是由实部虚部两部分组成的,因此必须要由两部分符号组成,而虚部的符号`i`或者`j`可以使用sympy自带的常量`I`表示,因此可以将复数作为一个算式来处理

```python
x = sp.symbols('x')
y = sp.symbols('y')
c = x+I*y
c
```

$\displaystyle x + i y$

### 声明一个函数符号

我们也可以指定符号为函数

```python
f = sp.symbols("f",cls=sp.Function)
```

```python
f = x+2*y+z
```

```python
f
```

$\displaystyle x + 2 y + z$

## 构建表达式

算式是由符号组成的运算过程,其构造过程可以有2种

### 使用符号构建表达式

符号间直接使用运算符或数学函数连接就可以构造表达式

```python
x+2*y+z
```

$\displaystyle x + 2 y + z$

### 使用字符串构建表达式

这种方式需要使用函数`sympify(exp_str,evaluate:bool)`其中`evaluate`这个参数决定是否进行化简

```python
sp.sympify("x+2*y+z")
```

$\displaystyle x + 2 y + z$

```python
exp1 = sp.sympify("x+y+z+y",evaluate=True)
exp1
```

$\displaystyle x + 2 y + z$

```python
exp2 = sp.sympify("x+y+z+y",evaluate=False)
exp2
```

$\displaystyle x + y + y + z$

### 一个例子: 推导欧拉公式

$$ e^{ix}=\cos x + i\sin x $$

这里用到了如下几个符号:
+ 运算符

```python
from sympy import cos,sin,exp
```

+ 常量

```python
from sympy import I,pi
```

+ 变量

```python
from sympy import symbols

x = symbols('x',real=False)
```

左边为:

```python
exp(I*x)
```

    exp(I*x)

右边算式为:

```python
right = cos(x)+I*sin(x)
```

```python
right
```

    I*sin(x) + cos(x)

我们用泰勒级数展开左边

```python
from sympy import series#泰勒展开函数
series(exp(I*x), x, 0, 10)#公式,变量
```

    1 + I*x - x**2/2 - I*x**3/6 + x**4/24 + I*x**5/120 - x**6/720 - I*x**7/5040 + x**8/40320 + I*x**9/362880 + O(x**10)

右边可以分为两部分,`I*sin(x)`和`cos(x)`

```python
p1 = series(I*sin(x), x, 0, 10)
p1
```

    I*x - I*x**3/6 + I*x**5/120 - I*x**7/5040 + I*x**9/362880 + O(x**10)

```python
p2 = series(cos(x), x, 0, 10)
p2
```

    1 - x**2/2 + x**4/24 - x**6/720 + x**8/40320 + O(x**10)

两部分的和为

```python
p1+p2
```

    1 + I*x - x**2/2 - I*x**3/6 + x**4/24 + I*x**5/120 - x**6/720 - I*x**7/5040 + x**8/40320 + I*x**9/362880 + O(x**10)

可见左右两边相同

```python
series(exp(I*x), x, 0, 10) == p1+p2
```

    True

src/数据分析篇/工具介绍/SymPy/声明符号和内置符号类型.ipynb

src/数据分析篇/工具介绍/SymPy/声明符号和内置符号类型.ipynb

src/数据分析篇/工具介绍/SymPy/声明符号和内置符号类型.ipynb


# Three-particle potentials

In this notebook we'll explore the Stillinger-Weber three-particle potential applied to a system of silicon (Si) atoms.
The Stillinger-Weber potential is given by a two-particle potential similar to Lennard-Jones and a three-particle potential.
\begin{align}
    U(r)
    = \sum_{ij} U(r_{ij})
    + \sum_{ijk} U(r_{ij}, r_{ik}, \theta_{ijk}).
\end{align}
The two-particle potential is given by
\begin{align}
    U(r_{ij})
    = A \epsilon\left[
        B \left(
            \frac{\sigma}{r_{ij}}
        \right)^{p}
        - \left(
            \frac{\sigma}{r_{ij}}
        \right)^{q}
    \right]
    \exp\left(
        \frac{\sigma}{r_{ij} - a\sigma}
    \right),
\end{align}
where $A$, $B$, $\epsilon$, $\sigma$, $p$, $q$ and $a$ are coefficients that can be adjusted depending on the system we are perusing.
The three-particle potential has the form
\begin{align}
    U(r_{ij}, r_{ik}, \theta_{ijk})
    = \lambda \epsilon \left[
        \cos(\theta_{ijk})
        - \cos(\theta_{0})
    \right]^2
    \exp\left(
        \frac{\gamma\sigma}{r_{ij} - a\sigma}
    \right)
    \exp\left(
        \frac{\gamma\sigma}{r_{ik} - a\sigma}
    \right),
\end{align}
where we've introduced the new parameters $\lambda$ and $\gamma$.
Note that we've simplified the potentials as we've removed the indices that can be associated with every coefficient to specify different types of interactions between atoms.
Below we use the given parameter file.

```python
import os
import re
import sys

sys.path.append("..")

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import sklearn.linear_model

from read_lammps_log import read_log, get_temp_lognames, read_rdf_log
from diffusion import get_diffusion_constant

sns.set(color_codes=True)
```

```python
%%writefile scripts/Si.sw
# DATE: 2007-06-11 CONTRIBUTOR: Aidan Thompson, athomps@sandia.gov
# CITATION: Stillinger and Weber,  Phys Rev B, 31, 5262, (1985)
# Stillinger-Weber parameters for various elements and mixtures
# multiple entries can be added to this file,
# LAMMPS reads the ones it needs
# these entries are in LAMMPS "metal" units:
#   epsilon = eV; sigma = Angstroms
#   other quantities are unitless
# format of a single entry (one or more lines):
#   element 1, element 2, element 3,
#   epsilon, sigma, a, lambda, gamma, costheta0, A, B, p, q, tol
# Here are the original parameters in metal units, for Silicon from:
#
# Stillinger and Weber,  Phys. Rev. B, v. 31, p. 5262, (1985)
#

Si # element 1
Si # element 2
Si # element 3
2.1683 # epsilon
2.0951 # sigma
1.80 # a
21.0 # lambda
1.20 # gamma
-0.333333333333 # cos(theta_0)
7.049556277 # A
0.6022245584 # B
4.0 # p
0.0 # q
0.0 # tol
```

    Overwriting scripts/Si.sw

In the Lennard-Jones-scripts we only specify $\epsilon$, $\sigma$ and $r_c$ (cut-off).
Furthermore, we see that for $A = 4$, $B = 1$, $p = 12$, and $q = 6$ we almost recover the Lennard-Jones potential in the two-particle potential if we remove the exponential cut-off function.
With $p = 4$ we lower the repulsive effect of the two-particle potential and by setting $q = 0$ we use a fixed attractive force between the particles.

```python
%%writefile scripts/si.in

units           metal
atom_style      atomic
atom_modify     map array
boundary        p p p
atom_modify     sort 0 0.0

# temperature
#variable T equal 1200.0

# diamond unit cell
variable myL equal 10
variable myscale equal 1.3
variable bins equal 200

variable a equal 5.431*${myscale}
lattice         custom $a               &
                a1 1.0 0.0 0.0          &
                a2 0.0 1.0 0.0          &
                a3 0.0 0.0 1.0          &
                basis 0.0 0.0 0.0       &
                basis 0.0 0.5 0.5       &
                basis 0.5 0.0 0.5       &
                basis 0.5 0.5 0.0       &
                basis 0.25 0.25 0.25    &
                basis 0.25 0.75 0.75    &
                basis 0.75 0.25 0.75    &
                basis 0.75 0.75 0.25

region          myreg block     0 ${myL} &
                                0 ${myL} &
                                0 ${myL}

create_box      1 myreg
create_atoms    1 region myreg

mass            1       28.06

group Si type 1

velocity all create ${T} 5287286 mom yes rot yes dist gaussian

pair_style      sw
pair_coeff * * scripts/Si.sw Si

neighbor        1.0 bin
neigh_modify    every 1 delay 10 check yes

timestep        1.0e-3
#fix             1 all nve
# Try using fix npt
fix             1 all nvt temp ${T} ${T} 0.05

# Run simulation
thermo          10
#dump 1 all custom 10 dat/si.lammpstrj id type x y z vx vy vz
run             1000

reset_timestep 0
variable time equal dt*step
compute msd all msd
compute myrdf all rdf ${bins}

fix 2 all ave/time 100 1 100 c_myrdf[*] file dat/si_rdf_${T}.log mode vector

thermo_style custom step v_time temp ke pe etotal press c_msd[4]

log dat/si_g_${T}.log

run 5000
```

    Overwriting scripts/si.in

```python
%%writefile scripts/run_si.in

export OMP_NUM_THREADS=4

for T in $(seq 1 200 6001); do
    mpirun -np 4 lmp -var T $T -in scripts/si.in
done
```

    Overwriting scripts/run_si.in

```python
temperature_list, file_list = get_temp_lognames("si_g")
```

```python
fig = plt.figure(figsize=(14, 10))

log_df_list = []

for T, filename in zip(temperature_list, file_list):
    log_df = read_log(filename)
    log_df_list.append(log_df)
    plt.plot(
        log_df["v_time"],
        log_df["c_msd[4]"],
        label=fr"$T = {T:.2f}$"
    )

plt.xlabel(r"$t$")
plt.ylabel(r"$\langle r^2(t)\rangle$")
plt.legend(loc="best")
plt.title(r"Mean squared displacement as a function of time")
plt.show()
```

```python
D_list = []
alpha_list = []

for T, log_df in zip(temperature_list, log_df_list):
    D, D_int, alpha, alpha_int = get_diffusion_constant(log_df)

    D_list.append(D)
    alpha_list.append(alpha)
```

```python
fig = plt.figure(figsize=(14, 10))

plt.plot(temperature_list, D_list)
plt.title(r"Plot of the diffusion constant as a function of temperature")
plt.xlabel(r"$T$")
plt.ylabel(r"$D(t)$")
plt.show()
```

```python
temperature_list, rdf_list = get_temp_lognames("si_rdf")
```

```python
g_r_dict = {}
bin_centers_dict = {}

for temp, filename in zip(temperature_list, rdf_list):
    bin_centers, g_r = read_rdf_log(filename)

    g_r_dict[temp] = g_r
    bin_centers_dict[temp] = bin_centers
```

```python
fig = plt.figure(figsize=(14, 10))

for i, temp in enumerate(temperature_list):
    key = max(g_r_dict[temp])

    plt.plot(
        bin_centers_dict[temp][key],
        g_r_dict[temp][key],
        label=fr"$T = {temp}$",
    )

plt.xlabel(r"$r$")
plt.ylabel(r"$g(r)$")
plt.legend(loc="best")
plt.title(r"Final radial distribution for varying temperatures")
plt.show()
```


```python
import numpy as np
import sympy as sp
import numpy.linalg as la
```

```python
def finite_difference_nd(f_str, s_str, xvec, h, scheme=0):
    '''
    scheme: 0 for forward, 1 for backward, 2 for central
    '''
    formula = sp.sympify(f_str)
    symbols = sp.symbols(s_str)
    result = []
    for i in range(len(xvec)):
        temp = formula # This kind of copy is enough
        temp_plus = formula
        temp_minus = formula
        for j in range(len(symbols)):
            temp = temp.subs(symbols[j], xvec[j])
            if i == j:
                temp_plus = temp_plus.subs(symbols[j], xvec[j]+h)
                temp_minus = temp_minus.subs(symbols[j], xvec[j]-h)
            else:
                temp_plus = temp_plus.subs(symbols[j], xvec[j])
                temp_minus = temp_minus.subs(symbols[j], xvec[j])
        if scheme == 0:
            result.append(float(((temp_plus-temp)/h).evalf()))
        elif scheme == 1:
            result.append(float(((temp-temp_minus)/h).evalf()))
        else:
            result.append(float(((temp_plus-temp_minus)/h/2).evalf()))
    return result
```

```python
# @Before
tol = 10**-7
```

```python
# Test case backward finite difference method
expected = np.array([1, 2.9, 2])
actual = np.array(finite_difference_nd('x*z+y**2*z+y', 'x y z', [1, 1, 1], 0.1, scheme=1))
assert la.norm(expected - actual) < tol
```

```python
# Test case forward finite difference method
expected = np.array([1, 3.1, 2])
actual = finite_difference_nd('x*y*z+y*y+z', 'x y z', [1, 1, 1], 0.1, 0)
assert la.norm(expected - actual) < tol
```

```python
# Workspace
finite_difference_nd('x*y*z**2+x+1', 'x y z', [1, 1, 1], 0.1, 1)
```

    [2.0000000000000018, 1.0000000000000009, 1.8999999999999995]

```python

```


## Griewank

The Griewank function has many widespread local minima, which are regularly distributed. The complexity is shown in the zoomed-in plots.

**Definition**

\begin{align}
\begin{split}
f(x) & = & \; \sum_{n=1}^n \; \frac{x_i^2}{4000} - \prod_{n=1}^n \cos \Bigg( \frac{x_i}{\sqrt{i}} \Bigg) + 1 \\[4mm]
&&-600 \leq x_i \leq 600 \quad i=1 \ldots,n \\[4mm]
\end{split}
\end{align}

**Optimum**

$$f(x^*) = 0 \; \text{at} \; x^* = (0,\ldots,0) $$

**Contour**

```python
import numpy as np
from pymoo.factory import get_problem, get_visualization

problem = get_problem("griewank", n_var=1)
plot = get_visualization("fitness-landscape", problem, _type="surface", n_samples=1000)
plot.do()
plot.apply(lambda ax: ax.set_xlim(-200,200))
plot.apply(lambda ax: ax.set_ylim(-1,13))
plot.show()
```

```python

```


Lab 1

PHYS 580 - Computational Physics

Professor Molnar

Ethan Knox

September 4, 2020

## Imports

```python
import numpy as np
from matplotlib import pyplot as plt
```

## Differential Equation

$$\frac{dN}{dt}=-\frac{N}{\tau}$$

```python
def f(y, x, tau = 1):
    return - y / tau
```

## Numerical Methods
Denoting: $\Delta x = x_{i+1}-x_i$

### Euler Method
$$y_{i+1} = y_i + f\left(y_i, x_i\right)\Delta x$$

```python
def euler(f, y0, x, *args):
    y = y0 * np.ones_like(x)

    for i in range(len(x) - 1):
        dx = x[i + 1] - x[i]
        k1 = f(y[i], x[i], *args) * dx
        y[i + 1] = y[i] + k1
    return y
```

### RK2 Method
$$
\begin{align}k_1 &= f\left(y_i, x_i\right)\Delta x\\
             k_2 &= f\left(y_i + k_1, x_i + \delta x\right)\Delta x\\
\end{align}
$$
$$y_{i+1} = y_i + \frac{1}{2}\left(k_1 + k_2\right)$$

```python
def rk2(f, y0, x, *args):
    y = y0 * np.ones_like(x)

    for i in range(len(x) - 1):
        dx = x[i + 1] - x[i]
        k1 = f(y[i], x[i], *args) * dx
        k2 = f(y[i] + k1, x[i] + dx, *args) * dx
        y[i + 1] = y[i] + 0.5 * (k1 + k2)
    return y
```

### RK4 Method
$$
\begin{align}k_1 &= \Delta x f\left(y_i, x_i\right)\\
             k_2 &= \Delta x f\left(y_i + \frac{k_1}{2}, x_i + \frac{\Delta x}{2}\right)\\
             k_3 &= \Delta x f\left(y_i + \frac{k_2}{2}, x_i + \frac{\Delta x}{2}\right)\\
             k_4 &= \Delta x f\left(y_i + k_3, x_i + \Delta x\right)\\
\end{align}
$$
$$y_{i + 1} = y_i + \frac{1}{6}\left(k_1 + 2\left(k_2 + k_3\right) + k_4\right)$$

```python
def rk4(f, y0, x, *args):
    y = y0 * np.ones_like(x)

    for i in range(len(x) - 1):
        dx = x[i + 1] - x[i]
        k1 = f(y[i], x[i], *args) * dx
        k2 = f(y[i] + 0.5 * k1, x[i] + 0.5 * dx, *args) * dx
        k3 = f(y[i] + 0.5 * k2, x[i] + 0.5 * dx, *args) * dx
        k4 = f(y[i] + k3, x[i] + dx, *args) * dx
        y[i + 1] = y[i] + (k1 + 2 * (k2 + k3) + k4) / 6
    return y
```

## Analytical Solution

$$N\left(t\right)=N_0e^{\frac{-x}{\tau}}$$

```python
def nexact(x, tau = 1, N0 = 1000):
    return N0 * np.exp(-x / tau)
```

## Error
### Global

```python
def global_error(calculated, exact):
    return np.cumsum(calculated - exact)
```

### Local

```python
def local_error(y_exact, y_approx, x):
    error = np.zeros_like(x)
    for i in np.arange(1, len(error)):
        error[i-1] = y_exact(x[i]) - y_exact(x[i-1]) - (y_approx[i] - y_approx[i-1])
    return error
```

## Code

#### Parameters

```python
N0 = 1000  # Initial Condition
tau = 1  # Rate Parameter

t_i = 0  # Initial Time
t_f = 5 * tau  # Final Time

ratios = 1.0e-1*np.asarray([0.05, 0.2, 0.5, 1.0, 1.5])
labels = [rf'$\Delta t/\tau = {ratio:0.4f}$' for ratio in ratios]
```

#### Accumulate Data

```python
t = [np.arange(t_i, t_f, ratio * tau) for ratio in ratios]

N_euler = [euler(f, N0, t_, tau) for t_ in t]
N_rk2 = [rk2(f, N0, t_, tau) for t_ in t]
N_rk4 = [rk4(f, N0, t_, tau) for t_ in t]
```

#### Plotting

```python
fig1, (ax1, ax2, ax3) = plt.subplots(3, 1, figsize=(16,16))

for i, label in enumerate(labels):
    ax1.plot(t[i] / tau, N_euler[i], label=label)
    ax2.plot(t[i] / tau, N_rk2[i], label=label)
    ax3.plot(t[i] / tau, N_rk4[i], label=label)

for ax in (ax1, ax2, ax3):
    ax.plot(t[0] / tau, nexact(t[0], tau, N0), c='k',ls='--', label="Exact")
    ax.set_xlabel(r'$t/\tau$')
    ax.legend()
    ax.grid()

ax1.set_ylabel(r'Euler $N(t)$')
ax2.set_ylabel(r'RK2 $N(t)$')
ax3.set_ylabel(r'RK4 $N(t)$')

plt.tight_layout()
plt.savefig("Lab1_results.png")
```

```python
plt.cla()
plt.clf()
```

```python
fig2, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(16,16))

for i, label in enumerate(labels):
    ax1.plot(t[i] / tau, np.absolute(local_error(nexact, N_euler[i], t[i]/tau)), label=label)
    ax2.plot(t[i] / tau, np.absolute(local_error(nexact, N_rk2[i], t[i]/tau)), label=label)
    ax3.plot(t[i] / tau, np.absolute(local_error(nexact, N_rk4[i], t[i]/tau)), label=label)

for ax in (ax1, ax2, ax3):
    ax.set_xlabel(r'$t/\tau$')
    ax.legend()
    ax.grid()

ax1.set_ylabel("Euler Local Error")
ax2.set_ylabel("RK2 Local Error")
ax3.set_ylabel("RK4 Local Error")

plt.tight_layout()
plt.savefig("Lab1_local_error.png")
```

```python
plt.cla()
plt.clf()
```

```python
fig3, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(16,16))

for i, label in enumerate(labels):
    ax1.plot(t[i] / tau, np.absolute(global_error(N_euler[i], nexact(t[i], tau, N0))), label=label)
    ax2.plot(t[i] / tau, np.absolute(global_error(N_rk2[i], nexact(t[i], tau, N0))), label=label)
    ax3.plot(t[i] / tau, np.absolute(global_error(N_rk4[i], nexact(t[i], tau, N0))), label=label)

for ax in (ax1, ax2, ax3):
    ax.set_xlabel(r'$t/\tau$')
    ax.legend()
    ax.grid()

ax1.set_ylabel("Euler Global Error")
ax2.set_ylabel("RK2 Global Error")
ax3.set_ylabel("RK4 Global Error")

plt.tight_layout()
plt.savefig("Lab1_global_error.png")
```

## Conclusion

It's very apparent that in both local and global errors, both Runge-Kutta methods outperformed Euler's method for a given timestep. Further, RK4 outperformed RK2 in the same manner.

```python

```

```python

```

```python
plt.cla()
plt.clf()
```

```python
fig3, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(16,16))

for i, label in enumerate(labels):
    ax1.plot(t[i] / tau, np.absolute(global_error(N_euler[i], nexact(t[i], tau, N0))), label=label)
    ax2.plot(t[i] / tau, np.absolute(global_error(N_rk2[i], nexact(t[i], tau, N0))), label=label)
    ax3.plot(t[i] / tau, np.absolute(global_error(N_rk4[i], nexact(t[i], tau, N0))), label=label)

for ax in (ax1, ax2, ax3):
    ax.set_xlabel(r'$t/\tau$')
    ax.legend()
    ax.grid()

ax1.set_ylabel("Euler Global Error")
ax2.set_ylabel("RK2 Global Error")
ax3.set_ylabel("RK4 Global Error")

plt.tight_layout()
plt.savefig("Lab1_global_error.png")
```

## Conclusion

It's very apparent that in both local and global errors, both Runge-Kutta methods outperformed Euler's method for a given timestep. Further, RK4 outperformed RK2 in the same manner.

```python

```


Quantum Fourier Transform

```
%load_ext sympyprinting
```

```
from sympy import sqrt, symbols, Rational
from sympy import expand, Eq, Symbol, simplify, exp, sin
from sympy.physics.quantum import *
from sympy.physics.quantum.qubit import *
from sympy.physics.quantum.gate import *
from sympy.physics.quantum.grover import *
from sympy.physics.quantum.qft import QFT, IQFT, Fourier
from sympy.physics.quantum.circuitplot import circuit_plot
```

QFT is useful for a quantum algorithm for factoring numbers which is exponentially faster than what is thought to be possible on a classical machine.

The transform does a DFT on the state of a quantum system

There is a simple decomposition of the QFT in terms of a few elementary gates.

QFT Gate and Circuit

Build a 3 qubit QFT and decompose it into primitive gates.

```
fourier = QFT(0,3).decompose(); fourier

```

```
circuit_plot(fourier, nqubits=3)
```

The QFT circuit can be represented in various symbolic forms.

```
m = represent(fourier, nqubits=3)
```

```
m
```

$$\left(\begin{smallmatrix}\frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2}\\\frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi} & \frac{1}{4} \sqrt{2} \mathbf{\imath} & \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi}\\\frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} & \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} \mathbf{\imath}\\\frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} & \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi} & \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi}\\\frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2}\\\frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi} & \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} & \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi}\\\frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} \mathbf{\imath} & \frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} \mathbf{\imath}\\\frac{1}{4} \sqrt{2} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} \mathbf{\imath} & - \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi} & - \frac{1}{4} \sqrt{2} & \frac{1}{4} \sqrt{2} \mathbf{\imath} e^{\frac{1}{4} \mathbf{\imath} \pi} & \frac{1}{4} \sqrt{2} \mathbf{\imath} & \frac{1}{4} \sqrt{2} e^{\frac{1}{4} \mathbf{\imath} \pi}\end{smallmatrix}\right)$$

```
represent(Fourier(0,3), nqubits=3)*4/sqrt(2)

```

$$\left(\begin{smallmatrix}1 & 1 & 1 & 1 & 1 & 1 & 1 & 1\\1 & \omega & \omega^{2} & \omega^{3} & \omega^{4} & \omega^{5} & \omega^{6} & \omega^{7}\\1 & \omega^{2} & \omega^{4} & \omega^{6} & 1 & \omega^{2} & \omega^{4} & \omega^{6}\\1 & \omega^{3} & \omega^{6} & \omega & \omega^{4} & \omega^{7} & \omega^{2} & \omega^{5}\\1 & \omega^{4} & 1 & \omega^{4} & 1 & \omega^{4} & 1 & \omega^{4}\\1 & \omega^{5} & \omega^{2} & \omega^{7} & \omega^{4} & \omega & \omega^{6} & \omega^{3}\\1 & \omega^{6} & \omega^{4} & \omega^{2} & 1 & \omega^{6} & \omega^{4} & \omega^{2}\\1 & \omega^{7} & \omega^{6} & \omega^{5} & \omega^{4} & \omega^{3} & \omega^{2} & \omega\end{smallmatrix}\right)$$

QFT in Action

Build a 3 qubit state to take the QFT of.

```
state = (Qubit('000') + Qubit('010') + Qubit('100') + Qubit('110'))/sqrt(4); state

```

Perform the QFT.

```
qapply(fourier*state)

```