```python
import numpy as np
import matplotlib.pyplot as plt
import scipy.stats as st
import warnings
warnings.filterwarnings("ignore")
```

* Please email me (ianzhang@connect.hku.hk) if you find any mistakes in the script.

* My consultation timeslot: Wednesday, 16:00 - 17:00, RR113.

# Data Augmentation

Given the model
$$p(\theta,z|\mathbf{y}_{\mathrm{obs}}),$$
we can obtain the **complete-data posterior distribution**
$$p(\theta|\mathbf{y}_{\mathrm{obs}}, z),$$
and the **conditional predictive distribution**
$$p(z|\mathbf{y}_{\mathrm{obs}}, \theta).$$
Our goal is to draw samples from $p(\theta|\mathbf{y}_{\mathrm{obs}})$, where
$$
p(\theta|\mathbf{y}_{\mathrm{obs}})=\int p(\theta,z|\mathbf{y}_{\mathrm{obs}})dz.
$$

## The original DA algorithm (multiple imputation)

I-step: Draw samples $\{\theta^{(k,j)}\}_{j=1}^m$ from $p_k(\theta|\mathbf{y}_{\mathrm{obs}})$, and obtain $\{z^{(k,j)}\}_{j=1}^m$, where $z^{(k,j)}\sim p(z|\mathbf{y}_{\mathrm{obs}},\theta^{(k,j)})$.

P-step: Update the posterior:
$$
p_{k+1}(\theta|\mathbf{y}_{\mathrm{obs}})=\frac{1}{m}\sum_{j=1}^mp(\theta|\mathbf{y}_{\mathrm{obs}},z^{(k,j)}).
$$

## The DA algorithm

Some insights about the original DA algorithm:
* The P-step can be integrated into the I-step.
* The I-step can be modified in the following way:

(1) Draw samples $\{\theta^{(k,j)}\}_{j=1}^m$ from $p_k(\theta|\mathbf{y}_{\mathrm{obs}})$, where
$$
p_k(\theta|\mathbf{y}_{\mathrm{obs}})=\frac{1}{m}\sum_{j=1}^mp(\theta|\mathbf{y}_{\mathrm{obs}},z^{(k-1,j)}).
$$
(2) Draw samples $\{z^{(k,j)}\}_{j=1}^m$ from $p_k(z|\mathbf{y}_{\mathrm{obs}})$, where
$$
p_k(z|\mathbf{y}_{\mathrm{obs}})=\frac{1}{m}\sum_{j=1}^mp(z|\mathbf{y}_{\mathrm{obs}},\theta^{(k,j)}).
$$
But the step (2) is based on **stratification**.

What if we let the step (1) to be based on stratification too? In that case we would have

(1) Draw samples $\{\theta^{(k,j)}\}_{j=1}^m$, where $\theta^{(k,j)}\sim p(\theta|\mathbf{y}_{\mathrm{obs}},z^{(k-1,j)})$.

(2) Draw samples $\{z^{(k,j)}\}_{j=1}^m$, where $z^{(k,j)}\sim p(z|\mathbf{y}_{\mathrm{obs}},\theta^{(k,j)})$.

And we find that this is equivalent to do $m$ **parallel Gibbs**!

So why not just simplify it to be one single Gibbs:

(1) Draw sample $\theta^{(k)}$, where $\theta^{(k)}\sim p(\theta|\mathbf{y}_{\mathrm{obs}},z^{(k-1)})$.

(2) Draw sample $z^{(k)}$, where $z^{(k)}\sim p(z|\mathbf{y}_{\mathrm{obs}},\theta^{(k)})$.

This is the **standard DA algrithm**. We can also draw $z$ first, just choose the convenient one.

## Example: grouped count data (missing data)

Given the observed data (\# of passages, \# of observations): \{(0,139),(1,128),(2,55),(3,25),(4+,13)\}

Assume the **Poisson model**:
$$
P(X=x)=\frac{e^{-\lambda}\lambda^x}{x!},\; x=0,1,\dots
$$
for the \# of passages. Try to estimate $\lambda$.

There are $347=139+128+55+25$ observations $\mathbf{x}_{\mathrm{obs}}$ and 13 missing data $z_{1:13}$.

If we assume the prior $p(\lambda)\propto 1/\lambda$, the model is
$$
p(\lambda,z_{1:13}|\mathbf{x}_{\mathrm{obs}})\propto\frac{1}{\lambda}e^{-347\lambda}\lambda^{128+55\times2+25\times3}\prod_{j=1}^{13}\frac{e^{-\lambda}\lambda^{z_j}}{z_j!}\mathbf{1}(z_j\ge4).
$$
so, we can obtain the complete-data posterior distribution
$$\lambda|\mathbf{x}_{\mathrm{obs}}, z_{1:13}\sim Gamma(313+\sum_{j=1}^{13}z_j,360),$$
and the conditional predictive distribution
$$p(z_j|\mathbf{y}_{\mathrm{obs}}, \lambda)=p(z_j|\lambda)\propto\frac{e^{-\lambda}\lambda^{z_j}}{z_j!}\mathbf{1}(z_j\ge4).$$

Based on the Gibbs, we can obtain the samples $\{\lambda^{(i)},z_{1:13}^{(i)}\}_{i=1}^{n}$.

* Assume $\lambda^{(0)}=\frac{313+13\times4}{360}$. Compare the two estimators of $\lambda$: (1) $\hat{\lambda}_1=\frac{1}{n}\sum_{i=1}^n\lambda^{(i)}$; (2) $\hat{\lambda}_2=\frac{1}{n}\sum_{i=1}^n\frac{313+\sum_{j=1}^{13}z_j^{(i)}}{360}$.

```python
np.random.seed(19971107)

def trunc_poisson(lbd, minval, size):
    cut = st.poisson(mu=lbd).cdf(minval - 1)
    u = st.uniform(loc=cut, scale=1 - cut).rvs(size=size)
    return st.poisson(mu=lbd).ppf(u)

lbd0 = (313 + 13*4) / 360
n = 5000
lbds = np.zeros(n)
lbds = np.append(lbd0, lbds)
z13s = np.zeros([n, 13])
for i in range(n):
    z13s[i] = trunc_poisson(lbd=lbds[i], minval=4, size=13)
    lbds[i + 1] = st.gamma(a=313 + z13s[i].sum(), scale=1 / 360).rvs()

lbds = lbds[1:]
z13s = (313 + z13s.sum(axis=1)) / 360

lbds_cummean = np.cumsum(lbds) / (np.arange(n) + 1)
z13s_cummean = np.cumsum(z13s) / (np.arange(n) + 1)

plt.plot(lbds_cummean, label='estimator 1')
plt.plot(z13s_cummean, label='estimator 2')
plt.legend()
plt.show()
```

## Example: probit model (latent variables)

Other than the logistic model, another classic model in binary classification is called the **probit model**:
$$
P(Y=1|x)=\Phi(x^T\beta),
$$
where $Y$ is a binary label, x is the vector of features, $\beta$ is the model parameter and $\Phi$ is the cdf of the standard normal distribution.

Given data $\{(y_1,x_1),\dots,(y_m,x_m)\}$, and the non-informative prior $p(\beta)\propto 1$, the model is
$$
p(\beta|y_{1:m},x_{1:m})\propto p(y_{1:m},\beta|x_{1:m})=p(y_{1:m}|\beta,x_{1:m})p(\beta)=\prod_{j=1}^m\Phi(x_j^T\beta)^{y_j}(1-\Phi(x_j^T\beta))^{1-y_j},
$$
and we can apply the Random Walk to do inference about $\beta$.

Another way is to use the data augmentation with the Gibbs sampler. To achieve this, we assume the latent variables $z_j=x_j^T\beta+\varepsilon_j$, $j=1,\dots,m$, where $\varepsilon_j\sim N(0,1)$. And we assume $y_j$ depends on $z_j$ deterministically that $y_j=\mathbf{1}(z_j>0)$. To see why this assumption holds, just notice that
$$
P(Y=1|x)=P(\mathbf{1}(Z>0)=1|x)=P(x^T\beta+\varepsilon>0)=P(-\varepsilon<x^T\beta)=\Phi(x^T\beta).
$$
So, the augmented model is
\begin{align}
p(\beta,z_{1:m}|y_{1:m},x_{1:m})&\propto p(y_{1:m},z_{1:m},\beta|x_{1:m})=p(y_{1:m}|z_{1:m})p(z_{1:m}|\beta,x_{1:m})p(\beta)\\
&=\prod_{j=1}^m(\mathbf{1}(z_j>0)\mathbf{1}(y_j=1)+\mathbf{1}(z_j\le0)\mathbf{1}(y_j=0))\exp(-\frac{(z_j-x_j^T\beta)^2}{2}).
\end{align}
Easy to derive that
$$
\beta|z_{1:m},y_{1:m},x_{1:m}\sim N((X^TX)^{-1}X^Tz_{1:m},(X^TX)^{-1}),
$$
where $X=[x_{1:m}]_{m\times p}$, and
$$
p(z_j|\beta,y_{1:m},x_{1:m})\propto(\mathbf{1}(z_j>0)\mathbf{1}(y_j=1)+\mathbf{1}(z_j\le0)\mathbf{1}(y_j=0))\exp(-\frac{(z_j-x_j^T\beta)^2}{2}).
$$

Based on the Gibbs, we can obtain the samples $\{\beta^{(i)},z_{1:m}^{(i)}\}_{i=1}^{n}$.

* Draw $x_j\sim N(\mathbf{0},I_2)$, $j=1,\dots,20$, and draw $y_{1:20}$ from the probit model with $\beta_0=[-1,2]^T$. Try to draw samples of $\beta$ given $x_{1:20},y_{1:20}$. Compare the trajectories of the Random Walk and the Gibbs. Which one is better?

```python
np.random.seed(19971107)
X = st.multivariate_normal(mean=np.zeros(2)).rvs(20)
beta0 = [-1, 2]
y = 1.0 * (X.dot(beta0) + st.norm.rvs(size=X.shape[0]) > 0)

def logtarget(beta):
    tmp = y * np.log(st.norm.cdf(X.dot(beta))) + (1 - y) * np.log(1 - st.norm.cdf(X.dot(beta)))
    return tmp.sum()

def trunc_norm(beta):
    lb = np.log(y == 1)
    ub = -np.log(y != 1)
    loc = X.dot(beta)
    return st.truncnorm(a=lb - loc, b=ub - loc, loc=loc).rvs()

def RW(size, step):
    beta = np.zeros([size + 1, 2])
    for i in range(size):
        beta_new = beta[i] + st.norm(scale=step).rvs(size=2)
        diff = logtarget(beta_new) - logtarget(beta[i])
        beta[i + 1] = beta_new if np.log(st.uniform.rvs()) <= diff else beta[i]

    return beta

def Gibbs(size):
    beta = np.zeros([size + 1, 2])
    Z = np.zeros([size, y.size])
    invXTX = np.linalg.inv(X.T.dot(X))
    for i in range(size):
        Z[i] = trunc_norm(beta[i])
        beta[i + 1] = st.multivariate_normal(mean=invXTX.dot(X.T).dot(Z[i]), cov=invXTX).rvs()

    return beta
```

```python
np.random.seed(19971107)

beta_RW = RW(2000, 0.8)
beta_Gibbs = Gibbs(2000)

fig, ax = plt.subplots(2, 2, figsize=[15, 10])
ax[0, 0].plot(-1 * np.ones(beta_RW.shape[0]))
ax[0, 1].plot(2 * np.ones(beta_RW.shape[0]))
ax[1, 0].plot(-1 * np.ones(beta_RW.shape[0]))
ax[1, 1].plot(2 * np.ones(beta_RW.shape[0]))

ax[0, 0].plot(beta_RW[:, 0], label='RW beta1')
ax[0, 1].plot(beta_RW[:, 1], label='RW beta2')
ax[1, 0].plot(beta_Gibbs[:, 0], label='Gibbs beta1')
ax[1, 1].plot(beta_Gibbs[:, 1], label='Gibbs beta2')
for a in ax.flatten():
    a.legend()

fig.show()
```

```python

```

# Simulating Annealing

For any distribution $\pi(x)$, its negative log function is called the corresponding **energy function**:
$$
h(x)=-\log(\pi(x)).
$$
In the simulated annealing, based on some intuitions from physics, we redefine the distribution with a new parameter $T$ - **the temperature**:
$$
\pi_T(x)=\exp(-\frac{h(x)}{T})(=\pi(x)^{1/T}).
$$

An example:

```python
pi = lambda x: 0.5 * st.norm(loc=-2, scale=0.2).pdf(x) + 0.5 * st.norm(loc=1, scale=0.4).pdf(x)
h = lambda x: -np.log(pi(x))

grid_x = np.linspace(-4, 4, 1000)
fig, ax = plt.subplots(2, 1, figsize=[8, 5])
ax[0].plot(grid_x, pi(grid_x))
ax[1].plot(grid_x, h(grid_x))
plt.show()
```

What if $T$ goes to 0 or infinity?

```python
piT = lambda x, T: np.exp(-h(x) / T)
hight = piT(-2, T=1)

grid_x = np.linspace(-4, 4, 1000)
fig, ax = plt.subplots(figsize=[8, 4])
ax.plot(grid_x, piT(grid_x, T=0.001) * hight / piT(-2, T=0.001), label='T=0.001')
ax.plot(grid_x, piT(grid_x, T=0.1) * hight / piT(-2, T=0.1), label='T=0.1')
ax.plot(grid_x, piT(grid_x, T=1), label='T=1')
ax.plot(grid_x, piT(grid_x, T=10) * hight / piT(-2, T=10), label='T=10')
ax.plot(grid_x, piT(grid_x, T=1000) * hight / piT(-2, T=1000), label='T=1000')
ax.legend()
plt.show()
```

The idea of **Simulating Annealing** is simply to run the **Random Walk** against a redefined target distribution $\pi_T(x)$, and gradually reduce $T$ as the algorithm proceeds. As the temperature decreases, the region around the maximum point would have higher and higher probability to capture the last sample point.

* Run the Random Walk against the above distribution with Gaussian kernel and step size 2, and set $T=2/n$, where $n$ is iteration number. Begin the walk at the local maximum $x_0=1$ and the total iteration time is $N=200$. What's the probability that the last sample point is trapped by the area around the global maximum point $x=-2$. Run 100 repetitions to estimate this probability and draw all the trajectories of the Random Walk.

```python
np.random.seed(19971107)

def walk(n, x_old):
    T = 2 / n
    x_new = x_old + st.norm(scale=2).rvs()
    ratio = piT(x_new, T) / piT(x_old, T)
    return x_new if st.uniform.rvs() <= ratio else x_old

def annealing(N):
    x = [1]
    for n in range(N):
        x.append(walk(n + 1, x[-1]))

    return x

rep = 100
N = 200
X = np.zeros([rep, N + 1])
for i in range(rep):
    if (i + 1) % 10 == 0:
        print(i + 1, end=' ')

    X[i] = annealing(N)

fig, ax = plt.subplots(2, 1, figsize=[8, 8])
ax[0].plot(X.T)
ax[1].hist(X[:, -1], bins=20)
plt.show()
```

**Think further:** Try different parameter settings to see how different parameters influence the annealing.

```python

```

# Root Finding

A common root ﬁnding problem in statistics: assume
$$
l(\theta)=\log(p(y_{\mathrm{obs}}|\theta)),
$$
and solve $f(\theta)=l'(\theta)=0$ to obtain MLE.

## Bisection method

The assumptions of the **Bisection method** are
1. $f(\theta)$ is one dimensional;
2. $f(\theta)$ is continuous;
3. We have initial points $a, b$ such that $f(a)f(b)<0$.

## Convex and concave functions

```python
x = np.linspace(-1, 1, 1000)
fv = lambda x: np.abs(x)
fc = lambda x: - x**2

fig, ax = plt.subplots(1, 2, figsize=[12, 4])
ax[0].plot(x, fv(x))
ax[0].set_title('convex')
ax[1].plot(x, fc(x))
ax[1].set_title('concave')
plt.show()
```

## Newton–Raphson method

Based on the **Taylor expansion**, we have
$$
f(\theta)=f(\theta_0)+f'(\theta_0)(\theta-\theta_0)+\mathcal{o}(\theta-\theta_0).
$$
If we ignore $\mathcal{o}(\theta-\theta_0)$, $f(\theta_1)=0$ induce
$$
\theta_1=\theta_0-\frac{f(\theta_0)}{f'(\theta_0)}.
$$
This makes an iteration of the **Newton–Raphson algorithm**.

### Multiple dimensions

If $\theta$ is $d$-dimensional, $f(\theta)=\nabla l(\theta)$ would be a multivalued function on a $d$-dimensional space, but we still have the Taylor expansion
$$
\nabla l(\theta)=\nabla l(\theta_0)+\nabla^2 l(\theta_0)(\theta-\theta_0)+\mathcal{o}(||\theta-\theta_0||),
$$
which results in an iteration of the Newton–Raphson algorithm in the $d$-dimensional space
$$
\theta_1=\theta_0+I(\theta_0)^{-1}\nabla l(\theta_0),
$$
where $I(\theta_0)=-\nabla^2 l(\theta_0)$ is called the **observed information matrix**. Different from the Bisection method, the Newton–Raphson algorithm can be easily generalized to high dimension space.

To speed up the algrithm, a simple idea called the **Fisher scoring algorithm** replaces $I(\theta_0)$ by $J(\theta_0)=\mathbb{E}[I(\theta_0)]$ which is called the **Fisher/expected information matrix**. This replacement is valid because
\begin{align}
\frac{1}{n}I(\theta_0)&=-\frac{1}{n}\nabla^2 l(\theta_0)=-\frac{1}{n}\nabla^2 \log(p(y_{\mathrm{obs}}|\theta_0))=-\frac{1}{n}\nabla^2 \log(\prod_{i=1}^np(y_i|\theta_0))=-\frac{1}{n}\sum_{i=1}^n\nabla^2\log(p(y_i|\theta_0))\\
&\approx-\mathbb{E}[\nabla^2\log(p(y_i|\theta_0))]=\frac{1}{n}\mathbb{E}[I(\theta_0)]=\frac{1}{n}J(\theta_0),
\end{align}
and
$$
I(\theta_0)^{-1}\nabla l(\theta_0)=(\frac{1}{n}I(\theta_0))^{-1}\frac{1}{n}\nabla l(\theta_0)\approx(\frac{1}{n}J(\theta_0))^{-1}\frac{1}{n}\nabla l(\theta_0)=J(\theta_0)^{-1}\nabla l(\theta_0).
$$

```python

```

STAT6011 Computational Statistics/Tut 9/Tut 9 (with answers).ipynb


Stability of defection, optimisation of strategies and the limits of memory in the PD.
-----------------------

## 1. Utility and Best responses to memory-one players

```python
import opt_mo
import numpy as np
import sympy as sym

sym.init_printing()

```

    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/sklearn/externals/joblib/__init__.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
      warnings.warn(msg, category=DeprecationWarning)

```python
import itertools
import axelrod as axl

from fractions import Fraction

```

```python
import pandas as pd

```

```python
import matplotlib.pyplot as plt
import matplotlib

```

```python
font = {"size": 10, "weight": "bold"}
matplotlib.rc("font", **font)

```

```python
p_1, p_2, p_3, p_4 = sym.symbols("p_1, p_2, p_3, p_4")
q_1, q_2, q_3, q_4 = sym.symbols("q_1, q_2, q_3, q_4")

```

```python
p = (p_1, p_2, p_3, p_4)
q = (q_1, q_2, q_3, q_4)

```

Utility
-------

The markov chain $M$.

```python
M = opt_mo.mem_one_match_markov_chain(player=p, opponent=q)

```

```python
with open("../tex/m_matrix.tex", "w") as f:
    f.write("M = {}".format(sym.latex(sym.Matrix(M))))

```

Let $v$ be the stationary vector of $M$, where $M$ is the transition matrix $M(p,q)$.

```python
pi_1, pi_2, pi_3, pi_4 = sym.symbols("pi_1, pi_2, pi_3, pi_4")
pi = (pi_1, pi_2, pi_3, pi_4)

```

```python
ss = opt_mo.steady_states(M, pi)

```

Once the markovian matrix is defined we can calculate the stable states of the matrix.
The stable states multiplied  by the scores of each state return the score of a player. The payoff matrices for $X$ and $Y$ equivilant are: $S_{p} = (R, S, T, P)$ & $S_{q} = (R, S, T, P)$

```python
v = sym.Matrix([[ss[pi_1]], [ss[pi_2]], [ss[pi_3]], [ss[pi_4]]])

```

```python
R, S, T, P = sym.symbols("R, S, T, P")
payoffs = np.array([R, S, T, P])

```

```python
utility = v.dot(np.array([3, 0, 5, 1]))
```

To test this we can have a defector play against a defector. We know that the score should be 1. Similar for two cooperators the score should be 3.

```python
assert (
    utility.subs({p_1: 0, p_2: 0, p_3: 0, p_4: 0, q_1: 0, q_2: 0, q_3: 0, q_4: 0}) == 1
)
assert (
    utility.subs({p_1: 0, p_2: 0, p_3: 0, p_4: 0, q_1: 1, q_2: 1, q_3: 1, q_4: 1}) == 5
)
assert (
    utility.simplify().subs(
        {p_1: 1, p_2: 1, p_3: 1, p_4: 1, q_1: 1, q_2: 1, q_3: 1, q_4: 1}
    )
    == 3
)

```

**Comparison to Press and Dyson formulation.**

In Press and Dyson's work on their 2012 article. They have managed to prove
that:

 $$v \times S_{p} = \frac{D (p, q, S_{p})} {D (p, q, 1)}$$

where $D$ is a $4 \times 4$ the determinant of a defined matrix $B$.

```python
S_p = sym.Matrix([[3, 0, 5, 1]])
B = opt_mo.make_B(S_p, (p_1, p_2, p_3, p_4), (q_1, q_2, q_3, q_4))

```

```python
def pd_utility(S_p, x, y):
    """
    Returns the utility as a function
    of p, q vectors. Based on Press and Dyson, 2012.
    """
    B = sym.Matrix(opt_mo.make_B(S_p, x, y))
    unit = sym.Matrix(opt_mo.make_B((1, 1, 1, 1), x, y))
    return B.det() / unit.det()

```

```python
x = (0, 0, 0, 0)
u = pd_utility(S_p, x, (q_1, q_2, q_3, q_4))

```

```python
assert (
    u == utility.subs({S: 0, R: 3, T: 5, P: 1, p_1: 0, p_2: 0, p_3: 0, p_4: 0}).factor()
)

```

We have studied the numerator and the denominator of $u$ separately.
Both the numerator and the denominator can be written as a general quadratic
form:

 $$\frac{1}{2}pQp^T + c p^T + a$$

where $p$ is an $n \times 1$ column vector of variables, $Q$ is a symetric $n \times n$ matrix and a real-valued, $n$-dimensional vector $c$.

Thus the objective function $u$ is the fraction/ratio of the two quadratic forms.

Information on quadratic programming can be found [here](https://neos-guide.org/content/quadratic-programming,http://www.numerical.rl.ac.uk/people/nimg/qp/qp.html).

**Our formulation.**

```python
S_p = sym.Matrix([[R, S, T, P]])
```

```python
utility = pd_utility(S_p, p, q)

```

```python
other_formulation = v.dot(np.array([R, S, T, P]))
```

```python
expr = other_formulation.simplify()
```

```python
(utility - expr).simplify()
```

```python
numerator = sym.fraction(utility.simplify())[0]
denominator = sym.fraction(utility.simplify())[1]

```

```python
numerator
```

```python
numerator_elements = [[numerator.coeff(f1 * f2).factor() * f1 * f2 for f2 in p] for f1 in p]
numerator_elements
```

```python
flat_elements = list(itertools.chain.from_iterable(numerator_elements))
```

```python
cross_prod = sum(flat_elements) / 2
```

```python
cross_prod
```

**Numerator**

```python
ps = np.array(p)
Q_num = np.array([[numerator.coeff(f1 * f2) for f2 in p] for f1 in p])
c_num = np.array(
    [
        numerator.subs({p_2: 0, p_3: 0, p_4: 0}).coeff(p_1),
        numerator.subs({p_1: 0, p_3: 0, p_4: 0}).coeff(p_2),
        numerator.subs({p_1: 0, p_2: 0, p_4: 0}).coeff(p_3),
        numerator.subs({p_1: 0, p_2: 0, p_3: 0}).coeff(p_4),
    ]
)

```

```python
expr = np.dot(ps, Q_num.dot(ps.T) * 1 / 2) + np.dot(c_num, ps.T)
constant = numerator.expand() - expr

```

```python
constant.simplify()

```

```python
assert (expr + constant).subs(
    {p_1: 0, p_2: 0, p_3: 0, p_4: 0, q_1: 1, q_2: 1, q_3: 1, q_4: 1}
) == T

```

```python
Q_num = sym.Matrix([[element.factor() for element in row] for row in Q_num])
c_num = sym.Matrix([element.factor() for element in c_num])

```

```python
Q_num
```

```python
c_num
```

```python
for file, name in zip(
    [Q_num, c_num, constant.simplify()],
    ["q_numerator", "c_numerator", "numerator_constant"],
):

    with open("../tex/{}.tex".format(name), "w") as f:
        f.write("{}".format(sym.latex(file)))

```

**Denominator**

```python
Q_den = np.array([[denominator.coeff(f1 * f2) for f2 in p] for f1 in p])
c_den = np.array(
    [
        denominator.subs({p_2: 0, p_3: 0, p_4: 0}).coeff(p_1),
        denominator.subs({p_1: 0, p_3: 0, p_4: 0}).coeff(p_2),
        denominator.subs({p_1: 0, p_2: 0, p_4: 0}).coeff(p_3),
        denominator.subs({p_1: 0, p_2: 0, p_3: 0}).coeff(p_4),
    ]
)

```

```python
expr = np.dot(ps, Q_den.dot(ps.T) * 1 / 2) + np.dot(c_den, ps.T)
den_constant = denominator.expand() - expr

```

```python
assert (expr + den_constant).subs(
    {p_1: 0, p_2: 0, p_3: 0, p_4: 0, q_1: 1, q_2: 1, q_3: 1, q_4: 1}
) == 1

```

```python
Q_den = sym.Matrix([[element.factor() for element in row] for row in Q_den])
c_den = sym.Matrix([element.factor() for element in c_den])

```

```python
Q_den
```

```python
for file, name in zip(
    [sym.Matrix(Q_den), sym.Matrix(c_den), den_constant.simplify()],
    ["q_denominator", "c_denominator", "denominator_constant"],
):

    with open("../tex/{}.tex".format(name), "w") as f:
        f.write("{}".format(sym.latex(file)))

```

**Validation**

```python
qs = [(1 / 3, 1 / 3, 1 / 3, i) for i in np.linspace(0, 1, 20)]

```

```python
for p, filename in zip(
    [[0, 1, 0, 1], [0, 2 / 3, 1 / 3, 0]], ["against_player_one", "against_player_two"]
):

    theoritical = [opt_mo.match_utility(p, q) for q in qs]
    simulated = [opt_mo.simulate_match_utility(p, q) for q in qs]

    xlabels = [str(Fraction(i).limit_denominator()) for i in np.linspace(0, 1, 20)]
    xlabels[1:-1] = [
        r"$\frac{" + i.split("/")[0] + "}{" + i.split("/")[1] + "}$"
        for i in xlabels[1:-1]
    ]

    title = str([str(Fraction(i).limit_denominator()) for i in p])
    title = title.replace("[", "(").replace("]", ")").replace("'", "")

    plt.figure(figsize=[6.8, 4])
    plt.plot(
        range(len(theoritical)),
        theoritical,
        linestyle="--",
        color="red",
        linewidth=2,
        zorder=1,
        label="$u_q(p)$",
    )
    plt.scatter(
        range(len(simulated)),
        simulated,
        marker="o",
        s=60,
        color="black",
        zorder=2,
        label="$U_q(p)$",
        linewidth="2",
    )
    plt.title(r"$p={}$".format(title)), plt.legend()

    plt.xticks(range(len(theoritical)), xlabels, rotation=0, fontsize=15), plt.xlabel(
        r"$q_4$"
    )

    plt.savefig("../img/validation_{}.pdf".format(filename), bbox_inches="tight")

```

**Tournament Utility**

```python
opponents = [(0, 0, 0, 0), (1, 0, 1, 0)]

```

```python
-opt_mo.tournament_utility([1, 1, 1, 1], opponents)

```

**Stewart \& Plotkin tournament**

```python
transitions = [
    (1, 1, 1, 1),
    (0, 0, 0, 0),
    (1 / 2, 1 / 2, 1 / 2, 1 / 2),
    (1, 0, 1, 0),
    (1, 0, 0, 0),
    (1, 1 / 3, 1, 1 / 3),
    (1, 0, 0, 1),
    (1, 1 / 8, 1, 1 / 4),
    (8 / 9, 1 / 2, 1 / 3, 0),
    (0.9, 0, 0.9, 0),
]

```

```python
ps = [(1 / 3, 1 / 3, 1 / 3, i) for i in np.linspace(0, 1, 20)]

```

```python
simulated_scores = []
for player in ps:
    score = opt_mo.simulate_tournament_utility(
        player, transitions, turns=100, repetitions=20
    )
    simulated_scores.append(score)

```

    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))
    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/axelrod/result_set.py:413: RuntimeWarning: invalid value encountered in true_divide
      / sum(map(np.array, self.match_lengths))

```python
theoretical_values = []
for player in ps:
    utility = opt_mo.tournament_utility(player, transitions)
    theoretical_values.append(utility)

```

```python
plt.figure(figsize=[6.8, 4])
plt.plot(
    theoretical_values,
    linestyle="--",
    color="red",
    linewidth=2,
    zorder=1,
    label=r"$\frac{1}{10} \sum_{i = 1} ^ {10} u_q^{(i)} (p)$",
)
plt.scatter(
    range(len(simulated_scores)),
    simulated_scores,
    marker="o",
    s=60,
    color="black",
    zorder=2,
    label=r"$\frac{1}{10} \sum_{i = 1} ^ {10} U_q^{(i)} (p)$",
    linewidth="2",
)
plt.legend()

plt.title("Tournament of $N=10$"), plt.legend()

xlabels = [str(Fraction(i).limit_denominator()) for i in np.linspace(0, 1, 20)]
xlabels[1:-1] = [
    r"$\frac{" + i.split("/")[0] + "}{" + i.split("/")[1] + "}$" for i in xlabels[1:-1]
]

plt.xticks(
    range(len(theoretical_values)), xlabels, rotation=0, fontsize=15
), plt.xlabel(r"$q_4$")

plt.savefig("../img/Stewart_tournament_results.pdf", bbox_inches="tight")

```

```python
players = [[Fraction(p[i]).limit_denominator() for p in ps] for i in range(4)]

```

```python
table = pd.DataFrame(
    [
        players[0],
        players[1],
        players[2],
        players[3],
        simulated_scores,
        theoretical_values,
    ]
).transpose()

```

```python
table.columns = [
    r"$p_1$",
    r"$p_2$",
    r"$p_3$",
    r"$p_4$",
    r"$\frac{1}{10} \sum\limits_{i = 1} ^ {10} u_q^{(i)} (p)$",
    r"$\frac{1}{10} \sum\limits_{i = 1} ^ {10} U_q^{(i)} (p)$",
]

```

```python
table.head()

```

      $p_1$
      $p_2$
      $p_3$
      $p_4$
      $\frac{1}{10} \sum\limits_{i = 1} ^ {10} u_q^{(i)} (p)$
      $\frac{1}{10} \sum\limits_{i = 1} ^ {10} U_q^{(i)} (p)$

      0
      1/3
      1/3
      1/3
      0
      1.97045
      1.90388

      1
      1/3
      1/3
      1/3
      1/19
      2.0014
      1.96031

      2
      1/3
      1/3
      1/3
      2/19
      2.015
      2.00053

      3
      1/3
      1/3
      1/3
      3/19
      2.0452
      2.0304

      4
      1/3
      1/3
      1/3
      4/19
      2.0887
      2.05327

```python
textfile = open("../tex/stewart_plotkin_simulation.tex", "w")
textfile.write(table.to_latex())
textfile.close()

```

**Mean player vs Average Utility**

```python
p_one, p_two = np.linspace(0, 1, 50), np.linspace(0, 1, 50)

```

```python
axl.seed(11)
opponents = [np.random.random(4) for _ in range(4)]
mean_player = [np.mean([opponents[i][j] for i in range(4)]) for j in range(4)]

av_utility = -opt_mo.tournament_utility((p_1, p_2, p_1, p_2), opponents)
mn_utility = opt_mo.match_utility((p_1, p_2, p_1, p_2), mean_player)

plt.figure(figsize=[6.8, 4])
expr = sym.lambdify((p_1, p_2), abs(av_utility - mn_utility).simplify())

X, Y = np.meshgrid(p_one, p_two)
Z = expr(X, Y)

plt.contourf(X, Y, Z)
cbar = plt.colorbar()
plt.title("Tournament size $N=4$"), plt.xlabel("$p_1$"), plt.ylabel("$p_2$")
cbar.ax.get_yaxis().labelpad = 15
cbar.set_label(
    r"$|\frac{1}{N} \sum_{i=1} ^ {N} {u_q}^{(i)} (p) - u_{\frac {1}{N} \sum_{i=1} ^ N q^{(i)}}(p)$|",
    rotation=270,
    labelpad=+45,
    y=0.45,
)
plt.tight_layout()
plt.savefig("../img/mean_vs_average_heatmap.pdf", bbox_inches="tight")

```

```python

```

nbs/1. Utility and Best responses to memory-one players.ipynb


# Support Vector Machines

    Authors: Jesús Cid Sueiro (jcid@tsc.uc3m.es)
             Jerónimo Arenas García (jarenas@tsc.uc3m.es)

This notebook is a compilation of material taken from several sources:

- The [sklearn documentation](href = http://scikit-learn.org/stable/modules/svm.html>)
- A [notebook](https://github.com/jakevdp/sklearn_pycon2015/blob/master/notebooks/03.1-Classification-SVMs.ipynb) by [Jake Vanderplas](https://github.com/jakevdp>)
- [Wikipedia](https://en.wikipedia.org/wiki/Support_vector_machine)

    Notebook version: 1.0 (Oct 28, 2015)
                      1.1 (Oct 27, 2016)
                      2.0 (Nov 2, 2017)
                      2.1 (Oct 20, 2018)
                      2.2 (Oct 20, 2019)

    Changes:
    v.1.0 - First version
    v.1.1 - Typo correction and illustrative figures for linear SVM
    v.2.0 - Compatibility with Python 3 (backcompatible with Python 2.7)
    v.2.1 - Minor corrections on the notation
    v.2.2 - Minor equation errors. Reformatted hyperlinks. Restoring broken visualization of images in some Jupyter versions.
    v.2.3 - Notation revision

```python
from __future__ import print_function

# To visualize plots in the notebook
%matplotlib inline

# Imported libraries
#import csv
#import random
#import matplotlib
import matplotlib.pyplot as plt
from mpl_toolkits import mplot3d
#import pylab

import numpy as np
#from sklearn.preprocessing import PolynomialFeatures
from sklearn import svm
from sklearn.datasets.samples_generator import make_blobs
from sklearn.datasets.samples_generator import make_circles

from ipywidgets import interact
```

## 1. Introduction

  [Source: [sklearn documentation](http://scikit-learn.org/stable/modules/svm.html)  ]

Support vector machines (SVMs) are a set of supervised learning methods used for classification, regression and outliers detection.

The advantages of support vector machines are:

- Effective in high dimensional spaces.
- Still effective in cases where number of dimensions is greater than the number of samples.
- Uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.
- Versatile: different Kernel functions can be specified for the decision function.

The disadvantages of support vector machines include:

- SVMs do not directly provide probability estimates.

## 2. Motivating Support Vector Machines

  [Source: A [notebook](https://github.com/jakevdp/sklearn_pycon2015/blob/master/notebooks/03.1-Classification-SVMs.ipynb) by [Jake Vanderplas](https://github.com/jakevdp>)]

Support Vector Machines (SVMs) are a kind of ***discriminative*** classifiers: that is, they draw a boundary between clusters of data without making any explicit assumption about the probability model underlying the data generation process.

Let's show a quick example of support vector classification. First we need to create a dataset:

```python
X, y = make_blobs(n_samples=50, centers=2, random_state=0, cluster_std=0.60)
plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='copper')
plt.xlabel("$x_0$", fontsize=14)
plt.ylabel("$x_1$", fontsize=14)
plt.axis('equal')
plt.show()
```

A discriminative classifier attempts to draw a line between the two sets of data. Immediately we see an inconvenience: such problem is ill-posed! For example, we could come up with several possibilities which perfectly discriminate between the classes in this example:

```python
xfit = np.linspace(-1, 3.5)
plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='copper')

for m, b in [(1, 0.65), (0.5, 1.6), (-0.2, 2.9)]:
    plt.plot(xfit, m * xfit + b, '-k')

plt.xlim(-1, 3.5);
plt.xlabel("$x_0$", fontsize=14)
plt.ylabel("$x_1$", fontsize=14)
plt.axis('equal')
plt.show()
```

These are three very different separators which perfectly discriminate between these samples. Depending on which you choose, a new data point will be classified almost entirely differently! How can we improve on this?

Support Vector Machines (SVM) select the boundary decision maximizing the ***margin***. The margin of a classifier is defined as twice the maximum signed distance between the decision boundary and the training data. By *signed* we mean that the distance to misclassified samples is counted negatively. Thus, if the classification problem is "separable" (i.e. if there exist a decision boundary with zero errors in the training set), the SVM will choose the zero-error decision boundary that is "as far as possible" from the training data.

In summary, what an SVM does is to not only draw a line, but consider the "sample free" region about the line. Here's an example of what it might look like:

```python
xfit = np.linspace(-1, 3.5)
plt.scatter(X[:,0], X[:,1], c=y, s=50, cmap='copper')

for m, b, d in [(1, 0.65, 0.33), (0.5, 1.6, 0.55), (-0.2, 2.9, 0.2)]:
    yfit = m * xfit + b
    plt.plot(xfit, yfit, '-k')
    plt.fill_between(xfit, yfit-d, yfit+d, edgecolor='none',
                     color='#AAAAAA', alpha=0.4)

plt.xlim(-1, 3.5)
plt.xlabel("$x_0$", fontsize=14)
plt.ylabel("$x_1$", fontsize=14)
plt.axis('equal')
plt.show()
```

Notice here that if we want to maximize this width, the middle fit is clearly the best. This is the intuition of the SVM, which optimizes a linear discriminant model in conjunction with a margin representing the perpendicular distance between the datasets.

## 3. Linear SVM

  [Source: adapted from [Wikipedia](https://en.wikipedia.org/wiki/Support_vector_machine)]

In order to present the SVM in a formal way, consider a training dataset $\mathcal{D} = \left\{ (\mathbf{x}_k, y_k) \mid \mathbf{x}_k\in \Re^M,\, y_k \in \{-1,1\}, k=0,\ldots, {K-1}\right\}$, where the binary symmetric label $y_k\in \{-1,1\}$ indicates the class to which the point $\mathbf{x}_k$ belongs. Each $\mathbf{x}_k$ is an $M$-dimensional real vector. We want to find the maximum-margin hyperplane that divides the points having $y_k=1$ from those having $y_k=-1$.

Any hyperplane can be written as the set of points $\mathbf{x}$ satisfying

$$
\mathbf{w}^\intercal \mathbf{x} - b=0,
$$

where ${\mathbf{w}}$ denotes the (not necessarily normalized) normal vector to the hyperplane. The parameter $\tfrac{b}{\|\mathbf{w}\|}$ determines the offset of the hyperplane from the origin along the normal vector ${\mathbf{w}}$.

If the training data are linearly separable, we can select two parallel hyperplanes in a way that they separate the data and there are no points between them, and then try to maximize their distance. The region bounded by them is called "the margin". These hyperplanes can be described by the equations

$$\mathbf{w}^\intercal \mathbf{x} - b=1$$
and
$$\mathbf{w}^\intercal \mathbf{x} - b=-1.$$

Note that the two equations above can represent any two parallel hyperplanes in $\Re^M$. Essentially, the direction of vector $\mathbf{w}$ determines the orientation of the hyperplanes, whereas parameter $b$ and the norm of $\mathbf{w}$ can be used to select their exact location.

To compute the distance between the hyperplanes, we can obtain the projection of vector ${\mathbf x}_1 - {\mathbf x}_2$, where ${\mathbf x}_1$ and ${\mathbf x}_2$ are points from each of the hyperplanes, onto a unitary vector orthonormal to the hyperplanes:

$$\text{Distance between hyperplanes} = \left[\frac{\mathbf{w}}{\|\mathbf{w}\|}\right]^\top ({\mathbf x}_1 - {\mathbf x}_2) = \frac{\mathbf{w}^\top {\mathbf x}_1 - \mathbf{w}^\top {\mathbf x}_2}{\|\mathbf{w}\|} = \frac{2}{\|\mathbf{w}\|}.$$

Therefore, to maximize the distance between the planes we want to minimize $\|\mathbf{w}\|$.

As we also have to prevent data points from falling into the margin, we add the following constraints: for each $k$ either

\begin{align}
\mathbf{w}^\top \mathbf{x}_k - b &\ge +1, \qquad\text{ if  }\;\;y_k=1, \qquad \text{or} \\
\mathbf{w}^\top \mathbf{x}_k - b &\le -1, \qquad\text{ if  }\;\;y_k=-1.
\end{align}

This can be rewritten as:

$$
y_k(\mathbf{w}^\top \mathbf{x}_k - b) \ge 1, \quad \text{ for all } 0 \le k \le K-1.
$$

We can put this together to get the optimization problem:

$$
(\mathbf{w}^*,b^*) = \arg\min_{(\mathbf{w},b)} \|\mathbf{w}\| \\
\text{subject to: }
y_k(\mathbf{w}^\top \mathbf{x}_k - b) \ge 1, \, \text{ for any } k = 0, \dots, {K-1}
$$

This optimization problem is difficult to solve because it depends on $\|\mathbf{w}\|$, the norm of $\mathbf{w}$, which involves a square root. Fortunately it is possible to alter the minimization objective $\|\mathbf{w}\|$ by substituting it with $\tfrac{1}{2}\|\mathbf{w}\|^2$ (the factor of $\frac{1}{2}$ being used for mathematical convenience) without changing the solution (the minimum of the original and the modified equation have the same $\mathbf{w}$ and $b$):

$$
(\mathbf{w}^*,b^*) = \arg\min_{(\mathbf{w},b)} \frac{1}{2}\|\mathbf{w}\|^2 \\
\text{subject to: }
y_k(\mathbf{w}^\top \mathbf{x}_k - b) \ge 1, \, \text{ for any } k = 0, \dots, {K-1}
$$

This is a particular case of a *quadratic programming* problem.

### 3.1. Primal form

The optimization problem stated in the preceding section can be solved by means of a generalization of the Lagrange method of multipliers for inequality constraints, using the so called Karush–Kuhn–Tucker (KKT) multipliers $\boldsymbol{\alpha}$. According to it, the constrained problem can be expressed as

$$(\mathbf{w}^*,b^*, \boldsymbol{\alpha}^*) = \arg\min_{\mathbf{w},b } \max_{\boldsymbol{\alpha}\geq 0 } \left\{ \frac{1}{2}\|\mathbf{w}\|^2 - \sum_{k=0}^{K-1}{\alpha_k\left[y_k(\mathbf{w}^\top \mathbf{x}_k - b)-1\right]} \right\}
$$

that is, we look for a *saddle point*.

A key result in convex optimization theory is that, for the kind of optimization problems discussed here (see [here](http://www.onmyphd.com/?p=kkt.karush.kuhn.tucker&ckattempt=1), for instance), the *max* and *min* operators are interchangeable, so that
$$
(\mathbf{w}^*,b^*, \boldsymbol{\alpha}^*) = \arg\max_{\boldsymbol{\alpha}\geq 0 } \min_{\mathbf{w},b } \left\{ \frac{1}{2}\|\mathbf{w}\|^2 - \sum_{k=0}^{K-1}{\alpha_k\left[y_k(\mathbf{w}^\top \mathbf{x}_k - b)-1\right]} \right\}
$$

Note that the inner minimization problem is now quadratic in $\mathbf{w}$ and, thus, the minimum can be found by differentiation:
$$
\mathbf{w}^* = \sum_{k=0}^{K-1}{\alpha_k y_k\mathbf{x}_k}.
$$

#### 3.1.1. Support Vectors

In view of the optimization problem, we can check that all the points which can be separated as $y_k(\mathbf{w}^\top \mathbf{x}_k - b) - 1 > 0 $ do not matter since we must set the corresponding $\alpha_k$ to zero. Therefore, only a few $\alpha_k$ will be greater than zero. The corresponding $\mathbf{x}_k$ are known as `support vectors`.

It can be seen that the optimum parameter vector $\mathbf{w}^\ast$ can be expressed in terms of the support vectors only:

$$
\mathbf{w}^* = \sum_{k\in {\cal{S}}_{SV}}{\alpha_k y_k\mathbf{x}_k}.
$$

where ${\cal{S}}_{SV}$ is the set of indexes associated to support vectors.

#### 3.1.2. The computation of $b$

Support vectors lie on the margin and satisfy $y_k(\mathbf{w}^\top \mathbf{x}_k - b) = 1$. From this condition, we can obtain the value of $b$, since for any support vector:

$$\mathbf{w}^\top\mathbf{x}_k - b = \frac{1}{y_k} = y_k \iff b = \mathbf{w}^\top\mathbf{x}_k - y_k
$$

This estimate of $b$, the centerpoint of the division, depends only on a single pair $y_k$ and $x_k$.  We may get a more robust estimate of the center by averaging over all of the $N_{SV}$ support vectors, if we believe the population mean is a good estimate of the midpoint, so in practice, $b$ is often computed as:

$$
b = \frac{1}{N_{SV}} \sum_{\mathbf{x}_k\in {\cal{S}}_{SV}}{(\mathbf{w}^\top\mathbf{x}_k - y_k)}
$$

### 3.2. Dual form

Writing the classification rule in its unconstrained dual form reveals that the *maximum-margin hyperplane* and therefore the classification task is only a function of the *support vectors*, the subset of the training data that lie on the margin.

Using the fact that $\|\mathbf{w}\|^2 = \mathbf{w}^\top \mathbf{w}$ and substituting $\mathbf{w} = \sum_{k=0}^{K-1}{\alpha_k y_k\mathbf{x}_k}$, we obtain

\begin{align}
(b^*, \boldsymbol{\alpha}^*)
   &= \arg\max_{\boldsymbol{\alpha}\geq 0 }
   \min_b \left\{
   \sum_{k=0}^{K-1}\alpha_k -
   \frac{1}{2}
   \sum_{k=0}^{K-1} \sum_{j=0}^{K-1} {\alpha_k \alpha_j y_k y_j \mathbf{x}_k^\top\mathbf{x}_j}
   + b \sum_{k=0}^{K-1}\alpha_k y_k
\right\}
\end{align}

Note that, if $\sum_{k=0}^{K-1}\alpha_k y_k \neq 0$ the optimal value  of $b$ is $+\infty$ of $-\infty$, and

\begin{align}
\min_b \left\{
\sum_{k=0}^{K-1}\alpha_k -
\frac{1}{2}
\sum_{k=0}^{K-1} \sum_{j=0}^{K-1} {\alpha_k \alpha_j y_k y_j \mathbf{x}_k^\top\mathbf{x}_j}
+ b \sum_{k=0}^{K-1}\alpha_k y_k
\right\} = -\infty.
\end{align}

Therefore, any $\boldsymbol{\alpha}$ satifying $\sum_{k=0}^{K-1}\alpha_k y_k \neq 0$ is suboptimal, so that the optimal multipliers must satisfy the condition  $\sum_{k=0}^{K-1}\alpha_k y_k = 0$.

Summarizing, the dual formulation of the optimization problem is

$$
\boldsymbol{\alpha}^* = \arg\max_{\boldsymbol{\alpha}\geq 0} \sum_{k=0}^{K-1} \alpha_k -
       \frac12 \sum_{k,j} \alpha_k \alpha_j y_k y_j\;\kappa(\mathbf{x}_k, \mathbf{x}_j)  \\
\text{subject to: } \qquad \sum_{k=0}^{K-1} \alpha_k y_k = 0.
$$

where the *kernel* $\kappa(\cdot)$ is defined by $\kappa(\mathbf{x}_k,\mathbf{x}_j)=\mathbf{x}_k^\top\mathbf{x}_j$.

Many implementations of the SVM use this dual formulation. They proceed in three steps:

1. Solve the dual problem to obtain $\boldsymbol{\alpha}^*$. Usually, only a small number of $\alpha_k^*$ are nonzero. The corresponding values of ${\bf x}_k$ are called the *support vectors*.

2. Compute $\mathbf{w}^* = \sum_{k=0}^{K-1} \alpha_k^* y_k\mathbf{x}_k$

3. Compute $b^* = \frac{1}{N_{SV}} \sum_{\alpha_k^*\neq 0}{(\mathbf{w}^{*\top}\mathbf{x}_k - y_k)}
$

## 4. Fitting a Support Vector Machine

  [Source: A [notebook](https://github.com/jakevdp/sklearn_pycon2015/blob/master/notebooks/03.1-Classification-SVMs.ipynb) by [Jake Vanderplas](https://github.com/jakevdp>)]

Now we'll fit a Support Vector Machine Classifier to these points.

```python
clf = svm.SVC(kernel='linear')
clf.fit(X, y)
```

To better visualize what's happening here, let's create a quick convenience function that will plot SVM decision boundaries for us:

```python
def plot_svc_decision_function(clf, ax=None):
    """Plot the decision function for a 2D SVC"""
    if ax is None:
        ax = plt.gca()
    x = np.linspace(plt.xlim()[0], plt.xlim()[1], 30)
    y = np.linspace(plt.ylim()[0], plt.ylim()[1], 30)
    Y, X = np.meshgrid(y, x)
    P = np.zeros_like(X)
    for i, xi in enumerate(x):
        for j, yj in enumerate(y):
            P[i, j] = clf.decision_function(np.array([xi, yj]).reshape(1,-1))
    # plot the margins
    ax.contour(X, Y, P, colors='k',
               levels=[-1, 0, 1], alpha=0.5,
               linestyles=['--', '-', '--'])

plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='copper')
plot_svc_decision_function(clf);
```

Notice that the dashed lines touch a couple of the points: these points are the *support vectors*. In scikit-learn, these are stored in the ``support_vectors_`` attribute of the classifier:

```python
plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1],
            s=200, marker='s');
plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='copper')
plot_svc_decision_function(clf)

```

Let's use IPython's interact functionality to explore how the distribution of points affects the support vectors and the discriminative fit.

```python
def plot_svm(N=10):
    X, y = make_blobs(n_samples=200, centers=2,
                      random_state=0, cluster_std=0.60)
    X = X[:N]
    y = y[:N]
    clf = svm.SVC(kernel='linear')
    clf.fit(X, y)
    plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='copper')
    plt.xlim(-1, 4)
    plt.ylim(-1, 6)
    plot_svc_decision_function(clf, plt.gca())
    plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1],
                s=200, facecolors='none')

interact(plot_svm, N=[10, 200], kernel='linear')
```

Notice the unique thing about SVM is that only the support vectors matter: that is, if you moved any of the other points without letting them cross the decision boundaries, they would have no effect on the classification results!

## 5. Non-separable problems.

  [Source: adapted from [Wikipedia](https://en.wikipedia.org/wiki/Support_vector_machine)]

In 1995, Corinna Cortes and Vladimir N. Vapnik suggested a modified maximum margin idea that allows for mislabeled examples. If there exists no hyperplane that can split the `positive` and `negative` samples, the `Soft Margin`
 method will choose a hyperplane that splits the examples as cleanly as possible, while still maximizing the distance to the nearest cleanly split examples. The method introduces non-negative slack variables, $\xi_k$, which measure the degree of misclassification of the data $\mathbf{x}_k$

$$y_k(\mathbf{w}^\top\mathbf{x}_k - b) \ge 1 - \xi_k \quad k=0,\ldots, K-1.
$$

The objective function is then increased by a function which penalizes non-zero $\xi_k$, and the optimization becomes a trade off between a large margin and a small error penalty. If the penalty function is linear, the optimization problem becomes:

$$(\mathbf{w}^*,\mathbf{\xi}^*, b^*) = \arg\min_{\mathbf{w},\mathbf{\xi}, b } \left\{\frac{1}{2} \|\mathbf{w}\|^2 + C \sum_{k=0}^{K-1} \xi_k \right\} \\
\text{subject to: } \quad y_k(\mathbf{w}^\intercal\mathbf{x}_k - b) \ge 1 - \xi_k, \quad \xi_k \ge 0, \quad  k=0,\ldots, K-1.
$$

This constraint along with the objective of minimizing $\|\mathbf{w}\|$ can be solved using KKT multipliers as done above. One then has to solve the following problem:

$$
\arg\min_{\mathbf{w}, \mathbf{\xi}, b } \max_{\boldsymbol{\alpha}, \boldsymbol{\beta} }
\left\{ \frac{1}{2}\|\mathbf{w}\|^2
+ C \sum_{k=0}^{K-1} \xi_k
- \sum_{k=0}^{K-1} {\alpha_k\left[y_k(\mathbf{w}^\top \mathbf{x}_k - b) -1 + \xi_k\right]}
- \sum_{k=0}^{K-1} \beta_k \xi_k \right \}\\
\text{subject to: } \quad
\alpha_k, \beta_k \ge 0.
$$

A similar analysis to that in the separable case can be applied to show that the dual formulation of the optimization problem is

$$
        \boldsymbol{\alpha}^* = \arg\max_{0 \leq \alpha_k \leq C} \sum_{k=0}^{K-1} \alpha_k -
       \frac12 \sum_{k,j} \alpha_k \alpha_j y_k y_j \;\kappa(\mathbf{x}_k, \mathbf{x}_j)  \\
\text{subject to: } \qquad \sum_{k=0}^{K-1} \alpha_k y_k = 0.
$$
Note that the only difference with the separable case is given by the constraints $\alpha_k \leq C$.

## 6. Nonlinear classification

  [Source: adapted from [Wikipedia](https://en.wikipedia.org/wiki/Support_vector_machine)]

The original optimal hyperplane algorithm proposed by Vapnik in 1963 was a linear classifier. However, in 1992, Bernhard E. Boser, Isabelle M. Guyon and Vladimir N. Vapnik suggested a way to create nonlinear classifiers by applying the *kernel trick* to maximum-margin hyperplanes. The resulting algorithm is formally similar, except that every dot product is replaced by a nonlinear kernel function. This allows the algorithm to fit the maximum-margin hyperplane in a transformed feature space. The transformation may be nonlinear and the transformed space high dimensional; thus though the classifier is a hyperplane in the high-dimensional feature space, it may be nonlinear in the original input space.

The kernel is related to the transform $\phi(\mathbf{x})$ by the equation $\kappa(\mathbf{x}, \mathbf{x}') = \phi(\mathbf{x})^\top \phi(\mathbf{x}')$. However, note that we do not need to explicitly compute $\phi(\mathbf{x})$, as long as we can express all necessary calculations in terms of the kernel function only, as it is the case for the optimization problem in the dual case.

The predictions of the SVM classifier can also be expressed in terms of kernels only, so that we never need to explicitely compute $\phi(\mathbf{x})$.

$$\begin{align}\hat y({\mathbf{x}}) & = {\mathbf {w^\ast}}^\top \phi(\mathbf{x}) - b^\ast \\ \\
& = \left[\sum_{k \in {\cal{S}}_{SV}} \alpha_k^* y_k \phi(\mathbf{x}_k)\right]^\top {\phi(\mathbf{x})} - b^\ast  \\ \\
& = - b^\ast + \sum_{k \in {\cal{S}}_{SV}} \alpha_k^* y_k \; \kappa(\mathbf{x}_k, {\mathbf{x}})
\end{align}$$

Some common kernels include:

* **Gaussian**: $\kappa(\mathbf{x},\mathbf{x}')=\exp(-\gamma \|\mathbf{x} - \mathbf{x}'\|^2)$, for $\gamma > 0$. Sometimes parametrized using $\gamma=\dfrac{1}{2 \sigma^2}$. This is by far the most widely used kernel.
* Polynomial (homogeneous): $\kappa(\mathbf{x},\mathbf{x}')=(\mathbf{x}^\top \mathbf{x}')^d$
* Polynomial (inhomogeneous): $\kappa(\mathbf{x},\mathbf{x}') = (\mathbf{x}^\top \mathbf{x}' + 1)^d$
* Hyperbolic tangent: $\kappa(\mathbf{x},\mathbf{x}') = \tanh(\gamma \mathbf{x}^\top \mathbf{x}'+c)$, for some (not every) $\gamma > 0$ and $c < 0$.

### 6.1. Example.

  [Source: A [notebook](https://github.com/jakevdp/sklearn_pycon2015/blob/master/notebooks/03.1-Classification-SVMs.ipynb) by [Jake Vanderplas](https://github.com/jakevdp>)]

Where SVM gets incredibly exciting is when it is used in conjunction with *kernels*.
To motivate the need for kernels, let's look at some data which is not linearly separable:

```python
X, y = make_circles(100, factor=.1, noise=.1)

clf = svm.SVC(kernel='linear').fit(X, y)
plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='copper')
plot_svc_decision_function(clf);
```

Clearly, no linear discrimination will ever separate these data.
One way we can adjust this is to apply a **kernel**, which is some functional transformation of the input data.

For example, one simple model we could use is a **radial basis function**

```python
r = np.exp(-(X[:, 0] ** 2 + X[:, 1] ** 2))
```

If we plot this along with our data, we can see the effect of it:

```python
def plot_3D(elev=30, azim=30):
    ax = plt.subplot(projection='3d')
    ax.scatter3D(X[:, 0], X[:, 1], r, c=y, s=50, cmap='spring')
    ax.view_init(elev=elev, azim=azim)
    ax.set_xlabel('x')
    ax.set_ylabel('y')
    ax.set_zlabel('r')

interact(plot_3D, elev=[-90, 90], azip=(-180, 180));
```

We can see that with this additional dimension, the data becomes trivially linearly separable!
This is a relatively simple kernel; SVM has a more sophisticated version of this kernel built-in to the process. This is accomplished by using the Gaussian kernel (``kernel='rbf'``), short for *radial basis function*:

```python
clf = svm.SVC(kernel='rbf', C=10)
clf.fit(X, y)

plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='copper')
plot_svc_decision_function(clf)
plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1],
            s=200, facecolors='none');
```

Here there are effectively $K$ basis functions: one centered at each point! Through a clever mathematical trick, this computation proceeds very efficiently using the "Kernel Trick", without actually constructing the matrix of kernel evaluations.

**Exercise**: Apply the linear SVM and the SVM with Gaussian kernel to the discrimination of classes `Versicolor` and `Virginica` in the Iris Dataset, using atributes $x_0$ and $x_1$ only. Plot the corresponding decision boundaries and the support vectors.

## 7. Hyperparameters

Note that the SVM formulation has several free parameters (hyperparameters) that must be selected out of the optimization problem:

- The free parameter $C$ used to solve non-separable problems.
- The kernel parameters (for instance, parameter $\gamma$ for the Gaussian kernel).

These parameters are usually adjusted using a cross-validation procedure.


# Quantum Teleportation with roqoqo & the use of conditional measurements

This notebook is designed to demonstrate the use of conditional measurements, by following through an example of quantum state teleportation.

In quantum teleportation there are two end users: The first user, Alice, wishes to send a particular quantum state to the second user, Bob. The protocol requires a total of three qubits, and the transmission of two classical bits.

The sender Alice controls qubits 0 and 1, and the reciever Bob controls qubit 2.

```Rust
:dep roqoqo = "0.9.0"
:dep roqoqo-quest = "0.2.0"
:dep num-complex = "0.4"
:dep qoqo_calculator = "0.5"
```

```Rust
extern crate num_complex;
extern crate roqoqo;
extern crate roqoqo_quest;

use roqoqo::{Circuit, operations as ops,prelude::*};
use roqoqo_quest::Backend;
use std::collections::HashMap;
use qoqo_calculator::CalculatorFloat;
use core::f64::consts::PI as Pi;
```

## State preparation

The first step is to prepare the quantum state which Alice will send to Bob. As an example, the most general single qubit quantum state is given by:
\begin{equation}
|\psi \rangle = cos(\frac{\theta}{2}) |0 \rangle + e^{i \phi} sin(\frac{\theta}{2}) |1 \rangle.
\end{equation}
This state can be prepared by a sequence of two single qubit rotations. In the code block below we first define a function that takes the angles $\theta$ and $\phi$ as input and prepares qubit 0 of a quantum register in the state $| \psi \rangle$.

Next we use an instance of the function with the angles $\theta=\frac{\pi}{2}$ and $\phi=0$ to create a circuit which prepares the state:
\begin{equation}
|\psi \rangle = \frac{1}{\sqrt{2}} \big ( |0 \rangle + |1 \rangle \big ) = | + \rangle.
\end{equation}

```Rust
fn prep_psi (angle_thet: CalculatorFloat, angle_phi: CalculatorFloat) -> Circuit
  {
    let mut circuit = Circuit::new();
    circuit += ops::RotateY::new(0, angle_thet);
    circuit += ops::RotateZ::new(0, angle_phi);
    return circuit
}

let init_circuit = prep_psi(CalculatorFloat::Float(Pi/2f64), CalculatorFloat::Float(0f64));
```

## Preparing an entangled resource state

Quantum teleportation requires that the end users initially share an entangled resource state,
\begin{equation}
|\Phi_{+} \rangle = \frac{1}{\sqrt(2)} \big ( |00 \rangle + |11 \rangle \big ) .
\end{equation}

The following circuit prepares the state $|\Phi_{+} \rangle$ between qubit 1, held by Alice, and qubit 2, held by Bob.

```Rust
let mut entangling_circ = Circuit::new();
entangling_circ += ops::Hadamard::new(1);
entangling_circ += ops::CNOT::new(1, 2);
```

## Encoding the state to be sent in the entangled resource state

The next step of the procedure is to encode the state of qubit 0, $\psi$, into the entangled resource state. This is accomplished by way of the circuit defined below, which is similar to that used to prepare the entangled resource.

```Rust
let mut encoding_circ = Circuit::new();
encoding_circ += ops::CNOT::new(0, 1);
encoding_circ += ops::Hadamard::new(0);
```

## State transfer part 1: Measurement

At this stage in the process both of Alice's qubits, 0 and 1, are measured. The measurement consumes the entangled resource and leaves the state of qubit 2,Bob's qubit, in a state that depends on the two measurement outcomes.

Let us call the classical bit which results from measuring qubit 0 'M1' and the bit resulting from measuring qubit 1 'M2'. The circuit below defines the classical register named 'M1M2', performs the measurement of qubits 0 and 1, and stores the results in the register 'M1M2'.

```Rust
let mut meas_circ = Circuit::new();
meas_circ += ops::DefinitionBit::new("M1M2".to_string(), 2, true);
meas_circ += ops::MeasureQubit::new(0, "M1M2".to_string(), 0);
meas_circ += ops::MeasureQubit::new(1, "M1M2".to_string(), 1);
```

## Defining the circuit for a conditional operation

Conditional operations in roqoqo have three inputs: the name of a classical register containing boolean values, the index of the register containing the value to be used to condition the operation, and the operation or sequence of operations to be performed if the boolean condition value is True.

To prepare the third input, it is necessary to create circuit snippets corresponding to the operations to be completed if the condition is True.

In the case of quantum teleportation, we need two conditional operations. The first is a Pauli Z acting on Bob's qubit, conditioned on the measurement result M1. The second is a Pauli X acting on Bob's qubit, conditioned on the measurement result M2.

Hence we prepare circuit snippets correspponding to a Pauli Z and a Pauli X operation.

```Rust
let mut conditional_z = Circuit::new();
conditional_z += ops::PauliZ::new(2);

let mut conditional_x = Circuit::new();
conditional_x += ops::PauliX::new(2);
```

## State transfer part 2: conditional operations

The final stage of the teleportation protocol is to perform corrections to the state of Bob's qubit 2, according to the measurement outcomes 'M1' and 'M2'.

The below circuit makes use of the circuit snippets defined above to perform the conditional corrections to the state of qubit 2. 2.

```Rust
let mut conditional_circ = Circuit::new();
conditional_circ += ops::PragmaConditional::new("M1M2".to_string(), 1, conditional_x);
conditional_circ += ops::PragmaConditional::new("M1M2".to_string(), 0, conditional_z);
```

## Putting it all together

Combining each of the circuits we have defined yeilds the full teleportation protocol. We can verify that the protocol is successful by reading out the final state vector and comparing it to the state which was to be sent, $|\psi \rangle$.

```Rust
let mut verification = Circuit::new();
verification += ops::DefinitionComplex::new("psi".to_string(), 8, true);
verification += ops::PragmaGetStateVector::new("psi".to_string(), Some(Circuit::new()));

let teleportation_circuit = init_circuit + entangling_circ + encoding_circ + meas_circ + conditional_circ + verification;

let backend = Backend::new(3);
let result_of_run = backend.run_circuit(&teleportation_circuit);
let (result_bit_registers, _result_float_registers, result_complex_registers)
= result_of_run.unwrap();

println!("Result bit registers :{:?}",result_bit_registers["M1M2"]);
println!("Result complex registers :{:?}",result_complex_registers["psi"]);
```

    Result bit registers :[[true, false]]
    Result complex registers :[[Complex { re: 0.0, im: 0.0 }, Complex { re: 0.7071067811865476, im: 0.0 }, Complex { re: 0.0, im: 0.0 }, Complex { re: 0.0, im: 0.0 }, Complex { re: -0.0, im: 0.0 }, Complex { re: 0.7071067811865475, im: -0.0 }, Complex { re: -0.0, im: 0.0 }, Complex { re: -0.0, im: 0.0 }]]


Contenido bajo licencia Creative Commons BY 4.0 y código bajo licencia MIT. © Juan Gómez y Nicolás Guarín-Zapata 2020. Este material es parte del curso Modelación Computacional en el programa de Ingeniería Civil de la Universidad EAFIT.

# Pórticos planos

## Introducción

En este Notebook se agrega un elemento adicional al programa de análisis estructural, iniciado con el problema simple de resortes unidimesionales y extendido para considerar armaduras planas. En particular, ahora se adicionan elementos tipo viga para abordar problemas en los cuales los elementos funcionen principalmente a flexión. Adicionalmente, en una implementación independiente, es posible acoplar el comportamiento a carga axial de la cercha con el comportamiento a flexión de la viga.

El modelo de viga que se discute en este notebook corresponde a un modelo de Euler-Bernoulli en el cual se desprecian las deformaciones por cortante y por lo tanto válido para elementos con secciones transversales de baja esbeltez.

**Al completar este notebook usted debería estar en la capacidad de:**

* Reconocer las modificaciones necesarias para convertir un programa fundamental de ensamblaje de resortes en uno para estructuras conformadas por vigas.

* Resolver problemas simples de estructuras conformadas por vigas sometidas a cargas puntuales.

### Ensamblaje de elementos tipo viga

Consideremos un elemento tipo viga en su sistema local de referencia como el mostrado en la figura. En el sistema local el elemento tiene 2 grados de libertad por nodo, correspondientes al desplazamiento transversal y una rotación con respecto a un eje perpendicular al plano de la imagen.

El vector de grados de libertad (o desplazamientos generalizados) del elemento es:

$$
u^T=\begin{bmatrix}v_1 &\theta_1 &v_2 &\theta_2\end{bmatrix}
$$

mientras que el vector de fuerza (momentos y cortantes) esta dado por:

$$
f^T=\begin{bmatrix}f_1 &m_1 &f_2 &m_2\end{bmatrix}
$$

En el sistema de referencia local la matriz de rigidez relacionando fuerzas con desplazamientos es:

$$
\begin{Bmatrix} f_1\\ m_1\\ f_2\\ m_2\end{Bmatrix} =
\begin{bmatrix}
12\frac{EI}{\mathcal l^3} &6\frac{EI}{\mathcal l^3} &-12\frac{EI}{\mathcal l^3} &6\frac{EI}{\mathcal l^2}\\
6\frac{EI}{\mathcal l^3}&4\frac{EI}{\mathcal l}&-6\frac{EI}{\mathcal l^2}&2\frac{EI}{\mathcal l}\\
12\frac{EI}{\mathcal l^3}&-6\frac{EI}{\mathcal l^2}&12\frac{EI}{\mathcal l^3}&-6\frac{EI}{\mathcal l^2}\\
6\frac{EI}{\mathcal l^2}&2\frac{EI}{\mathcal l}&-6\frac{EI}{\mathcal l^2}&4\frac{EI}{\mathcal l}
\end{bmatrix}
\begin{Bmatrix} v_1\\ \theta_1\\ v_2\\ \theta_2\end{Bmatrix}
$$

**Nota:** La matriz de rigidez puede formularse por diferentes métodos que serán cubiertos en el curso de Análisis de Estructuras.

Para obtener la matriz de rigidez global de la estructura es necesario considerar nuevamente la contribución de todos los elementos en el sistema **global** de referencia. Con ese propósito procedemos como con el problema de la cercha. Usando la matriz de transformación bajo rotación $\lambda$ se tiene que:

$$K = \lambda^T k\lambda$$

donde $K$ es la matriz de rigidez para el elemento tipo viga en el sistema global de referencia. Note que en este sistema el elemento tiene ahora 3 grados de libertad por nodo.

### Estructura aporticada simple

Considere el siguiente modelo simple conformado por un ensamblaje de 3 elementos. (Los archivos de datos de entrada están disponibles en la carpeta `files`).

Se requiere determinar el desplazamiento lateral de la estructura cuando es sometida a la carga lateral $P$.

Encuentre la matriz de transformación bajo rotación $\lambda$ requerida para la formulación de la matriz de rigidez en el sistema de referencia global.

Las modificacions que se deben aplicar al código de resortes (o al de cerchas) solo estan relacionadas con el hecho de que ahora hay 3 grados de libertad en cada nodo.

```python
import matplotlib.pyplot as plt
import numpy as np
import sympy as sym
```

```python
%matplotlib inline
```

Lea los archivos de entrada de la carpeta `files`.

```python
def readin():
    nodes = np.loadtxt('files/Fnodes.txt', ndmin=2)
    mats = np.loadtxt('files/Fmater.txt', ndmin=2)
    elements = np.loadtxt('files/Feles.txt', ndmin=2)
    loads = np.loadtxt('files/Floads.txt', ndmin=2)
    return nodes, mats, elements, loads
```

La función `eqcounter` cuenta ecuaciones y genera el arreglo de condiciones de frontera.

```python
def eqcounter(nodes):
    nnodes = nodes.shape[0]
    IBC = np.zeros([nnodes, 3], dtype=np.integer)
    for i in range(nnodes):
        for k in range(3):
            IBC[i, k] = int(nodes[i, k+3])
    neq = 0
    for i in range(nnodes):
        for j in range(3):
            if IBC[i, j] == 0:
                IBC[i, j] = neq
                neq = neq + 1
    return neq, IBC
```

Ahora, la función `DME` calcula la matriz de ensamblaje de ecuaciones.

```python
def DME(nodes, elements):

    nels = elements.shape[0]
    IELCON = np.zeros([nels, 2], dtype=np.integer)
    DME_mat = np.zeros([nels, 6], dtype=np.integer)
    neq, IBC = eqcounter(nodes)
    ndof = 6
    nnodes = 2
    for i in range(nels):
        for j in range(nnodes):
            IELCON[i, j] = elements[i, j+3]
            kk = IELCON[i, j]
            for l in range(3):
                DME_mat[i, 3*j+l] = IBC[kk, l]
    return DME_mat, IBC, neq
```

La función `assembly` usa el modelo y la matriz `DME_mat` para calcular la matriz de rigidez global.

```python
def assembly(elements, mats, nodes, neq, DME_mat):
    IELCON = np.zeros([2], dtype=np.integer)
    KG = np.zeros((neq, neq))
    nels = elements.shape[0]
    nnodes = 2
    ndof = 6
    for el in range(nels):
        elcoor = np.zeros([nnodes, 2])
        im = np.int(elements[el, 2])
        par0 = mats[im, 0]
        par1 = mats[im, 1]
        for j in range(nnodes):
            IELCON[j] = elements[el, j+3]
            elcoor[j, 0] = nodes[IELCON[j], 1]
            elcoor[j, 1] = nodes[IELCON[j], 2]
        kloc = uelbeam2DU(elcoor, par0, par1)
        dme = DME_mat[el, :ndof]
        for row in range(ndof):
            glob_row = dme[row]
            if glob_row != -1:
                for col in range(ndof):
                    glob_col = dme[col]
                    if glob_col != -1:
                        KG[glob_row, glob_col] = KG[glob_row, glob_col] +\
                            kloc[row, col]
    return KG
```

La función `uelbeam2D` usa las coordenadas de los nodos y los parámetros de material para calcular la matriz de rigidez local ya transformada al sistema de referencia global.

Agregue un comentario a cada línea relevante de los códigos de las siguientes funciones y úselos para escribir los pseudocódigos correspondientes. En particular identifique el calculo de la matriz de transformación bajo rotación $\lambda$.

```python
def uelbeam2DU(coord, I, Emod):
    """2D-2-noded beam element
       without axial deformation

    Parametros
    ----------
    coord : ndarray
      Cordenadas nodales (2, 2).
    A : float
      Area de la seccion transversal.
    Emod : float
      Modulo de elasticidad (>0).

    Returna
    -------
    kl : ndarray
      Matriz de rigidez local para el elemento (4, 4).

    """
    vec = coord[1, :] - coord[0, :]
    nx = vec[0]/np.linalg.norm(vec)
    ny = vec[1]/np.linalg.norm(vec)
    L = np.linalg.norm(vec)
    Q = np.array([
        [-ny, nx, 0,  0,  0, 0],
        [0,  0, 1.0,  0,  0, 0],
        [0,  0, 0, -ny, nx, 0],
        [0,  0, 0,  0,  0, 1.0]])
    kl = (I*Emod/(L*L*L)) * np.array([
        [12.0, 6, -12.0, 6*L],
        [6,  4*L*L, -6*L, 2*L*L],
        [-12.0,  -6*L, 12.0, -6*L],
        [6*L,  2*L*L, -6*L, 4*L*L]])
    kG = np.dot(np.dot(Q.T, kl), Q)
    return kG
```

La rutina `loadassem` forma el vector de cargas en los nodos.

```python
def loadasem(loads, IBC, neq, nl):
    RHSG = np.zeros([neq])
    for i in range(nl):
        il = int(loads[i, 0])
        ilx = IBC[il, 0]
        ily = IBC[il, 1]
        ilT = IBC[il, 2]
        if ilx != -1:
            RHSG[ilx] = loads[i, 1]
        if ily != -1:
            RHSG[ily] = loads[i, 2]
        if ilT != -1:
            RHSG[ilT] = loads[i, 3]
    return RHSG
```

El programa principal mantiene la misma estructura que el programa de resortes, es decir, se tienen siguientes pasos:

* Lee el modelo.

* Determina la matriz de ensamblaje `DME_mat`.

* Ensambla el sistema global de ecuaciones.

* Determina los desplazamientos globales `UG` tras resolver el sistema global.

```python
nodes, mats, elements, loads = readin()
DME_mat, IBC, neq = DME(nodes, elements)
KG = assembly(elements, mats, nodes, neq, DME_mat)
RHSG = loadasem(loads, IBC, neq, 1)
UG = np.linalg.solve(KG, RHSG)
print(UG)
```

    [ 2.25000000e+01  2.00000000e+01  1.27557539e-16 -1.25918779e-15
      2.00000000e+01  4.88970566e-16]

### Problemas propuestos

#### Problema 1

Implemente una función que calcule las fuerzas nodales en cada elemento y que verifique el equilibrio del sistema.

#### Problema 2

Determine la rigidez lateral de la estructura usando la siguiente expresión:

$$k = \frac{P}{\delta}\, .$$

#### Problema 3

Repotencie la estructura de tal forma que la rigidez lateral se incremente por un factor de 2.0

#### Problema 4

Repare el pórtico mostrado en la figura adicionando elementos y/o imponiendo restricciones apropiadas a los desplazamientos. (Cree un nuevo paquete de archivos de datos).

#### Problema 5

Para el portico de la figura asuma que la conexión del nudo 5 es articulada e indique como esta condición cambia los resultados.

## Referencias

* Bathe, Klaus-Jürgen. (2006) Finite element procedures. Klaus-Jurgen Bathe. Prentice Hall International.

* Juan Gómez, Nicolás Guarín-Zapata (2018). SolidsPy: 2D-Finite Element Analysis with Python, <https://github.com/AppliedMechanics-EAFIT/SolidsPy>.

## Formato del notebook

La siguiente celda cambia el formato del Notebook.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./nb_style.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>