$$\newcommand{\ket}[1]{\left|{#1}\right\rangle}$$
$$\newcommand{\bra}[1]{\left\langle{#1}\right|}$$

# Explanation of First-Generation QM Model

# Disclaimer

Ultimately, I would like to understand, and be able to explain, the quantum mechanics behind the entire process of simulating an NMR spectrum. For now, here is a "recipe" of the steps to arrive at the spin Hamiltonian, and how its eigensolution can be used to calculate frequencies and intensities.

Two sources in particular enabled this:
1. Materials by Ilya Kuprov at SpinDynamics.org, particularly [Module I, Lecture 5](http://spindynamics.org/Spin-Dynamics---Part-I---Lecture-05.php) and the Matlab code of [Module II, Lecture 05](http://spindynamics.org/Spin-Dynamics---Part-II---Lecture-05.php) and [06](http://spindynamics.org/Spin-Dynamics---Part-II---Lecture-06.php).
2. [Materials](http://www.users.csbsju.edu/~frioux/workinprogress.html#Spectroscopy) by Frank Rioux at St. John's University and College of St. Benedict. In particular, [*ABC Proton NMR Using Tensor Algebra*](http://www.users.csbsju.edu/~frioux/nmr/ABC-NMR-Tensor.pdf) was very helpful.

```python
from IPython.core.display import display, HTML
display(HTML("<style>.container { width:100% !important; }</style>"))
```

<style>.container { width:100% !important; }</style>

```python
import numpy as np
from math import sqrt
from scipy.linalg import eigh
from scipy.sparse import kron, csc_matrix, csr_matrix, lil_matrix, bmat

import bokeh.io
import bokeh.plotting

```

## Constructing the Hamiltonian From Scratch

Start with the Pauli matrices:
\begin{align}
\sigma_x = \begin{pmatrix}0& \frac{1}{2}\\ \frac{1}{2}&0\end{pmatrix},
\sigma_y = \begin{pmatrix}0& -\frac{i}{2}\\ \frac{i}{2}&0\end{pmatrix},
\sigma_z = \begin{pmatrix}\frac{1}{2}& 0\\ 0&-\frac{1}{2}\end{pmatrix}
\end{align}

plus the identity matrix $I = \begin{pmatrix}1&0\\0&1\end{pmatrix}$

```python
sigma_x = np.matrix([[0, 1 / 2], [1 / 2, 0]])
sigma_y = np.matrix([[0, -1j / 2], [1j / 2, 0]])
sigma_z = np.matrix([[1 / 2, 0], [0, -1 / 2]])
unit = np.matrix([[1, 0], [0, 1]])
```

The required inputs are a list of frequencies $\nu_i$ and a matrix of $J_{ij}$ coupling constants:

```python
freqlist = [10.0, 20.0]
couplings = np.array([[0, 5], [5, 0]])

```

Let's break the original hamiltonian function down into smaller functions and explain their roles:

```python
def hamiltonian(freqlist, couplings):
    """Calculate the Hamiltonian for a spin system (isotropic liquid).

    Arguments:
        freqlist: a list of n chemical shifts (in Hz)
        couplings: an n x n array of J coupling constants (in Hz)
    Return:
        H: numpy.ndarray spin Hamiltonian
    """
    Lx, Ly, Lz = create_krons(freqlist)
    Lproduct = cartesian_products(Lx, Ly, Lz)
    H_zeeman = hamiltonian_diagonal(freqlist, Lz)
    H_J = hamiltonian_off_diagonal(couplings, Lproduct)
    H = H_zeeman + H_J
    return H
```

### Step 1: Each spin gets its own $L_x$, $L_y$ and $L_z$ operators.

These are formed from Kronecker products between $\sigma_{x/y/z}$ and $I$ operators.

Each individual product, for n spins, uses 1$\sigma_{x/y/z}$ and (n - 1) $I$ operators. They all differ in where in the sequence the $\sigma_{x/y/z}$ operator is placed.

For 3 spins, and using $L_z$ for example:

\begin{align}
L_{z_1} &= \sigma_z \otimes I \otimes I\\
L_{z_2} &= I \otimes \sigma_z \otimes I\\
L_{z_3} &= I \otimes I \otimes \sigma_z
\end{align}

In the Python code, these individual $L_{x/y/z_n}$ operators get stored in a \[0, n\] array, Ln.

*{I'm not sure why a 2-D array of one row was used. It's possible that this could be simplified to a regular 1-D array, but it's also possible that there was an issue with some other operation, such as creating the Cartesian products. Need to check this.}*

```python
def create_krons(freqlist):
    nspins = len(freqlist)
    # The following empty arrays will be used to store the
    # Cartesian spin operators.
    Lx = np.empty((1, nspins), dtype='object')
    Ly = np.empty((1, nspins), dtype='object')
    Lz = np.empty((1, nspins), dtype='object')

    for n in range(nspins):
        Lx[0, n] = 1
        Ly[0, n] = 1
        Lz[0, n] = 1
        for k in range(nspins):
            if k == n:                                  # Diagonal element
                Lx[0, n] = np.kron(Lx[0, n], sigma_x)
                Ly[0, n] = np.kron(Ly[0, n], sigma_y)
                Lz[0, n] = np.kron(Lz[0, n], sigma_z)
            else:                                       # Off-diagonal element
                Lx[0, n] = np.kron(Lx[0, n], unit)
                Ly[0, n] = np.kron(Ly[0, n], unit)
                Lz[0, n] = np.kron(Lz[0, n], unit)

    return Lx, Ly, Lz
```

### Step 2: Create the sums of cartesian products of $L$ operators.

Eventually, the off-diagonal components of the Hamiltonian  $H$ require calculating Cartesian products of the $L$ operators. In an attempt to hopefully "vectorize" these for faster computation, all of these products were calculated at once. Then, when a particular result is required (e.g. $L_{x_1}L_{x_2}+L_{y_1}L_{y_2}+L_{z_1}L_{z_2}$) it can be plucked like a bonbon out of a tray when desired.

First, the $L_x, L_y, $ and $L_z$ operators were compiled into a 3 x n array of operators:
```python
Lcol = np.vstack((Lx, Ly, Lz)).real
```
created:
\begin{align}
L_{col} = \begin{pmatrix}
L_{x_1}& L_{x_2}&\dots & L_{x_n}\\
L_{y_1}& L_{y_2}&\dots & L_{y_n}\\
L_{z_1}& L_{z_2}&\dots & L_{z_n}
\end{pmatrix}
\end{align}

Its transpose created the n x 3 array of operators:

\begin{align}
L_{row} = \begin{pmatrix}
L_{x_1}& L_{y_1}&L_{z_1}\\
L_{x_2}& L_{y_2}&L_{z_2}\\
\vdots&\vdots&\vdots\\
L_{x_n}& L_{y_n}&L_{z_n}
\end{pmatrix}
\end{align}

The product of these two arrays gives an array of the Cartesian products:

\begin{align}
L_{product}&= L_{row} \cdot L_{col} \\
&=\Tiny\begin{pmatrix}
L_{x_1}L_{x_1}+L_{y_1}L_{y_1}+L_{z_1}L_{z_1}&L_{x_1}L_{x_2}+L_{y_1}L_{y_2}+L_{z_1}L_{z_2}&\dots&L_{x_1}L_{x_n}+L_{y_1}L_{y_n}+L_{z_1}L_{z_n}\\
L_{x_2}L_{x_1}+L_{y_2}L_{y_1}+L_{z_2}L_{z_1}&L_{x_2}L_{x_2}+L_{y_2}L_{y_2}+L_{z_2}L_{z_2}&\dots&L_{x_2}L_{x_n}+L_{y_2}L_{y_n}+L_{z_2}L_{z_n}\\
\vdots& &\ddots& \\
L_{x_n}L_{x_1}+L_{y_n}L_{y_1}+L_{z_n}L_{z_1}&L_{x_n}L_{x_2}+L_{y_n}L_{y_2}+L_{z_n}L_{z_2}&\dots&L_{x_n}L_{x_n}+L_{y_n}L_{y_n}+L_{z_n}L_{z_n}\\
\end{pmatrix}
\end{align}

```python
def cartesian_products(Lx, Ly, Lz):
    Lcol = np.vstack((Lx, Ly, Lz)).real
    Lrow = Lcol.T  # As opposed to sparse version of code, this works!
    Lproduct = np.dot(Lrow, Lcol)
    return Lproduct
```

### Step 3: Add the Zeeman (on-diagonal) terms to the Hamiltonian.

\begin{align}
H_{Zeeman} = \sum_{i=1}^n \nu_i L_{z_i}
\end{align}

```python
def hamiltonian_diagonal(freqlist, Lz):
    nspins = len(freqlist)
    # Hamiltonian operator
    H = np.zeros((2**nspins, 2**nspins))

    # Add Zeeman interactions:
    for n in range(nspins):
        H = H + freqlist[n] * Lz[0, n]

    return H

```

### Step 4: Add the J-coupling (off-diagonal) terms to the Hamiltonian.

\begin{align}
H_J &= \sum_{i=1}^n \sum_{j=1}^n \frac{J_{ij}}{2} (L_{x_i}L_{x_j}+L_{y_i}L_{y_j}+L_{z_i}L_{z_j})\\
H &= H_{Zeeman} + H_J
\end{align}

In an attempt to vectorize the calculation for better performance, this was accomplished by doing in-place multiplication of the matrix of $J$ coupling constants and the Cartesian products $L_{product}$.

```python
def hamiltonian_off_diagonal(couplings, Lproduct):

    nspins = len(couplings[0])
    # Hamiltonian operator
    H = np.zeros((2**nspins, 2**nspins))

    # Scalar couplings

    # Testing with MATLAB discovered J must be /2.
    # Believe it is related to the fact that in the SpinDynamics.org simulation
    # freqs are *2pi, but Js by pi only.
    scalars = 0.5 * couplings
    scalars = np.multiply(scalars, Lproduct)
    for n in range(nspins):
        for k in range(nspins):
            H += scalars[n, k].real

    return H
```

## Extracting Signal Frequencies and Intensities From the Hamiltonian

To simulate a "modern" NMR experiment, a 90° pulse and FID acquisition is simulated, followed by Fourier transform. This is the approach used in Kuprov's Matlab code, and should be the required approach for any experiment requiring a more elaborate pulse sequence.

For a simple NMR spectrum, we can adipt a "continuous wave spectrometer" approach. We can find the resonance frequencies and their relative intensities directly from the spin Hamiltonian. The time-independent Schrodinger equation $H\Psi = E\Psi$ is solved for eigenvectors and corresponding eigenvalues.

For each $\psi_i$, the eigenvectors are the coefficients $c_n$ for each pure spin state. For a two-spin system, for example,

$\psi_i = c_1\ket{\alpha\alpha} + c_2\ket{\alpha\beta} + c_3\ket{\beta\alpha} + c_4\ket{\beta\beta}$.

and the corresponding eigenvector would be \begin{bmatrix}c_1\\c_2\\c_3\\c_4\end{bmatrix}

For a one-spin system, the two states for "spin-up" ($\ \ket{\uparrow}$ or $\ket{\alpha}$) and for "spin-down" ($\ \ket{\downarrow}$ or $\ket{\beta}$) are represented by vectors $\begin{bmatrix}1\\0\end{bmatrix}$ and $\begin{bmatrix}0\\1\end{bmatrix}$, respectively. For "pure" multiple-spin states, their vectors are obtained by taking tensor products of these vectors. For example:

\begin{align}
\ket{\alpha\alpha} &=
\begin{bmatrix}1\\0\end{bmatrix}\otimes\begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}1\\0\\0\\0\end{bmatrix}
\\
\ket{\alpha\beta} &= \begin{bmatrix}1\\0\end{bmatrix}\otimes\begin{bmatrix}0\\1\end{bmatrix} = \begin{bmatrix}0\\1\\0\\0\end{bmatrix}
\\
\ket{\beta\alpha} &= \begin{bmatrix}0\\1\end{bmatrix}\otimes\begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}0\\0\\1\\0\end{bmatrix}
\\
\ket{\beta\beta} &= \begin{bmatrix}0\\1\end{bmatrix}\otimes\begin{bmatrix}0\\1\end{bmatrix} = \begin{bmatrix}0\\0\\0\\1\end{bmatrix}
\end{align}

A (coincidental?) consequence of this is that the index for $H$, expressed in binary form as a series of 0s and 1s, is the eigenvector for the associated pure spin state (cf. Rioux's *ABC Proton NMR Using Tensor Algebra*).

Since allowed transitions change the total spin of a system by $\pm$ 1, this is analogous to transitions only being allowed between spin states whose binary indices only differ at one bit. In computing terms, if the Hamming weight of the two indices differ by exactly 1, the transition is allowed.

```python
def popcount(n=0):
    """
    Computes the popcount (binary Hamming weight) of integer n
    input:
        :param n: an integer
    returns:
        popcount of integer (binary Hamming weight)

    """
    return bin(n).count('1')

def is_allowed(m=0, n=0):
    """
    determines if a transition between two spin states is allowed or forbidden.
    The transition is allowed if one and only one spin (i.e. bit) changes
    input: integers whose binary codes for a spin state
        :param n:
        :param m:
    output: 1 = allowed, 0 = forbidden

    """
    return popcount(m ^ n) == 1
```

Knowing this, we can create a transition probability matrix $T$, where $T_{ij} = 1$ if a transition between states $i$ and $j$ are allowed, and $0$ if not.

```python
def transition_matrix(n):
    """
    Creates a matrix of allowed transitions.
    The integers 0-n, in their binary form, code for a spin state (alpha/beta).
    The (i,j) cells in the matrix indicate whether a transition from spin state
    i to spin state j is allowed or forbidden.
    See the is_allowed function for more information.

    input:
        :param n: size of the n,n matrix (i.e. number of possible spin states)

    :returns: a transition matrix that can be used to compute the intensity of
    allowed transitions.
    """
    # function was optimized by only calculating upper triangle and then adding
    # the lower.
    T = lil_matrix((n, n))  # sparse matrix created
    for i in range(n - 1):
        for j in range(i + 1, n):
            if is_allowed(i, j):
                T[i, j] = 1
    T = T + T.T
    return T
```

The eigenvector solutions for the Hamiltonian include two pure states ("all-up/$\alpha$" and "all-down/$\beta$", plus mixed states. We can construct a matrix $V_{col}$ where each column of the matrix is an eigenvector solution, in their indexed order:

\begin{align}
V_{col} = \begin{pmatrix}
\ket{\psi_1} &\ket{\psi_2} &\dots &\ket{\psi_n}
\end{pmatrix}
=\begin{pmatrix}
\begin{bmatrix}c_1\\c_2\\\vdots\\c_n\end{bmatrix}_1
&\begin{bmatrix}c_1\\c_2\\\vdots\\c_n\end{bmatrix}_2
&\dots
&\begin{bmatrix}c_1\\c_2\\\vdots\\c_n\end{bmatrix}_n
\end{pmatrix}
\end{align}

and where its transpose $V_{row} = V_{col}^T$ has an eigenvector for each row:

\begin{align}
V_{row}=\begin{pmatrix}
\bra{\psi_1} \\
\bra{\psi_2} \\
\vdots\\
\bra{\psi_n} \\
\end{pmatrix}
=\begin{pmatrix}
\begin{bmatrix}c_1&c_2&\dots&c_n\end{bmatrix}_1\\
\begin{bmatrix}c_1&c_2&\dots&c_n\end{bmatrix}_2\\
\vdots\\
\begin{bmatrix}c_1&c_2&\dots&c_n\end{bmatrix}_n\\
\end{pmatrix}
\end{align}

The intensity matrix $I$ can be obtained by taking $V_{row}\cdot T \cdot V_{col}$ and squaring it element-wise, so that $I_{ij}$ is the relative probability of a transition between the $\psi_i$ and $\psi_j$ states. The difference in energy between the two states gives the frequency in Hz.

```python
def simsignals(H, nspins):
    """
    Solves the spin Hamiltonian H and returns a list of (frequency, intensity)
    tuples. Nuclei must be spin-1/2.
    Inputs:
        :param H: a Hamiltonian array
        :param nspins: number of nuclei
    :return spectrum: a list of (frequency, intensity) tuples.
    """
    # This routine was optimized for speed by vectorizing the intensity
    # calculations, replacing a nested-for signal-by-signal calculation.
    # Considering that hamiltonian was dramatically faster when refactored to
    # use arrays instead of sparse matrices, consider an array refactor to this
    # function as well.

    # The eigensolution calculation apparently must be done on a dense matrix,
    # because eig functions on sparse matrices can't return all answers?!
    # Using eigh so that answers have only real components and no residual small
    # unreal components b/c of rounding errors
    E, V = np.linalg.eigh(H)    # V will be eigenvectors, v will be frequencies
    print(E)
    # Eigh still leaves residual 0j terms, so:
    V = np.asmatrix(V.real)
    print(V)

    # Calculate signal intensities
    Vcol = csc_matrix(V)
    print('Vcol:')
    print(Vcol.todense())
    Vrow = csr_matrix(Vcol.T)
    print('Vrow:')
    print(Vrow.todense())
    m = 2 ** nspins
    T = transition_matrix(m)
    print('T:')
    print(T.todense())
    print('T•Vcol:')
    mid_t = T * Vcol
    print(mid_t.todense())
    I = Vrow * T * Vcol
    print('I:')
    print(I.todense())
    print('I squared:')
    I = np.square(I.todense())
    print(I)

    spectrum = []
    for i in range(m - 1):
        for j in range(i + 1, m):
            if I[i, j] > 0.01:  # consider making this minimum intensity
                                # cutoff a function arg, for flexibility
                v = abs(E[i] - E[j])
                spectrum.append((v, I[i, j]))

    return spectrum
```

```python
def hamiltonian_with_prints(freqlist, couplings):
    # The following empty arrays will be used to store the
    # Cartesian spin operators.
    Lx = np.empty((1, nspins), dtype='object')
    Ly = np.empty((1, nspins), dtype='object')
    Lz = np.empty((1, nspins), dtype='object')

    for n in range(nspins):
        Lx[0, n] = 1
        Ly[0, n] = 1
        Lz[0, n] = 1
        for k in range(nspins):
            if k == n:                                  # Diagonal element
                Lx[0, n] = np.kron(Lx[0, n], sigma_x)
                Ly[0, n] = np.kron(Ly[0, n], sigma_y)
                Lz[0, n] = np.kron(Lz[0, n], sigma_z)
            else:                                       # Off-diagonal element
                Lx[0, n] = np.kron(Lx[0, n], unit)
                Ly[0, n] = np.kron(Ly[0, n], unit)
                Lz[0, n] = np.kron(Lz[0, n], unit)

    Lcol = np.vstack((Lx, Ly, Lz)).real
    Lrow = Lcol.T  # As opposed to sparse version of code, this works!
    Lproduct = np.dot(Lrow, Lcol)
    print(Lcol)
    print('-'*10)
    print(Lrow)
    print('-'*10)
    print(Lproduct)

    # Hamiltonian operator
    H = np.zeros((2**nspins, 2**nspins))

    # Add Zeeman interactions:
    for n in range(nspins):
        H = H + freqlist[n] * Lz[0, n]

    # Scalar couplings

    # Testing with MATLAB discovered J must be /2.
    # Believe it is related to the fact that in the SpinDynamics.org simulation
    # freqs are *2pi, but Js by pi only.
    scalars = 0.5 * couplings
    scalars = np.multiply(scalars, Lproduct)
    for n in range(nspins):
        for k in range(nspins):
            H += scalars[n, k].real

    print('Lz: ', Lz)

    return H

```

```python
H = hamiltonian(freqlist, couplings)
H
```

    matrix([[ 16.25,   0.  ,   0.  ,   0.  ],
            [  0.  ,  -6.25,   2.5 ,   0.  ],
            [  0.  ,   2.5 ,   3.75,   0.  ],
            [  0.  ,   0.  ,   0.  , -13.75]])

```python
np.set_printoptions(threshold=np.nan)
nspins = len(freqlist)
simsignals(H, nspins)
```

    [-13.75        -6.84016994   4.34016994  16.25      ]
    [[ 0.          0.         -0.          1.        ]
     [ 0.         -0.97324899 -0.22975292  0.        ]
     [ 0.          0.22975292 -0.97324899  0.        ]
     [ 1.          0.         -0.          0.        ]]
    Vcol:
    [[ 0.          0.          0.          1.        ]
     [ 0.         -0.97324899 -0.22975292  0.        ]
     [ 0.          0.22975292 -0.97324899  0.        ]
     [ 1.          0.          0.          0.        ]]
    Vrow:
    [[ 0.          0.          0.          1.        ]
     [ 0.         -0.97324899  0.22975292  0.        ]
     [ 0.         -0.22975292 -0.97324899  0.        ]
     [ 1.          0.          0.          0.        ]]
    T:
    [[ 0.  1.  1.  0.]
     [ 1.  0.  0.  1.]
     [ 1.  0.  0.  1.]
     [ 0.  1.  1.  0.]]
    T•Vcol:
    [[ 0.         -0.74349607 -1.20300191  0.        ]
     [ 1.          0.          0.          1.        ]
     [ 1.          0.          0.          1.        ]
     [ 0.         -0.74349607 -1.20300191  0.        ]]
    I:
    [[ 0.         -0.74349607 -1.20300191  0.        ]
     [-0.74349607  0.          0.         -0.74349607]
     [-1.20300191  0.          0.         -1.20300191]
     [ 0.         -0.74349607 -1.20300191  0.        ]]
    I squared:
    [[ 0.         0.5527864  1.4472136  0.       ]
     [ 0.5527864  0.         0.         0.5527864]
     [ 1.4472136  0.         0.         1.4472136]
     [ 0.         0.5527864  1.4472136  0.       ]]

    [(6.9098300562505255, 0.55278640450004191),
     (18.090169943749473, 1.4472135954999577),
     (23.090169943749473, 0.55278640450004191),
     (11.909830056250525, 1.4472135954999577)]

What happens if we only have one nucleus?

```python
freqlist = [10.0]
couplings = np.array([[0]])
nspins = len(freqlist)
```

two uncoupled spins:

```python
freqlist = [10.0, 20.0]
couplings = np.array([[0, 0], [0, 0]])
nspins = len(freqlist)
```

```python
def spin2():
    v = np.array([150-7.5, 150+7.5])
    J = lil_matrix((2, 2))
    J[0, 1] = 12
    J = J + J.T
    return v, J
```

```python
v, J = spin2()
J = J.todense()
print(v)
print(J)
H = hamiltonian_with_prints(v, J)
H
```

    [ 142.5  157.5]
    [[  0.  12.]
     [ 12.   0.]]
    [[ matrix([[ 0. ,  0. ,  0.5,  0. ],
            [ 0. ,  0. ,  0. ,  0.5],
            [ 0.5,  0. ,  0. ,  0. ],
            [ 0. ,  0.5,  0. ,  0. ]])
      matrix([[ 0. ,  0.5,  0. ,  0. ],
            [ 0.5,  0. ,  0. ,  0. ],
            [ 0. ,  0. ,  0. ,  0.5],
            [ 0. ,  0. ,  0.5,  0. ]])]
     [ matrix([[ 0.+0.j ,  0.+0.j ,  0.-0.5j,  0.+0.j ],
            [ 0.+0.j ,  0.+0.j ,  0.+0.j ,  0.-0.5j],
            [ 0.+0.5j,  0.+0.j ,  0.+0.j ,  0.+0.j ],
            [ 0.+0.j ,  0.+0.5j,  0.+0.j ,  0.+0.j ]])
      matrix([[ 0.+0.j ,  0.-0.5j,  0.+0.j ,  0.-0.j ],
            [ 0.+0.5j,  0.+0.j ,  0.+0.j ,  0.+0.j ],
            [ 0.+0.j ,  0.-0.j ,  0.+0.j ,  0.-0.5j],
            [ 0.+0.j ,  0.+0.j ,  0.+0.5j,  0.+0.j ]])]
     [ matrix([[ 0.5,  0. ,  0. ,  0. ],
            [ 0. ,  0.5,  0. ,  0. ],
            [ 0. ,  0. , -0.5, -0. ],
            [ 0. ,  0. , -0. , -0.5]])
      matrix([[ 0.5,  0. ,  0. ,  0. ],
            [ 0. , -0.5,  0. , -0. ],
            [ 0. ,  0. ,  0.5,  0. ],
            [ 0. , -0. ,  0. , -0.5]])]]
    ----------
    [[ matrix([[ 0. ,  0. ,  0.5,  0. ],
            [ 0. ,  0. ,  0. ,  0.5],
            [ 0.5,  0. ,  0. ,  0. ],
            [ 0. ,  0.5,  0. ,  0. ]])
      matrix([[ 0.+0.j ,  0.+0.j ,  0.-0.5j,  0.+0.j ],
            [ 0.+0.j ,  0.+0.j ,  0.+0.j ,  0.-0.5j],
            [ 0.+0.5j,  0.+0.j ,  0.+0.j ,  0.+0.j ],
            [ 0.+0.j ,  0.+0.5j,  0.+0.j ,  0.+0.j ]])
      matrix([[ 0.5,  0. ,  0. ,  0. ],
            [ 0. ,  0.5,  0. ,  0. ],
            [ 0. ,  0. , -0.5, -0. ],
            [ 0. ,  0. , -0. , -0.5]])]
     [ matrix([[ 0. ,  0.5,  0. ,  0. ],
            [ 0.5,  0. ,  0. ,  0. ],
            [ 0. ,  0. ,  0. ,  0.5],
            [ 0. ,  0. ,  0.5,  0. ]])
      matrix([[ 0.+0.j ,  0.-0.5j,  0.+0.j ,  0.-0.j ],
            [ 0.+0.5j,  0.+0.j ,  0.+0.j ,  0.+0.j ],
            [ 0.+0.j ,  0.-0.j ,  0.+0.j ,  0.-0.5j],
            [ 0.+0.j ,  0.+0.j ,  0.+0.5j,  0.+0.j ]])
      matrix([[ 0.5,  0. ,  0. ,  0. ],
            [ 0. , -0.5,  0. , -0. ],
            [ 0. ,  0. ,  0.5,  0. ],
            [ 0. , -0. ,  0. , -0.5]])]]
    ----------
    [[ matrix([[ 0.75+0.j,  0.00+0.j,  0.00+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.75+0.j,  0.00+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.00+0.j,  0.75+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.00+0.j,  0.00+0.j,  0.75+0.j]])
      matrix([[ 0.25+0.j,  0.00+0.j,  0.00+0.j,  0.00+0.j],
            [ 0.00+0.j, -0.25+0.j,  0.50+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.50+0.j, -0.25+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.00+0.j,  0.00+0.j,  0.25+0.j]])]
     [ matrix([[ 0.25+0.j,  0.00+0.j,  0.00+0.j,  0.00+0.j],
            [ 0.00+0.j, -0.25+0.j,  0.50+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.50+0.j, -0.25+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.00+0.j,  0.00+0.j,  0.25+0.j]])
      matrix([[ 0.75+0.j,  0.00+0.j,  0.00+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.75+0.j,  0.00+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.00+0.j,  0.75+0.j,  0.00+0.j],
            [ 0.00+0.j,  0.00+0.j,  0.00+0.j,  0.75+0.j]])]]
    Lz:  [[ matrix([[ 0.5,  0. ,  0. ,  0. ],
            [ 0. ,  0.5,  0. ,  0. ],
            [ 0. ,  0. , -0.5, -0. ],
            [ 0. ,  0. , -0. , -0.5]])
      matrix([[ 0.5,  0. ,  0. ,  0. ],
            [ 0. , -0.5,  0. , -0. ],
            [ 0. ,  0. ,  0.5,  0. ],
            [ 0. , -0. ,  0. , -0.5]])]]

    matrix([[ 153. ,    0. ,    0. ,    0. ],
            [   0. ,  -10.5,    6. ,    0. ],
            [   0. ,    6. ,    4.5,    0. ],
            [   0. ,    0. ,    0. , -147. ]])

```python

```


# Comportamiento matrices de Jordan

```python
from sympy import *

x, y, z = symbols('a y n')

M = Matrix([[x,1,0,0,0], [0,x,1,0,0],[0,0,x,1,0],[0,0,0,x,1],[0,0,0,0,x]])

M**z
```

$\displaystyle \left[\begin{matrix}a^{n} & a^{n - 1} n & \frac{a^{n - 2} n \left(n - 1\right)}{2} & \frac{a^{n - 3} n \left(n - 2\right) \left(n - 1\right)}{6} & \frac{a^{n - 4} n \left(n - 3\right) \left(n - 2\right) \left(n - 1\right)}{24}\\0 & a^{n} & a^{n - 1} n & \frac{a^{n - 2} n \left(n - 1\right)}{2} & \frac{a^{n - 3} n \left(n - 2\right) \left(n - 1\right)}{6}\\0 & 0 & a^{n} & a^{n - 1} n & \frac{a^{n - 2} n \left(n - 1\right)}{2}\\0 & 0 & 0 & a^{n} & a^{n - 1} n\\0 & 0 & 0 & 0 & a^{n}\end{matrix}\right]$


# Ecommerce Optimization Code

### Formulation of a promotion optimization problem:

\begin{equation}
\begin{split}
\max~ & \sum_{j=1}^9\sum_{i=1}^n p_{ij}x_{ij}d_{ij} + \sum_{i=1}^n y_{i} \bar{p}_{i} \bar{d}_{i} \\
s.t. ~& \sum_{j=1}^9x_{ij}\leq 1, ~ i = 1,...,n\\
& \sum _{j=1}^{9}\sum _{i=1}^{n} p_{ij} x_{ij} \leq c\\
& y_{i}= 1-\sum_{j=1}^9 x_{ij}, ~ i = 1,...,n \\
& x_{ij} \in \{0,1\}
\end{split}
\end{equation}

c = User defined constant to limit number of promotion

n = number of products

j = different prices for each product

dij = Demand for product i at j price

xij = Binary Selection, if xij = 1 means select at product i and price j, 0 if otherwise

pij = jth Price for product i

```python
import csv
from gurobipy import *
import pandas as pd
import numpy as np
file = open('OptimizerInput.csv')
csvreader = csv.reader(file)
```

```python
header = []
header = next(csvreader)
header
```

    ['id', 'product', 'price', 'demand']

```python
rows = []
for row in csvreader:
        rows.append(row)
print(rows[1:12])

print(type(rows))
```

    [['1', '10 COLOUR SPACEBOY PEN', '0.821111111111111', '73.172647756993'], ['1', '10 COLOUR SPACEBOY PEN', '0.922222222222222', '54.126225061631'], ['1', '10 COLOUR SPACEBOY PEN', '1.02333333333333', '45.3624739654545'], ['1', '10 COLOUR SPACEBOY PEN', '1.12444444444444', '38.5223929109929'], ['1', '10 COLOUR SPACEBOY PEN', '1.22555555555556', '31.2091023340694'], ['1', '10 COLOUR SPACEBOY PEN', '1.32666666666667', '23.4628943416618'], ['1', '10 COLOUR SPACEBOY PEN', '1.42777777777778', '15.7804425707803'], ['1', '10 COLOUR SPACEBOY PEN', '1.52888888888889', '8.54204136171192'], ['1', '10 COLOUR SPACEBOY PEN', '1.63', '1.9488612255597'], ['2', '12 COLOURED PARTY BALLOONS', '0.65', '13.9609375'], ['2', '12 COLOURED PARTY BALLOONS', '0.716666666666667', '12.5208333333333']]
    <class 'list'>

```python
#We have 2395 products and 10 different price
#We will have 2 different set. 1 discounted set with 9 prices and 1 original set with 1 price
numofproducts = 2395

discprice = np.zeros((numofproducts, 9))
discrevenue = np.zeros((numofproducts, 9))

origprice = np.zeros(numofproducts)
origrevenue = np.zeros(numofproducts)

oneprice = np.ones(numofproducts)
prod = []

for row in range(0, len(rows), 10):
    pid = (int(rows[row][0]) - 1)
    for col in range(10):
        if col == 9:
            origprice[pid] = float(rows[row + col][2])
            origrevenue[pid] = float(rows[row + col][2]) * float(rows[row + col][3])
            prod.append(rows[row + col][1])
        else:
            discprice[pid,col] = float(rows[row + col][2])
            discrevenue[pid,col] = float(rows[row + col][2]) * float(rows[row + col][3])

```

```python
print(discprice.shape)
print(oneprice.shape)
print(discrevenue.shape)
print(origprice.shape)
print(origrevenue.shape)
```

    (2395, 9)
    (2395,)
    (2395, 9)
    (2395,)
    (2395,)

```python
#print(oneprice)
#print(discprice)
#print(origprice)
#print(revenue)
#print(prod)
```

```python
I, J = discrevenue.shape
print(I, J)
```

    2395 9

```python
tp = Model("promotion")

# User Defined Parameters, In this example we set c=4000
c = 268

# x variable is binary hence added vtype=GRB.BINARY
x = tp.addVars(I, J, vtype=GRB.BINARY)

# Set objective
tp.setObjective(quicksum(discrevenue[i,j]*x[i,j] for i in range(I) for j in range(J)) + quicksum(quicksum(1-x[i,j] for j in range(J))*origrevenue[i] for i in range(I)), GRB.MAXIMIZE)

# Add 1 chosen price constraints:
tp.addConstrs((quicksum(x[i,j] for j in range(J)) <= oneprice[i] for i in range(I)), "oneprice")

# Add sum of all the selected prices is equal to c constraints:
tp.addConstrs((quicksum(discprice[i,j]*x[i,j] for i in range(I) for j in range(J))) <= c for j in range(J))

# Solving the model
tp.optimize()

# Show Result
print('\nObj:', tp.objVal)
```

    Set parameter Username
    Academic license - for non-commercial use only - expires 2022-01-10
    Gurobi Optimizer version 9.5.0 build v9.5.0rc5 (win64)
    Thread count: 2 physical cores, 4 logical processors, using up to 4 threads
    Optimize a model with 2404 rows, 21555 columns and 215550 nonzeros
    Model fingerprint: 0xbb744248
    Variable types: 0 continuous, 21555 integer (21555 binary)
    Coefficient statistics:
      Matrix range     [4e-02, 3e+02]
      Objective range  [2e-14, 4e+03]
      Bounds range     [1e+00, 1e+00]
      RHS range        [1e+00, 3e+02]
    Found heuristic solution: objective 476123.43899
    Presolve removed 1622 rows and 16866 columns
    Presolve time: 0.20s
    Presolved: 782 rows, 4689 columns, 7920 nonzeros
    Variable types: 0 continuous, 4689 integer (4689 binary)
    Found heuristic solution: objective 486371.76507

    Root relaxation: objective 5.490079e+05, 1172 iterations, 0.08 seconds (0.09 work units)

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0 549007.912    0    1 486371.765 549007.912  12.9%     -    0s
    H    0     0                    548398.13901 549007.912  0.11%     -    0s
    H    0     0                    548956.46737 549007.912  0.01%     -    0s

    Explored 1 nodes (1172 simplex iterations) in 0.36 seconds (0.26 work units)
    Thread count was 4 (of 4 available processors)

    Solution count 4: 548956 548398 486372 476123

    Optimal solution found (tolerance 1.00e-04)
    Best objective 5.489564673688e+05, best bound 5.490079120080e+05, gap 0.0094%

    Obj: 548956.467368834

```python
result = np.zeros(I*J)
idx = 0
print(result.shape)

for i in range(I):
    for j in range(J):
        result[idx] = int(x[i,j].x)
        idx = idx + 1
```

    (21555,)

```python
print(type(result))
print(result[1:10])
#np.savetxt("promoResult.csv", result, delimiter=",")
```

    <class 'numpy.ndarray'>
    [0. 0. 0. 0. 0. 0. 0. 0. 0.]

```python
#Process the optimized solution into more user friendly rather than 0s and 1s

idx = 0
discitem = []
discpid = []
optprice = []
optrev = []

origitem =[]
origpid = []
pricemax = []
origrev = []

for row in range(0, len(result), 9):
    for col in range(9):
        #print(result[row+col])
        if result[row+col] == 1:
            discitem.append(prod[idx])
            discpid.append(idx+1)
            optprice.append(discprice[idx,col])
            optrev.append(discrevenue[idx,col])
            break
        #Keep it as original price if no discount
        if col == 8:
            origitem.append(prod[idx])
            origpid.append(idx+1)
            pricemax.append(origprice[idx])
            origrev.append(origrevenue[idx])

    idx = idx + 1
```

```python
d = {'id':discpid, 'Product':discitem, 'Optimized Price': optprice, 'Optimized Revenue': optrev}
df = pd.DataFrame(d)
df.to_csv('DiscountResult.csv', index=False)
```

```python
d2 = {'id':origpid, 'Product':origitem, 'Original Price': pricemax, 'Original Revenue': origrev}
df2 = pd.DataFrame(d2)
df2.to_csv('NonDiscountItem.csv', index=False)
```

```python
print(c)
sum(optrev) + sum(origrev)
```

    268

    136770.41537521078

```python

```


```python
import numpy as np
```

# Key

```python
key = np.array([
    [3, 10, 20],
    [20, 9, 17],
    [9, 4, 17]
])
```

```python
key.shape[0] == key.shape[1]
```

    True

```python
np.linalg.det(key) != 0
```

    True

```python
m = "attack is to night"
```

```python
import commons
m = commons.preprocessing(m)
```

```python
m
```

    'attackistonight'

```python
plaintext = commons.lettersOfPlaintext(m)
```

```python
plaintext
```

    ['a', 't', 't', 'a', 'c', 'k', 'i', 's', 't', 'o', 'n', 'i', 'g', 'h', 't']

```python
plaintext_idx = []
for i in plaintext:
    plaintext_idx.append(commons.letterToNumber(i))
```

```python
plaintext_idx
```

    [0, 19, 19, 0, 2, 10, 8, 18, 19, 14, 13, 8, 6, 7, 19]

```python
plaintext_matrix = np.array(plaintext_idx)
```

```python
plaintext_matrix
```

    array([ 0, 19, 19,  0,  2, 10,  8, 18, 19, 14, 13,  8,  6,  7, 19])

```python
plaintext_matrix.resize(5,3)
```

```python
plaintext_matrix
```

    array([[ 0, 19, 19],
           [ 0,  2, 10],
           [ 8, 18, 19],
           [14, 13,  8],
           [ 6,  7, 19]])

# Encryption

```python
encryption = np.matmul(plaintext_matrix, key) % 26
```

```python
encryption
```

    array([[ 5, 13, 22],
           [ 0,  6, 22],
           [ 9,  6,  9],
           [10,  3, 13],
           [17, 17, 16]], dtype=int32)

```python
encryption.resize(1, 15)
```

```python
ciphertext = []
for i in encryption.tolist()[0]:
    ciphertext.append(commons.numberToLetter(i))
```

```python
ciphertext
```

    ['f', 'n', 'w', 'a', 'g', 'w', 'j', 'g', 'j', 'k', 'd', 'n', 'r', 'r', 'q']

```python
plaintext
```

    ['a', 't', 't', 'a', 'c', 'k', 'i', 's', 't', 'o', 'n', 'i', 'g', 'h', 't']

# Decryption

```python
from sympy import Matrix
```

```python
inv_key = Matrix(key).inv_mod(26)
```

```python
inv_key = np.array(inv_key)
```

```python
inv_key
```

    array([[11, 22, 14],
           [7, 9, 21],
           [17, 0, 3]], dtype=object)

```python
inv_key = inv_key.astype(float)
```

```python
np.matmul(key, inv_key)%26
```

    array([[1., 0., 0.],
           [0., 1., 0.],
           [0., 0., 1.]])

```python
encryption.resize(5, 3)
```

```python
decryption = np.matmul(encryption, inv_key) % 26
```

```python
decryption
```

    array([[ 0., 19., 19.],
           [ 0.,  2., 10.],
           [ 8., 18., 19.],
           [14., 13.,  8.],
           [ 6.,  7., 19.]])

```python
decryption.resize(1, 15)
```

```python
restored_text = []
for i in decryption.tolist()[0]:
    restored_text.append(commons.numberToLetter(i))
```

```python
restored_text
```

    ['a', 't', 't', 'a', 'c', 'k', 'i', 's', 't', 'o', 'n', 'i', 'g', 'h', 't']


# Lake model

see also [this general introduction to the workbench](https://waterprogramming.wordpress.com/2017/11/01/using-the-exploratory-modelling-workbench/) as a source of inspiration for completing the assignment below

## the lake model
The exploratory modeling workbench includes an example folder. This folder contains a variety of examples that demonstrate the functionality of the workbench. Many of these examples have been drawn from published cases. Here, we use the Lake Problem as an example for demonstrating some of the key functionality of the workbench.

We demonstrate some of the key capabilities of the exploratory modeling workbench using the Lake problem. The lake problem is a stylized and hypothetical decision problem where the population of a city has to decide on the amount of annual pollution it will put into a lake. It the pollution in the lake passes a threshold, it will suffer irreversible eutrophication.

\begin{equation}
    X_{(t+1)}=X_t+a_t+\frac{(X_t^q)}{(1+X_t^q )}- bX_t+\epsilon_t
\end{equation}

where $X_t$ is the pollution at time $t$, $a_t$ is the rate of anthropogenic pollution at time $t$, $b$ is the lake’s natural removal rate, $q$ is the lake's natural recycling rate, $\epsilon_t$ is the rate of natural pollution at time $t$. The rate of anthropogenic pollution $a_t$ is the decision variable and is somewhere between 0, and 0.1. So $a_t \in [0,0.1]$. The natural pollution $\epsilon_t$ is modeled, following Singh et al. (2015), as a log normal distribution with mean $\mu$ and standard deviation $\sigma$.
There are four outcomes of interest. The first is the average concentration of phosphor in the lake.

\begin{equation}
    f_{phosphorus}=  \frac{1}{\left\vert{T}\right\vert} \sum\limits_{t\in{T}} X_t
\end{equation}

where $\left\vert{T}\right\vert$ is the cardinality of the set of points in time.
The second objective is the economic benefit derived from polluting the lake. Following Singh et al. (2015), this is defined as the discounted benefit of pollution mines the costs of having a polluted lake

\begin{equation}
    f_{economic} = \sum\limits_{t \in {T}}\alpha a_t \delta^t
\end{equation}

where $\alpha$ is the utility derived from polluting and $\delta$ is the discount rate. By default, $\alpha$ is 0.04.
The third objective is related to the year over year change in the anthropogenic pollution rate.

\begin{equation}
    f_{inertia} =\frac{1}{\left\vert{T}\right\vert-1} \sum\limits_{t=1}^{\left\vert{T}\right\vert} I(|a_{t}-a_{t-1} |>\tau)
\end{equation}

where $I$ is an indicator function that is 0 if the statement is false, and 1 if the statement is true, $\tau$ is the threshold that is deemed undesirable, and is for illustrative purposes et to 0.2. Effectively, f_{inertia} is the fraction of years where the absolute value of the change in anthropogenic pollution is larger then $\tau$.
The fourth objective is the fraction of years where the pollution in the lake is below the critical threshold.

\begin{equation}
    f_{reliability} =  \frac{1}{\left\vert{T}\right\vert} \sum\limits_{t \in T}I(X_{t}<X_{crit} )
\end{equation}

where $I$ is an indicator function that is 0 if the statement is false, and 1 if the statement is true, $X_{crit}$ is the critical threshold of pollution and is a function of both $b$ and $q$.

The lake problem is characterized by both stochastic uncertainty and deep uncertainty. The stochastic uncertainty arises from the natural inflow. To reduce this stochastic uncertainty, multiple replications are performed and the average over the replication is taken. Deep uncertainty is presented by uncertainty about the mean $\mu$ and standard deviation $sigma$ of the lognormal distribution characterizing the natural inflow, the natural removal rate of the lake $\beta$, the natural recycling rate of the lake $q$, and the discount rate $\delta$. The table below specifies the ranges for the deeply uncertain factors, as well as their best estimate or default values.

## Assignment
1. Given the Python implementation of the lake problem in lakemodel_function.py, adapt this code and connect it to the workbench

for the uncertainties, use the following table

|Parameter	|Range	        |Default value|
|-----------|--------------:|------------:|
|$\mu$    	|0.01 – 0.05	|0.02         |
|$\sigma$	|0.001 – 0.005 	|0.0017       |
|$b$      	|0.1 – 0.45	    |0.42         |
|$q$	    |2 – 4.5	    |2            |
|$\delta$	|0.93 – 0.99	|0.98         |

For now, assume that for each year a release decision is made. The release is between 0 and 0.1. Carefully look at line 23 in lake_model.py to identify the name to use for each lever.

```python

model.levers = [RealParameter(str(i), 0, 0.1) for i in range(100)]

levers = []
for i in range(100):
    levers.append(RealParameter(str(i), 0, 0.1))

model.levers = levers
```

2. Explore the behavior of the system in the absence of any release using 1000 scenarios, and the default sampling approach.
    * visualize the outcomes of interest, are there any apparent trade-offs?
    * can you visually identify the uncertainties that drive system behavior?

```python

```

3. Explore the behavior of the system over 1000 scenarios for 4 randomly sampled candidate strategies.
    * visualize the outcomes of interest
    * what can you say about how the release decision influences the system?

```python

```

4. If you have not used parallelization in the foregoing, try to adapt your code to use parallelization. The workbench comes with two evaluators for parallelization. The `MultiProcessingingEvaluator` and the `IpyparallelEvaluator`. When can you use each?
    * for windows users, carefully read [the python documentation](https://docs.python.org/3.6/library/multiprocessing.html). Can you use multiprocessing from within the notebook if the lake_model function is defined within the notebook?

```python

```


#
Logaritmos y trigonometría con Sympy.

```python
from sympy import *
init_printing()
x, y, z = symbols('x, y, z')
```

En otro *notebook* hemos trabajado con las funciones logarítmicas y trigonométricas en modo aproximado. Ahora vamos a trabajar en modo exacto. En particular Sympy es capaz de manejar las propiedades de las funciones.

Las propiedades fundamentales de los logaritmos sirven para desarrollar expresiones donde aparecen logaritmos. La función principal para desarrollar logaritmos es **expand_log**. A veces, para poder utilizar toda la potencia de esta función debemos añadir un argumento opcional: **force = True**. Otra manera de hacer esto es utilizar el comando **expand** y utilizar el argumento opcional **log = True**.

###
Comprueba con Sympy las siguientes propiedades de los logaritmos:

* $\log(x \cdot y) = \log(x) + \log(y)$

* $\log(x^z) = z\cdot \log(x)$

* $\displaystyle\log\left(\frac{x}{y}\right) = \log(x) -\log(y)$

* $\displaystyle \log(\sqrt{x}) = \frac{1}{2} \log(x)$

```python

```

```python

```

###
Calcula los siguientes logaritmos:

* $\log_2(1024)$

* $\log_2(\sqrt{8})$

* $\log_2(\sqrt[5]{16})$

```python

```

```python

```

```python

```

###
Realiza los siguientes cálculos trigonométricos:

* $\displaystyle \cos\left(\frac{\pi}{3}\right)$

* $\displaystyle \arcsin\left(\frac{1}{2}\right)$

* $\displaystyle 5\cos\left(\frac{\pi}{2}\right) +2\cos(\pi) -  \tan \left(\frac{2\pi}{3}\right)$

```python

```

```python

```

Las funciones **radians** y **deg2rad** convierten grados a radianes, pero en forma aproximada. Si queremos realizar cálculos exactos, debemos multiplicar los grados por $\pi$ y dividir el resultado entre $180$.

###
Realiza los siguientes cálculos trigonométricos:

* $\cos(15^\mathrm{o})$

* $\cos(30^\mathrm{o}) -3\tan(135^\mathrm{o})$

```python

```

```python

```

La función que sirve para desarrollar expresiones trigonométricas es **expand_trig**. También se puede utilizar **expand** con el argumento opcional **trig = True**.

###
Comprueba las siguientes propiedades trigonométricas.

* $\sin(x+y)=\sin(x)\cos(y)+\cos(x)\sin(y)$

* $\cos(x+y) = \cos(x)\cos(y)-\sin(x)\sin(y)$

* $\tan(x+y) = ?$

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

notebooks/09.- Logaritmos y trigonometria con Sympy.ipynb


## Solution {-}

\begin{equation}
  S_X(j\omega) = \frac{1}{(\omega^2 + 1)^2}
\end{equation}

Rewrite PSD in s-domain:

\begin{equation}
  S_X(s) = \frac{1}{(-s^2 + 1)^2}
\end{equation}

PSD can be written as:

\begin{equation}
  S_X = \frac{As + B}{(s+1)^2} + \frac{Cs + D}{(-s + 1)^2}
\end{equation}

Rewrite PSD again as partial fractions:

\begin{equation}
  S_X = \frac{C_1}{(s+1)^2} + \frac{C_2}{(s + 1)} + \frac{C_3}{(-s + 1)^2} + \frac{C_4}{(-s + 1)}
\end{equation}

Solve positive time part, i.e. $C_1$ and $C_2$:

\begin{align}
  C_1 =& \frac{1}{(s + 1)^2 (-s + 1)^2} (s + 1)^2 = \frac{1}{4}\\
  C_2 =& \frac{d}{ds} [\frac{1}{(s + 1)^2 (-s + 1)^2} (s + 1)^2] = \frac{1}{4}\\
\end{align}

Autocorrelation function $X(t)$:

\begin{equation}
  R_X(\tau) = \frac{1}{4}|\tau| e^{-|\tau|} + \frac{1}{4} e^{-|\tau|}
\end{equation}

```python
from sympy import symbols, inverse_fourier_transform

w, t = symbols('omega tau')

# Given Spectral Density Function
Sx = 1/(1 + w**2)**2

Rx = inverse_fourier_transform(Sx, w, t)
Rx
```

$\displaystyle - \frac{\pi \left(2 \pi \tau \sinh{\left(2 \pi \tau \right)} - 2 \pi \tau \cosh{\left(2 \pi \tau \right)} + \sinh{\left(2 \pi \tau \right)} - \cosh{\left(2 \pi \tau \right)}\right)}{2}$


### Plane Frame Member Local Stiffness Matrix
This page develops the various member stiffness relationships needed to analyze a planar frame by the matrix stiffness method.

#### Numbering and Sign Convention, Local Coordinates

The above figure shows the numbering and sign convention used for the six end displacements and forces for a general 2D frame member.  These are shown relative to the local coordinate system of the member, with origin at the left, or *j-end*, and *x*-axis coincident with the axis of the member.  The *y*-axis is positive upwards. It is sometimes necessary to distinguish member ends; the term *j-end* is used to refer to the left end or origin, and *k-end* is the other end (note that the CAL manual uses the *i-* and *j-* end for the same purpose).

Of course $f_0$ and $f_3$ are axial forces, $f_1$ and $f_4$ are shear forces, and $f_2$ and $f_5$ are bending moments.

$\newcommand{\mat}[1]{\left[\begin{matrix}#1\end{matrix}\right]}$

The local member stiffness matrix, $\mat{K_l}$, expresses the end forces as a function of the end displacements, thus:
$$
\mat{f_0\\ f_1\\ f_2\\ f_3\\ f_4\\ f_5} =
\mat{K_l}
\mat{\delta_0\\ \delta_1\\ \delta_2\\ \delta_3\\ \delta_4\\ \delta_5}
$$

### Derivation of the Local Member Stiffness Matrix, $\mat{K_l}$

```python
from sympy import *
init_printing(use_latex='mathjax')
from sympy.matrices import Matrix
```

#### Moment Flexibility Coefficients

We will first determine the inverse relationship for end moments -- the rotation flexibility matrix, $\mat{F}$, that expresses the end *rotations* as a function of the end *moments*, as shown in the following figure.

$$
\mat{\delta_2\\ \delta_5} = \mat{F} \mat{f_2\\f_5}
$$

Note that at each end, there are two other forces acting - axial and shear: $f_0$, $f_1$ and $f_3$, $f_4$. These are not
shown, for simplicity of figure.

The method of virtual work is probably the simplest and most direct way to do develop the rotation flexibility matrix.

The following figure shows unit moments placed individually at each end (at location #s 2 and 5).  Below each segment
is the corresponnding bending moment diagram, $m_i$:

The figure also
shows the resulting rotational displacements $\alpha_{ij}$ for each segment; these are the coefficients of the flexibility matrix:

$$
\mat{F} = \mat{\alpha_{22} & \alpha_{52}\\ \alpha_{25} & \alpha_{55}}
$$

The coefficients of $\mat{F}$, $\alpha_{ij}$, are the end displacements due to unit values of the end forces.
Specifically $\alpha_{ij}$ is the rotation at $i$ due to a unit moment at $j$ and is calculated using the method
of virtual work (integrating products of bending moments):

$$
\mat{F} = \mat{\int_0^L\frac{m_2 m_2}{EI} dx & \int_0^L\frac{m_2 m_5}{EI} dx \\
               \int_0^L\frac{m_5 m_2}{EI} dx & \int_0^L\frac{m_5 m_5}{EI} dx }
$$

```python
L,E,I,A,x = symbols('L E I A x')

m2 = x/L - 1     # unit moment at DOF 2
m5 = x/L         # unit moment at DOF 5
EI = E*I
F = Matrix([[m2*m2, m2*m5],
            [m5*m2, m5*m5]]).integrate((x,0,L))/EI
F
```

$$\left[\begin{matrix}\frac{L}{3 E I} & - \frac{L}{6 E I}\\- \frac{L}{6 E I} & \frac{L}{3 E I}\end{matrix}\right]$$

#### Moment Stiffness Coefficients
The moment stiffness matrix, $\mat{M}$ expresses the two end moments as functions
of the end rotations.

$$
\mat{f_2\\f_5} = \mat{M} \mat{\delta_2 \\ \delta_5}
$$

$$
\mat{M} = \mat{F}^{-1}
$$

and that is just the inverse of the *2x2* rotation flexibility matrix.

```python
M = F**-1
M
```

$$\left[\begin{matrix}\frac{4 E}{L} I & \frac{2 E}{L} I\\\frac{2 E}{L} I & \frac{4 E}{L} I\end{matrix}\right]$$

#### Total End Forces - Accounting For Axial and Shear Effects
The matrix $\mat{T_f}$ gives the total set of end forces in terms of the two end moments (i.e., the end forces that are in equilibrium with the end moments):

$$
\mat{f_0\\ f_1\\ f_2\\ f_3\\ f_4\\ f_5} = \mat{T_f} \mat{f_2\\f_5}
$$

```python
# transform end moments to end thrust,shear,moment
Tf = Matrix([[0,0],[1/L,1/L],[1,0],[0,0],[-1/L,-1/L],[0,1]])
Tf
```

$$\left[\begin{matrix}0 & 0\\\frac{1}{L} & \frac{1}{L}\\1 & 0\\0 & 0\\- \frac{1}{L} & - \frac{1}{L}\\0 & 1\end{matrix}\right]$$

```python
f2,f5 = symbols('f2 f5')  # confirm that that is OK
Tf*Matrix([[f2],
           [f5]])
```

$$\left[\begin{matrix}0\\\frac{f_{2}}{L} + \frac{f_{5}}{L}\\f_{2}\\0\\- \frac{f_{2}}{L} - \frac{f_{5}}{L}\\f_{5}\end{matrix}\right]$$

#### Total Stiffness Coefficients
Column $j$ of the member stiffness matrix $\mat{K_l}$ consists of all six end forces that
are consistent with a unit value of displacement $j$ and zero values of the other five displacements.

Therefore column 2 consists of the six end forces that occur when $\delta_2 = 1$
and all other displacements are 0.  That can be computed:

$$
\mat{T_f} \mat{M} \mat{1\\0}
$$

```python
Kl = zeros(6,6)   # build the member stiffness matrix one column at a time

Kl[:,2] = Tf * M * Matrix([[1],[0]])  # column 2
Kl[:,2]
```

$$\left[\begin{matrix}0\\\frac{6 E}{L^{2}} I\\\frac{4 E}{L} I\\0\\- \frac{6 E}{L^{2}} I\\\frac{2 E}{L} I\end{matrix}\right]$$

Column 5 is similar, but with $\delta_5 = 1$.

```python
Kl[:,5] =  Tf * M * Matrix([[0],[1]])   # column 5
Kl[:,5]
```

$$\left[\begin{matrix}0\\\frac{6 E}{L^{2}} I\\\frac{2 E}{L} I\\0\\- \frac{6 E}{L^{2}} I\\\frac{4 E}{L} I\end{matrix}\right]$$

The forces due to end translations are determined by mapping the unit translation to
end rotations of the elastic curve from the chord.

That is, for the case of $\delta_1 = 1$, (and notabley
$\delta_2 = \delta_5 = 0$), then the end moments will be consistent with end rotations
of $1/L$.  Column 1 will be calculated:

$$ \mat{T_f} \mat{M} \mat{1/L\\1/L} $$

```python
Kl[:,1] =  Tf * M * Matrix([[1/L],[1/L]])   # column 1
Kl[:,1]
```

$$\left[\begin{matrix}0\\\frac{12 E}{L^{3}} I\\\frac{6 E}{L^{2}} I\\0\\- \frac{12 E}{L^{3}} I\\\frac{6 E}{L^{2}} I\end{matrix}\right]$$

The end forces due to a unit value of $\delta_4$ are the same as those for $\delta_1$, but with reversed signs.

```python
Kl[:,4] =  Tf * M * Matrix([[-1/L],[-1/L]])   # column 4
Kl[:,4]
```

$$\left[\begin{matrix}0\\- \frac{12 E}{L^{3}} I\\- \frac{6 E}{L^{2}} I\\0\\\frac{12 E}{L^{3}} I\\- \frac{6 E}{L^{2}} I\end{matrix}\right]$$

Now the axial forces and displacements, which are de-coupled from shears and moments:

```python
ac = Matrix([E*A/L,0,0,-E*A/L,0,0])
ac
```

$$\left[\begin{matrix}\frac{A E}{L}\\0\\0\\- \frac{A E}{L}\\0\\0\end{matrix}\right]$$

```python
Kl[:,0] = ac     # column 0
Kl[:,3] = -ac    # column 3
```

### The member stiffness matrix in local coordinates, $\mat{K_l}$, is:

```python
Kl
```

$$\left[\begin{matrix}\frac{A E}{L} & 0 & 0 & - \frac{A E}{L} & 0 & 0\\0 & \frac{12 E}{L^{3}} I & \frac{6 E}{L^{2}} I & 0 & - \frac{12 E}{L^{3}} I & \frac{6 E}{L^{2}} I\\0 & \frac{6 E}{L^{2}} I & \frac{4 E}{L} I & 0 & - \frac{6 E}{L^{2}} I & \frac{2 E}{L} I\\- \frac{A E}{L} & 0 & 0 & \frac{A E}{L} & 0 & 0\\0 & - \frac{12 E}{L^{3}} I & - \frac{6 E}{L^{2}} I & 0 & \frac{12 E}{L^{3}} I & - \frac{6 E}{L^{2}} I\\0 & \frac{6 E}{L^{2}} I & \frac{2 E}{L} I & 0 & - \frac{6 E}{L^{2}} I & \frac{4 E}{L} I\end{matrix}\right]$$

And, for ease of copying into other software, here it is again.

```python
print(Kl)
```

    Matrix([[A*E/L, 0, 0, -A*E/L, 0, 0], [0, 12*E*I/L**3, 6*E*I/L**2, 0, -12*E*I/L**3, 6*E*I/L**2], [0, 6*E*I/L**2, 4*E*I/L, 0, -6*E*I/L**2, 2*E*I/L], [-A*E/L, 0, 0, A*E/L, 0, 0], [0, -12*E*I/L**3, -6*E*I/L**2, 0, 12*E*I/L**3, -6*E*I/L**2], [0, 6*E*I/L**2, 2*E*I/L, 0, -6*E*I/L**2, 4*E*I/L]])

```python

```


# Deflection with Moment area method

based on:

1. Yao_Mobasher2019_Analytical approach for the design of flexural elements made of reinforced ultra-high performance concrete
2. (Modern Concrete Technology) Alva Peled, Arnon Bentur, Barzin Mobasher - Textile Reinforced Concrete-CRC Press (2017)

## Determination of Deflection Response

The important methods used for finding out the slope and deflection at a section in a loaded beam are
1. Double integration method
2. Moment area method
3. Macaulay‟s method

With the moment-area method, the curvature diagram of a half-model according to the internal moment in Figure 1.1(c-d) is divided into several areas. By taking the moment of the areas around the left support, the mid-span deflection is obtained. A set of equations for calculating the mid-span deflection δ of the three-point and four-point bending at the first cracking (δcr), at ultimate (δu) when material has μ>μcrit are presented in Eqs.1 and qs.2. Mcr and Mu are the bending moments corresponding to the first cracking (φcr, δcr) and ultimate state (φu, δu), respectively.

figure 1.1
Moment–curvature distribution for 3PB and 4PB strain-hardening

## Import modules

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import sympy as sp
sp.init_printing()
from sympy import *

```

```python
from scipy.optimize import newton, brentq, root
```

```python
%run moment_curvature.ipynb
```

```python
model_data
```

```python
M,L ,L_p,phi,phi_u,phi_cr,M_u,M_cr,mu,mu_crit,delta_cr = sp.symbols(
    r'M,L,L_p,phi,phi_u,phi_cr,M_u,M_cr,mu,mu_crit,delta_cr',
    real=True, nonnegative=True
)
```

```python
load = sp.Piecewise(
     (M*4/L , True ),
)
load
```

```python
model_data = {
    phi_cr : 0.0035,
    L : 100,
    L_p : 25,
    M_cr : 25,
    mu_crit : 0.1
}
# The values and their sign should be determind!
```

```python
get_load = sp.lambdify(
    (M), load.subs(model_data), 'numpy'
)
```

```python
load = get_load(M_arr)
load
```

 Eq.1

```python
delta_3PB = sp.Piecewise(

    (Rational(1,12)*(L**2)*phi, phi <= phi_cr),
        ((L**2/(24*M**2))*((2*M**2-M*M_cr**2-M_cr**2)*phi + (M**2 + M*M_cr)*phi_cr) ,  phi > phi_cr),
)

delta_3PB
```

```python
delta_3PB.subs(model_data)
```

 Eq.2

```python
delta_4PB = sp.Piecewise(

    (Rational(23,216)*(L**2)*phi, phi <= phi_cr),
        ((L**2/(216*M**2))*((23*M**2-4*M*M_cr**2-4*M_cr**2)*phi + (4*M**2 + 4*M*M_cr)*phi_cr) ,  phi > phi_cr),
)

delta_4PB
```

```python
delta_4PB.subs(model_data)
```

```python
get_delta_3PB = sp.lambdify(
    (M, phi), delta_3PB.subs(model_data), 'numpy'
)
```

```python
get_delta_4PB = sp.lambdify(
    (M, phi), delta_4PB.subs(model_data), 'numpy'
)
```

```python
delta3PB = get_delta_3PB (M_arr, kappa_arr)
delta3PB
```

```python
delta4PB = get_delta_4PB (M_arr, kappa_arr)
delta4PB
```

```python
fig, ((ax1, ax2 , ax3)) = plt.subplots(1, 3, figsize=(15,5))
ax1.plot(kappa_arr, M_arr, 'tab:orange'),  ax1.set(title= 'Moment Curvature', xlabel=r'$\phi$', ylabel='$M$')
ax2.plot(delta3PB, load ,'tab:blue'),ax2.set(title= 'Load Deflection 3PB', xlabel=r'$\delta$', ylabel='$P$')
ax3.plot(delta4PB, load ,'tab:red'),ax3.set(title= 'Load Deflection 4PB', xlabel=r'$\delta$', ylabel='$P$')

fig. tight_layout()
fig.autofmt_xdate()

```

fig, ((ax1, ax2 , ax3)) = plt.subplots(1, 3, figsize=(15,5))
ax1.plot(kappa_arr, M_arr, 'tab:orange'),  ax1.set(title= 'Moment Curvature', xlabel=r'$\phi$', ylabel='$M$')
ax2.plot(delta3PB, load ,'tab:blue'),ax2.set(title= 'Load Deflection 3PB', xlabel=r'$\delta$', ylabel='$P$')
ax3.plot(delta4PB, load ,'tab:red'),ax3.set(title= 'Load Deflection 4PB', xlabel=r'$\delta$', ylabel='$P$')

fig. tight_layout()
fig.autofmt_xdate()

```python
delta_3PB = sp.Piecewise(

    (Rational(1,12)*(L**2)*phi, phi <= phi_cr),
        ((L**2/(24*M**2))*((2*M**2-M*M_cr**2-M_cr**2)*phi + (M**2 + M*M_cr)*phi_cr) ,  phi > phi_cr),
)

delta_3PB
```

```python
delta_3PB.subs(model_data)
```

 Eq.2

```python
delta_4PB = sp.Piecewise(

    (Rational(23,216)*(L**2)*phi, phi <= phi_cr),
        ((L**2/(216*M**2))*((23*M**2-4*M*M_cr**2-4*M_cr**2)*phi + (4*M**2 + 4*M*M_cr)*phi_cr) ,  phi > phi_cr),
)

delta_4PB
```

```python
delta_4PB.subs(model_data)
```

```python
get_delta_3PB = sp.lambdify(
    (M, phi), delta_3PB.subs(model_data), 'numpy'
)
```

```python
get_delta_4PB = sp.lambdify(
    (M, phi), delta_4PB.subs(model_data), 'numpy'
)
```

```python
delta3PB = get_delta_3PB (M_arr, kappa_arr)
delta3PB
```

```python
delta4PB = get_delta_4PB (M_arr, kappa_arr)
delta4PB
```

```python
fig, ((ax1, ax2 , ax3)) = plt.subplots(1, 3, figsize=(15,5))
ax1.plot(kappa_arr, M_arr, 'tab:orange'),  ax1.set(title= 'Moment Curvature', xlabel=r'$\phi$', ylabel='$M$')
ax2.plot(delta3PB, load ,'tab:blue'),ax2.set(title= 'Load Deflection 3PB', xlabel=r'$\delta$', ylabel='$P$')
ax3.plot(delta4PB, load ,'tab:red'),ax3.set(title= 'Load Deflection 4PB', xlabel=r'$\delta$', ylabel='$P$')

fig. tight_layout()
fig.autofmt_xdate()

```


# Constraint Handling

Constraint handling is often neglected in frameworks but is indeed an essential aspect of optimization.
Depending on the characteristics of the constraints, one method might be more suitable and lead to better results than another.
In principle, two ways exist how constraints can be dealt with: a) use an algorithm that can handle constraints internally; b) redefine the problem to have no constraints but still guarantee the optimum to be a feasible solution.
Both strategies will be elaborated in the following.

By default, the returned optimum is **always** required to be feasible. It might be required to obtain the **least infeasible** solution if no feasible could have been found in some situations. To enable this behavior, you can pass `return_least_infeasible=True` to the `minimize` function.

```python
from pymoo.algorithms.so_genetic_algorithm import GA
from pymoo.factory import get_problem
from pymoo.optimize import minimize

res = minimize(get_problem("g05"),
               GA(),
               return_least_infeasible=True,
               seed=1,
               verbose=False)

print("CV:", res.CV[0])
print("F:", res.F[0])
```

    CV: 0.01707583254530075
    F: 5186.280990822757

## Inequality Constraints

In pymoo, inequality constraints are always defined as "$\leq 0$" constraints. Thus, constraint violation is defined as follows: A solution is considered as **feasible** of all constraint violations are **less than** zero. A solution is considered as **infeasible** if at least one constraint violation is **larger than** zero.

$$
- x^4 \geq - 2
$$

Has to be converted to a "$\leq 0$" constraints to satisfy the implementation of constraints in pymoo:

$$
x^4  \leq 2\\
x^4 - 2 \leq 0
$$

Let us assume the whole optimization problem is then given by:

\begin{align}
\begin{split}
\min \;\; & f(x) = 1 + x^3 \\
\text{s.t.} \;\; & x^4 - 2\leq 0\\
& -2 \leq x \leq 2
\end{split}
\end{align}

The figure below shows the corresponding function and the optimum to be found. The constraint function is shown with a dashed line greater than zero (violated) and with a solid line where less or equal to zero (satisfied). The optimum is shown by a red cross, which is minimum in the feasible region.

```python
import numpy as np
import matplotlib.pyplot as plt
plt.rc('font', family='serif')

X = np.linspace(-2, 2, 10000)

F = 1 + X ** 3
G = X**4 - 2

plt.figure(figsize=(7, 5))

plt.plot(X, F, linewidth=2.0, color="blue", label="$f(x) = 1 + x^3$")

I = G<=0
plt.plot(X[I], G[I], linewidth=4.0, color="green")
plt.plot(X, G, linewidth=2.0, color="green", label="$g(x) = x^4 - 2$", linestyle='dotted')

plt.scatter([-1.18920654], [-0.6817904], color="red", s=100, marker="X", label="Optimum")
plt.xlim(-2, 2)
plt.xlabel("$x$")
plt.ylabel("$f$")

plt.legend(loc='upper center', bbox_to_anchor=(0.5, 1.12),
          ncol=4, fancybox=True, shadow=False)

plt.tight_layout()
plt.show()
```

The problem can be implemented straightforwardly:

```python
import numpy as np
from pymoo.util.misc import stack
from pymoo.model.problem import Problem

class MyProblem(Problem):

    def __init__(self):
        super().__init__(n_var=1,
                         n_obj=1,
                         n_constr=1,
                         xl=-2,
                         xu=2)

    def _evaluate(self, X, out, *args, **kwargs):
        out["F"] = 1 + X ** 3
        out["G"] = X**4 - 2
```

And then being solved as usually by initiating the problem, algorithm and running the minimize method:

```python
from pymoo.algorithms.so_pso import PSO
from pymoo.optimize import minimize

problem = MyProblem()

algorithm = PSO()

res = minimize(problem,
               algorithm,
               seed=1,
               verbose=False)

print("Best solution found: \nX = %s\nF = %s\nCV = %s" % (res.X, res.F, res.CV))
```

    Best solution found:
    X = [-1.18920654]
    F = [-0.6817904]
    CV = [0.]

For a bi-objective example with constraints, we like to refer to our [Getting Started Guide](../getting_started.ipynb) where a similar example is shown.

## Equality to Inequality Constraints

We got a couple of questions about how equality constraints should be handled in a genetic algorithm. In general, functions without any smoothness are challenging to handle for genetic algorithms. An equality constraint is an extreme case, where the constraint violation is 0 at precisely one point and otherwise 1.

Let us consider the following constraint $g(x)$ where $x$ represents a variable:

$g(x): x = 5$

An equality constraint can be expressed by an inequality constraint:

$g(x): |x - 5| \leq 0$

or

$g(x): (x-5)^2   \leq 0$

However, all of the constraints above are very strict and make **most search space infeasible**. Without providing more information to the algorithm, those constraints are challenging to satisfy.

Thus, the constraint can be smoothed by adding an epsilon to it and, therefore, having two inequality constraints:

$g'(x): 5 - \epsilon \leq x \leq 5 + \epsilon$

Also, it can be simply expressed in one inequality constraint by:

$g'(x): (x-5)^2  - \hat{\epsilon} \leq 0$

Depending on the $\epsilon$, the solutions will be more or less close to the desired value. However, the genetic algorithm does not know anything about the problem itself, making it challenging to handle and focus the search in the infeasible space.

## Parameter-less Approach / Feasibility First

Parameter-less constraint handling is the default constraint handling approach in **pymoo**. It strictly prefers feasible over infeasible solutions.

$$
F(x)=
\begin{cases}
f(x), \quad \text{if feasible} \\
f_{\max} + \sum_{j=1}^{|J|} \, \langle \, g_j (x) \, \rangle, \quad \rm{otherwise}
\end{cases}
$$

where $\langle \cdot \rangle$ is defined by

$$
\langle a \rangle =
\begin{cases}
a, \quad a \geq 0 \\
0, \quad \rm{otherwise}
\end{cases}
$$

This approach works well if an algorithm is based on comparisons. For instance, a genetic algorithm has to determine what individuals survive. Thus, this mapping can be utilized to create a rank for each individual.
If the $f_\max$ of the function is known beforehand, it can also be used to redefine the problem beforehand.
Redefining the problem is very similar to the approach discussed next.

## Penalty-based Approach / Lagrange Multipliers

The penalty-based constraint handling approach internally converts equality constraints to inequality constraints. Then, the violation of inequality and equality constraints penalize the objective value. This makes all non-constrained algorithms applicable; however, it also might create a more difficult fitness landscape to solve.

**NOTE**: Genetic algorithms are not made to handle equality constraints out of the box, but often need to be customized to satisfy the equality constraints directly. The approach below only works for a very simple and a very limited amount of equality constraints.

### Single-objective

\begin{align}
\begin{split}
\min \;\; & f(x) = x_1^2 + x_2^2 + x_3^2 \\
\text{s.t.} \;\; & x_1 \cdot x_2 \geq 1\\
& x_1 \cdot x_2 \leq 5\\
& x_2 + x_3 = 1\\
& 0 \leq x \leq 5
\end{split}
\end{align}

By converting the constraints to $\leq 0$ constraints the implementation of the problem looks as follows:

```python
def func_obj(x):
    return x[0] ** 2 + x[1] ** 2 + x[2] ** 2

constr_ieq = [
    lambda x: 1 - x[0] * x[1],
    lambda x: x[0] * x[1] - 5
]

constr_eq = [
    lambda x: 1 - x[1] - x[2]
]
```

For this example, we use the definition of a problem using the factory method `create_problem_from_funcs`:

```python
from pymoo.model.problem import FunctionalProblem
from pymoo.model.problem import ConstraintsAsPenaltyProblem

problem = FunctionalProblem(3,
                            func_obj,
                            xl=0,
                            xu=5,
                            constr_ieq=constr_ieq,
                            constr_eq=constr_eq
                            )

problem = ConstraintsAsPenaltyProblem(problem, penalty=1e6)
```

Then, an algorithm can be used to solve the problem as usual:

```python
from pymoo.algorithms.so_pso import PSO
from pymoo.optimize import minimize

algorithm = PSO(pop_size=50)

res = minimize(problem, algorithm, seed=1)

opt = res.opt[0]
X, F, CV = opt.get("X", "__F__", "__CV__")

print("Best solution found: \nX = %s\nF = %s\nCV = %s\n" % (X, F, CV))
```

    Best solution found:
    X = [1.25468499 0.79701394 0.20298647]
    F = [2.25066914]
    CV = [0.]

### Multi-objective

The almost equivalent could be applied for multi-objective optimization problems. The example below shows the [getting started guide](../getting_started.ipynb) example with a penalty being used to deal with the constraints:

#### Functions

```python
objs = [
    lambda x: x[0]**2 + x[1]**2,
    lambda x: (x[0]-1)**2 + x[1]**2

]

constr_ieq = [
    lambda x: 2*(x[0]-0.1) * (x[0]-0.9) / 0.18,
    lambda x: - 20*(x[0]-0.4) * (x[0]-0.6) / 4.8
]

constr_eq = []
```

#### Problem

```python
from pymoo.model.problem import FunctionalProblem
from pymoo.model.problem import ConstraintsAsPenaltyProblem

problem = FunctionalProblem(2,
                            objs,
                            xl=-2,
                            xu=2,
                            constr_ieq=constr_ieq,
                            constr_eq=constr_eq
                            )

problem = ConstraintsAsPenaltyProblem(problem, penalty=1e6)
```

#### Optimize

```python
from pymoo.algorithms.nsga2 import NSGA2
from pymoo.optimize import minimize

algorithm = NSGA2(pop_size=40)

res = minimize(problem,
               algorithm,
               seed=1)
```

#### Visualize

```python
from pymoo.visualization.scatter import Scatter

plot = Scatter(title = "Objective Space")
plot.add(res.F)
plot.show()
```

## Repair Operator

A simple approach is to handle constraints through a repair function. This is only possible if the equation of the constraint is known. The repair makes sure every solution that is evaluated is, in fact, feasible. Let us consider the following example where
the equality constraints need to consider more than one variable:

\begin{align}
\begin{split}
\min \;\; & f_1(x) = (x_1^2 + x_2^2) \\
\max \;\; & f_2(x) = -(x_1-1)^2 - x_2^2 \\[1mm]
\text{s.t.} \;\; & g_1(x_1, x_3) : x_1 + x_3 = 2\\[2mm]
& -2 \leq x_1 \leq 2 \\
& -2 \leq x_2 \leq 2 \\
& -2 \leq x_3 \leq 2
\end{split}
\end{align}

We implement the problem using by squaring the term and using an $\epsilon$ as we have explained above. The source code for the problem looks as follows:

```python
import numpy as np

from pymoo.model.problem import Problem

class MyProblem(Problem):

    def __init__(self):
        super().__init__(n_var=3,
                         n_obj=2,
                         n_constr=1,
                         xl=np.array([-2, -2, -2]),
                         xu=np.array([2, 2, 2]))

    def _evaluate(self, x, out, *args, **kwargs):
        f1 = x[:, 0] ** 2 + x[:, 1] ** 2
        f2 = (x[:, 0] - 1) ** 2 + x[:, 1] ** 2
        g1 = (x[:, 0] + x[:, 2] - 2) ** 2 - 1e-5

        out["F"] = np.column_stack([f1, f2])
        out["G"] = g1

```

As you might have noticed, the problem has similar characteristics to the problem in our getting started.
Before a solution is evaluated, a repair function is called. To make sure a solution is feasible, an approach would be to either set $x_3 = 2 - x_1$ or $x_1 = 2 - x_3$. Additionally, we need to consider that this repair might produce a variable to be out of bounds.

```python
from pymoo.model.repair import Repair

class MyRepair(Repair):

    def _do(self, problem, pop, **kwargs):
        for k in range(len(pop)):
            x = pop[k].X
            if np.random.random() < 0.5:

                x[2] = 2 - x[0]
                if x[2] > 2:
                    val = x[2] - 2
                    x[0] += val
                    x[2] -= val

            else:
                x[0] = 2 - x[2]
                if x[0] > 2:
                    val = x[0] - 2
                    x[2] += val
                    x[0] -= val

        return pop
```

Now the algorithm object needs to be initialized with the repair operator and then can be run to solve the problem:

```python
from pymoo.algorithms.nsga2 import NSGA2

algorithm = NSGA2(pop_size=100, repair=MyRepair(), eliminate_duplicates=True)

from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter

res = minimize(MyProblem(),
               algorithm,
               ('n_gen', 20),
               seed=1,
               verbose=True)

plot = Scatter()
plot.add(res.F, color="red")
plot.show()
```

In our case it is easy to verify if the constraint is violated or not:

```python
print(res.X[:, 0] + res.X[:, 2])
```

    [2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2.]

If you would like to compare the solution without a repair you will see how searching only in the feasible space helps:

```python
algorithm = NSGA2(pop_size=100, eliminate_duplicates=True)

res = minimize(MyProblem(),
               algorithm,
               ('n_gen', 20),
               seed=1,
               verbose=True)

plot = Scatter()
plot.add(res.F, color="red")
plot.show()

```

```python
print(res.X[:, 0] + res.X[:, 2])
```

    [1.99716258]

Here, in fact, the $\epsilon$ term is necessary to find any feasible solution at all.


## Exercise 1 : LDA Classification

From the last assignment, we have a basc understanding of how LDA works. Here, we want to use LDA on a practical example and see how it can help us in the classification process. Besides dimensionality reduction, LDA proides us with inforation of how important the new axes are for class discrimination and it offers a great way to show us if higher dimensional data is actually linearly seperable.

We are going to work with the Statlog(Vehical Silouttes) dataset throughout this exercise. It classifies four types of vehicles(OPEL, SAAB, BUS, VAN) based on the features exgtracted from the corresponding 2D silhoutte from an image of the vehicle(but only the silhoutte metrics are present in the dataset). In total, it measures k=18 features for n = 846 observations classified assigned to one of the four classes. The dataset is also summarized in Table 1.

There are two additional files accompanied by this exercise: train.csv which contains 752 observations supposed to be used for training and test.csv which contains 94 observations extracted to test the classifier.

Our goal is to test a classifier on this dataset and to use LDA as a preprocessing step. The general idea is to reduce the dimensions with LDA and pass the resulting projections to a classifier. For the latter, we are using and SVM.

In order to use LDA here, we first need to generalize the scatter metrics to C-class problems since the approach we have discussed so far is designed for a two-class problem only. The within-class matrix is easily extended by jst adding up every covariance matrix from each class
\begin{equation} S_W = \Sigma = \sum_{i=1}^C {\Sigma}_i \end{equation}

For the between-class scatter matrix, however, we need a new concept since we now have multiple mean vectors p; (one for each class). The idea is to compare the mean from each class with the overall mean
\begin{equation} \mu = \frac{1}{C} \sum_{i=1}^C\mu_i \end{equation}

of all observations (cf. Figure 1). Mathematically, we end up with something like

\begin{equation} S_B = \sum_{i=1}^C N_i(\mu_i - \mu)(\mu_i - \mu)^T \end{equation}

where we sum up all scatter from each class to the overall mean whereby weighting each matrix with the number N; of points in the class. This incorporates a weighting process where classes with more points contribute higher to the resulting matrix.

1. [Pen and Paper] In the previous assignment, the matrix Sp had rank 1 as there was only a single linearly independent vector in Sz. We now want to clarify how this situation changes when we use the general definition of Equation 3 which includes C classes. To answer this question, we need to find the number of linearly independent vectors of Sp. Here, the main ingredients of Sg are the difference vectors p; — pt (defined as column vectors). For simplicity, we discard the other terms and factors as they do not provide new information to our vector system. Then, we can define a new matrix which contains all information.
\begin{equation} A = (\mu_1 - \mu \hspace{0.5cm} \mu_2 - \mu \hspace{0.5cm} ... \hspace{0.5cm}\mu_C - \mu) \in \mathbb{R}^{k*C} \end{equation}

What is the highest possible rank of the matrix A? Hint: look at the column vectors of A
and Equation 2.

2. [Python] Let’s start with the implementation. We first want to write a function LDA(data, labels) which performs LDA on a labelled dataset (arbitrary dimension and class size). Similar to principal component analysis (covered later), this function should return three values:

    $\hspace{0.5cm}$*The coefficients of the new coordinate system. This is a matrix with the eigenvectors in the columns.*

    $\hspace{0.5cm}$*The projections of the data points relative to the new coordinate systems.*

    $\hspace{0.5cm}$*The contribution of each new axis to the class separability. This is called the latent and corresponds to the eigenvalues.*

    To compute these variables, we implement the procedure of the last assignment with the extensions to deal with multiple classes.

    a) Calculate the two scatter matrices Sy and Sz according to Equation 1 and Equation 3 by iterating over all classes and summing up the individual scatter matrices. Useful functions: np.mean, np. cov, np. outer.

      b) Set up the matrix Sy Sp. Use the (Moore-Penrose) pseudo-inverse (np.linalg.pinv) instead of the normal matrix inverse to improve the stability of your function.

      c) Calculate the eigenvalues and corresponding eigenvectors of the matrix Sj!Sg and sort the result descendingly by the eigenvalues. You can use the np. linalg.eig function but you have to do the sorting by yourself.

      d) With the help of the eigenvectors u; stored in the columns of the matrix U, compute the projections X for each data point x (stored in the rows of $X \in \mathbb{R}^{nxd}$)
      \begin{equation} \tilde{X} = X . U  =  \end{equation}

3. [Python] Write a function which loads the content of one of the *.csv files into your Python program. The last column of the array contains the class label of each observation (encoded as an integer, cf. Table 1). Store both, the data and the labels, in a Numpy array. Hint: set the data types to np. float64.

4. [Python] Training phase: use the content of the train.csv file in the following sub- tasks.

    a) The eigenvectors u; are the axes of the new coordinate system designed for class discrimination and the eigenvalues A; denote how important each axis actually is to discriminate the data. This is very useful to specify the dimension of our new feature space. Print the eigenvalues as “percentage explained”, i.e. what is the contribution of each eigenvalue relative to the available class discrimination
    \begin{equation} \tilde{\lambda}_i = |\frac{\lambda_i}{\sum_{j=1}^k \lambda_j}|\end{equation}

    b) Based on the previous result, choose the dimensionality d < C - 1 = 3 of the new feature space, ie. how many axes you want to use to describe your data in a class discriminative way.

    c) Train an SVM classifier with the d components of the projected data points x. In order to reduce the values to real numbers, you can use (here) the sum of both parts re(X) + im(x). For training, the svm object from the sk learn package with default parameters is suitable for the job (but you can experiment with different kernels if you like).

    d) Remember that in its raw form the dataset contains 18 different features. If you had to decide about the separability of the data, you would be left with a hard job since there is no direct way to visualize the data. Luckily, you chose d to be much smaller.

    Create a plot of the data you used to train the classifier, ie. the corresponding projections, as well as the resulting decision surfaces of the SVM. Figure 2 shows a possible solution and the following list contains some general recommendations:

    + Write a function as you need the same plot again later in the testing phase.

    + For the decision surfaces, it is helpful to generate a meshgrid of the relevant area and predict a label for each point in the grid. Then, the result can be visualized with the contour and contourf functions.

    + It is possible to combine multiple scatter plots. This is useful to plot all projec- tions of one class at a time.

5. [Python] Testing phase: use the content of the test . csv file in the following subtasks.

    a) Project all your test data to your d-dimensional LDA feature space.

    b) Predict labels for the test data with the help of the predict function.

    c) Since we also have labels for our test data, we can compare the predictions. A useful tool here is a confusion matrix where each entry cj; denotes how many observations from class i are classified to class j. The diagonal elements contain, therefore, the number of correctly classified elements for each class i. The function confusion_matrix from sklearn.metrics is suitable for this job.

    d) Plot the test projections together with the decision surfaces of the SVM (similar to the training phase).

    e) Explain why the confusion matrix is in coherence with the result of Figure 2.


## CCNSS 2018 Module 5: Whole-Brain Dynamics and Cognition
# Tutorial 1: Introduction to Complex Network Analysis (I)

*Please execute the cell bellow in order to initialize the notebook environment*

```python
!if [ ! -d data ]; then git clone https://github.com/ccnss/ccnss2018_students; \
                        cp -rf ccnss2018_students/module5/1_introduction_to_complex_network_analysis_1/data ./; fi
```

```python
import matplotlib.pyplot as plt    # import matplotlib
import numpy as np                 # import numpy
import math                        # import basic math functions
import random                      # import basic random number generator functions
import csv                         # import CSV(Comma Separated Values) file reading and writing
import scipy as sp                 # import scipy
from scipy import signal           # import signal module from scipy
from scipy import sparse           # import sparse module from scipy
import os                          # import basic os functions
import time                        # import time to measure real time
import collections                 # import collections
import networkx as nx              # import networkx

#fig_w, fig_h = (6, 4)
#plt.rcParams.update({'figure.figsize': (fig_w, fig_h)})

data_folder = 'data'

print('Available data files:', ' '.join(os.listdir(data_folder)))

data_file_1 = os.path.join(data_folder, 'gong_net_list.txt')
data_file_2 = os.path.join(data_folder, 'macaque_net_list.txt')
```

    Available data files: macaque_net_list.txt gong_net_list.txt

# Objectives

In this notebook we will introduce several important network analysis tools. Following up on the powerpoint tutorial, we will implement basic network measure using a human brain structural network as an example to apply the measures. We will compare the results with a random network with the same size.

## Background

Network theory is a powerful and universal tool which makes it possible to study the topology of complex systems. By observing the structural/functional network properties of the brain, we gain valuable insights into one of the most complex systems in the nature.

Using the structural human network as an example, we will study the following properties:

* Number of nodes ($N$) and number of links ($l$)
* Degree ($k$) of each node and degree distribution of the network
* Clustering coefficient ($C$)
* Characteristic path length ($L$) and efficiency ($E$)
* Betweenness centrality ($b$)

** EXERCISE 1: Connectivity list and connectivity matrix, random network**

The number of nodes $N$ and links $l$ in a network are the most fundamental characteristics of the network. When given a list of links between nodes, we can construct a connectivity matrix $A$ as follows:

\begin{align*}
\\
A_{ij} =
\begin{cases}
1 & \quad \quad \text{if } i \text{ and } j \text{ are connected}\\
0 & \quad \quad  \text{otherwise }\\
\end{cases}
\\
\end{align*}

In this definition, our resulting connectivity matrix will be symmetric. If two nodes are connected, they are called neighbours.

• Given a list of network connections representing a human brain network (Gong et al. network), construct a connectivity matrix: each row and column will correspond to a node, and each nonzero element of the matrix will correspond to the link between those nodes.

• Calculate the number of nodes in the network and the number of links between them. Simply, the number of rows (columns) of the matrix will be the number of nodes in the network, and the summation of the elements of the whole matrix devided by two will be the number of links in the network.

```python
net_gong_ut = [ row for row in csv.reader(open(data_file_1,'r'),delimiter='\t')]
net_gong_ut = np.array(net_gong_ut).astype(int)
net_gong_lt = net_gong_ut[:,(1,0,2)]
net_gong = np.concatenate( (net_gong_ut, net_gong_lt), axis=0)

net_row_gong = net_gong[:,0]-1
net_column_gong = net_gong[:,1]-1
net_value_gong = net_gong[:,2]
net_coord_gong = np.column_stack((net_row_gong,net_column_gong))
```

```python
# code for generating connectivity matrix from a connectivity list, for small or near-full network

def calculate_network_mat(net_list):
    len_net = np.amax(net_list)+1
    len_list = len(net_list)
    net_mat = np.zeros((len_net,len_net))
    for i in range(len_list):
        net_mat[ net_list[i,0] , net_list[i,1] ] = net_list[i,2]

    return net_mat
```

```python
# code for generating connectivity matrix from a connectivity list, for large yet sparse network

def calculate_network_mat2(net_list):
    len_net =  np.amax(net_list)+1
    net_mat = sp.sparse.coo_matrix((net_list[:,2], (net_list[:,0],net_list[:,1])), shape=(len_net,len_net) )
    # net_mat_csc = sp.sparse.csc_matrix(net_mat)

    return net_mat
```

```python
net_gong_m = net_gong.copy()
net_gong_m[:,:2] -= 1
net_gong_mat = calculate_network_mat(net_gong_m)
net_gong_mat2 = calculate_network_mat2(net_gong_m)

net_gong_mat_N = len(net_gong_mat)
net_gong_mat_l = np.sum(net_gong_mat)/2

print('# of nodes in Gong network N:', net_gong_mat_N)
print('# of links in Gong network l:', net_gong_mat_l)
```

**EXPECTED OUTPUT**
```
# of nodes in Gong network N: 78
# of links in Gong network l: 329.0
```

** EXERCISE 2: Degree and degree distribution**

The degree $k_i$ of each node $i$ is the number of links that node has:

$$ k_i = \sum_{j=1}^NA_{ij} ,$$

where $A_{ij}$ is the connectivity matrix.

• From the above connectivity matrix, calculate the degree of each node.

• After calculating the degree, we can calculate $p(k)$, probability of degree being k. Using this probability distribution $p(k)$, plot the **cumulative** distribution of node degrees $P(k)$, defined as follows:

$$ P(k) = \int _k ^\infty p(k) \ dk  $$

• Plot the above cumulative degree distribution $P(k)$ in a log-log plot.

• Repeat the above with a random network (with the provided code).

• Then, draw the Gong network and the random network (use the nx package with  the provided code).

```python
# random network generator #1
p = 329/(78*77/2)
net_rand_mat_l = 0
while net_rand_mat_l != 329:
    g = nx.erdos_renyi_graph(78,p)
    net_rand_mat = nx.to_numpy_matrix(g)
    net_rand_mat_N = len(net_rand_mat)
    net_rand_mat_l = np.sum(net_rand_mat)/2

net_rand_mat = np.array(net_rand_mat)
print(nx.is_connected(g))
print(net_rand_mat_N)
print(net_rand_mat_l)
print(type(net_rand_mat))
```

```python
# random network generator #2
G = nx.gnm_random_graph(78, 329)
net_rand2_mat = nx.to_numpy_matrix(G)
net_rand2_mat_N = len(net_rand2_mat)
net_rand2_mat_l = np.sum(net_rand2_mat)/2

net_rand2_mat = np.array(net_rand2_mat)
print(nx.is_connected(G))
print(net_rand2_mat_N)
print(net_rand2_mat_l)
print(type(net_rand_mat))

```

```python
# Write your code here
# computes degree of input network,
# and also the cumulative probability distribution for the degree, and prints the resulting figures
```

**EXPECTED OUTPUT**

```python
# draw Gong network and random network

Graph_gong = nx.from_numpy_matrix(net_gong_mat)
Graph_rand = nx.from_numpy_matrix(net_rand_mat)

plt.figure()
nx.draw(Graph_gong)
plt.show()

plt.figure()
nx.draw(Graph_rand)
plt.show()
```

**EXERCISE 3: Clustering coefficient **

The clustering coefficient ($C$) measures the local structure of a network, which can be calculated either for each node or the whole network. It is related to the number of a node's neighbours that are connected to each other.

• The clustering coefficient $C_i$ of node $i$ with degree $k_i$, can be defined as the ratio between the actual number of links between neighbours of $i\ (ln_i)$, and the maximum possible number of links between those neighbours:

$$ C_i = \frac{2 ln_i}{k_i(k_i - 1) }$$

• The clustering coefficient $C$ of whole network is the average of all individual clustering coefficients:

$$ C = \frac{1}{N}\sum_{i=1}^{N}{C_i }$$

• The clustering coefficient $C$ varies between 0 and 1. High $C$ means that neighboring nodes are well interconnected. This suggests redundancy in connections, which protects the network against random error, i.e. the loss of an individual node will have little impact on the structure of the network.

• Calculate the clustering coefficient $C$ for the human brain network and random network and compare the results.

• Confirm that your results are correct, by comparing it with a function from the networkx library, `nx.clustering()`.

```python
# insert your code here
# Write a function that calculates clustering coefficient of a given network and a node

def clustering_coeff(net_mat, node_number):
```

```python
# call the function and print the result (code #1)

net_gong_mat_cc = np.zeros(net_gong_mat_N)
for i in range(net_gong_mat_N):
    net_gong_mat_cc[i] = clustering_coeff(net_gong_mat, i)
net_gong_mat_cc_mean = np.mean(net_gong_mat_cc)
print('C of Gong network:',net_gong_mat_cc_mean)

net_rand_mat_cc = np.zeros(net_rand_mat_N)
for i in range(net_rand_mat_N):
    net_rand_mat_cc[i] = clustering_coeff(net_rand_mat, i)
net_rand_mat_cc_mean = np.mean(net_rand_mat_cc)
print('C of random network:',net_rand_mat_cc_mean)
```

```python
# call the function and print the result (code #2: using nx.clustering() )
graph = nx.to_networkx_graph(net_gong_mat)
gong_cc = nx.clustering(graph) # dictionary where key = node
gong_cc = np.array([gong_cc[i] for i in range(len(gong_cc))])
print('C of Gong network:', np.mean(gong_cc))

graph = nx.to_networkx_graph(net_rand_mat)
rand_cc = nx.clustering(graph) # dictionary where key = node
rand_cc = np.array([rand_cc[i] for i in range(len(rand_cc))])
print('C of random network:', np.mean(rand_cc))
```

**EXPECTED OUTPUT**

```
C of Gong network: 0.4917967040853441
C of random network: 0.10466937955625738
```

**EXERCISE 4: Characteristic path length**

The characteristic path length ($L$) is a measure indicating how integrated a network is and how easily information can flow. Calculate the following for both the human network and the random network and compare.

• The path length or distance $d_{ij}$ between two nodes $i$ and $j$ is the smallest number of links that connect $i$ to $j$. The characteristic path length $L$ of a network is the average distance over all pairs of nodes:

$$ L = \frac{1}{N(N-1)} \sum_{j,k \in N, \ i\neq j} {d_{ij}} $$

• The efficiency ($E$) of a network quantifies the exchange of information across the whole network where information is concurrently exchanged. It is defined as the average of the inverse distance over all nodes of the network.

$$ E = \frac{1}{N(N-1)} \sum_{j,k \in N, \ i\neq j} {1/d_{ij}} \\$$

Comparing the above two measures between a given network of interest and a random network of the same size gives you an idea of how economically the network is constructed.

• Write a code for a function calculating the distance matrix $d$ with $d_{ij}$ as its elements, given a connectivity matrix.

• Using the distance matrix, make a function for $L$ and $E$ for a given connectivity matrix.

• Hint: we can use the following useful property:

$$\\
\text{For connectivity matrix A}, {A^n}_{ij} \text{ is the number of paths of length $n$ from $i$ to $j$.}\\$$

Therefore, the smallest natural number $n$ such that the ${A^n}_{ij}$ becomes positive, gives you the distance $d_{i,j}$ between two nodes $i$ and $j$.

```python
# Write your code
# calculate distance matrix from a given connectivity matrix

def cal_net_mat_distance(net_mat):

    return net_mat_d

```

```python
# Write your code
# calculate characteristic path length and efficiency from a given distance matrix

def net_L_E(net_mat_d):

    return L, E
```

```python
net_gong_mat_d = net_mat_distance(net_gong_mat)
L, E = net_L_E(net_gong_mat_d)

print('L:', L,'\t', 'E:', E)
```

**EXPECTED OUTPUT**
```
L: 2.8071928071928074 	 E: 0.431013431013431
```

** EXERCISE 5: Betweenness centrality **

The betweenness centrality ($b$) is a measure of centrality in a network, based on shortest paths. For every pair of nodes in a connected network, there exists at least one shortest path between the nodes such that the number of links that the path passes through is minimized.

The betweenness centrality for each node ($b_i$) is the number of the shortest paths that pass through the nodes:

$$ b_i = \sum_{j,k \in N, \ j\neq k} {\frac{n_{jk}(i)}{n_{jk}}} $$

where $n_{jk}$ is the number of the shortest paths between nodes $j$ and $k$, and $n_{jk}(i)$ is the number of these paths that go through node $i$. One can also define betweenness centraity for the network:

$$ b = \frac{1}{N} \sum_{i = 1}^{N} b_i $$

• Write a code for a function that calculates $b_i$ for a given connectivity matrix and a given node $i$, using the `networkx` library. The functions `nx.to_networkx_graph()` and `nx.betweenness_centrality()` can be useful.

• Calculate the betweenness centrality for the human brain network and random network and compare the value.

• Fi you have time, calculate the betweenness centrality for each node of human network and identify the areas with highest betweenness centrality (choose top 5 nodes).

• What are the degrees and clustering coefficients ($k_i$ and $C_i$) of the nodes with high $b_i$?

• Repeat the calculation with the random network, and identify the characteristics of nodes with highest $b_i$ (choose top 5).

A node with high betweenness centrality would have more control over the network, because more information will pass through that node.

```python
# calculates betweenness centrality from a given connectivity matrix

def calculate_bc(net_mat):
    net_mat = np.matrix(net_mat)
    graph = nx.to_networkx_graph(net_mat)
    bc = nx.betweenness_centrality(graph,normalized=False) # dictionary where key = node
    bc = np.array([bc[i] for i in range(len(bc))])
    return bc
```

```python
net_gong_bc = calculate_bc(net_gong_mat)
net_gong_bc_mean = np.mean(net_gong_bc)
print('BC of Gong network:', net_gong_bc_mean)

net_rand_bc = calculate_bc(net_rand_mat)
net_rand_bc_mean = np.mean(net_rand_bc)
print('BC of random network:', net_rand_bc_mean)
```

**EXPECTED OUTPUT**

```
BC of Gong network: 69.57692307692308
BC of random network: 48.14102564102564
```

**EXERCISE 6: Comparison with other networks (optional, possible for mini project)**

Apparantly, the network characteristics of human brain network and random network are very different from each other. Repeat the comparison with macaque brain network data, and see if the macaque brain network exhibit similar properties compared with the human brain network.

```python

```

**EXERCISE 7: Random networks (optional, possible for mini project)**

We can use random network model of Erdős–Rényi to generate random networks.

An Erdős–Rényi network is constructed by connecting nodes randomly with probability $p$. An  Erdős–Rényi network $G(N, p)$ will have on average the following number of links:

$$l  = \binom{N}{2}p = \frac{N!}{2!(N-2)!}p .$$

In 1960, Erdős–Rényi showed that if the probability $p$ is above the following threshold, the graph $G(N,p)$ will almost surely be connected:

\begin{align} \text{if } p> \frac{\ln n}{n} &\text{, the the graph $G(N,p)$ is almost surely connected.} \\
\text{if } p< \frac{\ln n}{n} &\text{, the the graph $G(N,p)$ is almost surely disconnected.} \end{align}

• Construct a code generating random network $G$. One way of doing it is the following:

i) Give as the input: $N$ and $p$.

ii) Go through all the possible pairs of nodes of $G(N,p)$ with `for` loops.

iii) Generate a random number for each pair.

iv) Add the link of the specific pair to the connectivity list if the generated random number is lower than the value of $p$.

• With the same number of nodes and links as the human brain network, make a random network. For a fair compairson, one must first estimate $p$ from the data. Make sure that the resulting network  is connected. An easy way to do this is to run the code repeatedly until the output graph is connected.

• Is the calculated $p$ larger than $(\ln n) / n $?

```python

```


# Quantum Angular Momentum

This file will show how to use the various objects and methods in the `sympy.physics.quantum.spin` module, with some examples. Much of the work in this module is based off Varschalovich "Quantum Theory of Angular Momentum".

## Imports

```python
from IPython.display import display
```

```python
from sympy import init_printing
init_printing(use_latex=True)
```

```python
from sympy import factor, pi, S, Sum, symbols
from sympy.physics.quantum.spin import (
    Jminus, Jx, Jz, J2, J2Op, JzKet, JzKetCoupled, Rotation, WignerD, couple, uncouple
)
from sympy.physics.quantum import (
    Dagger, hbar, qapply, represent, TensorProduct
)
```

## Basic spin states and operators

We can define simple spin states and operators and manipulate them with standard quantum machinery.

Define a spin ket:

```python
jz = JzKet(1,1)
jz
```

Find the vector representation of the state:

```python
represent(jz)
```

Create and evaluate an innerproduct of a bra and a ket:

```python
ip = Dagger(jz)*jz
ip
```

```python
ip.doit()
```

Apply an angular momentum operator to the state:

```python
Jz*jz
```

```python
qapply(Jz*jz)
```

```python
Jminus*jz
```

```python
qapply(Jminus*jz)
```

We can also do this for symbolic angular momentum states:

```python
j, m = symbols('j m')
jz = JzKet(j, m); jz
```

```python
J2*jz
```

```python
qapply(J2*jz)
```

Find the matrix representation of a angular momentum operator:

```python
represent(Jz, j=1)
```

## Utilizing different bases

Angular momentum states and operators can be transformed between different spin bases. We can rewrite states as states in another basis:

```python
jz = JzKet(1, 1)
jz.rewrite("Jx")
```

Vector representation can also be done into different bases:

```python
represent(jz, basis=Jx)
```

When applying operators in another spin basis, any conversion necessary to apply the state is done, then the states are given back in the original basis. So in the following example, the state returned by `qapply` are in the $J_z$ basis:

```python
Jx*jz
```

```python
qapply(Jx*jz)
```

Rewriting states and applying operators between bases can also be done symbolically. In this case, the result is given in terms of Wigner-D matrix elements (see the next section for more information on the `Rotation` operator).

```python
jz = JzKet(j, m)
jz.rewrite("Jx")
```

## Rotation operator

Arbitrary rotations of spin states, written in terms of Euler angles, can be modeled using the rotation operator. These methods are utilized to go between spin bases, as seen in the section above.

Define an arbitrary rotation operator. The given angles are Euler angles in the `z-y-z` convention.

```python
a, b, g = symbols('alpha beta gamma')
Rotation(a, b, g)
```

Find the Wigner-D matrix elements of the rotation operator as given by $\langle j, m'|\mathcal{R}(\alpha, \beta, \gamma)|j,m\rangle$:

```python
mp = symbols('mp')
r = Rotation.D(j, m, mp, a, b, g)
r
```

Numerical matrix elements can be evaluated using the `.doit()` method:

```python
r = Rotation.D(1, 1, 0, pi, pi/2, 0)
r
```

```python
r.doit()
```

The Wigner small-d matrix elements give rotations when $\alpha=\gamma=0$. These matrix elements can be found in the same manner as above:

```python
r = Rotation.d(j, m, mp, b)
r
```

```python
r = Rotation.d(1, 1, 0, pi/2)
r
```

```python
r.doit()
```

You can also directly create a Wigner-D matrix element:

```python
WignerD(j, m, mp, a, b, g)
```

## Coupled and uncoupled states and operators

States and operators can also written in terms of coupled or uncoupled angular momentum spaces.

### Coupled states and operators

Define a simple coupled state of two $j=1$ spin states:

```python
jzc = JzKetCoupled(1, 0, (1, 1)); jzc
```

Note that the Hilbert space of coupled states is the direct sum of the coupled spin spaces. This can be seen in the matrix representation of coupled states:

```python
jzc.hilbert_space
```

```python
represent(jzc)
```

We can also couple more than two spaces together. See the `JzKetCoupled` documentation for more complex coupling schemes involving more than 2 spaces.

```python
jzc = JzKetCoupled(1, 1, (S(1)/2, S(1)/2, 1))
jzc
```

The normal operators are assumed to be diagonal in the corresponding coupled basis:

```python
qapply(Jz*jzc)
```

### Uncoupled states and operators

Uncoupled states are defined as tensor products of states:

```python
jzu = TensorProduct(JzKet(1, 1), JzKet(S(1)/2, -S(1)/2)); jzu
```

Vector representation of tensor product states gives the vector in the direct product space:

```python
represent(jzu)
```

Uncoupled operators are also defined as tensor products:

```python
jzopu = TensorProduct(Jz, 1)
jzopu
```

```python
qapply(jzopu*jzu)
```

Coupled operators which are diagonalized by uncoupled states (e.g. $J_z$ and uncoupled $J_z$ eigenstates) can also be applied:

```python
qapply(Jz*jzu)
```

Rewriting states works as before:

```python
jzu.rewrite("Jx")
```

### Coulping and Uncoupling States

The `couple` method will couple an uncoupled state:

```python
jzu = TensorProduct(JzKet(1, 1), JzKet(S(1)/2, -S(1)/2))
couple(jzu)
```

Similarly, the uncouple method will uncouple a coupled state:

```python
jzc = JzKetCoupled(2, 1, (1, S(1)/2, S(1)/2))
uncouple(jzc)
```

Uncoupling can also be done with the `.rewrite` method:

```python
jzc.rewrite("Jz", coupled=False)
```

The `uncouple` method can also uncouple normal states if given a set of spin bases to consider:

```python
jz = JzKet(2, 1)
uncouple(jz, (1, S(1)/2, S(1)/2))
```