# ***Introduction to Radar Using Python and MATLAB***
## Andy Harrison - Copyright (C) 2019 Artech House
<br/>

# Frank Code Ambiguity Function
***

Referring to Section 8.7.2,  Frank codes are polyphase codes representing a quadratic phase shift in a discrete manner.  To generate a Frank code, a single pulse is divided into $N$ groups.  Next, each group is divided into $N$ subpulses.  Therefore, the length of Frank codes are perfect squares, $L = N^2$.  Frank codes may be represented as a matrix of the form (Equation 8.72)

\begin{equation}
F_N = \frac{2\pi}{N}
\begin{bmatrix}
    0 & 0 & 0 & \dots  & 0 \\
    0 & 1 & 2 & \dots  & N-1 \\
    0 & 2 & 4 & \dots  & 2(N-1) \\
    \vdots & \vdots & \vdots & \ddots & \vdots \\
    0 & N-1 & 2(N-1) & \dots  & (N-1)^2
\end{bmatrix}\hspace{0.5in} \text{(rad)},
\end{equation}

where each row is the phase shift associated with the subpulses in that group.  Frank codes may also be expressed more compactly as (Equation 8.73)

\begin{equation}
    F_N = \frac{2\pi}{N}(m - 1)(n - 1) \hspace{0.2in} \text{for} \hspace{0.2in} m,n = 1 \dots N \hspace{0.5in} \text{(rad)}.
\end{equation}

For example, the Frank code for $N=3$ is (Equation 8.74)

\begin{equation}
    F_3 = \frac{2\pi}{3}[\;0\;\; 0\;\;  0\;\; 0\;\; 1\;\; 2\;\; 0\;\; 2\;\; 4\;] = \left[\;0\;\; 0\;\;  0\;\; 0\;\; \frac{2\pi}{3}\;\; \frac{4\pi}{3}\;\; 0\;\; \frac{4\pi}{3}\;\; \frac{2\pi}{3}\;\right].
\end{equation}

For phase shifts larger than $2\pi$, the phase wraps and the modulo with $2\pi$ is used.  The phase shift associated with each subpulse for Frank code $F_3$ is illustrated in Figure 8.21.

The ambiguity function of Frank code waveforms is similar to that of LFM waveforms.  This is expected as the Frank code is a discrete version of a quadratic phase shift.  For very long Frank codes, the phase shift between subpulses becomes small.  This can be problematic as phase instability of the system degrades the phase shift associated with each subpulse and reduces the performance of the waveform.  Other discrete polyphase waveforms have been developed to approximate quadratic phase shifts.
***

Begin by getting the library path

```python
import lib_path
```

Set the N-phase code and the chip width (s)

```python
n_phase = 3

chip_width = 0.1
```

Generate the N-phase Frank code

```python
from Libs.ambiguity.frank_code import n_phase_code

code = n_phase_code(n_phase)
```

Calculate the ambiguity function for the Frank coded waveform

```python
from Libs.ambiguity.ambiguity_function import phase_coded_wf

ambiguity, time_delay, doppler_frequency = phase_coded_wf(code, chip_width)
```

Display the zero-Doppler cut, the zero-range cut, and the 2D contour plot using the `matplotlib` routines

```python
from matplotlib import pyplot as plt

# Set the figure size

plt.rcParams["figure.figsize"] = (15, 10)

# Plot the ambiguity function

plt.plot(time_delay, ambiguity[round(len(doppler_frequency) / 2)], '')

# Set the time axis limits

plt.xlim(-len(code) * chip_width, len(code) * chip_width)

# Set the x and y axis labels

plt.xlabel("Time (s)", size=12)

plt.ylabel("Relative Amplitude", size=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the plot title and labels

plt.title('Frank Code Ambiguity Function', size=14)

# Set the tick label size

plt.tick_params(labelsize=12)
```

Create the zero-range cut

```python
 # Plot the ambiguity function

plt.plot(doppler_frequency,  ambiguity[:, round(len(time_delay) / 2)], '')

# Set the x and y axis labels

plt.xlabel("Doppler (Hz)", size=12)

plt.ylabel("Relative Amplitude", size=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the plot title and labels

plt.title('Frank Code Ambiguity Function', size=14)

# Set the tick label size

plt.tick_params(labelsize=12)
```

Create the two-dimensional contour plot

```python
from numpy import meshgrid

# Create the grid

t, f = meshgrid(time_delay, doppler_frequency)

# Plot the ambiguity function

plt.contour(t, f, ambiguity, 30, cmap='jet', vmin=-0.2, vmax=1.0)

# Set the time axis limits

plt.xlim(-len(code) * chip_width, len(code) * chip_width)

# Set the x and y axis labels

plt.xlabel("Time (s)", size=12)

plt.ylabel("Doppler (Hz)", size=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the plot title and labels

plt.title('Frank Code Ambiguity Function', size=14)

# Set the tick label size

plt.tick_params(labelsize=12)
```

```python

```


# Turbine isentropic efficiency

A steam turbine performs with an isentropic efficiency of $\eta_t = 0.84$. The inlet conditions are 4 MPa and 650°C, with a mass flow rate of 100 kg/s, and the exit pressure is 10 kPa. Assume the turbine is adiabatic.

Figure: Turbine

**Problem:**
 - Determine the power produced by the turbine
 - Determine the rate of entropy generation

```python
import numpy as np
import cantera as ct

from pint import UnitRegistry
ureg = UnitRegistry()
Q_ = ureg.Quantity
```

We can start by specifying state 1 and the other known quantities:

```python
temp1 = Q_(650, 'degC')
pres1 = Q_(4, 'MPa')

state1 = ct.Water()
state1.TP = temp1.to('K').magnitude, pres1.to('Pa').magnitude

mass_flow_rate = Q_(100, 'kg/s')
efficiency = 0.84

pres2 = Q_(10, 'kPa')
```

To apply the isentropic efficiency, we'll need to separately consider the real turbine and an equivalent turbine operating in a reversible manner. They have the same initial conditions and mass flow rate.

For the reversible turbine, an entropy balance gives:
\begin{equation}
s_{s,2} = s_1
\end{equation}
and then with $P_2$ and $s_{s,2}$ we can fix state 2 for the reversible turbine:

```python
state2_rev = ct.Water()
state2_rev.SP = state1.s, pres2.to('Pa').magnitude
state2_rev()
```

      water:

           temperature         319.003  K
              pressure           10000  Pa
               density         0.07464  kg/m^3
      mean mol. weight          18.016  amu
        vapor fraction        0.912916

                              1 kg            1 kmol
                           -----------      ------------
              enthalpy    -1.35945e+07       -2.449e+08     J
       internal energy    -1.37284e+07       -2.473e+08     J
               entropy         11017.2        1.985e+05     J/K
        Gibbs function     -1.7109e+07       -3.082e+08     J
     heat capacity c_p    inf              inf              J/K
     heat capacity c_v          100201        1.805e+06     J/K

Then, we can do an energy balance for the reversible turbine, which is also steady state and adiabatic:
\begin{equation}
\dot{m} h_1 = \dot{m} h_{s,2} + \dot{W}_{s,t}
\end{equation}
Then, recall that the isentropic efficiency is defined as
\begin{equation}
\eta_t = \frac{\dot{W}_t}{\dot{W}_{s,t}} \;,
\end{equation}
so we can obtain the actual turbine work using $\dot{W}_t = \eta_t \dot{W}_{s,t}$ :

```python
work_isentropic = mass_flow_rate * (
    Q_(state1.h, 'J/kg') - Q_(state2_rev.h, 'J/kg')
    )
work_actual = efficiency * work_isentropic
print(f'Actual turbine work: {work_actual.to(ureg.megawatt): .2f}')
```

    Actual turbine work: 118.73 megawatt

Then, we can perform an energy balance on the actual turbine:
\begin{equation}
\dot{m} h_1 = \dot{m} h_2 + \dot{W}_t \;,
\end{equation}
which we can use with the exit pressure to fix state 2.

```python
enthalpy2 = Q_(state1.h, 'J/kg') - (work_actual / mass_flow_rate)

state2 = ct.Water()
state2.HP = enthalpy2.to('J/kg').magnitude, pres2.to('Pa').magnitude
state2()
```

      water:

           temperature         328.393  K
              pressure           10000  Pa
               density       0.0661646  kg/m^3
      mean mol. weight          18.016  amu
        vapor fraction               1

                              1 kg            1 kmol
                           -----------      ------------
              enthalpy    -1.33683e+07       -2.408e+08     J
       internal energy    -1.35194e+07       -2.436e+08     J
               entropy         11725.3        2.112e+05     J/K
        Gibbs function    -1.72188e+07       -3.102e+08     J
     heat capacity c_p         1894.36        3.413e+04     J/K
     heat capacity c_v         1423.02        2.564e+04     J/K

Finally, we can perform an entropy balance on the actual turbine:
\begin{equation}
\dot{m} s_1 + \dot{S}_{\text{gen}} = \dot{m} s_2 \;,
\end{equation}
which allows us to find the rate of entropy generation.

```python
entropy_gen = mass_flow_rate * (
    Q_(state2.s, 'J/(kg K)') - Q_(state1.s, 'J/(kg K)')
    )
print(f'rate of entropy generation: {entropy_gen.to("kW/K"): .2f}')
```

    rate of entropy generation: 70.81 kilowatt / kelvin

```python

```


```python
import matplotlib.pyplot as plt
import matplotlib as mpl
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import scipy as sp
import scipy.linalg
import scipy.spatial
import sympy as sy
sy.init_printing()
```

#  The Dot Product

Consider two vectors

$$
\mathbf{u}=\left[\begin{array}{l}
u_{1} \\
u_{2} \\
\vdots \\
u_{n}
\end{array}\right] \quad \text { and } \bf{v}=\left[\begin{array}{l}
v_{1} \\
v_{2} \\
\vdots \\
v_{n}
\end{array}\right]
$$

The dot product of $\mathbf{u}$ and $\mathbf{v}$,i.e. $\mathbf{u}\cdot\mathbf{v}$ is defined as

$$
\left[\begin{array}{llll}
u_{1} & u_{2} & \cdots & u_{n}
\end{array}\right]\left[\begin{array}{c}
v_{1} \\
v_{2} \\
\vdots \\
v_{n}
\end{array}\right]=u_{1} v_{1}+u_{2} v_{2}+\cdots+u_{n} v_{n}
$$

We can generate two random vectors, then let's compare operations in NumPy.

```python
u = np.round(100*np.random.randn(10))
v = np.round(100*np.random.randn(10))
```

```python
u*v # this is element-wise multiplication
```

    array([ -846., -1700., -6486., -1886., -1100.,  6776., -6210.,  1586.,
            -245.,  2394.])

```python
u@v # matrix multiplication
```

```python
np.inner(u,v) # inner product here is the same as matrix multiplication
```

SymPy operation is the same like in linear algebra.

```python
u = sy.randMatrix(1, 10)
v = sy.randMatrix(1, 10)
```

```python
u*v.T
```

#  The Norm of a Vector

The norm is the length of a vector, definted by

$$
\|\mathbf{v}\|=\sqrt{\mathbf{v} \cdot \mathbf{v}}=\sqrt{v_{1}^{2}+v_{2}^{2}+\cdots+v_{n}^{2}}, \quad \text { and } \quad\|\mathbf{v}\|^{2}=\mathbf{v} \cdot \mathbf{v}
$$

A NumPy built-in ```np.linalg.norm()``` is for computing norms.The default setting is to compute the length of vectors from origins.

```python
a = [2, 6]
np.linalg.norm(a)
```

Verify the results.

```python
np.sqrt(2**2 + 6**2)
```

We can also compute a group of vectors' length, for instance $(2, 6)^T$, $(8, 2)^T$, $(9, 1)^T$

```python
A = np.array([[2, 8, 9],
              [6, 2, 1]])
np.linalg.norm(A, axis = 0)
```

    array([6.32455532, 8.24621125, 9.05538514])

#  Distance in $\mathbb{R}^n$

For $\mathbf{u}$ and $\mathbf{v}$ in $\mathbb{R}^{n}$, the distance between $\mathbf{u}$ and $\mathbf{v},$ written as dist $(\mathbf{u}, \mathbf{v}),$ is the length of the vector $\mathbf{u}-\mathbf{v} .$ That is,

$$
\operatorname{dist}(\mathbf{u}, \mathbf{v})=\|\mathbf{u}-\mathbf{v}\|
$$

Suppose we have two vectors $\mathbf{u} = (2, 9)$ and $\mathbf{v} = (-3, 4)$, compute the distance and visualize the results.

```python
u = np.array([2, 9])
v = np.array([-3, 4])
np.linalg.norm(u - v)
```

```python
fig, ax = plt.subplots(figsize = (12, 12))

vects = np.array([[2, 9], [-3, 4], [3, -4], [5, 5]])
col = ['red', 'blue', 'blue', 'green']
cordt = ['$(2, 9)$', '$(-3, 4)$', '$(3, -4)$', '$(5, 5)$']

for i in range(4):
    ax.arrow(0, 0, vects[i, 0], vects[i, 1], color = col[i], width = .08,
             length_includes_head = True,
             head_width = .3, # default: 3*width
             head_length = .6,
             overhang = .4)
    ax.text(x = vects[i][0], y = vects[i][1], s = cordt[i], size = 15)
ax.grid()

points = np.array([[2, 9], [5, 5], [3, -4], [-3, 4]])

ax.arrow(points[3][0], points[3][1], points[1][0], points[1][1], color = 'green', width = .08,
         length_includes_head = True,
         head_width = .3, # default: 3*width
         head_length = .6,
         overhang = .4)

line1 = np.array([points[0], points[1]])
ax.plot(line1[:,0], line1[:,1], c = 'b', lw = 3.5,alpha =0.5, ls = '--')

line2 = np.array([points[2], points[1]])
ax.plot(line2[:,0], line2[:,1], c = 'b', lw = 3.5,alpha =0.5, ls = '--')

ax.text(-2, 6, '$\|\|\mathbf{u}-\mathbf{v}\|\|$', size = 16)

###################### Axis, Spines, Ticks ##########################
ax.axis([-10, 10.1, -10.1, 10.1])
ax.spines['left'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['bottom'].set_position('center')
ax.spines['top'].set_color('none')
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')

ax.minorticks_on()
ax.tick_params(axis = 'both', direction = 'inout', length=12, width=2, which='major')
ax.tick_params(axis = 'both', direction = 'inout', length=10, width=1, which='minor')
```

From the graph, we know that the $\|\mathbf{u}-\mathbf{v}\|$ is $\sqrt{5^2 + 5^2}$.

```python
np.sqrt(5**2 + 5**2)
```

The same results as the ```np.linalg.norm(u - v)```.

#  Orthogonal Vectors

We have two vectors $\mathbf{u}$ and $\mathbf{v}$, and square the distance of $\|\mathbf{u}+\mathbf{v}\|$ and $\|\mathbf{u}-\mathbf{v}\|$

$$\begin{aligned}
\big[\operatorname{dist}(\mathbf{u},-\mathbf{v})\big]^{2} &=\|\mathbf{u}-(-\mathbf{v})\|^{2}=\|\mathbf{u}+\mathbf{v}\|^{2} \\
&=(\mathbf{u}+\mathbf{v}) \cdot(\mathbf{u}+\mathbf{v}) \\
&=\mathbf{u} \cdot(\mathbf{u}+\mathbf{v})+\mathbf{v} \cdot(\mathbf{u}+\mathbf{v}) \\
&=\mathbf{u} \cdot \mathbf{u}+\mathbf{u} \cdot \mathbf{v}+\mathbf{v} \cdot \mathbf{u}+\mathbf{v} \cdot \mathbf{v} \\
&=\|\mathbf{u}\|^{2}+\|\mathbf{v}\|^{2}+2 \mathbf{u} \cdot \mathbf{v}
\end{aligned}$$

$$\begin{aligned}
\big[\operatorname{dist}(\mathbf{u}, \mathbf{v})\big]^{2} &=\|\mathbf{u}\|^{2}+\|-\mathbf{v}\|^{2}+2 \mathbf{u} \cdot(-\mathbf{v}) \\
&=\|\mathbf{u}\|^{2}+\|\mathbf{v}\|^{2}-2 \mathbf{u} \cdot \mathbf{v}
\end{aligned}$$

Suppose $\mathbf{u} = (1, 4)$ and $\mathbf{v} = (-2, 2)$, visualize the vector and distances.

```python
fig, ax = plt.subplots(figsize = (12, 12))

vects = np.array([[1, 4], [-2, 2], [2, -2]])
colr = ['red', 'blue', 'blue']
cordt = ['$(1, 4)$', '$(-2, 2)$', '$(2, -2)$']
vec_name = ['$\mathbf{u}$', '$\mathbf{v}$','$\mathbf{-v}$']

for i in range(3):
    ax.arrow(0, 0, vects[i][0], vects[i][1], color = colr[i], width = .08,
             length_includes_head = True,
             head_width = .3, # default: 3*width
             head_length = .6,
             overhang = .4)
    ax.text(x = vects[i][0], y = vects[i][1], s = cordt[i], size = 15)
    ax.text(x = vects[i][0]/2, y = vects[i][1]/2, s = vec_name[i], size = 22)

ax.text(x = -1, y = 3, s = '$\|\|\mathbf{u}-\mathbf{v}\|\|$', size = 15)
ax.text(x = 1.5, y = 1, s = '$\|\|\mathbf{u}+\mathbf{v}\|\|$', size = 15)

############################### Dashed Line #######################
line1 = np.array([vects[0], vects[1]])
ax.plot(line1[:,0], line1[:,1], c = 'b', lw = 3.5,alpha =0.5, ls = '--')

line2 = np.array([vects[0], vects[2]])
ax.plot(line2[:,0], line2[:,1], c = 'b', lw = 3.5,alpha =0.5, ls = '--')

###################### Axis, Spines, Ticks ##########################
ax.spines['left'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['bottom'].set_position('center')
ax.spines['top'].set_color('none')
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')

ax.minorticks_on()
ax.tick_params(axis = 'both', direction = 'inout', length=12, width=2, which='major')
ax.tick_params(axis = 'both', direction = 'inout', length=10, width=1, which='minor')

ax.axis([-5, 5, -5, 5])
ax.grid()
```

Note that if $\big[\operatorname{dist}(\mathbf{u},-\mathbf{v})\big]^{2} = \big[\operatorname{dist}(\mathbf{u}, \mathbf{v})\big]^{2}$, $\mathbf{u}$ and $\mathbf{v}$ are orthogonal.According to equations above, it must be

$$\mathbf{u} \cdot \mathbf{v} = 0$$

This is one of the most important conclusion in linear algebra.

Suppose there is another vector $w = (2.5, 2.5)$, let's plot over the graph again.

```python
fig, ax = plt.subplots(figsize = (12, 12))

vects = np.array([[1, 4], [-2, 2], [2, -2], [2.5, 2.5]])
colr = ['red', 'blue', 'blue', 'green']
cordt = ['$(1, 4)$', '$(-2, 2)$', '$(2, -2)$', '$(2.5, 2.5)$']
vec_name = ['$\mathbf{u}$', '$\mathbf{v}$','$\mathbf{-v}$', '$\mathbf{w}$']

for i in range(4):
    ax.arrow(0, 0, vects[i][0], vects[i][1], color = colr[i], width = .08,
             length_includes_head = True,
             head_width = .3, # default: 3*width
             head_length = .6,
             overhang = .4)
    ax.text(x = vects[i][0], y = vects[i][1], s = cordt[i], size = 15)
    ax.text(x = vects[i][0]/2, y = vects[i][1]/2, s = vec_name[i], size = 22)

ax.text(x = -1, y = 3, s = '$\|\|\mathbf{u}-\mathbf{v}\|\|$', size = 15)
ax.text(x = 1.5, y = 1, s = '$\|\|\mathbf{u}+\mathbf{v}\|\|$', size = 15)

############################### Dashed Line #######################

line1 = np.array([vects[0], vects[1]])
ax.plot(line1[:,0], line1[:,1], c = 'b', lw = 3.5,alpha =0.5, ls = '--')

line2 = np.array([vects[0], vects[2]])
ax.plot(line2[:,0], line2[:,1], c = 'b', lw = 3.5,alpha =0.5, ls = '--')

line1 = np.array([vects[1], vects[3]])
ax.plot(line1[:,0], line1[:,1], c = 'k', lw = 3.5,alpha =0.5, ls = '--')

line2 = np.array([vects[2], vects[3]])
ax.plot(line2[:,0], line2[:,1], c = 'k', lw = 3.5,alpha =0.5, ls = '--')

###################### Axis, Spines, Ticks ##########################
ax.spines['left'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['bottom'].set_position('center')
ax.spines['top'].set_color('none')
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')

ax.minorticks_on()
ax.tick_params(axis = 'both', direction = 'inout', length=12, width=2, which='major')
ax.tick_params(axis = 'both', direction = 'inout', length=10, width=1, which='minor')

ax.axis([-5, 5, -5, 5])
ax.grid()
```

Use SciPy built-in function, construct two $2\times 2$ matrices for holding head and tail coordinates of the vector.

```python
a = np.array([[1, 4], [-2, 2]])
b = np.array([[1, 4], [2, -2]])
```

```python
distance = sp.spatial.distance.pdist(a, 'euclidean'); distance
```

    array([3.60555128])

```python
distance = sp.spatial.distance.pdist(b, 'euclidean');distance
```

    array([6.08276253])

Verify by NumPy ```.norm```.

```python
def dist(u, v):
    a = np.linalg.norm(u - v)
    return a
```

```python
u = np.array([1, 4])
v = np.array([-2, 2])
dist(u, v)
```

```python
u = np.array([1, 4])
v = np.array([2, -2])
dist(u, v)
```

Now Let's test if vector $(2.5, 2.5)^T$ is perpendicular to $(2, -2)^T$ and $(-2, 2)^T$.

```python
a = np.array([[2.5, 2.5], [-2, 2]])
b = np.array([[2.5, 2.5], [2, -2]])
distance1 = sp.spatial.distance.pdist(a, 'euclidean')
distance2 = sp.spatial.distance.pdist(b, 'euclidean')
```

```python
print(distance1, distance2)
```

    [4.52769257] [4.52769257]

They are the same length, which means $\mathbf{w}\perp \mathbf{v}$ and $\mathbf{w}\perp \mathbf{-v}$.

##  Orthogonal Complements

In general, the set of all vectors $\mathbf{z}$ that are orthogonal to subspace $W$ is called  orthogonal complement, or denoted as $W^\perp$.

The most common example would be

$$(\operatorname{Row} A)^{\perp}=\operatorname{Nul} A \quad \text { and } \quad(\operatorname{Col} A)^{\perp}=\operatorname{Nul} A^{T}$$

The nullspace of $A$ is perpendicular to the row space of $A$; the nullspace of $A^T$ is perpendicular to the column space of $A$.

#  Angles in $\mathbb{R}^n$

There is one formula of calculating angles in vector space, to derive it we need the law of cosine:

$$\|\mathbf{u}-\mathbf{v}\|^{2}=\|\mathbf{u}\|^{2}+\|\mathbf{v}\|^{2}-2\|\mathbf{u}\|\|\mathbf{v}\| \cos \vartheta$$

Rearrange, we get

$$\begin{aligned}
\|\mathbf{u}\|\|\mathbf{v}\| \cos \vartheta &=\frac{1}{2}\left[\|\mathbf{u}\|^{2}+\|\mathbf{v}\|^{2}-\|\mathbf{u}-\mathbf{v}\|^{2}\right] \\
&=\frac{1}{2}\left[u_{1}^{2}+u_{2}^{2}+v_{1}^{2}+v_{2}^{2}-\left(u_{1}-v_{1}\right)^{2}-\left(u_{2}-v_{2}\right)^{2}\right] \\
&=u_{1} v_{1}+u_{2} v_{2} \\
&=\mathbf{u} \cdot \mathbf{v}
\end{aligned}$$

In statistics, $\cos{\vartheta}$ is called  correlation coefficient .

$$
\cos{\vartheta}=\frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\|\|\mathbf{v}\|}
$$

##  Geometric Interpretation of Dot Product

You probably have notices that sometimes dot product and inner product are used interchangeably. Despite that, they do have different meanings.

Functions and polynomials can have inner product as well, but we usually use dot product to refer to inner product among vector space.

We will show that dot product has an interesting geometric interpretation.

Consider two vectors $\mathbf{a}$ and $\mathbf{u}$, pointing different direction, the angle form by them is $\vartheta$, also $\mathbf{u}$ is a unit vector. We would like to know how much $\mathbf{a}$ is pointing to $\mathbf{u}$'s direction.

And this value can be calculated by projecting $\mathbf{a}$ onto $\mathbf{u}$,

$$
 \|\mathbf{a}\|\cos{\vartheta}=\|\mathbf{a}\| \|\mathbf{u}\|\cos{\vartheta} = \mathbf{a} \cdot\mathbf{u}
$$

Any vector $\mathbf{b}$ can be normalized as a unit vector $\mathbf{u}$, then perform the calculation above, we can tell how much $\mathbf{b}$ is pointing the direction of $\mathbf{u}$.

#  Orthogonal Sets

If a set of vectors $S = \left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}$ in $\mathbb{R}^{n}$ has any arbitrary pair to be orthogonal, i.e. $\mathbf{u}_{i} \cdot \mathbf{u}_{j}=0$ whenever $i \neq j$, $S =\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}$ is called an orthogonal set.

Naturally, orthogonal set $S$ is linearly independent, they are also an  orthogonal basis for space spanned by $\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}$. Orthogonal basis has an advantage is that coordinates of the basis can be quickly computed.

For instance any $\mathbf{y}$ in $W$,

$$
\mathbf{y}=c_{1} \mathbf{u}_{1}+\cdots+c_{p} \mathbf{u}_{p}
$$

Because it is an orthogonal sets,

$$
\mathbf{y} \cdot \mathbf{u}_{1}=\left(c_{1} \mathbf{u}_{1}+c_{2} \mathbf{u}_{2}+\cdots+c_{p} \mathbf{u}_{p}\right) \cdot \mathbf{u}_{1}=c_{1}\left(\mathbf{u}_{1} \cdot \mathbf{u}_{1}\right)
$$

Thus

$$c_{j}=\frac{\mathbf{y} \cdot \mathbf{u}_{j}}{\mathbf{u}_{j} \cdot \mathbf{u}_{j}} \quad(j=1, \ldots, p)$$

#  Orthogonal Projection

For any $\mathbf{y}$ in $\mathbb{R}^n$, we want to decompose it as

$$
\mathbf{y} = \alpha \mathbf{u}+\mathbf{z} = \hat{\mathbf{y}}+\mathbf{z}
$$

where $ \mathbf{u}$ is perpendicular to $\mathbf{z}$, $\alpha$ is a scalar.And the subspace $L$ spanned by $\mathbf{u}$, the projection of $\mathbf{y}$ onto $L$ is denoted as

$$\hat{\mathbf{y}}=\operatorname{proj}_{L} \mathbf{y}=\frac{\mathbf{y} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}} \mathbf{u}$$

Becasue $\mathbf{u}\perp \mathbf{z}$, then $\mathbf{u}\cdot \mathbf{z} = 0$, replace $\mathbf{z}$ by $\mathbf{y}- \alpha \mathbf{u}$:

$$
(\mathbf{y}- \alpha \mathbf{u})\cdot \mathbf{u}= 0\\
\alpha = \frac{\mathbf{y}\cdot \mathbf{u}}{\mathbf{u}\cdot \mathbf{u}}
$$

Now we get the formula for projection onto $L$ spanned by $\mathbf{u}$.

##  A Visual Example in $\mathbb{R}^T$

Suppose we have $\mathbf{y} = (2, 5)^T$, $\mathbf{u} = (3, 1)^T$. Plot $\hat{\mathbf{y}}$ onto the subspace spanned $L$ by $\mathbf{u}$.

```python
fig, ax = plt.subplots(figsize = (12, 12))

vects = np.array([[4, 5], [2, 1]])
colr = ['red', 'blue']
cordt = ['$(4, 5)$', '$(2, 1)$']
vec_name = ['$\mathbf{y}$', '$\mathbf{u}$']

for i in range(2):
    ax.arrow(0, 0, vects[i][0], vects[i][1], color = colr[i], width = .03,
             length_includes_head = True,
             head_width = .1, # default: 3*width
             head_length = .2,
             overhang = .4)
    ax.text(x = vects[i][0], y = vects[i][1], s = cordt[i], size = 15)
    ax.text(x = vects[i][0]/2, y = vects[i][1]/2, s = vec_name[i], size = 22)

################################### Subspace L ############################
x = np.linspace(0, 8.1)
y = 1/2*x
ax.plot(x, y, lw = 3, color = 'red', alpha = .5)
ax.text(x = 6.5, y = 3, s = '$L = \operatorname{Span(\mathbf{u})}$', size = 19)

ax.axis([0, 8, 0, 8])
ax.grid()
```

Let's use formula to compute $\alpha$ and $\hat{\mathbf{y}}$.

```python
y = np.array([4, 5])
u = np.array([2, 1])
alpha = (y@u)/(u@u);alpha
```

```python
yhat = alpha*u;yhat
```

    array([5.2, 2.6])

With results above, we can plot the orthogonal projection.

```python
fig, ax = plt.subplots(figsize = (12, 12))

vects = np.array([[4, 5], [2, 1], [5.2, 2.6]])
colr = ['red', 'blue', 'green']
cordt = ['$(4, 5)$', '$(2, 1)$', '(5.2, 2.6)']
vec_name = ['$\mathbf{y}$', '$\mathbf{u}$', r'$\hat{\mathbf{y}} = \alpha\mathbf{u}$']

for i in range(3):
    ax.arrow(0, 0, vects[i][0], vects[i][1], color = colr[i], width = .03,
             length_includes_head = True,
             head_width = .1, # default: 3*width
             head_length = .2,
             overhang = .4, zorder = -i)
    ax.text(x = vects[i][0], y = vects[i][1], s = cordt[i], size = 19)
    ax.text(x = vects[i][0]/2, y = vects[i][1]/2, s = vec_name[i], size = 22)

##################################### Components of y orthogonal to u ##########################

point1 = [4, 5]
point2 = [5.2, 2.6]
line1 = np.array([point1, point2])
ax.plot(line1[:,0], line1[:,1], c = 'k', lw = 3.5,alpha =0.5, ls = '--')
ax.text(4.7, 3.8, '$\mathbf{z}$', size = 22)
################################### Subspace L ############################
x = np.linspace(0, 8.1)
y = 1/2*x
ax.plot(x, y, lw = 3, color = 'red', alpha = .5, zorder = -3)
ax.text(x = 6.5, y = 3, s = '$L = \operatorname{Span(\mathbf{u})}$', size = 19)

ax.axis([0, 8, 0, 8])
ax.grid()

#################################### Formula ################################
ax.text(x = 1, y = 7,
        s = r'$(\mathbf{y}- \alpha \mathbf{u})\cdot \mathbf{u}= 0$',
        size = 22, color = 'b')
ax.text(x = 1, y = 6.5,
        s = r'$\alpha = \frac{\mathbf{y}\cdot \mathbf{u}}{\mathbf{u}\cdot \mathbf{u}}$',
        size = 22, color = 'b')
ax.text(x = 1, y = 6,
        s = r'$\operatorname{proj}_{L}\mathbf{y}=\frac{\mathbf{y}\cdot\mathbf{u}}{\mathbf{u}\cdot\mathbf{u}}\mathbf{u}$',
        size = 22, color = 'b')
plt.show()
```

##  The Orthogonal Decomposition Theorem

To generalize the orthogonal projection in the higher dimension $\mathbb{R}^n$, we summarize the idea into the  orthogonal decomposition theorem.

Let $W$ be a subspace of $\mathbb{R}^{n}$. Then each $\mathbf{y}$ in $\mathbb{R}^{n}$ can be written uniquely in the form
$$
\mathbf{y}=\hat{\mathbf{y}}+\mathbf{z}
$$
where $\hat{\mathbf{y}}$ is in $W$ and $\mathbf{z}$ is in $W^{\perp} .$ In fact, if $\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}$ is any orthogonal basis of $W,$ then
$$
\hat{\mathbf{y}}=\frac{\mathbf{y} \cdot \mathbf{u}_{1}}{\mathbf{u}_{1} \cdot \mathbf{u}_{1}} \mathbf{u}_{1}+\cdots+\frac{\mathbf{y} \cdot \mathbf{u}_{p}}{\mathbf{u}_{p} \cdot \mathbf{u}_{p}} \mathbf{u}_{p}
$$
and $\mathbf{z}=\mathbf{y}-\hat{\mathbf{y}}$.

In $\mathbb{R}^{2}$, we project $\mathbf{y}$ onto subspace $L$ which is spanned by $\mathbf{u}$, here we generalize the formula for $\mathbb{R}^{n}$, that $\mathbf{y}$ is projected onto $W$ which is spanned by $\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}$.

##  A Visual Example in $\mathbb{R}^{3}$

A subspace $W=\operatorname{Span}\left\{\mathbf{u}_{1}, \mathbf{u}_{2}\right\}$, and a vector $\mathbf{y}$ is not in $W$, decompose $\mathbf{y}$ into $\hat{\mathbf{y}} + \mathbf{z}$, and plot them.

where

$$\mathbf{u}_{1}=\left[\begin{array}{r}
2 \\
5 \\
-1
\end{array}\right], \mathbf{u}_{2}=\left[\begin{array}{r}
-2 \\
1 \\
1
\end{array}\right], \text { and } \mathbf{y}=\left[\begin{array}{l}
1 \\
2 \\
3
\end{array}\right]$$

The projection onto $W$ in $\mathbb{R}^3$ is

$$
\hat{\mathbf{y}}=\frac{\mathbf{y} \cdot \mathbf{u}_{1}}{\mathbf{u}_{1} \cdot \mathbf{u}_{1}} \mathbf{u}_{1}+\frac{\mathbf{y} \cdot \mathbf{u}_{2}}{\mathbf{u}_{2} \cdot \mathbf{u}_{2}} \mathbf{u}_{2}=\hat{\mathbf{y}}_{1}+\hat{\mathbf{y}}_{2}
$$

The codes for plotting are quite redundent, however exceedingly intuitive.

```python
%matplotlib notebook

######################## Subspace W ##############################
s = np.linspace(-.5, .5, 10)
t = np.linspace(-.5, .5, 10)
S, T = np.meshgrid(s, t)

X1 = 2 * S - 2 * T
X2 = 5 * S + T
X3 = -S + T

fig = plt.figure(figsize = (7, 7))
ax = fig.add_subplot(projection='3d')
ax.plot_wireframe(X1, X2, X3, linewidth = 1.5, alpha = .3)

########################### vector y ###############################
y = np.array([1, 2, 3])
u1, u2 = np.array([2, 5, -1]), np.array([-2, 1, 1])

vec = np.array([[0, 0, 0, y[0], y[1], y[2]]])
X, Y, Z, U, V, W = zip(*vec)
ax.quiver(X, Y, Z, U, V, W, length=1, normalize=False, color = 'red', alpha = .6,arrow_length_ratio = .08, pivot = 'tail',
          linestyles = 'solid',linewidths = 3)

ax.text(y[0], y[1], y[2], '$\mathbf{y}$', size = 15)
########################### vector u1 and u2 ###############################

vec = np.array([[0, 0, 0, u1[0], u1[1], u1[2]]])
X, Y, Z, U, V, W = zip(*vec)
ax.quiver(X, Y, Z, U, V, W, length=1, normalize=False, color = 'red', alpha = .6,arrow_length_ratio = .08, pivot = 'tail',
          linestyles = 'solid',linewidths = 3)

vec = np.array([[0, 0, 0, u2[0], u2[1], u2[2]]])
X, Y, Z, U, V, W = zip(*vec)
ax.quiver(X, Y, Z, U, V, W, length=1, normalize=False, color = 'red', alpha = .6,arrow_length_ratio = .08, pivot = 'tail',
          linestyles = 'solid',linewidths = 3)

ax.text(u1[0], u1[1], u1[2], '$\mathbf{u}_1$', size = 15)
ax.text(u2[0], u2[1], u2[2], '$\mathbf{u}_2$', size = 15)
###########################  yhat ###############################

alpha1 = (y@u1)/(u1@u1)
alpha2 = (y@u2)/(u2@u2)

yhat1 = alpha1*u1
yhat2 = alpha2*u2
yhat = yhat1 + yhat2

vec = np.array([[0, 0, 0, yhat1[0], yhat1[1], yhat1[2]]])
X, Y, Z, U, V, W = zip(*vec)
ax.quiver(X, Y, Z, U, V, W, length=1, normalize=False, color = 'blue', alpha = .6,arrow_length_ratio = .08, pivot = 'tail',
          linestyles = 'solid',linewidths = 3, zorder= 3)

vec = np.array([[0, 0, 0, yhat2[0], yhat2[1], yhat2[2]]])
X, Y, Z, U, V, W = zip(*vec)
ax.quiver(X, Y, Z, U, V, W, length=1, normalize=False, color = 'blue', alpha = .6,arrow_length_ratio = .08, pivot = 'tail',
          linestyles = 'solid',linewidths = 3, zorder= 3)

vec = np.array([[0, 0, 0, yhat[0], yhat[1], yhat[2]]])
X, Y, Z, U, V, W = zip(*vec)
ax.quiver(X, Y, Z, U, V, W, length=1, normalize=False, color = 'pink', alpha = 1,arrow_length_ratio = .12, pivot = 'tail',
          linestyles = 'solid',linewidths = 3, zorder= 3)

ax.text(yhat1[0], yhat1[1], yhat1[2], '$\hat{\mathbf{y}}_1$', size = 15)
ax.text(yhat2[0], yhat2[1], yhat2[2], '$\hat{\mathbf{y}}_2$', size = 15)
ax.text(x = yhat[0], y = yhat[1], z = yhat[2],
        s = r'$\hat{\mathbf{y}}=\frac{\mathbf{y} \cdot \mathbf{u}_{1}}{\mathbf{u}_{1} \cdot \mathbf{u}_{1}} \mathbf{u}_{1}+\frac{\mathbf{y} \cdot \mathbf{u}_{2}}{\mathbf{u}_{2} \cdot \mathbf{u}_{2}} \mathbf{u}_{2}=\hat{\mathbf{y}}_{1}+\hat{\mathbf{y}}_{2}$', size = 15)
###########################  z ###############################
z = y - yhat
vec = np.array([[yhat[0], yhat[1], yhat[2], z[0], z[1], z[2]]])
X, Y, Z, U, V, W = zip(*vec)
ax.quiver(X, Y, Z, U, V, W, length=1, normalize=False, color = 'green', alpha = .6,arrow_length_ratio = .08, pivot = 'tail',
          linestyles = 'solid',linewidths = 3)

############################ Dashed Line ####################

line1 = np.array([y, yhat1])
ax.plot(line1[:,0], line1[:,1], line1[:,2], c = 'b', lw = 3.5,alpha =0.5, ls = '--')

line2 = np.array([y, yhat2])
ax.plot(line2[:,0], line2[:,1], line2[:,2], c = 'b', lw = 2.5,alpha =0.5, ls = '--')

line3 = np.array([yhat, yhat2])
ax.plot(line3[:,0], line3[:,1], line3[:,2], c = 'b', lw = 2.5,alpha =0.5, ls = '--')

line4 = np.array([yhat, yhat1])
ax.plot(line4[:,0], line4[:,1], line4[:,2], c = 'b', lw = 2.5,alpha =0.5, ls = '--')

############################# View Angel
ax.view_init(elev=-6, azim=12)
```

    <IPython.core.display.Javascript object>

#  Orthonormal Sets

An  orthonormal set is obtained from normalizing the orthogonal set, and also called orthonomal basis.

Matrices whose columns form an orthonormal set are important for matrix computation.

Let $U=\left[\begin{array}{lll}
\mathbf{u}_{1} & \mathbf{u}_{2} & \mathbf{u}_{3}
\end{array}\right]$, where $\mathbf{u}_{i}$'s are from an orthonormal set, the outer product, $U^TU$ is

$$U^{T} U=\left[\begin{array}{l}
\mathbf{u}_{1}^{T} \\
\mathbf{u}_{2}^{T} \\
\mathbf{u}_{3}^{T}
\end{array}\right]\left[\begin{array}{lll}
\mathbf{u}_{1} & \mathbf{u}_{2} & \mathbf{u}_{3}
\end{array}\right]=\left[\begin{array}{lll}
\mathbf{u}_{1}^{T} \mathbf{u}_{1} & \mathbf{u}_{1}^{T} \mathbf{u}_{2} & \mathbf{u}_{1}^{T} \mathbf{u}_{3} \\
\mathbf{u}_{2}^{T} \mathbf{u}_{1} & \mathbf{u}_{2}^{T} \mathbf{u}_{2} & \mathbf{u}_{2}^{T} \mathbf{u}_{3} \\
\mathbf{u}_{3}^{T} \mathbf{u}_{1} & \mathbf{u}_{3}^{T} \mathbf{u}_{2} & \mathbf{u}_{3}^{T} \mathbf{u}_{3}
\end{array}\right]$$

And $\mathbf{u}_{i}^{T} \mathbf{u}_{j}, i, j \in (1, 2, 3)$ is an inner product.

Because of orthonormal sets, we have

$$
\mathbf{u}_{1}^{T} \mathbf{u}_{2}=\mathbf{u}_{2}^{T} \mathbf{u}_{1}=0, \quad \mathbf{u}_{1}^{T} \mathbf{u}_{3}=\mathbf{u}_{3}^{T} \mathbf{u}_{1}=0, \quad \mathbf{u}_{2}^{T} \mathbf{u}_{3}=\mathbf{u}_{3}^{T} \mathbf{u}_{2}=0\\
\mathbf{u}_{1}^{T} \mathbf{u}_{1}=1, \quad \mathbf{u}_{2}^{T} \mathbf{u}_{2}=1, \quad \mathbf{u}_{3}^{T} \mathbf{u}_{3}=1
$$

which means

$$
U^TU = I
$$

Recall that we have a general projection formula

$$
\hat{\mathbf{y}}=\frac{\mathbf{y} \cdot \mathbf{u}_{1}}{\mathbf{u}_{1} \cdot \mathbf{u}_{1}} \mathbf{u}_{1}+\cdots+\frac{\mathbf{y} \cdot \mathbf{u}_{p}}{\mathbf{u}_{p} \cdot \mathbf{u}_{p}} \mathbf{u}_{p}
=\frac{\mathbf{y} \cdot \mathbf{u}_{1}}{{\mathbf{u}_{1}^T} \mathbf{u}_{1}} \mathbf{u}_{1}+\cdots+\frac{\mathbf{y} \cdot \mathbf{u}_{p}}{{\mathbf{u}_{p}^T} \mathbf{u}_{p}} \mathbf{u}_{p}\\
$$

If the $\{\mathbf{u}_1,...,\mathbf{u}_p\}$ is an orthonormal set, then

\begin{align}
\hat{\mathbf{y}}&=\left(\mathbf{y} \cdot \mathbf{u}_{1}\right) \mathbf{u}_{1}+\left(\mathbf{y} \cdot \mathbf{u}_{2}\right) \mathbf{u}_{2}+\cdots+\left(\mathbf{y} \cdot \mathbf{u}_{p}\right) \mathbf{u}_{p}\\
& = \left(\mathbf{u}_{1}^T\mathbf{y}\right) \mathbf{u}_{1}+\left(\mathbf{u}_{2}^T\mathbf{y}\right) \mathbf{u}_{2}+\cdots+\left(\mathbf{u}_{p}^T\mathbf{y}\right) \mathbf{u}_{p}\\
\end{align}

Chapter 15 - Innear Product and Orthogonality.ipynb


```python
from statsmodels.tsa.seasonal import seasonal_decompose
import matplotlib.pyplot as plt
import scipy.stats as st
import seaborn as sns
import pandas as pd
from scipy.stats import norm, uniform, expon, t, probplot
import scipy.stats as st
from scipy.integrate import quad
from sympy.solvers import solve
from sympy import Symbol
import numpy as np
from pandas import Series, DataFrame
from statsmodels.graphics.tsaplots import plot_acf
from statsmodels.tsa.stattools import acf
from pandas.plotting import lag_plot
from statsmodels.tsa.arima_process import ArmaProcess
```

```python
np.abs(np.roots([0.1, -0.5, 0.5, 1]))
```

    array([3.25642854, 3.25642854, 0.9430113 ])

```python
ar1 = np.array([1, -.5, .5, .1])
simulated_data = ArmaProcess(ar1, ma=[1]).generate_sample(nsample=200)
plt.plot(simulated_data)
plt.xlabel("Zeit")
plt.ylabel("Wert")
plt.title("AR(3) Prozess")
plt.show()
```

```python
lag = 25
acf_theor =ArmaProcess(ar = [1, -.5, .5, .1], ma=[1]).acf(lag)
x = np.arange(lag)
plt.bar(x, acf_theor , width=.2, color="black")
plt.plot(x, acf_theor, "ro", markerfacecolor="C1")
plt.xlabel("Index")
plt.ylabel("ACF")
plt.title("ACF eines AR(3) Prozesses")
plt.show()
```

```python

```


Complex Laplacian and its eigenmodes are parameterized by $\alpha$ and $k$.
---

Theory and math behind the eigenmodes:

The simplest possible dynamic behavior of a damped system is the first order differential equation with one term, and it's rate of exponential decay is governed by a rate contant $\beta$:

\begin{equation}
\label{dampedSys}
  \frac{dx_{1}(t)}{dt} = -\beta x_{1}(t)
\end{equation}

Where $x_{1}(t)$ is the average neuronal activation signal between all neurons within a region of interest. Thus we can interpret the above equation as the refractory period after neural discharge. But when viewing the brain as a network of interconnected regions, we want to introduce activities originating from other regions:

\begin{equation}
  \frac{dx_{i}(t)}{dt} = -\beta (x_{i}(t) - \frac{\alpha}{\pmb{deg_i}} \sum_{i,j} c_{i,j} x_{j}(t-\tau^{\nu}_{i,j}))
\end{equation}

The above equation introduces a connectivity $c_{i,j}$ term, which is normalized by a diagonal degree matrix and scaled by a coupling term $\alpha$. $\alpha$ acts as both a coupling constant as well as a parameter to distinguish the rate of diffusion from connected regions from $\beta$. By introducing connectivity, we also have to take into account the distance between each connected regions, therefore the term $\tau^{\nu}_{i,j}$ is introduced as delay, and is computed by dividing fiber tract distance between regions and signal transmission velocity. Now if we transform the above equation into the Fourier domain, we obtain the follow complex expression:

\begin{equation}
\begin{aligned}
    j\omega X(\omega)_{i} = -\beta X(\omega)_{i} + \frac{\alpha}{\pmb{deg_i}} \sum_j c_{i,j} e^{-j\omega \tau^{\nu}_{i,j}} X(\omega)\\
    j\omega \bar{X}(\omega) = -\beta (I - \alpha \Delta^{-\frac{1}{2}} C^{*}(\omega)) \bar{X}(\omega)\\
    j\omega \bar{X}(\omega) = -B\mathcal{L}\bar{X}(\omega)\\
\end{aligned}
\end{equation}

Here, we introduced a complex component to our structural connectivity term as delays become phases in the Fourier domain, specifically, $x(t-\tau^{\nu}_{i,j}) \to e^{-j\omega \tau^{\nu}_{i,j}} X(\omega)$, thus we can define a complex connectivity as a function of angular frequency $\omega$ as $C(\omega) = \frac{1}{\pmb{deg}}C^{*}(\omega)$, where $C^{*}(\omega) = c_{i,j}e^{-j\omega \tau^{\nu}_{i,j}}$. By redefining the connectivity term from above, the complex Laplacian $\mathcal{L}(\omega)$ is then defined as $\mathcal{L}(\omega) = I - \alpha C(\omega)$. Next we decompose the complex Laplacian matrix $\mathcal{L}$ into it's eigen modes and eigen values:

\begin{equation}
    \mathcal{L}(\omega) = \pmb(U)(\omega)\pmb{\Lambda}(\omega)\pmb{U}(\omega)^H
\end{equation}

Where $\pmb{\Lambda}(\omega) = diag([\lambda_1(\omega), ... , \lambda_N(\omega)])$ is a diagonal matrix consisting of the eigen values of the complex Laplacian matrix at angular frequency $\omega$, and $\pmb{U}(\omega)$ are the eigen modes of the complex Laplacian matrix at angular frequency $\omega$. We are going to see how these eigenmodes behave in their parameter space:

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# spectrome imports
from spectrome.brain import Brain
from spectrome.utils import functions, path
from spectrome.forward import eigenmode, runforward
```

```python
# Some house keeping:
data_dir = "../data"

# Define frequency range of interest
fmin = 2  # 2Hz - 45Hz signal range, filter for this with hbp
fmax = 45
fvec = np.linspace(fmin, fmax, 40)

# Load Pablo's Yeo 2017 canonical network maps
fc_dk = np.load("../data/com_dk.npy", allow_pickle=True).item()
fc_dk_normalized = pd.read_csv("../data/DK_dictionary_normalized.csv").set_index(
    "Unnamed: 0"
)

# Define variables for analysis:
alpha_vec = np.linspace(
    0.5, 4.5, 17
)  # np.linspace(0.5,5,10) # coupling strength values we are going to explore
k_vec = np.linspace(0, 100, 11)  # wave numbers we are going to explore
num_fc = 7  # 7 canonical networks
num_emode = 86  # number of eigenmodes, we are using 86 region DK atlas

default_k = 20  # default wave number
default_alpha = 0.1  # default alpha

# define list of canonical network names and re-order the dictionary using these names:
fc_names = [
    "Limbic",
    "Default",
    "Visual",
    "Fronto \n parietal",
    "Somato \n motor",
    "Dorsal \n Attention",
    "Ventral \n Attention",
]
fc_dk_normalized = fc_dk_normalized.reindex(
    [
        "Limbic",
        "Default",
        "Visual",
        "Frontoparietal",
        "Somatomotor",
        "Dorsal_Attention",
        "Ventral_Attention",
    ]
).fillna(0)

# turbo color map
turbo = functions.create_turbo_colormap()
```

#### Explore varying coupling strength while keeping wave number default first

```python
## Create Brain object from spectrome
alpha_brain = Brain.Brain()
alpha_brain.add_connectome(data_dir)
alpha_brain.reorder_connectome(alpha_brain.connectome, alpha_brain.distance_matrix)
alpha_brain.bi_symmetric_c()
alpha_brain.reduce_extreme_dir()

## Compute correlation values:
alpha_corr = np.zeros((num_emode, num_fc, len(alpha_vec)))
for a_ind in np.arange(0, len(alpha_vec)):
    alpha_brain.decompose_complex_laplacian(alpha=alpha_vec[a_ind], k=default_k)
    alpha_corr[:, :, a_ind] = eigenmode.get_correlation_df(
        alpha_brain.norm_eigenmodes, fc_dk_normalized, method="spearman"
    )
```

```python
## set-up some visualization details
dynamic_range = [0.35, 0.65]  # for colormap
# for coupling strength labels and number of ticks on x-axis:
alpha_labels = np.linspace(0.5, 4.5, 3)
n_ticks = 3

## PLOT
with plt.style.context("seaborn-paper"):
    alpha_corr_fig, alpha_ax = plt.subplots(1, 7, figsize=(7.0, 4.0), sharey=True)
    for i, ax in enumerate(alpha_corr_fig.axes):
        im = ax.imshow(alpha_corr[:, i, :], vmin=0, vmax=1, cmap=turbo, aspect="auto")
        ax.xaxis.set_major_locator(
            plt.LinearLocator(numticks=n_ticks)
        )  # LinearLocator(numticks = n_ticks)
        ax.set_yticklabels([0, 1, 10, 20, 30, 40, 50, 60, 70, 80])
        ax.xaxis.tick_top()
        ax.set_xticklabels(alpha_labels, linespacing=0.2)
        im.set_clim(dynamic_range)

    plt.suptitle("Coupling Strength", fontsize=12, fontweight="bold", y=1.025)

    #cbar_ax = alpha_corr_fig.add_axes([1, 0.15, 0.03, 0.7])
    #cb = alpha_corr_fig.colorbar(im, cax=cbar_ax, extend="both")
    #alpha_corr_fig.add_subplot(1, 1, 1, frameon=False)
    #plt.tick_params(labelcolor="none", which="both", axis = "both", top="off", bottom="off", left="off", right="off")
    #plt.grid(False)
    #plt.ylabel("Eigenmode Number", fontsize=12)

    alpha_corr_fig.text(-0.008, 0.25,  'Eigenmode Number', rotation = 'vertical', fontsize = 12)
    plt.tight_layout(w_pad = 0.30)
    plt.savefig("../figures/fig4/coupling_strength.png", dpi=300, bbox_inches="tight")
```

#### Set to default $\alpha$ and explore wave number now:

```python
## Brain object with spectrome
k_brain = Brain.Brain()
k_brain.add_connectome(data_dir)
k_brain.reorder_connectome(k_brain.connectome, k_brain.distance_matrix)
k_brain.bi_symmetric_c()
k_brain.reduce_extreme_dir()

# preallocate empty correlation df
k_corr = np.zeros((num_emode, num_fc, len(k_vec)))

## Compute correlations
for k in np.arange(0, len(k_vec)):
    k_brain.decompose_complex_laplacian(alpha=default_alpha, k=k_vec[k], num_ev=86)
    k_corr[:, :, k] = eigenmode.get_correlation_df(
        k_brain.norm_eigenmodes, fc_dk_normalized, method="spearman"
    )
```

```python
n_ticks = 3
k_labels = [0, 50, 100]

## PLOT
with plt.style.context("seaborn-paper"):
    k_corr_fig, k_ax = plt.subplots(1, 7, figsize=(6.25, 4.0), sharey=True)
    for i, ax in enumerate(k_corr_fig.axes):
        im = ax.imshow(k_corr[:, i, :], vmin=0, vmax=1, cmap=turbo, aspect="auto")
        ax.xaxis.set_major_locator(
            plt.LinearLocator(numticks=n_ticks)
        )  # LinearLocator(numticks = n_ticks)
        ax.set_yticklabels([0, 1, 10, 20, 30, 40, 50, 60, 70, 80])
        ax.xaxis.tick_top()
        ax.set_xticklabels(k_labels)
        im.set_clim(dynamic_range)
        if i < 3:
            ax.set_title(fc_names[i], y=-0.08, fontsize=8, weight="bold")
        else:
            ax.set_title(fc_names[i], y=-0.12, fontsize=8, weight="bold")

    plt.suptitle("Wave Number", fontsize=12, fontweight="bold", y=1.025)

    cbar_ax = k_corr_fig.add_axes([1, 0.15, 0.03, 0.7])
    cb = k_corr_fig.colorbar(im, cax=cbar_ax, extend="both")
    cb.set_label(r'Spatial Similarity to $\Psi_{CFN}$')
    #k_corr_fig.add_subplot(1, 1, 1, frameon=False)
    #plt.tick_params(labelcolor="none", top="off", bottom="off", left="off", right="off")
    #plt.grid(False)
    #plt.ylabel("Eigenmode Number", fontsize=12)
    k_corr_fig.text(-0.008, 0.35, 'Eigenmode Number', rotation = 'vertical', fontsize = 12)

    plt.tight_layout(w_pad = 0.25)
    plt.savefig("../figures/fig4/wave_number.png", dpi=300, bbox_inches="tight")
```

Complex Laplacian Eigenmodes
---

Find the highest spatial corelation values achieved by the best performing eigenmodes for each canonical network:

Compute Spearman correlation values:

```python
# pre-allocate an array for spearman R of best performing eigenmodes:
params_bestr = np.zeros((len(alpha_vec), len(k_vec), num_fc))

# Create brain object from spectrome with HCP connectome:
hcp_brain = Brain.Brain()
hcp_brain.add_connectome(data_dir)
hcp_brain.reorder_connectome(hcp_brain.connectome, hcp_brain.distance_matrix)
hcp_brain.bi_symmetric_c()
hcp_brain.reduce_extreme_dir()

# for each network, scan through alpha and k values, compute all eigenmode's spearman R
# then select the best performing eigenmode's spearman R
for i in np.arange(0, num_fc):
    print('Computing for {} network'.format(fc_dk_normalized.index[i]))
    for a_ind in np.arange(0, len(alpha_vec)):
        for k_ind in np.arange(0, len(k_vec)):
            # get eigenmodes of complex laplacian:
            hcp_brain.decompose_complex_laplacian(alpha = alpha_vec[a_ind], k = k_vec[k_ind])
            # compute spearman correlation
            spearman_eig = eigenmode.get_correlation_df(
                hcp_brain.norm_eigenmodes, fc_dk_normalized.iloc[[i]], method = 'spearman'
            )
            params_bestr[a_ind, k_ind, i] = np.max(spearman_eig.values)
```

Visualize in heatmap:

```python
dynamic_range = [0.30, 0.65]
k_ticks = 11
k_labels = np.linspace(0, 100, 11).astype(int)
a_ticks = 3
a_labels = np.linspace(0.5, 4.5, 3)

with plt.style.context("seaborn-paper"):
    corr_fig, corr_ax = plt.subplots(1,7, figsize = (8,5))
    for i, ax in enumerate(corr_fig.axes):
        im = ax.imshow(np.transpose(params_bestr[:,:,i]), vmin = 0, vmax = 1, cmap = turbo, aspect = 'auto')
        ax.yaxis.set_major_locator(plt.LinearLocator(numticks = k_ticks))
        ax.xaxis.tick_top()
        ax.set_yticklabels(k_labels)
        ax.xaxis.set_major_locator(plt.LinearLocator(numticks = a_ticks))
        ax.set_xticklabels(a_labels)
        im.set_clim(dynamic_range)
        if i < 3:
            ax.set_title(fc_names[i], y=-0.08, fontsize=8, weight="bold")
        else:
            ax.set_title(fc_names[i], y=-0.12, fontsize=8, weight="bold")

    plt.suptitle('Coupling Strength', fontsize = 12, y = 1)
    cbar_ax = corr_fig.add_axes([1, 0.15, 0.03, 0.7])
    cb = corr_fig.colorbar(im, cax=cbar_ax, extend="both")
    corr_fig.add_subplot(1, 1, 1, frameon=False)
    plt.tick_params(labelcolor="none", top="off", bottom="off", left="off", right="off")
    plt.grid(False)
    plt.ylabel('Wave Number', fontsize = 12)

    plt.tight_layout()
    plt.savefig('../figures/fig5/param_bestr.png', dpi = 300, bbox_inches = 'tight')
```

Note - global coupling doesn't affect the best performing eigenmode but may change which eigenmode is the best performing eigenmode as well as the other eigenmodes.

Split the wave number parameter into oscillatory frequency and signal transmission velocity since wave number $k$ is defined as $k = \frac{2 \pi f}{\nu}$. Then perform the same exploratory exercise as above:

```python
# define parameter ranges:
freq_vec = np.linspace(2, 47, 46)
nu_vec = np.linspace(1, 20, 21)

# define plotting visuals
dynamic_range = [0.3, 0.7]
f_ticks = 6
f_labels = np.linspace(2, 47, 6).astype(int)
nu_ticks = 3
nu_labels = np.linspace(0.5, 20, 3).astype(int)

#pre-allocate array for results
k_bestr = np.zeros((len(freq_vec), len(nu_vec), num_fc))

# compute spearman Rs:
for i in np.arange(0, num_fc):
    print('Computing for {} network'.format(fc_dk_normalized.index[i]))
    for f_ind in np.arange(0, len(freq_vec)):
        for v_ind in np.arange(0, len(nu_vec)):
            # get eigenmodes of complex laplacian:
            hcp_brain.decompose_complex_laplacian(alpha = default_alpha, k = None, f = freq_vec[f_ind], speed = nu_vec[v_ind])
            # compute spearman correlation
            spearman_eig = eigenmode.get_correlation_df(
                hcp_brain.norm_eigenmodes, fc_dk_normalized.iloc[[i]], method = 'spearman'
            )
            k_bestr[f_ind, v_ind, i] = np.max(spearman_eig.values)

# Plot as above:
with plt.style.context("seaborn-paper"):
    k_fig, k_ax = plt.subplots(1,7, figsize = (8,4))
    for i, ax in enumerate(k_fig.axes):
        im = ax.imshow(k_bestr[:,:,i], vmin = 0, vmax = 1, cmap = turbo, aspect = 'auto')
        ax.yaxis.set_major_locator(plt.LinearLocator(numticks = f_ticks))
        ax.xaxis.tick_top()
        ax.set_yticklabels(f_labels)
        ax.xaxis.set_major_locator(plt.LinearLocator(numticks = nu_ticks))
        ax.set_xticklabels(nu_labels)
        im.set_clim(dynamic_range)
        if i < 3:
            ax.set_title(fc_names[i], y=-0.08, fontsize=8, weight="bold")
        else:
            ax.set_title(fc_names[i], y=-0.12, fontsize=8, weight="bold")

    plt.suptitle('Transmission Velocity (m/s)', fontsize = 12, y = 1)
    cbar_ax = k_fig.add_axes([1, 0.15, 0.03, 0.7])
    cb = k_fig.colorbar(im, cax=cbar_ax, extend="both")
    k_fig.add_subplot(1, 1, 1, frameon=False)
    plt.tick_params(labelcolor="none", top="off", bottom="off", left="off", right="off")
    plt.grid(False)
    plt.ylabel('Frequency (Hz)', fontsize = 12)

    plt.tight_layout()
    plt.savefig('../figures/fig5/k_bestr.png', dpi = 300, bbox_inches = 'tight')
```


```python
import math
import numpy as np
import matplotlib.pyplot  as plt
from sympy import *
from pylab import mpl

def func(y):
    y = np.float64(y)
    return 1/(1 + y * y)

def draw_pic(x, y):
    fig=plt.figure()
    plt.plot(x, y, label='插值函数')
    plt.plot(x, func(x), label='原函数')
    plt.legend()
    plt.show()

def spline3_Parameters(x_vec):
        # parameter为二维数组，用来存放参数，size_of_Interval是用来存放区间的个数
        x_new = np.array(x_vec)
        parameter = []
        size_of_Interval = len(x_new) - 1;
        i = 1
        # 首先输入方程两边相邻节点处函数值相等的方程为2n-2个方程
        while i < len(x_new) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = x_new[i] * x_new[i] * x_new[i]
            data[(i - 1) * 4 + 1] = x_new[i] * x_new[i]
            data[(i - 1) * 4 + 2] = x_new[i]
            data[(i - 1) * 4 + 3] = 1
            data1 = np.zeros(size_of_Interval * 4)
            data1[i * 4] = x_new[i] * x_new[i] * x_new[i]
            data1[i * 4 + 1] = x_new[i] * x_new[i]
            data1[i * 4 + 2] = x_new[i]
            data1[i * 4 + 3] = 1

            parameter.append(data)
            parameter.append(data1)
            i += 1
        # 输入端点处的函数值。为两个方程, 加上前面的2n - 2个方程，一共2n个方程
        data = np.zeros(size_of_Interval * 4)
        data[0] = x_new[0] * x_new[0] * x_new[0]
        data[1] = x_new[0] * x_new[0]
        data[2] = x_new[0]
        data[3] = 1
        parameter.append(data)

        data = np.zeros(size_of_Interval * 4)
        data[(size_of_Interval - 1) * 4] = x_new[-1] * x_new[-1] * x_new[-1]
        data[(size_of_Interval - 1) * 4 + 1] = x_new[-1] * x_new[-1]
        data[(size_of_Interval - 1) * 4 + 2] = x_new[-1]
        data[(size_of_Interval - 1) * 4 + 3] = 1
        parameter.append(data)
        # 端点函数一阶导数值相等为n-1个方程。加上前面的方程为3n-1个方程。
        i = 1
        while i < size_of_Interval:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 3 * x_new[i] * x_new[i]
            data[(i - 1) * 4 + 1] = 2 * x_new[i]
            data[(i - 1) * 4 + 2] = 1
            data[i * 4] = -3 * x_new[i] * x_new[i]
            data[i * 4 + 1] = -2 * x_new[i]
            data[i * 4 + 2] = -1
            parameter.append(data)
            i += 1
        # 端点函数二阶导数值相等为n-1个方程。加上前面的方程为4n-2个方程。
        i = 1
        while i < len(x_new) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 6 * x_new[i]
            data[(i - 1) * 4 + 1] = 2
            data[i * 4] = -6 * x_new[i]
            data[i * 4 + 1] = -2
            parameter.append(data)
            i += 1
        #端点处的函数值的二阶导数为原函数的二阶导数，为两个方程。总共为4n个方程。
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6 * x_new[0]
        data[1] = 2
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-4] = 6 * x_new[-1]
        data[-3] = 2
        parameter.append(data)
        #df = pd.DataFrame(parameter)
        #df.to_csv('para.csv')
        return parameter

    # 功能：计算样条函数的系数。
    # 参数：parametes为方程的系数，y为要插值函数的因变量。
    # 返回值：三次插值函数的系数。

def solution_of_equation(parametes, x):
        size_of_Interval = len(x) - 1;
        result = np.zeros(size_of_Interval * 4)
        i = 1
        while i < size_of_Interval:
            result[(i - 1) * 2] = func(x[i])
            result[(i - 1) * 2 + 1] = func(x[i])
            i += 1
        result[(size_of_Interval - 1) * 2] = func(x[0])
        result[(size_of_Interval - 1) * 2 + 1] = func(x[-1])
        result[-2] = 5/13
        result[-1] = -5 / 13
        a = np.array(spline3_Parameters(x))
        b = np.array(result)
        #print(b)
        return np.linalg.solve(a, b)

    # 功能：根据所给参数，计算三次函数的函数值：
    # 参数:parameters为二次函数的系数，x为自变量
    # 返回值：为函数的因变量
def calculate(paremeters, x):
        result = []
        for data_x in x:
            result.append(
                paremeters[0] * data_x * data_x * data_x + paremeters[1] * data_x * data_x + paremeters[2] * data_x +
                paremeters[3])
        return result

x_init4 = np.arange(-5, 5.1, 1 )
result = solution_of_equation(spline3_Parameters(x_init4), x_init4)
#print(spline3_Parameters(x_init4))
#print(result)
x_axis4 = []
y_axis4 = []
for i in range(10):
    temp = np.arange(-5 + i, -4 + i, 0.01)
    x_axis4 = np.append(x_axis4, temp)
    y_axis4 = np.append(y_axis4, calculate(
        [result[4 * i], result[1 + 4 * i], result[2 + 4 * i], result[3 + 4 * i]], temp))
draw_pic(x_axis4, y_axis4)
```

```python
import math
import numpy as np
import matplotlib.pyplot  as plt
from sympy import *
from pylab import mpl

def func(y):
    y = np.float64(y)
    return 1/(1 + y * y)

def draw_pic1(x, y):
    fig=plt.figure()
    plt.plot(x, y, label='插值函数')
    plt.plot(x, func(x), label='原函数')
    plt.legend()
    plt.show()

def draw_pic2(x, y):
    fig=plt.figure()
    plt.plot(x, np.fabs(y-func(x)), label='误差')
    plt.legend()
    plt.show()

def spline3_Parameters(x_vec):
        parameter = []
        size_of_Interval = len(x_vec) - 1;
        i = 1

        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 2] = x_vec[i]
            data[(i - 1) * 4 + 3] = 1
            data1 = np.zeros(size_of_Interval * 4)
            data1[i * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data1[i * 4 + 1] = x_vec[i] * x_vec[i]
            data1[i * 4 + 2] = x_vec[i]
            data1[i * 4 + 3] = 1

            parameter.append(data)
            parameter.append(data1)
            i += 1

        data = np.zeros(size_of_Interval * 4)
        data[0] = x_vec[0] * x_vec[0] * x_vec[0]
        data[1] = x_vec[0] * x_vec[0]
        data[2] = x_vec[0]
        data[3] = 1
        parameter.append(data)

        data = np.zeros(size_of_Interval * 4)
        data[(size_of_Interval - 1) * 4] = x_vec[-1] * x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 1] = x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 2] = x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 3] = 1
        parameter.append(data)

        i = 1
        while i < size_of_Interval:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 3 * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = 2 * x_vec[i]
            data[(i - 1) * 4 + 2] = 1
            data[i * 4] = -3 * x_vec[i] * x_vec[i]
            data[i * 4 + 1] = -2 * x_vec[i]
            data[i * 4 + 2] = -1
            parameter.append(data)
            i += 1

        i = 1
        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 6 * x_vec[i]
            data[(i - 1) * 4 + 1] = 2
            data[i * 4] = -6 * x_vec[i]
            data[i * 4 + 1] = -2
            parameter.append(data)
            i += 1
        #the other two equations
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6 * x_vec[0]
        data[1] = 2
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-4] = 6 * x_vec[-1]
        data[-3] = 2
        parameter.append(data)
        return parameter

def solution_of_equation(parametes, x):
        size_of_Interval = len(x) - 1;
        result = np.zeros(size_of_Interval * 4)
        i = 1
        while i < size_of_Interval:
            result[(i - 1) * 2] = func(x[i])
            result[(i - 1) * 2 + 1] = func(x[i])
            i += 1
        result[(size_of_Interval - 1) * 2] = func(x[0])
        result[(size_of_Interval - 1) * 2 + 1] = func(x[-1])
        result[-2] = 5 / 13
        result[-1] = -5 / 13
        a = np.array(spline3_Parameters(x))
        b = np.array(result)
        #print(b)
        return np.linalg.solve(a, b)

def calculate(paremeters, x):
        result = []
        for data_x in x:
            result.append(
                paremeters[0] * data_x * data_x * data_x + paremeters[1] * data_x * data_x + paremeters[2] * data_x +
                paremeters[3])
        return result

x_init4 = np.arange(0, 1.01, 0.05)
result = solution_of_equation(spline3_Parameters(x_init4), x_init4)
#print(spline3_Parameters(x_init4))
#print(result)
x_axis4 = []
y_axis4 = []
for i in range(20):
    temp = np.arange(i/20, 0.05 + i/20, 0.01)
    x_axis4 = np.append(x_axis4, temp)
    y_axis4 = np.append(y_axis4, calculate(
        [result[4 * i], result[1 + 4 * i], result[2 + 4 * i], result[3 + 4 * i]], temp))
draw_pic1(x_axis4, y_axis4)
draw_pic2(x_axis4, y_axis4)
```

```python
        #二次导数和原函数相同
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6 * x_vec[0]
        data[1] = 2
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-4] = 6 * x_vec[-1]
        data[-3] = 2
        parameter.append(data)
```