# PUF model and probability distributions

We use the temperature-dependent model for SRAM-PUF as defined by Roel Maes.
\begin{align*}
r_i^{(j)}=\left\{\begin{matrix}
0 , &\text{if } m_i+d_i\cdot t^{(j)} +w\leq \tau,\\
1 , &\text{if } m_i+d_i \cdot t^{(j)} +w> \tau.
\end{matrix}\right.
\end{align*}
With process variable $m_i \sim \mathcal{N}(\mu_M,\sigma_M^2)$, temperature dependence variable $d_i \sim \mathcal{N}(0,\sigma_D^2)$, noise variable $w_i^{(j)} \sim \mathcal{N}(0,\sigma_N^2)$, and temperature $t^{(j)}$ at time $j$.
Roel also defines some parameters $\lambda_1 = \sigma_N/\sigma_M$, $\lambda_2 = (\tau-\mu_M)/\sigma_M$, and $\theta= \sigma_N / \sigma_D $.

We refer to the one-probability of cell $i$ as $\xi_i$ and in case of known state-variables, this is given as $\xi_i = \Pr(X=1) = 1- cdf_N(\tau - m_i-d_i \cdot t^{(j)} )=\Phi(\frac{-\tau + m_i + d_i \cdot t^{(j)} }{\sigma_N}) $.

It is our goal to estimate $\lambda_1,\lambda_2,\theta$ for our dataset.

We will use maximum-likelihood estimation to achieve this.

## Settings of parameters

```python
import numpy as np
from scipy.stats import norm
import matplotlib.pyplot as plt
import srampufparest
# set the parameters
lambda12 = [0.1,0.004]
theta = 45;
```

# verify srampufparest.loadUnique()
Does it give the same histogram as the code that I used before ?

```python
Nsamples = 60
Ndevices = 96
datapath = r'C:\Users\lkusters\surfdrive\python\server_008104\20181125\Unique'
folder = datapath+'\\002_Temp_025'
_,merged1 = srampufparest.loadUnique(folder)
import os
data25 = [[] for i in range(Nsamples)]
for ii in range(Ndevices):
    count = 0;
    for fname in os.listdir(folder):
        if fname.startswith("Unique_dev%03d"%(ii+1) ):
            data =  np.fromfile(folder+'\\'+fname, np.uint8)
            data_bits = np.unpackbits(data)
            data25[count].extend(data_bits)
            count += 1;
data25 = np.swapaxes(data25,0,1) # swap axes, s.t. 1st dim is cells, and second is observations
```

    Finished loading data for 96 devices, with 60 observations of 65536 cells
    returning observations [96,60,65536] and merged observations [6291456,60]

```python
# from srampufparest
from numpy.random import uniform
print(len(merged1))
N25 = [len(d) for d in merged1] # number of measurements per cell
sums25 = [sum(n) for n in merged1 ]
print(set(N25))
# now generate errorcount
errorcount25 = [sum25 if uniform(0,1,1)>(sum25/Nsamples) else Nsamples-sum25 for sum25 in sums25]
```

    6291456
    {60}

```python
# original method
from numpy.random import uniform
print(len(data25))
N25 = [len(d) for d in data25] # number of measurements per cell
sums25 = [sum(n) for n in data25 ]
print(set(N25))
# now generate errorcount
errorcount25 = [sum25 if uniform(0,1,1)>(sum25/Nsamples) else Nsamples-sum25 for sum25 in sums25]
```

    6291456
    {60}

```python
# ORIGINAL
# plot the histogram of the number of observed ones in each cell
from scipy.special import comb
pmfdata_sum25,edges = np.histogram(sums25,bins=Nsamples+1,range=(-.5,Nsamples+.5))
plt.plot(range(Nsamples+1),[count/len(data25) for count in pmfdata_sum25] ,'o', fillstyle='none')
plt.yscale('log')
plt.xlabel('k Number of ones for total %d measurements'%(Nsamples))
plt.ylabel('#cells with Number of ones = k ')
plt.title('$T=25^oC$')

# Functions srampuf
pmfhist,centers = srampufparest.getcounts1D(merged1)
plt.plot(centers,[count/len(merged1) for count in pmfhist] )

plt.show()
```

```python
print(centers)
```

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60]

# Verify fitting method

# Pdf of one-probability (no temperature)

We find that apparently the approximations by python (in integrator?) are not accurate enough, since the pdf does not sum to 1 (but instead 0.38). Therefore, we instead use the equations to approximate the cdf and then calculate the pdf by integrating (sum over small intervals).

## Pdf of the one-probability
\begin{equation}
        p_0(\xi|\lambda_1,\lambda_2)
        = \frac{\lambda_1 \cdot \phi( \Phi^{-1}(\xi) \cdot \lambda_1  +\lambda_2) }{\phi(\Phi^{-1}(\xi))}
\end{equation}

First, we show that indeed the total probability does not sum to 1:

## Cdf of the one-probability

The PDF can be approximated as
\begin{equation}
        p_0(\xi|\lambda_1,\lambda_2)
        = \Pr(\Xi \leq \xi + \Delta\xi) - \Pr(\Xi \leq \xi - \Delta\xi) ,
\end{equation}
with
\begin{equation}
        \Pr(\Xi \leq \xi)
        = \Phi( \Phi^{-1}(\xi) \cdot \lambda_1  +\lambda_2)
\end{equation}

Second show that we can use cdf and integrate to improve the approximation

```python
# First plot the pdf
l1 = lambda12[0]
l2 = lambda12[1]
NX = 1000 # number of steps

xx = [i for i in np.linspace(0,1,NX)]
yy= [norm.cdf(l1*norm.ppf(xi)+l2) for xi in xx]
pdf_one = np.diff(yy)
pdf_x = [x+0.5/NX for x in xx[:-1]]
plt.plot(pdf_x,pdf_one,'o', fillstyle='none')

NX = 1000 # number of steps

p0_p, p0_xi = srampufparest.pdfp0(l1,l2,NX)

plt.plot(p0_xi,p0_p)
plt.yscale('log')
plt.xlabel('Pr(X=1)=xi')
plt.ylabel('Pr(Xi=xi)')
plt.show()
pred = sum(p0_p) # should be 1
print(pred, 'should be equal to ONE ')
```

```python
# Then verify that we can reproduce the histogram of the one counts
# plot the histogram of the number of observed ones in each cell
from scipy.special import comb
pmfhist,centers = srampufparest.getcounts1D(merged1)
plt.plot(centers,[count/len(merged1) for count in pmfhist] ,'o', fillstyle='none')
plt.yscale('log')
plt.xlabel('k Number of ones for total %d measurements'%(Nsamples))
plt.ylabel('#cells with Number of ones = k ')
plt.title('$T=25^oC$')

# Now the fit
l1 = lambda12[0]
l2 = lambda12[1]
NX = 10000 # number of steps

p0_p, p0_xi = srampufparest.pdfp0(l1,l2,NX)

pmf_ones = [0]*(Nsamples+1)
for nones in range(Nsamples+1): # loop over number of ones
    pred = [(xi**nones)*((1-xi)**(Nsamples-nones))*pdfp for (xi,pdfp) in zip(p0_xi,p0_p) ]
    pmf_ones[nones] = comb(Nsamples,nones)*sum(pred)

plt.plot(range(Nsamples+1),pmf_ones)

plt.show()
pred = sum(p0_p) # should be 1
print(pred, 'should be equal to ONE ')
```

```python
# Now the temperature dependent behavior
# settings from Roel: 0.12,0.02,45
l1 = 0.12
l2 = 0.02
theta = 45
NX = 1000 # number of steps

idx = srampufparest.getcellskones(merged2,5); # select cells that observed 5 ones in T2
merged1_sel = [merged1[i] for i in idx] # find observations of these cells at T1
hist,centerpoints = srampufparest.getcounts1D(merged1_sel); # generate the histogram

p1 = 5/60 # one probability
L = 40 # number of observations at T1
dT = 65 # temperature difference between T1 and T2
predict,x = srampufparest.pmfpkones_temperature(NX,dT,l1,l2,theta,p1,L); # predict the histogram

plt.plot(x,predict)
plt.plot(centerpoints,hist,'o', fillstyle='none') # actual values
plt.yscale('log')
plt.xlabel('k Number of ones for total %d measurements'%(Nsamples))
plt.ylabel('#cells with Number of ones = k ')

```

    [0.31010513237186754, 0.10058079065038165, 0.021414246053697624, 0.007509174365105543, 0.005113879303033711, 0.004256694020444828, 0.003723684383966019, 0.0033391881623195127, 0.0030468769351748547, 0.002817159259993022, 0.0026321161129814253, 0.00248018010332088, 0.0023535354726852997, 0.00224670598903209, 0.002155738815462368, 0.0020777093525364064, 0.0020104084943440115, 0.0019521382792435193, 0.001901574438489749, 0.0018576716049877391, 0.0018195965162510464, 0.0017866800605659812, 0.0017583823008481784, 0.0017342666268835474, 0.0017139804573404471, 0.0016972407335269588, 0.0016838229890183372, 0.001673553145528366, 0.0016663014385726656, 0.00166197805605192, 0.0016605302043487897, 0.0016619404168654263, 0.0016662260009514534, 0.0016734395896692502, 0.0016836708320020091, 0.0016970493256213913, 0.0017137489773863103, 0.0017339940771341318, 0.0017580675018859808, 0.0017863216473234089, 0.001819192936825101, 0.001857221125029318, 0.0019010751557642824, 0.0019515881560259484, 0.002009805420579908, 0.0020770512619482056, 0.002155023893422279, 0.0022459330418780135, 0.0023527045873945815, 0.002479293844995779, 0.002631181756850738, 0.002816193112185558, 0.0030459131712021817, 0.003338299040791898, 0.0037230470696340955, 0.004257062598871042, 0.005121381785408889, 0.007573918461233495, 0.021774823635995335, 0.10113958971143032, 0.30592430516568947]

# Pdf of one-probability (with temperature)

## Pdf of the one-probability
\begin{equation}
        p_1(\xi_2|\xi_1,t_1,t_2,\lambda_1,\lambda_2,\theta)
        = \phi \left(\frac{\theta}{t_2-t_1} \cdot (\Phi^{-1}(\xi_{t_2})-\Phi^{-1}(\xi_{t_1})) \right)   \cdot \frac{\theta}{t_2-t_1}  \cdot  \frac{1 }{\phi(\Phi^{-1}(\xi_{t_2} )  ) }
\end{equation}

First, we show that indeed the total probability does not sum to 1:

## Cdf of the one-probability

The PDF can be approximated as
\begin{equation}
        p_1(\xi_2|\xi_1,t_1,t_2,\lambda_1,\lambda_2,\theta)
        = \Pr(\Xi_{t_2} \leq \xi_{t_2}+ \Delta\xi|\Xi_{t_1}=\xi_{t_1})  - \Pr(\Xi_{t_2} \leq \xi_{t_2}- \Delta\xi|\Xi_{t_1}=\xi_{t_1}) ,
\end{equation}
with
\begin{equation}
        \Pr(\Xi_{t_2} \leq \xi_{t_2}|\Xi_{t_1}=\xi_{t_1})
        = \Phi \left(\frac{\theta}{t_2-t_1} \cdot (\Phi^{-1}(\xi_{t_2})-\Phi^{-1}(\xi_{t_1})) \right)
\end{equation}

We show that we can use cdf and integrate to improve the approximation

```python
# use the CDF
l1 = lambda12[0]
l2 = lambda12[1]
th = theta
NX = 100 # number of steps

dT = 25 # temperature difference

# first we choose single value for pi
pi = 0.1 # xit1
xx = [i for i in np.linspace(0,1,NX)]
yy = [norm.cdf( (th/dT) * (norm.ppf(xi)-norm.ppf(pi)) )   for xi in xx]
p1_p = np.diff(yy)
p1_xi = [x+0.5/NX for x in xx[:-1]]
plt.plot(p1_xi,p1_p)
plt.yscale('log')
plt.xlabel('Pr(X=1)=xi')
plt.ylabel('Pr(Xi=xi|pi=%0.2f,dT=%d)'%(pi,dT))
plt.show()
pred = sum(p1_p)
xx[0]

print(pred, 'should be equal to ONE ')

# second we integrate over all possible pi, to get the average distribution
# we re-use p0_p and p0_xi
ytotal = [ sum( [ Pdfpi*norm.cdf( (th/dT) * (norm.ppf(xi)-norm.ppf(pi)) )  for (Pdfpi,pi) in zip(p0_p,p0_xi) ])   for xi in xx]
ytotal = np.diff(ytotal)
ytotalx = [x+0.5/NX for x in xx[:-1]]

plt.plot(ytotalx,ytotal)
plt.yscale('log')
plt.xlabel('Pr(X=1)=xi')
plt.ylabel('Pr(Xi=xi|dT=%d)'%dT)
plt.show()
pred = sum(ytotal)

print(pred, 'should be equal to ONE ')
```

```python
# use the CDF
l1 = lambda12[0]
l2 = lambda12[1]
th = theta
NX = 100 # number of steps

dT = 25 # temperature difference

# first we choose single value for pi
pi = 0.1 # xit1
p1_p, p1_xi = srampufparest.pdfp1(pi,dT,l1,l2,th,NX)
plt.plot(p1_xi,p1_p)
plt.yscale('log')
plt.xlabel('Pr(X=1)=xi')
plt.ylabel('Pr(Xi=xi|pi=%0.2f,dT=%d)'%(pi,dT))
plt.show()
pred = sum(p1_p)
xx[0]

print(pred, 'should be equal to ONE ')

```

```python
# plot the histogram of the number of observed ones in each cell
pmfdata_sum25,edges = np.histogram(sums25,bins=Nsamples+1,range=(-.5,Nsamples+.5))
plt.plot(range(Nsamples+1),[count/len(data25) for count in pmfdata_sum25] ,'o', fillstyle='none')
plt.yscale('log')
plt.xlabel('k Number of ones for total %d measurements'%(Nsamples))
plt.ylabel('#cells with Number of ones = k ')
plt.title('$T=25^oC$')

# now also plot the fit
pmf_est = calculatePmf_kones_noT(lambda12,Nsamples)
plt.plot(range(Nsamples+1),pmf_est)
fig1 = plt.gcf()
plt.show()
fig1.savefig('UNIQUE_kones_pdf.jpg', bbox_inches='tight')
```

# Build the grid for ML estimation

Now the probability that for any cell we observe $k_1$ ones at temperature $t_1$ and $k_2$ ones at temperature $t_2$, is
\begin{equation}
    \Pr_{k_1,k_2}(k_1,k_2 |t_1,t_2,\lambda_1,\lambda_2,\theta)
     ={K_1 \choose k_1} {K_2 \choose k_2}  \int_0^1  \xi_1^{k_1} (1-\xi_1)^{K_1 -k_1} p_0(\xi_1|\lambda_1,\lambda_2)   \int_0^1 \xi_2^{k_2} (1-\xi_2)^{K_2 -k_2} p_1(\xi_2|\xi_1,t_1,t_2 ,\lambda_1,\lambda_2,\theta) d \xi_2  d \xi_1 .
\end{equation}

```python

```

```python
from scipy.special import comb
# preparations
NX = 100
K=60 # number of observations
l1 = 0.1213;
l2 = 0.0210;
th = 45;
dT = 25 # difference between temperatures

xx = [i for i in np.linspace(0,1,NX)]
yy= [norm.cdf(l1*norm.ppf(xi)+l2) for xi in xx]
p0_p = np.diff(yy)
p0_xi = [x+0.5/NX for x in xx[:-1]]
```

```python
from scipy.special import comb
import sys
K=60

for th in [45,40,50,35,55]:
    print('Likelihood Pr_{k_1,k_2}(k_1,k_2 |dT,l1,l2,l2), with l1=%0.5f,l2=%0.5f,dT=%d,theta=%d'%(l1, l2, dT, th))

    Pk1k2 = [[0]*(K+1) for k in range(K+1)]
    for k1 in range(K+1):
        comb1 = comb(K,k1)
        for k2 in range(K+1):
            sys.stdout.write("\rCalculating k1=%2.0d,k2=%2.0d" % (k1,k2) )
            total = 0
            for (xi1,pdfxi1) in zip(p0_xi,p0_p):
                yy = [norm.cdf( (th/dT) * (norm.ppf(xi)-norm.ppf(xi1)) )   for xi in xx]
                p1_p = np.diff(yy)
                p1_xi = [x+0.5/NX for x in xx[:-1]]
                int2 = sum([(xi2**k2)*((1-xi2)**(K-k2))*pdfxi2 for (xi2,pdfxi2) in zip(p1_xi,p1_p)])
                total = total+ pdfxi1*int2*(xi1**k1)*((1-xi1)**(K-k1))
            Pk1k2[k1][k2] = comb1*comb(K,k2)*total
    np.savetxt('Pk1k1_(%d).txt.gz' %th , X, delimiter=' ', newline='\n', header='Likelihood Pr_{k_1,k_2}(k_1,k_2 |dT,l1,l2,l2), with l1=%0.5f,l2=%0.5f,dT=%d,theta=%d'%(l1, l2, dT, th), comments='# ')
```

    Likelihood Pr_{k_1,k_2}(k_1,k_2 |dT,l1,l2,l2), with l1=0.12130,l2=0.02100,dT=25,theta=45
    Calculating k1= 1,k2=50

```python
np.savetxt('Pk1k1_(%d).txt.gz' %th , X, delimiter=' ', newline='\n', header='Likelihood Pr_{k_1,k_2}(k_1,k_2 |dT,l1,l2,l2), with l1=%0.5f,l2=%0.5f,dT=%d,theta=%d'%(l1, l2, dT, th), comments='# ')
```

# Appendix : We can do the same for the pdf of Pe
Again we show that the pdf (approximate by python) does not sum to 1

```python
# can we also plot the pdf of Pe ?
lambda12 = [0.13,0.02]
l1 = lambda12[0]
l2 = lambda12[1]
NX = 1000 # number of steps

xx = [i for i in np.linspace(0,1,NX)]
y = [l1*(1-xi)*(norm.pdf(l1*norm.ppf(xi)+l2)+norm.pdf(l1*norm.ppf(xi)-l2))/norm.pdf(norm.ppf(xi)) for xi in xx]
plt.plot(xx,y)
plt.yscale('log')
plt.show()
pred = sum(y[1:-1])/NX
print(pred) # is not even close to 1 !
# this shows us that we are not accurate enough!
```

Again show that we can use cdf and integrate to improve the approximation

```python
# what if we use cdf instead?
lambda12 = [0.12,0.02]
l1 = lambda12[0]
l2 = lambda12[1]
NX = 100 # number of steps

xx = [i for i in np.linspace(0,1,NX)]
upperlim = [norm.ppf(i) for i in xx] # these are the upper limits of the integral I want to calculate
plt.plot(xx,upperlim)
plt.yscale('log');plt.xlabel('x');plt.ylabel(r'$\Phi^{-1}(x)$')
plt.title('Upper limit of integral')
plt.show()

# Now we generate the part inside the integral
Nu = 500000;
uu = np.linspace(-295,45,Nu) # outside of this range it is 0 anyway
y = [norm.cdf(-u)*(norm.pdf(l1*u+l2)+norm.pdf(l1*u-l2)) for u in uu];
plt.plot(uu,y)
#print(y[90:-355])
plt.yscale('log');
plt.xlabel('u');plt.ylabel(r'$\Phi(-u)\cdot(\phi(\dots)+\dots)$')
plt.title('Values of function inside integral')
plt.show()

# now I wil calculate the values
idxs = [np.searchsorted(uu, upper, side="left") for upper in upperlim];
cdfPe = [(340/Nu)*sum(y[:idx])*l1 for idx in idxs]
plt.plot(xx,cdfPe)
plt.yscale('log');plt.xlabel('x');plt.ylabel(r'$\lambda_1 \int_{-\infty}^{\Phi^{-1}(x)}\dots du$')
plt.title('$cdf_{Pe}$')
plt.show()

# ok! now I have cdf. Can I make pdf ?
pdfPe = np.diff(cdfPe)
plt.plot([x+0.5/NX for x in xx[:-1]],pdfPe)
plt.yscale('log');plt.xlabel('x');plt.ylabel('$\Delta cdf_{Pe}$')
plt.title('$pdf_{Pe}$')
plt.show()

print(sum(pdfPe))
```

```python

```


# 6. Force and the added mass

## Force

This part introduces the calculation of the force and moment on the body through the negative rate of change of impulse in the fluid. The continuous expressions for linear and angular impulse (about the origin) are, in two dimensions,

$\begin{align}
    \boldsymbol{P} &= \int_{V_f} \boldsymbol{x}\times \boldsymbol{\omega}\,\mathrm{d}V + \int_{S_b} \boldsymbol{x}\times \left( \boldsymbol{n}\times\boldsymbol{v}\right)\,\mathrm{d}S \\
    \boldsymbol{\Pi}_0 &= \frac{1}{2}\int_{V_f} \boldsymbol{x}\times \left(\boldsymbol{x}\times\boldsymbol{\omega}\right)\,\mathrm{d}V + \frac{1}{2} \int_{S_b} \boldsymbol{x}\times \left[ \boldsymbol{x}\times\left( \boldsymbol{n} \times \boldsymbol{v} \right) \right] \,\mathrm{d}S
\end{align}$

If there is only a single body, then the force and moment (about the origin) exerted by the fluid on that body are given by

$\boldsymbol{F} = -\rho \frac{\mathrm{d}\boldsymbol{P}}{\mathrm{d}t}, \qquad \boldsymbol{M}_{0} = -\rho \frac{\mathrm{d}\boldsymbol{\Pi}_{0}}{\mathrm{d}t},$

where $\rho$ is the fluid density. In the two-dimensional applications of this package, angular impulse and the moment have only a single component, e.g., $\boldsymbol{\Pi}_0 =\Pi_0\boldsymbol{e}_z$, where $\boldsymbol{e}_z$ is the unit vector out of the plane.

#### Point vortices past a cylinder

To verify our method of calculating the impulse, we create a model of two point vortices of equal and opposite circulation positioned at both sides of the $x$-axis, with the $x$-axis as axis of symmetry, and left of a circular cylinder positioned at the origin. If the top vortex has a positive circulation, the vortices propel each other towards the cylinder and convect past it. This configuration has an analytical solution for the trajecory and impulse.

We use a rectangular grid.

```julia
using GridPotentialFlow
using Plots
Lx = 10.0
Ly = 4.0
xlim = (-Lx/2,Lx/2)
ylim = (-Ly/2,Ly/2)
g = PhysicalGrid(xlim, ylim, 0.04)
```

    PhysicalGrid{2}((252, 104), (126, 52), 0.04, ((-5.0, 5.0), (-2.04, 2.04)))

```julia
Rc = 1
circle = Circle(Rc,2*cellsize(g))
pfb = PotentialFlowBody(circle)

Δs = dlength(circle);
```

The initial spacing between the vortices is `y∞`. After the vortices pass the cylinder, they should return to this spacing.

```julia
y∞ = Rc/2
v1 = Vortex(-Lx/2+10*cellsize(g),y∞,1.0);
v2 = Vortex(-Lx/2+10*cellsize(g),-y∞,-1.0);
```

Analytical trajectory

```julia
x(r) = sqrt(r^2-(y∞^2*(r^2-1)^2)/((r^2-1)^2-4*y∞^2))
y(r) = y∞*(r^2-1)/sqrt((r^2-1)^2-4*y∞^2)
s = 0:0.001:π/2
rmin = 1.4375649
c1 = rmin/(-Lx/2+rmin)
c2 = -Lx/2*c1
r = c2 ./ (cos.(s).-c1)
x_trajectory = -reverse(x.(r)); append!(x_trajectory,x.(r))
y_trajectory_upper = reverse(y.(r)); append!(y_trajectory_upper,y.(r))
y_trajectory_lower = -y_trajectory_upper;
```

Analytical impulse

```julia
Px_func(x,y) = y - y/(x^2+y^2) + y - y/(x^2+y^2);
Py_func(x,y) = -x + x/(x^2+y^2) + x - x/(x^2+y^2);
Px_exact = Px_func.(x_trajectory,y_trajectory_upper);
Py_exact = Py_func.(x_trajectory,y_trajectory_upper);
```

We create the vortex model with these two point vortices and the circle and advance the position of the point vortices over some time. We use `OrdinaryDiffEq.RK4()` for the time stepping and we compute the impulse history with `impulse` in a post-processing step.

```julia
model = VortexModel(g,bodies=[pfb],vortices=[v1,v2]);

function rhs(X,model,t)
    setvortexpositions!(model,X)
    Ẋ = vortexvelocities!(model)
    return Ẋ
end

import OrdinaryDiffEq
X = getvortexpositions(model)
prob = OrdinaryDiffEq.ODEProblem(rhs,X,(0.0,64.0),model);
sol = OrdinaryDiffEq.solve(prob,dt=0.1,OrdinaryDiffEq.RK4(),dense=false,adaptive=false);
```

Using the history of the vortex positions in `sol.u`, we can calculate the impulse history.

```julia
Px_numerical_hist = []
Py_numerical_hist = []
for u in sol.u
    setvortexpositions!(model,u)
    Px, Py = impulse(model)
    push!(Px_numerical_hist,Px)
    push!(Py_numerical_hist,Py)
end
```

When we compare the trajectories and impulse history, the numerical and analytical solution should match closely, which is indeed the case.

```julia
plot(circle,fillcolor=:black,fillrange=0,fillalpha=0.25,linecolor=:black,linewidth=2,xlabel="x",ylabel="y")
scatter!(model.vortices.x,model.vortices.y,color=:red)
plot!(x_trajectory,y_trajectory_upper,linecolor=:red,label="exact")
plot!(x_trajectory,y_trajectory_lower,linecolor=:red,label="")
plot!(map(s->s.u[1],sol.u),map(s->s.v[1],sol.u),color=:blue,linestyle=:dash,label="simulated")
plot!(map(s->s.u[2],sol.u),map(s->s.v[2],sol.u),color=:blue,linestyle=:dash,label="")
```

```julia
plot(x_trajectory,Px_exact,color=:red,label="exact",xlabel="x",ylabel="Px")
plot!(map(s->s.u[1],sol.u),Px_numerical_hist,color=:blue,linestyle=:dash,label="simulated")
```

#### Flat plate

We can apply this method also to our flat plate example and compare it to the Biot-Savart method from the `PotentialFlow.jl` package.

Because we create the `PotentialFlow.jl` model with a moving plate instead of a uniform flow, we specify a translational velocity. The `GridPotentialFlow.jl` model will still use a moving body coordinate system and thus a uniform flow that equals the negative translational velocity.

```julia
c = 1.0;   # chord length

ċ = -1.0;  # translational velocity
α = -π/3;   # angle of attack

σLE = 0.0;
σTE = 0.0;

Δt = 5e-2;
tf = 1.0;
T = 0.0:Δt:tf;
```

Potential Flow

```julia
using PotentialFlow
function compute_ẋ!(ẋ, x, t)
    plate, ambient_sys = x
    motion = ẋ[1]
    motion.ċ, motion.c̈, motion.α̇ = motion.kin(t)
    Plates.enforce_no_flow_through!(plate, motion, ambient_sys, t)
    reset_velocity!(ẋ, x)
    self_induce_velocity!(ẋ, x, t)
end
function shed_new_vorticity!(blobs, plate, motion, t, lesp = 0.0, tesp = 0.0)
    z₊ = (blobs[end-1].z + 2plate.zs[end])/3
    z₋ = (blobs[end].z + 2plate.zs[1])/3
    blob₊ = PotentialFlow.Vortex.Blob(z₊, 1.0, δ)
    blob₋ = PotentialFlow.Vortex.Blob(z₋, 1.0, δ)
    Plates.enforce_no_flow_through!(plate, motion, blobs, t)
    Γ₊, Γ₋, _, _ = Plates.vorticity_flux!(plate, blob₊, blob₋, t, lesp, tesp);
    push!(blobs, PotentialFlow.Vortex.Blob(z₊, Γ₊, blobs[1].δ), PotentialFlow.Vortex.Blob(z₋, Γ₋, blobs[1].δ))
end
N = 128 # number of plate control points (distributed along a extrema Chebyshev grid)
δ = 0.01
plate = PotentialFlow.Plate(N, c, zero(ComplexF64), α)
motion = Plates.RigidBodyMotion(ċ, 0.0);
```

    WARNING: using PotentialFlow.Vortex in module ##1127 conflicts with an existing identifier.

Initial step

```julia
Δz₀ = im*3Δt*exp(im*plate.α) # vectors perpendicular to the plate
z₋, z₊ = plate.zs[[1,N]] # LE and TE
blobs = PotentialFlow.Vortex.Blob.(Δz₀ .+ [z₊, z₋], 1.0, δ) # First two point vortices are placed close to the LE and TE with unit strength
Plates.enforce_no_flow_through!(plate, motion, (), 0)
Γ₊, Γ₋, _, _ = Plates.vorticity_flux!(plate, blobs[1], blobs[2], 0.0, σLE, σTE); # Determine strength of first two vortices
blobs = PotentialFlow.Vortex.Blob.(Δz₀ .+ [z₊, z₋], [Γ₊, Γ₋], δ) # Create first two point vortices now with calculated strengths
sys₀ = (plate, blobs)
sys = deepcopy(sys₀)
sys₊ = deepcopy(sys₀) # Used for storage during time-marching
ẋs = (motion, allocate_velocity(blobs))
imp = ComplexF64[];
t_hist = Float64[]
global t = 0
push!(t_hist,t)
```

    1-element Vector{Float64}:
     0.0

Time stepping

```julia
for tloc in T[2:end]
    global t += Δt
    global sys
    global sys₊
    push!(imp,Elements.impulse(sys))
    push!(t_hist,t)
    local plate, ambient_ω = sys
    local motion, ambient_u = ẋs
    resize!(sys₊[2], length(sys[2]))
    forward_euler!(sys₊, sys, t, Δt, compute_ẋ!, advect!, ẋs)
    sys, sys₊ = sys₊, sys
    shed_new_vorticity!(sys[2], sys[1], ẋs[1], t, σLE, σTE)
end
force = -diff(imp)/Δt;

Δx = 0.01
xlim = (-0.5,2)
ylim = (-1,1)
g = PhysicalGrid(xlim,ylim,Δx);
```

```julia
dsdx = 2
plate = RigidBodyTools.Plate(c,dsdx*Δx)
pfb = PotentialFlowBody(plate,edges=[1,length(plate)],σ=[SuctionParameter(σLE),SuctionParameter(σTE)])
transform = RigidTransform((0.0,0.0),-π/3)
transform(plate);
Δs = dlength(plate)
maximum(Δs/Δx);
```

As before, we create the initial vortices based on the time step and uniform flow and define a function that create the new point vortices for the subsequent time steps.

```julia
firstvLE = GridPotentialFlow.Vortex(plate.x[1]+3Δt*(-ċ)*cos(plate.α+π/2),plate.y[1]+3Δt*(-ċ)*sin(plate.α+π/2),0.0);
firstvTE = GridPotentialFlow.Vortex(plate.x[end]+3Δt*(-ċ)*cos(plate.α+π/2),plate.y[end]+3Δt*(-ċ)*sin(plate.α+π/2),0.0);
```

```julia
function createsheddedvortices(plate,oldvortices)

    vLE = GridPotentialFlow.Vortex(2/3*plate.x[1]+1/3*oldvortices[end-1].x,2/3*plate.y[1]+1/3*oldvortices[end-1].y,0.0)
    vTE = GridPotentialFlow.Vortex(2/3*plate.x[end]+1/3*oldvortices[end].x,2/3*plate.y[end]+1/3*oldvortices[end].y,0.0)

    return vLE,vTE
end
```

    createsheddedvortices (generic function with 1 method)

The model free stream should be the negative of the translational velocity of the plate.

```julia
model = VortexModel(g,bodies=[pfb],vortices=[firstvLE,firstvTE],U∞=(-ċ,0.0));
```

Then we enter a time stepping loop and record the impulse every time we create new vortices. Note that because we release new vortices at every time step, it is not straightforward to use a multi-step time integration method like RK4 to advance the positions of the point vortices. Instead, we use forward Euler and the size of the positions vector increases every time step by the number of new vortices we insert. To advance the system one time step, we first need to solve the saddle-point system, set the strengths of the new vortices, subtract that circulation from the bodies, and calculate the vortex velocities with the saddle-point system solution. For convenience, these four operations are carried out internally if we would call `vortexvelocities!(model)`. However, if we want to have access to the solution of the saddle-point system, we can explicitly perform each operation as is done in the following time stepping loop. Note that we have to set the strengths of the new vortices before calling the `impulse` function, otherwise the new vortices won't be included in the calculation.

```julia
Px_hist = Float64[];
Py_hist = Float64[];
sol = solve(model);
for tloc in T[2:end]
    X = getvortexpositions(model) # gets bigger every time step because we add vortices
    Ẋ = deepcopy(X)

    solve!(sol, model)
    setvortexstrengths!(model, sol.δΓ_vec, length(X.u)-1:length(X.u))
    subtractcirculation!(model.bodies, sol.δΓ_vec)
    Px, Py = impulse(model)

    vortexvelocities!(Ẋ, model, sol.ψ)
    X .= X .+ Ẋ*Δt
    setvortexpositions!(model, X)

    vLE, vTE = createsheddedvortices(plate,model.vortices)
    pushvortices!(model,vLE,vTE)
    push!(Px_hist,Px)
    push!(Py_hist,Py)
end
```

    ┌ Warning: Assignment to `X` in soft scope is ambiguous because a global variable by the same name exists: `X` will be treated as a new local. Disambiguate by using `local X` to suppress this warning or `global X` to assign to the existing global variable.
    └ @ string:5

The force history can be obtained by taking finite differences of the impulse history.

```julia
Fx_hist = -diff(Px_hist)/Δt;
Fy_hist = -diff(Py_hist)/Δt;
```

We can now compare the positions of the point vortices by shifting the `PotentialFlow.jl` solution by `-tf*ċ` such that the origin of the frame of reference coincides with the center plate. Superimposing the `GridPotentialFlow.jl` solution then shows that the positions of the point vortices agree very well.

```julia
plot(plate,fillcolor=:black,fillrange=0,fillalpha=0.25,linecolor=:black,linewidth=2,xlim=xlim,ylim=ylim,xlabel="x",ylabel="y")
scatter!(real.((v->v.z).(sys[2])).-tf*ċ,imag.((v->v.z).(sys[2])),color=:red,markersize=4,label="PotentialFlow.jl")
scatter!(model.vortices.x,model.vortices.y,color=:blue,markersize=2,label="GridPotentialFlow.jl")
```

The vertical impulse and the vertical force (lift) can also be compared and show good agreement as well.

```julia
plot(xlabel="t",ylabel="Py")
plot!(T[1:end-1],imag.(imp),color=:blue,label="PotentialFlow.jl")
plot!(T[1:end-1],Py_hist,color=:red,label="GridPotentialFlow.jl")
```

```julia
plot(xlabel="t",ylabel="Fy")
plot!(T[2:end-1],imag.(force),color=:blue,label="PotentialFlow.jl")
plot!(T[2:end-1],Fy_hist,color=:red,label="GridPotentialFlow.jl")
```

## Added mass

The added mass tensor provides a measure of the inertial influence of the fluid on the body in response to changes in the body's translational or rotational motion. The coefficients of the added mass tensor of a body are obtained by computing the impulse components associated with a unit-valued component of motion. The motion's influence is both direct, via the surface velocity, and indirect, in the bound vortex sheet that develops on the surface.

The added mass for simple geometries can easiliy be calculated in an analytical way. For example, the entries of the translational added mass tensor for an ellipse with semi-major axis $a$ and semi-minor axis $b$ are $m_{xx} = \rho \pi b^2$ for motion in the $x$ direction, $m_{yy} = \rho \pi a^2$ for motion in the $y$-direction, with the diagonal entries $m_{xy}=m_{yx}=0$. By using `addedmass` from this package we can approximate these results numerically. In this case, we have one body and the method will return a matrix with the entries

$\begin{bmatrix}
m_{xx} & m_{xy} \\
m_{yx} & m_{yy}
\end{bmatrix}.$

```julia
Lx = 2.0
xlim = (-Lx/2,Lx/2)
ylim = (-Lx/2,Lx/2)
g = PhysicalGrid(xlim,ylim,0.01)
Δx = cellsize(g);

a = 0.5
b = 0.25
ellipse = Ellipse(a,b,Δx)
pfb = PotentialFlowBody(ellipse)
model = VortexModel(g,bodies=[pfb])
M = GridPotentialFlow.addedmass(model)
```

    2×2 Matrix{Float64}:
      0.209533     -2.32453e-16
     -2.01445e-16   0.804095

We can compare the values using the `@test` macro.

```julia
using Test
@test isapprox(M[1,1], π*b^2, rtol=1e-1)
```

    Test Passed

```julia
@test isapprox(M[2,2], π*a^2, atol=1e-1)
```

    Test Passed

In the case of $N$ bodies, `addedmass` returns the translational added mass tensor of size $2N$-by-$2N$. As an example, we will compute the added mass tensor for an array of cylinders. To compare it with an analytically obtained solution, we will calculate the added mass matrix tensor for the following gap-to-radius ratios.

```julia
GRratios = [0.1,0.2,0.3,0.4,0.5,0.6,0.8,1.0,1.2,1.4,1.6];

function rectangulararray(rows,columns,spacing)
     centers = VectorData(rows*columns)
     for j in 1:columns
        for i in 1:rows
            xc = (i-1)*spacing
            yc = (j-1)*spacing
            centers.u[(j-1)*rows+i] = xc
            centers.v[(j-1)*rows+i] = yc
        end
    end
    centers.u .-= mean(centers.u)
    centers.v .-= mean(centers.v)
    return centers
 end
```

    rectangulararray (generic function with 1 method)

For this case, the array consists of three rows of three cylinders.

```julia
n = 100
rows = 3
columns = 3
N = rows*columns
R = 0.20
𝒱 = π*R^2
bodies = fill(Circle(R,2*cellsize(g)),N);
Δs = minimum(dlength(bodies[1]));
```

We loop over the gap-to-radius ratios, position the bodies, and compute the ratio of the largest eigenvalue to the largest diagonal element of the translational added mass tensor. To position the bodies, we defined the method `rectangulararray` (see notebook in the examples folder).

```julia
using Statistics: mean
using LinearAlgebra: eigen
```

```julia
λoverMratios = zeros(length(GRratios))
for idx in 1:length(GRratios)
    global bodies
    gap = GRratios[idx]*R
    spacing = 2*R+gap
    bodycenters = rectangulararray(rows,columns,spacing)
    for i in 1:N
        Tf = RigidTransform((bodycenters.u[i],bodycenters.v[i]),0.0)
        global bodies[i] = Tf(deepcopy(bodies[i]))
    end
    body_list = PotentialFlowBody.(bodies)
    vm = VortexModel(g,bodies=body_list);
    M = GridPotentialFlow.addedmass(vm)/𝒱
    eigM = eigen(M);
    max_eig_value_coef = maximum(real(eigM.values))
    max_self_added_mass_coef = maximum(M)
    λoverMratios[idx] = max_eig_value_coef/max_self_added_mass_coef
end
```

    ┌ Warning: Assignment to `M` in soft scope is ambiguous because a global variable by the same name exists: `M` will be treated as a new local. Disambiguate by using `local M` to suppress this warning or `global M` to assign to the existing global variable.
    └ @ string:13

The same ratios, obtained in an analytical way, are available in literature [^1] and can be used to verify the numerical values.

```julia
nineRodArrayChen = [2.3637,2.2092,2.1007,2.0120,1.9350,1.8665,1.7494,1.6531,1.5732,1.5066,1.4508];
plot(xlabel="GR",ylabel="λ/max(Mij)")
plot!(GRratios,nineRodArrayChen,label="Chen1975")
plot!(GRratios,λoverMratios,label="GridPotentialFlow.jl")
```

[^1]: Chen, S. S. (1975) "Vibration of nuclear fuel bundles," *Nuclear Engineering and Design*, 35 (3), 399-–422.

---

*This notebook was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl).*


# Scientific Computing with Python

This notebook by Xiaozhou Li is licensed under a Creative Commons Attribution 4.0 International License.
All code examples are also licensed under the [MIT license](http://opensource.org/licenses/MIT).

```python
%matplotlib inline
import matplotlib.pyplot as plt
```

## Numpy
### Introduction
The numpy package (module) is used in almost all numerical computation using Python. It is a package that provide high-performance vector, matrix and higher-dimensional data structures for Python. It is implemented in C and Fortran so when calculations are vectorized (formulated with vectors and matrices), performance is very good.

### Array Creation and Properties
NumPy arrays allow you to write fast (optimized) code that works on arrays of data.  To do this, there are some restrictions on arrays:

* all elements are of the same data type (e.g. float)
* the size of the array is fixed in memory, and specified when you create the array (e.g., you cannot grow the array like you do with lists)

Note that the traditional way to import numpy is to rename it np.  This saves on typing and makes your code a little more compact.

```python
import numpy as np
```

```python
a = np.arange(12)
```

```python
a
```

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])

```python
a = np.arange(12).reshape(3,4)
```

```python
a
```

    array([[ 0,  1,  2,  3],
           [ 4,  5,  6,  7],
           [ 8,  9, 10, 11]])

an array is an object of the `ndarray` class, and it has methods that know how to work on the array data.  Here, `.reshape()` is one of the many methods.

A NumPy array has a lot of meta-data associated with it describing its shape, datatype, etc.

```python
print(a.ndim)
print(a.shape)
print(a.size)
print(a.dtype)
print(a.itemsize)
print(type(a))
```

    2
    (3, 4)
    12
    int64
    8
    <class 'numpy.ndarray'>

```python
#help(a)
```

we can create a multi-dimensional array of a specified size initialized all to 0 easily, using the zeros() function.

```python
b = np.zeros((5,4))
b
```

    array([[0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.]])

There is also an analogous ones() and empty() array routine.  Note that here we can explicitly set the datatype for the array in this function if we wish.

```python
c = np.eye(10, dtype=np.float64)
c
```

    array([[1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
           [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
           [0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
           [0., 0., 0., 1., 0., 0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 1., 0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0., 1., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
           [0., 0., 0., 0., 0., 0., 0., 1., 0., 0.],
           [0., 0., 0., 0., 0., 0., 0., 0., 1., 0.],
           [0., 0., 0., 0., 0., 0., 0., 0., 0., 1.]])

`linspace` creates an array with evenly space numbers.  The `endpoint` optional argument specifies whether the upper range is in the array or not

```python
d = np.linspace(0, 1, 10, endpoint=False)
print(d)
d = np.linspace(0, 1, 10, endpoint=True)
print(d)
d = np.linspace(0, 1, 11)
print(d)
```

    [0.  0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9]
    [0.         0.11111111 0.22222222 0.33333333 0.44444444 0.55555556
     0.66666667 0.77777778 0.88888889 1.        ]
    [0.  0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1. ]

### Slicing

```python
a = np.arange(9)
a
```

    array([0, 1, 2, 3, 4, 5, 6, 7, 8])

Now look at accessing a single element vs. a range (using slicing)

Giving a single (0-based) index just references a single value

```python
a[2]
```

    2

Giving a range uses the range of the edges to return the values

```python
print(a[2:3])
```

    [2]

```python
a[2:4]
```

    array([2, 3])

The : can be used to specify all of the elements in that dimension

```python
a[:]
```

    array([0, 1, 2, 3, 4, 5, 6, 7, 8])

### Multidimensional Arrays

Multidimensional arrays are stored in a contiguous space in memory -- this means that the columns / rows need to be unraveled (flattened) so that it can be thought of as a single one-dimensional array.  Different programming languages do this via different conventions:

Storage order:

* Python/C use *row-major* storage: rows are stored one after the other
* Fortran/matlab use *column-major* storage: columns are stored one after another

The ordering matters when

* passing arrays between languages (we'll talk about this later this semester)
* looping over arrays -- you want to access elements that are next to one-another in memory
  * e.g, in Fortran:
  ```
  double precision :: A(M,N)
  do j = 1, N
     do i = 1, M
        A(i,j) = …
     enddo
  enddo
  ```

  * in C
  ```
  double A[M][N];
  for (i = 0; i < M; i++) {
     for (j = 0; j < N; j++) {
        A[i][j] = …
     }
  }
  ```

In python, using NumPy, we'll try to avoid explicit loops over elements as much as possible

Let's look at multidimensional arrays:

```python
a = np.arange(15).reshape(3,5)
a
```

    array([[ 0,  1,  2,  3,  4],
           [ 5,  6,  7,  8,  9],
           [10, 11, 12, 13, 14]])

Notice that the output of `a` shows the row-major storage.  The rows are grouped together in the inner `[...]`

Giving a single index (0-based) for each dimension just references a single value in the array

```python
a[1,1]
```

    6

Doing slices will access a range of elements.  Think of the start and stop in the slice as referencing the left-edge of the slots in the array.

```python
a[0:2,0:2]
```

    array([[0, 1],
           [5, 6]])

Access a specific column

```python
a[:,1]
```

    array([ 1,  6, 11])

Sometimes we want a one-dimensional view into the array -- here we see the memory layout (row-major) more explicitly

```python
a.flatten()
```

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])

we can also iterate -- this is done over the first axis (rows)

```python
for row in a:
    print(row)
```

    [0 1 2 3 4]
    [5 6 7 8 9]
    [10 11 12 13 14]

or element by element

```python
for e in a.flat:
    print(e)
```

    0
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14

## Scipy
The SciPy framework builds on top of the low-level NumPy framework for multidimensional arrays, and provides a large number of higher-level scientific algorithms.

SciPy is a collection of numerical algorithms with python interfaces.  In many cases, these interfaces are wrappers around standard numerical libraries that have been developed in the community and are used with other languages.  Usually detailed references are available to explain the implementation.

There are many subpackages generally, you load the subpackages separately, e.g.

```
from scipy import linalg, optimize
```
then you have access to the methods in those namespaces

## Matplotlib
### Introduction
Matplotlib is an excellent 2D and 3D graphics library for generating scientific figures.

### Reading and writing a panda

**(1) original figure**

```python
plt.figure()
img = plt.imread('../data/panda.jpg')
plt.imshow(img)
plt.imsave("original.jpg",img)
```

**(2) red channel displayed in grey**

```python
plt.figure()
img_red = img[:, :, 0]
plt.imshow(img_red, cmap=plt.cm.gray)
```

**(3) lower resolution (compression)**

```python
plt.figure()
img_tiny = img[::8, ::8]
plt.imshow(img_tiny, interpolation='nearest')
#plt.savefig("compressed.jpg")
plt.imsave("compressed.jpg",img_tiny)
```

### Mandlebrot Set (Mandelbrot fractal)

```python
def compute_mandelbrot(N_max, some_threshold, nx, ny):
    # A grid of c-values
    x = np.linspace(-2, 1, nx)
    y = np.linspace(-1.5, 1.5, ny)

    c = x[:,np.newaxis] + 1j*y[np.newaxis,:]

    # Mandelbrot iteration

    z = c
    for j in range(N_max):
        z = z**2 + c

    mandelbrot_set = (abs(z) < some_threshold)

    return mandelbrot_set

mandelbrot_set = compute_mandelbrot(50, 50., 601, 401)

plt.imshow(mandelbrot_set.T, extent=[-2, 1, -1.5, 1.5])
plt.gray()
plt.show()
```

### A simple example of 3D plotting
$$ z = \sin(\sqrt{x^2 + y^2}) $$

```python
from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure()
ax = Axes3D(fig)
X = np.arange(-4, 4, 0.25)
Y = np.arange(-4, 4, 0.25)
X, Y = np.meshgrid(X, Y)
R = np.sqrt(X ** 2 + Y ** 2)
Z = np.sin(R)

ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=plt.cm.hot)
ax.contourf(X, Y, Z, zdir='z', offset=-2, cmap=plt.cm.hot)
ax.set_zlim(-2, 2)

plt.show()
```

### An example displaying the contours of a function
$$ f(x,y) = \left(1 - \frac{x}{2} + x^5 + y^3\right)e^{-x^2-y^2}.$$

```python
def f(x,y):
    return (1 - x / 2 + x**5 + y**3) * np.exp(-x**2 -y**2)

n = 256
x = np.linspace(-3, 3, n)
y = np.linspace(-3, 3, n)
X,Y = np.meshgrid(x, y)

plt.axes([0.025, 0.025, 0.95, 0.95])

plt.contourf(X, Y, f(X, Y), 8, alpha=.75, cmap=plt.cm.hot)
C = plt.contour(X, Y, f(X, Y), 8, colors='black')
plt.clabel(C, inline=1, fontsize=10)

plt.xticks(())
plt.yticks(())
plt.show()
```

## Sympy - Symbolic algebra in Python

### Introduction
There are two notable Computer Algebra Systems (CAS) for Python:

* [SymPy](http://sympy.org/en/index.html) - A python module that can be used in any Python program, or in an IPython session, that provides powerful CAS features.
* [Sage](http://www.sagemath.org/) - Sage is a full-featured and very powerful CAS enviroment that aims to provide an open source system that competes with Mathematica and Maple. Sage is not a regular Python module, but rather a CAS environment that uses Python as its programming language.

Sage is in some aspects more powerful than SymPy, but both offer very comprehensive CAS functionality. The advantage of SymPy is that it is a regular Python module and integrates well with the Jupyter notebook.

```python
from sympy import *

init_printing()
```

### Expand, factor and simplify

```python
x, y = symbols('x y')

(x+1)*(x+2)*(x+3)*(x+4)*(x+5)
```

```python
expand((x+1)*(x+2)*(x+3)*(x+4)*(x+5))
```

```python
sin(x+y)
```

```python
expand(sin(x+y), trig=True)
```

```python
expand((x+y)**8)
```

```python
x**3 + 6 * x**2 + 11*x + 6
```

```python
factor(x**3 + 6 * x**2 + 11*x + 6)
```

```python
sin(x)**2 + cos(x)**2
```

```python
simplify(sin(x)**2 + cos(x)**2)
```

```python
cos(x)/sin(x)
```

```python
simplify(cos(x)/sin(x))
```

### Calculus
**(1) differentiation and integration**

$f(x) = (x+1)^2$

```python
f = (x+1)**2
f
```

Computing $\frac{d f}{dx}$, $\frac{d f^2}{dx}$

```python
diff(f,x)
```

```python
diff(f**2,x)
```

Computing $\frac{d \sin(f)}{dx}$, $\frac{d^2 \sin(f)}{dx^2}$

```python
diff(sin(f),x)
```

```python
diff(sin(f),x,2)
```

```python
diff(sin(f),x,4)
```

$$ f(x,y) = \sin(xy) + \cos(xy),$$
computing
$$ \frac{\partial^3 f}{\partial x \partial y^2},\quad \int f(x,y)\,dx,\quad \int_{-1}^{1}f(x,y)\,dx$$

```python
f = sin(x*y) + cos(y*x)
f
```

```python
diff(f, x, 1, y, 2)
```

```python
integrate(f, x)
```

```python
integrate(f, (x, -1, 1))
```

Computing $\int_{-\infty}^\infty e^{-x^2}\,dx$

```python
integrate(exp(-x**2), (x, -oo, oo))
```

**(2) limits**
$$ \lim\limits_{x\rightarrow 0}\frac{\sin(x)}{x},\quad \lim\limits_{x\rightarrow 0^{+}}\frac{1}{x},\quad \lim\limits_{x\rightarrow 0^{-}}\frac{1}{x}$$

```python
limit(sin(x)/x, x, 0)
```

```python
limit(1/x, x, 0, dir="+")
```

```python
limit(1/x, x, 0, dir="-")
```

**(3) series**

```python
exp(x)
```

```python
series(exp(x), x)
```

```python
series(exp(x), x, 1)
```

```python
series(sin(x), x, 0, 12)
```

```python
series(sin(x)*cos(x), x, 0, 8)
```

```python
series(sin(x)*cos(x)*exp(x), x, 0, 12)
```

### Linear algebra: Matrices

```python
m11, m12, m21, m22 = symbols("m11, m12, m21, m22")
b1, b2 = symbols("b1, b2")
```

```python
A = Matrix([[m11, m12],[m21, m22]])
A
```

```python
b = Matrix([[b1], [b2]])
b
```

```python
A**2
```

```python
A**3
```

```python
A * b
```

```python
A.det()
```

```python
A.inv()
```

### Solving equations
Solving
$$ x^2 - 1 = 0,\quad x^4 - x^2 - 1 = 0$$

```python
solve(x**2 - 1, x)
```

```python
solve(x**4 + x**3 - x**2 - 1, x)
```

Solving systems:
$$ x + y - 1 = 0,\quad x - y - 1 = 0,$$
and
$$ x + y - a = 0,\quad x - y - b = 0.$$

```python
solve([x + y - 1, x - y - 1], [x,y])
```

```python
a, b = symbols('a, b')
solve([x + y - a, x - y - b], [x,y])
```

```python

```


# Chapter 2 - Math Tools II (Matrices and Operators)

The previous chapter was concluded by describing how one could use a vector to describe a quantum state. This was followed by giving the example of the most basic form of quantum information, a qubit. However, given that a quantum computer's function is to manipulate and evolve a qubit's state according to certain instructions (also called algorithms), it would seem obvious that what we learned isn't sufficient enough to describe quantum information. In quantum mechanics, a system's state is evolved by what is called an operator, and similarly to how vectors are used to describe a system's state, matrices are used to describe how that state evolves once acted upon by an operator. Numpy will also be used for Python implementations.

 ## 2.1 Matrices

In Chapter 1, specifically in the sections related to Dirac Notation, vectors were described by either a column or a row matrix, and matrix multiplication was used to describe inner products. This chapter aims at describing Matrices in more detail and aims at explaining everything you would need to know about matrices to complete this series (which is arguably a complete introduction to basic quantum information theory). A matrix is essentially a 2D, or more specifically, a rectangular array of numbers. A matrix $M$, which has dimensions $m$ and $n$, would look like the following:

   \begin{equation}
  M = \begin{pmatrix} M_{11} & M_{12} & \ldots & M_{1n} \\\ M_{21} & M_{22} & \ldots & M_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ M_{m1} & M_{m2} & \ldots & M_{mn}\end{pmatrix}
   \end{equation}

Where the components of the matrix, $M_{ij}$ are numbers (for the sake of this series). If $M_{ij} \in R$, then the matrix is called a real matrix. Similarly, if $M_{ij} \in C$, the matrix is called a complex matrix.

For example,

   \begin{equation}
  M = \begin{pmatrix} 4 & i & 1 - i \\\ 2 & 0 & 2i\end{pmatrix}
   \end{equation}

   is a 2x3 complex matrix. $M_{22}$ = 0, $M_{21} = 2$, etc.

  ### Operations on matrices

  **Matrix Addition:**

  Condition: Two matrices, $A$ and $B$, with dimensions $m_A \times n_A$ and $m_B \times n_b$, can only be added if $m_A = m_B$ and $n_A = n_B$ (i.e if they have the same dimension).

  Given that they have the same dimensions, their sum would yield the following:

  \begin{equation}
  C = A + B \newline C = \begin{pmatrix} A_{11} & A_{12} & \ldots & A_{1n} \\\ A_{21} & A_{22} & \ldots & A_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ A_{m1} & A_{m2} & \ldots & A_{mn}\end{pmatrix} + \begin{pmatrix} B_{11} & B_{12} & \ldots & B_{1n} \\\ B_{21} & B_{22} & \ldots & B_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ B_{m1} & B_{m2} & \ldots & B_{mn}\end{pmatrix} \newline C = \begin{pmatrix} A_{11} + B_{11}  & A_{12} + B_{12} & \ldots & A_{1n} + B_{1n} \\\ A_{21} + B_{21} & A_{22} + B_{22} & \ldots & A_{2n} + B_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ A_{m1} + B_{m1} & A_{m2} + B_{m2} & \ldots & A_{mn} + B_{mn}\end{pmatrix}
  \end{equation}

  Or in simpler terms, $C$ is a matrix where $C_{ij} = A_{ij} + B_{ij}$.

  **Matrix Subtraction:**

  Condition: Similar to addition, the condition is that the dimensions of matrices $A$ and $B$ are equal

   Given that they have the same dimensions, their difference would yield the following:

  \begin{equation}
  C = A - B \newline C = \begin{pmatrix} A_{11} & A_{12} & \ldots & A_{1n} \\\ A_{21} & A_{22} & \ldots & A_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ A_{m1} & A_{m2} & \ldots & A_{mn}\end{pmatrix} - \begin{pmatrix} B_{11} & B_{12} & \ldots & B_{1n} \\\ B_{21} & B_{22} & \ldots & B_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ B_{m1} & B_{m2} & \ldots & B_{mn}\end{pmatrix} \newline C = \begin{pmatrix} A_{11} - B_{11}  & A_{12} - B_{12} & \ldots & A_{1n} - B_{1n} \\\ A_{21} - B_{21} & A_{22} - B_{22} & \ldots & A_{2n} - B_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ A_{m1} - B_{m1} & A_{m2} - B_{m2} & \ldots & A_{mn} - B_{mn}\end{pmatrix}
  \end{equation}

  Or in simpler terms, $C$ is a matrix where $C_{ij} = A_{ij} - B_{ij}$

  **Scalar Multiplication:**

  Multiplying a scalar $c$ with a matrix $A$ would yield, as with vectors, the same matrix where the components undergo the following transformation: $A_{ij} \rightarrow c*A_{ij}$, or in other words:

  \begin{equation}
  B = c*A \newline B = c\begin{pmatrix} A_{11} & A_{12} & \ldots & A_{1n} \\\ A_{21} & A_{22} & \ldots & A_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ A_{m1} & A_{m2} & \ldots & A_{mn}\end{pmatrix} \newline B = \begin{pmatrix} cA_{11} & cA_{12} & \ldots & cA_{1n} \\\ cA_{21} & cA_{22} & \ldots & cA_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ cA_{m1} & cA_{m2} & \ldots & cA_{mn}\end{pmatrix}
  \end{equation}

  **Matrix Multiplication:**

  Condition: For two matrices $A$ and $B$ to be multipliable, the following condition must apply: $n_A = m_b$, or in other words, the number of columns in $A$ must be equal to the number of rows in $B$. The result will be an $m_A \times n_B$ matrix. For example, if one is multiplying a $2 \times 4$ matrix by a $4 \times 2$ matrix, it will work out to be a $2 \times 2$ matrix.

  The result of multiplying an $m \times n$ with an $n \times p$ will be
  \begin{equation}
  C = AB \newline C = \begin{pmatrix} A_{11} & A_{12} & \ldots & A_{1n} \\\ A_{21} & A_{22} & \ldots & A_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ A_{m1} & A_{m2} & \ldots & A_{mn}\end{pmatrix}\begin{pmatrix} B_{11} & B_{12} & \ldots & B_{1p} \\\ B_{21} & B_{22} & \ldots & B_{2p} \\\ \vdots & \vdots & \ddots &\vdots \\\ B_{n1} & B_{n2} & \ldots & B_{np}\end{pmatrix} \newline C = \begin{pmatrix} C_{11} & C_{12} & \ldots & C_{1p} \\\ C_{21} & C_{22} & \ldots & C_{2p} \\\ \vdots & \vdots & \ddots &\vdots \\\ C_{m1} & C_{m2} & \ldots & C_{mp}\end{pmatrix}
  \end{equation}

  where $C_{ij} = A_{i1}A_{1j} + A_{i2}A_{2j} + \ldots A_{in}A_{nj} = \sum_{k=1}^n A_{ik}A_{kj}$.

  Note that in practice, it is much easier to get the hang of matrix multiplication than to think of it in terms of a generalized formula, so I'd suggest solving a few matrix multiplication problems to understand what this actually means. It is also important to note that matrix multiplication is not commutative (i.e $AB \neq BA$)

  **Matrix Division:**

  There isn't such a thing. One could argue that the closest thing to matrix division is multiplying a matrix $A$ by a matrix $B$'s inverse, $B^{-1}$. But that will be spoken about in Section 2.2

  ### Python Implementation

```python
#Importing necessary modules: numpy can be used to describe matrices in two ways: np.array and np.matrix
#Both were used in Chapter 1, but from now on, we will carry on with np.array
#We'll also import a sympy function so that we can visualize matrices in a clearer way. This does not affect any calculation

import numpy as np
from sympy import Matrix, init_printing

#This function uses SymPy to convert a numpy array into a clear matrix image

def view(mat):
    display(Matrix(mat))
```

```python
#Defining Matrices

A = np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]])
print('4x4 Matrix, A =\n', A)

print('\n')

print('Using SymPy, we can make this look clearer, like this:')
view(A)

#Getting individual components
#A very important thing to note is that python starts counting from 0, so A[2][3] would give us A_34

print('A_34 = ',A[2][3])

```

    4x4 Matrix, A =
     [[ 1  2  3  4]
     [ 5  6  7  8]
     [ 9 10 11 12]
     [13 14 15 16]]

    Using SymPy, we can make this look clearer, like this:

$\displaystyle \left[\begin{matrix}1 & 2 & 3 & 4\\5 & 6 & 7 & 8\\9 & 10 & 11 & 12\\13 & 14 & 15 & 16\end{matrix}\right]$

    A_34 =  12

```python
#Matrix Operation

#Defining two 2x3 matrices, A and B, and one scalar, c
A = np.array([[1,2,0],[1,3,4]])
B = np.array([[4,0,6],[3,0,1]])
C = np.array([[1,0],[9,4],[3,2]])
c = 5

#Addition
print('A+B =')
view(A+B)
print('\n')

#Subtraction
print('A-B = ')
view(A-B)
print('\n')

#Scalar Multiplication
print('cA = ')
view(c*A)
print('\n')
#Matrix Multiplication, where AB is written as A@B in python
print('AC = ')
view(A@C)
print('\n')
#Cocktail
print('-2A + 3B = ')
view(-2*A + 3*B)

#Excersise: Examine Multiplication for matrices A and B, where dim(A) =/= dim(B)

```

    A+B =

$\displaystyle \left[\begin{matrix}5 & 2 & 6\\4 & 3 & 5\end{matrix}\right]$

    A-B =

$\displaystyle \left[\begin{matrix}-3 & 2 & -6\\-2 & 3 & 3\end{matrix}\right]$

    cA =

$\displaystyle \left[\begin{matrix}5 & 10 & 0\\5 & 15 & 20\end{matrix}\right]$

    AC =

$\displaystyle \left[\begin{matrix}19 & 8\\40 & 20\end{matrix}\right]$

    -2A + 3B =

$\displaystyle \left[\begin{matrix}10 & -4 & 18\\7 & -6 & -5\end{matrix}\right]$

## 2.2 Square Matrices

A matrix that has an equal number of rows and columns is called a square matrix. Its dimensions are described as $n \times n$, where $n$ is what is known as the order of the matrix. For instance, the following matrix is described as a square matrix of order 2:

\begin{equation}
\begin{pmatrix} 1 & 2i \\\ 3 & -1 \end{pmatrix}
\end{equation}

The diagonal of a square matrix is formed from by set of numbers $A_{ii}$. For example, in the order 2 matrix example above, the diagonal has elements 1 and -1 ($A_{11}$ and $A_{22}$)

 Square matrices are very useful for describing linear transformations such as rotation, as well as getting the area formed by a set of vectors. In quantum information theory, a specific type of square matrix called a unitary matrix is used to describe the operators which act on a statevector to change the amplitudes.
  ### Determinant and Trace

 Determinants and traces are operations that are unique to square matrices. The result of a determinant is a number that affects how "scaled" an object is when transformed by a matrix. Similar to matrix multiplication, the general equation of a determinant is usually tedious to look at at first hand, but I'll present the specific cases for $2 \times 2$ and $3 \times 3$ matrices.

 For a $2 \times 2$ matrix, $A$, one would find the determinant to be:

 \begin{equation}
 det(A) = det\begin{pmatrix} a & b \\\ c & d \end{pmatrix} \newline = ad - bc
 \end{equation}

 For a $3 \times 3$ matrix, $A$, one would find the determinant to be:

  \begin{equation}
 det(A) = det\begin{pmatrix} a & b & c \\\ d & e & f \\\ g & h & i \end{pmatrix} \newline = a(ei - hf) - b(di -fg) + c(dh - eg)
 \end{equation}

 This is all the information I'll be giving on determinants because it is the only information required for this series. However, feel free to investigate this at your own speed.

 A trace of a square matrix is essentially the sum of all its diagonal's components. Formally, it is written as the following:

 \begin{equation}
 Tr(A) = \sum_{i=1}^n A_{ii}
 \end{equation}

  In Section 2.1, we described that matrix multiplication is not always commutative, and that in many cases, $AB \neq BA$. However, an interesting property is that $Tr(BA) = Tr(AB)$ regardless of whether the matrices commute or not. As the name suggests, matrices commute when $AB = BA$, and don't commute when $AB \neq BA$.

 ### Diagonal Matrices

 A diagonal matrix is a special case of a square matrix where every element in the matrix is zero except for its diagonal. The following is an example of a $3 \times 3$ diagonal matrix:

 \begin{equation}
 \begin{pmatrix} 1 & 0 & 0 \\\ 0 & 3 & 0 \\\ 0 & 0 & 2+i \end{pmatrix}
 \end{equation}

 ### Triangular Matrices
 It is said that the diagonal divides the matrix into two blocks. The upper block, consists of all elements, $A_{ij}$ where $i \leq j$. Similarly, the lower block consists of all $A_{ij}$ where $i \geq j$. An upper triangle matrix is any matrix where the upper block consists of any set of numbers and the lower block consits of only zeros. In other words, it is any matrix that takes the following form:

   \begin{equation}
  M = \begin{pmatrix} M_{11} & M_{12} & \ldots & M_{1n} \\\ 0 & M_{22} & \ldots & M_{2n} \\\ \vdots & \vdots & \ddots &\vdots \\\ 0 & 0 & \ldots & M_{nn}\end{pmatrix}
   \end{equation}

   Matrix $A$ is a $4 \times 4$ example:

   \begin{equation}
  A = \begin{pmatrix} M_{11} & M_{12}  & M_{13} & M_{14} \\\ 0 & M_{22} & M_{23} & M_{24} \\\ 0 & 0 & M_{33} & M_{34} \\\ 0 & 0 & 0 & M_{44}\end{pmatrix}
   \end{equation}

  On the other hand, we also have lower triangle matrices, where $A_{ij} = 0$ if $i>j$. It takes the following form:

   \begin{equation}
  M = \begin{pmatrix} M_{11} &0 & \ldots & 0 \\\ M_{21} & M_{22} & \ldots & 0 \\\ \vdots & \vdots & \ddots &\vdots \\\ M_{m1} & M_{m2} & \ldots & M_{mn}\end{pmatrix}
   \end{equation}

   Matrix $B$ is a $4 \times 4$ example:

   \begin{equation}
  B = \begin{pmatrix} M_{11} & 0  & 0 & 0 \\\ M_{21} & M_{22} & 0 & 0 \\\ M_{31} & M_{31} & M_{33} & 0 \\\ M_{41} & M_{42} & M_{43} & M_{44}\end{pmatrix}
   \end{equation}

A diagonal matrix can be viewed as either an upper or lower matrix

 ### Identity Matrix and Matrix Inversion

 An identity matrix, $I$, is any matrix where its diagonal elements are all equal to 1 and the rest of the elements are equal to 0. Formally, it could be represented by the Kronicker-Delta function described in Section 1.9 in the following way:

 \begin{equation}
 I_{ij} = \delta_{ij}
 \end{equation}

 where $\delta_{ij}$ is
 \begin{equation}
\delta_{ij} =
    \begin{cases}
            1, &         \text{if } i=j,\\
            0, &         \text{if } i\neq j.
    \end{cases}
\end{equation}

The following is a $3 \times 3$ identity matrix, $I_3$:

\begin{equation}
I_3 = \begin{pmatrix} 1 & 0 & 0 \\\ 0 & 1 & 0 \\\ 0 & 0 & 1\end{pmatrix}
\end{equation}

 One of the most notable properties of the identity matrix is that when any matrix $A$ is multiplied with it,  the results will be the same matrix. In other words, for an $m \times n$ matrix,

 \begin{equation}
 I_m A = AI_n = A
 \end{equation}

 and for a square matrix, the order of multiplication with the identity won't matter because $n = m$.

 In Section 1, an important point that was stated is that there is no such this as matrix division but that for square matrices, one can get matrix $A$'s inverse $A^{-1}$, where

  \begin{equation}
  AA^{-1} = I
  \end{equation}

   ### Python Implementation

```python
#Defining square matrices

A = np.array([[3,0,2], [2,0,-2],[0,1,1]])

#Determinant and Trace

trA = A.trace()
detA = np.linalg.det(A)

print('A = ')
view(A)

print('Tr(A) = ', trA)
print('det(A)', detA)

print('')
#Identity and Inverse

Ainv = np.linalg.inv(A)

#Multiplying A with its inverse. The result should be the I_3

print('I = AA^-1 = ')
view(A@Ainv)

print('The result is the identity. I_21 is approximately zero and is not exactly zero because of some negligable numerical error')

#Excercise: Create a function that identifies whether a square matrix is a diagonal matrix, upper triangle or lower triangle
#Hint: Start by creating a kronDelta(i,j) function

```

    A =

$\displaystyle \left[\begin{matrix}3 & 0 & 2\\2 & 0 & -2\\0 & 1 & 1\end{matrix}\right]$

    Tr(A) =  4
    det(A) 10.000000000000002

    I = AA^-1 =

$\displaystyle \left[\begin{matrix}1.0 & 0.0 & 0.0\\-5.55111512312578 \cdot 10^{-17} & 1.0 & 0.0\\0.0 & 0.0 & 1.0\end{matrix}\right]$

    The result is the identity. I_21 is approximately zero and is not exactly zero because of some negligable numerical error

## 2.3 Operators

As you most likely know, a function is a mathematical object that transforms an input number into an output number. Similarly, an operator is a mathematical object which transforms a function into a whole other function. An example is the derivative, which yields the rate at which a function is changing. The definition of an operator could be extended to vectors. In other word, an operator maps a vector $|v_1>$ to a new vector $|v_2>$. Notationally, this could be written as follows:

\begin{equation}
A|v_1> = |v_2>
\end{equation}

Where $A$ is an operator, $|v_1>$ is the "input " vector and $|v_2>$ is the "output vector".

In a quantum system, an operator $A|\psi>$ can represent an observable, which is essentially a physical quantity such as momentum and position. More specifically, an observable must be a linear operator, which we'll speak about soon.

### Operator Algebra

Let us say we have operators $A$ and $B$ acting on vector $|\psi>$. The following rules must apply:

**Adding operators:**

\begin{equation}
(A+B)|\psi> = A|\psi> + B|\psi>
\end{equation}

**Multiplying two operators (on state):**

\begin{equation}
(AB)|\psi> = A[B|\psi>]
\end{equation}

**Multiplying three operators:**

\begin{equation}
A(BC) = (AB)C
\end{equation}

**Commuation:**
\begin{equation}
[A,B]|\psi> = AB|\psi> - BA|\psi>
\end{equation}

If $[A,B]|\psi> = 0$, then the operators commutate and if $[A,B]|\psi> \neq 0$, then the operators don't commutate. This is similar to what was mentioned in 2.2. Commutation has a very important physical implication in quantum mechanics that will be described in Chapter 3.

### Linear Operators

An operator, $A$, was defined to be an object that transforms a vector $|v_1>$ to $|v_2>$. If vectors $|v_1>$ and $|v_2>$ are in the same vector space **V**, then $A$ is said to be a linear operator. Notationally, linear operator $A$,  is represented as follows:

\begin{equation}
A|v_1> = |v_2> ; |v_1>, and |v_2> \in V
\end{equation}

If $A$ is a linear operator, than it must have the following properties:

      1. A(|u> + |v>) = A|u> + A|v>
      2. A(c|u>) = cA|u>
and finally, from 1 and 2:

      3. A(c|u> + d|v>) = cA|u> + dA|v>

Where $c$ and $d$ are some constants and $c|u> + d|v>$ is a linear combination. In other words, a linear operator acting on a linear combination of vectors is equal to a linear combination of the vectors produced by the act of applying the operator on them. In concise terms, one can simply say that the linear combination is preserved

As mentioned before, linear operators are used to describe physical quantities (such as position and momentum)in quantum mechanics. In quantum computing, linear operators are used to describe the $\textit{quantum logic gates}$ which evolves the state of a qubit. There is also what is known as an antilinear operator, which has the following property:

\begin{equation}
A(c_1 |\psi> + c_2|\phi>) = c_1^* A|\psi> + c_2^* A|\phi>
\end{equation}

A final important quality of linear opertors is that given an operator $A$ and ket $|\psi>$, where $A|\psi> = |\phi>$,

\begin{equation}
BA|\psi> = B(A|\psi>) = B|\phi> = |\omega>
\end{equation}

This is just to assert that we start by evaluating the operator closest to $|\psi>$.

  ### Python Implementation

```python
#Operator of a function where O[f(x)] = f(x)*2 + 3
#If you don't know what a lambda function is in Python, check it out here: https://www.w3schools.com/python/python_lambda.asp

def operator(f):
    newFunc = lambda x: f(x)*2 + 3
    return newFunc

#g(x) = O[f(x)], where f(x) = 2*x. The result should be 4x + 3
#g(3) = 4*3 + 3 = 15

g = operator(lambda x: 2*x)

print('g(x) = O[f(x)] = O[2x] = 2*2x + 3 = 4x+3')
print('g(3) = 4*3 + 3 = ', g(3))

#Excercise: Create a derivative(f) operator that gets the derivative of an input function

```

    g(x) = O[f(x)] = O[2x] = 2*2x + 3 = 4x+3
    g(3) = 4*3 + 3 =  15

```python
#Operator of a vector where A|u> = |v>, where v_i = 2*u_i + 3

def A(vec):
    newVec = np.array([2*i + 3 for i in range(len(vec)+1)])
    return newVec

u = np.array([1,2,3,4])
v = A(u)

print('|u> = ', u)
print('')
print('|v> = A|u> = ', v)

#Excersise: Create linChecker(operator) function to check if an operator O is linear or not

```

    |u> =  [1 2 3 4]

    |v> = A|u> =  [ 3  5  7  9 11]

```python
#Commutation: Creating a function that checks whether operators A and B commutate on some dummy function f(x) = 2*x
#[A,B](2x) =?= 0

'''
Apparently, trying to define such a function violates something called The Rice theorem, so the next best thing
one can do is check whether A(B(f(n))) == B(A(f(n))), where n is a number. n = 5 is chosen arbitrarly.

Be carefuly about which number you select, though.
'''

def commutate(A,B,n):
    f = lambda x: 2*x
    if A(B(f(n))) == B(A(f(n))):
        print('A and B commute!')
    else:
        print('A and B do not commute!')

A = lambda f: f**2
B = lambda f: np.log(f)

print('A = f(x)^2, B = ln(f(x)):')
commutate(A,B,5)

print('')

C = lambda f: 2*f
D = lambda f: 3*f

print('A = 2*f(x), B = 3*f(x):')
commutate(C,D,5)

```

    A = f(x)^2, B = ln(f(x)):
    A and B do not commute!

    A = 2*f(x), B = 3*f(x):
    A and B commute!

## 2.4 Matrix Representation of Linear Operators

We have stated what a linear operator is and have presented some basic rules of operator algebra, but we still have not described how to properly represent them. For linear operators (i.e the operators we are concerned with), a very effective way to represent an operator which acts on a $n$-dimensional vector is by an $n \times n$ square matrix.

For instance, if one were to transform a 3D vector $|u>$ to $|v>$, as follows

\begin{equation}
\begin{pmatrix} a \\\ b \\\ c\end{pmatrix} \rightarrow \begin{pmatrix} a+b \\\ a-b \\\ 3c\end{pmatrix}
\end{equation}

they can use an operator $A$, represented by a $3 \times 3$ matrix, or

\begin{equation}
A|u> = |v>\newline \begin{pmatrix} 1 & 1 & 0 \\\ 1 & -1 & 0 \\\ 0 & 0 & 3 \end{pmatrix}
\begin{pmatrix} a \\\ b \\\ c\end{pmatrix} =\begin{pmatrix} a+b \\\ a-b \\\ 3c\end{pmatrix}
\end{equation}

Using the rules of matrix multiplication described in Section 1.1, one would find this to be true (in fact if you're new to this, I would encourage trying it out manually).

Generally, an $n \times n$ matrix transforms an $n$-vector where the $i$th column of the matrix is what the $i$th basis vector is mapped to. To demonstrate this, I will use new notation which makes it more obvious. Previously, an object in a matrix was described by $M_{ij}$, where $i$ is the row index and $j$ is the column index. Alternatively, a matrix component will now be written as $j_i$, where the column index is represented by a letter, not a number. For instance, the 3 $\times$ 3 example is

\begin{equation}
M|u> = |v> \newline
\begin{pmatrix} a_1 & b_1 & c_1 \\\ a_2 & b_2 & c_2 \\\ a_3 & b_3 & c_3 \end{pmatrix}\begin{pmatrix} x\\y\\z\end{pmatrix}  = \begin{pmatrix} a_1x + a_2y + a_3z \\\ b_1x + b_2y + b_3z \\\ c_1x + c_2y + c_3z \end{pmatrix}
\end{equation}

 As you can see, the $i$th components of the $|v>$ is the linear combination of the vector components of $|u>$, where the scalars are the components of the $i$th row of matrix (linear operator) $M$

 ### Python Implementation

```python
#Linear operator Examples
print('Example 1:')
#A|u> = |v>, where |u> = - |v>

u = np.array([1,2,3])

v = -1*u

A = np.array([[-1,0,0],[0,-1,0],[0,0,-1]])

print('A|u> = |v> = -|u>')
print('')

print('A|u>, where A is -I_3 = ')
view(A@u)
print('')

print('v = ')
view(v)

print('They are equal! \n')

print('Example 2:')

#AB|psi> = phi|> where |psi>, |phi> in R^4

A = np.array([[1,0,3,4],[1,1,1,1],[0,9,2,-1]])
B = np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]])
psi = np.array([1,2,3,4])
phi= A@B@psi

print('|phi> = ','AB|psi> = ')
view(phi)

```

    Example 1:
    A|u> = |v> = -|u>

    A|u>, where A is -I_3 =

$\displaystyle \left[\begin{matrix}-1\\-2\\-3\end{matrix}\right]$

    v =

$\displaystyle \left[\begin{matrix}-1\\-2\\-3\end{matrix}\right]$

    They are equal!

    Example 2:
    |phi> =  AB|psi> =

$\displaystyle \left[\begin{matrix}960\\360\\700\end{matrix}\right]$

```python
#Application to Quantum Computing

#X|0> = |1>, where, according to section 1.10, |0> = (1,0) and |1> = (0,1)

zero = np.array([1,0])
one = np.array([0,1])

X = np.array([[0,1],[1,0]])
print('X = ')
view(X)

print('')

#Checking to see if X|0>
XZero = X@zero
print('Is X|0> = |1>:')
#Prints true if all elements in both matrices are equal
print(XZero.all() == one.all())

print('')

print('X is a very important operator in quantum computing because it essentially acts as a quantum NOT gate. The implication of this tranformation is taking a qubit which is always 0 and making it always 1 and vice versa (X|1>=|0>).')
```

    X =

$\displaystyle \left[\begin{matrix}0 & 1\\1 & 0\end{matrix}\right]$

    Is X|0> = |1>:
    True

    X is a very important operator in quantum computing because it essentially acts as a quantum NOT gate. The implication of this tranformation is taking a qubit which is always 0 and making it always 1 and vice versa (X|1>=|0>).

## 2.5 Operations on Matrices

In Section 2.2, we discussed certain operations such as determinants and traces that produce a scalar out of a matrix. This section aims at explaining three important operations which transform matrix $M \rightarrow M'$ using certain rules. These operations are important not only for defining which linear transformations are allowed in a quantum system, but for doing these transformations. Note that all of these were defined briefly in Section 1.5, but will be further elaborated here

### Complex Conjugate

The first operation that will be discussed is the complex conjugate of a matrix, which, as we've discussed several times, simple transforms the matrix in the following way: $M_{ij} \rightarrow M_{ij}^*$, or in other words, replaces every component in matrix $M$ with it's complex conjugate. Explicitly,

\begin{equation}
  M^* = \begin{pmatrix} M_{11}^* & M_{12}^* & \ldots & M_{1n}^* \\\ M_{21}^* & M_{22}^* & \ldots & M_{2n}^* \\\ \vdots & \vdots & \ddots &\vdots \\\ M_{m1}^* & M_{m2}^* & \ldots & M_{mn}^*\end{pmatrix}
   \end{equation}

**Example:**

\begin{equation}
\begin{pmatrix} i+2 & -1 \\\ 2i & 5 \end{pmatrix}^* = \begin{pmatrix} -i+2 & -1 \\\ -2i & 5 \end{pmatrix}
\end{equation}

The complex conjugate of a real matrix is itself
### Transpose

A transpose of a matrix $M$ essentially performs the following transformation: $M_{ij} \rightarrow M_{ji}$. In other words it switches the row and column indices. It is easy to realize that all components in the matrix will be affected except for the diagonal compontents, who are affected by a transpose in the following way: $M_{ii} \rightarrow M_{ii}$. Explicitly,

\begin{equation}
  M^T = \begin{pmatrix} M_{11} & M_{21} & \ldots & M_{n1} \\\ M_{12} & M_{22} & \ldots & M_{n2} \\\ \vdots & \vdots & \ddots &\vdots \\\ M_{1m} & M_{2m} & \ldots & M_{nm}\end{pmatrix}
   \end{equation}

**Example:**

\begin{equation}
\begin{pmatrix} 1 & 2 & 3 \\\ 4 & 5 & 6 \\\ 7 & 8 & 9 \end{pmatrix}^T =
\begin{pmatrix} 1 & 4 & 7 \\\ 2 & 5 & 8 \\\ 3 & 6 & 9 \end{pmatrix}
\end{equation}

**Notable Relationships:**

    1.
   \begin{equation}
   (A^T)^T = (A^*)^* = A
   \end{equation}

    2.
   \begin{equation}
   (A \pm B)^T = A^T \pm B^T
   \end{equation}

    3.
   \begin{equation}
   (cA)^T = c(A^T)
   \end{equation}

    4.
   \begin{equation}
   (AB)^T = B^TA^T
   \end{equation}

### Hermitian Adjoint

A Hermitian Adjoint, also called a conjugate transpose, is essentially a combination of the two operators presented above. Its transformation on matrix $M$ would result in the following change in its components: $M_{ij} \rightarrow M_{ji}^*$. A Hermitian Adjoint on $M$ is denoted by $M^\dagger$. Explicitly,

\begin{equation}
  M^\dagger = (M^T)^* \begin{pmatrix} M_{11}^* & M_{21}^* & \ldots & M_{n1}^* \\\ M_{12}^* & M_{22}^* & \ldots & M_{n2}^* \\\ \vdots & \vdots & \ddots &\vdots \\\ M_{1m}^* & M_{2m}^* & \ldots & M_{nm}^*\end{pmatrix}
   \end{equation}

It is obvious to deduce that $M^\dagger = M^T$ if $M \in R^{m \times n}$ (i.e is real).

**Example:**

\begin{equation}
\begin{pmatrix} 1 + i & 2 \\\ 3-2i & 4 \end{pmatrix}^\dagger = \begin{pmatrix} 1 - i & 3+2i \\\ 2 & 4 \end{pmatrix}
\end{equation}

Recall that one of the most crucial points of Dirac notation is that a ket's dual (i.e a bra) is essentially its Hermitian conjugate. In other words $|\psi>^\dagger = <\psi|$ and vice versa.

**Adjoint of $A$ in Dirac Notation:**

Let's say we have states $|\psi>$ and $\psi$ and operator $A$. The operator, $A$'s linear adjoint can be written as follows:

\begin{equation}
 = <\phi|A\psi>
\end{equation}

Note that according to the symmetry of an inner product, this could be rewritten as

\begin{equation}
<\psi|A^\dagger \phi>^* = <\phi|A\psi>
\end{equation}

By conjugating both sides, we get

\begin{equation}
<\psi|A^\dagger\phi> = (<\phi|A\psi>)^*
\end{equation}

Which, by changing the orders of bras and kets, gives us

\begin{equation}
<\psi|A^\dagger \phi> =
\end{equation}

Removing |\phi> from both sides, the following can be obtained as a result:

\begin{equation}
<|A^\dagger \equiv <A\psi|
\end{equation}

The right hand side is what is known as the adjoint of $A$. This is important because remember that a linear operator $A$ acting on a ket produces a new ket, so the effect it has on the ket's corresponding bra is equivalent to $<\psi|A^\dagger$.

**Notable Relationships:**

Using a combination of the notable relationships of the complex conjugate (presented in 1.2) and the notable relationships of the transpose (presented here in 2.5), one can derive the following:

    1.
   \begin{equation}
   (A^\dagger)^\dagger = A
   \end{equation}

    2.
   \begin{equation}
   (aA)^\dagger = a^*A^\dagger
   \end{equation}

    3.
   \begin{equation}
   (A + B)^\dagger = A^\dagger + B^\dagger
   \end{equation}

    4.
   \begin{equation}
   (AB)^\dagger = B^\dagger A^\dagger
   \end{equation}

    5.
   \begin{equation}
   (AB|\psi>)^\dagger = <\psi|B^\dagger A^\dagger
   \end{equation}

  ### Python Implementation

```python
#Basic Operation Applications

A = np.array([[1j, 2+3j,4], [1,-2, 1j], [-2,3,0]])

AStar = np.conj(A)
ATrans = np.transpose(A)

#Defining hermitian conjugate as conjugate transpose
def hermitian(A):
    return np.conj(np.transpose(A))

ADagger = hermitian(A)

print('A = ')
view(A)

print('')

print('A* = ')
view(AStar)

print('')

print('AT = ')
view(ATrans)

print('A† = ')
view(ADagger)

```

    A =

$\displaystyle \left[\begin{matrix}1.0 i & 2.0 + 3.0 i & 4.0\\1.0 & -2.0 & 1.0 i\\-2.0 & 3.0 & 0\end{matrix}\right]$

    A* =

$\displaystyle \left[\begin{matrix}- 1.0 i & 2.0 - 3.0 i & 4.0\\1.0 & -2.0 & - 1.0 i\\-2.0 & 3.0 & 0\end{matrix}\right]$

    AT =

$\displaystyle \left[\begin{matrix}1.0 i & 1.0 & -2.0\\2.0 + 3.0 i & -2.0 & 3.0\\4.0 & 1.0 i & 0\end{matrix}\right]$

    A† =

$\displaystyle \left[\begin{matrix}- 1.0 i & 1.0 & -2.0\\2.0 - 3.0 i & -2.0 & 3.0\\4.0 & - 1.0 i & 0\end{matrix}\right]$

```python
#Testing The Five Notable Relationships of Hermitian Adjoints on matrices A, B, constant a, and vectors |psi>, <psi|

A = np.array([[1, 1j, 3], [1+2j, 0, 0], [3j, -1j,0]])
B = np.array([[3,-2j, 1j],[-3, 0, 0], [3j, -2j +3, 1]])
a = 5j
psi_ket = np.array([3j, 0, 2])
psi_bra = hermitian(psi_ket)

print('Is (A†)† = A:')
print(hermitian(hermitian(A)).all() == A.all())

print('')

print('Is (aA)† = a*A†:')
print(hermitian(a*A).all() == np.conj(a)*hermitian(A).all())

print('')

print('Is (A + B)† = A† + B†:')
print(hermitian(A+B).all() == (hermitian(A) + hermitian(B)).all())

print('')

print('Is (AB)† = B†A†:')
print(hermitian(A@B).all() == (hermitian(B)@hermitian(A)).any())

print('')

print('Is (AB|psi>)† = <psi|B†A†:')
print(hermitian(A@B@psi_ket).all() == (psi_bra@hermitian(B)@hermitian(A)).all())

#Excersise: Build a function that returns True if a matrix is Hermitian by testing it against the 5 relationships above
```

    Is (A†)† = A:
    True

    Is (aA)† = a*A†:
    True

    Is (A + B)† = A† + B†:
    True

    Is (AB)† = B†A†:
    True

    Is (AB|psi>)† = <psi|B†A†:
    True

## 2.6 Symmetric Matrices

Symmetric Matrices are are any square matrices $M$ where $M = M^T$. The following is an example:

\begin{equation}
M = M^T = \begin{pmatrix} 1 & 2 & 3 \\\ 2 & 5 & 6 \\\ 3 & 6 & 2 \end{pmatrix}
\end{equation}

    1. The transpose of a symmetric matrix is also symmetric
    2. A scalar times a symmetric matrix is also symmetric
    3. The inverse of a symmetric matrix is equal to the inverse of the transpose

 \begin{equation}
 (A^T)^{-1} = A^{-1}
 \end{equation}

    4. If (A+B)T = AT + BT = A + B, then A + B is a symmetric matrix
    5. The product of two symmetric matrices is also symmetric if the matrices commute (AB = BA)

A lot of the operators that we are concerned with in quantum mechanics are symmetrical. Namely, the two types of symmetrical matrices that we will deal with are called Hermitian operators (Section 2.9) and Unitary operators (Section 2.10)

```python
#Defining a function that tests whether a matrix is symmetrical or not

def symmCheck(a):
    return a.all() == np.transpose(a).any()

#A matrix we'd expect to be symmetrical
M = np.array([[1,2,3],[2,4,6],[3,6,2]])

#A matrix we'd expect not to be
N = np.array([[33,-2,0],[4,5,6],[62,3,3]])

#Printing symmCheck

print('Is M Symmetrical? ', symmCheck(M))
print('Is N Symmetrical? ', symmCheck(N))

#Excercise: Create a function that generates a random matrix M, with a condition that M is symmetrical

```

    Is M Symmetrical?  True
    Is N Symmetrical?  False

## 2.7 Orthogonal Matrices

Another property that a square matrix might have is being orthogonal. A matrix $M$ is orthogonal given that its inverse is equal to its transpose. Notationally,

\begin{equation}
A^T = A^{-1}
\end{equation}

Given the definition of a matrix's inverse (as stated in 2.2), one can see that the statement above is equivelent to the following:

\begin{equation}
A^T A = I
\end{equation}

**Example:**

\begin{equation}
A =\begin{pmatrix} 1 & 0 \\\ 0 & -1 \end{pmatrix}
\end{equation}

\begin{equation}
A^T =\begin{pmatrix} -1 & 0 \\\ 0 & 1 \end{pmatrix}
\end{equation}
Multiplying those $A$ with its transpose,
\begin{equation}
AA^T = \begin{pmatrix} 1 & 0 \\\ 0 & -1 \end{pmatrix}\begin{pmatrix} -1 & 0 \\\ 0 & 1 \end{pmatrix} \newline =
\begin{pmatrix} 1 & 0 \\\ 0 & 1 \end{pmatrix} \newline= I
\end{equation}

Therefore, we can deduce that $A^T = A^{-1}$, and that $A$ is orthogonal.

**Properties:**

    1. Orthogonal matrices are always symmetrical
    2. The inner product of two columns or two rows are always zero (i.e orthogonal, See Section 1.6)
    3. The product of two orthogonal matrices is also an orthogonal matrix

## 2.8 Identity Operator

A reasonable summary of everything that has been described in this section so far is the following: An operator is anything that transforms a vector into another. A linear operator, i.e an operator that preserves linear combination, can be represented by a matrix.

As mentioned before, an identity matrix is an $n \times n$ orthogonal matrix that consists of 1s in its diagonal and 0s elsewhere,(i.e $I_{ij} = \delta_{ij}$). When multiplied with another matrix $A$, the Identity matrix produces the same matrix $A$.

It can also be viewed as an operator that follows the same rules. Namely,

\begin{equation}
I_n|u> = |u>
\end{equation}

Where |u> is a vector, $n$ is the number of dimensions of the vector, and $I_n$ is an $n \times n$ Identity matrix. In other words, the Identity operator acting on a vector simply produces the same vector.

### Python Implementation

```python
#Identity operator function

def IOperator(vec):
    I = np.identity(len(vec))
    return I@vec

IOperator(np.array([1,2,3]))

#Excercise: Write code to show that the identity matrix is orthogonal, as I stated in 2.8

```

    array([1., 2., 3.])

## 2.9 Hermitian Operator

Section 2.3 started by explaining that the reason we're learning about operators is because they have very important applications to quantum mechanics. This is mainly because in QM, physical quantities which are usually described as functions of time now become linear operators, or more specifically, Hermitian operators. This is what is known as the third postulate of quantum mechanics. For instance, one may have heard of the Schrodinger Equation, which plays a key roll to modeling and understanding quantum systems as it can be solved to obtain the system's wavefunction ($\psi(x) \equiv <x|\psi>$). What one might not know is that it is essentially just an **eigienvector-eigenvalue** relationship of a Hermitian Operator called the Hamiltonian (more on that in chapter 3).

The topic of Hermitian operators and eigeinvalues will be revisited in Chapter 3, but for now, only a few basic properties will be stated. A Hermitian operator is represented by any matrix that is equivilent to its Hermitian Adjoint, or

\begin{equation}
H = H^{\dagger}
\end{equation}

where $H$ is a Hermitian operator.

One can realize that in a real matrix, a Hermitian operator is equivilent to Symmetric Operators described in 2.6. Also, an important, but simple thing to realize is that for a matrix to be Hermitian,

    1. Its diagonal must consist of real numbers
    2. H_ij = H_ji*

 **Example:**

 \begin{equation}
 H = \begin{pmatrix} 1 & 1 + i & 3 + 2i \\\ 1 - i & 5 & -i \\\ 3 - 2i & i & 7\end{pmatrix}
 \end{equation}

 (Look at Python Implementation for showing that this is Hermitian)

As you can see, the diagonal is real and the image of $H_{ij}$ , $H_{ji}$ is its complex conjugate. For example,

\begin{equation}
H_{12} = 1 + i, H_{21} = 1 - i \newline \dot{.\hspace{.095in}.}\hspace{.5in}H_{12} = H_{21}^*
\end{equation}

### Python Implementation

```python
#Defining the H used previously

H = np.array([[1,1+1j,3+2j],[1-1j,5,-1j],[3-2j,1j,7]])
HDagger = hermitian(H)

isHermitian = H.all() == HDagger.all()
HermOnVec = HDagger@np.array([1,1,1])

print('Hermitian Matrix, H = ')
view(H)

print('')

print('Hermitian Checker: ')
print(isHermitian)

print('')

print('H|u>, where |u> = [1,1,1]:')
view(HermOnVec)
```

    Hermitian Matrix, H =

$\displaystyle \left[\begin{matrix}1.0 & 1.0 + 1.0 i & 3.0 + 2.0 i\\1.0 - 1.0 i & 5.0 & - 1.0 i\\3.0 - 2.0 i & 1.0 i & 7.0\end{matrix}\right]$

    Hermitian Checker:
    True

    H|u>, where |u> = [1,1,1]:

$\displaystyle \left[\begin{matrix}5.0 + 3.0 i\\6.0 - 2.0 i\\10.0 - 1.0 i\end{matrix}\right]$

## 2.10 Unitary Operators

Another important type of operator in quantum mechanics is what is called a unitary operator. A key feature of a unitary operator, $U$, is that when acted on a vector, it produces another vector with the same norm, or

\begin{equation}
U|v_1> = |v_2>
\end{equation}

where $<v_1|v_1> = <v_2|v_2>$. This is extremely crucial because recall that in Section 1.10, we stated that vectors in quantum mechanics and quantum information theory represent amplitudes that when squared, give the probabilities of obtaining a specific state. Meaning that the norm (i.e the sum of all probabilities) must always equal 1, no matter how the statevector evolves. This being given, one should note that any operator which evolves (i.e updates) the statevector should be unitary.

A unitary matrix is essentially any matrix $U$, where the following condition applies:

\begin{equation}
UU^\dagger = I
\end{equation}

In other words, $U^\dagger = U^{-1}$. Similarly to how a Hermition matrix is an complex extension to a symmetric matrix, a Unitary matrix is a complex extension to an orthogonal matrix.

**Example:**

The following is an example of a unitary matrix:

\begin{equation}
U = \frac{1}{2}\begin{pmatrix} 1 + i & 1 - i \\\ 1 - i & 1 + i \end{pmatrix}
\end{equation}

This can be proved by multiplying $U$ with $U^\dagger$, as done below:

\begin{equation}
U = \frac{1}{4}\begin{pmatrix} 1 + i & 1 - i \\\ 1 - i & 1 + i \end{pmatrix} \begin{pmatrix} 1 - i & 1 + i \\\ 1 + i & 1 - i \end{pmatrix} \newline = \begin{pmatrix} 1 & 0 \\\ 0 & 1 \end{pmatrix} \newline = I
\end{equation}

I will not do the full matrix multiplication but rather will leave it to you, and will also use it as an example in the Python Implementation.

**Inner Product Preservation:**

I previously said that a unitary operator is special because once applying on a vector, its norm stays the same. While this is true, it is not a general fact and is simply a concequence of the fact that a unitary acted on two vectors will not change their inner product (and the norm invariance is simply a case where the two vectors are the same). Inner product preservation can be derived as follows:

Say we have vectors $|u>$ and $|v>$ that are both operated by U to produce $|\psi>$ and $\phi$, respectively. Or notationally,

\begin{equation}
|\psi> = U|u>
\end{equation}
and
\begin{equation}
|\phi> = U|v>
\end{equation}

Now let's say we want to get the following inner-product:

\begin{equation}
<\psi|\phi>
\end{equation}

Notice that according to basic bra-ket notation, this is equivelent to

\begin{equation}

\end{equation}

or
\begin{equation}

\end{equation}

which, by the definition of a unitary matrix $U$, reduces down to

\begin{equation}

\end{equation}

We essentially just derived that $<\psi|\phi> = $, or that the inner product is equal before and after the unitary transformation.

### Python Implementation

```python
#Example given above:

#Defining U and U†

U = 0.5*np.array([[1+1j, 1-1j],[1-1j, 1+1j]])
UDagger = hermitian(U)

#2x2 Identity matrix
I = np.identity(2)

#|v> and |psi> = U|v>
v = np.array([1,2])
psi = U@v

#Finding the norm of a vectors |v> and |psi>
normV = np.linalg.norm(v)
normPsi = np.linalg.norm(psi)

#UU† (Supposed to be I_2)

print('UU† = ')
view(U@UDagger)

print('')

#<v|v> vs <psi|psi>

print('Is <v|v> = <psi|psi>:')
normV == normPsi
```

    UU† =

$\displaystyle \left[\begin{matrix}1.0 & 0\\0 & 1.0\end{matrix}\right]$

    Is <v|v> = <psi|psi>:

    True

```python
#Application to Quantum Computing

#Defining |0>
zero = np.array([1,0])

#Defining a unitary matrix H, which stands for Hadamard (more info in chapter 4)
H = 1/np.sqrt(2)*np.array([[1,1],[1,-1]])

print('Hadamard Matrix= ')
view(H)

print('')

#Let's define a state vector, which is initialized at |0> (i.e 100% probability of a qubit being 0)

statevector = zero

print('Statevector before Hadamard:', statevector)

statevector = H@zero

print('Statevector after Hadamard:', statevector)

print('')
```

    Hadamard Matrix=

$\displaystyle \left[\begin{matrix}0.707106781186547 & 0.707106781186547\\0.707106781186547 & -0.707106781186547\end{matrix}\right]$

    Statevector before Hadamard: [1 0]
    Statevector after Hadamard: [0.70710678 0.70710678]

#### Interpretation:

                                    **WARNING!! MAY REQUIRE REFRESHER FROM SECTION 1.10**

This is a very crucial application to quantum computing which arguably summarizes everything we have learned so far!

We essentially took a qubit's statevector which is initialized at

\begin{equation}
|\psi> = |0> = \begin{pmatrix}1 \\\ 0 \end{pmatrix}
\end{equation}

A unitary operator $U$ was then applied on the qubit's state which evolved it into the following state

\begin{equation}
|\psi> = \begin{pmatrix}0.7071067 \\\ 0.7071067 \end{pmatrix} \newline = 0.7071067|0> + 0.7071067|1>
\end{equation}

Remember that the two terms in the statevector represent $\alpha$ and $\beta$. Also remember that $\alpha^2$ and $\beta^2$ represent the probabilities of obtaining a 0 and 1, respectively.

This means that before applying unitary operator $H$:

\begin{equation}
\alpha^2 = 1 \newline
\beta^2 = 0
\end{equation}

And after applying unitary operator $H$:

\begin{equation}
\alpha^2 = 0.7071067^2 = 0.5 \newline
\beta^2 = 0.7071067^2 = 0.5
\end{equation}

In other words, the qubit went from having 100% chance to be measured at 0 to a 50/50 percent chance to be a 0 or 1. Physically, what has happened is that we changed the probability of the qubit ending up in either the two possible states it can be in. And this is not the equivilent to a coin flip because a the result of a coin flip is based on how much force was applied on the coin and can be predicted every time using Newton's Laws and a measurement device that is accurate enough, whilst in this case we are exploiting the intrinsically random nature of the universe to set up a qubit state to do what we want!

Friendly advice: If you don't find this fascinating or mind-blowing, you might want to reconsider quantum computing as a whole.

## 2.11 Eigenvalues

One of the key takeaways of this chapter is that a matrix can be used as a "function" (more specifically, operator) in the sense where it can produce an output vector from an input one (by means of matrix multiplication). The topic of eigenvalues and eigenvectors arise when we consider only the linear transformations where the input and output vectors are in the same direction, as shown in the following figure:

Here, you can see that the only difference between $\vec{A}$ and $\vec{B}$ is the length, or in other words

\begin{equation}
\vec{B} = \lambda \vec{A}
\end{equation}

where $\lambda$ is a scalar that represents the ratio bewtween the length of $\vec{A}$ and the length of $\vec{B}$.

Now let us say we want to transform $\vec{A}$ to $\vec{B}$, which as described in Section 2.4, can be written out as follows:

\begin{equation}
O\vec{A} = \vec{B}
\end{equation}

where $O$ is a matrix.

But remember that $\vec{B}$ is simply a scalar product of $\vec{A}$, which means that we can write the linear transformation as

\begin{equation}
O\vec{A} = \lambda \vec{A}
\end{equation}

In other words, the effect of multiplying a vector by matrix $O$ is the same as multiplying it by constant $\lambda$. In such a case, $\lambda$ is what is called the eigenvalue of $O$.

Let's say we have vector $|\psi>$ and and matrix $M$, where the following relationship applies:

\begin{equation}
M|\psi> = \lambda |\psi>
\end{equation}

A key thing to notice is that the relationship above can be re-written as

\begin{equation}
(M - \lambda I_n)|\psi> = 0
\end{equation}

where $I_n$ is an $n \times n$ identity matrix.

This could be used to find $\lambda$ because $M - I_n \lambda$ is a matrix that, by definition, must have a determinant of 0. Using the fact that $det(M - I_n \lambda) = 0$, one can get a polynomial of degree $n$, where its roots are the eigeinvalues of $M$, as done here in a 3x3 case. Using the fundamental theorem of algebra, we can deduce that an $n \times n$ matrix will have $n$ eigeinvalues.

Another important relationship is that the sum of all eigenvalues is equal to the trace of $M$. Or,

\begin{equation}
\sum_{i = 1}^n \lambda_i = Tr(M)
\end{equation}

### Eigenvalue of Unitary Matrix

In Section 2.10, a specific type of square matrix was introduced, a Unitary matrix, which preserves the length of a vector when operated on it. It was also said that in quantum mechanics, all operators must be unitary because they must preserve the length of the statevector it acts on (which must be 1, as it represents the total probability).

The eigenvalue of a unitary matrix must always have a norm of 1. If $U \in R^{n \times n}$, then it's eigenvalue will be $\pm$ 1, but in the case of a complex matrix (i.e $U \in C^{n \times n}$), then its eigenvalues could be any value which lies in the unit circle in an argand diagram. This property is used in an imporant quantum algorithm called the Phase Estimation Algorithm (PEA) which will be discussed in a few chapters.

### Python Implementation

```python
#Finding the eigenvalues of 3x3 matrix M

M = np.array([[1,2,3],[1,5,0],[0,1,1]])
lambda_M = np.linalg.eigvals(M)

print('M = ')
view(M)

print('')
print('M has 3 eigenvalues:')

print(lambda_M)

print('')

#Checking to see whether the sum of eigenvalues is equal to the trace

if lambda_M.sum() == M.trace():
    print('Math works! Sum of eig(M) = Tr(M)')
else:
    print('Whoops.  math doesnt work... nothing does')

#Excercise: Construct a function that outputs the same result as np.linalg.eigvals. Use the method laid out above or
#any numerical method of your choice
```

    M =

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\1 & 5 & 0\\0 & 1 & 1\end{matrix}\right]$

    M has 3 eigenvalues:
    [5.5797401 +0.j         0.71012995+0.75566815j 0.71012995-0.75566815j]

    Math works! Sum of eig(M) = Tr(M)

## 2.12 Tensor Products

### Kronecker Product vs. Outer Product
Another important mathematical concept that one would very likely find themselves using in quantum computing and quantum information is the Tensor Product. Whilst the name may sound foreign to you, it is actually something that has already been discussed in this series. Specifically in Section 1.7, where the outer product was discussed. It was said that if one were to get the ket-bra of two vectors $|u>$ and $|v>$, say, both $\in R^3$, the following matrix would be the outcome:

   \begin{equation}
  |u><v| = \begin{pmatrix}v_1^*u_1 & v_2^*u_1 & v_3^*u_1 \\\ v_1^*u_2 & v_2^*u_2 & v_3^*u_2 \\\ v_1^*u_3 & v_2^*u_3& v_n^*u_3\end{pmatrix}
   \end{equation}

This is called the density matrix and it is result of the outer tensor product (also denoted by $\vec{u} \otimes_{outer} \vec{v}$). However, there is another type of tensor product called the Kronecker Product (denoted by $\vec{u} \otimes_{kron} \vec{v}$ or simply, $\vec{u} \otimes \vec{v}$), which is equivilent to the result obtained in the density matrix, but packaged in a different shape. When the word tensor product is used, it is most likely referring to the kronecker product

The outer product inputs two vectors of dimensions $n$ and $m$ and outputs an $n \times m$ matrix. The kronecker product, on the other hand, inputs two vectors of dimensions $n$ and $m$, and produces a new vector of dimensions $nm$.

### Calculating Tensor Products

Like many topics in linear algebra, one would find that tensor products are much easier to compute than to describe formally. For this reason, I will take a more mechanical and practical approach to explaining how to evaluate a tensor product, as its crude utilization is the only thing that will be necessary in this series.

To get the tensor product of vectors $\vec{u}$ and $\vec{v}$ ($\vec{u} \in R^n$ and $\vec{v} \in R^m$), all you have to do is scalar multiply the first component of $\vec{u}$ with vector $\vec{v}$, and repeat the process for all components of $\vec{u}$. You'll find that you have now made $n$ new vectors. Merge them. The result is the $\vec{u} \otimes \vec{v}$. In the case that the vectors are complex, do the same but with the complex conjugates of $u_i$.

If that description is not adequate enough for you to understand, hopefully this example with 2 and 3 dimensional complex vectors will work:

\begin{equation}
\vec{u} \otimes \vec{v} = \begin{pmatrix} u_1^* \\\ u_2^*\end{pmatrix} \otimes \begin{pmatrix} v_1 \\\ v_2 \\\ v_3\end{pmatrix} \newline  = \begin{pmatrix} u_1^* \begin{bmatrix} v_1 \\\ v_2 \\\ v_3\end{bmatrix} \\\ u_2^* \begin{bmatrix} v_1 \\\ v_2 \\\ v_3 \end{bmatrix} \end{pmatrix} = \begin{pmatrix} u_1^* v_1 \\\ u_1^* v_2 \\\ u_1^* v_3 \\\ u_2^* v_1 \\\ u_2^* v_2  \\\ u_2^* v_3\end{pmatrix}
\end{equation}

One can find that this is essentially stacking up the terms columns of the density matrix into one.

**Example:**

  The result is a 6-vector.

### Using Tensor Products to Extend to Multiple Qubit Systems

In Section 1.10, it was stated that one of the key differences between describing classical and quantum information is that classical information is described by a series of 1s and 0s, whilst quantum information is described by a vector which describes the probability of obtaining any possible series of 1s or 0s.

The basic unit of this quantum logic is called a qubit which is a linear combination of two basis vectors which represent the possible states: $|0>$ and $|1>$, where $|0> = \begin{pmatrix} 1 \\\ 0\end{pmatrix}$ and $|1> = \begin{pmatrix} 0 \\\ 1\end{pmatrix}$.

However, one cannot go very far with one qubit and nearly all quantum algorithms require multiple qubits, and this is where the tensor product comes in. Suppose one would want to describe the $|10>$ state (which is 2 in binary). To do this, one would have to take the tensor product of the two qubit states which make up |10>, |1> and |0>. So

\begin{equation}
|10> = |1> \otimes |0> \newline = \begin{pmatrix} 0 \\\ 1 \end{pmatrix} \otimes \begin{pmatrix} 1 \\\ 0 \end{pmatrix} \newline = \begin{pmatrix} 0 \begin{bmatrix} 1 \\\ 0\end{bmatrix} \\\ 1 \begin{bmatrix} 1 \\\ 0\end{bmatrix} \end{pmatrix} \newline = \begin{pmatrix}  0 \\\ 0 \\\ 1 \\\ 0 \end{pmatrix}
\end{equation}

The 4-dimensional statevector obtained above represents the basis vector |01>, where as you can see, all terms are zero except for the third component, which is 1. Generally, the result will be a $2^n$ dimensional statevector, where $n$ is the number of qubits and all the terms are zero except for the $i + 1$th index, where $i$ is the number in decimal.

Note to Adam: Rewrite this completely!!!

```python
#Tensor Product

u = np.array([1,2,3])
v = np.array([4,5])

outerProd = np.outer(u,v)
kronProd = np.kron(u,v)

print('u = [1 2 3], v = [4 5]')
print('u ⊗ v = ', kronProd)
print('|u><v| = ')
view(outerProd)

#Excercise: Create a function that does the same as np.kron() (Hint: Two for loops and np.append())
```

    u = [1 2 3], v = [4 5]
    u ⊗ v =  [ 4  5  8 10 12 15]
    |u><v| =

$\displaystyle \left[\begin{matrix}4 & 5\\8 & 10\\12 & 15\end{matrix}\right]$

```python
# Go nuts here:

```

Chapter 2 - Math Tools II (Matrices and Operators).ipynb