Load up the sympy python package to do symbolic mathematics

```
from IPython.display import display

from sympy.interactive import printing
printing.init_printing(use_latex=True)

from __future__ import division
import sympy as sym
from sympy import *

import matplotlib
matplotlib.rc('font',**{'family':'serif'})
```

# Band structure calculation for a 1D Lattice

We start by defining the function that calculates the truncated hamiltonian matrix.  I call this Hamiltonian, but it is really the set of linear equations for the coefficients that describe the wave functions in terms of plane waves.   We use the sympy package to define the matrix in terms of symbolic quantities.  This allows directly getting the latex output for the matrix, which comes in handy when including it in a document.

```
def H0_sin( Npoints, q, v0 ):
    def f(i,j):
        if i==j:
            return 4*(q+(i-Npoints))**2 + v0/2
        elif i-j == 1:
            return -v0/4
        elif j-i == 1:
            return -v0/4
        else:
            return 0.
    mdim = (2*Npoints+1)
    matrix = sym.Matrix(mdim,mdim,f)
    return matrix
q, v0 = sym.symbols("q V_{0}")
H0_sin(2,q,v0)
```

With this matrix in hand we can diagonalize it and get the eigenvalues, the eigenvalues of this matrix correspond to the energy for the given quasimomentum, lattice depth, and band index.  We define functions to diagonalize the matrix.  At the same time we define functions that calculate the energy of the harmonic oscillator states in each lattice site, when the siteis considered as an independent harmonic oscillator potential.

##Energy eigenvalues in 1D

```
def eigenvalsH0( q, v0,  NPoints=5):
    r"""Calculate energy in 1D lattice

    Given the quasimomentum and the lattice depth, this function
    will calculate the set of energy eigenvalues for a number of
    bands determined by NPoints.   The calculation is for a 1D
    lattice.

    Parameters
    ----------
    q : float
        The value of quasimomentum.  Units are 2\pi, so q can take
        values in (-0.5,0.5]
    v0 : float
        The value of the lattice depth in recoils
    NPoints : float, optional
        Plane waves e^{imGx} will be considered in the expansion of
        the Bloch states such that their momentum satisfies
        |m|<NPoints.  Here G is the unit reciprocal lattice vector
        of the 1D lattice:  G = 2*pi / a

        A total of 2*NPoints + 1 plane waves is used.

    Returns
    -------
    array
        An array with the sorted energy eigenvalues.  When they are
        sorted the index of the array corresponds to the band index.

    Other Parameters
    ----------------

    Raises
    ------

    See Also
    --------
    bandH0

    Notes
    -----

    References
    ----------
    For more information see [1]_.

    .. [1] O. Morsch and M. Oberthaler, “Dynamics of Bose-Einstein
       condensates in optical lattices,” Rev. Mod. Phys. 78, 179–215
       (2006).

    Examples
    --------
    >>> q=0.
    >>> v0=7.0
    >>> print eigenvalsH0(q, v0)

    """
    q_Sym, v0_Sym = sym.symbols("q V_{0}")
    h0 = np.array( np.array( H0_sin(NPoints,q_Sym,v0_Sym).subs( [(q_Sym,q), (v0_Sym,v0)] )), np.float )
    return np.sort( np.linalg.eigvalsh(h0) )

def bandH0( NBand, qs, v0, NPoints=5):
    r"""Version of eigenvalsH0 that works with an array input for q"""
    Eq = np.array( [ eigenvalsH0( q, v0, NPoints=NPoints)[NBand] for q in qs] )
    return Eq

def hosc( NBand, v0 ):
    r"""Energy of the NBand^th h.o. state in a lattice with depth v0"""
    return 2* np.sqrt(  v0 ) * (NBand + 0.5)
```

We then make a plot of the band structure,  energy as a function of quasimomentum

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

depths = [0., 4., 8., 12., 16.]
qs = np.linspace(-1,1,40)

fs = 12
fig = plt.figure(figsize=(2.16*len(depths),3.6))
gs = matplotlib.gridspec.GridSpec( 1,len(depths))
for i,d in enumerate(depths):
    ax = fig.add_subplot( gs[0,i])
    ax.set_xlabel("$q / (2\pi/a)$",fontsize=fs)
    ax.set_ylabel("$E_{q} / E_{r} $",fontsize=fs)
    ax.set_title('$%.1f E_{r}$'%d,fontsize=fs)
    ax.set_xlim(-0.5,0.5)
    ax.set_ylim(0.,16.)
    ax.fill_between(qs, 0., d, facecolor='gray', alpha=0.3)
    ax.plot( qs, hosc(0,d)*np.ones_like(qs), 'b:' )

    qs = np.linspace(-0.5,0.5,101)
    ax.plot( qs, bandH0( 0, qs, d, NPoints=5))
    ax.plot( qs, bandH0( 1, qs, d, NPoints=5))
    ax.plot( qs, bandH0( 2, qs, d, NPoints=5))

gs.tight_layout(fig, rect=[0.,0.,1.,1.0])
fig.savefig('BandStructure_figures/bands1d.png',dpi=300)
fig.savefig('BandStructure_figures/bands1d.pdf')
```

We define a way to calculate the effective mass of the system,  to do this we fit the behavior of $E(q)$ near $q=0$

```
depth = 40.
qs = np.linspace(-0.2,0.2,40)

fig = plt.figure(figsize=(5,5))
gs = matplotlib.gridspec.GridSpec( 1,1)

ax = fig.add_subplot( gs[0,0])
ax.set_xlabel("$q / (2\pi/a)$",fontsize=16)
ax.set_ylabel("$E_{q} / E_{R} $",fontsize=16)
ax.set_title('$%.1f E_{R}$'%depth,fontsize=16)
#ax.set_xlim(-0.1,0.1)
#ax.set_ylim(1.,2.)
es = bandH0( 0, qs, depth, NPoints=5)
ax.plot( qs, es)

z = np.polyfit( qs, es, 2)
print "Effective mass at %.3g Er = %.3g m " %( depth, 4./z[0])
p = np.poly1d(z)
ax.plot( qs, p(qs))

gs.tight_layout(fig, rect=[0.,0.,1.,1.0])
#fig.savefig('BandStructure_figures/bands1d.png',dpi=240)
#fig.savefig('BandStructure_figures/bands1d.pdf')
#plt.close()
```

We save some interpolation data to easily access the effective mass as a function of lattice depth

```
def effective_mass( v0):
    qs = np.linspace( -0.2, 0.2, 40 )
    es = bandH0( 0, qs, v0, NPoints=5)
    z = np.polyfit( qs, es, 2 )
    return 4./z[0]

v0s = np.linspace(0.001, 20., 120)
ms  = np.empty_like(v0s)
for ii,v0 in enumerate(v0s):
    ms[ii] = effective_mass(v0)

np.savetxt('banddat/interpdat_effective_mass.dat', np.column_stack((v0s,ms)))

```

We define a function that allows us to calculate the band width for a certain lattice depth

```
def bands1d( v0, NBand=0 ):
    r""" Returns an array of shape (2,) which has the bottom and top of the band energies

    Parameters
    ----------
    v0 : float
        Depth of the lattice in recoils
    NBand : optional
        Band index (default = 0 )
    """
    qbot = 0.
    qtop = 0.5
    Eq0 = eigenvalsH0( qbot, v0)[NBand]
    Eq1 = eigenvalsH0( qtop, v0)[NBand]
    if Eq0 > Eq1:
        return np.array([Eq1,Eq0])
    else:
        return np.array([Eq0,Eq1])

def bands1dvec( v0, NBand=0):
    r""" Returns an array of shape (len(v0,2) which has the bottom and top of the band energies

    Parameters
    ----------
    v0 : array-like
        Array of lattice depths. Depths are in recoils
    NBand : optional
        Band index (default = 0 )
    """
    band = numpy.empty( (len(v0), 2) )
    for i,v in enumerate(v0):
        band[i] = bands1d(v, NBand)
    return band
```

And we proceed to make a plot of the lattice band structure as a function of lattice depth

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

fs = 12
fig = plt.figure(figsize=(4.4,2.6))
gs = matplotlib.gridspec.GridSpec( 1,1, left=0.11, right=0.74, bottom=0.16, top=0.96)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=fs)
    ax.grid(alpha=0.5)
    ax.set_ylim(0.,15.)
    ax.set_xlim(0.,20.)

ax0.set_ylabel("$E_{R}$",fontsize=fs,rotation=0, labelpad=10)

v0 = np.linspace(0.1,20,80.)
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
for i in range(3):
    Eb = bands1dvec(v0,i)
    ax.fill_between( v0 , Eb[:,0], Eb[:,1], \
          color=c[i], facecolor=c[i], alpha=0.7, zorder=4)
    ax.plot( v0, Eb[:,1], color=c[i])
    ax.plot( v0, Eb[:,0], color=c[i],\
          label="Band = %d"%i)
    if i==2: labelstr = 'h.osc.'
    else: labelstr = None
    ax.plot( v0, hosc(i,v0), color='black', label=labelstr, zorder=1 )

ax0.legend( bbox_to_anchor=(1.03,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.8,1.0])
fig.savefig('BandStructure_figures/bands1d_V0.png',dpi=240)
fig.savefig('BandStructure_figures/bands1d_V0.pdf')

```

we then make plots to compare the band structure in the first quantized version and second quantized version of the Hamiltonian

```
from matplotlib import rc
rc('font', **{'family':'serif'})
rc('text', usetex=True)

fig = plt.figure(figsize=(3.24,2.7))
gs = matplotlib.gridspec.GridSpec( 1,1, left=0.16, right=0.95, bottom=0.17, top=0.94)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=12)
    ax.grid()
    ax.set_ylim(0.,10.)
    ax.set_xlim(0.,10.)

ax0.set_ylabel("$E_{R}$",fontsize=12,rotation=0)

v0 = np.linspace(0.1,10,80.)
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
for i in range(2):
    Eb = bands1dvec(v0,i)
    ax.fill_between( v0 , Eb[:,0], Eb[:,1], \
          color=c[i], facecolor=c[i], alpha=0.5, zorder=4)
    ax.plot( v0, Eb[:,1], color=c[i], lw=2.)
    ax.plot( v0, Eb[:,0], color=c[i], lw=2.,\
          label="Band = %d"%i)
    if i==1: labelstr = 'h.osc.'
    else: labelstr = None
    ax.plot( v0, hosc(i,v0), color='black', label=labelstr, zorder=1 )

ax0.legend( bbox_to_anchor=(0.,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':8}, handlelength=1.1, handletextpad=0.5 )

fig.savefig('BandStructure_figures/bands1d_V0_firstquant.png',dpi=300)
fig.savefig('BandStructure_figures/bands1d_V0_firstquant.pdf')

```

```
fig = plt.figure(figsize=(3.24,2.7))
gs = matplotlib.gridspec.GridSpec( 1,1, left=0.16, right=0.95, bottom=0.17, top=0.94)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=12)
    ax.grid()
    ax.set_ylim(-1.,6.)
    ax.set_xlim(0.,10.)

ax0.set_ylabel("$E_{R}$",fontsize=12,rotation=0)

v0 = np.linspace(0.1,10,80.)
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']

Eb = bands1dvec(v0,0)
E0 = (Eb[:,0] + Eb[:,1])/2

for i in range(2):
    Eb = bands1dvec(v0,i)
    ax.fill_between( v0 , Eb[:,0]-E0, Eb[:,1]-E0, \
          color=c[i], facecolor=c[i], alpha=0.5, zorder=4)
    ax.plot( v0, Eb[:,1]-E0, color=c[i], lw=2.)
    ax.plot( v0, Eb[:,0]-E0, color=c[i], lw=2.,\
          label="Band = %d"%i)
    if i==1: labelstr = 'h.osc.'
    else: labelstr = None
    ax.plot( v0, hosc(i,v0)-E0, color='black', label=labelstr, zorder=1 )

#ax0.legend( bbox_to_anchor=(1.03,1.00), \
#            loc='upper left', numpoints=1, \
#             prop={'size':12}, handlelength=1.1, handletextpad=0.5 )

fig.savefig('BandStructure_figures/bands1d_V0_secondquant.png',dpi=240)
fig.savefig('BandStructure_figures/bands1d_V0_secondquant.pdf')
```

## Energy eigenstates for a 1D lattice

To calculate the eigenstates in 1D we diagonlize the coefficient matrix and use the set of coefficient eigenvectors to build up the eigentates as a sum of plane waves

### We can build up the eigenstates using arrays:

```
def eigenfunsH0( q, v0, x, NPoints=5 ):
    r"""Calculate eigenfunctions in 1D lattice

    Given the quasimomentum and the lattice depth, this function
    will calculate the set of energy eigenfunctions for a number of
    bands determined by NPoints.   The calculation is for a 1D
    lattice.  The eigenfunctions will be returned as an array,
    evaluated at points given in the parameter x.

    Parameters
    ----------
    q : float
        The value of quasimomentum.  Units are 2\pi, so q can take
        values in (-0.5,0.5]
    v0 : float
        The value of the lattice depth in recoils
    x : array-like of floats
        The values of x at which the eigenfunctions will be evaluated
    NPoints : float, optional
        Plane waves e^{imGx} will be considered in the expansion of
        the Bloch states such that their momentum satisfies
        |m|<NPoints.  Here G is the unit reciprocal lattice vector
        of the 1D lattice:  G = 2*pi / a

        A total of 2*NPoints + 1 plane waves is used.

    Returns
    -------
    psi_bands : list
        A list with the eigenfuctions sorted by band index.  Each
        element of the list is an ndarray with length equal to len(x)
        len(psi_bands) will be equal to 2*NPoints + 1

    Notes
    -----
    The eigenfunctions are multiplied by an overall phase factor such
    that they have matching phases at x0, where x0 should be the position
    of a lattice site.  This function has a value of x0=0.5 hard-coded
    """
    q_Sym, v0_Sym = sym.symbols("q V_{0}")
    h0 = np.array( np.array( H0_sin(NPoints,q_Sym,v0_Sym).subs( [(q_Sym,q), (v0_Sym,v0)] )), np.float )
    w,v =  numpy.linalg.eigh(h0)
    a = 1.0 # We use x in units of the lattice spacing
    k = np.pi/a

    # m are the indices for the reciprocal lattice vectors
    m = np.linspace(-NPoints,NPoints,2*NPoints+1)

    # The sum of plane waves is carried out for each of the bands
    psi_bands = []
    for i in np.argsort(w):
        coefs = v[:,i][:,None]
        #if i==0:
        #    print coefs
        planewaves =  coefs * np.exp( 1j*(2*k)*np.outer(m,x) ) * np.exp(1j*q*2*k*x)
        psi = planewaves.sum(axis=0)

        # Adjust the wavefunctions so they have matching phases at x0
        # where x0 is the position of a lattice site.
        #
        # This will turn out to be important when defining the Wannier
        # wavefunctions.
        #
        x0 = 0.5
        phase = -1*np.exp( 1j* np.angle( np.sum(coefs * np.exp( 1j*(2*k)*m*x0 ) * np.exp(1j*q*k*x0))))

        psi = psi / phase
        psi_bands.append((psi,w[i]))
    return psi_bands

```

We can use the numeric version of the eigenfunctions to plot them for various lattice dephts

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.7
plt.rcParams['patch.linewidth'] = 0.4

depths = [4., 8., 12., 16.]
c = ['blue', 'green','red','black','purple']

fs=12
fig = plt.figure(figsize=(7.0,3))
gs = matplotlib.gridspec.GridSpec( 1,2,\
        left=0.10, right=0.86, bottom=0.14, top=0.93,
        hspace=0., wspace=0.40)

ax0 = fig.add_subplot( gs[0,0])
ax1 = fig.add_subplot( gs[0,1])

for ax in [ax0,ax1]:
    ax.set_xlabel("$x/a$",fontsize=fs)
    ax.set_ylabel("$|\psi_{q}^{n}(x)|^{2}$",fontsize=fs, rotation=90, labelpad=8)
    ax.set_xticks([-2, -1, 0, 1, 2])
    ax.grid(alpha=0.5)
ax0.set_title('$q = 0 $',fontsize=fs)
ax1.set_title('$q = \pi/a$',fontsize=fs)

qbot = 0.
qtop = 0.5

x = np.linspace(-2,2,200)
for i,d in enumerate(depths):
    psi0 = eigenfunsH0( qbot, d, x )[0]
    psi1 = eigenfunsH0( qtop, d, x )[0]
    ax0.plot( x, np.abs(psi0[0])**2, color = c[i] )
    ax1.plot( x, np.abs(psi1[0])**2, color = c[i], label='$%.1d E_{R}$'%d)

ax0.fill_between( x, 0., np.sin(np.pi*x)**2, facecolor='gray', alpha=0.3)
ax1.fill_between( x, 0., np.sin(np.pi*x)**2, facecolor='gray', alpha=0.3)

ax1.legend( bbox_to_anchor=(1.05,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.9,1.0])
fig.savefig('BandStructure_figures/eigenfuns1d.png',dpi=240)
fig.savefig('BandStructure_figures/eigenfuns1d.pdf')

```

# Calculation of Wannier states

We will consider the quasimomentun in the first Brilloun zone as $q \in [0,2\pi)\  $.   Numerically, we will consider a lattice with $L\ $ sites and quasimomentum values than run from $0\ $ to $(L-1)/L\ $.   In other words, we define the quasimomentum q without the factor of $2\pi\ $,  so we need to be careful to write the factor down when it is necessary.

The expansion of the eigenstates of the problem in terms of plane waves is given by

$$ \psi_{q}^{n}(x)  = \sum_{m \in \mathbb{Z} } c_{qm}^{n} e^{im2\pi x} $$

and the Wannier functions are defined in terms of the eigenstates as

$$ w_{j}^{n} = \frac{1}{L} \sum_{q} e^{-iqx_{j}} \psi_{q}^{n}(x) $$

Putting these two definitions together we arrive at

(see for instance http://alps.comp-phys.org/mediawiki/index.php/Documentation:Bosons_in_an_optical_lattice)

$$ w^{n} = \frac{1}{L} \sum_{m \in \mathbb{Z}} \sum_{q} c_{qm}^{n} e^{ix(mG+q)} $$

Here $G=2\pi\ $ , and since the units of $q$ are also $2\pi\ $, as was stated above, we can write this as

$$ w^{n} = \frac{1}{L} \sum_{m \in \mathbb{Z}} \sum_{q} c_{qm}^{n} e^{ix2\pi(m+q)} $$

The symmetry of the problem can be used to realize that

$$ c_{qm}^{n} = c_{q'm'}^{n} $$

if

$$q+m = -(q'+m')\  \ \ $$

which enables us to write the sum as
$$ w^{n} = \frac{1}{L} \left( c_{00}^{n}  + \sum_{m>0} \sum_{q>0} c_{qm}^{n} 2\cos(2\pi(m+q)x \right) $$

where we have to be careful to only sum over |(m+q)| once.   This is the expression that we use to numerically determine the Wannier functions

## The wannier states are defined below

```
def wannier( x, v0, band=0 ):
    L = 200. # Lattice size
    qset = np.arange( 0, L)/L
    qsetSize = len(qset)

    num_m = 20  # Number of plane waves
    mset = np.arange(-num_m, num_m+1)
    msetSize = len(mset)

    # Diagonalize for all values of q
    cCoefs = np.empty( (qsetSize, msetSize, msetSize) )
    for qi,q in enumerate(qset):
        h0 = np.array( np.array(H0_sin(num_m, q, v0)))
        w,v =  numpy.linalg.eigh(h0)
        for j in range(msetSize):
            if v[:,j][-1] > 0 :
                cCoefs[qi][j] = v[:,j]
            else:
                cCoefs[qi][j] = -1.*v[:,j]

    #Calculate Wannier
    wan = np.zeros_like( x, dtype=np.float)
    for qi,q in enumerate(qset):
        for mj,m in enumerate(mset):
            cqm = cCoefs[qi][band][mj]
            if q + m == 0:
                wan +=  cqm
            if q + m > 0:
                #print "added ",
                if band%2 == 0:
                    wan +=  cqm * 2* np.cos( 2*np.pi* (q+m) * x )
                else:
                    wan +=  cqm * 2* np.sin( 2*np.pi* (q+m) * x )
    wan = wan / L
    return wan
```

One can check that the Wannier functions come out (almost) properly normalized:

```
from scipy import integrate
x = linspace(-5, 5, 1000)
v0 = 5.
norm = integrate.simps(np.abs(wannier(x, v0))**2,x)
print norm
```

    0.999999999953

We then plot a few Wannier states for some lattice depths to show how they become increasingly localized

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.7
plt.rcParams['patch.linewidth'] = 0.4

fs=12
fig = plt.figure(figsize=(4.2,2.68))
gs = matplotlib.gridspec.GridSpec( 1,1,
        left=0.16, right=0.78, bottom=0.16, top=0.96)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$x/a$",fontsize=fs)
    ax.set_xticks([-3,-2, -1, 0, 1, 2, 3])
    ax.grid(alpha=0.5)

ax0.set_ylabel("$w^{n}(x-x_{j})$",fontsize=fs,rotation=90, labelpad=4)
#ax0.set_title('Real part',fontsize=12)
#ax1.set_title('Imaginary part',fontsize=12)

x = np.linspace(-3,3,1001)

depths = [1., 4., 7., 14. ]
for i,d in enumerate(depths):
    w = wannier(x, d)
    labelstr = '$%.0fE_{R}$'%d
    ax0.plot(x,np.real(w), label=labelstr,lw=1.8)

ax0.fill_between( x, 0., np.sin(np.pi*x)**2, facecolor='gray', alpha=0.3)
ax0.legend( bbox_to_anchor=(1.03,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.85,1.0])
fig.savefig('BandStructure_figures/wannier1d_V0.png',dpi=300)
fig.savefig('BandStructure_figures/wannier1d_V0.pdf')
```

We can also plot the wannier states for different bands

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.7
plt.rcParams['patch.linewidth'] = 0.4

fs=12
fig = plt.figure(figsize=(4.2,2.88))
gs = matplotlib.gridspec.GridSpec( 1,1,
        left=0.16, right=0.78, bottom=0.16, top=0.91)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$x/a$",fontsize=fs)
    ax.set_xticks([-3,-2, -1, 0, 1, 2, 3])
    ax.grid()

ax0.set_ylabel("$w^{n}(x-x_{j})$",fontsize=fs,rotation=90, labelpad=4)
ax0.set_title('$V_{0}=4 E_{R}$',fontsize=fs)
#ax1.set_title('Imaginary part',fontsize=12)

x = np.linspace(-3,3,1001)
v0 = 4.

bands = [0, 1, 2]
for i,b in enumerate(bands):
    w = wannier(x, v0, band=b)
    labelstr = 'Band %d'%b
    ax0.plot(x,np.real(w), label=labelstr, alpha=(1-0.25*b),lw=1.8, zorder=5-b)

ax0.fill_between( x, 0., np.sin(np.pi*x)**2, facecolor='gray', alpha=0.3)
ax0.legend( bbox_to_anchor=(1.01,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.82,1.0])
fig.savefig('BandStructure_figures/wannier1d_bands.png',dpi=300)
fig.savefig('BandStructure_figures/wannier1d_bands.pdf')
```

# Band structure for a 3D lattice

In a 3D optical lattice the hamiltonian is separable in the three cartesian coordinates.  So we can simply add up the energies and form products of the states from the solution of the 1D problem.

```
def bands3d( v0, NBand=0 ):
    r"""Returns an array of shape (2,) which has the bottom and top of the band energies

    Given the lattice depth it will calculate the bottom and the
    top of the band with band index equal to NBand.

    Parameters
    ----------
    v0 : array-like
        len(v0) must be equal to 3.  Each entry corresponds to the
        lattice depth in recoils along x, y, z respectively
    NBand :
        band index (default = 0 )

    Returns
    -------
    array-like
        array of shape (2,) which has the bottom of the band as
        first entry and the top of the band as second entry.
    """
    assert len(v0)==3
    order = np.argsort( v0 )
    l=0; m=0; n=0
    while NBand > 0:
        l+=1; NBand-=1
        if NBand > 0:
            m+=1; NBand-=1
        if NBand > 0:
            n+=1; NBand-=1
    band1dindex = np.array([ l, m, n] )

    qbot = 0.
    qtop = 0.5

    bandbot = 0.
    bandtop = 0.
    for i,o in enumerate(order):
        in1d = band1dindex[i]
        if in1d%2 ==0:
            bandbot += eigenvalsH0(qbot, v0[o])[in1d]
            bandtop += eigenvalsH0(qtop, v0[o])[in1d]
        else:
            bandbot += eigenvalsH0(qtop, v0[o])[in1d]
            bandtop += eigenvalsH0(qbot, v0[o])[in1d]
    return np.array([bandbot, bandtop])

def bands3dvec( v0, NBand=0):
    r""" Returns an array of shape (len(v0,2) which has the bottom and
    top of the band energies

    Parameters
    ----------
    v0 : array-like
        The shape of v0 is (len(v0,3).  Each row of v0 has 3
        values.  Each value corresponds to the lattice depth
        in recoils along x, y, z respectively.
    NBand : optional
        Band index (default = 0 )
    """
    band = numpy.empty( (len(v0), 2) )
    for i,v in enumerate(v0):
        band[i] = bands3d(v, NBand)
    return band

def hosc3d( NBand, v0 ):
    r"""Energy of the NBand^th excited state of the 3D harmonic oscillator"""
    return 2* np.sqrt(  v0 ) * (NBand + 1.5)
```

We make a plot with the band structure of the 3D lattice

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

fs = 12
fig = plt.figure(figsize=(4.4,2.6))
gs = matplotlib.gridspec.GridSpec( 1,1, left=0.11, right=0.74, bottom=0.16, top=0.96)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=fs)
    ax.grid()
    ax.set_ylim(0.,15.)
    ax.set_xlim(0.,20.)

ax0.set_ylabel("$E_{R}$",fontsize=fs,rotation=0, labelpad=10)

v0 = np.linspace(0.1,20,80.)
v03d = np.outer( v0, np.ones(3))
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
for i in range(3):
    Eb = bands3dvec(v03d,i)
    ax.fill_between( v0 , Eb[:,0], Eb[:,1], \
          color=c[i], facecolor=c[i], alpha=0.7, zorder=4)
    ax.plot( v0, Eb[:,1], color=c[i])
    ax.plot( v0, Eb[:,0], color=c[i],\
          label="Band = %d"%i)
    if i==2: labelstr = 'h.osc.'
    else: labelstr = None
    ax.plot( v0, hosc3d(i,v0), color='black', label=labelstr, zorder=1 )

ax0.legend( bbox_to_anchor=(1.03,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.8,1.0])
fig.savefig('BandStructure_figures/bands3d_V0.png',dpi=240)
fig.savefig('BandStructure_figures/bands3d_V0.pdf')
```

## Define an interpolation function to quickly calculate the band structure. The interpolation is for a 1D lattice and the 3D bands should be calculated by deciding which band index to use for each axis.

The interpolation data is saved to file for easy retrieval without having to evaluate the entire thing

```
v0 = np.linspace(0.,120,320.)
NPoints = 5
evals0 = np.empty( (len(v0), 2*NPoints+1) )
evals1 = np.empty( (len(v0), 2*NPoints+1) )
qbot = 0.
qtop = 0.5
for i,v in enumerate(v0):
    evals0[i] = eigenvalsH0(qbot, v, NPoints=NPoints)
    evals1[i] = eigenvalsH0(qtop, v, NPoints=NPoints)

np.savetxt('banddat/interpdat_B1D_v0.dat', v0)
for n in range( 2*NPoints+1 ):
    np.savetxt('banddat/interpdat_B1D_0_%d.dat'%n, evals0[:,n] )
    np.savetxt('banddat/interpdat_B1D_1_%d.dat'%n, evals1[:,n] )

```

The interpolation data is retrieved from disk and we use it to define functions that calculate the band structure in the 3D lattice.  The code that defines the interpolation functions is imported below

```
from Band3DInterp import *
```

## Compare the performance of the direct calculation and the interpolation

```
%timeit eigenvalsH0(0., 10.)
%timeit interp0[0](10.)
```

    100 loops, best of 3: 7 ms per loop
    10000 loops, best of 3: 28 us per loop

```
v0 = np.linspace(0.1,20,80.)
v03d = np.outer( v0, np.ones(3))
band = 0
%timeit bands3dvec(v03d, NBand=band)
%timeit bands3dinterpvec(v03d, NBand=band)
```

    1 loops, best of 3: 4.23 s per loop
    100 loops, best of 3: 16.7 ms per loop

```
print "Performance is increased by a factor of %d" % (4230 / 16.7)
```

    Performance is increased by a factor of 253

We can use the interpolation version of the 3D bands calculation to plot the band structure

```
fig = plt.figure(figsize=(6.5,3.8))
gs = matplotlib.gridspec.GridSpec( 1,1)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=16)
    ax.grid()
    ax.set_ylim(0.,15.)
    ax.set_xlim(0.,20.)

ax0.set_ylabel("$E_{R}$",fontsize=16,rotation=0)

v0 = np.linspace(0.1,20,80.)
v03d = np.outer( v0, np.ones(3))
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
for i in range(3):
    Eb = bands3dinterpvec(v03d,i)
    ax.fill_between( v0 , Eb[:,0], Eb[:,1], \
          color=c[i], facecolor=c[i], alpha=0.7, zorder=4)
    ax.plot( v0, Eb[:,1], color=c[i])
    ax.plot( v0, Eb[:,0], color=c[i],\
          label="Band = %d"%i)
    if i==2: labelstr = 'h.osc.'
    else: labelstr = None
    ax.plot( v0, hosc3d(i,v0), color='black', label=labelstr, zorder=1 )

ax0.legend( bbox_to_anchor=(1.03,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':12}, handlelength=1.1, handletextpad=0.5 )

gs.tight_layout(fig, rect=[0.,0.,0.8,1.0])
fig.savefig('BandStructure_figures/bands3d_V0_interp.png',dpi=240)
fig.savefig('BandStructure_figures/bands3d_V0_interp.pdf')
```

# Calculation of tunneling coefficient

## Using the sum over energy eigenvalues

```
def tCalc(v0, Xij=1., Nqs=40):
    r"""Calculates the tunneling matrix element

    Parameters
    ----------
    v0 : float
        Lattice depth in recoils
    Xij : float, optional,
        This is the separation between wannier states.  For
        nearest neighbors Xij = 1.   The type is a float, but
        really only integers should be fed into it.
        default value = 1.0 (nearest neighbors)
    Nqs : int, optional
        Then number of sites on the lattice (along one of the
        dimensions).  This number defines the number of quasi-momentum
        points that will be considered in the sum that calculates
        the tunnling.  The result should be independent of Nqs
        default value = 40

    Returns
    -------
    t: float
        tunneling matrix element in recoils
    """
    qset = np.arange( 0, Nqs)/Nqs
    t = 0.
    for q in qset:
        t += eigenvalsH0(q, v0)[0] * np.exp( 1j*2*np.pi*q*Xij)  / len(qset)
    return -1*np.real(t)
```

Calculate the value of the tunneling in the limit of zero lattice depth:

```
v0 = 0.1
print tCalc(v0 )
v0 = 0.01
print tCalc(v0 )
v0 = 0.001
print tCalc(v0 )
v0 = 0.00001
print tCalc(v0 )
```

    0.20215147731
    0.202994190322
    0.203053269912
    0.203059485994

We would like to have an interpolation function for the nearest neighbor tunneling matrix element.  Here we calculate the samples necessary to define the interpolation function.

```
v0 = np.linspace( 0.1, 80., 120.)
tv0 = np.array([tCalc(d) for d in v0])

np.savetxt('banddat/interpdat_t_v0.dat', v0)
np.savetxt('banddat/interpdat_t_tCalc.dat', tv0)
```

And below we load up the interpolation samples from disk and define the intepolartion function.

```
#Here the interpolation data is loaded from disk
from scipy.interpolate import interp1d
tInterp = interp1d( np.loadtxt('banddat/interpdat_t_v0.dat'), np.loadtxt('banddat/interpdat_t_tCalc.dat'))
```

We want to make a plot that illustrates the difference in tunneling matrix element between nearest-neighbor tunneling and beyond nearest-neighbor tunneling.   We calculate the samples first:

```
v0 = np.linspace(0.1,8,18)
tCalc_2sites = np.array([ np.abs(tCalc(v0val, Xij=2.)) for v0val in v0 ])
tCalc_3sites = np.array([ np.abs(tCalc(v0val, Xij=3.)) for v0val in v0 ])

```

And then we make the plot

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

fs=12.
v0 = np.linspace(0.1,8,18)
fig = plt.figure(figsize=(4.4,2.66))
gs = matplotlib.gridspec.GridSpec( 1,1,
        left=0.15, right=0.72, bottom=0.16, top=0.96)

ax0 = fig.add_subplot( gs[0,0])

for ax in [ax0]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=fs)
    ax.grid(alpha=0.5)
    ax.set_ylim(0.,0.2)
    #ax.set_xlim(0.,20.)

ax0.set_ylabel("$|t_{ij}|$ $(E_{R})$",fontsize=fs,rotation=90, labelpad=7)

c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
ax.plot( v0, tInterp(v0), label="$|\Delta_{ij}|/a=1$")
ax.plot( v0, tCalc_2sites, label="$|\Delta_{ij}|/a=2$" )
ax.plot( v0, tCalc_3sites, label="$|\Delta_{ij}|/a=3$ ")

ax0.legend( bbox_to_anchor=(1.02,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.8,1.0])
fig.savefig('BandStructure_figures/tightbinding_V0_interp.png',dpi=300)
fig.savefig('BandStructure_figures/tightbinding_V0_interp.pdf')
```

We show the tunneling (nearest-neighbor) in a logarithmic scale alongside the Mathieu approximation to the tunneling.  We use recoils and kHz

```
def tMathieu( v0 ):
    return 4./np.sqrt(np.pi) * v0**(3./4.) * np.exp( -2*np.sqrt(v0))

```

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

fs=12.
fig = plt.figure(figsize=(7.7,2.94))
gs = matplotlib.gridspec.GridSpec( 1,2,\
        left=0.09, right=0.84, bottom=0.15, top=0.96,
        wspace=0.35, hspace=0.)

ax0 = fig.add_subplot( gs[0,0])
ax1 = fig.add_subplot( gs[0,1])

for ax in [ax0,ax1]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=fs)
    ax.grid(alpha=0.5)
    #ax.set_xlim(0.,20.)

ax0.set_yscale("log")

ax0.set_ylim(0.002,0.6)
ax0.set_ylabel(r"$t/E_{r}$",fontsize=fs,rotation=90)

ax1.set_ylabel(r"$t/h\ \mathrm{(kHz)}$",fontsize=fs,rotation=90)
ax1.set_xlim(4., 10)
ax1.set_ylim(0.4, 2.6)

v0 = np.linspace(0.1,25,100.)
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
ax0.plot( v0, tInterp(v0), label="Diag.")
ax0.plot( v0, tMathieu(v0), label="Mathieu")
ax1.plot( v0, tInterp(v0) * 29.0, label="Diag.")  #29kHz = 1Er
ax1.plot( v0, tMathieu(v0) * 29.0, label="Mathieu")

ax1.legend( bbox_to_anchor=(1.03,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.88,1.0])
fig.savefig('BandStructure_figures/tunneling_V0_Mathieu.png',dpi=300)
fig.savefig('BandStructure_figures/tunneling_V0_Mathieu.pdf')
```

## Using the overlap between Wannier states

With the definition of the Wannier states that was defined above it is easy to compute derivatives of the Wannier functions:

$$ w^{n} = \frac{1}{L} \left( c_{00}^{n} + \sum_{m>0} \sum_{q>0} c_{qm}^{n} 2\cos(2\pi(m+q)x \right) $$

$$ \frac{\partial}{\partial x} w^{n} = -\frac{1}{L} \sum_{m>0} \sum_{q>0} c_{qm}^{n} 4\pi(m+q) \sin(2\pi(m+q)x) $$

$$ \frac{\partial^{2}}{\partial x^{2}} w^{n} = \frac{1}{L} \sum_{m>0} \sum_{q>0} c_{qm}^{n} 8\pi^{2}(m+q)^{2} \sin(2\pi(m+q)x) $$

We define the second derivative below

```
def wannierD2( x, v0, band=0 ):
    L = 200. # Lattice size
    qset = np.arange( 0, L)/L
    qsetSize = len(qset)

    num_m = 20  # Number of plane waves
    mset = np.arange(-num_m, num_m+1)
    msetSize = len(mset)

    # Diagonalize for all values of q
    cCoefs = np.empty( (qsetSize, msetSize, msetSize) )
    for qi,q in enumerate(qset):
        h0 = np.array( np.array(H0_sin(num_m, q, v0)))
        w,v =  numpy.linalg.eigh(h0)
        for j in range(msetSize):
            if v[:,j][-1] > 0 :
                cCoefs[qi][j] = v[:,j]
            else:
                cCoefs[qi][j] = -1.*v[:,j]

    #Calculate Wannier
    wan = np.zeros_like( x, dtype=np.float)
    for qi,q in enumerate(qset):
        for mj,m in enumerate(mset):
            cqm = cCoefs[qi][band][mj]
            if q + m > 0:
                wan += -1* cqm * 8*(( np.pi*(q+m) )**2)* np.cos( 2*np.pi* (q+m) * x )
    wan = wan / L
    return wan
```

The second derivative is useful in calculating the tunneling matrix element directely from the Wannier function overlap

$$ t = - \int w_{j}^{*}(x) \left( -\frac{\hbar^{2}}{2m} \frac{\partial^{2}}{\partial x^{2}} + V_{0}\sin^{2}(kx) \right)  w_{j+1}(x) $$

below we define a function that thas such a calculation and compare its results with the sum over energy eigenvalues method used above

```
from scipy import integrate

def tWannier( v0):
    x0 = np.linspace(-5, 5, 1000)
    x1 = np.linspace(-6, 4, 1000)

    w0 = wannier( x0, v0 )
    w1 = wannier( x1, v0 )
    w1D2 = wannierD2( x1, v0 )

    integrand = -1*( -1*w0*w1D2/(np.pi**2) + w0*v0*(np.sin(np.pi*x0)**2)*w1 )
    tInt = integrate.simps( integrand, x0)
    return tInt
```

```
v0list= [1.0, 3.0, 7.0, 10.]
print "v0\t"
print "----"
for v0 in v0list:
    print "%4.2f\t%4.5f\t%4.5f\t%4.5f\t%4.5f" % (v0, tCalc(v0), tInterp(v0), tMathieu(v0), tWannier(v0) )
```

    v0
    ----
    1.00	0.17812	0.17783	0.30542	0.17783
    3.00	0.11103	0.11134	0.16102	0.11103
    7.00	0.03942	0.03954	0.04889	0.03942
    10.00	0.01918	0.01923	0.02274	0.01918

We see that for lower lattice depths the two methods differ.    We look at the performance of the two methods and find that the one with the energy eigenvalues beats the other one by more than a factor of 10.

```
%timeit tWannier(7.0)
%timeit tCalc(7.0)
```

    1 loops, best of 3: 10.5 s per loop
    1 loops, best of 3: 738 ms per loop

# Calculation of on-site interactions

```
from scipy import integrate

def wF( v0 ):
    x0 = np.linspace(-5, 5, 1000)
    w0 = wannier( x0, v0 )
    return 8/np.pi * (integrate.simps( w0**4, x0 ))**3

def wFho( v0 ):
    return np.sqrt( 8*np.pi) * v0**(3./4.)
```

We calculate a table of values to be used in interpolation

```
v0 = np.hstack(( np.linspace(1e-3,0.1,4),np.linspace( 0.1, 80., 120.) ))
wFv0 = np.array([wF(d) for d in v0])

np.savetxt('banddat/interpdat_wF_v0.dat', v0)
np.savetxt('banddat/interpdat_wF_wF.dat', wFv0)
```

The interpolation data is loaded from disk and the function is defined

```
#Here the interpolation data is loaded from disk
from scipy.interpolate import interp1d
wFInterp = interp1d( np.loadtxt('banddat/interpdat_wF_v0.dat'), np.loadtxt('banddat/interpdat_wF_wF.dat'))
```

And we make a plot comparing the numerical result using Wannier functions and the harmonic approximation

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

fs=12.
fig = plt.figure(figsize=(7.7,2.94))
gs = matplotlib.gridspec.GridSpec( 1,2,\
        left=0.09, right=0.84, bottom=0.15, top=0.96,
        wspace=0.35, hspace=0.)

ax0 = fig.add_subplot( gs[0,0])
ax1 = fig.add_subplot( gs[0,1])

for ax in [ax0,ax1]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=fs)
    ax.grid()
    #ax.set_xlim(0.,20.)
    #ax.set_yscale("log")

#ax0.set_ylim(0.002,0.6)
ax0.set_ylabel("$U/a_{s}$ $(E_{R}/a)$",fontsize=fs,rotation=90)

ax1.set_ylabel("$U/a_{s}$ ($\mathrm{kHz}/a$)",fontsize=fs,rotation=90)
#ax1.set_xlim(5., 25)
#ax1.set_ylim(0.01, 4.)

v0 = np.linspace(0.1,25,100.)
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
ax0.plot( v0, wFInterp(v0), label="Wannier")
ax0.plot( v0, wFho(v0), label="H.osc.")
ax1.plot( v0, wFInterp(v0) * 29.0, label="Wannier")  #29kHz = 1Er
ax1.plot( v0, wFho(v0) * 29.0, label="H.osc")

ax1.legend( bbox_to_anchor=(1.03,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.88,1.0])
fig.savefig('BandStructure_figures/wFactor_V0.png',dpi=240)
fig.savefig('BandStructure_figures/wFactor_V0.pdf')
```

```
# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

fs=12.
fig = plt.figure(figsize=(7.7,2.94))
gs = matplotlib.gridspec.GridSpec( 1,2,\
        left=0.09, right=0.84, bottom=0.15, top=0.96,
        wspace=0.35, hspace=0.)

ax0 = fig.add_subplot( gs[0,0])
ax1 = fig.add_subplot( gs[0,1])

for ax in [ax0,ax1]:
    ax.set_xlabel("$V_{0} (E_{R})$",fontsize=fs)
    ax.grid()
    #ax.set_xlim(0.,20.)
    #ax.set_yscale("log")

#ax0.set_ylim(0.002,0.6)
ax0.set_ylabel(r'$\frac{U}{a_{s}}\ ( \frac{E_{r}}{100a_{0}})$', fontsize=fs, rotation=90 )

ax1.set_ylabel(r'$\frac{U/h}{a_{s}}\ ( \frac{\mathrm{kHz}}{100a_{0}})$', fontsize=fs, rotation=90 )
#ax0.set_ylabel(r'$U/a_{s}\ (E_{r}/100a_{0})$', fontsize=fs, rotation=90 )
#ax1.set_ylabel(r"$\frac{U}{a_{s}}$ $\left( \frac{\mathrm{kHz}}{100 a_{0}} \right)$",fontsize=fs,rotation=0)
#ax1.set_xlim(5., 25)
#ax1.set_ylim(0.01, 4.)

a0a = 100.* 5.29e-11 / (1064e-9/2)

v0 = np.linspace(0.1,12,100.)
c = ['blue', 'red', 'green']
fc = ['lightblue','pink','limegreen']
ax0.plot( v0, a0a*wFInterp(v0), label="Wannier")
ax0.plot( v0, a0a*wFho(v0), label="H.osc.")
ax1.plot( v0, a0a*wFInterp(v0) * 29.0, label="Wannier")  #29kHz = 1Er
ax1.plot( v0, a0a*wFho(v0) * 29.0, label="H.osc")

ax1.legend( bbox_to_anchor=(1.03,1.00), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=1.1, handletextpad=0.5 )

#gs.tight_layout(fig, rect=[0.,0.,0.88,1.0])
fig.savefig('BandStructure_figures/wFactor_V0_a0.png',dpi=300)
fig.savefig('BandStructure_figures/wFactor_V0_a0.pdf')
```

## Region of validity for the single band Hubbard model

```
# Calculation of the lattice depth at which next-nearest neighbor tunneling is 10 times slower than nearest neighbor tunneling

v0int = np.linspace( 2., 8., 40)
tun0 = tInterp(v0int)
tun1 = [ np.abs(tCalc(v0val, Xij=2.)) for v0val in v0int]

from scipy.interpolate import interp1d
tun1f = interp1d( v0int, tun1)

def g(x):
    return tun1f(x)*20 - tInterp(x)
print brentq( g, 2.1, 6.)
```

    5.08836237817

```

def bandgap( v0 ):
    v03d = np.outer( v0, np.ones(3))
    Eb0 = bands3dinterpvec(v03d,0 )[:,1]
    Eb1 = bands3dinterpvec(v03d,1 )[:,0]
    return Eb1-Eb0

def tunnel(v0):
    return tInterp(v0)

def Uonsite( v0, aS):
    a0a = 5.29e-11 / (1064e-9/2)
    return aS*a0a*wFInterp(v0)

# Find lines of constant U/t
def aSconstInt( v0, Ut):
    return  Ut * tInterp(v0) / Uonsite(v0,1.)

# Find  the line where W = multiple*U , where W is the band gap.
def aSlimit( v0, multiple):
    aS = bandgap(v0) / (multiple*Uonsite(v0,1.))
    return aS

# Find the lattice depth where 12t = W/10
def f(x):
    return  bandgap(x)[0]/10. -  12.*tunnel(x)
from scipy.optimize import brentq
print brentq( f, 1., 10.)

def g(x):
    return 2.*np.sqrt(x) - 12.*4/(np.sqrt(np.pi)) * (x**(3./4.)) * np.exp( -2.*np.sqrt(x))
print brentq( g, 1., 10.)

v0test = 3.
print type(tunnel(v0test))
print "v0 = {:0.2f}".format(v0test)
print " W = {:0.2f}".format(bandgap(v0test)[0])
print " U = {:0.2f}".format(Uonsite(v0test,200.))
print " t = {:0.2f}".format(float(tunnel(v0test)))
print " a_max = {:0.1f}".format( aSlimit(7.,10.)[0] )

# Start matplotlib
from matplotlib import rc
rc('text', usetex=True)
plt.rcParams['text.latex.preamble'] = [
       r'\usepackage{bm}',        # for bold math
       r'\usepackage{siunitx}',   # i need upright \micro symbols, but you need...
       r'\sisetup{detect-all}',   # ...this to force siunitx to actually use your fonts
       #r'\usepackage[textlf]{MyriadPro}',
       #r'\usepackage{sansmath}',  # load up the sansmath so that math -> helvet
       #r'\sansmath',               # <- tricky! -- gotta actually tell tex to use!
]
plt.rcParams['axes.linewidth'] = 0.6
plt.rcParams['patch.linewidth'] = 0.4

fs=12.
fig = plt.figure(figsize=(5.1,2.94))
gs = matplotlib.gridspec.GridSpec( 1,1,\
        left=0.12, right=0.66, bottom=0.15, top=0.96,
        wspace=0.35, hspace=0.)

ax = fig.add_subplot( gs[0])

v0 = np.linspace(2.24,20,1000)
dash = [(4.8,2.8), (6.4,2,2,2),(2,0.001),(2,2.10)]

a10 = aSlimit(v0,10.)
a05 = aSlimit(v0,5.)
a03 = aSlimit(v0,3.)
ax.plot( v0, a10, label=r'$\Delta/U=10$', color='k', dashes=dash[0])
ax.plot( v0, a05,  label=r'$\Delta/U=5$',  color='k', dashes=dash[1])
ax.plot( v0, a03,  label=r'$\Delta/U=3$',  color='k', dashes=dash[3])

ax.fill_between( v0, np.ones_like(v0), a10, lw=0., color='gray', alpha=0.0)
ax.fill_between( v0, a10, a05, lw=0., color='gray', alpha=0.30)
ax.fill_between( v0, a05, a03, lw=0., color='gray', alpha=0.60)
ax.fill_between( v0, a03, 1e3*np.ones_like(v0), lw=0., color='gray', alpha=0.90)
v0fill = np.linspace(0.,2.242,100)
ax.fill_between( v0fill, np.ones_like(v0fill), 1e3*np.ones_like(v0fill), lw=0., color='gray', alpha=0.90 )

#plt.axvline( 3.025 ) # for 10 times slower nnn than nn tunneling
ax.axvline( 5.088, color='gray', label=r'$t=20\,t_{2}$' ) # for 20 times slower nnn than nn tunneling

from colorChooser import rgb_to_hex, cmapCycle
colors = [ cmapCycle(matplotlib.cm.hot, i, lbound=-1.5, ubound=4.5) for i in range(4) ]
v0U = np.linspace(0.2, 20., 180)
ax.plot( v0U, aSconstInt(v0U,1.),  color=colors[3], label=r'$U/t=1$')
ax.plot( v0U, aSconstInt(v0U,5.),  color=colors[2], label=r'$U/t=5$')
ax.plot( v0U, aSconstInt(v0U,10.), color=colors[1], label=r'$U/t=10$')
ax.plot( v0U, aSconstInt(v0U,20.), color=colors[0], label=r'$U/t=20$')

v0nnn=np.linspace(0., 5.088)
fill_between(v0nnn, np.ones_like(v0nnn), 1e3*np.ones_like(v0nnn), color="none", hatch="XX", edgecolor="gray", linewidth=0.0)

ax.axhline( 73., color='b', lw=0.5, label=r'$\mathrm{narrow\ Feshbach}$'+'\n'+r'$\mathrm{resonance}$')
ax.axhspan( 560., 1000., color='blue', lw=0.1, alpha=0.4, label=r'$\mathrm{collisional}$'+'\n'+'$\mathrm{losses}$')
ax.axvline( 832. )

l0 = ax.legend( bbox_to_anchor=(1.04,1.035), \
            loc='upper left', numpoints=1, \
             prop={'size':10}, handlelength=2., handletextpad=0.5 )

#ax.axvline( 7.0 )
#ell = matplotlib.patches.Ellipse(xy=(7., 320.), width=0.2, height=480., angle=0., lw=0. )
ell = matplotlib.patches.FancyBboxPatch(
        (7.-0.2,316.-240.), 0.4, 480., lw=0.,
        boxstyle=matplotlib.patches.BoxStyle("Round", pad=0.02))

ax.add_artist(ell)
ell.set_clip_box(ax.bbox)
ell.set_facecolor('green')
ell.set_alpha(0.5)

ax.set_xlim(0., 20.)
ax.set_ylim(10, 1000)
ax.set_yscale('log')

ax.set_xlabel(r'$V_{0}\ (E_{r})$', fontsize=fs)
ax.set_ylabel(r'$a_{s}\ (a_{0})$', fontsize=fs)

fig.savefig('BandStructure_figures/singleband.png', dpi=300)
fig.savefig('BandStructure_figures/singleband.pdf')

```

```
class CustomXHatch(matplotlib.hatch.HorizontalHatch):
    def __init__(self, hatch, density):
        char_count = hatch.count('_')
        if char_count > 0:
            self.num_lines = int((1.0 / char_count) * density)
        else:
            self.num_lines = 0
        self.num_vertices = self.num_lines * 2
```

```
matplotlib.hatch.HatchPatternBase
```

    matplotlib.hatch.HatchPatternBase


# Approximate Singular Value Decomposition

### Synopsis

I use the eigenvalue decomposition method whenever I implement any Manifold Learning method. The general steps one must take in order to implement these methods would be:

1. Construct a Connectivity Matrix
2. Solve the Eigenvalue Decomposition Problem
3. Extract the k-smallest nontrivial eigenvalues and the corresponding eigenvectors to either extract the embedding or project the data into the new space where the embedding is present.

Now, step 2 is very computationally heavy so I am looking for methods which can reduce the computaitonal burden. From my understanding, the Approximate Singular Value Decomposition (ASVD) method uses the theory of random approximations of spectral decompositions to find the best rank-k  approximation of a matrix A. This sounds particularly useful because it will speed up the entire algorithm process and it is a method that has some pretty good theoretical guarantees to ensure that we have a good solution.

### Resources

I will be using the following papers to complete my self tutorial:

# Theory

* Randomized Approximate of Operators and their Spectral Decomposition for Diffusion Based Embeddings of Hetergeneous Data
    - Czaja et al. - [Paper][1]
* A randomized algorithm for principal component analysis
    - Rokhlin et al. - [Paper][2]
* Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions
    - Halko et al. - [Paper][3]
* A randomized algorithm for the decomposition of matrices
    - Martisson et al. - [Paper][8]

# Practical

* A Practical Guide to Randomized Algorithms in MATLAB
    - [Paper][10] | [Code][11]
* Course: Fast AI - Randomized SVD
    - [github](https://github.com/fastai/randomized-SVD)

# Source Codes

I will use the following code as a guide for implementing the steps in the paper:

* Github user - [gist][4]
* Github user - [github][6]
* Facebook: Fast Randomized PCA/SVD - [github][5]
* Scikit-Learn: Randomized SVD - [github][7]
* Mu Li: Randomized SVD - [github][9]

[1]: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7330267&tag=1
[2]: http://arxiv.org/pdf/0809.2274.pdf
[3]: https://arxiv.org/pdf/0909.4061
[4]: https://gist.github.com/alextp/662433
[5]: https://github.com/facebook/fbpca/blob/master/fbpca.py
[6]: https://github.com/ktaneishi/pyredsvd
[7]: https://github.com/scikit-learn/scikit-learn/blob/51a765acfa4c5d1ec05fc4b406968ad233c75162/sklearn/utils/extmath.py
[8]: https://math.berkeley.edu/~strain/273.F10/martinsson.tygert.rokhlin.randomized.decomposition.pdf
[9]: https://github.com/mli/nystrom/blob/master/rsvd.m
[10]: https://arxiv.org/pdf/1505.07570.pdf
[11]: https://github.com/wangshusen/RandMatrixMatlab

## Algorithm Walkthrough

Suppose we have a real matrix A of size $m\times n$. This can be typically decomposed via regular SVD as

$$A=U\Sigma V^T$$

where:
* $U$ is a real unitary $m\times m$ matrix
* $V$ is a real unitary $n \times n$ matrix
* $\Sigma$ is a real $m\times n$ diagonal matrix

A well known idea is that the best rank-$k$ approximation of A is given by

$$A=\tilde{U}\tilde{\Sigma}\tilde{V}^T$$

where:
* $\tilde{U}$ is a real unitary $m\times k$ matrix
* $\tilde{V}$ is a real unitary $n \times k$ matrix
* $\tilde{\Sigma}$ is a real $k\times k$ diagonal matrix
We now want to form a Gaussian matrix

### Step 1 - Gaussian Matrix

Let $A$ be a real $m\times n$ matrix.

```python
# get some random matrix
from sklearn.datasets import make_spd_matrix

A = make_spd_matrix(100, random_state=0)

m = A.shape[0]  # left dimension of A
n = A.shape[1]  # right dimension of A

# choose some k singular values we want
k = 10
```

Construct a matrix, $G$, of i.i.d. standard Gaussian random variables, s.t. the matrix is size $l\times m$ where $k<l\leq m-k$.

```python
# choose some l that satisfies the inequality listed above
l = 20

# assert that the inequality is satisfied
def check_l(l):
    assert l > k and l <= (m-k), "Your choice of l must satisfy the inequality."
    return l

# check my value of l
l = check_l(l)

# create gaussian matrix
import numpy as np

G = np.random.normal(size=(l,m))
```

### Step 2 - Compute Product

Compute the product of the following:

$$S=G(AA^T)^iA$$

where $i \in \mathbb{N}$. To get an idea of the dimensions, it is:

$$S=G_{l\times m}(A_{m \times n}A_{n \times m}^T)^iA_{m \times n}$$

This is actually a pretty involved product so I'm going to break down the steps even further.

##### Compute the square matrix

It would seem that the product $AA^T$ is only need because $A$ is an $m \times n$ matrix. This would not be needed if $A=A^T$, i.e. a symmetric matrix. In fact, it might be the case that doing $AA^T$ for a symmetric matrix would count as doing an extra iteration for the next step.

```python
# compute the square matrix
A_square = np.dot(A, A.T)
```

##### Compute the Power of the square matrix

This involves the $(AA^T)^{i}$. I'm pretty sure that $i \in \mathbb{N}$ and by increasing $i$ increases the accuracy of the method.

```python
# compute the matrix power
i = 2
A_power = np.linalg.matrix_power(A_square,i)
```

##### Now Compute the actual product

```python
# compute the product for step 2
S = np.dot(G,np.dot(A_square,A))
```

### Step 3. Compute the Pivoted QR-Decomposition

So, we need to use a pivoted $QR$-decomposition on the $S^T$ matrix computed in step 2 so that we get

$$S^T=QR$$

where $Q$ is an $n \times l$ matrix with orthonormal columns.

```python
# compute the QR-decomposition on S transpose
Q, R = np.linalg.qr(S.T,mode='complete')
```

There are so many options for the QR decomposition in the numpy package. The 'reduced' mode returns with reduced dimensions so I decided to use the 'complete' mode just in case it cuts some stuff out that we need for later. I could do some experiments later because I suspect that the reduced method may be a little bit faster.

**Note:** Not sure if they have a sparse package in the scipy library. May have to figure out another way to do it.

### Step 4. Compute the SVD of $T:=AQ$

Now we compute the following SVD

$$T=\tilde{U}\tilde{\Sigma}W^T$$

If we let $T:=AQ$, we get a much faster SVD computation than just performing SVD on $A$ alone because $T$ is size $m \times l$.

```python
T = np.dot(A,Q)
U_hat, Sigma_hat, W = np.linalg.svd(T)
```

### Step 5. Compute $\tilde{V}$

This is done by the following assignment:

$$\tilde{V}:=QW$$

```python
V_hat = np.dot(Q,W)
```

### Step 6. Extract $U$, $\Sigma$, and $V$

Now we can extract the necessary SVD from our algorithm.

* Extract upper left $m\times k$ block from $\tilde{U}$
* Extract upper left $n\times k$ block from $\tilde{V}$
* Extract upper left $k\times k$ block from $\tilde{\Sigma}$

```python
U = U_hat[:m, :k]
V = V_hat[:n, :k]
Sigma = Sigma_hat[:k]
```

## ASVD Function

To get away from the imperative style tutorial above, I'm defining a function that contains all of the elements that I outlined above.

```python
def asvd(A, n_components, size_gauss, pow_iter):

    # get components
    m = A.shape[0]; n = A.shape[1]
    k = n_components
    l = size_gauss
    i = pow_iter
    assert l > k and l <= (m-k), "Your choice of l must satisfy the inequality."

    # perform ASVD
    G = np.random.normal(size=(l,m))
    A_square = np.dot(A, A.T)
    A_power = np.linalg.matrix_power(A_square,i)
    S = np.dot(G,np.dot(A_square,A))
    Q, R = np.linalg.qr(S.T,mode='complete')
    T = np.dot(A,Q)
    U_hat, Sigma_hat, W = np.linalg.svd(T)
    V_hat = np.dot(Q,W)

    # extract n_components
    U = U_hat[:m, :k]
    V = V_hat[:n, :k]
    Sigma = Sigma_hat[:k]

    return U, Sigma, V
```

## More Compact ASVD Function

The next step is to reduce the memory used for this function. I don't need to be implementing all these  assignments when I can keep those contained within functions; really one of the highlights of python.

## Smallest Eigenvalues and Eigenvectors

So the above method computes the largest eigenvalues of the given matrix. In order to compute the smallest eigenvalues, we have to do a trick:

1. Let $k=1$.
2. Use ASVD to compute the largest $k$ eigenvalues of the matrix, i.e. $\sigma_1$.
3. Transform A like so: $\mathbf{A}' = 2\sigma_1 \mathbf{I}-\mathbf{A}$.
4. Use ASVD to solve for the largest $k$' eigenvalues and eigenvectors of the matrix, $\mathbf{A}'$.
5. The smallest eigenvalues and eigenvectors are given by: $\Sigma=2\sigma_1 \mathbf{I}-\Sigma'$

**Note**: We can actually keep the same eigenvectors as the $\mathbf{A}'$ matrix for the smallest eigenvalues.

Let's have the following eigenvalue formula:

\begin{equation}
\mathbf{A}\vec{x}=\lambda\vec{x}
\end{equation}

Now let's add some arbitrary scaling parameter $\gamma \mathbf{I}$ to the $\mathbf{A}$. Using the above equation, we can deduce that:

\begin{align}
\left(\gamma \mathbf{I+A}\right)\vec{x} &= \gamma \mathbf{I} \vec{x} + \mathbf{A}\vec{x} \\
    &= \gamma \vec{x} + \lambda \vec{x}\\
    &= \left( \gamma + \lambda \right) \vec{x}
\end{align}

Therefore, $\gamma + \lambda$ is an eigenvalue but $\vec{x}$ is still the same eigenvector.

##### Example

```python
# set parameters
n_components = 5  # i'm looking for the 5 smallest and largest eigenvectors for the same matrix A
size_gauss = 30
pow_iter = 20

# solve for the largest eigenvalue
_, Sigma_large, _ = asvd(A, n_components=n_components,
                   size_gauss=size_gauss,
                   pow_iter=pow_iter)

# transform the original matrix A to A'
A_new = 2*Sigma_large[0]*np.eye(A.shape[0])-A

# solve for the largest k components of the matrix A'
_, Sigma, _ = asvd(A_new, n_components=n_components,
                   size_gauss=size_gauss,
                   pow_iter=pow_iter)

# use transformation to find the smallest eigenvalues
Sigma_small = 2*Sigma_large[0]-Sigma

# find all of the eigenvalues
_, Sigma, _ = np.linalg.svd(A)

# just check to see if the results are the same
print('The largest {k} eigenvalues for A:\n{eig}'.format(k=n_components,eig=Sigma_large))
print('Using the SVD function:\n{eig}'.format(eig=Sigma[:n_components]))
print('The same? {ans}'.format(ans=np.linalg.norm(Sigma_large - Sigma[:n_components])<1E-10))
print('The smallest {k} eigenvalues for A:\n{eig}'.format(k=n_components,eig=Sigma_small))
print('Using the SVD function:\n{eig}'.format(eig=Sigma[::-1][:n_components]))
print('The same? {ans}'.format(ans=np.linalg.norm(Sigma_small - Sigma[::-1][:n_components])<1E-10))
```

    The largest 5 eigenvalues for A:
    [100.49708086   0.99603269   0.99283061   0.98404493   0.97890796]
    Using the SVD function:
    [100.49708086   0.99603269   0.99283061   0.98404493   0.97890796]
    The same? True
    The smallest 5 eigenvalues for A:
    [0.00258032 0.00694713 0.01430953 0.02844083 0.03210657]
    Using the SVD function:
    [0.00258032 0.00694713 0.01430953 0.02844083 0.03210657]
    The same? True

#### Where are the eigenvectors in an SVD?

I'll just do an experiment where I compute the solution to the generalized eigenvalue problem using SVD and some other method.

```python
from scipy.linalg import eig
U={}; Sigma={}; V={}
U['np'], Sigma['np'], V['np'] = np.linalg.svd(A)
U['asvd'], Sigma['asvd'], V['asvd'] = asvd(A, n_components=10,
                                           size_gauss=size_gauss,
                                           pow_iter=pow_iter)
eigVals = {}; eigVecs = {}
eigVals['scipy'], eigVecs['scipy'] = eig(a=A)
print(V['np'].shape, V['asvd'].shape, eigVecs['scipy'].shape)
print(V['np'][:5, :n_components])
print(V['asvd'][:5, :n_components])
print(eigVecs['scipy'][:5, :n_components])
print('\n\n\n')
print(Sigma['np'][:5])
print(Sigma['asvd'][:5])
print(eigVals['scipy'][:5])
```

    (100, 100) (100, 10) (100, 100)
    [[-0.05956788 -0.08791196 -0.17417145 -0.04740637  0.00759859]
     [-0.00759328 -0.23749074 -0.05009837 -0.08681157 -0.08235621]
     [ 0.11349362  0.0180285  -0.10913929 -0.17421939 -0.07026755]
     [-0.1997493   0.01389396  0.0518404   0.07702102 -0.17090324]
     [ 0.00489798  0.00536418  0.1232994   0.10461716 -0.19734486]]
    [[-0.05956139  0.16587084  0.19881521 -0.10309956 -0.12184645]
     [-0.08791162  0.07951956  0.00039253 -0.09658149  0.0015464 ]
     [-0.17417317  0.05115577  0.06641891  0.009945    0.18215605]
     [-0.04740436 -0.07802608  0.00304858  0.2878056   0.03702041]
     [ 0.00759136 -0.01111304 -0.20716936 -0.04216575  0.10729865]]
    [[ 0.05956788 -0.00759328  0.11349362 -0.1997493  -0.00489798]
     [ 0.08791196 -0.23749074  0.0180285   0.01389396 -0.00536418]
     [ 0.17417145 -0.05009837 -0.10913929  0.0518404  -0.1232994 ]
     [ 0.04740637 -0.08681157 -0.17421939  0.07702102 -0.10461716]
     [-0.00759859 -0.08235621 -0.07026755 -0.17090324  0.19734486]]

    [100.49708086   0.99603269   0.99283061   0.98404493   0.97890796]
    [100.49708086   0.99603269   0.99283061   0.98404493   0.97890796]
    [100.49708086+0.j   0.99603269+0.j   0.99283061+0.j   0.98404493+0.j
       0.97890796+0.j]

```python
# plot the two solutions and compare the values
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('ggplot')

fig, ax = plt.subplots()

ax.plot(Sigma['np'][1:10], label="SVD")
ax.plot(Sigma['asvd'][1:10], label='ASVD')
ax.plot(eigVals['scipy'][1:10], label='Generalized Eigenvalue Problem')
ax.set_title('First {k} Eigenvalues of Matrix A'.format(k=k))
ax.set_ylabel('$\lambda_i$')
ax.set_xlabel('Number $i$')
plt.legend(loc='upper right')

plt.show()
```

## Comparing Implementations

```python
# get some random matrix
from sklearn.datasets import make_spd_matrix

A = make_spd_matrix(100, random_state=0)

m = A.shape[0]  # left dimension of A
n = A.shape[1]  # right dimension of A

# choose some k singular values we want
k = 20

# set parameters
size_gauss = 60
pow_iter = 50

# initialize the dictionary of values
U={}; Sigma={}; V={}
```

##### Numpy's Implementation

```python
import numpy as np

U['np'], Sigma['np'], V['np'] = np.linalg.svd(A)

Sigma['np'] = Sigma['np'][:k]
V['np'] = V['np'].T[:,:k]
```

##### Facebook PCA's Implementation

```python
# import fbpca as fbpca

# Sigma['fbpca'], V['fbpca'] = fbpca.eigenn(A=A, k=k)
```

##### Scikit-Learn's Implementation

```python
from sklearn.utils.extmath import randomized_svd
# randomized SVD
U['rscikit'], Sigma['rscikit'], V['rscikit'] = randomized_svd(M=A, n_components=k,
                                                           n_iter=pow_iter)
V['rscikit'] = V['rscikit'].T
```

##### My Implementation

```python
U['asvd'], Sigma['asvd'], V['asvd'] = asvd(A, n_components=k,
                                           size_gauss=size_gauss,
                                           pow_iter=pow_iter)
```

##### Scipy Sparse Implementation

```python
from scipy.sparse.linalg import svds
from scipy.sparse import csr_matrix

U['scipy'], Sigma['scipy'], V['scipy'] = svds(A=csr_matrix(A),k=20,
                                              which='LM',
                                              maxiter=pow_iter)

V['scipy'] = V['scipy'].T
```

### Computing the Spectral Norms

```python
# Copied from FBPCA algorithm
import scipy

def diffsnorms(A, S, V, n_iter=20):
    """
    2-norm accuracy of a Schur decomp. of a matrix.
    Computes an estimate snorm of the spectral norm (the operator norm
    induced by the Euclidean vector norm) of A-VSV', using n_iter
    iterations of the power method started with a random vector;
    n_iter must be a positive integer.
    Increasing n_iter improves the accuracy of the estimate snorm of
    the spectral norm of A-VSV'.
    Notes
    -----
    To obtain repeatable results, reset the seed for the pseudorandom
    number generator.
    Parameters
    ----------
    A : array_like
        first matrix in A-VSV' whose spectral norm is being estimated
    S : array_like
        third matrix in A-VSV' whose spectral norm is being estimated
    V : array_like
        second matrix in A-VSV' whose spectral norm is being estimated
    n_iter : int, optional
        number of iterations of the power method to conduct;
        n_iter must be a positive integer, and defaults to 20
    Returns
    -------
    float
        an estimate of the spectral norm of A-VSV' (the estimate fails
        to be accurate with exponentially low probability as n_iter
        increases; see references DS1_, DS2_, and DS3_ below)
    Examples
    --------
    >>> from fbpca import diffsnorms, eigenn
    >>> from numpy import diag
    >>> from numpy.random import uniform
    >>> from scipy.linalg import svd
    >>>
    >>> A = uniform(low=-1.0, high=1.0, size=(2, 100))
    >>> A = A.T.dot(A)
    >>> (U, s, Va) = svd(A, full_matrices=False)
    >>> A = A / s[0]
    >>>
    >>> (d, V) = eigenn(A, 2)
    >>> err = diffsnorms(A, diag(d), V)
    >>> print(err)
    This example produces a rank-2 approximation V diag(d) V' to A
    such that the columns of V are orthonormal and the entries of d
    are nonnegative and are nonincreasing.
    diffsnorms(A, diag(d), V) outputs an estimate of the spectral norm
    of A - V diag(d) V', which should be close to the machine
    precision.
    References
    ----------
    .. [DS1] Jacek Kuczynski and Henryk Wozniakowski, Estimating the
             largest eigenvalues by the power and Lanczos methods with
             a random start, SIAM Journal on Matrix Analysis and
             Applications, 13 (4): 1094-1122, 1992.
    .. [DS2] Edo Liberty, Franco Woolfe, Per-Gunnar Martinsson,
             Vladimir Rokhlin, and Mark Tygert, Randomized algorithms
             for the low-rank approximation of matrices, Proceedings of
             the National Academy of Sciences (USA), 104 (51):
             20167-20172, 2007. (See the appendix.)
    .. [DS3] Franco Woolfe, Edo Liberty, Vladimir Rokhlin, and Mark
             Tygert, A fast randomized algorithm for the approximation
             of matrices, Applied and Computational Harmonic Analysis,
             25 (3): 335-366, 2008. (See Section 3.4.)
    See also
    --------
    eigenn, eigens
    """

    (m, n) = A.shape
    (m2, k) = V.shape
    (k2, k3) = S.shape

    assert m == n
    assert m == m2
    assert k == k2
    assert k2 == k3

    assert n_iter >= 1

    if np.isrealobj(A) and np.isrealobj(V) and np.isrealobj(S):
        isreal = True
    else:
        isreal = False

    # Promote the types of integer data to float data.
    dtype = (A * 1.0).dtype

    #
    # Generate a random vector x.
    #
    if isreal:
        x = np.random.normal(size=(n, 1)).astype(dtype)
    else:
        x = np.random.normal(size=(n, 1)).astype(dtype) \
            + 1j * np.random.normal(size=(n, 1)).astype(dtype)

    x = x / scipy.linalg.norm(x)

    #
    # Run n_iter iterations of the power method.
    #
    for it in range(n_iter):
        #
        # Set y = (A-VSV')x.
        #
        y = np.dot(A, x) - V.dot(S.dot(V.conj().T.dot(x)))
        #
        # Set x = (A'-VS'V')y.
        #
        x = np.dot(y.conj().T, A).conj().T \
            - V.dot(S.conj().T.dot(V.conj().T.dot(y)))

        #
        # Normalize x, memorizing its Euclidean norm.
        #
        snorm = scipy.linalg.norm(x)
        if snorm == 0:
            return 0
        x = x / snorm

    snorm = np.sqrt(snorm)

    return snorm
```

```python
# compute the norm for the difference between implementations
norm = {}

methods = ['rscikit', 'asvd', 'np', 'scipy']

for method in methods:
#     S = Sigma[method]
#     v = V[method]
#     print(S, v)
    norm[method] = diffsnorms(A, np.diag(Sigma[method]), V[method])
    print('Norm for {m} method: {n}'.format(m=method,n=norm[method]))

```

    Norm for rscikit method: 0.8394632337936275
    Norm for asvd method: 0.9374835480196004
    Norm for np method: 0.8347980612755976
    Norm for scipy method: 0.8270229685200706

I'll be honest, I have no idea what's a good or bad spectral norm value. Typically the lower the number the better. But both numbers are so close to 1 so maybe the higher the number the better? Who knows..

```python

```

```python

```

```python

```


# Load Packages for Julia

```julia
using DataFrames
using Distributions
using Plots
```

# Calculate Discrete Erlang Probabilities

The following function is to calculate the discrete truncated Erlang probability, given k and γ:

$$\begin{equation} p_i = \frac{1}{C(n^{E})} \Bigl(\sum_{j=0}^{k-1} \frac{e^{-(i-1)\gamma} \times ((i-1)\gamma)^{j}} {j!} -\sum_{j=0}^{k-1} \frac{e^{-i\gamma} \times (i\gamma)^{j}} {j!}\Bigr),\quad\text{for i=1,…,nE}. \end{equation} $$

where

$$\begin{equation} n^{E} = argmin_n\Bigl(C(n) = 1 - \sum_{j=0}^{k-1} \frac{e^{-n\gamma} \times (n\gamma)^{j}} {j!} > 0.99 \Bigr) \end{equation}$$

**N.B. The formula of pi here is slightly different from what is shown in the original paper because the latter (which is likely to be wrong) would lead to negative probabilities.**

```julia
#' @param k     The shape parameter of the Erlang distribution.
#' @param gamma The rate parameter of the Erlang distribution.
#' @return      A vector containing all p_i values, for i = 1 : n.
function compute_erlang_discrete_prob(k::Int64, gamma::Float64)
    n_bin = 0
    factorials = zeros(Int64, k + 1)
    factorials[1] = 1  ## factorials[1] = 0!
    for i = 1 : k
        factorials[i + 1] = factorials[i] * i  ## factorial[i + 1] = i!
    end

    one_sub_cummulative_probs = Float64[]
    cummulative_prob = 0
    while cummulative_prob <= 0.99
        n_bin = n_bin + 1
        push!(one_sub_cummulative_probs, 0)

        for j = 0 : (k - 1)
            one_sub_cummulative_probs[end] =
                one_sub_cummulative_probs[end] +
                (
                    exp( -n_bin * gamma )
                    * ( (n_bin * gamma) ^ j )
                    / factorials[j + 1]    ## factorials[j + 1] = j!
                )
        end
        cummulative_prob = 1 - one_sub_cummulative_probs[end]
    end
    one_sub_cummulative_probs = pushfirst!(one_sub_cummulative_probs, 1)

    density_prob = one_sub_cummulative_probs[1 : end - 1] - one_sub_cummulative_probs[2 : end]
    density_prob = density_prob / cummulative_prob

    return density_prob
end
```

    compute_erlang_discrete_prob (generic function with 1 method)

The implementation above calculates discrete probabilities pi’s base on the cummulative density function of the Erlang distribution:

$$\begin{equation} p_i = CDF_{Erlang}(x = i) - CDF_{Erlang}(x = i-1) \end{equation}$$

Meanwhile, the estimates of pi’s in the original paper seems to be based on the probability density function:

$$\begin{equation} p_i = PDF_{Erlang}(x = i) \end{equation}$$

While the two methods give slightly different estimates, they do not lead to any visible differences in the results of the subsequent simulations. This implementation uses the CDF function since it leads to faster runs.

# Simulate the SEIR Dynamics

The next function is to simulate the SEIR (susceptible, exposed, infectious, recovered) dynamics of an epidemic, assuming that transmission is frequency-dependent, i.e.

$$\begin{equation} \beta = \beta_0 \frac{I(t)}{N} \end{equation}$$

where N is the population size, I(t) is the number of infectious people at time t, and β0 is the base transmission rate.

This model does not consider births and deads (i.e. N is constant).

The rates at which individuals move through the E and the I classes are assumed to follow Erlang distributions of given shapes (kE, kI) and rates (γE, γI).

```julia
function seir_simulation( initial_state::Dict, parameters::Dict, max_time::Int64 )
    S_1 = get(initial_state, "S", -1)
    E_1 = get(initial_state, "E", -1)
    I_1 = get(initial_state, "I", -1)
    R_1 = get(initial_state, "R", -1)
    (S_1 != -1) || error("An initial value for S is required.")
    (E_1 != -1) || error("An initial value for E is required.")
    (I_1 != -1) || error("An initial value for I is required.")
    (R_1 != -1) || error("An initial value for R is required.")

    k_E     = get(parameters, "k_E", -1)
    gamma_E = get(parameters, "gamma_E", -1.0)
    k_I     = get(parameters, "k_I", -1)
    gamma_I = get(parameters, "gamma_I", -1.0)
    beta    = get(parameters, "beta", -1.0)
    (k_E != -1)       || error("Parameter k_E must be specified.")
    (gamma_E != -1.0) || error("Parameter gamma_E must be specified.")
    (k_I != -1)       || error("Parameter k_I must be specified.")
    (gamma_I != -1.0) || error("Parameter gamma_I must be specified.")
    (beta != -1.0)    || error("Parameter beta must be specified.")

    population_size = S_1 + E_1 + I_1 + R_1
    sim_data = DataFrame( S = Int64[], E = Int64[], I = Int64[], R = Int64[] )
    push!( sim_data, (S_1, E_1, I_1, R_1) )

    ## Initialise a matrix to store the states of the exposed sub-blocks over time.
    exposed_block_adm_rates = compute_erlang_discrete_prob(k_E, gamma_E)
    n_exposed_blocks = length(exposed_block_adm_rates)
    exposed_blocks = zeros(Int64, max_time, n_exposed_blocks)
    exposed_blocks[1, n_exposed_blocks] = sim_data[1, :E]

    ## Initialise a matrix to store the states of the infectious sub-blocks over time.
    infectious_block_adm_rates = compute_erlang_discrete_prob(k_I, gamma_I)
    n_infectious_blocks = length(infectious_block_adm_rates)
    infectious_blocks = zeros(Int64, max_time, n_infectious_blocks)
    infectious_blocks[1, n_infectious_blocks] = sim_data[1, :I]

    ## Run the simulation from time t = 2 to t = max_time
    for time = 2 : max_time
        transmission_rate = beta * sim_data[(time - 1), :I] / population_size
        exposure_prob = 1 - exp(-transmission_rate)
        distribution = Binomial(sim_data[(time - 1), :S], exposure_prob)
        new_exposed = rand(distribution)
        new_infectious = exposed_blocks[time - 1, 1]
        new_recovered  = infectious_blocks[time - 1, 1]

        if new_exposed > 0
            distribution =  Multinomial(new_exposed, exposed_block_adm_rates)
            exposed_blocks[time, :] = rand(distribution)
        end
        exposed_blocks[time, (1 : end - 1)] =
        exposed_blocks[time, (1 : end - 1)] + exposed_blocks[(time - 1), (2 : end)]

        if new_infectious > 0
            distribution =  Multinomial(new_infectious, infectious_block_adm_rates)
            infectious_blocks[time, :] = rand(distribution)
        end
        infectious_blocks[time, (1 : end - 1)] =
        infectious_blocks[time, (1 : end - 1)] + infectious_blocks[(time - 1), (2 : end)]

        push!( sim_data, ( sim_data[time - 1, :S] - new_exposed,
                           sum(exposed_blocks[time, :]),
                           sum(infectious_blocks[time, :]),
                           sim_data[time - 1, :R] + new_recovered )
        )
    end

    sim_data[:time] = collect(1 : max_time)
    return sim_data
end
```

    seir_simulation (generic function with 1 method)

To run a simulation, simply call the seirsimulation(…) method above.

Below is an example simulation where kE=5, γE=1, kI=10, γI=1, and β0=0.25 (R0=β0kIγI=2.5). The population size is N=10,000. The simmulation starts with 1 exposed case and everyone else belongs to the susceptible class. These settings are the same the the simulation 11 of the original paper.

**N.B. Since this is a stochastic model, there is chance for the outbreak not to occur even with a high R0.**

```julia
sim = seir_simulation( Dict("S" => 9999, "E" => 1, "I" => 0, "R" => 0),
                       Dict("k_E" =>  5, "gamma_E" => 1.0,
                            "k_I" => 10, "gamma_I" => 1.0,
                            "beta" => 0.25),
                       300 )
print(sim[end - 9 : end, :])
```

    10×5 DataFrame
    │ Row │ S     │ E     │ I     │ R     │ time  │
    │     │ Int64 │ Int64 │ Int64 │ Int64 │ Int64 │
    ├─────┼───────┼───────┼───────┼───────┼───────┤
    │ 1   │ 1000  │ 0     │ 0     │ 9000  │ 291   │
    │ 2   │ 1000  │ 0     │ 0     │ 9000  │ 292   │
    │ 3   │ 1000  │ 0     │ 0     │ 9000  │ 293   │
    │ 4   │ 1000  │ 0     │ 0     │ 9000  │ 294   │
    │ 5   │ 1000  │ 0     │ 0     │ 9000  │ 295   │
    │ 6   │ 1000  │ 0     │ 0     │ 9000  │ 296   │
    │ 7   │ 1000  │ 0     │ 0     │ 9000  │ 297   │
    │ 8   │ 1000  │ 0     │ 0     │ 9000  │ 298   │
    │ 9   │ 1000  │ 0     │ 0     │ 9000  │ 299   │
    │ 10  │ 1000  │ 0     │ 0     │ 9000  │ 300   │

```julia
gr(fmt = :png)
plot( sim[:, :time],
       [sim[:, :S], sim[:, :E], sim[:, :I], sim[:, :R]],
       label = ["Susceptible", "Exposed", "Infectious", "Recovered"],
       xlabel = "Time", ylabel = "Number of Individuals",
       linealpha = 0.5, linewidth = 3 )
```

```julia
using Random
Random.seed!(12345)
test_sim = seir_simulation( Dict("S" => 9999, "E" => 1, "I" => 0, "R" => 0),
                            Dict("k_E" =>  5, "gamma_E" => 1.0,
                                 "k_I" => 10, "gamma_I" => 1.0,
                                 "beta" => 0.25),
                            100 )
test_result = convert(Matrix, test_sim[end-2 : end, :])

correct_result = [ 6416  1017  1298  1269   98 ;
                   6210  1045  1379  1366   99 ;
                   6010  1086  1442  1462  100 ]

println("\n-------------------")
if correct_result == test_result
    println("    Test PASSED")
else
    println("    Test FAILED")
end
println("-------------------\n")
```

    -------------------
        Test PASSED
    -------------------


```python
from sympy.interactive import printing
printing.init_printing(use_latex= True)
from sympy import *
import sympy as sp

x =sp.Symbol('x')

f = sp.Function('f')(x)

diffeq = Eq(f.diff(x,x)-5*f,0)
display(diffeq)

dsolve(diffeq,f)
```

```python
import numpy as np
from scipy.integrate import odeint
%matplotlib inline
import matplotlib.pyplot as plt
```

```python
def firstorder(y,t,K,u):
    tau =5.0
    K = 2.0
    u = 1.0
    dydt =(-y +K*u)/tau
    return dydt

```

```python
t= np.linspace(0,10,11)
K = 2.0
u = 1.0

```

```python
y = odeint(firstorder,0,t,args=(K,u))

```

```python
plt.plot(t,y)
plt.title("Solving First Order D.E. in Python")
plt.xlabel("time")
plt.ylabel("values")
plt.show()
```

```python
import numpy as np
from scipy.integrate import odeint
%matplotlib inline
import matplotlib.pyplot as plt
```

```python
import numpy as np
import pandas as pd
from scipy.integrate import odeint
import matplotlib.pyplot as plt

def model(y,t):
    k = 0.045
    dydt = k * (100 - y)
    return dydt
y0 = 50

t = np.linspace(0,25)

y = odeint(model,y0,t)

plt.plot(t,y)
plt.xlabel('time')
plt.ylabel('y(t)')
plt.show()

Y = np.squeeze(np.asarray(y))
a = np.squeeze(np.asarray(t))
print(Y)
print(a)

```

```python
import numpy as np
import pandas as pd
from scipy.integrate import odeint
import matplotlib.pyplot as plt

def temperature(T,t):
    k = 0.045
    dTdt = k * (100 - T)
    return dTdt
y0 = 50

t = np.linspace(0,25)

y = odeint(temperature,y0,t)

plt.plot(t,y)
plt.xlabel('time')
plt.ylabel('T(t)')
plt.show()

Y = np.squeeze(np.asarray(T))
A = np.squeeze(np.asarray(t))
print(Y)
print(A)

```

```python
## 15 . 4 dakikada kaç derece olduğunu elle bulduk##
```

```python
import numpy as np
import pandas as pd
from scipy.integrate import odeint
import matplotlib.pyplot as plt

def model(y,t):
    k = 0.045
    dydt = k * (100 - y)
    return dydt
y0 = 68.41004006

t = np.linspace(10.20408163,15.4,2)

y = odeint(model,y0,t)

plt.plot(t,y)
plt.xlabel('time')
plt.ylabel('y(t)')
plt.show()

Y = np.squeeze(np.asarray(y))
T = np.squeeze(np.asarray(t))
print(Y)
print(T)

```

```python
## analitik solution https://youtu.be/xOzjeBaNfgo###
```

```python
from sympy.interactive import printing
printing.init_printing(use_latex= True)
from sympy import *
import sympy as sp

t =sp.Symbol('t')

y = sp.Function('y')(t)

diffeq = Eq(y.diff(t) * 10,-t*y)
display(diffeq)

dsolve(diffeq,y)
```

```python
## numeric solution https://youtu.be/xOzjeBaNfgo ##
```

```python
import numpy as np
import pandas as pd
from scipy.integrate import odeint
import matplotlib.pyplot as plt

def model(y,t):
    k = 10
    dydt = -t*y / k
    return dydt
y0 = 5

t = np.linspace(0,20,2)

y = odeint(model,y0,t)

plt.plot(t,y)
plt.xlabel('time')
plt.ylabel('y(t)')
plt.show()

Y = np.squeeze(np.asarray(y))
T = np.squeeze(np.asarray(t))
print(Y)
print(T)

```


```python
import numpy as np
import sympy as sym
from sympy.utilities.lambdify import lambdify
import matplotlib as mpl
import numpy as np
```

```python
sym.init_printing(use_unicode=True)
%matplotlib inline
mpl.style.use('seaborn-talk')
```

```python
x, t, z, = sym.symbols('x t z', real=True)
psi, phi,theta, pi, omega = sym.symbols('psi, phi theta, pi, omega', real=True)
alph1, alph2 = sym.symbols('\\alpha(\\theta), \\alpha(\\theta-1)', real=True)

c, rho, delta = sym.symbols('c, rho, delta', real=True, positive=True)
```

```python
alph1
```

```python
a = rho * x - rho * c * x**2
```

```python
a
```

```python
b = c * delta  * x + c**2 * omega * x**2
```

```python
alpha = psi * x - (1 - phi**2) * ( x**2 / 2)
```

```python
tmp = alpha.replace(x, -1)
```

```python
tmp
```

```python
b.replace(x, tmp + pi + alpha.replace(x, theta)).replace
```

```python
a.replace(x, tmp + pi + alpha.replace(x, theta))
```

```python
beta = alpha.replace(psi, psi - (1 - phi**2) / 2).replace(x, psi - (1 - phi**2))
```

```python
beta
```

```python
psi = sym.solveset(psi * rho + beta + 2 * psi**2 * c, psi)
```

```python
psi.replace(phi, 0)
```

```python
psi.replace(c,1)
```

```python
psi.replace(phi, 1).replace(c,1)
```

```python
psi.replace(rho, 0).replace(c,1)
```

```python
psi.replace(phi, 0).replace(c,1)
```

```python
psi_with_num = psi.replace(rho,.9).replace(c,1)
```

```python
psi0_func = lambdify(phi, psi_with_num.args[0])
psi1_func = lambdify(phi, psi_with_num.args[1])
```

```python
fig1, ax1 = mpl.pyplot.subplots()
xvals = np.linspace(-1,1,100)
yvals = [psi0_func(x) for x in xvals]
ax1.plot(xvals, yvals)
```

```python
fig2, ax2 = mpl.pyplot.subplots()
xvals = np.linspace(-1,1,100)
yvals = [psi1_func(x) for x in xvals]
ax2.plot(xvals, yvals)
```

```python
b
```

```python
psi
```

```python
gamma = b.replace(x, -x + ( 1- phi**2))
```

```python
psi2 = sym.solveset(x * rho + gamma + x**2 * c**2 * omega / 2, x)
```

```python
psi2.replace(c,1)
```

```python

```


$\def\*#1{\mathbf{#1}}$
$\DeclareMathOperator*{\argmax}{arg\,max}$

# Mathematical Models : the case of the Naive Bayes classifier

```python
from scipy import stats
import matplotlib.pyplot as plt
import numpy as np
from math import log
%matplotlib inline
```

## Classifier

A classifier is a function $M$ that predicts the class label $\hat{y} \in \{c_1, c_2, \ldots, c_k\}$ for a given point $\*x \in \mathbb{R}^d$ :

$$\hat{y} = M(\*x)$$

## Supervised Learning

1. Collect a **Training Data Set** composed of points along with their known classes
2. Train Classifier
3. Make Predictions

### Collect Training Data

A training set of points with their known classes :

| Day | Outlook | Temp | Humidity | Beach? |
|-----|---------|------|----------|--------|
| 1   | Sunny   | High | High     | Yes    |
| 2   | Sunny   | High | Normal   | Yes    |
| 3   | Sunny   | Low  | Normal   | No     |
| 4   | Sunny   | Mild | High     | Yes    |
| 5   | Rain    | Mild | Normal   | No     |
| 6   | Rain    | High | High     | No     |
| 7   | Rain    | Low  | Normal   | No     |
| 8   | Cloudy  | High | High     | No     |
| 9   | Cloudy  | High | Normal   | Yes    |
| 10  | Cloudy  | Mild | Normal   | No     |

## Probabilistic Classification

We consider a dataset $D = \{\*x_1, \ldots, \*x_n\}$, with $y_i \in \{c_1, c_2, \ldots, c_k\}$ for all $i = 1, 2, \ldots, n$.

The **predicted class** for a new data point $\*x$ is obtained as follows :

$$
\hat{y} = \argmax_{c_i}\{P(c_i | \*x)\}
$$

### Bayes' Theorem

$$
P(A|B) = \frac{P(B|A)P(A)}{P(B)}
$$

This allows to reduce question such as P(outcome|data) to P(data|outcome).

### Bayes Classifier

Based on this formula, The **predicted class** for a new data point $\*x$ is obtained as follows :
$$
\begin{align}
\hat{y} &= \argmax_{c_i}\{P(c_i | \*x)\}\\
        &= \argmax_{c_i}\Bigg\{\frac{P(\*x | c_i) P(c_i)}{P(\*x)}\Bigg\}\\
        &= \argmax_{c_i}\{P(\*x | c_i) P(c_i)\}
\end{align}
$$

**Prior Probability Estimator** :

$$
\hat{P}(c_i) = \frac{|\*D_i|}{|\*D|} = \frac{n_i}{n}
$$

where $\*D_i = \{\*x_j \in \*D : y_j = c_i\}$, $|\*D| = n$, and $|\*D_i| = n_i$.

### Naive Bayes Classifier
In this case we assume that attributes are independent. This allows to write the likelyhood as follows :

$$
P(\*x|c_i) = P(x_1, x_2, \ldots, x_n | c_i) = \prod_{j = 1}^d P(x_i|c_i)
$$

For **categorical attributes**,

$$
P(\*x|c_i) = \prod_{j = 1}^d P(x_i|c_i) \approx \prod_{j = 1}^d \hat{f}(\*v_j | c_i) = \prod_{j = 1}^d \frac{n_i(\*v_j)}{n_i}
$$

where $n_i(\*v_j)$ is the observed frequency of the value $\*v_j$ for the attribute $X_i$ in the class $c_i$.

| Day | Outlook | Temp | Humidity | Beach? |
|-----|---------|------|----------|--------|
| 1   | Sunny   | High | High     | Yes    |
| 2   | Sunny   | High | Normal   | Yes    |
| 3   | Sunny   | Low  | Normal   | No     |
| 4   | Sunny   | Mild | High     | Yes    |
| 5   | Rain    | Mild | Normal   | No     |
| 6   | Rain    | High | High     | No     |
| 7   | Rain    | Low  | Normal   | No     |
| 8   | Cloudy  | High | High     | No     |
| 9   | Cloudy  | High | Normal   | Yes    |
| 10  | Cloudy  | Mild | Normal   | No     |

| Outlook      | Beach | No Beach |
|--------------|-------|----------|
| Sunny        | 3/4   | 1/6      |
| Rain         | 0/4   | 3/6      |
| Cloudy       | 1/4   | 2/6      |

| Temperature  | Beach | No Beach |
|--------------|-------|----------|
| High         | 3/4   | 2/6      |
| Mild         | 1/4   | 2/6      |
| Low          | 0/4   | 2/6      |

| Humidity     | Beach | No Beach |
|--------------|-------|----------|
| High         | 2/4   | 2/6      |
| Normal       | 2/4   | 4/6      |

For example, based on the wather condition (Sunny, Mild, High), would this be a day to go to the beach ?

## Dealing with Zero counts

What happens when we consider (Sunny, Mild, Low), where the value of Low Humidity dow not appear in the training set ?

* P(Beach|(Sunny, Mild, Low)) = 0
* P(No Beach|(Sunny, Mild, Low)) = 0

Add-one **discounting** is a traditional statistical technique to address this problem : add one to the frequency of all outcomes. By applying this rules, the probabilities become what follows.

| Outlook      | Beach | No Beach |
|--------------|-------|----------|
| Sunny        | 4/7   | 2/9      |
| Rain         | 1/7   | 4/9      |
| Cloudy       | 2/7   | 3/9      |

| Temperature  | Beach | No Beach |
|--------------|-------|----------|
| High         | 4/7   | 3/9      |
| Mild         | 2/7   | 3/9      |
| Low          | 1/7   | 3/9      |

| Humidity     | Beach | No Beach |
|--------------|-------|----------|
| High         | 3/7   | 3/9      |
| Normal       | 3/7   | 5/9      |
| Low          | 1/7   | 1/9      |

## Multivariate bernoulli models

```python
# Sunny, Rain, Cloudy, Low, Mild, High, Normal, High
data = np.array([
    [1, 0, 0, 0, 0, 1, 0, 1],
    [1, 0, 0, 0, 0, 1, 1, 0],
    [1, 0, 0, 1, 0, 0, 1, 0],
    [1, 0, 0, 0, 1, 0, 0, 1],
    [0, 1, 0, 0, 1, 0, 1, 0],
    [0, 1, 0, 0, 0, 1, 0, 1],
    [0, 1, 0, 1, 0, 0, 1, 0],
    [0, 0, 1, 0, 0, 1, 0, 1],
    [0, 0, 1, 0, 0, 1, 1, 0],
    [0, 0, 1, 0, 1, 0, 1, 0]])

target = np.array([1, 1, 0, 1, 0, 0, 0, 0, 1, 0])

from sklearn.naive_bayes import BernoulliNB
bnb = BernoulliNB()
bnb.fit(data, target)
bnb.predict_proba([[1, 0, 0, 0, 1, 0, 0, 1]])
```

This model assume independence between some variables which is obviously wrong. Why ?

## Multinomial models

```python
from sklearn.naive_bayes import MultinomialNB
mnb = MultinomialNB(alpha=1.0e-10)
mnb.fit(data, target)
mnb.predict_proba([[1, 0, 0, 0, 1, 0, 0, 1]])
```

## Numerical Attributes

For *numerical attributes*, we can, for example, assume a normal distribution for each clas $c_i$ :

$$
P(x_j | c_i) \propto f(x_j) = \frac{1}{\sqrt{2\sigma_{ij}^2\pi} } \; e^{ -\frac{(x_-\mu_{ij})^2}{2\sigma_{ij}^2} }
$$

where $\mu_{ij}$ and $\sigma_{ij}^2$ denote the mean and variance for attribute $X_j$ and class $c_i$.

```python
from sklearn import datasets

iris = datasets.load_iris()

from sklearn.naive_bayes import GaussianNB

gnb = GaussianNB()

y_pred = gnb.fit(iris.data, iris.target).predict(iris.data)

print("Number of mislabeled points out of a total {} points : {}".format(iris.data.shape[0],
                                                                         (iris.target != y_pred).sum()))
```

## Classifier Evaluation

### Performance Measure

* $\*D$ : the data set composed of $n$ points in a $d$ dimensional space
* $k$ : the number of classes
* $M$ : the classifier
* $y_i$ : the "true" class that corresponds to $\*x_i \in \*D$
* $\hat{y}_i = M(\*x_i)$ : the predicted class

#### Error Rate

The fraction of incorrect predictions :

$$
Error\ Rate = \frac{1}{n} \sum_{i = 1}^n I(y_i \neq \hat{y}_i)
$$

#### Accuracy

The fraction of correct predictions :
$$
Accuracy = \frac{1}{n} \sum_{i = 1}^n I(y_i = \hat{y}_i) = 1 - Error\ Rate
$$

### Evaluation

The data set $\*D$ is split into two disjoint sets :

* **Training set** : used to train $M$
* **Testing set** : used to evaluate the performance measure (denoted $\theta$) of $M$

#### $K$-fold Cross-Validation

The data set $\*D$ is divided into $K$ equal-sized parts, denoted $\*D_1, \*D_2, \ldots, \*D_K$, called folds.

For each $i = 1,2,\ldots,K$, a model $M_i$ is trained and the corresponding measure $\theta_i$ is evaluated with respect to :

* $\*D_i$ : the testing set
* $\*D \backslash \*D_i = \cup_{j \neq i} \*D_j$ : the training set

This lead estimate the expected and variance values of the performance measure as follows,

* $\displaystyle \hat{\mu}_{\theta} = E[\theta] = \frac{1}{K} \sum_{i=1}^K \theta_i$
* $\displaystyle \hat{\sigma}_{\theta}^2 = \frac{1}{K} \sum_{i=1}^K (\theta_i - \hat{\mu}_{\theta})^2$

When $K = 1$ this validation method is called the **leave-one-out** cross-validation method.

## References :

* **The Data Science Design Manual**, by Steven Skiena, 2017, Springer;
* Python notebooks available at [http://data-manual.com/data](http://data-manual.com/data);
* Lectures slides available at [http://www3.cs.stonybrook.edu/~skiena/data-manual/lectures/](http://www3.cs.stonybrook.edu/~skiena/data-manual/lectures/).
* The Google Developers channel : Machine Learning Recipes
* Spring 2015 Boston University CS591 "Tools and Techniques for Data Mining and Applications" course