```julia
"""Tutorial: Overlap, Kinetic, and Dipole Integrals"""

__authors__   = ["D. Menendez", "Adam S. Abbott", "Boyi Zhang", "Justin M. Turney"]
__credit__    = ["D. Menendez", "Adam S. Abbott", "Boyi Zhang", "Justin M. Turney"]

__copyright__ = "(c) 2014-2020, The Psi4Julia Developers"
__license__   = "BSD-3-Clause"
__date__      = "2020-08-03"
```

# Overlap, Kinetic, and Dipole Integrals

In this tutorial we will compute the overlap and kinetic energy integrals encountered in Hartree-Fock. We will also compute the dipole integrals to obtain the molecular dipole moment.

## Recurrence formula for one-electron integrals over Gaussian functions
The direct calculation of every single integral in our arrays is not only a complicated procedure, but an expensive one too. Here, we will use the Obara-Saika recursion scheme for a simpler and more efficient implementation. For example, for the overlap integrals, given the value of $(s|s)$, one can recursively determine all other overlap integrals.

### Cartesian Gaussian functions
Denote the origin of a 3-dimensional cartesian gaussian function by the coordinates $\mathbf{R} = (R_x, R_y, R_z)$.
Let $\mathbf{r} = (x, y, z)$ be the coordinates of the electron, and $\alpha$ be the orbital exponent. We can now define an *unnormalized* Cartesian Gaussian function as:

\begin{equation}
\phi(\mathbf r; \alpha, \mathbf n, \mathbf R) = (x - R_x)^{n_x} (y - R_y)^{n_y} (z - R_z)^{n_z} \exp[-\alpha (\mathbf r - \mathbf R)^2]
\end{equation}

where $\alpha$ is the orbital exponent, and $\mathbf{n} = (n_x, n_y, n_z)$ is the angular momentum index vector. The sum $n_x + n_y + n_z = \lambda$ will hereafter be referred to as the **angular momentum**. We define a **shell** to be a set of functions (*components*) which share the same origin $\mathbf{R}$, and angular momentum $\lambda$.

The shells with $\lambda$  equal to $0, 1, 2,...,$ are referred to as the $s, p, d, ...$ shell. Each shell has $(\lambda + 1) (\lambda + 2)/2$ components. The $s$ shell, with angular momentum $\lambda = 0$ has one component usually designated as $s$. The $p$ shell ($\lambda = 1$) has three components, designated as $p_x, p_y, p_z$. The $d$ shell ($\lambda = 2$) has six components, designated as $d_{xx}, d_{yy}, d_{zz}, d_{xy}, d_{xz}, d_{yz}$.

In quantum chemistry, we typically represent a single component (an **atomic orbital**) by a linear combination of Gaussians ($c_1 \phi_1 + c_2 \phi_2 + c_3 \phi_3 ...$) hereon referred to as **primitives**. For example, the STO-3G basis set uses three primitives for each atomic orbital basis function. Each primitive is weighted by a coefficient $c$.

Using our angular momentum index vector $\mathbf{n}$ we denote the single component of the $s$ shell to have angular momentum index $\mathbf{n} = \mathbf{0} = (0, 0, 0)$. Since the $p$ shell has three components, we may compactly express the angular momentum index vector as $\mathbf{1}_i$ where $i$ may be $x$, $y$, or $z$, and $\mathbf{1}_i = (\delta_{ix}, \delta_{iy}, \delta_{iz})$. For example, $p_x$ may be represented as $\mathbf{1}_x = (1, 0, 0)$. For the six components of the $d$ shell, we require a sum of two angular momentum index vectors $\mathbf{1}_i + \mathbf{1}_j$, where $(i,j = x,y,z)$. In this notation, the $d_{xy}$ component is $\mathbf{1}_x + \mathbf{1}_y = (1,0,0) + (0,1,0) = (1,1,0)$. To obtain higher order angular momentum components, we add the appropriate number of $\mathbf{1}_i$'s ($\mathbf{n}$'s).

### Two-center overlap integrals
Two-center overlap integrals over unnormalized cartesian gaussian functions are of the form:
\begin{equation}
(\mathbf a|\mathbf b) = \int d\mathbf r\ \phi(\mathbf r; \alpha_a, \mathbf a, \mathbf A)\phi(\mathbf r; \alpha_b, \mathbf b, \mathbf B)
\end{equation}

Given $(\mathbf 0_A | \mathbf 0_B)$, we can use the Obara-Saika recursion relation to obtain overlap integrals between all basis functions. The overlap over $s$ functions is given by:

\begin{equation}
(\mathbf 0_A | \mathbf 0_B) = \left(\frac{\pi}{\alpha}\right)^{3/2} \exp[-\xi(\mathbf A-\mathbf B)^2]
\end{equation}

where $\alpha = \alpha_a + \alpha_b$ and $\zeta = \frac{\alpha_a\alpha_b}{\alpha}$.
The recursion relations are given below. For a full derivation, see the appendix, or the
[original paper](http://aip.scitation.org/doi/abs/10.1063/1.450106) by Obara and Saika. To increment the left side angular momentum:

\begin{equation}
(\mathbf a+\mathbf 1_i|\mathbf b) = (\mathbf{P - A})(\mathbf a|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf a)(\mathbf a-\mathbf 1_i|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf b)(\mathbf a|\mathbf b-\mathbf 1_i)
\end{equation}
and similarily, to increment the right side:
\begin{equation}
(\mathbf a|\mathbf b+\mathbf 1_i) = (\mathbf{P - B})(\mathbf a|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf a)(\mathbf a-\mathbf 1_i|\mathbf b) + \frac{1}{2\alpha} N_i(\mathbf b)(\mathbf a|\mathbf b-\mathbf 1_i)
\end{equation}

where \begin{equation}\mathbf{P} = \frac{\alpha_a \mathbf{A} + \alpha_b \mathbf{B}} {\alpha} \end{equation}

and ${N}_i(\mathbf{a})$, ${N}_i(\mathbf{b})$ are just the angular momenta of $\mathbf{a}$ and $\mathbf{b}$.

To fill in the first row, the seoncd term goes to zero, since we cannot have a negative angular momentum (cannot subtract $\mathbf 1_i$). To fill in the first column, the third term goes to zero for the same reason.

# Implementation
### Write the recursion function

The Obara-Saika recursion relationships depend on $\mathbf{P-A}$,  $\mathbf{P-B}$, $\alpha$, and the angular momentum values for $\mathbf a$ and $\mathbf b$, which we will denote as `PA`, `PB`, `alpha`, `AMa`, and `AMb`.
Let's write a function that takes these parameters and returns three matrices containing the x, y and z components of our unnormalized overlap integrals. These same components also can be used to construct our kinetic energy and dipole integrals later, as we will see.

For now, we will set $(\mathbf 0_A | \mathbf 0_B)$ to $1.0$ for simplicity. Later, we will set the value as defined above.

```julia
using PyCall: pyimport
psi4 = pyimport("psi4")
np   = pyimport("numpy") # used only to cast to Psi4 arrays

function os_recursion(PA, PB, alpha, AMa, AMb)
    if length(PA) != 3 || length(PB) != 3
       error("PA and PB must be xyz coordinates.")
    end

    # Allocate space x, y, and z matrices
    # We add one because the equation for the kinetic energy
    # integrals require terms one beyond those in the overlap
    x = zeros(AMa + 1, AMb + 1)
    y = zeros(AMa + 1, AMb + 1)
    z = zeros(AMa + 1, AMb + 1)

    # Define 1/2alpha factor for convenience
    oo2a = 1 / (2alpha)

    # Set initial conditions (0a|0b) to 1.0 for each cartesian component
    x[1, 1] = y[1, 1] = z[1, 1] = 1.0

    # BEGIN RECURSION
    # Fill in the [0,1] position with PB
    if AMb > 0
        x[1, 2] = PB[1]
        y[1, 2] = PB[2]
        z[1, 2] = PB[3]
    end

    # Fill in the rest of row zero
    for b in 2:AMb
        x[1, b+1] = PB[1] * x[1, b] + b * oo2a * x[1, b - 1]
        y[1, b+1] = PB[2] * y[1, b] + b * oo2a * y[1, b - 1]
        z[1, b+1] = PB[3] * z[1, b] + b * oo2a * z[1, b - 1]
    end

    # Now, we have for each cartesian component
    # | 1.0  PB #  #|
    # |  0   0  0  0|
    # |  0   0  0  0|
    # |  0   0  0  0|

    # Upward recursion in a for all b's
    # Fill in the [1,0] position with PA
    if AMa > 0
        x[2, 1] = PA[1]
        y[2, 1] = PA[2]
        z[2, 1] = PA[3]

    # Now, we have for each cartesian component
    # | 1.0  PB #  #|
    # |  PA  0  0  0|
    # |  0   0  0  0|
    # |  0   0  0  0|

        # Fill in the rest of row one
        for b in 2:AMb
            x[2, b] = PA[1] * x[1, b] + b * oo2a * x[1, b - 1]
            y[2, b] = PA[2] * y[1, b] + b * oo2a * y[1, b - 1]
            z[2, b] = PA[3] * z[1, b] + b * oo2a * z[1, b - 1]
        end

        # Now, we have for each cartesian component
        # | 1.0  PB #  #|
        # |  PA  #  #  #|
        # |  0   0  0  0|
        # |  0   0  0  0|

        # Fill in the rest of column 0
        for a in 2:AMa
            x[a + 1, 1] = PA[1] * x[a, 1] + a * oo2a * x[a - 1, 1]
            y[a + 1, 1] = PA[2] * y[a, 1] + a * oo2a * y[a - 1, 1]
            z[a + 1, 1] = PA[3] * z[a, 1] + a * oo2a * z[a - 1, 1]

        # Now, we have for each cartesian component
        # | 1.0  PB #  #|
        # |  PA  #  #  #|
        # |  #   0  0  0|
        # |  #   0  0  0|

        # Fill in the rest of the a'th row
            for b in 2:AMb
                x[a + 1, b] = PA[1] * x[a, b] + a * oo2a * x[a - 1, b] + b * oo2a * x[a, b - 1]
                y[a + 1, b] = PA[2] * y[a, b] + a * oo2a * y[a - 1, b] + b * oo2a * y[a, b - 1]
                z[a + 1, b] = PA[3] * z[a, b] + a * oo2a * z[a - 1, b] + b * oo2a * z[a, b - 1]
            end
        end

        # Now, we have for each cartesian component
        # | 1.0  PB #  #|
        # |  PA  #  #  #|
        # |  #   #  #  #|
        # |  #   #  #  #|
    end

    # Return the results
    (x, y, z)
end
```

# Overlap Integrals

Now that we have our recursion set up, we are ready to compute the integrals, starting with the overlap integrals. First we need a molecule and a basis set. We do a quick Hartree-Fock computation to obtain a wavefunction for easy access to the basis set. Later, we will also use the wavefunction's density matrix to compute the dipole moment.

By default, Psi4 uses spherical harmonic Gaussians, sometimes referred to as "pure angular momentum", whereas we will be using Cartesian Gaussians. We set `puream` to be off so that we can compare our integrals to Psi4's.

```julia
psi4.core.set_output_file("output.dat", false)

mol = psi4.geometry("""
                        O
                        H 1 1.1
                        H 1 1.1 2 104
                        symmetry c1
                        """)

psi4.set_options(Dict("basis"         => "sto-3g",
                      "scf_type"      => "pk",
                      "mp2_type"      => "conv",
                      "puream"        => 0,
                      "e_convergence" => 1e-8,
                      "d_convergence" => 1e-8))

scf_e, scf_wfn = psi4.energy("scf", return_wfn=true)

basis = scf_wfn.basisset()
```

The factored form of the two-center overlap integrals is
\begin{equation}
(\mathbf a | \mathbf b) = \space I_x(n_{ax},n_{bx},n_{cx})\ I_y(n_{ay},n_{by},n_{cy})\ I_z(n_{az},n_{bz},n_{cz})
\end{equation}

where $I_x$ is the x component of the unnormalized overlap matrix.

To obtain the overlap matrix, we must loop over all shells, and for each shell, loop over the primitive gaussians.

~~~julia
# make space to store the overlap integral matrix
S = zeros(basis.nao(),basis.nao())

# loop over the shells, basis.nshell is the number of shells
for i in 1:basis.nshell(), j in 1:basis.nshell()
    # basis.shell is a shell (1s, 2s, 2p, etc.)
    # for water, there are 5 shells: (H: 1s, H: 1s, O: 1s, 2s, 2p)
    ishell = basis.shell(i-1)
    jshell = basis.shell(j-1)
    # each shell has some number of primitives which make up each component of a shell
    # sto-3g has 3 primitives for every component of every shell.
    nprimi = ishell.nprimitive
    nprimj = jshell.nprimitive
~~~

At this point we access the information for each primitive basis set $\mathbf{a}$ and $\mathbf{b}$ in $(\mathbf{a}|\mathbf{b})$. The `basis.shell()` (which we have set to `ishell` and `jshell`) is a `psi4.core.GaussianShell` object, which contains all the information we need about the primitives within a shell, such as the exponent `(shell().exp())`, the coefficient `(shell().coef())`, the angular momentum `(shell().am)` and the atom number it is centered over `(shell().ncenter)`, which we can use to get the coordinates of the primitive center ($\mathbf{A}$ or $\mathbf{B}$) (*whew!*). We store all of these parameters as variables in our for-loops for easy access. Continuing our sample script:

~~~julia
    # loop over the primitives within a shell
    for a in 1:nprimi, b in 1:nprimj
        expa = ishell.exp(a-1) # primitive exponents
        expb = jshell.exp(b-1)
        coefa = ishell.coef(a-1)  # primitive coefficients
        coefb = jshell.coef(b-1)
        AMa = ishell.am  # angular momenta
        AMb = jshell.am
        # defining centers for each basis function:
        # mol.x() returns the x coordinate of the atom given by ishell.ncenter
        # we use this to define a coordinate vector for our centers
        A = [mol.x(ishell.ncenter), mol.y(ishell.ncenter), mol.z(ishell.ncenter)]
        B = [mol.x(jshell.ncenter), mol.y(jshell.ncenter), mol.z(jshell.ncenter)]
~~~

We are now ready to define `PA` and `PB` which are arguments for our recursion function.

~~~julia
        alpha = expa + expb
        zeta = (expa * expb) / alpha
        P = (expa * A + expb * B) / alpha
        PA = P - A
        PB = P - B
        AB = A - B
~~~

We also define the value of the $(s|s)$ overlap `start`, originally set to 1.0 in the recursion function.

\begin{equation}
s = (\mathbf 0_A | \mathbf 0_B) = \left(\frac{\pi}{\alpha}\right)^{3/2} \exp[-\zeta(\mathbf A-\mathbf B)^2]
\end{equation}

~~~julia
       start = (π / alpha)^(3 / 2) * exp(-zeta * (AB[1]^2 + AB[2]^2 + AB[3]^2))
~~~

Form the x-component, y-component, and z-component overlap matrices with our recursion function. Recall that we set $\mathbf{(0|0)} = 1.0$ and have ignored using the coefficients thus far. We will account for this later when we build each element of our overlap matrix.

~~~julia
       # call the recursion
       x, y, z = os_recursion(PA, PB, alpha, AMa+1, AMb+1)
~~~

We are now ready to assign the elements of our overlap matrix. Currently, we are only looping over the shells, but not the individual components of each shell (e.g. we are not distinguishing between $p_x$, $p_y$, and $p_z$). Given the total angular momentum of **a** (`AMa`) and **b** (`AMb`), we loop over all possible allocations of this total angular momentum to each angular momentum index in $\mathbf{n} = (n_x, n_y, n_z)$ which we denote as `l`, `m`, and `n`.

~~~julia
        # Basis function index where the shell begins
        i_idx = ishell.function_index
        j_idx = jshell.function_index

        # We use counters to keep track of which component (e.g., p_x, p_y, p_z)
        # within the shell we are on
        counta = 1

        for p in 1:AMa
            la = AMa - p                    # Let l take on all values, and p be the leftover a.m.
            for q in 1:p
                ma = p - q                  # distribute all leftover a.m. to m and n
                na = q
                countb = 1
                for r in 1:AMb
                    lb = AMb - r            # Let l take on all values, and r the leftover a.m.
                    for s in 1:r
                        mb = r - s          # distribute all leftover a.m. to m and n
                        nb = s

                        # set the value in the full overlap matrix
                        S[i_idx + counta, j_idx + countb] += start    *
                                                             coefa    *
                                                             coefb    *
                                                             x[la+1,lb+1] *
                                                             y[ma+1,mb+1] *
                                                             z[na+1,nb+1]

                        countb += 1
                    end
                end
                counta += 1
            end
        end
    end
end
~~~

## Computing the Full Overlap Integral Matrix

Putting all the above code snippets together we form our full overlap matrix:

```julia
# make space to store the overlap integral matrix
S = zeros(basis.nao(),basis.nao())

# loop over the shells, basis.nshell is the number of shells
for i in 1:basis.nshell(), j in 1:basis.nshell()
    # basis.shell is a shell (1s, 2s, 2p, etc.)
    # for water, there are 5 shells: (H: 1s, H: 1s, O: 1s, 2s, 2p)
    ishell = basis.shell(i-1)
    jshell = basis.shell(j-1)
    # each shell has some number of primitives which make up each component of a shell
    # sto-3g has 3 primitives for every component of every shell.
    nprimi = ishell.nprimitive
    nprimj = jshell.nprimitive
    # loop over the primitives within a shell
    for a in 1:nprimi, b in 1:nprimj
        expa = ishell.exp(a-1) # exponents
        expb = jshell.exp(b-1)
        coefa = ishell.coef(a-1)  # coefficients
        coefb = jshell.coef(b-1)
        AMa = ishell.am  # angular momenta
        AMb = jshell.am
        # defining centers for each basis function
        # mol.x() returns the x coordinate of the atom given by ishell.ncenter
        # we use this to define a coordinate vector for our centers
        A = [mol.x(ishell.ncenter), mol.y(ishell.ncenter), mol.z(ishell.ncenter)]
        B = [mol.x(jshell.ncenter), mol.y(jshell.ncenter), mol.z(jshell.ncenter)]

        alpha = expa + expb
        zeta = (expa * expb) / alpha
        P = (expa * A + expb * B) / alpha
        PA = P - A
        PB = P - B
        AB = A - B
        start = (π / alpha)^(3 / 2) * exp(-zeta * (AB[1]^2 + AB[2]^2 + AB[3]^2))

        # call the recursion
        x, y, z = os_recursion(PA, PB, alpha, AMa+1, AMb+1)

        # Basis function index where the shell begins
        i_idx = ishell.function_index
        j_idx = jshell.function_index

        # We use counters to keep track of which component (e.g., p_x, p_y, p_z)
        # within the shell we are on
        counta = 1

        for p in 1:AMa
            la = AMa - p                    # Let l take on all values, and p be the leftover a.m.
            for q in 1:p
                ma = p - q                  # distribute all leftover a.m. to m and n
                na = q
                countb = 1
                for r in 1:AMb
                    lb = AMb - r            # Let l take on all values, and r the leftover a.m.
                    for s in 1:r
                        mb = r - s          # distribute all leftover a.m. to m and n
                        nb = s

                        # set the value in the full overlap matrix
                        S[i_idx + counta, j_idx + countb] += start    *
                                                             coefa    *
                                                             coefb    *
                                                             x[la+1,lb+1] *
                                                             y[ma+1,mb+1] *
                                                             z[na+1,nb+1]

                        countb += 1
                    end
                end
                counta += 1
            end
        end
    end
end
```

Check our overlap against Psi4:

```julia
mints = psi4.core.MintsHelper(basis)
Spsi4 = np.asarray(mints.ao_overlap())
@assert np.allclose(S, Spsi4, 6)
```

# Kinetic energy integrals
Before proceeding to an analysis of the kinetic energy integral, it will prove convenient to establish a short-hand notation for integrals related to the overlap integral:
\begin{equation}
(0 \mid 0) = \int G(\alpha_a, \mathbf A, l_a, m_a, n_a) G(\alpha_b, \mathbf B, l_b, m_b, n_b)\ d\tau
\end{equation}
The symbol $(+1 \mid 0)_{\mathrm{x}}$ will denote an integral of the form given by the above equation, except that the quantum number $l_a$ has been incremented by 1. Similarly, $(0 \mid +1)_{\mathrm{x}}$ increments $l_b$ by 1. Quantum numbers $m$ and $n$ are incremented in the same way with subscripts $y$ and $z$.

The kinetic energy integral is defined as

\begin{equation}
T = - \frac{1}{2}  \int G(\alpha_a, \mathbf A, l_a, m_a, n_a) \left(\frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} + \frac{\partial^2}{\partial z^2}\right) G(\alpha_b, \mathbf B, l_b, m_b, n_b)\ d\tau
\end{equation}

The equation for this integral is "unsymmetric" because the quantum numbers of the Gaussian-type function centered on $\mathbf B$ are altered by the Laplacian while those of the Gaussian-type function centered on $\mathbf A$ are not. The symmetric $x$, $y$, and $z$ components are given by:

\begin{equation}
T_x = \frac{1}{2}\Big\{l_a l_b (-1|-1)_{\mathrm{x}} + 4\alpha_a \alpha_b (+1|+1)_{\mathrm{x}} - 2\alpha_a l_b (+1|-1)_{\mathrm{x}} - 2\alpha_b l_a (-1|+1)_{\mathrm{x}}\Big\}
\end{equation}

\begin{equation}
T_y = \frac{1}{2}\Big\{l_a l_b (-1|-1)_{\mathrm{y}} + 4\alpha_a \alpha_b (+1|+1)_{\mathrm{y}} - 2\alpha_a l_b (+1|-1)_{\mathrm{y}} - 2\alpha_b l_a (-1|+1)_{\mathrm{y}}\Big\}
\end{equation}

\begin{equation}
T_z = \frac{1}{2}\Big\{l_a l_b (-1|-1)_{\mathrm{z}} + 4\alpha_a \alpha_b (+1|+1)_{\mathrm{z}} - 2\alpha_a l_b (+1|-1)_{\mathrm{z}} - 2\alpha_b l_a (-1|+1)_{\mathrm{z}}\Big\}
\end{equation}

and the full kinetic energy integral is:

\begin{equation}
T =  (T_x + T_y + T_z)
\end{equation}

Within our same for-loop structure as for the overlap integrals, we can compute the kinetic energy integrals with:

~~~julia
Tx = (1 / 2) * (la * lb * x[lam, lbm] + 4 * expa * expb * x[la + 2, lb + 2] -
       2 * expa * lb * x[la + 2, lbm] - 2 * expb * la * x[lam, lb + 2])   *
       y[ma+1, mb+1] * z[na+1, nb+1]

Ty = (1 / 2) * (ma * mb * y[mam, mbm] + 4 * expa * expb * y[ma + 2, mb + 2] -
       2 * expa * mb * y[ma + 2, mbm] - 2 * expb * ma * y[mam, mb + 2])   *
       x[la+1, lb+1] * z[na+1, nb+1]

Tz = (1 / 2) * (na * nb * z[nam, nbm] + 4 * expa * expb * z[na + 2, nb + 2] -
       2 * expa * nb * z[na + 2, nbm] - 2 * expb * na * z[nam, nb + 2])   *
       x[la+1, lb+1] * y[ma+1, mb+1]

# incorporate the value of (0|0) and coefficients

T[i_idx + counta, j_idx + countb] += start * coefa * coefb * (Tx + Ty + Tz)
~~~

# Dipole moment integrals
We discuss these non-energy integrals here because they are frequently used and closely related to overlap and kinetic energy integrals. The dipole moment is defined with respect to a point in space $\mathbf C$, which is almost always taken to be the center of mass. Fortunately for our purposes, Psi4 automatically moves the molecule to be centered on the center of mass and thus $\mathbf C$ is $\mathbf 0$. The dipole moment integral is written for the $x$ direction as

\begin{equation}
d_{\mathrm{x}} =  \int G(\alpha_a, \mathbf A, l_a, m_a, n_a) G(\alpha_b, \mathbf B, l_b, m_b, n_b) \mathrm{x_\mathbf{C}}\ d\tau
\end{equation}

and similarly for the operators $\mathrm{y_\mathbf{C}}$ and $\mathrm{z_\mathbf{C}}$. Here, $x_\mathbf{C}$ is

\begin{equation}
\mathrm{x_\mathbf{C}} = \mathrm{x} - \mathrm{C_x}
\end{equation}

A convenient procedure is to redefine $\mathrm{x_\mathbf{C}}$ in terms of  $\mathrm{x_\mathbf{A}}$ or $\mathrm{x_\mathbf{B}}$ (we will use $\mathrm{x_\mathbf{A}}$):

\begin{equation}
\mathrm{x_\mathbf{C}}= (\mathrm{x - A_x}) + (\mathrm{A_x - C_x}) = \mathrm{x_\mathbf{A}} + (\mathrm{A_x - C_x})
\end{equation}

and we find

\begin{equation}
d_\mathrm{x} = (+1|0)_{\mathrm{x}} + (\mathrm{A_x - C_x}) (0|0)
\end{equation}

and the expressions are similar for $d_{\mathrm{y}}$ and $d_{\mathrm{z}}$. Putting this into code:

~~~julia
dx = (x[la + 2, lb+1] + A[1] * x[la+1, lb+1]) * y[ma+1, mb+1] * z[na+1, nb+1]
dy = (y[ma + 2, mb+1] + A[2] * y[ma+1, mb+1]) * x[la+1, lb+1] * z[na+1, nb+1]
dz = (z[na + 2, nb+1] + A[3] * z[na+1, nb+1]) * x[la+1, lb+1] * y[ma+1, mb+1]

# incorporate the (0|0) value and the coefficients:

Dx[i_idx + counta, j_idx + countb] += start * coefa * coefb * dx
Dy[i_idx + counta, j_idx + countb] += start * coefa * coefb * dy
Dz[i_idx + counta, j_idx + countb] += start * coefa * coefb * dz
~~~

# Full Implementation

Putting everything together, we get the overlap, kinetic, and dipole integrals:

```julia
# make space to store the overlap, kinetic, and dipole integral matrices
S = zeros(basis.nao(),basis.nao())
T = zeros(basis.nao(),basis.nao())
Dx = zeros(basis.nao(),basis.nao())
Dy = zeros(basis.nao(),basis.nao())
Dz = zeros(basis.nao(),basis.nao())

# loop over the shells, basis.nshell is the number of shells
for i in 1:basis.nshell(), j in 1:basis.nshell()
    # basis.shell is a shell (1s, 2s, 2p, etc.)
    # for water, there are 5 shells: (H: 1s, H: 1s, O: 1s, 2s, 2p)
    ishell = basis.shell(i-1)
    jshell = basis.shell(j-1)
    # each shell has some number of primitives which make up each component of a shell
    # sto-3g has 3 primitives for every component of every shell.
    nprimi = ishell.nprimitive
    nprimj = jshell.nprimitive
    # loop over the primitives within a shell
    for a in 1:nprimi, b in 1:nprimj
        expa = ishell.exp(a-1) # exponents
        expb = jshell.exp(b-1)
        coefa = ishell.coef(a-1)  # coefficients
        coefb = jshell.coef(b-1)
        AMa = ishell.am  # angular momenta
        AMb = jshell.am
        # defining centers for each basis function
        # mol.x() returns the x coordinate of the atom given by ishell.ncenter
        # we use this to define a coordinate vector for our centers
        A = [mol.x(ishell.ncenter), mol.y(ishell.ncenter), mol.z(ishell.ncenter)]
        B = [mol.x(jshell.ncenter), mol.y(jshell.ncenter), mol.z(jshell.ncenter)]
        alpha = expa + expb
        zeta = (expa * expb) / alpha
        P = (expa * A + expb * B) / alpha
        PA = P - A
        PB = P - B
        AB = A - B
        start = (π / alpha)^(3 / 2) * exp(-zeta * (AB[1]^2 + AB[2]^2 + AB[3]^2))
        # call the recursion
        x, y, z = os_recursion(PA, PB, alpha, AMa+1, AMb+1)

        # Basis function index where the shell begins
        i_idx = ishell.function_index
        j_idx = jshell.function_index

        # We use counters to keep track of which component (e.g., p_x, p_y, p_z)
        # within the shell we are on
        counta = 1

        for p in 1:AMa
            la = AMa - p                    # Let l take on all values, and p be the leftover a.m.
            for q in 1:p
                ma = p - q                  # distribute all leftover a.m. to m and n
                na = q
                countb = 1
                for r in 1:AMb
                    lb = AMb - r            # Let l take on all values, and r the leftover a.m.
                    for s in 1:r
                        mb = r - s          # distribute all leftover a.m. to m and n
                        nb = s

                        S[i_idx + counta, j_idx + countb] += start    *
                                                             coefa    *
                                                             coefb    *
                                                             x[la+1,lb+1] *
                                                             y[ma+1,mb+1] *
                                                             z[na+1,nb+1]

                        lam = la < 1 ? lastindex(x,1) : la
                        lbm = lb < 1 ? lastindex(x,1) : lb
                        nam = na < 1 ? lastindex(x,1) : na
                        nbm = nb < 1 ? lastindex(x,1) : nb
                        mam = ma < 1 ? lastindex(x,1) : ma
                        mbm = mb < 1 ? lastindex(x,1) : mb
                        Tx = (1 / 2) * (la * lb * x[lam, lbm] + 4 * expa * expb * x[la + 2, lb + 2] -
                               2 * expa * lb * x[la + 2, lbm] - 2 * expb * la * x[lam, lb + 2])   *
                               y[ma+1, mb+1] * z[na+1, nb+1]

                        Ty = (1 / 2) * (ma * mb * y[mam, mbm] + 4 * expa * expb * y[ma + 2, mb + 2] -
                               2 * expa * mb * y[ma + 2, mbm] - 2 * expb * ma * y[mam, mb + 2])   *
                               x[la+1, lb+1] * z[na+1, nb+1]

                        Tz = (1 / 2) * (na * nb * z[nam, nbm] + 4 * expa * expb * z[na + 2, nb + 2] -
                               2 * expa * nb * z[na + 2, nbm] - 2 * expb * na * z[nam, nb + 2])   *
                               x[la+1, lb+1] * y[ma+1, mb+1]

                        T[i_idx + counta, j_idx + countb] += start * coefa * coefb * (Tx + Ty + Tz)

                        dx = (x[la + 2, lb+1] + A[1] * x[la+1, lb+1]) * y[ma+1, mb+1] * z[na+1, nb+1]
                        dy = (y[ma + 2, mb+1] + A[2] * y[ma+1, mb+1]) * x[la+1, lb+1] * z[na+1, nb+1]
                        dz = (z[na + 2, nb+1] + A[3] * z[na+1, nb+1]) * x[la+1, lb+1] * y[ma+1, mb+1]

                        Dx[i_idx + counta, j_idx + countb] += start * coefa * coefb * dx
                        Dy[i_idx + counta, j_idx + countb] += start * coefa * coefb * dy
                        Dz[i_idx + counta, j_idx + countb] += start * coefa * coefb * dz

                        countb += 1
                    end
                end
                counta += 1
            end
        end
    end
end
```

Check the kinetic energy integrals:

```julia
Tpsi4 = np.asarray(mints.ao_kinetic())
@assert np.allclose(T, Tpsi4, 6)
```

# Computing the Dipole Moment

The x-component of the dipole moment is given by

\begin{align}
\mu_x =& - \mu_{\mathrm{elec}} + \mu_{\mathrm{nuclear}}\\
\mu_x =& - \sum_{\mu \nu}^{\mathrm{AO}} D_{\mu \nu} d_{\mu \nu}^{x} + \sum_A^N Z_A X_A
\end{align}

You can obtain the nuclear contribution to the dipole $\mu_{\mathrm{nuclear}}$ from Psi4 using

    psi4.core.molecule.nuclear_dipole()

The total dipole moment is given by

\begin{equation}
\mu = \sqrt{\mu_x^2 + \mu_y^2 + \mu_z^2}
\end{equation}

```julia
#Code to compute dipole moment

# Get density matrices
Da = np.asarray(scf_wfn.Da())
Db = np.asarray(scf_wfn.Db())
D = Da + Db

# Get nuclear dipole
nuc_dipole = mol.nuclear_dipole()

# Compute dipole moment components
mux = -sum(D .* Dx) + get(nuc_dipole,1)
muy = -sum(D .* Dy) + get(nuc_dipole,2)
muz = -sum(D .* Dz) + get(nuc_dipole,3)

# Compute the dipole moment
mu = sqrt(mux^2 + muy^2 + muz^2)

# Get Psi4's dipole moment components
mux_psi4 = psi4.core.variable("SCF DIPOLE X")
muy_psi4 = psi4.core.variable("SCF DIPOLE Y")
muz_psi4 = psi4.core.variable("SCF DIPOLE Z")

# Compute Psi4 dipole moment
mu_psi4 = sqrt(mux_psi4^2 + muy_psi4^2 + muz_psi4^2)

# Psi4 prints in Debye, ours is in a.u.
# 2.54174623 Debye in 1 a.u.
mu_psi4 *= 1/2.54174623

# Compare Psi4 dipole moment to ours

@assert np.allclose(mu, mu_psi4, 4)

println("Psi4 calculated molecular dipole magnitude: ", mu_psi4)
println("Here calculated molecular dipole magnitude: ", mu)
```

# References

1. Recursion scheme:
    > [[Obara:1986](http://aip.scitation.org/doi/abs/10.1063/1.450106)] S. Obara and A. Saika, *J. Chem. Phys* **84**, 3963 (1986)

2. T. Helgaker, P. Jorgensen, and J. Olsen, *Molecular Electronic Structure Theory*, John Wiley & Sons Inc, 2000.


# SMS Spam Detector

Link to slides: http://bit.ly/sms-spam-nodejs

## Read data using pandas

```python
import pandas as pd
df = pd.read_csv('data/SMSSpamCollection', delimiter='\t', header=None, names=['y', 'X'])

# Show the first 5 rows
df.head()
```

      y
      X

      0
      ham
      Go until jurong point, crazy.. Available only ...

      1
      ham
      Ok lar... Joking wif u oni...

      2
      spam
      Free entry in 2 a wkly comp to win FA Cup fina...

      3
      ham
      U dun say so early hor... U c already then say...

      4
      ham
      Nah I don't think he goes to usf, he lives aro...

```python
# Count how many of each category
df.y.value_counts()
```

    ham     4825
    spam     747
    Name: y, dtype: int64

## Split the data into training and test sets

Test set is set aside for independent testing

```python
from sklearn.model_selection import train_test_split

# shuffle and split the corpus into train and test sets
X_train, X_test, y_train, y_test = train_test_split(df.X, df.y, random_state=42)

# Count how many of each category for training and test sets
print(pd.Series(y_train).value_counts())
print(pd.Series(y_test).value_counts())
```

    ham     3618
    spam     561
    Name: y, dtype: int64
    ham     1207
    spam     186
    Name: y, dtype: int64

## Process text data to extract features

- Tokenize by spaces
- Remove stop words
- Convert each word into its count (how many times the word appears in the corpus)
- Usage of uppercase is common in spam messages, we will preserve the case by setting lowercase=False.

```python
from sklearn.feature_extraction.text import CountVectorizer

vectorizer = CountVectorizer(lowercase=False)
vectorizer.fit(X_train)

Z_train = vectorizer.transform(X_train)
Z_test = vectorizer.transform(X_test)

Z_train
```

    <4179x9329 sparse matrix of type '<class 'numpy.int64'>'
    	with 56614 stored elements in Compressed Sparse Row format>

```python
Z_test
```

    <1393x9329 sparse matrix of type '<class 'numpy.int64'>'
    	with 16839 stored elements in Compressed Sparse Row format>

## Train a Naive Bayes classifier

Naive Bayes classfiers are generally a good first start for a simple classification model.

This computes probabilities of spam vs. ham, using a Gaussian distribution to estimate the likelihood of each feature.

$$
\begin{align}\begin{aligned}P(y \mid x_1, \dots, x_n) \propto P(y) \prod_{i=1}^{n} P(x_i \mid y)\\\Downarrow\\\hat{y} = \arg\max_y P(y) \prod_{i=1}^{n} P(x_i \mid y),\end{aligned}\end{align}
$$

where

$$
P(x_i \mid y) = \frac{1}{\sqrt{2\pi\sigma^2_y}} \exp\left(-\frac{(x_i - \mu_y)^2}{2\sigma^2_y}\right)
$$

http://scikit-learn.org/stable/modules/naive_bayes.html#gaussian-naive-bayes

```python
from sklearn.naive_bayes import GaussianNB

clf = GaussianNB()
clf.fit(Z_train.toarray(), y_train)

print('Accuracy', clf.score(Z_test.toarray(), y_test))
```

    Accuracy 0.9267767408470926

```python
from sklearn.metrics import confusion_matrix

y_pred = clf.predict(Z_test.toarray())

confusion_matrix(y_test, y_pred)
```

    array([[1121,   86],
           [  16,  170]], dtype=int64)

Accuracy looks good (92.7%) for the test set, but in reality the number of misclassified "spam" is relatively high.

```
array([[1121,  86],
       [  16,  170]], dtype=int64)
```

- 1121 = number of correctly classified 'ham' SMS
- 170 = number of correctly classified 'spam' SMS
- 16 = number of 'spam' wrongly classified as 'ham'
- 86 = number of 'ham' wrongly classified as 'spam'

## Inspecting error cases

Let's take a look at some of the incorrectly labelled cases

```python
spam_classified_as_ham = X_test[(y_test == 'spam') & (y_test != y_pred)]

spam_classified_as_ham
```

    881     Reminder: You have not downloaded the content ...
    3864    Oh my god! I've found your number again! I'm s...
    2402    Babe: U want me dont u baby! Im nasty and have...
    4527    I want some cock! My hubby's away, I need a re...
    2663    Hello darling how are you today? I would love ...
    751     Do you realize that in about 40 years, we'll h...
    3463    Bloomberg -Message center +447797706009 Why wa...
    1126    For taking part in our mobile survey yesterday...
    227     Will u meet ur dream partner soon? Is ur caree...
    3755    Bloomberg -Message center +447797706009 Why wa...
    856     Talk sexy!! Make new friends or fall in love i...
    3360    Sorry I missed your call let's talk when you h...
    2770    Burger King - Wanna play footy at a top stadiu...
    731     Email AlertFrom: Jeri StewartSize: 2KBSubject:...
    1507    Thanks for the Vote. Now sing along with the s...
    68      Did you hear about the new "Divorce Barbie"? I...
    Name: X, dtype: object

```python
ham_classified_as_spam = X_test[(y_test == 'ham') & (y_test != y_pred)]

ham_classified_as_spam
```

    1859                     Sir, i am waiting for your call.
    2952                     Hey now am free you can call me.
    3142                       Customer place i will call you
    2422    Err... Cud do. I'm going to  at 8pm. I haven't...
    4937                             K..k.:)congratulation ..
    5519    Can you pls send me that company name. In saib...
    5339                  You'd like that wouldn't you? Jerk!
    315             You made my day. Do have a great day too.
    5349                                          I'm home...
    33      For fear of fainting with the of all that hous...
    1870                       Mom wants to know where you at
    4050                     Yeah that's the impression I got
    2067                           Then. You are eldest know.
    4677                              Ü ready then call me...
    3535                           Good evening! How are you?
    3689                           I'll meet you in the lobby
    4999                               Can you talk with me..
    132               Dear, will call Tmorrow.pls accomodate.
    1837                               Are you wet right now?
    893     Nutter. Cutter. Ctter. Cttergg. Cttargg. Ctarg...
    1612                                                  645
    534                                       I'll be late...
    3655                     Why i come in between you people
    1934                              R u over scratching it?
    889                              You unbelievable faglord
    2329        That day you asked about anand number. Why:-)
    3542             If you are not coughing then its nothing
    4357       Great. So should i send you my account number.
    1520                Check wid corect speling i.e. Sarcasm
    5307                              What you did in  leave.
                                  ...
    2561                     Are you still getting the goods.
    4796    Saw Guys and Dolls last night with Patrick Swa...
    69                     I plane to give on this month end.
    5470                           I thought slide is enough.
    2899          If you r @ home then come down within 5 min
    387                       Customer place i will call you.
    2603                  So when you gonna get rimac access
    805                             K I'll be there before 4.
    2107                     Thank you. I like you as well...
    4886                      Poor girl can't go one day lmao
    2963             NONE!NOWHERE IKNO DOESDISCOUNT!SHITINNIT
    2873                                      See you there!
    1553                                             U too...
    4293                                                G.W.R
    2451                              K..give back my thanks.
    596                              I am great! How are you?
    2807                                           Can a not?
    5151    No problem with the renewal. I.ll do it right ...
    1406                             K..k..any special today?
    3336            Sorry, got a late start, we're on the way
    2164    hi my darlin im on my way to London and we hav...
    4482                         No..its ful of song lyrics..
    122     here is my new address -apples&pairs&all that ...
    2298                            Draw va?i dont think so:)
    1501                   Host-based IDPS for linux systems.
    3877                               did u get that message
    1030                           Its good, we'll find a way
    2197           Not much, just some textin'. How bout you?
    2173     Yavnt tried yet and never played original either
    949                              Chk in ur belovd ms dict
    Name: X, Length: 86, dtype: object

```python

```

## Save model for deployment

We can optimize the model further (for example by using word vectors instead of counts), but for now let's save the model so that we can use it from our web server.

Scikit-learn models can be saved using pickle, which is Python's general purpose serialisation library.

http://scikit-learn.org/stable/modules/model_persistence.html

```python
import pickle

# Serialise the CountVectorizer
pickle.dump(vectorizer, open('model/vectorizer.pickle', 'wb'))

# Serialize the GaussianNB classifier
pickle.dump(clf, open('model/model.pickle', 'wb'))
```

```python
!dir model
```

     Volume in drive D is DATA
     Volume Serial Number is B200-6E0E

     Directory of D:\stackup-workshops\simple-ml\model

    04/10/2018  03:25 PM    <DIR>          .
    04/10/2018  03:25 PM    <DIR>          ..
    04/10/2018  03:25 PM           299,209 model.pickle
    04/10/2018  03:25 PM           399,600 vectorizer.pickle
                   2 File(s)        698,809 bytes
                   2 Dir(s)  782,077,120,512 bytes free

```python

```


```python
try:
    import openmdao.api as om
except ImportError:
    !python -m pip install openmdao[notebooks]
    import openmdao.api as om
```

# Cannonball Example with Euler Integration and an External Optimizer

This example will show you how to do the following:

1. Use an OpenMDAO Problem inside of a loop to perform a broader calculation (in this case, simple integration.)
2. Optimize a Problem using an external optimizer with a functional interface.
3. Perform a complex step across an OpenMDAO Problem.

In the example, we want to find the optimal angle to fire a cannon to maximize the distance it travels down
range. We already know that a 45 degree angle will maximize the range in ideal conditions, but the model
we will use also includes aerodynamic effects, so we will solve for the firing angle in the presence of a
small amount of drag force.

## Model

A very general set of dynamics for a simplified aircraft are given by these differential equations:

A very general set of dynamics for a simplified aircraft are given by these differential equations:

$$
  \begin{align}
    \frac{dv}{dt} &= \frac{T}{m} \cos \alpha - \frac{D}{m} - g \sin \gamma \\
    \frac{d\gamma}{dt} &= \frac{T}{m v} \sin \alpha + \frac{L}{m v} - \frac{g \cos \gamma}{v} \\
    \frac{dh}{dt} &= v \sin \gamma \\
    \frac{dr}{dt} &= v \cos \gamma \\
  \end{align}
$$

We will use these for the cannonball dynamics, though angle of attack, lift, and thrust will all be
zero.  We implement these equations into an OpenMDAO `ExplicitComponent` whose outputs are the
time rates of change of the states which are velocity 'v', flight path angle 'gam', altitude
'h' and range 'r'.

```python
import numpy as np

class FlightPathEOM2D(om.ExplicitComponent):
    """
    Computes the position and velocity equations of motion using a 2D flight path
    parameterization of states per equations 4.42 - 4.46 of _[1].

    References
    ----------
    .. [1] Bryson, Arthur Earl. Dynamic optimization. Vol. 1. Prentice Hall, p.172, 1999.
    """
    def initialize(self):
        self.options.declare('num_nodes', types=int)

    def setup(self):
        self.add_input(name='m', val=1.0, units='kg',
                       desc='aircraft mass')
        self.add_input(name='v', val=1.0, units='m/s',
                       desc='aircraft velocity magnitude')
        self.add_input(name='T', val=0.0, units='N',
                       desc='thrust')
        self.add_input(name='alpha', val=0.0, units='rad',
                       desc='angle of attack')
        self.add_input(name='L', val=0.0, units='N',
                       desc='lift force')
        self.add_input(name='D', val=0.0, units='N',
                       desc='drag force')
        self.add_input(name='gam', val=0.0, units='rad',
                       desc='flight path angle')

        self.add_output(name='v_dot', val=0.0, units='m/s**2',
                        desc='rate of change of velocity magnitude')
        self.add_output(name='gam_dot', val=0.0, units='rad/s',
                        desc='rate of change of flight path angle')
        self.add_output(name='h_dot', val=0.0, units='m/s',
                        desc='rate of change of altitude')
        self.add_output(name='r_dot', val=0.0, units='m/s',
                        desc='rate of change of range')

    def setup_partials(self):
        self.declare_partials('v_dot', ['T', 'D', 'm', 'gam', 'alpha'])
        self.declare_partials('gam_dot', ['T', 'L', 'm', 'gam', 'alpha', 'v'])
        self.declare_partials(['h_dot', 'r_dot'], ['gam', 'v'])

    def compute(self, inputs, outputs):
        g = 9.80665
        m = inputs['m']
        v = inputs['v']
        T = inputs['T']
        L = inputs['L']
        D = inputs['D']
        gam = inputs['gam']
        alpha = inputs['alpha']

        calpha = np.cos(alpha)
        salpha = np.sin(alpha)

        cgam = np.cos(gam)
        sgam = np.sin(gam)

        mv = m * v

        outputs['v_dot'] = (T * calpha - D) / m - g * sgam
        outputs['gam_dot'] = (T * salpha + L) / mv - (g / v) * cgam
        outputs['h_dot'] = v * sgam
        outputs['r_dot'] = v * cgam
```

We also need to compute the aerodynamic forces L and D, which are computed from the coefficients of lift and drag as well as the dynamic pressure.  This was implemented in two components:

```python
class DynamicPressureComp(om.ExplicitComponent):

    def setup(self):
        self.add_input(name='rho', val=1.0, units='kg/m**3',
                       desc='atmospheric density')
        self.add_input(name='v', val=1.0, units='m/s',
                       desc='air-relative velocity')

        self.add_output(name='q', val=1.0, units='N/m**2',
                        desc='dynamic pressure')

        self.declare_partials(of='q', wrt='rho')
        self.declare_partials(of='q', wrt='v')

    def compute(self, inputs, outputs):
        outputs['q'] = 0.5 * inputs['rho'] * inputs['v'] ** 2
```

```python
class LiftDragForceComp(om.ExplicitComponent):
    """
    Compute the aerodynamic forces on the vehicle in the wind axis frame
    (lift, drag, cross) force.
    """
    def initialize(self):
        self.options.declare('num_nodes', types=int)

    def setup(self):
        self.add_input(name='CL', val=0.0,
                       desc='lift coefficient')
        self.add_input(name='CD', val=0.0,
                       desc='drag coefficient')
        self.add_input(name='q', val=0.0, units='N/m**2',
                       desc='dynamic pressure')
        self.add_input(name='S', val=0.0, units='m**2',
                       desc='aerodynamic reference area')

        self.add_output(name='f_lift', shape=(1, ), units='N',
                        desc='aerodynamic lift force')
        self.add_output(name='f_drag', shape=(1, ), units='N',
                        desc='aerodynamic drag force')

        self.declare_partials(of='f_lift', wrt=['q', 'S', 'CL'])
        self.declare_partials(of='f_drag', wrt=['q', 'S', 'CD'])

    def compute(self, inputs, outputs):
        q = inputs['q']
        S = inputs['S']
        CL = inputs['CL']
        CD = inputs['CD']

        qS = q * S

        outputs['f_lift'] = qS * CL
        outputs['f_drag'] = qS * CD
```

Finally, we put it all together in our top model. Given any cannonball flight path angle, and velocity we can compute the rates of change of all states by running this model.

```python
class CannonballODE(om.Group):

    def setup(self):
        self.add_subsystem(name='dynamic_pressure',
                           subsys=DynamicPressureComp(),
                           promotes=['*'])

        self.add_subsystem(name='aero',
                           subsys=LiftDragForceComp(),
                           promotes_inputs=['*'])

        self.add_subsystem(name='eom',
                           subsys=FlightPathEOM2D(),
                           promotes=['*'])

        self.connect('aero.f_drag', 'D')
        self.connect('aero.f_lift', 'L')
```

## Time Integration

Given an initial angle and velocity for the cannonball, one way we can compute the range is by integrating the equations of motion that are provided by OpenMDAO. A simple way to do this is to use the [Euler integration](https://en.wikipedia.org/wiki/Euler_method). We can perform this by choosing a time step, running the Problem at the starting location to compute the state rates, and then use Euler's method to compute the new cannonball state.  We can do this sequentially until the height turns negative, which means we have hit the ground sometime between this time step and the previous one. Finally, we use linear interpolation between the final point and the previous one to find the location where the height is zero.

Note that Euler integration is a first-order method, so the accuracy will be linearly proportional to the step size. We use a 'dt' of 0.1 seconds here, which isn't particularly accurate, but runs quickly. In practice, you might use a higher order integration method here (e.g., Runge-Kutta.)

```python
import numpy as np
from scipy.optimize import minimize

from openmdao.test_suite.test_examples.cannonball.cannonball_ode import CannonballODE

def eval_cannonball_range(gam_init, prob, complex_step=False):
    """
    Compute distance given initial speed and angle of cannonball.

    Parameters
    ----------
    gam_init : float
        Initial cannonball firing angle in degrees.
    prob : <Problem>
        OpenMDAO problem that contains the equations of motion.
    complex_step : bool
        Set to True to perform complex step.

    Returns
    -------
    float
        Negative of range in m.
    """
    dt = 0.1        # Time step
    h_init = 1.0    # Height of cannon.
    v_init = 100.0  # Initial cannonball velocity.
    h_target = 0.0  #

    v = v_init
    gam = gam_init
    h = h_init
    r = 0.0
    t = 0.0

    if complex_step:
        prob.set_complex_step_mode(True)

    while h > h_target:

        # Set values
        prob.set_val('v', v)
        prob.set_val('gam', gam, units='deg')

        # Run the model
        prob.run_model()

        # Extract rates
        v_dot = prob.get_val('v_dot')
        gam_dot = prob.get_val('gam_dot', units='deg/s')
        h_dot = prob.get_val('h_dot')
        r_dot = prob.get_val('r_dot')

        h_last = h
        r_last = r

        # Euler Integration
        v = v + dt * v_dot
        gam = gam + dt * gam_dot
        h = h + dt * h_dot
        r = r + dt * r_dot
        t += dt
        # print(v, gam, h, r)

    # Linear interpolation between last two points to get the landing point accurate.
    r_final = r_last + (r - r_last) * h_last / (h_last - h)

    if complex_step:
        prob.set_complex_step_mode(False)

    #print(f"Distance: {r_final}, Time: {t}, Angle: {gam_init}")
    return -r_final
```

Here, we have placed the integration code inside of a function that takes the initial angle as an argument and returns the negative of the computed range. This is the objective we wish to optimize with an external optimizer which requires a function that it can call to evaluate the objective.

## Providing Derivatives for an External Optimizer

The optimizer also allows you to specify a function to evaluate the gradient. If you do not provide one, it will use finite difference. We can improve the accuracy by performing complex step instead. OpenMDAO allows you to run a model in complex mode. When the mode is enabled on the Problem, you can use 'set_val' to set complex values and 'get_val' to retrieve them. In the code for the objective evaluation above, we turn this feature on by calling `prob.set_complex_step_mode(True)`.  Likewise, it is important to turn it off when not needed, and it should only be used when you are performing a complex step to compute the derivatives.

The following function computes the total derivatives that the external optimizer needs.

```python
def gradient_cannonball_range(gam_init, prob):
    """
    Uses complex step to compute gradient of range wrt initial angle.

    Parameters
    ----------
    gam_init : float
        Initial cannonball firing angle in degrees.
    prob : <Problem>
        OpenMDAO problem that contains the equations of motion.

    Returns
    -------
    float
        Derivative of range wrt initial angle in m/deg.
    """
    step = 1.0e-14
    dr_dgam = eval_cannonball_range(gam_init + step * 1j, prob, complex_step=True)
    return dr_dgam.imag / step
```

## Running the Optimization

Now we can put everything together and run the optimization. Our optimizer is scipy.minimize for this example.

```python
import numpy as np
from scipy.optimize import minimize

import openmdao.api as om
from openmdao.test_suite.test_examples.cannonball.cannonball_ode import CannonballODE

def eval_cannonball_range(gam_init, prob, complex_step=False):
    """
    Compute distance given initial speed and angle of cannonball.

    Parameters
    ----------
    gam_init : float
        Initial cannonball firing angle in degrees.
    prob : <Problem>
        OpenMDAO problem that contains the equations of motion.
    complex_step : bool
        Set to True to perform complex step.

    Returns
    -------
    float
        Negative of range in m.
    """
    dt = 0.1        # Time step
    h_init = 1.0    # Height of cannon.
    v_init = 100.0  # Initial cannonball velocity.
    h_target = 0.0  #

    v = v_init
    gam = gam_init
    h = h_init
    r = 0.0
    t = 0.0

    if complex_step:
        prob.set_complex_step_mode(True)

    while h > h_target:

        # Set values
        prob.set_val('v', v)
        prob.set_val('gam', gam, units='deg')

        # Run the model
        prob.run_model()

        # Extract rates
        v_dot = prob.get_val('v_dot')
        gam_dot = prob.get_val('gam_dot', units='deg/s')
        h_dot = prob.get_val('h_dot')
        r_dot = prob.get_val('r_dot')

        h_last = h
        r_last = r

        # Euler Integration
        v = v + dt * v_dot
        gam = gam + dt * gam_dot
        h = h + dt * h_dot
        r = r + dt * r_dot
        t += dt
        # print(v, gam, h, r)

    # Linear interpolation between last two points to get the landing point accurate.
    r_final = r_last + (r - r_last) * h_last / (h_last - h)

    if complex_step:
        prob.set_complex_step_mode(False)

    #print(f"Distance: {r_final}, Time: {t}, Angle: {gam_init}")
    return -r_final

def gradient_cannonball_range(gam_init, prob):
    """
    Uses complex step to compute gradient of range wrt initial angle.

    Parameters
    ----------
    gam_init : float
        Initial cannonball firing angle in degrees.
    prob : <Problem>
        OpenMDAO problem that contains the equations of motion.

    Returns
    -------
    float
        Derivative of range wrt initial angle in m/deg.
    """
    step = 1.0e-14
    dr_dgam = eval_cannonball_range(gam_init + step * 1j, prob, complex_step=True)
    return dr_dgam.imag / step

prob = om.Problem(model=CannonballODE())
prob.setup(force_alloc_complex=True)

# Set constants
prob.set_val('CL', 0.0)                          # Lift Coefficient
prob.set_val('CD', 0.05)                         # Drag Coefficient
prob.set_val('S', 0.25 * np.pi, units='ft**2')   # Wetted Area (1 ft diameter ball)
prob.set_val('rho', 1.225)                       # Atmospheric Density
prob.set_val('m', 5.5)                           # Cannonball Mass

prob.set_val('alpha', 0.0)                       # Angle of Attack (Not Applicable)
prob.set_val('T', 0.0)                           # Thrust (Not Applicable)

result = minimize(eval_cannonball_range, 27.0,
                  method='SLSQP',
                  jac=gradient_cannonball_range,
                  args=(prob))

print(result['x'])
```

```python
from openmdao.utils.assert_utils import assert_near_equal

assert_near_equal(result['x'], 42.3810579, 1e-3)
```


### Implementation of schemes for the Heat Equation:
- Forward Time, Centered Space;
- Backward Time, Centered Space;
- Crank-Nicolson.

\begin{equation}
    \partial_{t}u = \partial^2_{x}u , \quad 0 < x < 1, \quad t > 0 \\
\end{equation}
\begin{equation}
    \partial_{x}u(0,t) = 0, \quad \partial_x{u}(1,t) = 0\\
\end{equation}
\begin{equation}
    u(x, 0) = cos(\pi x)
\end{equation}

### Exact Solution:
\begin{equation}
    u(x,t) = e^{-\pi^2t}cos(\pi x)
\end{equation}

```
#Numerical Differential Equations - Federal University of Minas Gerais
""" Utils """
import math, sys
import numpy as np
import sympy as sp
from scipy import sparse
from sympy import fourier_series, pi
from scipy.fftpack import *
from scipy.sparse import diags
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from os import path
count = 0
```

```
#Heat Diffusion in one dimensional wire within the Explicit Method
"""
λ = 2, λ = 1/2 e λ = 1/6
M = 4, M = 8, M = 10, M = 12 e M = 14
"""
#Heat function exact solution
def Solution(x, t):
    return np.exp((-np.pi**2)*t)*np.cos(np.pi*x)
```

```
# ---- Surface plot ----
def surfaceplot(U, Uexact, tspan, xspan, M):
    N = M**2
    #meshgrid : Return coordinate matrices from coordinate vectors
    X, T = np.meshgrid(tspan, xspan)

    fig = plt.figure(figsize=plt.figaspect(0.3))
    #fig2 = plt.figure(figsize=plt.figaspect(0.5))
    #fig3 = plt.figure(figsize=plt.figaspect(0.5))

    # ---- Exact Solution ----
    ax = fig.add_subplot(1, 4, 1,projection='3d')
    surf = ax.plot_surface(X, T, Uexact, linewidth=0, cmap=cm.jet, antialiased=True)
    ax.set_title('Exact Solution')
    ax.set_xlabel('Time')
    ax.set_ylabel('Space')
    ax.set_zlabel('U')

    # ---- Method Aproximation Solution ----
    ax1 = fig.add_subplot(1, 4, 2,projection='3d')
    surf = ax1.plot_surface(X, T, U, linewidth=0, cmap=cm.jet, antialiased=True)
    ax1.set_title('Approximation')
    ax1.set_xlabel('Time')
    ax1.set_ylabel('Space')
    ax1.set_zlabel('U')

    plt.tight_layout()
    ax.view_init(30,230)
    ax1.view_init(30,230)

    fig.savefig(path.join("plot_METHOD{0}.png".format(count)),dpi=600)
    plt.draw()

```

```
'''
Exact Solution for 1D reaction-diffusion equation:
        u_t = k * u_xx

with Neumann boundary conditions
at x=0: u_x(0,t) = 0
at x=L: u_x(L,t) = 0

with L = 1 and initial conditions:
u(x,0) = np.cos(np.pi*x)
'''

def ExactSolution(M, T = 0.5, L = 1):
    N = (M**2) #GRID POINTS on time interval

    xspan = np.linspace(0, L, M)
    tspan = np.linspace(0, T, N)

    Uexact = np.zeros((M, N))

    for i in range(0, M):
        for j in range(0, N):
            Uexact[i][j] = Solution(xspan[i], tspan[j])

    return (Uexact, tspan, xspan)
```

```
'''
Forward method to solve 1D reaction-diffusion equation:
        u_t = k * u_xx

with Neumann boundary conditions
at x=0: u_x(0,t) = 0 = sin(2*np.pi)
at x=L: u_x(L,t) = 0 = sin(2*np.pi)

with L = 1 and initial conditions:
u(x,0) = (1.0/2.0)+ np.cos(2.0*np.pi*x) - (1.0/2.0)*np.cos(3*np.pi*x)

u_x(x,t) = (-4.0*(np.pi**2))np.exp(-4.0*(np.pi**2)*t)*np.cos(2.0*np.pi*x) +
            (9.0/2.0)*(np.pi**2)*np.exp(-9.0*(np.pi**2)*t)*np.cos(3*np.pi*x))
'''

def ForwardEuler(M, lambd, T = 0.5, L = 1, k = 1):
    #Parameters needed to solve the equation within the explicit method
    #M = GRID POINTS on space interval
    N = (M**2) #GRID POINTS on time interval

    # ---- Length of the wire in x direction ----
    x0, xL = 0, L

    # ----- Spatial discretization step -----
    dx = (xL - x0)/(M-1)

    # ---- Final time ----
    t0,tF = 0, T

    # ----- Time step -----
    dt = (tF - t0)/(N-1)

    #lambd = dt*k/dx**2

    # ----- Creates grids -----
    xspan = np.linspace(x0, xL, M)
    tspan = np.linspace(t0, tF, N)

    # ----- Initializes matrix solution U -----
    U = np.zeros((M, N))

    # ----- Initial condition -----
    U[:,0] = np.cos(np.pi*xspan)

    # ----- Neumann boundary conditions -----
    """
    To implement these boundary conditions, we again use “false points”, x_0 and x_N+1 which are external points.
    We use a difference to approximate ∂u/∂x (xL,t) and set it equal to the desired boundary condition:
    """

    f = np.arange(1, N+1)
    f = (-3*U[0,:] + 4*U[1,:] - U[2,:])/2*dx
    U[0,:] = (4*U[1,:] - U[2,:])/3

    g = np.arange(1, N+1)
    g = (-3*U[-1,:] + 4*U[-2,:] - U[-3,:])/2*dx
    U[-1,:] = (4*U[-2,:] - U[-3,:])/3

    # -----  ftcs -----
    for k in range(0, N-1):
        for i in range(1, M-1):
            U[i, k+1] = lambd*U[i-1, k] + (1-2*lambd)*U[i,k] + lambd*U[i+1,k]

    return (U, tspan, xspan)

U, tspan, xspan = ForwardEuler(M = 14, lambd = 1.0/6.0)
Uexact, x, t = ExactSolution(M = 14)
surfaceplot(U, Uexact, tspan, xspan, M = 14)
```

```
'''
Backward method to solve 1D reaction-diffusion equation:
        u_t = k * u_xx

with Neumann boundary conditions
at x=0: u_x(0,t) = 0 = sin(2*np.pi)
at x=L: u_x(L,t) = 0 = sin(2*np.pi)

with L = 1 and initial conditions:
u(x,0) = (1.0/2.0)+ np.cos(2.0*np.pi*x) - (1.0/2.0)*np.cos(3*np.pi*x)

u_x(x,t) = (-4.0*(np.pi**2))np.exp(-4.0*(np.pi**2)*t)*np.cos(2.0*np.pi*x) +
            (9.0/2.0)*(np.pi**2)*np.exp(-9.0*(np.pi**2)*t)*np.cos(3*np.pi*x))
'''
def BackwardEuler(M, lambd, T = 0.5, L = 1, k = 1):
    #Parameters needed to solve the equation within the explicit method
    # M = GRID POINTS on space interval
    N = (M**2) #GRID POINTS on time interval

    # ---- Length of the wire in x direction ----
    x0, xL = 0, L

    # ----- Spatial discretization step -----
    dx = (xL - x0)/(M-1)

    # ---- Final time ----
    t0, tF = 0, T

    # ----- Time step -----
    dt = (tF - t0)/(N-1)

    # k = 1.0 Diffusion coefficient
    #lambd = dt*k/dx**2
    a = 1 + 2*lambd

    xspan = np.linspace(x0, xL, M)
    tspan = np.linspace(t0, tF, N)

    main_diag = (1 + 2*lambd)*np.ones((1,M))
    off_diag = -lambd*np.ones((1, M-1))
    a = main_diag.shape[1]
    diagonals = [main_diag, off_diag, off_diag]

    #Sparse Matrix diagonals
    A = sparse.diags(diagonals, [0,-1,1], shape=(a,a)).toarray()
    A[0,1] = -2*lambd
    A[M-1,M-2] = -2*lambd

    # --- Initializes matrix U -----
    U = np.zeros((M, N))

    # --- Initial condition -----
    U[:,0] = np.cos(np.pi*xspan)

    # ---- Neumann boundary conditions -----
    f = np.arange(1, N+1) #LeftBC
    #(-3*U[i,j] + 4*U[i-1,j] - U[i-2,j])/2*dx = 0
    f = U[0,:] = (4*U[1,:] - U[2,:])/3

    g = np.arange(1, N+1) #RightBC
    #(-3*U[N,j] + 4*U[N-1,j] - U[N-2,j])/2*dx = 0
    g = U[-1,:] = (4*U[-2,:] - U[-3,:])/3

    for i in range(1, N):
        c = np.zeros((M-2,1)).ravel()
        b1 = np.asarray([2*lambd*dx*f[i], 2*lambd*dx*g[i]])
        b1 = np.insert(b1, 1, c)
        b2 = np.array(U[0:M, i-1])
        b = b1 + b2  # Right hand side
        U[0:M, i] = np.linalg.solve(A,b)  # Solve x=A\b

    return (U, tspan, xspan)

U, tspan, xspan = BackwardEuler(M = 14, lambd = 1.0/6.0)
Uexact, x, t = ExactSolution(M = 14)
surfaceplot(U, Uexact, tspan, xspan, M = 14)
```

```
'''
Crank-Nicolson method to solve 1D reaction-diffusion equation:
        u_t = D * u_xx

with Neumann boundary conditions
at x=0: u_x = sin(2*pi)
at x=L: u_x = sin(2*pi)

with L=1 and initial condition:
u(x,0) = u(x,0) = (1.0/2.0)+ np.cos(2.0*np.pi*x) - (1.0/2.0)*np.cos(3*np.pi*x)
'''
def CrankNicolson(M, lambd, T = 0.5, L = 1, k = 1):
    #Parameters needed to solve the equation within the explicit method
    # M = GRID POINTS on space interval
    N = (M**2) #GRID POINTS on time interval

    # ---- Length of the wire in x direction ----
    x0, xL = 0, L

    # ----- Spatial discretization step -----
    dx = (xL - x0)/(M-1)

    # ---- Final time ----
    t0, tF = 0, T

    # ----- Time step -----
    dt = (tF - t0)/(N-1)

    #lambd = dt*k/(2.0*dx**2)
    a0 = 1 + 2*lambd
    c0 = 1 - 2*lambd

    xspan = np.linspace(x0, xL, M)
    tspan = np.linspace(t0, tF, N)

    maindiag_a0 = a0*np.ones((1,M))
    offdiag_a0 = (-lambd)*np.ones((1, M-1))

    maindiag_c0 = c0*np.ones((1,M))
    offdiag_c0 = lambd*np.ones((1, M-1))

    #Left-hand side tri-diagonal matrix
    a = maindiag_a0.shape[1]
    diagonalsA = [maindiag_a0, offdiag_a0, offdiag_a0]
    A = sparse.diags(diagonalsA, [0,-1,1], shape=(a,a)).toarray()

    A[0,1] = (-2)*lambd
    A[M-1,M-2] = (-2)*lambd

    #Right-hand side tri-diagonal matrix
    c = maindiag_c0.shape[1]
    diagonalsC = [maindiag_c0, offdiag_c0, offdiag_c0]

    Arhs = sparse.diags(diagonalsC, [0,-1,1], shape=(c,c)).toarray()
    Arhs[0,1] = 2*lambd
    Arhs[M-1,M-2] = 2*lambd

    # ----- Initializes matrix U -----
    U = np.zeros((M, N))

    #----- Initial condition -----
    U[:,0] = np.cos(np.pi*xspan)

    #----- Neumann boundary conditions -----
    #Add one line above and one line below using finit differences

    f = np.arange(1, N+1) #LeftBC
    #(-3*U[i,j] + 4*U[i-1,j] - U[i-2,j])/2*dx = 0
    f = U[0,:] = (4*U[1,:] - U[2,:])/3

    g = np.arange(1, N+1) #RightBC
    #(-3*U[N,j] + 4*U[N-1,j] - U[N-2,j])/2*dx = 0
    g = U[-1,:] = (4*U[-2,:] - U[-3,:])/3

    for k in range(1, N):
        ins = np.zeros((M-2,1)).ravel()
        b1 = np.asarray([4*lambd*dx*f[k], 4*lambd*dx*g[k]])
        b1 = np.insert(b1, 1, ins)
        b2 = np.matmul(Arhs, np.array(U[0:M, k-1]))
        b = b1 + b2  # Right hand side
        U[0:M, k] = np.linalg.solve(A,b)  # Solve x=A\b

    return (U, tspan, xspan)

U, tspan, xspan = CrankNicolson(M = 14, lambd = 1.0/6.0)
Uexact, x, t = ExactSolution(M = 14)
surfaceplot(U, Uexact, tspan, xspan, M = 14)
```


# Homework set 5, due Oct 6, 2017

## 1. LU Decomposition (5pts)
Find the LU decompositions of the following three matrices (no pivoting necessary) and use the results to determine
the determinants

$
\begin{pmatrix}
 2 & 1 \\
 -1 & 2
\end{pmatrix}
$
   ,
$
 \begin{pmatrix}
 2 & -1 & 0 \\
 -1 & 2 & -1 \\
 0 & -1 & 2 \\
 \end{pmatrix}
$
   ,
$
 \begin{pmatrix}
 3 & 0 & 0 \\
 -1 & 2 & 0 \\
 2 & 1 & 2 \\
 \end{pmatrix}
$

```python
from PIL import Image

image = Image.open('LU_decomposition1.png')
image.show()
image = Image.open('LU_decomposition2.png')
image.show()
image = Image.open('LU_decomposition3.png')
image.show()
```

## 2. QR Algorithm (15pts)

a) Follow the text on page 246-247 which lays out the algorithm to construct a QR decomposition for a $N\times N$ matrix and write a python function for it. Check your result using the following matrix

$$
\begin{pmatrix}
 1 & 4 & 8 & 4 \\
 4 & 2 & 3 & 7 \\
 8 & 3 & 6 & 9 \\
  4 & 7 & 9 & 2
 \end{pmatrix}
$$

b) Using your function to write a python program that determines the eigenvalues and eigenvectors of a real-valued symmetric matrix. Test it with the matrix above and test your answers with the appropriate numpy function.

```python
# -*- coding: utf-8 -*-
"""
Created on Wed Oct  7 02:07:39 2020
@author: Suzanne Conejos
"""
import numpy as np
from numpy import array
from numpy.linalg import eigh

#Using numpy.linalg.norm to get norm |u|
def q(u):
    norm = np.linalg.norm(u)
    q = []
    for i in range (0, len(u)):
        q.append((1/norm)*u[i])
    return np.array(q)

def printM(m):
    for i in range(0, len(m)):
        print (m[i])

def decompose(A):

    w = h = len(A)
    aVals = []
    uVals = []
    qVals = []

    Q = np.zeros((h,w),float)
    R = np.zeros((h,w),float)

    #Get values of A0 to An
    for i in range(0,w):
        temp = []
        for j in range(0,h):
            temp.append(A[j][i]);
        aVals.append(temp)

    #Get values of q using Gram-Schmidt process
    for i in range(0,w):
        #for i == 0, u0 = A0
        if (i == 0):
            u = np.array(aVals[i])
            uVals.append(u)
            qVals.append(q(u))
        else:
            #get Ui
            u = aVals[i]
            for j in range(1,i+1):
                u = np.subtract(u, np.dot(np.dot(aVals[i], qVals[j - 1]), qVals[j-1]))
            uVals.append(u)
            qVals.append(q(u))

    #Create Q using qVals
    for i in range(0,w):
        for j in range(0,h):
            Q[j][i] = qVals[i][j]
    #print("============================")
    #print("Matrix Q")
    #printM (Q)
    #print("============================")

    #Create R
    for i in range(0,h):
        for j in range(i,w):
            R[i][j] = np.dot(aVals[j], qVals[i])
    #print("============================")
    #print("Matrix R")
    #printM (R)
    #print("============================")

    #print("============================")
    #print("Matrix QR")
    #printM(np.dot(Q,R))
    #print("============================")

    #Calculating eigenvalues and eigenvectors
    B = array(A, float)
    x,V=eigh(B)
    #print("Eigenvalues")
    #print(x)
    #print("Eigenvectors")
    #print(V)
    #print("============================")

    return ("Matrix",A,"Q",Q,"R",R,"Matrix QR",np.dot(Q,R),"Eigenvalues",x,"Eigenvectors",V)

"""
Testing Function
"""

A = [[1, 4, 8, 4],
     [4, 2, 3, 7],
     [8, 3, 6, 9],
     [4, 7, 9, 2]]

decompose(A)

```

    ('Matrix',
     [[1, 4, 8, 4], [4, 2, 3, 7], [8, 3, 6, 9], [4, 7, 9, 2]],
     'Q',
     array([[ 0.10153462,  0.558463  ,  0.80981107,  0.1483773 ],
            [ 0.40613847, -0.10686638, -0.14147555,  0.8964462 ],
            [ 0.81227693, -0.38092692,  0.22995024, -0.37712564],
            [ 0.40613847,  0.72910447, -0.5208777 , -0.17928924]]),
     'R',
     array([[ 9.8488578 ,  6.49821546, 10.55960012, 11.37187705],
            [ 0.        ,  5.98106979,  8.4234836 , -0.484346  ],
            [ 0.        ,  0.        ,  2.74586406,  3.27671222],
            [ 0.        ,  0.        ,  0.        ,  3.11592335]]),
     'Matrix QR',
     array([[1., 4., 8., 4.],
            [4., 2., 3., 7.],
            [8., 3., 6., 9.],
            [4., 7., 9., 2.]]),
     'Eigenvalues',
     array([-8., -3.,  1., 21.]),
     'Eigenvectors',
     array([[-0.38357064, -0.77459667,  0.25819889, -0.43151697],
            [ 0.43151697, -0.25819889, -0.77459667, -0.38357064],
            [ 0.52740963,  0.25819889,  0.51639778, -0.62330229],
            [-0.62330229,  0.51639778, -0.25819889, -0.52740963]]))

# 3. Chain of Resistors (10pts)

Consider a long chain of resistors wired up like this

All resistors have the same resistane $R$. The battery delivers a voltage of $5V$. Find the voltages at the other internal points of the circuit.

a) Using Ohm's law and Kirchhoff's current law, which says that the total net current out of any junction in a circuit must be zero (otherwise charges would accumulate there) to show that the voltages $V_1,V_2,\ldots,V_N$ satisfy the linear set of equations

$$
\begin{align}
 3V_1 - V_2 - V_3 &= V_+  \\
 -V_1 + 4V_2 - V_3 - V_4 &= V_+ \\
 & \vdots \\
-V_{i-2} - V_{i-1} + 4V_i - V_{i+1} - V_{i+2}  &= 0 \\
 & \vdots \\
-V_{N-3} - V_{N-2} + 4V_{N-1} - V_N &= 0 \\
-V_{N-2} - V_{N-1} +3V_N &= 0
\end{align}
$$

Express these equations in vector form ${\bf A}{\bf V} = {\bf b}$ and find the coefficients of the matrix ${\bf A}$ and vector ${\bf b}$.

b) Write a Python program to solve for the voltages $V_i$ with a number of 10 nodes.

c) For larger number of nodes use the library-based function solve_banded to sove the equations above. Remember that you have to rewrite the coefficient matrix for that routine as described in your class notes. The Scipy document also provides an example which you can use as starting point. Solve for the voltages for 10000 nodes.

```python
from PIL import Image

image = Image.open('resistor1.jpg')
image.show()
image = Image.open('resistor2.jpg')
image.show()
```

```python
# -*- coding: utf-8 -*-
"""

"""
from numpy.linalg import solve
from numpy import  array

# Test with 10 elements

NumNodes = 10  #no. of nodes
Volt = 5   #given voltage

A = array([
	[3, -1, -1, 0, 0, 0, 0, 0, 0, 0],
	[-1, 4, -1, -1, 0, 0, 0, 0, 0, 0],
	[-1,-1, 4, -1, -1, 0, 0, 0, 0, 0],
	[0, -1,-1, 4, -1, -1, 0, 0, 0, 0],
	[0, 0, -1,-1, 4, -1, -1, 0, 0, 0],
	[0, 0, 0, -1,-1, 4, -1, -1, 0, 0],
	[0, 0, 0, 0, -1,-1, 4, -1, -1, 0],
	[0, 0, 0, 0, 0, -1,-1, 4, -1, -1],
	[0, 0, 0, 0, 0, 0, -1,-1, 4, -1],
	[0, 0, 0, 0, 0, 0, 0, -1, -1, 3]
],float)

v = array([Volt, Volt, 0, 0, 0, 0, 0, 0, 0, 0],float)

x = solve(A,v)
print(x)
print('=========================')

```

    [4.12573674 3.9194499  3.45776031 3.09430255 2.69155206 2.30844794
     1.90569745 1.54223969 1.0805501  0.87426326]
    =========================

```python
# -*- coding: utf-8 -*-
import numpy as np
from scipy.linalg import solve_banded

# Banded example

NumNodes = 10000
Volt = 5
Ab = zeros((5,NumNodes),float)

vb = zeros(NumNodes,float)

vb[[0,1]] = Volt
#print(vb)
Ab[:,:] = -1
#print(Ab)
Ab[2,:] = 4
#print(Ab)
Ab[2,0] = 3
#print(Ab)
Ab[2,NumNodes-1] = 3
#print(Ab)

x = solve_banded((2,2),Ab,vb)
print(x)
```

    [4.99888228e+00 4.99861842e+00 4.99802841e+00 ... 1.97158611e-03
     1.38158071e-03 1.11772227e-03]

```python

```


Határozzuk meg az alábbi ábrán látható tartó súlypontvonalának eltolódását leíró $v\left(x\right)$ függvényt végeselemes módszer használatával, síkbeli egyenes gerendalemek alkalmazásával.
Vizsgáljuk meg a végeselemes megoldással kapott hajlítónyomatéki igénybevétel hibáját az egyes szakaszokon.
Határozzuk meg az $x = a/2$ keresztmetszetben a hajlítónyomatéki igénybevétel nagyságát 2, illetve 3 síkbeli egyenes gerendaelem alkalmazásával.
A tartók két különböző átmérőjű ($d_1 = 2d$, illetve $d_2 = d$) kör keresztmetszetű tartókból
vannak összeépítve.
A tartók anyaga lineárisan rugalmas, homogén, izotrop. A $d_1$ átmérőjű rész rugalmassági modulusza $E$, míg a $d_2$ átmérővel rendelkező részé $4E$.

```python
import sympy as sp
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt

sp.init_printing()
```

```python
p, xi, L, I, E = sp.symbols("p, xi L I E")
```

```python
Nv = 1/8*sp.Matrix([2*(1-xi)**2*(2+xi),
                 L*(1-xi)**2*(1+xi),
                 2*(1+xi)**2*(2-xi),
                 L*(1+xi)**2*(xi-1)])
Nv
```

```python
Nfi=sp.diff(Nv,xi)*(2/L)
sp.simplify((1/L)**(-1)*Nfi)
```

```python
Bv=sp.diff(Nv,xi,2)*(2/L)**2
sp.simplify((1/L**2)**(-1)*Bv)
```

```python
KeBEAM1D = I*E*sp.integrate(Bv*(Bv.T),(xi,-1,1))*L/2
(I*E/L**3)**(-1)*KeBEAM1D
```

```python
Fp = p*sp.integrate(Nv,(xi,-1,1))*L/2
Fp
```

```python
def Bvfgv(xi,L):
    return np.multiply(1/L**2,np.array([6*xi, L*(3*xi-1),-6*xi,-L*(3*xi+1)]))
```

```python
def Nvfgv(xi,L):
    return np.array([1/4*(1-xi)**2*(2+xi),
                 L/8*(1-xi)**2*(1+xi),
                 1/4*(1+xi)**2*(2-xi),
                 L/8*(1+xi)**2*(xi-1)])
```

```python
def dddNvfgv(xi,L):
    return np.multiply(1/L**3,np.array([12, 6*L, -12, 6*L]))
```

```python
def Nfifgv(xi,L):
    return np.multiply(1/L,np.array([1.5*xi**2-1.5,
                     L*(0.75*xi+0.25)*(xi-1),
                     -1.5*xi**2+1.5,
                     L*(0.75*xi-0.25)*(xi+1)]))
```

```python
def KeBEAM1D(I,E,L):
    return np.multiply(I*E/L**3, np.array([[12,   6*L,    -12,  6*L],
                                           [6*L,  4*L**2, -6*L, 2*L**2],
                                           [-12,  -6*L,    12,  -6*L],
                                           [6*L, 2*L**2, -6*L, 4*L**2]]))
```

```python
def Fp(p,L):
    return np.multiply(L*p,np.array([1/2,L/12,1/2,-L/12]))
```

```python
a=800e-3
b=400e-3
d=20e-3
E=50e9
nu=0.3
F1=2500
M1=-500
p0=-5000
```

```python
I1 = (2*d)**4*np.pi/64
I2 = (d)**4*np.pi/64
E1 = E
E2 = 4*E
Ke1 = KeBEAM1D(I1,E1,a)
Ke2 = KeBEAM1D(I2,E2,b)
```

```python
elemSZF = np.array([[1,2,3,4],[3,4,5,6]]) - 1
```

```python
KG=np.zeros((6,6))
```

```python
eind=0
KG[np.ix_(elemSZF[eind],elemSZF[eind])]+=Ke1
```

```python
eind=1
KG[np.ix_(elemSZF[eind],elemSZF[eind])]+=Ke2
```

```python
FG=np.zeros(6)
FG[elemSZF[0]] += Fp(p0,a)
FG[elemSZF[0,2]] += F1
FG[elemSZF[1,3]] += M1
FG
```

    array([-2000.        ,  -266.66666667,   500.        ,   266.66666667,
               0.        ,  -500.        ])

```python
fixSZF=np.array([1,2,5])-1
```

```python
szabadSZF=[i for i in range(0,6) if i not in fixSZF]
szabadSZF
```

```python
KGkond = KG[np.ix_(szabadSZF,szabadSZF)]
```

```python
FGkond = FG[np.ix_(szabadSZF)]
```

```python
Umego = np.linalg.solve(KGkond,FGkond)
Umego
```

    array([ 0.01358122,  0.02122066, -0.0933709 ])

```python
UG = np.zeros(6)
UG[np.ix_(szabadSZF)] += Umego
UG
```

    array([ 0.        ,  0.        ,  0.01358122,  0.02122066,  0.        ,
           -0.0933709 ])

```python
FR = np.dot(KG,UG)-FG
FR
```

    array([  1.25000000e+03,  -2.00000000e+02,   0.00000000e+00,
             5.68434189e-14,   2.50000000e+02,  -1.13686838e-13])

# Eredmények ábrázolása

```python
xiLista = np.linspace(-1,1,num = 20)
cspxKRD=[[0,a],[a,a+b]]
cspxKRD
```

```python
x1Lista=[(cspxKRD[0][1]-cspxKRD[0][0])/2*(xi+1) + cspxKRD[0][0] for xi in xiLista]
x2Lista=[(cspxKRD[1][1]-cspxKRD[1][0])/2*(xi+1) + cspxKRD[1][0] for xi in xiLista]
xLista = np.concatenate((x1Lista,x2Lista))
```

```python
v1Lista = [np.dot(Nvfgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
v2Lista = [np.dot(Nvfgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
vLista = np.concatenate((v1Lista,v2Lista))
```

```python
figv = plt.figure(num = 1, figsize=(16/2.54,10/2.54))
axv = figv.add_subplot(111)

axv.plot(xLista,vLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$v \, \left[\mathrm{m}\right]$")

plt.grid()
plt.legend()
plt.show()
```

```python
fi1Lista = [np.dot(Nfifgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
fi2Lista = [np.dot(Nfifgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
fiLista = np.concatenate((fi1Lista,fi2Lista))
```

```python
figfi = plt.figure(num = 2, figsize=(16/2.54,10/2.54))
axfi = figfi.add_subplot(111)

axfi.plot(xLista,fiLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$\varphi \, \left[\mathrm{rad}\right]$")

plt.grid()
plt.legend()
plt.show()
```

```python
Mh1Lista = [-I1*E1*np.dot(Bvfgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
Mh2Lista = [-I2*E2*np.dot(Bvfgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
MhLista = np.concatenate((Mh1Lista,Mh2Lista))
```

```python
figMh = plt.figure(num = 3, figsize=(16/2.54,10/2.54))
axMh = figMh.add_subplot(111)

axMh.plot(xLista,MhLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$M_h \, \left[\mathrm{Nm}\right]$")

plt.grid()
plt.legend()
plt.show()
```

```python
V1Lista = [I1*E1*np.dot(dddNvfgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
V2Lista = [I2*E2*np.dot(dddNvfgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
VLista = np.concatenate((V1Lista,V2Lista))
```

```python
figV = plt.figure(num = 4, figsize=(16/2.54,10/2.54))
axV = figV.add_subplot(111)

axV.plot(xLista,VLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$V \, \left[\mathrm{N}\right]$")

plt.grid()
plt.legend()
plt.show()
```


Make sure you remove `raise NotImplementedError()` and fill in any place that says `# YOUR CODE HERE`, as well as your `NAME`, `ID`, and `LAB_SECTION` below:

```python
NAME = "FAHMID BIN KIBRIA"
ID = "19201063"
LAB_SECTION = "02"
```

---

# CSE330 Lab: Polynomial Interpolation using Lagrange Form
---

### Importing necessary libraries

```python
import numpy as np
import matplotlib.pyplot as plt
```

### The Lagrange_Polynomial class
General form of an $n$ degree Lagrange polynomial:

\begin{equation}
p_n(x) = \sum_{k=0}^{n} f(x_k)l_k(x) = \sum_{k=0}^{n} y_kl_k(x),\tag{1}
\end{equation}

where
\begin{equation}
l_k(x) = \prod_{j=0, j\neq k}^{n} \frac{x-x_j}{x_k-x_j}. \tag{2}
\end{equation}

Note that the Lagrange method is more efficient than the matrix method because *we do not need to calculate any inverse matrices*.

1. **The constructor `__init__(self, data_x, data_y)` is written for you.**

     * Here, we check whether the input vectors (numpy arrays) are equal or not.
     * We store `data_x` and `data_y`
     * We calculate and store the degree of the polynomial.
$$\$$

2. **The `_repr__(self)` function has been written for you.**

    * This is similar to the `toString()` method in Java. This returns a formatted string of the object whenever the object is printed.
$$\$$

3. **You have to implement the `l(self, k, x)` function.**
    * This function would take `k` and `x` as inputs and calculate the Lagrange basis using the Equation $(2)$.
$$\$$

4. **You have to implement the `__call__(self, x_arr)` function.**
    * This function makes an object of a class callable.
    * The function calculates the lagrange polynomial from a set of given nodes. `self.data_x` and `self.data_y` contains the coordinates of the given nodes of the original function. Using Equation $(1)$, you have to use `self.data_x`, `self.data_y`, and the `l(k, x_k, x)` function to find the interpolated output of the polynomial for all elements of `x_arr`.
`x_arr` is a numpy array containing points through which we want to plot our polynomial.

```python
class Lagrange_Polynomial:
    def __init__(self, data_x, data_y):
        '''
        First we need to check whether the input vectors (numpy arrays) are equal
        or not.
        assert (condition), "msg"
        this command checks if the condition is true or false. If true, the code
        runs normally. But if false, then the code returns an error message "msg"
        and stops execution
        '''
        assert len(data_x) == len(data_y), "length of data_x and data_y must be equal"

        '''
        Lagrange polynomials do not use coefficeints a_i, rather the nodes
        (x_i, y_i). Hence, we just need to store these inside the object
        '''

        self.data_x = data_x
        self.data_y = data_y

        self.degree = len(data_x) - 1
        # we assume that the inputs are numpy array, so we can perform
        # element wise operations

    def __repr__(self):
        # method for string representation
        # you don't need to worry about the following code if you don't understand
        strL = f"LagrangePolynomial of order {self.degree}\n"
        strL += "p(x) = "
        for i in range(len(self.data_y)):
            if self.data_y[i] == 0:
                continue
            elif self.data_y[i] >= 0:
                strL += f"+ {self.data_y[i]}*l_{i}(x) "
            else:
                strL += f"- {-self.data_y[i]}*l_{i}(x) "

        return strL

    def l(self, k, x):
        '''
        This metod implements the Lagrange Basis to be used for interpolation
        using Lagrange Polynomials.
        '''

        l_k = 1.0 # Initialization

        # --------------------------------------------
        # YOUR CODE HERE
        x_k = self.data_x[k]
        for j in range(self.degree + 1):
          x_j = self.data_x[j]
          if x_j != x_k:
            numerator = x - x_j
            denominator = x_k - x_j
            l_basis = numerator / denominator
            l_k *= l_basis
        #
        # HINT FOR LOOP METHOD: Should look like
        # x_k = self.data_x[k]
        # for j in range(self.degree + 1):
        #   l_k *= ?????
        #
        # HINT FOR VECTORIZED METHOD (no loops):
        #   Google how to use np.prod and np.delete
        # l_k = np.prod(?? np.delete(??) ??) /  np.prod(?? np.delete(??) ??)
        # --------------------------------------------
        return l_k

    def __call__(self, x_arr):
        """
        The method to make the object callable (see the code of the matrix method).
        'x_arr' is a set of given points (a numpy array). You have to use
        self.data_x and self.data_y to find the interpolated output of the
        polynomial for all elements of 'x_arr'.

        Implement as you wish but your 'total' numpy array where the i'th element
        p_x_arr[i] represents the interpolated value of p(x_arr[i]).
        """

        # initialize with zero
        p_x_arr  = np.zeros(len(x_arr))

        # --------------------------------------------
        # YOUR CODE HERE
        for i,x in enumerate(x_arr):
          for k in range(self.degree+1):
            p_x_arr[i] += self.data_y[k] * self.l(k,x)
        #
        # HINT: Should look like
        # for i, x in enumerate(x_arr):
        #    for k in range(self.degree + 1):
        #       ??????
        #       p_x_arr[i] = ??? self.data_y[k] ??? self.l(k, x)
        # --------------------------------------------

        return p_x_arr
```

### Calling the LagrangePolynomial object and plotting the polynomial.
First we create a lagrange polynomial object `p` by calling `Lagrange_Polynomial(data_x, data_y)`. Then, we call the object as a function, which is possible because we had implemented the `__call__` function in the Lagrange_Polynomial class, and pass in `x_arr`. `x_arr` is 1D numpy array (a vector), which we created using linspace function and contains $x_i$, i.e., the points through which we want to plot our polynomial. Calling the object as a function and inputting `x_arr` returns the corresponding $y_i$ values and stores them in the `p_x_arr` numpy array.

Finally, the polynomial is plotted by passing in `x_arr` and `p_x_arr` in plt.plot(), i.e., the $x_i$ and $y_i$ pairs.

*Note that in the plot the given nodes will be marked in red.*

```python
data_x = np.array([-3., -2., -1., 0., 1., 3., 4.])
data_y = np.array([-60., -80., 6., 1., 45., 30., 16.])

p = Lagrange_Polynomial(data_x, data_y)
print(p)

#generating 100 points from -3 to 4 in order to create a smooth line
x_arr = np.linspace(-3, 4, 50)
p_x_arr = p(x_arr)

# plot to see if your implementation is correct
#google the functions to understand what each parameters mean, if not apparent
plt.plot(x_arr, p_x_arr)
plt.plot(data_x, data_y, 'ro')
plt.legend(['interpolated', 'node points'], loc = 'lower right')
plt.xlabel('x')
plt.ylabel('y')
plt.title('Lagrange Polynomial')

plt.show()

```

```python

```


# 5.1 Bond behavior governed by damage

 * Define a bond-slip law governed by damage and loading history using unloading.
 * What is different in comparison to elastic-plastic models?
 * For what materials is unloading to zero realistic?
 * Example of a CFRP behavior

```python
%matplotlib widget
import matplotlib.pyplot as plt
import ibvpy.api as ib
```

## Construct the bond slip model

```python
bs1 = ib.MATSBondSlipMultiLinear(s_data="0,0.1,0.2,0.8", tau_data="0,8,4,1")
bs1.interact()
```

    HBox(children=(VBox(children=(Tree(layout=Layout(align_items='stretch', border='solid 1px black', display='fle…

```python
bs1 = ib.MATSBondSlipMultiLinear()
bs1.sim.tline.step=0.0005
bs1
```

        \begin{array}{lrrl}\hline
        \textrm{E_m} & E_\mathrm{m} = 28000.0 & \textrm{[MPa]} & \textrm{E-modulus of the matrix}  \\
                \textrm{E_f} & E_\mathrm{f} = 170000.0 & \textrm{[MPa]} & \textrm{E-modulus of the reinforcement}  \\
                \textrm{s_data} & s = 0,0.1,0.2,0.8 & \textrm{[mm]} & \textrm{slip values}  \\
                \textrm{tau_data} & \tau = 0,8,4,1 & \textrm{[MPa]} & \textrm{shear stress values}  \\
                \hline
        \hline
        \end{array}

```python
bs1.mats_eval.omega_fn_type = 'jirasek'
bs1.mats_eval
```

        \begin{array}{lrrl}\hline
        \textrm{E_m} & \textrm{30000.0} & & \textrm{None}  \\
                \textrm{E_f} & \textrm{200000.0} & & \textrm{None}  \\
                \textrm{E_b} & \textrm{223.18383360000004} & & \textrm{None}  \\
                \textrm{omega_fn_type} & \textrm{jirasek} & & \textrm{None}  \\
                \hline
        \textbf{omega_fn} & \textrm{JirasekDamageFn} & & \textrm{None} \\
            \hline
        \end{array}

```python
bs1.mats_eval.omega_fn.trait_set(s_f = 0.001, plot_max=0.05)
bs1.mats_eval.omega_fn
```

Damage function (Jirasek)
        \begin{align}
        \omega &= g(\kappa)
        = 1 - \left[\frac{s_0}{\kappa} \exp \left(- \frac{\kappa
        - s_0}{s_f - s_0}\right)\right]
        \end{align}
        where $\kappa$ is the state variable representing
        the maximum slip that occurred so far in
        in the history of loading.

        \begin{array}{lrrl}\hline
        \textrm{s_f} & s_\mathrm{f} = 0.001 & \textrm{[mm/mm]} & \textrm{derivative of the damage function at the onset of damage}  \\
                \textrm{s_0} & s_0 = 0.0004 & \textrm{[mm]} & \textrm{elastic strain limit}  \\
                \hline
        \hline
        \end{array}

```python
bs1.mats_eval.omega_fn.plot(plt.axes())
```

```python
bs1.w_max = 0.0015
bs1.loading_scenario.trait_set(loading_type='cyclic',
                               amplitude_type='increasing',
                               loading_range='symmetric',
                               )
bs1.loading_scenario.trait_set(number_of_cycles=10,
                               maximum_loading=1,
                               unloading_ratio =0.0)

```

        \begin{array}{lrrl}\hline
        \textrm{loading_type} & option = cyclic & \textrm{[-]} & \textrm{possible values: [monotonic, cyclic]}  \\
                \textrm{number_of_cycles} & n_\mathrm{cycles} = 10 & \textrm{[-]} & \textrm{for cyclic loading}  \\
                \textrm{maximum_loading} & \phi_{\max} = 1.0 & \textrm{[-]} & \textrm{load factor at maximum load level}  \\
                \textrm{unloading_ratio} & \phi_{\mathrm{unload}} = 0.0 & \textrm{[-]} & \textrm{fraction of maximum load at lowest load level}  \\
                \textrm{number_of_increments} & n_{\mathrm{incr}} = 20 & \textrm{[-]} & \textrm{number of values within a monotonic load branch}  \\
                \textrm{amplitude_type} & option = increasing & \textrm{[-]} & \textrm{possible values: [increasing, constant]}  \\
                \textrm{loading_range} & option = symmetric & \textrm{[-]} & \textrm{possible values: [non-symmetric, symmetric]}  \\
                \hline
        \hline
        \end{array}

```python
bs1.loading_scenario.plot(plt.axes())
plt.grid(True)
```

```python
bs1.sim.run()
```

```python
ax = plt.axes()
ax.grid(True)
bs1.hist.plot_Pw(ax,1)
```

### Evolution of the damage

```python
bs2 = BondSlipModel(mats_eval_type='damage')
bs2.sim.tline.step = 0.0005
bs2
```

        \begin{array}{lrrl}\hline
        \textrm{k_max} & k_{\max} = 400 & \textrm{[-]} & \textrm{maximum number of iterations}  \\
                \textrm{tolerance} & \epsilon = 0.0001 & \textrm{[-]} & \textrm{required accuracy}  \\
                \textrm{n_e_x} & n_\mathrm{E} = 1 & \textrm{[-]} & \textrm{number of finite elements along the embedded length}  \\
                \textrm{mats_eval_type} & \textrm{damage} & & \textrm{material model type}  \\
                \textrm{control_variable} & \textrm{u} & & \textrm{displacement or force control: [u|f]}  \\
                \textrm{w_max} & w_{\max} = 1.0 & \textrm{[mm]} & \textrm{maximum pullout slip}  \\
                \textrm{fixed_boundary} & \textrm{non-loaded end (matrix)} & & \textrm{which side of the specimen is fixed [non-loaded end [matrix], loaded end [matrix], non-loaded end [reinf]]}  \\
                \hline
        \textbf{loading_scenario} & \textrm{LoadingScenario} & & \textrm{object defining the loading scenario} \\
            \textbf{cross_section} & \textrm{CrossSection} & & \textrm{cross section parameters} \\
            \textbf{geometry} & \textrm{Geometry} & & \textrm{geometry parameters of the boundary value problem} \\
            \textbf{mats_eval} & \textrm{MATSBondSlipD} & & \textrm{material model of the interface} \\
            \hline
        \end{array}

```python
bs2.mats_eval.omega_fn_type = 'abaqus'
bs2.mats_eval
```

        \begin{array}{lrrl}\hline
        \textrm{E_m} & \textrm{30000.0} & & \textrm{None}  \\
                \textrm{E_f} & \textrm{200000.0} & & \textrm{None}  \\
                \textrm{E_b} & \textrm{223.18383360000004} & & \textrm{None}  \\
                \textrm{omega_fn_type} & \textrm{abaqus} & & \textrm{None}  \\
                \hline
        \textbf{omega_fn} & \textrm{AbaqusDamageFn} & & \textrm{None} \\
            \hline
        \end{array}

```python
bs2.mats_eval.omega_fn
```

Damage function (Abaqus)
        \begin{align}
        \omega = g(\kappa) = 1 -\left(\frac{s_0}{\kappa}\right)\left[ 1 - \frac{1 - \exp(- \alpha(\frac{\kappa - s_0}{s_u - s_0})}{1 - \exp(-\alpha)}  \right]
        \end{align}
        where $\kappa$ is the state variable representing
        the maximum slip that occurred so far in
        in the history of loading.

        \begin{array}{lrrl}\hline
        \textrm{s_u} & s_u = 0.003 & \textrm{[mm]} & \textrm{parameter of the damage function}  \\
                \textrm{alpha} & \alpha = 0.1 & \textrm{[-]} & \textrm{parameter controlling the slope of damage}  \\
                \textrm{s_0} & s_0 = 0.0004 & \textrm{[mm]} & \textrm{elastic strain limit}  \\
                \hline
        \hline
        \end{array}

```python
bs2.mats_eval.omega_fn.plot(plt.axes())
```

```python
bs2.loading_scenario.trait_set(loading_type='cyclic',
                               amplitude_type='increasing',
                               loading_range='symmetric',
                               )
bs2.loading_scenario.trait_set(number_of_cycles=8,
                               maximum_loading=0.0025,
                               unloading_ratio =0.0)
bs2.sim.run()
```

```python
bs2.loading_scenario.plot(plt.axes())
plt.grid(True)
```

```python
ax = plt.axes()
ax.grid(True)
bs2.hist.plot_Pw(ax,1)
```

## Tasks and Questions

### How to identify a damage function given an identified bond-slip curve experiment?

### Test the Abaqus damage function and the meaning of its parameters

```python

```


# 09 Errors

(See also *Computational Physics* (Landau, Páez, Bordeianu), Chapter 3)

These slides include material from *Computational Physics. eTextBook Python 3rd Edition.* Copyright © 2012 Landau, Rubin, Páez. Used under the Creative-Commons Attribution-NonCommerical-ShareAlike 3.0 Unported License.

## Stupidity or Incompetence

## Random errors
- cosmic rays
- random bit flips

## Approximation errors

"**algorithmic errors**"
   - simplifying and adapting mathematics to the computer
   - should decrease as $N$ increases

#### Example:
Approximate $\sin(x)$ with its truncated series expansion:
\begin{align}
\sin x &= \sum_{n=1}^{+\infty} \frac{(-1)^{n-1} x^{2n-1}}{(2n - 1)!}\\
  &\approx \sum_{n=1}^{N} \frac{(-1)^{n-1} x^{2n-1}}{(2n - 1)!} + \mathcal{E}(x, N)
\end{align}

## Round-off errors
- finite precision for storing floating-point numbers (32 bit, 64 bit)
- not known exactly (treat as uncertainty)
- can *accumulate* and lead to *garbage*

#### Example:
Assume you can only store only four decimals:

\begin{align}
     \text{storage}:&\quad \frac{1}{3} = 0.3333_c \quad\text{and}\quad \frac{2}{3} = 0.6667_c\\
     \text{exact}:&\quad 2\times\frac{1}{3} - \frac{2}{3} = 0\\
     \text{computer}:&\quad 2 \times 0.3333 - 0.6667 = -0.0001 \neq 0
\end{align}

... now imagine adding "$2\times\frac{1}{3} - \frac{2}{3}$" in a loop 100,000 times.

## The problems with *subtractive cancelation*
Model the computer representation $x_c$ of a number $x$ as

$$
x_c \simeq x(1+\epsilon_x)
$$

with the relative error $|\epsilon_x| \approx \epsilon_m$ (similar to machine precision).

What happens when we subtract two numbers $b$ and $c$:

$$a = b - c$$

\begin{gather}
a_c = b_c - c_c = b(1+\epsilon_b) - c(1+\epsilon_c)\\
\frac{a_c}{a} = 1 + \frac{b}{a}\epsilon_b - \frac{c}{a} \epsilon_c
\end{gather}

No guarantee that the errors cancel, and the relative error $|\frac{a_c}{a} - 1|$ can be huge for small $a$!

### Subtracting two nearly equal numbers

$$b \approx c$$ is bad!

\begin{align}
\frac{a_c}{a} &= 1 + \frac{b}{a}(\epsilon_b - \epsilon_c) \\
\left| \frac{a_c}{a} \right| &\leq 1 + \left| \frac{b}{a} \right| (|\epsilon_b| + |\epsilon_a|)
\end{align}

i.e. the large number $b/a$ magnifies the error.

# Beware of subtractions!

**If you subtract two large numbers and end up with a small one, then the small one is less significant than any of the large ones.**

## Round-off errors

Repeated calculations of quantities with errors beget new errors: In general, analyze with the rules of *error propagation*: function $f(x_1, x_2, \dots, x_N)$ with errors on the $x_i$ of $\epsilon_i$:
$$
\epsilon_f(x_1, x_2, \dots; \epsilon_1, \epsilon_2, \dots) =
   \sqrt{\sum_{i=1}^N \left(\epsilon_i \frac{\partial f}{\partial x_i}\right)^2}
$$

Example: division $a = b/c$ (... with short cut)

\begin{align}
a_c &= \frac{b_c}{c_c} = \frac{b(1+\epsilon_b)}{c(1+\epsilon_b)} \\
\frac{a_c}{a} &= \frac{1+\epsilon_b}{1+\epsilon_c}
    = \frac{(1+\epsilon_b)(1-\epsilon_c)}{1-\epsilon_c^2} \approx (1+\epsilon_b)(1-\epsilon_c)\\
    &\approx 1 + |\epsilon_b| + |\epsilon_c|
\end{align}

(neglected terms of order $\mathcal{O}(\epsilon^2)$); and same for multiplication.

**Errors accumulate with every operation.**

### Model for round-off error accumulation
View error in each calculation as a step in a *random walk*. The total "distance" (i.e. total error) $R$ over $N$ steps of length $r$ (the individual, "random" errors), is on average

$$ R \approx \sqrt{N} r $$

Total relative error $\epsilon_{\text{ro}}$ after $N$ calculations with error of the order of the machine precision $\epsilon_m$ is

$$ \epsilon_{\text{ro}} \approx \sqrt{N} \epsilon_m $$

(Only a model, depending on algorithm may be less or even $N!$...)

## Total error of an algorithm
What you need to know to evaluate an algorithm:
1. Does it converge? (What $N$ do I need?)
2. How precise are the connverged results (What is the error $\epsilon_\text{tot}$?)
3. What is its run time? (How fast is it for a given problem size?)

The total error contains *approximation* and *round off* errors:

\begin{gather}
\epsilon_\text{tot} = \epsilon_\text{app} + \epsilon_\text{ro}
\end{gather}

Model for the approximation error for an algorithm that takes $N$ steps (operations) to find a "good" answer:

$$
\epsilon_\text{app} \simeq \frac{\alpha}{N^\beta}
$$

and round off error as

$$
\epsilon_{\text{ro}} \approx \sqrt{N} \epsilon_m
$$

Model for total error:
$$
\epsilon_\text{tot} = \frac{\alpha}{N^\beta} + \sqrt{N} \epsilon_m
$$

Analyze $\log_{10} $ of the relative error (direct readout of number of significant decimals).

Image from Computational Physics. eTextBook Python 3rd Edition. Copyright © 2012 Landau, Rubin, Páez. Used under the Creative-Commons Attribution-NonCommerical-ShareAlike 3.0 Unported License.

### Example analysis
\begin{gather}
\epsilon_\text{app} = \frac{1}{N^2}, \quad \epsilon_\text{ro} = \sqrt{N}\epsilon_m\\
\epsilon_\text{tot} = \frac{1}{N^2} + \sqrt{N}\epsilon_m
\end{gather}

Total error is a *minimum* for

\begin{gather}
\frac{d\epsilon_\text{tot}}{dN} = -\frac{2}{N^{3}} + \frac{1}{2}\frac{\epsilon_m}{\sqrt{N}} = 0, \quad\text{thus} \quad
N^{5/2} = 4 \epsilon_m^{-1}\\
N = \left(\frac{4}{\epsilon_m}\right)^{2/5}
\end{gather}

What is the best $N$ for single precision $\epsilon_m \approx 10^{-7}$?

```python
import math
def N_opt(eps_m):
    return round(math.pow(4./eps_m, 2./5.))
def eps_app(N):
    return 1./(N*N)
def eps_ro(N, eps_m):
    return math.sqrt(N)*eps_m
```

```python
epsilon_m = 1e-7   # single precision

N = N_opt(epsilon_m)
err_app = eps_app(N)
err_ro = eps_ro(N, epsilon_m)
print("best N = {0}   (for eps_m={1})".format(N, epsilon_m))
print("eps_tot = {0:.3g}".format(err_app + err_ro))
print("eps_app = {0:.3g}, eps_ro = {1:.3g}".format(err_app, err_ro))
```

    best N = 1099   (for eps_m=1e-07)
    eps_tot = 4.14e-06
    eps_app = 8.28e-07, eps_ro = 3.32e-06

Single precision $\epsilon_m \approx 10^{-7}$:

$$
N \approx 1099\\
\epsilon_\text{tot} \approx 4 \times 10^{-6} \\
\epsilon_\text{app} = 8.28 \times 10^{-7} \\
\epsilon_\text{ro} = 3.32 \times 10^{-6}
$$

Here, most of the error is round-off error! What can you do?

* use double precision (delay round-off error)
* use a better algorithm, e.g. $\epsilon_\text{app}\simeq \frac{2}{N^4}$ (uses fewer steps)

**Better algorithms are always a good idea :-)**

Remember: trade-off between **approximation error** and **rounding error**.

```python

```