## setup and notebook configuration

```python
import numpy as np
import time
import quadpy
```

```python
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib.tri as mtri               # delaunay triangulation
from mpl_toolkits.mplot3d import Axes3D     # surface plotting
import seaborn as sns
import cmocean as cmo
sns.set()
%matplotlib inline
#%config InlineBackend.figure_format = 'retina'
#mpl.rcParams['figure.dpi']= 100
```

    /Users/corbin/virtual_envs/FEMexperiment_env/lib/python3.6/site-packages/cmocean/tools.py:76: MatplotlibDeprecationWarning: The is_string_like function was deprecated in version 2.1.
      if not mpl.cbook.is_string_like(rgbin[0]):

```python
%load_ext autoreload
%autoreload 2
```

```python
import src.fem_base.master.mk_master as mkm
import src.plot.matplotlib_utils as mplu
import src.fem_base.master.barycentric_coord_tools as bct
import src.fem_base.master.nodal_basis_1D as NB1D
import src.fem_base.master.nodal_basis_2D as nb2d
import src.fem_base.master.master_1D as m1d
import src.fem_base.master.polynomials_1D as p1d
import src.fem_base.master.polynomials_2D as p2d
```

# exposition

## why re-write? what's wrong with what's there

We will define our master triangle following the strange (but legacy) MPU convention of the points below, which is the opposite way from which the master element is usually defined. Note that
- MPU shifts the nodal points -- this is a good thing, but the user should have control over it
- there are serious performance issues with high order elements due to the use of sympy and symbolic gram-schmidt as an initialization technique. THERE MUST BE A BETTER WAY

We would like to re-write `Master_nodal` to be a bit more modular and flexible, as well as less differently-abled in terms of code organization

```python
mt = mkm.Master_nodal(order=3, dim=2, element=0)
```

```python
fig, ax = plt.subplots()
plt.scatter(mt.nodal_pts[:,0], mt.nodal_pts[:,1])
ax.axis('equal')
plt.show()
```

We can run the following benchmarking code (expensive), and generate a plot of wallclock time for master element initialization.

```python
master_init_times = []
for p in range(1, 11):
    start = time.clock()
    master_obj = mkm.Master_nodal(order=p, dim=2, element=0)
    end = time.clock()
    elapsed = end - start
    master_init_times.append(elapsed)
```

from which we plot

this is really slow. If we would like to play with higher order elements, we need a more efficient way of initializing these master elements.

# 2D triangle: basis and master element creation

## barycentric coordinate tools

In order to position points on the triangle, it will be convenient to work in [Barycentric coordinates](https://en.wikipedia.org/wiki/Barycentric_coordinate_system).
The following define the transformation to and from barycentric coordinates from cartesian coords.

```python
# %load -s cart2bary,bary2cart ../src/fem_base/master/barycentric_coord_tools.py
def cart2bary(verts, pts):
    """ retrns barycentric coords of pt on a tri w/ vertices verts
    @param verts  tuple of tuples specifying verts
    @param pt  numpy array of point on triangle shape (2, npts)
    """
    npts = pts.shape[1]
    _lambda = np.zeros((3, npts))
    (x1, y1), (x2, y2), (x3, y3) = verts
    T = np.array([[x1 - x3, x2 - x3],
                  [y1 - y3, y2 - y3]])
    r = np.copy(pts)
    r[0,:] -= x3
    r[1,:] -= y3

    _lambda[:2, :] = np.linalg.solve(T, r)
    _lambda[2, :] = 1 - _lambda[0,:] - _lambda[1,:]
    return _lambda

def bary2cart(verts, _lambda):
    """converts from barycentric to cartesian coordinates
    @param verts  tuple of tuples specifying triangle vertices
    @param _lambda array of barycentric points (3, npts)
    """
    npts = _lambda.shape[1]
    λ1, λ2, λ3 = _lambda[0,:], _lambda[1,:], _lambda[2,:]
    x, y = np.zeros(npts), np.zeros(npts)
    (x1, y1), (x2, y2), (x3, y3) = verts
    x = λ1*x1 + λ2*x2 + λ3*x3
    y = λ1*y1 + λ2*y2 + λ3*y3
    return x, y

```

The following generates the barycentric coordinates corresponding to uniform nodes on a triangle for a polynomial order $p$.

```python
# %load -s uniform_bary_coords ../src/fem_base/master/barycentric_coord_tools.py
def uniform_bary_coords(p):
    """ generate bary coords of uniform nodal pts for tri of order p"""
    N, Np = p, int((p+1)*(p+2)/2)
    node = 0
    bary = np.zeros((3, Np))
    for i in range(N+1):
        for j in range(N+1-i):
            λ1, λ2 = i/N, j/N
            λ3 = 1 - λ1 - λ2
            bary[:, node] = np.asarray([λ1, λ2, λ3])
            node += 1
    return bary

```

```python
# serves as a view mesh for the shape functions we are about to create
p = 24
Np = int((p+1)*(p+2)/2)
bary = uniform_bary_coords(p)
verts = ((-1, -1), (-1, 1), (1, -1))
x, y = bary2cart(verts, bary)
```

## modal basis: generating orthonormal polynomials

We need a way to generalize the 1D orthonormal polynomial basis to that over the standard simplex

$$T \equiv \{\mathbf{r}=(\xi, \eta) \mid (\xi, \eta) \leq -1; \xi+\eta \leq0\}$$

such that we can efficiently compute $\psi_m(\mathbf{r})$ over $T$. In Hesthaven and Warburton, the suggestion is to use the basis defined by

\begin{align}
   m(i,j) &= j + (N+1)i + 1 - \frac{i}{2}(i-1) \\
   \psi_m(\mathbf{r}) &= \sqrt{2} P_i(a) \, P_j^{(2i+1, 0)}(b)\, (1-b)^i \\
   &a = 2 \frac{1+\xi}{1-\eta} - 1, b=\eta
\end{align}

Where $P_n^{(\alpha, \beta)}$(x) is the familiar $n^{th}$ order Jacobi polynomial over [-1, 1] discussed in depth in the `1D_basis_and_master` tutorial. We leverage `orthopy` in order to generate these polynomials over barycentric coordinates, and supply the master element vertices to create the basis over $T$. However, we will need these relations in order to compute the derivatives of the shape functions.

```python
# %load -s xi_eta_to_ab ../src/fem_base/master/polynomials_2D.py
def xi_eta_to_ab(ξ, η):
    """ function to transform xi, eta coords to a, b
    see Hesthaven function 'rstoab'
    @param xi, eta vectors of xi, eta pts
    """
    a, b = np.zeros_like(ξ), np.zeros_like(η)
    singular = np.isclose(η, 1.0)
    nonsingular = np.logical_not(singular)
    a[nonsingular] = 2*(1. + ξ[nonsingular])/(1 - η[nonsingular]) - 1
    a[singular] = -1
    b = η
    return a, b

```

```python
# %load -s Simplex2DPoly ../src/fem_base/master/polynomials_2D.py
def Simplex2DPoly(a, b, i, j):
    """ generates the orthonormal polynomial over the master simplex (mapped to a, b coords)
    transcriber from Hesthaven 2008 (Simplex2DP.m)
    """
    h1 = p1d.JacobiP(a, 0, 0, i)
    h2 = p1d.JacobiP(b, 2*i+1, 0, j)
    P = np.sqrt(2.)*h1*h2*(1-b)**i
    return P

```

```python
# %load -s Vandermonde2D ../src/fem_base/master/polynomials_2D.py
def Vandermonde2D(N, ξ, η):
    a, b = xi_eta_to_ab(ξ, η)
    Np = int((N+1)*(N+2)/2)
    V2d = np.zeros((len(ξ), Np))
    counter = 0
    for i in range(N+1):
        for j in range(N-i+1):
            m = j + (N+1)*i + 1 - i/2.*(i-1)
            V2d[:,counter] = Simplex2DPoly(a, b, i,j)
            counter += 1
    return V2d

```

```python
V2d = Vandermonde2D(3, x, y)

print(V2d.shape)

fig = plt.figure(figsize=(9,6))
tri = mtri.Triangulation(x, y)

# data limits

for gg in range(10):
    ps2d = V2d[:, gg]
    ax = fig.add_subplot(3, 4, gg+1, projection='3d')
    ax.set_xticks([]); ax.set_yticks([]); ax.set_zticks([])
    cax = ax.plot_trisurf(tri, V2d[:,gg], cmap=cmo.cm.balance)
    ax.set_title(r'$P_{}$'.format(gg), y=1.06)
    ax.view_init(elev=90)
mplu.add_shared_colorbar(fig, ps2d, cmap=cmo.cm.balance)
plt.show()

```

Note that the Hesthaven polynomials are orthonormalized differently than the orthopy polynomials.

## modal basis: derivatives of orthonormal polynomials

\begin{align}
   m(i,j) &= j + (N+1)i + 1 - \frac{i}{2}(i-1) \\
   \psi_m(\mathbf{r}) &= \sqrt{2} P_i(a) \, P_j^{(2i+1, 0)}(b)\, (1-b)^i \\
   &a = 2 \frac{1+\xi}{1-\eta} - 1, b=\eta
\end{align}

So naturally

and the chain rule suggests that

\begin{align}
  \frac{d\psi_m}{d\xi} &= \frac{\partial a}{\partial\xi}\frac{\partial\psi_m}{\partial a}
  = \frac{2}{1 - \eta}\frac{\partial\psi_m}{\partial a}
  = \frac{2}{1 - b} \cdot \sqrt{2} \, P_j^{(2i+1, 0)}(b)\, (1-b)^i \frac{\partial}{\partial a}\left( P_i(a)  \right) \\
    &= 2(1-b)^{i-1}\cdot \sqrt{2} \, P_j^{(2i+1, 0)}(b)\,\frac{\partial}{\partial a}\left( P_i(a)  \right)
  \end{align}

  \begin{align}
  \frac{d\psi_m}{d\eta} &= \frac{\partial a}{\partial \eta}\frac{\partial \psi_m}{\partial a} + \frac{db}{d\eta}\frac{d\psi_m}{db}
  = \frac{2(1+\xi)}{(1-\eta)^2}\frac{\partial \psi_m}{\partial a} + \frac{d\psi_m}{db} \\
  &= \frac{2(1+\xi)}{(1-\eta)^2}\sqrt{2} \, P_j^{(2i+1, 0)}(b)\, (1-b)^i \frac{\partial}{\partial a}( P_i(a)  ) + \sqrt{2} \,P_i(a)\frac{\partial}{db}\left((1-b)^i\,P_j^{(2i+1, 0)}(b)\right) \\
%    &= \frac{2(1+\xi)}{(1-\eta)^2}\sqrt{2} \, P_j^{(2i+1, 0)}(b)\, (1-b)^i \frac{\partial}{\partial a}( P_i(a)  ) + \sqrt{2} \,P_i(a)\left[(1-b)^i\,\frac{\partial}{db}\left(P_j^{(2i+1, 0)}(b)\right) + P_j^{(2i+1, 0)}(b)\left(i(1-b)^{i-1}\right)\right] \\
\end{align}

We follow Hesthaven's simplified and normalized expression which is equivalent to the above.

```python
# %load -s Simplex2DPolyGradient,GradVandermonde2D ../src/fem_base/master/polynomials_2D.py
def Simplex2DPolyGradient(a, b, i, j):
    """ takes derivatives of modal basis polys w/r/t ξ, η
    transcribed from Hesthaven (GradSimplex2D.m)
    """
    Pa, dP_da = p1d.JacobiP(a,0,    0,i), p1d.GradJacobiP(a,0    ,0,i)
    Pb, dP_db = p1d.JacobiP(b,2*i+1,0,j), p1d.GradJacobiP(b,2*i+1,0,j)

    # d/dξ = da/dξ * d/da + db/dξ * d/db = 2/(1-b) d/da
    dψ_dξ = dP_da * Pb
    if i > 0:
        dψ_dξ *= ( (0.5*(1-b))**(i-1) )

    # d/dη = ((1+a)/2)/((1-b)/2)d/da + d/db
    dψ_dη = dP_da * (Pb * 0.5*(1+a))
    if i > 0:
        dψ_dη *= ((0.5*(1-b))**(i-1))

    tmp = dP_db * (0.5*(1-b))**(i)
    if i > 0:
        tmp -= 0.5*i*Pb * (0.5*(1-b))**(i-1)
    dψ_dη += Pa * tmp

    # normalize both derivatives
    dψ_dξ *= 2**(i+0.5)
    dψ_dη *= 2**(i+0.5)
    return [dψ_dξ, dψ_dη]

def GradVandermonde2D(p, ξ, η):
    """ compute the derivative vandermonde matrices in ξ, η directions """
    Np = int((p+1)*(p+2)/2)
    npts = len(ξ)
    a, b = xi_eta_to_ab(ξ, η)
    dVξ, dVη = np.zeros((npts, Np)), np.zeros((npts, Np))
    counter = 0
    for i in range(p+1):
        for j in range(p-i+1):
            dψ_dξ, dψ_dη = Simplex2DPolyGradient(a, b, i, j)
            dVξ[:,counter] = dψ_dξ
            dVη[:,counter] = dψ_dη
            counter += 1
    return [dVξ, dVη]

```

```python
# indices
p=2
gv = GradVandermonde2D(p, x, y)
V2d = Vandermonde2D(p, x, y)
```

Plot the polynomials, along with the derivative in each coordinate direction.

```python
tri = mtri.Triangulation(x, y)
nbasis_functions = V2d.shape[1]
grid=(1,3)

for m in range(nbasis_functions):

    fig = plt.figure(figsize=(8, 1.5))
    counter = 1
    data = np.vstack((V2d[:,m], gv[0][:,m], gv[1][:,m]))
    dmax = np.max(np.abs(data))
    #vmax, vmin = np.max(data), np.min(data)
    vmax, vmin = dmax, -1*dmax

    # original polynomial
    ax = fig.add_subplot(*grid, counter, projection='3d')
    if m==0:
        ax.set_xlabel(r'$\xi$')
        ax.set_ylabel(r'$\eta$')
    ax.set_xticks([-1, 1]); ax.set_yticks([-1, 1]); ax.set_zticks([])
    cax = ax.plot_trisurf(tri, V2d[:,m], vmin=vmin, vmax=vmax, cmap=cmo.cm.balance)
    ax.view_init(elev=90)
    counter+=1

    # derivative in ξ direction
    ax = fig.add_subplot(*grid, counter, projection='3d')
    ax.set_xticks([-1, 1]); ax.set_yticks([-1, 1]); ax.set_zticks([])
    cax = ax.plot_trisurf(tri, gv[0][:,m], vmin=vmin, vmax=vmax, cmap=cmo.cm.balance)
    ax.view_init(elev=90)
    counter+=1

    # derivative in η direction
    ax = fig.add_subplot(*grid, counter, projection='3d')
    ax.set_xticks([-1, 1]); ax.set_yticks([-1, 1]); ax.set_zticks([])
    cax = ax.plot_trisurf(tri, gv[1][:,m], vmin=vmin, vmax=vmax, cmap=cmo.cm.balance)
    ax.view_init(elev=90)
    counter+=1

    mplu.add_shared_colorbar(fig, np.linspace(vmin, vmax, 100), cmap=cmo.cm.balance,
                             cbar_ax=[0.92, 0.15, 0.01, 0.7])
    if m == 0:
        fig.suptitle(r'$\psi_m$,   $\frac{d\psi_m}{d\xi}$,   $\frac{d\psi_m}{d\eta}$',
                     y=1.2)
    plt.show()
```

Check that the magnitudes of the constant derivatives are correct

```python
# zero derivatives are zero
should_be_null = [gv[0][:,0], gv[1][:,0], gv[0][:,1], gv[0][:,2]]
assert np.all([np.allclose(arr, 0.) for arr in should_be_null])

# m = 1: dψ/dη constant is correcct
ψ1 = V2d[:,1]
slope = (np.max(ψ1) - np.min(ψ1))/2.
assert np.isclose(gv[1][:,1][0], slope)

# m = 3: dψ/dξ constant is correct
ψ3 = V2d[:,3]
slope = (np.max(ψ3) - np.min(ψ3))/2.
assert np.isclose(gv[0][:,3][0], slope)
```

## nodal basis: generating shape functions

Hesthaven makes the claim that $V^T \mathbf{\ell}(r) = \tilde{\mathbf{P}}(r)$. Just as we did when creating the 1D nodal shape functions, we can directly apply the relation

$$\ell = (V^T)^{-1} \tilde{\mathbf{P}}$$

which allows us to create our nodal shape functions from the orthonormal modal polynomial coefficients.

```python
p = 2
Np = int((p+1)*(p+2)/2)
bary = uniform_bary_coords(p)
verts = ((-1, -1), (-1, 1), (1, -1))
ξ, η = bary2cart(verts, bary)
V2D = Vandermonde2D(p, ξ, η)
print('shape of Vandermonde matrix: {}'.format(V2D.shape))

view_order = 32
ξ_view, η_view = bary2cart(verts, uniform_bary_coords(view_order))
P_tilde = Vandermonde2D(p, ξ_view, η_view)
Vti = np.linalg.inv(V2D.T)
nodal_shap = np.dot(Vti, P_tilde.T)
nodal_shap.shape
```

    shape of Vandermonde matrix: (6, 6)

    (6, 561)

```python
fig = plt.figure(figsize=(9, 4.5))
data = nodal_shap[4,:]

nb = nodal_shap.shape[0]
tri = mtri.Triangulation(ξ_view, η_view)
for i in range(nb):
    data = nodal_shap[i,:]
    ax = fig.add_subplot(2, 3, i+1, projection='3d')
    ax.set_xticks([-1, 0, 1]); ax.set_yticks([-1,0,1]); ax.set_zticks([]);
    cax = ax.plot_trisurf(tri, data, cmap=cmo.cm.balance)
    ax.view_init(elev=90)
fig = mplu.add_shared_colorbar(fig, data, cmo.cm.balance, ticks=[0,0.5,1])
plt.show()
```

honestly, a height plot isn't really the best way to show this. It's a bit funny. Probably need to get `cairo` `mpl` backend working and do flat plots with shading.

## `NodalBasis2DTriangle`

We load the `NodalBasis2DTriangle` in its entirety. The class is designed to evaluate the shape functions and their derivatives over the master triangle. The user has a choice as to uniformly space the nodal points, or warp them for better interpolation properties, discussed below.

```python
# %load -s NodalBasis2D ../src/fem_base/master/nodal_basis_2D.py
class NodalBasis2D(object): pass

```

```python
# %load -s NodalBasis2DTriangle ../src/fem_base/master/nodal_basis_2D.py
class NodalBasis2DTriangle(NodalBasis2D):
    verts = ((-1, -1), (1, -1), (-1, 1))
    def __init__(self, p, nodal_locations='UNIFORM'):
        """ creates a nodal basis over master element verts
        @param p  polynomial order of the nodal basis
        @param nodal_locations
            UNIFORM: uniformly spaced nodal points (in a barycentric sense)
            WARPED:  shifted points for better interpolation behavior (Hesthaven 2008)
        """
        self.p, self.nb = p, int((p+1)*(p+2)/2.)
        if nodal_locations == 'UNIFORM':
            self.nodal_pts = self.mk_uniform_nodal_pts()
        elif nodal_locations == 'WARPED':
            self.nodal_pts = self.mk_warped_nodal_pts()
        else: raise ValueError('node_spacing {} not recognized'.format(node_spacing))

    def shape_functions_at_pts(self, pts):
        """ computes values of shape functions at pts (npts, 2) """
        V = p2d.Vandermonde2D(N=self.p, ξ=self.nodal_pts[:,0], η=self.nodal_pts[:,1])
        VTi = np.linalg.inv(V.T)
        P_tilde = p2d.Vandermonde2D(N=self.p, ξ=pts[:,0], η=pts[:,1])
        shap = np.dot(VTi, P_tilde.T)
        return shap.T

    def shape_function_derivatives_at_pts(self, pts):
        """ compute the derivatives of shape fns in ξ, η directions at pts
        returns list of derivatives of shape functions indexed by coord direction
        """
        V = p2d.Vandermonde2D(N=self.p, ξ=self.nodal_pts[:,0], η=self.nodal_pts[:,1])
        dψ_dξ, dψ_dη = p2d.GradVandermonde2D(p=self.p, ξ=pts[:,0], η=pts[:,1])
        Vinv = np.linalg.inv(V)
        shap_der = [np.dot(dψ_dξ, Vinv), np.dot(dψ_dη, Vinv)]
        return shap_der

    def mk_uniform_nodal_pts(self):
        """ make uniformly spaced pts (barycentric sense) on the master element """
        uniform_bary_coords = bct.uniform_bary_coords(self.p)
        xp, yp = bct.bary2cart(self.verts, uniform_bary_coords)
        nodal_pts = np.vstack((xp, yp)).T
        return nodal_pts

    def mk_warped_nodal_pts(self, α=0.):
        """ compute warped nodal points on the order p master triangle
        @param α  blending constant, see Hesthaven p.179
        NOTE: Hesthaven uses scaled barycentric coords, so we transcribe
            Nodes2D.m, xytors.m directly rather than rescale things. This is done
            for readability in comparison to the Hesthaven text.
        """
        p = self.p
        λ1, λ2, λ3 = bct.uniform_bary_coords(p)
        x, y = -λ2 + λ3, (-λ2 - λ3 + 2*λ1) / np.sqrt(3)

        # warping and blending functions
        wt1, wt2, wt3 = self._w_t(p, λ3-λ2), 0.5*self._w_t(p, λ1-λ3), 0.5*self._w_t(p,λ2-λ1)
        b1, b2, b3 = 4*λ3*λ2, 4*λ3*λ1, 4*λ2*λ1
        w1, w2, w3 = wt1*b1*(1+(α*λ1)**2), wt2*b2*(1+(α*λ2)**2), wt3*b3*(1+(α*λ3)**2)

        # move the cartesian points on the equilateral triangle
        x += (1)*w1 + (-1)        *w2 + (-1)         *w3
        y += (0)*w1 + (np.sqrt(3))*w2 + (-np.sqrt(3))*w3

        # map back to master element from equilateral triangle
        λ1, λ2, λ3 = (np.sqrt(3)*y+1)/3, (-3*x-np.sqrt(3)*y+2)/6, (3*x-np.sqrt(3)*y+2)/6
        xp, yp = -λ2 + λ3 - λ1, -λ2 -λ3 + λ1
        nodal_pts = np.vstack((xp, yp)).T
        return nodal_pts

    def _w_t(self, p, pts):
        """ evaluate 1D warp factor w_tilde at order N at pts
        NOTE: see Hesthaven p.176
        """
        nb1d = NB1D.NodalBasis1D(p=p, node_spacing='EQUIDISTANT')
        shap = nb1d.shape_functions_at_pts(pts)

        r_eq = nb1d.nodal_pts
        r_LGL, _ = NB1D.LegendreGaussLobatto(nb1d.nb)

        numerator = np.dot(shap, r_LGL - r_eq)
        denom = 1 - pts**2
        denom[np.isclose(denom, 0.)] = 1  # don't divide by 0, numerator 0 here
        wr = numerator / denom
        return wr

```

## nodal basis: visualization

We define two `NodalBasis2DTriangle` instances. The first are the order $p=3$ uniform nodes whose shape functions we would like to plot. We will evaluate the shape functions at the uniform nodal points of a much higher order element.

```python
nb_plot = NodalBasis2DTriangle(p=3)
nbp_fine = NodalBasis2DTriangle(p=25)
view_pts = nbp_fine.nodal_pts
```

```python
nx, ny = nb_plot.nodal_pts.T
x, y = view_pts.T

fig = plt.figure(figsize=(9, 5))
ax = fig.add_subplot(1, 2, 1)
ax.scatter(nx, ny)
ax.axis('equal')
ax.set_title('nodal points')

ax = fig.add_subplot(1, 2, 2)
ax.scatter(x, y)
ax.set_title('view points')
ax.axis('equal')
plt.show()
```

```python
def plot_nodal_shape_functions(fig, grid, nodal_basis, view_points):
    shap = nodal_basis.shape_functions_at_pts(view_pts)
    x, y = view_pts.T
    tri = mtri.Triangulation(x, y)
    for i in range(nodal_basis.nb):
        data = shap[:,i]
        ax = fig.add_subplot(*grid, i+1, projection='3d')
        ax.set_xticks([-1, 0, 1]); ax.set_yticks([-1,0,1]); ax.set_zticks([]);
        cax = ax.plot_trisurf(tri, data, cmap=cmo.cm.balance, vmax=np.max(shap), vmin=np.min(shap))
        ax.view_init(elev=75, azim=-70)
    fig = mplu.add_shared_colorbar(fig, shap, cmo.cm.balance, ticks=[0,0.5,1])
    fig.suptitle(r'nodal shape functions, $p = {}$'.format(nodal_basis.p), y=0.92)
    return fig
```

Plot the nodal shape functions.

```python
fig = plt.figure(figsize=(9, 9))
grid = (4,3)
fig = plot_nodal_shape_functions(fig, grid, nb_plot, view_pts)
plt.show()
```

## nodal basis: node warping and blending

All of the developments so far have been agnostic to choice of where the nodal points were placed, but (as in 1D), the equally spaced points are not optimal for interpolation and are susceptible to Runge phenomenon. In this section, we consider a method by which we can perturb the nodal points for good interpolation behavior.

the warping function is quantity defined over the 1D master element [-1, 1] as

\begin{align}
    \tilde{w}(r) \equiv \frac{ \sum_{i=1}^{N_p} (r_i^{LGL} - r_i^{eq})\phi^{eq}_i(r)}{1 - r^2}
\end{align}

where $r_i^{LGL},\, r_i^{eq}$ are the $i^{th}$ LegendreGaussLobatto and equally spaced points, respectively, and $\phi_i^{eq}$ are the nodal 1D shape functions defined over the equally spaced points $r_i^{eq}$. Note that we can use leverage our `NodalBasis1D` class in order to do this in the class definition above.

Let's look at the difference between the warped points and the uniform points at different element orders:

```python
nb_plot = NodalBasis2DTriangle(p=5, nodal_locations='UNIFORM')
nbp_fine = NodalBasis2DTriangle(p=30)
view_pts = nbp_fine.nodal_pts

fig = plt.figure(figsize=(8, 10))
grid = (7,3)
fig = plot_nodal_shape_functions(fig, grid, nb_plot, view_pts)
plt.show()
```

We can start to see some Runge phenomena even at $p=5$. Note that the shape functions exceed 1 and dip far below 0. by contrast, warping the nodal locations makes the shape functions much better behaved.

```python
nb_plot = NodalBasis2DTriangle(p=5, nodal_locations='WARPED')
nbp_fine = NodalBasis2DTriangle(p=30)
view_pts = nbp_fine.nodal_pts

fig = plt.figure(figsize=(8, 10))
grid = (7,3)
fig = plot_nodal_shape_functions(fig, grid, nb_plot, view_pts)
plt.show()
```

So we see that it is in our interest to warp the nodal points when we are considering high-order elements.

## master triangle

### quadrature over the master triangle

We use the stable and open source library `quadpy` to obtain integration points over the triangle, via a small adapter function. By default, we use the Xiao Gimbutas schemes from 2010 (up to order 50).

```python
# %load -s triangle_quadrature ../src/fem_base/master/master_2D.py
def triangle_quadrature(n, verts):
    """ look up / compute quadrature rule over the triangle, order n
    @param n  the order of polynomial which should be integrated exactly
    @param verts  tuple of tuples defining the master element
    NOTE: leverages quadpy, 2*weights
    """
    if n > 50:
        raise NotImplementedError
    qr = quadpy.triangle.xiao_gimbutas.XiaoGimbutas(index=n)
    bary, weights = qr.bary, qr.weights
    xq, yq = bct.bary2cart(verts=verts, _lambda=bary.T)
    points = np.vstack((xq, yq)).T
    return points, 2*weights

```

```python
# %load -s Master2D ../src/fem_base/master/master_2D.py
class Master2D(object):

    def mk_shap_and_dshap_at_pts(self, pts):
        shap = self.basis.shape_functions_at_pts(pts)
        dshap = self.basis.shape_function_derivatives_at_pts(pts)
        return shap, dshap

```

```python
# %load -s Master2DTriangle ../src/fem_base/master/master_2D.py
class Master2DTriangle(Master2D):
    """ note vertex definitions in nodal_basis_2D.py """
    def __init__(self, p, nquad_pts=None, *args, **kwargs):
        self.p, self.dim = p, 2
        self.basis = nb2d.NodalBasis2DTriangle(self.p, **kwargs)
        self.nb, self.verts, self.n_ed = self.basis.nb, self.basis.verts, 3
        self.nodal_pts = self.basis.nodal_pts
        self.nq = 2*self.p +2 if nquad_pts is None else nquad_pts
        self.quad_pts, self.wghts = triangle_quadrature(self.nq, self.verts)

        # shape functions at nodal and quadrature points
        self.shap_quad,  self.dshap_quad = self.mk_shap_and_dshap_at_pts(self.quad_pts)
        _, self.dshap_nodal = self.mk_shap_and_dshap_at_pts(self.nodal_pts)

        # mass, stiffness matrices
        self.M, self.S, self.K = self.mk_M(), self.mk_S(), self.mk_K()
        self.Minv = np.linalg.inv(self.M)

        # edge data structures, master edge, nodes on the edge, lifting matrix, normals
        self.master_edge = [m1d.Master1D(p=self.p)]
        self.ids_ed = self.find_nodes_on_edges()
        self.nr = sum([len(ids) for ids in self.ids_ed])
        self.L = self.mk_L()
        self.edge_normals = np.array([[1/np.sqrt(2), 1/np.sqrt(2)], [-1, 0], [0, -1]])

    def mk_M(self):
        """ the mass matrix, M_ij = (phi_i, phi_j) """
        shapw = np.dot(np.diag(self.wghts), self.shap_quad)
        M = np.dot(self.shap_quad.T, shapw)
        return M

    def mk_S(self):
        """ the stiffness matrix, S[k]_ij = (phi_i, \frac{d\phi_j}{dx_k})
        returns a list indexed by coordinate direction on the master element
        """
        S = [None, None]
        for i in range(self.dim):
            dshapw = np.dot(np.diag(self.wghts), self.dshap_quad[i])
            S[i] = np.dot(self.shap_quad.T, dshapw)
        return S

    def mk_K(self):
        """ the stiffness matrix, K_ij = (\frac{d\phi_i}{dx_k}, \frac{d\phi_j}{dx_k})
        returns a list indexed by coordinate direction on the master element
        """
        K = [None, None]
        for i in range(self.dim):
            dshapw = np.dot(np.diag(self.wghts), self.dshap_quad[i])
            K[i] = np.dot(self.dshap_quad[i].T, dshapw)
        return K

    def find_nodes_on_edges(self):
        """ computes the node numbers (ids) on each edge
        the i^th barycentric coord of a point on a tri edge will be 0, find these pts
        @retval ids_ed  list of vectors indexed by edge number
        NOTE: we manually flip edges 0 and 2 to ensure CCW ordering
            of ed dof around the element
        """
        ids_ed = [None, None, None]
        bary_coords = bct.cart2bary(self.verts, self.nodal_pts.T)
        ids_ed[0] = np.where( np.isclose(bary_coords[0, :], 0.) )[0][::-1]
        ids_ed[1] = np.where( np.isclose(bary_coords[1, :], 0.) )[0]
        ids_ed[2] = np.where( np.isclose(bary_coords[2, :], 0.) )[0][::-1]
        return ids_ed

    def mk_L(self):
        """ makes the elemental lifting matrix """
        L = np.zeros((self.nb, self.nr), dtype=int)
        for ed_dof, interior_dof in enumerate(np.hstack(self.ids_ed)):
            L[interior_dof, ed_dof] = 1
        return L

```

### lifting matrix and edge data structures

barycentric coordinates provide an excellent way to find the nodal points on the edge of the element $\lambda_1, \lambda_2, \lambda_3$ will be 0 if the point is on edge 1, 2, 3, respectively.

```python
mt = Master2DTriangle(p=3, nodal_locations='UNIFORM')
xn, yn = mt.nodal_pts.T

# draw the interior nodes
fig = plt.figure(figsize=(12, 4))
ax = fig.add_subplot(1, 3, 1)
ax.axis('equal')
ax.scatter(xn, yn, s=12)
for idx, (xn, yn) in enumerate(mt.nodal_pts):
    ax.annotate(idx, (xn, yn))

# draw the element edge space nodes
ax = fig.add_subplot(1, 3, 2)
shift = 0.2
counter = 0
for edge, ids_ed in enumerate(mt.ids_ed):
    nx, ny = mt.edge_normals[edge].T
    edx, edy = mt.nodal_pts[ids_ed, :].T
    edx += shift * nx
    edy += shift * ny
    ax.scatter(edx, edy, c='r', s=12)
    for n in range(len(ids_ed)):
        ax.annotate(counter, (edx[n], edy[n]))
        counter += 1
    ax.scatter(mt.nodal_pts[:,0], mt.nodal_pts[:,1], c='b', s=12)
ax.axis('equal')
plt.show()
```

```python
# compute the points on each edge with bary coords
edge = 1
edge_indices = mt.ids_ed[edge]

fig, ax = plt.subplots(figsize=(4,4))
xn, yn = mt.nodal_pts.T
ax.scatter(xn, yn, alpha=0.4, label='nodal points')
ax.scatter(xn[edge_indices], yn[edge_indices], label='edge {} points'.format(edge))
ax.legend()
plt.show()
```

```python
interior_numbers = np.arange(mt.nb)
edge_numbers = np.ones(mt.nr, dtype=int)

interior2edge = np.dot(mt.L, edge_numbers)
print(interior2edge)

edge2interior = np.dot(mt.L.T, interior_numbers)
print(edge2interior)
```

    [2 1 1 2 1 0 1 1 1 2]
    [3 2 1 0 0 4 7 9 9 8 6 3]

### performance test for master element initialization

```python
# sort of unit test
for p in range(1, 5):
    mt = Master2DTriangle(p=p, nodal_locations='WARPED')
    mt2 = mkm.Master_nodal(order=p, dim=2, element=0)
    assert(np.allclose(mt.M, mt2.M))
```

```python
p=1
mt = Master2DTriangle(p=p, nodal_locations='WARPED')
mt2 = mkm.Master_nodal(order=p, dim=2, element=0)
print(mt2.K[0])
print()
print(mt.S[0].T)
print()
print(np.dot(mt.Minv, mt.dshap_nodal[0]).T)
```

    [[-0.33333333 -0.33333333 -0.33333333]
     [ 0.33333333  0.33333333  0.33333333]
     [ 0.          0.          0.        ]]

    [[ 0.33333333  0.33333333  0.33333333]
     [-0.33333333 -0.33333333 -0.33333333]
     [ 0.          0.          0.        ]]

    [[ 0.75  0.75  0.75]
     [-0.75 -0.75 -0.75]
     [ 0.    0.    0.  ]]

Let's integrate the volume of the master element

```python
np.sum(mt.M)
```

    2.0000000000000004

```python
master_init_times = []
for p in range(1, 21):
    start = time.clock()
    master_obj = Master2DTriangle(p=p)
    end = time.clock()
    elapsed = end - start
    master_init_times.append(elapsed)
```

```python
x = np.arange(1, 21)
plt.plot(x, master_init_times, '.--')
plt.xlabel('polynomial order')
plt.ylabel('initialization wallclock time [s]')
plt.xticks(x)
plt.show()
```

0.7 seconds for an order 20 element! We have reduced initialization time by 3 orders of magnitude!

Test differentiation matrices

## master quadrilateral

# 2D quad: basis and master element creation

Creation of a nodal basis and subsequent quadrilateral master element is intrinsically simpler than that of the master triangle, because we can form the relevant data structures by taking tensor products of the well conditioned 1D objects, due to the inherently cubic nature of the element. This is what Hesthaven suggests (citation needed).

# TODO

## functionality

## unit tests
- unit test where we transform from bary 2 cart 2 bary again
- unit test where we integrate the polys and recover the identity matrix
- unit test where we check Lesbesgue constant and compare to Hesthaven's
- hit polynomial functions with Dr to see if they converge to check derivatives
- unit test to test discrete differentiation matrices

```python

```


```python
import math
π = math.pi; x=20.1234; y=-4.56789
print("Jupyter notebooks combine the flexibility of an interactive notebook with the power of Python")
print(f"Python can work with Unicode characters directly. π is equal to {str(π)}")
print('Python can format output multiple ways.')
print('x is ' + '{:+.2f}'.format(x) + ' and y is ' + '{:+.2f}'.format(y))
```

    Jupyter notebooks combine the flexibility of an interactive notebook with the power of Python
    Python can work with Unicode characters directly. π is equal to 3.141592653589793
    Python can format output multiple ways.
    x is +20.12 and y is -4.57

Notebooks can have embedded latex equations
\begin{equation}
	f(t)=\frac{f_{end}-f_{start}}{T_c}t + f_{start}
    \Delta\phi(t) = 2\pi f(t)\Delta t
\end{equation}

and images such as this software defined radio

```python
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
#Scatter plot with line
fig = plt.figure(figsize=(5,3.5))
ax=fig.add_subplot(1,1,1)
color='r'
N=256
x=4*np.random.randn(N,1)
y=np.random.randn(N,1)
ax.plot(x,y,color + '.', label='points')
ax.plot(np.linspace(np.min(x), np.max(x), num=N),np.linspace(np.min(y), np.max(y), num=N),'b',label='line')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.legend()
ax.set_title('Scatter plot and line')
```


# Problem set 7: Solving the consumer problem with income risk

```python
import numpy as np
import scipy as sp
from scipy import linalg
from scipy import optimize
from scipy import interpolate
import sympy as sm

%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
from matplotlib import cm
from mpl_toolkits.mplot3d import Axes3D
```

# Tasks

```python
sm.init_printing(use_unicode=True)
```

## Optimization problem I

Consider the function

$$
f(\boldsymbol{x}) = f(x_1,x_2) = (x_1^2 - x_1x_2 + x_2^2)^2
$$

Define it in **sympy** by:

```python
x1 = sm.symbols('x_1')
x2 = sm.symbols('x_2')
f = (x1**2 - x1*x2 + x2**2)**2
f
```

The **Jacobian** is

```python
f1 = sm.diff(f,x1)
f2 = sm.diff(f,x2)
sm.Matrix([f1,f2])
```

The **Hessian**  is

```python
f11 = sm.diff(f,x1,x1)
f12 = sm.diff(f,x1,x2)
f21 = sm.diff(f,x2,x1)
f22 = sm.diff(f,x2,x2)
sm.Matrix([[f11,f12],[f21,f22]])
```

**Question A:** Create a 3D plot and a contour plot of $f(x_1,x_2)$ such as those in the answer below.

```python
_f = sm.lambdify((x1,x2),f)

# write your code here
```

**Answer:**

```python
# a. grids
x1_vec = np.linspace(-2,2,500)
x2_vec = np.linspace(-2,2,500)
x1_grid,x2_grid = np.meshgrid(x1_vec,x2_vec,indexing='ij')
f_grid = _f(x1_grid,x2_grid)

# b. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.plot_surface(x1_grid,x2_grid,f_grid,cmap=cm.jet)

# c. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$f$')

# d. invert xaxis
ax.invert_xaxis()

# e. add colorbar
fig.colorbar(cs);
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
levels = np.sort([j*10**(-i) for i in [-1,0,1,2,3,4] for j in [0.5,1,1.5]])
cs = ax.contour(x1_grid,x2_grid,f_grid,levels=levels,cmap=cm.jet)
fig.colorbar(cs);
```

**Question B:** Construct python functions for the jacobian and the hessian.

```python
f_python = lambda x: _f(x[0],x[1])

# write your code here
```

**Answer:**

```python
_f1 = sm.lambdify((x1,x2),f1)
_f2 = sm.lambdify((x1,x2),f2)
_f11 = sm.lambdify((x1,x2),f11)
_f12 = sm.lambdify((x1,x2),f12)
_f21 = sm.lambdify((x1,x2),f21)
_f22 = sm.lambdify((x1,x2),f22)

def f_jac(x):
    return np.array([_f1(x[0],x[1]),_f2(x[0],x[1])])

def f_hess(x):
    row1 = [_f11(x[0],x[1]),_f12(x[0],x[1])]
    row2 = [_f21(x[0],x[1]),_f22(x[0],x[1])]
    return np.array([row1,row2])

```

**Question C:** Minimize $f(x_1,x_2)$ using respectively

1. Nelder-Mead,
* BFGS without analytical jacobian,
* BFGS with analytical jacobian, and
* Newton-CG with analytical jacobian and hessian

Compare the results and discuss which optimizer you prefer.

**Optional:** If you wish, you can use the functions defined in the hidden cells below to also track how the optimizers converges to the solution.

```python
def collect(x):

    # globals used to keep track across iterations
    global evals # set evals = 0 before calling optimizer
    global x0
    global x1s
    global x2s
    global fs

    # a. initialize list
    if evals == 0:
        x1s = [x0[0]]
        x2s = [x0[1]]
        fs = [f_python(x0)]

    # b. append trial values
    x1s.append(x[0])
    x2s.append(x[1])
    fs.append(f_python(x))

    # c. increment number of evaluations
    evals += 1
```

```python
def contour():

    global evals
    global x1s
    global x2s
    global fs

    # a. contour plot
    fig = plt.figure(figsize=(10,4))
    ax = fig.add_subplot(1,2,1)
    levels = np.sort([j*10**(-i) for i in [-1,0,1,2,3,4] for j in [0.5,1,1.5]])
    cs = ax.contour(x1_grid,x2_grid,f_grid,levels=levels,cmap=cm.jet)
    fig.colorbar(cs)
    ax.plot(x1s,x2s,'-o',ms=4,color='black')
    ax.set_xlabel('$x_1$')
    ax.set_ylabel('$x_2$')

    # b. function value
    ax = fig.add_subplot(1,2,2)
    ax.plot(np.arange(evals+1),fs,'-o',ms=4,color='black')
    ax.set_xlabel('iteration')
    ax.set_ylabel('function value')
```

```python
x0 = [-2,-1] # suggested initial guess

# write your code here
```

**Answer:**

```python
print('Nelder-Mead:')
evals = 0
result = optimize.minimize(f_python,x0,method='Nelder-Mead',callback=collect,options={'disp':True})
contour()
```

```python
print('BFGS without analytical gradient:')

evals = 0
result = optimize.minimize(f_python,x0,method='BFGS',callback=collect,options={'disp':True})
contour()
```

```python
print('BFGS with analytical gradient:')

evals = 0
result = optimize.minimize(f_python,x0,jac=f_jac,method='BFGS',callback=collect,options={'disp':True})
contour()
```

```python
print('Newton-CG with analytical gradient and hessian:')

evals = 0
result = optimize.minimize(f_python,x0,jac=f_jac,hess=f_hess,method='Newton-CG',callback=collect,options={'disp':True})
contour()
```

## Optimization problem II

Consider the function

$$
f(x_1,x_2) = (4-2.1x_1^2 + \frac{x_1^4}{3})x_1^2 + x_1x_2 + (4x_2^2 - 4)x_2^2)
$$

Define it in **sympy** by:

```python
x1 = sm.symbols('x_1')
x2 = sm.symbols('x_2')
f = (4-2.1*x1**2 + (x1**4)/3)*x1**2 + x1*x2 + (4*x2**2 - 4)*x2**2
_f = sm.lambdify((x1,x2),f)
f
```

Create **3D plot**:

```python
# a. grids
x1_vec = np.linspace(-2,2,500)
x2_vec = np.linspace(-1,1,500)
x1_grid,x2_grid = np.meshgrid(x1_vec,x2_vec,indexing='ij')
f_grid = _f(x1_grid,x2_grid)

# b. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.plot_surface(x1_grid,x2_grid,f_grid,cmap=cm.jet)

# c. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$f$')

# d. invert xaxis
ax.invert_xaxis()

# e. remove background
ax.xaxis.pane.fill = False
ax.yaxis.pane.fill = False
ax.zaxis.pane.fill = False

# f. add colorbar
fig.colorbar(cs);
```

**Question A:** Find the minimum of the function starting from each of the suggested initial values below. Print the first 20 solutions, and all solutions aftwards, which is the best yet to be found. Save the solutions and associated function values in `xs` and `fs`.

```python
# a. python function for f
f_python = lambda x: _f(x[0],x[1])

# b. initial guesses
np.random.seed(1986)
K = 1000
x0s = np.empty((K,2))
x0s[:,0] = -2 + 4*np.random.uniform(size=K)
x0s[:,1] = -1 + 2*np.random.uniform(size=K)

# c. solutions and associated values
xs = np.empty((K,2))
fs = np.empty(K)

# write your code here
```

**Answer:**

```python
fopt = np.inf
xopt = np.nan
for i,x0 in enumerate(x0s):

    # a. optimize
    result = optimize.minimize(f_python,x0,method='BFGS')
    xs[i,:] = result.x
    fs[i] = result.fun

    # b. print first 20 or if better than seen yet
    if i < 20 or fs[i] < fopt: # plot 20 first or if improving
        if fs[i] < fopt:
            xopt = xs[i,:]
            fopt = fs[i]
        print(f'{i:4d}: x0 = ({x0[0]:6.2f},{x0[0]:6.2f})',end='')
        print(f' -> converged at ({xs[i][0]:6.2f},{xs[i][1]:6.2f}) with f = {fs[i]:.12f}')

# best solution
print(f'\nbest solution:\n x = ({xopt[0]:6.2f},{xopt[1]:6.2f}) -> f = {fopt:.12f}')
```

       0: x0 = (  0.28,  0.28) -> converged at (  0.09, -0.71) with f = -1.031628453485
       1: x0 = ( -1.69, -1.69) -> converged at ( -1.70,  0.80) with f = -0.215463824384
       2: x0 = (  0.43,  0.43) -> converged at (  0.09, -0.71) with f = -1.031628453490
       3: x0 = (  1.59,  1.59) -> converged at (  1.70, -0.80) with f = -0.215463824384
       4: x0 = (  0.18,  0.18) -> converged at (  0.09, -0.71) with f = -1.031628453490
       5: x0 = (  0.81,  0.81) -> converged at ( -0.09,  0.71) with f = -1.031628453490
       6: x0 = ( -0.46, -0.46) -> converged at ( -0.09,  0.71) with f = -1.031628453487
       7: x0 = (  0.61,  0.61) -> converged at (  0.09, -0.71) with f = -1.031628453490
       8: x0 = (  0.76,  0.76) -> converged at ( -0.09,  0.71) with f = -1.031628453484
       9: x0 = (  0.87,  0.87) -> converged at ( -0.09,  0.71) with f = -1.031628453490
      10: x0 = (  0.76,  0.76) -> converged at ( -0.09,  0.71) with f = -1.031628453489
      11: x0 = (  1.23,  1.23) -> converged at (  1.70, -0.80) with f = -0.215463824384
      12: x0 = ( -0.87, -0.87) -> converged at (  0.09, -0.71) with f = -1.031628453486
      13: x0 = (  1.03,  1.03) -> converged at (  0.09, -0.71) with f = -1.031628453490
      14: x0 = ( -0.77, -0.77) -> converged at ( -0.09,  0.71) with f = -1.031628453490
      15: x0 = ( -0.25, -0.25) -> converged at (  0.09, -0.71) with f = -1.031628453490
      16: x0 = (  0.21,  0.21) -> converged at (  0.09, -0.71) with f = -1.031628453488
      17: x0 = ( -0.27, -0.27) -> converged at (  0.09, -0.71) with f = -1.031628453490
      18: x0 = (  0.31,  0.31) -> converged at ( -0.09,  0.71) with f = -1.031628453490
      19: x0 = (  1.57,  1.57) -> converged at (  1.61,  0.57) with f = 2.104250310311
      24: x0 = ( -0.14, -0.14) -> converged at (  0.09, -0.71) with f = -1.031628453490
      27: x0 = ( -0.13, -0.13) -> converged at (  0.09, -0.71) with f = -1.031628453490
     155: x0 = (  0.60,  0.60) -> converged at (  0.09, -0.71) with f = -1.031628453490

    best solution:
     x = (  0.09, -0.71) -> f = -1.031628453490

**Question B:** Create a 3D scatter plot of where the optimizer converges, and color the dots by the associated function values.

```python
# write your code here
```

**Answer:**

```python
# a. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.scatter(xs[:,0],xs[:,1],fs,c=fs);

# b. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$f$')

# c. invert xaxis
ax.invert_xaxis()

# d. colorbar
fig.colorbar(cs);
```

**Question C:** Plot the function values at the solutions as a function of the starting values.

```python
# write your code here
```

**Answer:**

```python
# a. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.scatter(x0s[:,0],x0s[:,1],fs,c=fs);

# b. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$f$')

# c. invert xaxis
ax.invert_xaxis()

# d. colorbar
fig.colorbar(cs);
```

```python
sm.init_printing(pretty_printing=False)
```

# Problem: Solve the consumer problem with income risk I

Define the following **variables** and **parameters**:

* $m_t$ is cash-on-hand in period $t$
* $c_t$ is consumption in period $t$
* $y_t$ is income in period $t$
* $\Delta \in (0,1)$ is income risk
* $r$ is the interest rate
* $\beta > 0$, $\rho > 1$, $\nu > 0 $, $\kappa > 0$, $\xi > 0$ are utility parameters

In the **second period** the household solves:

$$
\begin{aligned}
v_{2}(m_{2}) &= \max_{c_{2}}\frac{c_{2}^{1-\rho}}{1-\rho}+\nu\frac{(m_{2}-c_{2}+\kappa)^{1-\rho}}{1-\rho} \\
 \text{s.t.} \\
c_{2} & \in [0,m_{2}]
\end{aligned}
$$

In the **first period** the household solves:

$$
\begin{aligned}
v_{1}(m_{1}) & =
\max_{c_{1}}\frac{c_{1}^{1-\rho}}{1-\rho}+\beta\mathbb{E}_{1}\left[v_2(m_2)\right] \\
\text{s.t.} \\
m_2 &= (1+r)(m_{1}-c_{1})+y_{2} \\
y_{2} &= \begin{cases}
1-\Delta & \text{with prob. }0.5\\
1+\Delta & \text{with prob. }0.5
\end{cases}\\
c_{1} & \in [0,m_{1}]\\
\end{aligned}
$$

The **basic functions** are:

```python
def utility(c,rho):
    return c**(1-rho)/(1-rho)

def bequest(m,c,nu,kappa,rho):
    return nu*(m-c+kappa)**(1-rho)/(1-rho)

def v2(c2,m2,rho,nu,kappa):
    return utility(c2,rho) + bequest(m2,c2,nu,kappa,rho)

def v1(c1,m1,rho,beta,r,Delta,v2_interp):

    # a. v2 value, if low income
    m2_low = (1+r)*(m1-c1) + 1-Delta
    v2_low = v2_interp([m2_low])[0]

    # b. v2 value, if high income
    m2_high = (1+r)*(m1-c1) + 1+Delta
    v2_high = v2_interp([m2_high])[0]

    # c. expected v2 value
    v2 = 0.5*v2_low + 0.5*v2_high

    # d. total value
    return utility(c1,rho) + beta*v2
```

The **solution functions** are:

```python
def solve_period_2(rho,nu,kappa,Delta):

    # a. grids
    m2_vec = np.linspace(1e-8,5,500)
    v2_vec = np.empty(500)
    c2_vec = np.empty(500)

    # b. solve for each m2 in grid
    for i,m2 in enumerate(m2_vec):

        # i. objective
        obj = lambda c2: -v2(c2,m2,rho,nu,kappa)

        # ii. initial value (consume half)
        x0 = m2/2

        # iii. optimizer
        result = optimize.minimize_scalar(obj,x0,method='bounded',bounds=[1e-8,m2])

        # iv. save
        v2_vec[i] = -result.fun
        c2_vec[i] = result.x

    return m2_vec,v2_vec,c2_vec

def solve_period_1(rho,beta,r,Delta,v1,v2_interp):

    # a. grids
    m1_vec = np.linspace(1e-8,4,100)
    v1_vec = np.empty(100)
    c1_vec = np.empty(100)

    # b. solve for each m1 in grid
    for i,m1 in enumerate(m1_vec):

        # i. objective
        obj = lambda c1: -v1(c1,m1,rho,beta,r,Delta,v2_interp)

        # ii. initial guess (consume half)
        x0 = m1*1/2

        # iii. optimize
        result = optimize.minimize_scalar(obj,x0,method='bounded',bounds=[1e-8,m1])

        # iv. save
        v1_vec[i] = -result.fun
        c1_vec[i] = result.x

    return m1_vec,v1_vec,c1_vec
```

**Question A:** Find optimal consumption in the first period as funcition of cash-on-hand, and plot it.

```python
rho = 8
kappa = 0.5
nu = 0.1
r = 0.04
beta = 0.94
Delta = 0.5

# b. solve
# write your code here

# c. plot
# write your code here
```

**Answer:**

```python
# b. solve
def solve(rho,beta,r,Delta,nu,kappa,v1):

    # a. solve period 2
    m2_vec,v2_vec,c2_vec = solve_period_2(rho,nu,kappa,Delta)

    # b. construct interpolator
    v2_interp = interpolate.RegularGridInterpolator((m2_vec,), v2_vec,
                                                    bounds_error=False,fill_value=None)

    # b. solve period 1
    m1_vec,v1_vec,c1_vec = solve_period_1(rho,beta,r,Delta,v1,v2_interp)

    return m1_vec,c1_vec

m1_vec,c1_vec = solve(rho,beta,r,Delta,nu,kappa,v1)

# c. plot
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(m1_vec,c1_vec)
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('consumption function in period 1')
ax.set_xlim([0,4])
ax.set_ylim([0,2.5]);
```

**Question B:** Find optimal consumption in the first period as funcition of cash-on-hand, and plot it, assuming that

$$
y_{2} = \begin{cases}
1-\sqrt{\Delta} & \text{with prob. }0.1\\
1-\Delta & \text{with prob. }0.4\\
1+\Delta & \text{with prob. }0.4\\
1+\sqrt{\Delta} & \text{with prob. }0.1
\end{cases}
$$

which add some low probability tail events, but does not change mean income. Give an interpretation of the change in the consumption function.

```python
# write your code here
```

**Answer:**

```python
def v1_alt(c1,m1,rho,beta,r,Delta,v2_interp):

    # a. expected v2 value
    Ra = (1+r)*(m1-c1)
    v2 = 0
    y2s = [1-np.sqrt(Delta),1-Delta,1+Delta,1+np.sqrt(Delta)]
    probs = [0.1,0.4,0.4,0.1]
    for y2,prob in zip(y2s,probs):
        m2 = Ra + y2
        v2 += prob*v2_interp([m2])[0]

    # b. total value
    return utility(c1,rho) + beta*v2

m1_vec_alt,c1_vec_alt = solve(rho,beta,r,Delta,nu,kappa,v1_alt)

# plot
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(m1_vec,c1_vec,label='original')
ax.plot(m1_vec_alt,c1_vec_alt,label='new')
ax.legend(loc='upper left')
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('consumption function in periode 1')
ax.set_xlim([0,4])
ax.set_ylim([0,2.5]);
```

# Problem: Solve the consumer problem with income risk II

Define the following **variables** and **parameters**:

* $m_t$ is cash-on-hand in period $t$
* $c_t$ is non-durable consumption in period $t$
* $d_t$ is durable consumption in period $t$ (only adjustable in period 1)
* $y_t$ is income in period $t$
* $\Delta \in (0,1)$ is income risk
* $r$ is the interest rate
* $\beta > 0$, $\rho > 1$, $\alpha \in (0,1)$, $\nu > 0 $, $\kappa > 0$, $\xi > 0$ are utility parameters

In the **second period** the household solves:

$$
\begin{aligned}
v_{2}(m_{2},d_{2}) &= \max_{c_{2}}\frac{c_{2}^{1-\rho}}{1-\rho}+\alpha\frac{d_{2}^{1-\rho}}{1-\rho}+\nu\frac{(m_{2}+d_{2}-c_{2}+\kappa)^{1-\rho}}{1-\rho} \\
\text{s.t.} \\
c_{2} & \in [0,m_{2}]
\end{aligned}
$$

In the **first period** the household solves:

$$
\begin{aligned}
v_{1}(m_{1}) &= \max_{c_{1},d_{1}}\frac{c_{1}^{1-\rho}}{1-\rho}+\alpha\frac{d_{1}^{1-\rho}}{1-\rho}+\beta\mathbb{E}_{1}\left[v_2(m_2,d_2)\right]\\&\text{s.t.}&\\
m_2 &= (1+r)(m_{1}-c_{1}-d_{1})+y_{2} \\
y_{2} &= \begin{cases}
1-\Delta & \text{with prob. }0.5\\
1+\Delta & \text{with prob. }0.5
\end{cases}\\
c_{1}+d_{1} & \in [0,m_{1}]\\
\end{aligned}
$$

Choose **parameters**:

```python
rho = 2
alpha = 0.1
kappa = 0.5
nu = 0.1
r = 0.04
beta = 0.94
Delta = 0.5

# b. solve
# write your code here

# c. plot
# write your code here
```

The **basic functions** are:

```python
def utility(c,d,alpha,rho):
    return c**(1-rho)/(1-rho) + alpha*d**(1-rho)/(1-rho)

def bequest(m,c,d,nu,kappa,rho):
    return nu*(m+d-c+kappa)**(1-rho)/(1-rho)

def v2(c2,d2,m2,alpha,rho,nu,kappa):
    return utility(c2,d2,alpha,rho) + bequest(m2,c2,d2,nu,kappa,rho)

def v1(c1,d1,m1,alpha,rho,beta,r,Delta,v2_interp):

    # a. v2 value, if low income
    m2_low = (1+r)*(m1-c1-d1) + 1-Delta
    v2_low = v2_interp([m2_low,d1])[0]

    # b. v2 value, if high income
    m2_high = (1+r)*(m1-c1-d1) + 1+Delta
    v2_high = v2_interp([m2_high,d1])[0]

    # c. expected v2 value
    v2 = 0.5*v2_low + 0.5*v2_high

    # d. total value
    return utility(c1,d1,alpha,rho) + beta*v2
```

The **solution function for period 2** is:

```python
def solve_period_2(alpha,rho,nu,kappa,Delta):

    # a. grids
    m2_vec = np.linspace(1e-8,5,200)
    d2_vec = np.linspace(1e-8,5,200)
    v2_grid = np.empty((200,200))
    c2_grid = np.empty((200,200))

    # b. solve for each m2 in grid
    for i,m2 in enumerate(m2_vec):
        for j,d2 in enumerate(d2_vec):

            # i. objective
            obj = lambda c2: -v2(c2,d2,m2,alpha,rho,nu,kappa)

            # ii. initial value (consume half)
            x0 = m2/2

            # iii. optimizer
            result = optimize.minimize_scalar(obj,x0,method='bounded',bounds=[1e-8,m2])

            # iv. save
            v2_grid[i,j] = -result.fun
            c2_grid[i,j] = result.x

    return m2_vec,d2_vec,v2_grid,c2_grid
```

**Question A:** Solve for consumption in period 2 and plot the consumption function.

```python
# write your code here
```

**Answer:**

```python
# a. solve
m2_vec,d2_vec,v2_grid,c2_grid = solve_period_2(alpha,rho,nu,kappa,Delta)

# b. grids
m2_grid,d2_grid = np.meshgrid(m2_vec,d2_vec,indexing='ij')

# c. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.plot_surface(m2_grid,d2_grid,c2_grid,cmap=cm.jet)

# d. add labels
ax.set_xlabel('$m_2$')
ax.set_ylabel('$d_2$')
ax.set_zlabel('$c_2$')

# e. invert xaxis
ax.invert_xaxis()

# f. add colorbar
fig.colorbar(cs);
```

**Question B:** Find optimal consumption and choices of durables in the first period as a function of cash-on-hand and plot it.

```python
# write your code here
```

**Answer:**

```python
# a. define solve function
def solve_period_1(alpha,rho,beta,r,Delta,v1,v2_interp):

    # a. grids
    m1_vec = np.linspace(1e-4,4,100)
    v1_vec = np.empty(100)
    c1_vec = np.empty(100)
    d1_vec = np.empty(100)

    # b. solve for each m1 in grid
    for i,m1 in enumerate(m1_vec):

        # i. objective
        obj = lambda x: -v1(x[0],x[1],m1,alpha,rho,beta,r,Delta,v2_interp)

        # ii. initial guess
        x0 = [m1*1/3,m1*1/3]

        # iii. bounds and constraitns
        bound = (1e-8,m1-1e-8)
        bounds = (bound, bound)
        ineq_con = {'type': 'ineq', 'fun': lambda x: m1-x[0]-x[1]}

        # iv. optimize
        result = optimize.minimize(obj,x0, method='SLSQP',
                                   bounds=bounds,
                                   constraints=[ineq_con])

        #result = optimize.minimize(obj,x0, method='Nelder-Mead')

        # v. save
        v1_vec[i] = -result.fun
        c1_vec[i] = result.x[0]
        d1_vec[i] = result.x[1]

    return m1_vec,v1_vec,c1_vec,d1_vec

# b. construct interpolator
v2_interp = interpolate.RegularGridInterpolator((m2_vec,d2_vec), v2_grid,
                                                bounds_error=False,fill_value=None)

# c. solve period 1
m1_vec,v1_vec,c1_vec,d1_vec = solve_period_1(alpha,rho,beta,r,Delta,v1,v2_interp)

# d. plot
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(m1_vec,c1_vec,label='non-durable consumption')
ax.plot(m1_vec_alt,d1_vec,label='durable consumption')
ax.legend(loc='upper left')
ax.set_xlabel('$m_1$')
ax.set_xlim([0,4])
ax.set_ylim([0,2.5]);
```


# Task for lecture 20

2. Regarding the distribution problem for the elbow muscles presented in this text:
    a. Test different initial values for the optimization.
    b. Test other values for the elbow angle where the results are likely to change.

3. In an experiment to estimate forces of the elbow flexors, through inverse dynamics it was found an elbow flexor moment of 10 Nm.
Consider the following data for maximum force (F0), moment arm (r), and pcsa (A) of the brachialis, brachioradialis, and biceps brachii muscles: F0 (N): 1000, 250, 700; r (cm): 2, 5, 4; A (cm$^2$): 33, 8, 23, respectively (data from Robertson et al. (2013)).
    a. Use static optimization to estimate the muscle forces.
    b. Test the robustness of the results using different initial values for the muscle forces.
    c. Compare the results for different cost functions.

```python
# import Python libraries
import numpy as np
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import sympy as sym
from sympy.plotting import plot
import pandas as pd
from IPython.display import display
from IPython.core.display import Math
```

### Muscle force estimation

Consider the following main flexors of the elbow joint (see figure below): biceps long head, biceps short head, and brachialis. Suppose that the elbow net joint torque determined using inverse dynamics is 20 Nm (flexor). How much each of these muscles contributed to the net torque?

Figure. A view in OpenSim of the arm26 model showing three elbow flexors (Biceps long and short heads and Brachialis).

For the optimization, we will need experimental data for the moment arm, maximum moment, and *pcsa* of each muscle. Let's import these data from the OpenSim arm26 model:

```python
# time elbow_flexion BIClong BICshort BRA
r_ef = np.loadtxt('./../data/r_elbowflexors.mot', skiprows=7)
f_ef = np.loadtxt('./../data/f_elbowflexors.mot', skiprows=7)
```

The maximum isometric force of these muscles are defined in the arm26 model as: Biceps long head: 624.3 N, Biceps short head: 435.56 N, and Brachialis: 987.26 N. Let's compute the mamimum torques that each muscle could produce considering a static situation at the different elbow flexion angles:

```python
m_ef = r_ef*1
m_ef[:, 2:] = r_ef[:, 2:]*f_ef[:, 2:]
```

These data don't have the *pcsa* value of each muscle. We will estimate the *pcsa* considering that the amount of maximum muscle force generated per area is constant and equal to 50N/cm$^2$. Consequently, the *pcsa* (in cm$^2$) for each muscle is:

```python
a_ef = np.array([624.3, 435.56, 987.26])/50  # 50 N/cm2
print(a_ef)
```

    [ 12.486    8.7112  19.7452]

### Solution of the optimization problem

For the present case, we are dealing with a problem of minimization, multidimensional (function of several variables), nonlinear, constrained, and we can't assume that the cost function is convex. Numerical optimization is hardly a simple task. There are many different algorithms and public and commercial software for performing optimization. For instance, look at [NEOS Server](http://www.neos-server.org/neos/), a free internet-based service for solving numerical optimization problems.
We will solve the present problem using the [scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html#module-scipy.optimize) package which provides several optimization algorithms. We will use the function `minimize`:

```python
scipy.optimize.minimize(fun, x0, args=(), method=None, jac=None, hess=None, hessp=None, bounds=None, constraints=(), tol=None, callback=None, options=None)
"""Minimization of scalar function of one or more variables."""
```

Now, let's write Python functions for each cost function:

```python
from scipy.optimize import minimize
```

```python
def cf_f1(x):
    """Cost function: sum of forces."""
    return x[0] + x[1] + x[2]

def cf_f2(x):
    """Cost function: sum of forces squared."""
    return x[0]**2 + x[1]**2 + x[2]**2

def cf_fpcsa2(x, a):
    """Cost function: sum of squared muscle stresses."""
    return (x[0]/a[0])**2 + (x[1]/a[1])**2 + (x[2]/a[2])**2

def cf_fmmax3(x, m):
    """Cost function: sum of cubic forces normalized by moments."""
    return (x[0]/m[0])**3 + (x[1]/m[1])**3 + (x[2]/m[2])**3
```

Let's also define the Jacobian for each cost function (which is an optional parameter for the optimization):

```python
def cf_f1d(x):
    """Derivative of cost function: sum of forces."""
    dfdx0 = 1
    dfdx1 = 1
    dfdx2 = 1
    return np.array([dfdx0, dfdx1, dfdx2])

def cf_f2d(x):
    """Derivative of cost function: sum of forces squared."""
    dfdx0 = 2*x[0]
    dfdx1 = 2*x[1]
    dfdx2 = 2*x[2]
    return np.array([dfdx0, dfdx1, dfdx2])

def cf_fpcsa2d(x, a):
    """Derivative of cost function: sum of squared muscle stresses."""
    dfdx0 = 2*x[0]/a[0]**2
    dfdx1 = 2*x[1]/a[1]**2
    dfdx2 = 2*x[2]/a[2]**2
    return np.array([dfdx0, dfdx1, dfdx2])

def cf_fmmax3d(x, m):
    """Derivative of cost function: sum of cubic forces normalized by moments."""
    dfdx0 = 3*x[0]**2/m[0]**3
    dfdx1 = 3*x[1]**2/m[1]**3
    dfdx2 = 3*x[2]**2/m[2]**3
    return np.array([dfdx0, dfdx1, dfdx2])
```

Perform the optimization using the previous initial condition looked in class

2. Regarding the distribution problem for the elbow muscles presented in this text:
    a. Test different initial values for the optimization.
    b. Test other values for the elbow angle where the results are likely to change.

```python
M = 20  # desired torque at the elbow
iang = 69  # which will give the closest value to 90 degrees
#iang = 34
r  = r_ef[iang, 2:]
f0 = f_ef[iang, 2:]
a  = a_ef
m  = m_ef[iang, 2:]
x0 = f_ef[iang, 2:]/10  # far from the correct answer for the sum of torques
print('M =', M)
print('x0 =', x0)
print('r * x0 =', np.sum(r*x0))
```

Inequality constraints (such as boundaries in our problem) can be entered with the parameter `bounds` to the `minimize` function:

```python
bnds = ((0, f0[0]), (0, f0[1]), (0, f0[2]))
```

Equality constraints (such as the sum of torques should equals the desired torque in our problem), as well as inequality constraints, can be entered with the parameter `constraints` to the `minimize` function (and we can also opt to enter the Jacobian of these constraints):

```python
# use this in combination with the parameter bounds:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)})
```

```python
# to enter everything as constraints:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[0]-x[0],
         'jac' : lambda x, r, f0, M: np.array([-1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[1]-x[1],
         'jac' : lambda x, r, f0, M: np.array([0, -1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[2]-x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, -1]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[0],
         'jac' : lambda x, r, f0, M: np.array([1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[1],
         'jac' : lambda x, r, f0, M: np.array([0, 1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, 1]), 'args': (r, f0, M)})
```

Although more verbose, if all the Jacobians of the constraints are also informed, this alternative seems better than informing bounds for the optimization process (less error in the final result and less iterations).

Given the characteristics of the problem, if we use the function `minimize` we are limited to the SLSQP (Sequential Least SQuares Programming) solver.

Finally, let's run the optimization for the four different cost functions and find the optimal muscle forces:

```python
f1r = minimize(fun=cf_f1, x0=x0, args=(), jac=cf_f1d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
f2r = minimize(fun=cf_f2, x0=x0, args=(), jac=cf_f2d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
fpcsa2r = minimize(fun=cf_fpcsa2, x0=x0, args=(a,), jac=cf_fpcsa2d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

```python
fmmax3r = minimize(fun=cf_fmmax3, x0=x0, args=(m,), jac=cf_fmmax3d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

```python
# Storing the results of the first optimization using previous initial conditions
opt1 = np.around(np.vstack((f1r.x, f2r.x, fpcsa2r.x, fmmax3r.x)), 1)
```

```python
dat = np.vstack((np.around(r*100,1), np.around(a,1), np.around(f0,0), np.around(m,1)))
er = ['-', '-', '-', '-',
      np.sum(r*f1r.x)-M, np.sum(r*f2r.x)-M, np.sum(r*fpcsa2r.x)-M, np.sum(r*fmmax3r.x)-M]
data = np.vstack((np.vstack((dat, opt1)).T, er)).T

rows = ['$\text{Moment arm}\;[cm]$', '$pcsa\;[cm^2]$', '$F_{max}\;[N]$', '$M_{max}\;[Nm]$',
        '$\sum F_i$', '$\sum F_i^2$', '$\sum(F_i/pcsa_i)^2$', '$\sum(F_i/M_{max,i})^3$']
cols = ['Biceps long head', 'Biceps short head', 'Brachialis', 'Error in M']
df = pd.DataFrame(data, index=rows, columns=cols)
print('\Solving the distribution problem for a desired torque of 20Nm and at 90 elbow flexion')
df
```

## 3. Regarding the distribution problem for the elbow muscles presented in this text:
    a. Test different initial values for the optimization.
    b. Test other values for the elbow angle where the results are likely to change.
Set different initial conditions such as consider a desired torque at the elbow of 40 Nm and a slightly higher initial sum of torques.

```python
M = 40  # desired torque at the elbow
iang = 69  # which will give the closest value to 90 degrees
r1  = r_ef[iang, 2:]
f01 = f_ef[iang, 2:]
a1  = a_ef
m1  = m_ef[iang, 2:]
x01 = f_ef[iang, 2:]/5  # far from the correct answer for the sum of torques
print('M =', M)
print('x0 =', x0)
print('r * x0 =', np.sum(r*x0))
```

Inequality constraints (such as boundaries in our problem) can be entered with the parameter `bounds` to the `minimize` function:

```python
bnds = ((0, f0[0]), (0, f0[1]), (0, f0[2]))
```

Equality constraints (such as the sum of torques should equals the desired torque in our problem), as well as inequality constraints, can be entered with the parameter `constraints` to the `minimize` function (and we can also opt to enter the Jacobian of these constraints):

```python
# use this in combination with the parameter bounds:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)})
```

```python
# to enter everything as constraints:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[0]-x[0],
         'jac' : lambda x, r, f0, M: np.array([-1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[1]-x[1],
         'jac' : lambda x, r, f0, M: np.array([0, -1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[2]-x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, -1]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[0],
         'jac' : lambda x, r, f0, M: np.array([1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[1],
         'jac' : lambda x, r, f0, M: np.array([0, 1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, 1]), 'args': (r, f0, M)})
```

Although more verbose, if all the Jacobians of the constraints are also informed, this alternative seems better than informing bounds for the optimization process (less error in the final result and less iterations).

Given the characteristics of the problem, if we use the function `minimize` we are limited to the SLSQP (Sequential Least SQuares Programming) solver.

Finally, let's run the optimization for the four different cost functions and find the optimal muscle forces:

```python
f1r = minimize(fun=cf_f1, x0=x0, args=(), jac=cf_f1d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
f2r = minimize(fun=cf_f2, x0=x0, args=(), jac=cf_f2d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
fpcsa2r = minimize(fun=cf_fpcsa2, x0=x0, args=(a,), jac=cf_fpcsa2d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

```python
fmmax3r = minimize(fun=cf_fmmax3, x0=x0, args=(m,), jac=cf_fmmax3d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

```python
opt2 = np.around(np.vstack((f1r.x, f2r.x, fpcsa2r.x, fmmax3r.x)), 1)
```

Displaying the results for the **second** optimization using different initial conditions basically changing the desired torque (twice as the previous) and initial sum of torques.

```python
dat = np.vstack((np.around(r*100,1), np.around(a,1), np.around(f0,0), np.around(m,1)))
er = ['-', '-', '-', '-',
      np.sum(r*f1r.x)-M, np.sum(r*f2r.x)-M, np.sum(r*fpcsa2r.x)-M, np.sum(r*fmmax3r.x)-M]
data = np.vstack((np.vstack((dat, opt2)).T, er)).T

rows = ['$\text{Moment arm}\;[cm]$', '$pcsa\;[cm^2]$', '$F_{max}\;[N]$', '$M_{max}\;[Nm]$',
        '$\sum F_i$', '$\sum F_i^2$', '$\sum(F_i/pcsa_i)^2$', '$\sum(F_i/M_{max,i})^3$']
cols = ['Biceps long head', 'Biceps short head', 'Brachialis', 'Error in M']
df = pd.DataFrame(data, index=rows, columns=cols)
print('\nSolving the distribution problem for a desired torque of 40Nm')
df
```

## 3. Regarding the distribution problem for the elbow muscles presented in this text:
    b. Test other values for the elbow angle where the results are likely to change.

Coinsider and angle of 45 degrees.

```python
M = 20  # desired torque at the elbow
iang = 34  # which will give the closest value to 45 degrees
#iang = 34
r  = r_ef[iang, 2:]
f0 = f_ef[iang, 2:]
a  = a_ef
m  = m_ef[iang, 2:]
x0 = f_ef[iang, 2:]/10  # far from the correct answer for the sum of torques
print('M =', M)
print('x0 =', x0)
print('r * x0 =', np.sum(r*x0))
```

Inequality constraints (such as boundaries in our problem) can be entered with the parameter `bounds` to the `minimize` function:

```python
bnds = ((0, f0[0]), (0, f0[1]), (0, f0[2]))
```

Equality constraints (such as the sum of torques should equals the desired torque in our problem), as well as inequality constraints, can be entered with the parameter `constraints` to the `minimize` function (and we can also opt to enter the Jacobian of these constraints):

```python
# use this in combination with the parameter bounds:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)})
```

```python
# to enter everything as constraints:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[0]-x[0],
         'jac' : lambda x, r, f0, M: np.array([-1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[1]-x[1],
         'jac' : lambda x, r, f0, M: np.array([0, -1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[2]-x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, -1]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[0],
         'jac' : lambda x, r, f0, M: np.array([1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[1],
         'jac' : lambda x, r, f0, M: np.array([0, 1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, 1]), 'args': (r, f0, M)})
```

Although more verbose, if all the Jacobians of the constraints are also informed, this alternative seems better than informing bounds for the optimization process (less error in the final result and less iterations).

Given the characteristics of the problem, if we use the function `minimize` we are limited to the SLSQP (Sequential Least SQuares Programming) solver.

Finally, let's run the optimization for the four different cost functions and find the optimal muscle forces:

```python
f1r = minimize(fun=cf_f1, x0=x0, args=(), jac=cf_f1d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
f2r = minimize(fun=cf_f2, x0=x0, args=(), jac=cf_f2d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
fpcsa2r = minimize(fun=cf_fpcsa2, x0=x0, args=(a,), jac=cf_fpcsa2d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

```python
fmmax3r = minimize(fun=cf_fmmax3, x0=x0, args=(m,), jac=cf_fmmax3d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

```python
opt3 = np.around(np.vstack((f1r.x, f2r.x, fpcsa2r.x, fmmax3r.x)), 1)
```

**Results of the optimization problem considering a desired torque of 20Nm and 45 degrees elbow flexion**

```python
dat = np.vstack((np.around(r*100,1), np.around(a,1), np.around(f0,0), np.around(m,1)))
er = ['-', '-', '-', '-',
      np.sum(r*f1r.x)-M, np.sum(r*f2r.x)-M, np.sum(r*fpcsa2r.x)-M, np.sum(r*fmmax3r.x)-M]
data = np.vstack((np.vstack((dat, opt3)).T, er)).T

rows = ['$\text{Moment arm}\;[cm]$', '$pcsa\;[cm^2]$', '$F_{max}\;[N]$', '$M_{max}\;[Nm]$',
        '$\sum F_i$', '$\sum F_i^2$', '$\sum(F_i/pcsa_i)^2$', '$\sum(F_i/M_{max,i})^3$']
cols = ['Biceps long head', 'Biceps short head', 'Brachialis', 'Error in M']
df = pd.DataFrame(data, index=rows, columns=cols)
print('\nSolving the distribution problem for a desired torque of 20Nm and at 45 elbow flexion')
df
```

4. In an experiment to estimate forces of the elbow flexors, through inverse dynamics it was found an elbow flexor moment of 10 Nm.
Consider the following data for maximum force (F0), moment arm (r), and pcsa (A) of the brachialis, brachioradialis, and biceps brachii muscles: F0 (N): 1000, 250, 700; r (cm): 2, 5, 4; A (cm$^2$): 33, 8, 23, respectively (data from Robertson et al. (2013)).
    a. Use static optimization to estimate the muscle forces.
    b. Test the robustness of the results using different initial values for the muscle forces.
    c. Compare the results for different cost functions.

```python
M = 10  # desired torque at the elbow
iang = 34  # which will give the closest value to 45 degrees
r  = np.array([0.02, 0.05, 0.04])
f0 = np.array([1000, 250, 700])
a  = np.array([33, 8, 23])
m  = r*f0
x0 = f0/20  # far from the correct answer for the sum of torques
print('M =', M)
print('x0 =', x0)
print('r * x0 =', np.sum(r*x0))
```

Inequality constraints (such as boundaries in our problem) can be entered with the parameter `bounds` to the `minimize` function:

```python
bnds = ((0, f0[0]), (0, f0[1]), (0, f0[2]))
```

Equality constraints (such as the sum of torques should equals the desired torque in our problem), as well as inequality constraints, can be entered with the parameter `constraints` to the `minimize` function (and we can also opt to enter the Jacobian of these constraints):

```python
# use this in combination with the parameter bounds:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)})
```

```python
# to enter everything as constraints:
cons = ({'type': 'eq',
         'fun' : lambda x, r, f0, M: np.array([r[0]*x[0] + r[1]*x[1] + r[2]*x[2] - M]),
         'jac' : lambda x, r, f0, M: np.array([r[0], r[1], r[2]]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[0]-x[0],
         'jac' : lambda x, r, f0, M: np.array([-1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[1]-x[1],
         'jac' : lambda x, r, f0, M: np.array([0, -1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: f0[2]-x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, -1]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[0],
         'jac' : lambda x, r, f0, M: np.array([1, 0, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[1],
         'jac' : lambda x, r, f0, M: np.array([0, 1, 0]), 'args': (r, f0, M)},
        {'type': 'ineq', 'fun' : lambda x, r, f0, M: x[2],
         'jac' : lambda x, r, f0, M: np.array([0, 0, 1]), 'args': (r, f0, M)})
```

Although more verbose, if all the Jacobians of the constraints are also informed, this alternative seems better than informing bounds for the optimization process (less error in the final result and less iterations).

Given the characteristics of the problem, if we use the function `minimize` we are limited to the SLSQP (Sequential Least SQuares Programming) solver.

Finally, let's run the optimization for the four different cost functions and find the optimal muscle forces:

```python
f1r = minimize(fun=cf_f1, x0=x0, args=(), jac=cf_f1d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
f2r = minimize(fun=cf_f2, x0=x0, args=(), jac=cf_f2d,
               constraints=cons, method='SLSQP',
               options={'disp': True})
```

```python
fpcsa2r = minimize(fun=cf_fpcsa2, x0=x0, args=(a,), jac=cf_fpcsa2d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

```python
fmmax3r = minimize(fun=cf_fmmax3, x0=x0, args=(m,), jac=cf_fmmax3d,
                   constraints=cons, method='SLSQP',
                   options={'disp': True})
```

## Comments

The results show that the estimations for the muscle forces depend on the cost function used in the optimization. Which one is correct? This is a difficult question and it's dependent on the goal of the actual task being modeled. Glitsch and Baumann (1997) investigated the effect of different cost functions on the optimization of walking and running and the predicted muscles forces were compared with the electromyographic activity of the corresponding muscles of the lower limb. They found that, among the analyzed cost functions, the minimization of the sum of squared muscle stresses resulted in the best similarity with the actual electromyographic activity.

In general, one should always test different algorithms and different initial values before settling for the solution found. Downey (2011), Kitchin (2013), and Kiusalaas (2013) present more examples on numerical optimization. The [NEOS Guide](http://neos-guide.org/) is a valuable source of information on this topic and [OpenOpt](http://openopt.org/) is a good alternative software for numerical optimization in Python.

## Exercises

1. Find the extrema in the function $f(x)=x^3-7.5x^2+18x-10$ analytically and determine if they are minimum or maximum.
2. Find the minimum in the $f(x)=x^3-7.5x^2+18x-10$ using the gradient descent algorithm.
2. Regarding the distribution problem for the elbow muscles presented in this text:
    a. Test different initial values for the optimization.
    b. Test other values for the elbow angle where the results are likely to change.

3. In an experiment to estimate forces of the elbow flexors, through inverse dynamics it was found an elbow flexor moment of 10 Nm.
Consider the following data for maximum force (F0), moment arm (r), and pcsa (A) of the brachialis, brachioradialis, and biceps brachii muscles: F0 (N): 1000, 250, 700; r (cm): 2, 5, 4; A (cm$^2$): 33, 8, 23, respectively (data from Robertson et al. (2013)).
    a. Use static optimization to estimate the muscle forces.
    b. Test the robustness of the results using different initial values for the muscle forces.
    c. Compare the results for different cost functions.

## References

- Bergstra B, Bengio Y (2012) [Random Search for Hyper-Parameter Optimization](http://jmlr.csail.mit.edu/papers/volume13/bergstra12a/bergstra12a.pdf). Journal of Machine Learning Research, 13, 281-305.
- Crowninshield RD, Brand RA (1981) [A physiologically based criterion of muscle force prediction in locomotion](http://www.ncbi.nlm.nih.gov/pubmed/7334039). Journal of Biomechanics, 14, 793–801.
- Downey AB (2014) [Physical Modeling in MATLAB](http://greenteapress.com/wp/physical-modeling-in-matlab-2e/). 2nd edition. Green Tea Press.
- Herzog W (1987) [Individual muscle force estimations using a non-linear optimal design](http://www.ncbi.nlm.nih.gov/pubmed/3682873). J Neurosci Methods, 21, 167-179.
- Glitsch U, Baumann W (1997) [The three-dimensional determination of internal loads in the lower extremity](http://www.ncbi.nlm.nih.gov/pubmed/9456380). Journal of Biomechanics, 30, 1123–1131.
- Kitchin J (2013) [pycse - Python Computations in Science and Engineering](http://kitchingroup.cheme.cmu.edu/pycse/).
- Kiusalaas (2013) [Numerical methods in engineering with Python 3](http://books.google.com.br/books?id=aJkXoxxoCoUC). 3rd edition. Cambridge University Press.
- Nigg BM and Herzog W (2006) [Biomechanics of the Musculo-skeletal System](https://books.google.com.br/books?id=hOIeAQAAIAAJ&dq=editions:ISBN0470017678). 3rd Edition. Wiley.
- Robertson G, Caldwell G, Hamill J, Kamen G (2013) [Research Methods in Biomechanics](http://books.google.com.br/books?id=gRn8AAAAQBAJ). 2nd Edition. Human Kinetics.
- Seireg A, Arvikar RJ (1973) [A mathematical model for evaluation of forces in lower extremeties of the musculo-skeletal system](http://www.ncbi.nlm.nih.gov/pubmed/4706941). Journal of Biomechanics, 6,  313–322, IN19–IN20, 323–326.

notebooks/.ipynb_checkpoints/Task for lecture 20-checkpoint.ipynb


# RSI Trading Strategy

Build a trading algorithm using RSI (Relative Strength Index) indicator
RSI indicator evaluates overbought or oversold conditions of a stock
Serves as a baseline before adding deep learning models to try to improve the strategy's performance

Run strategy on intraday time scale and RSI of 5 periods

Imports

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.ticker as mtick
import tensorflow as tf
```

Load intraday data from intraday_data.py

```python
df = pd.read_csv('FB_raw.csv')
df['date'] = pd.to_datetime(df['date']) # Format for datetime object
df['close'].fillna(method='ffill', inplace=True)

locator = mdates.MonthLocator() # Locate months for plotting

plt.title('FB stock price')
plt.plot(df['date'], df['close'])
plt.xticks(rotation=45)
x = plt.gca()
y = plt.gca()
x.xaxis.set_major_locator(locator)
x.xaxis.set_major_formatter(mdates.DateFormatter('%b %y'))
y.yaxis.set_major_formatter(mtick.StrMethodFormatter('${x:,.0f}'))
plt.show()
```

# Calculating RSI
## RSI formula
\begin{equation}
RSI = {100 - (\frac{100}{1+rs})}
\end{equation}
\begin{equation}
rs = {\frac{\text{smoothen average n*period up}}{\text{smoothen average n*period down}}}
\end{equation}

```python
period = 5 # 5 period RSI
close = df['close']
delta = close.diff()

up, down = delta.copy(), delta.copy()
up[up < 0] = 0 # Set 'up' value that are less than 0 to 0
down[down > 0] = 0 # Set 'down' value that are more than 0 to 0

# calculate the exponential weighted moving average
rolling_gain = up.ewm(span=period).mean()
rolling_loss = down.abs().ewm(span=period).mean()

rs = rolling_gain/rolling_loss
rsi = 100 - (100 / (1+rs))

df['rsi'] = rsi
```

# Conditions for RSI strategy
Buy when RSI is below 30, stock is oversold
Sell when RSI is above 70, stock is overbought

## Two conditions running simultaneously
* Long condtion with exit
* Short condition with exit

Vectorised to speed up operation
Similar to looping through conditions with each time step

```python
cond1 = (rsi <= 30) & (rsi.shift(1) > 30)
cond2 = (cond1 == True) & (rsi <= 70)
cond3 = (cond2 == True) & (cond2.shift(1) == False)
cond4 = (cond2 == False) & (cond2.shift(1) == True)

cond5 = (rsi >= 70) & (rsi.shift(1) < 70)
cond6 = (cond5 == True) & (rsi >= 30)
cond7 = (cond6 == True) & (cond6.shift(1) == False)
cond8 = (cond6 == False) & (cond6.shift(1) == True)

enterlong = close[cond3]
exitshort = close[cond4]

entershort = close[cond7]
exitlong = close[cond8]
```

Storing calculated values into dataframe

```python
df['long'] = enterlong
df['exit_long'] = exitlong

df['short'] = entershort
df['exit_short'] = exitshort
```

Calculating profit and loss of RSI strategy

```python
df['exit_long'].fillna(method='bfill', inplace=True)
long_pnl = df['exit_long'].shift(-1) - df['long']
df['long_pnl'] = long_pnl

df['exit_short'].fillna(method='bfill', inplace=True)
short_pnl = df['short'] - df['exit_short'].shift(-1)
df['short_pnl'] = short_pnl

df['total_pnl'] = df[['short_pnl', 'long_pnl']].sum(axis=1)
```

Trading strategy made 31221 trades from April 2019 to Decemember 2020

Both long and short conditions made around the same number of trades

```python
print(f"Total number of long trades: {df['long_pnl'].count()}")
print(f"Total number of short trades: {df['short_pnl'].count()}")
print(f"Total number of trades: {(df['short_pnl'].count() + df['long_pnl'].count())}")
```

    Total number of long trades: 15592
    Total number of short trades: 15629
    Total number of trades: 31221

# Visualising strategy's performance
Comparing holding 1 stock against RSI strategy of buy/sell 1 stock when condition is met, over the same period

* Note that slippage and trading costs are not accounted for in this current strategy

```python
performance_hold = df['close'].iloc[-1] - df['close'].iloc[1]
performance_long = df['long_pnl'].cumsum().fillna(method='ffill')
performance_short = df['short_pnl'].cumsum().fillna(method='ffill')
total_performance = df['total_pnl'].cumsum().fillna(method='ffill')

print(f"Profit from hold strategy: {performance_hold}")
print(f"Profit from short strategy: {(df['short_pnl'].sum())}")
print(f"Profit from long strategy: {df['long_pnl'].sum()}")
print(f"Profit from long & short strategies: {(df['total_pnl'].sum())}")
print(f"Max long draw down: {performance_long.min()}")
print(f"Max short draw down: {performance_short.min()}")
print(f"Max total draw down: {total_performance.min()}")

locator = mdates.MonthLocator()
fmt = mdates.DateFormatter('%b %y')

plt.plot(df['date'], df['close']-df['close'][0], alpha=0.3) #Performance of holding stock in the same time period
plt.plot(df['date'], performance_long, alpha=0.5)
plt.plot(df['date'], performance_short, alpha=0.5, color='red')
plt.plot(df['date'], total_performance, color='green')
x = plt.gca()
y = plt.gca()
x.xaxis.set_major_locator(locator)
x.xaxis.set_major_formatter(fmt)
y.yaxis.set_major_formatter(mtick.StrMethodFormatter('${x:,.0f}'))
plt.xticks(rotation=45)
plt.title('Comparing Performance of Strategies')
plt.legend(['Close', 'Long strat', 'Short strat', 'Total performance'])
plt.show()

```

Labeling each trade performance
* 1 for profitable trades
* 0 for losses

```python
df['long_labels'] = np.where(df['long_pnl'] <= 0, 0,
                        np.where(df['long_pnl'] > 0, 1, np.nan))

df['short_labels'] = np.where(df['short_pnl'] <= 0, 0,
                        np.where(df['short_pnl'] > 0, 1, np.nan))
```

Strategy yielded more winning trades than losing

Both long and short conditions made roughly the same amount of profitable trades against losing trades

```python
print(df['long_labels'].value_counts())
print(df['short_labels'].value_counts())
```

    1.0    9931
    0.0    5661
    Name: long_labels, dtype: int64
    1.0    9690
    0.0    5939
    Name: short_labels, dtype: int64

Export data to csv

```python
df.to_csv('fb_rsi.csv', index=False)
```

# Conclusion
Strategy trades very often and holds trade for a short time frame
Strategy made about twice as much winning trades against losing trades, next step is to try optimising the strategy to reduce losing trade with deep learning models

## Strategy's Limitation:
* Strategy only buys or sells 1 stock during each trade, strategy could be scaled up based on account size
* No stop-loss feature to limit loses in losing trades

## To-do List:
* Add risk management feature, perhaps Kelly criterion to allocate bet size
* Add stop-loss feature, limit amount on losing trades


_Latihan soal ini dari perkuliahan Analisis Sumber Daya Air (SI-6232) tahun 2018 Semester Genap diajarkan oleh Dr. Dhemi Harlan ST, MT, MSc._

# Metoda Calculus

## Calculus Single Variable

```python
from sympy import symbols, latex, simplify, init_printing, solve, solveset, Eq
from sympy import Rational as rat
init_printing()

sx = symbols('x')

fx = (1/4)*sx**4 - (1/2)*sx**2
fxd = fx.diff(sx)
fxdd = fx.diff(sx,sx)

def f(x):
    return float(fx.subs({sx: x}))

def fd(x):
    return float(fxd.subs({sx: x}))

def fdd(x):
    return float(fxdd.subs({sx: x}))
```

```python
import numpy as np
import matplotlib.pyplot as plt

axis = np.linspace(xa,xb,1000)
y, yd, ydd = [], [], []
for x in axis:
    y.append(f(x))
    yd.append(fd(x))
    ydd.append(fdd(x))

plt.rcParams["figure.figsize"] = (10, 12)
plt.plot(axis, y)
plt.plot(axis, yd)
plt.plot(axis, ydd)
plt.grid()

```

```python
result = solve(fxd,sx)

for x in result:
    print(fdd(x), '----', f(x))
```

    2.0 ---- -0.25
    -1.0 ---- 0.0
    2.0 ---- -0.25

```python
ex2f = sx**5
ex2fd = ex2f.diff(sx)
ex2fdd = ex2f.diff(sx, sx)
ex2fddd = ex2f.diff(sx, sx, sx)
ex2fdddd = ex2f.diff(sx, sx, sx, sx)
```

```python
ex3f = 12*sx**5 - 45*sx**4 + 40*sx**3 + 5
ex3f.diff(sx)
```

```python
hasil = solve(ex3f.diff(sx),sx)
print(hasil)
for x in hasil:
    print(ex3f.diff(sx,sx).subs({sx: x}))
```

    [0, 1, 2]
    0
    -60
    240

```python
ex3f.diff(sx, sx, sx).subs({sx: 0})
```

```python
from sympy import factor
factor(ex3f.diff(sx, sx, sx))
```

```python
ex3f.diff(sx, sx, sx).subs({sx: 0})
```

```python

```