# Implementing Discrete Cosine Transform (DCT) Using Python

## Table of Contents
* [Introduction](#Introduction)
* [Python Implementation](#Implementation)
* [Testing the Code](#Testing)

## Introduction

Discrete Cosine Transform (DCT) is one of the methods that transform an image in **space-domain** to its corresponding **frequency-domain**. The transformed image can also be returned back to its original format by using the inverse DCT. Discrete Fourier Transform (DFT) is a complex type of transform. That is, the output of DFT is a matrix of complex numbers having both the real and imaginary parts (cosine and sine). However, we may need to change the image only to its corresponding real-number transformation. So, DCT is the one that computes only the real (cosine) part of the transformation while keeping the similar format as the DFT.

Let the size of an input image be NxN.

The general formula for ***Forward DCT*** is:

$$
\begin{align}
C(u,v) = \tau(u,v)\sum_{x=0}^{N-1}\sum_{y=0}^{N-1}f(x,y) cos(\frac{(2x+1)}{2N}u\pi)cos(\frac{(2y+1)}{2N}v\pi) \; where \; u,v=0,1,2,...N-1
\end{align}
$$

$\hspace{7cm} \tau(u, v) = \frac{1}{N}$ for u=0, v=0

$\hspace{8.25cm}=\frac{2}{N}$ for otherwise

The ***forward DCT kernel*** is:
$$
\begin{align}
g(x,y,u,v) = \tau(u,v)\sum_{x=0}^{N-1}\sum_{y=0}^{N-1} cos(\frac{(2x+1)}{2N}u\pi)cos(\frac{(2y+1)}{2N}v\pi)
\end{align}
$$

Similarly, the ***Inverse DCT*** is:
$$
\begin{align}
f(x,y) = \sum_{u=0}^{N-1}\sum_{v=0}^{N-1}\tau(u,v)C(u,v) cos(\frac{(2x+1)}{2N}u\pi)cos(\frac{(2y+1)}{2N}v\pi) \; where \; x,y=0,1,2,...N-1
\end{align}
$$

In the next section, the forward DCT will be implemented in python. It will be tested with real images. Finally, its running time will be computed and visualized by using multiple images having different images.

## Python Implementation

First of all, let's import the necessary python libraries

```python
import numpy as np

import matplotlib.pyplot as plt
%matplotlib inline

#import matplotlib.image as img
import PIL.Image as Image

import math
import cmath

import time

import csv

from numpy import binary_repr

```

The functions below implement the common image handling

```python
def generateBlackAndWhiteSquareImage(imgSize):
    """
    Generates a square-sized black and white image with a given input size.

    Parameters
    ----------
    imgSize : int
        Input number that stores the dimension of the square image to be generated.

    Returns
    -------
    imge : ndarray
        The generated black and white square image.
    """

    #Creating a matrix with a given size where all the stored values are only zeros (for initialization)
    imge = np.zeros([imgSize, imgSize], dtype=int)

    #Starting and ending indices of the white part of the image.
    ind1 = imgSize/4
    ind2 = ind1 + (imgSize/2)

    #Make a part of the image as white (255)
    imge[ind1:ind2, ind1:ind2] = np.ones([imgSize/2, imgSize/2], dtype=int)*255

    #return the resulting image
    return imge

def generateImagesWithResizedWhite(imge):
    """
    Generates images with the same size as the original but with a resized white part of them.
    """

    N = imge.shape[0]

    imges = []
    i = N/2
    while i >= 4:
        j = (N - i)/2

        #Starting and ending indices for the white part.
        indx1 = j
        indx2 = j+i

        #Draw the image.
        imgeNew = np.zeros([N, N],dtype=int)
        imgeNew[indx1:indx2, indx1:indx2] = np.ones([i, i], dtype=int)*255

        #Add the image to the list.
        imges.append(imgeNew)

        i = i/2

    return imges

def resizeImage(imge, newSize):
    """
    Reduces the size of the given image.

    Parameters
    ----------
    imge : ndarray
        Input array that stores the image to be resized.

    Returns
    -------
    newSize : int
        The size of the newly generated image.
    """

    #Compute the size of the original image (in this case, only # of rows as it is square)
    N = imge.shape[0]

    #The ratio of the original image as compared to the new one.
    stepSize = N/newSize

    #Creating a new matrix (image) with a black color (values of zero)
    newImge = np.zeros([N/stepSize, N/stepSize])

    #Average the adjacent four pixel values to compute the new intensity value for the new image.
    for i in xrange(0, N, stepSize):
        for j in xrange(0, N, stepSize):
            newImge[i/stepSize, j/stepSize] = np.mean(imge[i:i+stepSize, j:j+stepSize])

    #Return the new image
    return newImge

def generateImages(imgSizes=[128, 64, 32, 16, 8]):
    """
    Generates images with different sizes
    """
    #Create an empty list of images to save the generated images with different sizes.
    images = []

    #Generate the first and biggest image
    imge = generateBlackAndWhiteSquareImage(imgSizes[0])

    #Add to the images list
    images.append(imge)

    #Generate the resized and smaller images with different sizes.
    for i in range(1, len(imgSizes)):
        size = imgSizes[i]
        images.append(resizeImage(imge, size))

    return images
```

The python class below implements the DCT transformation algorithm.

```python
class DCT(object):
    """
    This class DCT implements all the procedures for transforming a given 2D digital image
    into its corresponding frequency-domain image (Forward DCT Transform)
    """

    @classmethod
    def __computeSinglePoint2DCT(self, imge, u, v, N):
        """
        A private method that computes a single value of the 2D-DCT from a given image.

        Parameters
        ----------
        imge : ndarray
            The input image.

        u : ndarray
            The index in x-dimension.

        v : ndarray
            The index in y-dimension.

        N : int
            Size of the image.

        Returns
        -------
        result : float
            The computed single value of the DCT.
        """
        result = 0

        for x in xrange(N):
            for y in xrange(N):
                result += imge[x, y] * math.cos(((2*x + 1)*u*math.pi)/(2*N)) * math.cos(((2*y + 1)*v*math.pi)/(2*N))

        #Add the tau value to the result
        if (u==0) and (v==0):
            result = result/N
        elif (u==0) or (v==0):
            result = (math.sqrt(2.0)*result)/N
        else:
            result = (2.0*result)/N

        return result

    @classmethod
    def __computeSinglePointInverse2DCT(self, imge, x, y, N):
        """
        A private method that computes a single value of the 2D-DCT from a given image.

        Parameters
        ----------
        imge : ndarray
            The input image.

        u : ndarray
            The index in x-dimension.

        v : ndarray
            The index in y-dimension.

        N : int
            Size of the image.

        Returns
        -------
        result : float
            The computed single value of the DCT.
        """
        result = 0

        for u in xrange(N):
            for v in xrange(N):
                if (u==0) and (v==0):
                    tau = 1.0/N
                elif (u==0) or (v==0):
                    tau = math.sqrt(2.0)/N
                else:
                    tau = 2.0/N
                result += tau * dctImge[u, v] * math.cos(((2*x + 1)*u*math.pi)/(2*N)) * math.cos(((2*y + 1)*v*math.pi)/(2*N))

        return result

    @classmethod
    def computeForward2DDCT(self, imge):
        """
        Computes/generates the 2D DCT of an input image in spatial domain.

        Parameters
        ----------
        imge : ndarray
            The input image to be transformed.

        Returns
        -------
        final2DDFT : ndarray
            The transformed image.
        """

        # Assuming a square image
        N = imge.shape[0]
        final2DDCT = np.zeros([N, N], dtype=float)
        for u in xrange(N):
            for v in xrange(N):
                #Compute the DCT value for each cells/points in the resulting transformed image.
                final2DDCT[u, v] = DCT.__computeSinglePoint2DCT(imge, u, v, N)
        return final2DDCT

    @classmethod
    def computeInverse2DDCT(self, imge):
        """
        Computes/generates the 2D DCT of an input image in spatial domain.

        Parameters
        ----------
        imge : ndarray
            The input image to be transformed.

        Returns
        -------
        final2DDFT : ndarray
            The transformed image.
        """

        # Assuming a square image
        N = imge.shape[0]
        finalInverse2DDCT = np.zeros([N, N], dtype=float)
        for x in xrange(N):
            for y in xrange(N):
                #Compute the DCT value for each cells/points in the resulting transformed image.
                finalInverse2DDCT[x, y] = DCT.__computeSinglePointInverse2DCT(imge, x, y, N)
        return finalInverse2DDCT

    @classmethod
    def normalize2DDCTByLog(self, dctImge):
        """
        Computes the log transformation of the transformed DCT image to make the range
        of the DCT values b/n 0 to 255

        Parameters
        ----------
        dctImge : ndarray
            The input DCT transformed image.

        Returns
        -------
        dctNormImge : ndarray
            The normalized version of the transformed image.
        """

        #Normalize the DCT values of a transformed image:
        dctImge = np.absolute(dctImge)
        dctNormImge = (255/ math.log10(255)) * np.log10(1 + (255/(np.max(dctImge))*dctImge))

        return dctNormImge

```

## Testing the Code

### Testing the DCT Algorithm

First, we generate an 8-bit gray scale image as a 64x64 matrix and display it.

```python
imge = generateBlackAndWhiteSquareImage(64)
print "Image Size:", imge.shape
plt.figure(figsize=(3,3))
plt.imshow(imge, cmap=plt.get_cmap('gray'), vmin=0, vmax=255)
plt.show()
```

And we will also load another image from file:

```python
#Read an image file
imgeLena = Image.open("Images/lena_gray_256.tif") # open an image

#Convert the image file to a matrix
imgeLena = np.array(imgeLena)

#Convert the uint datatype of the matrix values into 'int' for using the negative values
imge = imge.astype('int')

#Display the image:
print "Image Size:", imgeLena.shape
plt.imshow(imgeLena, cmap=plt.get_cmap('gray'), vmin=0, vmax=255)
plt.show()
```

The image is then resized to 64x64 for making the DCT algorithm faster:

```python
imgeLena = resizeImage(imgeLena, 64)

print "Image Size:", imgeLena.shape

plt.figure(figsize=(3,3))
plt.imshow(imgeLena, cmap=plt.get_cmap('gray'), vmin=0, vmax=255)
plt.show()
```

Now, the DCT is computed for both the images as follows:

```python
#1. For image 1:
dctImge = DCT.computeForward2DDCT(imge)

#2. For image 2:
dctImgeLena = DCT.computeForward2DDCT(imgeLena)

```

Normalize the computed DCT results:

```python
#1. For image 1:
dctNormImge = DCT.normalize2DDCTByLog(dctImge)

#1. For image 2:
dctNormImgeLena = DCT.normalize2DDCTByLog(dctImgeLena)
```

Now, all the results are displayed for both the input images:

```python
fig, axarr = plt.subplots(2, 3, figsize=(10, 10))

#For image 1
axarr[0][0].imshow(imge, cmap=plt.get_cmap('gray'), vmin=0, vmax=255)
axarr[0][0].set_title('Original Image')

axarr[0][1].imshow(np.absolute(dctImge), cmap=plt.get_cmap('gray'))
axarr[0][1].set_title('Unnormalized DCT')

axarr[0][2].imshow(dctNormImge, cmap=plt.get_cmap('gray'))
axarr[0][2].set_title('Normalized DCT')

#For image 2
axarr[1][0].imshow(np.absolute(imgeLena), cmap=plt.get_cmap('gray'))
axarr[1][0].set_title('Original Image')

axarr[1][1].imshow(dctImgeLena, cmap=plt.get_cmap('gray'))
axarr[1][1].set_title('Unnormalized DCT')
axarr[1][2].imshow(dctNormImgeLena, cmap=plt.get_cmap('gray'))
axarr[1][2].set_title('Normalized DCT')
#fig.set_figwidth(15)

plt.show()
```

Now, the DCT results can be return back to their original form by using inverse DCT. (Only for the first image done).

Note: the result is rounded to the nearest integer to eliminate the very small decimall points which have come due to floating point precision.

```python
inverseImge = np.round(DCT.computeInverse2DDCT(dctImge))
plt.imshow(inverseImge, cmap=plt.get_cmap('gray'))
plt.show()
```

### Computing and Visualizing the DCT Running Time

After the implementation of the DCT algorithm is fully tested, its running time is computed for different image sizes.

First the images with different sizes are generated:

```python
#Generate images
imgSizes = [128, 64, 32, 16, 8]
images = generateImages(imgSizes)
```

After the images are generated, the DCT running time is analyzed as follows:

```python
# A list that stores the running time of the DCT algorithm for images with different size.
runningTimeDCT = []

#For each image...
for i, imge in enumerate(images):

    #Compute the image size
    N = imge.shape[0]

    print "Computing for ", N, "x", N, "image..."

    #Save the starting time.
    startTime = time.time()

    #Compute the DCT of the image.
    dctImge = DCT.computeForward2DDCT(imge)

    #Save the running time
    runningTimeDCT.append((time.time() - startTime)/60.0)
```

    Computing for  128 x 128 image...
    Computing for  64 x 64 image...
    Computing for  32 x 32 image...
    Computing for  16 x 16 image...
    Computing for  8 x 8 image...

```python
result = zip(imgSizes, runningTimeDCT)
np.savetxt("RunningTimes/runningTimeDCT.csv", np.array(result), delimiter=',')

#Load the running time for DFT from the separate post
runningTimeDFT = np.loadtxt("RunningTimes/runningTimeDFT.csv", delimiter =',')
```

Finally, the computation times are visualized by using line plot:

```python
#Plot the running times
plt.plot(xrange(len(runningTimeDCT)), runningTimeDCT, '-d')
plt.hold
plt.plot(xrange(len(runningTimeDFT)), runningTimeDFT, '-d')

xlabels = [str(imge.shape[0]) + 'x' + str(imge.shape[0]) for imge in images]
print xlabels
plt.xticks(xrange(len(runningTimeDCT)), xlabels)
plt.xlabel("Image Size(Pixels)")
plt.ylabel("Time(Sec)")
plt.legend(['DCT', 'DFT'])
plt.show()
```

From the above results, we can conclude that DCT is faster than DFT as it only calculates the real part.


# Chapter 2 - Linear Algebra

## 2.1 Scalars, Vectors, Matrices and Tensors

### Q1 [10 Points, M]
Denote the set of all n-dimensional binary vectors with Cartesian product notation

### Q2 [10 Points, S]
Given the vector
$
\boldsymbol{x}=\left[\begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4} \\
\end{array}\right]
$,
and the set
$
S = \{2, 4\}
$,
obtain the vectors $\boldsymbol{x}_{S}$ and $\boldsymbol{x}_{-S}$

### Q3 [20 Points, S]
Evaluate the following expressions with broadcasting rules,

$$
\left[\begin{array}{lll}
0 & 1 & 2
\end{array}\right]+[5]=
$$

$$
\left[\begin{array}{lll}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{array}\right]+\left[\begin{array}{lll}
0 & 1 & 1
\end{array}\right]=
$$

$$
\left[\begin{array}{l}
0 \\
1 \\
2
\end{array}\right]+\left[\begin{array}{ll}
0 & 1 & 2
\end{array}\right]=
$$

## 2.2-3 Multiplying Matrices and Vectors and Identity and Inverse Matrices

### Q4 [20 Points, H]
Let $A$ be a $2 \times 2$ matrix, if $A B=B A$ for every $B$ of the size $2 \times 2$, Prove that:
$$
A=\left[\begin{array}{ll}
a & 0 \\
0 & a
\end{array}\right], \
a \in \mathbb{R}
$$

## 2.4 Linear Dependence and Span

### Q5 [10 Points, H]
Prove that if a linear system of equations have two solutions, then it has infinitely many solutions.

### Q6 [5 Points, M]
Given $A x=0$, where $A \in \mathbb{R}^{m \times n}$ is any matrix, and $x \in \mathbb{R}^{n}$ is a vector of unknown variables to be solved, what is the condition such that there is infinitely many solutions?

## 2.5 Norms

### Q7 [15 Points, M]
Prove that **Max Norm** follows these conditions,
$$
\begin{align}
f(\boldsymbol{x})=0 \Rightarrow \boldsymbol{x}=\mathbf{0} \\
f(\boldsymbol{x}+\boldsymbol{y}) \leq f(\boldsymbol{x})+f(\boldsymbol{y}) \\
\forall \alpha \in \mathbb{R}, f(\alpha \boldsymbol{x})=|\alpha| f(\boldsymbol{x})
\end{align}
$$

## 2.6 Special Kinds of Matrices and Vectors

### Q8 [10 Points, M]
Solve the following system of equations,

$$
\frac{1}{2}\left[\begin{array}{cccc}
1 & 1 & 1 & 1 \\
1 & 1 & -1 & -1 \\
1 & -1 & 1 & -1 \\
1 & -1 & -1 & 1
\end{array}\right] \left[\begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4} \\
\end{array}\right] = \left[\begin{array}{c}
1 \\
2 \\
3 \\
4 \\
\end{array}\right]
$$


# How to Draw Ellipse of Covariance Matrix
Given a 2x2 covariance matrix, I explain how to draw the ellipse representing it. The following function explains the method to visualize multivariate normal distributions and correlation matrices. Formulae for radii & rotation are provided for covariance matrix shown below
\begin{align}
\Sigma = \begin{bmatrix} a & b \\ b & c \end{bmatrix}
\end{align}

## Radii and Rotation

\begin{align}
\lambda_{1,2} &= \frac{a+c}{2} \pm \sqrt{\left( \frac{a-c}{2} \right)^{2} + b^{2}} \\
\theta &= \begin{cases} 0 & \text{ if } b = 0 \text{ and } a \geq c \\
 \frac{\pi}{2} & \text{ if } b = 0 \text{ and } a < c \\
 \text{atan2}(\lambda_{1} - a, b) & \text{ if } b \neq 0
\end{cases}
\end{align}
Here, $\theta$ is the angle in radians from positive x-axis to the ellipse's major axis in the counterclockwise direction. $\sqrt{\lambda_{1}}$ is the radius of the major axis (the longer radius) and $\sqrt{\lambda_{2}}$ is the radius of the minor axis (shorter radius). In $atan2(\cdot, \cdot)$, the first parameter is $y$ and second is $x$.

```python
import random
import math
import numpy as np
import matplotlib.pyplot as plt
import time
from scipy.linalg import block_diag
from scipy.special import erfinv
from scipy.stats import t as tdist
from numpy.linalg import inv
from numpy import linalg as LA
from matplotlib.patches import Ellipse
```

```python
def GetAngleAndRadii(covar):
    """
    Given a covariance matrix, the function GetAngleAndRadii() calculates
    the major axis and minor axis radii and the orientation of the ellipse.

    Inputs:
    covar: 2x2 matrix

    Output:
    major_radius: Radius of the major axis of ellipse
    minor_radius: Radius of the minor axis of ellipse
    theta       : Orientation angle in radians from positive x-axis
                  to the ellipse's major axis in the counterclockwise direction
    """

    # Infer the a,b,c values
    a = covar[0,0]
    b = covar[0,1]
    c = covar[1,1]

    if b > a:
        raise Exception("Sorry, covariance matrix is invalid - Cov[0,1] should be < Cov[0,0] ")

    lambda_1 = (a+c)/2 + math.sqrt(((a-c)/2)**2 + b**2)
    lambda_2 = (a+c)/2 - math.sqrt(((a-c)/2)**2 + b**2)

    # Infer the radii
    major_radius = math.sqrt(lambda_1)
    minor_radius = math.sqrt(lambda_2)

    # Infer the rotation
    if b == 0:
        if a >= c:
            theta = 0
        else:
            theta = pi/2
    else:
        theta = math.atan2(lambda_1-a, b)

    return major_radius, minor_radius, theta

```

```python
# Check the above code
covar_check = np.array([[9,5],[5,4]])
major_radius_check, minor_radius_check, theta_check = GetAngleAndRadii(covar_check)
print('major axis radius = ', round(major_radius_check,2),
      'minor axis radius = ', round(minor_radius_check,2),
      'orientation = ', round(theta_check,2), 'rad')
```

    major axis radius =  3.48 minor axis radius =  0.95 orientation =  0.55 rad

```python
def plot_ellipse(center, cov = None):

    # Get the center of ellipse
    x_cent, y_cent = center

    print('center x at: ', x_cent)
    print('center y at: ', y_cent)

    # Get Ellipse Properties from cov matrix
    if cov is not None:
        major_radius, minor_radius, theta_orient = GetAngleAndRadii(cov)
        print('major axis radius = ', round(major_radius,2),
              'minor axis radius = ', round(minor_radius,2),
              'orientation = ', round(theta_orient,2), 'rad')
        eig_vec,eig_val,u = np.linalg.svd(cov)

    # Generate data for ellipse structure
    t    = np.linspace(0,2*np.pi,1000)
    x    = major_radius*np.cos(t)
    y    = minor_radius*np.sin(t)
    data = np.array([x,y])
    R    = np.array([[np.cos(theta_orient),-np.sin(theta_orient)],
                     [np.sin(theta_orient),np.cos(theta_orient)]])
    T    = np.dot(R,eig_vec)
    data = np.dot(T,data)

    # Center the ellipse at given center
    data[0] += x_cent
    data[1] += y_cent

    # Plot the ellipse
    fig,ax = plt.subplots()
    ax.plot(data[0],data[1],color='b',linestyle='-')
    ax.fill(data[0],data[1])
```

```python
covar_check = np.array([[9,4],[4,3]])
plot_ellipse(center = (1,2), cov=covar_check)
```

```python

```


# Fangohr, Hans. Introduction to Python for Computational Science and Engineering, 2015.
Embleton | 20160910 | Notes

### General Notes
* Use `help()` with a command for details
* Use `dir()` with a command for a list of available methods

## Chapter 2, A Powerful Calculator

```python
import math
```

```python
dir(math)

help(math.exp)

math.pi

math.e
```

    Help on built-in function exp in module math:

    exp(...)
        exp(x)

        Return e raised to the power of x.

    2.718281828459045

## Chapter 3, Data Types and Structures

* use cmath library to calculate complex results
* Strings are immutable, lists are mutable
* `dir("")` outputs a list of available methods
* Sequences
    * `a[i]` returns the ith element of a
    * `a[i:j]` returns elements i up to j-1
    * `len(a)` returns number of elements in a sequence
    * `min(a)` returns the smallest value in a seq.
    * `max(a)`
    * `x in a` returns True if x is an element in a
    * `a + b` concatenates seq. a and seq. b
    * `n * a` creates n copies of seq. a
* The `split()` method seperates the string where it finds white space or at a seperator character.
* The join method is the opposite of split
* Lists
    * Empty list given by `x = []`
    * You can mix objects within a list
    * You can add lists within lists
        * ? Is this the proper method for making tables?
            * You can use scipy.arrange() or pandas
    * `append()` to add an object to the end of a list, opposite is `remove()`
    * `range()` command common in for loops. Use: range(start, stop, step size)
        * Range is a type!
* Tuple
    * Immutable
    * Empty tuple given by `t = ()`
    * Tuple containing one object `t = (x,)`. The comma is required.
* Indexing
    * Use negative numbers to retrieve values from the back of the list
* Slicing
    * Slicing is different from indicing as it corresponds to the point between two indicies.
* Dictionaries
    * empty dictionary: `d = {}`
    * look for matches with: `d.has_key()` or `d.has_item()`
    * method `get(key, default)` to retrieve values or default if key not found
    * Keyword can be any immutable object
    * Dictionaries are very fast when retreiving values (when given the key)
* Passing arguments to functions
    * Modifications to values of an arguement in a function can affect the value of the original object
* Copying
    * `b = a` does not pass a copy of a to b and create two seperate objects.  Instead b and a refer to the same object.  Only the lable is copied. To create a copy of a with a differnt label, use something like `c = a[:]`
    * use `id(a)` to determine if objects are the same or different
* Equality Operators
    * <, >, ==, >=, <=, !=
    * Does not depend on type
    * To compare the id use, `a is b`.  Objects with different types will not have the same id

```python
import cmath

cmath.sqrt(-1)
```

    1j

```python
a = 'This is a test sentance'

print(a)

print(a.upper())

print(a.split())
```

    This is a test sentance
    THIS IS A TEST SENTANCE
    ['This', 'is', 'a', 'test', 'sentance']

```python
b = "The dog is hungry. The cat is bored. The snake is awake."
print(b)

s=b.split(".")

print(s)

print(".".join(s))

print(" STOP".join(s))
```

    The dog is hungry. The cat is bored. The snake is awake.
    ['The dog is hungry', ' The cat is bored', ' The snake is awake', '']
    The dog is hungry. The cat is bored. The snake is awake.
    The dog is hungry STOP The cat is bored STOP The snake is awake STOP

```python
a = [1, 2, 3]

print(a)

a.append(45)

print(a)

a.remove(2)

print(a)
```

    [1, 2, 3]
    [1, 2, 3, 45]
    [1, 3, 45]

```python
print(range(3, 10))

for i in range(3,11):
    print(i**2)
```

    range(3, 10)
    9
    16
    25
    36
    49
    64
    81
    100

```python
a = 100, 200, 'duck'
print(a)
print(type(a))

x, y = 10, 20
print(x)
print(y)

x, y = y, x
print(x)
print(y)
```

    (100, 200, 'duck')
    <class 'tuple'>
    10
    20
    20
    10

```python
a = 'dog cat mouse'
a = a.split()
print(a[0])
print(a[-1])
print(a[-2:])
```

    dog
    mouse
    ['cat', 'mouse']

```python
## Dictionaries
d = {}
d['today'] = [1, 2, 3]

d['yesterday'] = '19 deg C'

print(d.keys())
print(d.values())
print(d.items())

print(d)
print(d['today'])
print(d['today'][1])

# Other methods for creating dictionaries
d2 = {2:4, 3:9, 4:16}
print(d2)

d3 = dict(a=1, b=2, c=3)
print(d3)
print(d3['a'])

print(d.__contains__('today'))
d.get('today','unknown')
```

    dict_keys(['yesterday', 'today'])
    dict_values(['19 deg C', [1, 2, 3]])
    dict_items([('yesterday', '19 deg C'), ('today', [1, 2, 3])])
    {'yesterday': '19 deg C', 'today': [1, 2, 3]}
    [1, 2, 3]
    2
    {2: 4, 3: 9, 4: 16}
    {'a': 1, 'c': 3, 'b': 2}
    1
    True

    [1, 2, 3]

```python
# Dictionary Example

# create an empty directory
order = {}

# add orders as they come in
order['Peter'] = 'Pint of bitter'
order['Paul'] = 'Half pint of Hoegarden'
order['Mary'] = 'Gin Tonic'

# deliver order at bar
for person in order.keys():
    print(person, "requests", order[person])

```

    Paul requests Half pint of Hoegarden
    Mary requests Gin Tonic
    Peter requests Pint of bitter

```python
#Copying and Identity

a = [1, 2, 3, 4, 5]
b=a
b[0] = 42

print(a)

c = a[:]
c[1] = 99

print(a)
print(c)
print('id a: ', id(a))
print('id b: ', id(b))
print('id c: ', id(c))
```

    [42, 2, 3, 4, 5]
    [42, 2, 3, 4, 5]
    [42, 99, 3, 4, 5]
    id a:  72139272
    id b:  72139272
    id c:  67450824

## Chapter 4, Introspection

* Magic names start and end with a double underscore
* `isinstance(<unit>, <type space>)` Returns true if the given object is of the given type.
* `help(<object>)`
* `help()` Starts aand interactive help utility
* Provide a docstring for user defined functions

```python
# Example of documenting a user defined function and calling it.

def power2and3(x):
    """Returns the tuple (x**2, x**3)"""
    return x**2 ,x**3

print(power2and3(2))

print(power2and3.__doc__)

help(power2and3)
```

    (4, 8)
    Returns the tuple (x**2, x**3)
    Help on function power2and3 in module __main__:

    power2and3(x)
        Returns the tuple (x**2, x**3)

## Chapter 5, Input and Output

* String specifiers, reprinted table below
* Pg 54-55 for a more elegant method of string formatting used in Python 3
* `fileobject.readlines()` method returns a list of strings

```python
## Copied from pg 52.

AU = 149597870700  #Astronomical unit in [m]
"%g" %AU
```

    '1.49598e+11'

|Specifier|Style|Example Output for AU|
|:---:|:---:|:---|
|`%f`|Floating Point|149597870700.000000|
|`%e`|Exponential Notation|1.495979e+11|
|`%g`|Shorter of %e or %f|1.49598e+11|
|`%d`|Integer|149597870700|
|`%s`|String|149597870700|
|`%r`|repr|149597870700L|

```python
a = math.pi

print("Short pi = %.2f. longer pi = %.12f."  %(a, a))
```

    Short pi = 3.14. longer pi = 3.141592653590.

```python
## Reading and Writing Files

#1. Write a File
out_file = open("test.txt", "w")    # 'w' stands for Writing
out_file.write("Writing text to file. This is the first line.\n"
              "And the second lineasdfa.")
out_file.close()                    # close the file

#2. Read a File
in_file = open("test.txt", "r")       # 'r' stands for Reading
text = in_file.read()

in_file.close()

#3. Display Data
print (text)
```

    Writing text to file. This is the first line.
    And the second lineasdfa.

```python
## Readlines Example

myexp = open("myfile.txt", "w")    # 'w' stands for Writing
myexp.write("This is the first line.\n"
            "This is the second line.\n"
            "This is the third and last line.")
myexp.close()

f = open('myfile.txt', "r")
print(len(f.read()))
f.close()

f = open('myfile.txt', "r")
for line in f.readlines():
    print("%d characters" %len(line))
f.close()
```

    81
    24 characters
    25 characters
    32 characters

# Chapter 6, Control Flow

* If-then-else statements
* For loops
* use logical operators "`and`" and "`or`" to combine conditions
* Read chapter 8 for more on errors and exceptiosn, help('exceptions')

```python
a = 17

if a == 0:
    print("a is zero")
elif a < 0:
    print("a is negative")
else:
    print("a is positive")
```

    a is positive

```python
# for example
for animal in ['dog', 'cat', 'mouse']:
    print(animal, animal.upper())

for i in range(5,10):
    print(i)
```

    dog DOG
    cat CAT
    mouse MOUSE
    5
    6
    7
    8
    9

## Chapter 7, Functions and Modules

* A function takes an argument and returns a result or return value.
* Function parameter may have default values.
    * ie. `def print_multi_table(n, upto=10):`
* Common to have an `if __name__ == "__main__` to output results and capabilities only seen when program is runnin on its own.

Generic Function Format:

    def my_function(arg1, arg2, ..., argn):
        """Optional docstring."""

        #Implementation of the function

        return result #optional

    #this is not part of the function
    some_command

## Chapter 8, Functional Tools

* Examples using the tools `filter`, `reduce`, and `lamda`.
* An anonymous function is only needed once or needs no name
    * `lambda x : x**2`
    * `(lambda x, y, z: (x + y) * z)(10, 20, 2)`
* The map function applies function f to all elements in sequence s, `lst2 = map(f,s)`
    * `map(lambda x:x**2, range(10))`
* The filter function applies the function f to all elements in a sequence s, `lst2 = filter(f, lst)`
    * The filet function should return a true or false.
    * `filter(lambda x:x>5,range(11))`
* List comprehension is an expression followed by a for clause, then zero or more for or if clauses.  More consise then the above methods.

```python
## Maps

def f(x):
    return x**2

# Two methods to print
print(list(map(f, range(10))))

for ch in map(f, range(10)):
    print(ch)

#Combining with Lambda
print(list(map(lambda x:x**2,range(10))))

```

    [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
    0
    1
    4
    9
    16
    25
    36
    49
    64
    81
    [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

```python
## List Comprehensions

vec = [2, 4, 6]
print(vec)
print([3 * x for x in vec])
print([3 * x for x in vec if x >3])
print([3 * x for x in vec if x <2])
print([[x, x**2] for x in vec])
```

    [2, 4, 6]
    [6, 12, 18]
    [12, 18]
    []
    [[2, 4], [4, 16], [6, 36]]

## Chapter 9, Common Tasks

* Illustrates many ways to compute a series to illustrate the differnt methods possible and also illustrate the different methods we've previously discussed.  Includes a check method and doc file.
* `sorted` returns a copy of a sorted list while `sort` changes the list insto a sorted order of elements

## Chapter 10, Matlab to Python
* Common differences
* The extension library numpy provides matrix functionality similar to Matlab

## Chapter 11, Python Shells

* Useful features of different Python shells
* iPython, IPython Notebook, Spyder

## Chapter 12, Symbolic Computation

* SymPy is the Python Symbolic library, [SymPy Homepage](http://sympy.org) for full and up-to-date documentation
* Very slow compared to floating point opperation
* isympy an exexutable wrapper around python, convenient for figuring out new features or experementing interactively
* Rational type, Rational(1,2) represents 1/2.
    * Rational class works exactly as opposed to the standard float.
* If Sympy returns the result in an unfamiliar form, subtract it with the expected form to determine if they are equivalent.
* Calculate definite integrals with a tuple containing the variable of interest, lower, and upper bounds.
* Results from dsolve are an Equality class, function needs to be evaluated to a number to be plotted.
* Covers series expansion
* LaTeX and Pretty printing
* `preview()` allows you to display rendered output on the screen
* Automatic generation of C code via `codegen()`

```python
## Symbols

import sympy

x, y, z = sympy.symbols('x, y, z')

a = x + 2*y + 3*z - x

print(a)

print(sympy.sqrt(8))
```

    2*y + 3*z
    2*sqrt(2)

```python
x, y = sympy.symbols('x,y')
a = x + 2*y
print(a.subs(x, 10))
print(a.subs(x,10).subs(y,3))
print(a.subs({x:10, y:3}))

SS_77 =  -y + -23.625*x**3 - 5.3065*x**2 + 5.6633*x
SS_52 =  -y + -245.67*x**3 + 31.951*x**2 + 4.4341*x
SS_36 =  -y + -18.58*x**3 - 5.4025*x**2 + 2.1623*x

#print("t0 = 36, 0.5 Strain at %.2f MPa" % sympy.solve(SS_36.subs(y, 0.5),x)[0])
#print("t0 = 52, 0.5 Strain at %.2f MPa" % sympy.solve(SS_52.subs(y, 0.5),x)[0])
#print("t0 = 77, 0.5 Strain at %.2f MPa" % sympy.solve(SS_77.subs(y, 0.5),x)[0])

print("t0 = 36, 0.01 Stress at %.3f Strain" % sympy.solve(SS_36.subs(x, .01),y)[0])
print("t0 = 52, 0.01 Stress at %.3f Strain" % sympy.solve(SS_52.subs(x, .01),y)[0])
print("t0 = 77, 0.01 Stress at %.3f Strain" % sympy.solve(SS_77.subs(x, .01),y)[0])
```

    2*y + 10
    16
    16
    t0 = 36, 0.01 Stress at 0.021 Strain
    t0 = 52, 0.01 Stress at 0.047 Strain
    t0 = 77, 0.01 Stress at 0.056 Strain

```python
a = sympy.Rational(2,3)
print(a)
print(float(a))
print(a.evalf())
print(a.evalf(50))
```

    2/3
    0.6666666666666666
    0.666666666666667
    0.66666666666666666666666666666666666666666666666667

```python
# Differentiation

print(sympy.diff(3*x**4, x))
print(sympy.diff(3*x**4, x, x, x))
print(sympy.diff(3*x**4, x, 3))
```

    12*x**3
    72*x
    72*x

```python
## Integration

from sympy import integrate

print(integrate(sympy.sin(x), y))
print(integrate(sympy.sin(x), x))

# Definite Integrals
print(integrate(x*2, x))
print(integrate(x*2, (x, 0, 2)))
print(integrate(x**2, (x,0,2), (x, 0, 2), (y,0,1)))
```

    y*sin(x)
    -cos(x)
    x**2
    4
    16/3

```python
## Ordinary Differential Equations

from sympy import Symbol, dsolve, Function, Derivative, Eq

y = Function("y")
x = Symbol('x')
y_ = Derivative(y(x), x)
print(dsolve(y_ + 5*y(x), y(x)))
print(dsolve(Eq(y_ + 5*y(x), 0), y(x)))
print(dsolve(Eq(y_ + 5*y(x), 12), y(x)))
```

    Eq(y(x), C1*exp(-5*x))
    Eq(y(x), C1*exp(-5*x))
    Eq(y(x), C1*exp(-5*x)/5 + 12/5)

```python
## Linear Equations and Matrix Inversion

from sympy import symbols, Matrix

x, y, z = symbols('x,y,z')

A = Matrix(([3,7], [4,-2]))
print(A)

print(A.inv())
```

    Matrix([[3, 7], [4, -2]])
    Matrix([[1/17, 7/34], [2/17, -3/34]])

```python
## Solving Non Linear Equations

import sympy

x, y, z = sympy.symbols('x,y,z')
eq = x - x**2
print(sympy.solve(eq,x))
```

    [0, 1]

## Chapter 14, Numerical Calculation

* Limitations of the different number types: int, float, complex, and long.
* Comparing float and symbolic time to compute.

## Chapter 15, Numerical Python (numpy): arrays

* The data structure, `array`, allows efficient matrix and vector operation
* An array can only keep elements of the same type, as opposed to lists which can hold a mix.
* Convert a matrix back tp a list or tuple using, `list(s)` or `tuple(s)`.
* Computing eiganvectors and eiganvalues
* Numpy examples at [SciPy.org](http://www.scipy.org/Numpy_Example_List)

```python
## Vectors (1d-arrays)

import numpy as N

x = N.array([0, 0.5, 1, 1.5])

print(x)

print(N.zeros(4))
a = N.zeros((5,4))
print(a)
print(a.shape)

print(a[2,3])

random_matrix = N.random.rand(5,5)
print(random_matrix)

x = N.random.rand(5)

b = N.dot(random_matrix, x)
print("b= ", b)

```

    [ 0.   0.5  1.   1.5]
    [ 0.  0.  0.  0.]
    [[ 0.  0.  0.  0.]
     [ 0.  0.  0.  0.]
     [ 0.  0.  0.  0.]
     [ 0.  0.  0.  0.]
     [ 0.  0.  0.  0.]]
    (5, 4)
    0.0
    [[ 0.62986856  0.08414351  0.86029432  0.23673407  0.69039432]
     [ 0.09499312  0.46304194  0.83582097  0.80421487  0.99190126]
     [ 0.98594909  0.58822546  0.86016599  0.41493799  0.31856799]
     [ 0.91495891  0.38045604  0.67692051  0.39180708  0.22073492]
     [ 0.65115666  0.67522929  0.69594017  0.13819881  0.62083603]]
    b=  [ 0.39203078  0.83042469  0.85378184  0.62487284  0.88562137]

```python
## Curve Fitting of Polynomial Example

import numpy

# demo curve fitting: xdata and ydata are input data
xdata = numpy.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0])
ydata = numpy.array([0.0, 0.8, 0.9, 0.1, -0.8, -1.0])

#now fit for cubic (order = 3) polynomial
z = numpy.polyfit(xdata, ydata, 3)

#z is an array of coefficients, highest first, i.e.
#          x^3         X^2      X       0
#z=array([0.08703704, -0.8134, 1.693, -0.0396])

print("z = ", z)

#It is convenient to use `poly1d` objects for dealing with polynomials
p = numpy.poly1d(z)    #Creates a polynomial function p from coefficients and p can be evaluated for all x then.

print("p = ",p)

#Create a plot
xs = [0.1 * i for i in range(50)]
ys = [p(x) for x in xs]    # evaluates p(x) for all x in list xs

%matplotlib inline
import pylab
pylab.plot(xdata, ydata, 'o', label = 'data')
pylab.plot(xs, ys, label = 'fitted curve')
pylab.ylabel('y')
pylab.xlabel('x')
#pylab.savefig('polyfit.pdf')
pylab.show()
```

## Chapter 15, Visualizing Data

* Need to include all the useful links here
* IPython Inline mode via: `%matplotlib inline`, `%matplotlib qt`, or `%pylab`.
* `help(pylab.legend)` for legend placement information
* `help(pylab.plot)` for line style, color, thickness, etc calls
    * Colors can be called out in RGB, Hex, greyscale, etc.
* Subplot to call more than one plot in one figure, `pylab.subplot(numRows, numCols, plotNum)`.
* Multiple figures via: `pylab.figure(figNum)`.
* `pylab.close()` may be used to close one, some, or all figures.
* Use `pyplot.imshow()` to visualize matrix data (heat plot).
    * Use different color maps with the module `matplotlib.cm`
* Check out the contour_demo.py example for illustrating `z=f(x,y)`
* Visual Python is a module that allows you to create and animate 3D scenes.
    * Visual Python [Home Page](http://vpython.org)
    * Useful for illustrating time dependent data
* Visualising 2D and 3D fields as a function of time with the Visualization Toolkit, [VTK](http://vtk.org).
* Other modules: Mayavi, Paraview, and VisIt.

```python
## Plot details example

import pylab
import numpy as N

%matplotlib inline

x = N.arange(-3.14, 3.14, 0.01)
y1 = N.sin(x)
y2 = N.cos(x)

pylab.figure(figsize=(5,5))    #Sets figure size to 5 x 5 in.
pylab.plot(x, y1, label='sin(x)')
pylab.plot(x, y2, label='cos(x)')
pylab.legend()
pylab.grid()
pylab.xlabel('x')
pylab.title('This is the Title')
pylab.axis([-2,2,-1,1])
pylab.show()

```

```python
## Histogram Example

import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pylab as plt

# create data
mu, sigma = 100, 15
x = mu + sigma * np.random.randn(10000)

#histogram of the data
n, bins, patches = plt.hist(x, 50, normed=1, facecolor='green', alpha=0.75)

#fine tuning the plot
plt.xlabel('Smarts')
plt.ylabel('Probability')

#LaTeX strings for labels and titles
plt.title(r'$\mathrm{Histogram\ of\ IQ :}\ \mu=100,\ \sigma=15$')
plt.axis([40, 160, 0, 0.03])
plt.grid(True)

#add a best fit line curve
y = mlab.normpdf(bins, mu, sigma)
l = plt.plot(bins, y, 'r--', linewidth=1)

#Save to file
#plt.savefig('pylabhistogram.pdf')
#then display
plt.show()

```

## Chapter 16, Numerical Methods using Python (scipy)

* The `scipy` package privides numerous numerical algorithms
* All functionality from `numpy` may be available in `scipy`
* Using `help(scipy)` will detail the package structure.  You can call specific sections ie. `import scipy.integrate`.
* Use `scipy.quad()` to solve integrals of the form $\int_{a}^{b} f(x)dx$
    * Functions that approach +/- inf may be difficult to handle numerically.  Plot results with integand to check
* Use `scipy.odeint()` to solve differential equations of the type $\frac{\partial y}{\partial t}(t) = f(y,t)$
    * `help(scipy.integrate.odeint)` to explore differnet error tolerance options
* Using the `bisect()` method to find roots.  Requires arguments f(x), lower limit, and upper limit.  Optional xtol parameter.
* Using `fsolve()` to find roots is more efficient, but not garunteed to converge.  Input argument is a starting location suspected close to the root.
* The function `y0 = scipy.interpolate.interp1d(x, y, kind='nearest')` may be used to interpolate the data $(x_i, y_i)$ for all x.
* A generic curve fitting function is provided with `scipy.optimize.curve_fit()`
* FFT example below
* Optimization, using `scipy.optimize.fmin()` to find the minimum of a function. Arguments are the function and starting point.

```python
## Integral Example

from math import cos, exp, pi
from scipy.integrate import quad

#function  we want to integrate
def f(x):
    return exp(cos(-2 * x * pi)) + 3.2

#call quad to integrate f from -2 to 2
res, err = quad(f, -2, 2)

print("The numerical result is {:f} (+/-{:.2g})" .format(res, err))
```

    The numerical result is 17.864264 (+/-1.6e-11)

```python
## ODE Example

from scipy.integrate import odeint
import numpy as N

def f(y,t):
    """This returns the RHS of the ODE to integrate, i.e. dy/dt = f(y,t)"""
    return(-2 * y * t)

y0 = 1    # initial value
a = 0     # integration limits for t
b = 2

t = N.arange(a, b, 0.01)  # values of t for which we require the solution y(t)

y = odeint(f, y0, t)  # actual computation of y(t)

import pylab     # Plotting of results

%matplotlib inline
pylab.plot(t, y)
pylab.xlabel('t'); pylab.ylabel('y(t)')
pylab.show()
```

```python
## FFT exmaple
# Create a superposition of 50 and 70 Hz and plot the fft.

import scipy
import matplotlib.pyplot as plt
%matplotlib inline
pi = scipy.pi

signal_length = 0.5 # [seconds]
sample_rate = 500   # sampling rate [Hz]
dt = 1./sample_rate # delta t [s]

df = 1/signal_length    # frequency between points in the freq. domain [Hz]

t = scipy.arange(0, signal_length, dt) # the time vector
n_t = len(t)                           # length of the time vector

# create signal
y = scipy.sin(2*pi*50*t) + scipy.sin(2*pi*70*t + pi/4)

# compute the fourier transport
f = scipy.fft(y)

# work out meaningful frequencies in fourier transform
freqs = df*scipy.arange(0,(n_t-1)/2.,dtype='d')    #d = double precision float
n_freq = len(freqs)

# plot input data y against time
plt.subplot(2,1,1)
plt.plot(t, y, label='input data')
plt.xlabel('time [s]')
plt.ylabel('signal')

# plot frequency spectrum
plt.subplot(2,1,2)
plt.plot(freqs, abs(f[0:n_freq]), label='abs(fourier transform)')
plt.xlabel('frequency [Hz]')
plt.ylabel('abs(DFT(signal))')

# save plot to disk
#plt.savefig('fft1.pdf')
plt.show()
```

## Chapter 17, Where to go from here?

* A list of additional skills for computational science work.


```python
#@title Imports { display-mode: "form" }
from __future__ import print_function
from __future__ import division
from __future__ import absolute_import

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

#try:
#  tf.enable_eager_execution()
#  print('Eager execution enabled')
#except ValueError:
#  print('Already running in Eager mode')

#tfe = tf.contrib.eager
!pwd
```

    /Users/jchibueze/Downloads/Tutorials/James

## Matrix Multiplication

```python
# Define matrix A
A = np.array(
  [[1.0, 3.0],
   [2.0, 1.0],
   [4.0, 2.0]]
)

# Define matrix B
B = np.array(
  [[6.0, 2.0, 1.0],
   [3.0, 4.0, 5.0]]
)

# Define vector x
x = np.array([3.0, 2.0])

print('A.shape is:', A.shape, 'B.shape is:', B.shape, 'x.shape is:', x.shape)
A
```

### Matrix-vector multiplication

```python
# Using numpy dot
y = A.dot(x)

print('Using dot:\t y =', y, '\t y.shape =', y.shape)

# Using einsum
y = np.einsum('ij, j', A, x)

print('Using einsum:\t y =', y, '\t y.shape =', y.shape)

# Manual version 1
y = np.array([
    A[0,0] * x[0] + A[0,1] * x[1],
    A[1,0] * x[0] + A[1,1] * x[1],
    A[2,0] * x[0] + A[2,1] * x[1],
    ])
print('Manual 1:\t y =', y, '\t y.shape =', y.shape)

# Manual version 2:
# Matrix-vector multiplication can be thought of as a linear combination of the columns of of the matrix
y = x[0] * A[:,0]  +  x[1] * A[:, 1]

print('Manual 2:\t y =', y, '\t y.shape =', y.shape)
```

### Matrix-matrix multiplication

```python
# Using numpy dot
C = A.dot(B)

print('Using DOT: C= \n\n', C, '\n\nC.shape =', C.shape)

# Using einsum
C = np.einsum('ik, kj', A, B)
print('\n\nUsing einsum: C= \n\n', C, '\n\nC.shape =', C.shape)

# Note, the above einsum notation is equivalent to the following
C = np.einsum('ik, kj -> ij', A, B)

# And in Tensorflow
C = tf.matmul(A, B)
print('\n\nUsing Tensorflow: C= \n\n', C, '\n\nC.shape =', C.shape)
```

Matrix multiplication is not commutative

```python
# Matrix multiplication is not commutative:
C = B.dot(A)
print('C: \n', C)
print()
print('C.shape:', C.shape)
```

## Computing gradients with TensorFlow
$y = Ax$

In the code below, we use Tensorflow to calculate the following derivatives:

$\frac{dy}{dx}$

and

$\frac{\partial y}{\partial A}$

```python
A_tensor = tf.Variable(A)
x_tensor = tf.Variable(x)

with tf.GradientTape() as tape:
  y = tf.einsum('ij,j', A_tensor, x_tensor)

dydx, dydA = tape.gradient(y, [x_tensor, A_tensor])

print('dy/dx =', dydx)
print()
print('dy/dA =', dydA)
```

# Neural Network Gradient Example
In the following example, we compute the output of a 1 layer neural network and the gradients with respect to its parameters. We define an example input vector and parameters, but keep the computation generic. You can change the values and shapes of x, A and b below and run the rest of the code to compute the output and gradients for your own example.

```python
x = np.array([[-1.], [0.1], [2.1]])  # X has shape (3, 1)
A = np.array([              # A has shape (2, 3)
    [ 1.1, -2.5,  0.3],
    [-2.1,  0.2, -1.1]
])
b = np.array([[-1.0], [2.0]])      # b has shape (2)
```

Compute the neural network output
$\mathbf{f} = \operatorname{tanh}(A\mathbf{x} + \mathbf{b})$

```python
M, N = A.shape
z = A.dot(x) + b
f = np.tanh(z)

print('f =', f)
```

Compute the partial derivatives:

\begin{align}
\frac{d\mathbf{f}}{d\mathbf{z}} ; \frac{\partial\mathbf{z}}{\partial\mathbf{x}} ; \frac{\partial\mathbf{z}}{\partial\mathbf{b}} ; \frac{\partial\mathbf{z}}{\partial\mathbf{A}}
\end{align}

```python
# partial derivatives
dfdz = 1-f**2       # (derivative of tanh is 1-tanh^2)
print('df/dz =', dfdz, '\nshape:', dfdz.shape)
print()

dzdx = A
print('dz/dx =\n', dzdx, '\n\nshape:', dzdx.shape)
print()

dzdb = np.eye(M)
print('dz/db =\n', dzdb, '\n\nshape:', dzdb.shape)
print()

dzdA = np.zeros((M, M, N))  # Start with a tensor of zeros of the correct shape
for i in range(M):          # Then set the diagonal elements of dzdA
  dzdA[i,i,:] = x.T

print('dz/dA =\n', dzdA, '\n\nshape:', dzdA.shape)

```

Finally, we compute the gradients of the neural network output $f$ with respect to the parameters $A$ and $\mathbf{b}$ and the input $\mathbf{x}$ using the chain rule:

\begin{align}
\frac{\partial \mathbf{f}}{\partial \mathbf{x}} &= \frac{d \mathbf{f}}{d \mathbf{z}} \frac{\partial \mathbf{z}}{\partial \mathbf{x}} \ ; \
\frac{\partial \mathbf{f}}{\partial \mathbf{b}} = \frac{d \mathbf{f}}{d \mathbf{z}} \frac{\partial \mathbf{z}}{\partial \mathbf{b}} \ ; \
\frac{\partial \mathbf{f}}{\partial A} = \frac{d \mathbf{f}}{d \mathbf{z}} \frac{\partial \mathbf{z}}{\partial A}
\end{align}

```python
dfdx = np.einsum('il, lj', dfdz, dzdx)
print('df/dx =\n', dfdx, '\n\nshape:', dfdx.shape)
print()

dfdb = np.einsum('il, lj', dfdz, dzdb)
print('df/db =\n', dfdb, '\n\nshape:', dfdb.shape)
print()

dfdA = np.einsum('il, ljk', dfdz, dzdA)
print('df/dA =\n', dfdA, '\n\nshape:', dfdA.shape)
```

```python

```


# Part 1 - Scalars and Vectors

For the questions below it is not sufficient to simply provide answer to the questions, but you must solve the problems and show your work using python (the NumPy library will help a lot!) Translate the vectors and matrices into their appropriate python  representations and use numpy or functions that you write yourself to demonstrate the result or property.

```
import math
import matplotlib.pyplot as plt
import numpy as np
```

## 1.1 Create a two-dimensional vector and plot it on a graph

```
red = [1, 1]

plt.arrow(0,0, red[0], red[1],head_width=.05, head_length=0.05, color ='red')

plt.xlim(0,2)
plt.ylim(0,2)
plt.title("One Two-Dimentional Vector")

plt.show()
```

## 1.2 Create a three-dimensional vecor and plot it on a graph

```
from mpl_toolkits.mplot3d import Axes3D

red_3d = [1, 1, 1]

vectors = np.array([[0, 0, 0, 1, 1, 1]])

X, Y, Z, U, V, W = zip(*vectors)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.quiver(X, Y, Z, U, V, W, length=1, color='red')
ax.set_xlim([0, 1])
ax.set_ylim([0, 1])
ax.set_zlim([0, 1])
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
plt.show()

```

## 1.3 Scale the vectors you created in 1.1 by $5$, $\pi$, and $-e$ and plot all four vectors (original + 3 scaled vectors) on a graph. What do you notice about these vectors?

```
red = [1, 1]

blue = np.multiply(5, red)
green = np.multiply(np.pi, red)
yellow = np.multiply(-np.exp(1), red)

plt.arrow(0,0, red[0], red[1],head_width=.05, head_length=0.05, color ='red', width = 0.05)
plt.arrow(0,0, blue[0], blue[1],head_width=.05, head_length=0.05, color ='blue')
plt.arrow(0,0, green[0], green[1],head_width=.05, head_length=0.05, color ='green')
plt.arrow(0,0, yellow[0], yellow[1],head_width=.05, head_length=0.05, color ='yellow')

plt.xlim(-5,6)
plt.ylim(-5,6)
plt.title("Scaled Vectors")
plt.show()

# The vectors are all in one line, just different lengths.
# The -e is in the same line but pointed in opposite direction
```

## 1.4 Graph vectors $\vec{a}$ and $\vec{b}$ and plot them on a graph

\begin{align}
\vec{a} = \begin{bmatrix} 5 \\ 7 \end{bmatrix}
\qquad
\vec{b} = \begin{bmatrix} 3 \\4 \end{bmatrix}
\end{align}

```
a = [5, 7]
b = [3, 4]

plt.arrow(0,0, a[0], a[1],head_width=.05, head_length=0.05, color ='red')
plt.arrow(0,0, b[0], b[1],head_width=.05, head_length=0.05, color ='blue')

plt.xlim(0,6)
plt.ylim(0,8)
plt.title("Plot of a and b Vectors")
plt.show()
```

## 1.5 find $\vec{a} - \vec{b}$ and plot the result on the same graph as $\vec{a}$ and $\vec{b}$. Is there a relationship between vectors $\vec{a} \thinspace, \vec{b} \thinspace \text{and} \thinspace \vec{a-b}$

```
a = np.array([5, 7])
b = np.array([3, 4])

a_minus_b = np.subtract(a, b)

plt.arrow(0,0, a[0], a[1],head_width=.05, head_length=0.05, color ='red')
plt.arrow(0,0, b[0], b[1],head_width=.05, head_length=0.05, color ='blue')
plt.arrow(0,0, a_minus_b[0], a_minus_b[1],head_width=.05, head_length=0.05, color ='green')

plt.xlim(0,6)
plt.ylim(0,8)
plt.title("a Minus b in Green")
plt.show()

# The difference between a and b is intuitively represented by the green vector
# To get from the tip of blue (b) to the tip of red (a), you go over two and up 3
# This is exactly the green (a minus b) vector
```

## 1.6 Find $c \cdot d$

\begin{align}
\vec{c} = \begin{bmatrix}7 & 22 & 4 & 16\end{bmatrix}
\qquad
\vec{d} = \begin{bmatrix}12 & 6 & 2 & 9\end{bmatrix}
\end{align}

```
c = np.array([7, 22, 4, 16])
d = np.array([12, 6, 2, 9])

print('The dot product of c and d:')
np.dot(c, d)
```

    The dot product of c and d:

    368

##  1.7 Find $e \times f$

\begin{align}
\vec{e} = \begin{bmatrix} 5 \\ 7 \\ 2 \end{bmatrix}
\qquad
\vec{f} = \begin{bmatrix} 3 \\4 \\ 6 \end{bmatrix}
\end{align}

```
e = np.array([5, 7, 2])
f = np.array([3, 4, 6])

print('The cross product of e and f:')
np.cross(e, f)
```

    The cross product of e and f:

    array([ 34, -24,  -1])

## 1.8 Find $||g||$ and then find $||h||$. Which is longer?

\begin{align}
\vec{g} = \begin{bmatrix} 1 \\ 1 \\ 1 \\ 8 \end{bmatrix}
\qquad
\vec{h} = \begin{bmatrix} 3 \\3 \\ 3 \\ 3 \end{bmatrix}
\end{align}

```
g = np.array([1, 1, 1, 8])
h = np.array([3, 3, 3, 3])

print('Norm of g:')
print(np.linalg.norm(g))
print('\n')
print('Norm of h:')
np.linalg.norm(h)

#The Norm of g is longer
```

    Norm of g:
    8.18535277187245

    Norm of h:

    6.0

## 1.9 Show that the following vectors are orthogonal (perpendicular to each other):

\begin{align}
\vec{i} = \begin{bmatrix} 1 \\ 0 \\ -1  \end{bmatrix}
\qquad
\vec{j} = \begin{bmatrix} 1 \\ \sqrt{2} \\ 1 \end{bmatrix}
\end{align}

```
i = np.array([1, 0, -1])
j = np.array([1, np.sqrt(2), 1])

np.dot(i, j)

# These two vectors are orthogonal because their dot product equals zero.
# This means that these two vectors are linearly independent
# No way to scale one to get the other -- on different planes
```

    0.0

# Part 2 - Matrices

## 2.1 What are the dimensions of the following matrices? Which of the following can be multiplied together? See if you can find all of the different legal combinations.
\begin{align}
A = \begin{bmatrix}
1 & 2 \\
3 & 4 \\
5 & 6
\end{bmatrix}
\qquad
B = \begin{bmatrix}
2 & 4 & 6 \\
\end{bmatrix}
\qquad
C = \begin{bmatrix}
9 & 6 & 3 \\
4 & 7 & 11
\end{bmatrix}
\qquad
D = \begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
\qquad
E = \begin{bmatrix}
1 & 3 \\
5 & 7
\end{bmatrix}
\end{align}

```
# Matrices Dimensions
# A = 3x2
# B = 1x3
# C = 2x3
# D = 3x3
# E = 2x2

# A*C, A*E
# B*D
# C*A, C*D
# D*A
# E*C

# There are 7 possible combinations of multiplication

```

## 2.2 Find the following products: CD, AE, and BA. What are the dimensions of the resulting matrices? How does that relate to the dimensions of their factor matrices?

```
A = np.array([[1, 2],
              [3, 4],
              [5, 6]])

B = np.array([[2, 4, 6]])

C = np.array([[9, 6, 3],
              [4, 7, 11]])

D = np.array([[1, 0, 0],
              [0, 1, 0],
              [0, 0, 1]])

E = np.array([[1, 3],
              [5, 7]])
```

```
# CD
np.matmul(C, D)
```

    array([[ 9,  6,  3],
           [ 4,  7, 11]])

```
#AE
np.matmul(A, E)
```

    array([[11, 17],
           [23, 37],
           [35, 57]])

```
#BA
np.matmul(B, A)
```

    array([[44, 56]])

## 2.3  Find $F^{T}$. How are the numbers along the main diagonal (top left to bottom right) of the original matrix and its transpose related? What are the dimensions of $F$? What are the dimensions of $F^{T}$?

\begin{align}
F =
\begin{bmatrix}
20 & 19 & 18 & 17 \\
16 & 15 & 14 & 13 \\
12 & 11 & 10 & 9 \\
8 & 7 & 6 & 5 \\
4 & 3 & 2 & 1
\end{bmatrix}
\end{align}

```
F = np.array([[20, 19, 18, 17],
              [16, 15, 14, 13],
              [12, 11, 10, 9],
              [8, 7, 6, 5],
              [4, 3, 2, 1]])
```

```
F.T
```

    array([[20, 16, 12,  8,  4],
           [19, 15, 11,  7,  3],
           [18, 14, 10,  6,  2],
           [17, 13,  9,  5,  1]])

```
# The main diagonal of F and F^t are the same
# The the other numbers in F^t are flipped around the main diagonal
# The dimension of F is 5x4, while the dimension of F is 4x5
```

# Part 3 - Square Matrices

## 3.1 Find $IG$ (be sure to show your work) 😃

\begin{align}
G=
\begin{bmatrix}
12 & 11 \\
7 & 10
\end{bmatrix}
\end{align}

```
G = np.array([[12, 11],
              [7, 10]])
```

```
# Identity matrix
I = np.array([[1, 0],
              [0, 1]])
```

```
# The product of any matrix with its Identity Matrix is just the original matrix
np.matmul(G, I)
```

    array([[12, 11],
           [ 7, 10]])

## 3.2 Find $|H|$ and then find $|J|$.

\begin{align}
H=
\begin{bmatrix}
12 & 11 \\
7 & 10
\end{bmatrix}
\qquad
J=
\begin{bmatrix}
0 & 1 & 2 \\
7 & 10 & 4 \\
3 & 2 & 0
\end{bmatrix}
\end{align}

```
H = np.array([[12, 11],
              [7, 10]])

J = np.array([[0, 1, 2],
              [7, 10, 4],
              [3, 2, 0]])
```

```
# Determinant of H

np.linalg.det(H)
```

    43.000000000000014

```
# Determinant of J

np.linalg.det(J)
```

    -19.999999999999996

## 3.3 Find H^{-1} and then find J^{-1}

```
# Inverse of H

np.linalg.inv(H)
```

    array([[ 0.23255814, -0.25581395],
           [-0.1627907 ,  0.27906977]])

```
# Inverse of J

np.linalg.inv(J)
```

    array([[ 0.4 , -0.2 ,  0.8 ],
           [-0.6 ,  0.3 , -0.7 ],
           [ 0.8 , -0.15,  0.35]])

## 3.4 Find $HH^{-1}$ and then find $G^{-1}G$. Is $HH^{-1} == G^{-1}G$? Why or Why not?

```
#HH^-1

np.matmul(H, np.linalg.inv(H))
```

    array([[1.00000000e+00, 5.55111512e-16],
           [2.22044605e-16, 1.00000000e+00]])

```
#G^-1G

np.matmul(np.linalg.inv(G), G)
```

    array([[1.00000000e+00, 6.66133815e-16],
           [1.11022302e-16, 1.00000000e+00]])

```
# The two above results are the same because multiplying any matrix by its
# inverse will give you an indentiy matrix (i.e. 1's on main diagonal and 0's everywhere else)
```

# Stretch Goals:

A reminder that these challenges are optional. If you finish your work quickly we welcome you to work on them. If there are other activities that you feel like will help your understanding of the above topics more, feel free to work on that. Topics from the Stretch Goals sections will never end up on Sprint Challenges. You don't have to do these in order, you don't have to do all of them.

- Write a function that can calculate the dot product of any two vectors of equal length that are passed to it.
- Write a function that can calculate the norm of any vector
- Prove to yourself again that the vectors in 1.9 are orthogonal by graphing them.
- Research how to plot a 3d graph with animations so that you can make the graph rotate (this will be easier in a local notebook than in google colab)
- Create and plot a matrix on a 2d graph.
- Create and plot a matrix on a 3d graph.
- Plot two vectors that are not collinear on a 2d graph. Calculate the determinant of the 2x2 matrix that these vectors form. How does this determinant relate to the graphical interpretation of the vectors?

```
# Dot Product function

def dot_product(v1, v2):
  dotproduct = 0
  for i,j in zip(v1, v2):
    dotproduct += i * j

  return dotproduct
```

```
# Using same vectors above (c and d) to calculate dot product new function

dot_product(c, d)

# Same result as above with numpy
```

    368

```

```


# Softening function and damage function

```python
%reset -f
import sympy as sp
from sympy.plotting import plot as splot
sp.init_printing()
```

Let us consider a softening function of the form
\begin{align}
f(w) = c_1 exp( - c_2 w )
\end{align}
This is function should describe the decay of stress starting from the material tensile strength and continuously deminishing to zero.
The variable $w$ represents the crack opening and the parameters $c_1$ and $c_2$ are the material parameters.

```python
c_1, c_2, w = sp.symbols('c_1,c_2,w')
```

```python
f = c_1 * sp.exp(-c_2*w)
```

Let us plot the function to verify its shape for the material parameters set to the value 1

```python
f_w = f.subs({'c_1':1, 'c_2':1})
```

```python
splot(f_w, (w, 0, 10))
```

The function can be already used in this form. The question is however, how to set the material parameters $c_1$ and $c_2$. They can be directly associated to a particular type of material parameters - namely, to the tensile strength $f_\mathrm{t}$ and fracture energy $G_\mathrm{f}$

```python
f_t, G_f = sp.symbols('f_t, G_f')
```

Softening starts at the level of the material strength so that we can set $f(w = 0) = f_\mathrm{t}$  to obtain the equation

```python
Eq1 = f.subs({'w':0}) - f_t
Eq1
```

By solving for the $c_1$ we obtain the first substitution for our softening function.

```python
c_1_subs = sp.solve({Eq1}, c_1)
c_1_subs
```

Thus, $c_1$ is equivalent to the tensile strength $f_\mathrm{t}$

The second possible mechanical interpretation is provided by the statement that softening directly represents the energy dissipation of a unit crack area. Thus, for large $w \rightarrow \infty$ it is equivalent to the energy producing a stress-free crack. This is the meaning of fracture energy.

We can thus obtain the fracture energy represented by the softening function by evaluating its integral in the range $w \in (0, \infty)$.

```python
int_f_w = sp.integrate(f.subs(c_1_subs), w)
int_f_w
```

As $c_2 > 0$, only the second term matters.
The determinate integral
\begin{align}
\left[ - \frac{f_\mathrm{t}}{c_2}
\exp(-c_2 w) \right]_0^{\infty}
\end{align}
is zero for $w = \infty$, so that the value in $w = 0$ delivers the result of the integral
\begin{align}
\frac{f_\mathrm{t}}{c_2}
\end{align}

This integral is equal to the fracture energy $G_\mathrm{f}$.

```python
Eq2 = -int_f_w.args[1][0].subs({'w':0}) - G_f
Eq2
```

and the value of $c_2$ delivers the second substitution for the softening function

```python
c_2_subs = sp.solve({Eq2}, c_2)
c_2_subs
```

The softening function with strength and fracture energy as a parameter now obtains the form

```python
f_w = f.subs(c_1_subs).subs(c_2_subs)
f_w
```

Verify that the fracture energy is recovered at $w$ in infinity

```python
sp.integrate(f_w, w)
```

```python

```

## How to apply the softening function to a zone?

If we wish to embed the softening behavior into a finite element simulation we encounter the problem that the deformation is not described by the crack opening $w$. The finite element discretization assumes by definition a smooth stran field $verepilon$ and there is no notion of discontinuity.

As a consequence, we need to account for a length of the softening zone  $L_s$ which is actually equivalent to the size of the finite element.

```python
L_s, epsilon = sp.symbols('L_s, varepsilon')
```

The crack opening $w$ is simply substituted by
\begin{align}
 w = \varepsilon L_\mathrm{s}
\end{align}
to obtain the softening function in terms of strains

```python
f_epsilon = f_w.subs({'w' : epsilon * L_s})
f_epsilon
```

Note the important fact that the integral of the softening function over the strains scales the dissipated energy by the term $1/L_\mathrm{s}$.

```python
sp.integrate(f_epsilon, epsilon)
```

As a consequence, by changing the size of the softening zone we also change the total amount of the energy dissipated!!! This feature of the softening function will be exploited in the implementation of the finite elements.

## How to convert a softening function to damage function?

The shape of the softening function describes the process of deterioration - starting from an undamaged state and ending in a fully damage state. Let us establish an equivalence between softening and damage evolution by requiring that they describe the same kind of stress decay.

Considering the softening zone of the length $L_s$ let us describe the damage within this zone using the state variable $omega$ and the elastic modulus $E_c$

```python
omega, E_c = sp.symbols('omega, E_c')
```

Then, the constitute law to be used in a finite element of the zone is given as

```python
sigma = (1 - omega) * E_c * epsilon
```

The stress decay described by the damage law and by the softening law should be the same. Let them set equal and solve for the damage variable $omega$. Using the sympy solver we obtain the algebraic solution as

```python
omega_Gf = sp.solve(sigma - f_epsilon, omega)[0]
omega_Gf
```

This new damage function is defined using material parameters with a clear mechanical interpretation. Such kind of material law is attractive because it makes it possible to design tests that focus on an isolated phenomenon, i.e. the determination of the material strength, E modulus or fracture energy.

Let us now visually verify the shape of the damage function.
The set of parameters is assembled in a dictionary and then
they are all substituted into the damage function.

```python
data_f = dict(E_c = 28000, f_t = 3, L_s = 1, G_f = 0.01)
omega_Gf_epsilon = omega_Gf.subs(data_f)
omega_Gf_epsilon
```

The damage function is only valid in an inelastic regime. Therefore, we have to quantify the onset of inelasticity first as
\begin{align}
\varepsilon_0 = \frac{f_t}{E_c}
\end{align}

```python
epsilon_0 = (f_t / E_c).subs(data_f)
epsilon_0
```

Then the damage function can be plotted as

```python
splot(omega_Gf_epsilon, (epsilon,epsilon_0,epsilon_0*100))
```

The corresponding stress strain curve has then the form

```python
sigma = sp.Piecewise( (E_c * epsilon, epsilon < epsilon_0),
    (( 1 - omega_Gf ) * E_c * epsilon, epsilon >= epsilon_0))
sigma
```

```python
splot(sigma.subs(data_f), (epsilon,0,epsilon_0*100))
```

Note how the stress strain function scales with the change of the fracture energy and of the zone length.

Larger fracture energy makes the stress-strain response more ductile, while smaller makes it brittle.

On the other hand, larger size of the softening zone makes the softening behavior more brittle and smaller size of the zone makes it more ductile. Why?

```python
g_f = sp.integrate(sigma, epsilon)
```

```python

```

```python

```


> This is one of the 100 recipes of the [IPython Cookbook](http://ipython-books.github.io/), the definitive guide to high-performance scientific computing and data science in Python.

# 15.7. Analyzing a nonlinear differential system: Lotka-Volterra (predator-prey) equations

Here, we conduct a brief analytical study of a famous nonlinear differential system: the Lotka-Volterra equations, also known as predator-prey equations. This simple model describes the evolution of two interacting populations (e.g. sharks and sardines), where the predators eat the preys. This example illustrates how we can use SymPy to obtain exact expressions and results for fixed points and their stability.

```python
from sympy import *
init_printing()
```

```python
var('x y')
var('a b c d', positive=True)
```

The variables x and y represent the populations of the preys and predators, respectively. The parameters a, b, c and d are positive parameters (described more precisely in "How it works..."). The equations are:

$$\begin{align}
\frac{dx}{dt} &= f(x) = x(a-by)\\
\frac{dy}{dt} &= g(x) = -y(c-dx)
\end{align}$$

```python
f = x * (a - b*y)
g = -y * (c - d*x)
```

Let's find the fixed points of the system (solving f(x,y) = g(x,y) = 0).

```python
solve([f, g], (x, y))
```

```python
(x0, y0), (x1, y1) = _
```

Let's write the 2D vector with the two equations.

```python
M = Matrix((f, g)); M
```

Now we can compute the Jacobian of the system, as a function of (x, y).

```python
J = M.jacobian((x, y)); J
```

Let's study the stability of the two fixed points by looking at the eigenvalues of the Jacobian at these points.

```python
M0 = J.subs(x, x0).subs(y, y0); M0
```

```python
M0.eigenvals()
```

The parameters a and c are strictly positive, so the eigenvalues are real and of opposite signs, and this fixed point is a saddle point. Since this point is unstable, the extinction of both populations is unlikely in this model.

```python
M1 = J.subs(x, x1).subs(y, y1); M1
```

```python
M1.eigenvals()
```

The eigenvalues are purely imaginary so this fixed point is not hyperbolic, and we cannot draw conclusions about the qualitative behavior of the system around this fixed point from this linear analysis. However, one can show with other methods that oscillations occur around this point.

> You'll find all the explanations, figures, references, and much more in the book (to be released later this summer).

> [IPython Cookbook](http://ipython-books.github.io/), by [Cyrille Rossant](http://cyrille.rossant.net), Packt Publishing, 2014 (500 pages).


```python
import numpy as np
import matplotlib.pyplot as plt
import sympy as sy
import simtk.unit as unit
```

# The Lennard-Jones potential

The Lennard-Jones (LJ) potential between two particles is defined by the following equation, where $x$ is the distance between the particles, and $\sigma$ and $\epsilon$ are two parameters of the potential:

\begin{equation}
V(x) = 4 \epsilon \left[ \left( \frac{\sigma}{x} \right)^{12} - \left( \frac{\sigma}{x} \right)^6 \right]
\end{equation}

Lets see the shape of this function:

```python
def LJ (x, sigma, epsilon):

    t = sigma/x
    t6 = t**6
    t12 = t6**2

    return 4.0*epsilon*(t12-t6)
```

```python
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

xlim_figure = [0.01, 6.0]
ylim_figure = [-2.0, 10.0]

x = np.linspace(xlim_figure[0], xlim_figure[1], 100, True) * unit.angstrom
plt.plot(x, LJ(x, sigma, epsilon))
plt.xlim(xlim_figure)
plt.ylim(ylim_figure)
plt.xlabel('x [{}]'.format(x.unit.get_symbol()))
plt.ylabel('V [{}]'.format(epsilon.unit.get_symbol()))
plt.show()
```

The way the LJ potential is built, the $\sigma$ and $\epsilon$ parameters have a straightforward interpretation. The cut with $y=0$ is located in $x=\sigma$:

```python
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

xlim_figure = [0.01, 6.0]
ylim_figure = [-2.0, 10.0]

x = np.linspace(xlim_figure[0], xlim_figure[1], 100, True) * unit.angstrom
plt.plot(x, LJ(x, sigma, epsilon))
plt.hlines(0, xlim_figure[0], xlim_figure[1], linestyles='dotted', color='gray')
plt.vlines(sigma._value, ylim_figure[0], ylim_figure[1], linestyles='dashed', color='red')
plt.text(sigma._value+0.02*xlim_figure[1], 0.7*ylim_figure[1], '$\sigma$', fontsize=14)
plt.xlim(xlim_figure)
plt.ylim(ylim_figure)
plt.xlabel('x [{}]'.format(x.unit.get_symbol()))
plt.ylabel('V [{}]'.format(epsilon.unit.get_symbol()))
plt.show()
```

And $\epsilon$ is the depth of the minimum measured from $y=0$:

```python
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

xlim_figure = [0.01, 6.0]
ylim_figure = [-2.0, 10.0]

x = np.linspace(xlim_figure[0], xlim_figure[1], 100, True) * unit.angstrom
plt.plot(x, LJ(x, sigma, epsilon))
plt.hlines(0, xlim_figure[0], xlim_figure[1], linestyles='dotted', color='gray')
plt.hlines(-epsilon._value, xlim_figure[0], xlim_figure[1], linestyles='dashed', color='red')
plt.annotate(text='', xy=(1.0,0.0), xytext=(1.0,-epsilon._value), arrowprops=dict(arrowstyle='<->'))
plt.text(1.0+0.02*xlim_figure[1], -0.7*epsilon._value, '$\epsilon$', fontsize=14)
plt.xlim(xlim_figure)
plt.ylim(ylim_figure)
plt.xlabel('x [{}]'.format(x.unit.get_symbol()))
plt.ylabel('V [{}]'.format(epsilon.unit.get_symbol()))
plt.show()
```

Notice that the LJ potential has physical meaning when $\epsilon>0$ and $\sigma>0$ only. Actually, the potential vanishes whether $\epsilon=0$ or $\sigma=0$.

## The Lennard Jones minimum and the size of the particles

The LJ potential has a single minimum located in $x_{min}$. Lets equal to $0$ the first derivative of the potential to find the value of $x_{min}$:

```python
x, sigma, epsilon = sy.symbols('x sigma epsilon', real=True, positive=True)
V = 4.0*epsilon*((sigma/x)**12-(sigma/x)**6)
gradV = sy.diff(V,x)
roots=sy.solve(gradV, x)
x_min = roots[0]
```

```python
x_min
```

$\displaystyle 1.12246204830937 \sigma$

The minimum is then located in:

\begin{equation}
x_{min} = 2^{1/6} \sigma
\end{equation}

where the potential takes the value:

\begin{equation}
V(x_{min}) = -\epsilon
\end{equation}

```python
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

x_min = 2**(1/6)*sigma
y_min = -epsilon

xlim_figure = [x_min._value-0.4, x_min._value+0.4]
ylim_figure = [y_min._value-0.1, y_min._value+0.5]

x = np.linspace(xlim_figure[0], xlim_figure[1], 100, True) * unit.angstroms
plt.plot(x, LJ(x, sigma, epsilon))
plt.hlines(y_min._value, xlim_figure[0], xlim_figure[1], linestyles='dashed', color='gray')
plt.vlines(x_min._value, ylim_figure[0], ylim_figure[1], linestyles='dashed', color='gray')
plt.xlim(xlim_figure)
plt.ylim(ylim_figure)
plt.xlabel('x [{}]'.format(x.unit.get_symbol()))
plt.ylabel('V [{}]'.format(epsilon.unit.get_symbol()))
plt.show()
```

This way two particles in the equilibrium position will be placed at a $2^{1/6} \sigma$ distance. The potential is thereby modeling two "soft spheres" atracting each other very lightly. Their radii, given that both particles are equal, are equal to $r$:

\begin{equation}
r = \frac{1}{2} x_{min} = 2^{-5/6} \sigma
\end{equation}

And we say these spheres are "soft" because their volume is not limited by a hard-wall potential, they can penetrate each other suffering a not infinite repulsive force.

## Time period of the small harmonic oscillations around the minimum

If we want to perform a molecular simulation of this two particles we should wonder how big the integrator timestep must be. To answer this question we can study the harmonic approximation around the minimum. Lets calculate the time period, $\tau$, of a small harmonic oscillation around the minimum:

```python
x, sigma, epsilon = sy.symbols('x sigma epsilon', real=True, positive=True)
V = 4.0*epsilon*((sigma/x)**12-(sigma/x)**6)
gradV = sy.diff(V,x)
grad2V = sy.diff(V,x,x)

x_min = sy.solve(gradV,x)[0]
k_harm = grad2V.subs(x, x_min)
```

```python
k_harm
```

$\displaystyle \frac{57.1464378708551 \epsilon}{\sigma^{2}}$

The harmonic constant of the second degree Taylor polynomial of the LJ potential at $x=x_{min}$ is then:

\begin{equation}
k_{harm} = 36·2^{2/3} \frac{\epsilon}{\sigma^2}
\end{equation}

The oscillation period of a particle with $m$ mass in an harmonic potential defined by $\frac{1}{2} k x²$ is:

\begin{equation}
\tau = 2 \pi \sqrt{ \frac{m}{k}}
\end{equation}

As such, the period of the small harmonic oscillations around the LJ minimum of particle with $m$ mass is:

\begin{equation}
\tau = 2 \pi \sqrt{ \frac{m}{k_{harm}}} = \frac{\pi}{3·2^{1/3}} \sqrt{\frac{m\sigma^2}{\epsilon}}
\end{equation}

With the mass and parameters taking values of amus, angstroms and kilocalories per mole, the time period is in the order of:

```python
mass = 50.0 * unit.amu
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

k = 36 * 2**(2/3) * epsilon/sigma**2

tau = 2*np.pi * np.sqrt(mass/k)

print(tau)
```

    0.5746513694274475 ps

But, is this characteristic time a good threshold for a LJ potential? If the oscillations around the minimum are not small enough, the harmonic potential of the second degree term of the taylor expansion is easily overcome by the sharp left branch of the LJ potential:

```python
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

k = 36 * 2**(2/3) * epsilon/sigma**2

x_min = 2**(1/6)*sigma
y_min = -epsilon

xlim_figure = [x_min._value-0.2, x_min._value+0.2]
ylim_figure = [y_min._value-0.1, y_min._value+0.6]

x = np.linspace(xlim_figure[0], xlim_figure[1], 100, True) * unit.angstroms
plt.plot(x, LJ(x, sigma, epsilon))
plt.plot(x, 0.5*k*(x-x_min)**2+y_min)
plt.hlines(y_min._value, xlim_figure[0], xlim_figure[1], linestyles='dashed', color='gray')
plt.vlines(x_min._value, ylim_figure[0], ylim_figure[1], linestyles='dashed', color='gray')
plt.xlim(xlim_figure)
plt.ylim(ylim_figure)
plt.xlabel('x [{}]'.format(x.unit.get_symbol()))
plt.ylabel('V [{}]'.format(epsilon.unit.get_symbol()))
plt.show()
```

Let's imagine the following situation. Let a particle be in the harmonic potential at temperature of 300K. Will the particle be more constrained in space than in the well of the LJ potential? Will the particle feel the harmonic potential softer or sharper than the LJ? Lets make some numbers to evaluate if the oscillation time period of the harmonic approximation can be a good time threshold for the integration timestep of a molecular dynamics of the LJ potential.

The standard deviation of an harmonic oscillation with the shape $\frac{1}{2}k x^2$ in contact with a stochastic thermal bath can be computed as:

\begin{equation}
\beta = \frac{1}{k_{\rm B} T}
\end{equation}

\begin{equation}
Z_x = \int_{-\infty}^{\infty} {\rm e}^{- \beta \frac{1}{2}k x^2} = \sqrt{\frac{2 \pi}{\beta k}}
\end{equation}

\begin{equation}
\left< x \right> = \frac{1}{Z_x} \int_{-\infty}^{\infty} x {\rm e}^{-\beta \frac{1}{2}k x^2} = 0
\end{equation}

\begin{equation}
\left< x^2 \right> = \frac{1}{Z_x} \int_{-\infty}^{\infty} x^{2} {\rm e}^{-\beta \frac{1}{2}k x^2} = \frac{1}{Z_x}  \sqrt{\frac{2 \pi}{\beta³ k^3}} = \frac{1}{\beta k}
\end{equation}

\begin{equation}
{\rm std} = \left( \left< x^2 \right> -\left< x \right>^2 \right)^{1/2} = \sqrt{ \frac{k_{\rm B}T}{k} }
\end{equation}

This way, in the case of the harmonic potential obtained as the second degree term of the Taylor expansion around the LJ minimum:

```python
mass = 50.0 * unit.amu
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole
temperature = 300 * unit.kelvin
kB = unit.BOLTZMANN_CONSTANT_kB * unit.AVOGADRO_CONSTANT_NA

k = 36 * 2**(2/3) * epsilon/sigma**2
std = np.sqrt(kB*temperature/k)

x_min = 2**(1/6)*sigma
y_min = -epsilon

xlim_figure = [x_min._value-0.4, x_min._value+0.4]
ylim_figure = [y_min._value-0.1, y_min._value+0.6]

x = np.linspace(xlim_figure[0], xlim_figure[1], 100, True) * unit.angstroms
plt.plot(x, LJ(x, sigma, epsilon))
plt.plot(x, 0.5*k*(x-x_min)**2+y_min)
plt.hlines(y_min._value, xlim_figure[0], xlim_figure[1], linestyles='dashed', color='gray')
plt.vlines(x_min._value, ylim_figure[0], ylim_figure[1], linestyles='dashed', color='gray')
plt.axvspan(x_min._value - std._value, x_min._value + std._value, alpha=0.2, color='red')
plt.annotate(text='', xy=(x_min._value, y_min._value - 0.5*(y_min._value-ylim_figure[0])),
             xytext=(x_min._value-std._value, y_min._value - 0.5*(y_min._value-ylim_figure[0])),
             arrowprops=dict(arrowstyle='<->'))
plt.text(x_min._value-0.6*std._value, y_min._value - 0.4*(y_min._value-ylim_figure[0]), '$std$', fontsize=14)
plt.xlim(xlim_figure)
plt.ylim(ylim_figure)
plt.xlabel('x [{}]'.format(x.unit.get_symbol()))
plt.ylabel('V [{}]'.format(epsilon.unit.get_symbol()))
plt.show()
```

The harmonic potential is too soft as approximation. Its oscillation time used as threshold to choose the integration timestep can yield to numeric problems. Let's try with a stiffer potential, let's double the harmonic constant:

```python
mass = 50.0 * unit.amu
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole
temperature = 300 * unit.kelvin
kB = unit.BOLTZMANN_CONSTANT_kB * unit.AVOGADRO_CONSTANT_NA

k = 36 * 2**(2/3) * epsilon/sigma**2
std = np.sqrt(kB*temperature/k)

x_min = 2**(1/6)*sigma
y_min = -epsilon

xlim_figure = [x_min._value-0.4, x_min._value+0.4]
ylim_figure = [y_min._value-0.1, y_min._value+0.6]

x = np.linspace(xlim_figure[0], xlim_figure[1], 100, True) * unit.angstroms
plt.plot(x, LJ(x, sigma, epsilon))
plt.plot(x, 0.5*k*(x-x_min)**2+y_min)
plt.plot(x, k*(x-x_min)**2+y_min, label='2k_{harm}')
plt.hlines(y_min._value, xlim_figure[0], xlim_figure[1], linestyles='dashed', color='gray')
plt.vlines(x_min._value, ylim_figure[0], ylim_figure[1], linestyles='dashed', color='gray')
plt.axvspan(x_min._value - std._value, x_min._value + std._value, alpha=0.2, color='red')
plt.annotate(text='', xy=(x_min._value, y_min._value - 0.5*(y_min._value-ylim_figure[0])),
             xytext=(x_min._value-std._value, y_min._value - 0.5*(y_min._value-ylim_figure[0])),
             arrowprops=dict(arrowstyle='<->'))
plt.text(x_min._value-0.6*std._value, y_min._value - 0.4*(y_min._value-ylim_figure[0]), '$std$', fontsize=14)
plt.xlim(xlim_figure)
plt.ylim(ylim_figure)
plt.xlabel('x [{}]'.format(x.unit.get_symbol()))
plt.ylabel('V [{}]'.format(epsilon.unit.get_symbol()))
plt.show()
```

Lets take then, as reference, an harmonic potential with constant equal to $2k_{harm}$ could be a better idea. Lets compute then the new time threshold to choose the integration timestep:

\begin{equation}
\tau' = 2 \pi \sqrt{ \frac{m}{2k_{harm}}} = \frac{\pi}{3·2^{5/6}} \sqrt{\frac{m\sigma^2}{\epsilon}} = \frac{1}{\sqrt{2}} \tau
\end{equation}

```python
mass = 50.0 * unit.amu
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

k = 36 * 2**(2/3) * epsilon/sigma**2

tau = 2*np.pi * np.sqrt(mass/(2*k))

print(tau)
```

    0.4063398801402841 ps

It is an accepted rule of thumb that the integration timestep must be as large as $\tau / 10$, being $\tau$ the oscillation time period of the fastest possible vibration mode. So finally, in this case the integration time step should not be longer than:

```python
mass = 50.0 * unit.amu
sigma = 2.0 * unit.angstrom
epsilon = 1.0 * unit.kilocalories_per_mole

k = 36 * 2**(2/3) * epsilon/sigma**2

tau = 2*np.pi * np.sqrt(mass/(2*k))

print(tau/10.0)
```

    0.04063398801402841 ps


#### Distribución Uniforme

##### Distribución Uniforme Discreta

Decimos que una variable aleatoria X tiene una distribución uniforme discreta sobre el conjunto de n números {x1, x2, ..., xn} si **la probabilidad de que X tome cualquiera de estos valores es constante 1/n**.

Esta distribución surge en espacios de probabilidad **equiprobables**, esto es, en situaciones en donde tenemos n resultados diferentes y todos ellos tienen la misma probabilidad de ocurrir.

Se escribe

$X \sim \text{unif}\{x_1, x_2, ..., x_n\} $

en donde el símbolo "$\sim$" se lee "se distribuye como" o "tiene una distribución".

La distribución de probabilidad de esa variable aleatoria es:

\begin{equation}
  f(x)=\begin{cases}
    1/n, & \text{si x = $x_1, x_2, ..., x_n$}\\
    0, & \text{en otro caso}.
  \end{cases}
\end{equation}

\begin{equation}
n = 5  \text{    donde    } n = b − a + 1
\end{equation}

**Ejemplos**:

* X: puntuación en el lanzamiento de un dado regular

* X: resultado del lanzamiento de una moneda

* X: resultado de un juego de lotería

#### Distribución Uniforme Continua

Decimos que una variable aleatoria X tiene una distribución uniforme continua en el intervalo (a,b), donde a y b son números reales, si su función de densidad es

\begin{equation}
  f_X(x)=\begin{cases}
    \frac{1}{b-a}, & \text{si $a \lt x \lt b$} \\
    0, & \text{en otro caso}.
  \end{cases}
\end{equation}

a y b son los parámetros de la distribución uniforme continua.

La distribución uniforme asigna probabilidad positiva constante y mayor a cero sólo a valores de la variable aleatoria en determinado rango.

\begin{equation}
  F_X(x)=\begin{cases}
    0, & \text{si $x \lt a$} \\
    \frac{x - a}{b - a}, & \text{si $a \le x \lt b$} \\
    1, & \text{si $x \ge b$}
  \end{cases}
\end{equation}

**Ejemplos**:

* Una llamada telefónica llegó a in conmutador en un tiempo, al azar, dentro de un período de 1 minuto. El conmutador estuvo ocupado durante 15 segundos en ese minuto. ¿Cuál es la probabilidad de que la llamada haya llegado mientras el conmutador no estuvo ocupado?

* Dos amigos deben encontrarse en una parada de colectivo entre las 9:00 y las 10:00. Cada uno esperará un máximo de 10 minutos. ¿Cuál es la probabilidad de que no se encuentren si el amigo1 llegará a las 9:30?

---

Vamos a ver ahora cómo generar datos con estas distibuciones de probabilidad.

Necesitamos un generador de números aleatorios, que expone métodos para generar números aleatorios con alguna distribución de probabilidad especificada. Construimos este generador de este modo `np.random.default_rng()`

https://docs.scipy.org/doc/numpy/reference/random/generator.html

Estas son las distribuciones de probabilidad disponibles:
https://docs.scipy.org/doc/numpy/reference/random/generator.html#distributions

Para generar datos con distribución **uniforme discreta** emplearemos el método `choice`

https://docs.scipy.org/doc/numpy/reference/random/generated/numpy.random.Generator.choice.html

Para generar datos con distribución **uniforme continua** emplearemos el método `uniform`

https://docs.scipy.org/doc/numpy/reference/random/generated/numpy.random.Generator.uniform.html#numpy.random.Generator.uniform

```python
import numpy as np
random_generator = np.random.default_rng()
random_uniform_cont_data = random_generator.uniform(low = 3, high = 17, size = 30)

possible_values = [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
random_uniform_disc_data = random_generator.choice(possible_values, size = 30)
```

Usando la función `distribution_plotter` vamos a graficar los valores generados

```python
import seaborn as sns
def distribution_plotter(data, label, bins=None):
    sns.set(rc={"figure.figsize": (10, 7)})
    sns.set_style("white")
    dist = sns.distplot(data, bins= bins, hist_kws={'alpha':0.2}, kde_kws={'linewidth':5})
    dist.set_title('Distribucion de ' + label + '\n', fontsize=16)
```

```python
print(random_uniform_cont_data)
distribution_plotter(random_uniform_cont_data, "uniforme continua")
```

```python
print(random_uniform_disc_data)

distribution_plotter(random_uniform_disc_data, "uniforme discreta", bins=possible_values)
```

¿Estas distribuciones se parecen?

Prueben ahora generando, 100, 1000, 10000 y 100000 datos y comparen los gráficos. ¿Qué pueden concluir?

```python
# modificar este valor:
size_sample = 30

random_uniform_cont_data = random_generator.uniform(low = 3, high = 17, size = size_sample)

possible_values = [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
random_uniform_disc_data = random_generator.choice(possible_values, size = size_sample)

distribution_plotter(random_uniform_disc_data, "uniforme discreta")
distribution_plotter(random_uniform_cont_data, "uniforme continua")
```

#### Referencias

Gráficos: https://en.wikipedia.org/wiki/List_of_probability_distributions


# Some Linear Algebra Basics
## Python Imports
At the beginning of our Python source code we can import packages to extend the functionality of Python.

In this case we will be importing the NumPy library which supports *efficient* linear algebra calculations.

```
# Import the NumPy library for linear algebra

# we assign 'np' as an alias that we can use later when calling or using
# anything in the NumPy package
import numpy as np
```

## Summing a vector
The following cells will create a vector $$x = \begin{bmatrix}3&1&4&2\end{bmatrix}$$ and then calculate the sum of its elements, $$s = \sum_{i=1}^n x_i$$

```
# Create a vector, x

# np.array is used to define vectors or matrices in NumPy. Here we will
# initialize our arrays with python lists.
x = np.array([3, 1, 4, 2])

# Display x (using an f-string)
print(f"x =\n{x}")
```

    x =
    [3 1 4 2]

```
# Calculate the sum the elements of x
s = x.sum()

# Display the sum
print(f"Sum of elements of x = {s}")
```

    Sum of elements of x = 10

## Vector Dot-Product
In the next few cells we will create a new vector $y = \begin{bmatrix}1\\2\\3\\4\end{bmatrix}$.

Then we'll calculate the dot product of $x$ and $y$, $$d = xy = x \cdot y = \begin{bmatrix}3&1&4&2\end{bmatrix} \cdot \begin{bmatrix}1\\2\\3\\4\end{bmatrix}$$

*Please note: While linear algebra rules for the dot-product demand that number of columns in the first argument equal the number of rows in the second argument, NumPy will automatically transpose one of the vectors if two row vectors or two column vectors are passed. So, in NumPy, $\begin{bmatrix}3&1&4&2\end{bmatrix} \cdot \begin{bmatrix}1&2&3&4\end{bmatrix}$ or $\begin{bmatrix}3\\1\\4\\2\end{bmatrix} \cdot \begin{bmatrix}1\\2\\3\\4\end{bmatrix}$ will produce the same dot-product as shown above.*

```
# Create a column vector y
y = np.array([[1],
              [2],
              [3],
              [4]])

# Display our vectors
print(f"x =\n{x}\n\ny =\n{y}")
```

    x =
    [3 1 4 2]

    y =
    [[1]
     [2]
     [3]
     [4]]

```
# Calculate the dot product of x and y
d = x @ y

# Display the dot product
print(f"d = x . y = {d}")
```

    d = x . y = [25]

## Matrix Dot-Product
In the remaining cells we create two matrices $X = \begin{bmatrix}3&1\\4&2\end{bmatrix}$ and $Y = \begin{bmatrix}1&2\\3&4\end{bmatrix}$.

Then we'll calculate the dot product of $X$ and $Y$,$$D = XY = X\cdot Y = \begin{bmatrix}3&1\\4&2\end{bmatrix} \cdot \begin{bmatrix}1&2\\3&4\end{bmatrix}$$

```
# Create two matrices
X = np.array([[3, 1],
              [4, 2]])          # This definition is easier to read

Y = np.array([[1, 2], [3, 4]])  # This definition is more compact

# Display our matrices
print(f"X =\n{X}\n\nY =\n{Y}")
```

    X =
    [[3 1]
     [4 2]]

    Y =
    [[1 2]
     [3 4]]

```
# Calculate the dot product of X and Y
D = X @ Y

# Display the dot product
print(f"D = X . Y =\n{D}")
```

    D = X . Y =
    [[ 6 10]
     [10 16]]

## Exercises

### Exercise 1

Calculate and display $$\begin{bmatrix}3&1&4\end{bmatrix} \cdot \begin{bmatrix}1\\2\\3\end{bmatrix}$$

```
# Create the first vector

# Create the second vector

# Calculate the dot-product

# Display the dot-product

```

### Exercise 2

Calculate and display $$\begin{bmatrix}5&3\\1&2\end{bmatrix} \cdot \begin{bmatrix}7&4\\6&8\end{bmatrix}$$

```
# Create the first matrix

# Create the second matrix

# Calculate the dot-product

# Display the dot-product

```

### Exercise 3

Calculate and display $$\begin{align}x &= \begin{bmatrix}3&1&4\end{bmatrix}\\z &= \sum_{i=1}^n x_i\end{align}$$

```
# Create the vector
# x = ...

# Calculate the sum
# s = ...

# Display the dot-product
# print(f"... = {...}")
```


```python
import numpy as np
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import sympy as sym
from sympy.plotting import plot
import pandas as pd
from IPython.display import display
from IPython.core.display import Math
```

```python
# EXERCISE 1.Find the extrema in the function f(x)=x^3-7.5x^2+18x-10
# analytically and determine if they are minimum or maximum.

a = sym.symbols('a')
V = a**3 - 7.5*a**2 + 18*a - 10
Vdiff = sym.expand(sym.diff(V))
roots = sym.solve(Vdiff)
display(Math(sym.latex('Roots:') + sym.latex(roots)))
roots = np.asarray(roots)
```

$$Roots:\left [ 2.0, \quad 3.0\right ]$$

```python
def f(x):
    return x**3 - 7.5*x**2 + 18*x - 10

x = sym.symbols('x') # x como simbolo para realizar a derivada

fdiff = sym.diff(f(x),x)  #derivada

fdiff2 = sym.diff(fdiff, x)

for i in range(len(roots)):

    while fdiff2.evalf(subs ={x: roots[i]}) == 0: # fdiff.evalf(subs ={x: 0}) colocar 0 no lugar do simbolo x // caso a derivada der 0
        fdiff2 = sym.diff(fdiff2,x)

    print(fdiff2.evalf(subs ={x: roots[i]}))  # mostrar o valor da derivada se =/= 0

    if fdiff2.evalf(subs ={x: roots[i]}) <= 0: # determinar maxima ou minima
        print('maxima')
    else:
        print('minima')
```

    -3.00000000000000
    maxima
    3.00000000000000
    minima

```python
# EXERCISE 2. Find the minimum in the f(x)=x^3-7.5x^2+18x-10
# using the gradient descent algorithm.

cur_x = 6
gamma = 0.01
precision = 0.00001
step_size = 1
max_iters = 10000
iters = 0

f = lambda x: x**3 - 7.5*x**2 + 18*x - 10
df = lambda x: 3*x**2 - 15*x + 18

while (step_size > precision) & (iters < max_iters):
    prev_x = cur_x
    cur_x -= gamma*df(prev_x)
    step_size - abs(cur_x - prev_x)
    iters+=1

print('True local minimum at {} with function value {}.'.format(9/4, f(9/4)))
print('Local minimum by gradient descent at {} with function value {}.'.format(cur_x, f(cur_x)))
```

    True local minimum at 2.25 with function value 3.921875.
    Local minimum by gradient descent at 3.000000000000008 with function value 3.5.

```python

```


# Finding Pattern

```python
import numpy as np
```

## Given this array, find the next number in the sequence

```python
my_teaser_array = np.array([1, 7, 19, 37, 61, 91, 127, 169, 217, 271, 331])
my_teaser_array
```

    array([  1,   7,  19,  37,  61,  91, 127, 169, 217, 271, 331])

##### for information about numpy.diff, please see: http://docs.scipy.org/doc/numpy/reference/generated/numpy.diff.html

```python
np.diff(my_teaser_array)
```

    array([ 6, 12, 18, 24, 30, 36, 42, 48, 54, 60])

```python
np.diff(my_teaser_array, n=2)
```

    array([6, 6, 6, 6, 6, 6, 6, 6, 6])

```python
np.diff(my_teaser_array, n=3)
```

    array([0, 0, 0, 0, 0, 0, 0, 0])

##### Warning: imports should (usually) appear at top of notebook

If SymPy is not included on your computer; open a console and type 'conda sympy'

```python
from sympy import init_session
init_session()
```

    IPython console for SymPy 1.0 (Python 3.5.2-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at http://docs.sympy.org/1.0/

```python
diff(x**3)
```

```python
diff(x**3, x, 2)
```

```python
diff(x**3, x, 3)
```

```python
diff(x**3, x, 4)
```

```python
def my_guess(n):
    return (n+1)**3 - n**3
```

```python
my_guess(np.arange(20))
```

    array([   1,    7,   19,   37,   61,   91,  127,  169,  217,  271,  331,
            397,  469,  547,  631,  721,  817,  919, 1027, 1141], dtype=int32)

```python
my_teaser_array = np.array([1, 7, 19, 37, 61, 91, 127, 169, 217, 271, 331])
my_teaser_array
```

    array([  1,   7,  19,  37,  61,  91, 127, 169, 217, 271, 331])

```python

```


```python
from sympy import *
init_printing(use_unicode=True)
from IPython.display import display
```

# Task 1

```python
#v0, v1= Symbol('v_0'), Symbol('v_1')

v0 = Matrix([[0.5], [0.5], [0.5], [0.5]])
v1 = Matrix([[0.5], [0.5], [-0.5], [-0.5]])

v0.dot(v1)
```

#### Answer: 0

# Task 2.
If $\{\mathbf{v}^{(0)},\ \mathbf{v}^{(1)},\ \mathbf{v}^{(2)},\ \mathbf{v}^{(3)}\}$ is a full **orthonormal** basis, the following system of equations must be satisfied:

$$\begin{cases}
\langle \mathbf{v}^{(0)}, \mathbf{v}^{(3)}\rangle = 0 \\
\langle \mathbf{v}^{(1)}, \mathbf{v}^{(3)}\rangle = 0 \\
\langle \mathbf{v}^{(2)}, \mathbf{v}^{(3)}\rangle = 0 \\
\langle \mathbf{v}^{(3)}, \mathbf{v}^{(3)}\rangle = 1
\end{cases}$$

Let $\mathbf{v}^{(3)} = \begin{pmatrix} a & b & c & d \end{pmatrix}$.

```python
v2 = Matrix([[0.5], [-0.5], [0.5], [-0.5]])
a, b, c, d = symbols('a b c d')
v3 = Matrix([[a], [b], [c], [d]])
answer = solve([v0.dot(v3), v1.dot(v3), v2.dot(v3), v3.dot(v3)-1], (a, b, c, d))
answer
```

#### Answer: 2

# Task 3.

Let expansion coefficients vector $\mathbf x = \begin{pmatrix} a & b & c & d \end{pmatrix}$.

```python
v3 = Matrix(next(iter(answer)))
y = Matrix([2.5, 0.5, 1.5, -0.5])
x = (a, b, c, d)

basis_matrix = v0.row_join(v1).row_join(v2).row_join(v3)
linsolve((basis_matrix, y), x)
```

#### Answer: 2.0 1.0 2.0 0.0

# Task 4.
We just need to solve 4 sytems of linear equations in this task

```python
solve([y.dot(x), v1.dot(x), v2.dot(x), v3.dot(x)], (a, b, c, d))
```

```python
solve([y.dot(x), v0.dot(x), v1.dot(x), v2.dot(x)], (a, b, c, d))
```

```python
solve([y.dot(x), v0.dot(x), v2.dot(x), v3.dot(x)], (a, b, c, d))
```

```python
solve([y.dot(x), v1.dot(x), v2.dot(x), (v3-2*v1).dot(x)], (a, b, c, d))
```

#### Answer: cells 1, 3 and 4 must be selected

# Task 5.
Solution is very simple.

New $\mathbf{x'}=\begin{bmatrix} x_0-x_2 \\ x_1-x_0 \\ x_2-x_1 \end{bmatrix} = \begin{bmatrix} 1 \cdot x_0 + 0 \cdot x_1 -1 \cdot x_2 \\ -1 \cdot x_0 + 1 \cdot x_1 + 0 \cdot x_2 \\ 0 \cdot x_0 -1 \cdot x_1 + 1 \cdot x_2 \end{bmatrix} $. Now we can determine the matrix $\mathbf{F}=\begin{bmatrix} 1 && 0 && -1 \\ -1 && 1 && 0 \\ 0 && -1 && 1 \end{bmatrix}$.

#### Answer: 1 0 -1 -1 1 0 0 -1 1

# Task 6.
In the task 5 the matrix $\mathbf{D}$ was defined, that matrix represents delay by 1 (right shift). In this task the matrix $\mathbf{A}$ is a delay matrix again.

Thus $\mathbf{A}^4=\begin{bmatrix} 1 && 0 && 0 && 0 \\ 0 && 1 && 0 && 0 \\ 0 && 0 && 1 && 0 \\ 0 && 0 && 0 && 1 \end{bmatrix}$.

```python
A = Matrix([[0, 0, 0, 1], [1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0]])
A**4
```

#### Answer: 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1


# Ricci Tensor and Scalar Curvature calculations using Symbolic module

```python
import sympy
from sympy import cos, sin, sinh
from einsteinpy.symbolic import MetricTensor, RicciTensor, RicciScalar

sympy.init_printing()
```

### Defining the Anti-de Sitter spacetime Metric

```python
syms = sympy.symbols("t chi theta phi")
t, ch, th, ph = syms
m = sympy.diag(-1, cos(t) ** 2, cos(t) ** 2 * sinh(ch) ** 2, cos(t) ** 2 * sinh(ch) ** 2 * sin(th) ** 2).tolist()
metric = MetricTensor(m, syms)
```

### Calculating the Ricci Tensor(with both indices covariant)

```python
Ric = RicciTensor.from_metric(metric)
Ric.tensor()
```

$$\left[\begin{matrix}3 & 0 & 0 & 0\\0 & - 3 \cos^{2}{\left (t \right )} & 0 & 0\\0 & 0 & \left(\sin^{2}{\left (t \right )} - 1\right) \sinh^{2}{\left (\chi \right )} - 2 \cos^{2}{\left (t \right )} \sinh^{2}{\left (\chi \right )} & 0\\0 & 0 & 0 & \left(\sin^{2}{\left (t \right )} - 1\right) \sin^{2}{\left (\theta \right )} \sinh^{2}{\left (\chi \right )} - 2 \sin^{2}{\left (\theta \right )} \cos^{2}{\left (t \right )} \sinh^{2}{\left (\chi \right )}\end{matrix}\right]$$

### Calculating the Ricci Scalar(Scalar Curvature) from the Ricci Tensor

```python
R = RicciScalar.from_riccitensor(Ric)
R.expr
```

The curavture is -12 which is in-line with the theoretical results

docs/source/examples/Ricci Tensor and Scalar Curvature symbolic calculation.ipynb


# Poisson Distribution - Errors in Text

> This document is written in *R*.
>
> ***GitHub***: https://github.com/czs108

## Background

> In a certain long document there is an *average* of **0.5** typographical errors per **100** words of text.

## Question A

> What is the *mean* number of words between errors?

\begin{equation}
Mean = \frac{1}{0.5} \times 100 = 200
\end{equation}

## Question B

> What is the probability of finding **4** errors in a text of length **500** words?

\begin{equation}
\lambda = 0.5 \times \frac{500}{100} = 2.5
\end{equation}

\begin{equation}
\begin{split}
P(X = 4) &= \frac{e^{-\lambda} \cdot {\lambda}^{4}}{4!} \\
    &= \frac{e^{-2.5} \cdot {2.5}^{4}}{4!}
\end{split}
\end{equation}

Use the `dpois` function.

```R
dpois(x=4, lambda=2.5)
```

## Question C

> What is the probability of there being *at least* **300** words before the *1st* error?

\begin{equation}
\lambda = 0.5 \times \frac{300}{100} = 1.5
\end{equation}

\begin{equation}
\begin{split}
P(X = 0) &= \frac{e^{-\lambda} \cdot {\lambda}^{0}}{0!} \\
    &= e^{-1.5}
\end{split}
\end{equation}

```R
dpois(x=0, lambda=1.5)
```

0.22313016014843

Use the `exp` function.

```R
exp(-1.5)
```

0.22313016014843

## Question D

> What is the *minimum* number of words in which the probability of finding an error is *at least* **90%**?

We know that

\begin{equation}
P(X = 1) \geq 0.9
\end{equation}

So

\begin{equation}
P(X = 0) < 0.1
\end{equation}

Assume $n$ is the number of words.

\begin{equation}
\lambda = 0.5 \times \frac{n}{100}
\end{equation}

\begin{equation}
\begin{split}
P(X = 0) &= \frac{e^{-\lambda} \cdot {\lambda}^{0}}{0!} \\
    &= e^{-\lambda} \\
    &< 0.1
\end{split}
\end{equation}

Then we get

\begin{equation}
\ln 0.1 = -2.3
\end{equation}

```R
log(0.1)
```

-2.30258509299405

When $-\lambda < -2.3$, $P(X = 0) < 0.1$.

So $\lambda > 2.3$

\begin{equation}
n = 200 \times \lambda > 460
\end{equation}

## Question E

> How many words would there be on a page, if the probability of **0** errors on a page was **20%**?

Assume $n$ is the number of words on a page.

\begin{equation}
\lambda = 0.5 \times \frac{n}{100}
\end{equation}

\begin{equation}
\begin{split}
P(X = 0) &= \frac{e^{-\lambda} \cdot {\lambda}^{0}}{0!} \\
    &= e^{-\lambda} \\
    &= 0.2
\end{split}
\end{equation}

Then we get

\begin{align}
\ln 0.2 = -\lambda \\
\lambda = 1.609
\end{align}

```R
-log(0.2)
```

1.6094379124341

\begin{equation}
n = 200 \times \lambda = 322
\end{equation}

## Question F

> What is the probability of there being *at least* **2000** words before there are **10** errors?

\begin{equation}
\lambda = 0.5 \times \frac{2000}{100} = 10
\end{equation}

\begin{equation}
\begin{split}
P(X \leq 9) &= \sum_{i=0}^{9} P(X = i) \\
    &= \sum_{i=0}^{9} \frac{e^{-10} \cdot {10}^{i}}{i!}
\end{split}
\end{equation}

```R
sum(dpois(x=c(0:9), lambda=10))
```

0.457929714471852

Or use the `ppois` function.

```R
ppois(q=9, lambda=10)
```

0.457929714471852

exercises/Poisson Distribution - Errors in Text.ipynb