# Essential Math for Machine Learning: Python Edition

Course source: [LinkedIn Learning](https://www.linkedin.com/learning/essential-math-for-machine-learning-python-edition)

## 1 Vectors
Vectors, and vector spaces, are fundamental to *linear algebra*, and they're used in many machine learning models. Vectors describe spatial lines and planes, enabling you to perform calculations that explore relationships in multi-dimensional space.

### 1.1 What is a Vector
At its simplest, a vector is a numeric element that has both *magnitude* and *direction*. The magnitude represents a distance (for example, "2 miles") and the direction indicates which way the vector is headed (for example, "East"). Vectors are defined by an n-dimensional coordinate that describe a point in space that can be connected by a line from an arbitrary origin.

That all seems a bit complicated, so let's start with a simple, two-dimensional example. In this case, we'll have a vector that is defined by a point in a two-dimensional plane: A two dimensional coordinate consists of an *x* and a *y* value, and in this case we'll use **2** for *x* and **1** for *y*.

Our vector can be written as **v**=(2,1), but more formally we would use the following notation, in which the dimensional coordinate values for the vector are shown as a matrix:
\begin{equation}\vec{v} = \begin{bmatrix}2 \\ 1 \end{bmatrix}\end{equation}

So what exactly does that mean? Well, the coordinate is two-dimensional, and describes the movements required to get to the end point (of *head*) of the vector - in this case, we need to move 2 units in the *x* dimension, and 1 unit in the *y* dimension. Note that we don't specify a starting point for the vector - we're simply describing a destination coordinate that encapsulate the magnitide and direction of the vector. Think about it as the directions you need to follow to get to *there* from *here*, without specifying where *here* actually is!

It can help to visualize the vector, and with a two-dimensional vector, that's pretty straightforward. We just define a two-dimensional plane, choose a starting point, and plot the coordinate described by the vector relative to the starting point.

Run the code in the following cell to visualize the vector **v** (which remember is described by the coordinate (2,1)).

```python
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import math
```

```python
# We'll use a numpy array for our vector
v = np.array([2,1])

# and we'll use a quiver plot to visualize it.
origin = [0], [0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, *v, scale=10, color='r')
plt.show()
```

Note that we can use a numpy array to define the vector in Python; so to create our (2,1) vector, we simply create a numpy array with the elements [2,1]. We've then used a quiver plot to visualize the vector, using the point 0,0 as the starting point (or *origin*). Our vector of (2,1) is shown as an arrow that starts at 0,0 and moves 2 units along the *x* axis (to the right) and 1 unit along the *y* axis (up).

### 1.2 Calculating Vector Magnitude and Direction
We tend to work with vectors by expressing their components as *cartesian coordinates*; that is, *x* and *y* (and other dimension) values that define the number of units travelled along each dimension. So the coordinates of our (2,1) vector indicate that we must travel 2 units along the *x* axis, and *1* unit along the *y* axis.

However, you can also work with verctors in terms of their *polar coordinates*; that is coordinates that describe the magnitude and direction of the vector. The magnitude is the overall distance of the vector from tail to head, and the direction is the angle at which the vector is oriented.

#### 1.2.1 Calculating Magnitude
Calculating the magnitude of the vector from its cartesian coordinates requires measuring the distance between the arbitrary starting point and the vector head point. For a two-dimensional vector, we're actually just calculating the length of the hypotenuse in a right-angled triangle - so we could simply invoke Pythagorean theorum and calculate the square root of the sum of the squares of it's components, like this:

\begin{equation}\|\vec{v}\| = \sqrt{v_{1}\;^{2} + v_{2}\;^{2}}\end{equation}

The notation for a vector's magnitude is to surround the vector name with vertical bars - you can use single bars (for example, |**v**|) or double bars (||**v**||). Double-bars are often used to avoid confusion with absolute values. Note that the components of the vector are indicated by subscript indices (v1, v2,...v*n*),

In this case, the vector **v** has two components with values **2** and **1**, so our magnitude calculation is:

\begin{equation}\|\vec{v}\| = \sqrt{2^{2} + 1^{2}}\end{equation}

Which is:

\begin{equation}\|\vec{v}\| = \sqrt{4 + 1}\end{equation}

So:

\begin{equation}\|\vec{v}\| = \sqrt{5} \approx 2.24\end{equation}

You can run the following Python code to get a more precise result (note that the elements of a numpy array are zero-based)

```python
vMag = math.sqrt(v[0]**2 + v[1]**2)
print (vMag)
```

    2.23606797749979

This calculation works for vectors of any dimensionality - you just take the square root of the sum of the squared components:

\begin{equation}\|\vec{v}\| = \sqrt{v_{1}\;^{2} + v_{2}\;^{2} ... + v_{n}\;^{2}}\end{equation}

In Python, *numpy* provides a linear algebra library named **linalg** that makes it easier to work with vectors - you can use the **norm** function in the following code to calculate the magnitude of a vector:

```python
vMag = np.linalg.norm(v)
print (vMag)
```

    2.23606797749979

#### 1.2.2 Calculating Direction
To calculate the direction, or *amplitude*, of a vector from its cartesian coordinates, you must employ a little trigonometry. We can get the angle of the vector by calculating the *inverse tangent*; sometimes known as the *arctan* (the *tangent*  calculates an angle as a ratio - the inverse tangent, or **tan-1**, expresses this in degrees).

In any right-angled triangle, the tangent is calculated as the *opposite* over the *adjacent*. In a two dimensional vector, this is the *y* value over the *x* value, so for our **v** vector (2,1):

\begin{equation}tan(\theta) = \frac{1}{2}\end{equation}

This produces the result ***0.5***, from which we can use a calculator to calculate the inverse tangent to get the angle in degrees:

\begin{equation}\theta = tan^{-1} (0.5) \approx 26.57^{o}\end{equation}

Note that the direction angle is indicated as ***θ***.

Run the following Python code to confirm this:

```python
v = np.array([2,1])
vTan = v[1] / v[0]
print ('tan = ' + str(vTan))
vAtan = math.atan(vTan)
# atan returns the angle in radians, so convert to degrees
print('inverse-tan = ' + str(math.degrees(vAtan)))
```

    tan = 0.5
    inverse-tan = 26.56505117707799

There is an added complication however, because if the value for *x* or *y* (or both) is negative, the orientation of the vector is not standard, and a calculator can give you the wrong tan-1 value. To ensure you get the correct direction for your vector, use the following rules:
- Both *x* and *y* are positive: Use the tan-1 value.
- *x* is negative, *y* is positive: Add 180 to the tan-1 value.
- Both *x* and *y* are negative: Add 180 to the tan-1 value.
- *x* is positive, *y* is negative: Add 360 to the tan-1 value.

To understand why we need to do this, think of it this way. A vector can be pointing in any direction through a 360 degree arc.  Let's break that circle into four quadrants with the x and y axis through the center. Angles can be measured from the x axis in both the positive (counter-clockwise) and negative (clockwise) directions. We'll number the quadrants in the positive (counter-clockwise) direction (which is how we measure the *positive* angle) like this:

    2 | 1
    - o -
    3 | 4

OK, let's look at 4 example vectors

 1. Vector [2,4] has positive values for both x and y. The line for this vector travels through the point 0,0 from quadrant 3 to quadrant 1. Tan-1 of 4/2 is around 63.4 degrees, which is the positive angle from the x axis to the vector line - so this is the direction of the vector.
 2. Vector [-2,4] has a negative x and positive y. The line for this vector travels through point 0,0 from quadrant 4 to quadrant 2. Tan-1 of 4/-2 is around -64.4 degrees, which is the *negative* angle from x to the vector line; but in the wrong direction (as if the vector was travelling from quadrant 2 towards quadrant 4). So we need the opposite direction, which we get by adding 180.
 3. Vector [-2,-4] has negative x and y. The line for the vector travels through 0,0 from quadrant 1 to quadrant 3. Tan-1 of -4/-2 is around 63.4 degrees, which is the angle between the x axis and the line, but again in the opposite direction, from quadrant 3 to quadrant 1; we need to go a further 180 degrees to reflect the correct direction.
 4. Vector [2,-4] has positive x and negative y. It travels through 0,0 from quadrant 2 to quadrant 4. Tan-1 of -4/2 is around -64.4 degrees, which is the *negative* angle from the x axis to the vector line. Technically it's correct, the line is travelleing down and to the right at an angle of -63.4 degrees; but we want to express the *positive* (counter-clockwise) angle, so we add 360.

In the previous Python code, we used the *math.**atan*** function to calculate the inverse tangent from a numeric tangent. The *numpy* library includes a similar ***arctan*** function. When working with numpy arrays, you can also use the *numpy.**arctan2*** function to return the inverse tangent of an array-based vector in *radians*, and you can use the *numpy.**degrees*** function to convert this to degrees. The ***arctan2*** function automatically makes the necessary adjustment for negative *x* and *y* values.

```python
v = np.array([2,1])
print ('v: ' + str(np.degrees(np.arctan2(v[1], v[0]))))

s = np.array([-3,2])
print ('s: ' + str(np.degrees(np.arctan2(s[1], s[0]))))
```

    v: 26.56505117707799
    s: 146.30993247402023

### 1.3 Vector Addition
So far, we've worked with one vector at a time. What happens when you need to add two vectors.

Let's take a look at an example, we already have a vector named **v**, as defined here:
\begin{equation}\vec{v} = \begin{bmatrix}2 \\ 1 \end{bmatrix}\end{equation}
Now let's create a second vector, and called **s** like this:
\begin{equation}\vec{s} = \begin{bmatrix}-3 \\ 2 \end{bmatrix}\end{equation}

Run the cell below to create **s** and plot it together with **v**:

```python
v = np.array([2,1])
s = np.array([-3,2])
print (s)
```

    [-3  2]

```python
# Plot v and s
vecs = np.array([v,s])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['r', 'b'], scale=10)
plt.show()
```

You can see in the plot that the two vectors have different directions and magnitudes. So what happens when we add them together?

Here's the formula:
\begin{equation}\vec{z} = \vec{v}+\vec{s}\end{equation}

In terms of our vector matrices, this looks like this:
\begin{equation}\vec{z} = \begin{bmatrix}2 \\ 1 \end{bmatrix} + \begin{bmatrix}-3 \\ 2 \end{bmatrix}\end{equation}

Which gives the following result:
\begin{equation}\vec{z} = \begin{bmatrix}2 \\ 1 \end{bmatrix} + \begin{bmatrix}-3 \\ 2 \end{bmatrix} = \begin{bmatrix}-1 \\ 3 \end{bmatrix}\end{equation}

Let's verify that Python gives the same result:

```python
z = v + s
print(z)
```

    [-1  3]

So what does that look like on our plot?

```python
vecs = np.array([v,s,z])
origin = [0,0,0], [0,0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['r', 'b', 'g'], scale=10)
plt.show()
```

So what's going on here?
Well, we added the dimensions of **s** to the dimensions of **v** to describe a new vector **z**. Let's break that down:
- The dimensions of **v** are (2,1), so from our starting point we move 2 units in the *x* dimension (across to the right) and 1 unit in the *y* dimension (up). In the plot, if you start at the (0,0) position, this is shown as the red arrow.
- Then we're adding **s**, which has dimension values (-3, 2), so we move -3 units in the *x* dimension (across to the left, because it's a negative number) and then 2 units in the *y* dimension (up). On the plot, if you start at the head of the red arrow and make these moves, you'll end up at the head of the green arrow, which represents **z**.

The same is true if you perform the addition operation the other way around and add **v** to **s**, the steps to create **s** are described by the blue arrow, and if you use that as the starting point for **v**, you'll end up at the head of the green arrow, which represents **z**.

Note on the plot that if you simply moved the tail of the blue arrow so that it started at the head of red arrow, its head would end up in the same place as the head of the green arrow; and the same would be true if you moved tail of the red arrow to the head of the blue arrow.

## 2 Vector Multiplication
Vector multiplication can be performed in three ways:

- Scalar Multiplication
- Dot Product Multiplication
- Cross Product Multiplication

### 2.1 Scalar Multiplication
Let's start with *scalar* multiplication - in other words, multiplying a vector by a single numeric value.

Suppose I want to multiply my vector by 2, which I could write like this:

\begin{equation} \vec{w} = 2\vec{v}\end{equation}

Note that the result of this calculation is a new vector named **w**. So how would we calculate this?
Recall that **v** is defined like this:

\begin{equation}\vec{v} = \begin{bmatrix}2 \\ 1 \end{bmatrix}\end{equation}

To calculate 2v, we simply need to apply the operation to each dimension value in the vector matrix, like this:

\begin{equation}\vec{w} = \begin{bmatrix}2 \cdot 2 \\  2 \cdot 1 \end{bmatrix}\end{equation}

Which gives us the following result:

\begin{equation}\vec{w} = \begin{bmatrix}2 \cdot 2 \\  2 \cdot 1 \end{bmatrix} = \begin{bmatrix}4 \\ 2 \end{bmatrix}\end{equation}

In Python, you can apply these sort of matrix operations directly to numpy arrays, so we can simply calculate **w** like this:

```python
v = np.array([2,1])

w = 2 * v
print(w)

# Plot w
origin = [0], [0]
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, *w, scale=10)
plt.show()
```

The same approach is taken for scalar division.

Try it for yourself - use the cell below to calculate a new vector named **b** based on the following definition:

\begin{equation}\vec{b} = \frac{\vec{v}}{2}\end{equation}

```python
b = v / 2
print(b)

# Plot b
origin = [0], [0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, *b, scale=10)
plt.show()
```

### 2.2 Dot Product Multiplication
So we've seen how to multiply a vector by a scalar. How about multiplying two vectors together? There are actually two ways to do this depending on whether you want the result to be a *scalar product* (in other words, a number) or a *vector product* (a vector).

To get a scalar product, we calculate the *dot product*. This takes a similar approach to multiplying a vector by a scalar, except that it multiplies each component pair of the vectors and sums the results. To indicate that we are performing a dot product operation, we use the • operator:

\begin{equation} \vec{v} \cdot \vec{s} = (v_{1} \cdot s_{1}) + (v_{2} \cdot s_{2}) ... + \; (v_{n} \cdot s_{n})\end{equation}

So for our vectors **v** (2,1) and **s** (-3,2), our calculation looks like this:

\begin{equation} \vec{v} \cdot \vec{s} = (2 \cdot -3) + (1 \cdot 2) = -6 + 2 = -4\end{equation}

So the dot product, or scalar product, of **v** • **s** is **-4**.

In Python, you can use the *numpy.**dot*** function to calculate the dot product of two vector arrays:

```python
v = np.array([2,1])
s = np.array([-3,2])
d = np.dot(v,s)
print (d)
```

    -4

In Python 3.5 and later, you can also use the **@** operator to calculate the dot product:

```python
v = np.array([2,1])
s = np.array([-3,2])
d = v @ s
print (d)
```

    -4

#### 2.2.1 The Cosine Rule
An useful property of vector dot product multiplication is that we can use it to calculate the cosine of the angle between two vectors. We could write the dot products as:

$$ \vec{v} \cdot \vec{s} = \|\vec{v} \|\|\vec{s}\| \cos (\theta) $$

Which we can rearrange as:

$$ \cos(\theta) = \frac{\vec{v} \cdot \vec{s}}{\|\vec{v} \|\|\vec{s}\|} $$

So for our vectors **v** (2,1) and **s** (-3,2), our calculation looks like this:

$$ \cos(\theta) = \frac{(2 \cdot-3) + (-3 \cdot 2)}{\sqrt{2^{2} + 1^{2}} \times \sqrt{-3^{2} + 2^{2}}} $$

So:

$$\cos(\theta) = \frac{-4}{8.0622577483}$$

Which calculates to:

$$\cos(\theta) = -0.496138938357 $$

So:

$$\theta \approx 119.74 $$

Here's that calculation in Python:

```python
# define our vectors
v = np.array([2,1])
s = np.array([-3,2])

# get the magnitudes
vMag = np.linalg.norm(v)
sMag = np.linalg.norm(s)

# calculate the cosine of theta
cos = (v @ s) / (vMag * sMag)

# so theta (in degrees) is:
theta = math.degrees(math.acos(cos))

print(theta)
```

    119.74488129694222

### 2.3 Cross Product Multiplication
To get the *vector product* of multipying two vectors together, you must calculate the *cross product*. The result of this is a new vector that is at right angles to both the other vectors in 3D Euclidean space. This means that the cross-product only really makes sense when working with vectors that contain three components.

For example, let's suppose we have the following vectors:

\begin{equation}\vec{p} = \begin{bmatrix}2 \\ 3 \\ 1 \end{bmatrix}\;\; \vec{q} = \begin{bmatrix}1 \\ 2 \\ -2 \end{bmatrix}\end{equation}

To calculate the cross product of these vectors, written as **p** x **q**, we need to create a new vector (let's call it **r**) with three components (r1, r2, and r3). The values for these components are calculated like this:

\begin{equation}r_{1} = p_{2}q_{3} - p_{3}q_{2}\end{equation}
\begin{equation}r_{2} = p_{3}q_{1} - p_{1}q_{3}\end{equation}
\begin{equation}r_{3} = p_{1}q_{2} - p_{2}q_{1}\end{equation}

So in our case:

\begin{equation}\vec{r} = \vec{p} \times \vec{q} = \begin{bmatrix}(3 \cdot -2) - (1 \cdot 2) \\ (1 \cdot 1) - (2 \cdot -2) \\ (2 \cdot 2) - (3 \cdot 1) \end{bmatrix} = \begin{bmatrix}-6 - 2 \\ 1 - -4 \\ 4 - 3 \end{bmatrix} = \begin{bmatrix}-8 \\ 5 \\ 1 \end{bmatrix}\end{equation}

In Python, you can use the *numpy.**cross*** function to calculate the cross product of two vector arrays:

```python
p = np.array([2,3,1])
q = np.array([1,2,-2])
r = np.cross(p,q)
print (r)
```

    [-8  5  1]

## 3 Introduction to Matrices
In general terms, a matrix is an array of numbers that are arranged into rows and columns.

### 3.1 Matrices and Matrix Notation
A matrix arranges numbers into rows and columns, like this:

\begin{equation}A = \begin{bmatrix}
  1 & 2 & 3 \\
  4 & 5 & 6
 \end{bmatrix}
\end{equation}

Note that matrices are generally named as a capital letter. We refer to the *elements* of the matrix using the lower case equivalent with a subscript row and column indicator, like this:

\begin{equation}A = \begin{bmatrix}
  a_{1,1} & a_{1,2} & a_{1,3} \\
  a_{2,1} & a_{2,2} & a_{2,3}
 \end{bmatrix}
\end{equation}

In Python, you can define a matrix as a 2-dimensional *numpy.**array***, like this:

```python
import numpy as np
```

```python
A = np.array([[1,2,3],
              [4,5,6]])
print (A)
```

    [[1 2 3]
     [4 5 6]]

You can also use the **numpy.matrix** type, which is a specialist subclass of **array**:

```python
M = np.matrix([[1,2,3],
               [4,5,6]])
print (M)
```

    [[1 2 3]
     [4 5 6]]

There are some differences in behavior between ***array*** and ***matrix*** types - particularly with regards to multiplication (which we'll explore later). You can use either, but most experienced Python programmers who need to work with both vectors and matrices tend to prefer the ***array*** type for consistency.

### 3.2 Matrix Operations
Matrices support common arithmetic operations.

#### 3.2.1 Adding Matrices
To add two matrices of the same size together, just add the corresponding elements in each matrix:

\begin{equation}\begin{bmatrix}1 & 2 & 3 \\4 & 5 & 6\end{bmatrix}+ \begin{bmatrix}6 & 5 & 4 \\3 & 2 & 1\end{bmatrix} = \begin{bmatrix}7 & 7 & 7 \\7 & 7 & 7\end{bmatrix}\end{equation}

In this example, we're adding two matrices (let's call them ***A*** and ***B***). Each matrix has two rows of three columns (so we describe them as 2x3 matrices). Adding these will create a new matrix of the same dimensions with the values a1,1 + b1,1, a1,2 + b1,2, a1,3 + b1,3,a2,1 + b2,1, a2,2 + b2,2, and a2,3 + b2,3. In this instance, each pair of corresponding elements(1 and 6, 2, and 5, 3 and 4, etc.) adds up to 7.

Let's try that with Python:

```python
A = np.array([[1,2,3],
              [4,5,6]])
B = np.array([[6,5,4],
              [3,2,1]])
print(A + B)
```

    [[7 7 7]
     [7 7 7]]

#### 3.2.2 Subtracting Matrices
Matrix subtraction works similarly to matrix addition:

\begin{equation}\begin{bmatrix}1 & 2 & 3 \\4 & 5 & 6\end{bmatrix}- \begin{bmatrix}6 & 5 & 4 \\3 & 2 & 1\end{bmatrix} = \begin{bmatrix}-5 & -3 & -1 \\1 & 3 & 5\end{bmatrix}\end{equation}

Here's the Python code to do this:

```python
A = np.array([[1,2,3],
              [4,5,6]])
B = np.array([[6,5,4],
              [3,2,1]])
print (A - B)
```

    [[-5 -3 -1]
     [ 1  3  5]]

##### 3.2.2.1 Conformability
In the previous examples, we were able to add and subtract the matrices, because the *operands* (the matrices we are operating on) are ***conformable*** for the specific operation (in this case, addition or subtraction). To be conformable for addition and subtraction, the operands must have the same number of rows and columns. There are different conformability requirements for other operations, such as multiplication; which we'll explore later.

#### 3.2.3 Negative Matrices
The nagative of a matrix, is just a matrix with the sign of each element reversed:

\begin{equation}C = \begin{bmatrix}-5 & -3 & -1 \\1 & 3 & 5\end{bmatrix}\end{equation}

\begin{equation}-C = \begin{bmatrix}5 & 3 & 1 \\-1 & -3 & -5\end{bmatrix}\end{equation}

Let's see that with Python:

```python
C = np.array([[-5,-3,-1],
              [1,3,5]])
print (C)
print (-C)
```

    [[-5 -3 -1]
     [ 1  3  5]]
    [[ 5  3  1]
     [-1 -3 -5]]

#### 3.2.4 Matrix Transposition
You can *transpose* a matrix, that is switch the orientation of its rows and columns. You indicate this with a superscript **T**, like this:

\begin{equation}\begin{bmatrix}1 & 2 & 3 \\4 & 5 & 6\end{bmatrix}^{T} = \begin{bmatrix}1 & 4\\2 & 5\\3 & 6 \end{bmatrix}\end{equation}

In Python, both **numpy.array** and **numpy.matrix** have a **T** function:

```python
A = np.array([[1,2,3],
              [4,5,6]])
print(A.T)
```

    [[1 4]
     [2 5]
     [3 6]]

## 4 More Matrices
This notebook continues your exploration of matrices.

### 4.1 Matrix Multiplication
Multiplying matrices is a little more complex than the operations we've seen so far. There are two cases to consider, *scalar multiplication* (multiplying a matrix by a single number), and *dot product matrix multiplication* (multiplying a matrix by another matrix.
#### 4.1.1 Scalar Multiplication
To multiply a matrix by a scalar value, you just multiply each element by the scalar to produce a new matrix:

\begin{equation}2 \times \begin{bmatrix}1 & 2 & 3 \\4 & 5 & 6\end{bmatrix} = \begin{bmatrix}2 & 4 & 6 \\8 & 10 & 12\end{bmatrix}\end{equation}

In Python, you perform this calculation using the **\*** operator:

```python
import numpy as np
```

```python
A = np.array([[1,2,3],
              [4,5,6]])
print(2 * A)
```

    [[ 2  4  6]
     [ 8 10 12]]

#### 4.1.2 Dot Product Matrix Multiplication
To mulitply two matrices together, you need to calculate the *dot product* of rows and columns. This means multiplying each of the elements in each row of the first matrix by each of the elements in each column of the second matrix and adding the results. We perform this operation by applying the *RC* rule - always multiplying ***R***ows by ***C***olumns. For this to work, the number of ***columns*** in the first matrix must be the same as the number of ***rows*** in the second matrix so that the matrices are *conformable* for the dot product operation.

Sounds confusing, right?

Let's look at an example:

\begin{equation}\begin{bmatrix}1 & 2 & 3 \\4 & 5 & 6\end{bmatrix} \cdot \begin{bmatrix}9 & 8 \\ 7 & 6 \\ 5 & 4\end{bmatrix}\end{equation}

Note that the first matrix is 2x3, and the second matrix is 3x2. The important thing here is that the first matrix has two rows, and the second matrix has two columns. To perform the multiplication, we first take the dot product of the first ***row*** of the first matrix (1,2,3) and the first ***column*** of the second matrix (9,7,5):

\begin{equation}(1,2,3) \cdot (9,7,5) = (1 \times 9) + (2 \times 7) + (3 \times 5) = 38\end{equation}

In our resulting matrix (which will always have the same number of ***rows*** as the first matrix, and the same number of ***columns*** as the second matrix), we can enter this into the first row and first column element:

\begin{equation}\begin{bmatrix}38 & ?\\? & ?\end{bmatrix} \end{equation}

Now we can take the dot product of the first row of the first matrix and the second column of the second matrix:

\begin{equation}(1,2,3) \cdot (8,6,4) = (1 \times 8) + (2 \times 6) + (3 \times 4) = 32\end{equation}

Let's add that to our resulting matrix in the first row and second column element:

\begin{equation}\begin{bmatrix}38 & 32\\? & ?\end{bmatrix} \end{equation}

Now we can repeat this process for the second row of the first matrix and the first column of the second matrix:

\begin{equation}(4,5,6) \cdot (9,7,5) = (4 \times 9) + (5 \times 7) + (6 \times 5) = 101\end{equation}

Which fills in the next element in the result:

\begin{equation}\begin{bmatrix}38 & 32\\101 & ?\end{bmatrix} \end{equation}

Finally, we get the dot product for the second row of the first matrix and the second column of the second matrix:

\begin{equation}(4,5,6) \cdot (8,6,4) = (4 \times 8) + (5 \times 6) + (6 \times 4) = 86\end{equation}

Giving us:

\begin{equation}\begin{bmatrix}38 & 32\\101 & 86\end{bmatrix} \end{equation}

In Python, you can use the **numpy.dot** function or the **@** operator to multiply matrices and two-dimensional arrays:

```python
A = np.array([[1,2,3],
              [4,5,6]])
B = np.array([[9,8],
              [7,6],
              [5,4]])
print(np.dot(A,B))
print(A @ B)
```

    [[ 38  32]
     [101  86]]
    [[ 38  32]
     [101  86]]

This is one case where there is a difference in behavior between **numpy.array** and **numpy.matrix**, You can also use a regular multiplication (**\***) operator with a matrix, but not with an array:

```python
A = np.matrix([[1,2,3]
               ,[4,5,6]])
B = np.matrix([[9,8],
               [7,6],
               [5,4]])
print(A * B)
```

    [[ 38  32]
     [101  86]]

Note that, unlike with multiplication of regular scalar numbers, the order of the operands in a multiplication operation is significant. For scalar numbers, the *commmutative law* of multiplication applies, so for example:

\begin{equation}2 \times 4 = 4 \times 2\end{equation}

With matrix multiplication, things are different, for example:

\begin{equation}\begin{bmatrix}2 & 4 \\6 & 8\end{bmatrix} \cdot \begin{bmatrix}1 & 3 \\ 5 & 7\end{bmatrix} \ne \begin{bmatrix}1 & 3 \\ 5 & 7\end{bmatrix} \cdot \begin{bmatrix}2 & 4 \\6 & 8\end{bmatrix}\end{equation}

Run the following Python code to test this:

```python
A = np.array([[2,4],
              [6,8]])
B = np.array([[1,3],
              [5,7]])
print(A @ B)
print(B @ A)
```

    [[22 34]
     [46 74]]
    [[20 28]
     [52 76]]

### 4.2 Identity Matrices
An *identity* matrix (usually indicated by a capital **I**) is the equivalent in matrix terms of the number **1**. It always has the same number of rows as columns, and it has the value **1** in the diagonal element positions I1,1, I2,2, etc; and 0 in all other element positions. Here's an example of a 3x3 identity matrix:

\begin{equation}\begin{bmatrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{bmatrix} \end{equation}

Multiplying any matrix by an identity matrix is the same as multiplying a number by 1; the result is the same as the original value:

\begin{equation}\begin{bmatrix}1 & 2 & 3 \\4 & 5 & 6\\7 & 8 & 9\end{bmatrix} \cdot \begin{bmatrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{bmatrix} = \begin{bmatrix}1 & 2 & 3 \\4 & 5 & 6\\7 & 8 & 9\end{bmatrix} \end{equation}

If you doubt me, try the following Python code!

```python
A = np.array([[1,2,3],
              [4,5,6],
              [7,8,9]])
B = np.array([[1,0,0],
              [0,1,0],
              [0,0,1]])
print(A @ B)
```

    [[1 2 3]
     [4 5 6]
     [7 8 9]]

### 4.3 Matrix Division
You can't actually divide by a matrix; but when you want to divide matrices, you can take advantage of the fact that division by a given number is the same as multiplication by the reciprocal of that number. For example:

\begin{equation}6 \div 3 = \frac{1}{3}\times 6 \end{equation}

In this case, 1/3 is the reciprocal of 3 (which as a fraction is 3/1 - we "flip" the numerator and denominator to get the reciprocal). You can also write 1/3 as 3-1.

#### 4.3.1 Inverse of a Matrix
For matrix division, we use a related idea; we multiply by the *inverse* of a matrix:

\begin{equation}A \div B = A \cdot B^{-1}\end{equation}

The inverse of B is B-1 as long as the following equation is true:

\begin{equation}B \cdot B^{-1} = B^{-1} \cdot B = I\end{equation}

**I**, you may recall, is an *identity* matrix; the matrix equivalent of 1.

So how do you calculate the inverse of a matrix? For a 2x2 matrix, you can follow this formula:

\begin{equation}\begin{bmatrix}a & b\\c & d\end{bmatrix}^{-1} = \frac{1}{ad-bc}  \begin{bmatrix}d & -b\\-c & a\end{bmatrix}\end{equation}

What happened there?
- We swapped the positions of *a* and *d*
- We changed the signs of *b* and *c*
- We multiplied the resulting matrix by 1 over the *determinant* of the matrix (*ad-bc*)

Let's try with some actual numbers:

\begin{equation}\begin{bmatrix}6 & 2\\1 & 2\end{bmatrix}^{-1} = \frac{1}{(6\times2)-(2\times1)}  \begin{bmatrix}2 & -2\\-1 & 6\end{bmatrix}\end{equation}

So:

\begin{equation}\begin{bmatrix}6 & 2\\1 & 2\end{bmatrix}^{-1} = \frac{1}{10}  \begin{bmatrix}2 & -2\\-1 & 6\end{bmatrix}\end{equation}

Which gives us the result:

\begin{equation}\begin{bmatrix}6 & 2\\1 & 2\end{bmatrix}^{-1} = \begin{bmatrix}0.2 & -0.2\\-0.1 & 0.6\end{bmatrix}\end{equation}

To check this, we can multiply the original matrix by its inverse to see if we get an identity matrix. This makes sense if you think about it; in the same way that 3 x 1/3 = 1, a matrix multiplied by its inverse results in an identity matrix:

\begin{equation}\begin{bmatrix}6 & 2\\1 & 2\end{bmatrix} \cdot \begin{bmatrix}0.2 & -0.2\\-0.1 & 0.6\end{bmatrix} = \begin{bmatrix}(6\times0.2)+(2\times-0.1) & (6\times-0.2)+(2\times0.6)\\(1\times0.2)+(2\times-0.1) & (1\times-0.2)+(2\times0.6)\end{bmatrix} = \begin{bmatrix}1 & 0\\0 & 1\end{bmatrix}\end{equation}

Note that not every matrix has an inverse - for example, if the determinant works out to be 0, the inverse matrix is not defined.

In Python, you can use the **numpy.linalg.inv** function to get the inverse of a matrix in an **array** or **matrix** object:

```python
B = np.array([[6,2],
              [1,2]])

print(np.linalg.inv(B))
```

    [[ 0.2 -0.2]
     [-0.1  0.6]]

Additionally, the *matrix* type has an ***I*** method that returns the inverse matrix:

```python
B = np.matrix([[6,2],
              [1,2]])

print(B.I)
```

    [[ 0.2 -0.2]
     [-0.1  0.6]]

For larger matrices, the process to calculate the inverse is more complex. Let's explore an example based on the following matrix:

\begin{equation}\begin{bmatrix}4 & 2 & 2\\6 & 2 & 4\\2 & 2 & 8\end{bmatrix} \end{equation}

The process to find the inverse consists of the following steps:

1: Create a matrix of *minors* by calculating the *determinant* for each element in the matrix based on the elements that are not in the same row or column; like this:

\begin{equation}\begin{bmatrix}\color{blue}4 & \color{lightgray}2 & \color{lightgray}2\\\color{lightgray}6 & \color{red}2 & \color{red}4\\\color{lightgray}2 & \color{red}2 & \color{red}8\end{bmatrix}\;\;\;\;(2\times8) - (4\times2) = 8\;\;\;\;\begin{bmatrix}8 & \color{lightgray}? & \color{lightgray}?\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\end{bmatrix} \end{equation}

\begin{equation}\begin{bmatrix}\color{lightgray}4 & \color{blue}2 & \color{lightgray}2\\\color{red}6 & \color{lightgray}2 & \color{red}4\\\color{red}2 & \color{lightgray}2 & \color{red}8\end{bmatrix}\;\;\;\;(6\times8) - (4\times2) = 40\;\;\;\;\begin{bmatrix}8 & 40 & \color{lightgray}?\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\end{bmatrix}\end{equation}

\begin{equation}\begin{bmatrix}\color{lightgray}4 & \color{lightgray}2 & \color{blue}2\\\color{red}6 & \color{red}2 & \color{lightgray}4\\\color{red}2 & \color{red}2 & \color{lightgray}8\end{bmatrix}\;\;\;\;(6\times2) - (2\times2) = 8\;\;\;\;\begin{bmatrix}8 & 40 & 8\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\end{bmatrix} \end{equation}

\begin{equation}\begin{bmatrix}\color{lightgray}4 & \color{red}2 & \color{red}2\\\color{blue}6 & \color{lightgray}2 & \color{lightgray}4\\\color{lightgray}2 & \color{red}2 & \color{red}8\end{bmatrix}\;\;\;\;(2\times8) - (2\times2) = 12\;\;\;\;\begin{bmatrix}8 & 40 & 8\\12 & \color{lightgray}? & \color{lightgray}?\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\end{bmatrix} \end{equation}

\begin{equation}\begin{bmatrix}\color{red}4 & \color{lightgray}2 & \color{red}2\\\color{lightgray}6 & \color{blue}2 & \color{lightgray}4\\\color{red}2 & \color{lightgray}2 & \color{red}8\end{bmatrix}\;\;\;\;(4\times8) - (2\times2) = 28\;\;\;\;\begin{bmatrix}8 & 40 & 8\\12 & 28 & \color{lightgray}?\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\end{bmatrix} \end{equation}

\begin{equation}\begin{bmatrix}\color{red}4 & \color{red}2 & \color{lightgray}2\\\color{lightgray}6 & \color{lightgray}2 & \color{blue}4\\\color{red}2 & \color{red}2 & \color{lightgray}8\end{bmatrix}\;\;\;\;(4\times2) - (2\times2) = 4\;\;\;\;\begin{bmatrix}8 & 40 & 8\\12 & 28 & 4\\\color{lightgray}? & \color{lightgray}? & \color{lightgray}?\end{bmatrix} \end{equation}

\begin{equation}\begin{bmatrix}\color{lightgray}4 & \color{red}2 & \color{red}2\\\color{lightgray}6 & \color{red}2 & \color{red}4\\\color{blue}2 & \color{lightgray}2 & \color{lightgray}8\end{bmatrix}\;\;\;\;(2\times4) - (2\times2) = 4\;\;\;\;\begin{bmatrix}8 & 40 & 8\\12 & 28 & 4\\4 & \color{lightgray}? & \color{lightgray}?\end{bmatrix} \end{equation}

\begin{equation}\begin{bmatrix}\color{red}4 & \color{lightgray}2 & \color{red}2\\\color{red}6 & \color{lightgray}2 & \color{red}4\\\color{lightgray}2 & \color{blue}2 & \color{lightgray}8\end{bmatrix}\;\;\;\;(4\times4) - (2\times6) = 4\;\;\;\;\begin{bmatrix}8 & 40 & 8\\12 & 28 & 4\\4 & 4 & \color{lightgray}?\end{bmatrix} \end{equation}

\begin{equation}\begin{bmatrix}\color{red}4 & \color{red}2 & \color{lightgray}2\\\color{red}6 & \color{red}2 & \color{lightgray}4\\\color{lightgray}2 & \color{lightgray}2 & \color{blue}8\end{bmatrix}\;\;\;\;(4\times2) - (2\times6) = -4\;\;\;\;\begin{bmatrix}8 & 40 & 8\\12 & 28 & 4\\4 & 4 & -4\end{bmatrix} \end{equation}

2: Apply *cofactors* to the matrix by switching the sign of every alternate element in the matrix of minors:

\begin{equation}\begin{bmatrix}8 & -40 & 8\\-12 & 28 & -4\\4 & -4 & -4\end{bmatrix} \end{equation}

3: *Adjugate* by transposing elements diagonally:

\begin{equation}\begin{bmatrix}8 & \color{green}-\color{green}1\color{green}2 & \color{orange}4\\\color{green}-\color{green}4\color{green}0 & 28 & \color{purple}-\color{purple}4\\\color{orange}8 & \color{purple}-\color{purple}4 & -4\end{bmatrix} \end{equation}

4: Multiply by 1/determinant of the original matrix. To find this, multiply each of the top row elements by their corresponding minor determinants (which we calculated earlier in the matrix of minors), and then subtract the second from the first and add the third:

\begin{equation}Determinant = (4 \times 8) - (2 \times 40) + (2 \times 8) = -32\end{equation}

\begin{equation}\frac{1}{-32}\begin{bmatrix}8 & -12 & 4\\-40 & 28 & -4\\8 & -4 & -4\end{bmatrix} =  \begin{bmatrix}-0.25 & 0.375 & -0.125\\1.25 & -0.875 & 0.125\\-0.25 & 0.125 & 0.125\end{bmatrix}\end{equation}

Let's verify that the original matrix multiplied by the inverse results in an identity matrix:

\begin{equation}\begin{bmatrix}4 & 2 & 2\\6 & 2 & 4\\2 & 2 & 8\end{bmatrix} \cdot \begin{bmatrix}-0.25 & 0.375 & -0.125\\1.25 & -0.875 & 0.125\\-0.25 & 0.125 & 0.125\end{bmatrix}\end{equation}

\begin{equation}= \begin{bmatrix}(4\times-0.25)+(2\times1.25)+(2\times-0.25) & (4\times0.375)+(2\times-0.875)+(2\times0.125) & (4\times-0.125)+(2\times-0.125)+(2\times0.125)\\(6\times-0.25)+(2\times1.25)+(4\times-0.25) & (6\times0.375)+(2\times-0.875)+(4\times0.125) & (6\times-0.125)+(2\times-0.125)+(4\times0.125)\\(2\times-0.25)+(2\times1.25)+(8\times-0.25) & (2\times0.375)+(2\times-0.875)+(8\times0.125) & (2\times-0.125)+(2\times-0.125)+(8\times0.125)\end{bmatrix} \end{equation}

\begin{equation}= \begin{bmatrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{bmatrix} \end{equation}

As you can see, this can get pretty complicated - which is why we usually use a calculator or a computer program. You can run the following Python code to verify that the inverse matrix we calculated is correct:

```python
B = np.array([[4,2,2],
              [6,2,4],
              [2,2,8]])

print(np.linalg.inv(B))
```

    [[-0.25   0.375 -0.125]
     [ 1.25  -0.875  0.125]
     [-0.25   0.125  0.125]]

#### 4.3.2 Multiplying by an Inverse Matrix
Now that you know how to calculate an inverse matrix, you can use that knowledge to multiply the inverse of a matrix by another matrix as an alternative to division:

\begin{equation}\begin{bmatrix}1 & 2\\3 & 4\end{bmatrix} \cdot \begin{bmatrix}6 & 2\\1 & 2\end{bmatrix}^{-1} \end{equation}

\begin{equation}=\begin{bmatrix}1 & 2\\3 & 4\end{bmatrix} \cdot \begin{bmatrix}0.2 & -0.2\\-0.1 & 0.6\end{bmatrix}  \end{equation}

\begin{equation}=\begin{bmatrix}(1\times0.2)+(2\times-0.1) & (1\times-0.2)+(2\times0.6)\\(3\times0.2)+(4\times-0.1) & (3\times-0.2)+(4\times0.6)\end{bmatrix}\end{equation}

\begin{equation}=\begin{bmatrix}0 & 1\\0.2 & 1.8\end{bmatrix}\end{equation}

Here's the Python code to calculate this:

```python
A = np.array([[1,2],
              [3,4]])

B = np.array([[6,2],
              [1,2]])

C = A @ np.linalg.inv(B)

print(C)
```

    [[0.  1. ]
     [0.2 1.8]]

### 4.4 Solving Systems of Equations with Matrices
One of the great things about matrices, is that they can help us solve systems of equations. For example, consider the following system of equations:

\begin{equation}2x + 4y = 18\end{equation}
\begin{equation}6x + 2y = 34\end{equation}

We can write this in matrix form, like this:

\begin{equation}\begin{bmatrix}2 & 4\\6 & 2\end{bmatrix} \cdot \begin{bmatrix}x\\y\end{bmatrix}=\begin{bmatrix}18\\34\end{bmatrix}\end{equation}

Note that the variables (***x*** and ***y***) are  arranged as a column in one matrix, which is multiplied by a matrix containing the coefficients to produce as matrix containing the results. If you calculate the dot product on the left side, you can see clearly that this represents the original equations:

\begin{equation}\begin{bmatrix}2x + 4y\\6x + 2y\end{bmatrix} =\begin{bmatrix}18\\34\end{bmatrix}\end{equation}

Now. let's name our matrices so we can better understand what comes next:

\begin{equation}A=\begin{bmatrix}2 & 4\\6 & 2\end{bmatrix}\;\;\;\;X=\begin{bmatrix}x\\y\end{bmatrix}\;\;\;\;B=\begin{bmatrix}18\\34\end{bmatrix}\end{equation}

We already know that ***A • X = B***, which arithmetically means that ***X = B ÷ A***. Since we can't actually divide by a matrix, we need to multiply by the inverse; so we can find the values for our variables (*X*) like this: ***X = A-1 • B***

So, first we need the inverse of A:

\begin{equation}\begin{bmatrix}2 & 4\\6 & 2\end{bmatrix}^{-1} = \frac{1}{(2\times2)-(4\times6)}  \begin{bmatrix}2 & -4\\-6 & 2\end{bmatrix}\end{equation}

\begin{equation}= \frac{1}{-20}  \begin{bmatrix}2 & -4\\-6 & 2\end{bmatrix}\end{equation}

\begin{equation}=\begin{bmatrix}-0.1 & 0.2\\0.3 & -0.1\end{bmatrix}\end{equation}

Then we just multiply this with B:

\begin{equation}X = \begin{bmatrix}-0.1 & 0.2\\0.3 & -0.1\end{bmatrix} \cdot \begin{bmatrix}18\\34\end{bmatrix}\end{equation}

\begin{equation}X = \begin{bmatrix}(-0.1 \times 18)+(0.2 \times 34)\\(0.3\times18)+(-0.1\times34)\end{bmatrix}\end{equation}

\begin{equation}X = \begin{bmatrix}5\\2\end{bmatrix}\end{equation}

The resulting matrix (*X*) contains the values for our *x* and *y* variables, and we can check these by plugging them into the original equations:

\begin{equation}(2\times5) + (4\times2) = 18\end{equation}
\begin{equation}(6\times5) + (2\times2) = 34\end{equation}

These of course simplify to:

\begin{equation}10 + 8 = 18\end{equation}
\begin{equation}30 + 4 = 34\end{equation}

So our variable values are correct.

Here's the Python code to do all of this:

```python
A = np.array([[2,4],
              [6,2]])

B = np.array([[18],
              [34]])

C = np.linalg.inv(A) @ B

print(C)
```

    [[5.]
     [2.]]

## 5 Transformations, Eigenvectors, and Eigenvalues

Matrices and vectors are used together to manipulate spatial dimensions. This has a lot of applications, including the mathematical generation of 3D computer graphics, geometric modeling, and the training and optimization of machine learning algorithms. We're not going to cover the subject exhaustively here; but we'll focus on a few key concepts that are useful to know when you plan to work with machine learning.

### 5.1 Linear Transformations
You can manipulate a vector by multiplying it with a matrix. The matrix acts a function that operates on an input vector to produce a vector output. Specifically, matrix multiplications of vectors are *linear transformations* that transform the input vector into the output vector.

For example, consider this matrix ***A*** and vector ***v***:

$$ A = \begin{bmatrix}2 & 3\\5 & 2\end{bmatrix} \;\;\;\; \vec{v} = \begin{bmatrix}1\\2\end{bmatrix}$$

We can define a transformation ***T*** like this:

$$ T(\vec{v}) = A\vec{v} $$

To perform this transformation, we simply calculate the dot product by applying the *RC* rule; multiplying each row of the matrix by the single column of the vector:

$$\begin{bmatrix}2 & 3\\5 & 2\end{bmatrix} \cdot  \begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}8\\9\end{bmatrix}$$

Here's the calculation in Python:

```python
v = np.array([1,2])
A = np.array([[2,3],
              [5,2]])

t = A@v
print (t)
```

    [8 9]

In this case, both the input vector and the output vector have 2 components - in other words, the transformation takes a 2-dimensional vector and produces a new 2-dimensional vector; which we can indicate like this:

$$ T: \rm I\!R^{2} \to \rm I\!R^{2} $$

Note that the output vector may have a different number of dimensions from the input vector; so the matrix function might transform the vector from one space to another - or in notation, ${\rm I\!R}$n -> ${\rm I\!R}$m.

For example, let's redefine matrix ***A***, while retaining our original definition of vector ***v***:

$$ A = \begin{bmatrix}2 & 3\\5 & 2\\1 & 1\end{bmatrix} \;\;\;\; \vec{v} = \begin{bmatrix}1\\2\end{bmatrix}$$

Now if we once again define ***T*** like this:

$$ T(\vec{v}) = A\vec{v} $$

We apply the transformation like this:

$$\begin{bmatrix}2 & 3\\5 & 2\\1 & 1\end{bmatrix} \cdot  \begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}8\\9\\3\end{bmatrix}$$

So now, our transformation transforms the vector from 2-dimensional space to 3-dimensional space:

$$ T: \rm I\!R^{2} \to \rm I\!R^{3} $$

Here it is in Python:

```python
v = np.array([1,2])
A = np.array([[2,3],
              [5,2],
              [1,1]])

t = A@v
print (t)
```

    [8 9 3]

```python
v = np.array([1,2])
A = np.array([[1,2],
              [2,1]])

t = A@v
print (t)
```

    [5 4]

### 5.2 Transformations of Magnitude and Amplitude

When you multiply a vector by a matrix, you transform it in at least one of the following two ways:
* Scale the length (*magnitude*) of the matrix to make it longer or shorter
* Change the direction (*amplitude*) of the matrix

For example consider the following matrix and vector:

$$ A = \begin{bmatrix}2 & 0\\0 & 2\end{bmatrix} \;\;\;\; \vec{v} = \begin{bmatrix}1\\0\end{bmatrix}$$

As before, we transform the vector ***v*** by multiplying it with the matrix ***A***:

\begin{equation}\begin{bmatrix}2 & 0\\0 & 2\end{bmatrix} \cdot  \begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}2\\0\end{bmatrix}\end{equation}

In this case, the resulting vector has changed in length (*magnitude*), but has not changed its direction (*amplitude*).

Let's visualize that in Python:

```python
v = np.array([1,0])
A = np.array([[2,0],
              [0,2]])

t = A@v
print (t)

# Plot v and t
vecs = np.array([t,v])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
```

The original vector ***v*** is shown in orange, and the transformed vector ***t*** is shown in blue - note that ***t*** has the same direction (*amplitude*) as ***v*** but a greater length (*magnitude*).

Now let's use a different matrix to transform the vector ***v***:
\begin{equation}\begin{bmatrix}0 & -1\\1 & 0\end{bmatrix} \cdot  \begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}0\\1\end{bmatrix}\end{equation}

This time, the resulting vector has been changed to a different amplitude, but has the same magnitude.

```python
v = np.array([1,0])
A = np.array([[0,-1],
              [1,0]])

t = A@v
print (t)

# Plot v and t
vecs = np.array([v,t])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['orange', 'blue'], scale=10)
plt.show()
```

Now let's see change the matrix one more time:
\begin{equation}\begin{bmatrix}2 & 1\\1 & 2\end{bmatrix} \cdot  \begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}2\\1\end{bmatrix}\end{equation}

Now our resulting vector has been transformed to a new amplitude *and* magnitude - the transformation has affected both direction and scale.

```python
v = np.array([1,0])
A = np.array([[2,1],
              [1,2]])

t = A@v
print (t)

# Plot v and t
vecs = np.array([v,t])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['orange', 'blue'], scale=10)
plt.show()
```

#### 5.2.1 Afine Transformations
An Afine transformation multiplies a vector by a matrix and adds an offset vector, sometimes referred to as *bias*; like this:

$$T(\vec{v}) = A\vec{v} + \vec{b}$$

For example:

\begin{equation}\begin{bmatrix}5 & 2\\3 & 1\end{bmatrix} \cdot  \begin{bmatrix}1\\1\end{bmatrix} + \begin{bmatrix}-2\\-6\end{bmatrix} = \begin{bmatrix}5\\-2\end{bmatrix}\end{equation}

This kind of transformation is actually the basis of linear regression, which is a core foundation for machine learning. The matrix defines the *features*, the first vector is the *coefficients*, and the bias vector is the *intercept*.

here's an example of an Afine transformation in Python:

```python
v = np.array([1,1])
A = np.array([[5,2],
              [3,1]])
b = np.array([-2,-6])

t = A@v + b
print (t)

# Plot v and t
vecs = np.array([v,t])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['orange', 'blue'], scale=15)
plt.show()
```

### 5.3 Eigenvectors and Eigenvalues
So we can see that when you transform a vector using a matrix, we change its direction, length, or both. When the transformation only affects scale (in other words, the output vector has a different magnitude but the same amplitude as the input vector), the matrix multiplication for the transformation is the equivalent operation as some scalar multiplication of the vector.

For example, earlier we examined the following transformation that dot-mulitplies a vector by a matrix:

$$\begin{bmatrix}2 & 0\\0 & 2\end{bmatrix} \cdot  \begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}2\\0\end{bmatrix}$$

You can achieve the same result by mulitplying the vector by the scalar value ***2***:

$$2 \times \begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}2\\0\end{bmatrix}$$

The following python performs both of these calculation and shows the results, which are identical.

```python
v = np.array([1,0])
A = np.array([[2,0],
              [0,2]])

t1 = A@v
print (t1)
t2 = 2*v
print (t2)

fig = plt.figure()
a=fig.add_subplot(1,1,1)
# Plot v and t1
vecs = np.array([t1,v])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
a=fig.add_subplot(1,2,1)
# Plot v and t2
vecs = np.array([t2,v])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
```

In cases like these, where a matrix transformation is the equivelent of a scalar-vector multiplication, the scalar-vector pairs that correspond to the matrix are known respectively as eigenvalues and eigenvectors. We generally indicate eigenvalues using the Greek letter lambda (λ), and the formula that defines eigenvalues and eigenvectors with respect to a transformation is:

$$ T(\vec{v}) = \lambda\vec{v}$$

Where the vector ***v*** is an eigenvector and the value ***λ*** is an eigenvalue for transformation ***T***.

When the transformation ***T*** is represented as a matrix multiplication, as in this case where the transformation is represented by matrix ***A***:

$$ T(\vec{v}) = A\vec{v} = \lambda\vec{v}$$

Then  ***v*** is an eigenvector and ***λ*** is an eigenvalue of ***A***.

A matrix can have multiple eigenvector-eigenvalue pairs, and you can calculate them manually. However, it's generally easier to use a tool or programming language. For example, in Python you can use the ***linalg.eig*** function, which returns an array of eigenvalues and a matrix of the corresponding eigenvectors for the specified matrix.

Here's an example that returns the eigenvalue and eigenvector pairs for the following matrix:

$$A=\begin{bmatrix}2 & 0\\0 & 3\end{bmatrix}$$

```python
A = np.array([[2,0],
              [0,3]])
eVals, eVecs = np.linalg.eig(A)
print(eVals)
print(eVecs)
```

    [2. 3.]
    [[1. 0.]
     [0. 1.]]

So there are two eigenvalue-eigenvector pairs for this matrix, as shown here:

$$ \lambda_{1} = 2, \vec{v_{1}} = \begin{bmatrix}1 \\ 0\end{bmatrix}  \;\;\;\;\;\; \lambda_{2} = 3, \vec{v_{2}} = \begin{bmatrix}0 \\ 1\end{bmatrix} $$

Let's verify that multiplying each eigenvalue-eigenvector pair corresponds to the dot-product of the eigenvector and the matrix. Here's the first pair:

$$ 2 \times \begin{bmatrix}1 \\ 0\end{bmatrix} = \begin{bmatrix}2 \\ 0\end{bmatrix}  \;\;\;and\;\;\; \begin{bmatrix}2 & 0\\0 & 3\end{bmatrix} \cdot \begin{bmatrix}1 \\ 0\end{bmatrix} = \begin{bmatrix}2 \\ 0\end{bmatrix} $$

So far so good. Now let's check the second pair:

$$ 3 \times \begin{bmatrix}0 \\ 1\end{bmatrix} = \begin{bmatrix}0 \\ 3\end{bmatrix}  \;\;\;and\;\;\; \begin{bmatrix}2 & 0\\0 & 3\end{bmatrix} \cdot \begin{bmatrix}0 \\ 1\end{bmatrix} = \begin{bmatrix}0 \\ 3\end{bmatrix} $$

So our eigenvalue-eigenvector scalar multiplications do indeed correspond to our matrix-eigenvector dot-product transformations.

Here's the equivalent code in Python, using the ***eVals*** and ***eVecs*** variables you generated in the previous code cell:

```python
vec1 = eVecs[:,0]
lam1 = eVals[0]

print('Matrix A:')
print(A)
print('-------')

print('lam1: ' + str(lam1))
print ('v1: ' + str(vec1))
print ('Av1: ' + str(A@vec1))
print ('lam1 x v1: ' + str(lam1*vec1))

print('-------')

vec2 = eVecs[:,1]
lam2 = eVals[1]

print('lam2: ' + str(lam2))
print ('v2: ' + str(vec2))
print ('Av2: ' + str(A@vec2))
print ('lam2 x v2: ' + str(lam2*vec2))
```

    Matrix A:
    [[2 0]
     [0 3]]
    -------
    lam1: 2.0
    v1: [1. 0.]
    Av1: [2. 0.]
    lam1 x v1: [2. 0.]
    -------
    lam2: 3.0
    v2: [0. 1.]
    Av2: [0. 3.]
    lam2 x v2: [0. 3.]

You can use the following code to visualize these transformations:

```python
t1 = lam1*vec1
print (t1)
t2 = lam2*vec2
print (t2)

fig = plt.figure()
a=fig.add_subplot(1,1,1)
# Plot v and t1
vecs = np.array([t1,vec1])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
a=fig.add_subplot(1,2,1)
# Plot v and t2
vecs = np.array([t2,vec2])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
```

Similarly, earlier we examined the following matrix transformation:

$$\begin{bmatrix}2 & 0\\0 & 2\end{bmatrix} \cdot  \begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}2\\0\end{bmatrix}$$

And we saw that you can achieve the same result by mulitplying the vector by the scalar value ***2***:

$$2 \times \begin{bmatrix}1\\0\end{bmatrix} = \begin{bmatrix}2\\0\end{bmatrix}$$

This works because the scalar value 2 and the vector (1,0) are an eigenvalue-eigenvector pair for this matrix.

Let's use Python to determine the eigenvalue-eigenvector pairs for this matrix:

```python
A = np.array([[2,0],
              [0,2]])
eVals, eVecs = np.linalg.eig(A)
print(eVals)
print(eVecs)
```

    [2. 2.]
    [[1. 0.]
     [0. 1.]]

So once again, there are two eigenvalue-eigenvector pairs for this matrix, as shown here:

$$ \lambda_{1} = 2, \vec{v_{1}} = \begin{bmatrix}1 \\ 0\end{bmatrix}  \;\;\;\;\;\; \lambda_{2} = 2, \vec{v_{2}} = \begin{bmatrix}0 \\ 1\end{bmatrix} $$

Let's verify that multiplying each eigenvalue-eigenvector pair corresponds to the dot-product of the eigenvector and the matrix. Here's the first pair:

$$ 2 \times \begin{bmatrix}1 \\ 0\end{bmatrix} = \begin{bmatrix}2 \\ 0\end{bmatrix}  \;\;\;and\;\;\; \begin{bmatrix}2 & 0\\0 & 2\end{bmatrix} \cdot \begin{bmatrix}1 \\ 0\end{bmatrix} = \begin{bmatrix}2 \\ 0\end{bmatrix} $$

Well, we already knew that. Now let's check the second pair:

$$ 2 \times \begin{bmatrix}0 \\ 1\end{bmatrix} = \begin{bmatrix}0 \\ 2\end{bmatrix}  \;\;\;and\;\;\; \begin{bmatrix}2 & 0\\0 & 2\end{bmatrix} \cdot \begin{bmatrix}0 \\ 1\end{bmatrix} = \begin{bmatrix}0 \\ 2\end{bmatrix} $$

Now let's use Pythonto verify and plot these transformations:

```python
vec1 = eVecs[:,0]
lam1 = eVals[0]

print('Matrix A:')
print(A)
print('-------')

print('lam1: ' + str(lam1))
print ('v1: ' + str(vec1))
print ('Av1: ' + str(A@vec1))
print ('lam1 x v1: ' + str(lam1*vec1))

print('-------')

vec2 = eVecs[:,1]
lam2 = eVals[1]

print('lam2: ' + str(lam2))
print ('v2: ' + str(vec2))
print ('Av2: ' + str(A@vec2))
print ('lam2 x v2: ' + str(lam2*vec2))

# Plot the resulting vectors
t1 = lam1*vec1
t2 = lam2*vec2

fig = plt.figure()
a=fig.add_subplot(1,1,1)
# Plot v and t1
vecs = np.array([t1,vec1])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
a=fig.add_subplot(1,2,1)
# Plot v and t2
vecs = np.array([t2,vec2])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
```

Let's take a look at one more, slightly more complex example. Here's our matrix:

$$\begin{bmatrix}2 & 1\\1 & 2\end{bmatrix}$$

Let's get the eigenvalue and eigenvector pairs:

```python
A = np.array([[2,1],
              [1,2]])

eVals, eVecs = np.linalg.eig(A)
print(eVals)
print(eVecs)
```

    [3. 1.]
    [[ 0.70710678 -0.70710678]
     [ 0.70710678  0.70710678]]

This time the eigenvalue-eigenvector pairs are:

$$ \lambda_{1} = 3, \vec{v_{1}} = \begin{bmatrix}0.70710678 \\ 0.70710678\end{bmatrix}  \;\;\;\;\;\; \lambda_{2} = 1, \vec{v_{2}} = \begin{bmatrix}-0.70710678 \\ 0.70710678\end{bmatrix} $$

So let's check the first pair:

$$ 3 \times \begin{bmatrix}0.70710678 \\ 0.70710678\end{bmatrix} = \begin{bmatrix}2.12132034 \\ 2.12132034\end{bmatrix}  \;\;\;and\;\;\; \begin{bmatrix}2 & 1\\0 & 2\end{bmatrix} \cdot \begin{bmatrix}0.70710678 \\ 0.70710678\end{bmatrix} = \begin{bmatrix}2.12132034 \\ 2.12132034\end{bmatrix} $$

Now let's check the second pair:

$$ 1 \times \begin{bmatrix}-0.70710678 \\ 0.70710678\end{bmatrix} = \begin{bmatrix}-0.70710678\\0.70710678\end{bmatrix}  \;\;\;and\;\;\; \begin{bmatrix}2 & 1\\1 & 2\end{bmatrix} \cdot \begin{bmatrix}-0.70710678 \\ 0.70710678\end{bmatrix} = \begin{bmatrix}-0.70710678\\0.70710678\end{bmatrix} $$

With more complex examples like this, it's generally easier to do it with Python:

```python
vec1 = eVecs[:,0]
lam1 = eVals[0]

print('Matrix A:')
print(A)
print('-------')

print('lam1: ' + str(lam1))
print ('v1: ' + str(vec1))
print ('Av1: ' + str(A@vec1))
print ('lam1 x v1: ' + str(lam1*vec1))

print('-------')

vec2 = eVecs[:,1]
lam2 = eVals[1]

print('lam2: ' + str(lam2))
print ('v2: ' + str(vec2))
print ('Av2: ' + str(A@vec2))
print ('lam2 x v2: ' + str(lam2*vec2))

# Plot the results
t1 = lam1*vec1
t2 = lam2*vec2

fig = plt.figure()
a=fig.add_subplot(1,1,1)
# Plot v and t1
vecs = np.array([t1,vec1])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
a=fig.add_subplot(1,2,1)
# Plot v and t2
vecs = np.array([t2,vec2])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['blue', 'orange'], scale=10)
plt.show()
```

### 5.4 Eigendecomposition
So we've learned a little about eigenvalues and eigenvectors; but you may be wondering what use they are. Well, one use for them is to help decompose transformation matrices.

Recall that previously we found that a matrix transformation of a vector changes its magnitude, amplitude, or both. Without getting too technical about it, we need to remember that vectors can exist in any spatial orientation, or *basis*; and the same transformation can be applied in different *bases*.

We can decompose a matrix using the following formula:

$$A = Q \Lambda Q^{-1}$$

Where ***A*** is a trasformation that can be applied to a vector in its current base, ***Q*** is a matrix of eigenvectors that defines a change of basis, and ***Λ*** is a matrix with eigenvalues on the diagonal that defines the same linear transformation as ***A*** in the base defined by ***Q***.

Let's look at these in some more detail. Consider this matrix:

$$A=\begin{bmatrix}3 & 2\\1 & 0\end{bmatrix}$$

***Q*** is a matrix in which each column is an eigenvector of ***A***; which as we've seen previously, we can calculate using Python:

```python
A = np.array([[3,2],
              [1,0]])

l, Q = np.linalg.eig(A)
print(Q)
```

    [[ 0.96276969 -0.48963374]
     [ 0.27032301  0.87192821]]

So for matrix ***A***, ***Q*** is the following matrix:

$$Q=\begin{bmatrix}0.96276969 & -0.48963374\\0.27032301 & 0.87192821\end{bmatrix}$$

***Λ*** is a matrix that contains the eigenvalues for ***A*** on the diagonal, with zeros in all other elements; so for a 2x2 matrix, Λ will look like this:

$$\Lambda=\begin{bmatrix}\lambda_{1} & 0\\0 & \lambda_{2}\end{bmatrix}$$

In our Python code, we've already used the ***linalg.eig*** function to return the array of eigenvalues for ***A*** into the variable ***l***, so now we just need to format that as a matrix:

```python
L = np.diag(l)
print (L)
```

    [[ 3.56155281  0.        ]
     [ 0.         -0.56155281]]

So ***Λ*** is the following matrix:

$$\Lambda=\begin{bmatrix}3.56155281 & 0\\0 & -0.56155281\end{bmatrix}$$

Now we just need to find ***Q-1***, which is the inverse of ***Q***:

```python
Qinv = np.linalg.inv(Q)
print(Qinv)
```

    [[ 0.89720673  0.50382896]
     [-0.27816009  0.99068183]]

The inverse of ***Q*** then, is:

$$Q^{-1}=\begin{bmatrix}0.89720673 & 0.50382896\\-0.27816009 & 0.99068183\end{bmatrix}$$

So what does that mean? Well, it means that we can decompose the transformation of *any* vector multiplied by matrix ***A*** into the separate operations ***QΛQ-1***:

$$A\vec{v} = Q \Lambda Q^{-1}\vec{v}$$

To prove this, let's take vector ***v***:

$$\vec{v} = \begin{bmatrix}1\\3\end{bmatrix} $$

Our matrix transformation using ***A*** is:

$$\begin{bmatrix}3 & 2\\1 & 0\end{bmatrix} \cdot \begin{bmatrix}1\\3\end{bmatrix} $$

So let's show the results of that using Python:

```python
v = np.array([1,3])
t = A@v

print(t)

# Plot v and t
vecs = np.array([v,t])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['orange', 'b'], scale=20)
plt.show()
```

And now, let's do the same thing using the ***QΛQ-1*** sequence of operations:

```python
t = (Q@(L@(Qinv)))@v

# Plot v and t
vecs = np.array([v,t])
origin = [0,0], [0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['orange', 'b'], scale=20)
plt.show()
```

So ***A*** and ***QΛQ-1*** are equivalent.

If we view the intermediary stages of the decomposed transformation, you can see the transformation using ***A*** in the original base for ***v*** (orange to blue) and the transformation using ***Λ*** in the change of basis decribed by ***Q*** (red to magenta):

```python
t1 = Qinv@v
t2 = L@t1
t3 = Q@t2

# Plot the transformations
vecs = np.array([v,t1, t2, t3])
origin = [0,0,0,0], [0,0,0,0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, vecs[:,0], vecs[:,1], color=['orange', 'red', 'magenta', 'blue'], scale=20)
plt.show()
```

So from this visualization, it should be apparent that the transformation ***Av*** can be performed by changing the basis for ***v*** using ***Q*** (from orange to red in the above plot) applying the equivalent linear transformation in that base using ***Λ*** (red to magenta), and switching back to the original base using ***Q-1*** (magenta to blue).

### 5.5 Rank of a Matrix

The **rank** of a square matrix is the number of non-zero eigenvalues of the matrix. A **full rank** matrix has the same number of non-zero eigenvalues as the dimension of the matrix. A **rank-deficient** matrix has fewer non-zero eigenvalues as dimensions. The inverse of a rank deficient matrix is singular and so does not exist (this is why in a previous notebook we noted that some matrices have no inverse).

Consider the following matrix ***A***:

$$A=\begin{bmatrix}1 & 2\\4 & 3\end{bmatrix}$$

Let's find its eigenvalues (***Λ***):

```python
A = np.array([[1,2],
              [4,3]])
l, Q = np.linalg.eig(A)
L = np.diag(l)
print(L)
```

    [[-1.  0.]
     [ 0.  5.]]

$$\Lambda=\begin{bmatrix}-1 & 0\\0 & 5\end{bmatrix}$$

This matrix has full rank. The dimensions of the matrix is 2. There are two non-zero eigenvalues.

Now consider this matrix:

$$B=\begin{bmatrix}3 & -3 & 6\\2 & -2 & 4\\1 & -1 & 2\end{bmatrix}$$

Note that the second and third columns are just scalar multiples of the first column.

Let's examine it's eigenvalues:

```python
B = np.array([[3,-3,6],
              [2,-2,4],
              [1,-1,2]])
lb, Qb = np.linalg.eig(B)
Lb = np.diag(lb)
print(Lb)
```

    [[3.00000000e+00 0.00000000e+00 0.00000000e+00]
     [0.00000000e+00 5.23364153e-16 0.00000000e+00]
     [0.00000000e+00 0.00000000e+00 0.00000000e+00]]

$$\Lambda=\begin{bmatrix}3 & 0& 0\\0 & -6\times10^{-17} & 0\\0 & 0 & 3.6\times10^{-16}\end{bmatrix}$$

Note that matrix has only 1 non-zero eigenvalue. The other two eigenvalues are so extremely small as to be effectively zero. This is an example of a rank-deficient matrix; and as such, it has no inverse.

### 5.6 Inverse of a Square Full Rank Matrix
You can calculate the inverse of a square full rank matrix by using the following formula:

$$A^{-1} = Q \Lambda^{-1} Q^{-1}$$

Let's apply this to matrix ***A***:

$$A=\begin{bmatrix}1 & 2\\4 & 3\end{bmatrix}$$

Let's find the matrices for ***Q***, ***Λ-1***, and ***Q-1***:

```python
A = np.array([[1,2],
              [4,3]])

l, Q = np.linalg.eig(A)
L = np.diag(l)
print(Q)
Linv = np.linalg.inv(L)
Qinv = np.linalg.inv(Q)
print(Linv)
print(Qinv)
```

    [[-0.70710678 -0.4472136 ]
     [ 0.70710678 -0.89442719]]
    [[-1.  -0. ]
     [ 0.   0.2]]
    [[-0.94280904  0.47140452]
     [-0.74535599 -0.74535599]]

```python

```

Codes/Essential Math for Machine Learning/3. Vectors and Matrices.ipynb


Introducción a la computación numérica y simbólica con Python

# 9. Álgebra simbólica

Sympy ofrece multitud de posibilidades para manejar símbolos, y esa es la base del álgebra.

## Expresiones algebraicas

Con Sympy podemos factorizar, desarrollar y simplificar expresiones de la misma forma que cuando trabajamos con papel y lápiz.

```python
import numpy as np
import scipy as sci
import matplotlib.pyplot as plt
import sympy as sp
sp.init_printing()

expresion = sp.sympify('(x-1)*(x+2)*(x-5)')
print(expresion)
# Vamos a expandir el polinomio
print()
expresion = expresion.expand()
sp.pprint(expresion)
```

    (x - 5)*(x - 1)*(x + 2)

     3      2
    x  - 4⋅x  - 7⋅x + 10

`collect` obtiene factor común de las potencias de una expresión.

```python
x,y,z = sp.symbols('x y z')
polinomio = sp.sympify('8*(x-y)**3+(x+z)**3+2')
sp.pprint(polinomio)
print("=")
sp.pprint(polinomio.expand())
print("=")
sp.pprint(sp.collect(polinomio,x))
```

             3          3
    8⋅(x - y)  + (x + z)  + 2
    =
       3       2        2           2        2      3    3
    9⋅x  - 24⋅x ⋅y + 3⋅x ⋅z + 24⋅x⋅y  + 3⋅x⋅z  - 8⋅y  + z  + 2
    =
             3          3
    8⋅(x - y)  + (x + z)  + 2

`cancel` simplifica las expresiones fraccionarias.

```python
expresion = (2*x + x/4 - 2)/(x**2 - 4)
sp.pprint(expresion)
print("=")
sp.pprint(sp.cancel(expresion))
```

    9⋅x
    ─── - 2
     4
    ───────
      2
     x  - 4
    =
     9⋅x - 8
    ─────────
       2
    4⋅x  - 16

La función `apart` descompone en fracciones parciales

```python
expresion=(5*x+3)/(x**2+2*x-3)
sp.pprint(expresion)
print("=")
sp.pprint(expresion.apart())
```

      5⋅x + 3
    ────────────
     2
    x  + 2⋅x - 3
    =
      3       2
    ───── + ─────
    x + 3   x - 1

```python
# SymPy realiza la descomposición en fracciones parciales para integrar sin que tengamos que pedirlo de forma explícita

integral = sp.symbols('integral')
integral = sp.integrate(expresion, x)
print ("La integral indefinida de")
expresion
```

```python
print ("es")
integral
```

`trigsimp` y `expand_trig` son las funciones equivalentes a `factor`y `expand` cuando hay expresiones trigonométricas en el problema.

```python
expresiontrig = sp.cos(2*x)
sp.pprint(expresiontrig)
print("es igual a ")
sp.pprint(sp.expand_trig(expresiontrig))
print("")
expresiontrig = sp.sin(2*x)
sp.pprint(expresiontrig)
print("es igual a ")
sp.pprint(sp.expand_trig(expresiontrig))
print()
expresiontrig = sp.tan(2*x)
sp.pprint(expresiontrig)
print("es igual a ")
sp.pprint(sp.expand_trig(expresiontrig))

print()
expresiontrig = 2*sp.tan(x)/(1-sp.tan(x)*sp.sin(x)/sp.cos(x))
sp.pprint(expresiontrig)
print("es igual a ")
sp.pprint(sp.trigsimp(expresiontrig))

```

    cos(2⋅x)
    es igual a
         2
    2⋅cos (x) - 1

    sin(2⋅x)
    es igual a
    2⋅sin(x)⋅cos(x)

    tan(2⋅x)
    es igual a
      2⋅tan(x)
    ───────────
           2
    1 - tan (x)

          2⋅tan(x)
    ───────────────────
      sin(x)⋅tan(x)
    - ───────────── + 1
          cos(x)
    es igual a
    tan(2⋅x)

## Resolución de ecuaciones

SymPy encuentra las raíces de una ecuación usando la función `solve`

```python
raices = sp.solve(expresion)
print("raices de",expresion)
sp.pprint(raices)
print()
```

    raices de (5*x + 3)/(x**2 + 2*x - 3)
    [-3/5]

Hagamos los mismo con otro polinomio del que desconocemos sus factores

```python
expresion2 = sp.sympify('x**2-4*x-1')
raices2 = sp.solve(expresion2)
print("raices de",expresion2)
sp.pprint(raices2)

expresion3 = expresion/expresion2
print("expresion/expresion2",expresion/expresion2)
# Forma simplificada
print("Forma simplificada",sp.simplify(sp.factor(expresion)/sp.factor(expresion2)))
```

    raices de x**2 - 4*x - 1
    [2 - √5, 2 + √5]
    expresion/expresion2 (5*x + 3)/((x**2 - 4*x - 1)*(x**2 + 2*x - 3))
    Forma simplificada -(5*x + 3)/((x - 1)*(x + 3)*(-x**2 + 4*x + 1))

La función `solve` de SymPy tiene la ventaja de que no necesita una pista para localizar las raíces, y encuentra todas de una vez, pero es más lenta que los métodos numéricos y no siempre funciona.

```python
# Utilizamos el mismo ejemplo que en la lección de resolución numérica
expresion = sp.exp(x/3)/3+4*sp.sin(4*x)
sp.pprint(expresion)
print()

# Resolución numérica
def fuerzabruta(serie,x):
    raices = list()              # lista vacía
    lserie = len(serie)
    for i in range(lserie-1):
        if ((serie[i+1]*serie[i])<0):     # Producto negativo si hay cambio de signo
            raices.append((x[i+1]+x[i])/2)
    return raices

def frara(xn):
    return(np.exp(xn/3)-np.cos(4*xn)-0.7)

xn = np.linspace(0,np.pi,50)
yn = frara(xn)
raiceseq = fuerzabruta(yn,xn)
print("Raíces por el método numérico de fuerza bruta", raiceseq)

```

     x
     ─
     3
    ℯ
    ── + 4⋅sin(4⋅x)
    3

    Raíces por el método numérico de fuerza bruta [0.28851361104396056, 1.5066821910073496, 1.5707963267948966]

```python
# La resolución con SymPy no siempre es posible, hay ocasiones en que solve es incapaz de encontrar las raíces
# y produce una excepción. Como ejemplo, la misma ecuación que se ha resuelto numéricamente.

try:
    raices = sp.solve(expresion)
    print("Raíces con SymPy solve",raices)      # Mensaje de error porque es incapaz de encontrar la solución
except:
    print("Sympy: imposible resolver la ecuación "+str(expresion)+" =0")
```

    Sympy: imposible resolver la ecuación exp(x/3)/3 + 4*sin(4*x) =0

Tampoco puede resolver lo que algebraicamente es imposible como $cos(x)=x$, que sí pudimos resolver de forma numérica (número de Dotie).

```python
# Véase el mensaje de error

raices = sp.solve(sp.cos(x)-x)
```

## Matrices

SymPy tiene su propia clase de matrices, que no debe ser confundida con los arrays de NumPy.

```python
# Definición explícita con valores
from sympy.matrices import Matrix, eye, zeros, ones, diag, GramSchmidt
M = Matrix([[1,0,6], [-8,0,0]]);
M
```

```python
# Definición con una función
P = Matrix(2,3,lambda x,y:x*10+y)
P
```

```python
# Definición con contenido simbólico con una función
A = Matrix(3,3,lambda x,y:sp.symbols('a'+str((x+1)*10+y+1)))
A

```

```python
B = Matrix(3,3,lambda x,y:sp.symbols('b'+str((x+1)*10+y+1)))
A+B
```

`eye`crea una matriz con valor 1 en la diagonal (aunque no sea cuadrada) y el resto a 0

```python
E3 = eye(3,3)
E3
```

```python
A*E3                         # Al multiplicar por la matriz identidad se obtiene la matriz original
```

Los objetos de la clase matriz pueden manipularse mediante sus propios métodos, por ejemplo, `.inv` para inversión

```python
V = Matrix(2,2,lambda x,y:(x+1)*10+y+1)
V
```

```python
V.inv()
```

```python
V*(V.inv())
```

Con `det`se calcula el determinante

```python
V.det()                         # Cálculo con valores numéricos
```

```python
A.det()                         # Cálculo con símbolos
```

Con estas herramientas ya podemos resolver sistemas de ecuaciones lineales. Lo aplicamos al mismo ejemplo que usamos con NumPy.

```python
#  Encontrar el punto de corte de las rectas
# -3x + y = -3
#  4x + 2y = 5

AL = Matrix([[-3,1],[4,2]])
bL = Matrix([[-3,5]]).T

AL, bL
```

```python
r1 =  Matrix([[-3,1],[5,2]]).det()/AL.det()
r2 =  Matrix([[-3,-3],[4,5]]).det()/AL.det()

print("Las rectas se cortan en el punto x=",r1,"y=", r2)
```

    Las rectas se cortan en el punto x= 11/10 y= 3/10

Con el método `LUsolve`la ecuación se resuelve en un único paso.

```python
rs = AL.LUsolve(bL)
rs
```

Autovalores y autovectores

```python
Matrix([[1,2],[2,2]]).eigenvals()
```

```python
MC = Matrix(2,2,lambda x,y:sp.symbols('b'+str((x+1)*10+y+1)))
MC
```

```python
MC.eigenvals()
```

## Trabajando con vectores

En esta última sección se introduce el manejo de vectores con SymPy, tanto para manipulaciones algebraicas como para cálculo.

Empezamos importando el paquete de funciones vectoriales.

```python
from sympy.physics.vector import *

# Definición de una base ortonormal de dos dimensiones.

N = ReferenceFrame('N')               # Creación del marco de referencia, por defecto tridimensional
i = N.x                               # Definición de los vectores unitarios, por defecto ortonormales
j = N.y
k = N.z
i
```

```python
v1,v2,v3 = sp.symbols("v1 v2 v3")
v1 = -3*i+2*j
v2 = i+4*j+k
# Producto escalar
v1.dot(v2)
```

```python
# Aritmética vectorial
v3 = v1 + v2
v3
```

```python
# Producto vectorial
v1.cross(v2)
```

El momento de una fuerza respecto al punto $O$ se define como el producto vectorial de la fuerza por el vector desde el origen al punto de aplicación.

$\mathbf M_\text{O}=
\overrightarrow{\text{OP}} \times \mathbf{F}=
\mathbf{r} \times \mathbf{F} $

Imaginemos una esfera unida por una varilla de masa despreciable a un eje de rotación que tomamos como origen de coordenadas

```python
# Momento de una fuerza de 8N en sentido perpendicular al eje X aplicada en O = 6,0
# se expresa en N x m

OP, F, M = sp.symbols("OP F M")
OP = 6*i
F = 8*j
M = OP.cross(F)
M
```

Momento de la misma fuera pero aplicada con un ángulo de 45 grados respecto del eje X

```python
F = sp.sqrt(8)*i+sp.sqrt(8)*j
M = OP.cross(F)
M
```

La magnitud de este par es menor

```python
M.magnitude()
```

Si la fuerza se aplicase en el sentido del eje X el par es nulo

```python
F = 8*i
M = OP.cross(F)
M
```

Función genérica expresada según el ángulo alfa que forma la fuerza con el eje X
Para ello hay que descomponer la fuerza según los ejes X e Y
La magnitud es máxima cuando la fuerza se aplica perpendicularmente

```python
Fx, Fx, alfa = sp.symbols("FX FY alfa")
Fuerza = 8
M_modulo = []
angulo = np.linspace(-np.pi,np.pi,1000)
for alfa in angulo:
    Fx = Fuerza*sp.cos(alfa)
    Fy = Fuerza*sp.sin(alfa)
    F = Fx*i+Fy*j
    M = OP.cross(F)
    M_modulo.append(M.magnitude())
plt.title("Magnitud del momento de fuerza")
plt.plot(angulo,M_modulo)

```

---

    (c) 2020 Javier García Algarra. www.u-tad.com

Licensed under a Creative Commons Reconocimiento 4.0 Internacional License


1. 通过伯努利分布产生100个实验数据
2. 假定数据服从伯努利服务
3. 通过极大似然方法进行参数估计

```python
from scipy.stats import bernoulli

# 生成实验数据
data = bernoulli.rvs(0.5, size=100)
```

```python
# 假定数据服从伯努利分布，求似然函数
# p^k(1-p)^(n-k)
# 上述数据模拟了一次实现，n为100，看下出现1的个数即k
import pandas as pd
c = pd.value_counts(data)
print(c)
```

    1    54
    0    46
    dtype: int64

```python
# 从而可以得到似然函数，然后只需要求导，得到p就达到目的了

import sympy

p = sympy.symbols('p')
d = sympy.diff(p**54*(1-p)**46, p)

sympy.solve(d, p)
```

    [0, 27/50, 1]

```python
print(27/50)
# p = 0.54
# 1-p=0.46
```

    0.54
    0.45999999999999996

```python
# 从而就得到了分布公式
# f(n,k) = 0.54**k * 0.46**(n-k)
```


# Propeller selection
*Written by Marc Budinger (INSA Toulouse) and Scott Delbecq (ISAE-SUPAERO), Toulouse, France.*

**Sympy** package permits us to work with symbolic calculation.

```python
from math import pi

from sympy import Symbol
from sympy import *
```

## Design graph

The following diagram represents the design graph of the propeller’s selection. The max thrust is assumed to be known here.

> **Questions:**
* Give the main sizing problems you are able to detect.
* Propose one or multiple solutions (which can request equation manipulation, addition of design variables, addition of constraints)
* Orientate the arrows
* Give equations order, inputs/outputs at each step of this part of sizing procedure

### Sizing code and optimization

> Exercice: propose a sizing code for the selection of a propeller.

```python
# Specifications
rho_air=1.18# [kg/m^3] Air density
ND_max=105000./60.*.0254 #[Hz.m] Max speed limit (N.D max) for APC MR propellers

# Reference parameters for scaling laws
D_pro_ref=11.*.0254# [m] Reference propeller diameter
M_pro_ref=0.53*0.0283# [kg] Reference propeller mass

# Assumption

F_pro_to=15.0 #[N] Thrust for 1 propeller during Take Off
F_pro_hov=5.0 #[N] Thrust for 1 propeller during hover
```

Define the design variables as a symbol under `variableExample= Symbol('variableExample')`

```python
#Design variables
beta_pro=Symbol('beta_pro')#[-] ratio pitch-to-diameter propeller (0.3,0.6)
k_ND=Symbol('k_ND')#[-] undercoefficient max speed propeller (1,1000)
```

```python
#Equations:
#-----
C_t = (4.27e-02 + 1.44e-01 * beta_pro)  # Thrust coef with T=C_T.rho.n^2.D^4 - 0.8 for de-rating of APC catalog
C_p = -1.48e-03 + 9.72e-02 * beta_pro  # Power coef with P=C_p.rho.n^3.D^5

# Propeller selection with take-off scenario
D_pro = (F_pro_to / (C_t*rho_air*(ND_max/k_ND)**2.))**0.5  # [m] Propeller diameter
n_pro_to = ND_max / k_ND / D_pro # [Hz] Propeller speed
Omega_pro_to = n_pro_to * 2*pi # [rad/s] Propeller speed

M_pro = M_pro_ref * (D_pro/D_pro_ref)**2. # [kg] Propeller mass

P_pro_to = C_p * rho_air * n_pro_to**3. * D_pro**5. # [W] Power per propeller
T_pro_to = P_pro_to / Omega_pro_to # [N.m] Propeller torque

# Propeller torque & speed for hover
n_pro_hov = sqrt(F_pro_hov/(C_t * rho_air *D_pro**4.)) # [Hz] hover speed
Omega_pro_hov = n_pro_hov * 2.*pi # [rad/s] Propeller speed

P_pro_hov = C_p * rho_air * n_pro_hov**3. * D_pro**5. # [W] Power per propeller
T_pro_hov = P_pro_hov / Omega_pro_hov # [N.m] Propeller torque
```


# Inverse of a lower triangular matrix $L$

## Equations for each element of $L^{-1} = [q_{ii}]$

$
\begin{equation}
\begin{bmatrix}
    q_{11}&      0&     0 \\
    q_{21}& q_{22}&     0 \\
    q_{31}& q_{32}& q_{33}
\end{bmatrix}
\times
\begin{bmatrix}
    l_{11}&      0&     0 \\
    l_{21}& l_{22}&     0 \\
    l_{31}& l_{32}& l_{33}
\end{bmatrix}
=
\begin{bmatrix}
    1& 0& 0 \\
    0& 1& 0 \\
    0& 0& 1
\end{bmatrix}
\end{equation}
$

Therefore,

$
\begin{equation}
L^{-1} =
\begin{bmatrix}
    l_{11}^{-1}&      0&     0 \\
    -q_{22} l_{21} l_{11}^{-1}& l_{22}^{-1}&     0 \\
    -(q_{33}l_{31}+q_{32}l_{21}) l_{11}^{-1}& -q_{33}l_{32} l_{22}^{-1}& l_{33}^{-1}
\end{bmatrix}
=
\begin{bmatrix}
    l_{11}^{-1}&      0&     0 \\
    -l_{21} l_{22}^{-1} l_{11}^{-1}& l_{22}^{-1}&     0 \\
    -(-l_{21}l_{32}l_{33}^{-1} l_{22}^{-1}+l_{31}l_{33}^{-1}) l_{11}^{-1}& -l_{32}l_{33}^{-1} l_{22}^{-1}& l_{33}^{-1}
\end{bmatrix}
\end{equation}
$

```julia
function lowertriangularinverse(l)
    q = zeros(size(l))
    for i in 1:size(l,1)
        q[i,i] = 1/l[i,i]                               # diagonal element
        for j in 1:i-1
            j = i-j                           # moving reverse on each row
            for k in 0:i-j-1
                k = i-k
                q[i,j] -= q[i,k]*l[k,j]/l[j,j]         # sum of each term
            end
        end
    end
    return q
end
```

    lowertriangularinverse (generic function with 1 method)

## Test

```julia
using LinearAlgebra, Plots

function test(n)
    L,U,p = lu(rand(n,n))
    r = norm(lowertriangularinverse(L)*L-I(n))
    if r<1e-10 r=0 end
    return r
end

scatter(1:10,test.(1:10), label=:false, xlabel="n", ylabel="norm(r)")
```

# Complexity

Three for loops are used for this implementation. Therefore the complexity is $O(N^3)$

## Adjourn

```julia
using Dates
println("mahdiar")
Dates.format(now(), "Y/U/d HH:MM")
```

    mahdiar

    "2021/February/19 14:39"


## Principle of Monte-Carlo integration:

* Define a function that tests is a point is inside a region
* Generate $N_{total}$ random points over a know area $A_{total}$ that is larger than the region
* count the number of points $N_{in}$ that pass the test
* Compute the area of the region as:

\begin{equation}
A_{in} \approx \frac{N_{in}}{N_{total}} * A_{total}
\end{equation}

We will define a python module to compute pi by summing the results of N_jobs monte-carlo realizations that we will run in parallel.  Since multiprocessing only works for *modules* we can't really run this in a notebook, so we will write out a file first, and then run it

```python
print("hello")
```

```python
%%writefile multiproc.py

from multiprocessing import Pool
from concurrent.futures import ProcessPoolExecutor
from itertools import repeat
import time
import random
import math

A_TOTAL = 4.0

def monte_carlo_integrate(is_inside, n_total, range_x, range_y):
    """
    a slow (non numpy) way of computing an integral with
    Monte-Carlo sampling
    """

    n_inside = 0
    for i in range(n_total):
        x = random.uniform(*range_x)
        y = random.uniform(*range_y)
        if is_inside(x, y):
            n_inside += 1

    return n_inside

def monte_carlo_pi(n_total):
    return monte_carlo_integrate(
        is_inside=lambda x, y: math.sqrt(x ** 2 + y ** 2) < 1,
        n_total=n_total,
        range_x=[-1, 1],  # chosen to match A_total
        range_y=[-1, 1],
    )

def parallel_monte_carlo_pi(n_jobs, n_total_per_job=100_000, verbose=False):

    pool = Pool()   # note you cannot define this globally!
    results = pool.map(monte_carlo_pi, repeat(n_total_per_job, n_jobs))
    pi = sum(results)/(n_total_per_job*n_jobs)*A_TOTAL
    if verbose:
        print(f"Pool: {pool}")
        print(f"Results: {results}")
        print(f"Pi is {pi:.10f}")
    return pi

def parallel_monte_carlo_pi_2(n_jobs, n_total_per_job=100_000, verbose=False):
    with ProcessPoolExecutor() as pool:   # note you cannot define this globally!
        results = pool.map(monte_carlo_pi, repeat(n_total_per_job, n_jobs))
        pi = sum(results)/(n_total_per_job*n_jobs)*A_TOTAL
        if verbose:
            print(f"Pool: {pool}")
            print(f"Results: {results}")
            print(f"Pi is {pi:.10f}")
        return pi

if __name__ == "__main__":
    parallel_monte_carlo_pi(10)
```

Now we can import the module and run it locally:

```python
from multiproc import parallel_monte_carlo_pi, monte_carlo_pi
import matplotlib.pyplot as plt
plt.style.use("ggplot")
```

```python
parallel_monte_carlo_pi(n_jobs=10, n_total_per_job=100_000, verbose=True)
```

```python
t = {}
```

```python
t[1] = %timeit -o monte_carlo_pi(5_000_000)/5_000_000 * 4
```

```python
t[2] = %timeit -o parallel_monte_carlo_pi(n_jobs=2, n_total_per_job=5_000_000//2)
```

```python
t[5] = %timeit -o parallel_monte_carlo_pi(n_jobs=5, n_total_per_job=5_000_000//5)
```

```python
t[10] = %timeit -o parallel_monte_carlo_pi(n_jobs=10, n_total_per_job=5_000_000//10)
```

```python
t[20] = %timeit -o parallel_monte_carlo_pi(n_jobs=20, n_total_per_job=5_000_000//20)
```

```python
t[50] = %timeit -o parallel_monte_carlo_pi(n_jobs=50, n_total_per_job=5_000_000//50)
```

```python
t[100] = %timeit -o parallel_monte_carlo_pi(n_jobs=100, n_total_per_job=5_000_000//100)
```

```python
t[500] = %timeit -o parallel_monte_carlo_pi(n_jobs=500, n_total_per_job=5_000_000//500)
```

```python
t[1000] = %timeit -o parallel_monte_carlo_pi(n_jobs=1000, n_total_per_job=5_000_000//1000)
```

```python
fig, ax = plt.subplots(1,1)
plt.errorbar(
    x=t.keys(),
    y=[x.average for x in t.values()],
    yerr=[x.stdev for x in t.values()],
    lw=3,
    linestyle="dotted"
)
plt.xlabel("Number of Jobs")
plt.ylabel("Time to compute PI (5M samples)")
ax.set_yscale("log")
ax.set_xscale("log")
```

```python

```


#### Newton-Raphson Algorithmus

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

def f(x):
    return 4*x**4 - 3*x**3 - 2*x - 1

def df(x):
    return 16*x**3 - 9*x**2 - 2

xvalues = np.linspace(-0.55, 1.25, 100)
yvalues = f(xvalues)
df_yvalues = df(xvalues)
plt.figure(figsize=(10.0, 8.0))
plt.axhline(0)
plt.axvline(0)
plt.plot(xvalues, yvalues, label="f(x) = 4x^4 - 3x^3 - 2x - 1")
plt.plot(xvalues, df_yvalues, label="f'(x) = 16x^3 - 9x^2 - 2")
sin = 5*np.sin(10*xvalues)
plt.scatter(xvalues, sin, s=3, c='red', label='g(x) = 5sin(10x)')
plt.xlabel('X - Werte')
plt.ylabel('Y - Werte')
plt.grid(True)
plt.legend(loc='upper left')
plt.title('Polynom, Ableitung Polynom und Sinus-Funktion')
plt.show()
plt.close()

```

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

def f(x):
    return 4*x**4 - 3*x**3 - 2*x - 1

def df(x):
    return 16*x**3 - 9*x**2 - 2

delta = 1.0e-10
rvalues = np.linspace(-10, 10, 100000)
roots = []

for r in xvalues:
    guess = r
    for n in range(1000):
        next_guess = guess - f(guess)/df(guess)
        if abs(next_guess - guess) < delta:
            already_in = False
            for root in roots:
                if abs(next_guess - root) < delta:
                    already_in = True
                    break
            if not already_in:
                roots.append(next_guess)
            break
        guess = next_guess
print(roots)

```

    [-0.3780737706648974, 1.221912084397845]

#### Primzahlenberechnung mit der Methode "Sieb des Eratosthenes

```python

import numpy as np
from math import isqrt
import time
from sys import getsizeof

def primes_less_than(n):
    if n <= 2:
        return []
    is_prime = n * [True]
    # is_prime = np.array(n * [True], dtype=bool)
    is_prime[0] = False
    is_prime[1] = False
    print(f'Größe der Liste/Array = {getsizeof(is_prime):>15,d}')

    for i in range(2, isqrt(n)+1):
        if is_prime[i]:
            for x in range(i*i, n, i):
                is_prime[x] = False

    return [i for i in range(n) if is_prime[i]]

if __name__ == '__main__':
    t0 = time.perf_counter()
    # n = 10_000 # 0.01 Sekunden
    # n = 10_000_000 # 1.8 Sekunden
    n = 100_000_000  # 19.8 Sekunden
    # n = 1_000_000_000  # 218.0 Sekunden für 50,847,534 Primzahlen
    list_prim = primes_less_than(n)
    tdelta = time.perf_counter() - t0
    last_prim = list_prim[-1]
    anz_prim = len(list_prim)
    print(f'Laufzeit : {tdelta:6,.2f} Sekunden')
    print(
        f'Letzte Primzahl im Bereich bis {n:<,d} = {last_prim:>12,d} Anzahl = {anz_prim:12,d}')
    if n <= 10_000:
        with open('test.txt', 'w') as f:
            for n in list_prim:
                f.write(f'{n:>15,d}\n')

```

    Größe der Liste/Array =     800,000,056
    Laufzeit :  49.67 Sekunden
    Letzte Primzahl im Bereich bis 100,000,000 =   99,999,989 Anzahl =    5,761,455

#### Nullstellen einer Funktion mit sympy ermitteln

```python
from sympy import *
init_printing(use_unicode=True)
x = Symbol('x')
f = Function('f')('x')
a1 = x**3 + 4*x**2 - 2*x - 6
g = (Eq(f, a1))
plot(a1, (x, -5, 3))
g

```

```python
f = a1
e = solveset(Eq(f, 0))
e
```

```python
N(e)
```

#### Tests mit numpy

```python
import numpy as np
from sys import getsizeof
import random

temp = [random.randint(0, 200) for _ in range(1000)]
np_temp = np.array(temp)
print(f'Größe von Python list : {getsizeof(temp):>15,d}')
print(f'Größe von Numpy array : {getsizeof(np_temp):>15,d}')

```

    Größe von Python list :           8,856
    Größe von Numpy array :           4,112


# Factoring Polynomials with SymPy

Here is an example that uses [SymPy](http://sympy.org/en/index.html) to factor polynomials.

```
from IPython.html.widgets import interact
from IPython.display import display
```

```
from sympy import Symbol, Eq, factor, init_printing
init_printing(use_latex='mathjax')
```

```
x = Symbol('x')
```

```
def factorit(n):
    display(Eq(x**n-1, factor(x**n-1)))
```

Notice how the output of the `factorit` function is properly formatted LaTeX.

```
factorit(12)
```

$$x^{12} - 1 = \left(x - 1\right) \left(x + 1\right) \left(x^{2} + 1\right) \left(x^{2} - x + 1\right) \left(x^{2} + x + 1\right) \left(x^{4} - x^{2} + 1\right)$$

```
interact(factorit, n=(2,40));
```

$$x^{21} - 1 = \left(x - 1\right) \left(x^{2} + x + 1\right) \left(x^{6} + x^{5} + x^{4} + x^{3} + x^{2} + x + 1\right) \left(x^{12} - x^{11} + x^{9} - x^{8} + x^{6} - x^{4} + x^{3} - x + 1\right)$$

```

```


# Seasonal Time Dilation due to Earth's Eccentric Orbit

Just curious to see how large the effect is.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
```

Basic properties concerning Earth's orbit about the Sun and some of the physics parameters involved in the problem.

```python
d_perihelion = 1.47098074e11  # m
d_aphelion   = 1.52097701e11  # m
d_average    = 1.4959787e11   # m

GMsun  = 1.3271244002e20      # m^3 s^-2
MEarth = 5.9722e24            # kg

c = 2.99792458e8              # m s^-1
```

Based on these numbers, we can estimate the gravitational force at aphelion and perihelion to be

```python
g_perihelion = GMsun*MEarth/np.power(d_perihelion, 2)
g_aphelion   = GMsun*MEarth/np.power(d_aphelion, 2)
g_average    = GMsun*MEarth/np.power(d_average, 2)
```

Alternatively, we can base the estimate on the relative strength, which relieves uncertainties due to GMsun and MEarth.

```python
g_ratio = np.power(d_aphelion/d_perihelion, 2)
```

We see that the strength of gravitational force between the Earth and Sun is about 7% larger at perihelion as compared to aphelion.

```python
print g_ratio
```

    1.06913199477

Now we must convert the relative difference in gravitational force into a difference in time dilation for various processes. Since we're dealing with the Sun, we may adopt the Schwarzschild metric, which suggests
\begin{equation}
    t_0 = t_f \left( 1 - \frac{r_s}{r} \right)^{1/2}.
\end{equation}
The Schwarzschild radius for the Sun is

```python
r_s = 2.0*GMsun/np.power(c, 2)
```

which allows us to compute the relative difference in $t_0$ based on the variable Earth-Sun distance.

```python
t0_ratio = np.sqrt(1.0 - (r_s / d_perihelion)) / np.sqrt(1.0 - (r_s / d_aphelion))
t0_ratio_aphelion   = np.sqrt(1.0 - r_s / d_aphelion) / np.sqrt(1.0 - r_s / d_average)
t0_ratio_perihelion = np.sqrt(1.0 - r_s / d_perihelion) / np.sqrt(1.0 - r_s / d_average)
```

The ratio is

```python
print t0_ratio
```

    0.99999999967

which is small, but perhaps non-neligible when it comes to seasonal variations in the decay of Potassium-40, for example. In the case of the DAMA experiment, they have 250 kg of sodium iodide crystals with a small 20 ppb contamination of Potassium-40. Potassium-40 is radioactive with a long half-life of $1.251\times10^9$ years. There are two primary decay channels:
\begin{equation}
    ^{40}\textrm{K}\ \rightarrow\ ^{40}\textrm{Ca} + \beta^{-} + \bar{\nu}
\end{equation}
and
\begin{equation}
    ^{40}\textrm{K} + \beta^{-}\ \rightarrow\ ^{40}\textrm{Ar} + \gamma + \nu
\end{equation}
which occur about 90% and 10% of the time, respectively.

The molar mass of sodium iodide is 149.89 g mol$^{-1}$, meaning in 250 kg there are

```python
moles_NaI = 250.0e3/149.89
print moles_NaI, " moles of NaI,"
```

    1667.88978584  moles of NaI,

which translates into

```python
N_NaI = moles_NaI*6.022140857e23
print N_NaI, "NaI molecules."
```

    1.00442672243e+27 NaI molecules.

Assuming the maximum contamination fraction for Potassium-40 of 20 ppb, there are roughly

```python
N_40K = 20.0*N_NaI/1.0e9
print N_40K, "Potassium-40 atoms in the sample."
```

    2.00885344486e+19 Potassium-40 atoms in the sample.

Now our task is to estimate the approximate increase or decrease in Potassium-40 decays based on variations in the Sun-Earth gravitational potential.

During a single half-life for Potassium-40, approximately $1\times10^{19}$ atoms decay. Given a mean half-life of $1.251\times10^{9}$ years, there will be approximately

```python
decay_rate_average = 1.0e19/(1.251e9*60.0*60.0*24.0*365.256366)
print decay_rate_average, "decays per second on average."
```

    253.297646099 decays per second on average.

The decay rate at perihelion and aphelion will be slightly different due to the gravitational potential of the Earth.

```python
decay_rate_aphelion = 1.0e19/(1.251e9*60.0*60.0*24.0*365.256366*t0_ratio_aphelion)
print decay_rate_aphelion, "decays per second at aphelion."
```

    253.297646058 decays per second at aphelion.

```python
decay_rate_perihelion = 1.0e19/(1.251e9*60.0*60.0*24.0*365.256366*t0_ratio_perihelion)
print decay_rate_perihelion, "decays per second at perihelion."
```

    253.297646141 decays per second at perihelion.

Thus, during a given 24 hour period, one would expect

```python
decay_rate_aphelion = 1.0e19/(1.251e9*365.256366*t0_ratio_aphelion)
print decay_rate_aphelion, "decays per day at aphelion and"
```

    21884916.6194 decays per day at aphelion and

```python
decay_rate_perihelion = 1.0e19/(1.251e9*365.256366*t0_ratio_perihelion)
print decay_rate_perihelion, "decays per day at perihelion."
```

    21884916.6266 decays per day at perihelion.

which yields a difference of

```python
print decay_rate_perihelion - decay_rate_aphelion, "decays per day between summer and winter."
```

    0.00722143054008 decays per day between summer and winter.

which leads to a difference in

```python
print (decay_rate_perihelion - decay_rate_aphelion)/250.0, "decays per kg per day."
```

    2.88857221603e-05 decays per kg per day.

This level is far below the variation in counts per day observed by the DAMA experiment, which is approximately 0.05 cpd/kg/keV.


# 布尔代数

布尔代数又叫逻辑演算,是使用数学方法研究逻辑问题的学科,他用等式表示判断,把推理看作等式的变换.这种变换的有效性不依赖人们对符号的解释,只依赖于符号的组合规律.布尔代数究是非常重要的数学分支,电子,计算机等学科都是建立在其之上.

SymPy对逻辑演算也有支持,使用的是子模块`sympy.logic`下.

```python
from sympy import init_printing
init_printing(use_unicode=True)
```

## 基本的布尔运算

布尔代数研究的对象取值范围就是真值`true`和假值`false`,基本操作也只有4个:与`&(And)`,或`|(Or)`,非`~(Not)`,symbol对象可以直接使用对应运算符构造表达式

```python
from sympy import symbols
x, y = symbols('x,y')
```

```python
y | (x & y)
```

```python
x | y
```

```python
~x
```

要代入值计算布尔代数表达式只要使用`.subs`接口即可.

```python
from sympy.logic import true,false
```

```python
(y & x).subs({x: true, y: true})
```

$\displaystyle \text{True}$

注意Sympy下的true,false不等同于python中的True,False,要做判断需要使用`if`.千万不要使用`==`或`is`

```python
True if true else False
```

    True

```python
True if false else False
```

    False

```python
True == true
```

    True

```python
False == false
```

    True

```python
True is true
```

    False

```python
False is false
```

    False

## 扩展的布尔运算

在实际使用中布尔运算常用的运算符往往是上面三个基本运算符的组合,SymPy还支持的运算符包括:

运算符接口|运算符|含义
---|---|---
`^(Xor)`|异或|同则false,异则true
`Nand`|与非门|当输入均为true ,则输出为false;若输入中至少有一个为false,则输出为true
`Nor`|或非门|当输入均为false,则输出为true;若输入中至少有一个为true,则输出为false
` >> (Implies)`|蕴含|相当于`！A \| B`,`<<`代表逆蕴含
`Equivalent`|等价|相当于当一个为true时另一个也为true;当一个为false时另一个也为false

```python
x ^ y
```

```python
from sympy.logic import Nand,Nor,Equivalent
```

```python
Nand(x, y)
```

```python
Nor(x, y)
```

```python
x >> y
```

```python
Equivalent(x,y)
```

## ITE

这个方法在`sympy.logic.boolalg`下,`ITE`是一个三元运算符,含义就和python中的`xxx if xxx else ...`一样.

```python
from sympy.logic.boolalg import ITE
```

```python
ITE(true | false, true & true, true ^ true)
```

$\displaystyle \text{True}$

## 合取范式和析取范式

SymPy中可以使用`to_cnf(expr, simplify=False)`将逻辑表达式转换为合取范式,使用`is_cnf(expr)`判断表达式是否符合合取范式;使用`to_dnf(expr, simplify=False)`将逻辑表达式转换为析取范式,使用`is_dnf(expr)`判断表达式是否符合析取范式.

```python
from sympy.abc import A, B, D
from sympy.logic.boolalg import to_cnf,is_cnf
```

```python
expr = ~(A | B) | D
expr
```

```python
expr_cnf = to_cnf(expr)
expr_cnf
```

```python
is_cnf(expr)
```

    False

```python
is_cnf(expr_cnf)
```

    True

+ todo `SOP Form`和`POS Form`
+ 形式化简
+ 逻辑推断

```python

```