INTERPOLATIONS

Group 8

Gardyan Priangga Akbar (2301902296)

When do we need interpolation?

Interpolation is drawing conclusions from within a set of known information. For example, if we know that 0 is the lowest number and 10 being the maximum, we can determine that the number 5 must lie in between. Interpolation has many real-life applications, such as:

When you have the cost of catering for 25 and 100 people, but you need an estimate for the cost of catering for 50 people.

When deciding what laptop to buy and you know the price tag and capabilities of laptops at both the lower and higher ends, interpolation can be used to get the most optimal price and specs out of your budget.

Finding the amount of employees needed to complete a task with the most optimal cost.

And more...

There are a number of ways or methods to do interpolation. Two of them, for example, is Lagrange's method and Newton's Divided Difference method. In this notebook, we will be learning about these two and how we can implement them using Python. Strap youselves in, because at the end of every section there will be a playground area for you to explore and mess around! Let's go!

Lagrange Interpolation

The Theory

Lagrange's method is one of the ways for data interpolation from a set of known data points. With this method, we can interpolate the value of f(x) from any value of x from within the data set. Here is the formula:

Where:

n = the degree of polynomial (for linear n = 1, quadratic n = 2, and so on)

Li(x) = the weighting function

To get the weighting function, the formula is:

For some people this formula might seem quite daunting or scary even. However, this formula is just the equivalent of

Doing it in Python

First let's make a list of the data points we know.

```python
xy_values = []

#Initialize x and y values (make sure the X values are in order)
xy_values.append([0, 0])
xy_values.append([10, 227.04])
xy_values.append([15, 362.78])
xy_values.append([20, 517.35])
xy_values.append([22.5, 602.97])
xy_values.append([30, 901.67])

xy_values
```

    [[0, 0],
     [10, 227.04],
     [15, 362.78],
     [20, 517.35],
     [22.5, 602.97],
     [30, 901.67]]

Next let's decide on the order of polynomial to interpolate our data with. We will store it in a variable called n. For reference, to do a linear interpolation, we put our n value as 1. For quadratic n = 2, cubic n = 3, and so on.

```python
n = 1
```

Now let's choose a value of x to interpolate. Obviously, the value of x needs to be within our known data points, otherwise we won't be able to interpolate (that would be extrapolation).

```python
xVal = 16
```

Next we need to pick two points from our known data points that sandwhiches our xVal. We will be keeping track on the indexes. So if our xVal is 16, we will be picking the x values 15 and 20 because 16 lies between them. As we see in our xy_values list, 15 and 20 are positioned in the indexes 2 and 3 respectively. Hence, we take a note of that in a new list.

```python
def get_first2_indexes(xy_values, xVal):
    indexes = []
    for i in range(len(xy_values)-1):
        if xy_values[i][0] < xVal and xy_values[i+1][0] > xVal:
            indexes.append(i)
            indexes.append(i+1)
    return indexes

indexes = get_first2_indexes(xy_values, xVal)
indexes
```

    [2, 3]

If n = 1 (linear), we can go directly to finding the weighting function. However, when n > 1, we have to also select adjacent x values from our two chosen data points. Take note to always pick the data point closest to xVal.

For example when n = 3:
1. Compare 10 and 22.5
2. 10 is closer to 16 than 22.5. So we choose that.
3. indexes will now house [1, 2, 3]. Take note to keep track the indexes in ascending order.

For example when n = 4:
1. We add one more data point from when n = 3.
2. Compare 0 and 22.5
2. 22.5 is closer to 16 than 0. So we choose that.
3. indexes will now house [1, 2, 3, 4].

```python
def get_remaining_indexes(xy_values, indexes, xVal, n):
    for _ in range(n-1):
        #find the value nearest to xVal
        leftIndex = indexes[0]-1
        rightIndex = indexes[len(indexes)-1] + 1
        #Check if the adjacent index exists in the given xy_values data
        if (leftIndex > -1):
            if (rightIndex < len(xy_values)):
                #Check which one is closer to xVal
                if (abs(xy_values[leftIndex][0] - xVal) < abs(xy_values[rightIndex][0] - xVal)):
                    indexes.insert(0, leftIndex)
                else:
                    indexes.append(rightIndex)
            else:
                indexes.insert(0, leftIndex)
        elif (rightIndex < len(xy_values)):
            indexes.append(rightIndex)

get_remaining_indexes(xy_values, indexes, xVal, n)

indexes
```

    [2, 3]

Now we can go ahead and try to find the weighting functions. We will be using Sympy to help us keep track of variables and automatically calculate the final result. Let's start by importing Sympy library.

```python
import sympy as sp

x = sp.Symbol('x');
```

We will now proceed in determining the weighting function. Recall that the formula is

```python
def gather_weighting_functions(polynomial):
    wFunc = []      #Collection of Ln(x)
    for i in range(polynomial+1):
        subFunc = []    #Collection of individual (x - xj)/(xi-xj)
        for j in range(polynomial+1):
            #j != i
            if i != j:
                #(t - xj)/(xi-xj)
                #sub = [i, j]
                #sub[0] = xi
                #sub[1] = xj
                sub = []
                sub.append(i)
                sub.append(j)
                subFunc.append(sub)
        wFunc.append(subFunc)

    return wFunc

wFunc = gather_weighting_functions(n)
wFunc
```

    [[[0, 1]], [[1, 0]]]

The code above simply stores the values i and j in each of their respective iterations.

Recall the formula for lagrange's interpolation to be

We will now put fn(x) together with the code below (Sympy has the benefit of automatically simplifying our otherwise very long equation):

```python
def get_equation(xy_values, wFunc, indexes, x_symbol):
    total = 0
    for i in range(len(wFunc)):
        weight_function_prod = 1
        for a in range(len(wFunc[i])):
            iIndex = wFunc[i][a][0]
            index = indexes[iIndex]
            xi = xy_values[index][0]
            jIndex = wFunc[i][a][1]
            index = indexes[jIndex]
            xj = xy_values[index][0]
            sub = (x_symbol - xj)/ (xi - xj)
            weight_function_prod *= sub
        #Multiply by f(i)
        total += weight_function_prod * xy_values[indexes[i]][1]

    return sp.simplify(total)

equation = get_equation(xy_values, wFunc, indexes, x)
equation
```

$\displaystyle 30.914 x - 100.93$

We are not done however, because we are interested in the value of y when x is our xVal, which is 16. To solve this, we can call Sympy's evalf() function on our equation variable.

```python
#Solve for xVal
result = equation.evalf(subs={x : xVal})

result
```

$\displaystyle 393.694$

If we graph our findings it will look like this

```python
def graph_lagrange(xy_values, equation, xVal, result, x_symbol):
    #Graphing
    %matplotlib inline
    import matplotlib.pyplot as plt
    #split x and y
    x_values = []
    y_values = []

    for i in range(len(xy_values)):
        x_values.append(xy_values[i][0])
        y_values.append(xy_values[i][1])

    #Generate x and y
    new_x_values = []
    new_y_values = []
    for i in range(int(min(x_values) * 100), int(max(x_values) * 100), 1):
        new_x_values.append(i/100)
        new_y_values.append(equation.evalf(subs={x_symbol:i/100}))

    plt.plot(x_values, y_values, 'o', label='data')
    plt.plot(new_x_values, new_y_values, '-', label='equation')
    plt.plot([xVal], [result], '+', label="interpolated data")
    plt.legend()
    plt.xlabel("X")
    plt.ylabel("Y")

    print("y =", equation)
    plt.show()

graph_lagrange(xy_values, equation, xVal, result, x)
```

Notice how our interpolated xVal lies within our predicted equation, but not all of our known data points. There are some that are relatively quite far away from the equation line. In order to reduce this, we need to use a higher level polynomial (a higher value for n). Do take note that the highest level of polynomial you can do is equal to the number of data points you have minus 1. This is because you do not have enough data points to use a higher degree polynomial.

Try it Yourself!

Try experimenting with Lagrange's Interpolation yourself with your own data inputs.

Let's start with the data points that you know:

```python
xy_values = []

#Initialize x and y values (make sure the X values are in order)
xy_values.append([0, 0])
xy_values.append([10, 227.04])
xy_values.append([15, 362.78])
xy_values.append([20, 517.35])
xy_values.append([22.5, 602.97])
xy_values.append([30, 901.67])

xy_values
```

    [[0, 0],
     [10, 227.04],
     [15, 362.78],
     [20, 517.35],
     [22.5, 602.97],
     [30, 901.67]]

Now for what value of x do you want to find?

```python
xVal = 16
xVal
```

    16

And what order of polynomial would you like to use?

Note: Be sure to set your value of n to be the amount of data points -1. If you have 6 data points, your max value for n is 5.

```python
n = 3    #Order/degree of polynomial
#n = len(xy_values) - 1  #Use this to use the highest possible degree of polynomial
n
```

    3

Your inputs are now in! (Don't change anything in the code below)

```python
import sympy as sp

x = sp.Symbol('x');

indexes = get_first2_indexes(xy_values, xVal)
get_remaining_indexes(xy_values, indexes, xVal, n)
wFunc = gather_weighting_functions(n)
equation = get_equation(xy_values, wFunc, indexes, x)
result = equation.evalf(subs={x : xVal})

equation
```

$\displaystyle 0.00543466666666609 x^{3} + 0.132040000000028 x^{2} + 21.2655333333331 x - 4.25399999999809$

The code has been baked and here is the result!

```python
result
```

$\displaystyle 392.057168000002$

Now let's see how that looks like in a graph.

```python
graph_lagrange(xy_values, equation, xVal, result, x)
```

How did your graph turn out? Were you able to line up your known data points with your equation line? Maybe try with a higher value of n, or try with an entire different data set. Play around!

Newton Interpolation

The Theory

To do interpolation with Newton's method, we use Newton's Divided Difference Polynomial (NDDP) method. With this method, we wil be able to interpolate the equation of the line using the known data points. Here is the formula:

Where:

n = the degree of polynomial

an = is the divided difference

Solving for an is quite tricky, and so that part will be discussed as we learn to solve NDDP using Python.

Solving it with Python

Similar to what we did in Lagrange's, we first make a list of the data points we know.

```python
xy_values = []

#Initialize x and y values (make sure the X values are in order)
xy_values.append([0, 0])
xy_values.append([10, 227.04])
xy_values.append([15, 362.78])
xy_values.append([20, 517.35])
xy_values.append([22.5, 602.97])
xy_values.append([30, 901.67])
```

The beauty with Newton's method is that we do not need to specify what order of polynomial we want to use. That is determined by the amount of data points we have -1. So if we have 6 data points, we will be using a 5 degree polynomial. Our next step is to create the divided difference table, so let's do that.

```python
#Initialize divided difference table
def init_table():
    table = []

    for _ in range(len(xy_values)):
        temp = []
        for _ in range(len(xy_values) + 1):
            temp.append(-1)
        table.append(temp)

    #Insert x and y values to table
    for i in range(len(xy_values)):
        table[i][0] = xy_values[i][0]
        table[i][1] = xy_values[i][1]

    return table

table = init_table()
table
```

    [[0, 0, -1, -1, -1, -1, -1],
     [10, 227.04, -1, -1, -1, -1, -1],
     [15, 362.78, -1, -1, -1, -1, -1],
     [20, 517.35, -1, -1, -1, -1, -1],
     [22.5, 602.97, -1, -1, -1, -1, -1],
     [30, 901.67, -1, -1, -1, -1, -1]]

We first initialize an empty table by flagging all values with -1. Then we insert our known x and y values there. Our table will look something like this.

Next we need to populate the table and fill in the remaining empty cells. Here is the way to fill in the table in general:

Try taking a look at it carefully. You will see a particular pattern. Let's go ahead and fill the values in.

```python
#Do the divided difference table
def compute_table(table):
    y_bound = 1
    for col in range(2, len(table[0])):
        for row in range(y_bound, len(table)):
            try:
                delta = (table[row][col-1] - table[y_bound-1][col-1]) / (table[row][0] - table[y_bound-1][0])
            except:
                delta = 0
            #print(table[row][col-1], '-', table[y_bound-1][col-1], "divide", table[row][0], '-', table[y_bound-1][0], '=', delta)
            table[row][col] = delta
        y_bound += 1
compute_table(table)
table
```

    [[0, 0, -1, -1, -1, -1, -1],
     [10, 227.04, 22.704, -1, -1, -1, -1],
     [15, 362.78, 24.185333333333332, 0.29626666666666635, -1, -1, -1],
     [20, 517.35, 25.8675, 0.3163499999999999, 0.004016666666666713, -1, -1],
     [22.5,
      602.97,
      26.79866666666667,
      0.3275733333333335,
      0.004174222222222287,
      6.302222222222958e-05,
      -1],
     [30,
      901.67,
      30.055666666666664,
      0.36758333333333315,
      0.0047544444444444535,
      7.377777777777409e-05,
      1.434074074072601e-06]]

Next we need to get the values for our a0, a1, a2,...., an. Since we already did the divided difference table, we can just "steal" the values from it. This is how you can get the values from the table:

```python
#Get an values
def get_an_values(table):
    an = []
    col = 1
    for row in range(0, len(table)):
        an.append(table[row][col])
        col += 1
    return an

an = get_an_values(table)
an
```

    [0,
     22.704,
     0.29626666666666635,
     0.004016666666666713,
     6.302222222222958e-05,
     1.434074074072601e-06]

We now have all the pieces to put our NDDP puzzle together. Let's recall the formula again.

Alright let's do it in Python now.

```python
import sympy as sp
x = sp.Symbol('x')

def get_equation_newton(an, x_symb):
    func = 0
    for a in range(len(an)):
        product = an[a]
        for i in range(a):
            product *= (x_symb - xy_values[i][0])
        func += product

    func = sp.simplify(func)
    return func

func = get_equation_newton(an, x)

func
```

$\displaystyle x \left(1.4340740740726 \cdot 10^{-6} x^{4} - 3.3777777777671 \cdot 10^{-5} x^{3} + 0.00356481481481208 x^{2} + 0.211538888888918 x + 20.2515666666666\right)$

Great! We now have our equation line. Let's plot it and see how it looks.

```python
%matplotlib inline

import matplotlib.pyplot as plt

def graph_newton(xy_values, func, x_symb):
    #split n and y
    x_values = []
    y_values = []

    for i in range(len(xy_values)):
        x_values.append(xy_values[i][0])
        y_values.append(xy_values[i][1])

    #Generate x and y
    new_x_values = []
    new_y_values = []
    for i in range(int(min(x_values) * 100), int(max(x_values) * 100), 1):
        new_x_values.append(i/100)
        new_y_values.append(func.evalf(subs={x_symb:i/100}))

    plt.plot(x_values, y_values, 'o', label='data')
    plt.plot(new_x_values, new_y_values, '-', label='equation')
    plt.legend()
    plt.xlabel("X")
    plt.ylabel("Y")

    plt.show()

def graph_newton_with_interpolation(xy_values, func, x_symb, xVal, yVal):
    #split n and y
    x_values = []
    y_values = []

    for i in range(len(xy_values)):
        x_values.append(xy_values[i][0])
        y_values.append(xy_values[i][1])

    #Generate x and y
    new_x_values = []
    new_y_values = []
    for i in range(int(min(x_values) * 100), int(max(x_values) * 100), 1):
        new_x_values.append(i/100)
        new_y_values.append(func.evalf(subs={x_symb:i/100}))

    plt.plot(x_values, y_values, 'o', label='data')
    plt.plot(new_x_values, new_y_values, '-', label='equation')
    plt.plot([xVal], [yVal], '+', label="interpolated data")
    plt.legend()
    plt.xlabel("X")
    plt.ylabel("Y")

    plt.show()

print("y =", func)
graph_newton(xy_values, func, x)
```

Well would you look at that. The graph looks quite nicely; all the data points are inside the graph line. Much better than lagrange's method using a polynomial degree of 1. And we didn't even specify the order of polynomial with Newton's.

Try it Yourself!

Try experimenting with Newton's Divided Difference method yourself with your own datasets.

Let's start with the data points that you know:

```python
xy_values = []

#Initialize x and y values (make sure the X values are in order)
xy_values.append([0, 0])
xy_values.append([10, 227.04])
xy_values.append([15, 362.78])
xy_values.append([20, 517.35])
xy_values.append([22.5, 602.97])
xy_values.append([30, 901.67])

xy_values
```

    [[0, 0],
     [10, 227.04],
     [15, 362.78],
     [20, 517.35],
     [22.5, 602.97],
     [30, 901.67]]

Your inputs are now in! Let's process it.

```python
x = sp.Symbol('x')

table = init_table()
compute_table(table)
an = get_an_values(table)
func = get_equation_newton(an, x)
```

The code has been baked and here is the line equation you get:

```python
func
```

$\displaystyle x \left(1.4340740740726 \cdot 10^{-6} x^{4} - 3.3777777777671 \cdot 10^{-5} x^{3} + 0.00356481481481208 x^{2} + 0.211538888888918 x + 20.2515666666666\right)$

Let's see how it looks in a graph

```python
print("y =", func)
graph_newton(xy_values, func, x)
```

How does your graph look? Are all the data points lined up nicely? Maybe try with a higher value of n, or try with an entirely different data set. Play around!

Perhaps you would like to interpolate a value of y? What value for x would you like to try?

```python
xVal = 16
xVal
```

    16

```python
yVal = func.evalf(subs={x: xVal})

yVal
```

$\displaystyle 392.070578915556$

Let's see where does that lie in the graph

```python
graph_newton_with_interpolation(xy_values, func, x, xVal, yVal)
```

Does your interpolated data lie somewhere in the graph? Mess around with more interpolation and enjoy your graph :)

Conclusion

Alright, now that we've explored both Langrange's and Newton's method to do interpolations, let's do a recap!

Interpolation is all about making an educated guess from within a set of known data. Two methods we can use to do interpolation is Lagrange's and Newton's Divided Difference method. With Lagrange's we can control the degree of polynomial we want to use to produce different accuracies and therefore control the speed at which our program is going to run. Newton's method, on the other hand, would use the highest possible degree of polynomial to produce the most accurate and precise interpolation, but that means it will also use the max amount of time to compute. So which one to use? It all depends on you. Use the method that suits you best.


# Lecture 02 Elimination with Matrices

Today's lecture contains:
1. Elimination

2. Explaination of elimination

3. Permutation

4. Inverse Matrix

## 1. Elimination

Suppose we have equations with 3 unknown:
\begin{align}
\begin{cases}x&+2y&+z&=2\\3x&+8y&+z&=12\\&4y&+z&=2\end{cases}
\end{align}

Such equation can be expressed in the format of
\begin{align}
Ax=b
\end{align}
which is:
\begin{align}
\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}=\begin{bmatrix}2\\12\\2\end{bmatrix}
\end{align}

### 1.1 Process of elimination
The objective of elimination is to acquire an upper triangular matrix from $A$. What does it look like? It looks something like this:
\begin{align}
U(Upper Triangular Matrix): \begin{bmatrix}1&2&1\\0&8&1\\0&0&1\end{bmatrix}
\end{align}
Lower triangular matrix looks like this:
\begin{align}
L(Lower Triangular Matrix): \begin{bmatrix}1&0&0\\3&8&0\\2&6&1\end{bmatrix}
\end{align}

* (1) We wish we can eliminate all the $x$ from second and third equations. Taking the matrix $A$ as an example:
\begin{align}
A=\begin{bmatrix}\underline{1}&2&1\\3&8&1\\0&4&1\end{bmatrix}
\end{align}

The number with underscore is called **pivot** which is the coefficient of $x$ in the first equation. And we need to eliminate all the $x$ below which means **all numbers below pivot should be eliminated to zero**. Apparently, we can take $row_2$-**3**$row_1$

Therefore, the first step is:
\begin{align}
\begin{bmatrix}\underline{1}&2&1\\3&8&1\\0&4&1\end{bmatrix}\xrightarrow{row_2-3row_1}\begin{bmatrix}\underline{1}&2&1\\0&2&-2\\0&4&1\end{bmatrix}
\end{align}

* (2) In light of above logic, next step is to eliminate the second pivot which stands for $y$. That would be $row_3$-**2**$row_2$
\begin{align}
\begin{bmatrix}\underline{1}&2&1\\0&\underline{2}&-2\\0&4&1\end{bmatrix}\xrightarrow{row_3-2row_2}\begin{bmatrix}\underline{1}&2&1\\0&\underline{2}&-2\\0&0&\underline{5}\end{bmatrix}
\end{align}

* (3) Because we want to make step(1) and step(2) more intuitive, so we don't take $b$ into account for a moment. In the end , we can add the right hand side $b$ back to the above logic. This step is called **back substitution** and the matrix in the following format called **augmented matrix**.

\begin{align}
\left[\begin{array}{c|c}A&b\end{array}\right]=\left[\begin{array}{ccc|c}1&2&1&2\\3&8&1&12\\0&4&1&2\end{array}\right]\to\left[\begin{array}{ccc|c}1&2&1&2\\0&2&-2&6\\0&4&1&2\end{array}\right]\to\left[\begin{array}{ccc|c}1&2&1&2\\0&2&-2&6\\0&0&5&-10\end{array}\right]
\end{align}

* (4) In the end, we can convert matrix to equation.

\begin{align}
\begin{cases}x&+2y&+z&=2\\&2y&-2z&=6\\&&5z&=-10\end{cases}
\end{align}

$x$,$y$,$z$ can be easily solved:
\begin{align}
x=2, y=1, z=-2
\end{align}

### 1.2 Shape of multiplication
$matrix×column=column$
\begin{align}
\begin{bmatrix}...&...&...\\...&...&...\\...&...&...\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}=\begin{bmatrix}...\\...\\...\end{bmatrix}
\end{align}

$vector×matrix=vector$
\begin{align}
\begin{bmatrix}x&y&z\end{bmatrix}\begin{bmatrix}...&...&...\\...&...&...\\...&...&...\end{bmatrix}=\begin{bmatrix}...&...&...\end{bmatrix}
\end{align}

Sometime vector can be looked as 1×3 matrix.

## 2. Detail explaination of elimination

Before jumping into the explaination, we first need to figure out **how matrix multiply**?

Taking the following mutiplication as an example, where $I$ stands for **identity matrix**. One matrix multiple Identity matrix is itself.
\begin{align}
IA=A
\end{align}

\begin{align}
\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}
\end{align}

So what is going on in the above matrix multiplication?
* (1) Let's look from the perspective of the first matrix from the left.
\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}
can be decomposed to
\begin{bmatrix}1&0&0\end{bmatrix}
\begin{bmatrix}0&1&0\end{bmatrix}
\begin{bmatrix}0&0&1\end{bmatrix}

* (2) Each row of the left matrix stands for the coeffient of each row of the right matrix.

$\begin{bmatrix}1&0&0\end{bmatrix}$ means take $1$×$\begin{bmatrix}1&2&1\end{bmatrix}$, $0$×$\begin{bmatrix}3&8&1\end{bmatrix}$, $0$×$\begin{bmatrix}0&4&1\end{bmatrix}$

$\begin{bmatrix}0&1&0\end{bmatrix}$ means take $0$×$\begin{bmatrix}1&2&1\end{bmatrix}$, $1$×$\begin{bmatrix}3&8&1\end{bmatrix}$, $0$×$\begin{bmatrix}0&4&1\end{bmatrix}$

$\begin{bmatrix}0&0&1\end{bmatrix}$ means take $0$×$\begin{bmatrix}1&2&1\end{bmatrix}$, $0$×$\begin{bmatrix}3&8&1\end{bmatrix}$, $1$×$\begin{bmatrix}0&4&1\end{bmatrix}$

And sum them vertically!

### 2.1 Step(1)
So we look back to the elimination in part 1.
\begin{align}
\Bigg[\quad ?\quad \Bigg]\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&2&-2\\0&4&1\end{bmatrix}
\end{align}

We can easily understand the above multiplication should be the following:

\begin{align}
\begin{bmatrix}1&0&0\\-3&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&2&-2\\0&4&1\end{bmatrix}
\end{align}

In the meantime, we denote $\begin{bmatrix}1&0&0\\-3&1&0\\0&0&1\end{bmatrix}$ as $E_{21}$ since it eliminates all the component below the second row and first column.

### 2.2 Step(2)

The second elimination is:

\begin{align}
\Bigg[\quad ?\quad \Bigg]\begin{bmatrix}1&2&1\\0&2&-2\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&2&-2\\0&0&5\end{bmatrix}
\end{align}

And we can easily see that:

\begin{align}
\begin{bmatrix}1&0&0\\0&1&0\\0&-2&1\end{bmatrix}\begin{bmatrix}1&2&1\\0&2&-2\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&2&-2\\0&0&5\end{bmatrix}
\end{align}

We can denoted $\begin{bmatrix}1&0&0\\0&1&0\\0&-2&1\end{bmatrix}$ as $E_{32}$ since it eliminates all the component below the third row and second column.

### 2.3 Summary
Finally, the whole process can be written as followed:

\begin{align}
E_{32}(E_{12}A)=U
\end{align}

where $U$ stands for the right hand side from step two. $\begin{bmatrix}1&2&1\\0&2&-2\\0&0&5\end{bmatrix}$ **Upper triangular matrix**.

In the meantime, you may wonder **why the order of above multiplication starts from the right to left**?

It is $E_{32}(E_{12}A)$ rather than $A E_{12}E_{32}$?

Intuitively, I **think of this order as an order of function**. Like $g(f(x))$, we first apply $f(x)$ then $g()$. This is very similar to the syntax of Wolfram language.

## 3. Permutation

### 3.1	Exchange row1 and row2

Giving above knowledge, how can we do that:
\begin{align}
\Bigg[\quad ?\quad \Bigg]\begin{bmatrix}a&b\\c&d\end{bmatrix}=\begin{bmatrix}c&d\\a&b\end{bmatrix}
\end{align}

Very simple, that is:
\begin{align}
\begin{bmatrix}0&1\\1&0\end{bmatrix}\begin{bmatrix}a&b\\c&d\end{bmatrix}=\begin{bmatrix}c&d\\a&b\end{bmatrix}
\end{align}

### 3.2	Exchange column1 and column2
\begin{align}
\Bigg[\quad ?\quad \Bigg]\begin{bmatrix}a&b\\c&d\end{bmatrix}=\begin{bmatrix}b&a\\d&c\end{bmatrix}
\end{align}

That is impossible! While if we switch the position, and we can **see it in a column perspective**.
\begin{align}
\begin{bmatrix}a&b\\c&d\end{bmatrix}\begin{bmatrix}0&1\\1&0\end{bmatrix}=\begin{bmatrix}b&a\\d&c\end{bmatrix}
\end{align}

$\begin{bmatrix}\underline{0}&1\\\underline{1}&0\end{bmatrix}$ means take 0 first column and 1 second column.

## 4. Inverse Matrix
We take $E_{21}$ as example, What should be:
\begin{align}
\Bigg[\quad ?\quad \Bigg]\begin{bmatrix}1&0&0\\-3&1&0\\0&0&1\end{bmatrix}=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}
\end{align}

We can easily solve:
\begin{align}
\begin{bmatrix}1&0&0\\3&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&0&0\\-3&1&0\\0&0&1\end{bmatrix}=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}
\end{align}

We denote the inverse matrix of $E$ as $E^{-1}$. That said, $E^{-1}E=I$.

Lecture 02 Elimination with Matrices.ipynb


Math 267 Project #3

---

Name ________________________.    Student ID ____________________.  Enter your name and studet ID in this cell.

---

# Spring mass system.

The second order equation of an unforced spring/mass system with mass m, damping constant b and spring constant k is given by:

$$ mx''(t) + bx'(t) +kx(t) = 0. $$

Recall that $x(t)$ represents the displacement of the mass from its resting position ( with x>0 the spring is stretched and x<0 the spring is compressed). Therefore the velocity of the moving mass is $ x'(t)$ and the acceleration is $x’’(t)$. To approximate the solution for this system using Euler’s method we convert the equation into a system by introducing the variable

 $$ y = x’.$$

So our two unknowns are: $x$ for the position of the mass and $y$ for the velocity of the mass. As a system the second order equation becomes ( using m = 1)

\begin{align}
x'(t) &=  \; y(t) \\
y'(t) &= -k \cdot x(t) - b \cdot y(t)
\end{align}

and unlike the Predator-Prey model of exercise 2. this system is linear and can be represented in matrix notation:

\begin{align}
        \begin{pmatrix}
        x'(t) \\
        y'(t)
        \end{pmatrix}=
        \begin{pmatrix}
        0 & 1 \\
        -k & -b
        \end{pmatrix}\cdot
        \begin{pmatrix}
        x(t) \\
        y(t)
        \end{pmatrix}
    \end{align}

   ---

### Collaboration.  Students are allowed to work together on this project. Colaboration is encouraged.
 However you final submission must be your own work.

### Run the cell below to import the necessary libraries.

```python
# import libraries

import numpy as np
import matplotlib.pyplot as plt
from IPython.display import Image

# uncomment the line below if you are running a macbook
#%config InlineBackend.figure_format ='retina'
```

## Exericse 1.

For this exercise you will approximate the solution of an unforced mass-spring system with
mass m =1, damping constant b = 1 and spring constant k = 2. The second order equation for this system is given by:

$$x''(t) + 1x'(t) + 2x(t) = 0.$$

Use the initial conditions $x(0) = 0$ and $ x'(0)=y(0) = 1$. Use the improved Euler’s method to solve the system. You will need to convert the second order equation to a system as demonstrated above. Choose a Δt=h of 0.1 and a time interval of 12 seconds. Create two plots. One showing x(t) versus time and the second showing the solution curve "trajectory" for the system in the xy (phase) plane.

## Complete the code cell below to solve the sytstem.

```python
# define the slope functions

def f(x,y):
    return y

def g(x,y):
    return -1*y -2*x

h = 0.1  # set delta t
t = np.arange(0,12,h)

# Initialeze arrays to store the results.

x = np.zeros_like(t)
y = np.zeros_like(t)

# Set initial conditions:

x[0] = 0
y[0] = 1

# implement the Improved Euler's method

# Your code goes here, look at project #2

```

### Execute the cell below to graph your results.
You should see a damped sinusoid.

```python
plt.figure(figsize=(8,5))
plt.plot(t,x)
plt.grid()
plt.xlabel('t')
plt.ylabel('x');
plt.title("Damping b = 1");
```

### Execute the cell below to see the trajectory in the phase plane for this problem.
You should see a spiral.

```python
plt.figure(figsize=(8,5))
plt.plot(x,y,linewidth=2)
plt.xlim(-1.5,1.5)
plt.ylim(-1.5,1.5)
plt.grid()
plt.xlabel('x-displacement')
plt.ylabel('y-velocity');
plt.title("Trajectory for spring mass system b = 1");
```

### Repeat the steps above for b=3 and b=0. Copy and paste cells above and make the necessary modifications.  You should show time plots and phase plane plots for b=0 and b=3.

```python
#Your code here
```

---
# Exercise 2.

For this exercise you will approximate the solution of an undamped periodically forced spring-mass system with mass m =1 and spring constant k = 1, no damping. The second order equation for this system is given by:

$$ x''(t) + x(t) = Cos(\omega t).$$

Use the initial conditions $x(0) = 0$ and $ x'(0) = y(0) = 0$. Use the improved Euler’s method to solve the system. Note this system is not autonomous so the slope functions could depend on x,y and t.  Generate plots of $x(t)$ versus time for ω = 1.1 and ω = 1. Use a time interval of 50 seconds for $ \omega = 1 $ and 150 seconds for $ \omega = 1.1$. For both cases set Δt=0.1. Duplicate and modify the code above to generate the plots.  Label the plots appropriately. There are no phase plane plots for Exercise 2.

```python
#Your code here
```

```python

```

```python

```

## Answer the questions below.  Create a text cell to enter your answers.
1. Expain and discuss the results of exercise 1.
2. Compute the period of oscillation for exerice 1. b=0.  Compare to value obtained from the graph.
3. Discuss the results of exercise 2.  Hint: look at the envelopes.

```python

```


# 1. Introduction

Taxes play an important role for every individual in the economy. The size of the tax payment affects the disposible income, and thus the (possible) consumption level. The government collects taxes in order to finance its expenditures. The literatture has estimated that an increase in the marginal tax rate induces people to work less. This implies that the government cannot set a too high marginal tax rate in order to maximize its revenue. In the end, both workers and the government benefit from an optimal tax system. In this project, we examine the change of behaviour of a worker due to a change in the marginal tax rate. We will further try to estimate the tax rate that maximizes the tax revenue by drawing a sample of agents with different abilities and preferences using `np.random`.

In this project, we only consider changes on the intensive margin, i.e. how people adjust their work hours or/and effort due to changes in the marginal tax rate, and not if they change employment status due to a change in the marginal tax rate. Feldstein (1995) argues that taxable income as endogenous variable is better when estimating changes in behaviour due to changes in the marginal tax rate, since taxable income captures changes in hours worked and changes in work effort. We apply the same method as Feldstein (1995) and consider the change in taxable income from changes in the marginal tax rate.

# 2. Model description

We consider the utility function given by

$U_i = c_i - \frac{\alpha _i}{1+\frac{1}{\epsilon}} ( \frac{z _i}{ \alpha _i})^ {1+ \frac{1}{\epsilon}}$

where $c_i$ denotes consumption of the individual, while $z_i$ denotes the taxable income of the individual. $\alpha _i$ denotes the potential earnings level of the individual, and $\epsilon$ denotes the elasticity of taxable income with respect to the net-of-tax rate.

The consumption level is dependent of the taxable income. Thus, we consider the budget constraint given by

$c_i = z_i - T(z_i)$

where $T(z_i)$ is the tax payment. This implies that we assume that the individual uses all of his disposible income on consumption. The tax payment is given by

$T(z_i) = m*z_i$

where $m$ is the marginal tax rate chosen by the government. Since the tax rate is independent of earnings, we consider a proportional tax system.

We want to maximize the utility of the individual subject to his budget constraint, i.e.

$\max\limits_{z_i} U_i=c_i - \frac{\alpha _i}{1+\frac{1}{\epsilon}} ( \frac{z _i}{ \alpha _i})^ {1+ \frac{1}{\epsilon}}$ s.t. $c_i = z_i - T(z_i)$

# 3. Solutions

First, we find the analytical solution. Thus, we find the optimal taxable income of the individual subject to the tax rate.
Second, we show the graphical solution. Here, we see the optimum of the indifference curve, the budget constraint and the optimum of the individual.

## 3.1 Analytical solution

We import the `sympy`-package in order to define the variables and parameters. Afterwards, we use the `.init_printing`-function, so the variables are written in a pretty way. Lastly, we define the variables and parameters.

```python
# 1) Import sympy in order to write nice equations.
import sympy as sm

# 2) Define that we want the variables printed in a pretty way.
sm.init_printing(use_unicode=True)

# 3) Define the different variables and parameters.
U    = sm.symbols('U')
c    = sm.symbols('c')
z    = sm.symbols('z')
eps  = sm.symbols('epsilon')
T_z  = sm.symbols('T(z)')
mtax = sm.symbols('m')
alp  = sm.symbols('alpha')
```

We define the different functions. In total, we consider three different functions: the utility function, the budget constraint and the tax payment. The last two functions are functions of taxable income.

```python
# Define utility function.
utility = c - alp/(1+1/eps)*(z/alp)**(1+1/eps)
utility
```

```python
# Define budget constraint.
budget_constraint = sm.Eq(c, z - T_z)
budget_constraint
```

```python
# Define tax payment.
taxpay = sm.Eq(T_z, mtax*z)
taxpay
```

In the next step, we want to substitute the tax payment into the budget constraint.

```python
# 1) Isolate T_z in the tax payment (this is done already, but to make sure, we run this line of code).
tax_solve = sm.solve(taxpay, T_z)

# 2) Define that we want the tax payment substituted into the budget constraint.
budget_constraint_sub = budget_constraint.subs(T_z, tax_solve[0])

# 3) Ensure that the budget constraint is correct after the substitution.
budget_constraint_sub
```

Now, we want to substitute the budget constraint into the utility function. This is done in the same way as before.

```python
# 1) Isolate c in the budget constraint (this is done already, but to make sure, we run this line of code).
budget_constraint_solve = sm.solve(budget_constraint_sub, c)

# 2) Define that we want the budget constraint substituted into the utility function.
utility_sub = utility.subs(c, budget_constraint_solve[0])

# 3) Ensure that the utility function is correct after the substitution.
utility_sub
```

We have concluded that the utility function is defined in the right way. After substitution, the utility function does only depend of the taxable income (and parameters, but these are fixed).
In order to find the optimal level of taxable income, we differentiate the utility function with respect to taxable income. Afterwards, we isolate for taxable income to derive the optimal level of taxable income ($z^*$).

```python
# 1) Define that we want to calculate the derivate of the utility function with respect to taxable income.
foc = sm.diff(utility_sub, z)

# 2) Define that we want to isolate for taxable income to derive z*.
z_star = sm.solve(foc, z)

# 3) Define the optimal level of taxable income as a Python-function.
z_star_func = sm.lambdify((mtax, alp, eps), z_star)

# 4) Examine the optimal level of taxable income.
z_star
```

We note that the optimal level of taxable income depends on the marginal tax rate, the potential earnings and the elasticity of taxable income with respect to the net-of-tax rate. In the special case where the individual face no taxes, i.e. where $m=0$, his taxable income will equal his potential earnings ($z _i ^* = \alpha _i$).
Furthermore, we note that the taxable income is lowered more due to an increase in the tax rate when the elasticity of taxable income is high.

## 3.2 Graphical solution

We can show how people change behaviour due to a change in the marginal tax rate. As noticed before, the marginal tax rate and the elasticity of taxable income affect how much people deviate from their potential earnings. This can be shown in a graph where one can examine the changes in taxable income when the parameters are changed. In order to do that, we need to define different functions.

First, we import the `numpy`-package. Afterwards, we define a function for the budget constraint and the utility function. In the utility function, we insert the solution given by $z^*$ instead of $z$. We are then possible to determine the optimal consumption level which is a function of the optimal taxable income. Finally, we isolate consumption and keep utility fixed in order to draw an indifference curve .

```python
# 1) Import the numpy-package.
import numpy as np

# 2) Define the budget constraint.
def budget_con_fuc(m):
    return (1-m)*z
budget_con = np.array(range(2))

# 3) Define the utility function.
def value_of_choice(alpha, epsilon, m):
    # The utility is
    utility_f = (1-m)*(alpha*(1-m)**epsilon) - alpha/(1+1/epsilon)*(alpha*(1-m)**epsilon/alpha)**(1+1/epsilon)
    return utility_f

# 3) Define the consumption level determined by the optimal taxable income.
def y(budget_con, m):
    return budget_con*(1-m)

# 4) Define the consumption level as a function of the fixed utility and taxable income.
def utility_function(budget_con, alpha, epsilon, m):
    u = value_of_choice(alpha, epsilon, m)
    return u + alpha/(1+1/epsilon)*(budget_con/alpha)**(1+1/epsilon)
```

We want to plot the utility function, the budget constraint and the optimal taxable income. The optimal taxable income is given by the point where the budget constraint is tangent to the utility function (which is marked with a bullet on the graph).

In order to do so, we import the `matplotlib`-package in order to plot the graphs. Furthermore, we import the `ipywidgets`-package in order to generate interactive sliders.

```python
# Import packages in order to plot the graph and generate sliders.
import matplotlib.pyplot as plt
import ipywidgets as widgets
plt.style.use('ggplot')
```

```python
# 1) Generate the graph.
def graph(alpha, epsilon, tax=0.3):
    plt.figure(figsize=(12,6))
    budget_con = np.arange(0.0, alpha, 1)
    plt.plot(budget_con, utility_function(budget_con, alpha, epsilon, tax))
    plt.plot(budget_con, y(budget_con, tax))
    z_opt = (alpha*(1-tax)**epsilon)
    print("The optimal taxable income is: " + str(round(z_opt)))
    plt.plot(z_opt, (1-tax)*z_opt, 'ro')
    plt.ylabel('Consumption')
    plt.xlabel('Taxable income')
    plt.legend(['Indifference curve','Budget constraint', 'Optimum'])
    return

# 2) Generate the sliders.
widget_alpha = widgets.FloatSlider(
    value = 5000,
    min=100,
    max=10000,
    step=100,
    description='α:',
    readout_format='.0f',
    width = 1000,
    layout={'width': '500px'}
)

widget_epsilon = widgets.FloatSlider(
    value = 0.8,
    min=0.01,
    max=2,
    step=0.01,
    description='$\epsilon$:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

widget_m = widgets.FloatSlider(
    value = 0.5,
    min=0.0,
    max=1,
    step=0.01,
    description='$m$:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

# 3) Print the graph
widgets.interact(graph,
    alpha=widget_alpha, epsilon=widget_epsilon, tax=widget_m
);
```

    interactive(children=(FloatSlider(value=5000.0, description='α:', layout=Layout(width='500px'), max=10000.0, m…

The graph confirms the conclusion from section 3.1.
$z^*$ increases when $\alpha$ increases. This is obvious since one gets a higher income when his potential earnings increases everything else equal.
$z^*$ decreases when $m$ increases. When the marginal tax rate increases, leisure is cheaper, and hence people tend to substitute work for leisure which lowers the taxable income.
$z^*$ decreases when $\epsilon$ increases. When the elasticity of taxable income increases, people respond more to a change in the marginal tax rate. The greater the elasticity, the greater the response. Hence, an increase in the elasticity of taxable income lowers taxable income due to larger labor supply responses.

## 3.3 The tax rate that maximizes the tax revenue

Until know we have only looked at the individual agent's optimization problem. We will examine how the govenment should tax the individuals to maximize the tax revenue. We do this considering the following draw of $N$ individuals with the preferences and abilities as follows
    $$ \begin{eqnarray*}
     &  & \,\,\,\gamma^{j}=(|\alpha^{j}|,|\epsilon^{j}|)\\
     &  & \,\,\,\widehat{\gamma^{j}}=(\alpha^{j},\epsilon^{j}) \\
     &  & \,\,\,\widehat{\gamma^{j}} \sim \mathcal{N}(\mu,\Sigma) \\
     &  & \,\,\,\mu=(\mu_{\alpha},\mu_\epsilon) \\
     &  & \,\,\, \Sigma = \begin{bmatrix}
    \sigma_{\alpha}^2       & \sigma_{\alpha, \epsilon}  \\
    \sigma_{\alpha, \epsilon}   & \sigma_{\epsilon}^2
    \end{bmatrix}
    \end{eqnarray*} $$

Below we define a function which draws $N$ observations from the distribution defined above.

```python
def draw(N = 500, mu_alpha = 5000, mu_epsilon = 0.9, sigma_alpha = 1000000, sigma_epsilon = 0.1, correlation = -0.5):
    # 1) Define an array which contains mu_alpha and mu_epsilon
    mu = np.array([mu_alpha,mu_epsilon])

    # 2) Calculate the covariance between alpha and epsilon based on the correlation
    sigma_alp_eps = correlation * (sigma_epsilon * sigma_alpha)**0.5

    # 3) Define an array which contains the covariance matrix
    Sigma = np.array([[sigma_alpha, sigma_alp_eps], [sigma_alp_eps, sigma_epsilon]])

    # 4) Set the seed and draw the sample
    seed = 2019
    np.random.seed(seed)
    draw = np.random.multivariate_normal(mu, Sigma, size=N)

    # 5) Extract alphas and epsilons from the draw and make sure they are all positive
    global alphas, epsilons
    alphas = np.absolute(draw[:,0])
    epsilons = np.absolute(draw[:,1])
    return

# 6) Plot the distribution of alphas and epsilons
draw()
plt.figure(figsize=(12,6))
plt.scatter(alphas,epsilons)
plt.ylabel('Distribution of $\epsilon_i$')
plt.xlabel('Distribution of $α_i$')
plt.show()
```

We will now define a function which returns the tax revenue as a function of $\alpha_i$, $\epsilon_i$ and the tax rate.  We will then define another function, which returns the tax revenue times -1 as a function of only the tax rate given the values of $\alpha_i$ and $\epsilon_i$ from the draw. This function has the nice property that it is easy to minimize using `optimize.minimize`. The result from this minization process will be the tax rate which maximizes the tax revenue.

```python
from scipy import optimize

# 1) Define function which returns the tax revenue
def tax_revenue(alpha, epsilon, tax):
    # 1.1) Find the tax revenue from each agent
    tax_indi = alpha*(1-tax)**epsilon * tax

    # 1.2) Aggregate all the tax provenues
    tax_revenue = sum(tax_indi)
    return tax_revenue

# 2) Define function that return the tax revenue as a function of the tax rate for the alphas and epsilons drawn earlier
def tax_maximize(tax):
    t = tax_revenue(alphas, epsilons, tax) * -1
    return t

# 3) Define function which maximizes the tax provenue using the tax_maximize(tax) function
def find_maximizing_tax(initial_guess=0.1):
    # 3.1) Set the lower and upper bound
    low_bound = 0
    high_bound = 1-1e-5
    bounds = ((low_bound,high_bound),)

    # 3.2) Use the optimize.minimize to find the maximizing tax rate
    result = optimize.minimize(tax_maximize, initial_guess, bounds=bounds)
    return result

draw()

# 4) Store the optimal tax rate and the corresponding tax revenue.
results_max = find_maximizing_tax(initial_guess=0.1)
max_tax_rate_old = results_max.x
max_tax_rate = max_tax_rate_old[0]
max_tax_rev = results_max.fun
```

Given the standard parameters from the draw, we print the maximizing tax rate and the corresponding tax revenue.

```python
print(f'The optimal tax rate is given by {max_tax_rate:.2f}')
print(f'The corresponding tax revenue is given by {-max_tax_rev:.0f}')
```

    The optimal tax rate is given by 0.56
    The corresponding tax revenue is given by 710528

We will now use the tax_optimize function to plot a Laffer curve with slider to change the parameters of $\mu_\alpha$, $\mu_\epsilon$, $\sigma^2_\alpha$, $\sigma^2_\epsilon$, $\sigma_{\alpha,\epsilon}$

```python
import matplotlib

# 1) Define the interval for which the tax rate can be in [0,1], and turn this into a dataframe.
tax_rate_int = np.arange(0.0, 1.0, 0.01)
tax_table = pd.DataFrame({'taxrate':tax_rate_int})

# 2) Define a function which return the tax revenue for differnet tax levels.
def tax_revenue_df(row):
    t = tax_maximize(row['taxrate'])
    return -t

# 3) Define function which plot the Laffer curve.
def Laffer(mu_alpha = 5000, mu_epsilon = 0.9, sigma_alpha = 50000, sigma_epsilon = 0.1, correlation = 1):

    # 3.1) Make a draw given the parameters specified in this function
    draw(mu_alpha = mu_alpha, mu_epsilon = mu_epsilon, sigma_alpha = sigma_alpha, sigma_epsilon = sigma_epsilon, correlation = correlation)

    # 3.2) Calculate the tax revenues for different tax rates.
    tax_table[z] = tax_table.apply(tax_revenue_df, axis=1)

    # 3.3) Plot the tax revenues as a function of tax rates.
    plt.figure(figsize=(12,6))
    plt.plot(tax_table['taxrate'],tax_table[z], color='b')
    t = find_maximizing_tax(0.1)
    tax = t.x[0] * 100
    plt.plot(t.x[0],-t.fun,'ro')
    plt.axvline(x=t.x[0], linewidth=1, color='r', linestyle='dashed')
    plt.axhline(y=-t.fun, linewidth=1, color='r', linestyle='dashed')
    axes = plt.gca()
    axes.get_yaxis().set_major_formatter(
    matplotlib.ticker.FuncFormatter(lambda x, p: format(int(x), ',')))
    print(f'The tax revenue maximizing tax rate is: {tax:.3f} pct. with a tax revenue of {-t.fun:,.0f}')
    plt.show()
    return

# 4) Customize widgets
widget_mua = widgets.FloatSlider(
    value = 5000,
    min=10,
    max=10000,
    step=100,
    description='$\mu_a$:',
    readout_format='.0f',
    width = 1000,
    layout={'width': '500px'}
)

widget_mue = widgets.FloatSlider(
    value = 1,
    min=0,
    max=3,
    step=0.01,
    description='$\mu_{\epsilon}$:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

widget_sigmaa = widgets.FloatSlider(
    value = 50000,
    min=1000,
    max=100000,
    step=1000,
    description='$\sigma_a^2$:',
    readout_format='.0f',
    width = 1000,
    layout={'width': '500px'}
)

widget_sigmae = widgets.FloatSlider(
    value = 0.1,
    min=0,
    max=3,
    step=0.01,
    description='$\sigma^2_{ \epsilon}$:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

widget_corr = widgets.FloatSlider(
    value = -0.5,
    min=-1,
    max=1,
    step=0.01,
    description='$\sigma_{ a, \epsilon} $:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

# 5) Call the Laffer curve with widgets
widgets.interact(Laffer,
    mu_alpha = widget_mua, mu_epsilon = widget_mue, sigma_alpha = widget_sigmaa, sigma_epsilon = widget_sigmae, correlation = widget_corr,
);
```

    interactive(children=(FloatSlider(value=5000.0, description='$\\mu_a$:', layout=Layout(width='500px'), max=100…

The graph above shows that the tax revenue maximizing tax rate varies between around 25 pct. to 99,99 pct. depending on the parameters. The value of $\mu_\epsilon$ have a great influence on the tax revenue maximizing tax rate.

# 4. Further analysis

Previously, we considered a proportional tax system where everyone faced the same marginal tax rate for any income level. However, most Western contries have a progressive tax system where people with high incomes face a higher marginal tax rate. Thus, we examine the optimal taxable income when the individual face a tax system where the marginal tax rate to the right of a threshold is higher than to the left of the threshold.

## 4.1 Extension of the baseline model

We consider the same equations as in section 2. However, the function which defines the tax payment is different now. The tax payment is given by

$T_{new} (z_i) = min(z_i, K) * m + max(z_i - K, 0) * m_H$

where $K$ is the threshold, and $m_H$ is the marginal tax rate faced by high-income earners (people with an income above the threshold).

We note that if $z_i \leq K$ then there is no change from before, and the optimal tax rate is identical to the solution in section 3. An individual with an income above the threshold face the tax payment

$T_{H} (z_i) = K * m + (z_i - K) * m_H$

## 4.2 Analytical solution

As mentioned in section 4.1, the introduction of the new tax system does not change anything from before for people with an income below the threshold. High-income earners respond in another way than before, and hence we calculate the optimal taxable income for them.
The steps in this section is in most cases identical with the steps in section 3.2.

First, we define the new symbols and the new tax system for high-income earners.

```python
# 1) Define symbols.
T_H_z   = sm.symbols('T_{H}(z)')
m_H     = sm.symbols('m_H')
Kn      = sm.symbols('K')

# 2) Define new tax system.
taxhigh = sm.Eq(T_H_z, mtax*Kn + m_H*(z-Kn))
taxhigh
```

We insert the new function of tax payment into the budget constraint.

```python
# 1) Isolate T_z in the tax payment (this is done already, but to make sure, we run this line of code).
tax_high_solve = sm.solve(taxhigh, T_H_z)

# 2) Define that we want the tax payment substituted into the budget constraint.
budget_constraint_high_sub = budget_constraint.subs(T_z, tax_high_solve[0])

# 3) Ensure that the budget constraint is correct after the substitution.
budget_constraint_high_sub
```

We insert the new budget constraint into the utility function.

```python
# 1) Isolate c in the budget constraint (this is done already, but to make sure, we run this line of code).
budget_constraint_high_solve = sm.solve(budget_constraint_high_sub, c)

# 2) Define that we want the budget constraint substituted into the utility function.
utility_high_sub = utility.subs(c, budget_constraint_high_solve[0])

# 3) Ensure that the utility function is correct after the substitution.
utility_high_sub
```

We differentiate the utility function with respect to taxable income in order to derive the optimal level of taxable income.

```python
# 1) Define that we want to calculate the derivate of the utility function with respect to taxable income.
foc_high = sm.diff(utility_high_sub, z)

# 2) Define that we want to isolate for taxable income to derive z*.
z_star_high = sm.solve(foc_high, z)

# 3) Define the optimal level of taxable income as a Python-function.
z_star_func_high = sm.lambdify((m_H, alp, eps), z_star_high)

# 4) Examine the optimal level of taxable income.
z_star_high
```

We denote this level of optimal taxable income as $z_{high} ^*$. We note that this is the same optimal taxable income as in section 3.1 besides the marginal tax rate. However, it is not possible in this function to show that high-income earners tend to earn an income which is identical to the threshold. This is called "bunching" since people tend to bunch at kink points. If people with potential earnings above the threshold choose an optimal taxable income such that $z_{high}^*<K$ and $z^*>K$ then people will bunch at the kink. This is possible to show in a graphical analysis.

## Graphical analysis

In this section we will analyze the behavior of the worker grahpically. First, we will find the maximum utility of the worker, and then we will plot the indifference curve with the budget constraint the same way as in section 3.2.

```python
# 1) Define a function that return the maximum utility level given alpha, epsilon, low tax rate, high tax rate and the threshold
def value_of_choice_2(alpha, epsilon, tax_low, tax_high, kink):

    # 1.1) If the alpha is below the income threshold, the maximum utility is found using the old budget constraint without the tax rate for high income earners
    if alpha < kink:
        global z_opt_new
        z_opt_new = (alpha*(1-tax_low)**epsilon)
        tax_opt_new = z_opt_new * tax_low
        utility_f = (1-tax_low)*(alpha*(1-tax_low)**epsilon) - alpha/(1+1/epsilon)*(alpha*(1-tax_low)**epsilon/alpha)**(1+1/epsilon)
        return utility_f, z_opt_new, tax_opt_new

    # 1.2) Find the maximum utility if alpha is above the threshold
    if alpha >= kink:

        # 1.3) Find the utility for the budget constaint only below the threshold
        utility_l = (1-tax_low)*(alpha*(1-tax_low)**epsilon) - alpha/(1+1/epsilon)*(alpha*(1-tax_low)**epsilon/alpha)**(1+1/epsilon)
        z_opt_low = (alpha*(1-tax_low)**epsilon)
        tax_opt_low = z_opt_low * tax_low

        # 1.4) If the optimal income earned on the low budget set is above the threshold the low utility should be evaluated in the threshold
        if alpha*(1-tax_low)**epsilon>kink:
            utility_l = (1-tax_low)*kink - alpha/(1+1/epsilon)*(kink/alpha)**(1+1/epsilon)
            z_opt_low = kink
            tax_opt_low = kink * tax_low

        # 1.5) Find the utility for the budget constaint only above the threshold
        utility_h = (1-tax_low)*kink+(1-tax_high)*(alpha*(1-tax_high)**epsilon-kink) - alpha/(1+1/epsilon)*(alpha*(1-tax_high)**epsilon/alpha)**(1+1/epsilon)
        z_opt_high = (alpha*(1-tax_high)**epsilon)
        tax_opt_high = kink * tax_low + (z_opt_high - kink) * tax_high

        # 1.4) If the optimal income earned on the high budget set is below the threshold the high utility should be evaluated in the threshold
        if alpha*(1-tax_high)**epsilon<kink:
            utility_h = (1-tax_low)*kink - alpha/(1+1/epsilon)*(kink/alpha)**(1+1/epsilon)
            z_opt_high = kink

        # 1.5) return the values for the higest utility
        utility_g = max(utility_l, utility_h)
        if utility_g==utility_l:
            z_opt_new=z_opt_low
            tax_opt_new = tax_opt_low
        else:
            z_opt_new=z_opt_high
            tax_opt_new = tax_opt_high
        return utility_g, z_opt_new, tax_opt_new

# 2) Define the indifference curve
def indif_curve(budget_con, alpha, epsilon, tax_low, tax_high, k):
    u = value_of_choice_2(alpha, epsilon, tax_low, tax_high, k)[0]
    return u + alpha/(1+1/epsilon)*(budget_con/alpha)**(1+1/epsilon)

# 3) Define a function that makes a plot with the indifference curve, the budget constraint and the optimal behaviour.
def graph2(alpha, epsilon, tax_low=0.35, tax_high=0.7, kink=500):
    plt.figure(figsize=(12,6))
    budget_con = np.arange(0.0, alpha*1.1+100, 1)

    # 3.1) Plot the indifference curve
    plt.plot(budget_con, indif_curve(budget_con, alpha, epsilon, tax_low, tax_high, kink))

    # 3.2) Plot the budget constraints
    lower = np.arange(0.0, kink, 1)
    plt.plot(lower, lower*(1-tax_low))
    upper = np.arange(kink, alpha*1.1+100, 1)
    plt.plot(upper, (1-tax_low)*kink+(upper-kink)*(1-tax_high), color='g')
    plt.axvline(x=kink, color='black', linestyle=':')
    print("The optimal taxable income is: "+str(round(z_opt_new)))
    plt.plot(z_opt_new, indif_curve(z_opt_new, alpha, epsilon, tax_low, tax_high, kink), 'ro')
    plt.ylabel('Consumption')
    plt.xlabel('Taxable income')
    plt.legend(['Indifference curve','Budget constraint (low marginal tax)', 'Budget constraint (high marginal tax)', 'Threshold', 'Optimum'])
    return

# 4) Plot the graph with interactive widgets
widget_ml = widgets.FloatSlider(
    value = 0.5,
    min=0.0,
    max=1,
    step=0.01,
    description='$m_l$:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

widget_mh = widgets.FloatSlider(
    value = 0.5,
    min=0.0,
    max=1,
    step=0.01,
    description='$m_h$:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

widget_kink = widgets.FloatSlider(
    value = 2000,
    min=10,
    max=5000,
    step=10,
    description='$K$:',
    readout_format='.2f',
    width = 1000,
    layout={'width': '500px'}
)

widgets.interact(graph2,
    alpha=widget_alpha, epsilon=widget_epsilon, tax_low=widget_ml, tax_high=widget_mh, kink=widget_kink
);
```

    interactive(children=(FloatSlider(value=5000.0, description='α:', layout=Layout(width='500px'), max=10000.0, m…

We note some of the same results as in the conclusion in section 3.2. In this extended model, we note that people with a potential income slightly above the threshold will bunch at the kink. This is due to the fact that people are unwilling to work at the high marginal tax rate, but will like to work at the low marginal tax rate. Thus, they maximize their utility by having a taxable income exactly equal to the threshold.

## 4.3 The tax rate that maximizes the tax revenue

We will use the distribution of agents we defined in section 3.3 to calculate the tax revenue for different low and high taxes and different thresholds for the high tax. First, we make new draw from the draw function and then define a function which returns the total tax renevue as well as the tax payment from each agent

```python
# 1) Draws from the draw function
N=500
mu_alpha_a=5000
draw(N=N, mu_alpha=mu_alpha_a, mu_epsilon = 0.9, sigma_alpha = 1000000, sigma_epsilon = 0.1, correlation = -0.5)

# 2) Convert the numpy array with the alphas and epsilons into a list
alphas_list = alphas.tolist()
epsilons_list = epsilons.tolist()

# 3) Define a function which return total tax revenue as well as tax payment from each agent
def calc_tau(alphas_list, epsilons_list, tax_low, tax_high, kink):
    # 3.1) Create empty list with all the tax payments
    tau_list = []
    z_list = []
    # 3.2) Calculate the tax payment from every agent using a for loop
    for x in range(N):
        tau_list.append(value_of_choice_2(alphas_list[x], epsilons_list[x], tax_low, tax_high, kink)[2])
        z_list.append(value_of_choice_2(alphas_list[x], epsilons_list[x], tax_low, tax_high, kink)[1])

    return sum(tau_list), tau_list, z_list

# 4) Define function with calculate tax revenue times minus 1 as a function of tax_low, tax_high, kink -
    # given the current values of alphas_list and epsilons_list. The minimum of this function is the maximum tax revenue
def tax_kink_max(params):
    tax_low, tax_high, kink = params
    # 3.1) Create empty list with all the tax payments
    tau_list = []

    # 4.1) Calculate the tax payment from every agent using a for loop
    for x in range(N):
        tau_list.append(value_of_choice_2(alphas_list[x], epsilons_list[x], tax_low, tax_high, kink)[2])
    tau = - sum(tau_list)
    return tau
```

Below is a calculation of the tax revenue given $m=0.4$, $m_H=0.6$ and $K=1000$.

```python
params = [0.4,0.6,1000]
print(f'The tax revenue is: {-tax_kink_max(params):,.0f}')
```

    The tax revenue is: 609,344

We will now use the `optimize.minimize` function to find the tax rates and the threshold for the high tax that maximizes the tax revenue.

```python
# 1) Set the initial guess and the bounds for the tax rates
initial_guess = [0.5, 0.5, 200]
low_bound_tax = 0
high_bound_tax = 1-1e-5
bnds = ((low_bound_tax, high_bound_tax), (low_bound_tax, high_bound_tax), (0,100000))

# 2) Find the tax rate and kink that maximizes the tax revenue
draw(N=N, mu_alpha = mu_alpha_a, mu_epsilon = 0.9, sigma_alpha = 1000000, sigma_epsilon = 0.1, correlation = -0.5)
alphas_list = alphas.tolist()
epsilons_list = epsilons.tolist()
result = optimize.minimize(tax_kink_max, initial_guess, bounds=bnds)
print(f'mu_alpha = {mu_alpha_a:.0f}, mu_epsilon = 0.9, sigma_alpha = 1000000, sigma_epsilon = 0.1, correlation = -0.5 \n')
print(f'The maximizing low tax rate is:    {result.x[0]*100:.2f} pct.')
print(f'The maximizing high tax rate is:   {result.x[1]*100:.2f} pct.')
print(f'The maximizing kink rate is:       {result.x[2]:,.0f}')
print(f'The tax revenue is:                {-result.fun:,.0f} \n \n')
```

    mu_alpha = 5000, mu_epsilon = 0.9, sigma_alpha = 1000000, sigma_epsilon = 0.1, correlation = -0.5

    The maximizing low tax rate is:    100.00 pct.
    The maximizing high tax rate is:   13.27 pct.
    The maximizing kink rate is:       1,753
    The tax revenue is:                837,629

The result above shows that the government should tax all income below 1,752.5 with 100 pct. and then tax any income above 1,752.5 with 13.27 pct. The tax revenue is 837,629 compared to 710,528 in the linear tax system. [Not that this optimization is not completely accurate, in particular for extreme value of $\gamma$ and $\Sigma$]

We will now plot the tax revenue as a function of the low tax rate, the high tax rate and the kink in three different plot, respectivly.

```python
# 1) Define a function which return a list of tax revenues for differente low tax rates given the high tax rate and the kink
def laffer_fun_low(tax_high,kink):
    x=0
    listen = []
    xlist = []
    increment = 0.01
    while x<1:
        xlist.append(x)
        listen.append(calc_tau(alphas_list, epsilons_list, x, tax_high, kink)[0])
        x += increment
    return listen, xlist

# 2) Define a function which return a list of tax revenues for differente high tax rates given the low tax rate and the kink
def laffer_fun_high(tax_low,kink):
    x=0
    listen = []
    xlist = []
    increment = 0.01
    while x<1:
        xlist.append(x)
        listen.append(calc_tau(alphas_list, epsilons_list, tax_low, x, kink)[0])
        x += increment
    return listen, xlist

# 3) Define a function which return a list of tax revenues for differente kinks given the low tax rate and the high tax
def laffer_fun_kink(tax_low,tax_high):
    x=0
    listen = []
    xlist = []
    increment = 100
    while x<10000:
        xlist.append(x)
        listen.append(calc_tau(alphas_list, epsilons_list, tax_low, tax_high, x)[0])
        x += increment
    return listen, xlist

# 4) Make list containing the tax revenues for all low tax rate for differente high tax rates and kinks
x = laffer_fun_low(0,4752)[1]
low_1 = laffer_fun_low(result.x[1],result.x[2])[0]
low_2 = laffer_fun_low(result.x[1],result.x[2]*1.15)[0]
low_3 = laffer_fun_low(0,mu_alpha_a/2)[0]

# 5) Make list containing the tax revenues for all high tax rate for differente low tax rates and kinks
high_1 = laffer_fun_high(result.x[0],result.x[2])[0]
high_2 = laffer_fun_high(result.x[0],result.x[2]*1.15)[0]
high_3 = laffer_fun_high(0.5,mu_alpha_a/2)[0]

# 6) Make list containing the tax revenues for all kinks for differente low tax rates and high tax rates
x_kink = laffer_fun_kink(result.x[0],result.x[1])[1]
kink_1 = laffer_fun_kink(result.x[0],result.x[1])[0]
kink_2 = laffer_fun_kink(0.4,0.6)[0]
kink_3 = laffer_fun_kink(1,0)[0]

# 7) Print the plots
import matplotlib

fig, (ax1,ax2) = plt.subplots(1, 2, figsize=(15, 6))
ax1.get_yaxis().set_major_formatter(
matplotlib.ticker.FuncFormatter(lambda x, p: format(int(x), ',')))
ax2.get_yaxis().set_major_formatter(
matplotlib.ticker.FuncFormatter(lambda x, p: format(int(x), ',')))

# 8) Tax revenue as a function of the low tax rate
ax1.plot(x,low_1, color='r')
ax1.plot(x,low_2, color='b', linestyle=':')
ax1.plot(x,low_3, color='b', linestyle='--')
ax1.legend([f'High tax = {result.x[1]:.2f}, kink = {result.x[2]:.0f}',f'High tax = {result.x[1]:.2f}, kink = {result.x[2]*1.5:.0f}',f'High tax = 0.5, kink = {mu_alpha_a/2:.0f}'])
ax1.set_ylabel('Tax revenue')
ax1.set_xlabel('Tax rate')
ax1.set_title('Laffer curve for low tax rate')

# 9) Tax revenue as a function of the low tax rate
ax2.plot(x,high_1, color='r')
ax2.plot(x,high_2, color='b', linestyle=':')
ax2.plot(x,high_3, color='b', linestyle='--')
ax2.legend([f'Low tax = {result.x[0]:.2f}, kink = {result.x[2]:.2f}',f'Low tax = {result.x[0]:.2f}, kink = {result.x[2]*1.2:.2f}',f'Low tax = 0.5, kink = {mu_alpha_a/2:.2f}'])
ax2.set_ylabel('Tax revenue')
ax2.set_xlabel('Tax rate')
ax2.set_title('Laffer curve for high tax rate')
plt.show()

# 10) Tax revenue as a function of the kink
fig, ax3 = plt.subplots(1, 1, figsize=(15, 6))
ax3.get_yaxis().set_major_formatter(
matplotlib.ticker.FuncFormatter(lambda x, p: format(int(x), ',')))

ax3.plot(x_kink,kink_1, color='r')
ax3.plot(x_kink,kink_2, color='b', linestyle=':')
ax3.plot(x_kink,kink_3, color='b', linestyle='--')
ax3.set_ylabel('Tax revenue')
ax3.set_xlabel('Threshold for high tax (kink)')
ax3.set_title('Laffer curve for kink')
ax3.legend([f'Low tax = {result.x[0]:.2f}, high tax = {result.x[1]:.2f}',f'Low tax = 0.4, high tax = 0.6',f'Low tax = 0.8, high tax = 0.2'])

plt.show()

```

To see how the tax rate affect the agents behavior, we have below made a histogram which plots the distribution of the tax payments and the taxable income as a function of the three tax parameters. The function also prints the tax revenue.

```python
# 1) Make a function which plots a histogram over the tax payments and the taxable income
def histo(tax_low=1, tax_high=0, kink=4000):
    plt.figure(figsize=(12,8))
    graf = calc_tau(alphas_list, epsilons_list, tax_low, tax_high, kink)
    tax_max = max(graf[2])
    bins = np.linspace(0, tax_max, 50)
    plt.hist(graf[1], bins=bins, color='r', alpha=0.5)
    plt.hist(graf[2], bins=bins, color='b', alpha=0.5)
    plt.legend(['Tax payment', 'Taxable income'])
    plt.show()
    print(f'The tax provenue is {graf[0]:,.0f}')

widgets.interact(histo,
    tax_low=widget_ml, tax_high=widget_mh, kink=widget_kink
);
```

    interactive(children=(FloatSlider(value=0.5, description='$m_l$:', layout=Layout(width='500px'), max=1.0, step…

### Conclusion

In this project, we have examined how people react to changes in the marginal tax rate. We have found that a larger marginal tax rate and a higher elasticity affect the labor supply negatively in a proportional tax system. This is shown analytically and by a graph, where one can change the parameter values.

Most Western countries apply a progressive tax system, where high-income earners face a greater marginal tax rate. We show analyticcaly that initial top tax payers only are affected by the largest marginal tax rate. We have shown in the graph that people will tend to bunch at the kink, so they earn the highest possible income without having to pay the largest marginal tax rate. This is in line with the literature.


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, G.F. Forsyth, C. Cooper. Based on [CFDPython](https://github.com/barbagroup/CFDPython), (c)2013 L.A. Barba, also under CC-BY license.

# Space & Time

## 1-D Diffusion

Welcome back! This is the third IPython Notebook of the series *Space and Time — Introduction of Finite-difference solutions of PDEs*, the second module of ["Practical Numerical Methods with Python"](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about).

In the previous IPython notebooks of this series, we studied the numerical solution of the linear and non-linear convection equations using the finite-difference method, and learned about the CFL condition. Now, we will look at the one-dimensional diffusion equation:

\begin{equation}\frac{\partial u}{\partial t}= \nu \frac{\partial^2 u}{\partial x^2}\end{equation}

where $\nu$ is a constant known as the *diffusion coefficient*.

The first thing you should notice is that this equation has a second-order derivative. We first need to learn what to do with it!

### Discretizing 2nd-order derivatives

The second-order derivative can be represented geometrically as the line tangent to the curve given by the first derivative.  We will discretize the second-order derivative with a Central Difference scheme: a combination of forward difference and backward difference of the first derivative.  Consider the Taylor expansion of $u_{i+1}$ and $u_{i-1}$ around $u_i$:

$u_{i+1} = u_i + \Delta x \frac{\partial u}{\partial x}\big|_i + \frac{\Delta x^2}{2} \frac{\partial ^2 u}{\partial x^2}\big|_i + \frac{\Delta x^3}{3} \frac{\partial ^3 u}{\partial x^3}\big|_i + {\mathcal O}(\Delta x^4)$

$u_{i-1} = u_i - \Delta x \frac{\partial u}{\partial x}\big|_i + \frac{\Delta x^2}{2} \frac{\partial ^2 u}{\partial x^2}\big|_i - \frac{\Delta x^3}{3} \frac{\partial ^3 u}{\partial x^3}\big|_i + {\mathcal O}(\Delta x^4)$

If we add these two expansions, the odd-numbered derivatives will cancel out.  Neglecting any terms of ${\mathcal O}(\Delta x^4)$ or higher (and really, those are very small), we can rearrange the sum of these two expansions to solve for the second-derivative.

$u_{i+1} + u_{i-1} = 2u_i+\Delta x^2 \frac{\partial ^2 u}{\partial x^2}\big|_i + {\mathcal O}(\Delta x^4)$

And finally:

\begin{equation}\frac{\partial ^2 u}{\partial x^2}=\frac{u_{i+1}-2u_{i}+u_{i-1}}{\Delta x^2} + {\mathcal O}(\Delta x^2)\end{equation}

The central difference approximation of the 2nd-order derivative is 2nd-order accurate.

### Back to diffusion

We can now write the discretized version of the diffusion equation in 1D:

\begin{equation}\frac{u_{i}^{n+1}-u_{i}^{n}}{\Delta t}=\nu\frac{u_{i+1}^{n}-2u_{i}^{n}+u_{i-1}^{n}}{\Delta x^2}\end{equation}

As before, we notice that once we have an initial condition, the only unknown is $u_{i}^{n+1}$, so we re-arrange the equation to isolate this term:

\begin{equation}u_{i}^{n+1}=u_{i}^{n}+\frac{\nu\Delta t}{\Delta x^2}(u_{i+1}^{n}-2u_{i}^{n}+u_{i-1}^{n})\end{equation}

This discrete equation allows us to write a program that advances a solution in time—but we need an initial condition. Let's continue using our favorite: the hat function. So, at $t=0$, $u=2$ in the interval $0.5\le x\le 1$ and $u=1$ everywhere else.

### Stability of the diffusion equation

The diffusion equation is not free of stability constraints. Just like the linear and non-linear convection equations, there are a set of discretization parameters $\Delta x$ and $\Delta t$ that will make the numerical solution blow up. For the diffusion equation and the discretization used here, the stability condition for diffusion is

$$
\begin{equation}
\nu \frac{\Delta t}{\Delta x^2} \leq \frac{1}{2}
\end{equation}
$$

### And solve!

 We are ready to number-crunch!

The next two code cells initialize the problem by loading the needed libraries, then defining the solution parameters and initial condition. This time, we don't let the user choose just *any* $\Delta t$, though; we have decided this is not safe: people just like to blow things up. Instead, the code calculates a value of $\Delta t$ that will be in the stable range, according to the spatial discretization chosen! You can now experiment with different solution parameters to see how the numerical solution changes, but it won't blow up.

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

```python
nx = 41
dx = 2./(nx-1)
nt = 20
nu = 0.3   #the value of viscosity
sigma = .2
dt = sigma*dx**2/nu

x = numpy.linspace(0,2,nx)
ubound = numpy.where(x >= 0.5)
lbound = numpy.where(x <= 1)

u = numpy.ones(nx)
u[numpy.intersect1d(lbound, ubound)] = 2

un = numpy.ones(nx)
```

```python
for n in range(nt):
    un = u.copy()
    u[1:-1] = un[1:-1] + nu*dt/dx**2*(un[2:] -2*un[1:-1] +un[0:-2])

pyplot.plot(x, u, color='#003366', ls='--', lw=3)
pyplot.ylim(0,2.5);
```

## Animations

Looking at before-and-after plots of the wave in motion is helpful, but it's even better if we can see it changing! For displaying animated results within the notebook, there is a handy library called JSAnimation. To install it, [follow these instructions](https://github.com/numerical-mooc/numerical-mooc/wiki/HOWTO:-Install-JSAnimation-for-IPython-Notebook).

First, let's import the animation libraries:

```python
from JSAnimation.IPython_display import display_animation
from matplotlib import animation
```

Now we need to reset our initial conditions!

```python
nt = 50

u = numpy.ones(nx)
u[numpy.intersect1d(lbound,ubound)] = 2

un = numpy.ones(nx)
```

Now we have to create an animation.  This takes a few steps, but it's actually not hard to do!  First, we need to create a `figure` and add `axes`.  The `figure` defines the plot we want to animate.  The `axes` lets us create elements of the plot that we can change interactively.  Then we create a "blank" line that we will update at each iteration. Like this:

```Python
fig = pyplot.figure(figsize=(11,8))
ax = pyplot.axes(xlim=(0,2), ylim=(1,3))
line = ax.plot([], [], lw=2)[0]
```

**Note**: The `[0]` after the `ax.plot()` command is required, since `ax.plot` can (optionally) return several values.  Since we're only creating one line, we ask it for the "zeroth" (and only...) line.

Now that our `figure` is set up, we can define a function to evolve our solution plot.  This will be exactly the same as the code for the numerical solution, except that we also want to "update" the data of the solution line at each iteration.

```Python
def diffusion(i):
    line.set_data(x,u)

    un = u.copy()
    u[1:-1] = un[1:-1] + nu*dt/dx**2*\
            (un[2:] - 2*un[1:-1] + un[0:-2])
```

Finally, we can initialize our animation using `animation.FuncAnimation()`.  We're going to pass the following arguments:

*  `fig`: the name of our figure
*  `diffusion`: the name of our solver function
*  `frames`: the number of frames to draw (which we set equal to `nt`)
*  `interval`: the number of milliseconds each frame appears for

Let's put it all together. It's time to animate!

```python
fig = pyplot.figure(figsize=(8,5))
ax = pyplot.axes(xlim=(0,2), ylim=(1,2.5))
line = ax.plot([], [], color='#003366', ls='--', lw=3)[0]

def diffusion(i):
    line.set_data(x,u)

    un = u.copy()
    u[1:-1] = un[1:-1] + nu*dt/dx**2*(un[2:] -2*un[1:-1] +un[0:-2])

animation.FuncAnimation(fig, diffusion,
                        frames=nt, interval=100)
```

    <input id="_anim_sliderIGPBDYFDGDQDDAPB" type="range" style="width:350px" name="points" min="0" max="1" step="1" value="0" onchange="animIGPBDYFDGDQDDAPB.set_frame(parseInt(this.value));"></input>

    <button onclick="animIGPBDYFDGDQDDAPB.slower()">–</button>
    <button onclick="animIGPBDYFDGDQDDAPB.first_frame()"></button>
    <button onclick="animIGPBDYFDGDQDDAPB.previous_frame()"></button>
    <button onclick="animIGPBDYFDGDQDDAPB.reverse_animation()"></button>
    <button onclick="animIGPBDYFDGDQDDAPB.pause_animation()"></button>
    <button onclick="animIGPBDYFDGDQDDAPB.play_animation()"></button>
    <button onclick="animIGPBDYFDGDQDDAPB.next_frame()"></button>
    <button onclick="animIGPBDYFDGDQDDAPB.last_frame()"></button>
    <button onclick="animIGPBDYFDGDQDDAPB.faster()">+</button>
  <form action="#n" name="_anim_loop_selectIGPBDYFDGDQDDAPB" class="anim_control">
    <input type="radio" name="state" value="once" > Once </input>
    <input type="radio" name="state" value="loop" checked> Loop </input>
    <input type="radio" name="state" value="reflect" > Reflect </input>
  </form>

---

######The cell below loads the style of the notebook.

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>


# Exercise 4 - Logic Programming Using SymPy
## a) Checking for Prime numbers

### AIM:
To write a python program for checking prime numbers using SymPy.

### FUNCTIONS USED:
```
sympy.prime(n)
    Input : n - A number
   Output : nth prime number

sympy.isprime(n)
    Input : n - A number
   Output : Bool - whether n is a prime

sympy.primerange(start, end)
    Input : start - start of the prime range
            end - end of the prime range
   Output : Generator yeilding prime numbers in
            the range [start,end)

sympy.sieve.primerange(n)
    Same as sympy.primerange
    But uses a Sieve instance to store the
    previously computed values which is
    more efficient and consumes more space

sympy.randprime(start, end)
    Input : start - start of the  range
            end - end of the  range
   Output : A random prime number within
            the range [start,end)

sympy.prevprime(n)
    Input : n - A number
   Output : largest prime number less than n

sympy.nextprime(n)
    Input : n - A number
   Output : smallest prime number greater than n
```

### SOURCE CODE:

```python
import sympy
if __name__ == "__main__":
    print("The 5th prime is:",sympy.prime(5))
    print("The 13th prime is:",sympy.prime(13))
    print("Is 5 a prime number:",sympy.isprime(5))
    print("Is 6 a prime number:",sympy.isprime(6))
    print(
        "The prime numbers between 0 & 100 are:",
        list(sympy.primerange(0, 100)),sep="\n"
    )
    print(
        "The prime numbers between 0 & 100 are(Using Sieve):",
        list(sympy.sieve.primerange(0, 100)),sep="\n"
    )
    print("A random prime between 0 & 100:", sympy.randprime(0, 100))
    print("A random prime between 0 & 100:", sympy.randprime(0, 100))
    print("The largest prime less than 50 is:", sympy.prevprime(50))
    print("The smallest prime greater than 50 is:", sympy.nextprime(50))
```

    The 5th prime is: 11
    The 13th prime is: 41
    Is 5 a prime number: True
    Is 6 a prime number: False
    The prime numbers between 0 & 100 are:
    [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
    The prime numbers between 0 & 100 are(Using Sieve):
    [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
    A random prime between 0 & 100: 37
    A random prime between 0 & 100: 19
    The largest prime less than 50 is: 47
    The smallest prime greater than 50 is: 53

---

## b) Matching Mathematical Exprressions

### AIM:
To write a python program for matching mathematical exprressions using SymPy.

### FUNCTIONS USED:
```
sympy.diff(f, *variables)
    Input : f - A mathematical function
            variables - Mathematical variables
   Output : The deriviative of the function f
            with respect to the variables

sympy.integrate(f, *variables)
    Input : f - A mathematical function
            variables - Mathematical variables
   Output : The anti-deriviative or integral of the function f
            with respect to the variables

sympy.sin(x)
    Input : x - A number
   Output : Mathematical sine value of the number x

sympy.cos(x)
    Input : x - A number
   Output : Mathematical cosine value of the number x

sympy.exp(x)
    Input : x - A number
   Output : Mathematical exponential  value = e^x

sympy.limit(e, x, x0, dir)
    Input : e - expression, the limit of which is to be taken
            x -  variable in the limit.
            x0 - the value toward which x tends.
            dir - direction of approach of the limit
                  accepts the values ["+", "-", "+-"]
   Output : The mathematical limit of the expression e as
            the variable x tends to x0

sympy.solve(f,*variables)
    Input : f   - polynomial
                - transcendental
                - piecewise combinations of the above
                - systems of linear and polynomial equations
                - systems containing relational expressions
            variables - Mathematical variables
   Output : The required mathematical solution
            for the input(varies according to the input)
```

### SOURCE CODE:

```python
from sympy import (
    diff, integrate, sin, cos,
    exp, limit, solve, oo
)
from sympy.abc import x
if __name__ == "__main__":
    print("The derivative of sin(x)*e^x is :", diff(sin(x)*exp(x), x))
    print(
        "The indefinite integration of e^x*sin(x)+e^x*cos(x) is :",
        integrate(exp(x)*sin(x) + exp(x)*cos(x), x)
    )
    print(
        "The definite integration of sin(x^2) from -oo to oo is :",
        integrate(sin(x**2), (x, -oo, oo))
    )
    print(
        "The limit of sin(x)/x as x tends to 0 is :",
        limit(sin(x)/x, x, 0)
    )
    print(
        "The roots of the equation x^2-2 = 0 are :",
        solve(x**2 - 2, x)
    )
```

    The derivative of sin(x)*e^x is : exp(x)*sin(x) + exp(x)*cos(x)
    The indefinite integration of e^x*sin(x)+e^x*cos(x) is : exp(x)*sin(x)
    The definite integration of sin(x^2) from -oo to oo is : sqrt(2)*sqrt(pi)/2
    The limit of sin(x)/x as x tends to 0 is : 1
    The roots of the equation x^2-2 = 0 are : [-sqrt(2), sqrt(2)]

---


```python
import numpy as np

import matplotlib
import matplotlib.pyplot as plt
import matplotlib.ticker
from IPython.display import Image
%matplotlib inline

import seaborn as sns
```

Рассмотрим нелинейную систему

 \begin{cases}
   sin(x+2)-y=6\\
   x+ cos(y+2) = 2
 \end{cases}

```python
'''Корни согласно WolframAlpha'''
x_true = [1.9733057420725957950, -6.7390864096433335238] # WolframAlpha
```

```python
# невязка
def residual(u):
    x, y = u
    f1 = abs(np.sin(x+2) - y - 6)
    f2 = abs(x + np.cos(y+2) - 2)
    return [f1, f2]
```

## Метод простой итерации

Сведем систему к виду $u = f(u)$:

 \begin{cases}
   x = 2 - cos(y+2)\\
   y = sin(x+2)- 6\\
 \end{cases}

 $u^{n+1} = f(u^n)$ - МПИ, где $f(x, y) = \begin{pmatrix}
 2 - cos(y+2) \\
 sin(x+2) - 6
 \end{pmatrix}$

```python
def f(u):
    x, y = u
    return [2-np.cos(y+2), np.sin(x+2)-6]

def p(u1, u2):
    return ((u1[0]-u2[0])**2 + (u1[1]-u2[1])**2)**0.5
```

```python
# метод простой итерации
def msi(u0, eps, verbose=False, extend=False):
    x, y = u0
    fst_step = p(f([x,y]), [x,y])
    n = 0
    q = 0.5**0.5
    extend_xs, extend_ys = [x], [y]
    while not q**n*fst_step/(1-q) < eps:
        x, y = f([x, y])
        extend_xs.append(x)
        extend_ys.append(y)
        n += 1
    if verbose:
        print(f'Метод сошелся за {n} итераций')
    if extend:
        return [x, y, n]
    return [x, y]
```

```python
u0 = [0,0]
eps = 1e-6
res = msi(u0, eps, verbose=True)
print(f"Результат: {res}, ошибка в евк. норме: {p(res, x_true)}")
print(f"Невязка метода: {residual(res)}")
```

    Метод сошелся за 49 итераций
    Результат: [1.973330483019676, -6.739061348581915], ошибка в евк. норме: 3.5216065394487795e-05
    Невязка метода: [4.172660013157525e-05, 3.111920789944378e-07]

## Метод Ньютона

 \begin{cases}
   x - 2 + cos(y+2) = 0\\
   y - sin(x+2) + 6 = 0\\
 \end{cases}

 $u^{n+1} = u^n - J^{-1} F(u^n)$ - МПИ, где $F(x, y) = \begin{pmatrix}
 x - 2 + cos(y+2) \\
 y - sin(x+2) + 6
 \end{pmatrix}$ и
 $J = \begin{pmatrix}
 1 & -sin(y+2) \\
 -cos(x+2) & 1
 \end{pmatrix}$

 Отметим, что
 $J^{-1} = \frac{1}{1-sin(y+2)cos(x+2)} \begin{pmatrix}
 1 & sin(y+2) \\
 cos(x+2) & 1
 \end{pmatrix}$

 Рассмотрим окресность вблизи решения. В качестве критерия останова будем требовать выполнение условия на итерационную поправку:

 \begin{equation}
 ||x^{k}-x^{k-1}||\le \epsilon
 \end{equation}

 Будем требовать от метода сходимости за 50 итераций, т.е. если за 50 итераций критерий останова не выполнен считаем, что метод расходится.

```python
# вычисление обратной матрицы
def inv_J(u):
    x, y = u
    return np.dot([[1, np.sin(y+2)], [np.cos(x+2), 1]], 1/(1-np.sin(y+2)*np.cos(x+2)))
```

```python
# ф-ция F
def F(u):
    x, y = u
    return [x-2+np.cos(y+2), y-np.sin(x+2)+6]
```

```python
# метод Ньютона
def Newton_method(u0, eps, iter_num, verbose=False, extend=False):
    x, y = u0
    n = 0
    extend_xs, extend_ys = [x], [y]
    print('0-я итерация: %f,%f' % (extend_xs[0], extend_ys[0]))
    while True:
        x, y = x - np.dot(inv_J([x,y]), F([x,y]))[0], y - np.dot(inv_J([x,y]), F([x,y]))[1]
        extend_xs.append(x)
        extend_ys.append(y)
        n += 1
        print('%i-я итерация: %f,%f' % (n, extend_xs[n], extend_ys[n]))
        if (p([extend_xs[-2:-1][0], extend_ys[-2:-1][0]], [extend_xs[-1:][0], extend_ys[-1:][0]]) < eps) or (n > iter_num):
            break
    if verbose:
        print(f'Метод сошелся за {n} итераций')
    if extend:
        return [x, y, n]
    return [x, y]
```

```python
eps = 1e-6
res = Newton_method([2, -4], eps = eps, iter_num = 50, verbose = True)
print(f"Результат: {res}, ошибка в евк. норме: {p(res, x_true)}")
print(f"Невязка метода: {residual(res)}")
```

    0-я итерация: 2.000000,-4.000000
    1-я итерация: 9.205588,-11.466689
    2-я итерация: 3.134159,-8.243760
    3-я итерация: 1.019127,-7.778210
    4-я итерация: 1.711658,-6.565185
    5-я итерация: 1.962775,-6.751094
    6-я итерация: 1.973280,-6.739110
    7-я итерация: 1.973306,-6.739086
    8-я итерация: 1.973306,-6.739086
    Метод сошелся за 8 итераций
    Результат: [1.9733057420725957, -6.739086409643334], ошибка в евк. норме: 2.220446049250313e-16
    Невязка метода: [0.0, 0.0]

### Сравнительная характеристика сходимости методов

Для МПИ рассмотрим число итераций до остановки метода для разных элементов сетки

```python
a, N = 25, 26
xs = np.linspace(2-a, 2+a, N)
ys = np.linspace(-7-a, -7+a, N)

ns = []
for x in xs:
    string = []
    for y in ys:
        _, _, n = msi([x, y], 1e-5, extend=True)
        string.append(n)
    ns.append(string)
```

```python
plt.figure(figsize=(10,8))
sns.heatmap(ns, annot=True, xticklabels=xs, yticklabels=ys);
#plt.scatter([x_true[0]], [x_true[1]], c='white');
```

Будем рассматривать сходимость по сетке в случае метода Ньютона.

```python
a, N = 25, 26
xs = np.linspace(2-a, 2+a, N)
ys = np.linspace(-7-a, -7+a, N)

plt.figure(figsize=(8,6))

for x in xs:
    for y in ys:
        x0, y0 = Newton_method([x, y], 1e-6, 50)
        if p([x0, y0], x_true) < 1e-6:
            plt.scatter([x], [y], c='g', s=15, alpha=0.7)
        else:
            plt.scatter([x], [y], c='r', marker='x', alpha=0.7)

plt.scatter([x_true[0]], [x_true[1]], c='black');
plt.xlabel('$x$');
plt.ylabel('$y$');
```

##### Рассмотрим маленькую окресность решения (видим что для начальных значений, взятых из достаточно малой окресности решения метод сходится)

```python
a, N = 25, 26
xs = np.linspace(1.8, 2.15, N)
ys = np.linspace(-6.9, -6.5, N)

plt.figure(figsize=(8,6))

for x in xs:
    for y in ys:
        x0, y0 = Newton_method([x, y], 1e-6, 50)
        if p([x0, y0], x_true) < 1e-6:
            plt.scatter([x], [y], c='g', s=15, alpha=0.7)
        else:
            plt.scatter([x], [y], c='r', marker='x', alpha=0.7)
            print(x, y, p([x0, y0], x_true))

plt.scatter([x_true[0]], [x_true[1]], c='black');
plt.xlabel('$x$');
plt.ylabel('$y$');
```

Как видим в достаточно малой окресности решения для всех узлов метод Ньютона сходится.


# Práctica de multplicadores de Lagrange

En la clase pasada aprendimos la optimización por medio de multiplicadores de Lagrange cuando la función objetivo está sujeta a ciertas restricciones.

En resumen:

1. Obtener la función lagrangiana.
2. Derivar parcialmente.
3. Encontrar el punto crítico.
4. Evaluar el punto crítico.
5. Evaluar otro punto que cumpla la restricción.
6. Clasificar el punto crítico.

```python
from sympy import *    # cargar paquetes para hacer calculos matemáticos
init_printing(use_latex=True)
```

## Ejemplo aplicado

Una empresa puede elaborar
su producto en dos de sus plantas. El costo de producir
$x$ unidades en su primera planta y $y$ unidades en la segunda
planta está dado por la función conjunta de costo
$C(x, y) = x^2 +2y^2+5xy+700$. Si la empresa tiene una
orden de suministrar 500 unidades, ¿cuántas unidades debe
producir en cada planta con el objetivo de minimizar el
costo total?

```python
x,y,l =symbols("x,y,l")
```

```python
C=x**2+2*y**2+5*x*y+700
C
```

```python
g=x+y-500 # x+y=500 pero se debe igualar a cero
g
```

```python
L=C-l*g
L
```

```python
Lx=diff(L,x)
Ly=diff(L,y)
Ll=diff(L,l)
```

```python
pc=solve([Lx,Ly,Ll],[x,y,l])
pc
```

```python
C.subs([(x,125),(y,375)])
```

```python
solve(g.subs(x,100),y) # Punto para comparar
```

```python
C.subs([(x,100),(y,400)])
```

Esto implica que el punto crítico $(125, 375)$ es un máximo y 531.950 es el valor máximo de la función.

## Ejemplo de inversiones

Una inversión de $p$ dólares en las cuatro inversiones
A, B, C y D da como resultado un rendimiento
de $\sqrt{p}$, $\sqrt{1.2p}$, $\sqrt{1.3p}$ y $\sqrt{1.5p}$ dólares, respectivamente.
Una persona desea invertir \$12,000 en estas cuatro inversiones.
¿Cuánto deberá invertir en cada una de ellas para
maximizar el rendimiento anual?

```python
A,B,C,D,l =symbols("A,B,C,D,l")
```

A dinero invertido en el primer fondo

B dinero invertido en el segundo fondo

C dinero invertido en el tercer fondo

D dinero invertido en el cuarto fondo

```python
R=sqrt(A)+sqrt(1.2*B)+sqrt(1.3*C)+sqrt(1.5*D)
R
```

```python
g=A+B+C+D-12000 # A+B+C+D=12000
g
```

```python
L=R-l*g
L
```

```python
LA=diff(L,A)
LB=diff(L,B)
LC=diff(L,C)
LD=diff(L,D)
Ll=diff(L,l)

```

```python
pc=solve([LA,LB,LC,LD,Ll],[A,B,C,D,l])
pc
```

```python
round(R.subs([(A,2400),(B,2880),(C,3120), (D,3600)]),2)
```

```python
solve(g.subs([(A,2300),(B,2800),(C,3000)]),D) # Punto para comparar
```

```python
round(R.subs([(A,2300),(B,2800),(C,3000), (D,3900)]),2)
```

Se deben invertir 2400 dólares en el fondo A, 2880 dólares en el fondo B,3120 dólares en el fondo C,3600 US\$ en el fondo D para obtener un rendimiento máximo de 244.95 dólares al año.

# Ejercicios

Mediante el método de multiplicadores de Lagrange, determine y clasifique los puntos críticos de $f$ sujetos a las restricciones
dadas.

1. $f(x, y) =x^2  + y^2$  sujeto a $2x + 3y = 7$
2. $g(x, y) = x^2 + y^2 - 3xy$  sujeto a $2x + 3y= 31$.
3. $h(x, y) = 3x + 2y$  sujeto a $x^2 + y^2 = 13$.
4. $m(u, v, w, x) =3u^2 -v^2 + 2w^2 + x^2$  sujeto a $3u + v - 2w + 4x = 20$.

5.  La función de
producción de una empresa es $P(L, K) = 80L^{3/4}K^{1/4}$, en
donde $L$ y $K$ representan el número de unidades de mano de
obra y de capital utilizadas y $P$ es el número de unidades
elaboradas del producto. Cada unidad de mano de obra tiene
un costo de $\$60$ y cada unidad de capital cuesta $\$200$ y
la empresa dispone de $\$40,000$ destinados a producción.

Aplicando el método de multiplicadores de Lagrange
determine el número de unidades de mano de obra y de
capital que la empresa debe emplear para obtener una
producción máxima. ¿Cuál es esa producción máxima de la empresa?

6. Si se gastan $x$ miles de dólares en mano de obra y $y$ miles de dólares en equipo, la producción de cierta fábrica
será de
$Q(x, y) = 60x^{1/3}y^{2/3}$ unidades. Si hay US$120.000 disponibles, ¿Cómo debe distribuirse el dinero, entre mano de obra y equipo,
para general la mayor producción posible? ¿Cuál es la producción?

7. La función de producción de una empresa es $P(L, K) = 800\sqrt{3L^2+1.5K^2}$, en donde $L$ y $K$ representan el número de unidades de mano de
obra y de capital utilizadas y $P$ es el número de unidades elaboradas del producto. Los costos unitarios de la mano de obra y del capital son
de $\$250$ y $\$50$ y la empresa dispone de $\$6750$ para gastar en producción.

Aplicando el método de multiplicadores de Lagrange
determine el número de unidades de mano de obra y de
capital que la empresa debe emplear para obtener una
producción máxima. ¿Cuál es esa producción máxima de la empresa?

Taller multiplicadores de Lagrange para entregar.ipynb


## Density-based clustering

In density-based clustering the approach is different compared to distributed clustering. We need to implement all functions from scratch. The libraries that we are going to use are the same as in previous example, but in this case we have also the random package that is used to shuffle the objects in the data set.

```python
import random
import numpy as np
import pandas as pd
from math import sqrt
```

DBScan is an example of a density-based clustering method. The goal is to find all element where the neighborhood is defined as:
\begin{equation}
    N_{\epsilon}:{q|d(p,q)\leq\epsilon},
\end{equation}
where $p$ and $q$ are two elements of the training data set and $\epsilon$ is the neighborhood distance. For the data set used before and $\epsilon$ to 0.25 we get the regions like in figure below.

Let's setup the variables as in previous examples. The are three new ones like ```distance_matrix```, ```max_distance```, ```number_of_cluster```, and ``min_points``. The first one is clear, the second is a parameter that can be changed, depending on that how many neighborhood elements we would like to concider. The next variable is about the number of clusters that are calculated during clustering. It's not the exact number of clusters, but allow us count the clusters during clustering. Last variable is the number of points that needs to within a neighbourhood to be classified as non-border object. Boarded points are the points that are the farest points from the cluster, but it's not the noise.

```python
%store -r data_set

assignation = np.zeros(len(data_set))
distance_matrix = np.zeros((len(data_set), len(data_set)))
max_distance = 0.25
number_of_cluster = 0
min_points = 2
```

We need the distance function that we used in previous example to calculate the distance matrix:

```python
def calculate_distance(x,v):
    return sqrt((x[0]-v[0])**2+(x[1]-v[1])**2)
```

To calculate the distance matrix we use the calculate_distance that we used previously:

```python
def calculate_distance_matrix():
    distance_matrix = np.zeros((len(data_set),len(data_set)))
    for i in range(len(data_set)):
        for j in range(len(data_set)):
            distance_matrix[i, j] = calculate_distance(data_set[i], data_set[j])
    return distance_matrix
```

The next step is to get closest elements in the feature space:

```python
def get_closest_elements(distance_matrix, element_id):
    element_distances = distance_matrix[element_id]
    filtered = {}
    iter = 0
    for element in element_distances:
        if element < max_distance:
            filtered[iter] = element
        iter = iter + 1
    return filtered
```

The last step before cluster function is to define funtions that mark the elements in our data set that are known to be a noise or were already visited by our method.

```python
def set_as_noise(assignation,element_id):
    assignation[element_id] = -1
    return assignation

def set_visited(elements, assignation, number_of_clusters):
    for element_id in elements.keys():
        assignation[element_id] = number_of_clusters
    return assignation
```

Combine it all together:

```python
def cluster_density(assignation):
    number_of_cluster = 0
    distance_matrix = calculate_distance_matrix()
    element_ids = list(range(len(data_set)))
    random.shuffle(element_ids)
    for i in element_ids:
        if assignation[i] != 0:
            continue
        closest = get_closest_elements(distance_matrix, i)
        if len(closest) < min_points:
            assignation = set_as_noise(assignation,i)
        else:
            assignation = set_visited(closest, assignation, number_of_cluster)
            number_of_cluster = number_of_cluster + 1
    return assignation
```

Ready to cluster:

```python
new_assignation_density = cluster_density(assignation)
```

The number of cluster is the size of unique cluster ids that are in ``new_assignation_density`` minus noise.

```python
print("Number of clusters: "+ str(len(np.unique(new_assignation_density))-1))
```

    Number of clusters: 2

The noise is marked with -1. The other objects have the cluster number assigned.

```python
print(new_assignation_density)
```

    [ 1.  1.  1.  1.  1.  1.  2.  2. -1. -1.]

```python
%store new_assignation_density
```

    Stored 'new_assignation_density' (ndarray)

```python

```