preamble goes here with instruction run the cell
student name block

## Take Home Part

## Question 1:
- __When it is 8:00 in Lubbock,__
    - __It is 9:00 in New York__
    - __It is 14:00 in London__
    - __It is 15:00 in Cairo__
    - __It is 16:00 in Istanbul__
    - __It is 19:00 in Hyderabad__
    - __It is 22:00 in Tokyo__

__Write a function that reports the time in New York, London, Cairo, Istanbul, Hyderabad, and Tokyo based on the time in Lubbock. Use a 24-hour time format. Include error trapping that:__

1- Issues a message like "Please Enter A Number from 00 to 23" if the first input is numeric but outside the range of [0,23].

2- Takes any numeric input for "Lubbock time" selection , and forces it into an integer.

3- Issues an appropriate message if the user's selection is non-numeric.

__Check your function for these times:__
- 8:00
- 17:00
- 0:00

```python
#Suggested Solution:

def LBBtime():
    try:
        LBK = int(input('What hour is it in Lubbock?- Please enter a number from 0 to 23'))
        if LBK>23:
            print('Please Enter A Number from 00 to 23')
        if LBK<23 and LBK>=0:
            print("Time in New York is",LBK+1,":00")
            print("Time in London is",LBK+6,":00")
            print("Time in Cairo is",LBK+7,":00")
            print("Time in Istanbul is",LBK+8,":00")
            print("Time in Hyderabad is",LBK+11,":00")
            print("Time in Tokyo is",LBK+14,":00")
            return #null return
    except:
        print("Please Enter an Appropriate Input")
```

```python
LBBtime()
```

    Please Enter an Appropriate Input

## Question 2:
Follow the steps below. Add comments to your script and signify when each step and each task is done. *hint: For this problem you will need the numpy and pandas libraries.
- __STEP1: There are 8 digits in your R#. Define a 2x4 array with these 8 digits, name it "Rarray", and print it__
- __STEP2: Find the maximum value of the "Rarray" and its position__
- __STEP3: Sort the "Rarray" along the rows, store it in a new array named "Rarraysort", and print the new array out__
- __STEP4: Define and print a 4x4 array that has the "Rarray" as its two first rows, and "Rarraysort" as its next rows. Name this new array "DoubleRarray"__
- __STEP5: Slice and print a 4x3 array from the "DoubleRarray" that contains the last three columns of it. Name this new array "SliceRarray".__
- __STEP6: Define the "SliceRarray" as a panda dataframe:__
    - name it "Rdataframe",
    - name the rows as "Row A","Row B","Row C", and "Row D"
    - name the columns as "Column 1", "Column 2", and "Column 3"
- __STEP7: Print  the first few rows of the "Rdataframe".__
- __STEP8: Create a new dataframe object ("R2dataframe") by adding a column to the "Rdataframe", name it "Column X" and fill it with "None" values. Then, use the appropriate descriptor function and print the data model (data column count, names, data types) of the "R2dataframe"__
- __STEP9: Replace the **'None'** in the "R2dataframe" with 0. Then, print the summary statistics of each numeric column in the data frame.__
- __STEP10: Define a function based on the equation below, apply on the entire "R2dataframe", store the results in a new dataframe ("R3dataframe"), and print the results and the summary statistics again.__
$$ y = x^2 - 5x +7 $$
- __STEP11: Print the number of occurrences of each unique value in "Column 3"__
- __STEP12: Sort the data frame with respect to "Column 1" with a descending order and print it__
- __STEP13: Write the final format of the "R3dataframe" on a CSV file, named "Rfile.csv"__
- __STEP14: Read the "Rfile.csv" and print its content.__

** __Make sure to attach the "Rfile.csv" file to your midterm exam submission.__

```python
#Suggested Solution:

#Step0:
import numpy as np
import pandas as pd
#Step1:
Rarray = np.array([[1,6,7,4],[5,2,3,8]])     #Define Rarray
print(Rarray)
#Step2:
print(np.max(Rarray))          #Find the maximum Value
print(np.argmax(Rarray))       #Find the posirtion of the maximum value
#Step3:
Rarraysort = np.sort(Rarray,axis = 1) #Sort Rarray along the rows and define a new array
print(Rarraysort)
#Step4:
DoubleRarray = np.array([[1,6,7,4],[5,2,3,8],[1,4,6,7],[2,3,5,8]])     #Define DoubleRarray
print(DoubleRarray)
#Step5:
SliceRarray = DoubleRarray[0:4,1:4]     #Slice DoubleRarray and Define SliceRarray
print(SliceRarray)
#Step6:
myrowname = ["Row A","Row B","Row C","Row D"]
mycolname = ["Column 1", "Column 2","Column 3"]
Rdataframe = pd.DataFrame(SliceRarray,myrowname,mycolname)     #Define Rdataframe
#Step7:
print(Rdataframe.head())     #Print the first few rows of the Rdataframe
#Step8:
Rdataframe['Column X']= None     #Add a new column, "Column X"
R2dataframe = Rdataframe         #Define R2dataframe
print(R2dataframe.info())               #Get the info
#Step9:
R2dataframe = R2dataframe.fillna(0)     #Replace NAs with 0
print(R2dataframe.describe())                 #Get the summary statistics
#Step10:
def myfunc(x):  # A user-built function
    y = (x**2) - (10*x) +7
    return(y)
R3dataframe = R2dataframe.apply(myfunc)     #Apply the function on the entire R2dataframe
print(R3dataframe)
print(R3dataframe.describe())
#Step11:
print(R3dataframe['Column 3'].value_counts())  #Returns the number of occurences of each unique value in Column 3
#Step12:
print(R3dataframe.sort_values('Column 1', ascending = False)) #Sorting based on Column 1
#Step13:
R3dataframe.to_csv('Rfile.csv')     #Write R3dataframe on a CSV file
#Step14:
readfilecsv = pd.read_csv('Rfile.csv') #Read the Rfile.csv
print(readfilecsv)         #Print the contents of the Rfile.csv
```

    [[1 6 7 4]
     [5 2 3 8]]
    8
    7
    [[1 4 6 7]
     [2 3 5 8]]
    [[1 6 7 4]
     [5 2 3 8]
     [1 4 6 7]
     [2 3 5 8]]
    [[6 7 4]
     [2 3 8]
     [4 6 7]
     [3 5 8]]
           Column 1  Column 2  Column 3
    Row A         6         7         4
    Row B         2         3         8
    Row C         4         6         7
    Row D         3         5         8
    <class 'pandas.core.frame.DataFrame'>
    Index: 4 entries, Row A to Row D
    Data columns (total 4 columns):
     #   Column    Non-Null Count  Dtype
    ---  ------    --------------  -----
     0   Column 1  4 non-null      int64
     1   Column 2  4 non-null      int64
     2   Column 3  4 non-null      int64
     3   Column X  0 non-null      object
    dtypes: int64(3), object(1)
    memory usage: 160.0+ bytes
    None
           Column 1  Column 2  Column 3  Column X
    count  4.000000  4.000000  4.000000       4.0
    mean   3.750000  5.250000  6.750000       0.0
    std    1.707825  1.707825  1.892969       0.0
    min    2.000000  3.000000  4.000000       0.0
    25%    2.750000  4.500000  6.250000       0.0
    50%    3.500000  5.500000  7.500000       0.0
    75%    4.500000  6.250000  8.000000       0.0
    max    6.000000  7.000000  8.000000       0.0
           Column 1  Column 2  Column 3  Column X
    Row A       -17       -14       -17         7
    Row B        -9       -14        -9         7
    Row C       -17       -17       -14         7
    Row D       -14       -18        -9         7
            Column 1   Column 2   Column 3  Column X
    count   4.000000   4.000000   4.000000       4.0
    mean  -14.250000 -15.750000 -12.250000       7.0
    std     3.774917   2.061553   3.947573       0.0
    min   -17.000000 -18.000000 -17.000000       7.0
    25%   -17.000000 -17.250000 -14.750000       7.0
    50%   -15.500000 -15.500000 -11.500000       7.0
    75%   -12.750000 -14.000000  -9.000000       7.0
    max    -9.000000 -14.000000  -9.000000       7.0
    -9     2
    -14    1
    -17    1
    Name: Column 3, dtype: int64
           Column 1  Column 2  Column 3  Column X
    Row B        -9       -14        -9         7
    Row D       -14       -18        -9         7
    Row A       -17       -14       -17         7
    Row C       -17       -17       -14         7
      Unnamed: 0  Column 1  Column 2  Column 3  Column X
    0      Row A       -17       -14       -17         7
    1      Row B        -9       -14        -9         7
    2      Row C       -17       -17       -14         7
    3      Row D       -14       -18        -9         7

## Problem 3

Graphing Functions Special Functions

Consider the two functions listed below:

\begin{equation}
f(x) = e^{-\alpha x}
\label{eqn:fofx}
\end{equation}

\begin{equation}
g(x) = \gamma sin(\beta  x)
\label{eqn:gofx}
\end{equation}

Prepare a plot of the two functions on the same graph.

Use the values in Table below for $\alpha$, $\beta$, and $\gamma$.

|Parameter|Value|
|:---|---:|
|$\alpha$|0.50|
|$\beta$|3.00|
|$\gamma$|$\frac{\pi}{2}$|

The plot should have $x$ values ranging from $0$ to $10$ (inclusive) in sufficiently small increments to see curvature in the two functions as well as to identify the number and approximate locations of intersections. In this problem, intersections are locations in the $x-y$ plane where the two "curves" cross one another of the two plots.

```python
# By-hand evaluate f(x) for x=1, alpha = 1/2
```

```python
# By-hand evaluate g(x) for x=3.14, beta = 1/2, gamma = 2
```

```python
# Define the first function f(x,alpha), test the function using your by hand answer
def f(x,alpha):
    import math
    f = math.exp(-1.0*alpha*x)
    return f

f(1,0.5)
```

    0.6065306597126334

```python
# Define the second function g(x,beta,gamma), test the function using your by hand answer
def g(x,beta,gamma):
    import math
    f = gamma*math.sin(beta*x)
    return f

g(3.14,0.5,2.0)
```

    1.9999993658636692

```python
# Built a list for x that ranges from 0 to 10, inclusive, with adjustable step sizes for plotting later on
howMany = 10
scale = 10.0/howMany
xvector = []
for i in range(0,howMany+1):
    xvector.append(scale*i)
#xvector # activate to display
```

```python
# Build a plotting function that plots both functions on the same chart
alpha = 0.5
beta = 0.5
gamma = 2.0
yf = []
yg = []
for i in range(0,howMany+1):
    yf.append(f(xvector[i],alpha))
    yg.append(g(xvector[i],beta,gamma))

def plot2lines(list11,list21,list12,list22,strx,stry,strtitle): # plot list1 on x, list2 on y, xlabel, ylabel, title
    from matplotlib import pyplot as plt # import the plotting library from matplotlibplt.show()
    plt.plot( list11, list21, color ='green', marker ='s', linestyle ='dashdot' , label = "Observed" ) # create a line chart, years on x-axis, gdp on y-axis
    plt.plot( list12, list22, color ='red', marker ='o', linestyle ='solid' , label = "Model") # create a line chart, years on x-axis, gdp on y-axis
    plt.legend()
    plt.title(strtitle)# add a title
    plt.ylabel(stry)# add a label to the x and y-axes
    plt.xlabel(strx)
    plt.show() # display the plot
    return #null return

plot2lines(xvector,yf,xvector,yg,'x-value','y-value','plot of f and g')
```

```python
# Using the plot as a guide, find the values of x where the two curves intercept (i.e. f(x) = g(x))

xguess = float(input('my guess for x')) # ~0.7, and 6.25
alpha = 0.5
beta = 0.5
gamma = 2.0
result = f(xguess,alpha) - g(xguess,beta,gamma)
print('f(x) - g(x) =', result)
```

    my guess for x 6.25

    f(x) - g(x) = 0.010753149164711609

## Bonus Problem 1. Extra Credit (You must complete the regular problems)!
__create a class to compute the average grade (out of 10) of the students based on their grades in Quiz1, Quiz2, the Mid-term, Quiz3, and the Final exam.__

| Student Name  |   Quiz 1   |   Quiz 2   |   Mid-term   |   Quiz 3   |  Final Exam  |
| ------------- | -----------| -----------| -------------| -----------| -------------|
| Harry         | 8          | 9          | 8            | 10         | 9            |
| Ron           | 7          | 8          | 8            | 7          | 9            |
| Hermione      | 10         | 10         | 9            | 10         | 10           |
| Draco         | 8          | 7          | 9            | 8          | 9            |
| Luna          | 9          | 8          | 7            | 6          | 5            |

1. __Use docstrings to describe the purpose of the class.__
2. __Create an object for each car brand and display the output as shown below.__

"Student Name": **Average Grade**

3. __Create and print out a dictionary with the student names as keys and their average grades as data.__

```python
#Suggested Solution:

class Hogwarts:
    """This class calculates the average grade of the students"""
    def __init__(self, Name,Quiz1,Quiz2,MidTerm,Quiz3,Final):
        self.Name = Name
        self.Quiz1 = Quiz1
        self.Quiz2 = Quiz2
        self.MidTerm = MidTerm
        self.Quiz3 = Quiz3
        self.Final= Final

    def average(self):
        return (self.Quiz1 + self.Quiz2 + self.MidTerm + self.Quiz3 + self.Final) /5

S1 = Hogwarts('Harry',8,9,8,10,9)                      #Fill the instances
S2 = Hogwarts('Ron',7,8,8,7,9)
S3 = Hogwarts('Hermione',10,10,9,10,10)
S4 = Hogwarts('Draco',8,7,9,8,9)
S5 = Hogwarts('Luna',9,8,7,6,5)

print("Harry", S1.average())
print("Ron", S2.average())
print("Hermione", S3.average())
print("Draco", S4.average())
print("Luna", S5.average())

GradeDict = {"Harry":S1.average(),"Ron":S2.average(),"Hermione":S3.average(),"Draco":S4.average(),"Luna":S5.average()}
print(GradeDict)
```

    Harry 8.8
    Ron 7.8
    Hermione 9.8
    Draco 8.2
    Luna 7.0
    {'Harry': 8.8, 'Ron': 7.8, 'Hermione': 9.8, 'Draco': 8.2, 'Luna': 7.0}

## Bonus 2 Extra credit (You must complete the regular problems)!
#### Write the VOLUME Function to compute the volume of Cylinders, Spheres, Cones, and Rectangular Boxes. This function should:
- First, ask the user about __the shape of the object__ of interest using this statement:

*"Please choose the shape of the object. Enter 1 for "Cylinder", 2 for "Sphere", 3 for "Cone", or 4 for "Rectangular Box""*

- Second, based on user's choice in the previous step, __ask for the right inputs__.
- Third, print out an statement with __the input values and the calculated volumes__.

#### Include error trapping that:

1. Issues a message that *"The object should be either a Cylinder, a Sphere, a Cone, or a Rectangular Box. Please Enter A Number from 1,2,3, and 4!"* if the first input is non-numeric.
2. Takes any numeric input for the initial selection , and force it into an integer.
4. Issues an appropriate message if the user's selection is numeric but outside the range of [1,4]
3. Takes any numeric input for the shape characteristics , and force it into a float.
4. Issues an appropriate message if the object characteristics are as non-numerics.

#### Test the script for:
1. __Sphere, r=10__
2. __r=10 , Sphere__
3. __Rectangular Box, w=5, h=10, l=0.5__

- __Volume of a Cylinder = πr²h__
- __Volume of a Sphere = 4(πr3)/3__
- __Volume of a Cone = (πr²h)/3__
- __Volume of a Rectangular Box = whl__

```python
#Suggested Solution:

import numpy as np # import NumPy: for large, multi-dimensional arrays and matrices, along with  high-level mathematical functions to operate on these arrays.
pi = np.pi #pi value from the np package

def VOLUME():
    try:
        UI = input('Please choose the shape of the object. Enter 1 for "Cylinder", 2 for "Sphere", 3 for "Cone", or 4 for "Rectangular Box"')
        UI =int(UI)
        if UI==1:
                try:
                    UI2 = input('Please enter the radius of the Cylinder')
                    r= float(UI2)
                    UI3 = input('Please enter the height of the Cylinder')
                    h= float(UI3)
                    V= pi*h*r**2
                    print("The volume of the Cylinder with the radius of ",r," and the height of ",h," is equal to", V)
                except:
                    print("The radius and height of the Cylinder must be numerics. Please Try Again!")
        elif UI==2:
                try:
                    UI2 = input('Please enter the radius of the Sphere')
                    r= float(UI2)
                    V= (4*pi*r**3)/3
                    print("The volume of the Sphere with the radius of ",r," is equal to", V)
                except:
                    print("The radius of the Sphere must be numeric. Please Try Again!")
        elif UI==3:
                try:
                    UI2 = input('Please enter the radius of the Cone')
                    r= float(UI2)
                    UI3 = input('Please enter the height of the Cone')
                    h= float(UI3)
                    V= (pi*h*r**2)/3
                    print("The volume of the Cone with the radius of ",r," and the height of ",h," is equal to", V)
                except:
                    print("The radius and height of the Cone must be numerics. Please Try Again!")
        elif UI==4:
                try:
                    UI2 = input('Please enter the width of the Rectangular Box')
                    w= float(UI2)
                    UI3 = input('Please enter the height of the Rectangular Box')
                    h= float(UI3)
                    UI4 = input('Please enter the length of the Rectangular Box')
                    l= float(UI4)
                    V= w*h*l
                    print("The volume of the Rectangular Box with the width of ",w," and the height of ",h," and the length of ",l," is equal to", V)
                except:
                    print("The width, height, and length of the Rectangular Box must be numerics. Please Try Again!")
        else:
            print("Please Enter A Number from 1,2,3, and 4!")

    except:
        print("The object should be either a Cylinder, a Sphere, a Cone, or a Rectangular Box. Please Enter A Number from 1,2,3, and 4!")

```

```python
VOLUME()
```

    Please choose the shape of the object. Enter 1 for "Cylinder", 2 for "Sphere", 3 for "Cone", or 4 for "Rectangular Box" 1
    Please enter the radius of the Cylinder 3
    Please enter the height of the Cylinder 3

    The volume of the Cylinder with the radius of  3.0  and the height of  3.0  is equal to 84.82300164692441

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```


# Numerical Integration
## CH EN 2450 - Numerical Methods
**Prof. Tony Saad (www.tsaad.net)
Department of Chemical Engineering
University of Utah**

The purpose of numerical integration is to compute the area under a curve or a specified set of data points. The two general applications of numerical integration comprise the integration of (a) complex continuous functions, and (b) discrete data. Many times, the function to be integrated does not admit an antiderivative, such as $f(x) = e^{-x^2}$.

## Newton-Cotes Integration
The general idea behind Newton-Cotes Integration is to approximate the function using a polynomial and then integrating the polynomial.

### Left, Right, and Midpoint Rules
The left, right, and midpoint rules are the simplest of all numerical integration methods. They are based on approximating the function as a constant
\begin{equation}
\int_a^b f(x) \text{d}x \approx \int_a^b C \text{d}x = C\times(b-a)
\end{equation}
The constant can be chosen in several ways, the most common are
1. Left point rule: choose $C = f(a)$
2. Right point rule: choose $C = f(b)$
3. Midpoint rule: choose $C = f(\frac{a+b}{2})$

The three options are shown on the plots below.

```python
import numpy as np
%matplotlib inline
%config InlineBackend.figure_format = 'svg'
import matplotlib.pyplot as plt
```

```python
def myf(x):
    return 1.0/np.sqrt(np.pi) * np.exp(-x**2)
```

```python
xalot = np.linspace(0,2,200)

npts = 5

x = np.linspace(0,2, npts)
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, sharey=True)
fig.set_size_inches((12,3))
a = 0.5
b = 1.0

ax = ax1
ax.plot(xalot, myf(xalot))
ax.set_title('Leftpoint Rule')
# ax1.fill_between([a,b],[myf(a),myf(a)],  edgecolor='k', facecolor='lightgray',linewidth=1)
# plot left Riemann Sum
for i in range(0, len(x)-1):
    a = x[i]
    b = x[i+1]
    ax.fill_between([a,b],[myf(a),myf(a)],  edgecolor='k', facecolor='lightgray',linewidth=1)

ax = ax2
ax.plot(xalot, myf(xalot))
# ax3.fill_between([a,b],[myf( (a+b)/2) ,myf( (a+b)/2)],  edgecolor='k', facecolor='lightgray',linewidth=1)
# plot midpoint rule
ax.set_title('Midpoint Rule')
for i in range(0, len(x)-1):
    a = x[i]
    b = x[i+1]
    h = myf((a+b)/2)
    ax.fill_between([a,b],[h,h],edgecolor='k', facecolor='lightgray',linewidth=1)

ax = ax3
ax.plot(xalot, myf(xalot))
# ax2.fill_between([a,b],[myf(b),myf(b)],  edgecolor='k', facecolor='lightgray',linewidth=1)
# plot right Riemann Sum
ax.set_title('Rightpoint Rule')
for i in range(0, len(x)-1):
    a = x[i]
    b = x[i+1]
    ax.fill_between([a,b],[myf(b),myf(b)],  edgecolor='k', facecolor='lightgray',linewidth=1)

plt.ylim(bottom=0.0)
```

    (0.0, 0.5923990627251441)

Here we define all three rules

```python
def leftpoint(f, a, b, npts):
    '''
    f: Any Python function
    a: Lower integral bound
    b: Upper integral bound
    npts: Number of quadrature points

    Returns the integral of f(x) based on the leftpoint rule
    '''
    x = np.linspace(a,b,npts)
    sum = 0.0
    for i in range(0,len(x)-1):
        a = x[i]
        b = x[i+1]
        sum += (b-a)*(f(a) )
    return sum

def rightpoint(f, a, b, npts):
    '''
    f: Any Python function
    a: Lower integral bound
    b: Upper integral bound
    npts: Number of quadrature points

    Returns the integral of f(x) based on the rightpoint rule
    '''
    x = np.linspace(a,b,npts)
    sum = 0.0
    for i in range(0,len(x)-1):
        a = x[i]
        b = x[i+1]
        sum += (b-a)*(f(b) )
    return sum

def midpoint(f, a, b, npts):
    '''
    f: Any Python function
    a: Lower integral bound
    b: Upper integral bound
    npts: Number of quadrature points

    Returns the integral of f(x) based on the midpoint rule
    '''
    x = np.linspace(a,b,npts)
    sum = 0.0
    for i in range(0,len(x)-1):
        a = x[i]
        b = x[i+1]
        sum += (b-a)*(f( (a+b)/2 ) )
    return sum
```

#### Example

Use the left, right, and midpoints rules to compute the integral
\begin{equation}
\int_0^3 e^{-x^2} \text{d} x
\end{equation}
Compute the absolute relative approximate error for a set of integration points ranging from 2 to 200. Plot the error in all cases.

```python
numpts = np.arange(2,50)
a = 0.0
b = 3.0

lpvals = []
rpvals = []
mpvals = []
for npts in numpts:
    v = leftpoint(myf,a,b,npts)
    lpvals.append(v)
    v = rightpoint(myf,a,b,npts)
    rpvals.append(v)
    v = midpoint(myf,a,b,npts)
    mpvals.append(v)

plt.plot(numpts,lpvals, numpts,rpvals, numpts, mpvals)
plt.grid()
```

Now let's look at how the absolute approximate relative error behaves

```python
lpvals = np.array(lpvals)
e1 =abs( (lpvals[1:] - lpvals[:-1])/lpvals[1:] )
plt.loglog(numpts[1:],e1, label="Leftpoint rule")

rpvals = np.array(rpvals)
e2 =abs( (rpvals[1:] - rpvals[:-1])/rpvals[1:])
plt.loglog(numpts[1:],e2, label="Rightpoint rule" )

mpvals = np.array(mpvals)
e3 =abs( (mpvals[1:] - mpvals[:-1])/mpvals[1:])
plt.loglog(numpts[1:],e3, label="Midpoint rule" )

plt.grid()
plt.legend()
```

    <matplotlib.legend.Legend at 0x11b2990b8>

### Trapezoid Rule

The trapezoidal rule fits a straight line between each two points. It is given by the formula
\begin{equation}
\int_a^b f(x) \text{d} x \approx \frac{b-a}{2}[f(a) + f(b)]
\end{equation}

```python
def myf(x):
    return 1.0/np.sqrt(np.pi) * np.exp(-x**2)

xalot = np.linspace(0,2,200)

npts = 5

x = np.linspace(0,2, npts)

plt.plot(xalot, myf(xalot))
plt.title('Trapezoidal Rule')
# ax1.fill_between([a,b],[myf(a),myf(a)],  edgecolor='k', facecolor='lightgray',linewidth=1)
# plot left Riemann Sum
for i in range(0, len(x)-1):
    a = x[i]
    b = x[i+1]
    plt.fill_between([a,b],[myf(a),myf(b)],  edgecolor='k', facecolor='lightgray',linewidth=1)

plt.ylim(bottom=0.0)
```

    (0.0, 0.5923990627251441)

Implementation of the Trapezoidal rule is straightforward as shown below. In this case, we will implement the trapezoidal rule for both continuous functions and for discrete datasets.

```python
def traprule(x,y):
    '''
    Integrates a discrete set of data using the Trapezoidal rule.

    x: Values of the independent variable
    y: Values of the dependent variable

    Returns the integral of f(x) based on the midpoint rule
    '''

    sum = 0.0
    for i in range(0,len(x)-1):
        sum += 0.5*(x[i+1]-x[i])*(y[i+1] + y[i])
    return sum

def traprulef(f, a, b, npts):
    '''
    Computes the integral of f(x) using the Trapezoidal rule.

    f: Any Python function
    a: Lower integral bound
    b: Upper integral bound
    npts: Number of quadrature points

    Returns the integral of f(x) based on the midpoint rule
    '''
    x = np.linspace(a,b,npts)
    sum = 0.0
    for i in range(0,len(x)-1):
        a = x[i]
        b = x[i+1]
        sum += 0.5*(b-a)*(f(a) + f(b))
    return sum
```

#### Example
Use the Trapezoidal rule to compute the integral
\begin{equation}
\int_0^3 e^{-x^2} \text{d} x
\end{equation}
Compute the absolute relative approximate error for a set of integration points ranging from 2 to 200. Plot the error.

```python
numpts = np.arange(2,30)
a = 0.0
b = 3.0

trapvals = []
for npts in numpts:
    v = traprulef(myf,a,b,npts)
    trapvals.append(v)

plt.plot(numpts,trapvals)
plt.grid()
```

```python
trapvals = np.array(trapvals)
e1 =abs( (trapvals[1:] - trapvals[:-1])/trapvals[1:] )
plt.loglog(numpts[1:],e1, label="Trapezoidal rule")
plt.grid()
```

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

CSS style adapted from https://github.com/barbagroup/CFDPython. Copyright (c) Barba group
<link href='http://fonts.googleapis.com/css?family=Merriweather' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Bitter' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Oxygen' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Lora' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>

```python

```


Differential

1. limit

```python
import sympy as sp
```

```python
x, a, c, h, n, = sp.symbols('x a c h n')
```

```python
fx=(x**2-x-6)/(x-3)
```

```python
sp.limit(fx,x,3)
```

$\displaystyle 5$

```python
sp.limit(fx,x,3,"+")
```

$\displaystyle 5$

```python
sp.limit(fx,x,3,"-")
```

$\displaystyle 5$

2. Derivative

```python
from sympy import Derivative
```

```python
fx = x**2
```

```python
Derivative(fx,x).doit()
```

$\displaystyle 2 x$

```python
Derivative(fx,x).doit().subs({x:2})
```

$\displaystyle 4$

```python
fxr = -x+2
fxl = x**2 - x + 2
```

```python
fx0 = fxl.subs({x:0})
```

```python
fx0
```

$\displaystyle 2$

```python
sp.limit(fxr,x,0,'+')
```

$\displaystyle 2$

```python
sp.limit(fxr,x,0,'-')
```

$\displaystyle 2$

```python
sp.limit((fxr-fx0)/(x-0),x,0,'+')
```

$\displaystyle -1$

```python
sp.limit((fxr-fx0)/(x-0),x,0,'-')
```

$\displaystyle -1$

```python
fx = c
fxh = fx.subs({x:x*h})
sp.limit((fxh-fx)/h,h,0)

```

$\displaystyle 0$

```python
fx = x**3
fxh = fx.subs({x:x*h})
sp.limit((fxh - fx)/h,h,0)
```

$\displaystyle - \infty \operatorname{sign}{\left(x^{3} \right)}$

```python
Derivative(fx,x).doit()
```

$\displaystyle 3 x^{2}$

```python
fx=sp.log(x)
fxh = fx.subs({x:x*h})
sp.limit((fxh-fx)/h,h,0)
```

$\displaystyle -\infty$

```python
Derivative(fx,x).doit()
```

$\displaystyle \frac{1}{x}$

```python
fx = sp.log(x)
fxh = fx.subs({x:x*h})
sp.limit((fxh-fx)/h,h,0)
```

$\displaystyle -\infty$

```python
Derivative(fx,x).doit()
```

$\displaystyle \frac{1}{x}$

```python
fx = sp.log(x, 10)
fxh = fx.subs({x:x*h})
sp.limit((fxh-fx)/h,h,0)
```

$\displaystyle -\infty$

```python
Derivative(fx,x).doit()
```

$\displaystyle \frac{1}{x \log{\left(10 \right)}}$

```python
fx = sp.exp(x)
fxh = fx.subs({x:x*h})
sp.limit((fxh-fx)/h,h,0)
```

$\displaystyle - \infty \operatorname{sign}{\left(e^{x} - 1 \right)}$

```python
Derivative(fx,x).doit()
```

$\displaystyle e^{x}$

```python
fx = a**x
fxh = fx.subs({x:x+h})
sp.limit((fxh-fx)/h,h,0)
```

$\displaystyle a^{x} \log{\left(a \right)}$

```python
Derivative(fx,x).doit()
```

$\displaystyle a^{x} \log{\left(a \right)}$

```python

```


# 15.4. Computing exact probabilities and manipulating random variables

https://ipython-books.github.io/154-computing-exact-probabilities-and-manipulating-random-variables/

### Ref

* SymPy stats at http://docs.sympy.org/latest/modules/stats.html
* Probability lectures on Awesome Math, at https://github.com/rossant/awesome-math/#probability-theory
* Statistics lectures on Awesome Math, at https://github.com/rossant/awesome-math/#statistics

```python
from sympy import *
from sympy.stats import *
init_printing()
```

```python
X, Y = Die('X', 6), Die('Y', 6)
```

```python
P(Eq(X, 3))
```

```python
P(X > 3)
```

```python
P(X > Y)
```

```python
P(X + Y > 6, X < 5)
```

```python
Z = Normal('Z', 0, 1)  # Gaussian variable
```

```python
P(Z > pi)
```

```python
E(Z**2), variance(Z**2)
```

```python
f = density(Z)
```

```python
var('x')
f(x)
```

```python
%matplotlib inline
plot(f(x), (x, -6, 6))
```

```python
Eq(Integral(f(x), (x, pi, oo)),
   simplify(integrate(f(x), (x, pi, oo))))
```

```python

```

```python

```

```python

```


```python
import math
import numpy as np
import matplotlib.pyplot  as plt
from sympy import *
from pylab import mpl

def func(y):
    y = np.float64(y)
    return 1/(1 + y * y)

def draw_pic1(x, y):
    fig=plt.figure()
    plt.plot(x, y, label='插值函数')
    plt.plot(x, func(x), label='原函数')
    plt.legend()
    plt.show()

def draw_pic2(x, y):
    fig=plt.figure()
    plt.plot(x, np.fabs(y-func(x)), label='误差')
    plt.legend()
    plt.show()

def spline3_Parameters(x_vec):
        parameter = []
        size_of_Interval = len(x_vec) - 1;
        i = 1

        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 2] = x_vec[i]
            data[(i - 1) * 4 + 3] = 1
            data1 = np.zeros(size_of_Interval * 4)
            data1[i * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data1[i * 4 + 1] = x_vec[i] * x_vec[i]
            data1[i * 4 + 2] = x_vec[i]
            data1[i * 4 + 3] = 1

            parameter.append(data)
            parameter.append(data1)
            i += 1

        data = np.zeros(size_of_Interval * 4)
        data[0] = x_vec[0] * x_vec[0] * x_vec[0]
        data[1] = x_vec[0] * x_vec[0]
        data[2] = x_vec[0]
        data[3] = 1
        parameter.append(data)

        data = np.zeros(size_of_Interval * 4)
        data[(size_of_Interval - 1) * 4] = x_vec[-1] * x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 1] = x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 2] = x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 3] = 1
        parameter.append(data)

        i = 1
        while i < size_of_Interval:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 3 * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = 2 * x_vec[i]
            data[(i - 1) * 4 + 2] = 1
            data[i * 4] = -3 * x_vec[i] * x_vec[i]
            data[i * 4 + 1] = -2 * x_vec[i]
            data[i * 4 + 2] = -1
            parameter.append(data)
            i += 1

        i = 1
        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 6 * x_vec[i]
            data[(i - 1) * 4 + 1] = 2
            data[i * 4] = -6 * x_vec[i]
            data[i * 4 + 1] = -2
            parameter.append(data)
            i += 1
        #the other two equations
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6 * x_vec[0]
        data[1] = 2
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-4] = 6 * x_vec[-1]
        data[-3] = 2
        parameter.append(data)
        return parameter

def solution_of_equation(parametes, x):
        size_of_Interval = len(x) - 1;
        result = np.zeros(size_of_Interval * 4)
        i = 1
        while i < size_of_Interval:
            result[(i - 1) * 2] = func(x[i])
            result[(i - 1) * 2 + 1] = func(x[i])
            i += 1
        result[(size_of_Interval - 1) * 2] = func(x[0])
        result[(size_of_Interval - 1) * 2 + 1] = func(x[-1])
        result[-2] = 0
        result[-1] = 0
        a = np.array(spline3_Parameters(x))
        b = np.array(result)
        #print(b)
        return np.linalg.solve(a, b)

def calculate(paremeters, x):
        result = []
        for data_x in x:
            result.append(
                paremeters[0] * data_x * data_x * data_x + paremeters[1] * data_x * data_x + paremeters[2] * data_x +
                paremeters[3])
        return result

x_init4 = np.arange(0, 1.01, 0.05)
result = solution_of_equation(spline3_Parameters(x_init4), x_init4)
#print(spline3_Parameters(x_init4))
#print(result)
x_axis4 = []
y_axis4 = []
for i in range(20):
    temp = np.arange(i/20, 0.05 + i/20, 0.01)
    x_axis4 = np.append(x_axis4, temp)
    y_axis4 = np.append(y_axis4, calculate(
        [result[4 * i], result[1 + 4 * i], result[2 + 4 * i], result[3 + 4 * i]], temp))
draw_pic1(x_axis4, y_axis4)
draw_pic2(x_axis4, y_axis4)
print(np.fabs([result[4 * 0] * 0.03**3 + result[1 + 4 * 0] * 0.03**2 +
               result[2 + 4 * 0] * 0.03 + result[3 + 4 * 0]] - func(0.03)))
print(np.fabs([result[4 * 19] * 0.97**3 + result[1 + 4 * 19] * 0.97**2 +
               result[2 + 4 * 19] * 0.97 + result[3 + 4 * 19]] - func(0.97)))
```

```python
import math
import numpy as np
import matplotlib.pyplot  as plt
from sympy import *
from pylab import mpl

def func(y):
    y = np.float64(y)
    return 1/(1 + y * y)

def draw_pic1(x, y):
    fig=plt.figure()
    plt.plot(x, y, label='插值函数')
    plt.plot(x, func(x), label='原函数')
    plt.legend()
    plt.show()

def draw_pic2(x, y):
    fig=plt.figure()
    plt.plot(x, np.fabs(y-func(x)), label='误差')
    plt.legend()
    plt.show()

def spline3_Parameters(x_vec):
        parameter = []
        size_of_Interval = len(x_vec) - 1;
        i = 1

        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 2] = x_vec[i]
            data[(i - 1) * 4 + 3] = 1
            data1 = np.zeros(size_of_Interval * 4)
            data1[i * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data1[i * 4 + 1] = x_vec[i] * x_vec[i]
            data1[i * 4 + 2] = x_vec[i]
            data1[i * 4 + 3] = 1

            parameter.append(data)
            parameter.append(data1)
            i += 1

        data = np.zeros(size_of_Interval * 4)
        data[0] = x_vec[0] * x_vec[0] * x_vec[0]
        data[1] = x_vec[0] * x_vec[0]
        data[2] = x_vec[0]
        data[3] = 1
        parameter.append(data)

        data = np.zeros(size_of_Interval * 4)
        data[(size_of_Interval - 1) * 4] = x_vec[-1] * x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 1] = x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 2] = x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 3] = 1
        parameter.append(data)

        i = 1
        while i < size_of_Interval:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 3 * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = 2 * x_vec[i]
            data[(i - 1) * 4 + 2] = 1
            data[i * 4] = -3 * x_vec[i] * x_vec[i]
            data[i * 4 + 1] = -2 * x_vec[i]
            data[i * 4 + 2] = -1
            parameter.append(data)
            i += 1

        i = 1
        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 6 * x_vec[i]
            data[(i - 1) * 4 + 1] = 2
            data[i * 4] = -6 * x_vec[i]
            data[i * 4 + 1] = -2
            parameter.append(data)
            i += 1
        #the other two equations
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6 * x_vec[0]
        data[1] = 2
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-4] = 6 * x_vec[-1]
        data[-3] = 2
        parameter.append(data)
        return parameter

def solution_of_equation(parametes, x):
        size_of_Interval = len(x) - 1;
        result = np.zeros(size_of_Interval * 4)
        i = 1
        while i < size_of_Interval:
            result[(i - 1) * 2] = func(x[i])
            result[(i - 1) * 2 + 1] = func(x[i])
            i += 1
        result[(size_of_Interval - 1) * 2] = func(x[0])
        result[(size_of_Interval - 1) * 2 + 1] = func(x[-1])
        result[-2] = (-50)
        result[-1] = (-50*26+5000)/26**3
        a = np.array(spline3_Parameters(x))
        b = np.array(result)
        #print(b)
        return np.linalg.solve(a, b)

def calculate(paremeters, x):
        result = []
        for data_x in x:
            result.append(
                paremeters[0] * data_x * data_x * data_x + paremeters[1] * data_x * data_x + paremeters[2] * data_x +
                paremeters[3])
        return result

x_init4 = np.arange(0, 1.01, 0.05)
result = solution_of_equation(spline3_Parameters(x_init4), x_init4)
#print(spline3_Parameters(x_init4))
#print(result)
x_axis4 = []
y_axis4 = []
for i in range(20):
    temp = np.arange(i/20, 0.05 + i/20, 0.01)
    x_axis4 = np.append(x_axis4, temp)
    y_axis4 = np.append(y_axis4, calculate(
        [result[4 * i], result[1 + 4 * i], result[2 + 4 * i], result[3 + 4 * i]], temp))
draw_pic1(x_axis4, y_axis4)
draw_pic2(x_axis4, y_axis4)
print(np.fabs([result[4 * 0] * 0.03**3 + result[1 + 4 * 0] * 0.03**2 +
               result[2 + 4 * 0] * 0.03 + result[3 + 4 * 0]] - func(0.03)))
print(np.fabs([result[4 * 19] * 0.97**3 + result[1 + 4 * 19] * 0.97**2 +
               result[2 + 4 * 19] * 0.97 + result[3 + 4 * 19]] - func(0.97)))
```

```python
import math
import numpy as np
import matplotlib.pyplot  as plt
from sympy import *
from pylab import mpl

def func(y):
    y = np.float64(y)
    return 1/(1 + y * y)

def draw_pic1(x, y):
    fig=plt.figure()
    plt.plot(x, y, label='插值函数')
    plt.plot(x, func(x), label='原函数')
    plt.legend()
    plt.show()

def draw_pic2(x, y):
    fig=plt.figure()
    plt.plot(x, np.fabs(y-func(x)), label='误差')
    plt.legend()
    plt.show()

def spline3_Parameters(x_vec):
        parameter = []
        size_of_Interval = len(x_vec) - 1;
        i = 1

        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 2] = x_vec[i]
            data[(i - 1) * 4 + 3] = 1
            data1 = np.zeros(size_of_Interval * 4)
            data1[i * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data1[i * 4 + 1] = x_vec[i] * x_vec[i]
            data1[i * 4 + 2] = x_vec[i]
            data1[i * 4 + 3] = 1

            parameter.append(data)
            parameter.append(data1)
            i += 1

        data = np.zeros(size_of_Interval * 4)
        data[0] = x_vec[0] * x_vec[0] * x_vec[0]
        data[1] = x_vec[0] * x_vec[0]
        data[2] = x_vec[0]
        data[3] = 1
        parameter.append(data)

        data = np.zeros(size_of_Interval * 4)
        data[(size_of_Interval - 1) * 4] = x_vec[-1] * x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 1] = x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 2] = x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 3] = 1
        parameter.append(data)

        i = 1
        while i < size_of_Interval:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 3 * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = 2 * x_vec[i]
            data[(i - 1) * 4 + 2] = 1
            data[i * 4] = -3 * x_vec[i] * x_vec[i]
            data[i * 4 + 1] = -2 * x_vec[i]
            data[i * 4 + 2] = -1
            parameter.append(data)
            i += 1

        i = 1
        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 6 * x_vec[i]
            data[(i - 1) * 4 + 1] = 2
            data[i * 4] = -6 * x_vec[i]
            data[i * 4 + 1] = -2
            parameter.append(data)
            i += 1
        #the other two equations
        data = np.zeros(size_of_Interval * 4)
        data[0] = 3 * x_vec[0] * x_vec[0]
        data[1] = 2 * x_vec[0]
        data[2] = 1
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-4] = 3 * x_vec[0] * x_vec[0]
        data[-3] = 2 * x_vec[0]
        data[-2] = 1
        parameter.append(data)
        return parameter

def solution_of_equation(parametes, x):
        size_of_Interval = len(x) - 1;
        result = np.zeros(size_of_Interval * 4)
        i = 1
        while i < size_of_Interval:
            result[(i - 1) * 2] = func(x[i])
            result[(i - 1) * 2 + 1] = func(x[i])
            i += 1
        result[(size_of_Interval - 1) * 2] = func(x[0])
        result[(size_of_Interval - 1) * 2 + 1] = func(x[-1])
        result[-2] = 0
        result[-1] = (-50)/26**2
        a = np.array(spline3_Parameters(x))
        b = np.array(result)
        #print(b)
        return np.linalg.solve(a, b)

def calculate(paremeters, x):
        result = []
        for data_x in x:
            result.append(
                paremeters[0] * data_x * data_x * data_x + paremeters[1] * data_x * data_x + paremeters[2] * data_x +
                paremeters[3])
        return result

x_init4 = np.arange(0, 1.01, 0.05)
result = solution_of_equation(spline3_Parameters(x_init4), x_init4)
#print(spline3_Parameters(x_init4))
#print(result)
x_axis4 = []
y_axis4 = []
for i in range(20):
    temp = np.arange(i/20, 0.05 + i/20, 0.01)
    x_axis4 = np.append(x_axis4, temp)
    y_axis4 = np.append(y_axis4, calculate(
        [result[4 * i], result[1 + 4 * i], result[2 + 4 * i], result[3 + 4 * i]], temp))
draw_pic1(x_axis4, y_axis4)
draw_pic2(x_axis4, y_axis4)
print(np.fabs([result[4 * 0] * 0.03**3 + result[1 + 4 * 0] * 0.03**2 +
               result[2 + 4 * 0] * 0.03 + result[3 + 4 * 0]] - func(0.03)))
print(np.fabs([result[4 * 19] * 0.97**3 + result[1 + 4 * 19] * 0.97**2 +
               result[2 + 4 * 19] * 0.97 + result[3 + 4 * 19]] - func(0.97)))
```

```python
import math
import numpy as np
import matplotlib.pyplot  as plt
from sympy import *
from pylab import mpl

def func(y):
    y = np.float64(y)
    return 1/(1 + y * y)

def draw_pic1(x, y):
    fig=plt.figure()
    plt.plot(x, y, label='插值函数')
    plt.plot(x, func(x), label='原函数')
    plt.legend()
    plt.show()

def draw_pic2(x, y):
    fig=plt.figure()
    plt.plot(x, np.fabs(y-func(x)), label='误差')
    plt.legend()
    plt.show()

def spline3_Parameters(x_vec):
        parameter = []
        size_of_Interval = len(x_vec) - 1;
        i = 1

        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 2] = x_vec[i]
            data[(i - 1) * 4 + 3] = 1
            data1 = np.zeros(size_of_Interval * 4)
            data1[i * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data1[i * 4 + 1] = x_vec[i] * x_vec[i]
            data1[i * 4 + 2] = x_vec[i]
            data1[i * 4 + 3] = 1

            parameter.append(data)
            parameter.append(data1)
            i += 1

        data = np.zeros(size_of_Interval * 4)
        data[0] = x_vec[0] * x_vec[0] * x_vec[0]
        data[1] = x_vec[0] * x_vec[0]
        data[2] = x_vec[0]
        data[3] = 1
        parameter.append(data)

        data = np.zeros(size_of_Interval * 4)
        data[(size_of_Interval - 1) * 4] = x_vec[-1] * x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 1] = x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 2] = x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 3] = 1
        parameter.append(data)

        i = 1
        while i < size_of_Interval:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 3 * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = 2 * x_vec[i]
            data[(i - 1) * 4 + 2] = 1
            data[i * 4] = -3 * x_vec[i] * x_vec[i]
            data[i * 4 + 1] = -2 * x_vec[i]
            data[i * 4 + 2] = -1
            parameter.append(data)
            i += 1

        i = 1
        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 6 * x_vec[i]
            data[(i - 1) * 4 + 1] = 2
            data[i * 4] = -6 * x_vec[i]
            data[i * 4 + 1] = -2
            parameter.append(data)
            i += 1
        #the other two equations
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6 * x_vec[0]
        data[1] = 2
        data[4] = 6 * x_vec[1]
        data[5] = 2
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-8] = 6 * x_vec[-2]
        data[-7] = 2
        data[-4] = 6 * x_vec[-1]
        data[-3] = 2
        parameter.append(data)
        return parameter

def solution_of_equation(parametes, x):
        size_of_Interval = len(x) - 1;
        result = np.zeros(size_of_Interval * 4)
        i = 1
        while i < size_of_Interval:
            result[(i - 1) * 2] = func(x[i])
            result[(i - 1) * 2 + 1] = func(x[i])
            i += 1
        result[(size_of_Interval - 1) * 2] = func(x[0])
        result[(size_of_Interval - 1) * 2 + 1] = func(x[-1])
        result[-2] = 0
        result[-1] = 0
        a = np.array(spline3_Parameters(x))
        b = np.array(result)
        #print(b)
        return np.linalg.solve(a, b)

def calculate(paremeters, x):
        result = []
        for data_x in x:
            result.append(
                paremeters[0] * data_x * data_x * data_x + paremeters[1] * data_x * data_x + paremeters[2] * data_x +
                paremeters[3])
        return result

x_init4 = np.arange(0, 1.01, 0.05)
result = solution_of_equation(spline3_Parameters(x_init4), x_init4)
#print(spline3_Parameters(x_init4))
#print(result)
x_axis4 = []
y_axis4 = []
for i in range(20):
    temp = np.arange(i/20, 0.05 + i/20, 0.01)
    x_axis4 = np.append(x_axis4, temp)
    y_axis4 = np.append(y_axis4, calculate(
        [result[4 * i], result[1 + 4 * i], result[2 + 4 * i], result[3 + 4 * i]], temp))
draw_pic1(x_axis4, y_axis4)
draw_pic2(x_axis4, y_axis4)
print(np.fabs([result[4 * 0] * 0.03**3 + result[1 + 4 * 0] * 0.03**2 +
               result[2 + 4 * 0] * 0.03 + result[3 + 4 * 0]] - func(0.03)))
print(np.fabs([result[4 * 19] * 0.97**3 + result[1 + 4 * 19] * 0.97**2 +
               result[2 + 4 * 19] * 0.97 + result[3 + 4 * 19]] - func(0.97)))
```

```python
import math
import numpy as np
import matplotlib.pyplot  as plt
from sympy import *
from pylab import mpl

def func(y):
    y = np.float64(y)
    return 1/(1 + y * y)

def draw_pic1(x, y):
    fig=plt.figure()
    plt.plot(x, y, label='插值函数')
    plt.plot(x, func(x), label='原函数')
    plt.legend()
    plt.show()

def draw_pic2(x, y):
    fig=plt.figure()
    plt.plot(x, np.fabs(y-func(x)), label='误差')
    plt.legend()
    plt.show()

def spline3_Parameters(x_vec):
        parameter = []
        size_of_Interval = len(x_vec) - 1;
        i = 1

        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 2] = x_vec[i]
            data[(i - 1) * 4 + 3] = 1
            data1 = np.zeros(size_of_Interval * 4)
            data1[i * 4] = x_vec[i] * x_vec[i] * x_vec[i]
            data1[i * 4 + 1] = x_vec[i] * x_vec[i]
            data1[i * 4 + 2] = x_vec[i]
            data1[i * 4 + 3] = 1

            parameter.append(data)
            parameter.append(data1)
            i += 1

        data = np.zeros(size_of_Interval * 4)
        data[0] = x_vec[0] * x_vec[0] * x_vec[0]
        data[1] = x_vec[0] * x_vec[0]
        data[2] = x_vec[0]
        data[3] = 1
        parameter.append(data)

        data = np.zeros(size_of_Interval * 4)
        data[(size_of_Interval - 1) * 4] = x_vec[-1] * x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 1] = x_vec[-1] * x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 2] = x_vec[-1]
        data[(size_of_Interval - 1) * 4 + 3] = 1
        parameter.append(data)

        i = 1
        while i < size_of_Interval:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 3 * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = 2 * x_vec[i]
            data[(i - 1) * 4 + 2] = 1
            data[i * 4] = -3 * x_vec[i] * x_vec[i]
            data[i * 4 + 1] = -2 * x_vec[i]
            data[i * 4 + 2] = -1
            parameter.append(data)
            i += 1

        i = 1
        while i < len(x_vec) - 1:
            data = np.zeros(size_of_Interval * 4)
            data[(i - 1) * 4] = 6 * x_vec[i]
            data[(i - 1) * 4 + 1] = 2
            data[i * 4] = -6 * x_vec[i]
            data[i * 4 + 1] = -2
            parameter.append(data)
            i += 1
        #the other two equations
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6
        data[4] = 6
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-8] = 6
        data[-4] = 6
        parameter.append(data)
        return parameter

def solution_of_equation(parametes, x):
        size_of_Interval = len(x) - 1;
        result = np.zeros(size_of_Interval * 4)
        i = 1
        while i < size_of_Interval:
            result[(i - 1) * 2] = func(x[i])
            result[(i - 1) * 2 + 1] = func(x[i])
            i += 1
        result[(size_of_Interval - 1) * 2] = func(x[0])
        result[(size_of_Interval - 1) * 2 + 1] = func(x[-1])
        result[-2] = 0
        result[-1] = 0
        a = np.array(spline3_Parameters(x))
        b = np.array(result)
        #print(b)
        return np.linalg.solve(a, b)

def calculate(paremeters, x):
        result = []
        for data_x in x:
            result.append(
                paremeters[0] * data_x * data_x * data_x + paremeters[1] * data_x * data_x + paremeters[2] * data_x +
                paremeters[3])
        return result

x_init4 = np.arange(0, 1.01, 0.05)
result = solution_of_equation(spline3_Parameters(x_init4), x_init4)
#print(spline3_Parameters(x_init4))
#print(result)
x_axis4 = []
y_axis4 = []
for i in range(20):
    temp = np.arange(i/20, 0.05 + i/20, 0.01)
    x_axis4 = np.append(x_axis4, temp)
    y_axis4 = np.append(y_axis4, calculate(
        [result[4 * i], result[1 + 4 * i], result[2 + 4 * i], result[3 + 4 * i]], temp))
draw_pic1(x_axis4, y_axis4)
draw_pic2(x_axis4, y_axis4)
print(np.fabs([result[4 * 0] * 0.03**3 + result[1 + 4 * 0] * 0.03**2 +
               result[2 + 4 * 0] * 0.03 + result[3 + 4 * 0]] - func(0.03)))
print(np.fabs([result[4 * 19] * 0.97**3 + result[1 + 4 * 19] * 0.97**2 +
               result[2 + 4 * 19] * 0.97 + result[3 + 4 * 19]] - func(0.97)))
```

```python
        #端点处的函数值的二阶导数为原函数的二阶导数
        data = np.zeros(size_of_Interval * 4)
        data[0] = 6 * x_vec[0]
        data[1] = 2
        parameter.append(data)
        data = np.zeros(size_of_Interval * 4)
        data[-4] = 6 * x_vec[-1]
        data[-3] = 2
        parameter.append(data)
        result[-2] = 0
        result[-1] = 0

        data[(i - 1) * 4] = 3 * x_vec[i] * x_vec[i]
            data[(i - 1) * 4 + 1] = 2 * x_vec[i]
            data[(i - 1) * 4 + 2] = 1
```


# Logistic Regression

***

* Formulation

***

Logistic regression (also called **log**istic un**it**, logit) is a statistical model that in its basic form uses a logistic function to model a binary dependent variable, although many more complex extensions exist. The logistic function is a monotonous and defferentiable function used to connect the true label $y$ and predicted value $f(\mathbf{x})$. The most popular logistic function is the so-called sigmoid function $\displaystyle \sigma(\mathbf{z}) = \frac{1}{1+e^{-\mathbf{z}}}$. Then the logistic regression assume

\begin{equation}
\displaystyle y = \frac{1}{1+e^{-f(\mathbf{x})}} = \frac{1}{1+e^{-(\mathbf{w}^T\mathbf{x} + b)}}   \tag{1}
\end{equation}

This can be rewritten as

\begin{equation}
\displaystyle ln \left(\frac{y}{1-y} \right) = \mathbf{w}^T\mathbf{x} + b   \tag{2}
\end{equation}

This means, in Logistic regression, the linear regression results $\mathbf{w}^T\mathbf{x} + b$ is used to approach the log-odds of true label $\displaystyle ln \left(\frac{y}{1-y} \right)$.

* Algorithm

Given features $\mathbf{x}$ and label $y$, how to determine $\mathbf{w}$ and $b$?

We may use the similar cost function as linear regression method $E_{(w, b)} = \sum\limits_{i=1}^m (y_i - f(x_i))^2$. However, note that we use the sigmoid function in logistic regression, at this time, the cost function $E_{(w, b)}$ is not a convex function. That means the results may not easily converge to an optimal point if gradient descent is applied.

An alternative way is to use the maximum likelihood method. Let's view $y$ as the probability of positive cases, and $1-y$ as the probability of positive cases.

\begin{equation}
\displaystyle y = p(y=1 \vert \mathbf{x} ) = p_1 = \frac{e^{\mathbf{w}^T\mathbf{x} + b}}{1 + e^{\mathbf{w}^T\mathbf{x} + b}}   \tag{3}
\end{equation}

\begin{equation}
\displaystyle 1-y = p(y=0 \vert \mathbf{x} ) = p_0 = \frac{1}{1 + e^{\mathbf{w}^T\mathbf{x} + b}}   \tag{4}
\end{equation}

Set $\mathbf{w}^T\mathbf{x} + b = \beta^T \hat{\mathbf{x}}$, where $\beta = (\mathbf{w}, b)$, and $\hat{\mathbf{x}} = (\mathbf{x}; 1)$. Based on Bernoulli Distribution, the likelihood of the logistic regression

\begin{equation}
\displaystyle L(\beta) = \prod_{i=1}^{m} p(y=y_i \vert \mathbf{x}_i; \mathbf{w}, b) = \prod_{i=1}^{m} p_1^{y_i}  +  p_0^{(1-y_i)} \tag{5}
\end{equation}

For easier calculations, the likelihood is updated to the log-likelihood

\begin{equation}
\displaystyle LL(\beta) = \prod_{i=1}^{m} ln p(y=y_i \vert \mathbf{x}_i; \mathbf{w}, b) = \prod_{i=1}^{m}  y_i ln p_1  + (1-y_i) ln p_0 \tag{6}
\end{equation}

The corrsponding cross-entropy or cost function is the invert of log-likelihood

\begin{equation}
\displaystyle J(\beta) = \prod_{i=1}^{m}  - y_i ln p_1  - (1-y_i) ln p_0 \tag{7}
\end{equation}

The optimal solution is then

\begin{equation}
\displaystyle \beta^* = \underset{\mathbf{\beta}}{\arg\min} J(\beta) \tag{8}
\end{equation}

* Hand-on examples

```python
import numpy as np
import pandas as pd
import math as m
import matplotlib.pyplot as plt

def train_test_split(X, Y, train_size, shuffle):
    ''' Perform tran/test datasets splitting '''
    if shuffle:
        randomize = np.arange(len(X))
        np.random.shuffle(randomize)
        X = X[randomize]
        Y = Y[randomize]
    s_id = int(len(Y) * train_size)
    X_train, X_test = X[:s_id], X[s_id:]
    Y_train, Y_test = Y[:s_id], Y[s_id:]

    return X_train, X_test, Y_train, Y_test

def train_test_split_po(X, Y, train_size, shuffle):
    ''' Perform tran/test datasets splitting '''
    if shuffle:
        randomize = np.arange(len(X))
        np.random.shuffle(randomize)
        X = X[randomize]
        Y = Y[randomize]
    s_id = int(len(Y) * train_size)
    X_train, X_test = X[:s_id], X[s_id:]
    Y_train, Y_test = Y[:s_id], Y[s_id:]
    Y_train = Y_train.reshape((-1, 1))
    X_train1 = np.append(X_train, Y_train, axis = 1)
    X_train1 = X_train1[np.argsort(X_train1[:, 0])]
    Y_test = Y_test.reshape((-1, 1))
    X_test1 = np.append(X_test, Y_test, axis = 1)
    X_test1 = X_test1[np.argsort(X_test1[:, 0])]
    X_train, X_test = X_train1[:,:-1], X_test1[:,:-1]
    Y_train, Y_test = X_train1[:,-1], X_test1[:,-1]
    Y_train=np.squeeze(Y_train)
    Y_test=np.squeeze(Y_test)
    return X_train, X_test, Y_train, Y_test

def metric_mse(Y_label, Y_pred):
    ''' Evaluate mean squared error (MSE) '''
    return np.mean(np.power(Y_label - Y_pred, 2))

def metric_rmse(Y_label, Y_pred):
    ''' Evaluate root mean squared error (RMSE) '''
    return m.sqrt(np.mean(np.power(Y_label - Y_pred, 2)))

def metric_accuracy(Y_label, Y_pred):
    '''Evaluate the accuracy'''
    correct_amount = 0
    for i in range(np.size(Y_pred)) :   #np.size: Number of elements in the array
        if Y_label[i] == Y_pred[i] :
            correct_amount = correct_amount + 1
    return correct_amount / np.size(Y_pred) * 100

def readin_csv_data(path):
    df = pd.read_csv(path)
    X = df.iloc[:,:-1].values
    Y = df.iloc[:,-1].values
    return X, Y

def generate_dataset_MVND():
    '''generate a dataset that satisfies the multivariate normal distribution'''
    np.random.seed(24) # Fixing random state for reproducibility
    num_observations = 2000
    x1 = np.random.multivariate_normal([0, 0], [[1, .75],[.75, 1]], num_observations)
    x2 = np.random.multivariate_normal([1, 4], [[1, .75],[.75, 1]], num_observations)
    X = np.vstack((x1, x2)).astype(np.float32)
    Y = np.hstack((np.zeros(num_observations),np.ones(num_observations)))
    return X, Y

def generate_dataset_simple(beta, n, std_dev):
    ''' Generate dataset '''
    X = np.random.rand(n)
    e = np.random.randn(n) * std_dev
    Y = X * beta + e
    X = X.reshape((n,1))
    return X, Y

def generate_dataset_polynomial(beta, n, std_dev):
    ''' Generate polynomial dataset '''

    e = np.random.randn(n) * std_dev/n
    X = np.random.random_sample(n)
    X = np.sort(X)
    Y = 1- 6*X +36*X**2 - 53*X**3 + 22*X**5 + e
    X = X.reshape((n,1))
    return X, Y

def standardization(X,degree):
    """ A scaling technique where the values
    are centered around the mean with
    a unit standard deviation.
    This means that the mean of the attribute
    becomes zero and the resultant distribution
    has a unit standard deviation.
    ----------------------------------------
    degree: polynomial regression degree
    """
    X[:, :(degree)] = (X[:, :(degree)] - np.mean(X[:, :(degree)], axis = 0))/ \
    np.std(X[:, :(degree)], axis = 0)
    return X

def normalization(X,degree):
    """ A scaling technique in which values
    are shifted and rescaled so that they
    end up ranging between 0 and 1.
    It is also known as Min-Max scaling
    ----------------------------------------
    degree: polynomial regression degree, or attribute/feature number
    """
    X[:, :(degree)] = (X[:, :(degree)] - np.amin(X[:, :(degree)], axis = 0))/ \
    (np.amax(X[:, :(degree)], axis = 0) - np.amin(X[:, :(degree)], axis = 0))
    return X

def transformation(m, X, degree):
    tmp = np.zeros([m, 1])
    for j in range(degree + 1):
        if j != 0:
            x_pow = np.power(X, j)
            tmp = np.append(tmp, x_pow.reshape(-1, 1), axis = 1)
    tmp = np.append(tmp, np.ones((m, 1)), axis = 1)
    Xt=tmp[:,1:]
    return Xt

def transformation_predict(m, X, degree):
    tmp = np.zeros([m, 1])
    for j in range(degree + 1):
        if j != 0:
            x_pow = np.power(X, j)
            tmp = np.append(tmp, x_pow.reshape(-1, 1), axis = 1)
    tmp = np.append(tmp, np.ones((m, 1)), axis = 1)
    Xt=tmp[:,1:-1]
    return Xt

def sigmoid(z):
    '''sigmoid( z ) = 1 / ( 1 + e^( - z ) )'''
    return 1/(1 + np.exp(-z))

#def log_likelihood(X, W, Y):
#    return np.sum(- Y*np.dot(X, W) + np.log(1 + np.exp(np.dot(X, W)) )

class LogisticRegression() :
    ''' Logistic Regression model for classification '''

    def __init__(self, iterations, learning_rate):
        self.lr = learning_rate
        self.it = iterations

    def fit(self, X, Y):
        # m instances, d atrributes
        self.m, self.d = X.shape
        # weight initialization
        self.W = np.zeros(self.d+1)
        self.X = X
        self.XX = np.ones((self.m, self.d+1))
        self.XX[:,:-1] = self.X
        self.Y = Y
        for i in range(self.it):
            self.update_weights()
        return self

    def update_weights(self):
        # calculate gradients
        dW = (self.XX.T).dot(sigmoid(self.XX.dot(self.W)) - self.Y)/self.m
        # update weights
        self.W = self.W - self.lr * dW
        return self

    def predict(self, X):
        return np.around(sigmoid(X.dot(self.W))).astype(int)

def main():
    # Import data
    X, Y = generate_dataset_MVND()
    # Splitting dataset into train and test set
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, train_size=.8, shuffle=True)
    # Model Learning
    model = LogisticRegression(learning_rate = 0.02, iterations = 1000)
    model.fit(X_train, Y_train)
    # Model Working
    M, D = X_test.shape
    TEST = np.ones((M, D+1))
    TEST[:,:-1] = X_test
    Y_pred = model.predict(TEST)
    #Statistics
    print( 'Accuracy: ', metric_accuracy(Y_test, Y_pred), '%' )
    # Visualization
    plt.figure(figsize = (12, 8))
    plt.scatter(X_test[:, 0], X_test[:, 1], c = Y_test, marker='o', cmap=plt.cm.jet, alpha = .5, s = 20, label = 'Test Points')
    c = np.ma.masked_where(Y_test == Y_pred, Y_test)
    plt.scatter(X_test[:, 0], X_test[:, 1],c = c, marker='x', cmap=plt.cm.jet, alpha = .8, s = 100, label = 'Wrong Predicted')

    x1 = np.arange(-4, 4, 0.1)
    x2 = -(model.W[2] + model.W[0]*x1)/model.W[1]
    plt.plot(x1, x2, c='k', label='Decision Boundary')
    #plt.legend((p1, p2), ('Test Points', 'Wrong Predicted Points'), scatterpoints=1, loc='upper left',
    #       ncol=3, fontsize=15)
    plt.grid(True)
    plt.legend()
    plt.show()

if __name__ == '__main__':
    main()
```

```python

def main():
    # Import data
    X, Y = readin_csv_data('Logistic_dataset_1.csv')
    X = normalization(X,2)
    # Splitting dataset into train and test set
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, train_size=.6, shuffle=True)
    # Model Learning
    model = LogisticRegression(learning_rate = 0.02, iterations = 1000)
    model.fit(X_train, Y_train)
    # Model Working
    M, D = X_test.shape
    TEST = np.ones((M, D+1))
    TEST[:,:-1] = X_test
    Y_pred = model.predict(TEST)
    #Statistics
    print( 'Accuracy: ', metric_accuracy(Y_test, Y_pred), '%' )
    # Visualization
    plt.figure(figsize = (12, 8))
    X0 = X[np.where(Y == 0)]
    X1 = X[np.where(Y == 1)]
    p1=plt.scatter(X0[:, 0], X0[:, 1], c = 'r', alpha = .5, s = 50, label = 'y = 0')
    p2=plt.scatter(X1[:, 0], X1[:, 1], c = 'b', alpha = .5, s = 50, label = 'y = 1')
    x1 = np.arange(0, 1, 0.1)
    x2 = -(model.W[2] + model.W[0]*x1)/model.W[1]
    p3 = plt.plot(x1, x2, c='k', label = 'Decision Boundary')
    plt.grid(True)
    plt.legend()
    plt.show()

if __name__ == '__main__':
    main()
```

```python

```

```python

```


## Equation Solving
### Importing modules:

```python
from scipy import linalg as la
from scipy import optimize
import sympy
sympy.init_printing()
import numpy as np
import matplotlib.pyplot as plt
```

### Linear Equation Systems:

In general, a linear equation system can be written on the form:

\begin{gather*}
\Large a_{11}x_{1}  + a_{12}x_{2} + ... + a_{1n}x_{n} = b_{1}, \\
\Large a_{21}x_{1}  + a_{22}x_{2} + ... + a_{2n}x_{n} = b_{2}, \\
\newline
\Large a_{m1}x_{1}  + a_{m2}x_{2} + ... + a_{mn}x_{n} = b_{m}
\end{gather*}

### Square Systems:

\begin{gather*}
\Large \frac{ \| \delta x\| }{\|x\|}=\frac{\|A^{-1} \delta b \|}{\|x\|} \leq  \frac{\|A^{-1}\| . \| \delta b\|}{\|x\|}= \frac{\|A^{-1}\| . \|b\|}{\|x\|}. \frac{\|\delta b\|}{\|b\|}  \leq \|A^{-1}\| . \|A\| .  \frac{\|\delta b\|}{\|b\|}
\end{gather*}

#### Example:

\begin{gather*}
\Large 2x_{1} + 3x_{2} = 4 \\
\Large 5x_{1} + 4x_{2} = 3
\end{gather*}

```python
# Using SymPy
A = sympy.Matrix([
    [2, 3],
    [5, 4]
])
b = sympy.Matrix([4, 3])

print(f"Rank of A: {A.rank()}")
print(f"Condition number of A: {A.condition_number()}")
print(f"Norm of A: {A.norm()}")
```

    Rank of A: 2
    Condition number of A: sqrt(2*sqrt(170) + 27)/sqrt(27 - 2*sqrt(170))
    Norm of A: 3*sqrt(6)

```python
# Using Numpy/SciPy
A = np.array([
    [2, 3],
    [5, 4]
])
b = np.array([4, 3])

print(f"Rank of A: {np.linalg.matrix_rank(A)}")
print(f"Condition number of A: {np.linalg.cond(A)}")
print(f"Norm of A: {np.linalg.norm(A)}")
```

    Rank of A: 2
    Condition number of A: 7.582401374401516
    Norm of A: 7.3484692283495345

### LU factorization
#### In SymPy:

```python
A = sympy.Matrix([
    [2, 3],
    [5, 4]
])
b = sympy.Matrix([4, 3])

L, U, _ = A.LUdecomposition()
print(f"L: {L}")
print(f"U: {U}")
print(f"L * U: {L * U}")

x = A.solve(b)  # equivalent to A.LUsolve(b)
print(f"x: {x}")
```

    L: Matrix([[1, 0], [5/2, 1]])
    U: Matrix([[2, 3], [0, -7/2]])
    L * U: Matrix([[2, 3], [5, 4]])
    x: Matrix([[-1], [2]])

```python
A = np.array([
    [2, 3],
    [5, 4]
])
b = np.array([4, 3])

P, L, U = la.lu(A)
print(f"L: \n{L}\n")
print(f"U: \n{U}\n")
print(f"P.dot(L.dot(U)): \n{P.dot(L.dot(U))}\n")
x = la.solve(A, b)
print(f"x: {x}")
```

    L:
    [[1.  0. ]
     [0.4 1. ]]

    U:
    [[5.  4. ]
     [0.  1.4]]

    P.dot(L.dot(U)):
    [[2. 3.]
     [5. 4.]]

    x: [-1.  2.]

### Rectangular Systems:

```python
# defining true model
x = np.linspace(-1, 1, 100)
a, b, c = 1, 2, 3
y_exact = a + b * x + c * x ** 2
```

```python
# simulate noisy data
m = 100
X = 1 - 2 * np.random.randn(m)
Y = a + b * X  + c * x ** 2 + np.random.randn(m)
```

```python
# fit the data to the model using linear least squares
A = np.vstack([X**0, x**1, x**2])
sol, r, rank, sv = la.lstsq(A.T, Y)

y_fit = sol[0] + sol[1] * x + sol[2] * x**2
fig, ax = plt.subplots(figsize=(12, 4))

ax.plot(X, Y, 'go', alpha=0.5, label='Simulated data')
ax.plot(x, y_exact, 'k', lw=2, label='True value $y = 1 + 2x + 3x^2$')
ax.plot(x, y_fit, 'b', lw=2, label='Least square fit')
ax.set_xlabel(r"$x$", fontsize=18)
ax.set_ylabel(r"$y$", fontsize=18)
ax.legend(loc=2)
```

```python
# fit the data to the model using linear least square:
# first order polynomial
A = np.vstack([X**n for n in range(2)])
sol, r , rank, sv = la.lstsq(A.T, Y)
y_fit1 = sum([s * x**n for n, s in enumerate(sol)])

# 15th order polynomial
A = np.vstack([X**n for n in range(16)])
sol, r, rank, sv = la.lstsq(A.T, Y)
y_fit15 = sum([s * x**n for n, s in enumerate(sol)])

fig, ax = plt.subplots(figsize=(12, 4))
ax.plot(X, Y, 'go', alpha=0.5, label='Simulated data')
ax.plot(x, y_exact, 'k', lw=2, label='True value $y = 1 + 2x + 3x^2$')
ax.plot(x, y_fit1, 'b', lw=2, label='Least square fit [1st order]')
ax.plot(x, y_fit15, 'm', lw=2, label='Least square fit [15th order]')
ax.set_xlabel(r"$x$", fontsize=18)
ax.set_ylabel(r"$y$", fontsize=18)
ax.legend(loc=2)
```

### EigenValue Problems:

```python
# Example_01
A = np.array([
    [1, 3, 5],
    [3, 5, 3],
    [5, 3, 9]
])
e_vals, e_vecs = la.eig(A)
print(f"e_vals: {e_vals}\n")
print(f"e_vecs: \n{e_vecs}\n")
print(f"la.eigvalsh: {la.eigvalsh(A)}")
```

    e_vals: [13.35310908+0.j -1.75902942+0.j  3.40592034+0.j]

    e_vecs:
    [[ 0.42663918  0.90353276 -0.04009445]
     [ 0.43751227 -0.24498225 -0.8651975 ]
     [ 0.79155671 -0.35158534  0.49982569]]

    la.eigvalsh: [-1.75902942  3.40592034 13.35310908]

### Non-Linear Equations:
$$a + bx + cx^{2} = 0$$

```python
# Solving using sympy:

x, a, b, c = sympy.symbols("x, a, b, c")
sympy.solve(a + b*x + c*x**2, x)
```

```python
# trigonometric equations:
sympy.solve(a * sympy.cos(x) - b * sympy.sin(x), x)
```

```python
# Example for not solvable equation:
try:
    sympy.solve(sympy.sin(x) - x, x)
except NotImplementedError:
    print("No algorithms are implemented to solve equation")
```

    No algorithms are implemented to solve equation

```python
# Example_02

x = np.linspace(-2, 2, 1000)

# four examples of nonlinear functions
f1 = x**2 - x - 1
f2 = x**3 - 3 * np.sin(x)
f3 = np.exp(x) - 2
f4 = 1 - x**2 + np.sin(50 / (1 + x**2))

# plot each function
fig, axes = plt.subplots(1, 4, figsize=(12, 3), sharey=True)

for n, f in enumerate([f1, f2, f3, f4]):
    axes[n].plot(x, f, lw=1.5)
    axes[n].axhline(0, ls=':', color='k')
    axes[n].set_ylim(-5, 5)
    axes[n].set_xticks([-2, -1, 0, 1, 2])
    axes[n].set_xlabel(r'$x$', fontsize=18)

axes[0].set_ylabel(r'$f(x)$', fontsize=18)

titles = [r'$f(x)=x^2-x-1$', r'$f(x)=x^3-3\sin(x)$',
          r'$f(x)=\exp(x)-2$', r'$f(x)=\sin\left(50/(1+x^2)\right)+1-x^2$']
for n, title in enumerate(titles):
    axes[n].set_title(title)
```

```python
# Example_03
# The bisection method with a graphical visualization of each step:

# define a function, desired tolerance and starting interval [a, b]
f = lambda x: np.exp(x) - 2
tol = 0.1
a, b = -2, 2
x = np.linspace(-2.1, 2.1, 1000)

# graph the function f
fig, ax = plt.subplots(1, 1, figsize=(12, 4))

ax.plot(x, f(x), lw=1.5)
ax.axhline(0, ls=':', color='k')
ax.set_xticks([-2, -1, 0, 1, 2])
ax.set_xlabel(r'$x$', fontsize=18)
ax.set_ylabel(r'$f(x)$', fontsize=18)

# find the root using the bisection method and visualize
# the steps in the method in the graph
fa, fb = f(a), f(b)

ax.plot(a, fa, 'ko')
ax.plot(b, fb, 'ko')
ax.text(a, fa + 0.5, r"$a$", ha='center', fontsize=18)
ax.text(b, fb + 0.5, r"$b$", ha='center', fontsize=18)

n = 1
while b - a > tol:
    m = a + (b - a)/2
    fm = f(m)

    ax.plot(m, fm, 'ko')
    ax.text(m, fm - 0.5, r"$m_%d$" % n, ha='center')
    n += 1

    if np.sign(fa) == np.sign(fm):
        a, fa = m, fm
    else:
        b, fb = m, fm

ax.plot(m, fm, 'r*', markersize=10)
ax.annotate("Root approximately at %.3f" % m,
            fontsize=14, family="serif",
            xy=(a, fm), xycoords='data',
            xytext=(-150, +50), textcoords='offset points',
            arrowprops=dict(arrowstyle="->", connectionstyle="arc3, rad=-.5"))

ax.set_title("Bisection method")
```

```python
# Example_04
# Newton's method
# Using sympy:

# defining a function, desired tolernce and starting point xk
tol = 0.01
xk = 2

s_x = sympy.symbols("x")
s_f = sympy.exp(s_x) - 2

f = lambda x: sympy.lambdify(s_x, s_f, 'numpy')(x)
fp = lambda x: sympy.lambdify(s_x, sympy.diff(s_f, s_x), 'numpy')(x)

x = np.linspace(-1, 2.1, 1000)

# setting up a graph for visualizing the root finding steps
fig, ax = plt.subplots(1, 1, figsize=(12, 4))
ax.plot(x, f(x))
ax.axhline(0, ls=":", color="k")

# iterating newton's method until convergence to the desired tolerance
# has been reached
n = 0
while f(xk) > tol:
    xk_new = xk - f(xk) / fp(xk)
    ax.plot([xk, xk], [0, f(xk)], color='k', ls=':')
    ax.plot(xk, f(xk), 'ko')
    ax.text(xk, -.5, r'$x_%d$' % n, ha='center')
    ax.plot([xk, xk_new], [f(xk), 0], 'k-')

    xk = xk_new
    n += 1

ax.plot(xk, f(xk), 'r*', markersize=15)
ax.annotate("Root approximately at %.3f" % xk,
            fontsize=14, family="serif",
            xy=(xk, f(xk)), xycoords='data',
            xytext=(-150, +50), textcoords='offset points',
            arrowprops=dict(arrowstyle="->",
                             connectionstyle="arc3, rad=-.5"))

ax.set_title("Newtown's method")
ax.set_xticks([-1, 0, 1, 2])
```

```python
# Example_05
# Using SciPy

optimize.bisect(lambda x: np.exp(x) - 2, -2, 2)
```

```python
# Example_06
# finding the root of the equation expx - 2 = 0
# with and without specifying its derivative.

x_root_guess = 2
f = lambda x: np.exp(x) -2
fprime = lambda x:np.exp(x)
print(f"optimize.newton(f, x_root_guess): {optimize.newton(f, x_root_guess)}")
print(f"optimize.newton(f, x_root_guess, fprime=fprime): "
      f"{optimize.newton(f, x_root_guess, fprime=fprime)}")
```

    optimize.newton(f, x_root_guess): 0.6931471805599455
    optimize.newton(f, x_root_guess, fprime=fprime): 0.6931471805599453

```python
# Example_07
# finding the root of the equation expx - 2 = 0
# using optimize.brentq(), optimize.brenth()

print(f"optimize.brentq(lambda x: np.exp(x) - 2, -2, 2): "
      f"{optimize.brentq(lambda x: np.exp(x) - 2, -2, 2)}")

print(f"optimize.brenth(lambda x: np.exp(x) - 2, -2, 2): "
      f"{optimize.brenth(lambda x: np.exp(x) - 2, -2, 2)}")
```

    optimize.brentq(lambda x: np.exp(x) - 2, -2, 2): 0.6931471805599453
    optimize.brenth(lambda x: np.exp(x) - 2, -2, 2): 0.6931471805599381

### Systems of Non-Linear Equations:

$$\begin{cases}y - x^{3} - 2x^{2} + 1 = 0\\y + x^{2} - 1 = 0\end{cases} $$

which can be represented by the vector-valued function:

$$f([x_{1}, x_{2}]) = [x_{2} - x_{1}^{3} - 2x_{1}^{2} + 1, x_{2} - x_{1}^{2} - 1]$$

```python
# Example_08

def f(x):
    return [x[1] - x[0]**3 - 2*x[0]**2 + 1, x[1] + x[0]**2 - 1]

optimize.fsolve(f, [1, 1])
```

    array([0.73205081, 0.46410162])

```python
# Example_09

x, y = sympy.symbols("x, y")
f_mat = sympy.Matrix([y - x**3 -2*x**2 + 1, y + x**2 - 1])
f_mat.jacobian(sympy.Matrix([x, y]))
```

$\displaystyle \left[\begin{matrix}- 3 x^{2} - 4 x & 1\\2 x & 1\end{matrix}\right]$

```python
# Example_10

def f_jacobian(x):
    return [[-3*x[0]**2-4*x[0], 1], [2*x[0], 1]]

optimize.fsolve(f, [1, 1], fprime=f_jacobian)
```

    array([0.73205081, 0.46410162])

```python
# Example_11

x = np.linspace(-3, 2, 5000)
y1 = x**3 + 2 * x**2 -1
y2 = -x**2 + 1

fig, ax = plt.subplots(figsize=(8, 4))

ax.plot(x, y1, 'b', lw=1.5, label=r'$y = x^3 + 2x^2 - 1$')
ax.plot(x, y2, 'g', lw=1.5, label=r'$y = -x^2 + 1$')

x_guesses = [[-2, 2], [1, -1], [-2, -5]]
for x_guess in x_guesses:
    sol = optimize.fsolve(f, x_guess)
    ax.plot(sol[0], sol[1], 'r*', markersize=15)

    ax.plot(x_guess[0], x_guess[1], 'ko')
    ax.annotate("", xy=(sol[0], sol[1]), xytext=(x_guess[0], x_guess[1]),
                arrowprops=dict(arrowstyle="->", linewidth=2.5))

ax.legend(loc=0)
ax.set_xlabel(r'$x$', fontsize=18)
fig.tight_layout()
```

```python
# Example_12
# build a visualization of how different initial
# guesses converge to different solutions:

fig, ax = plt.subplots(figsize=(8, 4))

ax.plot(x, y1, 'k', lw=1.5, label=r'$y = x^3 + 2x^2 - 1$')
ax.plot(x, y2, 'k', lw=1.5, label=r'$y = -x^2 + 1$')

sol1 = optimize.fsolve(f, [-2,  2])
sol2 = optimize.fsolve(f, [ 1, -1])
sol3 = optimize.fsolve(f, [-2, -5])

colors = ['r', 'b', 'g']
for idx, s in enumerate([sol1, sol2, sol3]):
    ax.plot(s[0], s[1], colors[idx]+'*', markersize=15)

for m in np.linspace(-4, 3, 80):
    for n in np.linspace(-15, 15, 40):
        x_guess = [m, n]
        sol = optimize.fsolve(f, x_guess)

        for idx, s in enumerate([sol1, sol2, sol3]):
            if abs(s-sol).max() < 1e-8:
                # ax.plot(sol[0], sol[1], colors[idx]+'*', markersize=15)
                ax.plot(x_guess[0], x_guess[1], colors[idx]+'.')

ax.set_xlabel(r'$x$', fontsize=18)
fig.tight_layout()

```


# Vector Laplacian in curvilinear coordinates

The vector Laplacian is

$$
\nabla^2 \vec{u} = \nabla \cdot \nabla \vec{u}
$$

A vector identity gives the vector Laplacian as

$$
\nabla^2 \vec{u} = \nabla \nabla \cdot \vec{u} - \nabla \times \nabla \times \vec{u}
$$

We will check if this identity holds for shenfun using both cylindrical and spherical coordinates.

For reference, the vector Laplacian is given [here](https://en.wikipedia.org/wiki/Del_in_cylindrical_and_spherical_coordinates)

Cylinder coordinates are mapped to Cartesian through

$$
\begin{align*}
x &= r \cos \theta \\
y &= r \sin \theta \\
z &= z
\end{align*}
$$

and we use a domain $(r, \theta, z) \in [0, 1] \times [0, 2 \pi] \times [0, 2 \pi]$.

Spherical coordinates are mapped as

$$
\begin{align*}
x &= r \sin(\theta) \cos(\phi)\\
y &= r \sin(\theta) \sin(\phi)\\
z &= r \cos(\theta)
\end{align*}
$$

for a domain $(r, \theta, \phi) \in [0, 1] \times [0, \pi] \times [0, 2 \pi]$.

This is all we need to know for using these coordinate systems with shenfun.

# Cylinder coordinates

```python
from shenfun import *
from IPython.display import Math
import sympy as sp

r, theta, z = psi = sp.symbols('x,y,z', real=True, positive=True)
rv = (r*sp.cos(theta), r*sp.sin(theta), z)

N = 10
F0 = FunctionSpace(N, 'F', dtype='d')
F1 = FunctionSpace(N, 'F', dtype='D')
L = FunctionSpace(N, 'L', domain=(0, 1))
T = TensorProductSpace(comm, (L, F1, F0), coordinates=(psi, rv))
V = VectorSpace(T)
u = TrialFunction(V)
```

```python
du = div(u)
Math(du.tolatex(funcname='u', symbol_names={r: 'r', theta: '\\theta', z: 'z'}))
```

```python
du.tosympy(basis=(r*sp.cos(theta), sp.sin(theta), z), psi=psi)
```

The vector Laplacian can now be found as

```python
du = div(grad(u))
```

We can look at `du` using the following

```python
Math((du).tolatex(funcname='u', symbol_names={r: 'r', theta: '\\theta', z: 'z'}))
```

Note that the basis vectors $\mathbf{b}_i$ are not unit vectors (i.e., of length 1). For this reason the equation does not look exactly like the one [here](https://en.wikipedia.org/wiki/Del_in_cylindrical_and_spherical_coordinates). The basis vectors are

```python
Math(T.coors.latex_basis_vectors(covariant=True, symbol_names={r: 'r', theta: '\\theta', z: 'z'}))
```

Notice that $|\mathbf{b}_{\theta}|=r$. Shenfun uses non-normalized covariant basis vectors for describing all vectors and higher order tensors. The vector components are contraviariant and as such use a superscript $u^{\theta}$ and not subscript $u_{\theta}$. Note that for orthogonal coordinates the scaled unit vectors are the same for either contra- or covariant basis vectors and as such this distinction is not necessary here. The distinction is only required for non-orthogonal coordinate systems. Shenfun can handle both orthogonal and non-orthogonal coordinates, but requires that equations to be solved are separable.

Now check the vector identity

$$
\nabla^2 \vec{u} = \nabla \nabla \cdot \vec{u} - \nabla \times \nabla \times \vec{u}
$$

```python
dv = grad(div(u)) - curl(curl(u))
dv.simplify()
Math((dv).tolatex(funcname='u', symbol_names={r: 'r', theta: '\\theta', z: 'z'}))
```

We see that the order is different, but the vector is actually identical to the previous one (du). To show that they are equal we can subtract one from the other and simplify.

```python
dw = du-dv
dw.simplify()
Math(dw.tolatex(funcname='u', symbol_names={r: 'r', theta: '\\theta', z: 'z'}))
```

If you are not convinced we can assemble some matrices and check that `du` and `dv` behave the same way.

```python
v = TestFunction(V)
A0 = inner(v, du)
A1 = inner(v, dv)
```

`A0` and `A1` now contains lists of tensor product matrices, because the vector identities contain a lot of different terms (as we have seen above). To check that `A0` and `A1` are identical, we test the vector product of the matrices with a random vector. Since we are working with vectors we use a `BlockMatrix` for the combined tensor product matrices.

```python
u_hat = Function(V)
u_hat[:] = np.random.random(u_hat.shape) + np.random.random(u_hat.shape)*1j
a0 = BlockMatrix(A0)
a1 = BlockMatrix(A1)
b0 = Function(V)
b1 = Function(V)
b0 = a0.matvec(u_hat, b0)
b1 = a1.matvec(u_hat, b1)
print('Error ', np.linalg.norm(b0-b1))
```

# Spherical coordinates

We now turn to spherical coordinates and run the same test.

```python
r, theta, phi = psi = sp.symbols('x,y,z', real=True, positive=True)
rv = (r*sp.sin(theta)*sp.cos(phi), r*sp.sin(theta)*sp.sin(phi), r*sp.cos(theta))
N = 6
F = FunctionSpace(N, 'F', dtype='d')
L0 = FunctionSpace(N, 'L', domain=(0, 1))
L1 = FunctionSpace(N, 'L', domain=(0, np.pi))
T = TensorProductSpace(comm, (L0, L1, F), coordinates=(psi, rv, sp.Q.positive(sp.sin(theta))))
V = VectorSpace(T)
u = TrialFunction(V)
du = div(grad(u))
dv = grad(div(u)) - curl(curl(u))
dv.simplify()
dw = du-dv
dw.simplify()
```

```python
Math(dw.tolatex(funcname='u', symbol_names={r: 'r', theta: '\\theta', phi: '\\phi'}))
```

This proves that for shenfun the vector identity $\nabla^2 \vec{u} = \nabla \nabla \cdot \vec{u} - \nabla \times \nabla \times \vec{u}$ holds true also for spherical coordinates.

```python

```


# 1.3.2 Constructing Procedure Using `Lambda`

In general, `lambda` is used to create procedures in the same way as `define`, except that no name is specified foir the procedure.

```scheme
(define plus4 (lambda (x) (+ x 4)))
```

```scheme
(plus4 3)
```

    7

## Using `let` to create local variables

```scheme
(define f (+ 2 3))
```

```scheme
f
```

    5

```scheme
(define (g) (+ 2 3))
```

```scheme
(g)
```

    5

$f(x, y) = x(1 + xy)^2 + y(1 - y) + (1 + xy)(1 - y)$

$$
\begin{align}
a &= 1 + xy \\
b &= 1 - y \\
f(x, y) &= xa^2 + yb + ab
\end{align}
$$

```scheme
(define (square x) (* x x))
```

```scheme
(define (f x y)
  (let
    ((a (+ 1 (* x y)))
     (b (- 1 y)))
    (+ (* x (square a))
       (* y b)
       (* a b))))
```

```scheme
(f 2 4)
```

    123

> A `let` expression is simply syntatic sugar for the underlying `lambda` expression.

```scheme

```


# Predavanje 5

## Sympy

```python
import sympy as sp
sp.init_printing()
```

### Računanje s brojevima

```python
x = sp.sqrt(8)
x
```

```python
sp.srepr(x)
```

    'Mul(Integer(2), Pow(Integer(2), Rational(1, 2)))'

```python
print(x)
```

    2*sqrt(2)

```python
x.func, x.args
```

    (sympy.core.mul.Mul, (2, sqrt(2)))

```python
x.args[1].args[0].func
```

    sympy.core.numbers.Integer

```python
sp.srepr(sp.sqrt(2) - sp.sqrt(3))
```

    'Add(Mul(Integer(-1), Pow(Integer(3), Rational(1, 2))), Pow(Integer(2), Rational(1, 2)))'

```python
- sp.sqrt(3) + sp.sqrt(2) == sp.sqrt(2) - sp.sqrt(3)
```

    True

```python
x.n(50)
```

```python
x**2
```

### Simboli

```python
x = sp.Symbol('x')
```

```python
x, y = sp.symbols('x, y')
```

```python
izraz = x + 2*y
```

```python
sp.srepr(izraz)
```

    "Add(Symbol('x'), Mul(Integer(2), Symbol('y')))"

```python
izraz.args[1].args
```

```python
izraz.subs({x: sp.E, y: y-3})
```

```python
sp.srepr(izraz.subs({x: sp.E, y: y-3}))
```

    "Add(Mul(Integer(2), Symbol('y')), Integer(-6), E)"

```python
izraz.n(50, subs={x:5, y:sp.pi})
```

### Analiza

```python
sp.Derivative(sp.sin(x+y) * sp.E**y, y, 2, x, y)
```

```python
der = sp.Derivative(sp.sin(x+y) * sp.E**y, x, y, 3)
der
```

```python
der = der.doit()
der
```

```python
intg = sp.Integral(der, y, y, y, x)
intg
```

```python
%time intg.doit()
```

```python
sp.Integral(sp.sin(x**2), (x, 0, sp.oo)).doit()
```

```python
lim = sp.Limit((sp.cos(x) - 1) / x**2, x, 0, '-')
lim
```

```python
lim.doit()
```

```python
sp.Limit(3**x / 2**x, x, sp.oo).doit()
```

```python
red = sp.Sum(x**-2, (x, 4, sp.oo))
red
```

```python
konačna_suma = sp.Sum(x**-2, (x, 4, 10))
sp.Eq(konačna_suma, konačna_suma.doit())
```

```python
čvor = sp.Add(1, 1, evaluate=False)
sp.Eq(čvor, čvor.doit(), evaluate=False)
```

```python
red.doit()
```

```python
sp.Integral(sp.sqrt(x**2 - 1)).doit()
```

### Rješavanje jednadžbi

```python
jednadžba = sp.Eq(x**3 + 8)
jednadžba
```

```python
sp.solve(jednadžba)
```

```python
sp.solve(sp.Eq(x**2 + x + 1, 0), x)
```

```python
sp.solve([sp.Eq(x**2 + y**2, 10), sp.Eq(x - y, 1)], [x, y], dict=True)
```

```python
y, t = sp.Function('y'), sp.Symbol('t')
```

```python
difj = sp.Eq(sp.Derivative(y(t), t, 2) - y(t), sp.E ** t)
difj
```

```python
sp.classify_ode(difj, y(t))
```

    ('nth_linear_constant_coeff_undetermined_coefficients',
     'nth_linear_constant_coeff_variation_of_parameters',
     'nth_linear_constant_coeff_variation_of_parameters_Integral')

```python
sp.dsolve(difj, y(t))
```

### Linearna algebra

```python
sp.Determinant(sp.Matrix([[1,2,3], [4,5,6], [1,3,4]])**-1).doit()
```

```python
M = sp.Matrix([[1, 2, 3], [2, 2, 2], [1, 2, 3]])
```

```python
M.eigenvals()
```

```python
M.eigenvects()
```

### Razne manipulacije s izrazima

```python
print(sp.latex(intg))
```

    \iiiint \left(- 2 \left(\sin{\left (x + y \right )} + \cos{\left (x + y \right )}\right) e^{y}\right)\, dy\, dy\, dy\, dx

```python
[sp.cos(x).rewrite(funkcija) for funkcija in (sp.sin, sp.tan, sp.csc, sp.exp, sp.acos)]
```

```python
with sp.assuming(sp.Q.real(x), sp.Q.nonzero(x)):
    print(sp.ask(sp.Q.positive(x ** 2)))
```

    True

```python
with sp.assuming(sp.Q.integer(x)):
    print(sp.ask(sp.Q.prime(4 * x)))
```

    False

```python
(x**2 + 2*x + 1).equals((x+1)**2)
```

    True

```python
x + 1 / 3
```

```python
x + sp.S(1) / 3
```

```python
izraz = 0
for ponavljanje in range(5):
    izraz = x ** izraz
izraz
```

```python
izraz = x
for ponavljanje in range(3):
    izraz = izraz.subs(x, x**x)
izraz
```

```python
izraz = sp.sin(2*x) + sp.cos(2*x)
```

```python
izraz.expand(trig=True)
```

```python
def parcijalna_zamjena(izraz, podizraz, transformacija):
    return izraz.subs(podizraz, transformacija(podizraz))
```

```python
parcijalna_zamjena(izraz, sp.sin(2*x), sp.expand_trig)
```

```python
x, y = sp.symbols('x, y')
polinom = x**4 - 4*x**3 + 4*x**2 - 2*x + 3
zamjene = {x**i : y**i for i in range(5) if not i % 2}
zamjene
```

```python
polinom.subs(zamjene)
```

```python
sp.S('x^2+y^3-1/3+pi+alpha')
```

```python
from sympy.printing.dot import dotprint
```

```python
from sympy.abc import a, b, c, x
print(dotprint(sp.solve(a * x**2 + b * x + c, x)[~0]))
```

    digraph{

    # Graph style
    "ordering"="out"
    "rankdir"="TD"

    #########
    # Nodes #
    #########

    "Mul(Rational(-1, 2), Pow(Symbol(a), NegativeOne()), Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())))_()" ["color"="black", "label"="Mul", "shape"="ellipse"];
    "Rational(-1, 2)_(0,)" ["color"="black", "label"="-1/2", "shape"="ellipse"];
    "Pow(Symbol(a), NegativeOne())_(1,)" ["color"="black", "label"="Pow", "shape"="ellipse"];
    "Symbol(a)_(1, 0)" ["color"="black", "label"="a", "shape"="ellipse"];
    "NegativeOne()_(1, 1)" ["color"="black", "label"="-1", "shape"="ellipse"];
    "Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half()))_(2,)" ["color"="black", "label"="Add", "shape"="ellipse"];
    "Symbol(b)_(2, 0)" ["color"="black", "label"="b", "shape"="ellipse"];
    "Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())_(2, 1)" ["color"="black", "label"="Pow", "shape"="ellipse"];
    "Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c)))_(2, 1, 0)" ["color"="black", "label"="Add", "shape"="ellipse"];
    "Pow(Symbol(b), Integer(2))_(2, 1, 0, 0)" ["color"="black", "label"="Pow", "shape"="ellipse"];
    "Symbol(b)_(2, 1, 0, 0, 0)" ["color"="black", "label"="b", "shape"="ellipse"];
    "Integer(2)_(2, 1, 0, 0, 1)" ["color"="black", "label"="2", "shape"="ellipse"];
    "Mul(Integer(-4), Symbol(a), Symbol(c))_(2, 1, 0, 1)" ["color"="black", "label"="Mul", "shape"="ellipse"];
    "Integer(-4)_(2, 1, 0, 1, 0)" ["color"="black", "label"="-4", "shape"="ellipse"];
    "Symbol(a)_(2, 1, 0, 1, 1)" ["color"="black", "label"="a", "shape"="ellipse"];
    "Symbol(c)_(2, 1, 0, 1, 2)" ["color"="black", "label"="c", "shape"="ellipse"];
    "Half()_(2, 1, 1)" ["color"="black", "label"="1/2", "shape"="ellipse"];

    #########
    # Edges #
    #########

    "Mul(Rational(-1, 2), Pow(Symbol(a), NegativeOne()), Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())))_()" -> "Rational(-1, 2)_(0,)";
    "Mul(Rational(-1, 2), Pow(Symbol(a), NegativeOne()), Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())))_()" -> "Pow(Symbol(a), NegativeOne())_(1,)";
    "Mul(Rational(-1, 2), Pow(Symbol(a), NegativeOne()), Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())))_()" -> "Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half()))_(2,)";
    "Pow(Symbol(a), NegativeOne())_(1,)" -> "Symbol(a)_(1, 0)";
    "Pow(Symbol(a), NegativeOne())_(1,)" -> "NegativeOne()_(1, 1)";
    "Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half()))_(2,)" -> "Symbol(b)_(2, 0)";
    "Add(Symbol(b), Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half()))_(2,)" -> "Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())_(2, 1)";
    "Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())_(2, 1)" -> "Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c)))_(2, 1, 0)";
    "Pow(Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c))), Half())_(2, 1)" -> "Half()_(2, 1, 1)";
    "Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c)))_(2, 1, 0)" -> "Pow(Symbol(b), Integer(2))_(2, 1, 0, 0)";
    "Add(Pow(Symbol(b), Integer(2)), Mul(Integer(-4), Symbol(a), Symbol(c)))_(2, 1, 0)" -> "Mul(Integer(-4), Symbol(a), Symbol(c))_(2, 1, 0, 1)";
    "Pow(Symbol(b), Integer(2))_(2, 1, 0, 0)" -> "Symbol(b)_(2, 1, 0, 0, 0)";
    "Pow(Symbol(b), Integer(2))_(2, 1, 0, 0)" -> "Integer(2)_(2, 1, 0, 0, 1)";
    "Mul(Integer(-4), Symbol(a), Symbol(c))_(2, 1, 0, 1)" -> "Integer(-4)_(2, 1, 0, 1, 0)";
    "Mul(Integer(-4), Symbol(a), Symbol(c))_(2, 1, 0, 1)" -> "Symbol(a)_(2, 1, 0, 1, 1)";
    "Mul(Integer(-4), Symbol(a), Symbol(c))_(2, 1, 0, 1)" -> "Symbol(c)_(2, 1, 0, 1, 2)";
    }

```python
from IPython.display import SVG
```

```python
SVG('izraz.svg')
```

```python
(sp.gamma(x + 2) / sp.gamma(x)).simplify()
```

```python
(x ** 2 + 2 * x + 1).simplify(), (x ** 2 + 2 * x + 1).factor()
```

```python
((x + 1) * (x - 2) - (x + 3) * (x - 1)).expand()
```

```python
from sympy.abc import z
(x**2*z + 4*x*y*z + 4*y**2*z).factor()
```

```python
(sp.cos(x)**2 + sp.sin(x)**2 - 2*sp.sin(x)*sp.cos(x)).factor()
```

```python
izraz = x*y + x - 3 + 2*x**2 - z*x**2 + x**3
```

```python
izraz.coeff(x, 2)
```

```python
izraz = 1 / x + (3*x/2 - 2) / (x - 4)
izraz
```

```python
izraz.together()
```

```python
izraz.expand()
```

```python
izraz.cancel()
```

```python
izraz.apart()
```

```python
izraz.together().expand().together()
```

```python
izraz.simplify()
```

```python
izraz = (x*y**2 - 2*x*y*z + x*z**2 + y**2 - 2*y*z + z**2)/(x**2 - 1)
```

```python
izraz, izraz.cancel(), izraz.factor()
```

```python
((4*x**3 + 21*x**2 + 10*x + 12)/(x**4 + 5*x**3 + 5*x**2 + 4*x)).apart()
```

```python
(sp.acos(sp.cos(7)) - (7 - 2*sp.pi)).n()
```

```python
sp.trigsimp(sp.acos(sp.cos(7)))
```

```python
sp.acos(sp.cos(7)).simplify()  # Krivo!
```

```python
sp.trigsimp(sp.sinh(x)**4 - sp.cosh(x)**4)
```

```python
sp.expand_trig(sp.cos(1 - x))
```

```python
sp.trigsimp(_)
```

```python
from sympy.abc import a, b, c, x, y, z
```

```python
sp.powsimp(x**a * x**b)
```

```python
sp.powsimp(2**a * 3**a)
```

```python
ar, br, cr, xr, yr, zr = sp.symbols('a b c x y z', real=True)
```

```python
ap, bp, cp, xp, yp, zp = sp.symbols('a b c x y z', nonnegative=True)
```

```python
sp.powsimp(xp ** ar * yp ** ar)
```

```python
sp.powsimp(x ** a * y ** a, force=True)
```

```python
sp.powsimp(x ** 2 * y ** 2)
```

```python
2*(x+y), (x*y)**2
```

```python
sp.expand_power_exp(x ** (a + b))
```

```python
sp.expand_power_base((xp * yp) ** ar)
```

```python
sp.powdenest((xp**ar)**br)
```

```python
bi = sp.Symbol('b', integer=True)
```

```python
sp.powdenest((x**a)**bi)
```

```python
from sympy.abc import x, y
xp, yp = sp.symbols('x y', positive=True)
sp.expand_log(sp.log(xp * yp))
```

```python
sp.exp(sp.log(x))
```

```python
sp.log(sp.exp(x))
```

```python
sp.expand_log(sp.log(xp ** ar))
```

```python
sp.logcombine(2 * sp.log(xp))
```

```python
sp.Eq(sp.factorial(x), sp.gamma(x + 1))
```

```python
_.simplify()
```

```python
sp.Integral(t**z * sp.E**-t, (t, 0, sp.oo)).doit().rewrite(sp.factorial)
```

```python
from sympy.abc import n, k
sp.binomial(n, k)
```

```python
sp.binomial(n, k).rewrite(sp.factorial)
```

```python
sp.combsimp(sp.binomial(n, n-2))
```

```python
sp.combsimp(sp.factorial(x) * sp.factorial(-x))
```

```python
sp.series(sp.tan(x), x, sp.pi / 2, 9)
```

```python
sp.solveset(sp.cos(x), x)
```

```python
sp.solveset(x**2 - x + 1, x)
```

```python
sp.solveset(sp.Eq(x - y, x + y + 2), x)
```

```python
sp.solveset(sp.sin(x) <= 0, x, sp.S.Reals)  # Krivo!
```

```python
sp.srepr(_)
```

    'Union(FiniteSet(Integer(0)), Interval(pi, Mul(Integer(2), pi), S.false, S.false))'

```python
M = sp.Matrix([[2, 3], [4, 5], [7, 1]])
```

```python
M.col(0)
```

```python
sp.Matrix([1, 9])
```

```python
M * _
```

```python
(M.T * M) ** n
```

```python
sp.eye(5)
```

```python
sp.zeros(3, 7)
```

```python
(sp.ones(3, 3) + sp.eye(3)) ** -1
```

```python
n = sp.Symbol('n', positive=True)
sp.simplify(((M * M.T) ** n).subs({n: 4}).expand())
```

```python
sp.Matrix([[1, 0, 1, 3], [2, 3, 4, 7], [-1, -3, -3, -4]]).rref()
```

```python

```


```julia
import Pkg; Pkg.add(Pkg.PackageSpec(url="https://github.com/JuliaComputing/JuliaAcademyData.jl"))
using JuliaAcademyData; activate("Foundations of machine learning")
```

# Modeling data 2

## Building a model

Recall that in notebook 3, we saw that we could use a mathematical function to classify an image as an apple or a banana, based on the average amount of green in an image:

A common function for performing this kind of **classification** is the sigmoid that we saw in the last notebook, and that we will now extend by adding two **parameters**, $w$ and $b$:

$$\sigma(x; w, b) := \frac{1}{1 + \exp(-wx + b)}$$

$$ x = \mathrm{data} $$

\begin{align}
\sigma(x;w,b) &\approx 0 \implies \mathrm{apple} \\
\sigma(x;w,b) &\approx 1 \implies \mathrm{banana}
\end{align}

In our mathematical notation above, the `;` in the function differentiates between the **data** and the **parameters**. `x` is the data and is determined from the image. The parameters, `w` and `b`, are numbers which we choose to make our function match the results it should be modeling.

Note that in the code below, we don't distinguish between data and parameters - both are just inputs to our function, σ!

```julia
using Images, Statistics

apple = load(datapath("data/10_100.jpg"))
banana = load(datapath("data/104_100.jpg"))

apple_green_amount = mean(Float64.(green.(apple)))
banana_green_amount = mean(Float64.(green.(banana)))

println("Average green for apple = $apple_green_amount")
println("Average green for banana = $banana_green_amount")
```

```julia
σ(x, w, b) = 1 / (1 + exp(-w * x + b))
```

What we want is that when we give σ as input the average green for the apple, roughly `x = 0.3385`, it should return as output something close to 0, meaning "apple". And when we give σ the input `x = 0.8808`, it should output something close to 1, meaning "banana".

By changing the parameters of the function, we can change the shape of the function, and hence make it represent, or **fit**, the data better!

## Data fitting by varying parameters

We can understand how our choice of `w` and `b` affects our model by seeing how our values for `w` and `b` change the plot of the $\sigma$ function.

```julia
using Plots; gr()   # GR works better for interactive manipulations
```

Run the code in the next cell. You should see two "sliders" appear, one for `w` and one for `b`.

**Game**:
Change w and b around until the blue curve, labeled "model", which is the graph of the `\sigma` function, passes through *both* of the data points at the same time.

```julia
w = 10.0 # try manipulating w between -10 and 30
b = 10.0 # try manipulating b between 0 and 20

plot(x -> σ(x, w, b), xlim=(-0,1), ylim=(-0.1,1.1), label="model", legend=:topleft, lw=3)

scatter!([apple_green_amount],  [0.0], label="apple", ms=5)   # marker size = 5
scatter!([banana_green_amount], [1.0], label="banana", ms=5)
```

Notice that the two parameters do two very different things. The **weight**, `w`, determines *how fast* the transition between 0 and 1 occurs. It encodes how trustworthy we think our data  actually is, and in what range we should be putting points between 0 and 1 and thus calling them "unsure". The **bias**, `b`, encodes *where* on the $x$-axis the switch should take place. It can be seen as shifting the function left-right. We'll come to understand these *parameters* more in notebook 6.

Here are some parameter choices that work well:

```julia
w = 25.58; b = 15.6

plot(x -> σ(x, w, b), xlim=(0,1), ylim=(-0.1,1.1), label="model", legend=:topleft, lw=3)

scatter!([apple_green_amount], [0.0], label="apple")
scatter!([banana_green_amount],[1.0], label="banana")
```

(Note that in this problem there are many combinations of `w` and `b` that fit the data well.)

Once we have a model, we have a computational representation for how to choose between "apple" and "banana". So let's pull in some new images and see what our model says about them!

```julia
apple2 = load(datapath("data/107_100.jpg"))
```

```julia
green_amount = mean(Float64.(green.(apple2)))
@show green_amount

scatter!([green_amount], [0.0], label="new apple")
```

Our model successfully says that our new image is an apple! Pat yourself on the back: you've actually just trained your first neural network!

#### Exercise 1

Load the image of a banana in `data/8_100.jpg` as `mybanana`. Edit the code below to calculate the amount of green in `mybanana` and to overlay data for this image with the existing model and data points.

#### Solution

To get the desired overlay, the code we need is

```julia
mybanana = load(datapath("data/8_100.jpg"))
mybanana_green_amount = mean(Float64.(green.(banana)))
scatter!([mybanana_green_amount], [1.0], label="my banana")
```

## Closing remarks: bigger models, more data, more accuracy

That last apple should start making you think: not all apples are red; some are yellow. "Redness" is one attribute of being an apple, but isn't the whole thing. What we need to do is incorporate more ideas into our model by allowing more inputs. However, more inputs would mean more parameters to play with. Also, we would like to have the computer start "learning" on its own, instead of modifying the parameters ourselves until we think it "looks right". How do we take the next step?

The first thing to think about is, if you wanted to incorporate more data into the model, how would you change the sigmoid function? Play around with some ideas. But also, start thinking about how you chose parameters. What process did you do to finally end up at good parameters? These two problems (working with models with more data and automatically choosing parameters) are the last remaining step to understanding deep learning.