Polynomial Regression
==========

In this week's programming exercise, you are asked to implement the basic building blocks for polynomial regression step-by-step. We will do the following:

- **a)** Load a very simple, noisy dataset and inspect it.
- **b)** Construct a design matrix for polynomial regression of degree m: the Vandermonde matrix.
- **c)** Calculate the Moore-Penrose pseudoinverse of the design matrix.
- **d)** Calculate a vector of coefficients that minimizes the squared error of an n-degree polynomial on our given set of measurements (data).
- **e)** Use this coefficient (weight) vector to construct a polynomial that predicts the underlying function the noisy data is drawn from.
- **f)** With the work we have done before, we look at a polynomials of different degrees we fit using the provided data.

Before you start, make sure that you have downloaded the file *poly_data.csv* from stud.ip and put it in the same folder as this notebook! You are supposed to implement the functions yourself!

```python
# the usual imports.
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

a)
------------
1. Use the numpy function **loadtxt** to load the data from *poly_data.csv* into a variable data. Data should now be a $n\times n$ **ndarray** matrix. You can check the type and size of data yourself using the **type** function and the **shape** attribute of the matrix.
2. The first row and second row correspond to the [independent and dependent variable](https://en.wikipedia.org/wiki/Dependent_and_independent_variables) respectively. Store them in two different, new variables **X** (independent) and **Y** (dependent).
3. Use a scatterplot to take a look at the data. It has been generated by sampling a function $f$ and adding Gaussian noise:
\begin{align}
y_i &= f(x_i) + \epsilon \\
\epsilon &\sim \mathcal{N}(0, \sigma^2)
\end{align}

You can use execute the second cell below to take a look at $f(x)$.

```python
# ~~ your code for a) here ~~
data = np.loadtxt('poly_data.csv', delimiter = ',') #load data from csv data points are seperated by a ","" "
X = data[0] #store independent variables in X
Y = data[1] #store dependent variable in Y

plt.plot(X,Y,'o',color = "royalblue")
# setting and labeling your axis
plt.ylabel('dependent variable')
plt.xlabel('independent variable')
plt.xlim((0,10));
```

```python
# Taking a loog at f(x)
def target_func(x):
    return 1.5*np.sin(x) - np.cos(x/2) + 0.5

x = np.linspace(0,10, 101)
y = target_func(x)
plt.plot(x,y);
```

b)
--------

In the lecture, you have derived the formula for linear regression with arbitrary basis functions and normal distributed residuals $\epsilon$. Here, we choose polynomial basis functions and therefore will try and approximate the function above via a polynomial of degree $m$:
$$y = \alpha_0 + \alpha_1x + \alpha_2x^2 + \alpha_3x^3 + \dots + \alpha_mx^m + \epsilon$$
Due to our choice of basis functions, this is called polynomial regression.

The simplest version of polynomial regression uses monomial basis functions $\{1, x, x^2, x^3, \dots \}$ in the design matrix. Such a matrix is called the [Vandermonde matrix](https://en.wikipedia.org/wiki/Vandermonde_matrix) in linear algebra. Implement a function that takes the observed, independent variables $x_i$ stored in **X** and constrcuts a design matrix of the following form:

$$ \Phi = \begin{bmatrix} 1 & x_1 & x_1^2 & \dots & x_1^m \\ 1 & x_2 & x_2^2 & \dots & x_2^m \\ 1 & x_3 & x_3^2 & \dots & x_3^m \\ \vdots & \vdots & \vdots & & \vdots \\ 1 & x_n & x_n^2 & \dots & x_n^m \end{bmatrix}$$

We have provided the function's doc string as well as two quick lines that test your implementation in the notebook cell below.

```python
def poly_dm(x, m):
    """
    Generate a design matrix with monomial basis functions.

    Parameters
    ----------
    x : array_like
        1-D input array.
    m : int
        degree of the monomial used for the last column.

    Returns
    -------
    phi : ndarray
        Design matrix.
        The columns are ``x^0, x^1, ..., x^m``.
    """
    # create an array of shape length of input vector x m and fill it with zero
    #note: we use m + 1 because the amount of rows needs to be degree + the first row
    design_matrix = np.zeros((len(x),m+1))

    for i in range(m+1): #fill all rows with the same data set (complete values each row)
       design_matrix[:,i] = x

    design_matrix[:,0] = 1 #change all entries of row 0 to 1

    for index, x in np.ndenumerate(design_matrix): # for all entries x with the index(column,row)
       exponent = index[1]   # we set the variable exponent to entry 1 of the index -> row_nr or degree
       design_matrix[index] = x ** exponent #the entry at this index is now the original entry to the power of the row_nr

    return design_matrix #return the design matrix
try:
    print('poly_dm:',(lambda a=np.random.rand(10):'O.K.'if np.allclose(poly_dm(a,3),np.vander(a,4,True))else'Something went wrong! (Your result does not match!)')())
except:
    print('poly_dm: Something went horribly wrong! (an error was thrown)')
example_array = np.array([1,2,3,4,5])
poly_dm(example_array,3)
```

    poly_dm: O.K.

    array([[  1.,   1.,   1.,   1.],
           [  1.,   2.,   4.,   8.],
           [  1.,   3.,   9.,  27.],
           [  1.,   4.,  16.,  64.],
           [  1.,   5.,  25., 125.]])

c)
--------

According to the lecture, it is quite usefull to calculate the Moore-Penrose pseudoinverse $A^\dagger$ of a matrix:
$$ A^\dagger = (A^T A)^{-1}A^T$$
where $M^T$ means transpose of matrix $M$ and $M^{-1}$ denotes its inverse.

According to the docstring in the cell below, implement a function that returns $A^\dagger$ for a matrix $A$, and test your implementation against the small test that is included.

```python
def pseudoinverse(A):
    """
    Compute the (Moore-Penrose) pseudo-inverse of a matrix.

    Parameters
    ----------
    A : (M, N) array_like
      Matrix to be pseudo-inverted.

    Returns
    -------
    A_plus : (N, M) ndarray
      The pseudo-inverse of `a`.
    """
    #print("Our original design-matrix:")
    #print(A)

    #print("Transposed:")
    A_transposed = A.transpose()
    #print(A_transposed)

    #Matrix product of A and A_transposed:
    A_to_be_inversed = np.matmul(A_transposed,A)
    #print("Matrix product of A and A_transposed:")
    #print(A_to_be_inversed)

    #The inverse of the Matrix product
    A_inversed = np.linalg.inv(A_to_be_inversed)
    #print("The inverse of the Matrix product")
    #print(A_inversed)

    #The Resulting pseudo_inverse of our given matrix
    pseudo_inverse = np.matmul(A_inversed, A_transposed)
    #print("The Resulting pseudo_inverse of our given matrix")
    #print(pseudo_inverse)

    return pseudo_inverse

# the lines below test the pseudo_inverse function
try:
    print('pseudo_inverse:',(lambda m=np.random.rand(9,5):'Good Job!'if np.allclose(pseudoinverse(m),np.linalg.pinv(m))else'Not quite! (Your result does not match!)')())
except:
    print('pseudo_inverse: Absolutely not! (an error was thrown)')

```

    pseudo_inverse: Good Job!

d)
-------
To estimate the parameters $\alpha_i$ up to a chosen degree $m$, we use call the vector containing all the $\alpha_i$ $w$ and solve the following formula presented in class:
\begin{align}
y &= \Phi w \\
w &= \Phi^\dagger y
\end{align}
where $\Phi$ is the design matrix and $\Phi^\dagger$ its pseudoinverse and $y$ is the vector of dependent variables we observed in our dataset and stored in **Y**.

Implement a function that calculates $w$ according to the docstring given below. Again, a short test of your implementation is provided.

```python
def poly_regress(x, y, deg):
    """
    Least squares polynomial fit.

    Parameters
    ----------
    x : array, shape (M,)
        x-coordinates of the M sample points.

    y : array, shape (M,)
        y-coordinates of the sample points.

    deg : int
        Degree of the fitting polynomial.

    Returns
    -------
    w : array, shape (deg+1,)
        Polynomial coefficients, highest power last.
    """
    # variable to store our design matrix
    design_matrix = poly_dm(x, deg)

    #variavle to store our pseudo_inverse
    inverse_dm = pseudoinverse(design_matrix)

    # variable to store the parameters
    # parameters (beta-coefficients) are calculated by the Matrix product of our dm and it`s pseudo-inverse
    our_parameters = np.matmul(inverse_dm,y)

    #print("The Parameters are")
    #print(our_parameters)

    return our_parameters

# the lines below test the poly_regress function
try:
    print('poly_regress:',(lambda a1=np.random.rand(9),a2=np.random.rand(9):'Ace!'if
                           np.allclose(poly_regress(a1,a2,2),np.polyfit(a1,a2,2)[::-1])else'Almost! (Your result does not match!)')())
except:
    print('poly_regress: Not nearly! (an error was thrown)')
```

    poly_regress: Ace!

e)
--------
The last function we will write will use the vector of coefficients we can now calculate to construct a polynomial function and evaluate it at any point we choose to. Remember, the form of this polynomial is given by:
$$y = \alpha_0 + \alpha_1x + \alpha_2x^2 + \alpha_3x^3 + \dots + \alpha_mx^m$$

This is the model we assumed above, but we do not need to include the noise term here! Again, the function is specified in a docstring and tested in the little {try - catch} block below. *Hint:* The order of the polynomial you need to calculate is inherently given by the length of **w**, the number of coefficients.

```python
def polynom(x, w):
    """ Evaluate a polynomial.

    Parameters
    ----------
    x : 1d array
        Points to evaluate.

    w : 1d array
        Coefficients of the monomials.

    Returns
    -------
    y : 1d array
        Polynomial evaluated for each cell of x.
    """
    y_values = np.zeros(len(x)) #create an array containing with the same length as x
    for i in range(len(x)): # for every entry in x
        for j in range(len(w)): # for every value(beta-coefficient) in our vector w
            y_values[i] += w[j] * (x[i]**j) # the corresponding y value is the sum of all beta-coefficients times the x value to the power of j

    return y_values # retunr the y values

# the lines below test the polynom function
try:
    print('polynom:',(lambda a1=np.random.rand(9),a2=np.random.rand(9):'OK'if np.allclose(polynom(a1,a2),np.polyval(a2[::-1],a1))else'Slight failure! (Your result does not match!)')())
except:
    print('polynom: Significant failure! (an error was thrown)')
```

    polynom: OK

f)
------
f, as in finally. We can now use all the functions we have written above to investigate how well a polynomial of a degree $m$ fits the noisy data we are given. For $m \in \{1,2,10\}$, estimate a polynomial function on the data. Evaluate the three functions on a vector of equidistant points between 0 and 10 (*linearly spaced*). Additionally, plot the original target function $f(x)$, as well as the scatter plot of the data samples. Make sure every graph and the scatter appear in the same plot. Label each graph by adding a label argument to the **plt.plot** function. This allows the use of the **legend()** function and makes the plot significantly more understandable!

```python
plt.figure(figsize = (14, 10))

generated_numbers = np.linspace(0, 10, 1001) # generates 1000 evenly spaced number between 0 and 10
target_function = target_func(generated_numbers) # variable to store given target function

plt.scatter(X, Y, color = "white") #generate scatter plot with white dots
plt.plot(generated_numbers, target_function, label = 'target function') #plot target function with our generated numbers
ax = plt.gca() #select graph axis (used for backgroundcolor)
ax.set_facecolor("black") #set background color to black

for degree in [1,2,10]: # generate our polynomial given our data_set for degree 1, 2 and 10
    w = poly_regress(X, Y, degree) # call above defined functions
    y = polynom(generated_numbers, w)
    plt.plot(generated_numbers, y, label = 'degree: ' + str(degree)) # plot polynomials and add labels for the legend

    """
    for degree in [5]:
    w = poly_regress(X, Y, degree)
    y = polynom(generated_numbers, w)
    plt.plot(generated_numbers, y, label = 'degree: ' + str(degree))
    """

plt.xlim((0, 10)) # zoom in
plt.legend(); # show legend
```

```python

```

Task_37-Group_26_Polynomial-Regression/Task 37 Group 26 Polynomial Regression.ipynb


---
author: Nathan Carter (ncarter@bentley.edu)
---

This answer assumes you have imported SymPy as follows.

```python
from sympy import *                   # load all math functions
init_printing( use_latex='mathjax' )  # use pretty math output
```

Let's assume we've defined a variable and created a formula, as covered
in how to create symbolic variables.

```python
var( 'x' )
formula = x**2 + x
formula
```

$\displaystyle x^{2} + x$

We can substitute a value for $x$ using the `subs` function.
You provide the variable and the value to substitute.

```python
formula.subs( x, 8 )  # computes 8**2 + 8
```

$\displaystyle 72$

If you had to substitute values for multiple variables, you can use
multiple `subs` calls or you can pass a dictionary to `subs`.

```python
var( 'y' )
formula = x/2 + y/3
formula
```

$\displaystyle \frac{x}{2} + \frac{y}{3}$

```python
formula.subs( x, 10 ).subs( y, 6 )
```

$\displaystyle 7$

```python
formula.subs( { x: 10, y: 6 } )
```

$\displaystyle 7$

database/tasks/How to substitute a value for a symbolic variable/Python, using SymPy.ipynb


```python
import sympy as sp
sp.init_printing()
```

```python
x = sp.symbols('x')
sp.solve(x**2 - 8*x + 25)
```

```python
x = 1 + 2*sp.I

print(f'x = {x}, Re(x) = {sp.re(x)}, Imag(x) = {sp.im(x)}, modulus is {sp.Abs(x)}, arg is {sp.arg(x)}')
```

```python
x = sp.symbols('x')
sp.integrate(sp.log(x),x)
```

```python
sp.integrate(x**3 * sp.exp(-x),(x,0,sp.oo))
```

# Matrices

```python
A = sp.Matrix([[1, 2, -3], [4, 5, 6], [7, 8, 9]])
A.inv()

```

```python
a = sp.symbols('a')
A = sp.Matrix([[a, 0, 0], [1, a, 0], [0, 1, a]])
A.inv()
```

```python
C = sp.Matrix([[1, 2],[3, 4],[5, 6]])
D = sp.Matrix([4, 2])
# C * D.transpose()
C
```

```python

```


# Maxwell Boltzman distribution of speed

\begin{equation}
f(v,T)=4\pi v^2(\frac{m}{2\pi kT})^{3/2}e^{\frac{-mv^2}{2 KT}}
\end{equation}

\begin{equation}
RMS=\sqrt{\frac{3kT}{m}}
\end{equation}

\begin{equation}
MPS=\sqrt{\frac{2kT}{m}}
\end{equation}

\begin{equation}
MS=\sqrt{\frac{8kT}{\pi m}}
\end{equation}

```python
import math  as mt
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
def maxwell(M,v, T):
    NA = 6.02e+23
    k = 1.38e-23
    m=M/NA
    a = m/(2*k)
    b =(a /(np.pi))**1.5
    prob =b*T**(-1.5) *4*np.pi*v**2*np.exp(-a*v**2/T)
    return prob
```

```python
def maxwell1(M,v, T):
    NA = 6.02e+23
    k = 1.38e-23
    m=M/NA
    a = m/(2*k)*v**2
    b =(a /(np.pi))**1.5
    prob2 =b*T**(-1.5) *4*np.pi*v**2
    return prob2
```

```python
def maxwell2(M,v, T):
    NA = 6.02e+23
    k = 1.38e-23
    m=M/NA
    a = m/(2*k)*v**2
    b =(a /(np.pi))**1.5
    prob3 =np.exp(-a*v**2/T)
    return prob3
```

```python
Mo=16e-3 #mass of oxygen
Mh=2e-3 # mass of hydrogen
Mn=28e-3 # mass of nitrogen
velocity = np.arange(100, 750, 10)
```

```python
# probability  for oxygen at 200K,300K,400K
prob200 = maxwell(Mo,velocity, 100.0)
prob300 = maxwell(Mo,velocity, 200.0)
prob400 = maxwell(Mo,velocity, 300.0)
```

```python
# probability  for oxygen at 200K,300K,400K
prob2001 = maxwell1(Mo,velocity, 100.0)
prob3001 = maxwell1(Mo,velocity, 200.0)
prob4001 = maxwell1(Mo,velocity, 300.0)
```

```python
# probability  for oxygen at 200K,300K,400K
prob2002 = maxwell2(Mo,velocity, 100.0)
prob3002 = maxwell2(Mo,velocity, 200.0)
prob4002 = maxwell2(Mo,velocity, 300.0)
```

```python
plt.plot(velocity, prob200, 'g-',label='100k')
plt.plot(velocity, prob300, 'b-',label='200k')
plt.plot(velocity, prob400, 'r-',label='300k')
plt.xlabel('Velocity(m/s)')
plt.ylabel('Probability')
plt.title('Maxwell Boltzman distribution of speed for Oxygen')
plt.legend()
plt.savefig("image/BoltmanO.png", dpi = 600) # dpi dot per inch
plt.show()
```

```python
plt.plot(velocity, prob2001, 'g-',label='100k')
plt.plot(velocity, prob3001, 'b-',label='200k')
plt.plot(velocity, prob4001, 'r-',label='300k')
plt.xlabel('Velocity(m/s)')
plt.ylabel('Probability')
plt.title('Maxwell Boltzman distribution of speed for Oxygen')
plt.legend()
plt.savefig("image/Boltman1.png", dpi = 600) # dpi dot per inch
plt.show()
```

```python
plt.plot(velocity, prob2002, 'g-',label='100k')
plt.plot(velocity, prob3002, 'b-',label='200k')
plt.plot(velocity, prob4002, 'r-',label='300k')
plt.xlabel('Velocity(m/s)')
plt.ylabel('Probability')
plt.title('Maxwell Boltzman distribution of speed for Oxygen')
plt.legend()
plt.savefig("image/Boltman2.png", dpi = 600) # dpi dot per inch
plt.show()
```

```python
plt.semilogy(velocity, prob300, 'g-',label='original')
plt.semilogy(velocity, prob3001, 'b-',label='v2')
plt.semilogy(velocity, prob3002, 'r-',label='exp')
plt.xlabel('Velocity(m/s)')
plt.ylabel('Probability')
plt.title('Maxwell Boltzman distribution of speed for Oxygen')
plt.legend()
plt.savefig("image/Boltman3.png", dpi = 600) # dpi dot per inch
plt.show()
```

```python
M = {"Velocity(m/s)": velocity,"Probability(100K)": prob200,"Probability(200K)": prob300,"Probability(300K)": prob400}
```

```python
DF = pd.DataFrame(M)
DF
```

      Velocity(m/s)
      Probability(100K)
      Probability(200K)
      Probability(300K)

      0
      100
      0.000612
      0.000227
      0.000126

      1
      110
      0.000726
      0.000272
      0.000151

      2
      120
      0.000845
      0.000320
      0.000178

      3
      130
      0.000969
      0.000371
      0.000208

      4
      140
      0.001094
      0.000425
      0.000239

      ...
      ...
      ...
      ...
      ...

      60
      700
      0.000295
      0.001104
      0.001319

      61
      710
      0.000265
      0.001061
      0.001297

      62
      720
      0.000237
      0.001019
      0.001274

      63
      730
      0.000212
      0.000977
      0.001250

      64
      740
      0.000189
      0.000935
      0.001225

65 rows × 4 columns

```python
DF.to_csv("data/Boltzman.csv")
```

```python
# probability  for hydrogen,nitrogen ,oxygen at 300K
probh = maxwell(Mh,velocity, 300) # for hydrogen
probn = maxwell(Mn,velocity, 300) # for nitrogen
probo = maxwell(Mo,velocity, 300) #for oxygen
```

```python
plt.plot(velocity, probh, 'g-',label='Hydrogen')
plt.plot(velocity, probn, 'b-',label='Nitrogen')
plt.plot(velocity, probo, 'r-',label='Oxygen')
plt.xlabel('Velocity(m/s)')
plt.ylabel('Probability')
plt.title('Maxwell Boltzman distribution of speed for gas at ')
plt.legend()
plt.savefig("image/Boltmanal.png", dpi = 600) # dpi dot per inch
plt.show()
```

```python
def rms(M,T):
    NA = 6.02e+23
    k = 1.38e-23
    m=M/NA
    vel =np.sqrt(3*k*T/m)
    return vel
```

```python
temp = np.arange(200, 500, 10)
```

```python
# rms for gas
rmsh = rms(Mh,temp) #for hydrogen
rmso = rms(Mo,temp) #for oxygen
rmsn = rms(Mn,temp) #for nitrogen
```

```python
plt.plot(temp, rmsh, 'g-',label='Hydrogen')
plt.plot(temp, rmso, 'b-',label='Oxygen')
plt.plot(temp, rmsn, 'r-',label='Nitrogen')
plt.xlabel('Temperature(K)')
plt.ylabel('RMS(m/s)')
plt.title('RMS speed for gas')
plt.legend()
plt.savefig("image/rms.png", dpi = 600) # dpi dot per inch
plt.show()
```

# Planck's formula of radiation

\begin{equation}
f(\lambda,T)=\frac{2hc^2}{\lambda^5}\frac{1}{e^{\frac{hc}{\lambda KT}}-1}
\end{equation}

```python
def planck(wav, T):
    h = 6.626e-34
    c = 3.0e+8
    k = 1.38e-23
    a = 2.0*h*c**2
    b = h*c/(wav*k*T)
    intensity = a/ ( (wav**5) * (np.exp(b) - 1.0) )
    return intensity
```

```python
wavelengths = np.arange(6e-9, 2e-6, 1e-9) # wavelength in m
```

```python
# intensity at 4000K, 5000K, 6000K, 7000K
intensity4000 = planck(wavelengths, 4000)
intensity5000 = planck(wavelengths, 5000)
intensity6000 = planck(wavelengths, 6000)
intensity7000 = planck(wavelengths, 7000)
```

```python
plt.plot(wavelengths*1e9, intensity4000, 'r-',label='4000k')
plt.plot(wavelengths*1e9, intensity5000, 'g-',label='5000k')
plt.plot(wavelengths*1e9, intensity6000, 'k-',label='6000k')
plt.xlabel('Wavelength(nm)')
plt.ylabel('Spetral energy densiry(W/m$^3$)')
plt.title('Black body radiation')
plt.legend()
plt.savefig("image/planck.png", dpi = 600) # dpi dot per inch
plt.show()
```

```python
M1 = {"wavelength(m)": wavelengths,"Intensity(4000K)": intensity4000,"Intensity(5000K)": intensity5000,"Intensity(6000K)": intensity6000}
```

```python
DF1 = pd.DataFrame(M1)
DF1
```

      wavelength(m)
      Intensity(4000K)
      Intensity(5000K)
      Intensity(6000K)

      0
      6.000000e-09
      3.391580e-236
      4.586434e-184
      2.603298e-149

      1
      7.000000e-09
      2.704885e-199
      1.305948e-154
      8.037306e-125

      2
      8.000000e-09
      1.173680e-171
      1.471820e-132
      1.711556e-106

      3
      9.000000e-09
      3.428211e-150
      1.945881e-115
      2.873970e-92

      4
      1.000000e-08
      4.822859e-133
      9.161117e-102
      6.521953e-81

      ...
      ...
      ...
      ...
      ...

      1989
      1.995000e-06
      7.428806e+11
      1.165626e+12
      1.618849e+12

      1990
      1.996000e-06
      7.418241e+11
      1.163810e+12
      1.616189e+12

      1991
      1.997000e-06
      7.407693e+11
      1.161998e+12
      1.613535e+12

      1992
      1.998000e-06
      7.397163e+11
      1.160190e+12
      1.610887e+12

      1993
      1.999000e-06
      7.386651e+11
      1.158385e+12
      1.608244e+12

1994 rows × 4 columns

```python
DF1.to_csv("data/planck.csv")
```


```python
from sympy import *

half = S(1)/2

B, q, l, xi = var("B, q, l, xi")

N1 = -xi**3 + 2*xi**2 - xi
N2 = 2*xi**3 - 3*xi**2 + 1
N3 = -xi**3 + xi**2
N4 = -2*xi**3 + 3*xi**2
N5 = xi**4/24 - xi**3/12 + xi**2/24

A = Matrix([l*N1, N2, l*N3, N4, l**4/B*N5])

A4 = Matrix([l*N1, N2, l*N3, N4])
dA4 = diff(A4, xi) / l

pprint("\nDistributed Load:")
q1 = 2*q*xi
q2 = 2*q*(1 - xi)
pprint("q₁, q₂:")
pprint(q1)
pprint(q2)
fq  = integrate(q1*A4*l, (xi, 0, half))
fq += integrate(q2*A4*l, (xi, half, 1))
pprint("leads to fq / ql:")
pprint(fq/(q*l))

pprint("\nConcentrated Moment:")
pprint("M at xi=1/2")
Mxi = var("M")
fM  = - Mxi * dA4.subs(xi, half)
pprint("leads to fM / M:")
pprint(fM/M)

pprint("\nConcentrated Force:")
pprint("F at xi=1/2")
Fxi = var("F")
fF  = Fxi * A4.subs(xi, half)
pprint("leads to fF / F:")
pprint(fF/F)

# Distributed Load:
# q₁, q₂:
# 2⋅q⋅ξ
# 2⋅q⋅(-ξ + 1)
# leads to fq / ql:
# ⎡-5⋅l ⎤
# ⎢─────⎥
# ⎢  96 ⎥
# ⎢     ⎥
# ⎢ 1/4 ⎥
# ⎢     ⎥
# ⎢ 5⋅l ⎥
# ⎢ ─── ⎥
# ⎢  96 ⎥
# ⎢     ⎥
# ⎣ 1/4 ⎦
#
# Concentrated Moment:
# M at xi=1/2
# leads to fM / M:
# ⎡-1/4⎤
# ⎢    ⎥
# ⎢ 3  ⎥
# ⎢─── ⎥
# ⎢2⋅l ⎥
# ⎢    ⎥
# ⎢-1/4⎥
# ⎢    ⎥
# ⎢-3  ⎥
# ⎢─── ⎥
# ⎣2⋅l ⎦
#
# Concentrated Force:
# F at xi=1/2
# leads to fF / F:
# ⎡-l ⎤
# ⎢───⎥
# ⎢ 8 ⎥
# ⎢   ⎥
# ⎢1/2⎥
# ⎢   ⎥
# ⎢ l ⎥
# ⎢ ─ ⎥
# ⎢ 8 ⎥
# ⎢   ⎥
# ⎣1/2⎦

```


# Revisiting error propagation with automatic differentiation

By Kyle Cranmer, March 2, 2020

This notebook is dedicated to Feeman Dyson, who died on February 28, 2020 in Princeton, NJ at the age of 96.

“New directions in science are launched by new tools much more often than by new concepts. The effect of a concept-driven revolution is to explain old things in new ways. The effect of a tool-driven revolution is to discover new things that have to be explained.”

-- Freeman Dyson

)

## Reminder of propagation of errors

This notebook was made to investigate the propagation of errors formula.
We imagine that we have a function $q(x,y)$ and we want to propagate the
uncertainty on $x$ and $y$ (denoted $\sigma_x$ and $\sigma_y$, respectively) through to the quantity $q$.

The most straight forward way to do this is just randomly sample $x$ and $y$, evaluate $q$ and look at it's distribution. This is really the definition of what we mean by propagation of uncertianty. It's very easy to do with some simply python code.

The calculus formula for the propagation of errors is really an approximation. This is the formula for a general $q(x,y)$
\begin{equation}
\sigma_q^2 = \left( \frac{\partial q}{\partial x} \sigma_x \right)^2 + \left( \frac{\partial q}{\partial y}\sigma_y \right)^2
\end{equation}

In the special case of addition  $q(x,y) = x\pm y$ we have $\sigma_q^2 = \sigma_x^2 + \sigma_y^2$.

In the special case of multiplication $q(x,y) = x y$ and division $q(x,y) = x / y$ we have $(\sigma_q/q)^2 = (\sigma_x/x)^2 + (\sigma_y/y)^2$, which we can rewrite as $\sigma_q = (x/y) \sqrt{(\sigma_x/x)^2 + (\sigma_y/y)^2}$

Let's try out these formulas and compare the direct approach of making the distribution to the prediction from these formulas

## Automatic Differentiation

Excerpts from the Wikipedia article: https://en.wikipedia.org/wiki/Automatic_differentiation

In mathematics and computer algebra, automatic differentiation (AD), also called algorithmic differentiation or computational differentiation,[1][2] is a set of techniques to numerically evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc.) and elementary functions (exp, log, sin, cos, etc.). By applying the chain rule repeatedly to these operations, derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor more arithmetic operations than the original program.

Usually, two distinct modes of AD are presented, forward accumulation (or forward mode) and reverse accumulation (or reverse mode). Forward accumulation specifies that one traverses the chain rule from inside to outside (that is, first compute
${\displaystyle dw_{1}/dx}$ and then
${\displaystyle dw_{2}/dw_{1}}$ and at last
${\displaystyle dy/dw_{2}})$, while reverse accumulation has the traversal from outside to inside (first compute
${\displaystyle dy/dw_{2}}$ and then
${\displaystyle dw_{2}/dw_{1}}$ and at last
${\displaystyle dw_{1}/dx})$. More succinctly,

 * forward accumulation computes the recursive relation:
${\displaystyle {\frac {dw_{i}}{dx}}={\frac {dw_{i}}{dw_{i-1}}}{\frac {dw_{i-1}}{dx}}}$ with
${\displaystyle w_{3}=y}$, and,

 * reverse accumulation computes the recursive relation:
${\displaystyle {\frac {dy}{dw_{i}}}={\frac {dy}{dw_{i+1}}}{\frac {dw_{i+1}}{dw_{i}}}}$ with
${\displaystyle w_{0}=x}$.

<!--

Fundamental to AD is the decomposition of differentials provided by the chain rule. For the simple composition

\begin{equation}
{\displaystyle {\begin{aligned}y&=f(g(h(x)))=f(g(h(w_{0})))=f(g(w_{1}))=f(w_{2})=w_{3}\\w_{0}&=x\\w_{1}&=h(w_{0})\\w_{2}&=g(w_{1})\\w_{3}&=f(w_{2})=y\end{aligned}}}
\end{equation}
the chain rule gives

\begin{equation}
{\displaystyle {\frac {dy}{dx}}={\frac {dy}{dw_{2}}}{\frac {dw_{2}}{dw_{1}}}{\frac {dw_{1}}{dx}}}
\end{equation}
Usually, two distinct modes of AD are presented, forward accumulation (or forward mode) and reverse accumulation (or reverse mode). Forward accumulation specifies that one traverses the chain rule from inside to outside (that is, first compute ${\displaystyle dw_{1}/dx}$ and then
${\displaystyle dw_{2}/dw_{1}}$ and at last
${\displaystyle dy/dw_{2}})$, while reverse accumulation has the traversal from outside to inside (first compute
${\displaystyle dy/dw_{2}}$ and then
${\displaystyle dw_{2}/dw_{1}}$ and at last
${\displaystyle dw_{1}/dx})$. More succinctly,
forward accumulation computes the recursive relation:

\begin{equation}
{\displaystyle {\frac {dw_{i}}{dx}}={\frac {dw_{i}}{dw_{i-1}}}{\frac {dw_{i-1}}{dx}}}
\end{equation}
with
${\displaystyle w_{3}=y}$, and,
reverse accumulation computes the recursive relation:

\begin{equation}
{\displaystyle {\frac {dy}{dw_{i}}}={\frac {dy}{dw_{i+1}}}{\frac {dw_{i+1}}{dw_{i}}}}
\end{equation}
with ${\displaystyle w_{0}=x}$.
Generally, both forward and reverse accumulation are specific manifestations of applying the operator of program composition, fixing the appropriate one of the two mappings

\begin{equation}
{\displaystyle (w,y)}.
\end{equation}

### Forward mode

In forward accumulation AD, one first fixes the independent variable with respect to which differentiation is performed and computes the derivative of each sub-expression recursively. In a pen-and-paper calculation, this involves repeatedly substituting the derivative of the inner functions in the chain rule:

\begin{equation}
{\displaystyle {\frac {\partial y}{\partial x}}={\frac {\partial y}{\partial w_{n-1}}}{\frac {\partial w_{n-1}}{\partial x}}={\frac {\partial y}{\partial w_{n-1}}}\left({\frac {\partial w_{n-1}}{\partial w_{n-2}}}{\frac {\partial w_{n-2}}{\partial x}}\right)={\frac {\partial y}{\partial w_{n-1}}}\left({\frac {\partial w_{n-1}}{\partial w_{n-2}}}\left({\frac {\partial w_{n-2}}{\partial w_{n-3}}}{\frac {\partial w_{n-3}}{\partial x}}\right)\right)=\cdots }
\end{equation}
This can be generalized to multiple variables as a matrix product of Jacobians.

### Reverse mode

In reverse accumulation AD, the dependent variable to be differentiated is fixed and the derivative is computed with respect to each sub-expression recursively. In a pen-and-paper calculation, the derivative of the outer functions is repeatedly substituted in the chain rule:

\begin{equation}
{\frac  {\partial y}{\partial x}}={\frac  {\partial y}{\partial w_{1}}}{\frac  {\partial w_{1}}{\partial x}}=\left({\frac  {\partial y}{\partial w_{2}}}{\frac  {\partial w_{2}}{\partial w_{1}}}\right){\frac  {\partial w_{1}}{\partial x}}=\left(\left({\frac  {\partial y}{\partial w_{3}}}{\frac  {\partial w_{3}}{\partial w_{2}}}\right){\frac  {\partial w_{2}}{\partial w_{1}}}\right){\frac  {\partial w_{1}}{\partial x}}=\cdots
\end{equation}

-->

We will use
https://jax.readthedocs.io/en/latest/notebooks/autodiff_cookbook.html

```python
from jax import grad, jacfwd
import jax.numpy as np
```

Now here are 3 lines of code for the propagation of uncertainty formula

\begin{equation}
\sigma_q = \sqrt{\left( \frac{\partial q}{\partial x} \sigma_x \right)^2 + \left( \frac{\partial q}{\partial y}\sigma_y \right)^2}
\end{equation}

```python
def error_prop_jax_gen(q,x,dx):
    jac = jacfwd(q)
    return np.sqrt(np.sum(np.power(jac(x)*dx,2)))
```

## Setup two observations with uncertainties

Below I'll use $x$ and $y$ for symbols, but they will be stored in the array `x` so that `x[0]=`$x$ and `x[1]=$y$.

```python
x_ = np.array([2.,3.])
dx_ = np.array([.1,.1])
```

    /Users/cranmer/anaconda3/envs/jax-md/lib/python3.6/site-packages/jax/lib/xla_bridge.py:120: UserWarning: No GPU/TPU found, falling back to CPU.
      warnings.warn('No GPU/TPU found, falling back to CPU.')

## Addition and Subtraction

In the special case of addition  $q(x,y) = x\pm y$ we have $\sigma_q^2 = \sigma_x^2 + \sigma_y^2$.

```python
def q(x):
    return x[0]+x[1]
```

```python
def error_prop_classic(x, dx):
    # for q = x[0]*x[1]
    ret = dx[0]**2 + dx[1]**2
    return np.sqrt(ret)
```

```python
print('q = ', q(x_), '+/-', error_prop_classic(x_, dx_))
```

    q =  5.0 +/- 0.14142136

```python
print('q = ', q(x_), '+/-', error_prop_jax_gen(q, x_, dx_))
```

    q =  5.0 +/- 0.14142136

## Multiplication and Division

In the special case of multiplication
\begin{equation}
q(x,y) = x y
\end{equation}
and division
\begin{equation}
q(x,y) = \frac{x}{y}
\end{equation}

\begin{equation}
(\sigma_q/q)^2 = (\sigma_x/x)^2 + (\sigma_y/y)^2
\end{equation}
which we can rewrite as
\begin{equation}
\sigma_q = (x/y) \sqrt{\left(\frac{\sigma_x}{x}\right)^2 + \left(\frac{\sigma_y}{y}\right)^2}
\end{equation}

```python
def q(x):
    return x[0]*x[1]
```

```python
def error_prop_classic(x, dx):
    # for q = x[0]*x[1]
    ret = (dx[0]/x[0])**2 + (dx[1]/x[1])**2
    return (x[0]*x[1])*np.sqrt(ret)
```

```python
print('q = ', q(x_), '+/-', error_prop_classic(x_, dx_))
```

    q =  6.0 +/- 0.36055514

```python
print('q = ', q(x_), '+/-', error_prop_jax_gen(q, x_, dx_))
```

    q =  6.0 +/- 0.36055514

```python
def q(x):
    return x[0]/x[1]
```

```python
def error_prop_classic(x, dx):
    # for q = x[0]*x[1]
    ret = (dx[0]/x[0])**2 + (dx[1]/x[1])**2
    return (x[0]/x[1])*np.sqrt(ret)
```

```python
print('q = ', q(x_), '+/-', error_prop_classic(x_, dx_))
```

    q =  0.6666667 +/- 0.040061682

```python
print('q = ', q(x_), '+/-', error_prop_jax_gen(q, x_, dx_))
```

    q =  0.6666667 +/- 0.040061682

## Powers

$q(x,y) = x^m y^n$
we have

\begin{equation}
(\sigma_q/q)^2 = \left(|m|\frac{\sigma_x}{x}\right)^2 + \left(|n|\frac{\sigma_y}{y}\right)^2
\end{equation}
which we can rewrite as
\begin{equation}
\sigma_q = x^m y^n \sqrt{\left(|m|\frac{\sigma_x}{x}\right)^2 + \left(|n|\frac{\sigma_y}{y}\right)^2}
\end{equation}

```python
def q(x, m=2, n=3):
    return np.power(x[0],m)*np.power(x[1],n)
```

```python
x_ = np.array([1.5, 2.5])
dx_ = np.array([.1, .1])
q(x_)
```

    DeviceArray(35.15625, dtype=float32)

```python
def error_prop_classic(x, dx):
    # for q = x[0]*x[1]
    dq_ = q(x_)*np.sqrt(np.power(2*dx_[0]/x_[0],2)+np.power(3*dx_[1]/x_[1],2))
    return dq_
```

```python
print('q = ', q(x_), '+/-', error_prop_classic(x_, dx_))
```

    q =  35.15625 +/- 6.3063865

```python
print('q = ', q(x_), '+/-', error_prop_jax_gen(q, x_, dx_))
```

    q =  35.15625 +/- 6.3063865

## Misc Examples

See some examples here:

http://www.geol.lsu.edu/jlorenzo/geophysics/uncertainties/Uncertaintiespart2.html

Example:  `w = (4.52 ± 0.02) cm, A = (2.0 ± 0.2), y = (3.0 ± 0.6) cm`. Find

\begin{equation}
z=\frac{wy^2}{\sqrt{A}}
\end{equation}

The second relative error, (Dy/y), is multiplied by 2 because the power of y is 2.
The third relative error, (DA/A), is multiplied by 0.5 since a square root is a power of one half.

So Dz = 0.49 (28.638  ) = 14.03  which we round to 14

z = (29 ± 14)
Using Eq. 3b,
z=(29 ± 12)
Because the uncertainty begins with a 1, we keep two significant figures and round the answer to match.

```python
def q(x):
    return x[0]*x[2]*x[2]/np.sqrt(x[1])
```

```python
x_ = np.array([4.52, 2., 3.]) #w,A,y
dx_ = np.array([.02, .2, .6])

print('q = ', q(x_), '+/-', error_prop_jax_gen(q, x_, dx_))
```

    q =  28.765104 +/- 11.596283

### Check with a plot

```python
import numpy as onp  #using jax as np right now
w_ = onp.random.normal(x_[0], dx_[0], 10000)
A_ = onp.random.normal(x_[1], dx_[1], 10000)
y_ = onp.random.normal(x_[2], dx_[2], 10000)
x__ = np.vstack((w_, A_, y_))
z_ = q(x__)
print('mean =', np.mean(z_), 'std = ', np.std(z_))
```

    mean = 30.050316 std =  11.813263

```python
import matplotlib.pyplot as plt
```

```python
_ = plt.hist(z_, bins=50)
```

## Example 2

also taken from http://www.geol.lsu.edu/jlorenzo/geophysics/uncertainties/Uncertaintiespart2.html

`w = (4.52 ± 0.02) cm, x = (2.0 ± 0.2) cm, y = (3.0 ± 0.6) cm. `

Find
\begin{equation}
z = w x +y^2
\end{equation}

We have v = wx = (9.0 ± 0.9) cm.
The calculation of the uncertainty in  is the same as that shown to the left. Then from Eq. 1b
Dz =  3.7
z = (18 ± 4) .

```python
def q(x):
    # [w,x,y]
    return x[0]*x[1]+x[2]*x[2]
```

```python
x_ = np.array([4.52, 2., 3.]) #w,x,y
dx_ = np.array([.02, .2, .6])
```

```python
print(q(x_),'+/-', error_prop_jax_gen(q, x_, dx_))
```

    18.04 +/- 3.711983

## An example with many inputs

The code we used for `error_prop_jax_gen` is generic and supports functions `q` on any number of variables

```python
def q(x):
    return np.sum(x)
```

```python
x_ = 1.*np.arange(1,101) #counts from 1-100 (and 1.* to make them floats)
dx_ = 0.1*np.ones(100)
```

The sum from $1 to N$ is $N*(N+1)/2$  (see [the story of Gauss](https://hsm.stackexchange.com/questions/384/did-gauss-find-the-formula-for-123-ldotsn-2n-1n-in-elementary-school)), so we expect q(x)=5050. And the uncertainty should be $\sqrt{100}*0.1$ = 1.

```python
print(q(x_),'+/-', error_prop_jax_gen(q, x_, dx_))
```

    5050.0 +/- 1.0

another toy example... product from 1 to 10

```python
def q(x):
    return np.product(x)
```

```python
x_ = 1.*np.arange(1,11) #counts from 1-100 (and 1.* to make them floats)
dx_ = 0.1*np.ones(10)
print(q(x_),'+/-', error_prop_jax_gen(q, x_, dx_))
```

    3628800.0 +/- 451748.1

Checking this is an exercise left to the reader :-)

```python

```


# Elliptic PDE: Illustrating the Maximum Principle

Copyright (C) 2010-2020 Luke Olson

Copyright (C) 2020 Andreas Kloeckner

MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

```python
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import matplotlib.pyplot as plt
import numpy as np

import sympy as sym
```

## Checking the Solution Symbolically

```python
sx = sym.Symbol("x")
sy = sym.Symbol("y")
sr = sym.sqrt(sx**2 + sy**2)
sphi = sym.atan2(sy, sx)
ssol = sr**2 * sym.cos(2*sphi)

sym.simplify(sym.diff(ssol, sx, 2) + sym.diff(ssol, sy, 2))
```

$\displaystyle 0$

## Plotting the Solution

```python
step = 0.04
maxval = 1.0

r = np.linspace(0, 1.25, 50)
p = np.linspace(0, 2*np.pi, 50)
R, P = np.meshgrid(r, p)
X, Y = R*np.cos(P), R*np.sin(P)

Z = R**2 * np.cos(2*P)

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.YlGnBu_r,
                       linewidth=0, antialiased=False)
```

For the domain (and any given subdomain):
* Where are maximum and minimum attained?

```python

```

demos/intro/Elliptic PDE Illustrating the Maximum Principle.ipynb


# Vector Multiplication
Vector multiplication can be performed in three ways:

- Scalar Multiplication
- Dot Product Multiplication
- Cross Product Multiplication

## Scalar Multiplication
Let's start with *scalar* multiplication - in other words, multiplying a vector by a single numeric value.

Suppose I want to multiply my vector by 2, which I could write like this:

\begin{equation} \vec{w} = 2\vec{v}\end{equation}

Note that the result of this calculation is a new vector named **w**. So how would we calculate this?
Recall that **v** is defined like this:

\begin{equation}\vec{v} = \begin{bmatrix}2 \\ 1 \end{bmatrix}\end{equation}

To calculate 2v, we simply need to apply the operation to each dimension value in the vector matrix, like this:

\begin{equation}\vec{w} = \begin{bmatrix}2 \cdot 2 \\  2 \cdot 1 \end{bmatrix}\end{equation}

Which gives us the following result:

\begin{equation}\vec{w} = \begin{bmatrix}2 \cdot 2 \\  2 \cdot 1 \end{bmatrix} = \begin{bmatrix}4 \\ 2 \end{bmatrix}\end{equation}

In Python, you can apply these sort of matrix operations directly to numpy arrays, so we can simply calculate **w** like this:

```python
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt
import math

v = np.array([2,1])

w = 2 * v
print(w)

# Plot w
origin = [0], [0]
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, *w, scale=10)
plt.show()
```

The same approach is taken for scalar division.

Try it for yourself - use the cell below to calculate a new vector named **b** based on the following definition:

\begin{equation}\vec{b} = \frac{\vec{v}}{2}\end{equation}

```python
b = v / 2
print(b)

# Plot b
origin = [0], [0]
plt.axis('equal')
plt.grid()
plt.ticklabel_format(style='sci', axis='both', scilimits=(0,0))
plt.quiver(*origin, *b, scale=10)
plt.show()
```

## Dot Product Multiplication
So we've seen how to multiply a vector by a scalar. How about multiplying two vectors together? There are actually two ways to do this depending on whether you want the result to be a *scalar product* (in other words, a number) or a *vector product* (a vector).

To get a scalar product, we calculate the *dot product*. This takes a similar approach to multiplying a vector by a scalar, except that it multiplies each component pair of the vectors and sums the results. To indicate that we are performing a dot product operation, we use the • operator:

\begin{equation} \vec{v} \cdot \vec{s} = (v_{1} \cdot s_{1}) + (v_{2} \cdot s_{2}) ... + \; (v_{n} \cdot s_{n})\end{equation}

So for our vectors **v** (2,1) and **s** (-3,2), our calculation looks like this:

\begin{equation} \vec{v} \cdot \vec{s} = (2 \cdot -3) + (1 \cdot 2) = -6 + 2 = -4\end{equation}

So the dot product, or scalar product, of **v** • **s** is **-4**.

In Python, you can use the *numpy.**dot*** function to calculate the dot product of two vector arrays:

```python
import numpy as np

v = np.array([2,1])
s = np.array([-3,2])
d = np.dot(v,s)
print (d)
```

    -4

In Python 3.5 and later, you can also use the **@** operator to calculate the dot product:

```python
import numpy as np

v = np.array([2,1])
s = np.array([-3,2])
d = v @ s
print (d)
```

    -4

### The Cosine Rule
An useful property of vector dot product multiplication is that we can use it to calculate the cosine of the angle between two vectors. We could write the dot products as:

$$ \vec{v} \cdot \vec{s} = \|\vec{v} \|\|\vec{s}\| \cos (\theta) $$

Which we can rearrange as:

$$ \cos(\theta) = \frac{\vec{v} \cdot \vec{s}}{\|\vec{v} \|\|\vec{s}\|} $$

So for our vectors **v** (2,1) and **s** (-3,2), our calculation looks like this:

$$ \cos(\theta) = \frac{(2 \cdot-3) + (-3 \cdot 2)}{\sqrt{2^{2} + 1^{2}} \times \sqrt{-3^{2} + 2^{2}}} $$

So:

$$\cos(\theta) = \frac{-4}{8.0622577483}$$

Which calculates to:

$$\cos(\theta) = -0.496138938357 $$

So:

$$\theta \approx 119.74 $$

Here's that calculation in Python:

```python
import math
import numpy as np

# define our vectors
v = np.array([2,1])
s = np.array([-3,2])

# get the magnitudes
vMag = np.linalg.norm(v)
sMag = np.linalg.norm(s)

# calculate the cosine of theta
cos = (v @ s) / (vMag * sMag)

# so theta (in degrees) is:
theta = math.degrees(math.acos(cos))

print(theta)

```

    119.74488129694222

## Cross Product Multiplication
To get the *vector product* of multipying two vectors together, you must calculate the *cross product*. The result of this is a new vector that is at right angles to both the other vectors in 3D Euclidean space. This means that the cross-product only really makes sense when working with vectors that contain three components.

For example, let's suppose we have the following vectors:

\begin{equation}\vec{p} = \begin{bmatrix}2 \\ 3 \\ 1 \end{bmatrix}\;\; \vec{q} = \begin{bmatrix}1 \\ 2 \\ -2 \end{bmatrix}\end{equation}

To calculate the cross product of these vectors, written as **p** x **q**, we need to create a new vector (let's call it **r**) with three components (r1, r2, and r3). The values for these components are calculated like this:

\begin{equation}r_{1} = p_{2}q_{3} - p_{3}q_{2}\end{equation}
\begin{equation}r_{2} = p_{3}q_{1} - p_{1}q_{3}\end{equation}
\begin{equation}r_{3} = p_{1}q_{2} - p_{2}q_{1}\end{equation}

So in our case:

\begin{equation}\vec{r} = \vec{p} \times \vec{q} = \begin{bmatrix}(3 \cdot -2) - (1 \cdot 2) \\ (1 \cdot 1) - (2 \cdot -2) \\ (2 \cdot 2) - (3 \cdot 1) \end{bmatrix} = \begin{bmatrix}-6 - 2 \\ 1 - -4 \\ 4 - 3 \end{bmatrix} = \begin{bmatrix}-8 \\ 5 \\ 1 \end{bmatrix}\end{equation}

In Python, you can use the *numpy.**cross*** function to calculate the cross product of two vector arrays:

```python
import numpy as np

p = np.array([2,3,1])
q = np.array([1,2,-2])
r = np.cross(p,q)
print (r)
```

    [-8  5  1]


# Modelo de motor DC

El par $\tau_r$ en el eje de un motor DC con constante de par $K_{tr}$, momento de inercia del rotor $I_r$, y coeficiente de rozamiento viscoso $b_r$, cuando circula una corriente $i$ por sus bobinados y gira a una velocidad $\omega_r$, será:

$$
\begin{equation}
    \tau_r = K_{tr}i - I_r\dot\omega_r - b_r\omega_r
\end{equation}
$$

Al introducir una reductora con relación de transmisión $n$ y rendimiento $\eta$, la velocidad de giro a la salida $\omega_g$ se reduce en un factor $n$. El par, por el contrario, es multiplicado por el mismo factor, además de verse afectado por las pérdidas mecánicas en los engranajes, representadas a través del rendimiento $\eta$:

$$
\begin{equation}
    \omega_g = \frac{\omega_r}{n} \qquad
    \tau_g = \eta n\tau_r
\end{equation}
$$

Sustituyendo ambas expresiones en la primera ecuación, se obtiene la relación entre par y velocidad a la salida de la reductora:

$$
\begin{equation}
    \tau_g = \eta nK_{tr}i - \eta n^2I_r\dot\omega_g - \eta n^2b_r\omega_g
\end{equation}
$$

En esta expresión, es fácil identificar los parámetros equivalentes del conjunto motor-reductora, visto desde el lado de la carga:

$$
\begin{equation}
    K_{tg} = \eta nK_{tr} \qquad
    b_g = \eta n^2b_r     \qquad
    I_g = \eta n^2I_r
\end{equation}
$$

Del mismo modo, se puede proceder con la ecuación del circuito eléctrico. El voltaje $V$ en los bornes del motor es:

$$
\begin{equation}
    V = L\frac{\mathrm{d}i}{\mathrm{d}t} + Ri + K_{vr}\omega_r
\end{equation}
$$

donde $L$ y $R$ son la inductancia y la resistencia del bobinado, respectivamente, y $K_{vr}$ es la constante de velocidad del motor (relación entre velocidad de giro y fuerza contraelectromotriz), que en un motor DC coincide con la constante de par $K_{tr}$. Si introducimos este último hecho en la ecuación, junto con la relación de velocidades:

$$
\begin{equation}
    V = L\frac{\mathrm{d}i}{\mathrm{d}t} + Ri + nK_{tr}\omega_g
\end{equation}
$$

podemos deducir que la constante de velocidad equivalente, vista desde la salida de la reductora, es:

$$
\begin{equation}
    K_{vg} = nK_{tr} = \frac{K_{tg}}{\eta}
\end{equation}
$$

## Identificación de parámetros del motor [Pololu 2215](https://www.pololu.com/product/2215)

Con estas ecuaciones, podemos proceder a identificar los parámetros del motor. El fabricante proporciona, en la hoja de características del motor, unas curvas par-velocidad y par-corriente en régimen permanente para una tensión nominal de 6V, medidas en el eje de salida de la reductora. A partir de estas curvas, podemos obtener la mayoría de los parámetros del motor. Como los ensayos se han realizado en régimen permanente, los efectos de la inductancia $L$ y el momento de inercia equivalente del rotor $I_g$ no aparecen en los resultados, y por lo tanto no se pueden identificar. Si se eliminan estas dos constantes, las ecuaciones que sirven para extraer los parámetros de las curvas del fabricante serán:

$$
\begin{align}
    \tau_g &= K_{tg}i - b_g\omega_g \\
    6 &= Ri + K_{vg}\omega_g
\end{align}
$$

En estas ecuaciones aparecen las cuatro constantes que sí podremos identificar a partir de las curvas: $K_{tg}$, $K_{vg}$, $b_g$ y $R$.

Las curvas par-velocidad y par-corriente correspondientes a los ensayos del fabricante para el modelo 2215 son las siguientes:

$$
\begin{align}
    \omega_g &= 410 - 32\tau_g \\
    i &= 0.073 + 0.11\tau_g
\end{align}
$$

donde el par está en kg·mm, la velocidad en rpm, y la corriente en A. En estas dos ecuaciones hay cuatro constantes, así que si se cogen las dos ecuaciones del motor, y se expresan también en forma par-velocidad y par-corriente, se pueden identificar los cuatro parámetros, que deberán ser convertidos posteriormente a unidades del SI.

Las dos constantes que faltan se pueden obtener a partir de ensayos dinámicos, midiendo la respuesta a una entrada de voltaje determinada con una carga de inercia conocida. Respecto a la inductancia $L$, a partir de aquí ignoraremos su efecto, dado que la dinámica de la parte eléctrica es mucho más rápida que la de la parte mecánica, y no tiene un efecto relevante en el control. Para la inercia equivalente del rotor $I_g$ se usará el siguiente valor, estimado a partir de un ensayo básico:

$$
    I_g = 5·10^{-5} \mathrm{kg·m}^2
$$

## Efecto de la reductora adicional del robot

El robot que vamos a controlar tiene una reductora adicional de relación $n_r$ 41:25, así que si queremos obtener un modelo del sistema motor-reductora completo, deberemos aplicar una segunda conversión a los parámetros obtenidos para la primera reductora. En este caso, consideraremos que las pérdidas mecánicas de la segunda reductora son despreciables frente a las de la primera, que tiene una relación de velocidades mucho más elevada (38437:507):

$$
\begin{equation}
    K_{tw} = n_rK_{tg} \qquad
    K_{vw} = n_rK_{vg} \qquad
    b_w = n_r^2b_g     \qquad
    I_w = n_r^2I_g
\end{equation}
$$

De este modo, las ecuaciones del motor, considerando el par $\tau_w$ y la velocidad $\omega_w$ ya en la rueda, serán:

$$
\begin{align}
    \tau_w &= K_{tw}i - I_w\dot\omega_w - b_w\omega_w \\
    V &= Ri + K_{vw}\omega_w
\end{align}
$$

## Introducción del segundo motor

Hasta ahora hemos considerado un sólo motor, pero nuestro robot tiene dos ruedas, con un motor independiente en cada una. Para evitar tener que desarrollar un modelo completo en tres dimensiones, en la primera fase de diseño del controlador se simplificará el sistema, tratándolo como un modelo plano. Eso implica que el robot se moverá en línea recta, con las dos ruedas girando siempre a la misma velocidad $\omega_m$, mientras ambos motores reciben el mismo voltaje de entrada. Bajo estos supuestos, se puede considerar que el par total $\tau_m$ será dos veces el par de una sola rueda $\tau_w$. Como los motores van conectados en paralelo a la tensión de entrada $V$, la segunda ecuación no cambia:

$$
\begin{align}
    \tau_m &= 2K_{tw}i - 2I_w\dot\omega_m - 2b_w\omega_m \\
    V &= Ri + K_{vw}\omega_m
\end{align}
$$

En estas ecuaciones se pueden identificar los parámetros correspondientes al modelo completo, que será el utilizado para el diseño del controlador. Para mayor comodidad, se muestran ya los valores en función de los parámetros identificados previamente a la salida de la primera reductora:

$$
\begin{equation}
    K_t = 2n_rK_{tg} \qquad
    K_v = n_rK_{vg}  \qquad
    b = 2n_r^2b_g    \qquad
    I_m = 2n_r^2I_g
\end{equation}
$$

Así, las ecuaciones definitivas que usaremos para diseñar el controlador resultan:

$$
\begin{align}
    \tau_m &= K_ti - I_m\dot\omega_m - b\omega_m \\
    V &= Ri + K_v\omega_m
\end{align}
$$

## Funcionamiento como generador

Todo lo desarollado hasta ahora se basa en la suposición de que el motor produce una potencia positiva, que es transmitida a la rueda a través de la reductora. Pero cuando el motor está frenando, la potencia fluye en sentido contrario, de la rueda al motor. En esta situación, lo que antes era la salida de la reductora pasa a ser la entrada, y esto implica que las pérdidas de par se producirán en el lado opuesto. En general, en función del signo de la potencia mecánica, se cumplirá que:

$$
\begin{equation}
    \left\{
    \begin{matrix}
        \tau_g = \eta n\tau_r && \dot W > 0 \\
        \eta\tau_g = n\tau_r && \dot W < 0
    \end{matrix}
    \right.
\end{equation}
$$

Esto quiere decir que los parámetros del sistema serán diferentes dependiendo de si el motor está trabajando como generador o como motor. Es fácil demostrar que los parámetros del conjunto motor-reductora, cuando trabaja en modo generador, pasan a ser:

$$
\begin{equation}
    K_t^G = \frac{K_t}{\eta^2} \qquad
    K_v^G = K_v                \qquad
    b^G = \frac{b}{\eta^2}     \qquad
    I_m^G = \frac{I_m}{\eta^2}
\end{equation}
$$

donde el rendimiento de la primera reductora $\eta$ se puede obtener fácilmente dividiendo $K_{tg}$ entre $K_{vg}$.

Si los parámetros del modelo son diferentes en cada caso, lo correcto sería diseñar dos controladores, uno para modo motor y otro para modo generador. Durante el funcionamiento, habría que detectar qué está haciendo el motor en cada momento, y aplicar las ganancias del controlador correspondiente. Para conocer el signo de la potencia mecánica, basta con comparar el signo del par $\tau_m$ con el de la velocidad de giro $\omega_m$. Para poder estimar el par sin tener que medir la corriente, se puede despejar ésta en una ecuación y sustituir el resultado en la otra:

$$
\begin{equation}
    \tau_m = \frac{K_t}{R}V - \left(\frac{K_tK_v}{R} + b\right)\omega_m - I_m\dot\omega_m
\end{equation}
$$

Si hacemos lo mismo en modo generador, sustituyendo los valores correspondientes de las constantes, se demuestra que:

$$
\begin{equation}
    \tau_m^G = \frac{\tau_m}{\eta^2}
\end{equation}
$$

En nuestra aplicación, el efecto del rendimiento de la reductora no es muy relevante, así que no merece la pena todo el trabajo de diseñar dos controladores e implementar en el microcontrolador la detección del modo de funcionamiento. El controlador LQR es bastante robusto frente a errores en los parámetros, y se puede comprobar que las ganancias en modo motor y en modo generador son muy parecidas, así que se podrán utilizar los parámetros originales sin problema o, si se quiere afinar algo más, un valor promedio entre modo motor y modo generador.


```python
import sympy as sm
```

```python
sm.init_printing()
```

# Symbols and Functions

```python
a, b, th, gamma, x, t, y, z = sm.symbols('a, b, theta, gamma, x, t, y, z')
```

```python
%whos
```

```python
a, b, th, gamma, x, t, y, z
```

```python
f = sm.Function('f')
```

```python
f(t)
```

```python
f(x, y, z)
```

```python
a1, a2, a3 = sm.symbols('a1, a2, a3')
```

```python
a1, a2, a3
```

# Expressions

```python
expr1 = a + b - x
expr1
```

```python
expr2 = f(t) + 2*f(x, y, z) + a/b
expr2
```

```python
expr3 = sm.sin(f(t)) - sm.tan(a/b)/sm.log(gamma)
expr3
```

# Printing

```python
print(expr3)
```

```python
repr(expr3)
```

```python
sm.srepr(expr1)
```

```python
sm.pprint(expr3)
```

```python
print(sm.latex(expr3))
```

```python
sm.ccode(expr1)
```

```python
print(sm.octave_code(expr3))
```

# Derivatives

```python
expr3
```

```python
sm.diff(expr3, a)
```

```python
part1 = sm.diff(expr3, a)
```

```python
part2 = sm.diff(part1, b)
```

```python
part2
```

```python
expr3.diff(a)
```

```python
expr3.diff(t)
```

```python
expr3.diff(t, 2)
```

```python
expr3.diff(t).diff(t)
```

# Numerical Evaluation

```python
expr1
```

```python
repl = {a: 5, b: -38, x: 102}
repl
```

```python
expr1.subs(repl)
```

```python
expr1.xreplace(repl)
```

```python
type(expr1.subs(repl))
```

```python
type(-135)
```

```python
type(int(expr1.subs(repl)))
```

```python
expr4 = sm.pi/4 + sm.sin(x*y)
expr4
```

```python
expr4.xreplace({x: 12, y: 24})
```

```python
expr4.evalf()
```

```python
expr4.evalf(subs={x: 12, y:24})
```

```python
type(expr4.evalf(subs={x: 12, y:24}))
```

```python
type(float(expr4.evalf(subs={x: 12, y:24})))
```

```python
expr4.evalf(subs={x: 12, y:24}, n=1000)
```

```python
expr1
```

```python
eval_expr1 = sm.lambdify((a, b, x), expr1)
```

```python
eval_expr1(12.0, 34.3, -2.0)
```

```python
type(eval_expr1(12.0, 34.3, -2.0))
```

# Matrices & Linear Algebra

```python
mat1 = sm.Matrix([[1, 2], [3, 4]])
mat1
```

```python
mat1.shape
```

```python
mat1.det()
```

```python
mat2 = sm.Matrix([[expr1, expr2], [expr3, expr4]])
mat2
```

```python
mat2.diff(t)
```

```python
mat1 + mat2
```

```python
mat1 * mat2
```

```python
sm.hadamard_product(mat1, mat2)
```

```python
mat1**2
```

```python
mat1 * mat1
```

```python
sm.eye(5)
```

```python
sm.zeros(2,4)
```

# Linear systems

```python
lin_expr_1 = a*x + b**2*y + sm.sin(gamma)*z
lin_expr_1
```

```python
lin_expr_2 = sm.sin(f(t))*x + sm.log(f(t))*z
lin_expr_2
```

```python
sm.Eq(lin_expr_1, 0)
```

```python
sm.Eq(lin_expr_2, 0)
```

```python
res = sm.solve([lin_expr_1, lin_expr_2], x, z, dict=True)
res
```

```python
res_dict = res[0]
res_dict
```

```python
sm.Eq(x, res_dict[x])
```

```python
lin_mat_exprs = sm.Matrix([lin_expr_1, lin_expr_2])
lin_mat_exprs
```

```python
A = lin_mat_exprs.jacobian([x, z])
A
```

```python
b = -lin_mat_exprs.xreplace({x: 0, z: 0})
b
```

```python
A.LUsolve(b)
```

# Simplification

```python
sm.simplify(A.LUsolve(b))
```

```python
sm.cos(gamma)**2 + sm.sin(gamma)**2
```

```python
sm.trigsimp(sm.cos(gamma)**2 + sm.sin(gamma)**2)
```

```python
sub_exprs, simp_expr = sm.cse(A.LUsolve(b).diff(t))
```

```python
simp_expr
```

```python
sub_exprs
```


# Contravariant & Covariant indices in Tensors (Symbolic)

```python
import sympy
from einsteinpy.symbolic import ChristoffelSymbols, RiemannCurvatureTensor
from einsteinpy.symbolic.predefined import Schwarzschild

sympy.init_printing()
```

### Analysing the schwarzschild metric along with performing various operations

```python
sch = Schwarzschild()
sch.tensor()
```

```python
sch_inv = sch.inv()
sch_inv.tensor()
```

```python
sch.order
```

```python
sch.config
```

    'll'

### Obtaining Christoffel Symbols from Metric Tensor

```python
chr = ChristoffelSymbols.from_metric(sch_inv) # can be initialized from sch also
chr.tensor()
```

```python
chr.config
```

    'ull'

### Changing the first index to covariant

```python
new_chr = chr.change_config('lll') # changing the configuration to (covariant, covariant, covariant)
new_chr.tensor()
```

```python
new_chr.config
```

    'lll'

### Any arbitary index configuration would also work!

```python
new_chr2 = new_chr.change_config('lul')
new_chr2.tensor()
```

### Obtaining Riemann Tensor from Christoffel Symbols and manipulating it's indices

```python
rm = RiemannCurvatureTensor.from_christoffels(new_chr2)
rm[0,0,:,:]
```

```python
rm.config
```

    'ulll'

```python
rm2 = rm.change_config("uuuu")
rm2[0,0,:,:]
```

```python
rm3 = rm2.change_config("lulu")
rm3[0,0,:,:]
```

```python
rm4 = rm3.change_config("ulll")
rm4.simplify()
rm4[0,0,:,:]
```

#### It is seen that `rm` and `rm4` are same as they have the same configuration

docs/source/examples/Playing with Contravariant and Covariant Indices in Tensors(Symbolic).ipynb


# 15.3. Analyzing real-valued functions

```python
from sympy import *
init_printing()
```

```python
var('x z')
```

```python
f = 1 / (1 + x**2)
```

```python
f.subs(x, 1)
```

```python
diff(f, x)
```

```python
limit(f, x, oo)
```

```python
series(f, x0=0, n=9)
```

```python
integrate(f, (x, -oo, oo))
```

```python
integrate(f, x)
```

```python
fourier_transform(f, x, z)
```


## Visualizing a space curve

**Vector Functions on  Intervals**

When a particle moves through space during a time interval $I$, we think of the particle's
coordinates as functions defined on $I$:

\begin{equation}
   x = f(t), \ y = g(t),\ z = h(t) \ \ \ \mbox{for}\ \ t\in I = [a, b].
\end{equation}

  The points $(x, y, z) = (f(t), g(t), h(t))$ make up the curve in space that we call a
   **particle's path**.

A curve in space can also be represented in *vector form*:
\begin{equation}
  \vec{r}(t)  =   f(t)\vec{i}+g(t)\vec{j}+h(t)\vec{k} =  <f(t), g(t), h(t)>
 \end{equation}
where $\vec{i}, \vec{j}$, and $\vec{k}$ are *standard unit vectors*.

Equation (2) defines $\vec{r}$ as a vector function of the real variable $t$
on the interval $I$. More
generally, a **vector-valued function** or **vector function** on a domain set $D$
is a rule that
assigns a vector in space to each element in $D$. For now, the domains will be intervals of
real numbers resulting in a *space curve*. Domains can be regions
in the plane. Vector functions will then represent surfaces in space.
Vector functions on a
domain in the plane or space also give rise to *vector fields*,
which are important to the
study of the flow of a fluid, gravitational fields, and electromagnetic phenomena.

We want to visualize vectors functions on intervals using a python package `Sympy`.

```python
# Import necessary packages
%matplotlib inline
from sympy import symbols, cos, sin
from sympy.plotting import plot3d_parametric_line as ppl3
from sympy.plotting import plot_parametric as ppl
import numpy as np

t = symbols('t')
```

**Example 1** Visualize $\vec{r}(t) = <t^2-3t, t+2>$ for $t\in [-2\pi, 3\pi]$.

```python
ppl(t**2-3*t, t+2, (t, -2*np.pi, 3*np.pi))
```

**Example 2** Visualize $\vec{r}(t) = \cos t\,\vec{i} + \sin t\,\vec{j} + t\,\vec{k}$ for
$t\in [0, 6\pi]$.

```python
ppl3(cos(t), sin(t), t, (t, 0, 6*np.pi))
```

**Exercises**

Visualize vector functions.

1. $\vec{r}(t) = (\sin 3t)(\cos t)\vec{i} + (\sin 3t)(\sin t)\vec{j} +t\vec{k}$
2. $\vec{r}(t) = (\cos t)\vec{i} + (\sin t)\vec{j} + (\sin 2t)\vec{k}$
3. $\vec{r}(t) = (4+\sin 20t)(\cos t)\vec{i} + (4+\sin 20t)(sin t) + (\cos 20t)\vec{k}$

```python

```


<!-- dom:TITLE: Demo - Integration of functions -->
# Demo - Integration of functions
<!-- dom:AUTHOR: Mikael Mortensen Email:mikaem@math.uio.no at Department of Mathematics, University of Oslo. -->
<!-- Author: -->
**Mikael Mortensen** (email: `mikaem@math.uio.no`), Department of Mathematics, University of Oslo.

Date: **August 7, 2020**

**Summary.** This is a demonstration of how the Python module [shenfun](https://github.com/spectralDNS/shenfun) can be used to
integrate over 1D curves and 2D surfaces in 3D space.
We make use of
curvilinear coordinates, and reproduce some integrals
performed by Behnam Hashemi with [Chebfun](http://www.chebfun.org/examples/approx3/SurfaceIntegral3D.html).

**Notice.**

For all the examples below we could just as well
use Legendre polynomials instead of Chebyshev.
Just replace 'C' with 'L' when creating function spaces.
The accuracy ought to be similar.

## The inner product

A lesser known fact about [shenfun](https://github.com/spectralDNS/shenfun) is
that it can be used to perform regular, unweighted, integrals with
spectral accuracy. With the newly added curvilinear coordinates
feature, we can now also integrate over highly complex lines and surfaces
embedded in a higher dimensional space.

To integrate over a domain in shenfun we use the
[inner()](https://shenfun.readthedocs.io/en/latest/shenfun.forms.html#shenfun.forms.inner.inner)
function, with a constant test function. The [inner()](https://shenfun.readthedocs.io/en/latest/shenfun.forms.html#shenfun.forms.inner.inner)
function in shenfun is defined as an integral over the
entire domain $\Omega$ in question

$$
(u, v)_w = \int_{\Omega} u \overline{v} w d\Omega,
$$

for trial function $u$, test function $v$ and weight $w$.
Also, $\overline{v}$ represents the complex conjugate of $v$, in case
we are working with complex functions (like Fourier exponentials).

The functions and weights take on different form, but if
the test function $v$ is chosen to be a constant, e.g., $v=1$,
then the weight is also constant, $w=1$, and the inner product becomes
an unweighted integral of $u$ over the domain

$$
(u, 1)_w = \int_{\Omega} u d\Omega
$$

### Curve integrals

For example, if we create some function space on the line from
0 to 1, then we can get the length of this domain using `inner`

```
from shenfun import *
B = FunctionSpace(10, 'C', domain=(0, 1))
u = Array(B, val=1)
length = inner(u, 1)
print('Length of domain =', length)
```

Note that we cannot simply do `inner(1, 1)`, because the
`inner` function does not know about the domain, which is part
of the [FunctionSpace](https://shenfun.readthedocs.io/en/latest/shenfun.forms.html#shenfun.forms.arguments.FunctionSpace). So to integrate `u=1`, we need to
create `u` as an [Array](https://shenfun.readthedocs.io/en/latest/shenfun.forms.html#shenfun.forms.arguments.Array) with the constant value 1.

Since the function space `B` is Cartesian the computed
length is simply the domain length.
Not very impressive, but the same goes for multidimensional
tensor product domains

```
F = FunctionSpace(10, 'F', domain=(0, 2*np.pi))
T = TensorProductSpace(comm, (B, F))
area = inner(1, Array(T, val=1))
print('Area of domain =', area)
```

Still not very impressive, but moving to curvilinear coordinates
it all starts to become more interesting. Lets
look at a spiral $C$ embedded in $\mathbb{R}^3$, parametrized
by one single parameter $t$

$$
\begin{align*}
x(t) &= \sin 2t \\
y(t) &= \cos 2t \\
z(t) &= \frac{t}{2} \\
0 \le & t \le 2\pi
\end{align*}
$$

What is the length of this spiral? The spiral can be
seen as the red curve in the figure a few cells below.

The integral over the parametrized curve $C$ can
be written as

$$
\int_C ds = \int_{t=0}^{2\pi} \sqrt{\left(\frac{d x}{d t}\right)^2 + \left(\frac{d y}{d t}\right)^2 + \left(\frac{d z}{d t}\right)^2} dt.
$$

We can find this integral easily using shenfun. Create
a function space in curvilinear coordinates, providing
the position vector $\mathbf{r} = x(t)\mathbf{i} + y(t) \mathbf{j} + z(t) \mathbf{k}$
as input. Also, choose to work with covariant basis vectors, which
is really not important unless you work with vector equations. The
alternative is the default 'normal', where the basis vectors
are normalized to unit length.

```
import sympy as sp
from shenfun import *
config['basisvectors'] = 'covariant'
t = sp.Symbol('x', real=True, positive=True)
rv = (sp.sin(2*t), sp.cos(2*t), 0.5*t)
C = FunctionSpace(100, 'C', domain=(0, 2*np.pi), coordinates=((t,), rv))
```

Then compute the arclength using [inner()](https://shenfun.readthedocs.io/en/latest/shenfun.forms.html#shenfun.forms.inner.inner), again by using a constant
testfunction 1, and a constant [Array](https://shenfun.readthedocs.io/en/latest/shenfun.forms.html#shenfun.forms.arguments.Array) `u=1`

```
length = inner(1, Array(C, val=1))
print('Length of spiral =', length)
```

The arclength is found to be slightly longer than $4 \pi$. Looking at the
spiral below, the result looks reasonable.

```
%matplotlib inline

import matplotlib.pyplot as plt
fig = plt.figure(figsize=(4, 3))
X = C.cartesian_mesh(uniform=True)
ax = fig.add_subplot(111, projection='3d')
p = ax.plot(X[0], X[1], X[2], 'r')
hx = ax.set_xticks(np.linspace(-1, 1, 5))
hy = ax.set_yticks(np.linspace(-1, 1, 5))
```

The term $\sqrt{\left(\frac{d x}{d t}\right)^2 + \left(\frac{d y}{d t}\right)^2 + \left(\frac{d z}{d t}\right)^2}$
is actually here a constant $\sqrt{4.25}$, found in shenfun as

```
C.coors.sg
```

We could also integrate a non-constant function over the spiral.
For example, lets integrate the function $f(x, y, z)= \sin^2 x$

$$
\int_C \sin^2 x ds = \int_{t=0}^{2\pi} \sin^2 (\sin 2t) \sqrt{\left(\frac{d x}{d t}\right)^2 + \left(\frac{d y}{d t}\right)^2 + \left(\frac{d z}{d t}\right)^2} dt
$$

```
inner(1, Array(C, buffer=sp.sin(rv[0])**2))
```

which can be easily verified using, e.g., Wolfram Alpha

```
from IPython.display import IFrame
IFrame("https://www.wolframalpha.com/input/?i=integrate+sin%5E2%28sin%282t%29%29+sqrt%284.25%29+from+t%3D0+to+2pi", width="500px", height="350px")
```

### Surface integrals

Consider a 3D function $f(x,y,z) \in \mathbb{R}^3$ and
a 2D surface (not neccessarily plane) $S(u, v)$,
parametrized in two new coordinates $u$ and $v$. A position
vector $\mathbf{r}$ can be used to parametrize $S$

$$
\mathbf{r} = x(u, v) \,\mathbf{i} + y(u, v) \,\mathbf{j} + z(u, v) \,\mathbf{k},
$$

where $\mathbf{i}, \mathbf{j}, \mathbf{k}$ are the Cartesian unit vectors.
The two new coordinates $u$ and $v$ are functions of $x, y, z$,
and they each have a one-dimensional domain

$$
u \in D_u \quad v \in D_v.
$$

The exact size of the domain depends on the problem at hand. The computational
domain of the surface $S$ is $D=D_u \times D_v$.

A surface integral of $f$ over $S$ can now be written

$$
\int_S f(x, y, z) dS = \int_D f(x(u, v), y(u, v), z(u, v)) \left|\frac{\partial \mathbf{r}}{\partial u} \times \frac{\partial \mathbf{r}}{\partial v} \right| dudv,
$$

where $dS$ is a surface area element. With shenfun such integrals
are trivial, even for highly complex domains.

## Example 1

Consider first the surface integral of $f(x,y,z)=x^2$
over the unit sphere. We use regular spherical coordinates,

$$
\begin{align*}
0 &\le \theta \le \pi \\
0 &\le \phi \le 2\pi \\
x(\theta, \phi) &= \sin \theta \cos \phi \\
y(\theta, \phi) &= \sin \theta \sin \phi \\
z(\theta, \phi) &= \cos \theta
\end{align*}
$$

The straight forward implementation of a function space for
the unit sphere reads

```
import sympy as sp

theta, phi = psi =sp.symbols('x,y', real=True, positive=True)
rv = (sp.sin(theta)*sp.cos(phi), sp.sin(theta)*sp.sin(phi), sp.cos(theta))

B0 = FunctionSpace(0, 'C', domain=(0, np.pi))
B1 = FunctionSpace(0, 'F', dtype='d')
T = TensorProductSpace(comm, (B0, B1), coordinates=(psi, rv, sp.Q.positive(sp.sin(theta))))
```

where `sp.Q.positive(sp.sin(theta))` is a restriction that
helps `Sympy` in computing the Jacobian required for the integral.
We can now approximate the function $f$ on this surface

```
f = Array(T, buffer=rv[0]**2)
```

and we can integrate over $S$

```
I = inner(1, f)
```

and finally compare to the exact result, which is $4 \pi / 3$

```
print('Error =', abs(I-4*np.pi/3))
```

Note that we can here achieve better accuracy by using
more quadrature points. For example by refining `f`

```
T = T.get_refined(2*np.array(f.global_shape))
f = Array(T, buffer=rv[0]**2)
print('Error =', abs(inner(1, f)-4*np.pi/3))
```

Not bad at all:-)

To go a little deeper into the integral, we can get the
term $\left|\frac{\partial \mathbf{r}}{\partial u} \times \frac{\partial \mathbf{r}}{\partial v} \right|$
as

```
print(T.coors.sg)
```

Here the printed variable is `x`, but this is because `theta`
is named `x` internally by `Sympy`. This is because of the definition
used above: `theta, phi = sp.symbols('x,y', real=True, positive=True)`.

Note that $\mathbf{b}_u = \frac{\partial \mathbf{r}}{\partial u}$ and
$\mathbf{b}_v = \frac{\partial \mathbf{r}}{\partial v}$ are the two
basis vectors used by shenfun for the surface $S$. The basis
vectors are obtainable as `T.coors.b`, and can also be printed
in latex using:

```
from IPython.display import Math
Math(T.coors.latex_basis_vectors(symbol_names={theta: '\\theta', phi: '\\phi'}))
```

where we tell latex to print `theta` as $\theta$, and not `x`:-)

From the basis vectors it should be easy to see that $\left| \mathbf{b}_{\theta} \times \mathbf{b}_{\phi} \right| = \sin \theta$.

## Example 2

Next, we solve [Example 5](http://www.math24.net/surface-integrals-of-first-kind.html)
from the online resources at math24.net. Here

$$
f = \sqrt{1+x^2+y^2}
$$

and the surface is defined by

$$
\mathbf{r} = u \cos v \mathbf{i} + u \sin v \mathbf{j} + v \mathbf{k}
$$

with $0 \le u \le 2, 0 \le v \le 2\pi$.

The implementation is only a few lines, and we end by comparing
to the exact solution $14 \pi /3$

```
u, v = psi =sp.symbols('x,y', real=True, positive=True)
rv = (u*sp.cos(v), u*sp.sin(v), v)
B0 = FunctionSpace(0, 'C', domain=(0, 2))
B1 = FunctionSpace(0, 'C', domain=(0, np.pi))
T = TensorProductSpace(comm, (B0, B1), coordinates=(psi, rv))
f = Array(T, buffer=sp.sqrt(1+rv[0]**2+rv[1]**2))
print('Error =', abs(inner(1, f)-14*np.pi/3))
```

In this case the integral measure is

```
print(T.coors.sg)
```

## Example 3

In this third example we use a surface that
looks like a seashell. Again, the example is taken from
[chebfun](http://www.chebfun.org/examples/approx3/SurfaceIntegral3D.html).

The surface of the seashell is parametrized with position
vector

$$
\begin{align*}
\mathbf{r} &= \left(\left(\frac{5}{4}-\frac{5 v}{8 \pi}\right)  \cos 2v(1+\cos u) + \cos 2v \right) \mathbf{i} \\
  &+\left(\left(\frac{5}{4}-\frac{5 v}{8 \pi}\right) \sin 2v (1+\cos u) + \sin 2v \right) \mathbf{j},\\
  &+\left(\frac{10 v}{2 \pi} + \left(\frac{5}{4}-\frac{5 v}{8 \pi}\right) \sin u + 15\right) \mathbf{k}
\end{align*}
$$

for $0 \le u \le 2 \pi, -2 \pi \le v \le 2 \pi$.

The function $f$ is now defined as

$$
f(x,y,z) = x+y+z
$$

The implementation is

```
rv = (5*(1-v/(2*sp.pi))*sp.cos(2*v)*(1+sp.cos(u))/4 + sp.cos(2*v),
      5*(1-v/(2*sp.pi))*sp.sin(2*v)*(1+sp.cos(u))/4 + sp.sin(2*v),
      10*v/(2*sp.pi) + 5*(1-v/(2*sp.pi))*sp.sin(u)/4 + 15)

B0 = FunctionSpace(100, 'C', domain=(0, 2*np.pi))
B1 = FunctionSpace(100, 'C', domain=(-2*np.pi, 2*np.pi))
T = TensorProductSpace(comm, (B0, B1), coordinates=(psi, rv, sp.Q.positive(v-2*sp.pi)))

f = rv[0]+rv[1]+rv[2]
fb = Array(T, buffer=f)
I = inner(1, fb)
print(I)
```

which agrees very well with chebfun's result. The basis vectors
for the surface of the seashell are

```
Math(T.coors.latex_basis_vectors(symbol_names={u: 'u', v: 'v'}))
```

which, if nothing else, shows the power of symbolic
computing in Sympy.

We can plot the
seashell using either plotly or mayavi. Here we choose
plotly since it integrates well with the executable
jupyter book.

```
import plotly
fig = surf3D(fb, colorscale=plotly.colors.sequential.Jet)
fig.update_layout(scene_camera_eye=dict(x=1.6, y=-1.4, z=0))
fig.show()
```

<!-- ======= Bibliography ======= -->


sergazy.nurbavliyev@gmail.com © 2021

## Which game would you choose

Question: Assume you and your friend decided to play the game in a casino. There are two games that you can play.
Game 1. You roll two uniform fair dice. You can get the amount of dollars that is equivalent to the product of the numbers shown on both dies.
Game 2. You roll one uniform fair die. You will get the dollar amount of the square of the value shown on the die. Which game will you choose? To be more clear: Which game has the higher expected value?

## Answer

Without calculating the real expected values. (This part is left to the reader.) We can stil answer the question.

Let $X$ be the random variable that shows the outcome of the uniform fair die. Then the game 1 is asking to find $\mathbb{E}[X]*\mathbb{E}[X]$. In other words, the product of the expectation of the two independent rolls. However the game 2 is asking to find $\mathbb{E}[X^2]$. In other words the expectation of the square of the single roll.  Remember the definition of the variance. We know it is always non-negative. That is
$Var(X)\geq 0$.
\begin{equation}
Var(X)=\mathbb{E}[X^2]-\mathbb{E}[X]^2\geq 0
\end{equation}
This implies that $\mathbb{E}[X^2]\geq \mathbb{E}[X]^2$. Indeed, $\mathbb{E}[X^2]> \mathbb{E}[X]^2$ unless the two games are exactly the same. This implies that the second game has a higher expected value and we should choose that one.

```python

```

```python

```

## Python code for exact value

```python
import sympy as S
from sympy.stats import E, Die,variance
x=Die('D1',6)
y=Die('D2',6)
```

```python
E(x),E(x**2),variance(x)
```

    (7/2, 91/6, 35/12)

```python
E(y),E(y**2),variance(y)
```

    (7/2, 91/6, 35/12)

```python
z =x*y
```

```python
E(z),E(z**2),variance(z)
```

    (49/4, 8281/36, 11515/144)

```python
E(z)**2<E(z**2)
```

$\displaystyle \text{True}$

```python

```

```python

```

Which game would you choose March 9 2021.ipynb


# Занятие 5
# Прикладная алгебра и численные методы
## Интерполяция: полиномы Лагранжа, сплайны и кривые Безье

```
import numpy as np
import scipy.linalg
import sympy
from sympy import S
import matplotlib.pyplot as plt
from copy import deepcopy
%matplotlib inline
```

## Многочлен Лагранжа
$$
f(x) = L(x) = \sum_{i=1}^n y_i
\frac{(x - x_0)(x - x_1)...(x - x_{i-1})(x - x_{i+1})...(x - x_n)}{(x_i - x_0)(x_i - x_1)...(x_i - x_{i-1})(x_i - x_{i+1})...(x_i - x_n)},
\quad y_i = f(x_i), \quad i = 0, ..., n.
$$
## Пример 1
Построить параболу по трем точкам (-1, -2), (0, -1), (1, 2).

```
x = S('x')
X = (-1, 0, 1)
Y = (-2, -1, 2)
L = 0
for i in range(3):
    Li = Y[i]
    for j in range(3):
        if i != j:
            Li *= (x - X[j])/(X[i] - X[j])
    L += Li
display(L, sympy.simplify(sympy.expand(L)))
```

$\displaystyle 2 x \left(\frac{1}{2} - \frac{x}{2}\right) + x \left(x + 1\right) + \left(1 - x\right) \left(- x - 1\right)$

$\displaystyle x^{2} + 2 x - 1$

## Пример 2 Линейная регрессия
Для данных Примера 1 построим параболу методом линейной регрессии (исключительно для тренировки и повторения, в жизни так не делают!)

Составим матрицу  из $A$ из значений $x$ и $x^2$ и матрицу $Y$, далее все как обычно - псевдообратная матрица и произведение псевдообратной к  $A$ и  $Y$:

```
X2 = [ -1, 0, 1]
A2 = np.array([[1, item, item**2] for item in X2])
Y2 = np.array([[-2], [-1], [2]])
res2 = np.matmul(np.linalg.pinv(A2), Y2)
a2, b2, c2 = [round(item, 2) for item in  res2[:, 0]]
display('A', A2,'Y', Y)
print('a =', a2, 'b =', b2, 'c =', c2)
```

    'A'

    array([[ 1, -1,  1],
           [ 1,  0,  0],
           [ 1,  1,  1]])

    'Y'

    (-2, -1, 2)

    a = -1.0 b = 2.0 c = 1.0

Изобразим на графике точки исходных данных и полученную линию регрессии:

```
X_ls2 = np.linspace(X2[0], X2[-1])
plt.plot(X_ls2, a2 + b2*X_ls2 + c2*X_ls2**2, label='$y = {a} + {b}x + {c}x^2$'.format(a=round(a2),
                                                                                b=round(b2), c=round(c2)))
plt.scatter(X2, Y2, color='red', label='data')
plt.grid()
plt.legend()
```

## Пример 3 Линейная регрессия
Для данных точек построим наилучшую в смысле среднеквадратического отклонения параболу методом линейной регрессии.
$$
\begin{matrix}
X & -3 & -1 & 0 & 1 & 3\\
Y & -4 & -0.8 & 1.6 & 2.3 & 1.5
\end{matrix}
$$

```
X3 = [ -3, -1, 0, 1, 3]
A3 = np.array([[1, item, item**2] for item in X3])
Y3 = np.array([[-4], [-0.8], [1.6], [2.3], [1.5]])
res3 = np.matmul(np.linalg.pinv(A3), Y3)
a3, b3, c3 = [round(item, 2) for item in  res3[:, 0]]
print('a =', a3, 'b =', b3, 'c =', c3)
```

    a = 1.23 b = 0.98 c = -0.28

Построим график, отметим на нем исходные точки.

```
X_ls = np.linspace(X3[0], X3[-1])
plt.plot(X_ls, a3 + b3*X_ls + c3*X_ls**2)
plt.scatter(X3, Y3, color='red')
plt.grid()
```

## Пример 4
Построим полином Лагранжа для данных Примера 3:

```
x = S('x')
Y3 = [-4, -S(8)/10, S(16)/10, S(23)/10, S(15)/10]
L3 = 0
n = len(X3)
for i, Li in enumerate(Y3):
    for j in range(n):
        if i != j:
            Li *= (x - X3[j])/(X3[i] - X3[j])
    L3 += Li
L3 = sympy.simplify(sympy.expand(L3))
display(L3)
X_ls = np.linspace(X3[0], X3[-1])
Y_regr = [L3.subs(x, item) for item in X_ls]
plt.plot(X_ls, Y_regr, label=sympy.latex(sympy.Eq(S('y'), L3), mode='inline'))
plt.scatter(X3, Y3, color='red', label='data')
plt.grid()
plt.legend()
```

## Многочлен Лагранжа с scipy.interpolate.lagrange
## Пример 5
Построим полином Лагранжа для данных Примера 3 с помощью scipy.interpolate.lagrange

```
from scipy.interpolate import lagrange
from numpy.polynomial.polynomial import Polynomial
```

```
poly = lagrange(X3, Y3)
Polynomial(poly).coef
```

    array([0.0666666666666667, -0.0791666666666667, -0.916666666666667,
           1.62916666666667, 1.60000000000000], dtype=object)

Сравним с полученными ранее коэффициентами:

```
coeffs = L3.as_coefficients_dict()
res = [round(coeffs[item], 8) for item in [x**k for k in range(len(coeffs))]]
res.reverse()
res
```

    [0.06666667, -0.07916667, -0.91666667, 1.62916667, 1.60000000]

## Квадратичный сплайн

## Пример 7
Аппроксимировать $f(x) = x^3 - 6x^2 + 11 x - 6$ на отрезке $[0, 4]$ квадратичным сплайном с узлами 0, 2, 4.

```
def spl2(a0, a1, a2, x):
    return a0 + a1*x + a2*x**2
def dspl2(a0, a1, a2, x, x0):
    return spl2(a0, a1, a2, x).diff(x).subs(x, x0)
def f(x):
    return x**3 - 6*x**2 + 11*x - 6
def df(x, x0):
    return f(x).diff(x).subs(x, x0)
x0 = 0
x1 = 2
x2 = 4
f0, f1, f2 = [f(xi) for xi in (x0, x1, x2)]
df0 = df(x, x0)
a0, a1, a2, x = sympy.symbols('a0:3 x')
sys1 = [sympy.Eq(dspl2(a0, a1, a2, x, x0), df0), sympy.Eq(spl2(a0, a1, a2, x0), f0), sympy.Eq(spl2(a0, a1, a2, x1), f1)]
display(*sys1)
```

$\displaystyle a_{1} = 11$

$\displaystyle a_{0} = -6$

$\displaystyle a_{0} + 2 a_{1} + 4 a_{2} = 0$

```
res1 = sympy.solve(sys1)
spline1 = res1[a0] + res1[a1]*x + res1[a2]*x**2
display(res1, spline1)
df1 = spline1.diff(x).subs(x, x1)
sys2 = [sympy.Eq(dspl2(a0, a1, a2, x, x1), df1),
                   sympy.Eq(spl2(a0, a1, a2, x1), f1),
                   sympy.Eq(spl2(a0, a1, a2, x2), f2)]
res2 = sympy.solve(sys2)

spline2 = res2[a0] + res2[a1]*x + res2[a2]*x**2
display(*sys2, spline2)
```

    {a0: -6, a1: 11, a2: -4}

$\displaystyle - 4 x^{2} + 11 x - 6$

$\displaystyle a_{1} + 4 a_{2} = -5$

$\displaystyle a_{0} + 2 a_{1} + 4 a_{2} = 0$

$\displaystyle a_{0} + 4 a_{1} + 16 a_{2} = 6$

$\displaystyle 4 x^{2} - 21 x + 26$

```
x0, x1, x2 = (0, 2, 4)
X_ls1 = np.linspace(x0, x1)
X_ls2 = np.linspace(x1, x2)
X_ls12 = np.linspace(x0, x2)
Y1 = [spline1.subs(x, item) for item in X_ls1]
Y2 = [spline2.subs(x, item) for item in X_ls2]
plt.plot(X_ls1, Y1, color='green')
plt.plot(X_ls2, Y2, color='red')
plt.plot(X_ls12, f(X_ls12), color='black')
```

## Кубический сплайн
$$
f_i(x) = a_ix^3 + b_ix^2 + c_ix + d,
\quad
\left\{
\begin{matrix}
f_i(x_{i - 1}) = y_{i - 1}\\
f_i(x_{i}) = y_{i}\\
f_i'(x_{i - 1}) = f'_{i - 1}(x_{i - 1})\\
f_i''(x_{i - 1}) = f''_{i - 1}(x_{i - 1})
\end{matrix}
\right.
$$
## Пример 8
Аппроксимировать $f(x) = \sin(x)$ на отрезке $\left[0, \frac{2\pi}{3}\right]$ квадратичным сплайном с узлами
0, $\frac{\pi}{3}$, $\frac{2\pi}{3}$.

```
X = [np.pi*k/3 for k in range(3)]
Y = [np.sin(xk) for xk in X]
plt.scatter(X, Y)
```

Вначале определим несколько вспомогательных функций:

spl3(a0, a1, a2, a3, x) кубическая парабола с коэффициентами a0, a1, a2, a3 и аргументом x

dspl3(a1, a2, a3, x) - первая производная кубического сплайна

d2spl3(a2, a3, x) - вторая производная кубического сплайна

sys_spl(f, x, xi, xi_1, yi, yi_1, d1spl, d2spl) - система уравнений относительно коэффициентов сплайна

```
def spl3(a0, a1, a2, a3, x):
    return a0 + a1*x + a2*x**2 + a3*x**3
def dspl3(a1, a2, a3, x):
    return a1 + 2*a2*x + 3*a3*x**2
def d2spl3(a2, a3, x):
    return 2*a2 + 6*a3*x
def sys_spl(f, x, xi, xi_1, yi, yi_1, d1spl, d2spl):
    return [sympy.Eq(spl3(a0, a1, a2, a3, xi_1), yi_1),
            sympy.Eq(spl3(a0, a1, a2, a3, xi), yi),
            sympy.Eq(dspl3(a1, a2, a3, xi_1), d1spl),
            sympy.Eq(d2spl3(a2, a3, xi_1), d2spl)]
```

Нам понадобится $\sin(x)$ из numpy для построения графика и из sympy для получения сплайна (аналитически), поэтому опишем функцию fx(x, lib='sympy'), в которой по умолчанию $\sin(x)$ берется из из sympy, но можно изменить значение по умолчанию параметра lib на 'numpy'.

```
def fx(x, lib='sympy'):
    if lib == 'sympy':
        return sympy.sin(x)
    if lib == 'numpy':
        return np.sin(x)
```

Введем переменные в математическом смысле - символы a0, a1, a2, a3, x.

Обозначим xi_1 горизонтальную координату первой точки, xi - второй, yi_1 и yi - соответствующие вертикальные координаты,
d1spl и d2spl - значения первой и второй производной на левом конце отрезка, т.е. в xi_1, они для самого левого отрезка равны производным самой функции в этой точке.

Для построения сплайна воспользуемся решением с помощью solve соответствующей системы уравнений относительно коэффициентов сплайна.

Полученные значения коэффициентов используем для получения выражения (expression) для сплайна на левом отрезке spline3

```
a0, a1, a2, a3, x = sympy.symbols('a0:4 x')
xi_1 = X[0]
xi = X[1]
yi_1 = fx(xi_1)
yi = fx(xi)
d1spl = fx(x).diff(x).subs(x, xi_1)
d2spl = fx(x).diff(x, 2).subs(x, xi_1)
sys3 = sys_spl(fx, x, xi, xi_1, yi, yi_1, d1spl, d2spl)
display(*sys3)
res3 = sympy.solve(sys3)
spline3 = res3[a0] + res3[a1]*x + res3[a2]*x**2 + res3[a3]*x**3
display(spline3)
```

$\displaystyle a_{0} = 0$

$\displaystyle a_{0} + 1.0471975511966 a_{1} + 1.09662271123215 a_{2} + 1.14838061778888 a_{3} = 0.866025403784439$

$\displaystyle a_{1} = 1$

$\displaystyle 2 a_{2} = 0$

$\displaystyle - 0.1577631532662 x^{3} + 1.0 x$

Повторим те же действия со следующим отрезком, т.е. $\left[\frac{\pi}{3}, \frac{2\pi}{3}\right]$.

Отличие в том, что производные первого и второго порядков считаются не по исходной функции, а по построенному куску сплайна, т.е. считаем производные от  spline3 в точке $\frac{\pi}{3}$.

```
xi_1 = X[1]
xi = X[2]
yi_1 = fx(xi_1)
yi = fx(xi)
d1spl = spline3.diff(x).subs(x, xi_1)
d2spl = spline3.diff(x, 2).subs(x, xi_1)
sys4 = sys_spl(fx, x, xi, xi_1, yi, yi_1, d1spl, d2spl)
display(*sys4)
res4 = sympy.solve(sys4)
spline4 = res4[a0] + res4[a1]*x + res4[a2]*x**2 + res4[a3]*x**3
display(spline4)
```

$\displaystyle a_{0} + 1.0471975511966 a_{1} + 1.09662271123215 a_{2} + 1.14838061778888 a_{3} = 0.866025403784439$

$\displaystyle a_{0} + 2.0943951023932 a_{1} + 4.3864908449286 a_{2} + 9.18704494231105 a_{3} = 0.866025403784439$

$\displaystyle a_{1} + 2.0943951023932 a_{2} + 3.28986813369645 a_{3} = 0.480980029398058$

$\displaystyle 2 a_{2} + 6.28318530717959 a_{3} = -0.991255126616512$

$\displaystyle 0.0346882668161555 x^{3} - 0.604603967503653 x^{2} + 1.63313979421358 x - 0.221007480688529$

Построим на графике исходную функцию и весь сплайн (из двух кусков)

```
x0, x1, x2 = X
X_ls1 = np.linspace(x0, x1)
X_ls2 = np.linspace(x1, x2)
X_ls12 = np.linspace(x0, x2)
Y1 = [spline3.subs(x, item) for item in X_ls1]
Y2 = [spline4.subs(x, item) for item in X_ls2]
plt.plot(X_ls1, Y1, color='green', label='spline1')
plt.plot(X_ls2, Y2, color='red', label='spline2')
plt.plot(X_ls12, fx(X_ls12, lib='numpy'), color='black', linestyle=':', label='sin(x)')
plt.legend()
```

## Сплайны scipy.interpolate: interp1d, splrep, InterpolatedUnivariateSpline
https://docs.scipy.org/doc/scipy/reference/interpolate.html#module-scipy.interpolate
#### Кубический сплайн:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.CubicSpline.html#scipy.interpolate.CubicSpline
####  interp1d
https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html#scipy.interpolate.interp1d
####  Полиномы Бернштейна
https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.BPoly.html#scipy.interpolate.BPoly

Теперь будем строить сплайны средствами scipy.interpolate, вначале подключим необходимые функции:

```
from scipy.interpolate import interp1d, splrep, splev, InterpolatedUnivariateSpline, BPoly, CubicSpline
```

## interp1d
Параметры $x$ и  $y$ - координаты узлов сплайна,
kind - str или int, необязательный параметр, определяющий тип сплайна, по умолчанию линейный (‘linear’), можно сделать ‘nearest’, ‘nearest-up’, ‘zero’, ‘slinear’, ‘quadratic’, ‘cubic’, ‘previous’ или ‘next’.

‘zero’, ‘slinear’, ‘quadratic’ и ‘cubic’ обозначают интерполяцию нулевого, первого, второго или третьего порядка,

‘previous’ и ‘next’ просто возвращают предыдущее или следующее значение функции

‘nearest-up’ и ‘nearest’ различаются обработкой полуцелых чисел, таких как 0.5, 1.5.

## Пример 9
Аппроксимировать $f(x) = \sin(x)$ на отрезке $\left[0, 2\pi\right]$ квадратичным сплайном с узлами
$\frac{\pi k}{4}$, $k = 0, ..., 8$.

```
X = [np.pi*k/4 for k in range(9)]
Y = [np.sin(xk) for xk in X]
plt.scatter(X, Y, color='red')
spl1 = interp1d(X, Y)
spl2 = interp1d(X, Y, kind='cubic')
xs = np.linspace(X[0], X[-1], 1000)
plt.plot(xs, np.sin(xs), 'c-', xs, spl1(xs), 'g:', xs, spl2(xs), 'm--', lw=3)
```

## Еще один способ построения сплайна:

splrep функция, которая возвращает представление  B-spline кривой

splev вычисляет значения B-spline или его производных

Есть еще полезные функции:

sproot находит корни кубического B-spline

splint вычисляет определенный интеграл  B-spline на отрезке

spalde вычисляет все производные B-spline

splrep, splev работают на основе FITPACK (написан на Фортране)

## Пример 10.
Построим сплайн для Примера 7 с помощью splrep и splev.

```
X = [np.pi*k/4 for k in range(9)]
Y = [np.sin(xk) for xk in X]
tck = splrep(X, Y, s=0)
xnew = np.linspace(X[0], X[-1], 1000)
ynew = splev(xnew, tck, der=0)
plt.plot(X, Y, 'ro', xnew, ynew, 'c--', xnew, np.sin(xnew), 'k:', lw=3)
plt.legend(['Linear', 'Cubic Spline', 'sin(x)'])
```

## InterpolatedUnivariateSpline
тоже работает на FITPACK.
## Пример 11.
Построим сплайн для Примера 7 с помощью InterpolatedUnivariateSpline (объектно-ориентированная обертка для функций FITPACK).

```
X = [np.pi*k/4 for k in range(9)]
Y = [np.sin(xk) for xk in X]
spl = InterpolatedUnivariateSpline(X, Y)
xs = np.linspace(X[0], X[-1], 1000)
plt.scatter(X, Y, color='red')
plt.plot(xs, np.sin(xs), 'c-', xs, spl(xs), 'g--', lw=3)
```

## Пример 12
Построим сплайн для Примера 7 с помощью CubicSpline

```
X = [np.pi*k/4 for k in range(9)]
Y = [np.sin(xk) for xk in X]
cs = CubicSpline(X, Y)
xs = np.linspace(X[0], X[-1], 1000)
plt.scatter(X, Y, color='red')
plt.plot(xs, np.sin(xs), 'c-', xs, cs(xs), 'g--', lw=3)
```

## Кривые Безье
#### Линейная кривая Безье
$P_0$ и $P_1$ две различные точки, тогда линейная кривая Безье - отрезок, соединяющий эти точки:
$$
B(t) = P_0 + t(P_1 - P_0) = (1 - t)P_0 + tP_1,\quad 1\le t \le 1
$$
#### Квадратичная кривая Безье
$P_0$, $P_1$ и $P_2$ три различные точки, тогда квадратичная кривая Безье - отрезок, соединяющий эти точки:
$$
B(t) = (1 - t)^2P_0 + 2(1 - t)tP_1 + t^2P_2 = P_1 + (1 - t)^2(P_0 - P_1) + t^2(P_2 - P_1),\quad 1\le t \le 1
$$
#### Кубическая кривая Безье
$P_0$, $P_1$, $P_2$ и $P_3$ четыре различные точки, тогда кубическая кривая Безье - отрезок, соединяющий эти точки:
$$
B(t) = (1 - t)^3P_0 + 3(1 - t)^2tP_1 + 3(1 - t)t^2P_2 +t^3P_3,\quad 1\le t \le 1
$$
#### Общий вид кривой Безье
$P_0$, $P_1$, ...  и $P_n$ - $n$ различных точек, тогда линейная кривая Безье - отрезок, соединяющий эти точки:
$$
B(t) = \sum_{k=0}^n C_n^k(1 - t)^{n - k}t^kP_k,\quad 1\le t \le 1
$$

https://bezier.readthedocs.io/en/stable/python/reference/bezier.curve.html

## Пример 13
Построим кривую Безье по точкам $P_1(-1, 3)$, $P_2(0, 4)$, $P_3(3, 2)$.

Для каждой точки построим полином Бернштейна $C_n^k(1 - t)^{n - k}t^k$, $1\le t \le 1$ и составим сумму произведений соответствующих координат точки на полиномы  Бернштейна.

```
from scipy.special import comb
def my_B(X, Y):
    n = len(X)
    def my_B_x(t):
        return sum([X[k]*comb(n - 1, k)*(1 - t)**(n - k - 1)*t**k for k in range(n)])
    def my_B_y(t):
        return sum([Y[k]*comb(n - 1, k)*(1 - t)**(n - k - 1)*t**k for k in range(n)])
    return (my_B_x, my_B_y)
```

```
P1 = (-1, 3)
P2 = (0, 4)
P3 = (3, 2)
X = []
Y = []
for point in (P1, P2, P3):
    X.append(point[0])
    Y.append(point[1])
B_x, B_y = my_B(X, Y)
X1 = np.array(X).reshape(3, 1)
Y1 = np.array(Y).reshape(3, 1)
x = [0, 1]
bpX = BPoly(X1, x)
bpY = BPoly(Y1, x)
t_linspace = np.linspace(0, 1)
plt.plot(bpX(t_linspace), bpY(t_linspace), 'c-', B_x(t_linspace), B_y(t_linspace), 'r:', lw=3)
plt.scatter(X, Y)
```

## Пример 14
Построим кривую Безье с помощью модуля bezier:

https://bezier.readthedocs.io/en/stable/python/reference/bezier.curve.html

```

```

```
import bezier
```

```
nodes = np.asfortranarray([
...     [0.0, 0.625, 1.0],
...     [0.0, 0.5  , 0.5],
... ])
>>> curve = bezier.Curve(nodes, degree=2)
>>> curve
```


# __Fundamentos de programación__

Hecho por: Juan David Argüello Plata

## __1. Variables__

Una variable es el nombre con el que se identifica información de interés.

```
nom_variable = contenido
```

El contenido de una variable puede cambiar de naturaleza; por eso se dice que Python es un lenguaje dinámico.

### __1.1. Naturaleza de las variables__

| Naturaleza | Ejemplo  |
|----------|---|
| Numérico | `x = 5`  |
| Textual  |  `text = 'Esta es una frase'` |
| Lista    | `lista = [0,1,2,"texto"]`  |
| Tupla | `tupla = (0,1,2,"texto")` |
| Diccionario | `dic = {"num":5, "text": "hola"}`  |

### __1.2. Variable numérica__

La forma en como se define una variable numérica y el tipo de operaciones básicas que se pueden emplear con ellas se muestra a continuación.

```python
#Declarar una variable numérica es igual que en el álgebra...
x = 1
print(x)
```

```python
x = 5
w = 10
z = 20
print("x = ", x, ", w = ", w, ", z = ", z)  #Podemos ser más específicos a la hora de imprimir información
```

También se pueden hacer operaciones matemáticas, pero _cuidado_: es importante escribir bien las ecuaciones.

Si se quisiera resolver:

$$
\begin{equation}
  y = \frac{x}{w \, z}
\end{equation}
$$

Se debe escribir el algoritmo así:

```python
y = x/(w*z)
print(y)
```

Porque si se escribe y ejecuta así:

```python
y = x/w*z
print(y)
```

Se estaría realmente resolviendo:

$$
\begin{equation}
  y = \frac{x}{w} z
\end{equation}
$$

Ejercicio:

Resuelve la siguiente ecuación:

$$
\begin{equation}
  y = \frac{m \, n}{m ^{2}} \frac{n +1}{ \left(n^{-2} m \right) ^{3}}
\end{equation}
$$

Dónde:

* $n = 2$
* $m = 10$

```python

```

### __1.2. Variable de texto__

A continuación, se puede observar la naturaleza de las variables textuales.

```python
t = "Esta es una oración"   #De igual manera que la variable numérica.
print(t)
```

```python
#Es posible adicionar texto
t2 = ", ¿o no?"
frase_completa = t+t2
print(frase_completa)
```

```python
#Podemos también acceder a las letras en un texto
print(frase_completa[0])
```

```python
#Y a fragmentos de una oración
print(frase_completa[2:])
```

### __1.3. Listas__

Variables _dinámicas_ con contenido de cualquier naturaleza.

```python
#Ejemplo de lista
l = ['a','b','c', [0,1]]
print(l)
```

```python
#¿Cómo accedemos a la información?
print(l[0])  #Recuerda: el contenido de la lista empieza desde 0, 1, 2, ...
```

```python
#Podemos redefinir el contenido de la siguiente manera:
l[0] = 'z'
print(l)  #De esta manera, la lista se cambia su valor
```

```python
print(l[3][0])  #También podemos leer la información de una lista dentro de otra lista
```

### __1.4. Tuplas__

Variables _estáticas_ con contenido de cualquier naturaleza.

```python
t = ('a',0,20,'2', ('Hola', 'Adiós'))  #Similar a la lista
print(t)
```

```python
#También podemos acceder a su contenido... y jugar con él
print('¿' + t[4][0] + '?, ' + t[4][1])
```

```python
#Pero si lo intentamos cambiar...
t[0] = 1
```

### __1.5. Diccionarios__

Tipo de variable usada en programación web. Facilita la lectura de código al darle _"nombres"_ a su contenido.

```python
#Si vamos al súper mercado
lista_mercado = {
    'manzana':2,
    'peras':3,
    'uvas': 4
}
print(lista_mercado)
```

```python
#Podemos ser aún más específicos...
lista_mercado = {
    'Frutas': {
        'Manzanas': {'Unidades': 'Un', 'Cant': 2},
        'Peras': {'Unidades': 'Un', 'Cant': 1},
        'Uvas': {'Unidades': 'Lb', 'Cant': 4}
    }
}
print(lista_mercado)
```

```python
#Se accede a la información de la siguiente manera:
print(lista_mercado['Frutas']['Manzanas'])
```


# Boltzmann distribution for a harmonic oscillator

```python
from sympy import *
init_printing()
```

```python
k, beta, T, p, n, omega, hbar = symbols('k, beta, T, p, n, omega, hbar', positive=True)
```

```python
p = exp(-n*hbar*omega/k/T)*(1-exp(-hbar*omega/k/T))
```

```python
p
```

```python
p.subs(n,2)
```

```python
solveset(p.subs([(n,2),(omega,3.0e4*3.14),(k,1.38e-23),(hbar,1.05e-34)])-0.25,T)
```

```python
x = symbols('x', positive=True)
```

```python
solveset(x**2-x**3-0.08,x)
```

```python
(hbar*omega/k*log(1/0.42)).subs([(n,2),(omega,3.0e4*3.14),(k,1.38e-23),(hbar,1.05e-34)])
```

```python

```


# Einstein Tensor calculations using Symbolic module

```python
import numpy as np
import pytest
import sympy
from sympy import cos, simplify, sin, sinh, tensorcontraction
from einsteinpy.symbolic import EinsteinTensor, MetricTensor, RicciScalar

sympy.init_printing()
```

### Defining the Anti-de Sitter spacetime Metric

```python
syms = sympy.symbols("t chi theta phi")
t, ch, th, ph = syms
m = sympy.diag(-1, cos(t) ** 2, cos(t) ** 2 * sinh(ch) ** 2, cos(t) ** 2 * sinh(ch) ** 2 * sin(th) ** 2).tolist()
metric = MetricTensor(m, syms)
```

### Calculating the Einstein Tensor (with both indices covariant)

```python
einst = EinsteinTensor.from_metric(metric)
einst.tensor()
```

docs/source/examples/Einstein Tensor symbolic calculation.ipynb