# Agenda

- Introduction to regression analysis

    - Dependent variable, independent variables
    - Prediction, forecasting, inference
    - Types of regression analysis

- Introduction to linear regression

    - Discussion of Continuous, ordinal and nominal data
    - Simple linear regression
    - Formalization of the problem
    - Objective Function
    - Using Python for linear regression
    - Linear regression with more than one predictors

# Regression Analysis

Regression analysis is a __set of statistical processes__ for estimating the relationship between a __dependent variable__ and one or more __independent variables__.

- Goal: Understand the distribution of the response variable (Y) conditioned on predictors (X).

- Can be used for prediction or forecasting or inference.

__Prediction__

- Make educated guesses about a population (whole) from data (sample)

__Forecast__

- Make systematic and educated guesses about future from knowledge gain from past

__Inference__

Is there any casual relationship between a dependent variable and independent variables in a fix dataset.

Source: Statistical Learning From Regression Perspective 2nd edition p3

## Types of Regression Analysis

- Linear Regression

- Binomial Regression

- Poisson Regression

- Robust Regression

- Time Series

- And many other variations

# Linear Regression

## Measurement Scales

__Nominal Scales (Categorical Data)__

- They are labels more than scales

- Examples: Male, Female, Republican, Democrat, Independent

- Note that they can show up as numerical values too:{"Math":1, "Economics": 2, "Philosophy": 3}.

__Ordinal Scales__

- Orders - Rank

- Military: commander< captain < admiral

- Another example: Pain Level: 0 < 1 < 2 < 3 < 4 < ... < 10

__Interval Scales__

- Differences makes sense but ratios don't!

- Example: 10°F, 20°F, 40°F, 80°F

__Ratio Scales__

- There is an absolute zero point.

- 5 seconds 10 seconds

- 5 meters 10 meters

- 5 lb, 10 lb

## Simple Linear Regression

__Understanding the task__

For this part of the lecture, we will follow the textbook [Introduction to Statistical Learning with R, 7th edition.](https://faculty.marshall.usc.edu/gareth-james/ISL/ISLR%20Seventh%20Printing.pdf) (ISLR)

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

```

```python
## load data is also supporting code
import load_data
```

```python
# read data to a dataframe
data = load_data.data()[['TV', 'Sales']]

print(data.shape)
data.head()
```

    (200, 2)

      TV
      Sales

      1
      230.1
      22.1

      2
      44.5
      10.4

      3
      17.2
      9.3

      4
      151.5
      18.5

      5
      180.8
      12.9

__About this dataset:__ The dataset contains statistics about the sales of a product in 200 different markets, together with advertising budgets in each of these markets for different media channels: TV, radio and newspaper.

The sales are in thousands of units and the budget is in thousands of dollars.

[For more informatio about this dataset](https://cran.r-project.org/web/packages/ISLR/ISLR.pdf)

__Some interesting questions we can try to answer__

- Is there any relationship between advertising budget and sales?

- If there is any relation how strong is the relationship?

- Which media contribute to sales?

For more read ISLR p60.

```python
## plot data
data.plot(kind = 'scatter', x = 'TV', y= 'Sales')
plt.show()
```

```python
## utils is a py file that I added a homemade functions
from utils import draw_line

tv = data.TV.values
sales = data.Sales.values

```

```python
## draw line will draw a line with a given slope and intercept
draw_line(tv, sales, slope= 0.07, intercept = 6.8, figsize = (10,6) )
```

__Formalization of the Process__

- In simple linear regression we assume that if we would have population of both X and y variables then we would see the following relation:

$$ Y = \beta_{0} + \beta_{1}X + \epsilon$$

- $ \beta_{0}, \beta_{1}$ are parameters of the model and called the __intercept__ and __coefficient__ of the linear model respectively.

- $\epsilon$ is the irreducible error term. Depend on the problem at hand we might assume that these errors are coming from measurement mistakes, personal beliefs, recording errors, etc.

__Note__

The same equation above can appear in different notations. We choose to stick to the notation in ISLR.

Some alternatives:

$$ Y = \theta_{0} + \theta_{1}X + \epsilon$$

$$ Y = w_{0} + w_{1}X + \epsilon$$

## Assumptions

* Assume $\mbox{Sales} \approx \mbox{TV}$  linearly.

* There is an initial value (baseline) of the Sales.

* Data might not be fit on a line but the errors are random.

* Observations(rows) are independent of each other

__Task?__

Let's discuss

__Task?__

- Finding $\beta_{0}$, $\beta_{1}$ that gives us the best line!

Is research problem specific enough?

__Task?__

- Finding $\beta_{0}$, $\beta_{1}$ that gives the line which minimizes the prediction error!

Is research problem specific enough?

__Task?__

- Finding $\beta_{0}$, $\beta_{1}$ that gives the line which minimizes the sum of individual errors!

Is research problem specific enough?

__Task?__

- Finding $\beta_{0}$, $\beta_{1}$ that gives the line which minimizes the sum of squares of individual errors!

This approach is called __Least Squares Approach__

__The same statement just fancier__

__Formula (Model)__

$$ Y = \beta_{0} + \beta_{1}X $$

__Objective Function (Cost Function)__

$$ J(\beta_{0}, \beta_{1}) = \frac{1}{2N}\sum_{i = 1}^{N} \left( e_{i} \right)^{2} $$

__Goal:  minimize $J(\beta_{0}, \beta_{1})$ with respect to $\beta_{0}, \beta_{1}$__

$$ e_i = y_{i} - (\beta_{0} + \beta_{1}x_{i})$$

__Full Picture__

```python
import matplotlib.image as mpimg

f, axes = plt.subplots(nrows= 1, ncols = 3)
for i in range(3):
    axes[i].axis('off')
    if i==0:
        img = mpimg.imread(f'images/simple{i}.jpg')
    else:
        img = mpimg.imread(f'images/simple{i}.png')
    axes[i].imshow(img)
plt.gcf().set_size_inches(20, 20)
```

__Solution__

$$ \boldsymbol{\beta} = (X^{T}X)^{-1}X^{T}\boldsymbol{Y}$$

__Finding $\beta$'s with Python__

```python
## Equation 3.4 from ISLR

def least_squares(X,y):
    x_bar = X.mean()
    y_bar = y.mean()
    b1_num = (X - x_bar).dot(y - y_bar)
    b1_denom = ((X - x_bar)**2).sum()
    b1 = b1_num /b1_denom
    b0 = y_bar - b1*x_bar
    J = 0.5 *((y - (b0 + b1*X))**2).mean()
    return  b0,b1, J

least_squares(data.TV, data.Sales)
```

    (7.032593549127698, 0.047536640433019736, 5.256326457828378)

```python
## Four Step workflow with sklearn
## 1. import the tool
from sklearn.linear_model import LinearRegression

## 2. instantiate the tool you would like to use.
lr = LinearRegression()
X = data[['TV']]
y = data.Sales
## 3. fit the model
lr.fit(X, y)

## 4. check-out the results or make predictions
print(lr.intercept_, lr.coef_)
```

    7.032593549127693 [0.04753664]

```python
## R_squared Score

lr.score(X, y)
```

    0.611875050850071

$R^{2}:$ The proportion of the variability in $Y$ that can be explained using X with the linear model

## More than one Variables

- When we have more than one variable our linear model will look like as:

$$ Y = \beta_{0} + \beta_{1} X_{1} + \beta_{2}X_{2} + \cdots \beta_{p}X_{p} + \epsilon$$

- Cost function (objective function) become:

\begin{align}
RSS &= \sum\limits_{i=1}^{N} (y_{i} - \hat{y}_{i})^{2} \\
&= \sum\limits_{i=1}^{N} (y_{i} - \beta_{0} - \beta_{1} x_{i1}  - \beta_{2}x_{i2} - \cdots  - \beta_{p}x_{ip})^{2}
\end{align}

__The same statement just fancier__

__Formula (Model)__

$$ Y = \beta_{0} + \beta_{1} X_{1} + \beta_{2}X_{2} + \cdots \beta_{p}X_{p} + \epsilon$$

__Objective Function (Cost Function)__

$$ J(\beta_{0}, \beta_{1}, \cdots, \beta_{p}) = \frac{1}{2N}\sum_{i = 1}^{N} \left( e_{i} \right)^{2} $$

__Goal:  minimize $J(\beta_{0}, \beta_{1}, \cdots, \beta_{p})$ with respect to $\beta_{0}, \beta_{1}, \cdots, \beta_{p}$__

```python
## This time we will use statsmodels api

import statsmodels.api as sm
data = load_data.data()
X = data[['TV', 'Radio', 'Newspaper']]
y = data.Sales
X = sm.add_constant(X)
mod = sm.OLS(y, X, hasconst= True)
res = mod.fit()
res.summary()
```

OLS Regression Results

  Dep. Variable:	          Sales	        R-squared:         	    0.897

  Model:	                   OLS	         Adj. R-squared:    	    0.896

  Method:	             Least Squares	    F-statistic:       	    570.3

  Date:	             Thu, 01 Oct 2020	   Prob (F-statistic):	 1.58e-96

  Time:	                 18:41:41	       Log-Likelihood:    	  -386.18

  No. Observations:	         200	        AIC:               	    780.4

  Df Residuals:	             196	        BIC:               	    793.6

  Df Model:	                   3

  Covariance Type:	      nonrobust

      	         coef	     std err	      t	      P>|t|	  [0.025	    0.975]

  const	         2.9389	     0.312	     9.422	  0.000	     2.324	     3.554

  TV	            0.0458	     0.001	    32.809	  0.000	     0.043	     0.049

  Radio	         0.1885	     0.009	    21.893	  0.000	     0.172	     0.206

  Newspaper	    -0.0010	     0.006	    -0.177	  0.860	    -0.013	     0.011

  Omnibus:	       60.414	   Durbin-Watson:     	    2.084

  Prob(Omnibus):	  0.000	   Jarque-Bera (JB):  	  151.241

  Skew:	          -1.327	   Prob(JB):          	 1.44e-33

  Kurtosis:	       6.332	   Cond. No.          	     454.

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

__Q__ Is there a relationship between predictors and response?

__Answer__ Check the F-stats

__Q__ Which factors are important?

This is called variable selection and not a very straightforward process.

- Ideally we would check all possible prediction combinations and then compare them with AIC, BIC, $C_P$

__Q__: Predictions

```python
print(res.predict(X.loc[13].values), y.loc[13])

X.loc[13]
```

    [10.57712073] 9.2

    const         1.0
    TV           23.8
    Radio        35.1
    Newspaper    65.9
    Name: 13, dtype: float64

```python
residuals = res.resid

y_pred = res.predict(X)
plt.scatter(y_pred, residuals)
plt.axhline(y=0, color = 'red', label = '0')

plt.xlabel('predicted values')

plt.ylabel('residuals')

plt.tight_layout()
```

__Q__ Model Fit

- We can check $R^{2}$ or adjusted $R^{2}$

######  Play Ground

```python
import numpy as np
ncols = 14
nrows = 15
X = np.zeros(shape = (nrows,ncols))
for i in range(ncols):
    X[:,i] = np.random.normal(size = nrows)

y = np.random.normal(size = nrows)
```

```python
import statsmodels.api as sm

mod = sm.OLS(y, X, hasconst= True)
res = mod.fit()
print(res.summary())
```

                                OLS Regression Results
    ==============================================================================
    Dep. Variable:                      y   R-squared:                       1.000
    Model:                            OLS   Adj. R-squared:                  0.997
    Method:                 Least Squares   F-statistic:                     420.4
    Date:                Thu, 01 Oct 2020   Prob (F-statistic):             0.0382
    Time:                        18:59:08   Log-Likelihood:                 44.707
    No. Observations:                  15   AIC:                            -61.41
    Df Residuals:                       1   BIC:                            -51.50
    Df Model:                          13
    Covariance Type:            nonrobust
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    x1             0.4817      0.018     27.328      0.023       0.258       0.706
    x2            -0.9031      0.112     -8.093      0.078      -2.321       0.515
    x3            -0.4771      0.055     -8.682      0.073      -1.175       0.221
    x4             0.1321      0.042      3.158      0.195      -0.399       0.663
    x5            -0.8227      0.049    -16.873      0.038      -1.442      -0.203
    x6            -0.6451      0.057    -11.380      0.056      -1.366       0.075
    x7            -0.1962      0.130     -1.515      0.371      -1.842       1.450
    x8             0.6703      0.024     27.698      0.023       0.363       0.978
    x9             0.5629      0.068      8.334      0.076      -0.295       1.421
    x10            0.1525      0.115      1.330      0.411      -1.305       1.610
    x11            1.0894      0.140      7.800      0.081      -0.685       2.864
    x12            0.2359      0.028      8.485      0.075      -0.117       0.589
    x13            0.8089      0.067     12.014      0.053      -0.047       1.664
    x14           -0.1957      0.049     -4.003      0.156      -0.817       0.425
    ==============================================================================
    Omnibus:                        1.174   Durbin-Watson:                   2.182
    Prob(Omnibus):                  0.556   Jarque-Bera (JB):                0.798
    Skew:                          -0.533   Prob(JB):                        0.671
    Kurtosis:                       2.623   Cond. No.                         37.5
    ==============================================================================

    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

__Important Note__

- As we mentioned in the beginning of the lecture linear regression is a basic approach but not necessarily simple one.

- Especially when the assumptions do not hold things results might not be reliable.

- For more details on this please read ISLR 3.3.3 very carefully.

## Further Reading

- [What is the parameter copy_X in sklearn](https://stackoverflow.com/questions/52541717/what-is-the-parameter-copy-x-in-sklearn-linear-regression)

- https://newonlinecourses.science.psu.edu/stat501/node/252/

- https://towardsdatascience.com/simple-and-multiple-linear-regression-in-python-c928425168f9

- https://www.statsmodels.org/stable/examples/notebooks/generated/ols.html

- https://lagunita.stanford.edu/assets/courseware/v1/e4f2f4823252ead8c1a68722e3415c28/c4x/HumanitiesSciences/StatLearning/asset/linear_regression-handout.pdf

- [Statsmodels API](https://www.statsmodels.org/stable/discretemod.html)

"Some of the figures in this presentation are taken from "An Introduction to Statistical Learning, with applications in R"  (Springer, 2013) with permission from the authors: G. James, D. Witten,  T. Hastie and R. Tibshirani "


# Sine-Gordon equation

* Physical space
\begin{align}
\phi_{tt} = \phi_{xx} - sin(\phi)
\end{align}
* Discretize equation on a Chebyshev polynomial grid
* $D$ is a differentiation matrix, $I$ is an identity matrix, and $0$ is a matrix of zeroes

\begin{equation}
\begin{bmatrix}
\phi \\ u
\end{bmatrix}_{t} =
\begin{bmatrix}
0 & I \\
D^2 & 0
\end{bmatrix}
\begin{bmatrix}
\phi \\ u
\end{bmatrix} +
\begin{bmatrix}
0 \\ -sin(\phi)
\end{bmatrix}
\end{equation}

```python
from rkstiff.etd35 import ETD35
from rkstiff.grids import construct_x_Dx_cheb
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

# Linear and nonlinear operators

```python
N = 100 # N Chebyshev polynomial degree (N+1 points)
a = 20 # grid end points
x,Dx = construct_x_Dx_cheb(N,-a,a) # x has length(N+1), Dx is a matrix of size (N+1) x (N+1)

# Discretize and propagate on interior Chebyshev points, enforce B.C. explicitly
x_interior = x[1:-1]
nx = len(x_interior)

D = Dx.dot(Dx)[1:-1,1:-1] # differentiate on interior points
I = np.eye(nx)
Z = np.zeros(shape=(nx,nx))
L1 = np.c_[Z,I]
L2 = np.c_[D,Z]
linop = np.r_[L1,L2]

def NL_kink(w):
    u = w[:nx].real # u in first N-1 components of w, the derivate of u is in the last N-1 components
    return np.r_[np.zeros(nx),-np.sin(u+(x_interior+a)*np.pi/a)]

def NL_antikink(w):
    u = w[:nx].real
    return np.r_[np.zeros(nx),-np.sin(u+np.pi*(a-x_interior)/a)]

def NL_kink_kink(w):
    u = w[:nx].real
    return np.r_[np.zeros(nx),-np.sin(u+2*(x_interior+a)*np.pi/a)]

def NL_kink_antikink(w):
    u = w[:nx].real
    return np.r_[np.zeros(nx),-np.sin(u+2*np.pi)]
```

# Initial value profiles

```python
# kink solitons of form 4tan^{-1}(F(x)/G(t)), with F(x;v) = exp(...), G(t;v) = exp(...)
# here they are evaluated at zero for the initial profile

v = 0.2
psi0_kink = 4*np.arctan(np.exp(x_interior/np.sqrt(1-v**2)))
dpsi0_kink = (-2*v/np.sqrt(1-v**2))/np.cosh(x_interior/np.sqrt(1-v**2))

psi0_antikink = 4*np.arctan(np.exp(-x_interior/np.sqrt(1-v**2)))
dpsi0_antikink = (-2*v/np.sqrt(1-v**2))/np.cosh(x_interior/np.sqrt(1-v**2))

psi0_kink_kink = 4*np.arctan(np.exp( (x_interior+a/2)/np.sqrt(1-v**2))) + \
        4*np.arctan(np.exp( (x_interior-a/2) /np.sqrt(1-v**2)))
dpsi0_kink_kink = -2*v/(np.cosh((x_interior+a/2)/np.sqrt(1-v**2))*np.sqrt(1-v**2)) + \
     2*v/(np.cosh((x_interior-a/2)/np.sqrt(1-v**2))*np.sqrt(1-v**2))

psi0_kink_antikink = 4*np.arctan(np.exp( (x_interior+a/2) /np.sqrt(1-v**2))) + \
        4*np.arctan(np.exp( -(x_interior-a/2) /np.sqrt(1-v**2)))
dpsi0_kink_antikink = -2*v/(np.cosh((x_interior+a/2)/np.sqrt(1-v**2))*np.sqrt(1-v**2)) + \
     -2*v/(np.cosh((x_interior-a/2)/np.sqrt(1-v**2))*np.sqrt(1-v**2))

IVP = [psi0_kink,psi0_antikink,psi0_kink_kink,psi0_kink_antikink]
labels = ['kink','anti-kink','kink+kink','kink+anti-kink']

fig = plt.figure(figsize=(12,6))
for i,soliton in enumerate(IVP):
    ax = fig.add_subplot(2,2,i+1)
    ax.plot(x[1:-1],soliton)
    ax.set_title(labels[i])
    ax.set_xlabel('x')
    ax.set_ylabel('$\phi$')
plt.tight_layout()
```

# Transform boundaries to zeroes

```python
psi0_kink = psi0_kink - (x_interior+a)*np.pi/a
psi0_antikink = psi0_antikink  - (a - x_interior)*np.pi/a
psi0_kink_kink = psi0_kink_kink - 2*(x_interior+a)*np.pi/a
psi0_kink_antikink = psi0_kink_antikink - 2*np.pi

IVP = [psi0_kink,psi0_antikink,psi0_kink_kink,psi0_kink_antikink]
labels = ['kink','anti-kink','kink+kink','kink+anti-kink']

fig = plt.figure(figsize=(12,6))
for i,soliton in enumerate(IVP):
    ax = fig.add_subplot(2,2,i+1)
    ax.plot(x[1:-1],soliton)
    ax.set_title(labels[i])
    ax.set_xlabel('x')
    ax.set_ylabel('$\phi$')
plt.tight_layout()
```

# Propagate with ETD35

```python
tf = 80
params = {'epsilon': 1e-4, 'contour_radius' : 1, 'contour_points' : 32, 'diagonalize' : True  }
solver_kink = ETD35(linop=linop,NLfunc=NL_kink,**params)
solver_antikink = ETD35(linop=linop,NLfunc=NL_antikink,**params)
solver_kink_kink = ETD35(linop=linop,NLfunc=NL_kink_kink,**params)
solver_kink_antikink = ETD35(linop=linop,NLfunc=NL_kink_antikink,**params)

solver_kink.evolve(np.r_[psi0_kink,dpsi0_kink],t0=0,tf=tf)
solver_antikink.evolve(np.r_[psi0_antikink,dpsi0_antikink],t0=0,tf=tf)
solver_kink_kink.evolve(np.r_[psi0_kink_kink,dpsi0_kink_kink],t0=0,tf=tf)
_ = solver_kink_antikink.evolve(np.r_[psi0_kink_antikink,dpsi0_kink_antikink],t0=0,tf=tf)

```

    Warning: linear matrix array has a large condition number of 1969407.88,
                method may be unstable
    Warning: linear matrix array has a large condition number of 1969407.88,
                method may be unstable
    Warning: linear matrix array has a large condition number of 1969407.88,
                method may be unstable
    Warning: linear matrix array has a large condition number of 1969407.88,
                method may be unstable

# Recover boundaries and process for graphing

```python
Ukink = []
for wv in solver_kink.u:
    wint = wv[:nx].real
    uint = wint + (x_interior+a)*np.pi/a
    u = np.r_[0,uint,2*np.pi]
    Ukink.append(u)
Ukink = np.array(Ukink);
t = np.array(solver_kink.t)
Tkink,Xkink = np.meshgrid(t,x,indexing='ij')

Uantikink = []
for wv in solver_antikink.u:
    wint = wv[:nx].real
    uint = wint + (a - x_interior)*np.pi/a
    u = np.r_[2*np.pi,uint,0]
    Uantikink.append(u)
Uantikink = np.array(Uantikink)
t = np.array(solver_antikink.t)
Tantikink,Xantikink = np.meshgrid(t,x,indexing='ij')

Ukink_kink = []
for wv in solver_kink_kink.u:
    wint = wv[:nx].real
    uint = wint + (x_interior+a)*2*np.pi/a
    u = np.r_[0,uint,4*np.pi]
    Ukink_kink.append(u)
Ukink_kink = np.array(Ukink_kink);
t = np.array(solver_kink_kink.t)
Tkink_kink,Xkink_kink = np.meshgrid(t,x,indexing='ij')

Ukink_antikink = []
for wv in solver_kink_antikink.u:
    wint = wv[:nx].real
    uint = wint + 2*np.pi
    u = np.r_[2*np.pi,uint,2*np.pi]
    Ukink_antikink.append(u)
Ukink_antikink = np.array(Ukink_antikink)
t = np.array(solver_kink_antikink.t)
Tkink_antikink,Xkink_antikink = np.meshgrid(t,x,indexing='ij')
```

# Plot results

```python
fig = plt.figure(figsize=(16,16))
ax = fig.add_subplot(2,2,1,projection='3d')
ax.plot_wireframe(Xkink,Tkink,Ukink,color='black')
ax.view_init(elev=28,azim=-135)
ax.set_title('Kink soliton')
ax = fig.add_subplot(2,2,2,projection='3d')
ax.plot_wireframe(Xantikink,Tantikink,Uantikink,color='black')
ax.view_init(elev=28,azim=-48)
ax.set_title('Anti-Kink soliton')
ax = fig.add_subplot(2,2,3,projection='3d')
ax.plot_wireframe(Xkink_kink,Tkink_kink,Ukink_kink,color='black')
ax.view_init(elev=28,azim=-135)
ax.set_title('Kink-kink soliton interaction')
ax = fig.add_subplot(2,2,4,projection='3d')
ax.plot_wireframe(Xkink_antikink,Tkink_antikink,Ukink_antikink,color='black')
ax.view_init(elev=50,azim=-120)
ax.set_title('Kink-antikink soliton interaction')

for ax in fig.axes:
    ax.set_xlabel('x')
    ax.set_ylabel('t')
    ax.set_zlabel('z')
    ax.set_facecolor('white')

```


```
%matplotlib inline
from IPython.display import display
from sympy import *
from sympy.abc import x
init_printing()
```

```
omega = exp(-x**2)
q0 = 1
scalarprod_q0 = integrate(omega * x * q0 * q0, x)
norm_q0 = integrate(omega*q0*q0, x)
beta_0 = scalarprod_q0/norm_q0
display(beta_0)
```

```
q1 = x - beta_0
scalarprod_q1 = integrate(omega * x * q1 * q1, x)
norm_q1 = integrate(omega*q1*q1, x)
```

```

```


# Cardano Formula

```python
print("I'm going to solve a cubic equation for you")
```

    I'm going to solve a cubic equation for you

```python
def discriminant():
    print("Ensure the coefficient of x^3 is 1")
    a = int(input("The coefficient of x^2 "))
    b = int(input("The coefficient of x "))
    c = int(input("The constant term "))
    discriminant = ((a**2)*(b**2)) + (18*a*b*c) - (4*(b**3)) - (4*(a**2)*c) - (27*(c**2))
    if discriminant > 0:
        print("The roots are real and distinct")
    elif discriminant == 0:
        print("The roots are real and equal")
    else:
        print("There is 1 real root and 2 imaginary roots")
    return discriminant;
discriminant()

```

    Ensure the coefficient of x^3 is 1
    The coefficient of x^2 1
    The coefficient of x 2
    The constant term 3
    There is 1 real root and 2 imaginary roots

    -175

```python
import math

def Cardano():
    print("Ensure the coefficient of x^3 is 1")
    a = float(input("The coefficient of x^3 "))
    b = float(input("The coefficient of x^2 "))
    c = float(input("The coefficient of x "))
    d = float(input("The constant term "))
    if a !=1:
        print("Please let a be equal to a by dividing the equation by a")
    else:
        while a ==1:

            discriminant = ((b**2)*(c**2)) + (18*b*c*d) - (4*(c**3)) - (4*(b**2)*d) - (27*(d**2))

            Q = (3*c - b)/9
            R = ((9*b*c)-(27*d) - (2 * pow(b,3)))/54
            S = ((R + math.sqrt((Q**3)+ (R**2))))**(1/3)
            T = ((R - math.sqrt((Q**3)+ (R**2))))**(1/3)

            if discriminant < 0:
                print (Q,R,S,T)

            else:
                print("Can't use cardano")
    return
Cardano()
```

```python
# importing library sympy
from sympy import symbols, Eq, solve

# defining symbols used in equations
# or unknown variables
x, y = symbols('x,y')

# defining equations
eq1 = Eq((x+y), 1)
print("Equation 1:")
print(eq1)
eq2 = Eq((x-y), 1)
print("Equation 2")
print(eq2)

# solving the equation
print("Values of 2 unknown variable are as follows:")

print(solve((eq1, eq2), (x, y)))
```

    Equation 1:
    Eq(x + y, 1)
    Equation 2
    Eq(x - y, 1)
    Values of 2 unknown variable are as follows:
    {x: 1, y: 0}

```python
import math

def cuberoot(x):
    if x < 0:
        x = abs(x)
        cube_root = x**(1/3)*(-1)
    else:
        cube_root = x**(1/3)
    return cube_root

def Cardano():
    print("Ensure the coefficient of x^3 is 1")
    a = float(input("The coefficient of x^3 "))
    b = float(input("The coefficient of x^2 "))
    c = float(input("The coefficient of x "))
    d = float(input("The constant term "))
    if a !=1:
        print("Please let a be equal to a by dividing the equation by a")
    else:
        if a ==1:

            discriminant = ((b**2)*(c**2)) + (18*b*c*d) - (4*(c**3)) - (4*(b**2)*d) - (27*(d**2))

            Q = (3*c - pow(b,2))/9
            R = ((9*b*c)-(27*d) - (2 * pow(b,3)))/54
            u = (Q**3)+ (R**2)
            u = math.sqrt(u)
            x = R+u
            y = R-u
            if R < 0:
                S = cuberoot(x)
                T = cuberoot(y)
                print (S)
            else:
                print('Tired')
                #S = (R + math.sqrt(u))**(1/3)
            #u = (Q**3)+ (R**2)
            #S = (abs(R) + math.sqrt(u))**(1/3)
            #T = (R - u)**(0.33)

            if discriminant < 0:
                print (Q,R)

            else:
                print("Can't use cardano")
    return
Cardano()
```

    Ensure the coefficient of x^3 is 1
    The coefficient of x^3 1
    The coefficient of x^2 3
    The coefficient of x 0
    The constant term 0.5
    -0.7937005259840998
    -1.0 -1.25

```python

```


```python
%matplotlib inline
```

```python
%run proof_setup
```

```python
import numpy as np
import sympy as sm

def rotate(sinw, cosw, sini, cosi, x2, y2, z2):
    Rwinv = sm.Matrix([[cosw, sinw, 0], [-sinw, cosw, 0], [0, 0, 1]])
    Riinv = sm.Matrix([[1, 0, 0], [0, cosi, sini], [0, -sini, cosi]])
    v2 = sm.Matrix([[x2], [y2], [z2]])
    v0 = Rwinv * Riinv * v2
    return sm.simplify(v0), sm.simplify(v2)

def get_quadratic_eqs(circular=False, edge=False, printer=None, wcase=False):
    if printer is None:
        printer = lambda x: x

    semimajor, ecc, w, incl, x, y, z, L = sm.symbols("a, e, omega, i, x, y, z, L")
    sinw = sm.sin(w)
    cosw = sm.cos(w)
    sini = sm.sin(incl)
    cosi = sm.cos(incl)

    if edge:
        cosi = 0
        sini = 1

    y = z * cosi / sini

    if wcase:
        sinw = 0
        cosw = 1

    if circular:
        ecc = 0

    v0, v2 = rotate(sinw, cosw, sini, cosi, x, y, z)

    print("x0 =", printer(v0[0]))
    print("y0 =", printer(v0[1]))
    print("z0 =", printer(v0[2]))
    print()

    eq = (v0[0] - semimajor*ecc)**2 + v0[1]**2/(1-ecc**2) - semimajor**2

    eq1 = sm.poly(eq, x, z)

    denom = (ecc**2 - 1)
    print("A =", printer(sm.simplify(denom * eq1.coeff_monomial(x**2))))
    print("B =", printer(sm.cancel(denom * eq1.coeff_monomial(x*z))))
    print("C =", printer(sm.simplify(denom * eq1.coeff_monomial(z**2))))
    print("D =", printer(sm.simplify(denom * eq1.coeff_monomial(x))))
    print("E =", printer(sm.simplify(denom * eq1.coeff_monomial(z))))
    print("F =", printer(sm.simplify(denom * eq1.coeff_monomial(1))))

    return (
        sm.simplify(denom * eq1.coeff_monomial(x**2)),
        sm.simplify(denom * eq1.coeff_monomial(x*z)),
        sm.simplify(denom * eq1.coeff_monomial(z**2)),
        sm.simplify(denom * eq1.coeff_monomial(x)),
        sm.simplify(denom * eq1.coeff_monomial(z)),
        sm.simplify(denom * eq1.coeff_monomial(1)),
    )
```

```python
get_quadratic_eqs(printer=sm.latex)
print()
print()
get_quadratic_eqs(circular=True, printer=sm.latex);
print()
print()
get_quadratic_eqs(wcase=True, printer=sm.latex);
print()
print()
get_quadratic_eqs(edge=True, printer=sm.latex);
```

    x0 = x \cos{\left (\omega \right )} + \frac{z \sin{\left (\omega \right )}}{\sin{\left (i \right )}}
    y0 = - x \sin{\left (\omega \right )} + \frac{z \cos{\left (\omega \right )}}{\sin{\left (i \right )}}
    z0 = 0

    A = e^{2} \cos^{2}{\left (\omega \right )} - 1
    B = \frac{2 e^{2} \sin{\left (\omega \right )} \cos{\left (\omega \right )}}{\sin{\left (i \right )}}
    C = \frac{e^{2} \sin^{2}{\left (\omega \right )} - 1}{\sin^{2}{\left (i \right )}}
    D = 2 a e \left(- e^{2} + 1\right) \cos{\left (\omega \right )}
    E = - \frac{2 a e \left(e^{2} - 1\right) \sin{\left (\omega \right )}}{\sin{\left (i \right )}}
    F = a^{2} \left(e^{2} - 1\right)^{2}

    x0 = x \cos{\left (\omega \right )} + \frac{z \sin{\left (\omega \right )}}{\sin{\left (i \right )}}
    y0 = - x \sin{\left (\omega \right )} + \frac{z \cos{\left (\omega \right )}}{\sin{\left (i \right )}}
    z0 = 0

    A = -1
    B = 0
    C = - \frac{1}{\sin^{2}{\left (i \right )}}
    D = 0
    E = 0
    F = a^{2}

    x0 = x
    y0 = \frac{z}{\sin{\left (i \right )}}
    z0 = 0

    A = e^{2} - 1
    B = 0
    C = - \frac{1}{\sin^{2}{\left (i \right )}}
    D = 2 a e \left(- e^{2} + 1\right)
    E = 0
    F = a^{2} \left(e^{2} - 1\right)^{2}

    x0 = x \cos{\left (\omega \right )} + z \sin{\left (\omega \right )}
    y0 = - x \sin{\left (\omega \right )} + z \cos{\left (\omega \right )}
    z0 = 0

    A = e^{2} \cos^{2}{\left (\omega \right )} - 1
    B = 2 e^{2} \sin{\left (\omega \right )} \cos{\left (\omega \right )}
    C = e^{2} \sin^{2}{\left (\omega \right )} - 1
    D = 2 a e \left(- e^{2} + 1\right) \cos{\left (\omega \right )}
    E = 2 a e \left(- e^{2} + 1\right) \sin{\left (\omega \right )}
    F = a^{2} \left(e^{2} - 1\right)^{2}

```python
def get_quartic_expr(circular=False, edge=False, printer=None, wcase=False):
    if printer is None:
        printer = lambda x: x

    A, B, C, D, E, F, T, L, x = sm.symbols("A, B, C, D, E, F, T, L, x", real=True)
    if edge:
        A, B, C, D, E, F = get_quadratic_eqs(edge=True)

    p0 = T
    p1 = 0
    p2 = x**2 - L**2

    q0 = C
    q1 = B*x + E
    q2 = A*x**2 + D*x + F

    quartic = sm.Poly((p0*q2 - p2*q0)**2 - (p0*q1 - p1*q0)*(p1*q2 - p2*q1), x)

    if circular:
        args = {A:-1, B: 0, D:0, E: 0}
    elif wcase:
        args = {B: 0, E: 0}
        quartic = sm.factor(quartic.subs(args))
        print(quartic)
        return
    else:
        args = {}

    for i in range(5):
        print("a_{0} =".format(i), printer(sm.factor(sm.simplify(quartic.coeff_monomial(x**i).subs(args)))))
```

```python
get_quartic_expr(printer=sm.latex)
print()
print()
get_quartic_expr(circular=True, printer=sm.latex)
print()
print()
get_quartic_expr(wcase=True, printer=sm.latex)
```

    a_0 = C^{2} L^{4} + 2 C F L^{2} T - E^{2} L^{2} T + F^{2} T^{2}
    a_1 = - 2 T \left(B E L^{2} - C D L^{2} - D F T\right)
    a_2 = 2 A C L^{2} T + 2 A F T^{2} - B^{2} L^{2} T - 2 C^{2} L^{2} - 2 C F T + D^{2} T^{2} + E^{2} T
    a_3 = 2 T \left(A D T + B E - C D\right)
    a_4 = A^{2} T^{2} - 2 A C T + B^{2} T + C^{2}

    a_0 = \left(C L^{2} + F T\right)^{2}
    a_1 = 0
    a_2 = - 2 \left(C + T\right) \left(C L^{2} + F T\right)
    a_3 = 0
    a_4 = \left(C + T\right)^{2}

    (A*T*x**2 + C*L**2 - C*x**2 + D*T*x + F*T)**2

```python

```

```python

```

```python
def balance_companion_matrix(companion_matrix):
    diag = np.array(np.diag(companion_matrix))
    companion_matrix[np.diag_indices_from(companion_matrix)] = 0.0
    degree = len(diag)

    # gamma <= 1 controls how much a change in the scaling has to
    # lower the 1-norm of the companion matrix to be accepted.
    #
    # gamma = 1 seems to lead to cycles (numerical issues?), so
    # we set it slightly lower.
    gamma = 0.9

    scaling_has_changed = True
    while scaling_has_changed:
        scaling_has_changed = False

        for i in range(degree):
            row_norm = np.sum(np.abs(companion_matrix[i]))
            col_norm = np.sum(np.abs(companion_matrix[:, i]))

            # Decompose row_norm/col_norm into mantissa * 2^exponent,
            # where 0.5 <= mantissa < 1. Discard mantissa (return value
            # of frexp), as only the exponent is needed.
            _, exponent = np.frexp(row_norm / col_norm)
            exponent = exponent // 2

            if exponent != 0:
                scaled_col_norm = np.ldexp(col_norm, exponent)
                scaled_row_norm = np.ldexp(row_norm, -exponent)
                if scaled_col_norm + scaled_row_norm < gamma * (col_norm + row_norm):
                    # Accept the new scaling. (Multiplication by powers of 2 should not
                    # introduce rounding errors (ignoring non-normalized numbers and
                    # over- or underflow))
                    scaling_has_changed = True
                    companion_matrix[i] *= np.ldexp(1.0, -exponent)
                    companion_matrix[:, i] *= np.ldexp(1.0, exponent)

    companion_matrix[np.diag_indices_from(companion_matrix)] = diag
    return companion_matrix

def solve_companion_matrix(poly):
    poly = np.atleast_1d(poly)
    comp = np.eye(len(poly) - 1, k=-1)
    comp[:, -1] = -poly[:-1] / poly[-1]
    return np.linalg.eigvals(balance_companion_matrix(comp))

def _get_quadratic(a, e, cosw, sinw, cosi, sini):
    e2 = e*e
    e2mo = e2 - 1
    return (
        (e2*cosw*cosw - 1),
        2*e2*sinw*cosw/sini,
        (e2mo - e2*cosw*cosw)/(sini*sini),
        -2*a*e*e2mo*cosw,
        -2*a*e*e2mo*sinw/sini,
        a**2*e2mo*e2mo,
    )

def _get_quartic(A, B, C, D, E, F, T, L):
    A2 = A*A
    B2 = B*B
    C2 = C*C
    D2 = D*D
    E2 = E*E
    F2 = F*F
    T2 = T*T
    L2 = L*L
    return (
        C2*L2*L2 + 2*C*F*L2*T - E2*L2*T + F2*T2,
        -2*T*(B*E*L2 - C*D*L2 - D*F*T),
        2*A*C*L2*T + 2*A*F*T2 - B2*L2*T - 2*C2*L2 - 2*C*F*T + D2*T2 + E2*T,
        2*T*(A*D*T + B*E - C*D),
        A2*T2 - 2*A*C*T + B2*T + C2,
    )

def _get_roots_general(a, e, omega, i, L, tol=1e-8):
    cosw = np.cos(omega)
    sinw = np.sin(omega)
    cosi = np.cos(i)
    sini = np.sin(i)

    f0 = 2 * np.arctan2(cosw, 1 + sinw)

    quad = _get_quadratic(a, e, cosw, sinw, cosi, sini)
    A, B, C, D, E, F = quad
    T = cosi / sini
    T *= T
    quartic = _get_quartic(A, B, C, D, E, F, T, L)

    roots = solve_companion_matrix(quartic)
    roots = roots[np.argsort(np.real(roots))]

    # Deal with multiplicity
    roots[0] = roots[:2][np.argmin(np.abs(np.imag(roots[:2])))]
    roots[1] = roots[2:][::-1][np.argmin(np.abs(np.imag(roots[2:])[::-1]))]
    roots = roots[:2]

    # Only select real roots
    roots = np.clip(np.real(roots[np.abs(np.imag(roots)) < tol]), -L, L)
    if len(roots) < 2:
        return np.empty(0)

    angles = []
    for x in roots:
        b0 = A*x*x + D*x + F
        b1 = B*x + E
        b2 = C
        z1 = -0.5 * b1 / b2
        arg = b1*b1 - 4*b0*b2
        if arg < 0:
            continue
        z2 = 0.5 * np.sqrt(arg) / b2
        for sgn in [-1, 1]:
            z = z1 + sgn * z2
            if z > 0:
                continue
            y = z * cosi / sini

            x0 = x*cosw + z*sinw/sini
            y0 = -x*sinw + z*cosw/sini
            angle = np.arctan2(y0, x0) - np.pi
            if angle < -np.pi:
                angle += 2*np.pi
            angles.append(angle - f0)

    angles = np.sort(angles)

    # Wrap the roots properly to span the transit
    if len(angles) == 2:
        if np.all(angles > 0):
            angles = np.array([angles[1] - 2*np.pi, angles[0]])
        if np.all(angles < 0):
            angles = np.array([angles[1], angles[0] + 2*np.pi])
    else:
        angles = np.array([-np.pi, np.pi])

    return angles + f0

def check_roots(a, e, omega, i, L, tol=1e-8):
    L /= a
    a = 1.0
    roots = _get_roots_general(a, e, omega, i, L, tol=tol)

    for f in roots:
        b2 = a**2*(e**2 - 1)**2*(np.cos(i)**2*(np.cos(omega)*np.sin(f) + np.sin(omega)*np.cos(f))**2 + (np.cos(omega)*np.cos(f) - np.sin(omega)*np.sin(f))**2)/(e*np.cos(f) + 1)**2
        print("b2 = ", b2, " L2 = ", L**2)

    print(roots)
```

```python
check_roots(10.0, 0.5, -0.15, 0.5*np.pi - 0.01, 1.0)
```

    b2 =  0.010014687059509328  L2 =  0.010000000000000002
    b2 =  0.010019159137027812  L2 =  0.010000000000000002
    [1.58853526 1.83656275]

```python
check_roots(100.0, 0.0, np.pi, 0.5*np.pi, 1.5)
```

    b2 =  0.00022499999999999343  L2 =  0.000225
    b2 =  0.000225000000000003  L2 =  0.000225
    [-1.58579689 -1.55579576]

```python
get_quadratic_eqs()
print()
get_quartic_expr();
```

    x0 = x*cos(omega) + z*sin(omega)/sin(i)
    y0 = -x*sin(omega) + z*cos(omega)/sin(i)
    z0 = 0

    A = e**2*cos(omega)**2 - 1
    B = 2*e**2*sin(omega)*cos(omega)/sin(i)
    C = (e**2*sin(omega)**2 - 1)/sin(i)**2
    D = 2*a*e*(-e**2 + 1)*cos(omega)
    E = -2*a*e*(e**2 - 1)*sin(omega)/sin(i)
    F = a**2*(e**2 - 1)**2

    a_0 = C**2*L**4 + 2*C*F*L**2*T - E**2*L**2*T + F**2*T**2
    a_1 = -2*T*(B*E*L**2 - C*D*L**2 - D*F*T)
    a_2 = 2*A*C*L**2*T + 2*A*F*T**2 - B**2*L**2*T - 2*C**2*L**2 - 2*C*F*T + D**2*T**2 + E**2*T
    a_3 = 2*T*(A*D*T + B*E - C*D)
    a_4 = A**2*T**2 - 2*A*C*T + B**2*T + C**2

```python
get_quadratic_eqs(edge=True)
print()
print()
get_quadratic_eqs(circular=True, printer=sm.latex, edge=True);
```

    x0 = x*cos(omega) + z*sin(omega)
    y0 = -x*sin(omega) + z*cos(omega)
    z0 = -y

    b_0 = (a**2*e**4 - 2*a**2*e**2 + a**2 - 2*a*e**3*x*cos(omega) + 2*a*e*x*cos(omega) + e**2*x**2*cos(omega)**2 - x**2)/(e**2 - 1)
    b_1 = (-2*a*e**3*sin(omega) + 2*a*e*sin(omega) + 2*e**2*x*sin(omega)*cos(omega))/(e**2 - 1)
    b_2 = (e**2*sin(omega)**2 - 1)/(e**2 - 1)

    x0 = x
    y0 = z
    z0 = - y

    b_0 = - a^{2} + x^{2}
    b_1 = 0
    b_2 = 1

```python

```


(c) Juan Gomez 2019. Thanks to Universidad EAFIT for support. This material is part of the course Introduction to Finite Element Analysis

# SOLUTION: Interpolation in Two-Dimensional Domains

## By Juan Gomez

### Class activity
**Problem 1**

Extend the 2D interpolation scheme discussed above to the case of a vector valued function in the context of linear elasticity. For that purpose:

* Assume that at each nodal point of the square domain the displacement vector with horizontal and vertical componentes denoted by $u$ and $v$ respectively is known.
* Using these nodal values compute the horizontal and vertical displacement components over the element. Note that the contribution from the nodal dispalcements from the Q-th node can be written like:

$$
\begin{bmatrix}u\\v\end{bmatrix}=\left[\cdots\begin{array}{cc}N^Q&0\\0&N^Q\end{array}\cdots\right]\begin{bmatrix}\vdots\\u^Q\\v^Q\\\vdots\end{bmatrix}\\
$$

where $N^Q$ is the corresponding shape function.

* Using these same nodal values compute the strain field given by:

$$\varepsilon_{xx}=\frac12\left(\frac{\partial u}{\partial x}\right)$$

$$\varepsilon_{yy}=\frac12\left(\frac{\partial v}{\partial y}\right)$$

$$\gamma_{yy}=\left(\frac{\partial u}{\partial y}+\frac{\partial v}{\partial x}\right)$$

It can be observed that in order to find the strain components we need first derivatives of $u$ and $v$ with respect to $x$ and $y$. These derivatives satisfy:

$$\begin{bmatrix}\frac{\partial u}{\partial x}\\\frac{\partial u}{\partial x}\end{bmatrix} = \left[\cdots\begin{array}{cc}\frac{\partial N^Q}{\partial x}&0\\0&\frac{\partial N^Q}{\partial x}\end{array}\cdots\right] \begin{bmatrix}\vdots\\u^Q\\v^Q\\\vdots\end{bmatrix}$$

$$\begin{bmatrix}\frac{\partial u}{\partial y}\\\frac{\partial u}{\partial y}\end{bmatrix} = \left[\cdots\begin{array}{cc}\frac{\partial N^Q}{\partial y}&0\\0&\frac{\partial N^Q}{\partial y}\end{array}\cdots\right] \begin{bmatrix}\vdots\\u^Q\\v^Q\\\vdots\end{bmatrix}$$

thus the vector storing the strain components satisfy:

$$\begin{bmatrix}\varepsilon_{xx}\\\varepsilon_{yy}\\\gamma_{xy}\end{bmatrix}=\begin{bmatrix}&\frac{\partial N^Q}{\partial x}&0&\\\cdots&0&\frac{\partial N^Q}{\partial y}&\cdots\\&\frac{\partial N^Q}{\partial y}&\frac{\partial N^Q}{\partial x}&\end{bmatrix}\begin{bmatrix}\vdots\\u^Q\\v^Q\\\vdots\end{bmatrix}$$.

The expression above can be written in matrix form after considering the contribution from all nodal points in the element as:

$$\left(\varepsilon\right)=\left[B\right]\left(u\right)$$

where we recognize matrix $\left[B\right]$ to be given by:

$$B = \begin{bmatrix}&\frac{\partial N^Q}{\partial x}&0&\\\cdots&0&\frac{\partial N^Q}{\partial y}&\cdots\\&\frac{\partial N^Q}{\partial y}&\frac{\partial N^Q}{\partial x}&\end{bmatrix}$$

```python
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
import sympy as sym
from scipy import interpolate
import math
```

The first subroutine (taken from SolidsPy) stores the interpolation shape functions for the element in the 2D array $N(x , y)$. In this case the element is assumed to be a size 2.0 perfect square.

```python
def sha4(x, y):
    """Shape functions for a 4-noded quad element

    Parameters
    ----------
    x : float
      x coordinate for a point within the element.
    y : float
      y coordinate for a point within the element.

    Returns
    -------
    N : Numpy array
      Array of interpolation functions.

    Examples
    --------
    We can check evaluating at two different points, namely (0, 0) and
    (1, 1). Thus

    >>> N = sha4(0, 0)
    >>> N_ex = np.array([
    ...    [1/4, 0, 1/4, 0, 1/4, 0, 1/4, 0],
    ...    [0, 1/4, 0, 1/4, 0, 1/4, 0, 1/4]])
    >>> np.allclose(N, N_ex)
    True

    and

    >>> N = sha4(1, 1)
    >>> N_ex = np.array([
    ...    [0, 0, 0, 0, 1, 0, 0, 0],
    ...    [0, 0, 0, 0, 0, 1, 0, 0]])
    >>> np.allclose(N, N_ex)
    True

    """
    N = np.zeros((2, 8))
    H = 0.25*np.array(
        [(1 - x)*(1 - y),
         (1 + x)*(1 - y),
         (1 + x)*(1 + y),
         (1 - x)*(1 + y)])
    N[0, ::2] = H
    N[1, 1::2] = H

    return N
```

This second subroutine (adaptade from SolidsPy) stores the shape functions derivatives in the strain matrix $B$

```python
def stdm4NQ(x, y):
    """Strain-displacement interpolator B for a 4-noded quad element

    Parameters
    ----------
    r : float
      r component in the natural space.
    s : float
      s component in the natural space.
    coord : ndarray
      Coordinates of the nodes of the element (4, 2).

    Returns
    -------
    ddet : float
      Determinant evaluated at `(r, s)`.
    B : ndarray
      Strain-displacement interpolator evaluated at `(r, s)`.

    """
    nn = 4
    B = np.zeros((3, 2*nn))
    dhdx = 0.25*np.array([
            [y- 1, -y + 1, y + 1, -y - 1],
            [x - 1, -x - 1, x + 1, -x + 1]])

    B[0,  ::2] = 0.5*dhdx[0, :]
    B[1, 1::2] = 0.5* dhdx[1, :]
    B[2,  ::2] = dhdx[1, :]
    B[2, 1::2] = dhdx[0, :]
    return B
```

We are now ready to conduct the interpolation assuming the vector of nodal displacements for the element is known. The response at a single point is computed like:

```python
v = sym.Matrix(8, 1, [-1.0 , -0.5 , +0.5 , +0.4 , -1.0 , -0.5 ,+0.5 ,+0.4])
x = 0.25
y = 0.25
N = sha4(x, y)
B = stdm4NQ(x, y)
u_sol = N*v
e_sol = B*v
print(u_sol)
print(e_sol)
```

    Matrix([[-0.296875000000000], [-0.0781250000000000]])
    Matrix([[-0.0937500000000000], [-0.0562500000000000], [-0.300000000000000]])

**Problem 2**

In a realistic finite element problem and due to geometric irregularities in the computational domain all the elements of the mesh would have different geometric paramters. These differences not only complicate the interpolation process but even create serious problems on coding a systematic approach. In practice every real (distorted) element is transformed to a canonical element in such a way that the interpolation process is conducted in the canonical space. The figure below shows the relation between these two spaces emphasizing the fact that there is a one to one connecion between a point in both spaces. The mathematical details of the transformation are provided in the **Class Notes.**

* Compute the transformation between the distorted and canonical element spaces required to conduct two-dimensional interpolation in a realistic case.

To conduct the transformation we assume that the relation between the physical space $\overrightarrow x$ and canonical (or mathematical space) $\overrightarrow\eta$ is given by:

$$\overrightarrow x=f(\overrightarrow\eta)$$

In the finite element method this transformation is established using also an interpolation scheme. It then follows that the physical space coordinates can be obtained in terms of the mathematical space coordinates like:

$$\begin{bmatrix}x\\y\end{bmatrix}=\left[\cdots\begin{array}{cc}N^Q&0\\0&N^Q\end{array}\cdots\right]\begin{bmatrix}\vdots\\x^Q\\y^Q\\\vdots\end{bmatrix}$$

and where $x^Q$ and $y^Q$ are the coordinates of the Q-th nodal point in the physical space. The corresponding function for the 4-noded element is then

```python
def sha4nat(r,s):
    """
    Compute the shape functions for bi-linear
    square element of size 2.0.
    """
    sh=sym.zeros(4)
    N=sym.zeros(2 , 8)
    sh[0] =(1.0/4.0)*(1 - r)*(1 - s)
    sh[1] =(1.0/4.0)*(1 + r)*(1 - s)
    sh[2] =(1.0/4.0)*(1 + r)*(1 + s)
    sh[3] =(1.0/4.0)*(1 - r)*(1 + s)

    N[0,0]=sh[0]
    N[0,2]=sh[1]
    N[0,4]=sh[2]
    N[0,6]=sh[3]

    N[1,1]=sh[0]
    N[1,3]=sh[1]
    N[1,5]=sh[2]
    N[1,7]=sh[3]
#
    return N
```

Let us define the nodal points coordinates

```python
coord = sym.Matrix(8, 1, [0.0 , 0.0 , 1.0 , 0.0 , 1.0 , 1.0 , 0.0 , 1.0])
```

Now conduct the interpolation:

```python
r=0.0
s=0.0
N = sha4nat(r,s)
X = N*coord
print(X)
```

    Matrix([[0.500000000000000], [0.500000000000000]])

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./nb_style.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>

```python

```


# Angewandte Mathematik (WS2020)

Leeres Jupyter Notebook als Ausgangspunkt für die Übung.

    Wenn Sie das Notebook über mybinder.org geöffnet haben (Link auf der Kurswebseite), dann können Sie auf dem Server keine Ergebnisse speichern!
    Nach 10 Minuten Inaktivität löscht mybinder alle Ergebnisse. Laden Sie sich das Notebook daher regelmäßig auf Ihren Rechner.
    Gehen Sie dazu auf wie folgt vor:

Wählen Sie in der Menüleiste File → Download as

Wählen Sie als Format Notebook (.ipynb)

Je nach Browsereinstellungen lädt Ihr Browser das Notebook direkt herunter oder fragt Sie wo Sie die Datei speichern wollen.

Zunächst importieren wir einige Pythonmodule die wir immer wieder benötigen werden:
* [NumPy](https://numpy.org): Numerische Basisroutinen ([Dokumentation][np-doc])
* [SymPy][sp]: Symbolisches Rechnen ([Dokumentation][sympy-doc]

[sp]: https://www.sympy.org/en/index.html
[sp-doc]: https://docs.sympy.org/latest/index.html
[np]: https://numpy.org/
[np-doc]: https://numpy.org/doc/stable/

```python
import numpy as np
import sympy as sp
```

```python
sp.init_printing(use_latex=True)
```

Zur Erinnerung:
* Sie können Zellen mit Quellcode ausführen indem Sie die Zelle auswählen und **Shift** + **Return** drücken.
* Mit dern Pfeiltasten können Sie zwischen Zellen wechseln.
* Um eine Zelle zu bearbeiten, wählen Sie diese aus und drücken **Return**.
* Um Kommentare oder Freitext einzufügen, ändern Sie den Typ einer Zelle in *Markdown*. Wählen Sie dazu im Menü **Cell** → **Cell Type** **Markdown** aus.
* Wenn Sie die letzte Zeile ausführen, fügt Jupyter automatisch eine neue Zelle am Ende des Notebooks an.

```python
x, y, t = sp.symbols('x y t')
poly = x**2 - 4*x*y + 4*y**2
poly
```

```python
factors = sp.factor(poly)
factors
```

```python
sp.expand(factors)
```

Löse das bestimmte Integral:

$$\int_0^\infty \exp(-x)\mathop{}\!\mathrm{d}x$$

```python
sp.integrate(sp.exp(-x), (x, 0, sp.oo))
```

Bestimme die Stammfunktion von

$$ \int 2 \exp(-\frac{x}{2})\mathop{}\!\mathrm{d}x$$

```python
f = 2 * sp.exp(-x/2)
F = sp.integrate(f, x)
display(F)
```

```python
(sp.sin(x**2)).series(x, n=10)
```

```python

```

```python

```


# MathematicalProgram Tutorial

For instructions on how to run these tutorial notebooks, please see the [README](https://github.com/RobotLocomotion/drake/blob/master/tutorials/README.md).

## Background
Many engineering problems can be formulated as mathematical optimization problems, and solved by numerical solvers. A generic mathematical optimization problem can be formulated as
\begin{align}
\begin{array}{rl}
                       \min_x \;  &  f(x)
   \\[0.1pc]\text{subject to}  \;  &  x \in\mathcal{S}
   \end{array}
   \hspace{1.45em}
   {\scriptsize
   \begin{array}{|ll|}
             \hline \text{The real-valued decision variable is}      & x
       \\[-0.21pc]  \text{The real-valued cost function is}          & f(x)
       \\[-0.21pc]  \text{The constraint set is}                     & \mathcal{S}
       \\[-0.21pc]  \text{The optimal $x$ that minimizes the cost function is} & x^*
       \\[0.03pc]\hline
   \end{array}
   }
\end{align}

where $x$ is the real-valued decision variable(s), $f(x)$ is the real-valued *cost function*, $\mathcal{S}$ is the constraint set for $x$. Our goal is to find the optimal $x^*$ within the constraint set $\mathcal{S}$, such that $x^*$ minimizes the cost function $f(x)$.

For example, the following optimization problem determines the value of $x$
that minimizes $x^3 + 2x + 1$ subject to $x \ge 1$.
\begin{align}
\begin{array}{rl}
                       \min_x  &  x^3 + 2x + 1
   \\[0.1pc] \text{subject to}  &  x \ge 1
\end{array}
\hspace{1.45em}
{\scriptsize
\begin{array}{|ll|}
              \hline \text{The real-valued decision variable is}         &  x
   \\[-0.21pc]       \text{The real-valued cost function $f(x)$ is} &  x^3 + 2x + 1
   \\[-0.21pc]       \text{The set $\mathcal{S}$ of constraints is}      &  x \ge 1
   \\[-0.21pc]       \text{The value that minimizes the cost function is}    &  x^* = 1
   \\[0.03pc]\hline
\end{array}}
\end{align}

In general, how an optimization problem is solved depends on its categorization (categories include Linear Programming, Quadratic Programming, Mixed-integer Programming, etc.). Categorization depends on properties of both the cost function $f(x)$ and the constraint set $\mathcal{S}$. For example, if the cost function $f(x)$ is a linear function of $x$, and the constraint $\mathcal{S}$ is a linear set $\mathcal{S} = \{x | Ax\le b\}$, then we have a *linear programming* problem, which is efficiently solved with certain solvers.

There are multiple solvers for each category of optimization problems,
but each solver has its own API and data structures.
Frequently, users need to rewrite code when they switch solvers.
To remedy this, Drake provides a common API through the *MathematicalProgram* class.
In addition to avoiding solver-specific code,
the constraint and cost functions can be written in symbolic form (which makes code more readable).
In these ways, Drake's MathematicalProgram is akin to [YALMIP](https://yalmip.github.io/) in MATLAB or [JuMP](https://github.com/JuliaOpt/JuMP.jl) in Julia, and we support both Python and C++.

 Note: Drake supports many [solvers](https://drake.mit.edu/doxygen_cxx/group__solvers.html)
(some are open-source and some require a license).

Drake can formulate and solve the following categories of optimization problems
* Linear programming
* Quadratic programming
* Second-order cone programming
* Nonlinear nonconvex programming
* Semidefinite programming
* Sum-of-squares programming
* Mixed-integer programming (mixed-integer linear programming, mixed-integer quadratic programming, mixed-integer second-order cone programming).
* Linear complementarity problem

This tutorial provides the basics of Drake's MathematicalProgram.
Advanced tutorials are available at the [bottom](#Advanced-tutorials) of this document.

## Basics of MathematicalProgram class
Drake's MathematicalProgram class contains the mathematical formulation of an optimization problem, namely the decision variables $x$, the cost function $f(x)$, and the constraint set $\mathcal{S}$.

### Initialize a MathematicalProgram object

 To initialize this class, first create an empty MathematicalProgram as

```python
%matplotlib notebook
```

```python
from pydrake.solvers.mathematicalprogram import MathematicalProgram
import numpy as np
import matplotlib.pyplot as plt

# Create an empty MathematicalProgram named prog (with no decision variables,
# constraints or cost function)
prog = MathematicalProgram()
```

### Adding decision variables
Shown below, the function `NewContinuousVariables` adds two new continuous decision variables to `prog`.  The newly added variables are returned as `x` in a numpy array.

 Note the range of the variable is a continuous set, as opposed to binary variables which only take discrete value 0 or 1.

```python
x = prog.NewContinuousVariables(2)

```

The default names of the variable in *x* are "x(0)" and "x(1)".  The next line prints the default names and types in `x`, whereas the second line prints the symbolic expression "1 + 2x[0] + 3x[1] + 4x[1]".

```python
print(x)
print(1 + 2*x[0] + 3*x[1] + 4*x[1])
```

To create an array `y` of two variables named "dog(0)"" and "dog(1)", pass the name "dog" as a second argument to `NewContinuousVariables()`. Also shown below is the printout of the two variables in `y` and a symbolic expression involving `y`.

```python
y = prog.NewContinuousVariables(2, "dog")
print(y)
print(y[0] + y[0] + y[1] * y[1] * y[1])
```

To create a $3 \times 2$ matrix of variables named "A", type

```python
var_matrix = prog.NewContinuousVariables(3, 2, "A")
print(var_matrix)
```

### Adding constraints
There are many ways to impose constraints on the decision variables. This tutorial shows a few simple examples. Refer to the links at the [bottom](#Advanced-tutorials) of this document for other types of constraints.

#### AddConstraint
The simplest way to add a constraint is with `MathematicalProgram.AddConstraint()`.

```python
# Add the constraint x(0) * x(1) = 1 to prog
prog.AddConstraint(x[0] * x[1] == 1)
```

You can also add inequality constraints to `prog` such as

```python
prog.AddConstraint(x[0] >= 0)
prog.AddConstraint(x[0] - x[1] <= 0)
```

`prog` automatically analyzes these symbolic inequality constraint expressions and determines they are all *linear* constraints on $x$.

### Adding Cost functions
In a complicated optimization problem, it is often convenient to write the total cost function $f(x)$ as a sum of individual cost functions
\begin{align}
f(x) = \sum_i g_i(x)
\end{align}

#### AddCost method.
The simplest way to add an individual cost function $g_i(x)$ to the total cost function $f(x)$ is with the `MathematicalProgram.AddCost()` method (as shown below).

```python
# Add a cost x(0)**2 + 3 to the total cost. Since prog doesn't have a cost before, now the total cost is x(0)**2 + 3
prog.AddCost(x[0] ** 2 + 3)
```

To add another individual cost function $x(0) + x(1)$ to the total cost function $f(x)$, simply call `AddCost()` again as follows

```python
prog.AddCost(x[0] + x[1])
```

now the total cost function becomes $x(0)^2 + x(0) + x(1) + 3$.

`prog` can analyze each of these individual cost functions and determine that $x(0) ^ 2 + 3$  is a convex quadratic function, and $x(0) + x(1)$ is a linear function of $x$.

### Solve the optimization problem
Once all the decision variables/constraints/costs are added to `prog`, we are ready to solve the optimization problem.

#### Automatically choosing a solver
The simplest way to solve the optimization problem is to call `Solve()` function. Drake's MathematicalProgram analyzes the type of the constraints/costs, and then calls an appropriate solver for your problem. The result of calling `Solve()` is stored inside the return argument. Here is a code snippet

```python
"""
Solves a simple optimization problem
       min x(0)^2 + x(1)^2
subject to x(0) + x(1) = 1
           x(0) <= x(1)
"""
from pydrake.solvers.mathematicalprogram import Solve
# Set up the optimization problem.
prog = MathematicalProgram()
x = prog.NewContinuousVariables(2)
prog.AddConstraint(x[0] + x[1] == 1)
prog.AddConstraint(x[0] <= x[1])
prog.AddCost(x[0] **2 + x[1] ** 2)

# Now solve the optimization problem.
result = Solve(prog)

# print out the result.
print("Success? ", result.is_success())
# Print the solution to the decision variables.
print('x* = ', result.GetSolution(x))
# Print the optimal cost.
print('optimal cost = ', result.get_optimal_cost())
# Print the name of the solver that was called.
print('solver is: ', result.get_solver_id().name())
```

Notice that we can then retrieve optimization result from the return argument of `Solve`. For example, the solution $x^*$ is retrieved from `result.GetSolution()`, and the optimal cost from `result.get_optimal_cost()`.

Some optimization solution is infeasible (doesn't have a solution). For example in the following code example, `result.get_solution_result()` will not report `kSolutionFound`.

```python
"""
An infeasible optimization problem.
"""
prog = MathematicalProgram()
x = prog.NewContinuousVariables(1)[0]
y = prog.NewContinuousVariables(1)[0]
prog.AddConstraint(x + y >= 1)
prog.AddConstraint(x + y <= 0)
prog.AddCost(x)

result = Solve(prog)
print("Success? ", result.is_success())
print(result.get_solution_result())
```

#### Manually choosing a solver

If you want to choose a solver yourself, rather than Drake choosing one for you, you could instantiate a solver explicitly, and call its `Solve` function. There are two apporaches to instantiate a solver. For example, if I want to solve a problem using the open-source solver [IPOPT](https://github.com/coin-or/Ipopt), I can instantiate the solver using either of the two approaches:
1. The simplest approach is to call `solver = IpoptSolver()`
2. The second approach is to construct a solver with a given solver ID as `solver = MakeSolver(IpoptSolver().solver_id())`

```python
"""
Demo on manually choosing a solver
Solves the problem
min x(0)
s.t x(0) + x(1) = 1
    0 <= x(1) <= 1
"""
from pydrake.solvers.ipopt import IpoptSolver
prog = MathematicalProgram()
x = prog.NewContinuousVariables(2)
prog.AddConstraint(x[0] + x[1] == 1)
prog.AddConstraint(0 <= x[1])
prog.AddConstraint(x[1] <= 1)
prog.AddCost(x[0])

# Choose IPOPT as the solver.
# First instantiate an IPOPT solver.

solver = IpoptSolver()
# The initial guess is [1, 1]. The third argument is the options for Ipopt solver,
# and we set no solver options.
result = solver.Solve(prog, np.array([1, 1]), None)

print(result.get_solution_result())
print("x* = ", result.GetSolution(x))
print("Solver is ", result.get_solver_id().name())
print("Ipopt solver status: ", result.get_solver_details().status,
      ", meaning ", result.get_solver_details().ConvertStatusToString())

```

Note that `solver.Solve()` expects three input arguments, the optimization program `prog`, the initial guess of the decision variable values (`[1, 1]` in this case), and an optional setting for the solver (`None` in this case, we use the default IPOPT setting). If you don't have an initial guess, you could call `solver.Solve(prog)`. Drake will choose a default initial guess (a 0-valued vector), but this initial guess might be a bad starting point for optimization. Note from the following example code, with the default initial guess, the solver cannot find a solution, even though a solution exists (and could be found with initial guess [1, 1]).

```python
from pydrake.solvers.mathematicalprogram import MakeSolver
solver = MakeSolver(IpoptSolver().solver_id())
result = solver.Solve(prog)
print(result.get_solution_result())
print("x* = ", result.GetSolution(x))
```

Also note that if we know which solver is called, then we can access some solver-specific result, by calling `result.get_solver_details()`. For example, `IpoptSolverDetails` contains a field `status`, namely the status code of the IPOPT solver, we could access this info by

```python
print("Ipopt solver status: ", result.get_solver_details().status,
      ", meaning ", result.get_solver_details().ConvertStatusToString())
```

Each solver has its own details. You should refer to `FooSolverDetails` class on what is stored inside the return argument of `result.get_solver_details()`. For example, if you know that IPOPT is called, then refer to `IpoptSolverDetails` class; for OSQP solver, refer to `OsqpSolverDetails`, etc.

### Using an initial guess
Some optimization problems, such as nonlinear optimization, require an initial guess. Other types of problems, such as quadratic programming, mixed-integer optimization, etc,  can be solved faster if a good initial guess is provided. The user could provide an initial guess as an input argument in `Solve` function. If no initial guess is provided, Drake will use a zero-valued vector as the initial guess.

In the example below, we show that an initial guess could affect the result of the problem. Without an user-provided initial guess, the solver might be unable to find the solution.

```python
from pydrake.solvers.ipopt import IpoptSolver
prog = MathematicalProgram()
x = prog.NewContinuousVariables(2)
prog.AddConstraint(x[0]**2 + x[1]**2 == 100.)
prog.AddCost(x[0]**2-x[1]**2)
solver = IpoptSolver()
# The user doesn't provide an initial guess.
result = solver.Solve(prog, None, None)
print(f"Without a good initial guess, the result is {result.is_success()}")
print(f"solution {result.GetSolution(x)}")
# Pass an initial guess
result = solver.Solve(prog, [-5., 0.], None)
print(f"With a good initial guess, the result is {result.is_success()}")
print(f"solution {result.GetSolution(x)}")
```

For more details on setting the initial guess, the user could refer to [Nonlinear program](./nonlinear_program.ipynb) section `Setting the initial guess`.

## Add callback
Some solvers support adding a callback function in each iteration. One usage of the callback is to visualize the solver progress in the current iteration. `MathematicalProgram` supports this usage through the function `AddVisualizationCallback`, although the usage is not limited to just visualization, the callback function can do anything. Here is an example

```python
# Visualize the solver progress in each iteration through a callback
# Find the closest point on a curve to a desired point.

fig = plt.figure()
curve_x = np.linspace(1, 10, 100)
ax = plt.gca()
ax.plot(curve_x, 9./curve_x)
ax.plot(-curve_x, -9./curve_x)
ax.plot(0, 0, 'o')
x_init = [4., 5.]
point_x, = ax.plot(x_init[0], x_init[1], 'x')
ax.axis('equal')

def update(x):
    global iter_count
    point_x.set_xdata(x[0])
    point_x.set_ydata(x[1])
    ax.set_title(f"iteration {iter_count}")
    fig.canvas.draw()
    fig.canvas.flush_events()
    # Also update the iter_count variable in the callback.
    # This shows we can do more than just visualization in
    # callback.
    iter_count += 1
    plt.pause(0.1)

iter_count = 0
prog = MathematicalProgram()
x = prog.NewContinuousVariables(2)
prog.AddConstraint(x[0] * x[1] == 9)
prog.AddCost(x[0]**2 + x[1]**2)
prog.AddVisualizationCallback(update, x)
result = Solve(prog, x_init)
```

## Advanced tutorials
[Setting solver parameters](./solver_parameters.ipynb)

[Updating costs and constraints (e.g. for efficient solving of many similar programs)](./updating_costs_and_constraints.ipynb)

[Debugging tips](./debug_mathematical_program.ipynb)

[Linear program](./linear_program.ipynb)

[Quadratic program](./quadratic_program.ipynb)

[Nonlinear program](./nonlinear_program.ipynb)

[Sum-of-squares optimization](./sum_of_squares_optimization.ipynb)


# Non-Linear Shooting Method

To numerically approximate the non-linear Boundary Value Problem
$$
y^{''}=f(x,y,y^{'}) \ \ \ a < x < b $$
$$y(a)=\alpha$$
$$y(b) =\beta$$

The Boundary Value Problem is divided into two Initial Value Problems:
1. The first 2nd order non-linear Intial Value Problem is the same as the original Boundary Value Problem with an extra initial condtion $y_1^{'}(a)=\lambda_0$.
\begin{equation}
 y^{''}_1=f(x,y,y^{'}), \ \    y_1(a)=\alpha, \ \ \color{red}{y^{'}_1(a)=\lambda_0},\\
\end{equation}
2. The second 2nd order Intial Value Problem is the homogenous form of the original Boundary Value Problem with the initial condtions $z(a)=0$ and $z^{'}(a)=1$.

\begin{equation}
z^{''}=\frac{\partial f}{\partial y^{'}}z^{'}(x,\lambda)+\frac{\partial f}{\partial y}z(x,\lambda), \ \ \color{green}{z(a)=0, \ \ z^{'}(a)=1}.
\end{equation}
combining these results together to get the unique solution
\begin{equation}
y(x)=y_1(x).
\end{equation}
Unlike the linear method, the non-linear shooting method is iterative to get the value of $\lambda$ that results in the same solution as the Boundary Value Problem.

The first choice of $\lambda_0$ is a guess. Then after the first iteration a Newton Raphson method is used to update $\lambda$

$$\lambda_k=\lambda_{k-1}-\frac{y(b,\lambda_{k-1})-\beta}{\frac{dy}{d \lambda}(b,\lambda_{k-1})}$$
until $\lambda_{k+1}-\lambda_{k+1}<tol$.

```python
import numpy as np
import math
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings("ignore")
class ListTable(list):
    """ Overridden list class which takes a 2-dimensional list of
        the form [[1,2,3],[4,5,6]], and renders an HTML Table in
        IPython Notebook. """
from IPython.core.display import HTML

def _repr_html_(self):
    html = ["<table>"]
    for row in self:
        html.append("<tr>")

        for col in row:
            html.append("<td>{0}</td>".format(col))

        html.append("</tr>")
    html.append("</table>")
    return ''.join(html)
from IPython.display import HTML
HTML('''
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>''')
```

<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>

## Example Non-linear Boundary Value Problem
To illustrate the shooting method we shall apply it to the non-linear Boundary Value Problem:
$$ y^{''}=-2yy^{'}, $$
with boundary conditions
$$y(0) = -2.5, $$
$$y(1) = 3. $$
The __boundary value problem__ is broken into two second order __Initial Value Problems:__
1. The first 2nd order Intial Value Problem is the same as the original Boundary Value Problem with an extra initial condtion $y^{'}(0)=\lambda_0$.
\begin{equation}
u^{''} =-2uu', \ \ \ \ u(0)=-2.5, \ \ \ \color{green}{u^{'}(0)=\lambda_0}
\end{equation}
2. The second 2nd order Intial Value Problem is the homogenous form of the original Boundary Value Problem with the initial condtions $z^{'}(0)=0$ and $z^{'}(0)=1$.
\begin{equation}
z^{''} =-2y^{'}z-2yz^{'}, \ \ \ \ \color{green}{z(1)=0}, \ \ \ \color{green}{z^{'}(1)=1}
\end{equation}

The first choice of $\lambda$ we guess $\lambda_0=0.2$. Then after the first iteration a Newton Raphson method is used to update $\lambda$

$$\lambda_k=\lambda_{k-1}-\frac{y(b,\lambda_{k-1})-\beta}{\frac{dy}{d \lambda}(b,\lambda_{k-1})}$$
until $\lambda_{k+1}-\lambda_{k+1}<0.001$.

## Discrete Axis
The stepsize is defined as
$$h=\frac{b-a}{N}$$
here it is
$$h=\frac{1-0}{10}$$
giving
$$x_i=0+0.1 i$$
for $i=0,1,...10.$

```python
## BVP
N=10
h=1/N
x=np.linspace(0,1,N+1)
fig = plt.figure(figsize=(10,4))
plt.plot(x,0*x,'o:',color='red')
plt.xlim((0,1))
plt.xlabel('x',fontsize=16)
plt.title('Illustration of discrete time points for h=%s'%(h),fontsize=32)
plt.show()
```

## Initial conditions
The initial conditions for the discrete equations are:
$$ u_1[0]=-2.5$$
$$ u_2[0]=\lambda_0$$
$$ z_1[0]=0$$
$$ z_2[0]=1$$
$$ \beta=3$$
Let $\lambda_0=0.2$

```python
U1=np.zeros(N+1)
U2=np.zeros(N+1)
Z1=np.zeros(N+1)
Z2=np.zeros(N+1)

lambda_app=[0.2]
U1[0]=-2.5
U2[0]=lambda_app[0]

Z1[0]=0
Z2[0]=1
beta=3
```

## Numerical method
The Euler method is applied to numerically approximate the solution of the system of the two second order non-linear initial value problems they are converted in to two pairs of two first order non-linear initial value problems:
1. Discrete form of Equation 1
$$u_{1 }[i+1]=u_{1}[i] + h u_{2 }[i]$$
$$u_{2 }[i+1]=u_{2 }[i] + h (-2u_{1}[i]u_{1 }[i])$$
with the initial conditions $u_{1}[0]=-2.5$ and $u_{2}[0]=\lambda_0.$

2. Discrete form of Equation 2
$$z_{1}[i+1]=z_{1 }[i] + h z_{2 }[i]$$
$$z_{2}[i+1]=z_{2 }[i] + h (-2z_{2}[i]y_{1 }[i]-2z_{1 }[i] y_{2 }[i])$$
with the initial conditions $z_{1}[0]=0$ and $z_{2}[0]=1$.

At the end of each iteration
$$ U1[i] \approx y(x_i).$$
The initial choice of $\lambda_0=0.2$ is up to the user then for each iteration of the system $\lambda$ is updated using the formula:
$$\lambda_k=\lambda_{k-1}-\frac{u_1[10]-3}{z_1[10]}.$$

The plot below shows the numerical approximation of the solution to the non-linear Boundary Value Problem for each iteration.

The stopping criteria for the iterative process is
$$|\lambda_k-\lambda_{k-1}|<tol$$
with $tol=0.0001.$

```python
tol=0.0001
k=0
fig = plt.figure(figsize=(10,4))
while  k<10:
    k=k+1
    for i in range (0,N):
        U1[i+1]=U1[i]+h*(U2[i])
        U2[i+1]=U2[i]+h*(-2*U2[i]*U1[i])

        Z1[i+1]=Z1[i]+h*(Z2[i])
        Z2[i+1]=Z2[i]+h*(-2*U2[i]*Z1[i]-2*Z2[i]*U1[i])

    lambda_app.append(lambda_app[k-1]-(U1[N]-beta)/Z1[N])

    plt.plot(x,U1,':o',label=r"$\lambda$ %s"%(k))
    plt.xlabel('x',fontsize=16)
    plt.ylabel('U1',fontsize=16)
    U2[0]=lambda_app[k]
    if abs(lambda_app[k]-lambda_app[k-1])<tol:
        break

plt.legend(loc='best')
plt.title(r"Iterations of $\lambda_k$",fontsize=32)
plt.show()
```

## Results
The plot below shows the final iteration of the numerical approximation for the non-linear boundary value problem.

```python
fig = plt.figure(figsize=(10,4))
plt.rc('text', usetex=True)
plt.rc('font', family='serif')
plt.grid(True)
plt.plot(x,U1,'b:o')
plt.title(r"Numerical Approximation of the non-linear Boundary Value Problem",fontsize=32)
plt.xlabel('x',fontsize=16)
plt.ylabel(r"$u_1$",fontsize=16)
plt.show()

```

## $\lambda$ Iteration
The plot below shows the iterations of $\lambda_k$.

```python
fig = plt.figure(figsize=(10,4))
plt.rc('text', usetex=True)
plt.rc('font', family='serif')
plt.grid(True)
plt.plot(lambda_app,'o')
plt.title(r"Values of $\lambda$ for each interation ",fontsize=32)
plt.xlabel('Iterations (k)',fontsize=16)
plt.ylabel(r"Values of $\lambda_k$",fontsize=16)
plt.show()
```

```python

```

Chapter 06 - Boundary Value Problems/.ipynb_checkpoints/Non-Linear Shooting Method-checkpoint.ipynb


# SymPy Exercises

```python
import sympy as sym
from sympy import init_session
init_session()
```

    IPython console for SymPy 1.6 (Python 3.8.3-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at https://docs.sympy.org/1.6/

## Q1: Creating an expression

Create the expression:

$$f = x e^{-x} + x (1-x)$$

Then evaluate it for

$$x = 0, 0.1, 0.2, 0.4, 0.8$$

```python

```

## Q2: Factoring a polynomial, and finding its roots

Factor

$$x^{4} - 6 x^{3} + x^{2} + 24 x + 16$$

Then find its zeros.

```python

```

## Q3: Integratation and differentiation

Integrate the function:

$$
f = \sin(x) e^{-x}
$$

Then differentiate the result to see if you get back the original function

```python

```

## Q4: Parsing an expression

Write a program that reads in a mathematical expression as a string (e.g., `"sin(2*pi*x)"`), converts it to a SymPy expression, and then evaluates it as needed.

Have your program either make a plot of the entered function, or use the input function as the function to fit a dataset to using curvefit.

The following will be helpful:

`parse_expr()` will convert a string into a SymPy expression

```python
from sympy.parsing.sympy_parser import parse_expr
```

```python
s = "sin(2*pi*x)"
a = parse_expr(s)
a
```

`sympy.lambdify()` will convert a SymPy expression into a function that is callable by python.  You can make it a numpy-compatible function too (this means, e.g., that any `sin()` in your SymPy expression will be evaluate using `np.sin()`)

```python
f = sym.lambdify(x, a, "numpy")
```

```python
f(1.0)
```

```python
help(lambdify)
```

    Help on function lambdify in module sympy.utilities.lambdify:

    lambdify(args, expr, modules=None, printer=None, use_imps=True, dummify=False)
        Convert a SymPy expression into a function that allows for fast
        numeric evaluation.

        .. warning::
           This function uses ``exec``, and thus shouldn't be used on
           unsanitized input.

        Explanation
        ===========

        For example, to convert the SymPy expression ``sin(x) + cos(x)`` to an
        equivalent NumPy function that numerically evaluates it:

        >>> from sympy import sin, cos, symbols, lambdify
        >>> import numpy as np
        >>> x = symbols('x')
        >>> expr = sin(x) + cos(x)
        >>> expr
        sin(x) + cos(x)
        >>> f = lambdify(x, expr, 'numpy')
        >>> a = np.array([1, 2])
        >>> f(a)
        [1.38177329 0.49315059]

        The primary purpose of this function is to provide a bridge from SymPy
        expressions to numerical libraries such as NumPy, SciPy, NumExpr, mpmath,
        and tensorflow. In general, SymPy functions do not work with objects from
        other libraries, such as NumPy arrays, and functions from numeric
        libraries like NumPy or mpmath do not work on SymPy expressions.
        ``lambdify`` bridges the two by converting a SymPy expression to an
        equivalent numeric function.

        The basic workflow with ``lambdify`` is to first create a SymPy expression
        representing whatever mathematical function you wish to evaluate. This
        should be done using only SymPy functions and expressions. Then, use
        ``lambdify`` to convert this to an equivalent function for numerical
        evaluation. For instance, above we created ``expr`` using the SymPy symbol
        ``x`` and SymPy functions ``sin`` and ``cos``, then converted it to an
        equivalent NumPy function ``f``, and called it on a NumPy array ``a``.

        Parameters
        ==========

        args : List[Symbol]
            A variable or a list of variables whose nesting represents the
            nesting of the arguments that will be passed to the function.

            Variables can be symbols, undefined functions, or matrix symbols.

            >>> from sympy import Eq
            >>> from sympy.abc import x, y, z

            The list of variables should match the structure of how the
            arguments will be passed to the function. Simply enclose the
            parameters as they will be passed in a list.

            To call a function like ``f(x)`` then ``[x]``
            should be the first argument to ``lambdify``; for this
            case a single ``x`` can also be used:

            >>> f = lambdify(x, x + 1)
            >>> f(1)
            2
            >>> f = lambdify([x], x + 1)
            >>> f(1)
            2

            To call a function like ``f(x, y)`` then ``[x, y]`` will
            be the first argument of the ``lambdify``:

            >>> f = lambdify([x, y], x + y)
            >>> f(1, 1)
            2

            To call a function with a single 3-element tuple like
            ``f((x, y, z))`` then ``[(x, y, z)]`` will be the first
            argument of the ``lambdify``:

            >>> f = lambdify([(x, y, z)], Eq(z**2, x**2 + y**2))
            >>> f((3, 4, 5))
            True

            If two args will be passed and the first is a scalar but
            the second is a tuple with two arguments then the items
            in the list should match that structure:

            >>> f = lambdify([x, (y, z)], x + y + z)
            >>> f(1, (2, 3))
            6

        expr : Expr
            An expression, list of expressions, or matrix to be evaluated.

            Lists may be nested.
            If the expression is a list, the output will also be a list.

            >>> f = lambdify(x, [x, [x + 1, x + 2]])
            >>> f(1)
            [1, [2, 3]]

            If it is a matrix, an array will be returned (for the NumPy module).

            >>> from sympy import Matrix
            >>> f = lambdify(x, Matrix([x, x + 1]))
            >>> f(1)
            [[1]
            [2]]

            Note that the argument order here (variables then expression) is used
            to emulate the Python ``lambda`` keyword. ``lambdify(x, expr)`` works
            (roughly) like ``lambda x: expr``
            (see :ref:`lambdify-how-it-works` below).

        modules : str, optional
            Specifies the numeric library to use.

            If not specified, *modules* defaults to:

            - ``["scipy", "numpy"]`` if SciPy is installed
            - ``["numpy"]`` if only NumPy is installed
            - ``["math", "mpmath", "sympy"]`` if neither is installed.

            That is, SymPy functions are replaced as far as possible by
            either ``scipy`` or ``numpy`` functions if available, and Python's
            standard library ``math``, or ``mpmath`` functions otherwise.

            *modules* can be one of the following types:

            - The strings ``"math"``, ``"mpmath"``, ``"numpy"``, ``"numexpr"``,
              ``"scipy"``, ``"sympy"``, or ``"tensorflow"``. This uses the
              corresponding printer and namespace mapping for that module.
            - A module (e.g., ``math``). This uses the global namespace of the
              module. If the module is one of the above known modules, it will
              also use the corresponding printer and namespace mapping
              (i.e., ``modules=numpy`` is equivalent to ``modules="numpy"``).
            - A dictionary that maps names of SymPy functions to arbitrary
              functions
              (e.g., ``{'sin': custom_sin}``).
            - A list that contains a mix of the arguments above, with higher
              priority given to entries appearing first
              (e.g., to use the NumPy module but override the ``sin`` function
              with a custom version, you can use
              ``[{'sin': custom_sin}, 'numpy']``).

        dummify : bool, optional
            Whether or not the variables in the provided expression that are not
            valid Python identifiers are substituted with dummy symbols.

            This allows for undefined functions like ``Function('f')(t)`` to be
            supplied as arguments. By default, the variables are only dummified
            if they are not valid Python identifiers.

            Set ``dummify=True`` to replace all arguments with dummy symbols
            (if ``args`` is not a string) - for example, to ensure that the
            arguments do not redefine any built-in names.

        Examples
        ========

        >>> from sympy.utilities.lambdify import implemented_function
        >>> from sympy import sqrt, sin, Matrix
        >>> from sympy import Function
        >>> from sympy.abc import w, x, y, z

        >>> f = lambdify(x, x**2)
        >>> f(2)
        4
        >>> f = lambdify((x, y, z), [z, y, x])
        >>> f(1,2,3)
        [3, 2, 1]
        >>> f = lambdify(x, sqrt(x))
        >>> f(4)
        2.0
        >>> f = lambdify((x, y), sin(x*y)**2)
        >>> f(0, 5)
        0.0
        >>> row = lambdify((x, y), Matrix((x, x + y)).T, modules='sympy')
        >>> row(1, 2)
        Matrix([[1, 3]])

        ``lambdify`` can be used to translate SymPy expressions into mpmath
        functions. This may be preferable to using ``evalf`` (which uses mpmath on
        the backend) in some cases.

        >>> import mpmath
        >>> f = lambdify(x, sin(x), 'mpmath')
        >>> f(1)
        0.8414709848078965

        Tuple arguments are handled and the lambdified function should
        be called with the same type of arguments as were used to create
        the function:

        >>> f = lambdify((x, (y, z)), x + y)
        >>> f(1, (2, 4))
        3

        The ``flatten`` function can be used to always work with flattened
        arguments:

        >>> from sympy.utilities.iterables import flatten
        >>> args = w, (x, (y, z))
        >>> vals = 1, (2, (3, 4))
        >>> f = lambdify(flatten(args), w + x + y + z)
        >>> f(*flatten(vals))
        10

        Functions present in ``expr`` can also carry their own numerical
        implementations, in a callable attached to the ``_imp_`` attribute. This
        can be used with undefined functions using the ``implemented_function``
        factory:

        >>> f = implemented_function(Function('f'), lambda x: x+1)
        >>> func = lambdify(x, f(x))
        >>> func(4)
        5

        ``lambdify`` always prefers ``_imp_`` implementations to implementations
        in other namespaces, unless the ``use_imps`` input parameter is False.

        Usage with Tensorflow:

        >>> import tensorflow as tf
        >>> from sympy import Max, sin, lambdify
        >>> from sympy.abc import x

        >>> f = Max(x, sin(x))
        >>> func = lambdify(x, f, 'tensorflow')

        After tensorflow v2, eager execution is enabled by default.
        If you want to get the compatible result across tensorflow v1 and v2
        as same as this tutorial, run this line.

        >>> tf.compat.v1.enable_eager_execution()

        If you have eager execution enabled, you can get the result out
        immediately as you can use numpy.

        If you pass tensorflow objects, you may get an ``EagerTensor``
        object instead of value.

        >>> result = func(tf.constant(1.0))
        >>> print(result)
        tf.Tensor(1.0, shape=(), dtype=float32)
        >>> print(result.__class__)
        <class 'tensorflow.python.framework.ops.EagerTensor'>

        You can use ``.numpy()`` to get the numpy value of the tensor.

        >>> result.numpy()
        1.0

        >>> var = tf.Variable(2.0)
        >>> result = func(var) # also works for tf.Variable and tf.Placeholder
        >>> result.numpy()
        2.0

        And it works with any shape array.

        >>> tensor = tf.constant([[1.0, 2.0], [3.0, 4.0]])
        >>> result = func(tensor)
        >>> result.numpy()
        [[1. 2.]
         [3. 4.]]

        Notes
        =====

        - For functions involving large array calculations, numexpr can provide a
          significant speedup over numpy. Please note that the available functions
          for numexpr are more limited than numpy but can be expanded with
          ``implemented_function`` and user defined subclasses of Function. If
          specified, numexpr may be the only option in modules. The official list
          of numexpr functions can be found at:
          https://numexpr.readthedocs.io/en/latest/user_guide.html#supported-functions

        - In previous versions of SymPy, ``lambdify`` replaced ``Matrix`` with
          ``numpy.matrix`` by default. As of SymPy 1.0 ``numpy.array`` is the
          default. To get the old default behavior you must pass in
          ``[{'ImmutableDenseMatrix':  numpy.matrix}, 'numpy']`` to the
          ``modules`` kwarg.

          >>> from sympy import lambdify, Matrix
          >>> from sympy.abc import x, y
          >>> import numpy
          >>> array2mat = [{'ImmutableDenseMatrix': numpy.matrix}, 'numpy']
          >>> f = lambdify((x, y), Matrix([x, y]), modules=array2mat)
          >>> f(1, 2)
          [[1]
           [2]]

        - In the above examples, the generated functions can accept scalar
          values or numpy arrays as arguments.  However, in some cases
          the generated function relies on the input being a numpy array:

          >>> from sympy import Piecewise
          >>> from sympy.testing.pytest import ignore_warnings
          >>> f = lambdify(x, Piecewise((x, x <= 1), (1/x, x > 1)), "numpy")

          >>> with ignore_warnings(RuntimeWarning):
          ...     f(numpy.array([-1, 0, 1, 2]))
          [-1.   0.   1.   0.5]

          >>> f(0)
          Traceback (most recent call last):
              ...
          ZeroDivisionError: division by zero

          In such cases, the input should be wrapped in a numpy array:

          >>> with ignore_warnings(RuntimeWarning):
          ...     float(f(numpy.array([0])))
          0.0

          Or if numpy functionality is not required another module can be used:

          >>> f = lambdify(x, Piecewise((x, x <= 1), (1/x, x > 1)), "math")
          >>> f(0)
          0

        .. _lambdify-how-it-works:

        How it works
        ============

        When using this function, it helps a great deal to have an idea of what it
        is doing. At its core, lambdify is nothing more than a namespace
        translation, on top of a special printer that makes some corner cases work
        properly.

        To understand lambdify, first we must properly understand how Python
        namespaces work. Say we had two files. One called ``sin_cos_sympy.py``,
        with

        .. code:: python

            # sin_cos_sympy.py

            from sympy import sin, cos

            def sin_cos(x):
                return sin(x) + cos(x)

        and one called ``sin_cos_numpy.py`` with

        .. code:: python

            # sin_cos_numpy.py

            from numpy import sin, cos

            def sin_cos(x):
                return sin(x) + cos(x)

        The two files define an identical function ``sin_cos``. However, in the
        first file, ``sin`` and ``cos`` are defined as the SymPy ``sin`` and
        ``cos``. In the second, they are defined as the NumPy versions.

        If we were to import the first file and use the ``sin_cos`` function, we
        would get something like

        >>> from sin_cos_sympy import sin_cos # doctest: +SKIP
        >>> sin_cos(1) # doctest: +SKIP
        cos(1) + sin(1)

        On the other hand, if we imported ``sin_cos`` from the second file, we
        would get

        >>> from sin_cos_numpy import sin_cos # doctest: +SKIP
        >>> sin_cos(1) # doctest: +SKIP
        1.38177329068

        In the first case we got a symbolic output, because it used the symbolic
        ``sin`` and ``cos`` functions from SymPy. In the second, we got a numeric
        result, because ``sin_cos`` used the numeric ``sin`` and ``cos`` functions
        from NumPy. But notice that the versions of ``sin`` and ``cos`` that were
        used was not inherent to the ``sin_cos`` function definition. Both
        ``sin_cos`` definitions are exactly the same. Rather, it was based on the
        names defined at the module where the ``sin_cos`` function was defined.

        The key point here is that when function in Python references a name that
        is not defined in the function, that name is looked up in the "global"
        namespace of the module where that function is defined.

        Now, in Python, we can emulate this behavior without actually writing a
        file to disk using the ``exec`` function. ``exec`` takes a string
        containing a block of Python code, and a dictionary that should contain
        the global variables of the module. It then executes the code "in" that
        dictionary, as if it were the module globals. The following is equivalent
        to the ``sin_cos`` defined in ``sin_cos_sympy.py``:

        >>> import sympy
        >>> module_dictionary = {'sin': sympy.sin, 'cos': sympy.cos}
        >>> exec('''
        ... def sin_cos(x):
        ...     return sin(x) + cos(x)
        ... ''', module_dictionary)
        >>> sin_cos = module_dictionary['sin_cos']
        >>> sin_cos(1)
        cos(1) + sin(1)

        and similarly with ``sin_cos_numpy``:

        >>> import numpy
        >>> module_dictionary = {'sin': numpy.sin, 'cos': numpy.cos}
        >>> exec('''
        ... def sin_cos(x):
        ...     return sin(x) + cos(x)
        ... ''', module_dictionary)
        >>> sin_cos = module_dictionary['sin_cos']
        >>> sin_cos(1)
        1.38177329068

        So now we can get an idea of how ``lambdify`` works. The name "lambdify"
        comes from the fact that we can think of something like ``lambdify(x,
        sin(x) + cos(x), 'numpy')`` as ``lambda x: sin(x) + cos(x)``, where
        ``sin`` and ``cos`` come from the ``numpy`` namespace. This is also why
        the symbols argument is first in ``lambdify``, as opposed to most SymPy
        functions where it comes after the expression: to better mimic the
        ``lambda`` keyword.

        ``lambdify`` takes the input expression (like ``sin(x) + cos(x)``) and

        1. Converts it to a string
        2. Creates a module globals dictionary based on the modules that are
           passed in (by default, it uses the NumPy module)
        3. Creates the string ``"def func({vars}): return {expr}"``, where ``{vars}`` is the
           list of variables separated by commas, and ``{expr}`` is the string
           created in step 1., then ``exec``s that string with the module globals
           namespace and returns ``func``.

        In fact, functions returned by ``lambdify`` support inspection. So you can
        see exactly how they are defined by using ``inspect.getsource``, or ``??`` if you
        are using IPython or the Jupyter notebook.

        >>> f = lambdify(x, sin(x) + cos(x))
        >>> import inspect
        >>> print(inspect.getsource(f))
        def _lambdifygenerated(x):
            return (sin(x) + cos(x))

        This shows us the source code of the function, but not the namespace it
        was defined in. We can inspect that by looking at the ``__globals__``
        attribute of ``f``:

        >>> f.__globals__['sin']
        <ufunc 'sin'>
        >>> f.__globals__['cos']
        <ufunc 'cos'>
        >>> f.__globals__['sin'] is numpy.sin
        True

        This shows us that ``sin`` and ``cos`` in the namespace of ``f`` will be
        ``numpy.sin`` and ``numpy.cos``.

        Note that there are some convenience layers in each of these steps, but at
        the core, this is how ``lambdify`` works. Step 1 is done using the
        ``LambdaPrinter`` printers defined in the printing module (see
        :mod:`sympy.printing.lambdarepr`). This allows different SymPy expressions
        to define how they should be converted to a string for different modules.
        You can change which printer ``lambdify`` uses by passing a custom printer
        in to the ``printer`` argument.

        Step 2 is augmented by certain translations. There are default
        translations for each module, but you can provide your own by passing a
        list to the ``modules`` argument. For instance,

        >>> def mysin(x):
        ...     print('taking the sin of', x)
        ...     return numpy.sin(x)
        ...
        >>> f = lambdify(x, sin(x), [{'sin': mysin}, 'numpy'])
        >>> f(1)
        taking the sin of 1
        0.8414709848078965

        The globals dictionary is generated from the list by merging the
        dictionary ``{'sin': mysin}`` and the module dictionary for NumPy. The
        merging is done so that earlier items take precedence, which is why
        ``mysin`` is used above instead of ``numpy.sin``.

        If you want to modify the way ``lambdify`` works for a given function, it
        is usually easiest to do so by modifying the globals dictionary as such.
        In more complicated cases, it may be necessary to create and pass in a
        custom printer.

        Finally, step 3 is augmented with certain convenience operations, such as
        the addition of a docstring.

        Understanding how ``lambdify`` works can make it easier to avoid certain
        gotchas when using it. For instance, a common mistake is to create a
        lambdified function for one module (say, NumPy), and pass it objects from
        another (say, a SymPy expression).

        For instance, say we create

        >>> from sympy.abc import x
        >>> f = lambdify(x, x + 1, 'numpy')

        Now if we pass in a NumPy array, we get that array plus 1

        >>> import numpy
        >>> a = numpy.array([1, 2])
        >>> f(a)
        [2 3]

        But what happens if you make the mistake of passing in a SymPy expression
        instead of a NumPy array:

        >>> f(x + 1)
        x + 2

        This worked, but it was only by accident. Now take a different lambdified
        function:

        >>> from sympy import sin
        >>> g = lambdify(x, x + sin(x), 'numpy')

        This works as expected on NumPy arrays:

        >>> g(a)
        [1.84147098 2.90929743]

        But if we try to pass in a SymPy expression, it fails

        >>> try:
        ...     g(x + 1)
        ... # NumPy release after 1.17 raises TypeError instead of
        ... # AttributeError
        ... except (AttributeError, TypeError):
        ...     raise AttributeError() # doctest: +IGNORE_EXCEPTION_DETAIL
        Traceback (most recent call last):
        ...
        AttributeError:

        Now, let's look at what happened. The reason this fails is that ``g``
        calls ``numpy.sin`` on the input expression, and ``numpy.sin`` does not
        know how to operate on a SymPy object. **As a general rule, NumPy
        functions do not know how to operate on SymPy expressions, and SymPy
        functions do not know how to operate on NumPy arrays. This is why lambdify
        exists: to provide a bridge between SymPy and NumPy.**

        However, why is it that ``f`` did work? That's because ``f`` doesn't call
        any functions, it only adds 1. So the resulting function that is created,
        ``def _lambdifygenerated(x): return x + 1`` does not depend on the globals
        namespace it is defined in. Thus it works, but only by accident. A future
        version of ``lambdify`` may remove this behavior.

        Be aware that certain implementation details described here may change in
        future versions of SymPy. The API of passing in custom modules and
        printers will not change, but the details of how a lambda function is
        created may change. However, the basic idea will remain the same, and
        understanding it will be helpful to understanding the behavior of
        lambdify.

        **In general: you should create lambdified functions for one module (say,
        NumPy), and only pass it input types that are compatible with that module
        (say, NumPy arrays).** Remember that by default, if the ``module``
        argument is not provided, ``lambdify`` creates functions using the NumPy
        and SciPy namespaces.

```python

```

## Q5: Units

SymPy can deal with physical units.  See:

http://docs.sympy.org/latest/modules/physics/units/quantities.html

Let's try this out.  Newton's 2nd law is
$$F = ma$$

Create a mass of 1 kg and an acceleration of 10 m/s$^2$, and compute the force, $F$, and express the result in Newtons.

Note: the `convert_to` function was added in SymPy 1.1, so if you are using an earlier version, you will need to divide by the target unit to do the conversion.

```python

```