# Logistic Regression

Researchers are often interested in setting up a model to analyze the relationship between predictors (i.e., independent variables) and it's corresponsing response (i.e., dependent variable). Linear regression is commonly used when the response variable is continuous.  One assumption of linear models is that the residual errors follow a normal distribution. This assumption fails when the response variable is categorical, so an ordinary linear model is not appropriate. This newsletter presents a regression model for response variable that is dichotomous–having two categories. Examples are common: whether a plant lives or dies, whether a survey respondent agrees or disagrees with a statement, or whether an at-risk child graduates or drops out from high school.

In ordinary linear regression, the response variable (Y) is a linear function of the coefficients (B0, B1, etc.) that correspond to the predictor variables (X1, X2, etc.,). A typical model would look like:

    Y = B0 + B1*X1 + B2*X2 + B3*X3 + … + E

For a dichotomous response variable, we could set up a similar linear model to predict individual category memberships if numerical values are used to represent the two categories. Arbitrary values of 1 and 0 are chosen for mathematical convenience. Using the first example, we would assign Y = 1 if a plant lives and Y = 0 if a plant dies.

This linear model does not work well for a few reasons. First, the response values, 0 and 1, are arbitrary, so modeling the actual values of Y is not exactly of interest. Second, it is the probability that each individual in the population responds with 0 or 1 that we are interested in modeling. For example, we may find that plants with a high level of a fungal infection (X1) fall into the category “the plant lives” (Y) less often than those plants with low level of infection. Thus, as the level of infection rises, the probability of plant living decreases.

Thus, we might consider modeling P, the probability, as the response variable. Again, there are problems. Although the general decrease in probability is accompanied by a general increase in infection level, we know that P, like all probabilities, can only fall within the boundaries of 0 and 1. Consequently, it is better to assume that the relationship between X1 and P is sigmoidal (S-shaped), rather than a straight line.

It is possible, however, to find a linear relationship between X1 and function of P. Although a number of functions work, one of the most useful is the logit function. It is the natural log of the odds that Y is equal to 1, which is simply the ratio of the probability that Y is 1 divided by the probability that Y is 0. The relationship between the logit of P and P itself is sigmoidal in shape. The regression equation that results is:

    ln[P/(1-P)] = B0 + B1*X1 + B2*X2 + …

Although the left side of this equation looks intimidating, this way of expressing the probability results in the right side of the equation being linear and looking familiar to us. This helps us understand the meaning of the regression coefficients. The coefficients can easily be transformed so that their interpretation makes sense.

The logistic regression equation can be extended beyond the case of a dichotomous response variable to the cases of ordered categories and polytymous categories (more than two categories).

# Mathematics behind Logistic Regression

## Notation

The problem structure is the classic classification problem. Our data set $\mathcal{D}$ is composed of $N$ samples. Each sample is a tuple containing a feature vector and a label. For any sample $n$ the feature vector is a $d+1$ dimensional column vector denoted by ${\bf x}_n$ with $d$ real-valued components known as features. Samples are represented in homogeneous form with the first component equal to $1$: $x_0=1$. Vectors are bold-faced. The associated label is denoted $y_n$ and can take only two values: $+1$ or $-1$.

$$
\mathcal{D} = \lbrace ({\bf x}_1, y_1), ({\bf x}_2, y_2), ..., ({\bf x}_N, y_N) \rbrace \\
{\bf x}_n = \begin{bmatrix} 1 & x_1 & ... & x_d \end{bmatrix}^T
$$

## Learning Algorithm

The learning algorithm is how we search the set of possible hypotheses (hypothesis space $\mathcal{H}$) for the best parameterization (in this case the weight vector ${\bf w}$). This search is an optimization problem looking for the hypothesis that optimizes an error measure.

There is no sophisticted, closed-form solution like least-squares linear, so we will use gradient descent instead. Specifically we will use batch gradient descent which calculates the gradient from all data points in the data set.

Luckily, our "cross-entropy" error measure is convex so there is only one minimum. Thus the minimum we arrive at is the global minimum.

Gradient descent is a general method and requires twice differentiability for smoothness. It updates the parameters using a first-order approximation of the error surface.

$$
{\bf w}_{i+1} = {\bf w}_i + \nabla E_\text{in}({\bf w}_i)
$$

To learn we're going to minimize the following error measure using batch gradient descent.

$$
e(h({\bf x}_n), y_n) = \ln \left( 1+e^{-y_n \; {\bf w}^T {\bf x}_n} \right) \\
E_\text{in}({\bf w}) = \frac{1}{N} \sum_{n=1}^{N} e(h({\bf x}_n), y_n) = \frac{1}{N} \sum_{n=1}^{N} \ln \left( 1+e^{-y_n \; {\bf w}^T {\bf x}_n} \right)
$$

We'll need the derivative of the point loss function and possibly some abuse of notation.

$$
\frac{d}{d{\bf w}} e(h({\bf x}_n), y_n)
= \frac{-y_n \; {\bf x}_n \; e^{-y_n {\bf w}^T {\bf x}_n}}{1 + e^{-y_n {\bf w}^T {\bf x}_n}}
= -\frac{y_n \; {\bf x}_n}{1 + e^{y_n {\bf w}^T {\bf x}_n}}
$$

With the point loss derivative we can determine the gradient of the in-sample error:

$$
\begin{align}
\nabla E_\text{in}({\bf w})
&= \frac{d}{d{\bf w}} \left[ \frac{1}{N} \sum_{n=1}^N e(h({\bf x}_n), y_n) \right] \\
&= \frac{1}{N} \sum_{n=1}^N \frac{d}{d{\bf w}} e(h({\bf x}_n), y_n) \\
&= \frac{1}{N} \sum_{n=1}^N \left( - \frac{y_n \; {\bf x}_n}{1 + e^{y_n {\bf w}^T {\bf x}_n}} \right) \\
&= - \frac{1}{N} \sum_{n=1}^N \frac{y_n \; {\bf x}_n}{1 + e^{y_n {\bf w}^T {\bf x}_n}} \\
\end{align}
$$

Our weight update rule per batch gradient descent becomes

$$
\begin{align}
{\bf w}_{i+1} &= {\bf w}_i - \eta \; \nabla E_\text{in}({\bf w}_i) \\
&= {\bf w}_i - \eta \; \left( - \frac{1}{N} \sum_{n=1}^N \frac{y_n \; {\bf x}_n}{1 + e^{y_n {\bf w}_i^T {\bf x}_n}} \right) \\
&= {\bf w}_i + \eta \; \left( \frac{1}{N} \sum_{n=1}^N \frac{y_n \; {\bf x}_n}{1 + e^{y_n {\bf w}_i^T {\bf x}_n}} \right) \\
\end{align}
$$

where $\eta$ is our learning rate.

### Enough with the theory, now jump to the implimentation. We will look at 2 libraries for the same.

## Logistic Regression with statsmodel

We'll be using the same dataset as UCLA's Logit Regression tutorial to explore logistic regression in Python. Our goal will be to identify the various factors that may influence admission into graduate school.

The dataset contains several columns which we can use as predictor variables:

   * gpa
   * gre score
   * rank or prestige of an applicant's undergraduate alma mater
   * The fourth column, admit, is our binary target variable. It indicates whether or not a candidate was admitted our not.

```python
import numpy as np
import pandas as pd
import pylab as pl
import statsmodels.api as sm

```

```python
df = pd.read_csv("binary.csv")
#df = pd.read_csv("https://stats.idre.ucla.edu/stat/data/binary.csv")
```

```python
df.head()
```

      admit
      gre
      gpa
      rank

      0
      0
      380
      3.61
      3

      1
      1
      660
      3.67
      3

      2
      1
      800
      4.00
      1

      3
      1
      640
      3.19
      4

      4
      0
      520
      2.93
      4

```python
#RENAMING THE RANK COLUMN
df.columns = ["admit", "gre", "gpa", "prestige"]
df.head()
#df.shape
```

      admit
      gre
      gpa
      prestige

      0
      0
      380
      3.61
      3

      1
      1
      660
      3.67
      3

      2
      1
      800
      4.00
      1

      3
      1
      640
      3.19
      4

      4
      0
      520
      2.93
      4

### Summary Statistics & Looking at the data
Now that we've got everything loaded into Python and named appropriately let's take a look at the data. We can use the pandas function which describes a summarized view of everything. There's also function for calculating the standard deviation, std.

A feature I really like in pandas is the pivot_table/crosstab aggregations. crosstab makes it really easy to do multidimensional frequency tables. You might want to play around with this to look at different cuts of the data.

```python
pd.crosstab(df["admit"],df["prestige"],rownames = ["admit"])
```

      prestige
      1
      2
      3
      4

      admit

      0
      28
      97
      93
      55

      1
      33
      54
      28
      12

```python
df.hist()
pl.show()
```

```python

```

 ### dummy variables
pandas gives you a great deal of control over how categorical variables can be represented. We're going dummify the "prestige" column using get_dummies.

get_dummies creates a new DataFrame with binary indicator variables for each category/option in the column specified. In this case, prestige has four levels: 1, 2, 3 and 4 (1 being most prestigious). When we call get_dummies, we get a dataframe with four columns, each of which describes one of those levels.

```python
dummy_ranks = pd.get_dummies(df["prestige"], prefix = "prestige")
```

```python
dummy_ranks.head()
```

      prestige_1
      prestige_2
      prestige_3
      prestige_4

      0
      0
      0
      1
      0

      1
      0
      0
      1
      0

      2
      1
      0
      0
      0

      3
      0
      0
      0
      1

      4
      0
      0
      0
      1

```python
# CREATING A CLEAN DATA FRAME
cols_to_keep = ["admit", "gre", "gpa"]
data = df[cols_to_keep].join(dummy_ranks.loc[:,"prestige_2":])
data.head()
```

      admit
      gre
      gpa
      prestige_2
      prestige_3
      prestige_4

      0
      0
      380
      3.61
      0
      1
      0

      1
      1
      660
      3.67
      0
      1
      0

      2
      1
      800
      4.00
      0
      0
      0

      3
      1
      640
      3.19
      0
      0
      1

      4
      0
      520
      2.93
      0
      0
      1

Once that's done, we merge the new dummy columns with the original dataset and get rid of the prestige column which we no longer need.

Lastly we're going to add a constant term for our logistic regression. The statsmodels function we would use requires intercepts/constants to be specified explicitly.

### Performing the regression
Actually doing the logistic regression is quite simple. Specify the column containing the variable you're trying to predict followed by the columns that the model should use to make the prediction.

In our case we'll be predicting the admit column using gre, gpa, and the prestige dummy variables prestige_2, prestige_3 and prestige_4. We're going to treat prestige_1 as our baseline and exclude it from our fit. This is done to prevent multicollinearity, or the dummy variable trap caused by including a dummy variable for every single category.

```python
#ADDING THE INTERCEPT MANUALLY
data["intercept"] = 1.0
data.head()
```

      admit
      gre
      gpa
      prestige_2
      prestige_3
      prestige_4
      intercept

      0
      0
      380
      3.61
      0
      1
      0
      1.0

      1
      1
      660
      3.67
      0
      1
      0
      1.0

      2
      1
      800
      4.00
      0
      0
      0
      1.0

      3
      1
      640
      3.19
      0
      0
      1
      1.0

      4
      0
      520
      2.93
      0
      0
      1
      1.0

```python
train_cols = data.columns[1:]

logit = sm.Logit(data["admit"], data[train_cols])
```

```python
results = logit.fit()
```

    Optimization terminated successfully.
             Current function value: 0.573147
             Iterations 6

Since we're doing a logistic regression, we're going to use the statsmodels Logit function. For details on other models available in statsmodels, check out their docs here.

### Interpreting the results
One of my favorite parts about statsmodels is the summary output it gives. If you're coming from R, I think you'll like the output and find it very familiar too.

```python
ironman = results.predict([800,4,0,0,0,1.0])
```

```python
print(ironman)
```

    [0.73840825]

```python
results.summary()
```

Logit Regression Results

  Dep. Variable:	         admit	        No. Observations:  	     400

  Model:	                 Logit	        Df Residuals:      	     394

  Method:	                 MLE	         Df Model:          	       5

  Date:	            Fri, 20 Nov 2020	   Pseudo R-squ.:     	  0.08292

  Time:	                13:11:55	       Log-Likelihood:    	  -229.26

  converged:	             True	         LL-Null:           	  -249.99

  Covariance Type:	     nonrobust	      LLR p-value:       	 7.578e-08

       	         coef	     std err	      z	      P>|z|	  [0.025	    0.975]

  gre	            0.0023	     0.001	     2.070	  0.038	     0.000	     0.004

  gpa	            0.8040	     0.332	     2.423	  0.015	     0.154	     1.454

  prestige_2	    -0.6754	     0.316	    -2.134	  0.033	    -1.296	    -0.055

  prestige_3	    -1.3402	     0.345	    -3.881	  0.000	    -2.017	    -0.663

  prestige_4	    -1.5515	     0.418	    -3.713	  0.000	    -2.370	    -0.733

  intercept	     -3.9900	     1.140	    -3.500	  0.000	    -6.224	    -1.756

```python

```


# SYMPY

```python
# las funciones que usaremos durante la práctica
from sympy import Symbol, symbols, factor, expand, simplify, solve, pprint, plot, Derivative, Integral, sin
```

```python
x = Symbol('x')
```

```python
x + x + 1
```

    2*x + 1

```python
type(2*x)
```

    sympy.core.mul.Mul

```python
type(2*x+1)
```

    sympy.core.add.Add

## Atajo para describir incógnitas de una vez

```python
x = Symbol('x')
x
```

    x

```python
y = Symbol('y')
y
```

    y

```python
z = Symbol('z')
z
```

    z

```python
# equivalente a
x, y, z = symbols('x,y,z')
x, y, z
```

    (x, y, z)

## Simplificaciones básicas

```python
x*y+x*y
```

    2*x*y

```python
x * (x+x)
```

    2*x**2

```python
x * y / y
```

    x

## Expandir y factorizar

```python
factor(x**2 - y**2)
```

    (x - y)*(x + y)

```python
factor(x**3 + 3*x**2*y + 3*x*y**2 + y**3)
```

    (x + y)**3

```python
expand((x - y)*(x + y))
```

    x**2 - y**2

```python
expand((x - 1)*(x + 1)*(2*x+3))
```

    2*x**3 + 3*x**2 - 2*x - 3

## Evaluar

```python
# evaluar en un punto f(1, 2)
f = x*x + x*y + x*y + y*y
f.subs({x: 1, y: 2})
```

    9

```python
# cambio de variable x -> 1-y
f.subs({x: 1-y})
```

    y**2 + 2*y*(-y + 1) + (-y + 1)**2

```python
simplify(_)
```

    1

```python
# cambio de variable x -> 2-y**2
f.subs({x: 2-y**2})
```

    y**2 + 2*y*(-y**2 + 2) + (-y**2 + 2)**2

```python
simplify(_)
```

    y**2 - 2*y*(y**2 - 2) + (y**2 - 2)**2

```python
expand(_)
```

    y**4 - 2*y**3 - 3*y**2 + 4*y + 4

## Despejando

```python
a, b, c = symbols("a,b,c")
```

```python
f = x**2 + 5*x + 4      # f(x) = 0
solve(f, dict=True)     # despeja (resuelve) x
```

    [{x: -4}, {x: -1}]

```python
f = a*x**2 + b*x + c    # f(x, a, b, c) = 0
solve(f, x, dict=True)  # despeja x
```

    [{x: (-b + sqrt(-4*a*c + b**2))/(2*a)}, {x: -(b + sqrt(-4*a*c + b**2))/(2*a)}]

```python
pprint(_)
```

    ⎡⎧           _____________⎫  ⎧    ⎛       _____________⎞ ⎫⎤
    ⎢⎪          ╱           2 ⎪  ⎪    ⎜      ╱           2 ⎟ ⎪⎥
    ⎢⎨   -b + ╲╱  -4⋅a⋅c + b  ⎬  ⎨   -⎝b + ╲╱  -4⋅a⋅c + b  ⎠ ⎬⎥
    ⎢⎪x: ─────────────────────⎪, ⎪x: ────────────────────────⎪⎥
    ⎣⎩            2⋅a         ⎭  ⎩             2⋅a           ⎭⎦

```python
pprint(solve(f, a, dict=True))
```

    ⎡⎧   -(b⋅x + c) ⎫⎤
    ⎢⎪a: ───────────⎪⎥
    ⎢⎨         2    ⎬⎥
    ⎢⎪        x     ⎪⎥
    ⎣⎩              ⎭⎦

## Representando

```python
plot(2*x**2 + 3*x)
```

```python
plot(2*x**2 + 3*x, (x, -5, 5))
```

## Derivadas e integrales

```python
Derivative(2*x**2+3*x+1, x).doit()
```

    4*x + 3

```python
Derivative(2*sin(x)*x**2, x).doit()
```

    2*x**2*cos(x) + 4*x*sin(x)

```python
Integral(4*x + 3, x).doit()
```

    2*x**2 + 3*x

```python
Integral(sin(x)**2, x).doit()
```

    x/2 - sin(x)*cos(x)/2


# 04 Scipy

**For a better understanding of this notebook, the 03 Scipy notebook should be known.**
This script will go beyond descriptive statistics as shown in the previous notebook. One main purpose is the filling of gaps in data set. First, by applying some easy (and not so easy) interpolation algorithms. Second, by setting up a linear regression to another data set and use other data to fill the gap (which will be done as a project). Lastly, finding derivatives and antiderivatives will be introduced.

```python
%matplotlib inline
import pandas as pd
import numpy as np
import requests
import matplotlib.pyplot as plt
```

## Interpolation

### 1D interpolation

The Scipy module offers the interpolate submodule, that can be used for interpolation. Procedural functions are offered along with high level interpolator classes, that offer moethods beyond simple gap filling. From simple linear interpolations to piecewise cubic splines most common interpolation algorithms can be found. In order to illustrate the algorithm itself, the interpolation will be performed on generated data.

```python
from scipy.interpolate import interp1d
interp1d??
```

```python
# generate an example data set
x = np.linspace(0, 10, num=11)
y = np.sin(-x**2/13.)

# interpolate
f = interp1d(x,y, kind='linear')
xi = np.linspace(0, 10, num=100)

# plot
fig, ax = plt.subplots(1,1)
ax.plot(x,y,'ob')
ax.plot(xi, f(xi), '-r', label='linear')
plt.legend(loc='upper left')
```

```python
fq = interp1d(x,y,kind='quadratic')
fc = interp1d(x,y,kind='cubic')
fn = interp1d(x,y,kind='nearest')

# plot
fig, ax = plt.subplots(1,1)
ax.plot(x,y,'ob')
ax.plot(xi, fq(xi), '-r', label='quadratic')
ax.plot(xi, fc(xi), '-g', label='cubic')
ax.plot(xi, fn(xi), '--k', label='nearest')
plt.legend(loc='upper left')
```

In terms of  filling gaps in environmental data, the univariate linear, quadratic and cubic functions, along with the nearest neighbour interpolator shall fit most requirements. Although it might not seem to be very useful in the given example, the nearest neighbour algorithm is especially usefull in case the set of observations shall be preserved and no new values shall be added. This is almost always the case for classified data.
The cubic and quadratic interpolations are in fact spline interpolators. The interp1d can also apply higher splines by giving the order instead of the method as the *kind* argument.

```python
x = np.linspace(0,10, num=15)
np.random.seed(938)
y = np.random.gamma(1.5, 7, size=15)

# plot
fig, ax = plt.subplots(1,1)
ax.plot(x,y, 'ob')
```

```python
xi = np.linspace(0,10, num=100)

# plot
fig, axes = plt.subplots(3,2, figsize=(12,9))

for i,deg in zip(range(6), [1,2,3,5,7,9]):
    f = interp1d(x,y,kind=deg)
    axes.flatten()[i].plot(x,y,'ob')
    axes.flatten()[i].plot(xi, f(xi), '-g')
    axes.flatten()[i].set_title('%d splines' % deg)
```

### Linear Regression

In contrast to the interpolation algorithms a linear regression will find the best suitable parameter set to fit a predefined function to the set of data. In most implementations the *best* fit will be evaluated on the basis of either the *least squre method* or a *maximum likelihood* approach. This should not be confused with an interpolation, where **unknown** values are tried to be guessed based on the known values.

A short example shall illustrate this. We will use the scipy module to fint different function to the observations from above, then the difference shall be clear. The first example is a linear model, then a polynomial expression.

```python
from scipy.optimize import curve_fit
```

```python
# define the linear model
lin = lambda x,m,b: m*x + b

# fit
cof, cov = curve_fit(lin,x,y)

# plot
fig, ax = plt.subplots(1,1)

ax.plot(x,y, 'ob')
ax.plot(xi,[lin(_, *cof) for _ in xi], '--g', lw=2)
```

```python
# prepare the functions
poly2 = lambda x,a,b,c: a*x**2 + b*x + c
poly3 = lambda x,a,b,c,d: a*x**3 + b*x**2 + c*x + d
poly4 = lambda x,a,b,c,d,e: a*x**4 + b*x**3 + c*x**2 + d*x + e

# plot
fig, ax = plt.subplots(1,1)
ax.plot(x,y,'ob')

for model, c in zip((poly2, poly3, poly4), ('b', 'g', 'k')):
    cof, cov = curve_fit(model, x, y)
    ax.plot(xi, [model(_, *cof) for _ in xi], '--%s' % c, lw=2)
```

```python
# poly 7
poly7 = lambda x,a,b,c,d,e,f,g,h: a*x**7 + b*x**6 + c*x**5 + d*x**4 + e*x**3 + f*x**2 + g*x + h

# fit
cof, cov = curve_fit(poly7, x, y)

# plot
fig, ax = plt.subplots(1,1)

ax.plot(x,y,'ob')
ax.plot(xi, [poly7(_, *cof) for _ in xi], '-g', lw=2)
```

### 2D interpolation

The Scipy stack includes a number of approaches, functions and classes which are intended for multivariate interpolation, with an even stronger subset for 2D interpolation. Most of these classes need an in-depth understanding of the math behind the algorithm in order to avoid misuage and errors. These kinds of algorithms and approaches go beyond the scope of this notebook and will not be introduced. However, the usage of the respective classes and functions is very similar to the ones introduced here.
Therefore, let's generate a continous field.

```python
# define a field
rf = lambda x, y: np.sin(x) + np.sin(y)
```

At first, we need a regular aranged grid, where we can apply the field to. This grid will represent the *truth* of our observation. In an application of these techniques, this field is what we are actually interested in. This example will illustrate how near we can come using the interpolators for this very easy approach of an mathematical describeable field.

```python
# generate a meshgrid with range()
t = np.linspace(-3, 3, 100)
xx,yy = np.meshgrid(t,t)
#xx, yy = np.mgrid[-3:3:100j, -3:3:100j]
zz = rf(xx,yy)

fig, ax = plt.subplots(1,1, figsize=(6,6))
ax.pcolor(xx,yy,zz)

# add contour lines
contour = ax.contour(xx,yy,zz, colors='k')
ax.clabel(contour)
```

Next, we will randomly select some samples from this environment and check, how different interpolations perform here.

```python
# select sample points
n = 30
sample = np.random.rand(n,2) * 6 - 3

# unstack
xi = sample[:, 0]
yi = sample[:, 1]
zi = rf(xi, yi)

fig, ax = plt.subplots(1,1, figsize=(6,6))
ax.pcolor(xx,yy,zz)
ax.plot(xi, yi, 'or')
contour = ax.contour(xx,yy,zz, colors='k')
ax.clabel(contour)
```

Similar to the interp1d, the scipy package offers an interp2d algorithm, which in fact does not interpolate but approximate. The actual interpolation algorithm is called *griddata*, which can be used. It has to be mentioned, that this is an blackbox function, that just takes the input data.

```python
from scipy.interpolate import griddata, SmoothBivariateSpline
```

```python
# interpolate using the nearest neighbors
Zn = griddata(sample, zi, (xx,yy), method='nearest')
Zl = griddata(sample, zi, (xx,yy), method='linear')
Zc = griddata(sample, zi, (xx,yy), method='cubic')

fig, axes = plt.subplots(1,3, figsize=(12,4))

for ax, Z in zip(axes, (Zn, Zl, Zc)):
    # plot
    ax.matshow(Z, origin='lower', cmap='RdYlBu')
    ax.scatter((xi + 3) / 6 * 100, (yi + 3) / 6 * 100,25,(1,0,0))
```

```python
griddata(sample, zi, (xx,yy), method='nearest')
```

    array([[-1.13391809, -1.13391809, -1.13391809, ...,  0.12989957,
             0.12989957,  0.12989957],
           [-1.13391809, -1.13391809, -1.13391809, ...,  0.12989957,
             0.12989957,  0.12989957],
           [-1.13391809, -1.13391809, -1.13391809, ...,  0.12989957,
             0.12989957,  0.12989957],
           ...,
           [ 0.81530046,  0.81530046, -0.62004579, ...,  1.62501462,
             1.62501462,  1.62501462],
           [ 0.81530046, -0.62004579, -0.62004579, ...,  1.62501462,
             1.62501462,  1.62501462],
           [ 0.81530046, -0.62004579, -0.62004579, ...,  1.62501462,
             1.62501462,  1.62501462]])

As a last interpolation example, one of the classes is used: SmoothBivariareSpline. This class is quite handy as the order of splines can be set for each variable independently, e.g. you might use cubic splines in x-direction but a linear interpolation on the y-axis.

```python
# use a spline of order 3
interpolant = SmoothBivariateSpline(xi, yi, zi, kx=3, ky=3)

fig, ax = plt.subplots(1,1, figsize=(6,6))
ax.pcolor(xx,yy,interpolant(t, t))

# add contour lines
contour = ax.contour(xx,yy, interpolant(t, t), colors='k')
ax.clabel(contour)

# add the sample
ax.scatter(xi, yi, 25, zi)
plt.xlim(-3, 3)
plt.ylim(-3, 3)
```

```python
# calculate the residuals
residuals = np.abs(zz - interpolant(t,t))

fig, ax = plt.subplots(1,1)
ax.pcolor(xx,yy,residuals, cmap='RdBu_r')
ax.scatter(xi, yi, 25, (1,0,0))
plt.xlim(-3, 3)
plt.ylim(-3, 3)
```

```python
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm

# calc the colormap
cmap = cm.jet(interpolant(t,t) / np.amax(interpolant(t,t)))
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_surface(xx,yy,interpolant(t,t), alpha=0.5, facecolors=cmap)
```

### Integration

Integration is a common tool is several environmental models and can be used to evaluate the integrative information of a function, especially in non-linear cases. In principle, scipy offer different approaches for solving an integration problem, numerical integration and symbolic integration.

  * **numerical integration:** these algorithms will find a numerical approximation of the integral. This is kind of *always working*,but these are apprximations and will introduce an error to the result. This error might or might not be significant.
  * **symbolic integration:** by symbolic integration scipy will try to solve the integral analytically. Obviously this will only work if there is an analytical solution for the problem and a solver implemented. If available, this is the most correct solution as no approximations are used.
  * **class specific integration:** some of scipy's interpolation classes also implement methods for finding derivatives and integrals. These methods are very specific and therefore are usually the best choice.

#### polynomial integration

For polynomials, the numpy packages offers a really fast and exact solver based on the Fundamental Theorem of Calculus. This is a method of the polynomial object. Let's solve an rather easy example:
$$ f(x) = -x^2 + 5 $$

```python
# create the polynom
p = np.poly1d([-1, 0, 5])
#p = np.poly1d([2])

# plot
x = np.linspace(-3,3, 100)
fig, ax = plt.subplots(1, 1)
ax.plot(x, p(x), '-b', lw=2)
plt.grid()
```

```python
# get the integral coefficent
print(p.integ())
```

             3
    -0.3333 x + 5 x

```python
# get the integral on [0,2]
_x = np.linspace(0, 2, 100)

# plot
fig, ax = plt.subplots(1, 1)
ax.plot(x, p(x), '-b', lw=2)
ax.fill_between(_x, 0, p(_x), facecolor='yellow', alpha=0.5)
plt.grid()

p.integ()(2)
```

#### sympy

**sympy** is an important package for **sym**bolic **py**thon, that can solve a lot of problems related to symbolic mathematics. One application is solving symbolic integration problems. One of the main advantages (like with the polnomial's integ method) is the set up of antiderivative. Unlike the polynomial class from numpy, the sympy functions can also be used to solve non-polynomial functions. Let's illustrate this on two easy examples.

\begin{equation}
f(x) = -x^2 + 5
\end{equation}

\begin{equation}
f(x) = sin\left(\frac{-x^2}{13}\right)
\end{equation}

```python
from sympy import latex,pprint, integrate, symbols, sin as Sin
```

In order to make a variable *symbolic* and therefore for sympy useable, these variables have to be instantiated as *Symbol* class instances. The *symbols* class factory can be used for this.
The integrate method can then return the antiderivative and, if a interval is given, the definite solution of the antiderivative on this integral.

```python
# make z a symbol
z = symbols('z', real=True)

#print the antiderivate
print('Antiderivative:')
pprint(integrate(-z**2 + 5))

# solve for [0:3]
print('Solution [0,3):', integrate(-z**2 + 5, (z, 0,3)))
```

    Antiderivative:
       3
      z
    - ── + 5⋅z
      3
    Solution [0,3): 6

```python
integrate(-z**2 + 5)
```

    -z**3/3 + 5*z

The

```python
from sympy.plotting import plot as sym_plot
# make z2 an symbol
z = symbols('z', real=True)

# print the antiderivative
print('Antiderivative:')
pprint(integrate(-Sin(z**2 / 13.)))

sym_plot(-Sin(z**2 / 13.))

# solve for [-5,5]
print('Solution [-5,5):', integrate(-Sin(z**2 / 13.), (z, -5, 5)))
```

```python
integrate(-Sin(z**2 / 13.))
```

    -1.91213231759729*sqrt(pi)*fresnels(0.392232270276368*z/sqrt(pi))*gamma(3/4)/gamma(7/4)

#### numerical integration

Any type of numerical solution for an integration problem is solved by finding an approximation. Obviously there are tons of solutions with different advantages and disadvantages. Some approximations are fast, others are almost impossible to solve within a reasonable time span.
The scipy.interpolate package has two implementations that will fit almost all requirements of environmental scientist when it comes to integration. The function *cumtrapz* is an implementation of the trapezodial rule and the *simp* function, which is an implementation of the Simpson's rule.

```python
from scipy.integrate import cumtrapz, simps
```

```python
# setup the function
f = lambda x : -np.sin(-x**2 / 13.)
x = np.linspace(-5,5, num=100)

print('trazezodial:\t', cumtrapz(f(x), x)[-1])
print('Simpson\'s:\t', simps(f(x), x))
```

    trazezodial:	 4.90337498374
    Simpson's:	 4.90377460764

```python
# performance
f = lambda x : x**3 + x**2
z = symbols('z', real=True)

%timeit integrate(z**3+z**2, (z, -5, 5))
%timeit cumtrapz(f(x), x)
%timeit simps(f(x), x)
```

    The slowest run took 12.69 times longer than the fastest. This could mean that an intermediate result is being cached.
    1000 loops, best of 3: 739 µs per loop
    10000 loops, best of 3: 29.9 µs per loop
    10000 loops, best of 3: 95.7 µs per loop

##### Test scenario for integration precision using polynomials

$$ p(x) = 5*x^5 + x^4 + 3*x^3 + \frac{1}{3}*x^2 - \frac{3}{4}*x + 6$$

```python
fp = lambda x: 5*x**5 + x**4 + 3*x**3 + (1/3) *x**2 - (3/4)*x + 6
nodes = np.linspace(0,3, num=10)
p = np.poly1d([5, 1, 3, (1/3), -(3/4), 6])

print('Real: ',p.integ()(3))
print('Trpz: ', cumtrapz(fp(nodes), nodes)[-1])
print('Simps:', simps(fp(nodes), nodes))
```

    Real:  734.475
    Trpz:  754.945987654
    Simps: 738.397290809

```python
# plot
%matplotlib inline
fig, ax = plt.subplots(1,1)

tests = np.logspace(1,5,num=30)
align = lambda n: np.linspace(0,3,num=n)

ax.axhline(y=734.475, color='r', lw=2)
ax.semilogx(tests, [cumtrapz(fp(align(n)), align(n))[-1] for n in tests], '.-b')
plt.grid(which='both')
```


# Sobel filter edge detection

This project utilises Sobel filter for an edge detection problem. This is often used in image processing, which can be applied in face recognition systems, extracting buildings from remotely senses images of cities or surface cracks caused by earthquakes from satelite images. A sample image of the Old Town in Warsaw is filtered using Sobel operator.

```python
import numpy as np
import matplotlib.pyplot as plt
import imageio
```

```python
#plotting images
def make_image(inputname, title,sizeX,sizeY):
    n_images = len(inputname)
    plt.figure(figsize=(sizeX,sizeY))
    for i in range(n_images):
        plt.subplot(1,n_images,i+1)
        fig = plt.imshow(inputname[i])
        fig.axes.get_xaxis().set_visible(False)
        fig.axes.get_yaxis().set_visible(False)
        fig.set_cmap('gray') # ignore for rgb images
        plt.title(title[i])

#convert RGB to grayscale
def rgb2gray(rgb):
    grayscale = np.dot(rgb[...,:3], [0.299, 0.587, 0.114])
    return np.rint(grayscale)
```

```python
#read and display the image
original_image = imageio.imread('data/image.png')
make_image([original_image],["Original image"],7,7)
```

Given a function $\mathbf{B}$ and a filter function $\mathbf{f}$, most filtering operations are implemented using a convolution
integral to produce an output $\mathbf{O}$.

\begin{equation}
\mathbf{O}(t) =\sum_{y=-\infty}^{y=\infty}\sum_{x=-\infty}^{x=\infty} \mathbf{B} (\tau_{x,y}) \mathbf{f} (t_{x,y}-\tau_{x,y})d\tau_{x,y}
\end{equation}

Here, the convulation is performed by sliding the filter kernel $\mathbf{f}$ over the image data $\mathbf{f}$. This is started from the top left corner and goes across all the positiontions of the pixels, where the filter fits entirely.

Let's look at the Sobel filter kernels:

\begin{equation*}
\begin{aligned}[c]
\mathbf{f}_x=\begin{bmatrix}
    +1 & 0 & -1 \\
    +2 & 0 & -2 \\
    +1 & 0 & -1
  \end{bmatrix}
\end{aligned}
\begin{aligned}[c]
\mathbf{f}_y=\begin{bmatrix}
    +1 & +2 & +1 \\
    0 & 0 & 0 \\
    -1 & -2 & -1
  \end{bmatrix}
\end{aligned}
\end{equation*}

Therefore, the convuation can be rewritten as:

\begin{equation}
\mathbf{O}(t) =\sum_{k=1}^{m-1}\sum_{l=1}^{n-1} \mathbf{B}(i+k,j+l)\mathbf{f}(k,l)
\end{equation}

where, $i$, $j$ are locations in $\mathbf{B}$ and $\mathbf{O}$; $k$, $l$ are locations in $\mathbf{f}$; and $m$ and $n$ are the number of rows and columns in the kernel respectively.

The filter is applied in both directions, to the original image. To get the fully filtered image the output of both kernels are combined using the root of the squares of the individual outputs:

\begin{equation}
\mathbf{O} = \sqrt{\mathbf{O}_x^2 + \mathbf{O}_y^2}
\end{equation}

```python
image=rgb2gray(original_image) # convert rgb image to grayscale

kernelX = np.array([[+1, 0 ,-1], [+2, 0, -2], [+1, 0, -1]]) #create kernel X filter
kernelY = np.array([[+1, +2, +1], [0, 0, 0], [-1, -2, -1]]) #create kernel Y filter

# Sobel-filtered variants of the original image
sobelX = np.array([[np.sum(image[j-1:j+2, i-1:i+2] * kernelX) for i in range(1, image.shape[1]-1)]
                   for j in range(1, image.shape[0]-1)])
sobelY = np.array([[np.sum(image[j-1:j+2, i-1:i+2] * kernelY) for i in range(1, image.shape[1]-1)]
                   for j in range(1, image.shape[0]-1)])

sobel = np.sqrt(sobelX**2 + sobelY**2)
```

## Display images

```python
make_image([original_image,sobel],["Original image","Sobel filtered image"],14,14)
```

```python
make_image([sobelX,sobelY],["Sobel X","Sobel Y"],14,14)
```

```python

```


# Rate Equations

Recall, a rate equation looks like this:
$$\frac{dy}{dt} = f(y,t).$$
* $y$ is the dependent variable and is a function of $t$, that is $y(t)$.
* $t$ is the independent variable.
* $f(y,t)$ is some function of $y$ and $t$; that is, some formula written in terms of $y$ and possibly $t$.
    * We call this function the *rate* or the *right hand side function* (RHS).
* There is also an initial condition $$y(0)=y_0.$$

### Example Radioactive decay.

$$\frac{dC}{dt} = -\frac{C}{\tau}.$$
The exact solution is
$$C(t) = C_0e^{-t/\tau},$$
where $C_0$ is the initial concentration.

### Pretest: ```odeint```

* New function ```odeint```
* ```from scipy.integrate import odeint```
* Questions:
    * What arguments will this need?
    * What will it return?
    * Will you need to define a function and what form will it have?
    * What does the *solution* look like? (Like, what do you get as an *answer*?)
    * What were the things we needed/did in Excel?
* Try to solve the above ODE with $\tau=1$, $C_0=1$ and $t_{end}=5$.

#### Numerical Solution:

$$\frac{dC}{dt} = -\frac{C}{\tau}.$$

1. define the function ```f(y,t)```
2. set the desired solution times ```t```
3. set the initial condition ```C0```
4. call ```odeint``` as ```C=odeint(f, C0, times)```
    * ```from scipy.integrate import odeint```
5. plot the results

Solve this problem with $\tau=1$, $C_0=1$, and t_end=5.

```python
import numpy             as np
import matplotlib.pyplot as plt
%matplotlib inline
from scipy.integrate     import odeint
```

```python
#------ Step 1, define the function

def rhsf(C, t, τ):
    return -C/τ

#------ Step 2, set times to get the solution at

t = np.linspace(0, 5, 1000)

#------ Step 3, set initial condition

C0 = 1

#------ Step 4, solve the problem

τ=1
C = odeint(rhsf, C0, t, args=(τ,))

#------ Step 5 plot the answer

plt.rc('font', size=14)
plt.plot(t,C)
plt.xlabel('t')
plt.ylabel('C(t)');

#------ Step 6: does the answer make sense? Analyse it.
```

### Exercise

Solve the rate equation for $v(t)$ at discrete points $t$ and plot the result:

$$\frac{dv}{dt} = g - cv^2.$$

* $g=9.81$, $c=1$.
* $v_0 = 0$.
* Solve to $t=2$.

```python
def rhsf(v, t):
    g = 9.81
    c = 1
    dvdt = g - c*v**2
    return dvdt

v0 = 0.0
t = np.linspace(0,2,200)
v = odeint(rhsf, v0, t)

plt.plot(t,v)
plt.xlabel('t')
plt.ylabel('v(t)')

```

## Multiple equations

What is we have multiple rate equations in multiple variables?

If we have multiple rate equations in multiple unknowns, we solve the problem in the same way, but using vector notation.
* This is similar to what we do for ```fsolve```.

### Example

Solve for the velocity of a falling object:
\begin{align}
\frac{dv}{dt} &= g - cv^2, \\
\frac{dx}{dt} &= v.
\end{align}

Here, g is gravitational acceleration, x is position, v is velocity, and c is a drag coefficient.

We first cast this in the form $$\frac{d\vec{y}}{dt} = \vec{f}(\vec{y},t).$$

This is equivalent to
\begin{align}
\frac{dy_0}{dt} &= f_0(y_0,y_1,t) \\
\frac{dy_1}{dt} &= f_1(y_0,y_1,t)
\end{align}

For our example problem $v\equiv y_0$, $x\equiv y_1$, $g-cv^2=f_0(v,x,t)$, and $v=f_1(v,x,t)$

* Note, similiar to ```fsolve```:
    * Pass in an array of variables,
        * use a readable name,
    * Recover variables from the array,
    * Compute each rate in terms of the variables,
    * Return an array of the rates.

#### Numerical solution

\begin{align}
\frac{dv}{dt} &= g - cv^2, \\
\frac{dx}{dt} &= v.
\end{align}

* c = 1 m$^{-1}$
* g = 9.81 (m/s$^2$)
* t_end = 2 (s)
* $v_0 = 0$ (m/s)
* $x_0 = 0$ (m)

```python
#------ Step 1, define the function

def rhsf(vx, t):
    v = vx[0]
    x = vx[1]

    g = 9.81
    c = 1

    dvdt = g - c*v*v
    dxdt = v

    return np.array([dvdt, dxdt])

#------ Step 2, set times to get the solution at

t = np.linspace(0,2,1001)

#------ Step 3, set initial condition

vx0 = np.array([0, 0])

#------ Step 4, solve the problem, recover v, x arrays

vx = odeint(rhsf, vx0, t)

#------ Step 5 plot the answer

plt.plot(t, vx[:,0], label='v')
plt.plot(t, vx[:,1], label='x')
plt.xlabel('t')
plt.ylabel('v(t), x(t)')
plt.legend(frameon=False);

#------ Step 6: does the answer make sense? Analyse it.

```

### Exercise

Solve the following rate equation for a coal particle in a furnace.
$$ \frac{dT}{dt} = \left(\frac{hA}{m c_p}(T_f - T) + \frac{\sigma A}{m c_p}(T_f^4-T^4)\right).$$

The initial particle temperature is $T_0=500$ K.

Use D=100 $\mu$m, tend = 0.05 s.

The following data is given:

| Variable | Value  | Units        |
|----------|--------|-------       |
| $\rho_p$ | 1000   | kg/m$^3$     |
| $c_p$    | 1380   | J/kg$\cdot$K |
| $k$      | 0.1    | W/m$\cdot$K  |
| $Nu$     | 2      | --           |
| $\sigma$ | 5.67E-8| W/m$^2$K$^4$ |
| $T_f$    | 1500   | K            |

Also, the area, mass and Nusselt number $Nu$ are given, respectively, by:
$$ A = \pi D^2, $$
$$ m = \frac{\pi}{6}D^3\rho_p,$$
$$ Nu = \frac{hD}{k}.$$

```python

```

### Exercise

We are performing a chemical reaction as follows.
$$Rxn 1:  A+B\rightarrow C $$
$$Rxn 2: B+C\rightarrow D  $$
Here, symbols $A$, $B$, $C$, $D$ denote species concentrations in mol/L.

The initial concentrations are $A_0=1$, $B_0=1$, $C_0=0$, $D_0=0$. Also, $k_1=1\,L/mol\cdot s$, and $k_2=1.5\,L/mol\cdot s$.

The concentrations obey the following rate equations:

\begin{align}
\frac{dA}{dt} &= -k_1AB, \\
\frac{dB}{dt} &= -k_1AB - k_2BC, \\
\frac{dC}{dt} &= k_1AB - k_2BC, \\
\frac{dD}{dt} &= k_2BC.
\end{align}

Solve for the concentrations of $A$, $B$, $C$, and $D$ as functions of time to $t=3$ s.

Plot the concentrations of A, B, C, and D as functions of time on the same plot. Label the axes as "time (s)" and “concentration (mol/L)”.

```python

```


# 2-Variables and Basic Types

Always run this statement first, when working with this book:

```python
from scipy import *
from matplotlib.pyplot import *
%matplotlib   inline   # makes that plots appear within this notebook
```

## Variables

```python
diameter=3.
height=5.
cylinder=[diameter,height] # reference to a list
```

```python
a = b = c = 1
print(a)
```

    1

```python
a = 1
```

```python
a = a + 1 # a gets the value 2
print(a)
```

    2

```python
a = 3 * a
print(a)
```

    6

```python
a += 1 # same as a = a + 1
a *= 3 # same as a = 3 * a
print(a)
```

    21

## Numeric Types

### Integers

```python
6 // 2  # 3
```

    3

```python
7 // 2  # 3
```

    3

```python
7 / 2   # 3.5
```

    3.5

### Floating Point Numbers

```python
0.4 - 0.3 # returns 0.10000000000000003
```

    0.10000000000000003

```python
0.4 - 0.3 == 0.1  # returns False
```

    False

####  Infinite and Not a Number

```python
exp(1000.)  #  inf
```

    /home/claus/anaconda3/lib/python3.5/site-packages/ipykernel/__main__.py:1: RuntimeWarning: overflow encountered in exp
      if __name__ == '__main__':

    inf

```python
a = inf
```

```python
3 - a       # -inf
```

    -inf

```python
3 + a       #  inf
```

    inf

```python
a+a  # inf
```

    inf

```python
a-a  # nan
```

    nan

```python
a/a  # nan
```

    nan

```python
x = nan
```

```python
x < 0 # False
```

    False

```python
x > 0 # False
```

    False

```python
x == x # False
```

    False

```python
0 < inf # True
```

    True

```python
inf <= inf # True
```

    True

```python
inf == inf # True
```

    True

```python
-inf < inf # True
```

    True

```python
inf - inf  # nan
```

    nan

```python
exp(-inf)  # 0
```

    0.0

```python
exp(1/inf) # 1
```

    1.0

#### Underflow: Machine Epsilon

```python
import sys
sys.float_info.epsilon # 2.220446049250313e-16 (depending on your system)
```

    2.220446049250313e-16

#### Other float types in NumPy

```python
a  = pi # returns   3.141592653589793
a
```

    3.141592653589793

```python
a1 = float64(a)  # returns   3.1415926535897931
a1
```

    3.1415926535897931

```python
a2=float32(a)  # returns   3.1415927
a2
```

    3.1415927

```python
a - a1  # returns   0.0
```

    0.0

```python

a - a2  # returns   -8.7422780126189537e-08
```

    -8.7422780126189537e-08

```python
f32 = finfo(float32)
```

```python
f32.precision #  6 (decimal digits)
```

    6

```python
f64 = finfo(float64)
```

```python
f64.precision # 15 (decimal digits)
```

    15

```python
f   = finfo(float)
```

```python
f.precision  # 15 (decimal digits)
```

    15

```python
f64.max  # 1.7976931348623157e+308 (largest number)
```

    1.7976931348623157e+308

```python
f32.max  # 3.4028235e+38 (largest number)
```

    3.4028235e+38

### Complex Numbers

```python
b=5.2
```

```python
z=bj  # returns a (name) error
```

```python
z=b*j # returns a (name) error
```

```python
z=b*1j  # is correct
print(z)
```

    5.2j

```python
z=3.2+5.2j
z.conjugate()  # returns (3.2-5.2j)
```

    (3.2-5.2j)

#### Real and Imaginary Parts

```python
z = 1j
```

```python
z.real # 0
```

    0.0

```python
z.imag # 1
```

    1.0

```python
z.imag = 2 # error
```

```python
N = 10

# the following vector contains the Nth roots of unity:
unity_roots = array([exp(1j*2*pi*k/N) for k in range(N)])

# access all the real or imaginary parts with real or imag:
plot(unity_roots.real, unity_roots.imag, 'o')
axis('equal')

allclose(unity_roots**N, 1) # True
```

```python
z=3.2+5.2j
```

```python
(z+z.conjugate())/2. # returns (3.2+0j)
```

    (3.2+0j)

```python
((z+z.conjugate())/2.).real  # returns 3.2
```

    3.2

```python
(z-z.conjugate())/2.# returns 5.2j
```

    5.2j

```python
((z-z.conjugate())/2.).imag  # returns 5.2
```

    5.2

```python
sqrt(z*z.conjugate()) # returns  (6.1057350089894991+0j)
```

    (6.1057350089894991+0j)

## Booleans

```python
a = True
b = 30>45  # b gets the value False
print(a,b)
```

    True False

```python
x=3
if x>0:
   print("positive")
else:
   print("nonpositive")
```

    positive

### Boolean Operators

```python
True and False # False
```

    False

```python
False or True # True
```

    True

```python
(30 > 45) or (27 < 30) # True
```

    True

```python
not True # False
```

    False

```python
not (3 > 4) # True
```

    True

```python
a=3; b=4; c=-1

a < b < c # same as: a < b and b < c
```

    False

```python
a == b == c # same as: a == b and b == c
```

    False

### Boolean Casting

```python
bool([]) # False
```

    False

```python
bool(0) # False
```

    False

```python
bool(' ') # True
```

    True

```python
bool('') # False
```

    False

```python
bool('hello') # True
```

    True

```python
bool(1.2) # True
```

    True

```python
bool(array([1])) # True
```

    True

```python
bool(array([1,2])) # Exception raised!
```

#### Automatic Boolean Casting

```python
L=[]
if L:
    print("list not empty")
else:
    print("list is empty")
```

    list is empty

```python
n=23
if n % 2:
    print("n is odd")
else:
    print("n is even")
```

    n is odd

```python
def and_as_function(x,y):
    if not x:
        return x
    else:
        return y

and_as_function(True,False)
```

    False

```python
def or_as_function(x,y):
    if x:
        return x
    else:
        return y

or_as_function(True,False)
```

    True

```python
True or x_not_defined

```

    True

```python
False and x_not_defined
```

    False

```python
[1] or 'a' # produces [1]
```

    [1]

```python
'a' or [1] # produces 'a'
```

    'a'

### Booleans and Integers

```python
def print_ispositive(x):
    possibilities=['nonpositive', 'positive']
    return "x is {}".format(possibilities[x>0])

print_ispositive(-23)
```

    'x is nonpositive'

```python
print_ispositive(7)
```

    'x is positive'

```python
True+13
```

    14

## Strings

```python
name = 'Johan Carlsson'
child = "Åsa is Johan Carlsson's daughter"
book = """Aunt Julia
and the Scriptwriter"""

print(name)
print(child)
print(book)
```

    Johan Carlsson
    Åsa is Johan Carlsson's daughter
    Aunt Julia
    and the Scriptwriter

```python
book[-1] # returns 'r'
```

    'r'

```python
book[-12:] # returns 'Scriptwriter'
```

    'Scriptwriter'

```python
book[1]='a'
```

```python
print('Temperature:\t20\tC\nPressure:\t5\tPa')
```

    Temperature:	20	C
    Pressure:	5	Pa

```python
a="""
A multiline
example"""
a # returns '\nA multiline\nexample'
```

    '\nA multiline\nexample'

```python
latexfontsize="\\tiny"
print(latexfontsize)
```

    \tiny

```python
latexfs=r"\tiny"  # returns "\\tiny"
latexfontsize == latexfs  # returns True
```

    True

### Operations on strings and string methods

```python
last_name='Carlsson'
first_name='Johanna'
Full_name=first_name+' '+last_name  # returns 'Johanna Carlsson'

print(Full_name)
```

    Johanna Carlsson

```python
game=2*'Yo' # returns 'YoYo'
print(game)
```

    YoYo

```python
'Anna' > 'Arvid' # returns false
```

    False

```python
'ANNA' < 'anna'  # returns true
```

    True

```python
'10B' < '11A' # returns true
```

    True

```python
text='quod erat    demonstrandum'
text.split() # returns ['quod', 'erat', 'demonstrandum']
```

    ['quod', 'erat', 'demonstrandum']

```python
table='Johan;Carlsson;19890327'
table.split(';') # returns ['Johan','Carlsson','19890327']
```

    ['Johan', 'Carlsson', '19890327']

```python
king='CarlXVIGustaf'
king.split('XVI')  # returns ['Carl','Gustaf']
```

    ['Carl', 'Gustaf']

```python
sep=';'
sep.join(['Johan','Carlsson','19890327']) # returns 'Johan;Carlsson;19890327'
```

    'Johan;Carlsson;19890327'

```python
birthday='20101210'
birthday.find('10') # returns 2
```

    2

### String Formatting

```python
course_code = "NUMA21"
print("This course's name is {}".format(course_code)) # This course's name is NUMA21
```

    This course's name is NUMA21

```python
quantity = 33.45
print("{:f}".format(quantity)) # 33.450000
print("{:1.1f}".format(quantity)) # 33.5
print("{:.2e}".format(quantity)) # 3.35e+01
```

    33.450000
    33.5
    3.35e+01

```python
print("{name} {value:.1f}".format(name="quantity",value=quantity)) # "quantity 33.5"
```

    quantity 33.5

```python
r"we {} in LaTeX \begin{{equation}}".format('like')
```

    'we like in LaTeX \\begin{equation}'


# Multiple Traveling Salesman Problem

## 1. Mathematical Problem

In this notebook, we show how to solve the Multiple Traveling Salesman Problem (mTSP) using CVXPY. The problem considers $m$ traveling salesmen. To solve it, I'm going to use the Miller-Tucker-Zemlin formulation, which follows:

The cities are identified with the numbers $1, \ldots, n$, with which we define:

$$
x_{ij} = \begin{cases} 1 & \text{the path goes from the city} \, i \text{ to city } j \\ 0 & \text{otherwise} \end{cases}
$$

For $i = 1, \ldots, n$, $m$ is the number of salesmen, $u_{i}$ is an auxiliary variable and $c_{i, j}$ is the distance from city $i$ to the city $j$. Then the mTSP can be written as the following integer linear programming problem:

$$
\begin{align}
\min &\sum_{i=1}^n \sum_{j\ne i,j=1}^n c_{i,j}x_{i,j}\colon &&  \\
     & \sum_{i=2}^n x_{i,1} = m \\
     & \sum_{j=2}^n x_{1,j} = m \\
     & \sum_{i=2,i\ne j}^n x_{i,j} = 1 && j=1, \ldots, n; \\
     & \sum_{j=2,j\ne i}^n x_{i,j} = 1 && i=1, \ldots, n; \\
     & u_{i}-u_{j} + 1 \le (n-1)(1-x_{i,j}) && 2 \le i \ne j \le n;  \\
     & 0 \le u_{i} \le n && 2 \le i \le n; \\
     & x_{i,j} \in \{0,1\}  && i,j=1, \ldots, n; \\
     & u_{i} \in \mathbf{Z} && i=2, \ldots, n. \\
\end{align}
$$

The objective function minimizes the distance of each salesman's routes. The first two restrictions guarantee that there is a number of salesmen $m$ who leave and return to the city of origin. The following two restrictions ensure that each city is reached from exactly one other city and that from each city there is an exit to exactly one other city. The constraints of the auxiliary variables force that no salesman passes through the same city twice.

## 2. Example for a single Traveling Salesman

We are going to find the optimal paths (shortest distance) for a single salesman that starts from the point of origin (O) and must pass through all points A to F and return to point O.

| Point |	Coordinates |
| :----------: | :----------: |
| Origin(O) | (-12.059296, -76.975893) |
| A | (-12.079575, -77.009686) |
| B | (-12.087303, -76.996620) |
| C | (-12.084391, -76.975651) |
| D | (-12.063603, -76.960483) |
| E | (-12.056762, -77.014452) |
| F | (-12.011531, -77.002383) |

```python
################################################
# Loading libraries
################################################
!pip install --quiet geopy

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cvxpy as cp
from geopy import distance # Library for geographical calculations

################################################
# Original Data
################################################
points = [(-12.059296, -76.975893),
          (-12.079575, -77.009686),
          (-12.087303, -76.996620),
          (-12.084391, -76.975651),
          (-12.063603, -76.960483),
          (-12.056762, -77.014452),
          (-12.011531, -77.002383)]

################################################
# Building distance matrix
################################################
n = len(points)
C = np.zeros((n,n))

for i in range(0, n):
    for j in range(0, len(points)):
        C[i,j] = distance.distance(points[i], points[j]).km

# Showing distance matrix
print('Distance Matrix is:\n')
print(np.round(C,4))
```

    Distance Matrix is:

    [[0.     4.3092 3.8329 2.7762 1.7441 4.2076 6.0199]
     [4.3092 0.     1.6596 3.7435 5.6408 2.5764 7.5691]
     [3.8329 1.6596 0.     2.3055 4.7278 3.8966 8.4056]
     [2.7762 3.7435 2.3055 0.     2.8311 5.2142 8.5694]
     [1.7441 5.6408 4.7278 2.8311 0.     5.9246 7.3483]
     [4.2076 2.5764 3.8966 5.2142 5.9246 0.     5.1733]
     [6.0199 7.5691 8.4056 8.5694 7.3483 5.1733 0.    ]]

```python
################################################
# Solving the integer programming problem
################################################

# Defining variables
X = cp.Variable(C.shape, boolean=True)
u = cp.Variable(n, integer=True)
ones = np.ones((n,1))

# Defining the objective function
objective = cp.Minimize(cp.sum(cp.multiply(C, X)))

# Defining the constraints
constraints = []
constraints += [X @ ones == ones]
constraints += [X.T @ ones == ones]
constraints += [cp.diag(X) == 0]
constraints += [u[1:] >= 2]
constraints += [u[1:] <= n]
constraints += [u[0] == 1]

for i in range(1, n):
    for j in range(1, n):
        if i != j:
            constraints += [ u[i] - u[j] + 1  <= (n - 1) * (1 - X[i, j]) ]

# Solving the problem
prob = cp.Problem(objective, constraints)
prob.solve(verbose=False)

# Transforming the solution to a path
X_sol = np.argwhere(X.value==1)
orden = X_sol[0].tolist()

for i in range(1, n):
    row = orden[-1]
    orden.append(X_sol[row,1])

# Showing the optimal path
print('The path is:\n')
print( ' => '.join(map(str, orden)))
```

    The path is:

    0 => 6 => 5 => 1 => 2 => 3 => 4 => 0

```python
################################################
# Plotting the optimal path
################################################

# Transforming the points to the xy plane approximately
xy_cords = np.zeros((n,2))

for i in range(0, n):
    xy_cords[i,0] = distance.distance((points[0][1],0), (points[i][1],0)).km
    xy_cords[i,1] = distance.distance((0,points[0][0]), (0,points[i][0])).km

# Plotting the points
fig, ax = plt.subplots(figsize=(14,7))

for i in range(n):
    ax.annotate(str(i), xy=(xy_cords[i,0], xy_cords[i,1]+0.1))

ax.scatter(xy_cords[:,0],xy_cords[:,1])
ax.plot(xy_cords[orden,0], xy_cords[orden,1])
```

```python
# Showing the optimal distance
distance = np.sum(np.multiply(C, X.value))
print('The optimal distance is:', np.round(distance,2), 'km')
```

    The optimal distance is: 22.31 km

## 3. Example for three Traveling Salesmen

Continuing with the previous example, now we are going to find the optimal paths (shortest distance) for three salesmen that start from the point of origin (O) and between all of them must go through points A to F and return to point O.

| Point |	Coordinates |
| :----------: | :----------: |
| Origin(O) | (-12.059296, -76.975893) |
| A | (-12.079575, -77.009686) |
| B | (-12.087303, -76.996620) |
| C | (-12.084391, -76.975651) |
| D | (-12.063603, -76.960483) |
| E | (-12.056762, -77.014452) |
| F | (-12.011531, -77.002383) |

```python
################################################
# Solving the integer programming problem
################################################

# Defining the variables
X = cp.Variable(C.shape, boolean=True)
u = cp.Variable(n, integer=True)
m = 3
ones = np.ones((n,1))

# Defining the objective function
objective = cp.Minimize(cp.sum(cp.multiply(C, X)))

# Defining the constraints
constraints = []
constraints += [X[0,:] @ ones == m]
constraints += [X[:,0] @ ones == m]
constraints += [X[1:,:] @ ones == 1]
constraints += [X[:,1:].T @ ones == 1]
constraints += [cp.diag(X) == 0]
constraints += [u[1:] >= 2]
constraints += [u[1:] <= n]
constraints += [u[0] == 1]

for i in range(1, n):
    for j in range(1, n):
        if i != j:
            constraints += [ u[i] - u[j] + 1  <= (n - 1) * (1 - X[i, j]) ]

# Solving the problem
prob = cp.Problem(objective, constraints)
prob.solve(verbose=False)

# Transforming the solution to paths
X_sol = np.argwhere(X.value==1)

ruta = {}
for i in range(0, m):
    ruta['Salesman_' + str(i+1)] = [0]
    j = i
    a = 10e10
    while a != 0:
        a = X_sol[j,1]
        ruta['Salesman_' + str(i+1)].append(a)
        j = np.where(X_sol[:,0] == a)
        j = j[0][0]
        a = j

# Showing the paths
for i in ruta.keys():
    print('The path of ' + i + ' is:\n')
    print( ' => '.join(map(str, ruta[i])))
    print('')
```

    The path of Salesman_1 is:

    0 => 3 => 0

    The path of Salesman_2 is:

    0 => 4 => 0

    The path of Salesman_3 is:

    0 => 6 => 5 => 1 => 2 => 0

```python
################################################
# Plotting the optimal path
################################################

# Plotting the points
fig, ax = plt.subplots(figsize=(14,7))

for i in range(n):
    ax.annotate(str(i), xy=(xy_cords[i,0], xy_cords[i,1]+0.1))

ax.scatter(xy_cords[:,0],xy_cords[:,1])
for i in ruta.keys():
    ax.plot(xy_cords[ruta[i],0], xy_cords[ruta[i],1], label = i)
    ax.legend(loc='best')
```

```python
# Showing the optimal distance
distance = np.sum(np.multiply(C, X.value))
print('The optimal distance is:', np.round(distance,2), 'km')
```

    The optimal distance is: 28.3 km

It can be observed that depending on the location of the points that salesmen must travel, in terms of distance it may not be optimal to add more salesmen.


```python
import sympy as sp
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
```

```python
x = sp.symbols('x')
f = 1 / (x**2 + 1)
a, b = -1, 1
new_node = 0.15
order = 7
```

# Let's plot it!

```python
x_plot = np.linspace(a, b, 100, endpoint=True)
y_plot = sp.lambdify(x, f, 'numpy')(x_plot)
dy_plot = sp.lambdify(x, sp.diff(f), 'numpy')(x_plot)

sns.set_style('whitegrid')
plt.figure(figsize=(12, 6))
sns.lineplot(x_plot, y_plot).set_title('Runge function');
```

# Interpolation using Chebyshev's polynomial

```python
# Compute Chebyshev nodes
nodes = [np.cos(np.pi * (2 * k + 1) / (2 * order)) for k in range(order)]
values = [f.evalf(subs={x: n}) for n in nodes]
```

> Note: The code below is mostly taken from task-1

```python
# Helper function to compute divided differences
def div_diffs(nodes, values, max_order):
    if len(nodes) != len(values):
        raise ValueError("nodes and values lists must have the same length")
    diffs = [values]
    for order in range(max_order):
        current_diffs = []
        for i in range(1, len(nodes) - order):
            current_diffs.append((diffs[-1][i] - diffs[-1][i-1]) / (nodes[i + order] - nodes[i-1]))
        diffs.append(current_diffs)
    return diffs
```

```python
# Helper function to nicely print divided differences
def print_diffs(diffs):
    rows = 2 * len(diffs[0]) - 1
    for r in range(rows):
        row = '\t' if r % 2 == 1 else ''
        for i in range(r % 2, min(rows - r, min(r + 1, len(diffs))), 2):
            row += f'{float(diffs[i][(r - i) // 2]):.4f}\t\t'
        print(row[:-2])
```

```python
dds = div_diffs(nodes, values, order - 1)
print_diffs(dds)
```

    0.5127
    	-0.5590
    0.6206		0.1404
    	-0.6350		0.4981
    0.8416		-0.3451		0.0637
    	-0.3651		0.4084		-0.2611
    1.0000		-0.8416		0.5223		-0.2678
    	0.3651		-0.4084		0.2611
    0.8416		-0.3451		0.0637
    	0.6350		-0.4981
    0.6206		0.1404
    	0.5590
    0.5127

```python
for pos in range(len(nodes)):
    if nodes[pos] > new_node:
        break
# pos is where we'd put the new node among the others
```

```python
# Find position of the closest by value nodes
i, j = pos - 1, pos
pos_around_new = []
while (j - i) <= order:
    if i < 0 and j >= len(nodes):
        break
    if j >= len(nodes) or new_node - nodes[i] <= nodes[j] - new_node:
        pos_around_new.append(i)
        i -= 1
    else:
        pos_around_new.append(j)
        j += 1
```

```python
nodes_around_new = [nodes[i] for i in pos_around_new]
```

```python
polynom = 0
mult = 1
for o in range(order):
    m = min(pos_around_new[:o+1])
    polynom += dds[o][m] * mult
    mult *= x - nodes[pos_around_new[o]]
chebyshev_estimated_value = polynom.evalf(subs={x: new_node})
```

    Estimated value: 0.978528748953975
    True value: 0.977995110024450

# Interpolation using equally spaced nodes

> It only supports forward interpolation, thus may fail with some orders. I'll fix it soon :)

```python
def finite_differences(nodes, values, max_order):
    if len(nodes) != len(values):
        raise ValueError("nodes and values lists must have the same length")
    diffs = [values]
    for order in range(max_order):
        current_diffs = []
        for i in range(1, len(nodes) - order):
            current_diffs.append(diffs[-1][i] - diffs[-1][i-1])
        diffs.append(current_diffs)
    return diffs
```

```python
nodes = np.linspace(a, b, order)
step = (b - a) / (order - 1)
values = sp.lambdify(x, f, 'numpy')(nodes)

for n, v in zip(nodes, values):
    print(f'f({n:.3g}) = {v:.5g}')
```

    f(-1) = 0.5
    f(-0.667) = 0.69231
    f(-0.333) = 0.9
    f(0) = 1
    f(0.333) = 0.9
    f(0.667) = 0.69231
    f(1) = 0.5

```python
fds = finite_differences(nodes, values, order - 1)
```

```python
print_diffs(fds)
```

    0.5000
    	0.1923
    0.6923		0.0154
    	0.2077		-0.1231
    0.9000		-0.1077		0.0308
    	0.1000		-0.0923		0.1538
    1.0000		-0.2000		0.1846		-0.3077
    	-0.1000		0.0923		-0.1538
    0.9000		-0.1077		0.0308
    	-0.2077		0.1231
    0.6923		0.0154
    	-0.1923
    0.5000

```python
nearest_pos, nearest_node = None, None
for i, n in enumerate(nodes):
    if n < new_node and new_node <= n + step/2:
        nearest_pos, nearest_node = i, n
```

```python
t = (x - nearest_node) / step
```

```python
ns = [1]
mult = -1
for i in range(1, order):
    ns.append(ns[-1] * (t - mult * (i // 2)) / i)
    mult *= -1
```

```python
ns, len(ns)
```

    ([1,
      3.0*x,
      1.5*x*(3.0*x - 1),
      0.5*x*(3.0*x - 1)*(3.0*x + 1),
      0.125*x*(3.0*x - 2)*(3.0*x - 1)*(3.0*x + 1),
      0.025*x*(3.0*x - 2)*(3.0*x - 1)*(3.0*x + 1)*(3.0*x + 2),
      0.00416666666666667*x*(3.0*x - 3)*(3.0*x - 2)*(3.0*x - 1)*(3.0*x + 1)*(3.0*x + 2)],
     7)

```python
polynom = 0
for i in range(order):
    polynom += ns[i] * fds[i][nearest_pos - i // 2]
esn_estimated_value = polynom.evalf(subs={x: new_node})
```

```python
true_value = f.evalf(subs={x: new_node})
```

```python
pd.DataFrame([
        [chebyshev_estimated_value, true_value - chebyshev_estimated_value],
        [esn_estimated_value, true_value - esn_estimated_value],
        [true_value, 0]
    ], index=['Estimation using Chebyshev nodes', 'Estimation using equally spaced nodes', 'True value'],
    columns=[f'{order} nodes', 'Error']
)
```

      7 nodes
      Error

      Estimation using Chebyshev nodes
      0.978528748953975
      -0.000533638929525071

      Estimation using equally spaced nodes
      0.978245658112981
      -0.000250548088530866

      True value
      0.977995110024450
      0


# **Derivation of the threshold function**

```python
# import dill
# dill.settings['recurse'] = True
```

```python
%matplotlib widget
import sympy as sp
import numpy as np
import matplotlib.pylab as plt
from mpl_toolkits.mplot3d import Axes3D
import ipywidgets as ipw
sp.init_printing()
from cymbol import Cymbol, ccode
```

# Variables

**Stress variables**

$x$ corresponds to $\sigma_N$ and $y$ corresponds to $\sigma_T$

```python
x, y = sp.symbols('x, y')
```

**Unknown parameters of the ellipse cap function**

```python
x_c = sp.symbols('x_c')  # center point of an ellipse
a = sp.symbols('a', nonnegative=True)
b = sp.symbols('b', nonnegative=True)
c = sp.symbols('c', positive=True)
```

**Material parameters**

```python
x_0 = sp.symbols('x_0')
x_bar, y_bar = sp.symbols(r'\bar{x}, \bar{y}', nonnegative=True )
m = Cymbol(r'm', codename='m_', real=True, nonnegative=True)
```

# Mid part of the threshold function

The linear part of the thresshold function is introduced as
\begin{align}
f_\mathrm{lin} :=
|y| - \bar{y} + m(x - x_0)
\end{align}
where $x_0$ denotes the reference position at which $y = \pm \bar{y}$

```python
f_mid_ = sp.sqrt(y**2) - (y_bar - m * (x-x_0))
#get_f_mid = sp.lambdify((x, y, y_bar, m), f_mid_)
f_mid_
```

# Cap function

It is introduced in form of an ellipse centered at the position ($x_c, 0$), i.e.
\begin{align}
f_\mathrm{ell} := \sqrt{\dfrac{y^2}{b^2} + \dfrac{(x - x_0 - x_\mathrm{c})^2}{a^2}} - c
\end{align}
The parameters $a, b, c$ and $x_\mathrm{c}$ are determined based on the compatibility and continuity conditions along the transition between $f_\mathrm{lin}$ and $f_\mathrm{ell}$.

```python
f_cap_ = sp.sqrt((x-x_0-x_c)**2/a**2 + y**2/b**2) - c
f_cap_
```

Construct the derivatives

```python
df_cap_dx_ = f_cap_.diff(x)
df_cap_dy_ = f_cap_.diff(y)
```

# Compatibility and smoothness conditions specified in the figure above

\begin{align}
\left. f_\mathrm{ell} \right|_{x = \bar{x}, y = 0} &= 0 \\
\left. f_\mathrm{ell} \right|_{x=x_0, y=\bar{y}} &= 0 \\
\left.
\dfrac{f_{\mathrm{ell},x}}{f_{\mathrm{ell},y}
}
\right|_{x=x_\mathrm{c}, y=\bar{y}}
&= -m
\end{align}

```python
eq1 = sp.Eq(f_cap_.subs({x: x_bar, y: 0}), 0)
eq2 = sp.Eq(f_cap_.subs({x: x_0, y: y_bar}), 0)
eq3 = sp.Eq((-df_cap_dx_ / df_cap_dy_).subs({x: x_0, y: y_bar}), -m)
eq1, eq2, eq3
```

Solve for $a, b$ and $x_c$

```python
abx_subs = sp.solve({eq1, eq2, eq3},{a, b, x_c})[0]
abx_subs
```

# Continuity between $f_\mathrm{mid}$ and $f_\mathrm{cap}$

**Require an identical value for**
\begin{align}
f_\mathrm{mid}(x_c, 0) = f_\mathrm{cap}(x_c,0)
\end{align}

```python
f_mid_c_ = f_mid_.subs({x: abx_subs[x_c]+x_0,y:0})
f_cap_abx_ = f_cap_.subs(abx_subs)
f_cap_abxc_ = f_cap_abx_.subs({x: abx_subs[x_c]+x_0,y:0})
eq4 = sp.Eq(f_cap_abxc_, f_mid_c_)
eq4
```

Which can be used to express $c$

```python
c_solved = sp.solve(eq4, c)[0]
sp.simplify(c_solved)
```

```python
abx_subs[x_c]
```

```python
x_hat = sp.symbols(r'\hat{x}', nonnegative=True)
```

```python
c_paper = y_bar + (m**2 * (x_hat) **2)/(2*m*(x_hat) + y_bar)
sp.simplify(c_paper.subs(x_hat, x_0 - x_bar) - c_solved)
```

```python
a_solved = abx_subs[a]
sp.simplify(a_solved)
```

```python
x_hat_ = (x_bar - x_0)
```

$$
-\frac{\bar{y} - m \hat{x} } {\bar{y} - 2m\hat{x}} \cdot \frac{\hat{x}}{c}
$$

```python
a_paper = - (-m*x_hat_ + y_bar)/((-2*m*x_hat_ + y_bar)) * x_hat_ / c
a_paper
```

```python
sp.simplify(a_paper - a_solved)
```

```python
b_solved = abx_subs[b]
sp.simplify(b_solved)
```

```python
b_paper = (sp.sqrt(y_bar)/c *
           (-m * x_hat_ + y_bar)/(sp.sqrt((-2*m*x_hat_ + y_bar))))
b_paper
```

$$
\frac{\bar{y} - m \hat{x}}{\sqrt{\bar{y} - 2m \hat{x}}} \cdot
\frac{\sqrt{y}}{c}
$$

```python
sp.simplify(b_paper**2 - b_solved**2)
```

```python
x_c_solved = abx_subs[x_c]
sp.simplify(x_c_solved)
```

```python
x_c_paper = -(m*x_hat**2)/(2*m*x_hat + y_bar)
sp.simplify(x_c_paper.subs(x_hat, x_0 - x_bar) - x_c_solved)
```

Substitute back the $f_\mathrm{cap}$ to obtain its resolved form

```python
f_cap_solved_1 = f_cap_abx_.subs(c, c_solved)
f_cap_solved_1
```

```python
f_cap_solved_ = f_cap_solved_1 # dill.loads(dill.dumps(f_cap_solved_1))
```

```python
f_cap_solved_
```

```python
x_bar, y_bar = sp.symbols(r'\bar{x}, \bar{y}', nonnegative=True )

```

# Test the results

```python
_x_0=-3
_x_bar=-5
_y_bar=3
_m=.3
```

The value in the denominator must not be equal to 0

```python
m_limit_ = sp.solve(2*x_bar * m + y_bar, m)[0]
_m_limit = m_limit_.subs({x_bar: _x_bar, y_bar: _y_bar})
_m_limit
```

```python
if _m < _m_limit * sp.sign(_x_bar - _x_0):
    print('Take care')
```

Test the obtained position of $x_c$

```python
x_c_ = abx_subs[x_c]
x_c_
```

```python
get_x_c = sp.lambdify((x_bar, y_bar, m, x_0), x_c_, 'numpy' )
_x_c = get_x_c(_x_bar, _y_bar, _m, _x_0)
_x_c
```

# Domain separation for $f_\mathrm{cap}$ and $f_\mathrm{mid}$

Define the transition between cap and mid domains by defining a connection
line between [$x_c$,0]-[0,$\bar{\tau}$].

```python
(-y_bar / (x_c) * (x - x_0 -x_c))
```

```python
y_trans_ = (-y_bar / (x_c) * (x - x_0 - x_c)).subs(x_c, x_c_)
f_cap_domain_ = sp.sign(x_bar-x_0) * sp.sign(-m) * (sp.Abs(y) - y_trans_) > 0
```

```python
x_trans = np.linspace(_x_c, _x_0, 10)
get_y_trans = sp.lambdify((x, x_bar, y_bar, m, x_0), y_trans_)
y_trans = get_y_trans(x_trans, _x_bar, _y_bar, _m, _x_0)
x_trans, y_trans
```

    (array([-0.28571429, -0.58730159, -0.88888889, -1.19047619, -1.49206349,
            -1.79365079, -2.0952381 , -2.3968254 , -2.6984127 , -3.        ]),
     array([31.5       , 28.33333333, 25.16666667, 22.        , 18.83333333,
            15.66666667, 12.5       ,  9.33333333,  6.16666667,  3.        ]))

```python
get_y_trans(_x_c, _x_bar, _y_bar, _m, _x_0)
```

# Composed smooth level set function $f_\mathrm{full}(x,y)$

```python
f_full_ = sp.Piecewise(
    (f_cap_solved_, f_cap_domain_),
    (f_mid_, True)
)
get_f_full_lambdified = sp.lambdify((x, y, x_bar, y_bar, m, x_0), f_full_, 'numpy')
```

# Composed level set function with two caps

```python
f_t = Cymbol(r'f_\mathrm{t}', codename='f_t_')
f_c = Cymbol(r'f_\mathrm{c}', codename='f_c_')
f_c0 = Cymbol(r'f_\mathrm{c0}', codename='f_c0_')
f_s = Cymbol(r'f_\mathrm{s}', codename='f_s_')
```

```python
subs_tension = {x_0: 0, x_bar: f_t, y_bar: f_s}
subs_shear = {y_bar: f_s, x_0: 0}
subs_compression = {x_0: -f_c0, x_bar: -f_c,  y_bar: f_s-m*(-f_c0) }
```

```python
f_solved_ = sp.Piecewise(
    (f_cap_solved_.subs(subs_tension), f_cap_domain_.subs(subs_tension)),
    (f_cap_solved_.subs(subs_compression), f_cap_domain_.subs(subs_compression)),
    (f_mid_.subs(subs_shear), True)
)
f_solved_
```

```python
get_f_solved_lambdified = sp.lambdify((x, y, f_t, f_c, f_c0, f_s, m), f_solved_, 'numpy')
get_f_solved = get_f_solved_lambdified # dill.loads(dill.dumps(get_f_solved_lambdified))
```

```python
# f_file = open('f.dill', 'wb')
# dill.dump(get_f_solved_lambdified, f_file)
# f_file.close()
```

```python
# f_file = open('f.dill', 'rb')
# get_f_solved = dill.load(f_file)
# f_file.close()
```

```python
X_a, Y_a = np.mgrid[-100:20:210j,-30:30:210j]
mp = {'f_t_': 5, 'f_c_': 80, 'f_c0_': 70, 'f_s_': 5, 'm_': 0.1}
Z_a = get_f_solved(X_a,Y_a,**mp)
```

```python
fig = plt.figure(figsize=(8,4))
ax = fig.add_subplot(1, 1, 1)
ax.contour(X_a,Y_a,Z_a, levels=8)
ax.plot([_x_0], [_y_bar], marker='o')
ax.plot([_x_bar], [0], marker='o', color='red')
ax.plot([_x_c+_x_0], [0], marker='o', color='green')
ax.set_aspect('equal')
```

```python

```

```python

```

```python

```


# Introduction to Python

## By John Chang [GitHub](https://eve-ning.github.io/)

## Math to Python Translation

It's easier to understand python by looking at it as math

Here are some **math-to-python** translations

### Translating Math Operations

### Example 1.1

\begin{align}
x = 40;
y = 3x * 4
\end{align}
\begin{align}
y = {?}
\end{align}

```python
x = 40
y = 3 * x * 4
```

*How do you know if the code worked? You have to `print` them*

```python
print(x)
print(y)
```

    40
    480

You cannot print something that doesn't exist yet, it will "throw" a `NameError: name is not defined` Error

```python
# print(a)
```

### Example 1.2

\begin{align}
x = 50, y = 100
\end{align}
\begin{align}
z = {?}
\end{align}

```python
x = 50
y = 100
z = x / y

print(z)
```

    0.5

### Example 1.3

A plant branch, on average 15 leaves, the plant has 12 branches. Approximate the total amount of leaves

```python
leaves = 15
branches = 12
print(leaves * branches)
```

    180

### Tutorial 1.1

Find the value of Voltage in the following equation with Python

\begin{align}
Current = 14.6;
Resistance = 39.0 * 10 ^{-3}
\end{align}
\begin{align}
Voltage = Current * Resistance
\end{align}

```python
# print(voltage)
```

### Tutorial 1.2

Find the value z of the following equation with Python

\begin{align}
x = 15;
y = 5;
\end{align}
\begin{align}
z = \frac{10y^2+10}{5xy+15} * \frac{3}{2}
\end{align}

```python
# print(z)
```

### Translating Math Functions

Sometimes, you would want to use the same complex function multiple times. So you define a function for it

\begin{align}
GST(Base) = Base * 0.07
\end{align}
\begin{align}
GST(100) = 7
\end{align}
\begin{align}
GST(300) = 21
\end{align}

\begin{align}
sin(degrees) = {...}
\end{align}
\begin{align}
sin(90) = 1
\end{align}
\begin{align}
sin(180) = 0
\end{align}

It's rather rare in math to define your own functions, however, it's unavoidable for programming

### Example 2.1

\begin{align}
f(x) = 5 x ^ {3}
\end{align}
\begin{align}
f(10) = {?}
\end{align}
\begin{align}
\frac{f(5)}{f(2)} = {?}
\end{align}

```python
def f(x):
    return 5 * x * x * x

print(f(10))
print(f(5)/f(2))
```

    5000
    15.625

\begin{align}
f(f(2)) = {?}
\end{align}

```python
print(f(f(2)))
```

    320000

#### Pre-built functions

In math, there are functions such as **ln**, **sin**, etc. Only sometimes, base Python will have the capability to execute the functions on demand

```python
# sin(40)
```

```python
# ln(1.4)
```

`NameError: name is not defined` means it just doesn't exist yet!

You don't have to define these functions, just **google**

Google 1st Result: [`python sin`](https://docs.python.org/2/library/math.html)

When you see a result that is **name**.*function*, it means it's in a package that you need to `import`

```python
import math
math.cos(math.pi)
```

    -1.0

### Example 2.2

\begin{align}
sin(cos(\pi^{9})) = {?}
\end{align}

```python
import math
math.sin(math.cos(pow(math.pi, 9)))
```

    -0.0971323539481088

`pow` is an in-built function, `math.pow` also works.

### Tutorial 2.1

\begin{align}
sin2(x,y) = sin(x) * sin(y)
\end{align}
\begin{align}
cos2(x,y) = cos(x) * cos(y)
\end{align}
\begin{align}
sin2(3\pi,3) * cos2(2\pi,2) = {?}
\end{align}

```python
import math
def sin2(x,y):
    return
def cos2(x,y):
    return
print()
```

### Tutorial 2.2

\begin{align}
ln(-sin2(e^\pi,1)) = {?}
\end{align}

```python
# print()
```

We will be mainly using these few packages for *Data Preparation*

- pandas
- numpy

# Answers

# Tutorials

\begin{align}
Current = 14.6;
Resistance = 39.0 * 10 ^{-3}
\end{align}
\begin{align}
Voltage = Current * Resistance
\end{align}

```python
14.6 * 39.0 * 0.001
```

    0.5694

\begin{align}
x = 15;
y = 5;
\end{align}
\begin{align}
z = \frac{10y^2+10}{5xy+15} * \frac{3}{2}
\end{align}

```python
x = 15
y = 5
z = (10 * y * y + 10) / (5 * x * y + 15) * 3 / 2
print(z)
```

    1.0

\begin{align}
sin2(x,y) = sin(x) * sin(y)
\end{align}
\begin{align}
cos2(x,y) = cos(x) * cos(y)
\end{align}
\begin{align}
sin2(3\pi,3) * cos2(2\pi,2) = {?}
\end{align}

```python
import math
def sin2(x, y):
    return math.sin(x) * math.sin(y)
def cos2(x, y):
    return math.cos(x) * math.cos(y)
sin2(3 * math.pi, 3) * cos2(2 * math.pi, 2)
```

    -2.1575819320582253e-17

\begin{align}
ln(-sin2(e^\pi,1)) = {?}
\end{align}

```python
print(math.log(-math.sin(math.exp(math.pi)*1)))
```

    -0.0914821575035909

```python

```


**Tarea Nro. 3 - LinAlg + Sympy**

- Nombre y apellido: Ivo Andrés Astudillo
- Fecha: 7 de diciembre de 2020

# Producto punto

## Use un producto punto y la lista de compras de la tabla 9.4 para determinar su cuenta total en la tienda.

```python
import numpy as np
```

```python
articulo = np.array([2, 1, 2, 5, 1])
costo = np.array([3.50, 1.25, 4.25, 1.55, 3.15])
print("Cuenta total:", np.dot(articulo, costo))
```

    Cuenta total: 27.65

# Multiplicación matricial

## Con un calorímetro de bomba se realizó una serie de experimentos. En cada experimento se usó una cantidad diferente de agua. Calcule la capacidad calorífica total para el calorímetro en cada uno de los experimentos, mediante multiplicación matricial, los datos de la tabla 9.8 y la información acerca de la capacidad calorífica que sigue a la tabla.

```python
bomba = np.array([
    [110, 250, 10],
    [100, 250, 10],
    [101, 250, 10],
    [98.6, 250, 10],
    [99.4, 250, 10]
]).reshape(5, 3)

capacidad_calorica = np.array([4.2, 0.45, 0.90]).reshape(3, 1)

capacidad_calorica_total = np.dot(bomba, capacidad_calorica)

i=1
for c in capacidad_calorica_total:
    print(f'Capacidad calórica del experimento {i} = {float(c)}')
    i+=1
```

    Capacidad calórica del experimento 1 = 583.5
    Capacidad calórica del experimento 2 = 541.5
    Capacidad calórica del experimento 3 = 545.7
    Capacidad calórica del experimento 4 = 535.62
    Capacidad calórica del experimento 5 = 538.98

# Determinantes e inversos

## Recuerde que no todas las matrices tienen inverso. Una matriz es singular (es decir: no tiene inverso) si su determinante es igual a 0 (es decir, |A| = 0). Use la función determinante para probar si cada una de las siguientes matrices tiene inverso:

Si existe un inverso, calcúlelo.

```python
def detInv(m, nombre):
    det = np.linalg.det(m)
    if det != 0:
        inv = np.linalg.inv(m)
        print(f'El determinante de la matriz {nombre} es: {det}')
        print(f'El inverso de la matriz {nombre} es:')
        print(inv)
        print('\n')
    else:
        print(f'El determinante de la matriz {nombre} es: {det} por lo tanto no tiene inverso.')
        print('\n')

A = np.array([
    [2, -1],
    [4, 5]
]).reshape(2,2)

B = np.array([
    [4, 2],
    [2, 1]
]).reshape(2,2)

C = np.array([
    [2, 0, 0],
    [1, 2, 2],
    [5, 4, 0]
]).reshape(3, 3)

detInv(A, 'A')
detInv(B, 'B')
detInv(C, 'C')
```

    El determinante de la matriz A es: 14.000000000000004
    El inverso de la matriz A es:
    [[ 0.35714286  0.07142857]
     [-0.28571429  0.14285714]]

    El determinante de la matriz B es: 0.0 por lo tanto no tiene inverso.

    El determinante de la matriz C es: -15.999999999999998
    El inverso de la matriz C es:
    [[ 0.5    0.     0.   ]
     [-0.625 -0.     0.25 ]
     [ 0.375  0.5   -0.25 ]]

# Resolución de sistemas ecuaciones lineales

## Resuelva el siguiente sistema de ecuaciones

```python
M = np.array([
    [3, 4, 2, -1, 1, 7, 1],
    [2, -2, 3, -4, 5, 2, 8],
    [1, 2, 3, 1, 2, 4, 6],
    [5, 10, 4, 3, 9, -2, 1],
    [3, 2, -2, -4, -5, -6, 7],
    [-2, 9, 1, 3, -3, 5, 1],
    [1, -2, -8, 4, 2, 4, 5],
]).reshape(7, 7)

res = np.array([42, 32, 12, -5, 10, 18, 17]).reshape(7, 1)

print('Solución del sistema de ecuaciones:\n')
print(np.linalg.solve(M, res))
```

    Solución del sistema de ecuaciones:

    [[-0.18899493]
     [ 2.54589061]
     [-3.28057396]
     [-6.75778176]
     [ 1.32124449]
     [ 4.31944831]
     [ 0.62940585]]

# Cálculo

La capacidad calorífica Cp de un gas se puede modelar con la ecuación empírica

\begin{equation}
C_p = a + bT + cT^2+dT^3
\end{equation}

donde a, b, c y d son constantes empíricas y T es la temperatura en grados Kelvin. El cambio en entalpía (una medida de energía) conforme el gas se caliente de T 2  es la integral de esta ecuación con respecto a T:

\begin{equation}
\bigtriangleup h = \int_{T_{1}}^{T_{2}} C_{p}\,dT
\end{equation}

## Encuentre el cambio en entalpía del oxígeno gaseoso conforme se calienta de 300 K a 1000 K. Los valores de a, b, c y d para el oxígeno son

\begin{equation}
\begin{split}
a & = 25.48 \\
b & = 1.520 x 10^{-2} \\
c & = -0.7155 x 10^{-5} \\
d & = 1.312 x 10^{-9}
\end{split}
\end{equation}

```python
a, b, c, d, C_p, T, T_1, T_2, h = sp.symbols('a b c d C_p T T_1 T_2 h')
```

```python
a = 25.480
b = 1.52 * (10**-2)
c = -0.7155 * (10**-5)
d = 1.312 * (10**-9)
T_1 = 300
T_2 = 1000
```

```python
C_p = a + b*(T_2) + c*(T_2**2) + d*(T_2**3)
C_p
```

    34.836999999999996

```python
h = sp.Integral(C_p, (T, T_1, T_2))
h
```

$\displaystyle \int\limits_{300}^{1000} 34.837\, dT$

```python
h.doit()
```

$\displaystyle 24385.9$


# The Hydrogenic atom

By separating the angular and radial components of the Wavefunction

\begin{align}
\Psi(x, y, z) \rightarrow R(r) Y(\phi, \theta)
\end{align}

the radial wavefunction for a Hydrogenic atom takes the form

\begin{align}
-\frac{1}{2 \mu} \frac{d^2 u(r)}{dr^2} + V_{eff}u(r) &= E u(r) \\
u(r) &= rR(r) \\
V_{eff}(r) &= -\frac{1}{r} + \frac{\hbar^2 l (l + 1)}{2 \mu r^2}
\end{align}

where $\mu$ is the reduced mass.

## Exercise 1

Plot the effective potential $V_{eff}(r)$ for different values of $l$. Note how different the potential is for $l = 0$ and for $l \neq 0$. Can you explain what happens?

## Exercise 2

Find the analytic expression for $R_{nl}(r)$ at [Explain Everything](https://drive.explaineverything.com/thecode/FNPFRHMN) and make Python Functions for
n = 1, l = 0, n = 2, l = 0, 1 and n = 3, l = 0, 1, 2.

Note that on Expliain Everything the functions are given as a function of $\rho$. The relation between $\rho$ and $r$ is: $\rho = \frac{2 Z}{n a}r$ where $a = 4\pi \epsilon_0 \frac{ \hbar^2 }{\mu e^2}$

```python
def R10(r):
    # n = 1, l = 0
```
Bonus points to those who can make a General R(n, l, r).
```python
def R(n, l, r):

```
I was not able to do this so please tell me if you come up with a solution.

## Exercise 3

Now let's study the radial probablilty density $P(r)$, the total wavefunction is
$\Psi_{nlm_l}$. To study the radial probability density we take the probability density $|\Psi_{nlm_l}|$ and integrate the non-radial parts ($\theta, \phi$).

\begin{align}
P(r) = \int_{surface} |\Psi_{nlm_l}(\theta, \phi, r)| d\tau = \int_0^\pi \int_0^{2 \pi} R_{nl}(r)^2 |Y_{lm_l}(\theta, \phi)| r^2\sin (\theta) d\theta d \phi
\end{align}

we then find $P(r) = R_{nl}(r)^2 r^2$

Plot the Quantity $P(r) = R_{nl}(r)^2 r^2 $ for various values of Z.

What happens when Z increases, why do you think this happens?

## Exercise 4

The total wavefunciton $Y_{n l m_l} = R_{nl}(r) Y_{lm_l}(\theta, \phi)$ where $R_{nl}$ is the solution to the Radial wavefunciton and

Make a function that returns the probability density of finding the particle within a given volume $\theta \in (\theta_0, \theta_1)$ $r \in (r_0, r_1)$$\phi \in (\phi_0, \phi_1)$

```python
def two_pz_probability(theta_0, theta_1, phi_0, phi_1, r_0, r_1):
```

```python

```


```python
import sympy as sp

print(f"SymPy Version: {sp.__version__}")

# 数式をキレイに表示する
sp.init_printing()
```

    SymPy Version: 1.8

### シンボルの生成

- 複数のシンボルを同時に作成する場合は、`sympy.symbols()`を使用する。
  + str型の引数として、シンボルをスペース区切りで指定する。
- 単一のシンボルを生成する場合は、`sympy.Symbol`クラスを使用出来る。

```python
x, y, z = sp.symbols('x y z')
x, y, z
```

```python
a = sp.Symbol('a')
a
```

### 数式を組み立てる

- 生成したシンボルのインスタンスを使用して、Pythonの演算子を使って式を定義するだけでよい
- 生成された式は、整形されたものになっているので、展開した数式が欲しい場合は`sympy.expand()`を使用する
- 逆に展開されたものを因数分解する場合は、`sympy.factor()`を使用する。
  + 式を整形する`sympy.simplify()`と合わせて使用されることが多い。

```python
f = (a * x + y) ** 2
print(f"式の型: {type(f)}")
f
```

```python
g = sp.expand(f)
print(f"式の型: {type(g)}")
g
```

```python
sp.factor(g)
```

### 数式を整形する

- あるシンボルについて整理する場合は、`sympy.collect()`を使用する

```python
exp = (x + y + z) ** 3
exp
```

```python
exp = sp.expand(exp)
exp
```

```python
# x について式を整理する
sp.collect(exp, x)
```

### 数式に値を代入する

- 生成した数式のインスタンスに対して、`数式.subs()`を呼び出す。
  + 引数は辞書型のインスタンスになる

```python
exp = (x + y + z) ** 3
exp = sp.expand(exp)
exp
```

```python
exp.subs({x: 2})
```

```python
exp.subs({x: 2, y: 1})
```

```python
exp.subs({x: 2, y: 1, z: 0.5})
```

### 数式から係数を抽出する

- 生成した数式のインスタンスに対して、`数式.coeff()`を呼び出す

```python
exp = (x + y + z) ** 3
exp = sp.expand(exp)
exp = sp.simplify(exp)
exp
```

```python
sp.collect(exp, x).coeff(x, 1)
```

### 分数を取り扱う

- 分数を代数的に扱う場合は、`sympy.Rational()`を使用する。
  + 引数は分子, 分母の順で指定する。

```python
sp.Rational(1, 3) + sp.Rational(1, 2)
```

### 数列を取り扱う

- 数列を定義する場合は、`sympy.sequence()`を使用する
  + 引数に、数列の式を与えるだけなので、等差数列や等比数列も扱うことができる。
- 作成した数列のインスタンスに対して、`数列.formula`を呼び出すと、定義された式を取り出せる
- 数列の和を計算する場合は、`sympy.summation()`を使用する。
- また、積和を計算する場合は、`sympy.product()`を使用する。

```python
n = sp.Symbol('n')
an = sp.sequence(3 * n + 1)
an
```

```python
an.formula
```

```python
sp.summation(an.formula, (n, 0, 5))
```

```python
sp.product(an.formula, (n, 0, 5))
```

### 連立方程式を取り扱う

- 方程式の定義には、`sympy.Eq`を使用する。
  + 引数には、左辺, 右辺の順で2つの式を与える
- 方程式を解く場合は、`sympy.solve()`を使う

```python
eq1 = sp.Eq(    x + y +     z, 4)
eq2 = sp.Eq(2 * x + y +     z, 6)
eq3 = sp.Eq(    x - y + 2 * z, 3)

equations = [eq1, eq2, eq3]
equations
```

```python
sp.solve(equations)
```

```python

```