# Sympy - Symbolic algebra in Python

J.R. Johansson (robert@riken.jp) http://dml.riken.jp/~rob/

The latest version of this [IPython notebook](http://ipython.org/notebook.html) lecture is available at [http://github.com/jrjohansson/scientific-python-lectures](http://github.com/jrjohansson/scientific-python-lectures).

The other notebooks in this lecture series are indexed at [http://jrjohansson.github.com](http://jrjohansson.github.com).

```
%pylab inline
```

    Welcome to pylab, a matplotlib-based Python environment [backend: module://IPython.kernel.zmq.pylab.backend_inline].
    For more information, type 'help(pylab)'.

## Introduction

There are two notable Computer Algebra Systems (CAS) for Python:

* [SymPy](http://sympy.org/en/index.html) - A python module that can be used in any Python program, or in an IPython session, that provides powerful CAS features.
* [Sage](http://www.sagemath.org/) - Sage is a full-featured and very powerful CAS enviroment that aims to provide an open source system that competes with Mathematica and Maple. Sage is not a regular Python module, but rather a CAS environment that uses Python as its programming language.

Sage is in some aspects more powerful than SymPy, but both offer very comprehensive CAS functionality. The advantage of SymPy is that it is a regular Python module and integrates well with the IPython notebook.

In this lecture we will therefore look at how to use SymPy with IPython notebooks. If you are interested in an open source CAS environment I also recommend to read more about Sage.

To get started using SymPy in a Python program or notebook, import the module `sympy`:

```
from sympy import *
```

To get nice-looking $\LaTeX$ formatted output run:

```
init_printing()

# or with older versions of sympy/ipython, load the IPython extension
#%load_ext sympy.interactive.ipythonprinting
# or
#%load_ext sympyprinting
```

## Symbolic variables

In SymPy we need to create symbols for the variables we want to work with. We can create a new symbol using the `Symbol` class:

```
x = Symbol('x')
```

```
(pi + x)**2
```

$$\left(x + \pi\right)^{2}$$

```
# alternative way of defining symbols
a, b, c = symbols("a, b, c")
```

```
type(a)
```

    sympy.core.symbol.Symbol

We can add assumptions to symbols when we create them:

```
x = Symbol('x', real=True)
```

```
x.is_imaginary
```

$$False$$

```
x = Symbol('x', positive=True)
```

```
x > 0
```

$$True$$

### Complex numbers

The imaginary unit is denoted `I` in Sympy.

```
1+1*I
```

$$1 + i$$

```
I**2
```

$$-1$$

```
(x * I + 1)**2
```

$$\left(i x + 1\right)^{2}$$

### Rational numbers

There are three different numerical types in SymPy: `Real`, `Rational`, `Integer`:

```
r1 = Rational(4,5)
r2 = Rational(5,4)
```

```
r1
```

$$\frac{4}{5}$$

```
r1+r2
```

$$\frac{41}{20}$$

```
r1/r2
```

$$\frac{16}{25}$$

## Numerical evaluation

SymPy uses a library for artitrary precision as numerical backend, and has predefined SymPy expressions for a number of mathematical constants, such as: `pi`, `e`, `oo` for infinity.

To evaluate an expression numerically we can use the `evalf` function (or `N`). It takes an argument `n` which specifies the number of significant digits.

```
pi.evalf(n=50)
```

$$3.1415926535897932384626433832795028841971693993751$$

```
y = (x + pi)**2
```

```
N(y, 5) # same as evalf
```

$$\left(x + 3.1416\right)^{2}$$

When we numerically evaluate algebraic expressions we often want to substitute a symbol with a numerical value. In SymPy we do that using the `subs` function:

```
y.subs(x, 1.5)
```

$$\left(1.5 + \pi\right)^{2}$$

```
N(y.subs(x, 1.5))
```

$$21.5443823618587$$

The `subs` function can of course also be used to substitute Symbols and expressions:

```
y.subs(x, a+pi)
```

$$\left(a + 2 \pi\right)^{2}$$

We can also combine numerical evolution of expressions with NumPy arrays:

```
import numpy
```

```
x_vec = numpy.arange(0, 10, 0.1)
```

```
y_vec = numpy.array([N(((x + pi)**2).subs(x, xx)) for xx in x_vec])
```

```
fig, ax = subplots()
ax.plot(x_vec, y_vec);
```

However, this kind of numerical evolution can be very slow, and there is a much more efficient way to do it: Use the function `lambdify` to "compile" a Sympy expression into a function that is much more efficient to evaluate numerically:

```
f = lambdify([x], (x + pi)**2, 'numpy')  # the first argument is a list of variables that
                                         # f will be a function of: in this case only x -> f(x)
```

```
y_vec = f(x_vec)  # now we can directly pass a numpy array and f(x) is efficiently evaluated
```

The speedup when using "lambdified" functions instead of direct numerical evaluation can be significant, often several orders of magnitude. Even in this simple example we get a significant speed up:

```
%%timeit

y_vec = numpy.array([N(((x + pi)**2).subs(x, xx)) for xx in x_vec])
```

    10 loops, best of 3: 20.4 ms per loop

```
%%timeit

y_vec = f(x_vec)
```

    100000 loops, best of 3: 3.67 µs per loop

## Algebraic manipulations

One of the main uses of an CAS is to perform algebraic manipulations of expressions. For example, we might want to expand a product, factor an expression, or simply an expression. The functions for doing these basic operations in SymPy are demonstrated in this section.

### Expand and factor

The first steps in an algebraic manipulation

```
(x+1)*(x+2)*(x+3)
```

$$\left(x + 1\right) \left(x + 2\right) \left(x + 3\right)$$

```
expand((x+1)*(x+2)*(x+3))
```

$$x^{3} + 6 x^{2} + 11 x + 6$$

The `expand` function takes a number of keywords arguments which we can tell the functions what kind of expansions we want to have performed. For example, to expand trigonometric expressions, use the `trig=True` keyword argument:

```
sin(a+b)
```

$$\sin{\left (a + b \right )}$$

```
expand(sin(a+b), trig=True)
```

$$\sin{\left (a \right )} \cos{\left (b \right )} + \sin{\left (b \right )} \cos{\left (a \right )}$$

See `help(expand)` for a detailed explanation of the various types of expansions the `expand` functions can perform.

The opposite a product expansion is of course factoring. The factor an expression in SymPy use the `factor` function:

```
factor(x**3 + 6 * x**2 + 11*x + 6)
```

$$\left(x + 1\right) \left(x + 2\right) \left(x + 3\right)$$

### Simplify

The `simplify` tries to simplify an expression into a nice looking expression, using various techniques. More specific alternatives to the `simplify` functions also exists: `trigsimp`, `powsimp`, `logcombine`, etc.

The basic usages of these functions are as follows:

```
# simplify expands a product
simplify((x+1)*(x+2)*(x+3))
```

$$\left(x + 1\right) \left(x + 2\right) \left(x + 3\right)$$

```
# simplify uses trigonometric identities
simplify(sin(a)**2 + cos(a)**2)
```

$$1$$

```
simplify(cos(x)/sin(x))
```

$$\frac{1}{\tan{\left (x \right )}}$$

### apart and together

To manipulate symbolic expressions of fractions, we can use the `apart` and `together` functions:

```
f1 = 1/((a+1)*(a+2))
```

```
f1
```

$$\frac{1}{\left(a + 1\right) \left(a + 2\right)}$$

```
apart(f1)
```

$$- \frac{1}{a + 2} + \frac{1}{a + 1}$$

```
f2 = 1/(a+2) + 1/(a+3)
```

```
f2
```

$$\frac{1}{a + 3} + \frac{1}{a + 2}$$

```
together(f2)
```

$$\frac{2 a + 5}{\left(a + 2\right) \left(a + 3\right)}$$

Simplify usually combines fractions but does not factor:

```
simplify(f2)
```

$$\frac{2 a + 5}{\left(a + 2\right) \left(a + 3\right)}$$

## Calculus

In addition to algebraic manipulations, the other main use of CAS is to do calculus, like derivatives and integrals of algebraic expressions.

### Differentiation

Differentiation is usually simple. Use the `diff` function. The first argument is the expression to take the derivative of, and the second argument is the symbol by which to take the derivative:

```
y
```

$$\left(x + \pi\right)^{2}$$

```
diff(y**2, x)
```

$$4 \left(x + \pi\right)^{3}$$

For higher order derivatives we can do:

```
diff(y**2, x, x)
```

$$12 \left(x + \pi\right)^{2}$$

```
diff(y**2, x, 2) # same as above
```

$$12 \left(x + \pi\right)^{2}$$

To calculate the derivative of a multivariate expression, we can do:

```
x, y, z = symbols("x,y,z")
```

```
f = sin(x*y) + cos(y*z)
```

$\frac{d^3f}{dxdy^2}$

```
diff(f, x, 1, y, 2)
```

$$- x \left(x y \cos{\left (x y \right )} + 2 \sin{\left (x y \right )}\right)$$

## Integration

Integration is done in a similar fashion:

```
f
```

$$\sin{\left (x y \right )} + \cos{\left (y z \right )}$$

```
integrate(f, x)
```

$$x \cos{\left (y z \right )} + \begin{cases} 0 & \text{for}\: y = 0 \\- \frac{\cos{\left (x y \right )}}{y} & \text{otherwise} \end{cases}$$

By providing limits for the integration variable we can evaluate definite integrals:

```
integrate(f, (x, -1, 1))
```

$$2 \cos{\left (y z \right )}$$

and also improper integrals

```
integrate(exp(-x**2), (x, -oo, oo))
```

$$\sqrt{\pi}$$

Remember, `oo` is the SymPy notation for inifinity.

### Sums and products

We can evaluate sums and products using the functions: 'Sum'

```
n = Symbol("n")
```

```
Sum(1/n**2, (n, 1, 10))
```

$$\sum_{n=1}^{10} n^{-2}$$

```
Sum(1/n**2, (n,1, 10)).evalf()
```

$$1.54976773116654$$

```
Sum(1/n**2, (n, 1, oo)).evalf()
```

$$1.64493406684823$$

Products work much the same way:

```
Product(n, (n, 1, 10)) # 10!
```

$$\prod_{n=1}^{10} n$$

## Limits

Limits can be evaluated using the `limit` function. For example,

```
limit(sin(x)/x, x, 0)
```

$$1$$

We can use 'limit' to check the result of derivation using the `diff` function:

```
f
```

$$\sin{\left (x y \right )} + \cos{\left (y z \right )}$$

```
diff(f, x)
```

$$y \cos{\left (x y \right )}$$

$\displaystyle \frac{\mathrm{d}f(x,y)}{\mathrm{d}x} = \frac{f(x+h,y)-f(x,y)}{h}$

```
h = Symbol("h")
```

```
limit((f.subs(x, x+h) - f)/h, h, 0)
```

$$y \cos{\left (x y \right )}$$

OK!

We can change the direction from which we approach the limiting point using the `dir` keywork argument:

```
limit(1/x, x, 0, dir="+")
```

$$\infty$$

```
limit(1/x, x, 0, dir="-")
```

$$-\infty$$

## Series

Series expansion is also one of the most useful features of a CAS. In SymPy we can perform a series expansion of an expression using the `series` function:

```
series(exp(x), x)
```

$$1 + x + \frac{1}{2} x^{2} + \frac{1}{6} x^{3} + \frac{1}{24} x^{4} + \frac{1}{120} x^{5} + \mathcal{O}\left(x^{6}\right)$$

By default it expands the expression around $x=0$, but we can expand around any value of $x$ by explicitly include a value in the function call:

```
series(exp(x), x, 1)
```

$$e + e x + \frac{1}{2} e x^{2} + \frac{1}{6} e x^{3} + \frac{1}{24} e x^{4} + \frac{1}{120} e x^{5} + \mathcal{O}\left(x^{6}\right)$$

And we can explicitly define to which order the series expansion should be carried out:

```
series(exp(x), x, 1, 10)
```

$$e + e x + \frac{1}{2} e x^{2} + \frac{1}{6} e x^{3} + \frac{1}{24} e x^{4} + \frac{1}{120} e x^{5} + \frac{1}{720} e x^{6} + \frac{1}{5040} e x^{7} + \frac{1}{40320} e x^{8} + \frac{1}{362880} e x^{9} + \mathcal{O}\left(x^{10}\right)$$

The series expansion includes the order of the approximation, which is very useful for keeping track of the order of validity when we do calculations with series expansions of different order:

```
s1 = cos(x).series(x, 0, 5)
s1
```

$$1 - \frac{1}{2} x^{2} + \frac{1}{24} x^{4} + \mathcal{O}\left(x^{5}\right)$$

```
s2 = sin(x).series(x, 0, 2)
s2
```

$$x + \mathcal{O}\left(x^{2}\right)$$

```
expand(s1 * s2)
```

$$x + \mathcal{O}\left(x^{2}\right)$$

If we want to get rid of the order information we can use the `removeO` method:

```
expand(s1.removeO() * s2.removeO())
```

$$\frac{1}{24} x^{5} - \frac{1}{2} x^{3} + x$$

But note that this is not the correct expansion of $\cos(x)\sin(x)$ to $5$th order:

```
(cos(x)*sin(x)).series(x, 0, 6)
```

$$x - \frac{2}{3} x^{3} + \frac{2}{15} x^{5} + \mathcal{O}\left(x^{6}\right)$$

## Linear algebra

### Matrices

Matrices are defined using the `Matrix` class:

```
m11, m12, m21, m22 = symbols("m11, m12, m21, m22")
b1, b2 = symbols("b1, b2")
```

```
A = Matrix([[m11, m12],[m21, m22]])
A
```

$$\left[\begin{smallmatrix}m_{11} & m_{12}\\m_{21} & m_{22}\end{smallmatrix}\right]$$

```
b = Matrix([[b1], [b2]])
b
```

$$\left[\begin{smallmatrix}b_{1}\\b_{2}\end{smallmatrix}\right]$$

With `Matrix` class instances we can do the usual matrix algebra operations:

```
A**2
```

$$\left[\begin{smallmatrix}m_{11}^{2} + m_{12} m_{21} & m_{11} m_{12} + m_{12} m_{22}\\m_{11} m_{21} + m_{21} m_{22} & m_{12} m_{21} + m_{22}^{2}\end{smallmatrix}\right]$$

```
A * b
```

$$\left[\begin{smallmatrix}b_{1} m_{11} + b_{2} m_{12}\\b_{1} m_{21} + b_{2} m_{22}\end{smallmatrix}\right]$$

And calculate determinants and inverses, and the like:

```
A.det()
```

$$m_{11} m_{22} - m_{12} m_{21}$$

```
A.inv()
```

$$\left[\begin{smallmatrix}\frac{1}{m_{11}} + \frac{m_{12} m_{21}}{m_{11}^{2} \left(m_{22} - \frac{m_{12} m_{21}}{m_{11}}\right)} & - \frac{m_{12}}{m_{11} \left(m_{22} - \frac{m_{12} m_{21}}{m_{11}}\right)}\\- \frac{m_{21}}{m_{11} \left(m_{22} - \frac{m_{12} m_{21}}{m_{11}}\right)} & \frac{1}{m_{22} - \frac{m_{12} m_{21}}{m_{11}}}\end{smallmatrix}\right]$$

## Solving equations

For solving equations and systems of equations we can use the `solve` function:

```
solve(x**2 - 1, x)
```

$$\begin{bmatrix}-1, & 1\end{bmatrix}$$

```
solve(x**4 - x**2 - 1, x)
```

$$\begin{bmatrix}- i \sqrt{- \frac{1}{2} + \frac{1}{2} \sqrt{5}}, & i \sqrt{- \frac{1}{2} + \frac{1}{2} \sqrt{5}}, & - \sqrt{\frac{1}{2} + \frac{1}{2} \sqrt{5}}, & \sqrt{\frac{1}{2} + \frac{1}{2} \sqrt{5}}\end{bmatrix}$$

System of equations:

```
solve([x + y - 1, x - y - 1], [x,y])
```

$$\begin{Bmatrix}x : 1, & y : 0\end{Bmatrix}$$

In terms of other symbolic expressions:

```
solve([x + y - a, x - y - c], [x,y])
```

$$\begin{Bmatrix}x : \frac{1}{2} a + \frac{1}{2} c, & y : \frac{1}{2} a - \frac{1}{2} c\end{Bmatrix}$$

## Quantum mechanics: noncommuting variables

How about non-commuting symbols? In quantum mechanics we need to work with noncommuting operators, and SymPy has a nice support for noncommuting symbols and even a subpackage for quantum mechanics related calculations!

```
from sympy.physics.quantum import *
```

## States

We can define symbol states, kets and bras:

```
Ket('psi')
```

$${\left|\psi\right\rangle }$$

```
Bra('psi')
```

$${\left\langle \psi\right|}$$

```
u = Ket('0')
d = Ket('1')

a, b = symbols('alpha beta', complex=True)
```

```
phi = a * u + sqrt(1-abs(a)**2) * d; phi
```

$$\alpha {\left|0\right\rangle } + \sqrt{- \left\lvert{\alpha}\right\rvert^{2} + 1} {\left|1\right\rangle }$$

```
Dagger(phi)
```

$$\overline{\alpha} {\left\langle 0\right|} + \overline{\sqrt{- \left\lvert{\alpha}\right\rvert^{2} + 1}} {\left\langle 1\right|}$$

```
Dagger(phi) * d
```

$$\left(\overline{\alpha} {\left\langle 0\right|} + \overline{\sqrt{- \left\lvert{\alpha}\right\rvert^{2} + 1}} {\left\langle 1\right|}\right) {\left|1\right\rangle }$$

Use `qapply` to distribute a mutiplication:

```
qapply(Dagger(phi) * d)
```

$$\overline{\alpha} \left\langle 0 \right. {\left|1\right\rangle } + \overline{\sqrt{- \left\lvert{\alpha}\right\rvert^{2} + 1}} \left\langle 1 \right. {\left|1\right\rangle }$$

```
qapply(Dagger(phi) * u)
```

$$\overline{\alpha} \left\langle 0 \right. {\left|0\right\rangle } + \overline{\sqrt{- \left\lvert{\alpha}\right\rvert^{2} + 1}} \left\langle 1 \right. {\left|0\right\rangle }$$

### Operators

```
A = Operator('A')
B = Operator('B')
```

Check if they are commuting!

```
A * B == B * A
```

$$False$$

```
expand((A+B)**3)
```

$$A B A + A \left(B\right)^{2} + \left(A\right)^{2} B + \left(A\right)^{3} + B A B + B \left(A\right)^{2} + \left(B\right)^{2} A + \left(B\right)^{3}$$

```
c = Commutator(A,B)
c
```

$$\left[A,B\right]$$

We can use the `doit` method to evaluate the commutator:

```
c.doit()
```

$$A B - B A$$

We can mix quantum operators with C-numbers:

```
c = Commutator(a * A, b * B)
c
```

$$\alpha \beta \left[A,B\right]$$

To expand the commutator, use the `expand` method with the `commutator=True` keyword argument:

```
c = Commutator(A+B, A*B)
c.expand(commutator=True)
```

$$- \left[A,B\right] B + A \left[A,B\right]$$

```
Dagger(Commutator(A, B))
```

$$- \left[A^{\dagger},B^{\dagger}\right]$$

```
ac = AntiCommutator(A,B)
```

```
ac.doit()
```

$$A B + B A$$

#### Example: Quadrature commutator

Let's look at the commutator of the electromagnetic field quadatures $x$ and $p$. We can write the quadrature operators in terms of the creation and annihilation operators as:

$\displaystyle x = (a + a^\dagger)/\sqrt{2}$

$\displaystyle p = -i(a - a^\dagger)/\sqrt{2}$

```
X = (A + Dagger(A))/sqrt(2)
X
```

$$\frac{1}{2} \sqrt{2} \left(A^{\dagger} + A\right)$$

```
P = -I * (A - Dagger(A))/sqrt(2)
P
```

$$- \frac{1}{2} \sqrt{2} i \left(- A^{\dagger} + A\right)$$

Let's expand the commutator $[x,p]$

```
Commutator(X, P).expand(commutator=True).expand(commutator=True)
```

$$- i \left[A^{\dagger},A\right]$$

Here we see directly that the well known commutation relation for the quadratures

$[x,p]=i$

is a directly related to

$[A, A^\dagger]=1$

(which SymPy does not know about, and does not simplify).

For more details on the quantum module in SymPy, see:

* http://docs.sympy.org/0.7.2/modules/physics/quantum/index.html
* http://nbviewer.ipython.org/urls/raw.github.com/ipython/ipython/master/docs/examples/notebooks/sympy_quantum_computing.ipynb

## Further reading

* http://sympy.org/en/index.html - The SymPy projects web page.
* https://github.com/sympy/sympy - The source code of SymPy.
* http://live.sympy.org - Online version of SymPy for testing and demonstrations.

## Versions

```
%reload_ext version_information

%version_information numpy, sympy
```

Software	Version

Python	3.3.2+ (default, Feb 28 2014, 00:52:16) [GCC 4.8.1]

IPython	2.2.0

OS	posix [linux]

numpy	1.8.2

sympy	0.7.5

Tue Aug 26 22:57:37 2014 JST


# Down-And-Out Put

```python
import pyvacon.analytics as analytics
import datetime as dt
import pyvacon.tools.converter as converter
import pyvacon.tools.enums as enums
import pyvacon.marketdata.testdata as mkt_testdata
import pyvacon.instruments.testdata as ins_testdata
import math
import pandas as pd
from scipy.stats import norm
import pyvacon.marketdata.plot as mkt_plot #import module for plotting functionality
#the next lin is a jupyter internal command to show the matplotlib graphs within the notebook
%matplotlib inline
# alternative library for plots (offline version)
```

## Definition of a Down-And-Out Put

A down-and-out put belongs to the class of barrier options. It is a put option that ceases to exist if the underlying's spot prices reaches a barrier which is less that the current asset price.

## Down-And-Out Put Pricing Formulas

The value of a down-and-out put in the case of a barrier which es less than the strike price can be calculated subtracting the price of a down-and-in put from a plain vanilla put. In the case of a barrier which is greater than or equal to the strike price, the price of the down-and-out put is by definition zero. Because the value of a down-and-in put with a barrier greater or equal than the strike equals the value of a plain vanilla put option, the formula proves this finding.

$$p_{do} = p - p_{di}$$

The price of a down-and-in put $p_{di}$ is defined as:

$$p_{di}=-S_0N(-x_1)e^{-qT}+Ke^{-rT}N(-x_1+\sigma\sqrt{T})+S_0e^{-qT}(H/S_0)^{2\lambda}[N(y)-N(y_1)]-Ke^{-rT}(H/S_0)^{2\lambda-2}[N(y-\sigma\sqrt{T})-N(y_1-\sigma\sqrt{T})]$$

with $S_0$ as the underlying asset's spot price, $K$ as the option's strike price, $H$ as the option's barrier, $N(x)$ as the cumulative probability distribution function for a standardized normal distribution, $r$ as the risk-free interest rate, $q$ as the borrowing rate, $\sigma$ as the underlying's volatility and $T$ as the option's maturity as year fraction

where

\begin{align}
\lambda &=\frac{r-q+\sigma^2/2}{\sigma^2} \\
\\
y &=\frac{\ln[H^2/(S_0K)]}{\sigma\sqrt{T}}+\lambda\sigma\sqrt{T} \\
\\
x_1 &=\frac{\ln(S_0/H)}{\sigma\sqrt{T}}+\lambda\sigma\sqrt{T} \\
\\
y_1 &=\frac{\ln(H/S_0)}{\sigma\sqrt{T}}+\lambda\sigma\sqrt{T} \\
\end{align}

The following code defines the formula to value an down-and-out put.

```python
def dopPricer(S0, K, H, r, sigma, T, q=0, t=0):
    _lambda = (r-q+sigma**2/2)/sigma**2
    y = (math.log(H**2/(S0*K)))/(sigma*math.sqrt(T-t))+_lambda*sigma*math.sqrt(T-t)
    x1 = (math.log(S0/H))/(sigma*math.sqrt(T-t))+_lambda*sigma*math.sqrt(T-t)
    y1 = (math.log(H/S0))/(sigma*math.sqrt(T-t))+_lambda*sigma*math.sqrt(T-t)
    pdi = -S0*norm.cdf(-x1)*math.exp(-q*(T-t))+K*math.exp(-r*(T-t))*norm.cdf(-x1+sigma*math.sqrt(T-t))+S0*math.exp(-q*(T-t))*(H/S0)**(2*_lambda)*(norm.cdf(y)-norm.cdf(y1))-K*math.exp(-r*(T-t))*(H/S0)**(2*_lambda-2)*(norm.cdf(y-sigma*math.sqrt(T-t))-norm.cdf(y1-sigma*math.sqrt(T-t)))
    d1= (math.log(S0/K)+(r+sigma**2/2)*(T-t))/(sigma*math.sqrt(T-t))
    d2 = d1-sigma*math.sqrt(T-t)
    p = -1*(S0*norm.cdf(-1*d1)-K*math.exp(-r*(T-t))*norm.cdf(-1*d2))
    pdo = p - pdi
    if H>=K:
        return 0
    if H<K and S0>H:
        return pdo
    if H<K and S0<=H:
        return 0
```

## Manual Pricing of a Down-And-Out Put Option

In the following code, we use the defined formulas above and calculate the price of a down-and-out put assuming a non-dividend paying stock.

```python
S0 = 50.0
K = 50.0
H = 40.0
r = 0.05
q = 0
sigma = 0.3
T = 1.0
t = 0

print('Down-and-out put price =',dopPricer(S0, K, H, r, sigma, T, q, t))
```

**Plotting the price profile of a down-and-out put**

The following code constructs a plot which shows the option price dependent on the spot price. Additionally, the further plot shows the delta function of the down-and-out put.

```python
#Create different spots
spots = []
n=0.5
while n<=100:
    spots.append(n)
    n=n+0.1

#Calculate option prices for the individual spots assuming everything else remains the same
prices = []
for s in range(len(spots)):
    prices.append(dopPricer(spots[s], K, H, r, sigma, T, q, t))

prices_list = {'Spots': spots,
              'Prices': prices}

dop_prices = pd.DataFrame(prices_list, index = spots)

# Delta
nominator = []
for s in range(len(spots)):
    nominator.append(dopPricer(spots[s], K, H, r, sigma, T, q, t)-dopPricer(spots[s-1], K, H, r, sigma, T, q, t))

denominator = []
for s in range(len(spots)):
    denominator.append(spots[s]-spots[s-1])

value = []
for s in range(len(nominator)):
    value.append(nominator[s]/denominator[s])

#Alternative way to calculate the deltas
#deltas = analytics.vectorDouble()
#for i in range(len(spots)):
#    deltas.append(norm.cdf((math.log(spots[s]/K)+(r+sigma**2/2)*(T-t))/(sigma*math.sqrt(T-t)))-1)

delta_list = {'Spots': spots,
              'Deltas': value}

dop_deltas = pd.DataFrame(delta_list, index = spots)

# Plotting

fig, (dop, dop_delta) = mkt_plot.plt.subplots(1,2, figsize=(12,4),dpi=100,num=1)
dop.plot(dop_prices['Spots'], dop_prices['Prices'], 'k', label='Down-and-out put price')
dop.set_title('Down-and-out Put Prices')
dop.set_xlabel('Spot')
dop.set_ylabel('Price')
dop.axvline(x=K, label='Strike', ls= '--', c='g')
dop.axvline(x=H, label='Barrier', ls=':', c='r')
legend = dop.legend(loc='best', shadow=True, fontsize='medium')

#fig, cdi2 = mkt_plot.plt.subplots()
dop_delta.plot(dop_deltas['Spots'], dop_deltas['Deltas'], 'k', label='Down-and-out put delta')
dop_delta.set_title('Down-and-out Put Deltas')
dop_delta.set_xlabel('Spot')
dop_delta.set_ylabel('Price')
dop_delta.axvline(x=K, label='Strike', ls= '--', c='g')
dop_delta.axvline(x=H, label='Barrier', ls=':', c='r')
legend = dop_delta.legend(loc='best', shadow=True, fontsize='medium')
```

## Option Pricing using pyvacon

### Create the necessary market data
As a first step, we need to create the market data necessary to undertake the valuation. Therefore, we need to construct a *discount-*, *funding-* and *borrowing curve*, a *forward-curve* as well as a *volatility surface*.

#### Create a discount-, funding-, and borrowing-curve

```python
object_id = "TEST_DC"
refdate = analytics.ptime(2017,1,1,0,0,0)
days_to_maturity = [1, 180, 365, 720, 3*365, 4*365, 10*365]
dates = converter.createPTimeList(refdate, days_to_maturity)
# discount factors from constant rate
disc_rate = 0.05
borrow_rate = 0.0
funding_rate = 0.05

disc_df = analytics.vectorDouble()
for d in days_to_maturity:
    disc_df.append(math.exp(-d/365.0*disc_rate))
dc = analytics.DiscountCurve(object_id, refdate,dates, disc_df, enums.DayCounter.ACT365_FIXED, enums.InterpolationType.HAGAN_DF, enums.ExtrapolationType.NONE)

borrow_df = analytics.vectorDouble()
for d in days_to_maturity:
    borrow_df.append(math.exp(-d/365.0*borrow_rate))
bc = analytics.DiscountCurve(object_id, refdate, dates, borrow_df, enums.DayCounter.ACT365_FIXED,
                             enums.InterpolationType.HAGAN_DF, enums.ExtrapolationType.NONE)

funding_df = analytics.vectorDouble()
for d in days_to_maturity:
    funding_df.append(math.exp(-d/365.0*funding_rate))
fc = analytics.DiscountCurve(object_id, refdate, dates, funding_df, enums.DayCounter.ACT365_FIXED,
                             enums.InterpolationType.HAGAN_DF, enums.ExtrapolationType.NONE)
```

#### Create a forward curve
An equity forward curve has to be created to determine the future spot price. The procedure follows the description in the [equity forward curves](../marketdata/equity_forwardcurve.ipynb) notebook. To make the results comparable, we assume that the underlying is a non-dividend paying stock.

```python
# Creation of a Forward curve
#div table
object_id = "Div_Table"
refdate = analytics.ptime(2017,1,1,0,0,0) #dates which enters analytics objects must be analytics ptimes.

ex_dates = converter.createPTimeList(refdate, [dt.datetime(2018,3,29), dt.datetime(2019,3,29), dt.datetime(2020,3,29), dt.datetime(2021,3,29)])
pay_dates = converter.createPTimeList(refdate, [dt.datetime(2018,4,1), dt.datetime(2019,4,1), dt.datetime(2020,4,1), dt.datetime(2021,4,1)])
tax_factors = analytics.vectorDouble([1.0, 1.0, 1.0, 1.0])
div_yield = analytics.vectorDouble([0, 0.00, 0.0, 0.0])
div_cash = analytics.vectorDouble([0.0, 0.0, 0.0, 0.0])

div_table=analytics.DividendTable(object_id, refdate, ex_dates, div_yield, div_cash, tax_factors, pay_dates)

spot = S0

forward_curve = analytics.EquityForwardCurve(refdate, spot, fc, bc, div_table)
```

#### Create a volatility surface
The procedure follows the description in the [equity volatilities](../marketdata/equity_volatilities.ipynb) notebook. In order to compare the results with the model above, we use a flat volatility surface.

```python
#expiries = [1.0/12.0, 1.0, 2.0, 3.0]
#atm_vols =  [0.3, 0.28, 0.25, 0.24]
#term_param = analytics.VolatilityParametrizationTerm(expiries,atm_vols)
flat_param = analytics.VolatilityParametrizationFlat(0.3)

obj_id = 'TEST_SURFACE'
vol_surf = analytics.VolatilitySurface(obj_id, refdate, forward_curve, enums.DayCounter.ACT365_FIXED, flat_param)

fwdspot = forward_curve.value(converter.getLTime(refdate),converter.getLTime(refdate))
```

### Setup the specification

The down-and-out put specification requires the following parameters:

- spot price of the underlying
- currency
- expiry relative to the reference date in days
- barrier level relative to the spot
- rebate level relative to the spot
- strike level relative to the spot
- underlying
- reference date
- barrier at start

```python
rel_expiry = 365 # expiry in number of days
rel_level = (H/spot) #barrier level relative to spot # Achtung, hier stand 0, was wird hier genau erwartet?
rel_rebate = 0 # rebate relative to spot
rel_strike = K/spot # strike relative to spot
spot=fwdspot
udl = 'DBK'
dop = ins_testdata.DOP.__create_DOP__(spot, 'EUR', rel_expiry, rel_level, rel_strike, rel_rebate,
                                        udl,converter.getLTime(refdate))
#help(ins_testdata.DOP.__create_DOP__)
```

### Setup the pricing data
A product may be priced in two different ways: One may either fill the respective pricing data needed for a special pricer (which inherits from the BasePricingData) and use the respective price method where just the pricing data is given. Another possibility is to use the price-method where the storages are given. In this case, the pricer will fill the needed pricing data according to the underlying and other data as specified in the product specification.

**Here we show the approach explicitly setting the pricing data.**

```python
dop_pricing_data = analytics.LocalVolPdePricingData()
#help(analytics.LocalVolPdePricingData)
dop_pricing_data.pricer = 'LocalVolPdePricer'
dop_pricing_data.valDate = converter.getLTime(refdate)
dop_pricing_data.pricingRequest = analytics.PricingRequest()
dop_pricing_data.vol = vol_surf
dop_pricing_data.spec = dop
dop_pricing_data.param = analytics.PdePricingParameter()
dop_pricing_data.dsc = dc
```

### Pricing

After all necessary information has been set, the price of the option can be calculated using the *analytics.price* function.

```python
pr = analytics.price(dop_pricing_data)
pr.getPrice()
```

---


# DC motor with observer

```python
import numpy as np
import sympy as sy
sy.init_printing()
```

```python
s = sy.symbols('s', real=False)
J, gr, b, k, kv, tau, tauc = sy.symbols('J, gr, b, k, k_v, tau, tau_c', real=True, positive=True)
l1, l2, k1, k2 = sy.symbols('l_1, l_2, k_1, k_2', real=True)
```

```python
# Using physical states x = [omega, theta]
A = sy.Matrix([[-1/tau, 0], [1, 0]])
B = sy.Matrix([[k/tau], [0]])
Bv = sy.Matrix([[1/J], [0]])
C = sy.Matrix([[0, 1]])

```

```python
# Charact polynomial
L = sy.Matrix([[l1, l2]])
M = s*sy.eye(2) - (A-B*L)
char_pol = sy.poly(sy.det(M), s)
char_pol

```

```python
# Desired closed-loop poles
wc = 1/tauc
p1 = wc/sy.sqrt(2)*(-1 + sy.I)
p2 = wc/sy.sqrt(2)*(-1 - sy.I)
char_pol_des = sy.poly((s-p1)*(s-p2), s)
char_pol_des
```

```python
eqns = [c_ - cd_ for c_, cd_ in zip(char_pol.coeffs(), char_pol_des.coeffs())]
sol = sy.solve(eqns, [l1, l2])
sol
```

## Observer design

```python
d = sy.symbols('d', real=True, positive=True)
po1 = d*p1
po2 = d*p2
char_pol_des_o = sy.poly((s-po1)*(s-po2), s)
char_pol_des_o
```

```python
K = sy.Matrix([[k1], [k2]])
char_pol_o = sy.Poly(sy.det(s*sy.eye(2)-(A-K*C)), s)
char_pol_o
```

```python
eqns = [c_ - cd_ for c_, cd_ in zip(char_pol_o.coeffs(), char_pol_des_o.coeffs())]
sol_o = sy.solve(eqns, [k1, k2])
sol_o
```

```python
K*C
```

```python
A-K*C
```

```python
s*sy.eye(2)-(A-K*C)
```

## Observer gains to eliminate constant input disturbance

Plant dynamics, reference signal $r=0$
$$ \dot{x} = Ax - BL\hat{x} + Bv $$

Observer dynamics
$$ \dot{\hat{x}} = A\hat{x} - BL\hat{x} + KCx - KC\hat{x} + Bv = (A-KC)\hat{x} + KCx + Bv $$

Error dynamics, $\tilde{x} = x-\hat{x}$
$$ \dot{\tilde{x}} = (A-KC)\tilde{x} + Bv $$

Steady-state estimation error
$$ \tilde{x}_{ss} = -(A-KC)^{-1}Bv. $$
Steady-state state
$$0 = Ax_{ss} - BL\hat{x}_{ss} + Bv$$
Steady-state output
\begin{align}
y_{ss} = Cx_{ss} = C\hat{x}_{ss} + C\tilde{x}_ss
\end{align}

```python
kv = Bv[0,0]/B[0,0]
kv

```

```python
AminKC = A-K*C
AminKCinv = AminKC.inv()
AminKC, AminKCinv
```

```python
lhs = L*AminKC.inv()*B
lhs
```

```python
sol = sy.solve(lhs[0,0]+1,k1)
sol
```

```python
sol = sy.solve(lhs[0,0].subs([(k2, tau*k1)])+1,k1)
sol
```

```python
a, b = sy.symbols('a, b')
sol = sy.solve(lhs[0,0].subs([(k2, tau*k1), (k, b*tau),(tau, 1/a)])+1,k1)
sol[0]

```

```python
Kk = sy.Matrix([sol[0], 1/a*sol[0]])
Kk
```

```python
AA = A.subs([(k, b*tau),(tau, 1/a)])
BB = B.subs([(k, b*tau),(tau, 1/a)])
BB
AminKkC = AA-Kk*C
AminKkCinv = AminKkC.inv()
AA, AminKkC, AminKkCinv
```

```python
sy.simplify(L*AminKkCinv*BB)
```

```python
L
```

```python

```

modern-control/notebooks/DC motor with observer.ipynb


Construir o gráfico e encontrar o foco e uma equação da diretriz.

10. $x = -\frac{y^2}{8}$

Arrumando a equação

$y^2 = -8x$

$2p = -8$, logo

$p =  -4$

Calculando o foco

$F = \frac{p}{2}$

$F = -\frac{4}{2}$

$F =  -2$

$F(-2,0)$

Calculando a diretriz

$D = -\frac{p}{2}$

$D : x = -2$

$V(0,0)$

$F(-2,0)$

Gráfico da parábola

```python
from sympy import *
from sympy.plotting import plot_implicit
x, y = symbols("x y")
plot_implicit(Eq((y-0)**2, -8*(x+0)), (x,-10,10), (y,-10,10),
title=u'Gráfico da parábola', xlabel='x', ylabel='y');
```

Problemas Propostos. Pag. 172 - 175/10.ipynb


# 初始化环境

```python
from IPython.display import display, Math, Latex
from sympy import *
init_printing()

from helper import comparator_factory, comparator_eval_factory, comparator_method_factory

x,y,z = symbols('x y z')

func_comparator = comparator_factory('使用{}前:','使用后:')
method_comparator = comparator_method_factory('调用{}前:','调用后:')
eval_comparator = comparator_eval_factory('计算前:','计算后:')
```

# 矩阵

## 创建矩阵
### 从list创建矩阵
如果要在Sympy中创建一个矩阵， 可以通过行向量(内嵌的list)构成的list初始化一个```Matrix```类。

```python
m = Matrix(
    [
        [1, -1],
        [3, 4],
        [0, 2]
    ]
)

m
```

$$\left[\begin{matrix}1 & -1\\3 & 4\\0 & 2\end{matrix}\right]$$

如果只传入单层的list，那么会创建一个$n \times 1$矩阵。

```python
m = Matrix([1,2,3])

m
```

$$\left[\begin{matrix}1\\2\\3\end{matrix}\right]$$

## 构造常用的矩阵

有一些创建常用矩阵的方法。

### 单位矩阵
用```eye(n)```创建单位矩阵。

```python
M = eye(3)

M
```

$$\left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{matrix}\right]$$

### 零矩阵

通过```zero(n)```去创建零矩阵。

```python
M = zeros(3,4)

display(M)
```

$$\left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right]$$

### Matrix of Ones
通过```ones(n)```去创建数值全为1的矩阵。

```python
M = ones(3,2)

M
```

$$\left[\begin{matrix}1 & 1\\1 & 1\\1 & 1\end{matrix}\right]$$

### 对角矩阵
传入数个矩阵或者数值去构造对角矩阵。 数值会被处理成$1 \times 1$矩阵。

```python
M = diag(-1, ones(2, 2), Matrix([5, 7, 5]))

M
```

$$\left[\begin{matrix}-1 & 0 & 0 & 0\\0 & 1 & 1 & 0\\0 & 1 & 1 & 0\\0 & 0 & 0 & 5\\0 & 0 & 0 & 7\\0 & 0 & 0 & 5\end{matrix}\right]$$

## 基本操作

### 获得形状

用```shape```获得矩阵的形状。

```python
M.shape
```

### 获取行或者列
使用```row()```和```col()```获取单行或者单列。

```python
M.row(0)
```

$$\left[\begin{matrix}-1 & 0 & 0 & 0\end{matrix}\right]$$

获取单列。

```python
M.col(-1)
```

$$\left[\begin{matrix}0\\0\\0\\5\\7\\5\end{matrix}\right]$$

### 删除行列
用```row_del```或```col_del```删除行或列。 这些操作会直接修改矩阵。

例如，我们要删除第一行和最后一列。

```python
M = Matrix([[1, 2, 3], [3, 2, 1]])

display('删除前:',M)

M.row_del(0)
M.col_del(-1)

display('删除后:',M)
```

    '删除前:'

$$\left[\begin{matrix}1 & 2 & 3\\3 & 2 & 1\end{matrix}\right]$$

    '删除后:'

$$\left[\begin{matrix}3 & 2\end{matrix}\right]$$

### 矩阵转置
使用```T```

```python
M = Matrix([[1, 2, 3], [4, 5, 6]])

display('原矩阵:',M)
display('转职后矩阵:',M.T)
```

    '原矩阵:'

$$\left[\begin{matrix}1 & 2 & 3\\4 & 5 & 6\end{matrix}\right]$$

    '转职后矩阵:'

$$\left[\begin{matrix}1 & 4\\2 & 5\\3 & 6\end{matrix}\right]$$

### 插入行列

如果要插入行或者列，用```row_insert```或```col_insert```。 这些操作并不会直接修改原矩阵而是返回数值被修改的一个新矩阵。

```row_insert()```接受位置和$1 \times n$矩阵。

```python
M = Matrix([[1, 2, 3], [3, 2, 1]])

method_comparator(M, 'row_insert', 0, Matrix([[6,6,6]]))
```

    调用row_insert()前:

$$\left[\begin{matrix}1 & 2 & 3\\3 & 2 & 1\end{matrix}\right]$$

    调用后:

$$\left[\begin{matrix}6 & 6 & 6\\1 & 2 & 3\\3 & 2 & 1\end{matrix}\right]$$

```col_insert()```接受位置和$n \times 1$矩阵。

```python
M = Matrix([[1, 2, 3], [3, 2, 1]])

method_comparator(M, 'col_insert', -1,  Matrix([6,6]))
```

    调用col_insert()前:

$$\left[\begin{matrix}1 & 2 & 3\\3 & 2 & 1\end{matrix}\right]$$

    调用后:

$$\left[\begin{matrix}1 & 2 & 6 & 3\\3 & 2 & 6 & 1\end{matrix}\right]$$

## 矩阵计算
基本的矩阵计算像加法，减法，乘法，求逆可以通过pythhon的算符```+```,```-```,```*```,```**```来实现。

### 加法

```python
M = Matrix([[1, 3], [-2, 3]])
N = Matrix([[0, 3], [0, 7]])

print('M')
display(M)
print('N')
display(N)
print('M+N')
display(M+N)
```

    M

$$\left[\begin{matrix}1 & 3\\-2 & 3\end{matrix}\right]$$

    N

$$\left[\begin{matrix}0 & 3\\0 & 7\end{matrix}\right]$$

    M+N

$$\left[\begin{matrix}1 & 6\\-2 & 10\end{matrix}\right]$$

### 减法

```python
M = Matrix([[1, 3], [-2, 3]])
N = Matrix([[0, 3], [0, 7]])

print('M')
display(M)
print('N')
display(N)
print('M-N')
display(M-N)
```

    M

$$\left[\begin{matrix}1 & 3\\-2 & 3\end{matrix}\right]$$

    N

$$\left[\begin{matrix}0 & 3\\0 & 7\end{matrix}\right]$$

    M-N

$$\left[\begin{matrix}1 & 0\\-2 & -4\end{matrix}\right]$$

### 乘法

```python
M = Matrix([[1, 2, 3], [3, 2, 1]])
N = Matrix([0, 1, 1])

print('M')
display(M)
print('N')
display(N)
display('MxN')
display(M*N)
```

    M

$$\left[\begin{matrix}1 & 2 & 3\\3 & 2 & 1\end{matrix}\right]$$

    N

$$\left[\begin{matrix}0\\1\\1\end{matrix}\right]$$

    'MxN'

$$\left[\begin{matrix}5\\3\end{matrix}\right]$$

### 求逆

```python
M = Matrix([[1, 3], [-2, 3]])

print('M')
display(M)
print('inverse of M:')
display(M**(-1))
```

    M

$$\left[\begin{matrix}1 & 3\\-2 & 3\end{matrix}\right]$$

    inverse of M:

$$\left[\begin{matrix}\frac{1}{3} & - \frac{1}{3}\\\frac{2}{9} & \frac{1}{9}\end{matrix}\right]$$

## 行列式
用```det```计算矩阵的行列式

```python
M = Matrix([[1, 0, 1], [2, -1, 3], [4, 3, 2]])

method_comparator(M,'det')
```

### 行阶梯型矩阵
用```rref()```将矩阵处理成行阶梯型矩阵。 它返回两个结果，第一个是行阶梯型矩阵， 第二个是主元列的index。

```python
M = Matrix([[1, 0, 1, 3], [2, 3, 4, 7], [-1, -3, -3, -4]])

M_rref, pivot_columns = M.rref()

print('原矩阵:')
display(M)

print('阶梯型:')
display(M_rref)

print('主元index:')
display(pivot_columns)
```

### 零空间

```nullspace()```返回一个能够张成矩阵零空间的向量组成的list。

```python
M = Matrix([[1, 2, 3, 0, 0], [4, 10, 0, 0, 1]])

print('原矩阵M:')
display(M)
print('矩阵M的零空间的张成向量:')
display(M.nullspace())
```

    原矩阵M:

$$\left[\begin{matrix}1 & 2 & 3 & 0 & 0\\4 & 10 & 0 & 0 & 1\end{matrix}\right]$$

    矩阵M的零空间的张成向量:

$$\left [ \left[\begin{matrix}-15\\6\\1\\0\\0\end{matrix}\right], \quad \left[\begin{matrix}0\\0\\0\\1\\0\end{matrix}\right], \quad \left[\begin{matrix}1\\- \frac{1}{2}\\0\\0\\1\end{matrix}\right]\right ]$$

### 列空间

```columnspace()```返回一个能够张成矩阵列空间的向量组成的list。

```python
M = Matrix([[1, 1, 2], [2 ,1 , 3], [3 , 1, 4]])

print('矩阵M:')
display(M)
print('张成矩阵M的列空间的向量:')
display(M.columnspace())
```

    矩阵M:

$$\left[\begin{matrix}1 & 1 & 2\\2 & 1 & 3\\3 & 1 & 4\end{matrix}\right]$$

    张成矩阵M的列空间的向量:

$$\left [ \left[\begin{matrix}1\\2\\3\end{matrix}\right], \quad \left[\begin{matrix}1\\1\\1\end{matrix}\right]\right ]$$

### 特征值和特征向量
要计算矩阵的特征值，可以使用```eigenvals```。它返回由特征值:代数重数对组成的dict。

```python
M = Matrix([[3, -2,  4, -2], [5,  3, -3, -2], [5, -2,  2, -2], [5, -2, -3,  3]])

print('矩阵M')
display(M)

print('矩阵M的特征值和它们的代数重数:')
display(M.eigenvals())
```

```eigenvectors```返回由特征值，几何重数，特征向量列表组成的tuple。

```python
M = Matrix(
    [
        [3, -2,  4, -2],
        [5,  3, -3, -2],
        [5, -2,  2, -2],
        [5, -2, -3,  3]
    ]
)

print('矩阵M')
display(M)

print('特征值，几何重数，特征向量:')
display(M.eigenvects())
```

    矩阵M

$$\left[\begin{matrix}3 & -2 & 4 & -2\\5 & 3 & -3 & -2\\5 & -2 & 2 & -2\\5 & -2 & -3 & 3\end{matrix}\right]$$

    特征值，几何重数，特征向量:

$$\left [ \left ( -2, \quad 1, \quad \left [ \left[\begin{matrix}0\\1\\1\\1\end{matrix}\right]\right ]\right ), \quad \left ( 3, \quad 1, \quad \left [ \left[\begin{matrix}1\\1\\1\\1\end{matrix}\right]\right ]\right ), \quad \left ( 5, \quad 2, \quad \left [ \left[\begin{matrix}1\\1\\1\\0\end{matrix}\right], \quad \left[\begin{matrix}0\\-1\\0\\1\end{matrix}\right]\right ]\right )\right ]$$

如果只是想获得特征多项式， 用```charpoly()```和```factor()```

```python
print('矩阵M')
display(M)

lamda = symbols('lamda')
p = M.charpoly(lamda).factor()

print('矩阵M的特征多项式')
display(p)
```

## 对角化

如果要对角化一个矩阵，用```diagonalize()```。 它返回($P$,$D$)组成的tuple，其中$D$是对角矩阵并且满足$M = PDP^{-1}$。

```python
M = Matrix(
    [
        [3, -2,  4, -2],
        [5,  3, -3, -2],
        [5, -2,  2, -2],
        [5, -2, -3,  3]
    ]
)

P, D = M.diagonalize()

print('矩阵M')
display(M)

print('矩阵P')
display(P)

print('矩阵D')
display(D)

display(Latex('$PDP^{-1}$'))
display(P*D*P**-1)
```

    矩阵M

$$\left[\begin{matrix}3 & -2 & 4 & -2\\5 & 3 & -3 & -2\\5 & -2 & 2 & -2\\5 & -2 & -3 & 3\end{matrix}\right]$$

    矩阵P

$$\left[\begin{matrix}0 & 1 & 1 & 0\\1 & 1 & 1 & -1\\1 & 1 & 1 & 0\\1 & 1 & 0 & 1\end{matrix}\right]$$

    矩阵D

$$\left[\begin{matrix}-2 & 0 & 0 & 0\\0 & 3 & 0 & 0\\0 & 0 & 5 & 0\\0 & 0 & 0 & 5\end{matrix}\right]$$

$PDP^{-1}$

$$\left[\begin{matrix}3 & -2 & 4 & -2\\5 & 3 & -3 & -2\\5 & -2 & 2 & -2\\5 & -2 & -3 & 3\end{matrix}\right]$$

# 等式求解

## 基本求解
代数等式求解的主要函数是```solveset()```。 它的语法是```solveset(equation, variable=None, domain=S.Complexes)```

```python
eq = Eq(x**2-1, 0)

func_comparator(eq, solveset, x)
```

如果```solveset()```中传入一个表达式而非等式， 那么它会被假设等于0,。

```python
expr = x**2-1

func_comparator(expr, solveset, x)
```

求解结果可以是无限集合。

```python
eq = Eq(sin(x) - 1)

func_comparator(eq, solveset, x)
```

求解结果可以是空集。

```python
eq = Eq(exp(x))

func_comparator(eq, solveset, x)
```

如果求解失败，会返回一个条件集合。

```python
eq = Eq(cos(x) - x)

func_comparator(eq, solveset, x)
```

```solveset()```只会显示每个解一次。

```python
eq = Eq(x**3 - 6*x**2 + 9*x,0)

func_comparator(eq, solveset, x)
```

如果要获得多项等式每个解的重数，可以用```roots()```

```python
eq = Eq(x**3 - 6*x**2 + 9*x,0)

func_comparator(eq, roots, x)
```

## 线性系统

```linsolve()```用于求解线性系统. 线性系统有不同的定义方法。

### 等式list

```python
ls = [
    Eq(x + y + z - 1),
    Eq(x + y + 2*z - 3)
]

func_comparator(ls,linsolve, x, z)
```

等式list可以简化成表达式list， 如果这些表达式都等于0。

```python
ls = [
    x + y + z - 1,
    x + y + 2*z - 3
]

func_comparator(ls,linsolve, x, z)
```

### 增广矩阵形式

```python
M = Matrix(
    [
        [1, 1, 1, 1],
        [1, 1, 2, 3]
    ]
)

func_comparator(M,linsolve, x, y, z)
```

### $Ax = b$ 形式

```python
M = Matrix(
    [
        [1, 1, 1, 1],
        [1, 1, 2, 3]
    ]
)
ls = A, b = M[:, :-1], M[:, -1]

func_comparator(ls,linsolve, x, y, z)
```

## 非线性系统

```solvset()```不能解非线性多元系统。 遇到这样的需求用```solve()```代替。

```python
eq_list = [
    Eq(x*y - 1, 0),
    Eq(x-2, 0)
]

func_comparator(eq_list, solve, x, y)
```

## 微分方程
先定义函数符号变量。

```python
f = symbols('f', cls = Function)
```

然后定义微分方程，例如，$f''(x) - 2f'(x) + f(x) = \sin(x)$

```python
diffeq = Eq(f(x).diff(x, 2) - 2*f(x).diff(x) + f(x), sin(x))
```

然后用```dsolve()```求解。

```python
func_comparator(diffeq, dsolve, f(x))
```

如果无法得到显式的解， ```dsolve()```返回一个隐函数形式。

```python
diffeq = Eq(f(x).diff(x)*(1 - sin(f(x))), 0)

func_comparator(diffeq, dsolve, f(x))
```

# 参考资料
[Sympy Documentation](http://docs.sympy.org/latest/index.html)

# 相关文章
* [Sympy笔记I]({filename}0026_sympy_intro_1_ch.ipynb)
* [Sympy笔记II]({filename}0027_sympy_intro_2_ch.ipynb)
* [Sympy笔记III]({filename}0028_sympy_intro_3_ch.ipynb)
* [Sympy笔记IV]({filename}0029_sympy_intro_4_ch.ipynb)


# Lecture 12:  Fourier Series

## Reading and Reference

* Essential Mathematical Methods for Physicists, H. Weber and G. Arfken, Academic Press, 2003
* Advanced engineering Mathematics, E. Kreyszig, John wiley and Sons, 2010
* Numerical Recipes, W. Press, Cambridge University Press, 1986
* C. Hammond, The Basics of Crystallography and Diffraction, Oxford Science Publications, 4th ed.
* B. Gustafsson, Fundamentals of Scientific Computing, Springer, 2011
* S. Farlow, Partial Differential Equations for Scientists and Engineers, Dover, 1993

## What to Learn?

* A wave can be described by two parameters.
* More complex functions can be constructed from combinations of individual waves.
* Sines and cosines are basis vectors in Fourier space.
* A generalized inner product of functions can be used to compute Fourier coefficients.

## What to do?

* Compute Fourier coefficients using the inner product of functions.
* Learn how to shift the functions represented to arbitrary center points and domain widths.
* Demonstrate that Fourier basis vectors are orthogonal.
* Use a Fourier series to represent a sawtooth wave.

### Introduction
----

A summation of sines and cosines can be used to approximate periodic functions.  The Fourier Series is one such series and is given by:

$$
f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty} a_n cos(nx) + \sum_{m=1}^{\infty} b_m sin(mx)
$$

The coefficients are related to the original function by:

$$
a_n = \frac{1}{\pi} \int^{2\pi}_0 f(s) \cos ns \; ds, \;\; n = 0,1,2,...
$$

and

$$
b_m = \frac{1}{\pi} \int^{2\pi}_0 f(s) \sin ms \; ds, \;\; m = 1,2,...
$$

This series with coefficients determined by the integrals may be used in the solution of [ordinary differential](https://en.wikipedia.org/wiki/Ordinary_differential_equation) and [partial differential equations](https://en.wikipedia.org/wiki/Partial_differential_equation).  In materials engineering you will sometimes see diffusion problems use series solutions to describe the evolution of a concentration field where there is a factor composed of an infinite series and a factor containing an exponential in time.  Together they are selected to describe the diffusive evolution of a system.  A classic example of a [Fourier series](https://en.wikipedia.org/wiki/Fourier_series) in the solution to a diffusion problem is in Jackson and Hunt's paper on eutectic solidification.  In that paper the boundary condition was represented by a Fourier series to model the composition profile across eutectic lamellae.

### Representations of a Wave
---

A wave:

*  is represented by a frequency and amplitude.
*  is periodic on some domain, usually 0 to 2$\pi$ but can also be $-\pi$ to $\pi$ or anything else.
*  can be summed in combination with other waves to construct more complex functions.

```python
# Note the form of the import statements.  Keep the namespaces from colliding.
%matplotlib inline
import numpy as np
import sympy as sp

def plotSine(amplitude=2.4, frequency=np.pi/3.0, npoints=200):
    """
    Plots a sine function with a user specified amplitude and frequency.

    Parameters
    ----------
    amplitude : amplitude of the sine wave.

    frequency : the frequency of the sine wave.

    npoints : the number of points to use when plotting the wave.

    Returns
    -------
    A plot.

    """

    import matplotlib.pyplot as plt
    import numpy as np

    t = np.linspace(0, 2*np.pi, npoints)
    f = amplitude*np.sin(2*np.pi*frequency*t)
    fname = r"$A(t) = A \sin(2 \pi f t)$"

    fig, ax = plt.subplots()
    ax.plot(t, f, label=fname)
    ax.legend(loc='upper right')
    ax.set_xlabel(r'$t$', fontsize=18)
    ax.set_ylabel(r'$A$', fontsize=18)
    ax.set_title('A Sine Wave');

    plt.show()

    return
```

```python
plotSine()
```

All the properties of the wave are specified with these three pieces of information:

* It is a sine wave
* It has amplitude 2.4
* It has frequency $\pi$/3

In the previous plot we know that the frequency of $2\pi/3$ and coefficient (amplitue) of $2.4$ were linked through the `sin` function.  So it isn't hard to extrapolate to a situation where we might have MANY functions each with their own amplitude.  We could also imagine having many `sin` functions each with a different frequency - so let us make a list of amplitudes and frequencies (numerically) that we can use for plotting.  The following histogram plots the amplitudes for each frequency.

```python
def plotPower(amplitudes=[0,0,1.0,2.0,0,0,0], period=2.0*np.pi, npoints=200):
    """
    Plots a power series and the associated function assuming that the amplitudes
    provided are equally divided over the period of 2\pi unless specified.  Can also
    change the number of points to represent the function if necessary.
    """
    import matplotlib.pyplot as plt
    import numpy as np

    fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(12,5))
    fig.subplots_adjust(bottom=0.2)

    frequencies = np.linspace(0, period, len(amplitudes))
    t = np.linspace(0, period, npoints)

    # Reminder: zip([1,2,3],[4,5,6]) --> [(1,4),(2,5),(3,6)]
    f = sum([amplitude*np.sin(2*np.pi*frequency*t) for (amplitude, frequency) in zip(amplitudes, frequencies)])

    ax[0].bar(frequencies, amplitudes)
    ax[0].set_xlabel(r'$f$', fontsize=12)
    ax[0].set_ylabel(r'$A$', fontsize=12)
    ax[0].set_title(r'Power Spectrum')

    ax[1].plot(t, f)
    ax[1].set_xlabel(r'$f$', fontsize=12)
    ax[1].set_ylabel(r'$A$', fontsize=12)
    ax[1].set_title(r'Constructed Function')

    plt.show()

    return
```

```python
plotPower()
```

The plot above is one common way of visualizing the amplitudes of each term in a series.  Each bar represents the amplitude of a particular frequency in the reconstructed function.

### A Vector Space and Dot Products
----

A vector is an element of a _vector space_.  A vector space is the set of all vectors having dimension N.

We are introduced to the Euclidian vectors $\hat{i}$, $\hat{j}$, and $\hat{k}$ in physical problems and we gain a physical intuition for orthogonality.  We also learn a mechanism for computing the [dot product](https://en.wikipedia.org/wiki/Dot_product) in Euclidian systems, but other generalizations are possible.  One such generalization is the dot product of functions.

This dot product of functions can be used to determine Fourier coefficients.

```python
t = sp.symbols('t')
sp.init_printing()

def signal(x):
    return (x*(2 - x)*(1 - x)**2)
```

```python
sp.plot(signal(t), (t,0,2));
```

Is there a way to approximate the function above?  For real functions, the dot product can be generalized by the inner product, defined as:

$$ < f(x) | g(x) > = \int_{-L}^{L} f(x) g(x) dx $$

If this quantity is zero, then the functions are orthogonal. If the functions are orthogonal then they form a function space and can be used to approximate other functions.

The dot product for vectors v and w in Euclidian space has a geometric interpretation:

$$
\mathbf{v} \cdot \mathbf{w} = |v||w| \cos{\theta}
$$

This scalar quantity tells you how much of the vector v points along w, i.e., the magnitude of a vector pointing in the direction of $\hat{v}$ you need to add to some other (mutually orthogonal) vectors in order to reproduce w as a summation. When generalized to functions we write:

$$
< f(x) | g(x) > = \int_{-L}^{L} f(x) g(x) dx
$$

This computes how much of function $f(x)$ is projected onto $g(x)$.  Using a point $x = a$, compute $f(a)$ and $g(a)$. $f(a)$ and $g(a)$ represent the height of each function above/below the x-axis, so a vector from (a, 0) to (a, f(a)) can be dotted with a vector from (a, 0) to (a, g(a)). They are necessarily parallel along the space that contains the x-axis, so their dot product is just the product of their magnitudes:  $f(a)$ times $g(a)$. Now, multiply this by dx to keep the contribution from position $x=a$ proportional to how many additional x-positions you'll do this for. Take this dot product over and over, at each x-position, always scaling by $dx$ to keep it all in proportion. The sum of these dot products is the projection of $f(x)$ onto $g(x)$ (or vice-versa).

### Interactive Visualization of the Dot Product of Functions
----

```python
import matplotlib as mpl
import matplotlib.pyplot as plt
from ipywidgets import interact, fixed

# Somehow we want to add this text to the plot...
# dot_prod_value = sp.integrate(sp.sin(2*x)*sp.sin(x), (x, 0, 2*sp.pi))

def npf(x):
    return np.sin(2*x)

def npg(x):
    return np.sin(x)

def spf(x):
    return sp.sin(2*x)

def spg(x):
    return sp.sin(x)

# Make ff and gg tuples of np/sp functions? - or we can lambdafy the sp functions.
def myfig(ff,gg,a):
    """
    This function's docstring explaining the function.
    """
    x = np.linspace(0, 2*np.pi, 100)
    y1 = ff(x)
    y2 = gg(x)
    y3 = ff(x)*gg(x)
    fig = plt.figure(figsize=(8,5))
    axes = fig.add_axes([0.1, 0.1, 0.8, 0.8])
    axes.plot(x, y1, 'r', label=r"$f(x)$")
    axes.arrow(a, 0, 0, ff(a), length_includes_head=True, head_length=0.1, head_width=0.1, color='r')
    axes.plot(x, y2, 'g', label=r"$g(x)$")
    axes.arrow(a, 0, 0, gg(a), length_includes_head=True, head_length=0.1, head_width=0.1, color='g')
    axes.plot(x, y3, 'b', label=r"$f(x) \cdot g(x)$")
    axes.arrow(a, 0, 0, ff(a)*gg(a), length_includes_head=True, head_length=0.1, head_width=0.1, color='b')
    axes.legend()
    axes.grid(True)
    plt.show()
    return
```

```python
interact(myfig, ff=fixed(npf), gg=fixed(npg), a=(0,np.pi*2,0.05))
```

Using `scipy` we can perform this and other integrations numerically.  Two examples are given for the following functions:

```python
from scipy import integrate
import numpy as np

def myfunc1(x):
    return np.sin(4*x)

def myfunc2(x):
    return np.sin(x)

def myfunc3(x):
    return myfunc1(x)*myfunc2(x)

def myfuncx2(x):
    return x**2
```

```python
[integrate.quad(myfuncx2, 0, 4), 4.0**3/3.0]
```

```python
integrate.quad(myfunc3, 0, 2*np.pi)
```

```python
import sympy as sp
sp.init_printing()

n, m = sp.symbols('n m', Integer=True)
x = sp.symbols('x')

def f(x):
    return sp.sin(n*x)

def g(x):
    return sp.sin(m*x)

# scope of variables in def is local.
def func_dot(f, g, lb, ub):
    return sp.integrate(f(x)*g(x), (x, lb, ub))

func_dot(f, g, 0, 2*sp.pi)
```

### DIY:  Demonstrate the Inner Product of Certain Functions are Zero
----

Identify the conditions under which the inner product of:

$$
<\sin{4x}, \sin{x}>
$$

and

$$
<\sin{nx}, \sin{mx}>
$$

are zero.

### The Fourier Series Definied on Arbitrary Ranges

This discussion is derived from Sean Mauch's open source Applied Mathematics textbook.  If $f(x)$ is defined over $c-L \leq x \leq c+L $ and $f(x+2L) = f(x)$ then $f(x)$ can be written as:

$$
f(x) = \frac{a_0}{2} + \sum_{n=1}^{\infty} a_n cos \left( \frac{n \pi (x+c)}{L} \right) + \sum_{m=1}^{\infty} b_m sin \left( \frac{m \pi (x+c)}{L} \right)
$$

and the coefficients:

$$ a_n = \langle f(x) | \cos \left( \frac{n\pi (x+c)}{L} \right) \rangle = \frac{1}{L}\int^{c+L}_{c-L} f(x) \cos \frac{n \pi (x+c)}{L} $$

and

$$ b_m = \langle f(x) | \sin \left( \frac{m\pi (x+c)}{L} \right) \rangle = \frac{1}{L}\int^{c+L}_{c-L} f(x) \sin \frac{m \pi (x+c)}{L} $$

Using our generalized dot product for functions as defined above we can compute the Fourier coefficients.  The code for this follows in functions `a_n_amplitudes` and `b_m_amplitudes`.

### Computing the Fourier Coefficients by Hand
----

Note:  These next couple of cells take a few seconds to run.

```python
import sympy as sp
import numpy as np

x = sp.symbols('x')
dum = sp.symbols('dum')
sp.init_printing()

lam = 2
center = 1

def signal(x):
    return (x*(2 - x)*(1 - x)**2)

def mySpecialFunction(x):
    return sp.sin(2*x)

def b_m_amplitudes(n, funToProject, center, lam):
    return (2/lam)*sp.integrate(funToProject(dum)*sp.sin(2*n*sp.pi*dum/lam), (dum,center-lam/2,center+lam/2))

def a_n_amplitudes(m, funToProject, center, lam):
    return (2/lam)*sp.integrate(funToProject(dum)*sp.cos(2*m*sp.pi*dum/lam), (dum,center-lam/2,center+lam/2))

def b_m_vectorspace_element(n, var, lam):
    return sp.sin(2*n*sp.pi*var/lam)

def a_n_vectorspace_element(m, var, lam):
    if m==0:
        return sp.Rational(1,2)
    elif m!=0:
        return sp.cos(2*m*sp.pi*var/lam)
```

```python
terms = 3
funToProject = signal

an_vectors = [a_n_vectorspace_element(n, x, lam) for n in range(terms)]
an_amplitudes = [a_n_amplitudes(n, funToProject, center, lam) for n in range(terms)]
bm_vectors = [b_m_vectorspace_element(m, x, lam) for m in range(terms)]
bm_amplitudes = [b_m_amplitudes(m, funToProject, center, lam) for m in range(terms)]
```

We use a list comprehension to collect the basis vectors and amplitudes into a useful data structure through the `zip` function.

```python
truncatedSeries = (sum([a*b for a,b in zip(an_vectors,an_amplitudes)])
                   + sum([c*d for c,d in zip(bm_vectors,bm_amplitudes)]))
truncatedSeries
```

We can now plot this series and see the comparison of the signal (blue) and the series representation (red).  We can quantitatively describe the accuracy between the approximation and the function.

```python
p = sp.plot(signal(x), truncatedSeries, (x, 0, 2), show=False, title=r'Comparison of Series (Red) and Function (Blue)')
p[0].line_color = 'blue'
p[1].line_color = 'red'
p.show()
```

It is also possible to unpack the series above and look at the plot of each individual term's contribution to the approximate function.

```python
test = [c*d for c,d in zip(an_vectors,an_amplitudes)]

p2 = sp.plot(test[0],(x,0,2), show=False)

#[p2.append(sp.plot(test[i], (x,0,2), show=False)[0]) for i in range(1,5,1)]

[p2.append(sp.plot(i, (x,0,2), show=False)[0]) for i in test]

for i in range(1,terms,1):
    #p = sp.plot(test[i], (x,0,2), show=False)
    #p2.append(p[0])
    p2[i].line_color = 1.0-i/5.0,i/5.0,0.3

[p2.append(sp.plot(test[i], (x,0,2), show=False)[0])]

p2.show()
```

### Computing the Fourier Coefficients using Sympy
----

Here we use `sympy`'s `fourier_series` function to build a truncated series. We plot the series so that you can explore what happens when you change the number of terms.  The `interact` command creates a widget you can use to explore the effect of changing the nubmer of terms.

```python
import sympy as sp
import matplotlib.pyplot as plt
from ipywidgets import interact, fixed
```

```python
sp.fourier_series(x**2)
```

```python
sp.init_printing()

x = sp.symbols('x')

def myAwesomeFunction(a):
    return a

def fsMyFunc(terms, var):
    return sp.fourier_series(myAwesomeFunction(var), (var, -sp.pi, sp.pi)).truncate(n=terms)

def plotMyFunc(terms):
    p1 = sp.plot(fsMyFunc(terms,x),(x,-sp.pi, sp.pi), show=False, line_color='r')
    p2 = sp.plot(myAwesomeFunction(x), (x,-sp.pi,sp.pi), show=False, line_color='b')
    p2.append(p1[0])
    p2.show()
    return None

plt.rcParams['lines.linewidth'] = 3
plt.rcParams['figure.figsize'] = 8, 6
```

```python
interact(plotMyFunc, terms=(1,10,1));
```

### DIY:  Series for a Sawtooth Wave
----

Using a Fourier series, represent the following periodic function:

$$f(x) = \left\{
\begin{array}{ll}
 x, & 0 \leq x \leq \pi, \\
 x-2\pi, & \pi \leq x \leq 2\pi,
\end{array}
\right.$$

### DIY:  Compute Your Own
----

Compute a Fourier series for a function of your choosing.  Think about the restrictions on the use of the Fourier series.


# Linear Error Propagation

#### MMU Installation

`mmu` can be installed using:
```bash
pip install git+https://github.com/RUrlus/ModelMetricUncertainty.git
```

```python
%load_ext autoreload
%autoreload 2
```

```python
import arviz as az
import numpy as np
import pandas as pd
import seaborn as sns

import mmu
import mmur
from mmur.viz import _set_plot_style, plot_logstic_dgp, plot_probas, plot_ci_violin
```

```python
%matplotlib inline
COLORS = _set_plot_style()
```

```python
def plot_metric_distributions(estimated_metrics, gt_metrics, coverage=None, label_alt='simulated'):
    fig, axs = plt.subplots(ncols=5, figsize=(25, 5))
    for i, c in enumerate(estimated_metrics.columns):
        sns.kdeplot(estimated_metrics[c], ax=axs[i], label='estimated')
        if coverage is not None:
            sns.kdeplot(coverage[c], ax=axs[i], label=label_alt)
        axs[i].axvline(gt_metrics[c][0], c='grey', lw=2, ls='--', label='population mean')
    axs[0].legend()
    return fig, axs
```

### Logistic process

Let $X \subset \mathbb{R}^{Nx2}$
where:

$$\begin{align}
X_{i, 1} &= 1.~\forall~i \in N\\
X_{i, 2} &\sim \mathrm{Uniform}(-10, 10)~\forall~i \in N\\
L &= \beta_{1}X_{1} + \beta_{2}X_{2}\\
P &= \mathrm{sigmoid}(L)\\
L_{\mathrm{noisy}} &= L + \mathrm{Normal}(0, \sigma)\\
P_{\mathrm{noisy}} &= \mathrm{sigmoid}(L_{\mathrm{noisy}})\\
y &\sim \mathrm{Bernoulli}(P_{\mathrm{noisy}})\\
\end{align}$$

```python
fig, ax = plot_logstic_dgp()
```

## Logistic Model generator

We simulate a scenario where you have trained a Logistic Regression model and we want to see how well the uncertainty of the metrics is modelled

1. Generate train, test and holdout samples from logistic process
2. Fit Logistic regression on train set
3. Using fitted model predict probabilities on:
    a. test set
    b. all hold out sets
4. Compute confusion matrix test
5. Model uncertainty on the metrics based on the test set
6. Compare against distribution of metrics on the hold out sets

### Generate data

```python
generator = mmur.LogisticGenerator()
outp = generator.fit_transform(
    train_samples=10000,
    test_samples=10000,
    holdout_samples=10000,
    n_sets=10000,
    noise_sigma=0.3,
    random_state=123456
)

# Select the test sets
y_test = outp['test']['y']
probas_test = outp['test']['proba']
```

We only consider the below metrics for now

```python
target_metrics = [
    'pos.precision', 'pos.recall'
]
```

#### Test set performance

Compute the confusion matrix and metrics on the test-set

```python
test_conf_mat, test_metrics = mmu.binary_metrics_proba(
    y_test, probas_test, threshold=0.5
)
test_conf_mat = test_conf_mat.flatten()
test_metrics = mmu.metrics_to_dataframe(test_metrics)[target_metrics]

mmu.confusion_matrix_to_dataframe(test_conf_mat)
```

```python
test_metrics
```

```python
gt_proba_test = outp['ground_truth']['test']
```

We know the ground truth probability

Compute the ground truth confusion matrix and metrics

```python
gt_conf_mat, gt_metrics = mmu.binary_metrics_proba(
    y_test, gt_proba_test, threshold=0.5
)
mmu.confusion_matrix_to_dataframe(gt_conf_mat)
```

```python
gt_metrics = mmu.metrics_to_dataframe(gt_metrics)[target_metrics]
gt_metrics
```

### Hold-out set

We compare a sample from the holdout set to the ground truth probability

```python
y_holdout = outp['holdout']['y']
proba_holdout = outp['holdout']['proba']
```

```python
fig, ax = plot_probas(proba_holdout, gt_proba_test)
```

Compute metrics on this set

```python
holdout_conf_mat, holdout_metrics = mmu.binary_metrics_runs(y=y_holdout, proba=proba_holdout, threshold=0.5)
holdout_metrics = mmu.metrics_to_dataframe(holdout_metrics)[target_metrics]
```

At this point you can compare the observed metrics in `holdout_metrics` with the estimation coming from the method.

Univariate uncertainties can be validated using the below cell as an example

```python
from mmur.models import pr_uni_err_prop
```

```python
lep_ci = pd.DataFrame(
    np.vstack(pr_uni_err_prop(test_conf_mat)),
    columns=['mu', 'sigma', 'lb', 'ub'],
    index=['pos.precision', 'pos.recall']
)
```

```python
fig, ax = plot_ci_violin(lep_ci, holdout_metrics)
```


# 2 DOF System

The dynamical system in figure is composed of two massless, uniform beams
supporting a dimensionless body of mass $m$.

With $\omega_0^2=(EJ)/(mL^3)$ determine the system's eigenvalues
$\omega_i^2=\lambda_i^2\,\omega_0^2$ and the mass normalized eigenvectors.

The system is subjected to a horizontal motion $u_\mathcal{B}= u_\mathcal{B}(t)$,
determine the mass displacements
$\boldsymbol x_\mathcal{B} = \boldsymbol r\,u_\mathcal{B}$ and write the modal equations of dynamic equilibrium as
$$\ddot q_i + \lambda_i^2\omega_0^2\,q_i = \alpha_i\,\ddot u_\mathcal{B}$$
determining the numerical values of the $\alpha_i$.
## Structural matrices

The flexibility is computed using the Principle of Virtual Displacements, the stiffness is computed by inversion and the mass matrix is the unit matrix multiplied by $m$, $\ \boldsymbol M=m\,\boldsymbol I$.

```python
l = [1, 1, 1, 2, 1]
m = [[p( 2/3, 0), p(-1/3, 1), p(1, 0), p(2/3, 0), p(4/3, 0)],
     [p(-2/3, 0), p(-2/3, 0), p(0, 0), p(1/3, 0), p(2/3, 0)]]

F = array([[vw(emme, chi, l) for emme in m] for chi in m])
K = inv(F)
M = eye(2)

dl(dmat(r'\boldsymbol{F}=\frac{1}{27}\frac{L^3}{EJ}', F*27, r','))
dl(dmat(r'\boldsymbol{K}=\frac{1}{53}\frac{EJ}{L^3}', K*53, r','))
dl(dmat(r'\boldsymbol{M}=m', M,'.', fmt='%d'))
```

\begin{align}\boldsymbol{F}=\frac{1}{27}\frac{L^3}{EJ}\begin{bmatrix}+80&+13\\+13&+20\end{bmatrix},\end{align}

\begin{align}\boldsymbol{K}=\frac{1}{53}\frac{EJ}{L^3}\begin{bmatrix}+20&-13\\-13&+80\end{bmatrix},\end{align}

\begin{align}\boldsymbol{M}=m\begin{bmatrix}1&0\\0&1\end{bmatrix}.\end{align}

## The eigenvalues problem

```python
wn2, Psi = eigh(K, M) ; Psi[:,0] *= -1
Lambda2 = diag(wn2)

dl(dmat(r'\boldsymbol{\Lambda^2}=', Lambda2, r'.'))
dl(dmat(r'\boldsymbol{\Psi}=', Psi,  r'.'))
```

\begin{align}\boldsymbol{\Lambda^2}=\begin{bmatrix}+0.326499&+0\\+0&+1.56029\end{bmatrix}.\end{align}

\begin{align}\boldsymbol{\Psi}=\begin{bmatrix}+0.979172&-0.203032\\+0.203032&+0.979172\end{bmatrix}.\end{align}

## Mass Displacements and Inertial Forces

We downgrade the  hinge in $\mathcal B$ to permit a unit horizontal displacement and observe that the centre of instantaneous rotation for the lower beam is at the intersection of the vertical in $\mathcal B$ and the line connecting $\mathcal A$ and $\mathcal C$.

The angle of rotation of the lower beam is $\theta_2=1/3L$, anti-clockwise and the angle of rotation of the upper beam (that rotates about the hinge in $\mathcal A$) is equal but clockwise, due to the continuity of displacements in the internal hinge $\mathcal C$.

Knowing the angle of rotation of the upper beam, the mass displacements are
$$ x_1 = 2L\frac{1}{3L}=\frac{2}{3}\quad
\text{ and }\quad x_2=L\frac{1}{3L}=\frac{1}{3}.$$

We can eventually write
$$ \boldsymbol x_\text{tot} = \boldsymbol x + \begin{Bmatrix}
   2/3\\1/3 \end{Bmatrix}\,u_\mathcal{B}$$

and the inertial force can be written as

$$\boldsymbol f_\text{I} =
\boldsymbol M \, \ddot{\boldsymbol x}_\text{tot} =
\boldsymbol M \, \ddot{\boldsymbol x} +
\boldsymbol M \, \begin{Bmatrix}2/3\\1/3\end{Bmatrix}\,\ddot u_\mathcal{B}
$$

## The Modal Equations of Motion

The equation of motion, in structural coordinates, is

$$
\boldsymbol M \, \ddot{\boldsymbol x} +
\boldsymbol K \, \boldsymbol x =
- \boldsymbol M \, \begin{Bmatrix}2/3\\1/3\end{Bmatrix}\,\ddot u_\mathcal{B}
$$

or, because $\boldsymbol M = m\, \boldsymbol I$,

$$
\boldsymbol M \, \ddot{\boldsymbol x} +
\boldsymbol K \, \boldsymbol x =
- m\, \begin{Bmatrix}2/3\\1/3\end{Bmatrix}\,\ddot u_\mathcal{B}
$$

Using the modal expansion, $\boldsymbol x = \boldsymbol\Psi\, \boldsymbol q$ and premultiplying  term by term by $\boldsymbol\Psi^T$ we have, because the eigenvectors are normalized w/r to the mass matrix,

$$
m\,\boldsymbol I \, \ddot{\boldsymbol q} + m\omega_0^2\,
\boldsymbol \Lambda^2 \, \boldsymbol q =
- m\,\boldsymbol\Psi^T\, \begin{Bmatrix}2/3\\1/3\end{Bmatrix}\,\ddot u_\mathcal{B}.
$$

```python
r = array((2/3, 1/3))
a = -Psi.T@r
print('The modal equations of motion are')

for i, (ai, l2i) in enumerate(zip(a, wn2), 1):
    dl(r'$$\ddot q_{%d} %+.6f\,\omega_0^2\,q_{%d} = %+.6f\,\ddot u_\mathcal{B}$$' %
        (i, l2i, i, ai))
```

    The modal equations of motion are

$$\ddot q_{1} +0.326499\,\omega_0^2\,q_{1} = -0.720459\,\ddot u_\mathcal{B}$$

$$\ddot q_{2} +1.560294\,\omega_0^2\,q_{2} = -0.191036\,\ddot u_\mathcal{B}$$

## Initialization

```python
from numpy import array, diag, eye, poly1d
from scipy.linalg import eigh, inv

from IPython.display import Latex, HTML
display(HTML(open('01.css').read()))

def p(*l): return poly1d(l)

def vw(M, Χ, L):
    return sum(p(l)-p(0) for (m, χ, l) in zip(M, Χ, L) for p in ((m*χ).integ(),))

def dmat(pre, mat, post, mattype='b', fmt='%+.6g'):
    s = r'\begin{align}' + pre + r'\begin{%smatrix}'%mattype
    s += r'\\'.join('&'.join(fmt%val for val in row) for row in mat)
    s += r'\end{%smatrix}'%mattype + post + r'\end{align}'
    return s

def dl(ls):
    display(Latex(ls))
    return None
```

<style>

@font-face {
    font-family: 'Charis SIL';
    src: url('fonts/CharisSILEur-R.woff') format('woff');
}
@font-face {
    font-family: 'Architects Daughter';
    font-style: normal;
    src: url(https://fonts.gstatic.com/s/architectsdaughter/v6/RXTgOOQ9AAtaVOHxx0IUBM3t7GjCYufj5TXV5VnA2p8.woff2) format('woff2');
    unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000;
}
.prompt{display: None;}
div.cell{margin: auto;width:720px;}

code{font-family: Consolas}

div #notebook { /* centre the content */
    background: #fdfffa;
    margin: auto;
    padding: 0em;
    padding-top: 1em;
}
div #notebook_container {width: 700px!important;}
#notebook li { /* More space between bullet points */
    margin-top:0.2em;
}

div.text_cell_render{
    font-family: "Charis SIL", Cambria,  serif;
    line-height: 150%;
    font-size: 140%;
}
.CodeMirror{
    width: 700px!important;
    font-family: Consolas,monospace;
    font-size: 100%;
    background-color:#fefffc!important;
}

.output_area {
    font-family: Consolas,monospace;font-size: 100%;
    background-color:#ffffff!important;
    margin-top:0.8em;
}

div.output_latex{overflow:hidden}

h1 {
    font-family: 'Architects Daughter', serif;
    font-size:   32pt!important;
    text-align:  center;
    text-shadow: 4px 4px 4px #aaa;
    padding-bottom: 32pt;
}

.warning{color: rgb( 240, 20, 20 )}
</style>


```python
# Initial Setups and Imports

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import sympy as sp
import scipy.signal as signal
import scipy.optimize as optimise

pd.plotting.register_matplotlib_converters()

plt.style.use('seaborn')
plt.rcParams['figure.dpi'] = 100
plt.rcParams['font.family'] = "Ubuntu Mono"
```

```python
dataset = pd.read_csv('dataset.csv')

time = dataset['time']
velocity = dataset['velocity']

w = np.linspace(0.1, 10, 1000)
```

```python
periodogram = signal.lombscargle(time, velocity, w, normalize=True, precenter=True)

wdom = w[np.argmax(periodogram)]

timeperiod =  2 * np.pi / wdom
```

```python
plt.subplot(211)

plt.plot(w, periodogram, c='r')
plt.xlabel("Angular Frequency")
plt.ylabel("LombScargle Power")
plt.title("LombScargle Periodogram")
plt.text(6,0.85,'Maxima = {:.5f}'.format(wdom), bbox = dict(facecolor = 'gold', alpha = 0.5))

plt.subplot(212)

plt.plot(2 * np.pi / w, periodogram, color='g')
plt.xlim(0, 10)
plt.xlabel("Period")
plt.ylabel("LombScargle Power")
plt.text(6,0.85,'Maxima = {:.5f}'.format(timeperiod),bbox = dict(facecolor = 'gold', alpha = 0.5))

plt.tight_layout()
```

```python
def folding(array,timeperiod):
    """Returns the folded time array"""

    tarray = []

    for x in array:
        if x < timeperiod:
            tarray.append(x)
        else:
            while x > timeperiod:
                x -= timeperiod
            tarray.append(x)

    return tarray
```

```python
time = [x - time.values[0] for x in time.values]
tfolded = folding(time, timeperiod)
```

```python
sns.scatterplot(x = tfolded,y = velocity,marker = "X",color = 'r')

plt.title('Velocity // Folded-Time')
plt.xlabel('Time (Folded)')
plt.ylabel('Velocity')

plt.xlim(0,timeperiod)
plt.ylim(-80,80)
```

## Equation

```python
# Data

j = 1.396
x = -10
p = timeperiod * 86400
a = 3131006.710264267
e = 0.013

# Sine - Wave

th      = np.linspace(0, 4 * np.pi, 1000)
semiamp = 2 * np.pi * a * np.sin(j) / (p * np.sqrt(1 - e ** 2))
v       = semiamp * (np.cos(1*(th + x)) + e * np.cos(x))
```

```python
# Plotting the equation
plt.plot(th, v, c='green')

plt.xlabel('Theta')
plt.ylabel('Velocity (m/s)')

plt.xlim(0,timeperiod)

plt.tight_layout()
```

```python
# Superimposing

sns.scatterplot(x = tfolded,y = velocity, marker = 'X')

plt.plot(th, v, c = 'g')
plt.xlim(0, timeperiod)
```

## Least Square Fit

```python
# defining the function

def f(x,a,b,c):
    return a * np.sin (x + c) + b
```

```python
xdata = tfolded
ydata = velocity
```

```python
predict, pcov = optimise.curve_fit(f,xdata,ydata)
```

```python
plt.plot(xdata, f(xdata, *predict),'X', label = "Least Square Fit")

sns.scatterplot(x = xdata, y = velocity, marker = "X", color = 'r', label = "Folded Scatter Plot")

plt.plot(th, v, '--', label = 'Fit using the equation')

plt.xlim(0, timeperiod)

plt.xlabel('Time')
plt.ylabel('Velocity')

plt.legend();
```

$$
V = \frac{2\pi a * \sin{i}}{P \sqrt{1-e^2}} [Cos(\theta+\omega) + e Cos(\omega)]
$$

```python
i = 1
P = timeperiod * 86400
thdata = np.linspace(0, 4 * np.pi, 1000)
```

```python
def vel(th,e, w, a):
    return ((2 * np.pi * a * np.sin(i))/(P * np.sqrt(1 - e**2))) * (np.cos(th + w) + e*np.cos(w))
```

```python
e0 = 0.88
w0 = 2.23234
a0 = 175798129.9
popt, pcov = optimise.curve_fit(vel,thdata, v,p0=(e0,w0,a0))
```

```python
f"e = {popt[0]}  w = {popt[1]} a = {popt[2]}"
```

    'e = 0.01299999999999999  w = 2.566370614359173 a = 3664174.311343284'

```python
e = popt[0]
w = popt[1]
a = popt[2]
(2 * np.pi * a * np.sin(i)) / (P * np.sqrt(1 - e**2))
```

    53.083839093067915

```python
th_,e_,w_,a_,i_,P_ = sp.symbols(r'\theta e w a i P')
((2 * sp.pi * a_ * sp.sin(i_))/(P_ * sp.sqrt(1 - e_**2))) * (sp.cos(th_ + w_) + e_*sp.cos(w_))
```

$\displaystyle \frac{2 \pi a \left(e \cos{\left(w \right)} + \cos{\left(\theta + w \right)}\right) \sin{\left(i \right)}}{P \sqrt{1 - e^{2}}}$

```python
def vv(th):
    return ((2 * np.pi * a * np.sin(i))/(P * np.sqrt(1 - e**2))) * (np.cos(th + w) + e*np.cos(w))
```

```python
th = 1/wdom * np.linspace(0,4*np.pi,1000)
plt.plot(th,vv(th),'r--')
plt.scatter(xdata,velocity)
plt.xlabel(r'$\theta$')
plt.ylabel(r'$V_r$')
plt.xlim(0,timeperiod)
```

```python
np.sqrt(np.diag(pcov))
```

    array([1.29393123e-17, 1.53542699e-17, 5.62513983e-11])

```python
df = dataset
```

```python
df.iloc[3].values
```

    array([  3.62695, -19.85   ,   2.9    ])

```python

```


# Non Linear Regression Analysis

## Objectives

After completing this lab you will be able to:

*   Differentiate between linear and non-linear regression
*   Use non-linear regression model in Python

If the data shows a curvy trend, then linear regression will not produce very accurate results when compared to a non-linear regression since linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example in python. In this notebook, we fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014.

Importing required libraries

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

Although linear regression can do a great job at modeling some datasets, it cannot be used for all datasets. First recall how linear regression, models a dataset. It models the linear relationship between a dependent variable y and the independent variables x. It has a simple equation, of degree 1, for example y = $2x$ + 3.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
#plt.figure(figsize=(8,6))
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

Non-linear regression is a method to model the non-linear relationship between the independent variables $x$ and the dependent variable $y$. Essentially any relationship that is not linear can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).  For example:

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like exponentials, logarithms, fractions, and so on. For example: $$ y = \log(x)$$

We can have a function that's even more complicated such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic

$$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

### Exponential

An exponential function with base c is defined by $$ Y = a + b c^X$$ where b ≠0, c > 0 , c ≠ 1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

Y= np.exp(X)
y_noise = 5*np.random.normal(size=x.size)
y_data = y+y_noise
plt.plot(x, y_data,'bo')
plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

### Logarithmic

The response $y$ is a results of applying the logarithmic map from the input $x$ to the output $y$. It is one of the simplest form of **log()**: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be a polynomial representation of the $x$ values. In general form it would be written as\
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

### Sigmoidal/Logistic

$$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = 1-4/(1+np.power(3, X-2))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

# Non-Linear Regression example

For an example, we're going to try and fit a non-linear model to the datapoints corresponding to China's GDP from 1960 to 2014. We download a dataset with two columns, the first, a year between 1960 and 2014, the second, China's corresponding annual gross domestic income in US dollars for that year.

```python
import numpy as np
import pandas as pd

df = pd.read_csv("china_gdp.csv")
df.head(10)
```

      Year
      Value

      0
      1960
      5.918412e+10

      1
      1961
      4.955705e+10

      2
      1962
      4.668518e+10

      3
      1963
      5.009730e+10

      4
      1964
      5.906225e+10

      5
      1965
      6.970915e+10

      6
      1966
      7.587943e+10

      7
      1967
      7.205703e+10

      8
      1968
      6.999350e+10

      9
      1969
      7.871882e+10

```python

```

### Plotting the Dataset

This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it decelerate slightly in the 2010s.

```python
plt.figure(figsize=(8,5))
x_data, y_data = (df["Year"].values, df["Value"].values)
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

### Choosing a model

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Independent Variable')
plt.show()
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta\_1(X-\beta\_2)}}$$

$\beta\_1$: Controls the curve's steepness,

$\beta\_2$: Slides the curve on the x-axis.

### Building The Model

Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data
xdata =x_data/max(x_data)
ydata =y_data/max(y_data)
```

#### How we find the best parameters for our fit line?

we can use **curve_fit** which uses non-linear least squares to fit our sigmoid function, to data. Optimal values for the parameters so that the sum of the squared residuals of sigmoid(xdata, \*popt) - ydata is minimized.

popt are our optimized parameters.

```python
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
```

     beta_1 = 690.451711, beta_2 = 0.997207

Now we plot our resulting regression model.

```python
x = np.linspace(1960, 2015, 55)
x = x/max(x)
plt.figure(figsize=(8,5))
y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')
plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

## Practice

Can you calculate what is the accuracy of our model?

```python
msk = np.random.rand(len(df)) < 0.8
train_x = xdata[msk]
test_x = xdata[~msk]
train_y = ydata[msk]
test_y = ydata[~msk]

# build the model using train set
popt, pcov = curve_fit(sigmoid, train_x, train_y)

# predict using test set
y_hat = sigmoid(test_x, *popt)

# evaluation
print("Mean absolute error: %.2f" % np.mean(np.absolute(y_hat - test_y)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_hat - test_y) ** 2))
from sklearn.metrics import r2_score
print("R2-score: %.2f" % r2_score(y_hat , test_y) )

```

    Mean absolute error: 0.03
    Residual sum of squares (MSE): 0.00
    R2-score: 0.95

Click here for the solution

```python
# split data into train/test
msk = np.random.rand(len(df)) < 0.8
train_x = xdata[msk]
test_x = xdata[~msk]
train_y = ydata[msk]
test_y = ydata[~msk]

# build the model using train set
popt, pcov = curve_fit(sigmoid, train_x, train_y)

# predict using test set
y_hat = sigmoid(test_x, *popt)

# evaluation
print("Mean absolute error: %.2f" % np.mean(np.absolute(y_hat - test_y)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_hat - test_y) ** 2))
from sklearn.metrics import r2_score
print("R2-score: %.2f" % r2_score(y_hat , test_y) )

```


# General

```python
import os
import sys
module_path = os.path.abspath(os.path.join('..\\..'))
if module_path not in sys.path:
    sys.path.append(module_path)
```

```python
import civilpy
```

### Units in CivilPy are handled by the "pint" package, more examples of it's use can be found in the [documentation](https://pint.readthedocs.io/en/stable/).

### Kinetic Energy

```python
v = 27000 * (civilpy.unit.ft / civilpy.unit.sec)
m = 4000 * civilpy.unit.lb
kinetic = civilpy.general.kinetic_energy(m, v)
kinetic.to_compact()

print(f'{round(kinetic, 3):.2E}')
```

    4.53E+10 foot * pound

### Potential Energy

```python
m = 2 * civilpy.unit.kg
z = 0.4 * civilpy.unit.m
potential = civilpy.general.potential_energy(m, z, system='metric')
potential.ito(civilpy.unit.joule)

print(f'{round(potential, 3)}')
```

    7.848 joule

### Weight/Force Calculation

```python
m = 10 * civilpy.unit.kg
weight = civilpy.general.mass_to_weight(m, system='metric')
weight.ito(civilpy.unit.newton)

print(f'{round(weight, 1)}')
```

    98.1 newton

### Note: civilpy.general.math.solver is just the "sympy" package, more examples of it's use can be found in the [documentation](https://www.sympy.org/en/index.html).

### Quadratic Equation

```python
x = civilpy.general.solver.symbols('x')
eq = 2*x**2 - 7*x + 3
solution = civilpy.general.equation_solver(eq, 'x')
solution
```

    [1/2, 3]

### Polynomial Equations

```python
eq = 2*x**4 - 2*x**3 - 14*x**2 + 2*x + 12

solution = civilpy.general.equation_solver(eq, 'x')
solution
```

    [-2, -1, 1, 3]

### Exponential Equations

```python
t = civilpy.general.solver.symbols('t')

eq = civilpy.general.solver.exp(-0.005*t) - 0.07
solution = civilpy.general.equation_solver(eq, 't')[0]
print(f'{solution:.0f}')
```

    532

### Partial Fraction Decomposition

```python
f = (x**2 + 2*x + 3)/(x**4 + x**3 + 2*x**2)

temp = civilpy.general.solver.apart(f)
temp = civilpy.general.solver.together(temp)
temp = civilpy.general.solver.numer(f) / civilpy.general.solver.expand(civilpy.general.solver.denom(f))
temp = civilpy.general.solver.factor(temp)

temp
```

$\displaystyle \frac{x^{2} + 2 x + 3}{x^{2} \left(x^{2} + x + 2\right)}$

```python
civilpy.general.solver.var('A:D')

p1 = A/x
p2 = B/x**2
p3 = (C + D*x)/(x**2 + x + 2)

eqs = p1, p2, p3

h = sum(eqs)

rational_function = civilpy.general.solver.together(h)

temp = civilpy.general.solver.factor(rational_function, x)

civilpy.general.solver.Eq(temp, f)
```

$\displaystyle \frac{2 B + x^{3} \left(A + D\right) + x^{2} \left(A + B + C\right) + x \left(2 A + B\right)}{x^{2} \left(x^{2} + x + 2\right)} = \frac{x^{2} + 2 x + 3}{x^{4} + x^{3} + 2 x^{2}}$

```python
eq = civilpy.general.solver.Eq(civilpy.general.solver.numer(_.lhs), civilpy.general.solver.numer(_.rhs))
solution = civilpy.general.solver.solve_undetermined_coeffs(eq, [A, B, C, D], x)

solution
```

    {A: 1/4, B: 3/2, C: -3/4, D: -1/4}

### Simultaneous Linear Equations

```python
y = civilpy.general.solver.symbols('y')
p1 = 2*x + 3*y - 12
p2 = 3*x + 4*y - 8

eqs = p1, p2

solution = civilpy.general.solver.solve(eqs, [x, y])

solution
```

    {x: -24, y: 20}

### Limits

$ \lim_{x \to 3} \frac{x^3 - 27}{x^2 - 9} $

```python
eq = (x**3 - 27) / (x**2 - 9)
limit_expr = civilpy.general.solver.limit(eq, x, 3)
print(limit_expr)
```

    9/2

$ \lim_{x \to \infty} \frac{3x - 2}{4x + 3} $

```python
eq = (3*x - 2) / (4*x + 3)
inf = civilpy.general.inf

limit_expr = civilpy.general.solver.limit(eq, x, inf)
print(limit_expr)
```

    3/4

$ \lim_{x \to 2} \frac{x^2 + x - 6}{x^2 - 3x + 2} $

```python
eq = (x**2 + x - 6) / (x**2 - 3*x + 2)
limit_expr = civilpy.general.solver.limit(eq, x, 2)
print(limit_expr)
```

    5

### Sequence and Progressions

$ \lim_{n \to \infty} \frac{e^n}{n} $

```python
n = civilpy.general.solver.symbols('n')
eq = (civilpy.general.solver.exp(n)) / (n)
limit_expr = civilpy.general.solver.limit(eq, n, inf)
print(limit_expr)
```

    oo

```python
# Apply L'Hopital's Rule
eq = (civilpy.general.solver.exp(n)) / (1)
limit_expr = civilpy.general.solver.limit(eq, n, inf)
print(limit_expr)
```

    oo

### Arithmetic Growth Rate

```python
p_0 = 1000
R = 10
T = 5

solution = civilpy.general.arithmetic_growth(p_0, R, T)
solution
```

    1050

### Geometric Growth Rate

```python
p_0 = 245000
R = 0.03
T = 6

solution = civilpy.general.geometric_growth(p_0, R, T)
solution
```

    292542.81264960504

### Matrix Math (array is just numpy.array)

```python
x = civilpy.general.np.array([3,2])
y = civilpy.general.np.array([5,1])
z = x + y
z
```

    array([8, 3])

### Determinate of a Matrix

```python
np = civilpy.general.np
A = np.array([[2, 3, -4], [3, -1, -2], [4, -7, -6]])
print(round(np.linalg.det(A),0))
```

    82.0

### Matrix Multiplication

```python
A = np.array([[1, 4, 3], [5, 2, 6]])
B = np.array([[7, 12], [11, 8], [9, 10]])

C = np.matmul(A, B)
C
```

    array([[ 78,  74],
           [111, 136]])

### Transposing Matrix

```python
A = np.array([[1, 6, 9], [2, 3, 4], [7, 1, 5]])
A_trans = np.matrix.transpose(A)
A_trans
```

    array([[1, 2, 7],
           [6, 3, 1],
           [9, 4, 5]])

### Matrix Rank

```python
A = np.array([[7, 4, 9, 1], [0, 2, -5, 3], [0, 4, -10, 6]])
rank = np.linalg.matrix_rank(A)
rank
```

    2

### Classical Adjoint

```python
A = np.array([[2, 3, -4], [0, -4, 2], [1, -1, 5]])

adjoint = np.matrix.transpose(np.linalg.inv(A).T * np.linalg.det(A))
adjoint
```

    array([[-18., -11., -10.],
           [  2.,  14.,  -4.],
           [  4.,   5.,  -8.]])

### Matrix Inverse

```python
A = np.array([[4, 5], [2, 3]])

A_inverse = np.linalg.inv(A)
A_inverse
```

    array([[ 1.5, -2.5],
           [-1. ,  2. ]])

```python
identity = np.matmul(A, A_inverse)
identity
```

    array([[1., 0.],
           [0., 1.]])

### Simultaneous Linear Equations in Matrix Form

$ a_{11}x_1 + a_{12}x_2 = b_1 $

$ a_{21}x_1 + a_{22}x_2 = b_2 $

```python
A = np.array([['a11', 'a12'],['a21', 'a22']])
X = np.array([['x1'], ['x2']])
B = np.array([['b1'], ['b2']])

print(A)
print(X)
print(B)
```

    [['a11' 'a12']
     ['a21' 'a22']]
    [['x1']
     ['x2']]
    [['b1']
     ['b2']]

### Solving Linear Equations

$ 2x + 3y - 4z = 1 $

$ 3x - y - 2z = 4 $

$ 4x - 7y - 6z = -7 $

```python
A = np.array([[2, 3, -4], [3, -1, -2], [4, -7, -6]])
B = np.array([[1], [4], [-7]])

X = np.linalg.solve(A, B)
print(X)
```

    [[3.]
     [1.]
     [2.]]

### Eigenvalues and Eigenvectors

```python
A = civilpy.general.solver.Matrix([[2, 4], [6, 4]])

eigenvalues = A.eigenvals()
eigenvectors = A.eigenvects(simplify=True)

print('Eigenvalues:')
for count, key in enumerate(eigenvalues):
    print(f'{count+1}: {key}')

print('\nEigenvectors:')
for value in eigenvectors:
    print(f'\n {value[2][0]}')
```

    Eigenvalues:
    1: 8
    2: -2

    Eigenvectors:

     Matrix([[-1], [1]])

     Matrix([[2], [3]])

### Vectors

Angle between 2 vectors

```python
v_1 = [-np.sqrt(3), 1]
v_2 = [2*np.sqrt(3), 2]

unit_vect_1 = v_1 / np.linalg.norm(v_1)
unit_vect_2 = v_2 / np.linalg.norm(v_2)
dot_product = np.dot(unit_vect_1, unit_vect_2)
angle = np.arccos(dot_product)

print(round(np.degrees(angle), 0))
```

    120.0

Find vector orthogonal to two vectors

```python
v_1 = np.array([1, -1, 2])
v_2 = np.array([0, 3, -1])

solution = np.cross(v_1, v_2)
vector_length = np.linalg.norm(solution)
unit_vector = solution / vector_length

unit_vector
```

    array([-0.84515425,  0.16903085,  0.50709255])

### Analytic Geometry

```python
unit = civilpy.unit
d = 2 * unit.inch
dia = 2 * unit.inch
radius = civilpy.water_resources.hydraulic_radius(depth=d, dia=dia)
print(round(radius, 2))
```

    0.5 inch

```python
radius = civilpy.water_resources.hydraulic_radius(depth=1.8 * unit.meter, b1=6.1 * unit.meter, shape='rectangle')
round(radius, 2)
```

1.13 meter

```python
radius = civilpy.water_resources.hydraulic_radius(depth=1.8 * unit.meter, slope=2, b1=6.1 * unit.meter, shape='trapezoid')
round(radius, 2)
```

1.23 meter

```python
radius = civilpy.water_resources.hydraulic_radius(depth=1.8 * unit.meter, slope=0.577, shape='triangle')
round(radius, 2)
```

0.45 meter

### Plotting an implicit equation

```python
import matplotlib.pyplot
from numpy import arange
from numpy import meshgrid

delta = 0.025
xrange = arange(-5.0, 1, delta)
yrange = arange(-5.0, 9.0, delta)
X, Y = meshgrid(xrange,yrange)

# F is one side of the equation, G is the other
F = 4*Y**2 - 12*Y + 16*X + 41
G = 0

plt = matplotlib.pyplot.contour(X, Y, (F - G), [0])
plt.axes.set_aspect('equal')
matplotlib.pyplot.show()
```

```python
delta = 0.025
xrange = arange(-5.0, 5, delta)
yrange = arange(-5.0, 5.0, delta)
X, Y = meshgrid(xrange,yrange)

# F is one side of the equation, G is the other
F = X**2 - 10*X*Y + Y**2 + X + Y + 1
G = 0

plt = matplotlib.pyplot.contour(X, Y, (F - G), [0])
plt.axes.set_aspect('equal')
matplotlib.pyplot.show()
```

```python
delta = 0.025
xrange = arange(-5.0, 5, delta)
yrange = arange(-5.0, 5.0, delta)
X, Y = meshgrid(xrange,yrange)

# F is one side of the equation, G is the other
F = X**2 + 4*Y**2 + 2*X - 8*Y + 1
G = 0

plt = matplotlib.pyplot.contour(X, Y, (F - G), [0])
plt.axes.set_aspect('equal')
matplotlib.pyplot.show()
```

### Plotting with sympy for when matplotlib fails

```python
from sympy import symbols, plot_implicit, Eq
x, y = symbols('x y')
plot_implicit(Eq(x**2 + y**2 - 6*x + 8*y + 20, 3), (x, 0, 9), (y, -9, 0), aspect_ratio=[1,1])

```

```python
import matplotlib.pyplot as plt

n = 1000
r = 1
k = 1
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

# Plot a helix along the x-axis
theta_max = 8 * np.pi
theta = np.linspace(0, theta_max, n)
x = r * np.cos(theta)
y = r * np.sin(theta)
z = k * theta
ax.plot(x, y, z, 'b', lw=2)

matplotlib.pyplot.show()
```

### Differential Calculus

```python
sym = civilpy.general.solver
x = sym.Symbol('x')
sym.diff(x**5)
```

$\displaystyle 5 x^{4}$

slope @ x = 3 of curve $f(x) = x^3 - 2x$

```python
f_prime = sym.diff(x**3 - 2*x)
f_prime.subs(x, 3)
```

$\displaystyle 25$

```python
f = 5*x**(5/3)
f_prime = sym.diff(f)
f_prime
```

$\displaystyle 8.33333333333333 x^{0.666666666666667}$

```python
f = sym.sin(x) * sym.cos(x) ** 2
f_prime = sym.diff(f)
f_prime
```

$\displaystyle - 2 \sin^{2}{\left(x \right)} \cos{\left(x \right)} + \cos^{3}{\left(x \right)}$

```python
f = sym.ln(sym.cos(sym.exp(x)))
f_prime = sym.diff(f)
sym.simplify(f_prime)
```

$\displaystyle - e^{x} \tan{\left(e^{x} \right)}$

Find the global extrema of $ f(x) = x^3 + x^2 - x + 1 $on the interval \[-2, +2\]

```python
f = x**3 + x**2 - x + 1
x1 = -2
x2 = 2
x1, x2, dict1 = civilpy.general.find_global_extrema_order_3(f, x1, x2)
```

    The first derivative is
    3*x**2 + 2*x - 1 the second is
     6*x + 2

    The first root is -1, the second root is 1/3

    -1 is a local maximum

    1/3 is a local minimum

    The inflection point is -1/3

    Global min: -2, Global max: 2

### Derivatives of parametric equations

```python
x, y, t = sym.symbols('x y t')
f1 = 5 * sym.cos(t) - x
f2 = 5 * sym.sin(t) - y

dx_dt = sym.diff(f1, t)
dy_dt = sym.diff(f2, t)

dy_dx = dy_dt / dx_dt
sym.simplify(dy_dx)
```

$\displaystyle - \frac{1}{\tan{\left(t \right)}}$

### Partial derivative

```python
x, y, z = sym.symbols('x y z')
f = 3*x**2 - 6*y**2 + x*y + 5*y - 9
dz_dx = sym.diff(f, x)
dz_dx
```

$\displaystyle 6 x + y$

### Implicit Differentiation

```python
f = x**2 + x*y + y**3

df_dx = sym.diff(f, x)
df_dy = sym.diff(f, y)
dy_dx = -df_dx / df_dy
dy_dx
```

$\displaystyle \frac{- 2 x - y}{x + 3 y^{2}}$

```python
f = x**2 + y**2 + z**2 - 9
df_dx = sym.diff(f, x)
df_dz = sym.diff(f, z)
dz_dx = - df_dx / df_dz

result = dz_dx.subs(x, 1).subs(y, 2).subs(z, 2)
result
```

$\displaystyle - \frac{1}{2}$

### Integration

```python
f = (2*x**2 + 3)**2 / x
integral = sym.integrate(f, x)
integral
```

$\displaystyle x^{4} + 6 x^{2} + 9 \log{\left(x \right)}$

```python
f = (3*x + 2) / (3*x - 2)
integral = sym.integrate(f, x)
integral
```

$\displaystyle x + \frac{4 \log{\left(3 x - 2 \right)}}{3}$

```python
f = x**2 * sym.exp(x) * sym.cos(x)
integral = sym.integrate(f, x)
integral
```

$\displaystyle \frac{x^{2} e^{x} \sin{\left(x \right)}}{2} + \frac{x^{2} e^{x} \cos{\left(x \right)}}{2} - x e^{x} \sin{\left(x \right)} + \frac{e^{x} \sin{\left(x \right)}}{2} - \frac{e^{x} \cos{\left(x \right)}}{2}$

```python
f = sym.sin(x) * sym.cos(x)
integral = sym.integrate(f, x)
integral
```

$\displaystyle \frac{\sin^{2}{\left(x \right)}}{2}$

```python
f = x**2 + (y**3)*x
integral = sym.integrate(f, x)
d_integral = sym.integrate(integral, y)
d_integral
```

$\displaystyle \frac{x^{3} y}{3} + \frac{x^{2} y^{4}}{8}$

### Definite Integral

```python
f = sym.sin(x)
# first value = bottom value in eq.
solution = sym.integrate(f, (x, sym.pi / 4, sym.pi / 3))
round(solution.evalf(), 3)
```

$\displaystyle 0.207$

```python
f = x ** 2
integral = sym.integrate(f, (x, 0, 4))
integral
```

$\displaystyle \frac{64}{3}$

```python

```