# Symbolic Python

In standard mathematics we routinely write down abstract variables or concepts and manipulate them without ever assigning specific values to them. An example would be the quadratic equation

$$  a x^2 + b x + c = 0 $$

and its roots $x_{\pm}$: we can write down the solutions of the equation and discuss the existence, within the real numbers, of the roots, without specifying the particular values of the parameters $a, b$ and $c$.

In a standard computer programming language, we can write *functions* that encapsulate the solutions of the equation, but calling those functions requires us to specify values of the parameters. In general, the value of a variable must be given before the variable can be used.

However, there *do* exist *Computer Algebra Systems* that can perform manipulations in the "standard" mathematical form. Through the university you will have access to Wolfram Mathematica and Maple, which are commercial packages providing a huge range of mathematical tools. There are also freely available packages, such as SageMath and `sympy`. These are not always easy to use, as all CAS have their own formal languages that rarely perfectly match your expectations.

Here we will briefly look at `sympy`, which is a pure Python CAS. `sympy` is not suitable for complex calculations, as it's far slower than the alternatives. However, it does interface very cleanly with Python, so can be used inside Python code, especially to avoid entering lengthy expressions.

# sympy

## Setting up

Setting up `sympy` is straightforward:

```python
import sympy
sympy.init_printing()
```

The standard `import` command is used. The `init_printing` command looks at your system to find the clearest way of displaying the output; this isn't necessary, but is helpful for understanding the results.

To do *anything* in `sympy` we have to explicitly tell it if something is a variable, and what name it has. There are two commands that do this. To declare a single variable, use

```python
x = sympy.Symbol('x')
```

To declare multiple variables at once, use

```python
y, z0 = sympy.symbols(('y', 'z_0'))
```

Note that the "name" of the variable does not need to match the symbol with which it is displayed. We have used this with `z0` above:

```python
z0
```

Once we have variables, we can define new variables by operating on old ones:

```python
a = x + y
b = y * z0
print("a={}. b={}.".format(a, b))
```

    a=x + y. b=y*z_0.

```python
a
```

In addition to variables, we can also define general functions. There is only one option for this:

```python
f = sympy.Function('f')
```

## In-built functions

We have seen already that mathematical functions can be found in different packages. For example, the $\sin$ function appears in `math` as `math.sin`, acting on a single number. It also appears in `numpy` as `numpy.sin`, where it can act on vectors and arrays in one go. `sympy` re-implements many mathematical functions, for example as `sympy.sin`, which can act on abstract (`sympy`) variables.

Whenever using `sympy` we should use `sympy` functions, as these can be manipulated and simplified. For example:

```python
c = sympy.sin(x)**2 + sympy.cos(x)**2
```

```python
c
```

```python
c.simplify()
```

Note the steps taken here. `c` is an object, something that `sympy` has created. Once created it can be manipulated and simplified, using the methods on the object. It is useful to use tab completion to look at the available commands. For example,

```python
d = sympy.cosh(x)**2 - sympy.sinh(x)**2
```

Now type `d.` and then tab, to inspect all the available methods. As before, we could do

```python
d.simplify()
```

but there are many other options.

## Solving equations

Let us go back to our quadratic equation and check the solution. To define an *equation* we use the `sympy.Eq` function:

```python
a, b, c, x = sympy.symbols(('a', 'b', 'c', 'x'))
quadratic_equation = sympy.Eq(a*x**2+b*x+c, 0)
sympy.solve(quadratic_equation)
```

What happened here? `sympy` is not smart enough to know that we wanted to solve for `x`! Instead, it solved for the first variable it encountered. Let us try again:

```python
sympy.solve(quadratic_equation, x)
```

This is our expectation: multiple solutions, returned as a list. We can access and manipulate these results:

```python
roots = sympy.solve(quadratic_equation, x)
xplus, xminus = sympy.symbols(('x_{+}', 'x_{-}'))
xplus = roots[0]
xminus = roots[1]
```

We can substitute in specific values for the parameters to find solutions:

```python
xplus_solution = xplus.subs([(a,1), (b,2), (c,3)])
xplus_solution
```

We have a list of substitutions. Each substitution is given by a tuple, containing the variable to be replaced, and the expression replacing it. We do not have to substitute in numbers, as here, but could use other variables:

```python
xminus_solution = xminus.subs([(b,a), (c,a+z0)])
xminus_solution
```

```python
xminus_solution.simplify()
```

We can use similar syntax to solve *systems* of equations, such as

$$ \begin{aligned}  x + 2 y &= 0, \\ xy & = z_0. \end{aligned} $$

```python
eq1 = sympy.Eq(x+2*y, 0)
eq2 = sympy.Eq(x*y, z0)
sympy.solve([eq1, eq2], [x, y])
```

## Differentiation and integration

### Differentiation

There is a standard function for differentiation, `diff`:

```python
expression = x**2*sympy.sin(sympy.log(x))
sympy.diff(expression, x)
```

A parameter can control how many times to differentiate:

```python
sympy.diff(expression, x, 3)
```

Partial differentiation with respect to multiple variables can also be performed by increasing the number of arguments:

```python
expression2 = x*sympy.cos(y**2 + x)
sympy.diff(expression2, x, 2, y, 3)
```

There is also a function representing an *unevaluated* derivative:

```python
sympy.Derivative(expression2, x, 2, y, 3)
```

These can be useful for display, building up a calculation in stages, simplification, or when the derivative cannot be evaluated. It can be explicitly evaluated using the `doit` function:

```python
sympy.Derivative(expression2, x, 2, y, 3).doit()
```

### Integration

Integration uses the `integrate` function. This can calculate either definite or indefinite integrals, but will *not* include the integration constant.

```python
integrand=sympy.log(x)**2
sympy.integrate(integrand, x)
```

```python
sympy.integrate(integrand, (x, 1, 10))
```

The definite integral is specified by passing a tuple, with the variable to be integrated (here `x`) and the lower and upper limits (which can be expressions).

Note that `sympy` includes an "infinity" object `oo` (two `o`'s), which can be used in the limits of integration:

```python
sympy.integrate(sympy.exp(-x), (x, 0, sympy.oo))
```

Multiple integration for higher dimensional integrals can be performed:

```python
sympy.integrate(sympy.exp(-(x+y))*sympy.cos(x)*sympy.sin(y), x, y)
```

```python
sympy.integrate(sympy.exp(-(x+y))*sympy.cos(x)*sympy.sin(y),
                (x, 0, sympy.pi), (y, 0, sympy.pi))
```

Again, there is an unevaluated integral:

```python
sympy.Integral(integrand, x)
```

```python
sympy.Integral(integrand, (x, 1, 10))
```

Again, the `doit` method will explicitly evaluate the result where possible.

## Differential equations

Defining and solving differential equations uses the pattern from the previous sections. We'll use the same example problem as in the `scipy` case,

$$  \frac{\text{d} y}{\text{d} t} = e^{-t} - y, \qquad y(0) = 1. $$

First we define that $y$ is a function, currently unknown, and $t$ is a variable.

```python
y = sympy.Function('y')
t = sympy.Symbol('t')
```

`y` is a general function, and can be a function of anything at this point (any number of variables with any name). To use it consistently, we *must* refer to it explicitly as a function of $t$ everywhere. For example,

```python
y(t)
```

We then define the differential equation. `sympy.Eq` defines the equation, and `diff` differentiates:

```python
ode = sympy.Eq(y(t).diff(t), sympy.exp(-t) - y(t))
ode
```

Here we have used `diff` as a method applied to the function. As `sympy` can't differentiate $y(t)$ (as it doesn't have an explicit value), it leaves it unevaluated.

We can now use the `dsolve` function to get the solution to the ODE. The syntax is very similar to the `solve` function used above:

```python
sympy.dsolve(ode, y(t))
```

This is simple enough to solve, but we'll use symbolic methods to find the constant, by setting $t = 0$ and $y(t) = y(0) = 1$.

```python
general_solution = sympy.dsolve(ode, y(t))
value = general_solution.subs([(t,0), (y(0), 1)])
value
```

We then find the specific solution of the ODE.

```python
ode_solution = general_solution.subs([(value.rhs,value.lhs)])
ode_solution
```

## Plotting

`sympy` provides an interface to `matplotlib` so that expressions can be directly plotted. For example,

```python
%matplotlib inline
from matplotlib import rcParams
rcParams['figure.figsize']=(12,9)
```

```python
sympy.plot(sympy.sin(x));
```

We can explicitly set limits, for example

```python
sympy.plot(sympy.exp(-x)*sympy.sin(x**2), (x, 0, 1));
```

We can plot the solution to the differential equation computed above:

```python
sympy.plot(ode_solution.rhs, xlim=(0, 1), ylim=(0.7, 1.05));
```

This can be *visually* compared to the previous result. However, we would often like a more precise comparison, which requires numerically evaluating the solution to the ODE at specific points.

## lambdify

At the end of a symbolic calculation using `sympy` we will have a result that is often long and complex, and that is needed in another part of another code. We could type the appropriate expression in by hand, but this is tedious and error prone. A better way is to make the computer do it.

The example we use here is the solution to the ODE above. We have solved it symbolically, and the result is straightforward. We can also solve it numerically using `scipy`. We want to compare the two.

First, let us compute the `scipy` numerical result:

```python
from numpy import exp
from scipy.integrate import odeint
import numpy

def dydt(y, t):
    """
    Defining the ODE dy/dt = e^{-t} - y.

    Parameters
    ----------

    y : real
        The value of y at time t (the current numerical approximation)
    t : real
        The current time t

    Returns
    -------

    dydt : real
        The RHS function defining the ODE.
    """

    return exp(-t) - y

t_scipy = numpy.linspace(0.0, 1.0)
y0 = [1.0]

y_scipy = odeint(dydt, y0, t_scipy)
```

We want to evaluate our `sympy` solution at the same points as our `scipy` solution, in order to do a direct comparison. In order to do that, we want to construct a function that computes our `sympy` solution, without typing it in. That is what `lambdify` is for: it creates a function from a sympy expression.

First let us get the expression explicitly:

```python
ode_expression = ode_solution.rhs
ode_expression
```

Then we construct the function using `lambdify`:

```python
from sympy.utilities.lambdify import lambdify

ode_function = lambdify((t,), ode_expression, modules='numpy')
```

The first argument to `lambdify` is a tuple containing the arguments of the function to be created. In this case that's just `t`, the time(s) at which we want to evaluate the expression. The second argument to `lambdify` is the expression that we want converted into a function. The third argument, which is optional, tells `lambdify` that where possible it should use `numpy` functions. This means that we call the function using `numpy` arrays, it will calculate using `numpy` array expressions, doing the whole calculation in a single call.

We now have a function that we can directly call:

```python
print("sympy solution at t=0: {}".format(ode_function(0.0)))
print("sympy solution at t=0.5: {}".format(ode_function(0.5)))
```

    sympy solution at t=0: 1.0
    sympy solution at t=0.5: 0.9097959895689501

And we can directly apply this function to the times at which the `scipy` solution is constructed, for comparison:

```python
y_sympy = ode_function(t_scipy)
```

Now we can use `matplotlib` to plot both on the same figure:

```python
from matplotlib import pyplot
pyplot.plot(t_scipy, y_scipy[:,0], 'b-', label='scipy')
pyplot.plot(t_scipy, y_sympy, 'k--', label='sympy')
pyplot.xlabel(r'$t$')
pyplot.ylabel(r'$y$')
pyplot.legend(loc='upper right')
pyplot.show()
```

We see good visual agreement everywhere. But how accurate is it?

Now that we have `numpy` arrays explicitly containing the solutions, we can manipulate these to see the differences between solutions:

```python
pyplot.semilogy(t_scipy, numpy.abs(y_scipy[:,0]-y_sympy))
pyplot.xlabel(r'$t$')
pyplot.ylabel('Difference in solutions');
```

The accuracy is around $10^{-8}$ everywhere - by modifying the accuracy of the `scipy` solver this can be made more accurate (if needed) or less (if the calculation takes too long and high accuracy is not required).

# Further reading

`sympy` has [detailed documentation](http://docs.sympy.org/latest/index.html) and a [useful tutorial](http://docs.sympy.org/dev/tutorial/index.html).

## Exercise : systematic ODE solving

We are interested in the solution of

$$  \frac{\text{d} y}{\text{d} t} = e^{-t} - y^n, \qquad y(0) = 1, $$

where $n > 1$ is an integer. The "minor" change from the above examples mean that `sympy` can only give the solution as a power series.

### Exercise 1

Compute the general solution as a power series for $n = 2$.

### Exercise 2

Investigate the help for the `dsolve` function to straightforwardly impose the initial condition $y(0) = 1$ using the `ics` argument. Using this, compute the specific solutions that satisfy the ODE for $n = 2, \dots, 10$.

### Exercise 3

Using the `removeO` command, plot each of these solutions for $t \in [0, 1]$.


# SymPy Basics
Adapted from: https://github.com/sympy/sympy/wiki/Quick-examples

```python
from sympy import *
from IPython.display import display
init_printing(order="lex",use_latex='mathjax')
```

# Symbolic Expressions and Calculations

```python
x, y, z, t = symbols('x y z t')
k, m, n = symbols('k m n', integer=True)
#f, g, h = map(Function, 'fgh')
```

```python
eqn = Rational(3,2)*pi + exp(I*x) / (x**2 + y)
eqn
```

$$\frac{3 \pi}{2} + \frac{e^{i x}}{x^{2} + y}$$

```python
eqn.subs(x,3)
```

$$\frac{3 \pi}{2} + \frac{e^{3 i}}{y + 9}$$

```python
exp(I*x).subs(x,pi).evalf()
```

$$-1.0$$

```python
expr = x + 2*y
expr.args
```

$$\left ( x, \quad 2 y\right )$$

```python
exp(pi * sqrt(163)).evalf(50)
```

$$262537412640768743.99999999999925007259719818568888$$

```python
N(pi,100)
```

$$3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117068$$

```python
latex(S(eqn,evaluate=False))
```

    '\\frac{3 \\pi}{2} + \\frac{e^{i x}}{x^{2} + y}'

$$ \frac{3 \pi}{2} + \frac{e^{i x}}{x^{2} + y}$$

## Algebra

```python
((x+y)**2 * (x+1)).expand()
```

$$x^{3} + 2 x^{2} y + x^{2} + x y^{2} + 2 x y + y^{2}$$

```python
a = 1/x + (x*sin(x) - 1)/x
a
```

$$\frac{1}{x} \left(x \sin{\left (x \right )} - 1\right) + \frac{1}{x}$$

```python
a.simplify()
```

$$\sin{\left (x \right )}$$

```python
eqn = Eq(x**3 + 2*x**2 + 4*x + 8, 0)
eqn
```

$$x^{3} + 2 x^{2} + 4 x + 8 = 0$$

```python
solve(eqn,x)
```

$$\left [ -2, \quad - 2 i, \quad 2 i\right ]$$

```python
eq1 = Eq(x + 5*y, 2)
eq2 = Eq(-3*x + 6*y, 15)
display(eq1)
display(eq2)
sln = solve([eq1, eq2], [x, y])
sln
```

$$x + 5 y = 2$$

$$- 3 x + 6 y = 15$$

$$\left \{ x : -3, \quad y : 1\right \}$$

```python
display(eq1.subs(sln))
display(eq2.subs(sln))
```

$$\mathrm{True}$$

$$\mathrm{True}$$

## Recurrence Relations

$$
\large\begin{align}
y_0 & =1 \\
y_1 & =4 \\
y_n & =y_n-2y_{n-1}+5y_{n-2}
\end{align}
$$

```python
f=y(n)-2*y(n-1)-5*y(n-2)
f
```

$$y{\left (n \right )} - 5 y{\left (n - 2 \right )} - 2 y{\left (n - 1 \right )}$$

```python
sln = rsolve(f,y(n),[1,4])
sln
```

$$\left(\frac{1}{2} + \frac{\sqrt{6}}{4}\right) \left(1 + \sqrt{6}\right)^{n} + \left(- \sqrt{6} + 1\right)^{n} \left(- \frac{\sqrt{6}}{4} + \frac{1}{2}\right)$$

```python
for i in range(0,10):
    print(sln.subs(n,i).simplify())
```

    1
    4
    13
    46
    157
    544
    1873
    6466
    22297
    76924

## Sums and Products

```python
a, b = symbols('a b')
s = Sum(6*n**2 + 2**n, (n, a, b))
s
```

$$\sum_{n=a}^{b} \left(2^{n} + 6 n^{2}\right)$$

```python
s.doit()
```

$$- 2^{a} + 2^{b + 1} - 2 a^{3} + 3 a^{2} - a + 2 b^{3} + 3 b^{2} + b$$

```python
s.subs({b:3,a:1}).doit()
```

$$98$$

```python
Sum(b, (b, 1, n)).doit().factor()
```

$$\frac{n}{2} \left(n + 1\right)$$

```python
Sum(n*(n+1)/2,(n, 1, b)).doit()
```

$$\frac{b^{3}}{6} + \frac{b^{2}}{2} + \frac{b}{3}$$

```python
for i in range(1,10):
    print(Sum(n*(n+1)/2, (n, 1, b)).doit().subs(b,i))
```

    1
    4
    10
    20
    35
    56
    84
    120
    165

```python
Sum(n, (n, a, b)).subs(a,1).doit()
```

$$\frac{b^{2}}{2} + \frac{b}{2}$$

```python
(x**3/6 + x**2/2 +x/3).factor()
```

$$\frac{x}{6} \left(x + 1\right) \left(x + 2\right)$$

```python
product(n*(n+1), (n, 1, b))
```

$${2}^{\left(b\right)} b!$$

```python
f=Function('f')
ex=Eq(f(1/x)-3*f(x),x)
```

## Calculus

$$\lim_{x\to 0} \frac{\sin x - x}{x^3} = -\frac{1}{6}$$

```python
((sin(x)-x)/x**3).limit(x,0)
```

$$- \frac{1}{6}$$

```python
(x**2+5*x**3).diff(x)
```

$$15 x^{2} + 2 x$$

```python
(-x).limit(x,oo)
```

$$-\infty$$

$$\int x^2 \cos x  \ dx$$

```python
(x**2 * cos(x)).integrate(x)
```

$$x^{2} \sin{\left (x \right )} + 2 x \cos{\left (x \right )} - 2 \sin{\left (x \right )}$$

$$\int_0^{\pi/2} x^2 \cos x \ dx$$

```python
integrate(x**2 * cos(x), (x, 0, pi/2))
##(x**2 * cos(x)).integrate(x, 0, pi/2) does not work.
```

$$-2 + \frac{\pi^{2}}{4}$$

$$  \large f''(x) + 9 f(x) = 1 $$

```python
fn = dsolve(Eq(Derivative(f(x),x,x) + 9*f(x), 1), f(x))
fn
```

$$f{\left (x \right )} = C_{1} \sin{\left (3 x \right )} + C_{2} \cos{\left (3 x \right )} + \frac{1}{9}$$

```python
fla = 3*sin(3*x)+3*cos(3*x)+1/9
fla.diff(x).diff(x).subs(x,3)+9*fla.subs(x,3)
```

$$1.0$$

## Linear Algebra


# "Differentiation and Integration with SymPy"
> "Differentiation and Integration are some of the most important topics in maths. This post walks through how python can be used to solve these kinds of problems."

 - toc:false
 - badges: true
 - comments: true
 - author: Rohit Thakur
 - categories: [maths, calculus]

Differentiation and Integration and the building blocks of calculus. They are usually taught in Calculus I.
These two concept are heavily used in alot of fields, including machine learning.

If you don't remember, here is a high level introduction of what these two concepts are:

Differentiation (also called derivation) measures the rate of change of a function. Think of it like calculating slope in linear models. Only difference here is that you can calculate the slope at every point and for functions that are non linear.
For Example: if `f(x) = x**2` then, the derivative of this function is `f'(x) = 2x`. The derivative function is read "f prime of x".

Intergration on the other hand measures the area under the curve.
For the above function, the integration or often called antiderivative will be `x**3 / 3`

Calculating Derivatives and Integrals is sometimes confusing so we'll use [SymPy](https://www.sympy.org/en/index.html).
We will start by installing the library.

```python
#collapse-output
!pip install sympy
```

    Requirement already satisfied: sympy in c:\users\rohit\miniconda3\envs\math\lib\site-packages (1.8)
    Requirement already satisfied: mpmath>=0.19 in c:\users\rohit\miniconda3\envs\math\lib\site-packages (from sympy) (1.2.1)

Next we import everything we need.

```python
from sympy import Symbol, diff, integrate
```

Since we will be calculating for `x`, we'll have to create a symbol for it.

```python
x = Symbol('x')
```

Lets start by writing our function

```python
f = 2*x**3+6
f
```

$\displaystyle 2 x^{3} + 6$

Now we calculate the derivative of the function using sympy.

```python
f_prime = f.diff(x)
f_prime
```

$\displaystyle 6 x^{2}$

So now we have the derivative of our function. Lets calculate the integral.

```python
f_int = integrate(f_prime, x)
f_int
```

$\displaystyle 2 x^{3}$

So this is how we calculate both the derivative and anti-derivative of a function using sympy. There are other libraries as well that work really well,
e.g scipy.

>Important: These aren't actual functions. In order to use these, you need to use "lambdify" them. We'll start by importing it.

```python
from sympy.utilities.lambdify import lambdify
```

```python
f = lambdify(x, f)
f_prime = lambdify(x, f_prime)
f_int = lambdify(x, f_int)
```

```python
print(f"The value of function at 3 is: {f(3)} and its derivative is {f_prime(3)}")
```

    The value of function at 3 is: 60 and its derivative is 54


###Objective
**Given a set of equilibria and their stability, construct a system of equations that produces the same dynamics as those determined by the equilibria.**

Take a general two dimensional system of the following form

$$\begin{align}
\dot x &= f(x,y) \\
\dot y &= g(x,y)
\end{align}$$

What form should $f(x,y)$ and $g(x,y)$ take? I will show that all dynamics in two dimensions can be reproduced by a coupled system of polynomials.

$$\begin{align}
f(x,y) &= y+P_1(x) \\
g(x,y) &= -x+P_2(y)
\end{align}$$

where $P_1(x)$ and $P_2(y)$ are arbitrary polynomials of any degree. This form was chosen so that the nullclines could be explicitly defined and would be condusive to analysis. The nullclines are given by

$$\begin{align}
y &= -P_1(x) \\
x &= P_2(y)
\end{align}$$

The Jacobian of the system is given by.

$$J(x,y) = \begin{bmatrix}
       p_1 & 1 \\
       -1 & p_2
     \end{bmatrix}$$

Where for notational brevity we are referring to the derivatives as $\frac{dP_1}{dx}(x,y) \equiv p_1$ and $\frac{dP_2}{dy}(x,y) \equiv p_2$. We can quickly see the trace and determinant as

$$\begin{align}
Trace(J) &= \tau(p_1,p_2) = p_1 + p_2 \\
Det(J) &= \Delta(p_1,p_2) = p_1 p_2 +1
\end{align}$$

The eigen values of the Jacobian determine the type and stability of nonhyperbolic equilibria. The characteristic for this system is given by

$$\begin{vmatrix}
     p_1-\lambda & 1 \\
     -1 & p_2-\lambda
   \end{vmatrix} = (p_1-\lambda)(p_2-\lambda)+1
                 = \lambda^2 - (p_1+p_2)\lambda + p_1 p_2 +1
                 = \lambda^2 - \tau \lambda + \Delta
$$

Given the characteristic we find the eigenvalues as

$$\lambda = \frac{\tau \pm \sqrt{\tau^2-4\Delta}}{2}$$

In conventional dynamics theory the type of equilibria is determined by the sign of the radicand $\tau^2-4\Delta$ and their stability is determined by the sign of $\tau$ and $\Delta$. The regions of stability in $\tau$ vs $\Delta$ space is shown below.

In general $\tau$ and $\Delta$ can take on all values and collectively describe all equilibria in two dimensions. Given that $P_1(x)$ and $P_2(y)$ are polynomials we know that their derivatives $p_1,p_2 \hspace{3pt} \epsilon \hspace{3pt} (-\infty,\infty)$. Also, because our general system above produces continuous $\tau$ and $\Delta$, we see that it is capable of reproducing all equilibria in two dimensions.

**TO DO: add the $p_1$ vs $p_2$ plane showing regions of stability**

The slopes of nullclines in phase space are given by

$$\begin{align}
-\frac{f_x}{f_y} &= -p_1 \\
-\frac{g_x}{g_y} &= \frac{1}{p_2}
\end{align}$$

In the $y$ vs $x$ phase space we'll define a vector tangent to each nullcline as

$$\begin{align}
\vec{T_x} &= 1\hat{i} + -p_1\hat{j} \\
\vec{T_y} &= p_2\hat{i} + 1\hat{j}
\end{align}$$

We are interested in the angle between these vectors at an equilibria, therefore we will use the following formula...

$$\theta = \cos^{-1}(\frac{\vec{T_x}\cdot \vec{T_y}}{\|T_x\| \|T_y\|})$$

Starting with the numerator we get...

$$\vec{T_x}\cdot \vec{T_y} = p_2-p_1$$

And in the denominator

$$\|T_x\| \|T_y\| = \sqrt{1+p^2_1}\sqrt{1+p^2_2}$$

The objective here is to find the angle between the tangent vectors in terms of the eigenvalues of the system. We will need to convert the numerator and denominator into expressions of only eigenvalues. For this we will be using the handy fact from linear algebra that the trace of a matrix is equal to the sum of its eigenvalues, $\tau=\lambda_1+\lambda_2$, and the determinant of a matrix is equal to the product of its eigenvalues, $\Delta = \lambda_1 \lambda_2$.

**TO DO: elegant algebra leading to the following formulas (don't have time now)**

Thus we find the angle between the tangent vectors at the equilibria is given by

$$\theta(\lambda_1,\lambda_2) = \cos^{-1}\Bigg (\pm \sqrt{\frac{(\lambda_1-\lambda_2)^2+4}{(\lambda_1+\lambda_2)^2+(\lambda_1 \lambda_2-2)^2}}\Bigg )$$

Or, in terms of the trace and determinant

$$\theta(\lambda_1,\lambda_2) = \cos^{-1}\Bigg (\pm \sqrt{\frac{\tau^2+4(1-\Delta)}{\tau^2+(\Delta-2)^2}}\Bigg )$$

We will show that this is defined for all nonzero $\lambda_1$ and $\lambda_2$. We needn't worry about the degenerate case as the Hartman-Grobman theory tells us the Jacobian is not guaranteed to correctly determine the stability of non-hyperbolic equilibria. We need to show three things.

1. The denominator is never zero
2. The radicand is always positive
3. The magnitude of the argument to $\cos^{-1}$ is bounded by one

The second equation is found by algebraic manipulations and subsitutions of the first. Therefore showing any of the properties for one equation implies it is true for the other.

**(1)**
By inspection we see that both terms in the denominator are squared (in both equations). Thus, the denominator is nonzero for nonzero eigenvalues.

**(2)**
By inspecting the first equation we see that all terms in the radicand are positive, and we are done.

**(3)**
Looking at the second equation...

$$\frac{\tau^2+4(1-\Delta)}{\tau^2+(\Delta-2)^2} < 1 \hspace{10pt} \rightarrow \hspace{10pt} \tau^2+4(1-\Delta) < \tau^2+(\Delta-2)^2$$

Recalling that the denominator is nonzero we do not flip the inequality. We expand both sides,

$$\tau^2-4\Delta+4 < \tau^2+\Delta^2-4\Delta+4 \hspace{10pt} \rightarrow \hspace{4pt} 0 < \Delta^2$$

which is always true for nonzero eigenvalues.

**TO DO: Demonstrate with some small examples**

```

```

```

```

cmb_phase_plane/research/.ipynb_checkpoints/Phase Diagram - Inverse Problem-checkpoint.ipynb


# Particle in  multistable potentials

## Phase portrait for a one-dimensional system

The Newton equation  for a point particle in a one-dimensional potential $U(x)$ can be written as a set of two first-order differential equations:

$$ \begin{cases} \dot x = v \\ \dot v = - \frac{1}{m}U'(x) \end{cases} $$

We can draw a phase portrait, i.e. parametric solutions $(x(t),v(t))$ and a vector field defined by the right hand sides of equations in the $(x,v)$-phase space.

## Example: motion in the $U(x) = x^3-x^2$ potential

```python
var('v')
m = 1
U(x) = x^3-x^2
xmax,xmin = sorted([s_.rhs() for s_ in solve(U.diff(x)==0,x)])
Emin = U(xmin)
Etot = 1/2*m*v^2 + U(x)

plot(U(x),(x,-0.4,1.1),figsize=4) +\
 point([xmin,U(xmin)],color='red',size=40)+\
 point([xmax,U(xmax)],color='red',size=40)
```

The potential $U(x) =x^3-x^2$.

```python
pkt = point((xmin,0),size=20,color='black')
plt = sum([ implicit_plot(Etot==E0,(x,-1/2,1.2),(v,-1,1),color='blue')\
          for E0 in srange(Emin,0.0,0.02)])
plt +=implicit_plot(Etot==0,(x,-1/2,1.2),(v,-1,1),color='black') +pkt
plt +=sum([ implicit_plot(Etot==E0,(x,-1/2,1.2),(v,-1,1),color='green')\
           for E0 in srange(0.02,-2*Emin,0.04)])
plt.show()
```

The phase curves $(x, v)$ depends on initial conditions $(x_0, v_0)$. There are two types of phase curves: closed (periodic motion of the  particle in a bounded interval) and open (the motion is unbounded: the particle can escape to -infinity or can return from -infinity.

```python
vector_field = vector([v,-U.diff(x)])
plt  + plot_vector_field(vector_field.normalized(),(x,-1/2,1.2),(v,-1,1))
```

The vector field shows direction of motion of the particle on the $(x, v)$-plane.

## Harmonic  oscillations  limit  for  one-dimensional systems

Consider a conservative one-dimensional system. In this case, the force $f(x)$ can always be represented as gradient of the potential $U(x)$, namely,
$$f(x) = -\frac{\partial U(x)}{dx}.$$
Consider a certain potential that has a minium at some point $x_0$. The necessary condition for minimum of the function is its zero first derivative  at this point.  Let's expand the potential in the Taylor series around the minimum. We obtain:
$$ U(x) = U(x_0) + \underbrace{U'(x_0)( x-x_0)}_{=0}+\frac{1}{2} U''(x_0)(x-x_0)^2+...$$
For small deviation from the minimum this series can be approximated by the function
$$U(x) = \frac{1}{2} k (x-x_0)^2,$$

The Newton equation for such  motion is as follows:

$$m \ddot x = m  a  = F = -U'(x)  =  -k (x-x_0)$$

For the new variable $y=x-x_0$ it takes the form

$$m \ddot y =-ky$$

This is the already known equation for the harmonic oscillator with the shifted equilibrium point $x_0$.

Now,  let us return to the system with the potential $U(x) = x^3-x^2$.

```python
var('x v')
Etot = 1/2*v^2 + U(x)
Elin = 1/2*v^2 + U(xmin)+1/2*U.diff(x,2).subs(x==xmin)*(x-xmin)^2
show(Etot)
show(Elin)
```

```python
Emin = Etot(x=xmin,v=0)
Emin
```

Let's have a look at  the trajectories for the  exact system with $U(x) = x^3-x^2$ and the linearized system with $U(x)=(1/2) x^2$. The blue line below is a separatrix - i.e. a solution with $E=0$

```python
plt = sum([ implicit_plot(Etot==E0,(x,.4,.91),(v,-.3,.3),color='red') \
          for E0 in srange(Emin+1e-3,-0.1,0.005)])
plt += implicit_plot(Etot==0.00,(x,0,1.1), (v,-.6,.6),color='blue')

plt_lin =sum([ implicit_plot(Elin==E0+1e-3,(x,.4,.91),(v,-.3,.3),color='gray') \
              for E0 in srange(Emin,-0.1,0.005)])
plt+plt_lin
```

For larger ones, there is a growing discrepancy:

 - for the nonlinear system, above certain energy, there are open trajectories
 - motion in an linearized system is always an ellipse. The period does not depend on the amplitude.

## Period of oscillation around potential minimum

We take a particle with  energy by $dE$ larger than minimum of the potential:

```python
dE = 0.01
```

```python
E0 = U(xmin)+dE
E0
```

In order to analyze eqaution of motion, we need to solve: $$U(x)=E_0$$ and obtain the extreme values of position to the left and right from potentiall minimum. In our case it con be done analytically.

However, note that it requires to neglect imaginary part, which is approximated small value of the order of [machine epsilon](https://en.wikipedia.org/wiki/Machine_epsilon).

```python
_, x1,x2 = sorted( [s_.rhs().n().real() for s_ in solve(U(x)==E0,x)])
x1,x2
```

Now, having $x_{1,2}$ we can integrate numerically equation for period (\ref{eq:1d_TE}) and obtain:

```python
period = 2*sqrt(m/2.)*\
  integral_numerical(1/sqrt(E0-U(x)) , x1,x2, algorithm='qags')[0]
period
```

Let is plot this situation:

```python
U(x) = x^3-x^2
xmax,xmin = sorted([s_.rhs() for s_ in solve(U.diff(x)==0,x)])
plot(U(x),(x,-0.4,1.1),figsize=4,gridlines=[None,[U(xmin),E0]])+\
 point([xmin,U(xmin)],color='red',size=40)+\
 point([xmax,U(xmax)],color='red',size=40)+\
 point([x1,U(x1)],color='green',size=40)+\
 point([x2,U(x2)],color='green',size=40)
```

We can write a function which computes the period based on previous steps:

```python
def T(E0):
    m = 1
    _, x1,x2 = sorted( [s_.rhs().n().real() \
                        for s_ in solve(U(x)==E0,x)])
    integral, error = \
      integral_numerical(1/sqrt(E0-U(x)), x1,x2, algorithm='qags')
    period = 2*sqrt(m/2.) * integral
    return period
```

```python
period_num = T(U(xmin)+dE)
period_num
```

We known exactly the period of oscillation for small neiborhood of potential minimum. It is simply period of the harmonic oscillator with frequecy $\omega=\sqrt{U''(x_{min})}$ (for $m=1$).

```python
omega = sqrt(U(x).diff(x,2).subs(x==xmin.n()))
period_harm = 2*pi.n()/omega
period_harm
```

#### How does period depent on energy?

We can easily compute plot $T(E_0)$:

```python
TonE = [(E_,T(E_)) for E_ in srange(U(xmin)+1e-6,-1e-5,0.001)]
```

```python
line(TonE, figsize=(6,2),gridlines=[None,[period_harm]])
```

#### How does the tracjectory look like?

Let us integrate numerically equation of motion:

```python
t_lst = srange(0, period_num, 0.01, include_endpoint=True)
sol = desolve_odeint([v,-U.diff(x)],[x2,.0],t_lst,[x,v])
```

We have in phase space $(x,\dot x)$:

```python
line(sol[::10,:],marker='o',figsize=4)
```

Position oscillates with time:

```python
line(zip(t_lst,sol[:,0]),figsize=(6,2))
```

## Time to reach the hill

The top of the potential hill  is at the beginning of the coordinate system $(x,E)$. Then we examine the limit $E\to0$ boundary

Near zero, we can approximate the potential by  the reverse parabola. Then the time to reach the hill from a certain point (for example $x=1$) reads:

```python
var('E')
assume(E>0)
integrate(-1/sqrt(E+x^2),x,1,0)
```

This result is divergent for $E\to0$:

```python
limit( arcsinh(1/x),x=0)
```

It means that time needed to climb a hill with *just* enough kinetic energy is **infinite**. It is valid only for potential hills which have zero derivative at the top. On the other hand for potential barriers which do not have this property, for example, $U(x)=-|x|$, the particle can reach the top with just enought energy in  finite time.

Let analyze it:

```python
U1(x) = -abs(x)
E0 = 0
```

we can plot velocity and potential:

```python
plot([U1(x), sqrt(2*(E0 - U1(x)))],(x,-1,1),figsize=4)
```

the time of travel from $x=-1$ to $x=0$ is given by:

\begin{equation}
\label{eq:1d_TE}
t=\sqrt{\frac{m}{2}} \; \int_{-x1}^{x1}{\frac{dx}{(\sqrt(E-U(x)}}
\end{equation}

which in this case is:

```python
sqrt(m/2.)*integrate(1/sqrt((E0- U1(x))),x,-1,0).n()
```

In the case of potentials which behave like $|x|^\alpha$, for $\alpha>1$ we can calculate time  of travel if we particle total energy is by $dE$ larger than potential barrier.

```python
def t_hill(E0):
    m = 1

    x2, = [s_.rhs().n().real() for s_ in solve(U(x)==E0,x) if s_.rhs().n().imag().abs()<1e-6]

    integral, error = \
      integral_numerical(1/sqrt(E0-U(x)), 0,x2, algorithm='qags')
    m = 1
    period = 2*sqrt(m/2.) * integral
    return period
```

```python
t_hill(9.1)
```

```python
import numpy as np
t_E = [(E_,t_hill(E_)) for E_ in np.logspace(-6,1,120)]
line(t_E,axes_labels=['$E$','$t$'], figsize=(6,2))
```

```python
t_E = [(log(E_),t_hill(E_)) for E_ in np.logspace(-6,16,120)]
line(t_E,axes_labels=['$\log_{10} E$','$t$'], figsize=(6,2))
```

#### Experiment with Sage!
Examine in a similar way the system corresponding to the movement in the $U(x) = -\cos(x)$ potential - this is a physical pendulum.

\newpage


# Lecture 11: Poisson Distribution and Approximation

## Sympathetic Magic, or _[Confusing the map for the territory](http://nobeliefs.com/MapandTerritory.htm)_

### Don't mistake a random variable for its distribution

As a case in point, say we have two random variables $X$ and $Y$.

\begin{align}
  P(X + Y) \ne P(X=x) + P(Y=y)
\end{align}

Claiming equality is clearly nonsense, since $X + Y$ is itself a random variable, and it should be recognized that $P(X=x) + P(Y=y)$ can exceed 1.

### Think of them as houses & blueprints

Metaphorically, the distribution is the _blueprint_, and the random variable is the _house_.

----

## Poisson Distribution

### Description

The single most often used real-world discrete model. Applicable when there are a _large number of trials_ with a _small probability of success in each trial_.

Note that the probability of success in each trial does _not_ have to be the same in each trial (unlike $Bin(n,p)$).

The Poisson distribution expresses the probability of a given number of events occurring in a fixed interval of time and/or space, if these events occur with a known average _rate_ and independently of the time since the last event. The Poisson distribution can also be used for the number of events in other specified intervals such as distance, area or volume.

Examples might be counting:

- e-mails coming into your mailbox in an hour
- buses following the same route arriving at a bus-stop in a day
- natural phenomena, like earthquakes or meteor sightings, etc., in a month
- chips in a chocolate-chip cookie (area/volume)

### Notation

$X \sim Pois(\lambda)$

### Parameters

$\lambda$ - the average number of events per interval (rate), where $\lambda \gt 0$

### Probability mass function

\begin{align}
  P(X = k) = e^{-\lambda} \frac{\lambda^k}{k!} ~~ \text{, where } k \in \{0,1,2,\dots,\}
\end{align}

Checking the validity of this PMF it is easy to see that it is always non-negative.

Furthermore,

\begin{align}
  \sum_{k=0}^{\infty} P(X = k) &= \sum_{k=0}^{\infty} e^{-\lambda} \frac{\lambda^k}{k!} \\
  &= e^{-\lambda} \sum_{k=0}^{\infty} \frac{\lambda^k}{k!} & &\text{since } e^{-\lambda} \text{ is constant} \\
  &= e^{-\lambda} e^{\lambda} & &\text{recall Taylor series for } e^{\lambda} \\
  &= 1 ~~~~ \blacksquare
\end{align}

### Expected value

\begin{align}
  \mathbb{E}(X) &= \sum_{k=0}^{\infty} k ~~ e^{-\lambda} \frac{\lambda^k}{k!} \\
  &= e^{-\lambda} \sum_{k=1}^{\infty} k \frac{\lambda^k}{k!} \\
  &= e^{-\lambda} \sum_{k=1}^{\infty} \frac{\lambda^k}{(k-1)!} \\
  &= \lambda e^{-\lambda} \sum_{k=1}^{\infty} \frac{\lambda^{k-1}}{(k-1)!} \\
  &= \lambda e^{-\lambda} e^{\lambda} \\
  &= \lambda ~~~~ \blacksquare
\end{align}

----

## Poisson Paradigm (Approximate)

Suppose we have:

1. a lot of events $A_1, A_2, \dots, A_n$, with $P(A_j) = p$
1. number of trials $n$ is very large
1. $p$ is very small

The events could be _independent_ (knowing that $A_1$ occurred has no bearing whatsoever on $A_2$.

They could even be _weakly dependent_ ($A_1$ occurring has some level of affect on the likelihood of $A_2$).

Either way, the expected number of events $A_j$ occurring is approximately $Pois(\lambda)$, and by Linearity

\begin{align}
  \mathbb{E}(A) &= \lambda = \sum_{j=1}^n p_j
\end{align}

### Relating the Binomial distribution to the Poisson

This example will relate the Binomial distribution to the Poisson distribution.

Suppose we have $X \sim Bin(n,p)$, and let $n \rightarrow \infty$, $p \rightarrow 0$.

We will hold $\lambda = np$ constant, which will let us see what happens to the Binomial distribution when the number of trails approaches $\infty$ while the probabability of success $p$ gets very small.

Specifically let's see what happens to the Binomial PMF under these conditions.

\begin{align}
  P(X = k) &= \binom{n}{k} p^k q^{n-k} ~~~~ \text{where k is fixed} \\
  &= \underbrace{\frac{n (n-1) \dots (n-k+1)}{k!} \left(\frac{\lambda}{n}\right)^k}_{\text{n terms cancel out}} ~~ \underbrace{\left(1 - \frac{\lambda}{n}\right)^n}_{\text{continuous compounding and e}} ~~ \underbrace{\left(1 - \frac{\lambda}{n}\right)^{-k}}_{\text{goes to 1}}   \\
  \\
  &\rightarrow \frac{\lambda^k}{k!} e^{-\lambda} ~~ \text{which is the Poisson PMF at k}
\end{align}

### Thought-experiment: counting raindrops

Let's say we want to count the number of raindrops that fall on a certain area on the ground.

We could divide this area with a grid, with each grid cell being very, very tiny.

Therefore:

- the number grid cells is very, very large
- the probability of a rain drop landing in an arbitrary cell is very, very small

_What distribution could we use to model this situation?_

#### How about $X \sim Bin(n,p)$?
Considering the Binomial distribution, we would have to make certain assumptions:

1. the event of a raindrop hitting a cell is independent of other events
1. the probability $p$ of a raindrop hitting an arbitrary cell is identical.

While it might be OK to make these assumptions, but one **big** stumbling block in using the Binomial is that we would end up calculating factorials of a very large $n$, which even for a computer would be problematic.

#### How about $X \sim Pois(\lambda)$?

The Poisson distribution is a better model approximation than the Binomial, since

1. no factorials involved, so much simpler than the Binomial
1. we can deal with the case where more than one raindrop lands in a cell
1. we don't need to make any assumption on $p$ being identical

### Birthdays, revisited: triple matches

Suppose we have $n$ people, and we want to know the _approximate probability_ that there are 3 people who share the same birthday.

Doing this the way we did in Lecture 3 is possible, but it would get very messy.

Let's set ourselves up first.

- there are $\binom{n}{3}$ triplets
- we used indicator r.v. $I_{ijk} \text{, where } i < j < k$
- $\Rightarrow \mathbb{E}(\# \text{ triple matches}) = \binom{n}{3} \frac{1}{365^2}$

But we are interested in the _approximate probability_ of triple matches, so now let's use the Poisson. Why?

1. we expect the total number of trials $\binom{n}{3}$ to be very large
1. the probability $P(I_{ijk}$ is very small
1. events are _weakly dependent_, since if persons $1$ and $2$ are already known to share a birthday, then  $I_{123} \text{, } I_{124}$ are not completely independent

We claim that these circumstances are _approximately_ $Pois(\lambda)$ with $\lambda = \mathbb{E}(\# \text{ triple matches})$.

Let $X = \text{# triple matches}$

\begin{align}
  P(X \ge 1) &= 1 - P(X=0) \\
  &\approx 1 - e^{-\lambda} \frac{\lambda^0}{0!} \\
  &= 1 - e^{-\lambda} ~~~~ \text{where you just plug in } \lambda = \binom{n}{3} \frac{1}{365^2} ~~~~ \blacksquare
\end{align}

----


# Predefined Metrics in Symbolic Module

### Importing some of the predefined tensors. All the metrics are comprehensively listed in EinsteinPy documentation.

```python
from einsteinpy.symbolic.predefined import Schwarzschild, DeSitter, AntiDeSitter, Minkowski, find
from einsteinpy.symbolic import RicciTensor, RicciScalar
import sympy
from sympy import simplify

sympy.init_printing()  # for pretty printing
```

### Printing the metrics for visualization
All the functions return instances of :py:class:`~einsteinpy.symbolic.metric.MetricTensor`

```python
sch = Schwarzschild()
sch.tensor()
```

```python
Minkowski(c=1).tensor()
```

```python
DeSitter().tensor()
```

```python
AntiDeSitter().tensor()
```

### Calculating the scalar (Ricci) curavtures
They should be constant for De-Sitter and Anti-De-Sitter spacetimes.

```python
scalar_curvature_de_sitter = RicciScalar.from_metric(DeSitter())
scalar_curvature_anti_de_sitter = RicciScalar.from_metric(AntiDeSitter())
```

```python
scalar_curvature_de_sitter.expr
```

```python
scalar_curvature_anti_de_sitter.expr
```

On simplifying the expression we got above, we indeed obtain a constant

```python
simplify(scalar_curvature_anti_de_sitter.expr)
```

### Searching for a predefined metric
find function returns a list of available functions

```python
find("sitter")
```

    ['AntiDeSitter', 'AntiDeSitterStatic', 'DeSitter']

EinsteinPy/Predefined Metrics in Symbolic Module.ipynb


Math 267 Project #3

---

## Solution

---

# Spring mass system.

The second order equation of an unforced spring/mass system with mass m, damping constant b and spring constant k is given by:

$$ mx''(t) + bx'(t) +kx(t) = 0. $$

Recall that $x(t)$ represents the displacement of the mass from its resting position ( with x>0 the spring is stretched and x<0 the spring is compressed). Therefore the velocity of the moving mass is $ x'(t)$ and the acceleration is $x’’(t)$. To approximate the solution for this system using Euler’s method we convert the equation into a system by introducing the variable

 $$ y = x’.$$

So our two unknowns are: $x$ for the position of the mass and $y$ for the velocity of the mass. As a system the second order equation becomes ( using m = 1)

\begin{align}
x'(t) &=  \; y(t) \\
y'(t) &= -k \cdot x(t) - b \cdot y(t)
\end{align}

and unlike the Predator-Prey model of exercise 2. this system is linear and can be represented in matrix notation:

\begin{align}
        \begin{pmatrix}
        x'(t) \\
        y'(t)
        \end{pmatrix}=
        \begin{pmatrix}
        0 & 1 \\
        -k & -b
        \end{pmatrix}\cdot
        \begin{pmatrix}
        x(t) \\
        y(t)
        \end{pmatrix}
    \end{align}

   ---

### Collaboration.  Students are allowed to work together on this project. Colaboration is encouraged.
 However you final submission must be your own work.

### Run the cell below to import the necessary libraries.

```python
# import libraries

import numpy as np
import matplotlib.pyplot as plt
from IPython.display import Image

# uncomment the line below if you are running a macbook
%config InlineBackend.figure_format ='retina'
```

## Exericse 1.

For this exercise you will approximate the solution of an unforced mass-spring system with
mass m =1, damping constant b = 1 and spring constant k = 2. The second order equation for this system is given by:

$$x''(t) + 1x'(t) + 2x(t) = 0.$$

Use the initial conditions $x(0) = 0$ and $ x'(0)=y(0) = 1$. Use the improved Euler’s method to solve the system. You will need to convert the second order equation to a system as demonstrated above. Choose a Δt=h of 0.1 and a time interval of 12 seconds. Create two plots. One showing x(t) versus time and the second showing the solution curve "trajectory" for the system in the xy (phase) plane.

## Complete the code cell below to solve the sytstem.

```python
# define the slope functions

def f(x,y):
    return y

def g(x,y):
    return -1*y -2*x

h = 0.1  # set delta t
t = np.arange(0,12,h)

# Initialeze arrays to store the results.

x = np.zeros_like(t)
y = np.zeros_like(t)

# Set initial conditions:

x[0] = 0
y[0] = 1

# implement Euler's method

for i in range (len(t)-1):

    F1 = f(x[i],y[i])
    G1 = g(x[i],y[i])

    F2 = f(x[i]+F1*h,y[i]+G1*h)
    G2 = g(x[i]+F1*h,y[i]+G1*h)

    slope1 = (F1+F2)/2
    slope2 = (G1+G2)/2

    x[i+1] = x[i] + slope1 * h
    y[i+1] = y[i] + slope2 * h

```

### Execute the cell below to graph your results.
You should see a damped sinusoid.

```python
plt.figure(figsize=(8,5))
plt.plot(t,x)
plt.grid()
plt.xlabel('t')
plt.ylabel('x');
plt.title("Damping b = 1");
```

### Execute the cell below to see the trajectory in the phase plane for this problem.
You should see a spiral.

```python
plt.figure(figsize=(8,5))
plt.plot(x,y,linewidth=2)
plt.xlim(-1.5,1.5)
plt.ylim(-1.5,1.5)
plt.grid()
plt.xlabel('x-displacement')
plt.ylabel('y-velocity');
plt.title("Trajectory for spring mass system b = 1");
```

### Repeat the steps above for b=3 and b=0. Copy and paste cells above and make the necessary modifications.  You should show time plots and phase plane plots for b=0 and b=3.

```python
# b = 0

# define the slope functions

def f(x,y):
    return y

def g(x,y):
    return  -2*x

h = 0.1  # set delta t
t = np.arange(0,12,h)

# Initialeze arrays to store the results.

x = np.zeros_like(t)
y = np.zeros_like(t)

# Set initial conditions:

x[0] = 0
y[0] = 1

# implement Euler's method

for i in range (len(t)-1):

    F1 = f(x[i],y[i])
    G1 = g(x[i],y[i])

    F2 = f(x[i]+F1*h,y[i]+G1*h)
    G2 = g(x[i]+F1*h,y[i]+G1*h)

    slope1 = (F1+F2)/2
    slope2 = (G1+G2)/2

    x[i+1] = x[i] + slope1 * h
    y[i+1] = y[i] + slope2 * h

```

```python
plt.figure(figsize=(8,5))
plt.plot(t,x)
plt.grid()
plt.xlabel('t')
plt.ylabel('x');
plt.xticks(np.arange(0,13,.5))
plt.title("Damping b = 0");
```

```python
plt.figure(figsize=(8,5))
plt.plot(x,y,linewidth=2)
plt.xlim(-1.5,1.5)
plt.ylim(-1.5,1.5)
plt.grid()
plt.xlabel('x-displacement')
plt.ylabel('y-velocity');
plt.title("Trajectory for spring mass system b = 0");
```

```python
# b=3

# define the slope functions

def f(x,y):
    return y

def g(x,y):
    return -3*y -2*x

h = 0.1  # set delta t
t = np.arange(0,12,h)

# Initialeze arrays to store the results.

x = np.zeros_like(t)
y = np.zeros_like(t)

# Set initial conditions:

x[0] = 0
y[0] = 1

# implement Euler's method

for i in range (len(t)-1):

    F1 = f(x[i],y[i])
    G1 = g(x[i],y[i])

    F2 = f(x[i]+F1*h,y[i]+G1*h)
    G2 = g(x[i]+F1*h,y[i]+G1*h)

    slope1 = (F1+F2)/2
    slope2 = (G1+G2)/2

    x[i+1] = x[i] + slope1 * h
    y[i+1] = y[i] + slope2 * h

```

```python
plt.figure(figsize=(8,5))
plt.plot(t,x)
plt.grid()
plt.xlabel('t')
plt.ylabel('x');
plt.title("Damping b = 3");
```

```python
plt.figure(figsize=(8,5))
plt.plot(x,y,linewidth=2)
plt.xlim(-1.5,1.5)
plt.ylim(-1.5,1.5)
plt.grid()
plt.xlabel('x-displacement')
plt.ylabel('y-velocity');
plt.title("Trajectory for spring mass system b = 3");
```

---
# Exercise 2.

For this exercise you will approximate the solution of an undamped periodically forced spring-mass system with mass m =1 and spring constant k = 1, no damping. The second order equation for this system is given by:

$$ x''(t) + x(t) = Cos(\omega t).$$

Use the initial conditions $x(0) = 0$ and $ x'(0) = y(0) = 0$. Use the improved Euler’s method to solve the system. Note this system is not autonomous so the slope functions could depend on x,y and t.  Generate plots of $x(t)$ versus time for ω = 1.1 and ω = 1. Use a time interval of 50 seconds for $ \omega = 1 $ and 150 seconds for $ \omega = 1.1$. For both cases set Δt=0.1. Duplicate and modify the code above to generate the plots.  Label the plots appropriately. There are no phase plane plots for Exercise 2.

## Solution for $ \omega = 1. $

```python
# define the slope functions

def f(x,y):
    return y

def g(x,y,t):
    return  -1*x + np.cos(t)

h = 0.1  # set delta t
t = np.arange(0,50,h)

# Initialeze arrays to store the results.

x = np.zeros_like(t)
y = np.zeros_like(t)

# Set initial conditions:

x[0] = 0
y[0] = 0

# implement Euler's method

for i in range (len(t)-1):

    F1 = f(x[i],y[i])
    G1 = g(x[i],y[i],t[i])

    F2 = f(x[i]+F1*h,y[i]+G1*h)
    G2 = g(x[i]+F1*h,y[i]+G1*h,t[i+1])

    slope1 = (F1+F2)/2
    slope2 = (G1+G2)/2

    x[i+1] = x[i] + slope1 * h
    y[i+1] = y[i] + slope2 * h

```

```python
plt.figure(figsize=(8,5))
plt.plot(t,x)
plt.grid()
plt.xlabel('t')
plt.ylabel('x');
plt.title("Forced Spring-Mass system no damping $\omega = 1.$");
```

## Solution for $ \omega = 1.1 $

```python
# define the slope functions

def f(x,y):
    return y

def g(x,y,t):
    return  -1*x + np.cos(1.1*t)

h = 0.1  # set delta t
t = np.arange(0,150,h)

# Initialeze arrays to store the results.

x = np.zeros_like(t)
y = np.zeros_like(t)

# Set initial conditions:

x[0] = 0
y[0] = 0

# implement Euler's method

for i in range (len(t)-1):

    F1 = f(x[i],y[i])
    G1 = g(x[i],y[i],t[i])

    F2 = f(x[i]+F1*h,y[i]+G1*h)
    G2 = g(x[i]+F1*h,y[i]+G1*h,t[i+1])

    slope1 = (F1+F2)/2
    slope2 = (G1+G2)/2

    x[i+1] = x[i] + slope1 * h
    y[i+1] = y[i] + slope2 * h

```

```python
plt.figure(figsize=(12,5))
plt.plot(t,x)
plt.grid()
plt.xlabel('t')
plt.ylabel('x');
plt.title("Forced Srping-Mass system no damping $\omega = 1.1$");
```

## Answer the questions below.  Create a text cell to enter your answers.
1. Expain and discuss the results of exercise 1.

    For exercise \#1 we see the spring behaviour under three damping conditions.

    b = 0 ~ we see no damping and the spring oscillates.

    b = 1 ~ we see an underdamped behaviour and the oscillation decays.

    b = 3 ~ we see overdamped and the spring returns to equilibrium with no oscillations.

2. Compute the period of oscillation for exerice 1. b=0.  Compare to value obtained from the graph.

        The period is 2*pi/omega = sqrt(2)*pi = 4.44.  This is close to the value from the graph.

3. Discuss the results of exercise 2.  Hint: look at the envelopes.

        For omega = 1, we are exciting the system at the resonant frequency and thus see RESONANCE, that is the osicllation are increasing.

        For omega = 1.1, the exciting frequencey is "close" to the resonant frequency and we observe the BEATING phenomenon.

```python

```


# Modeling and Simulation in Python

SymPy code for Chapter 16

Copyright 2017 Allen Downey

License: [Creative Commons Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0)

### Mixing liquids

We can figure out the final temperature of a mixture by setting the total heat flow to zero and then solving for $T$.

```python
from sympy import *

init_printing()
```

```python
C1, C2, T1, T2, T = symbols('C1 C2 T1 T2 T')

eq = Eq(C1 * (T - T1) + C2 * (T - T2), 0)
eq
```

```python
solve(eq, T)
```

### Analysis

We can use SymPy to solve the cooling differential equation.

```python
T_init, T_env, r, t = symbols('T_init T_env r t')
T = Function('T')

eqn = Eq(diff(T(t), t), -r * (T(t) - T_env))
eqn
```

Here's the general solution:

```python
solution_eq = dsolve(eqn)
solution_eq
```

```python
general = solution_eq.rhs
general
```

We can use the initial condition to solve for $C_1$.  First we evaluate the general solution at $t=0$

```python
at0 = general.subs(t, 0)
at0
```

Now we set $T(0) = T_{init}$ and solve for $C_1$

```python
solutions = solve(Eq(at0, T_init), C1)
value_of_C1 = solutions[0]
value_of_C1
```

Then we plug the result into the general solution to get the particular solution:

```python
particular = general.subs(C1, value_of_C1)
particular
```

We use a similar process to estimate $r$ based on the observation $T(t_{end}) = T_{end}$

```python
t_end, T_end = symbols('t_end T_end')
```

Here's the particular solution evaluated at $t_{end}$

```python
at_end = particular.subs(t, t_end)
at_end
```

Now we set $T(t_{end}) = T_{end}$ and solve for $r$

```python
solutions = solve(Eq(at_end, T_end), r)
value_of_r = solutions[0]
value_of_r
```

We can use `evalf` to plug in numbers for the symbols.  The result is a SymPy float, which we have to convert to a Python float.

```python
subs = dict(t_end=30, T_end=70, T_init=90, T_env=22)
r_coffee2 = value_of_r.evalf(subs=subs)
type(r_coffee2)
```

    sympy.core.numbers.Float

```python
r_coffee2 = float(r_coffee2)
r_coffee2
```

```python

```


```python
%matplotlib inline
from IPython.display import Markdown, display
import numpy as np
import matplotlib.dates as mdates
import matplotlib.pyplot as plt
import time
from scipy.stats import norm
from sympy import Symbol, symbols, Matrix, sin, cos, integrate, diff, Q, refine, simplify, factor, expand_trig, exp, latex, Integral
from sympy import init_printing
from sympy.utilities.codegen import codegen
init_printing(use_latex=True)
```

# Motion models
In tracking we can use different motion models. Here, we will derive a couple of those:

## CATR - constant acceleration and turn rate model
This model assumes the state $[x, y, \theta, v, \omega, a]^\top$, where $x$ and $y$ are coordinates of the object, $\theta$ is it CCW orientation, $v$ is its speed along the vector spawned by $\theta$, $\omega$ is the angular velocity (aka turn rate), and $a$ is the acceleration along the vector spawned by $\theta$. Furthermore, we assume that the object has constant acceleration $a$ and constant turn rate $\omega$. This is a useful model to model differential-drive systems.

### Derivation:

```python
speed, yaw, dt, x, y, acceleration, t, d_sin, d_cos = symbols('v \\theta \Delta{t} x y a t d_{sin} d_{cos}')
turn_rate_non_zero = Symbol('\hat{\omega}', nonzero=True, finite=True)
zero_turn_rate = Symbol('\omega_0', zero=True)
turn_rate = Symbol('\omega')

state_catr = Matrix([x, y, yaw, speed, turn_rate, acceleration])
print("CART state is:")
display(state_catr)

def get_next_state(turn_rate):
    # Specify the functions for yaw, speed, x, and y.
    yaw_func = yaw + turn_rate * t
    speed_func = speed + acceleration * t
    x_speed_func = speed_func * cos(yaw_func)
    y_speed_func = speed_func * sin(yaw_func)

    # Get next state by integrating the functions.
    next_speed = speed + integrate(acceleration, (t, 0, dt))
    next_yaw = yaw + integrate(turn_rate, (t, 0, dt))
    next_x = x + integrate(x_speed_func, (t, 0, dt))
    next_y = y + integrate(y_speed_func, (t, 0, dt))

    return Matrix([next_x, next_y, next_yaw, next_speed, turn_rate, acceleration])

# There is a difference in computation betwee the cases when the turn rate is allowed to be zero or not
print("Assuming a non-zero turn rate, the next state is:")
next_state = get_next_state(turn_rate_non_zero)
display(next_state)

substitutes = {x:42, y:23, yaw:0.5, speed:2, acceleration:2, dt:0.1, turn_rate_non_zero:2, zero_turn_rate:0}
display('Plugging in the numbers:', substitutes)
display(next_state.evalf(subs=substitutes))

state = Matrix([x,y,yaw,speed,turn_rate_non_zero,acceleration])
print("Jacobian of the next state with respect to the previous state:")
J = next_state.jacobian(state)
display(J)
display('Plugging in the numbers:', substitutes)
display(J.evalf(subs=substitutes))

print("Assuming a zero turn rate, state is:")
next_state = get_next_state(zero_turn_rate)
display(next_state)
display('Plugging in the numbers:', substitutes)
display(next_state.evalf(subs=substitutes))

state = Matrix([x,y,yaw,speed,zero_turn_rate,acceleration])
print("Jacobian of the next state with respect to the previous state with 0 turn rate:")
J = next_state.jacobian(state)
display(J)

display('Plugging in the numbers:', substitutes)
display(J.evalf(subs=substitutes))

```

## CVTR - constant velocity and turn rate model
This model assumes the state $[x, y, \theta, v, \omega]^\top$, where $x$ and $y$ are coordinates of the object, $\theta$ is it CCW orientation, $v$ is its speed along the vector spawned by $\theta$, $\omega$ is the angular velocity (aka turn rate). Furthermore, we assume that the object has constant speed $v$ and constant turn rate $\omega$. This is also a useful model to model differential-drive systems. It is a bit simpler than the CATR one.

### Derivation:

```python
speed, yaw, dt, x, y, t = symbols('v \\theta \Delta{t} x y t')
turn_rate_non_zero = Symbol('\omega', nonzero=True, finite=True)
zero_turn_rate = Symbol('\omega_0', zero=True)

state_cvtr = Matrix([x, y, yaw, speed, turn_rate_non_zero])
print("CVRT state is:")
display(state_cvtr)

def get_next_state(turn_rate):
    # Specify the functions for yaw, x, and y.
    yaw_func = yaw + turn_rate * t
    x_speed_func = speed * cos(yaw_func)
    y_speed_func = speed * sin(yaw_func)

    # Get next state by integrating the functions.
    next_yaw = yaw + integrate(turn_rate, (t, 0, dt))
    next_x = x + integrate(x_speed_func, (t, 0, dt))
    next_y = y + integrate(y_speed_func, (t, 0, dt))

    return Matrix([next_x, next_y, next_yaw, speed, turn_rate])

# There is a difference in computation betwee the cases when the turn rate is allowed to be zero or not
print("Assuming a non-zero turn rate, next state is:")
next_state = get_next_state(turn_rate_non_zero)
display(next_state)
substitutes = {x:42, y:23, yaw:0.5, speed:2, dt:0.1, turn_rate_non_zero:2, zero_turn_rate:0}
display(Markdown('Plugging in the numbers:'), substitutes)
display(next_state.evalf(subs=substitutes))

state = Matrix([x,y,yaw,speed,turn_rate_non_zero])
print("Jacobian of the next state with respect to the previous state:")
J = next_state.jacobian(state)
display(J)

display(Markdown('Plugging in the numbers:'), substitutes)
display(J.evalf(subs=substitutes))

print("Assuming a zero turn rate, next state is:")
next_state = get_next_state(zero_turn_rate)
display(next_state)
display(Markdown('Plugging in the numbers:'), substitutes)
display(next_state.evalf(subs=substitutes))

state = Matrix([x,y,yaw,speed,zero_turn_rate])
print("Jacobian of the next state with respect to the previous one with zero turn rate:")
J = next_state.jacobian(state)
display(J)

display(Markdown('Plugging in the numbers:'), substitutes)
display(J.evalf(subs=substitutes))
```


# Ejercicios de canales de comunicación

## Ejercicio 1

La fuente de entrada a un canal de comunicación ruidoso es una variable aleatoria $X$ que contiene los símbolos $\{a, b, c, d\}$. La salida de este canal es una variable aleatoria $Y$ sobre estos mismos cuatro símbolos. La distribución conjunta de estas dos variables aleatorias es la siguiente:

Para hacer más sencillos los cálculos, se le aplicará una transposición a la matriz para que las distribuciones de $X$ queden como filas y las de $Y$ como columnas:

```python
symbols = ('a', 'b', 'c', 'd')

import numpy as np

matrix = np.array([
    [1/8, 1/16, 1/16, 1/4],
    [1/16, 1/8, 1/16, 0],
    [1/32, 1/32, 1/16, 0],
    [1/32, 1/32, 1/16, 0]
])
joint_distribution = matrix.transpose()
joint_distribution
```

    array([[0.125  , 0.0625 , 0.03125, 0.03125],
           [0.0625 , 0.125  , 0.03125, 0.03125],
           [0.0625 , 0.0625 , 0.0625 , 0.0625 ],
           [0.25   , 0.     , 0.     , 0.     ]])

Para calcular la información mutua (en bits) de ambas variables aleatorias usaremos la siguiente fórmula, la cual toma en cuenta las distribuciones marginales de cada variable aleatoria y las conjuntas de ambas:

$$
I(X;Y) = \sum_{x \in X}{ \sum_{y \in Y}{
    p(x,y) \log_{2}{ \frac{ p(x,y) }{ p(x)p(y) }}
}}
$$

La distribución marginal de $X$, $p(X)$ es:

```python
def get_x_marginal(joint_distribution):
    # sumar toda la fila
    return [sum(x) for x in joint_distribution]

x_marginal = get_x_marginal(joint_distribution)
f'p(X) = {x_marginal}'
```

    'p(X) = [0.25, 0.25, 0.25, 0.25]'

La distribución marginal de $Y$, $p(Y)$ es:

```python
def get_y_marginal(joint_distribution):
    # sumar cada columna
    return [sum(x[y] for x in joint_distribution) for y in range(len(joint_distribution))]

y_marginal = get_y_marginal(joint_distribution)
f'p(Y) = {y_marginal}'
```

    'p(Y) = [0.5, 0.25, 0.125, 0.125]'

Ahora pasaremos a codificar una función que nos permita calcular la información mutua a partir de las distribuciones marginales:

```python
from math import log
from itertools import chain, cycle

def get_mutual_information(joint_distribution, x_marginal, y_marginal, base=2):
    return sum(
        p_xy * log(p_xy / (p_x * p_y), base)
        for p_xy, p_x, p_y in zip(
            # aplanar matriz para iterar sobre cada valor
            chain.from_iterable(joint_distribution),
            # repetir marginales porque no tienen el mismo tamaño que la matriz aplanada
            cycle(x_marginal),
            cycle(y_marginal)
        )
        if p_xy > 0
    )

mutual_info = get_mutual_information(joint_distribution, x_marginal, y_marginal)
f'I(X;Y) = {mutual_info} bits'
```

    'I(X;Y) = 0.375 bits'

## Ejercicio 2

Vamos a considerar un canal de comunicación binario simétrico, cuya fuente de entrada es el alfabeto $X = \{0, 1\}$ con probabilidades $\{0.5, 0.5\}$, cuyo alfabeto de salida es $Y = \{0, 1\}$, y cuya matriz de canales es:

\begin{equation}
    \begin{pmatrix}
        1 - \epsilon && \epsilon \\
        \epsilon && 1 - \epsilon
    \end{pmatrix}
\end{equation}

donde $\epsilon$ es la probabilidad de transmisión de error. Para este caso vamos a suponer que $\epsilon = 0.35$, por lo que la matriz del canal es:

```python
error = 0.35
channel_matrix = [
    [1 - error, error],
    [error, 1 - error]
]
np.array(channel_matrix)
```

    array([[0.65, 0.35],
           [0.35, 0.65]])

Y la distribución conjunta, $p(X,Y)$:

```python
x_probabilities = (0.5, 0.5)
joint_distribution = [
    [p_x * col for p_x, col in zip(x_probabilities, row)]
    for row in channel_matrix
]
np.array(joint_distribution)
```

    array([[0.325, 0.175],
           [0.175, 0.325]])

Finalmente podemos calcular la información mutua del canal, ya que usaremos la fórmula del ejercicio anterior y ya contamos con la distribución conjunta. Debido a que nosotros calculamos $p(X,Y)$, las distribuciones marginales para este caso son las probabilidades dadas por el problema, es decir $\{0.5, 0.5\}$:

```python
i = get_mutual_information(joint_distribution, x_probabilities, x_probabilities)
f'I(X;Y) = {i} bits'
```

    'I(X;Y) = 0.06593194462450899 bits'


I'm a little confused by lognormal distributions.

Let's say I have a parameter that is distributed as:

\begin{equation}
V \sim \mathcal{N}(5, 2) \, ,
\end{equation}

how would I represent this in a lognormal distribution so it can't fall below zero? A lognormal distribution is defined as:

\begin{equation}
\ln(V) \sim \ln(\mathcal{N}(5, 2)) \, ,
\end{equation}

> In probability theory, a lognormal distribution is a ... probability distribution o a random variable who's *logarithm is normally distributed*.

> Thus if $X$ is log-normally distributed, then $Y - \ln(X)$ is normally distributed.

```python
import numpy as np
import seaborn as sns
import pylab as plt
```

```python
npts = 5000
mu = 10
sig = 2
v = np.random.normal(mu, sig, npts)
```

```python
sns.distplot(v)
plt.xlabel('V')
plt.axvline(mu)
```

This extends below 0, which we don't want it to do. What does the logarithm look like?

```python
sns.distplot(np.log(v))
plt.xlabel(r'$\ln(V)$')
plt.axvline(np.log(mu))
plt.show()
```

If we wanted a lognormal distribution to recapture this same distribution, should we be using

\begin{equation}
\ln(V) \sim \log\mathcal{N}(\ln(10), \ln(2)) \, ?
\end{equation}

```python
lnv = np.random.lognormal(np.log(mu), np.log(2), npts)
```

```python
sns.distplot(lnv, label='LogNormal')
sns.distplot(np.log(v), label='ln(V)')
plt.legend()
```

Okay, these don't line up...

```python
sns.distplot(lnv, label='LogNormal')
sns.distplot(v, label='V')
```

I guess this only truly works if the logarithm of V is normally distributed, but in this case we want it to be the other way round?

```python
import mystyle as ms
with plt.style.context(ms.ms):
    sns.distplot(v)
    sns.distplot(lnv)
    plt.axvline(np.median(v), c='r', label='Median V')
    plt.axvline(np.median(lnv), c='g', label='Median LnV')
    plt.axvline(np.mean(lnv), c='b', label='Mean LnV')
    plt.legend()
```

In both cases the median values intersect at $\mu$, but with $V$ distributed lognormally, it can not fall below zero.

Let's try a more realistic example.

```python
npts = 10000
mu = 0.7
sigma = .1
v = np.random.normal(mu, sigma, npts)
lnv = np.random.lognormal(np.log(mu), sigma, npts)
```

```python
with plt.style.context(ms.ms):
    sns.distplot(v, label='Normally distributed')
    sns.distplot(lnv, label='Lognormally distributed')
    plt.legend()
print(f'Median of Normal: {np.median(v)}')
print(f'Median of LogNormal: {np.median(lnv)}')
```

This example indicates that $\sigma$ should remain as is.

The equations seem to agree. Let's check our previous example:

```python
npts = 10000
mu = 10.
sigma = 2.
v = np.random.normal(mu, sigma, npts)
lnv = np.random.lognormal(np.log(mu), sigma, npts)
```

```python
with plt.style.context(ms.ms):
    sns.distplot(v, label='Normally distributed')
    sns.distplot(lnv, label='Lognormally distributed')
    plt.legend()
print(f'Median of Normal: {np.median(v)}')
print(f'Median of LogNormal: {np.median(lnv)}')
```

Yep, the plots are unsightly but otherwise okay.

Conclusions: our parameters that can't go below zero should be distributed as

\begin{equation}
V \sim \rm{LogNormal}(\ln(\mu), \sigma)
\end{equation}


#
Números complejos

```python
from sympy import *
init_printing()
x = symbols('x')
```

Python puede trabajar de modo nativo con los números complejos. Para construir un número complejo se puede utilizara la función **complex** o bien escribir el número complejo en forma binómica, teniendo en cuenta que la unidad imaginaria se denota por **j** en Python. Sin embargo nosotros vamos a trabajar con la biblioteca Sympy y en ella la unidad imaginaria se denota con la letra **I** mayúscula.

###
Calcula la raíz cuadrada de -1 y resuelve la ecuación $x^2+1=0$.

```python

```

```python

```

```python

```

###
Dados los números complejos $z=3+5i$ y $w=3-2i$, calcula su suma, su resta, multiplicación, división, inverso y potencias.

```python

```

```python

```

```python

```

```python

```

Para calcular el módulo de un número complejo empleamos **abs**, para obtener el argumento (en radianes) **arg**, para el conjugado **conj**, y para las partes real e imaginaria **re** e **im**. Con el módulo y el argumento podemos ya escribir el número en forma polar.

###
Calcula el módulo, el argumento, el conjugado, la parte real e imaginaria de $w$.

```python

```

```python

```

Si un número complejo $z$ tiene módulo $m$ y argumento $\theta$ (en radianes), entonces, **según la fórmula de Euler**, el número se puede escribir como $z = m\cdot exp(i\theta)$. También se puede escribir como $z= m\cos(\theta) + i m\sin(\theta)$.

###
Utilizando la fórmula de Euler, construye el número complejo $5_{30^0}$ y calcula el conjugado, la parte real, la imaginaria y el módulo.

```python

```

```python

```

```python

```

###
Calcula distintas raíces del número $3+5i$, comprobando el resultado.

```python

```

```python

```

```python

```


# PRAKTIKUM 8
`Curve Fitting / Data Fitting / Regresi`

# 1 Regresi Linear
 Koefisien garis regresi linear
\begin{equation}
y=Ax+B
\end{equation}
merupakan solusi dari sistem persamaan linear berikut, yang disebut _normal equations_.
\begin{align}
\begin{split}
\left( \sum_{k=1}^{N}{x_k^2} \right) A + \left( \sum_{k=1}^{N}{x_k} \right) B &= \sum_{k=1}^{N}{x_ky_k}\\
\left( \sum_{k=1}^{N}{x_k} \right) A + NB &= \sum_{k=1}^{N}{y_k}
\end{split}
\end{align}
solusi sistem tersebut adalah
\begin{align}
A &= \frac{\sum_{k=1}^{N}{(x_k-\bar{x})(y_k-\bar{y})}}{\sum_{k=1}^{N}{(x_k-\bar{x})^2}} \\
B &= \bar{y}-A\bar{x}
\end{align}

```julia
using Statistics
```

```julia
function reglin(X,Y)
    # Hitung nilai rataan data x dan y
    xmean = mean(X);
    ymean = mean(Y);
    # Hitung nilai jumlah dari xy dan x^2
    sumxy = (X.-xmean)'*(Y.-ymean)
    sumx2 = (X.-xmean)'*(X.-xmean)
    # Hitung nilau koefisien garis regresi linear Y=Ax+B
    A = sumxy/sumx2;
    B = ymean .- A*xmean;
    return A,B
end
```

### Contoh 1:
Diberikan data seperti berikut.

|x||-1 | 0 | 1 | 2 | 3 | 4 | 5 | 6 |
|-||  -|  -|  -|  -|  -|  -|  -|  -|
|y||10 | 9 | 7 | 5 | 4 | 3 | 0 | -1 |

Cari dan gambarkan garis regresi tersebut

```julia
x = [-1, 0, 1, 2, 3, 4, 5, 6]
y = [10, 9, 7, 5, 4, 3, 0, -1];
```

```julia
A,B = reglin(x,y)
```

Persamaan Regresi $ y = -1.6071 x + 8.6429 $

```julia
using Plots
```

```julia
f(x) = A*x+B;
xk = -1:0.1:6
plt1 = plot(xk,f.(xk),label = "Garis Regresi")
scatter!(x,y,label = "Data")
```

```julia
rmse(yk,yduga) = sqrt(mean((yk.-yduga).^2));
yduga = f.(x);
galat = rmse(y,yduga)
```

#### Cara Lain:

```julia
P = [x ones(length(x),1)]
```

```julia
P'*P   # Matriks Koefisien dari SPL
P'*y  # Ruas Kanan SPL
```

```julia
C = inv(P'*P)*(P'*y) # Solusi SPL
```

```julia
f(x) = C[1]*x+C[2];
xk = -1:0.1:6
plot(xk,f.(xk),label = "Garis Regresi")
scatter!(x,y,label = "Data")
```

# 2 Regresi Pangkat
Koefisien $ A $ dari fungsi pangkat $$ y=Ax^M $$ adalah
\begin{equation}\label{eq:8 pangkat}
A=\frac{ \sum_{k=1}^{N}{x_k^My_k}}{ \sum_{k=1}^N{x_k^{2M}}}
\end{equation}

```julia
function regpower(X,Y,m)
  # Hitung nilai jumlah dari x^m*y dan x^2m
  sumxy = (X.^m)'*Y
  sumx2 = (X.^m)'*(X.^m)
  # Hitung nilau koefisien garis regresi pangkat y = Ax^m
  A = sumxy/sumx2
end
```

### Contoh 2:
Pada suatu praktikum fisika, seorang mahasiswa mendapatkan kumpulan data percobaan seperti berikut.

|	Waktu, t_k | Jarak,  d_k |
|-|-|
|	0.2 | 0.1960|
|	0.4 | 0.7850|
|	0.6 | 1.7665|
|	0.8 | 3.1405|
|	1.0 | 4.9075|

   Hubungan dari data tersebut adalah $ d=\frac{1}{2}gt^2 $ dengan $ d $ merupakan jarak dalam meter dan $ t $ merupakan waktu dalam detik. Carilah koefisien gravitasi $ g $.

```julia
tk = [0.2,0.4,0.6,0.8,1.0];
dk = [0.1960,0.7850,1.7665,3.1405,4.9075];
```

Didefinisikan $A = 1/2 g$

```julia
A = regpower(tk,dk,2)
```

```julia
g = 2*A
```

#### Cara Lain

```julia
P = tk.^2
```

```julia
A = inv(P'*P)*(P'*dk) # Solusi SPL
```

# 3 Regresi Non-Linear
## Regresi Eksponensial
Misalkan bahwa diberikan titik $ (x_1,y_1) $, $ (x_2,y_2) $, $\dots$, $ (x_N,y_N) $ dan ingin dicocokan dengan grafik eksponensial dengan bentuk
\begin{equation}\label{eq:8 eks}
y = Ce^{Ax}
\end{equation}
Langkah pertama adalah me-logaritma-kan kedua sisi, sehingga diperoleh
\begin{align}
\begin{split}
&\ln(y)=\ln(Ce^{Ax})\\
\Leftrightarrow\ &\ln(y)=\ln(e^{Ax})+\ln(C)\\
\Leftrightarrow\ &\ln(y)=Ax+\ln(C)
\end{split}
\end{align}
Selanjutnya, perubahan peubah didefinisikan sebagai berikut.
\begin{align}\label{eq:8 eks 1}
Y=\ln(y),\ \ \ \ X=x,\ \ \ \ \text{dan}\ \ \ \ B=\ln(C)
\end{align}

### Contoh 3:
Gunakan metode linearisasi data untuk mencari kurva regresi eksponensial $$ y=Ce^{Ax} $$ untuk 5 titik data, yaitu $ (0,1.5) $, $ (1,2.5) $, $ (2,3.5) $, $ (3,5.0) $, dan $ (4,7.5) $.

```julia
x = [  0,   1,   2,   3,   4]
y = [1.5, 2.5, 3.5, 5.0, 7.5]
Y = log.(y);
```

```julia
A,B = reglin(x,Y)
C = exp(B)
(A,C)
```

```julia
f(x) = C*exp(A*x);
xk = -1:0.1:5
plot(xk,f.(xk),label = "Garis Regresi",legend = :topleft)
scatter!(x,y,label = "Data")
```

# 4 Regresi Polinomial
## Regresi Parabola
Koefisien dari garis regresi parabola
\begin{equation}\label{eq:8 para}
y=Ax^2+Bx+C
\end{equation}
merupakan nilai solusi $ A $, $ B $, dan $ C $ dari sistem linear
\begin{align}\label{eq:8 para1}
\begin{split}
\left( \sum_{k=1}^{N}x_k^4 \right) A +
\left( \sum_{k=1}^{N}x_k^3 \right) B +
\left( \sum_{k=1}^{N}x_k^2 \right) C &=
\sum_{k=1}^{N}y_kx_k^2\\
\left( \sum_{k=1}^{N}x_k^3 \right) A +
\left( \sum_{k=1}^{N}x_k^2 \right) B +
\left( \sum_{k=1}^{N}x_k \right) C &=
\sum_{k=1}^{N}y_kx_k\\
\left( \sum_{k=1}^{N}x_k^2 \right) A +
\left( \sum_{k=1}^{N}x_k \right) B +
N C &=
\sum_{k=1}^{N}y_k
\end{split}
\end{align}

Dalam notasi matriks, ruas kanan dari sistem linear akan setara dengan nilai dari $ F^TY $, yaitu
\begin{align}
F^TY
=\begin{bmatrix}
&&&&\\[-0.5em]
(x_1)^0 & (x_2)^0 & (x_3)^0 & \dots & (x_N)^0 \\[0.5em]
(x_1)^1 & (x_2)^1 & (x_3)^1 & \dots & (x_N)^1 \\[0.5em]
(x_1)^2 & (x_2)^2 & (x_3)^2 & \dots & (x_N)^2 \\[0.5em]
\end{bmatrix}
\begin{bmatrix}
y_1\\y_2\\y_3\\\vdots \\y_N
\end{bmatrix}
\end{align}
Sementara itu, ruas kiri dari sistem linear akan setara dengan nilai dari $ F^TF $, yaitu
\begin{align}
F^TF
=\begin{bmatrix}
&&&&\\[-0.5em]
(x_1)^0 & (x_2)^0 & (x_3)^0 & \dots & (x_N)^0 \\[0.5em]
(x_1)^1 & (x_2)^1 & (x_3)^1 & \dots & (x_N)^1 \\[0.5em]
(x_1)^2 & (x_2)^2 & (x_3)^2 & \dots & (x_N)^2 \\[0.5em]
\end{bmatrix}
\begin{bmatrix}
(x_1)^0 & (x_1)^1 & (x_1)^2 \\
(x_2)^0 & (x_2)^1 & (x_2)^2 \\
(x_3)^0 & (x_3)^1 & (x_3)^2 \\
\vdots  &  \vdots  &  \vdots  \\
(x_N)^0 & (x_N)^1 & (x_N)^2 \\
\end{bmatrix}
\end{align}
Dengan demikian, nilai koefisien regresi $ C $ dapat dicari dengan menyelesaikan sistem linear
\begin{equation}
(F^TF)\ C=F^TY
\end{equation}

## Regresi Polinomial berderajat m

```julia
function regpoly(X,Y,m)
  #% Hitung matriks F
  F = zeros(length(X),m+1)
  for k = 1: m+1
    F[:,k]=X.^(k-1);
  end
  #% Hitung matriks A dan matriks B serta koefisien regresi pangkat
  A = F'*F;
  B = F'*Y;
  C = A\B; # sama saja dengan inv(A)*B
  C = reverse(C,dims=1)
end
```

### Contoh 4:
Carilah persamaan regresi parabola dari empat titik data $ (-3,3) $, $ (0,1) $, $ (2,1) $, dan $ (4,3) $.

```julia
x = [-3, 0, 2, 4]
y = [ 3, 1, 1, 3]
C = regpoly(x,y,2)
```

```julia
f(x) = C[1]*x^2 + C[2]*x + C[3]
xk = -3:0.1:4
plt = plot(xk,f.(xk),label = "Garis Regresi",legend=:top)
scatter!(x,y,label = "Data")
```

```julia
rmse(yk,yduga) = sqrt(mean((yk.-yduga).^2));
yduga = f.(x);
galat = rmse(y,yduga)
```

#### Cara Lain:

```julia
P = [x.^2 x ones(length(x),1)]
```

```julia
C = inv(P'*P)*(P'*y)
```

# Soal Latihan
Kerjakan soal berikut pada saat kegiatan praktikum berlangsung.

`Nama: ________`

`NIM: ________`

### Soal 1
Ulangi langkah-langkah pada **Contoh 1** untuk membentuk garis regresi linear dari data berikut.

| $x_k$ | 0.0 |  1.0 |  2.0 |   4.0 |   5.0|
|-|-|-|-|-|-|
| $y_k$ | 3.0 |  5.1 |  6.9 |  10.0 |  13.0|

Gambarkan garis regresi beserta titik-titik data, kemudian hitung nilai RMSE dari garis regresi tersebut.

```julia

```

### Soal 2
Diberikan data hasil pengamatan seperti berikut.

| $d_k$ | 1.0 |  2.0 |  3.0 |   4.0 |   5.0|
|-|-|-|-|-|-|
| $t_k$ | 0.45 |  0.63 |  0.79 |  0.90 |  1.01 |

Hitunglah nilai koefisien grafitasi tempat pengamatan berdasarkan data tersebut dengan mengikuti cara pada **Contoh 2**.

```julia

```

### Soal 3

Ulangi langkah-langkah pada **Contoh 3** untuk membentuk garis regresi eksponensial dari data berikut.

| $x_k$ | 0.0 | 1.0 |  2.0 |  4.0 |  5.0|
|-|-|-|-|-|-|
| $y_k$ | 2.00 |  0.75 |  0.38 |  0.15 |  0.10 |

Gambarkan garis regresi beserta titik-titik data, kemudian hitung nilai RMSE dari garis regresi tersebut.

```julia

```

### Soal 4
Ulangi langkah-langkah pada **Contoh 4** untuk membentuk garis regresi kuadratik dari data berikut.

| $x_k$ | 0.0 | 1.0 |  2.0 |  4.0 |  5.0|
|-|-|-|-|-|-|
| $y_k$ | 7.10 |  3.00 |  3.60 |  11.00 |  24.00 |

Gambarkan garis regresi beserta titik-titik data, kemudian hitung nilai RMSE dari garis regresi tersebut.

```julia

```


# Symbolic mathematics with Sympy

[Sympy](http://www.sympy.org/en/index.html) is described as a:

> "... Python library for symbolic mathematics."

This means it can be used to:

- Manipulate symbolic expressions;
- Solve symbolic equations;
- Carry out symbolic Calculus;
- Plot symbolic function.

It has other capabilities that we will not go in to in this handbook. But you can read more about it here: http://www.sympy.org/en/index.html

## Manipulating symbolic expressions

Before we can start using the library to manipulate expressions, we need to import it.

```python
import sympy as sym
```

The above imports the library and gives us access to it's commands using the shortand `sym` which is conventially used.

If we wanted to get Python to check that $x - x = 0$ we would get an error if we did not tell Python what $x$ was:

```python
x - x
```

This is where Sympy comes in, we can tell Python to create $x$ as a symbolic variable:

```python
x = sym.symbols('x')
```

Now we can calculate $x - x$:

```python
x - x
```

    0

We can create and manipulate expressions in Sympy. Let us for example verify:

$$(a + b) ^ 2 = a ^ 2 + 2ab + b ^2$$

First, we create the symbolic variables $a, b$:

```python
a, b = sym.symbols('a, b')
```

Now let's create our expression:

```python
expr = (a + b) ** 2
expr
```

    (a + b)**2

**Note** we can get Sympy to use LaTeX so that the output looks nice in a notebook:

```python
sym.init_printing()
```

```python
expr
```

Let us expand our expression:

```python
expr.expand()
```

Note that we can also get Sympy to produce the LaTeX code for future use:

```python
sym.latex(expr.expand())
```

    'a^{2} + 2 a b + b^{2}'

---
**EXERCISE** Use Sympy to verify the following expressions:

- $(a - b) ^ 2 = a ^ 2 - 2 a b + b^2$
- $a ^ 2 - b ^ 2 = (a - b) (a + b)$ (instead of using `expand`, try `factor`)

## Solving symbolic equations

We can use Sympy to solve symbolic expression. For example let's find the solution in $x$ of the quadratic equation:

$$a x ^ 2 + b x + c = 0$$

```python
# We only really need to define `c` but doing them all again.
a, b, c, x = sym.symbols('a, b, c, x')
```

The Sympy command for solving equations is `solveset`. The first argument is an expression for which the root will be found. The second argument is the value that we are solving for.

```python
sym.solveset(a * x ** 2 + b * x + c, x)
```

---
**EXERCISE** Use Sympy to find the solutions to the generic cubic equation:

$$a x ^ 3 + b x ^ 2 + c  x + d = 0$$

---

It is possible to pass more arguments to `solveset` for example to constrain the solution space. Let us see what the solution of the following is in $\mathbb{R}$:

$$x^2=-1$$

```python
sym.solveset(x ** 2 + 1, x, domain=sym.S.Reals)
```

---
**EXERCISE** Use Sympy to find the solutions to the following equations:

- $x ^ 2 == 2$ in $\mathbb{N}$;
- $x ^ 3 + 2 x = 0$ in $\mathbb{R}$.

---

## Symbolic calculus

We can use Sympy to compute limits. Let us calculate:

$$\lim_{x\to 0^+}\frac{1}{x}$$

```python
sym.limit(1/x, x, 0, dir="+")
```

---
**EXERCISE** Compute the following limits:

1. $\lim_{x\to 0^-}\frac{1}{x}$
2.  $\lim_{x\to 0}\frac{1}{x^2}$

---

We can use also Sympy to differentiate and integrate. Let us experiment with differentiating the following expression:

$$x ^ 2 - \cos(x)$$

```python
sym.diff(x ** 2 - sym.cos(x), x)
```

Similarly we can integrate:

```python
sym.integrate(x ** 2 - sym.cos(x), x)
```

We can also carry out definite integrals:

```python
sym.integrate(x ** 2 - sym.cos(x), (x, 0, 5))
```

---

**EXERCISE** Use Sympy to calculate the following:

1. $\frac{d\sin(x ^2)}{dx}$
2. $\frac{d(x ^2 + xy - \ln(y))}{dy}$
3. $\int e^x \cos(x)\;dx$
4. $\int_0^5 e^{2x}\;dx$

## Plotting with Sympy

Finally Sympy can be used to plot functions. Note that this makes use of another Python library called [matplotlib](http://matplotlib.org/). Whilst Sympy allows us to not directly need to make use of matplotlib it could be worth learning to use as it's a very powerful and versatile library.

Before plotting in Jupyter we need to run a command to tell it to display the plots directly in the notebook:

```python
%matplotlib inline
```

Let us plot $x^2$:

```python
expr = x ** 2
p = sym.plot(expr);
```

We can directly save that plot to a file if we wish to:

```python
p.save("x_squared.pdf");
```

---
**EXERCISE** Plot the following functions:

- $y=x + cos(x)$
- $y=x ^ 2 - e^x$ (you might find `ylim` helpful as an argument)

Experiment with saving your plots to a file.

---

## Summary

This section has discussed using Sympy to:

- Manipulate symbolic expressions;
- Calculate limits, derivates and integrals;
- Plot a symbolic expression.

This just touches the surface of what Sympy can do.

Let us move on to using [Numpy](02 - Linear algebra with Numpy.ipynb) to do Linear Algebra.


## Eigenvalues and Eigenvectors

### Learning to obtain eigenvalues and eigenvectors using python.

#### [4,3,2],[1,4,1],[3,10,4]

```
import numpy as np
I=np.array([[4,3,2],[1,4,1],[3,10,4]])
print("#### input matrix")
print(I)
print()
x=np.linalg.eigvals(I)
y=np.linalg.eig(I)
print("#### Eigen values")
print(x)
print()
print("#### Eigen vectors")
print(y)
```

    #### input matrix
    [[ 4  3  2]
     [ 1  4  1]
     [ 3 10  4]]

    #### Eigen values
    [8.98205672 2.12891771 0.88902557]

    #### Eigen vectors
    (array([8.98205672, 2.12891771, 0.88902557]), array([[-0.49247712, -0.82039552, -0.42973429],
           [-0.26523242,  0.14250681, -0.14817858],
           [-0.82892584,  0.55375355,  0.89071407]]))

#### [1,-3,3],[3,-5,3],[6,-6,4]

```
import numpy as np
I=np.array([[1,-3,3],[3,-5,3],[6,-6,4]])
print("#### input matrix")
print(I)
print()
x=np.linalg.eigvals(I)
y=np.linalg.eig(I)
print("#### Eigen values")
print(x)
print()
print("#### Eigen vectors")
print(y)
```

    #### input matrix
    [[ 1 -3  3]
     [ 3 -5  3]
     [ 6 -6  4]]

    #### Eigen values
    [ 4.+0.00000000e+00j -2.+1.10465796e-15j -2.-1.10465796e-15j]

    #### Eigen vectors
    (array([ 4.+0.00000000e+00j, -2.+1.10465796e-15j, -2.-1.10465796e-15j]), array([[-0.40824829+0.j        ,  0.24400118-0.40702229j,
             0.24400118+0.40702229j],
           [-0.40824829+0.j        , -0.41621909-0.40702229j,
            -0.41621909+0.40702229j],
           [-0.81649658+0.j        , -0.66022027+0.j        ,
            -0.66022027-0.j        ]]))

### Properties of eigenvalues and eigenvectors:
1. For a nxn matrix, the number of eigen values is n.
2. The sum of eigen values is equal to the sum of the diagonal elements of matrix.
3. The product of eigenvalues is equal to the determinant of the matrix.
4. The eigen value for an identity matrix is 1.
5. The eigenvalue of a triangular matrix is same as the diagonal elements of a matrix.
6. For a skew symmetric matrix, the eigenvalues are imaginary.
7. For orthogonal matrix eigenvalues is 1.(Orthogonal matrix---> A.(A)ˆt=I).
8. For indempotent matrix the eigenvalues are 0 and 1(Aˆ2=identity matrix).

```
import math
from math import *
import numpy as np
from numpy import *
```

#### property 2

```
A=array([[1,2,3],[2,3,5],[3,1,1]])
print("#### Input matrix")
print(A)
print()
X=A[0][0]+A[1][1]+A[2][2]
print("#### X")
print(X)
print()
Y=sum(linalg.eigvals(A))
Z=round(Y)
print("#### Z")
print(Z)
print()
print("The sum of eigen values is equal to the sum of the diagonal elements of matrix")
equal(X,Z)
```

    #### Input matrix
    [[1 2 3]
     [2 3 5]
     [3 1 1]]

    #### X
    5

    #### Z
    5

    The sum of eigen values is equal to the sum of the diagonal elements of matrix

    True

Property 2 is verified.

#### property 3

```
B=array([[1,2,3],[1,3,5],[4,1,2]])
print("#### B")
print(B)
print()
M=linalg.det(B)
print("#### M")
print(M)
print()
Q=prod(linalg.eigvals(B))
P=np.round(Q)
print("#### P")
print(P)
print()
print("The product of eigenvalues is equal to the determinant of the matrix.")
equal(P,M)
```

    #### B
    [[1 2 3]
     [1 3 5]
     [4 1 2]]

    #### M
    4.000000000000002

    #### P
    (4+0j)

    The product of eigenvalues is equal to the determinant of the matrix.

    False

Property 3 is verified.

#### Property 4

```
I=array([[1,0,0],[0,1,0],[0,0,1]])
print("#### Input matrix")
print(I)
print()
print("The eigen value for an identity matrix is 1")
linalg.eigvals(I)
```

    #### Input matrix
    [[1 0 0]
     [0 1 0]
     [0 0 1]]

    The eigen value for an identity matrix is 1

    array([1., 1., 1.])

Property 4 is verified.

#### Property 5

```
T=array([[4,0,0],[2,3,0],[1,2,3]])
print("#### Input matrix")
print(T)
print()
print("The eigenvalue of a triangular matrix is same as the diagonal elements of the matrix.")
linalg.eigvals(T)
```

    #### Input matrix
    [[4 0 0]
     [2 3 0]
     [1 2 3]]

    The eigenvalue of a triangular matrix is same as the diagonal elements of the matrix.

    array([3., 3., 4.])

Property 5 is verified

#### Property 6

```
E=(B-B.transpose())/2
print("#### skew matrix")
print(E)
print()
print("For a skew symmetric matrix, the eigenvalues are imaginary.")
linalg.eigvals(E)
```

    #### skew matrix
    [[ 0.   0.5 -0.5]
     [-0.5  0.   2. ]
     [ 0.5 -2.   0. ]]

    For a skew symmetric matrix, the eigenvalues are imaginary.

    array([0.+0.j        , 0.+2.12132034j, 0.-2.12132034j])

Property 6 is verified.

#### Property 7

```
F=array([[1,0],[0,-1]])
print("#### orthogonal matrix")
print(F)
print()
print("For orthogonal matrix eigenvalues is 1.(Orthogonal matrix---> A.(A)ˆt=I).")
linalg.eigvals(F)
```

    #### orthogonal matrix
    [[ 1  0]
     [ 0 -1]]

    For orthogonal matrix eigenvalues is 1.(Orthogonal matrix---> A.(A)ˆt=I).

    array([ 1., -1.])

Property 7 verified.

### Diagonalization of sqaure matrix

```
import numpy as np
from math import *
A= np.mat([[2,-2,3],[1,1,1],[1,3,-1]])
X,P=np.linalg.eig(A)
I=np.linalg.inv(P)
Z=np.around(I*A*P)
for i in range(len(Z)):
    for j in range(len(Z)):
        if Z[i,j]==-0:
            Z[i,j]=0
print("The final diagonalized matrix is")
print(Z)
print()

print("Eigen vectors")
print(P)
print()

print("Eigen values")
print(X)
print()
```

    The final diagonalized matrix is
    [[ 3.  0.  0.]
     [ 0.  1.  0.]
     [ 0.  0. -2.]]

    Eigen vectors
    [[ 0.57735027  0.57735027 -0.61684937]
     [ 0.57735027 -0.57735027 -0.05607722]
     [ 0.57735027 -0.57735027  0.78508102]]

    Eigen values
    [ 3.  1. -2.]

### Cayley-Hamilton Theorem

```
A=np.mat([[2, 3],[4, 5]])
from math import *
X=np.poly(A)
print("The co-efficients of the characteristic equation are",X)

trace_A = np.trace(A)
det_A = np.linalg.det(A)
I = np.eye(len(A))
A*A - trace_A * A + det_A * I

from sympy import *
from math import *
from numpy import *
print("Enter elements of the matrix: ")
A=mat(input())
s=0
print()
print("The matrix is: ")
print(A)
print()
I=eye(len(A),len(A))
print("The identity matrix is: ")
print(I)
print()
ce=poly(A)
ce=ce.round()
print("The coefficients of the characteristic equation=",ce)
for i in range (len(ce)):
    eq=ce[i]*I*(A**(len(ce)-i))
    s=s+eq
print()
s
```

    The co-efficients of the characteristic equation are [ 1. -7. -2.]
    Enter elements of the matrix:
    1 2 3 4; 3 7 9 2; 1 4 7 8; 1 5 9 2

    The matrix is:
    [[1 2 3 4]
     [3 7 9 2]
     [1 4 7 8]
     [1 5 9 2]]

    The identity matrix is:
    [[1. 0. 0. 0.]
     [0. 1. 0. 0.]
     [0. 0. 1. 0.]
     [0. 0. 0. 1.]]

    The coefficients of the characteristic equation= [  1. -17. -38. 116. -32.]

    matrix([[0., 0., 0., 0.],
            [0., 0., 0., 0.],
            [0., 0., 0., 0.],
            [0., 0., 0., 0.]])

### Conclusion:
The extraction of eigenvalues and eigenvectors of matrices were obtained. The extraction rows and columns in matrix was learnt. We also learnt the propertities of eigen values and eigen vectors, the process of diagonalization of matrix, solving system of equation using Cramer’s rule. The Cayley-Hamilton
Theorem was verified in Python in this lab. The extraction of coefficients of the characteristic equation of matrix was learned.

```

```


# RSA Cryptography Algorithm

## About RSA Algorithm
RSA (Rivest–Shamir–Adleman) is one of the first public-key cryptosystems and is widely used for secure data transmission. In such a cryptosystem, the encryption key is public and distinct from the decryption key which is kept secret (private).

## Algorithm

1. Select 2 Prime Numbers - **p & q**
2. Calculate **n = p x q**
3. Calculate Euler's Totient Function of n,  **φ(n) = (p-1) x (q-1)**
4. Select PUBLIC KEY, **e** such that **e & φ(n) are Co-primes**  i.e, **gcd(e , φ(n))=1**
5. Calculate PRIVATE KEY, **d** such that **(d x e) mod φ(n) = 1**

## Public and Private Keys

1. The Public  key is { e , n }, which is known to all in the network.
2. The Private key is { d , n }, which is known ONLY to the User to whom message is to be sent.

## Encryption & Decryption

#### Encryption Algorithm

The Cipher Text, C is generated from the plaintext, M using the public key, e as:

**C = Me mod n**

#### Decryption Algorithm

The Plain Text, M is generated from the ciphertext, C using the private key, d as:

**M = Cd mod n**

## Implementation of RSA using Python

```python
from sympy import *
import math

#Generate p and q
p = randprime(1, 10)
q = randprime(11, 20)

#Generate n and l(n)
n = p*q
l = (p-1)*(q-1)

#Function to test Co-Primality for generation of list of Public Keys
def isCoPrime(x):
    if math.gcd(l,x)==1:
        return True
    else:
        return False

#Function to find mod Inverese of e withl(n) to generate d
def modInverse(e, l) :
    e = e % l;
    for x in range(1, l) :
        if ((e * x) % l == 1) :
            return x
    return 1

#List for Co-Primes
listOfCP = []
for i in range(1, l):
    if isCoPrime(i) == True:
        listOfCP.append(i)

#Print values of P, Q, N, L
print("Value of P = ", p)
print("Value of Q = ", q)
print("Value of N = ", n)
print("Value of L = ", l)

print(" ")

#Print List of Co-Primes for e
print("List of Available Public Keys")
print(listOfCP)

print(" ")

#select a Public Key from list of Co-Primes
e = int(input("Select Public Key from the Above List ONLY: "))

#Value of d
d = modInverse(e, l)

print(" ")

#Print Public and Private Keys
print("PUBLIC KEY  : { e , n } = {", e ,",", n , "}")
print("PRIVATE KEY : { d , n } = {", d ,",", n , "}")

print(" ")

#Encryption Algorithm
def encrypt(plainText):
    return (plainText**e)%n

#Decryption Algorithm
def decrypt(cipherText):
    #pvtKey = int(input("Enter your Private Key: "))
    return (cipherText**d)%n

#Driver Code

#Message Input
pt = int(input('Enter the Plain Text: '))
print("CipherText: ", encrypt(pt))

print(" ")

#CipherText Input
ct = int(input('Enter the Cipher Text: '))
print("PlainText: ", decrypt(ct))
```

    Value of P =  7
    Value of Q =  19
    Value of N =  133
    Value of L =  108

    List of Available Public Keys
    [1, 5, 7, 11, 13, 17, 19, 23, 25, 29, 31, 35, 37, 41, 43, 47, 49, 53, 55, 59, 61, 65, 67, 71, 73, 77, 79, 83, 85, 89, 91, 95, 97, 101, 103, 107]

    Select Public Key from the Above List ONLY: 31

    PUBLIC KEY  : { e , n } = { 31 , 133 }
    PRIVATE KEY : { d , n } = { 7 , 133 }

    Enter the Plain Text: 89
    CipherText:  110

    Enter the Cipher Text: 110
    PlainText:  89

#### Tanmoy Sen Gupta
tanmoysg.com | +91 9864809029 | tanmoysps@gmail.com


```python
import sympy as sp
sp.init_printing(use_unicode=True)
```

```python
flowMat = sp.Matrix([[1,1,1,0,0,0,500], [1,0,0,1,0,1,400], [0,0,1,0,1,-1,100], [0,1,0,-1,-1,0,0]])
flowMat
```

$\displaystyle \left[\begin{matrix}1 & 1 & 1 & 0 & 0 & 0 & 500\\1 & 0 & 0 & 1 & 0 & 1 & 400\\0 & 0 & 1 & 0 & 1 & -1 & 100\\0 & 1 & 0 & -1 & -1 & 0 & 0\end{matrix}\right]$

```python
flowMat.rref()
```

$\displaystyle \left( \left[\begin{matrix}1 & 0 & 0 & 1 & 0 & 1 & 400\\0 & 1 & 0 & -1 & -1 & 0 & 0\\0 & 0 & 1 & 0 & 1 & -1 & 100\\0 & 0 & 0 & 0 & 0 & 0 & 0\end{matrix}\right], \  \left( 0, \  1, \  2\right)\right)$

```python
coffeeMat = sp.Matrix([[0.3,0.6,0.1], [0,0.8,0.2], [0.5,0.5,0]]).T
coffeeMat
```

$\displaystyle \left[\begin{matrix}0.3 & 0 & 0.5\\0.6 & 0.8 & 0.5\\0.1 & 0.2 & 0\end{matrix}\right]$

```python
drinks = sp.Matrix([1,0,0])
drinks
```

$\displaystyle \left[\begin{matrix}1\\0\\0\end{matrix}\right]$

```python
for i in range(10):
    drinks = coffeeMat*drinks
    print(drinks)
```

    Matrix([[0.300000000000000], [0.600000000000000], [0.100000000000000]])
    Matrix([[0.140000000000000], [0.710000000000000], [0.150000000000000]])
    Matrix([[0.117000000000000], [0.727000000000000], [0.156000000000000]])
    Matrix([[0.113100000000000], [0.729800000000000], [0.157100000000000]])
    Matrix([[0.112480000000000], [0.730250000000000], [0.157270000000000]])
    Matrix([[0.112379000000000], [0.730323000000000], [0.157298000000000]])
    Matrix([[0.112362700000000], [0.730334800000000], [0.157302500000000]])
    Matrix([[0.112360060000000], [0.730336710000000], [0.157303230000000]])
    Matrix([[0.112359633000000], [0.730337019000000], [0.157303348000000]])
    Matrix([[0.112359563900000], [0.730337069000000], [0.157303367100000]])