# SymPy

`SymPy` is a package for symbolic calculations in python, similar to *Mathematica*. It works with expressions containing symbols.

```python
from sympy import *
init_printing()
```

Symbols are basic bricks used to construct expressions.
Each symbol has a name used for printing expressions. Objects of the class `Symbol` should be created and assigned to python variables in order to be used in expressions. The symbol name and the name of the variable to which this symbol is assigned are two independent things, and one may write `abc=Symbol('xyz')`. But then one has to write `abc` in input expressions, while `SymPy` will write `xyz` in output ones, producing unnecessary confusion. The python variable name should better be the same as the symbol name.

In languages specifically designed for symbolic calculations, such as *Mathematica*, if a variable to which nothing has been assigned is used, it automatically means a symbol with the same name. Python has not been designed for symbolic calculations. If you use a variable to which nothing has been assigned, you will get an error message. Symbol objects have to be created explicitly.

```python
x=Symbol('x')
```

```python
a=x**2-1
a
```

Several symbols can be defined at once. The string is split at spaces.

```python
y,z=symbols('y z')
```

Let's substitute $y+1$ for $x$.

```python
a.subs(x,y+1)
```

## Polynomials and rational functions

`SymPy` does not expand brackets automatically. The function `expand` is used for this.

```python
a=(x+y-z)**6
a
```

```python
a=expand(a)
a
```

Degree of the polynomial $a$ in $x$.

```python
degree(a,x)
```

Let's collect terms with same power of $x$ together.

```python
collect(a,x)
```

Any polynomial with integer coefficients can be factorized into polynomials with integer coefficients (which cannot be factorized further). There exist efficient algorithms to do this.

```python
a=factor(a)
a
```

`SymPy` does not automatically cancel ratios of polynomials by their greatest common divisor. The function `cancel` is used for this.

```python
a=(x**3-y**3)/(x**2-y**2)
a
```

```python
cancel(a)
```

`SymPy` does not automatically bring sums of rational expressions to common denominator. The function `together` is used for this.

```python
a=y/(x-y)+x/(x+y)
a
```

```python
together(a)
```

The function `simplify` tries to rewrite an expression *in a simplest form*. This concept is not well defined (different forms may be considered simplest in different contexts), and there exists no algorithm for such simplification. The function `simplify` works heuristically, and it is not possible to guess in advance what simplifications it will try. It is very convenient in interactive sessions in order to check if it will succeed in rewriting an expression in some reasonably good form. But it is not desirable to use it in programs. There one should better use more specialized functions which perform well defined expression transformations.

```python
simplify(a)
```

Partial fraction decomposition with respect to $x$.

```python
apart(a,x)
```

Let's substitute some values for the symbils $x$ and $y$.

```python
a=a.subs({x:1,y:2})
a
```

And how much is it numerically?

```python
a.n()
```

## Elementary functions

`SymPy` automatically applies simplifications of elementary functions which are correct everywhere.

```python
sin(-x)
```

```python
cos(pi/4),tan(5*pi/6)
```

`SymPy` can work with floating point numbers having arbitrarily high precision. Here is $\pi$ with 100 significant digits.

```python
pi.n(200)
```

`E` is the base of natural logarithms.

```python
log(1),log(E)
```

```python
exp(log(x)),log(exp(x))
```

Why not $x$? Try $x=2\pi i$.

```python
sqrt(0)
```

```python
sqrt(x)**4,sqrt(x**4)
```

Why not $x^2$? Try $x=i$.

Symbols can have certain properties. E.g., they can be positive. Then `SymPy` can simplify square roots better.

```python
p,q=symbols('p q',positive=True)
sqrt(p**2)
```

```python
sqrt(12*x**2*y),sqrt(12*p**2*y)
```

Let the symbol $n$ be integer (`I` is the imaginary unit).

```python
n=Symbol('n',integer=True)
exp(2*pi*I*n)
```

The method `rewrite` tries to rewrite an expression in terms of a given function.

```python
cos(x).rewrite(exp),exp(I*x).rewrite(cos)
```

```python
asin(x).rewrite(log)
```

The function `trigsimp` tries to rewrite a trigonometric expression *in a simplest form*. In programs it is better to use more specialized functions.

```python
trigsimp(2*sin(x)**2+3*cos(x)**2)
```

The function `expand_trig` expands sines and cosines of sums and multiple angles.

```python
expand_trig(sin(x-y)),expand_trig(sin(2*x))
```

The inverse transformation, rewriting products and powers of sines and cosines into expressions linear in these functions, is needed more often. Suppose we work with a truncated Fourier series.

```python
a1,a2,b1,b2=symbols('a1 a2 b1 b2')
a=a1*cos(x)+a2*cos(2*x)+b1*sin(x)+b2*sin(2*x)
a
```

We want to square it and get a truncated Fourier series again.

```python
a=(a**2).rewrite(exp).expand().rewrite(cos).expand()
a
```

```python
a.collect([cos(x),cos(2*x),cos(3*x),sin(x),sin(2*x),sin(3*x)])
```

The function `expand_log` transforms logarithms of products and powers (of positive quantities) into sums of logarithms; `logcombine` performs the inverse transformation.

```python
a=expand_log(log(p*q**2))
a
```

```python
logcombine(a)
```

The function `expand_power_exp` rewrites powers whose exponents are sums via products of powers.

```python
expand_power_exp(x**(p+q))
```

The function `expand_power_base` rewrites powers whose bases are products via products of powers.

```python
expand_power_base((x*y)**n)
```

The function `powsimp` performs the inverse transformations.

```python
powsimp(exp(x)*exp(2*y)),powsimp(x**n*y**n)
```

New symbolic functions can be introduced. They may have an arbitrary numbers of arguments.

```python
f=Function('f')
f(x)+f(x,y)
```

## Expression structure

Internally expressions are are trees. The function `srepr` returns a string representing this tree.

```python
srepr(x+1)
```

    "Add(Symbol('x'), Integer(1))"

```python
srepr(x-1)
```

    "Add(Symbol('x'), Integer(-1))"

```python
srepr(x-y)
```

    "Add(Symbol('x'), Mul(Integer(-1), Symbol('y')))"

```python
srepr(2*x*y/3)
```

    "Mul(Rational(2, 3), Symbol('x'), Symbol('y'))"

```python
srepr(x/y)
```

    "Mul(Symbol('x'), Pow(Symbol('y'), Integer(-1)))"

One may use the functions `Add`, `Mul`, `Pow`, etc. instead of the binary operations `+`, `*`, `**`, etc.

```python
Mul(x,Pow(y,-1))
```

```python
srepr(f(x,y))
```

    "Function('f')(Symbol('x'), Symbol('y'))"

The attribute `func` is the top-level function of an expression, and `args` is the list of its agruments.

```python
a=2*x*y**2
a.func
```

    sympy.core.mul.Mul

```python
a.args
```

```python
for i in a.args:
    print(i)
```

    2
    x
    y**2

The function `subs` substitutes an expression for a symbol.

```python
a.subs(y,2)
```

It can perform substitutions for several symbols. To this end, one calls it with a list of tuples or a dictionary.

```python
a.subs([(x,pi),(y,2)])
```

```python
a.subs({x:pi,y:2})
```

It can substitute not only for a symbol but also for a subexpression - a function with arguments.

```python
a=f(x)+f(y)
a.subs(f(y),1)
```

```python
(2*x*y*z).subs(x*y,z)
```

```python
(x+x**2+x**3+x**4).subs(x**2,y)
```

Substitutions are performed sequentially. In this case, first $x$ is replaced by $y$ producing $y^3+y^2$; then $y$ is replaced by $x$ in this result.

```python
a=x**2+y**3
a.subs([(x,y),(y,x)])
```

Interchanging these substitutions leads to a different result.

```python
a.subs([(y,x),(x,y)])
```

But if one calls `subs` with the keyword parameter `simultaneous=True`, all substitutions are preformed simultaneously. In this way one can, e.g., interchange $x$ and $y$.

```python
a.subs([(x,y),(y,x)],simultaneous=True)
```

A function can be replaced by another function.

```python
g=Function('g')
a=f(x)+f(y)
a.subs(f,g)
```

The method `replace` searches for subexpressions matching a pattern (with wildcards) and replaces them by a given expression.

```python
a=Wild('a')
(f(x)+f(x+y)).replace(f(a),a**2)
```

```python
(f(x,x)+f(x,y)).replace(f(a,a),a**2)
```

```python
a=x**2+y**2
a.replace(x,x+1)
```

Only a complete subtree can match a pattern, not a subset of factors in a product or a smaller power in a larger one.

```python
a=2*x*y*z
a.replace(x*y,z)
```

```python
(x+x**2+x**3+x**4).replace(x**2,y)
```

## Solving equations

```python
a,b,c,d,e,f=symbols('a b c d e f')
```

An equation is represented by the function `Eq` with two arguments. The function `solve` returns a list of solutions.

```python
solve(Eq(a*x,b),x)
```

Instead of equations, one may pass just expressions to `solve`;
they mean equations `<expression>=0`.

```python
solve(a*x+b,x)
```

A square equation has 2 solutions.

```python
solve(a*x**2+b*x+c,x)
```

A system of linear equations.

```python
solve([a*x+b*y-e,c*x+d*y-f],[x,y])
```

The function `roots` returns roots of a polynomial together with their multiplicities.

```python
roots(x**3-3*x+2,x)
```

The function `solve_poly_system` solves systems of polynomial equations by constructing their Gröbner bases.

```python
p1=x**2+y**2-1
p2=4*x*y-1
solve_poly_system([p1,p2],x,y)
```

## Series

```python
exp(x).series(x,0,5)
```

A series can start from a negative power.

```python
cot(x).series(x,n=5)
```

And even run over half-integer powers.

```python
sqrt(x*(1-x)).series(x,n=5)
```

```python
log(gamma(1+x)).series(x,n=6).rewrite(zeta)
```

Let's prepare 3 series.

```python
sinx=series(sin(x),x,0,8)
sinx
```

```python
cosx=series(cos(x),x,n=8)
cosx
```

```python
tanx=series(tan(x),x,n=8)
tanx
```

Products and ratios of series are not calculated automatically, the function `series` should be applied to them.

```python
series(tanx*cosx,n=8)
```

```python
series(sinx/cosx,n=8)
```

And this series should be equal to 1. But since `sinx` and `cosx` are known only with a limited accuracy, we obtain 1 with the same accuracy.

```python
series(sinx**2+cosx**2,n=8)
```

Here the leading terms have canceled, and the result can be obtained only with a lower accuracy.

```python
series((1-cosx)/x**2,n=6)
```

Series can be differentiated and integrated.

```python
diff(sinx,x)
```

```python
integrate(cosx,x)
```

A series (starting from a small term) can be substituted for an expansion variable in another series. Here are $\sin(\tan(x))$ and $\tan(\sin(x))$.

```python
st=series(sinx.subs(x,tanx),n=8)
st
```

```python
ts=series(tanx.subs(x,sinx),n=8)
ts
```

```python
series(ts-st,n=8)
```

It is not possible to substitute a numerical value for the expansion variable in a series (and hence to plot it). To this end one has to remove the $\mathcal{O}$ term first, transforming a series into a polynomial.

```python
sinx.removeO()
```

## Derivatives

```python
a=x*sin(x+y)
diff(a,x)
```

```python
diff(a,y)
```

The second derivative in $x$ and the first one in $y$.

```python
diff(a,x,2,y)
```

Expressions with undefined functions can be differentiated.

```python
a=x*f(x**2)
b=diff(a,x)
b
```

What's this?

```python
print(b)
```

    2*x**2*Subs(Derivative(f(_xi_1), _xi_1), (_xi_1,), (x**2,)) + f(x**2)

The function `Derivative` represents an unevaluated derivative. It can be evaluated by the method `doit`.

```python
a=Derivative(sin(x),x)
Eq(a,a.doit())
```

## Integrals

```python
integrate(1/(x*(x**2-2)**2),x)
```

```python
integrate(1/(exp(x)+1),x)
```

```python
integrate(log(x),x)
```

```python
integrate(x*sin(x),x)
```

```python
integrate(x*exp(-x**2),x)
```

```python
a=integrate(x**x,x)
a
```

This is an unevaluated integral.

```python
print(a)
```

    Integral(x**x, x)

```python
a=Integral(sin(x),x)
Eq(a,a.doit())
```

Definite integrals.

```python
integrate(sin(x),(x,0,pi))
```

`oo` means $\infty$.

```python
integrate(exp(-x**2),(x,0,oo))
```

```python
integrate(log(x)/(1-x),(x,0,1))
```

## Summing series

```python
summation(1/n**2,(n,1,oo))
```

```python
summation((-1)**n/n**2,(n,1,oo))
```

```python
summation(1/n**4,(n,1,oo))
```

An unevaluated sum is denoted `Sum`.

```python
a=Sum(x**n/factorial(n),(n,0,oo))
Eq(a,a.doit())
```

## Limits

```python
limit((tan(sin(x))-sin(tan(x)))/x**7,x,0)
```

This limit is easy: just expand the numerator and the denominator into series. Things become more difficult if $x=0$ is an essential singularity. Let's calculate one-sided limits.

```python
limit((tan(sin(x))-sin(tan(x)))/(x**7+exp(-1/x)),x,0,'+')
```

```python
limit((tan(sin(x))-sin(tan(x)))/(x**7+exp(-1/x)),x,0,'-')
```

## Differential equations

```python
t=Symbol('t')
x=Function('x')
p=Function('p')
```

First order.

```python
dsolve(diff(x(t),t)+x(t),x(t))
```

Second order.

```python
dsolve(diff(x(t),t,2)+x(t),x(t))
```

A system of first-order equations.

```python
dsolve((diff(x(t),t)-p(t),diff(p(t),t)+x(t)))
```

## Linear algebra

```python
a,b,c,d,e,f=symbols('a b c d e f')
```

A matrix can be constructed from a list of lists.

```python
M=Matrix([[a,b,c],[d,e,f]])
M
```

```python
M.shape
```

A row matrix.

```python
Matrix([[1,2,3]])
```

A column matrix.

```python
Matrix([1,2,3])
```

A matrix can be constructed from a function.

```python
def g(i,j):
    return Rational(1,i+j+1)
Matrix(3,3,g)
```

Or from an undefined function.

```python
g=Function('g')
M=Matrix(3,3,g)
M
```

```python
M[1,2]
```

```python
M[1,2]=0
M
```

```python
M[2,:]
```

```python
M[:,1]
```

```python
M[0:2,1:3]
```

A unit matrix.

```python
eye(3)
```

A zero matrix.

```python
zeros(3)
```

```python
zeros(2,3)
```

A diagonal matrix.

```python
diag(1,2,3)
```

```python
M=Matrix([[a,1],[0,a]])
diag(1,M,2)
```

Operations with matrices.

```python
A=Matrix([[a,b],[c,d]])
B=Matrix([[1,2],[3,4]])
A+B
```

```python
A*B,B*A
```

```python
A*B-B*A
```

```python
simplify(A**(-1))
```

```python
det(A)
```

### Eigenvalues and eigenvectors

```python
x=Symbol('x',real=True)
```

```python
M=Matrix([[(1-x)**3*(3+x),4*x*(1-x**2),-2*(1-x**2)*(3-x)],
          [4*x*(1-x**2),-(1+x)**3*(3-x),2*(1-x**2)*(3+x)],
          [-2*(1-x**2)*(3-x),2*(1-x**2)*(3+x),16*x]])
M
```

```python
det(M)
```

This means that this matrix has a null space (this matrix transforms vectors from this subspace into 0). Let's find a basis of this subspace.

```python
v=M.nullspace()
len(v)
```

It is one-dimensional.

```python
v=simplify(v[0])
v
```

Let's check.

```python
simplify(M*v)
```

The eigenvalues and their multiplicities.

```python
M.eigenvals()
```

If both eigenvalues and corresponding eigenvectors are needed, the method `eigenvects` is used. It returns a list of tuples. In each tuple the zeroth element is an eigenvalue, the first one is its multiplicity, and the last one is a list of corresponding basis eigenvectors (their number is the multiplicity).

```python
v=M.eigenvects()
len(v)
```

```python
for i in range(len(v)):
    v[i][2][0]=simplify(v[i][2][0])
v
```

Let's check.

```python
for i in range(len(v)):
    z=M*v[i][2][0]-v[i][0]*v[i][2][0]
    pprint(simplify(z))
```

    ⎡0⎤
    ⎢ ⎥
    ⎢0⎥
    ⎢ ⎥
    ⎣0⎦
    ⎡0⎤
    ⎢ ⎥
    ⎢0⎥
    ⎢ ⎥
    ⎣0⎦
    ⎡0⎤
    ⎢ ⎥
    ⎢0⎥
    ⎢ ⎥
    ⎣0⎦

### Jordan normal form

```python
M=Matrix([[Rational(13,9),-Rational(2,9),Rational(1,3),Rational(4,9),Rational(2,3)],
          [-Rational(2,9),Rational(10,9),Rational(2,15),-Rational(2,9),-Rational(11,15)],
          [Rational(1,5),-Rational(2,5),Rational(41,25),-Rational(2,5),Rational(12,25)],
          [Rational(4,9),-Rational(2,9),Rational(14,15),Rational(13,9),-Rational(2,15)],
          [-Rational(4,15),Rational(8,15),Rational(12,25),Rational(8,15),Rational(34,25)]])
M
```

The method `M.jordan_form()` returns a couple of matrices, the transformation matrix $P$ and the Jordan form $J$: $M = P J P^{-1}$.

```python
P,J=M.jordan_form()
J
```

```python
P=simplify(P)
P
```

Let's check.

```python
Z=P*J*P**(-1)-M
simplify(Z)
```

## Plots

`SymPy` uses `matplotlib`. However, it distributes $x$ points adaptively, not uniformly.

```python
%matplotlib inline
```

A single function.

```python
plot(sin(x)/x,(x,-10,10))
```

Several functions.

```python
plot(x,x**2,x**3,(x,0,2))
```

Some additional plotting functions can be imported from `sympy.plotting`.

```python
from sympy.plotting import (plot_parametric,plot_implicit,
                            plot3d,plot3d_parametric_line,
                            plot3d_parametric_surface)
```

A parametric plot - a Lissajous curve.

```python
t=Symbol('t')
plot_parametric(sin(2*t),cos(3*t),(t,0,2*pi),
                title='Lissajous',xlabel='x',ylabel='y')
```

An implicit plot - a circle.

```python
plot_implicit(x**2+y**2-1,(x,-1,1),(y,-1,1))
```

A surface. If it is not inline but in a separaye window, you can rotate it with your mouse.

```python
plot3d(x*y,(x,-2,2),(y,-2,2))
```

Several surfaces.

```python
plot3d(x**2+y**2,x*y,(x,-2,2),(y,-2,2))
```

A parametric space curve - a spiral.

```python
a=0.1
plot3d_parametric_line(cos(t),sin(t),a*t,(t,0,4*pi))
```

A parametric surface - a torus.

```python
u,v=symbols('u v')
a=0.3
plot3d_parametric_surface((1+a*cos(u))*cos(v),
                          (1+a*cos(u))*sin(v),a*sin(u),
                          (u,0,2*pi),(v,0,2*pi))
```


## Finding expressions for elastic moduli

I want to find more expressions to fill in the gaps in [my matrix](http://www.subsurfwiki.org/wiki/Template:Elastic_modulus):

We can use [`sympy`](http://docs.sympy.org/dev/tutorial/basic_operations.html) to help!

```python
import sympy
sympy.__version__
```

    '1.1.1'

```python
sympy.init_printing(use_latex='mathjax')
```

We must define our symbols. We'll use `alpha` and `beta` for Vp and Vs.

```python
alpha, beta, gamma = sympy.symbols("alpha, beta, gamma")
lamda, mu, E, K, M, rho = sympy.symbols("lamda, mu, E, K, M, rho")
nu = sympy.symbols("nu")
```

## Expression for Vp (alpha) in terms of K, E

```python
from sympy import sqrt
alpha_expr = sqrt((mu * (E - 4*mu)) / (rho * (E - 3*mu)))
alpha_expr
```

$$\sqrt{\frac{\mu \left(E - 4 \mu\right)}{\rho \left(E - 3 \mu\right)}}$$

```python
print(sympy.latex(alpha_expr))
```

    \sqrt{\frac{\mu \left(E - 4 \mu\right)}{\rho \left(E - 3 \mu\right)}}

```python
mu_expr = (3 * K * E) / (9 * K - E)
```

```python
subs = alpha_expr.subs(mu, mu_expr)
subs
```

$$\sqrt{3} \sqrt{\frac{E K \left(- \frac{12 E K}{- E + 9 K} + E\right)}{\rho \left(- E + 9 K\right) \left(- \frac{9 E K}{- E + 9 K} + E\right)}}$$

```python
print(sympy.latex(subs))
```

    \sqrt{3} \sqrt{\frac{E K \left(- \frac{12 E K}{- E + 9 K} + E\right)}{\rho \left(- E + 9 K\right) \left(- \frac{9 E K}{- E + 9 K} + E\right)}}

```python
from sympy import simplify
simplify(subs)
```

```python
print(sympy.latex(simplify(subs)))
```

    \sqrt{3} \sqrt{\frac{K \left(E + 3 K\right)}{\rho \left(- E + 9 K\right)}}

## Expression for Vs (beta) in terms of K, E

```python
beta_expr = sqrt(mu/rho)
simplify(beta_expr.subs(mu, mu_expr))
```

$$\sqrt{3} \sqrt{- \frac{E K}{\rho \left(E - 9 K\right)}}$$

```python
simpl = simplify(beta_expr.subs(mu, mu_expr))
```

```python
print(sympy.latex(simpl))
```

    \sqrt{3} \sqrt{- \frac{E K}{\rho \left(E - 9 K\right)}}

## Expression for Vp/Vs (gamma) in terms of K, E

```python
gamma_expr = sqrt((K + (4*mu/3)) / mu)
simpl = simplify(gamma_expr.subs(mu, mu_expr))
simpl
```

$$\sqrt{\frac{1}{E} \left(E + 3 K\right)}$$

```python
print(sympy.latex(simpl))
```

    \sqrt{\frac{1}{E} \left(E + 3 K\right)}

## Expression for Vp/Vs (gamma) in terms of E, mu

Not totally sure why I have to use that hacky way to get the terms to cnacel properly.

```python
gamma_emu_expr = alpha_expr / beta_expr
simpl = sqrt(gamma_emu_expr**2)
simpl
```

$$\sqrt{\frac{E - 4 \mu}{E - 3 \mu}}$$

```python
print(sympy.latex(simpl))
```

    \sqrt{\frac{E - 4 \mu}{E - 3 \mu}}

## Expression for Vp/Vs (gamma) in terms of PR, mu

```python
e_expr = 2 * mu * (1 + nu)
```

```python
simplify(sqrt(gamma_emu_expr.subs(E, e_expr)**2))
```

$$\sqrt{2} \sqrt{\frac{\nu - 1}{2 \nu - 1}}$$

```python
print(sympy.latex(simplify(sqrt(gamma_emu_expr.subs(E, e_expr)**2))))
```

    \sqrt{2} \sqrt{\frac{\nu - 1}{2 \nu - 1}}

## Expression for Vp in terms of E, lamda

```python
vp_expr = sympy.Eq(alpha, sqrt(mu*(E - 4*mu) / (rho*(E - 3*mu))))
vp_expr
```

$$\alpha = \sqrt{\frac{\mu \left(E - 4 \mu\right)}{\rho \left(E - 3 \mu\right)}}$$

```python
alpha_expr = sqrt(mu*(E - 4*mu) / (rho*(E - 3*mu)))
```

```python
mu_expr = (rho * alpha**2 - lamda)/2
```

```python
new_expr = simplify(vp_expr.subs(mu, mu_expr))
```

```python
new_expr.subs(alpha, alpha_expr)
```

$$\sqrt{\frac{\mu \left(E - 4 \mu\right)}{\rho \left(E - 3 \mu\right)}} = \sqrt{\frac{\left(- \lambda + \frac{\mu \left(E - 4 \mu\right)}{E - 3 \mu}\right) \left(E + 2 \lambda - \frac{2 \mu \left(E - 4 \mu\right)}{E - 3 \mu}\right)}{\rho \left(2 E + 3 \lambda - \frac{3 \mu \left(E - 4 \mu\right)}{E - 3 \mu}\right)}}$$

OK, I give up. Trying Wolfram Alpha...

```python
vp_wolfram = simplify(sqrt(-lamda/rho+sqrt(9*lamda**2+E**2+2*lamda*E)/rho+E/rho)/sqrt(2))**2
sqrt(vp_wolfram)
```

$$\frac{\sqrt{2}}{2} \sqrt{\frac{1}{\rho} \left(E - \lambda + \sqrt{E^{2} + 2 E \lambda + 9 \lambda^{2}}\right)}$$

That's better!

```python
print(sympy.latex(sqrt(vp_wolfram)))
```

    \frac{\sqrt{2}}{2} \sqrt{\frac{1}{\rho} \left(E - \lambda + \sqrt{E^{2} + 2 E \lambda + 9 \lambda^{2}}\right)}

Another for Wolfram Alpha: Vs

    solve V = sqrt((a/(2*((y-2 r V^2)/(2 r V^2))*r)) - (a/r)) for V

```python
simplify(sqrt(sqrt(9*lamda**2+2*lamda*E+E**2)/rho-(3*lamda)/rho+E/rho)/2)**2
```

$$\frac{1}{4 \rho} \left(E - 3 \lambda + \sqrt{E^{2} + 2 E \lambda + 9 \lambda^{2}}\right)$$

Another for Wolfram Alpha: Gamma, or Vp/Vs

    solve G = sqrt(((4/3)*a - 2*((y*G^2 - y)/3))/(a - ((y*G^2 - y)/3))) for G

```python
simplify(sqrt(sqrt(9*lamda**2+2*lamda*E+E**2)/E+(3*lamda)/E+3)/sqrt(2))**2
```

$$\frac{1}{2 E} \left(3 E + 3 \lambda + \sqrt{E^{2} + 2 E \lambda + 9 \lambda^{2}}\right)$$

## Expression for Vs (beta) in terms of nu, K

```python
mu_expr = 3*K*(1 - 2*nu)/(2 + 2*nu)
simpl = simplify((sqrt(mu/rho)).subs(mu,mu_expr)**2)
simpl
```

$$- \frac{3 K \left(2 \nu - 1\right)}{2 \rho \left(\nu + 1\right)}$$

```python
print(sympy.latex(simpl))
```

    - \frac{3 K \left(2 \nu - 1\right)}{2 \rho \left(\nu + 1\right)}

## Expression for Vp (alpha) in terms of nu, K

```python
vp_expr = sqrt(lamda * (1 - nu) / (rho * nu))
l_expr = 3 * K * nu / (1 + nu)
vp_expr.subs(lamda, l_expr)
```

$$\sqrt{3} \sqrt{\frac{K \left(- \nu + 1\right)}{\rho \left(\nu + 1\right)}}$$

## Expression for Vp (alpha) in terms of K, rho, and Vs (beta)

```python
vp_expr = sqrt((K + 1.3333333*mu) / rho)
mu_expr = rho * beta**2
simplify(vp_expr.subs(mu, mu_expr))
```

$$\sqrt{\frac{K}{\rho} + 1.3333333 \beta^{2}}$$

```python

```


# Packages

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import yfinance as yf
import yahoofinancials
import seaborn as sn
from scipy.stats import anderson,kstest,shapiro, probplot
```

# Masalah.
Misalkan portofolio saham dengan komposisi saham TLKM, BBNI, dan ASII. Akan dicari nilai return dengan variansi minimal dan juga dicari komposisi dari masing-masing saham.

```python
stock = ['TLKM.JK', 'BBNI.JK','ASII.JK', 'ANTM.JK', 'INDF.JK']
df = pd.DataFrame(columns=stock)
for i in stock :
    temp = yf.Ticker(i).history(period = '2y', interval = '1mo',actions=False)
    temp.dropna(inplace = True)
    temp['log_return'] = np.log(temp['Close']) - np.log(temp['Close'].shift(1))
    temp['log_return'].dropna(inplace = True)
    df[i] =temp['log_return'].dropna()
```

```python
mu = df.mean().values
C = df.cov().values
e = np.ones(len(mu))
print(f'Vektor return {mu} dan \n matriks kovariansi \n {C} \n dan vektor e {e}')

```

    Vektor return [0.0173399  0.03236004 0.01910487 0.06699543 0.00259145] dan
     matriks kovariansi
     [[ 0.00587657  0.00456284 -0.0007402   0.00245725  0.00064832]
     [ 0.00456284  0.01116214  0.00167183  0.00427448  0.00392362]
     [-0.0007402   0.00167183  0.00748455  0.00698968 -0.00113314]
     [ 0.00245725  0.00427448  0.00698968  0.03106407 -0.00126756]
     [ 0.00064832  0.00392362 -0.00113314 -0.00126756  0.00482781]]
     dan vektor e [1. 1. 1. 1. 1.]

```python
# mu = np.array([1.1, 1.2, 1.3])
# C = np.array([[0.01, 0, 0], [0, 0.05, 0], [0, 0, 0.07]])
# e = np.ones(len(mu))
# print(f'Vektor return {mu} dan \n matriks kovariansi \n {C} \n dan vektor e {e}')

```

Dari masalah
\begin{equation}
\min\left\{ -t \mu^T \mathbf{x} +  \frac{1}{2}\mathbf{x}^T C  \mathbf{x} | \mathbf{e}^T \mathbf{x} = 1 \right\}
\end{equation}
dengan $t$ adalah suatu parameter tak negatif.
dapat diperoleh bahwa persamaan untuk efficient frontiernya adalah :
\begin{equation}
\sigma^2_P - \beta_0 = \frac{(\mu_P - \alpha_0)^2}{\alpha_1}
\end{equation}
dengan :
\begin{align}
\alpha_0 = \mu^T h_0 \\
\alpha_1 = \mu^T h_1 \\
\beta_0 = h_0^TCh_0
\end{align}
dan
\begin{align}
h_0 &= \frac{C^{-1}e}{e^TC^{-1}e} \\
h_1 &= C^{-1}\mu - \frac{e^TC^{-1}\mu}{e^TC^{-1}e} C^{-1}e
\end{align}

```python
h_0 = (np.linalg.inv(C).dot(e))/(e.dot(np.linalg.inv(C)).dot(e))
h_1 = np.linalg.inv(C).dot(mu) -((e.dot(np.linalg.inv(C)).dot(mu))/(e.dot(np.linalg.inv(C)).dot(e)))*(np.linalg.inv(C).dot(e))
print(f'Diperoleh \n h_0 {h_0}\n dan \n h_1 {h_1}')
```

    Diperoleh
     h_0 [ 0.4332845  -0.29924553  0.36359046 -0.01483422  0.51720478]
     dan
     h_1 [-1.1251155   3.44389107 -1.07559641  1.72769534 -2.97087449]

```python
alpha_0 = mu.dot(h_0)
alpha_1 = mu.dot(h_1)
beta_0 = h_0.dot(C).dot(h_0)
beta_2 = h_1.dot(C).dot(h_1)
print(f'Diperoleh \n alpha_0 : {alpha_0}, \n alpha_1 : {alpha_1}, \n beta_0 : {beta_0}\n beta_2 : {beta_2}')
```

    Diperoleh
     alpha_0 : 0.005122348163682106,
     alpha_1 : 0.1794347481375964,
     beta_0 : 0.0012105522636421777
     beta_2 : 0.17943474813759647

Sehingga persamaan Efficient Frontier - nya menjadi :
\begin{equation}
\sigma^2_P - 0.00875 = \frac{(\mu_P - 1.125)^2}{0.035}
\end{equation}

```python
t = np.linspace(0,0.5, 1000)
sigma_P = beta_0 + t**2 *beta_2
mu_p = alpha_0 + t*alpha_1
mu_p_bawah = alpha_0 - t*alpha_1
plt.plot(sigma_P, mu_p)
plt.plot(sigma_P, mu_p_bawah, '--')
plt.xlabel('sigma^2_P')
plt.ylabel('mu_P')
plt.grid(True)
plt.title('Kurva Efficient Frontier')
plt.show()
print(f'Nilai return dari variansi yang minimum adalah : {np.min(mu_p)}')
print(f'Dengan masing-masing proporsi saham TLKM, BBNI dan ASII adalah sebagai berikut : {h_0}')
```

Misalkan ditambahkan sebuah asset \textit{risk-free} dengan \textit{rate of return} $r = 1.02$. Maka diperoleh capital market linenya sebagai berikut :


```python
%matplotlib inline
```

```python
import numpy as np
import matplotlib.pyplot as plt
import random
from time import time
import secrets
from numpy.polynomial import Polynomial as Poly
from sympy.ntheory import factorint
from sympy.ntheory import primefactors
from turtle import *

```

# Basic Algebra Exercise
## Functions, Polynomials, Complex Numbers. Applications of Abstract Algebra

### Problem 1. Polynomial Interpolation
We know that if we have a set of $n$ data points with coordinates $(x_1; y_1), (x_2; y_2), \dots, (x_n; y_n)$, we can try to figure out what function may have generated these points.

Please note that **our assumptions about the data** will lead us to choosing one function over another. This means that our results are as good as our data and assumptions. Therefore, it's extremely important that we write down our assumptions (which sometimes can be difficult as we sometimes don't realize we're making them). It will be better for our readers if they know what those assumptions and models are.

In this case, we'll state two assumptions:
1. The points in our dataset are generated by a polynomial function
2. The points are very precise, there is absolutely no error in them. This means that the function should pass **through every point**

This method is called *polynomial interpolation* (*"polynomial"* captures assumption 1 and *"interpolation"* captures assumption 2).

It can be proved (look at [Wikipedia](https://en.wikipedia.org/wiki/Polynomial_interpolation) for example) that if we have $n$ data points, there is only one polynomial of degree $n-1$ which passes through them. In "math speak": "the vector spaces of $n$ points and polynomials of degree $n-1$ are isomorphic (there exists a bijection mapping one to the other)".

There are a lot of ways to do interpolation. We can also write the function ourselves if we want but this requires quite a lot more knowledge than we already covered in this course. So we'll use a function which does this for us. `numpy.polyfit()` is one such function. It accepts three main parameters (there are others as well, but they are optional): a list of $x$ coordinates, a list of $y$ coordinates, and a polynomial degree.

Let's say we have these points:
```python
points = np.array([(0, 0), (1, 0.8), (2, 0.9), (3, 0.1), (4, -0.8), (5, -1.0)])
```

First, we need to "extract" the coordinates:
```python
x = points[:, 0]
y = points[:, 1]
```

Then, we need to calculate the interpolating polynomial. For the degree, we'll initially set $n-1$:
```python
coefficients = np.polyfit(x, y, len(points) - 1)
poly = np.poly1d(coefficients)
```

After that, we need to plot the function. To do this, we'll create a range of $x$ values and evaluate the polynomial at each value:
```python
plot_x = np.linspace(np.min(x), np.max(x), 1000)
plot_y = poly(plot_x)
```

Finally, we need to plot the result. We'll plot both the fitting polynomial curve (using `plt.plot()`) and the points (using `plt.scatter`). It's also nice to have different colors to make the line stand out from the points.
```python
plt.plot(plot_x, plot_y, c = "green")
plt.scatter(x, y)
plt.xlabel("x")
plt.ylabel("y")
plt.show()
```
Don't forget to label the axes!

```python
points = np.array([(0, 0), (1, 0.8), (2, 0.9), (3, 0.1), (4, -0.8), (5, -1.0)])
x = points[:, 0]
y = points[:, 1]
coefficients = np.polyfit(x, y, len(points) - 1)
poly = np.poly1d(coefficients)
plot_x = np.linspace(np.min(x), np.max(x), 1000)
plot_y = poly(plot_x)
plt.plot(plot_x, plot_y, c = "green")
plt.scatter(x, y)
plt.title("title")
plt.xlabel("x")
plt.ylabel("y")
plt.show()
```

Your task now is to **wrap the code in a function**. It should accept a list of points, the polynomial degree, min and max value of $x$ used for plotting. **Be extremely careful to ensure that the function uses its parameters!** Of course, you can extract other utility functions if you wish (e.g., separating plotting from the rest is a good idea).

We'll use this function to try some other cases.

```python
def interpolate_polynomial(points, degree, min_x, max_x, title=None, add_func=None):
    """
    Interpolates a polynomial of the specified degree through the given points and plots it
    points - a list of points (x, y) to plot
    degree - the polynomial degree
    min_x, max_x - range of x values used to plot the interpolating polynomial
    """
    pts = np.array(points)
    x = pts[:, 0]
    y = pts[:, 1]
    p = Poly.fit(x, y, degree)
    p = p.convert(domain=[min_x, max_x])
    p_plot = p.linspace(1000)
    plt.plot(p_plot[0], p_plot[1], c="green")

    if add_func:
        v_y = np.vectorize(add_func)
        x2 = np.linspace(min_x, max_x, 1000)
        y2 = v_y(x2)
        plt.plot(x2, y2, c="blue")

    if title:
        plt.title(title)
    else:
        plt.title(f'Polynomial of degree {degree}')
    plt.xlabel("x")
    plt.ylabel("y")
    plt.scatter(x, y)
    plt.show()
```

```python
points = np.array([(0, 0), (1, 0.8), (2, 0.9), (3, 0.1), (4, -0.8), (5, -1.0)])
interpolate_polynomial(points, len(points) - 1, np.min(points[:, 0]), np.max(points[:, 0]))
```

We see this is a very nice fit. This is expected, of course. Let's try to expand our view a little. Let's try to plot other values of $x$, further than the original ones. This is **extrapolation**.

```python
interpolate_polynomial(points, len(points) - 1, -5, 10)
```

Hmmm... it seems our polynomial goes a little wild outside the original range. This is to show how **extrapolation can be quite dangerous**.

Let's try a lower polynomial degree now. We used 4, how about 3, 2 and 1?
**Note:** We can add titles to every plot so that we know what exactly we're doing. Te title may be passed as an additional parameter to our function.

```python
interpolate_polynomial(points, 3, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 2, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 1, np.min(points[:, 0]), np.max(points[:, 0]))
```

We see the fitting curves (or line in the last case) struggle more and more and they don't pass through every point. This breaks our assumptions but it can be very useful.

Okay, one more thing. How about increasing the degree? Let's try 5, 7 and 10. Python might complain a little, just ignore it, everything is fine... sort of :).

```python
interpolate_polynomial(points, 5, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 7, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 10, np.min(points[:, 0]), np.max(points[:, 0]))
```

Those graphs look pretty much the same. But that's the point exactly. I'm being quite sneaky here. Let's try to expand our view once again and see what our results really look like.

```python
interpolate_polynomial(points, 5, -10, 10)
interpolate_polynomial(points, 7, -10, 10)
interpolate_polynomial(points, 10, -10, 10)
```

Now we see there are very wild differences. Even though the first two plots look quite similar, look at the $y$ values - they're quite different.

So, these are the dangers of interpolation. Use a too high degree, and you get "the polynomial wiggle". These are all meant to represent **the same** data points but they look insanely different. Here's one more comparison.

```python
interpolate_polynomial(points, len(points) - 1, -2, 7)
interpolate_polynomial(points, len(points) + 1, -2, 7)
```

Now we can see what big difference even a small change in degree can make. This is why we have to choose our interpolating functions very carefully. Generally, a lower degree means a simpler function, which is to be preferred. See [Occam's razor](https://en.wikipedia.org/wiki/Occam%27s_razor).

And also, **we need to be very careful about our assumptions**.

```python
points = np.array([(-5, 0.03846), (-4, 0.05882), (-3, 0.1), (-2, 0.2), (-1, 0.5), (0, 1), (1, 0.5), (2, 0.2), (3, 0.1), (4, 0.05882), (5, 0.03846)])
interpolate_polynomial(points, len(points) - 1, np.min(points[:, 0]), np.max(points[:, 0]))
```

This one definitely looks strange. Even stranger, if we remove the outermost points... ($x = \pm 5$), we get this

```python
points = np.array([(-4, 0.05882), (-3, 0.1), (-2, 0.2), (-1, 0.5), (0, 1), (1, 0.5), (2, 0.2), (3, 0.1), (4, 0.05882)])
interpolate_polynomial(points, len(points) - 1, np.min(points[:, 0]), np.max(points[:, 0]))
```

This is because the generating function is not a polynomial. It's actually:
$$ y = \frac{1}{1 + x^2} $$

Plot the polynomial interpolation and the real generating function **on the same plot**. You may need to modify the original plotting function or just copy its contents.

```python
points = np.array([(-4, 0.05882), (-3, 0.1), (-2, 0.2), (-1, 0.5), (0, 1), (1, 0.5), (2, 0.2), (3, 0.1), (4, 0.05882)])
interpolate_polynomial(points, len(points) - 1, np.min(points[:, 0]), np.max(points[:, 0]), add_func=lambda x: 1 / (1 + x**2))
```

### Problem 2. Complex Numbers as Vectors
We saw that a complex number $z = a + bi$ is equivalent to (and therefore can be represented as) the ordered tuple $(a; b)$, which can be plotted in a 2D space. So, complex numbers and 2D points are equivalent. What is more, we can draw a vector from the origin of the coordinate plane to our point. This is called a point's **radius-vector**.

Let's try plotting complex numbers as radius vectors. Don't forget to label the real and imaginary axes. Also, move the axes to the origin. Hint: These are called "spines"; you'll need to move 2 of them to the origin and remove the other 2 completely. Hint 2: You already did this in the previous lab.

We can use `plt.quiver()` to plot the vector. It can behave a bit strangely, so we'll need to set the scale of the vectors to be the same as the scale on the graph axes:
```python
plt.quiver(0, 0, z.real, z.imag, angles = "xy", scale_units = "xy", scale = 1)
```

Other than that, the main parameters are: $x_{begin}$, $y_{begin}$, $x_{length}$, $y_{length}$ in that order.

Now, set the aspect ratio of the axes to be equal. Also, add grid lines. Set the axis numbers (called ticks) to be something like `range(-3, 4)` for now.
```python
plt.xticks(range(-3, 4))
plt.yticks(range(-3, 4))

```

If you wish to, you can be a bit more clever with the tick marks. Find the minimal and maximal $x$ and $y$ values and set the ticks according to them. It's a good practice not to jam the plot too much, so leave a little bit of space. That is, if the actual x-range is $[-2; 2]$, set the plotting to be $[-2.5; 2.5]$ for example. Otherwise, the vector heads (arrows) will be "jammed" into a corner or side of the plot.

```python
def plot_complex_numbers(numbers, colors=None):
    """
    Plots the given complex numbers as radius vectors in the 2D space
    """
    # Write your code here
    ax = plt.gca()
    ax.set_aspect("equal")

    X = [0 for _ in range(len(numbers))]
    Y = X
    U = np.array([n.real for n in numbers])
    V = np.array([n.imag for n in numbers])
    if not colors:
        colors = 'green'
    x_min = min(U.min(), 0) - 0.5
    x_max = max(U.max(), 0) + 0.5
    y_min = min(V.min(), 0) - 0.5
    y_max = max(V.max(), 0) + 0.5

    ax.quiver(X, Y, U, V, angles = "xy", scale_units = "xy", scale = 1, color=colors)
    ax.set_xlim(x_min, x_max)
    ax.set_ylim(y_min, y_max)
    plt.xlabel("real")
    plt.ylabel("imaginary")
    plt.show()

plot_complex_numbers([2 + 3j])
```

How about many numbers? We'll need to get a little bit more creative. First, we need to create a 2D array, each element of which will be a 4-element array: `[0, 0, z.real, z.imag]`. Next, `plt.quiver()` can accept a range of values. Look at [this StackOverflow post](https://stackoverflow.com/questions/12265234/how-to-plot-2d-math-vectors-with-matplotlib) for details and adapt your code.

```python
plot_complex_numbers([2 + 3j, -2 - 1j, -3, 2j], ["green", "red", "blue", "orange"])
```

Now let's add two numbers and plot the result.

```python
z1 = 2 + 3j
z2 = 1 - 1j
plot_complex_numbers([z1, z2, z1 + z2], ["red", "blue", "green"])
```

We can see that adding the complex numbers is equivalent to adding vectors (remember the "parallelogram rule"). As special cases, let's try adding pure real and pure imaginary numbers:

```python
z1 = 2 + 3j
z2 = 2 + 0j
plot_complex_numbers([z1, z2, z1 + z2], ["red", "blue", "green"])
```

```python
z1 = 2 + 3j
z2 = 0 + 2j
plot_complex_numbers([z1, z2, z1 + z2], ["red", "blue", "green"])
```

How about multiplication? First we know that multiplying by 1 gives us the same vector and mulpiplying by -1 gives us the reversed version of the same vector. How about multiplication by $\pm i$?

```python
z = 2 + 3j
plot_complex_numbers([z, z * 1], ["red", "blue"])
plot_complex_numbers([z, z * -1], ["red", "blue"])
plot_complex_numbers([z, z * 1j], ["red", "blue"])
plot_complex_numbers([z, z * -1j], ["red", "blue"])
```

So, multiplication by $i$ is equivalent to 90-degree rotation. We can actually see the following equivalence relationships between multiplying numbers and rotation about the origin:

| Real | Imaginary | Result rotation |
|------|-----------|-----------------|
| 1    | 0         | $0^\circ$       |
| 0    | 1         | $90^\circ$      |
| -1   | 0         | $180^\circ$     |
| 0    | -1        | $270^\circ$     |

Once again, we see the power of abstraction and algebra in practice. We know that complex numbers and 2D vectors are equivalent. Now we see something more: addition and multiplication are equivalent to translation (movement) and rotation!

Let's test the multiplication some more. We can see the resulting vector is the sum of the original vectors, but *scaled and rotated*:

```python
z1 = 2 + 3j
z2 = 1 - 2j
plot_complex_numbers([z1, z2, z1 * z2], ["red", "blue", "green"])
```

### Problem 3. Recursion and Fractals

> "To understand recursion, you first need to understand recursion."

There are three main parts to a recursive function:
1. Bottom - when the recursion should finish
2. Operation - some meaningful thing to do
3. Recursive call - calling the same function
4. Clean-up - returning all data to its previous state (this reverses the effect of the operation)

Let's do one of the most famous recursion examples. And I'm not talking about Fibonacci here. Let's draw a tree using recursive functions.

The figure we're going to draw is called a **fractal**. It's self-similar, which means that if you zoom in on a part of it, it will look the same. You can see fractals everywhere in nature, with broccoli being one of the prime examples. Have a look:

First, we need to specify the recursive part. In order to draw a tree, we need to draw a line of a given length (which will be the current branch), and then draw two more lines to the left and right. By "left" and "right", we should mean "rotation by a specified angle".

So, this is how to draw a branch: draw a line and prepare to draw two more branches to the left and right. This is going to be our recursive call.

To make things prettier, more natural-looking (and have a natural end to our recursion), let's draw each "sub-branch" a little shorter. If the branch becomes too short, it won't have "child branches". This will be the bottom of our recursion.

There's one more important part of recursion, and this is **"the clean-up"**. After we did something in the recursive calls, it's very important to return the state of everything as it was **before** we did anything. In this case, after we draw a branch, we go back to our starting position.

Let's first import the most import-ant (no pun intended...) Python drawing library: `turtle`! In order to make things easier, we'll import all methods directly.
```python
from turtle import *
```

You can look up the docs about turtle if you're more interested. The basic things we're going to use are going forward and backward by a specified number of pixels and turning left and right by a specified angle (in degrees).

Let's now define our recursive function:
```python
def draw_branch(branch_length, angle):
    if branch_length > 5:
        forward(branch_length)
        right(angle)
        draw_branch(branch_length - 15, angle)
        left(2 * angle)
        draw_branch(branch_length - 15, angle)
        right(angle)
        backward(branch_length)
```

And let's call it:
```python
draw_branch(100, 20)
```

We need to start the tree not at the middle, but toward the bottom of the screen, so we need to make a few more adjustments. We can wrap the setup in another function and call it. Let's start one trunk length below the center (the trunk length is the length of the longest line).

**Note:** It's important to call `done()` after the drawing is finished. If you miss it, the window with the turtle drawing will freeze and throw an exception.

```python
def draw_tree(trunk_length, angle):
    speed("fastest")
    left(90)
    up()
    backward(trunk_length)
    down()
    draw_branch(trunk_length, angle)
    done()
```

Note that the graphics will show in a separate window. Also note that sometimes you might get bugs. If you do, go to Kernel > Restart.

```python
# Write your code here
def draw_branch(branch_length, angle):
    if branch_length > 5:
        forward(branch_length)
        right(angle)
        draw_branch(branch_length - 15, angle)
        left(2 * angle)
        draw_branch(branch_length - 15, angle)
        right(angle)
        backward(branch_length)

def draw_tree(trunk_length, angle):
    speed("fastest")
    left(90)
    up()
    backward(trunk_length)
    down()
    draw_branch(trunk_length, angle)
    done()
```

```python
draw_tree(100, 20)
```

Experiment with different lengths and angles. Especially interesting angles are $30^\circ$, $45^\circ$, $60^\circ$ and $90^\circ$.

```python
draw_tree(100, 30)
```

```python
draw_tree(100, 45)
```

```python
draw_tree(100, 90)
```

Now modify the original function a little. Draw the lines with different thickness. Provide the trunk thickness at the initial call. Similar to how branches go shorter, they should also go thinner.

```python
# Write your code here
def draw_branch(branch_length, angle, thickness):
    if branch_length > 5:
        width(thickness)
        forward(branch_length)
        right(angle)
        draw_branch(branch_length - 15, angle, thickness * .75)
        left(2 * angle)
        draw_branch(branch_length - 15, angle, thickness * .75)
        right(angle)
        backward(branch_length)

def draw_tree(trunk_length, angle, thickness):
    speed("fastest")
    color("tan4")
    width(thickness)
    left(90)
    up()
    backward(trunk_length)
    down()
    draw_branch(trunk_length, angle, thickness)
    done()

draw_tree(100, 25, 10)
```

#### * Optional problem
Try to draw another kind of fractal graphic using recursion and the `turtle` library. Two very popular examples are the "Koch snowflake" and the "Sierpinski triangle". You can also modify the original tree algorithm to create more natural-looking trees. You can, for example, play with angles, number of branches, lengths, and widths. The Internet has a lot of ideas about this :). Hint: Look up **"L-systems"**.

```python
def draw_branch(branch_length, angle, thickness):
    if branch_length > 5:
        width(thickness)
        forward(branch_length)
        right(angle)
        draw_branch(branch_length * random.random(), angle, thickness * .75)
        left(2 * angle)
        draw_branch(branch_length * .7, angle, thickness * .75)
        right(angle)
        backward(branch_length)

def draw_tree(trunk_length, angle, thickness):
    speed("fastest")
    color("tan4")
    width(thickness)
    left(90)
    up()
    backward(trunk_length)
    down()
    draw_branch(trunk_length, angle, thickness)
    done()

draw_tree(100, 25, 10)
```

### Problem 4. Run-length Encoding
One application of algebra and basic math can be **compression**. This is a way to save data in less space than it originally takes. The most basic form of compression is called [run-length encoding](https://en.wikipedia.org/wiki/Run-length_encoding).

Write a function that encodes a given text. Write another one that decodes.

We can see that RLE is not very useful in the general case. But it can be extremely useful if we have very few symbols. An example of this can be DNA and protein sequences. DNA code, for example, has only 4 characters.

Test your encoding and decoding functions on a DNA sequence (you can look up some on the Internet). Measure how much your data is compressed relative to the original.

```python
def encode(text):
    """
    Returns the run-length encoded version of the text
    (numbers after symbols, length = 1 is skipped)
    """
    retval = text[0]
    n = 1
    for i in range(1, len(text)):
        if text[i] == retval[-1]:
            n += 1
        else:
            if n > 1:
                retval += str(n)
            retval += text[i]
            n = 1
    if n > 1:
        retval += str(n)
    return retval

def decode(text):
    """
    Decodes the text using run-length encoding
    """
    retval = ''
    for ch in text:
        if ch.isnumeric():
            retval += retval[-1] * (int(ch) - 1)
        else:
            retval += ch
    return retval
```

```python
# Tests
# Test that the functions work on their own
assert encode("AABCCCDEEEE") == "A2BC3DE4"
assert decode("A2BC3DE4") == "AABCCCDEEEE"

# Test that the functions really invert each other
assert decode(encode("AABCCCDEEEE")) == "AABCCCDEEEE"
assert encode(decode("A2BC3DE4")) == "A2BC3DE4"
```

```python
test_sequence = """
ctatcagctatatttttcttagtagttccagcgcctgtcgcggcaatcgtgaaatgtgct
tggagtcgactgtctgggactcggctacagctcctatccgaagctagtcggtaataagat
catcgaaacgatggctttaacggtgtttcccaggagccggttgtccagtcagctcgtatt
actctacttggggtacatcagcttttaaccaaagctcctacgcacaacaaactgcggtac
agtgtgcggtgcgtgcccctagaatatttcacgggtgcagggactatctcccgtgaactg
ccgcagtcagactattggggctcaggtacaggttgtaaacttgcgtatacgacactcatc
caagtttgtgatcccgcgacattgaacaacaagaatgagacggtaaacataagaaacaat
atcatgacatttttacacattttattcgggtcagataggacatgtgactcgactacagat
gtcaactgatcctgctcccacggactgaatgataaaagtccttacagagccggagaatat
tacctgggcacctttggagccagaaagatgattttagtacttgaggcaaaaccgtcccct
aagactaacattgggattcacccgctgtttcggtacccggtgacgatgacccgttacgac
agcgaacgtaacttccccttgctaagttgaacaggatgtacgccagaggtcactcactac
gacgggttcgataactgcatgttattgcaagaacgccctgcacggcgactcgagaaccct
gttgatgacaaacccattcggatgcgattacagttttccagcatatgcataaacgataag
caacttgttgtggtattattacaaataggctgagttacgggacatctgcgacccattccc
ttacgaaagcagagagcccgagcaataagatacgcctctctcatgcaactttaacccgtc
cttcgtagtacacggcctattacccggtacccacctgagt"""

result = len(test_sequence)/len(encode(test_sequence))

print(f'Compression ratio: {result:.2f}')
```

    Compression ratio: 1.08

### * Problem 5. Function Invertibility and Cryptography
As we already saw, some functions are able to be inverted. That is, if we know the output, we can see what input generated it directly. This is true if the function is **one-to-one correspondence** (bijection).

However, not all functions are created the same. Some functions are easy to compute but their inverses are extremely difficult. A very important example is **number factorization**. It's relatively easy (computationally) to multiply numbers but factoring them is quite difficult. Let's run an experiment.

We'll need a function to generate random n-bit numbers. One such function can be found in the `random` package:
```python
import random
random.getrandbits(n_bits)
```

It works perfectly for this example. However, to make a point about random generators and their security, we may want to import a similar function from the `secrets` module instead. If you're interested, you can look up "cryptographic pseudo-random number generators" / "PRNGs".
```python
import secrets
secrets.randbits(n_bits)

```

We could, of course, write our factorization by hand but we'll use `sympy`
```python
from sympy.ntheory import factorint
factorint(1032969399047817906432668079951) # {3: 2, 79: 1, 36779: 1, 7776252885493: 1, 5079811103: 1}
```

This function returns a `dict` where the keys are the factors, and the values - how many times they should be multiplied.

We'll also need a tool to accurately measure performance. Have a look at [this one](https://docs.python.org/3/library/time.html#time.time) for example.

Specify a sequence of bit lengths, in increasing order. For example, you might choose something like `[10, 20, 25, 30, 32, 33, 35, 38, 40]`. Depending on your computer's abilities you can go as high as you want. For each bit length, generate a number. See how much time it takes to factor it. Then see how much time it takes to multiply the factors. Be careful how you measure these. You shouldn't include the number generation (or any other external functions) in your timing.

In order to have better accuracy, don't do this once per bit length. Do it, for example, five times, and average the results.

Plot all multiplication and factorization times as a function of the number of bits. You should see that factorization is much, much slower. If you don't see this, just try larger numbers :D.

```python
# Write your code here
def measure_time(verbose=False):
    def wrapper(func):
        def inner(*args, **kwargs):
            start = time.time()
            retval = func(*args, **kwargs)
            exec_time = time.time() - start
            if verbose:
                print(f"Function '{func.__name__}' runtime (sec): {exec_time}")
            return exec_time, retval
        return inner
    return wrapper

def list_generator(start: int, step: int, number: int) -> list:
    """
    Generates a list with 'number' of integers.
    The first one equals to 'start' and every consequtive is increased by 'step'.
    """
    i = start
    retval = [i]
    for n in range(number):
        i += step
        retval.append(i)
    return retval

@measure_time()
def get_factors(number: int) -> dict:
    return factorint(number)

@measure_time()
def multiply_factors(factors: dict) -> int:
    retval = 1
    for k, v in factors.items():
        retval *= k * v
    return retval

def get_runtimes(n_bits: int, samples: int=1) -> tuple:
    fact_list = []
    prod_list = []

    for i in range(samples):
        number = secrets.randbits(n_bits)
        factor_time, factors = get_factors(number)
        fact_list.append(factor_time)
        multiply_time, result = multiply_factors(factors)
        prod_list.append(multiply_time)
    return sum(fact_list)/samples, sum(prod_list)/samples

def main(verbose=False):
    SAMPLE_SIZE = 10
    bitsizes = [16, 32, 48, 64, 80, 96, 112, 128]

    retval = []
    for n_bits in bitsizes:
        factor_time, multiply_time = get_runtimes(n_bits, SAMPLE_SIZE)
        retval.append((n_bits, factor_time, multiply_time))
        if verbose:
            print(f'\n{n_bits} bits number\n\tavg. factoring time (sec):{factor_time}\n\tavg. product time (sec):{multiply_time}')
    return retval

p5_data = main(True)
```

    16 bits number
    	avg. factoring time (sec):0.0
    	avg. product time (sec):0.0

    32 bits number
    	avg. factoring time (sec):0.00039992332458496096
    	avg. product time (sec):0.0

    48 bits number
    	avg. factoring time (sec):0.0015977144241333008
    	avg. product time (sec):0.0

    64 bits number
    	avg. factoring time (sec):0.003502678871154785
    	avg. product time (sec):0.0

    80 bits number
    	avg. factoring time (sec):0.0015972375869750977
    	avg. product time (sec):0.0

    96 bits number
    	avg. factoring time (sec):0.17370858192443847
    	avg. product time (sec):0.00010306835174560547

    112 bits number
    	avg. factoring time (sec):1.033999490737915
    	avg. product time (sec):0.0

    128 bits number
    	avg. factoring time (sec):1.1454092025756837
    	avg. product time (sec):0.0

```python
x = [el[0] for el in p5_data]
y1 = [el[1] for el in p5_data]
y2 = [el[2] for el in p5_data]

plt.scatter(x, y1)
plt.scatter(x, y2)
plt.title("Time to execute")
plt.xlabel("bitsize")
plt.ylabel("seconds")
plt.show()
```

### * Problem 6. Diffie - Hellman Simulation
As we already saw, there are functions which are very easy to compute in the "forward" direction but really difficult (computationally) to invert (that is, determine the input from the output). There is a special case: the function may have a hidden "trap door". If you know where that door is, you can invert the function easily. This statement is at the core of modern cryptography.

Look up **Diffie - Hellman key exchange** (here's a [video](https://www.youtube.com/watch?v=cM4mNVUBtHk) on that but feel free to use anything else you might find useful).

Simulate the algorithm you just saw. Generate large enough numbers so the difference is noticeable (say, factoring takes 10-15 seconds). Simulate both participants in the key exchange. Simulate an eavesdropper.

First, make sure after both participants run the algorithm, they have *the same key* (they generate the same number).

Second, see how long it takes for them to exchange keys.

Third, see how long it takes the eavesdropper to arrive at the correct shared secret.

You should be able to see **the power of cryptography**. In this case, it's not that the function is irreversible. It can be reversed, but it takes a really long time (and with more bits, we're talking billions of years). However, if you know something else (this is called a **trap door**), the function becomes relatively easy to invert.

#### Solution
This solution has some classes in it since I wanted to make the simulation like a program.
The first element is a function to check if a number if prime or not as we will need to work with prime numbers.

```python
def is_prime(n: int, k=10) -> bool:
    """
    Function to test if a number is prime using basic rules and a Miller test.
    """
    low_primes = [3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97
                  , 101, 103, 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179
                  , 181, 191, 193, 197, 199, 211, 223, 227, 229, 233, 239, 241, 251, 257, 263, 269
                  , 271, 277, 281, 283, 293, 307, 311, 313, 317, 331, 337, 347, 349, 353, 359, 367
                  , 373, 379, 383, 389, 397, 401, 409, 419, 421, 431, 433, 439, 443, 449, 457, 461
                  , 463, 467, 479, 487, 491, 499, 503, 509, 521, 523, 541, 547, 557, 563, 569, 571
                  , 577, 587, 593, 599, 601, 607, 613, 617, 619, 631, 641, 643, 647, 653, 659, 661
                  , 673, 677, 683, 691, 701, 709, 719, 727, 733, 739, 743, 751, 757, 761, 769, 773
                  , 787, 797, 809, 811, 821, 823, 827, 829, 839, 853, 857, 859, 863, 877, 881, 883
                  , 887, 907, 911, 919, 929, 937, 941, 947, 953, 967, 971, 977, 983, 991, 997]

    def power(x: int, y: int, p: int) -> int:
        res = 1
        x = x % p
        while y > 0:
            if y & 1:
                res = (res * x) % p
            y = y >> 1
            x = (x * x) % p
        return res

    def miller_test() -> bool:
        d = n - 1
        while d % 2 == 0:
            d //= 2

        a = 2 + random.randint(1, n - 4)
        x = power(a, d, n)
        if x == 1 or x == n - 1:
            return True
        while d != n - 1:
            x = (x * x) % n
            d *= 2
            if x == 1:
                return False
            if x == n - 1:
                return True
        return False

    if n == 2 or n in low_primes:
        return True
    if n % 2 == 0 or ((n ** 2) - 1) % 12 != 0:
        return False
    if any(n % p == 0 for p in low_primes):
        return False
    for i in range(k):
        if not miller_test():
            return False
    return True

```

Next is a class to simulate the connection between the parties

```python
class Connection:
    """
    The connection object stores the public key and the list of exchnaged messages
    that each user ran read (including the man in the middle).
    """

    def __init__(self, pbk_nbits: int) -> None:
        """
        Initiate new connection class object with the size of the public key in number of bits.
        """
        self.pbk_nbits = pbk_nbits
        self.pbk = self.__gen_pbk()
        self.messages = []

    def send_message(self, sender_name: str, message: str) -> None:
        self.messages.append((sender_name, message))

    def __gen_pbk(self) -> tuple:
        p = self.__gen_prime()
        pf = primefactors(p - 1)
        for n in range(2, p):
            is_primitive = True
            if any(n ** ((p-1)//i) % p == 1 for i in pf):
                is_primitive = False
            if is_primitive:
                return p, n

    def __str__(self):
        return f'Connection public key (prime, base): {self.pbk}'

    def __gen_prime(self) -> int:
        while True:
            n = secrets.randbits(self.pbk_nbits)
            if n % 2 == 0:
                continue
            if is_prime(n):
                return n
```

Next is a class to simuilate the parties participating in the connection

```python
class User:
    """
    The user class stores the public and private keys of each party.
    The class has methods to generate private and public keys and exchange keys
    """

    def __init__(self, name: str) -> None:
        self.name = name
        self.connection = None
        self.keys = dict()

    def connect(self, connection: Connection) -> None:
        """
        Sets the connection to be used by the user.
        Deletes old keys and generate new ones.
        """
        if self.connection != connection:
            self.connection = connection
            self.keys.clear()
            self.generate_keys()

    def generate_keys(self) -> None:
        """
        Generates public and private key for the user
        """
        if self.connection:
            p, g = self.connection.pbk
            pk = secrets.randbelow(p)
            pbk = g ** pk % p
            self.keys['private'] = pk
            self.keys['public'] = pbk

    def exchange_pbk(self) -> None:
        """
        Sends the users public key over the connection
        """
        if self.connection:
            pbk = self.keys.get('public')
            self.connection.send_message(self.name, f'pbk:{pbk}')

    def generate_sk(self):
        """
        Generates the shared key
        """
        if self.connection:
            messages = [msg for sender, msg in self.connection.messages if sender != self.name]
            other_pbk = next(el.split(':')[1] for el in messages if 'pbk:' in el)
            if other_pbk:
                p = self.connection.pbk[0]
                pk = self.keys.get('private')
                sk = (int(other_pbk) ** pk) % p
                self.keys['shared'] = sk

    def __str__(self):
        return f'\nUser: {self.name}' \
               f'\nKeys: {self.keys}'
```

The last one is the class that will simulate the Diffie Hellman exchange.

```python
class Simulation:
    """
    Simulate the Diffie Hellman exchange.
    Stores information for the connection and the connected users.
    Has methods to add user, generate shared key, and crack the shared key.
    """

    def __init__(self, connection: Connection) -> None:
        self.connection = connection
        self.users = []

    def add_person(self, name: str):
        if name not in [user.name for user in self.users]:
            self.users.append(User(name))

    def generate_sks(self, verbose=False):
        """
        Generates shared keys and measures the time to do so
        """
        start = time()
        for user in self.users:
            user.connect(self.connection)
            user.exchange_pbk()
        for user in self.users:
            user.generate_sk()
            if verbose:
                print(user)
        exec_time = time() - start
        if verbose:
            print(f"Time to generate shared keys (sec): {exec_time}")

    def crack_sk(self, verbose=False):
        """
        Cracks the shared keys and measures teh time to do so.
        """
        start = time()
        pbks = {user.name: user.keys.get('public') for user in self.users}
        p, g = self.connection.pbk
        for pk_guess in range(1, p):
            pbk = g ** pk_guess % p
            if pbk in pbks.values():
                exec_time = time() - start
                if verbose:
                    username = next(name for name, key in pbks.items() if key == pbk)
                    other_pbk = next(item for item in pbks.values() if item != pbk)
                    sk = other_pbk ** pk_guess % p
                    print(f"Found {username}'s pk: {pk_guess}")
                    print(f"Shared key: {sk}")
                    print(f"Time to find shared key (sec): {exec_time}")
                return exec_time

```

Now to run the simulation.

```python
def run_sims(n_bits, attempts, verbose=False):
    results = []
    for i in range(attempts):
        conn = Connection(n_bits)
        if verbose:
            print(conn)
        sim = Simulation(conn)
        sim.add_person('Alice')
        sim.add_person('Bob')
        sim.generate_sks(verbose)
        results.append(sim.crack_sk(verbose))
    return results

# Uncomment the code below to run.
# Run with 'verbose=True' to see the actual keys for the connection and each party

# target_average_time = 15
# avg_crack_time = 0
# nbits = 8
# while avg_crack_time <= target_average_time:
#     results = run_sims(nbits, 10, False)
#     avg_crack_time = sum(results) / attempts
#     print(f'N_bits: {nbits}\nAverage crack time: {avg_crack_time}')
#     nbits += 2
```

<table >
    <tr><th > Number size (bits) <th><th> Avg. crack time (sec) <th><tr>
    <tr><td> 8 <td><td> 0.0000 <td><tr>
    <tr><td> 10 <td><td> 0.0002 <td><tr>
    <tr><td> 12 <td><td> 0.0014 <td><tr>
    <tr><td> 14 <td><td> 0.0624 <td><tr>
    <tr><td> 16 <td><td> 0.9004 <td><tr>
    <tr><td> 18 <td><td> 34.0687 <td><tr>
<table>

### ** Problem 7. The Galois Field in Cryptography
Research about the uses of the Galois field. What are its properties? How can it be used in cryptography? Write a simple cryptosystem based on the field.

You can use the following questions to facilitate your research:
* What is a field?
* What is GF(2)? Why is it an algebraic field?
* What is perfect secrecy? How does it relate to the participants in the conversation, and to the outside eavesdropper?
* What is symmetrical encryption?
* How to encrypt one-bit messages?
* How to extend the one-bit encryption system to many buts?
* Why is the system decryptable? How do the participants decrypt the encrypted messages?
* Why isn't the eavesdropper able to decrypt?
* What is a one-time pad?
    * How does the one-time pad achieve perfect secrecy?
* What happens if we try to use a one-time pad many times?
    * Provide an example where you break the "many-time pad" security
* What are some current enterprise-grade applications of encryption over GF(2)?
* Implement a cryptosystem based on GF(2). Show correctness on various test cases

### ** Problem 8. Huffman Compression Algorithm
Examine and implement the **Huffman algorithm** for compressing data. It's based on information theory and probability theory. Document your findings and provide your implementation.

This algorithm is used for **lossless compression**: compressing data without loss of quality. You can use the following checklist:

* What is the difference between lossless and lossy compression?
* When can we get away with lossy compression?
* What is entropy?
* How are Huffman trees constructed?
    * Provide a few examples
* How can we get back the uncompressed data from the Huffman tree?
* How and where are Huffman trees stored?
* Implement the algorithm. Add any other formulas / assumptions / etc. you might need.
* Test the algorithm. A good measure would be percentage compression: $$\frac{\text{compressed}}{\text{uncompressed}} * 100\%$$
* How well does Huffman's algorithm perform compared to other compression algorithms (e.g. LZ77)?

Math/Basic Algebra/Basic Algebra Exercise.ipynb


Before you turn this problem in, make sure everything runs as expected. First, **restart the kernel** (in the menubar, select Kernel$\rightarrow$Restart) and then **run all cells** (in the menubar, select Cell$\rightarrow$Run All).

**NB. Do not add new or remove/cut cells in the notebook. Additionally, do not change the filename of this notebook.**

Make sure you fill in any place that says `YOUR CODE HERE` or "YOUR ANSWER HERE", as well as your student number below:

```python
STUDENT_NUMBER = "141927"
```

---

# Exercises Topic 4

```python
# imports

import numpy as np
from matplotlib import pyplot as plt
```

## Exercise 4.1: Detecting periodicity (4 points)

In the folder that you obtained this notebook there is a file called
`sunspots.txt`, which contains the observed number of sunspots on the
Sun for each month since January 1749.  The file contains two columns of
numbers, the first representing the month and the second being the sunspot
number.

(a) Write a program that reads the data in the file and makes a graph of
  sunspots as a function of time.  You should see that the number of
  sunspots has fluctuated on a regular cycle for as long as observations
  have been recorded.  Make an estimate of the length of the cycle in
  months.

```python
def read_sunspot_data(filename):
    """Reads and returns sunspot data

    Args:
        filename (str): name of the file

    Returns:
        numpy array: the data"""

    data = None

    # YOUR CODE HERE
    data = np.loadtxt(filename)
    #raise NotImplementedError()

    return data

data = read_sunspot_data('sunspots.txt')
# plotting
# YOUR CODE HERE
plt.plot(data[:,0], data[:,1])
#raise NotImplementedError()
plt.xlabel('time (months)')
plt.ylabel('sunspot (#)')
```

### Answer (Double click to edit)
There seems to 24 peaks in a time period of 3143 months, or about 272 years.
This means that a peak occurs every ~131 months, or once in every 11 years.

```python
# validation
fn = read_sunspot_data('sunspots.txt')
assert isinstance(fn, np.ndarray), 'bad function'
assert fn.shape == (3143, 2), 'bad shape'
assert fn.dtype == float, 'bad type'
```

(b) Modify your program to calculate the Fourier transform of the sunspot
  data and then make a graph of the magnitude squared $|c_k|^2$ of the
  Fourier coefficients as a function of $k$---also called the
  *power spectrum* of the sunspot signal.  You should see that
  there is a noticeable peak in the power spectrum at a nonzero value
  of~$k$.  The appearance of this peak tells us that there is one frequency
  in the Fourier series that has a higher amplitude than the others around
  it---meaning that there is a large sine-wave term with this frequency,
  which corresponds to the periodic wave you can see in the original data.

```python
def discrete_fourier_transform(y):
    """Calculates coefficient of the discrete trasnform of 1D y

    Args:
        y (numpy array): 1D data

    Returns:
        numpy array: 1D array of coefficients after fourier transform"""
    # coefficients
    c = None

    # YOUR CODE HERE
    N = len(y)
    n = np.arange(N)
    c = np.zeros(N//2+1,complex)
    for k in range(N//2+1):
        c[k] = np.sum(y*np.exp(-2j*np.pi*k*n/N))
    #raise NotImplementedError()
    return c
```

```python
# validation
n = 10
fn = np.sin(np.arange(n))
dft = discrete_fourier_transform(fn)
c_k2 = np.abs(dft)**2
assert abs(c_k2[0] - 3.82284412) <= 1e-4, 'bad function'
assert abs(c_k2[1] - 10.28600735) <= 1e-4, 'bad function'
```

```python
fft_data = discrete_fourier_transform(data[:, 1])

#plotting
# YOUR CODE HERE
plt.plot(np.square(abs(fft_data)))
#raise NotImplementedError()
plt.xlabel('k')
plt.ylabel('$c_k^2$')
```

```python
plt.plot(np.square(abs(fft_data[0:50])))  # Zoom closer to the non-zero peak
#raise NotImplementedError()
plt.xlabel('k')
plt.ylabel('$c_k^2$')

k = 24
N = data.shape[0]
print(f"k = {k}: freq = {N}/{k} = {N/k}")
```

(c) Find the approximate value of $k$ to which the peak corresponds.
  What is the period of the sine wave with this value of $k$?  You should
  find that the period corresponds roughly to the length of the cycle that
  you estimated in part~(a).

This kind of Fourier analysis is a sensitive method for detecting
periodicity in signals.  Even in cases where it is not clear to the eye
that there is a periodic component to a signal, it may still be possible to
find one using a Fourier transform.

### Answer (Double click to edit)

From the plot, it can be visually estimated that the peak occurs approximately at $k=24$. When the total number of data points, 3143, is divided by $k$, we get the frequency: $N/k = 3143$ months $/24 \approx 130.96$ months, which is very close to the approximated value 131 months.

## Exercise 4.2: Fourier filtering and smoothing (6 points)

In the folder that you obtained this notebook you'll find a file called `dow.txt`.
It contains the daily closing value for each business day from late 2006
until the end of 2010 of the Dow Jones Industrial Average, which is a
measure of average prices on the US stock market.

Write a program to do the following:

(a) Read in the data from `dow.txt` and plot them on a graph.

```python
def read_dow_data(filename):
    """read dow data from filename

    Args:
        filename (str): name of the file

    Returns:
        numpy array: dow data"""
    data = None

    # YOUR CODE HERE
    data = np.loadtxt(filename)
    #raise NotImplementedError()

    return data

data = read_dow_data('dow.txt')

# plotting
# YOUR CODE HERE
plt.plot(data)
#raise NotImplementedError()
plt.xlabel('days')
plt.ylabel('closing value')
```

```python
# validation
fn = read_dow_data('dow.txt')
assert isinstance(fn, np.ndarray), 'bad function'
assert fn.shape == (1024,), 'bad function'
assert fn.dtype == float, 'bad function'
```

(b) Calculate the coefficients of the discrete Fourier transform of the
  data using the function `rfft` from `numpy.fft`, which produces
  an array of $\frac{1}{2} N+1$ complex numbers.

```python
def dft_numpy(y):
    """Perform numpy rfft

    Args:
        y (array): input data
    Returns:
        numpy array: dft of data
    """
    dft = None
    # modifications on y should not change incomming data too
    y = np.copy(y)

    # YOUR CODE HERE
    dft = np.fft.rfft(y)
    #raise NotImplementedError()

    return dft

# compute dft
dft_data = dft_numpy(data)
```

```python
# validation
n = 10
fn = np.sin(np.arange(n))
dft = dft_numpy(fn)
c_k2 = np.abs(dft)**2
assert abs(c_k2[0] - 3.82284412) <= 1e-4, 'bad function'
assert abs(c_k2[1] - 10.28600735) <= 1e-4, 'bad function'
```

(c) Now set all but the first 10\% of the elements of this array to zero
  (i.e.,~set the last 90\% to zero but keep the values of the first 10\%).

```python
def data_trim(y, x):
    """trim data, by making all but first x% elements to zero

    Args:
        y (numpy array): fft data
        x (float): percentage of first elements to be the same
    Returns:
        numpy array: trimmed data
    """
    partc_out = None
    # modifications on y should not change incomming data too
    y = np.copy(y)

    # YOUR CODE HERE
    i = int(len(y)*x/100)
    partc_out = np.zeros_like(y)
    partc_out[:i] = y[:i]
    #raise NotImplementedError()

    return partc_out

dft_data_trimmed = data_trim(dft_data, 10)
```

```python
# validation
a = np.arange(1, 11)
a_trimmed = data_trim(a, 10)
assert len(a_trimmed) == len(a), 'bad function'
for i in range(1, 10):
    assert len(np.where(data_trim(a, i*10)==0)[0]) == 10 - i, 'bad function at i = {}'.format(i)
```

(d) Calculate the inverse Fourier transform of the resulting array, zeros
  and all, using the function \verb|irfft|, and plot it on the same graph
  as the original data.  You may need to vary the colors of the two curves
  to make sure they both show up on the graph.  Comment on what you see.
  What is happening when you set the Fourier coefficients to zero?

```python
def idft_numpy(y):
    """Perform numpy irfft

    Args:
        y (array): input data
    Returns:
        numpy array: dft of data
    """
    idft = None
    # modifications on y should not change incomming data too
    y = np.copy(y)

    # YOUR CODE HERE
    idft = np.fft.irfft(y)
    #raise NotImplementedError()

    return idft

# compute idft
data_idft = idft_numpy(dft_data_trimmed)

# plotting
# YOUR CODE HERE
plt.plot(data, label='original')
plt.plot(data_idft, label='irfft')
plt.legend()
#raise NotImplementedError()
plt.xlabel('days')
plt.ylabel('closing value')
```

### Answer (Double click to edit)
The inverse FFT adapts to the original data quite well. Increasing the percentage x of the data used to perform the inverse FFT, i.e. decreasing the number of zeros, makes the irfft data more accurate. In other words, decreasing the percentage makes the function smoother. 10% data usage already seems to be enough to get very good approximation of the original data.

```python
# validation
n = 10
fn = np.sin(np.arange(n))
dft = dft_numpy(fn)
fn_idft = idft_numpy(dft)
assert np.mean(np.abs(fn - fn_idft)) <= 1e-4, 'bad function'
```

(e) Modify your program so that it sets all but the first 2\% of the
  coefficients to zero and run it again.

```python
# get trimmed data
# YOUR CODE HERE
dft_data_trimmed_2 = data_trim(dft_data, 2)
data_idft_2 = idft_numpy(dft_data_trimmed_2)
#raise NotImplementedError()

# plotting
# YOUR CODE HERE
plt.plot(data, label='original')
plt.plot(data_idft_2, label='irfft')
plt.legend()
#raise NotImplementedError()
plt.xlabel('days')
plt.ylabel('closing value')
```

## Exercise 4.3: Comparison of the DFT and DCT (3 points)

Exercise 4.2 looked at data representing the variation of the Dow Jones
Industrial Average, colloquially called "the Dow," over time.  The
particular time period studied in that exercise was special in one sense:
the value of the Dow at the end of the period was almost the same as at the
start, so the function was, roughly speaking, periodic.  In the folder that you obtained this notebook there is another file called `dow2.txt`, which also contains
data on the Dow but for a different time period, from 2004 until 2008.
Over this period the value changed considerably from a starting level
around 9000 to a final level around 14000.

(a) Write a program in
  which you read the data in the file `dow2.txt` and plot it on a
  graph.  Then smooth the data by calculating its Fourier transform,
  setting all but the first 2\% of the coefficients to zero, and inverting
  the transform again, plotting the result on the same graph as the
  original data.  You should see that the data are
  smoothed, but now there will be an additional artifact.  At the beginning
  and end of the plot you should see large deviations away from the true
  smoothed function.  These occur because the function is required to be
  periodic---its last value must be the same as its first---so it needs to
  deviate substantially from the correct value to make the two ends of the
  function meet.  In some situations (including this one) this behavior is
  unsatisfactory.  If we want to use the Fourier transform for smoothing,
  we would certainly prefer that it not introduce artifacts of this kind.

```python
# use already made functions in 4.2
# YOUR CODE HERE
data = read_dow_data('dow2.txt')
dft_data = dft_numpy(data)
dft_data_trimmed = data_trim(dft_data, 2)
data_idft = idft_numpy(dft_data_trimmed)
#raise NotImplementedError()

# plotting
# YOUR CODE HERE
plt.plot(data, label='original')
plt.plot(data_idft, label='irfft')
plt.legend()
#raise NotImplementedError()
plt.xlabel('days')
plt.ylabel('DOW closing values')
```

(b) Modify your program to repeat the same analysis using discrete cosine
  transforms.  You can use the functions from `dcst.py` (in the folder that you obtained this notebook) to perform the
  transforms if you wish.  Again discard all but the first 2\% of the
  coefficients, invert the transform, and plot the result.  You should see
  a significant improvement, with less distortion of the function at the
  ends of the interval.  This occurs because the cosine transform does not force the value of the
  function to be the same at both ends.

It is because of the artifacts introduced by the strict periodicity
of the DFT that the cosine transform is favored for many technological
applications, such as audio compression.  The artifacts can degrade the
sound quality of compressed audio and the cosine transform generally gives
better results.

The cosine transform is not wholly free of artifacts itself however.  It's
true it does not force the function to be periodic, but it does force the
gradient to be zero at the ends of the interval (which the ordinary Fourier
transform does not).  You may be able to see this in your calculations for
part (b) above.  Look closely at the smoothed function and you should see
that its slope is flat at the beginning and end of the interval.  The
distortion of the function introduced is less than the distortion in
part (a), but it's there all the same.  To reduce this effect, audio
compression schemes often use overlapped cosine transforms, in which
transforms are performed on overlapping blocks of samples, so that the
portions at the ends of blocks, where the worst artifacts lie, need not be
used.

```python
# use already made functions in 4.2 and dcst.py
from dcst import dct, idct
# YOUR CODE HERE
data = read_dow_data('dow2.txt')
dct_data = dct(data)
dct_data_trimmed = data_trim(dct_data, 2)
data_idct = idct(dct_data_trimmed)
#raise NotImplementedError()

# plotting
# YOUR CODE HERE
plt.plot(data, label='original')
plt.plot(data_idct, label='idct')
plt.legend()
#raise NotImplementedError()
plt.xlabel('days')
plt.ylabel('DOW closing values')

```

## Exercise 4.4: Image deconvolution (7 points)

You've probably seen it on TV, in one of those crime drama shows.
They have a blurry photo of a crime scene and they click a few buttons on
the computer and magically the photo becomes sharp and clear, so you can
make out someone's face, or some lettering on a sign.  Surely (like almost
everything else on such TV shows) this is just science fiction?  Actually,
no.  It's not.  It's real and in this exercise you'll write a program that
does it.

When a photo is blurred each point on the photo gets smeared out
according to some "smearing distribution," which is technically called a
*point spread function*.  We can represent this smearing
mathematically as follows.  For simplicity let's assume we're working with
a black and white photograph, so that the picture can be represented by a
single function $a(x,y)$ which tells you the brightness at each
point $(x,y)$.  And let us denote the point spread function by $f(x,y)$.
This means that a single bright dot at the origin ends up appearing as
$f(x,y)$ instead.  If $f(x,y)$ is a broad function then the picture is
badly blurred.  If it is a narrow peak then the picture is relatively
sharp.

In general the brightness $b(x,y)$ of the blurred photo at point $(x,y)$ is
given by

$$\begin{equation}
b(x,y) = \int_0^K \int_0^L a(x',y') f(x-x',y-y') \>d x'\>d y',
\end{equation}$$

where $K\times L$ is the dimension of the picture.  This equation is called
the *convolution* of the picture with the point spread
function.

Working with two-dimensional functions can get complicated, so to get the
idea of how the math works, let's switch temporarily to a one-dimensional
equivalent of our problem.  Once we work out the details in 1D we'll return
to the 2D version.  The one-dimensional version of the convolution above
would be

$$\begin{equation}
b(x) = \int_0^L a(x') f(x-x') \>d x'.
\end{equation}$$

The function $b(x)$ can be represented by a Fourier series as in Eq. (7.5):

$$\begin{equation}
b(x) = \sum_{k=-\infty}^\infty
         \tilde{b}_k \exp\biggl( i {2\pi k x\over L} \biggr),
\end{equation}$$

where

$$\begin{equation}
\tilde{b}_k = {1\over L} \int_0^L b(x)
                         \exp\biggl( -i {2\pi k x\over L} \biggr) \>d x
\end{equation}$$

are the Fourier coefficients.  Substituting for $b(x)$ in this equation
gives

$$\begin{align*}
\tilde{b}_k &= {1\over L} \int_0^L \int_0^L a(x') f(x-x')
                         \exp\biggl( -i {2\pi k x\over L} \biggr)
                         \>d x'\>d x \\
            &= {1\over L} \int_0^L \int_0^L a(x') f(x-x')
                         \exp\biggl( -i {2\pi k (x-x')\over L} \biggr)
                         \exp\biggl( -i {2\pi k x'\over L} \biggr)
                         \>d x'\>d x.
\end{align*}$$

Now let us change variables to $X=x-x'$, and we get

$$\begin{equation}
\tilde{b}_k = {1\over L} \int_0^L a(x')
                         \exp\biggl( -i {2\pi k x'\over L} \biggr)
                         \int_{-x'}^{L-x'} f(X)
                         \exp\biggl( -i {2\pi k X\over L} \biggr) \>d X
                         \>d x'.
\end{equation}$$

If we make $f(x)$ a periodic function in the standard fashion by repeating
it infinitely many times to the left and right of the interval from 0
to~$L$, then the second integral above can be written as

$$\begin{align*}
\int_{-x'}^{L-x'} f(X) \exp\biggl( -i {2\pi k X\over L} \biggr) \>d X
&= \int_{-x'}^0 f(X) \exp\biggl( -i {2\pi k X\over L} \biggr) \>d X
   \\
&\hspace{5em}{} + \int_0^{L-x'} f(X) \exp\biggl( -i {2\pi k X\over L}
   \biggr) \>d X \\
&\hspace{-12em} {} = \exp\biggl( i {2\pi k L\over L} \biggr)
   \int_{L-x'}^L f(X) \exp\biggl( -i {2\pi k X\over L} \biggr) \>d X
   + \int_0^{L-x'} f(X) \exp\biggl( -i {2\pi k X\over L} \biggr) \>d X
   \\
&\hspace{-12em} {} = \int_0^L f(X)
                    \exp\biggl( -i {2\pi k X\over L} \biggr) \>d X,
\end{align*}$$

which is simply $L$ times the Fourier transform $\tilde{f}_k$ of $f(x)$.
Substituting this result back into our equation for $\tilde{b}_k$ we then
get

$$\begin{align*}
\tilde{b}_k = \int_0^L a(x')
              \exp\biggl( -i {2\pi k x'\over L} \biggr)
              \tilde{f}_k \>d x'
            = L\,\tilde{a}_k\tilde{f}_k.
\end{align*}$$

In other words, apart from the factor of $L$, the Fourier transform of the
blurred photo is the product of the Fourier transforms of the unblurred
photo and the point spread function.

Now it is clear how we deblur our picture.  We take the blurred
picture and Fourier transform it to get $\tilde{b}_k =
L\,\tilde{a}_k\tilde{f}_k$.  We also take the point spread function and
Fourier transform it to get~$\tilde{f}_k$.  Then we divide one by the
other:

$$\begin{equation}
{\tilde{b}_k\over L\tilde{f}_k} = \tilde{a}_k
\end{equation}$$

which gives us the Fourier transform of the \emph{unblurred} picture.
Then, finally, we do an inverse Fourier transform on $\tilde{a}_k$ to get
back the unblurred picture.  This process of recovering the unblurred
picture from the blurred one, of reversing the convolution process, is
called *deconvolution*.

Real pictures are two-dimensional, but the mathematics follows through
exactly the same.  For a picture of dimensions $K\times L$ we find that the
two-dimensional Fourier transforms are related by

$$\begin{equation}
\tilde{b}_{kl} =  KL\tilde{a}_{kl}\tilde{f}_{kl}\,,
\end{equation}$$

and again we just divide the blurred Fourier transform by the Fourier
transform of the point spread function to get the Fourier transform of the
unblurred picture.

In the digital realm of computers, pictures are not pure functions $f(x,y)$
but rather grids of samples, and our Fourier transforms are discrete
transforms not continuous ones.  But the math works out the same again.

The main complication with deblurring in practice is that we don't usually
know the point spread function.  Typically we have to experiment with
different ones until we find something that works.  For many cameras it's a
reasonable approximation to assume the point spread function is Gaussian:

$$\begin{equation}
f(x,y) = \exp\biggl( -{x^2+y^2\over2\sigma^2} \biggr),
\end{equation}$$

where $\sigma$ is the width of the Gaussian.  Even with this assumption,
however, we still don't know the value of $\sigma$ and we may have to
experiment to find a value that works well.  In the following exercise, for
simplicity, we'll assume we know the value of $\sigma$.

(a) In the folder that you obtained this notebook you will find a file called `blur.txt` that
  contains a grid of values representing brightness on a black-and-white
  photo---a badly out-of-focus one that has been deliberately blurred using
  a Gaussian point spread function of width $\sigma=25$.  Write a program
  that reads the grid of values into a two-dimensional array of real
  numbers and then draws the values on the screen of the computer as a
  density plot.  You should see the photo appear.  If you get something
  wrong it might be upside-down.  Work with the details of your program
  until you get it appearing correctly.  (Hint: The picture has the sky,
  which is bright, at the top and the ground, which is dark, at the
  bottom.)

```python
def read_2D_data(filename):
    """Reads 2D data

    Args:
        filename (str): name of the file

    Returns:
        numpy array: the data"""

    data = None

    # YOUR CODE HERE
    data = np.loadtxt(filename)
    #raise NotImplementedError()

    return data

data = read_2D_data('blur.txt')

# plotting
# YOUR CODE HERE
plt.imshow(data, cmap='gray')
plt.show()
#raise NotImplementedError()
```

```python
# validation

data = read_2D_data('blur.txt')
assert isinstance(data, np.ndarray), 'bad function'
assert data.dtype == float, 'bad function'
assert data.shape == (1024, 1024), 'bad function'
```

(b) Write another program that creates an array, of the same size as the
  photo, containing a grid of samples drawn from the Gaussian~$f(x,y)$
  above with $\sigma=25$.  Make a density plot of these values on the
  screen too, so that you get a visualization of your point spread
  function.  Remember that the point spread function is periodic (along
  both axes), which means that the values for negative $x$ and $y$ are
  repeated at the end of the interval.  Since the Gaussian is centered on
  the origin, this means there should be bright patches in each of the four
  corners of your picture, something like this:

Note: This photo has a smaller shape, than the blur.txt, therefore the
spreads will not be as drastic as in the photo.

```python
def get_point_spread_function(data, sigma):
    """Retruns a gaussian point spread function

    Args:
        data (numpy array): the photo array to get the size of the spread function
        sigma (float): sigma of gaussian

    Returns:
        numpy array: the point spread function"""
    spread = np.zeros_like(data)

    # YOUR CODE HERE
    w = 2*sigma**2

    # Upper left corner
    x,y = np.meshgrid(np.arange(data.shape[0]), np.arange(data.shape[1]))
    d = x*x + y*y
    spread += np.exp(-d/w)

    # Upper right corner
    x,y = np.meshgrid(np.arange(data.shape[0]-1,-1,-1), np.arange(data.shape[1]))
    d = x*x + y*y
    spread += np.exp(-d/w)

    # Lower left corner
    x,y = np.meshgrid(np.arange(data.shape[0]), np.arange(data.shape[1]-1,-1,-1))
    d = x*x + y*y
    spread += np.exp(-d/w)

    # Lower right corner
    x,y = np.meshgrid(np.arange(data.shape[0]-1,-1,-1), np.arange(data.shape[1]-1,-1,-1))
    d = x*x + y*y
    spread += np.exp(-d/w)
    #raise NotImplementedError()

    return spread

point_func = get_point_spread_function(data, 25)

# plotting
# YOUR CODE HERE
plt.imshow(point_func, cmap='gray')
plt.show()
#raise NotImplementedError()
```

```python
# validation
fn = read_2D_data('blur.txt')
pt_fn = get_point_spread_function(fn, 25)
assert pt_fn.shape == fn.shape, 'shape mismatch'
# checking 90 deg rotational symmetry, since the gaussian is even in all corners
assert np.mean(np.abs(pt_fn - np.rot90(pt_fn, k=1))) <= 1e-4, 'bad function'
```

(c) Combine your two programs and add Fourier transforms using the
  functions `rfft2` and `irfft2` from `numpy.fft`, to make a
  program that does the following:

i) Calculates Fourier transform of both, photo and point spread function

ii) Divides one by the other

iii) Performs an inverse transform to get the unblurred photo

iv) Displays the unblurred photo on the screen

When you are done, you should be able to make out the scene in
the photo, although probably it will still not be perfectly sharp.

Hint: One thing you'll need to deal with is what happens when the Fourier
transform of the point spread function is zero, or close to zero.  In that
case if you divide by it you'll get an error (because you can't divide by
zero) or just a very large number (because you're dividing by something
small).  A workable compromise is that if a value in the Fourier transform
of the point spread function is smaller than a certain amount $\epsilon$
you don't divide by it---just leave that coefficient alone.  The value of
$\epsilon$ is not very critical but a reasonable value seems to be $10^{-3}$.

```python
def deconvolve(data, point_function):
    """Deconvolves point_function from the data

    Args:
        data (numpy array): the 2D data to deconvolve
        point_dunction (numpy array): the point function

    Returns:
        numpy array: Deconvolved array"""

    deconv_data = np.zeros_like(data)
    epsilon = 1e-3

    # YOUR CODE HERE
    # FFT for the blurred data
    rfft_data = np.fft.rfft2(data)

    # FFT for the spread function
    rfft_pf = np.fft.rfft2(point_function)

    # Assign elements lower than epsilon to 1 in spread func fft
    # (division by 1 = No division)
    rfft_pf[rfft_pf < epsilon] = 1

    # a_kl = b_kl / (f_kl * K * L)
    conv_data = rfft_data / (rfft_pf * data.shape[0] * data.shape[1])

    # Inverse FFT for the convoluted data
    deconv_data = np.fft.irfft2(conv_data)
    #raise NotImplementedError()

    return deconv_data

data = read_2D_data('blur.txt')
# try different values of sigma to get the best deconvolution without artefacts
point_func = get_point_spread_function(data, 19)
deconc_data = deconvolve(data, point_func)

# plotting
# YOUR CODE HERE
plt.imshow(deconc_data, cmap='gray')
plt.show()
#raise NotImplementedError()
```

(d) Bearing in mind this last point about zeros in the Fourier transform,
  what is it that limits our ability to deblur a photo?  Why can we not
  perfectly unblur any photo and make it completely sharp?

We have seen this process in action here for a normal snapshot, but it is
also used in many physics applications where one takes photos.  For
instance, it is used in astronomy to enhance photos taken by telescopes.
It was famously used with images from the Hubble Space Telescope
after it was realized that the telescope's main mirror had a serious
manufacturing flaw and was returning blurry photos---scientists managed to
partially correct the blurring using Fourier transform techniques.

### Answer (double click to edit)

If the values in the fourier transformed point function become very small, those points cannot be divided, i.e. it is left "untreated", which decreases the unblurring. If enough points are left untouched, the image cannot be properly unblurred. Because the fourier transform produces very small values for at least some of the points, the unblurring can never be perfect. Also, if the point function parameter sigma is increased too much, artifacts are introduced to the image. While increasing the sigma makes the image sharper, at some point it start to draw grid-like dots on the image, which occurs because the Gaussians become too wide.

```python

```