# Lab Assignment 2

### Sam Daucey, s2028017

## Presentation and coding style (3 marks)

In this assignment, some marks are allocated to your coding style and presentation. Try to make your code more readable using the tips given in your computer lab 2. Make sure your figures have good quality, right size, good range and proper labels.

## Task 1 (4 marks)

In this task we try to use several method from Lab 2 to solve the initial value problem

\begin{equation}
y' = 3y-4t, \quad y(0)=1,
\end{equation}

Set the step size to $h = 0.05$ and numerically solve this ODE from $t=0$ to $0.5$ using the following methods:

- Forward Euler

- Adams–Bashforth order 2

- Adams–Bashforth order 3 (we did not code this method in the computer lab, but you can find the formula on [this wikipedia page](https://en.wikipedia.org/wiki/Linear_multistep_method)). For this method, you need to build the very first two steps using other methods. For the first step, use the Euler scheme. For the second step, use Adams–Bashforth order 2.

Plot the three different approximations, and display the values in a table.

```python
# Import packages
import math
import numpy as np
import matplotlib.pyplot as plt

def AB_s_coeffs(s):
    """Returns the coefficients for the s-step Adams-Bashforth method up to s=3."""
    return {
        1: [1],
        2: [-1/2, 3/2],
        3: [5/12, -16/12, 23/12]
    }[s]

def AB_s_step(s, dy_dt, y_tup, t_tup, dt):
    """
    Returns an approximation for y_(n+1) using the s-step Adams-Bashforth method.
    ----------------------------------------------------------
    inputs:
        s: the order of the AB method
        dy_dt: the RHS function in the system of ODE
        y_tup: an s-tuple containing [y_(n-s+1), y_(n-s+2) ... y_n]
        t_tup: an s-tuple containing [t_(n-s+1), t_(n-s+2) ... t_n]
        dt: the step size
    """
    # Retreive the s-step coefficients
    coeffs =  AB_s_coeffs(s)

    # Compute the change in y using the Adams-Bashforth formula
    dy = dt * sum(
        c * dy_dt(y_i, t_i) for c, y_i, t_i in zip(coeffs, y_tup, t_tup)
    )

    # Use the change in y to return y_(n+1)
    y_n = y_tup[-1]
    return y_n + dy

def AB_s_solve(s, dy_dt, y_0, t_0, t_end, h):
    """
    Solves the differential equation t' = dy_dt(y, t) numerically using the s-step Adams-Bashforth method.
    ----------------------------------------------------------
    inputs:
        s: the order of the AB method
        dy_dt: the RHS function in the system of ODE
        y_0: initial condition on y,  y(t_0) = y_0
        t_0: initial time
        t_end: end time
        h: step size
    output:
        y: the solution of ODE.
        t: the times at which this solution was evaluated at: y[i] ~ y(t[i])"""

    # We need to add an extra step to account for the fact we want to evaluate at the start and end points.
    step_count = math.ceil((t_end - t_0)/h) + 1
    y = np.zeros(step_count)
    t = np.linspace(t_0, t_end, step_count)
    y[0] = y_0

    # Generate y[1], y[2] ... y[s-1] using the 1, 2 ... s-1 step AB methods
    for i in range(1, s):
        y_tup = y[:i]
        t_tup = t[:i]
        y[i] = AB_s_step(i, dy_dt, y_tup, t_tup, h)

    # Generate y[s], y[s+1] ... using the s step AB method
    for i in range(s, step_count):
        y_tup = y[i-s:i]
        t_tup = t[i-s:i]
        y[i] = AB_s_step(s, dy_dt, y_tup, t_tup, h)

    return t, y
```

```python
#  defining the function in the RHS of the ODE given in the question
def y_prime(y, t):
    return 3*y - 4*t
```

```python

# Note that Euler's method is just the one-step Adams-Bashforth method,
# Hence we can use s-step Adams-Bashforth to get all the solutions.
euler_soln = AB_s_solve(1, y_prime, 1, 0, 0.5, 0.05)
AB2_soln = AB_s_solve(2, y_prime, 1, 0, 0.5, 0.05)
AB3_soln = AB_s_solve(3, y_prime, 1, 0, 0.5, 0.05)

# Plot the results
fig, ax = plt.subplots(figsize=(14, 8))

ax.set_xlabel("t")
ax.set_ylabel("y")

ax.plot(*euler_soln, label="Euler")
ax.plot(*AB2_soln, label="2-step Adams-Bashforth")
ax.plot(*AB3_soln, label="3-step Adams-Bashforth")

ax.legend()
```

```python
from pandas import DataFrame

# printing the solution in a table:
DataFrame({
    "Euler": euler_soln[1],
    "2-step Adams-Bashforth": AB2_soln[1],
    "3-step Adams-Bashforth": AB3_soln[1]
}, index=euler_soln[0])

```

      Euler
      2-step Adams-Bashforth
      3-step Adams-Bashforth

      0.00
      1.000000
      1.000000
      1.000000

      0.05
      1.150000
      1.150000
      1.150000

      0.10
      1.312500
      1.318750
      1.318750

      0.15
      1.489375
      1.504219
      1.505391

      0.20
      1.682781
      1.708762
      1.711315

      0.25
      1.895198
      1.935417
      1.939662

      0.30
      2.129478
      2.187728
      2.194139

      0.35
      2.388900
      2.469811
      2.478979

      0.40
      2.677235
      2.786439
      2.799086

      0.45
      2.998820
      3.143152
      3.160162

      0.50
      3.358643
      3.546378
      3.568827

## Task 2 (3 marks)

Use `SymPy` to solve the differential equation $y' = 3y-4t$, with $y(0)=1$, present the analytical solution, and check the exact value of $y(0.5)$.

Compare the result with the approximations from the three methods in Task 1. You may use a table to show the results of each method at $y(0.5)$. Which method is the most/least accurate? Why?

```python
# standard setup
import sympy as sym
sym.init_printing()
from IPython.display import display_latex
import sympy.plotting as sym_plot

# Define symbols to use with sympy
t = sym.symbols("t")
y = sym.Function("y")
y_prime = y(t).diff(t)

# Define and solve the differential equation
diff_eq = sym.Eq(y_prime, 3*y(t) - 4*t)

sol = sym.dsolve(diff_eq, ics={y(0):1})

# Print the solution into the console
print("The solution is:")
display_latex(sol)

print("evaluated at 0.5:")
display_latex(sol.subs(t, 0.5))

# Substitute t into our symbolic solution for each t in our range
t_eval = np.linspace(0, 0.5, 11)
y_expr = sol.rhs
exact_soln = [y_expr.subs(t, t_i) for t_i in t_eval]

```

    The solution is:

$\displaystyle y{\left(t \right)} = \left(\frac{4 \left(3 t + 1\right) e^{- 3 t}}{9} + \frac{5}{9}\right) e^{3 t}$

    evaluated at 0.5:

$\displaystyle y{\left(0.5 \right)} = 3.60093837241004$

We can see that our most accurate approximation of $y(0.5)$ came from the order-3 Adams-Bashforth method, followed by the order-2 Adams Bashforth method,  with the least accurate being Euler's method. This is unsuprising, as the higher order methods will eventually have a lower global truncation error for sufficiently small h.

On top of this, the step size $h=0.05$ is still relatively large in comparison to how precise floating point numbers are, so the higher round-off error associated with doing more computations for the Adams-Bashforth methods had negligible effect. Hence the majority of the total global error was truncation error, which the higher order Adams-Bashforth methods performed better on.

```python

```

```python

```

```python

```

```python

```


```python
import numpy as np
import matplotlib.pyplot as plt
from sympy import Symbol, integrate
%matplotlib notebook
```

### Smooth local paths
We will use cubic spirals to generate smooth local paths. Without loss of generality, as $\theta$ smoothly changes from 0 to 1, we impose a condition on the curvature as follows

$\kappa = f'(\theta) = K(\theta(1-\theta))^n $

This ensures curvature vanishes at the beginning and end of the path. Integrating, the yaw changes as
$\theta = \int_0^x f'(\theta)d\theta$

With $n = 1$ we get a cubic spiral, $n=2$ we get a quintic spiral and so on. Let us use the sympy package to find the family of spirals

1. Declare $x$ a Symbol

2. You want to find Integral of $f'(x)$

3. You can choose $K$ so that all coefficients are integers

Verify if $\theta(0) = 0$ and $\theta(1) = 1$

```python
K =   30#choose for cubic/quintic
n =  2#choose for cubic/ quintic
x =  Symbol('x')#declare as Symbol
print(integrate(K*(x*(1-x))**n, x)) # complete the expression
```

    6*x**5 - 15*x**4 + 10*x**3

```python
#write function to compute a cubic spiral
#input/ output can be any theta
def cubic_spiral(theta_i, theta_f, n=10):
    x = np.linspace(0, 1, num=n)
    theta = (-2*x**3 + 3*x**2) * (theta_f-theta_i) + theta_i
    return theta
    # pass

def quintic_spiral(theta_i, theta_f, n=10):
    x = np.linspace(0, 1, num=n)
    theta = (6*x**5 - 15*x**4 + 10*x**3)* (theta_f-theta_i) + theta_i
    return theta
    # pass
def circular_spiral(theta_i, theta_f, n=10):
    x = np.linspace(0, 1, num=n)
    theta = x* (theta_f-theta_i) + theta_i
    return theta
```

### Plotting
Plot cubic, quintic spirals along with how $\theta$ will change when moving in a circular arc. Remember circular arc is when  $\omega $ is constant

```python
theta_i = 1.57
theta_f = 0
n = 10
x = np.linspace(0, 1, num=n)
plt.figure()
plt.plot(x,circular_spiral(theta_i, theta_f, n),label='Circular')
plt.plot(x,cubic_spiral(theta_i, theta_f, n), label='Cubic')
plt.plot(x,quintic_spiral(theta_i, theta_f, n), label='Quintic')

plt.grid()
plt.legend()
```

## Trajectory

Using the spirals, convert them to trajectories $\{(x_i,y_i,\theta_i)\}$. Remember the unicycle model

$dx = v\cos \theta dt$

$dy = v\sin \theta dt$

$\theta$ is given by the spiral functions you just wrote. Use cumsum() in numpy to calculate {}

What happens when you change $v$?

```python
v = 1
dt = 0.1
theta_i = 1.57
theta_f = 0
n = 100
theta_cubic = cubic_spiral(theta_i, theta_f, n)
theta_quintic = quintic_spiral(theta_i, theta_f, int(n+(23/1000)*n))
theta_circular = circular_spiral(theta_i, theta_f, int(n-(48/1000)*n))
# print(theta)
def trajectory(v,dt,theta):
    dx = v*np.cos(theta) *dt
    dy = v*np.sin(theta) *dt
    # print(dx)
    x = np.cumsum(dx)
    y = np.cumsum(dy)
    return x,y

# plot trajectories for circular/ cubic/ quintic
plt.figure()
plt.plot(*trajectory(v,dt,theta_circular), label='Circular')
plt.plot(*trajectory(v,dt,theta_cubic), label='Cubic')
plt.plot(*trajectory(v,dt,theta_quintic), label='Quintic')

plt.grid()
plt.legend()
```

## Symmetric poses

We have been doing only examples with $|\theta_i - \theta_f| = \pi/2$.

What about other orientation changes? Given below is an array of terminal angles (they are in degrees!). Start from 0 deg and plot the family of trajectories

```python
dt = 0.1
thetas = [15, 30, 45, 60, 90, 120, 150, 180] #convert to radians
plt.figure()
for tf in thetas:
    t = cubic_spiral(0, np.deg2rad(tf),50)
    x = np.cumsum(np.cos(t)*dt)
    y = np.cumsum(np.sin(t)*dt)
    plt.plot(x, y, label=f'0 to {tf} degree')
plt.grid()
plt.legend()
# On the same plot, move from 180 to 180 - theta
#thetas =
plt.figure()
for tf in thetas:
    t = cubic_spiral(np.pi, np.pi-np.deg2rad(tf),50)
    x = np.cumsum(np.cos(t)*dt)
    y = np.cumsum(np.sin(t)*dt)
    plt.plot(x, y, label=f'180 to {180-tf} degree')

plt.grid()
plt.legend()
```

Modify your code to print the following for the positive terminal angles $\{\theta_f\}$
1. Final x, y position in corresponding trajectory: $x_f, y_f$
2. $\frac{y_f}{x_f}$ and $\tan \frac{\theta_f}{2}$

What do you notice?
What happens when $v$ is doubled?

```python
dt = 0.1
thetas = [15, 30, 45, 60, 90, 120, 150, 180] #convert to radians
# plt.figure()
for tf in thetas:
    t = cubic_spiral(0, np.deg2rad(tf),50)
    x = np.cumsum(np.cos(t)*dt)
    y = np.cumsum(np.sin(t)*dt)
    print(f'tf: {tf} x_f : {x[-1]} y_f: {y[-1]} y_f/x_f : {y[-1]/x[-1]} tan (theta_f/2) : {np.tan(np.deg2rad(tf)/2)}')

```

    tf: 15 x_f : 4.936181599941893 y_f: 0.6498606361772978 y_f/x_f : 0.13165249758739583 tan (theta_f/2) : 0.13165249758739583
    tf: 30 x_f : 4.747888365557456 y_f: 1.2721928533042435 y_f/x_f : 0.2679491924311227 tan (theta_f/2) : 0.2679491924311227
    tf: 45 x_f : 4.444428497864582 y_f: 1.8409425608129912 y_f/x_f : 0.41421356237309487 tan (theta_f/2) : 0.41421356237309503
    tf: 60 x_f : 4.040733895009051 y_f: 2.3329188020071205 y_f/x_f : 0.5773502691896257 tan (theta_f/2) : 0.5773502691896257
    tf: 90 x_f : 3.0152040529843056 y_f: 3.0152040529843065 y_f/x_f : 1.0000000000000002 tan (theta_f/2) : 0.9999999999999999
    tf: 120 x_f : 1.8653713069408235 y_f: 3.230917878602665 y_f/x_f : 1.7320508075688772 tan (theta_f/2) : 1.7320508075688767
    tf: 150 x_f : 0.8004297415440109 y_f: 2.9872444633314705 y_f/x_f : 3.732050807568873 tan (theta_f/2) : 3.7320508075688776
    tf: 180 x_f : 5.551115123125783e-17 y_f: 2.3817721504025933 y_f/x_f : 4.290619267613818e+16 tan (theta_f/2) : 1.633123935319537e+16

These are called *symmetric poses*. With this spiral-fitting approach, only symmetric poses can be reached.

In order to move between any 2 arbitrary poses, you will have to find an intermediate pose that is pair-wise symmetric to the start and the end pose.

What should be the intermediate pose? There are infinite possibilities. We would have to formulate it as an optimization problem. As they say, that has to be left for another time!

```python

```


# Damped, driven linear oscillator: various cases

The examples shown in this notebook were taken from M. Tabor, Chaos and Integrability in Nonlinear Dynamics, Chap. 1

Here, we consider numerical solutions to ODEs which represent a system subjected to some form of external time-dependent force $F(t)$. Particularly interesting are those cases where $F(t)$ is a periodic function, for example, $F(t) \approx cos(\Omega t)$. Consider as an example the damped, driven linear oscillator

$$
\ddot{x}+\lambda \dot{x}+\omega^2 x = \epsilon F(t)
$$

where $\epsilon$ can be thought of as a 'coupling parameter' --in the limit $\epsilon\rightarrow 0$, the system becomes autonomous again. We can re-write this differential equation as:

$$
\begin{align}
    \begin{split}
        \dot{x}&=y \\
        \dot{y}&=-\lambda y -\omega^2 x + \epsilon \cos(\Omega t) \\
    \end{split}
\end{align}
$$

We will study this system, and consider several distinct cases. We will integrate it using `TaylorIntegration`, and visualize the results using `Plots`:

```julia
using TaylorIntegration, Plots
pyplot()
```

    WARNING: Method definition macroexpand(Module, Any) in module Compat at /Users/Jorge/.julia/v0.6/Compat/src/Compat.jl:1491 overwritten in module MacroTools at /Users/Jorge/.julia/v0.6/MacroTools/src/utils.jl:64.

    Plots.PyPlotBackend()

Before we start, we fix some integration parameters (the order of Taylor expansions and the local absolute error tolerance):

```julia
const abstol = 1e-30
const order = 28
```

    28

## 1. The driven oscillator (non-resonant)

First, we will consider the non-resonant, frictionless, linear oscillator:

$$
\ddot{x}+\omega^2x=\epsilon\cos(\Omega t),
$$
which we will rewrite as:

$$
\begin{align}
\begin{split}
\dot{x}_1 &= x_2 \\
\dot{x}_2 &= -\omega^2x_1+\epsilon\cos(x_3) \\
\dot{x}_3 &= \Omega
\end{split}
\end{align}
$$

This ODE is subject to initial conditions $x(0)=x_0$ and $\dot{x}(0)=v_0$. Here, we have $\lambda=0$ (no friction), and $\omega \neq \Omega$ (non-resonant condition).

```julia
const Ω = 1.0 #forcing frequency
const ϵ = 0.5 #forcing amplitude
const ω = 1.1 #"natural" frequency
const T = 2π/ω #period associated to oscillator's ω

#the driven linear oscillator ODE:
function drivenosc!(t, x, dx)
    dx[1] = x[2]
    dx[2] = -(ω^2)*x[1]+ϵ*cos(Ω*t)
    nothing
end
```

    drivenosc! (generic function with 1 method)

The initial time, initial condition and final time are:

```julia
const t0 = 0.0 #initial time
const x0 = [1.0,0.0] #initial condition
#const x0 = [1.0,0.0,Ω*t0] #initial condition
const tmax = 100*T # 200*T #final time of integration
```

    571.1986642890532

Then, the particular solution is:

$$
x_\mathrm{part}(t)=\frac{\epsilon}{\omega^2-\Omega^2} \cos(\Omega t)
$$

```julia
x_part(t) = ϵ*cos(Ω*t)/(ω^2-Ω^2)
```

    x_part (generic function with 1 method)

The overall solution is:

$$
x_\mathrm{gral}(t) = a\sin(\omega t +\delta)+\frac{\epsilon}{\omega^2-\Omega^2} \cos(\Omega t)
$$

where

$$
\begin{align}
\begin{split}
a &= \sqrt{ \left( x_0-\frac{\epsilon}{\omega^2-\Omega^2}\right)^2+\left( \frac{\dot{x}_0}{\omega} \right)^2 } \\
\delta &= \arctan\left( \frac{ x_0-\frac{\epsilon}{\omega^2-\Omega^2} }{ \dot{x}_0/\omega } \right)
\end{split}
\end{align}
$$

```julia
const a = sqrt( (x0[1]-ϵ/(ω^2-Ω^2))^2+(x0[2]^2/ω^2) ) #amplitude
const δ = atan((ω*(x0[1]-ϵ/(ω^2-Ω^2))/x0[2])) #phase shift

x_gral(t) = a*sin(ω*t+δ)+x_part(t)

a, δ
```

    (1.3809523809523787, -1.5707963267948966)

We can check if our expression for `x_gral ` is consistent, using `TaylorSeries`:

```julia
t_poly = TaylorSeries.Taylor1([0.0, 1.0], 3) #A Taylor polynomial which represents time
x_gral(t_poly) #evaluate x_gral at t_poly
```

     1.0 + 9.30148402209536e-17 t - 0.3550000000000001 t² - 1.8757992777892314e-17 t³ + 𝒪(t⁴)

We can get the numerical value of the derivatives of `x_gral` up to any order we want! In particular, its 0-th and 1st order coefficients are consistent, within machine epsilon, with the initial conditions $x_0=1$, $\dot{x}_0=0$.

Now, let's integrate the ODE:

```julia
@time tT, xT = taylorinteg(drivenosc!, x0, t0, tmax, order, abstol, maxsteps=50000);
```

      0.371958 seconds (612.46 k allocations: 56.206 MiB, 4.59% gc time)

In this case, the solution represents the sum of two sinusoidal signals with different frequencies and therefore we can observe the phenomenon of beats:

```julia
# x vs t, the first steps
firsti =1 # length(tT2)-20
lasti =10 # length(tT2)
myrange=firsti:lasti
lint = linspace(tT[firsti], tT[lasti], 10*length(myrange))
plot(
lint/T,
x_gral.(lint),
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
label="analytical"
#leg=false
)
plot!(
tT[myrange]/T,
xT[myrange,1],
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
label="numerical",
leg=false
)
scatter!(
tT[myrange]/T,
xT[myrange,1],
label="numerical",
leg=true,
ms=3.0
)
```

```julia
# x vs t
plot(
tT/T,
xT[:,1],
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
leg=false
)
```

Now, how does the numerical solution compare to the analytical solution? Well, we can measure this via the difference between the analytical solution and the numerical solution at each time step:

$$
\Delta x(t) = x_\mathrm{num}(t)-x_\mathrm{gral}(t)
$$

We measure this in machine epsilons (`eps()`):

```julia
#the absolute error in machine epsilons
Δx = (xT[:,1]-x_gral.(tT))/eps()
#absolute error, numerical solution, analytic solution, at last time step:
Δx[end], xT[end,1], x_gral(tT[end])
```

    (945.5, 0.6220322210267172, 0.6220322210265072)

At the end of the integration, the error in the numerical solution is less than 700`eps()`! Now, let's plot the absolute error as a function of time:

```julia
plot(
tT/T,
Δx, #the absolute error in machine epsilons
xaxis="time, t",
yaxis="absolute error (machine epsilons)",
title="x vs t",
leg=false
)
```

## 2. The driven oscillator: resonance

When $\Omega \rightarrow \omega$, the solution

$$
\begin{align}
x_\mathrm{gral}(t) &= a\sin(\omega t +\delta)+\frac{\epsilon}{\omega^2-\Omega^2} \cos(\Omega t)
\end{align}
$$

given before, breaks down. This condition is an example of resonance. In this case, the ODE we are trying to solve is:

$$
\begin{align}
  \begin{split}
    \dot{x}_1 &= x_2 \\
    \dot{x}_2 &= -\omega^2x_1+\epsilon\cos(\omega t) \\
  \end{split}
\end{align}
$$

and the general solution (for $x_1$) is:

$$
x_\mathrm{gral}(t) = a\sin(\omega t +\delta)+\frac{\epsilon t}{2\omega} \cos(\omega t)
$$

Note that, once we know $x_1$, then $x_2$ can be calculated straightforward; $x_3(t)=\omega t$ by definition. The coefficients $a$ and $\delta$ are given in this case by:

$$
\begin{align}
\begin{split}
a &= \sqrt{x_0^2+\dot{x}_0^2/\omega^2} \\
\delta &= \arctan \left( \frac{\omega x_0}{\dot{x}_0} \right)
\end{split}
\end{align}
$$

```julia
#the resonant driven linear oscillator ODE:
function drivenoscres!(t, x, dx)
    dx[1] = x[2]
    dx[2] = -(ω^2)*x[1]+ϵ*cos(ω*t)
    nothing
end
```

    drivenoscres! (generic function with 1 method)

```julia
const a′ = sqrt( x0[1]^2+(x0[2]^2/ω^2) ) #amplitude
const δ′ = atan((ω*x0[1]/x0[2])) #phase shift

x_gral_res(t) = a′*sin(ω*t+δ′)+ϵ*t*sin(ω*t)/(2ω) #overall solution with resonance
```

    x_gral_res (generic function with 1 method)

If we want to test the consistency of our expression for `x_gral`, we can do so by evaluating it using `TaylorSeries`:

```julia
x_gral_res(TaylorSeries.Taylor1([0.0, 1.0], 3))
```

     1.0 + 6.735557395310444e-17 t - 0.3550000000000001 t² - 1.3583374080542732e-17 t³ + 𝒪(t⁴)

We're ready to perform a Taylor integration for the resonant driven linear oscillator:

```julia
const tmax = 100*T # 200*T
```

    571.1986642890532

```julia
@time tT2, xT2 = taylorinteg(drivenoscres!, x0, t0, tmax, order, abstol, maxsteps=50000);
```

      0.127499 seconds (518.14 k allocations: 54.571 MiB, 22.92% gc time)

```julia
# x vs t, the first steps
firsti2 =1 # length(tT2)-20
lasti2 =10 # length(tT2)
myrange2=firsti2:lasti2
lint2 = linspace(tT2[firsti2], tT2[lasti2], 10*length(myrange2))
plot(
lint2/T,
x_gral_res.(lint2),
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
leg=false
)
plot!(
tT2[myrange2]/T,
xT2[myrange2,1],
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
leg=false,
)
scatter!(
tT2[myrange2]/T,
xT2[myrange2,1],
leg=false,
ms=3.0
)
```

```julia
# x vs t
plot(
tT2/T,
xT2[:,1],
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
leg=false
)
plot!(
tT2/T,
x_gral_res.(tT2),
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
leg=false
)
```

The solution grows unbounded with time! Physically, this means that the external source is imparting energy to the system, and there are no losses of energy due to any damping ($\lambda = 0$).

Once again we ask: how does this numerical solution compare to the analytical solution?

```julia
#the absolute error in machine epsilons
Δx2 = (xT2[:,1]-x_gral_res.(tT2))/eps();
Δx2[end], xT2[end,1], x_gral_res(tT2[end])
```

    (237767.0, 1.000000000053305, 1.00000000000051)

```julia
plot(
tT2/T,
Δx2, #the absolute error in machine epsilons
xaxis="time, t",
yaxis="absolute error (machine epsilons)",
title="x vs t",
leg=false
)
```

Now, try doing longer integrations for the system above. How does the absolute error behave for longer times of integration? Does it grow systematically? Is there a way to know if the error in our integrations is dominated by roundoff errors?

## 3. The damped driven oscillator ($\lambda \neq 0$)

We return to the original problem:

$$
\begin{align}
    \begin{split}
        \dot{x}_1&=x_2 \\
        \dot{x}_2&=-\lambda x_2 -\omega^2 x_1 + \epsilon \cos(\Omega t) \\
    \end{split}
\end{align}
$$

i.e., here $\lambda \neq 0$. In this case, the overall solution may be written as:

$$
x(t) = Ae^{-\lambda t/2}\cos(\nu t + \Delta)+\frac{\epsilon\cos(\Omega t+\Delta_\mathrm{ext})}{\sqrt{(\omega^2-\Omega^2)^2+\lambda^2\Omega^2}}
$$

where the intrinsic frequency $\nu$ is

$$
\nu = \frac{1}{2}\sqrt{4\omega^2-\lambda^2}
$$

and the "external" phase $\Delta_\mathrm{ext}$ is given by

$$
\Delta_\mathrm{ext} = \arctan \left( \frac{ -\lambda\Omega }{ (\omega^2-\Omega^2) } \right)
$$

We actually have three sub-cases,  depending on the value of $4\omega^2-\lambda$:

+ $4\omega^2-\lambda>0$: sub-damping
+ $4\omega^2-\lambda=0$: critical damping
+ $4\omega^2-\lambda<0$: super-damping

Here, we will analyze only the first case, but we only need to change the values of $\omega$ and $\lambda$ and then run the integration again to analyze those other cases too!

The amplitud $A$ and phase $\Delta$ of the homogeneous part are determined by the initial conditions and are given by

$$
\begin{align}
\begin{split}
A &= \sqrt{(A\cos\Delta)^2+(A\sin\Delta)^2} \\
\Delta &= \arctan\left( \frac{A\sin\Delta}{A\cos\Delta} \right)
\end{split}
\end{align}
$$

where

$$
\begin{align}
\begin{split}
A\sin\Delta &= \frac{1}{\nu}( \frac{\lambda}{2}(\alpha-x_0)-(\dot{x}_0+\beta) ) \\
A\cos\Delta &= x_0-\alpha
\end{split}
\end{align}
$$

and, in turn,

$$
\begin{align}
\begin{split}
\alpha &= \frac{\epsilon\cos(\Delta_\mathrm{ext})}{\sqrt{(\omega^2-\Omega^2)^2+\lambda^2\Omega^2}} \\
\beta &= \frac{\epsilon\Omega\sin(\Delta_\mathrm{ext})}{\sqrt{(\omega^2-\Omega^2)^2+\lambda^2\Omega^2}}
\end{split}
\end{align}
$$

```julia
const Ω = 2.0 #forcing frequency
const ϵ = 0.5 #forcing amplitude
const ω = 1.1 #"natural" frequency
const λ = 0.2 #damping
```

    WARNING: redefining constant Ω

    0.2

```julia
#the driven, damped linear oscillator ODE:

function drivdamposc!(t, x, dx)
    dx[1] = x[2]
    dx[2] = -λ*x[2]-(ω^2)*x[1]+ϵ*cos(Ω*t)
    nothing
end
```

    drivdamposc! (generic function with 1 method)

```julia
#two-variable versions of atan(y/x)
myatan(x, y) = y>=zero(x)?( x>=zero(x)?atan(y/x):(atan(y/x)+pi) ):( x>=zero(x)?(atan(y/x)+2pi):(atan(y/x)+pi) )
myatan2(x, y) = y>=zero(x)?( x>=zero(x)?atan(y/x):(atan(y/x)-pi) ):( x>=zero(x)?(atan(y/x)):(atan(y/x)+pi) )

const t0 = 0.0
const x0 = [5.0,0.0] #initial condition

const ν = 0.5*sqrt(4(ω^2)-λ^2) #the intrinsic frequency
const Text = 2π/Ω #period associated to external frequency Ω
const Δext = myatan2( (ω^2-Ω^2), (-λ*Ω) ) #the "external" phase

#some auxiliary variables... (see text)
const α = ϵ*cos(Δext)/sqrt( (ω^2-Ω^2)^2+(λ^2)*(Ω^2) )
const β = ϵ*Ω*sin(Δext)/sqrt( (ω^2-Ω^2)^2+(λ^2)*(Ω^2) )
const AcosΔ = x0[1]-α
const AsinΔ = ( (λ/2)*(α-x0[1])-(x0[2]+β) )/ν

const A = sqrt( AsinΔ^2+AcosΔ^2 ) #the homogeneous amplitude
#we have to be careful with the homogeneous phase, when inverting tan:
#if atan( AsinΔ./AcosΔ ) < 0
#    const Δ = atan( AsinΔ./AcosΔ )+π #homogeneous phase, case 1
#else
#    const Δ = atan( AsinΔ./AcosΔ ) #homogeneous phase, case 2
#end

const Δ = myatan2( AcosΔ , AsinΔ )

println("A=", A)
println("Δ=", Δ)
println("AcosΔ=", AcosΔ)
println("AsinΔ=", AsinΔ)
println("Δext=", Δext)
```

    A=5

    WARNING: redefining constant x0

    .193145415819222
    Δ=-0.08222027678306153
    AcosΔ=5.175602019108521
    AsinΔ=-0.42650093744798
    Δext=3.2839914642983628

```julia
#the general solution to the damped driven linear oscillator:
x_ddo(t) = A*exp(-λ*t/2)*cos(ν*t+Δ)+ϵ*cos(Ω*t+Δext)/sqrt( (ω^2-Ω^2)^2+(λ^2)*(Ω^2) )
```

    x_ddo (generic function with 1 method)

Again, we check for consistency of our expression between the analytical solution, `x_ddo`, and the given initial conditions:

```julia
x_ddo(TaylorSeries.Taylor1([0.0, 1.0], 3))
```

     5.0 - 7.632783294297951e-17 t - 2.7750000000000004 t² + 0.18500000000000005 t³ + 𝒪(t⁴)

The final time of integration is:

```julia
const tmax = 100*Text
```

    WARNING: redefining constant tmax

    314.1592653589793

We're now ready to integrate:

```julia
@time tT3, xT3 = taylorinteg(drivdamposc!, x0, t0, tmax, order, abstol, maxsteps=50000);
```

      0.170335 seconds (523.14 k allocations: 55.079 MiB, 11.53% gc time)

How does the solution $x(t)$ look like during the first few steps? Let's plot it!

```julia
# x vs t, the first firsti3-lasti3 steps, starting from the firsti3-th step

firsti3 = 1 # length(tT3)-200 #1 # length(tT3)-20
lasti3 = 600 # length(tT3) #10 # length(tT3)
myrange3=firsti3:lasti3
lint3 = linspace(tT3[firsti3], tT3[lasti3], 10*length(myrange3))
plot(
lint3/Text,
x_ddo.(lint3),
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
leg=false
)
plot!(
tT3[myrange3]/Text,
xT3[myrange3,1],
xaxis="time, t",
yaxis="x(t)",
title="x vs t",
leg=false,
)
scatter!(
tT3[myrange3]/Text,
xT3[myrange3,1],
leg=false,
ms=3.0
)
```

As $t$ increases, the homogeneous (exponential) part of the solution decays and one is finally left with

$$
\lim_{t\to\infty} x(t) = \frac{\epsilon\cos(\Omega t+\Delta_\mathrm{ext})}{\sqrt{(\omega^2-\Omega^2)^2+\lambda^2\Omega^2}}
$$

Note that, unlike the case $\lambda=0$, although the amplitude of the motion becomes large at the resonance $\Omega\to\omega$, it does not diverge. Physically, this asymptotic solution corresponds to steady oscillations of constant energy, which represents a balance between the energy pumped to the system and the energy dissipated by friction.

As always, we ask, just how good, quantitatively, is the numerical solution compared to the analytical solution? To answer that (again, as always), we will plot the absolute error as a function of time:

```julia
#the absolute error in machine epsilons
Δx3 = (xT3[:,1]-x_ddo.(tT3))/eps();
Δx3[end], xT3[end,1], x_ddo(tT3[end])
```

    (63.75, -0.17560201910850107, -0.17560201910851522)

```julia
plot(
tT3/Text,
Δx3, #the absolute error in machine epsilons
xaxis="t (natural periods)",
yaxis="absolute error (machine epsilons)",
title="Absolute error vs time",
leg=false
)
```

```julia

```


# A first example how to use Function_from_Expression

Determine the equation of a linear function

$$
  f(x) = a\,x + b
$$

where the points $P(2|3)$ and $Q(-2,5)$ belong to the graph

$$
  y=f(x)
$$

of $f$ and plot the graph $y=f(x)$ for $-4\le x\le 4$.

## Solution

To solve this, we first need some initialisations:

```python
from sympy import *
init_printing()

import matplotlib.pyplot as plt

# usually you want this
%matplotlib inline

# useful for OS X
%config InlineBackend.figure_format='retina'

import numpy as np

from IPython.display import display, Math

from fun_expr import Function_from_Expression as FE
```

In the next step, the function $f$ is defined with two unknown coefficients $a$, $b$:

```python
# define the function

# the variable of the function
x = Symbol('x')

# the unknown coefficients:
a,b = symbols('a,b')

# the function
f = FE(x, a*x+b)

# display result
f
```

```python
# or, better
Math("f(x)="+latex(f(x)))
```

$$f(x)=a x + b$$

The unknown coefficients $a$, $b$ need to be determined. To do this, a system of linear equations is derived from the given points:

```python
# define points:
x_p,y_p = 2,3
x_q,y_q = -2,5

pts = [(x_p,y_p),(x_q,y_q)]

# define equations
eqns = [Eq(f(x_p),y_p),
        Eq(f(x_q),y_q)]

# display result
for eq in eqns:
    display(eq)
```

The resulting system of equations is solved:

```python
# solve equations
sol = solve(eqns)

# display result
sol
```

The solution is substituted into the function.

```python
# substitute results into f
f = f.subs(sol)

# display result
Math("f(x)="+latex(f(x)))
```

$$f(x)=- \frac{x}{2} + 4$$

The resulting function is plotted over the interval $-4\le x \le 4$

```python
# define new plot
fig, ax = plt.subplots()

# the interval along the x-axis
lx = np.linspace(-4,4)

# plot f(x)
ax.plot(lx,f.lambdified(lx),label=r"$y={f}$".format(f=latex(f(x))))

# insert the given points to the graph
ax.scatter(*zip(*pts))

# some additional commands
ax.grid(True)
ax.axhline(0,c='k')
ax.axvline(0,c='k')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.legend(loc='upper right')
```


# Exercise: beam bending with the least squares method

The differential equation for beam bending
$$
(EI w'')'' - q = 0
$$
can be integrated analytically by specifying the linear loading as $q = q_0\frac{x}{L}$ and the boundary conditions
$$
w(0) = 0\\
w(L) = 0\\
w''(0)=0\\
w''(L) = 0
$$
to yield the deflection solution ("Biegelinie"):
$$
w(x) = \frac{q_0 L^4}{360 EI} \left[ 3\left(\frac{x}{L}\right)^5 - 10 \left(\frac{x}{L}\right)^3 + 7 \left(\frac{x}{L}\right)\right]
$$
We now seek to approximate this solution numerically using the least squares method.

```python
import numpy as np #numerical methods
import sympy as sp #symbolic operations
import matplotlib.pyplot as plt #plotting
sp.init_printing(use_latex='mathjax') #makes sympy output look nice

#Some plot settings
plt.style.use('seaborn-deep')
plt.rcParams['lines.linewidth']= 2.0
plt.rcParams['lines.color']= 'black'
plt.rcParams['legend.frameon']=True
plt.rcParams['figure.figsize'] = (8, 6)
plt.rcParams['font.family'] = 'serif'
plt.rcParams['legend.fontsize']=14
plt.rcParams['font.size'] = 14
plt.rcParams['axes.spines.right'] = False
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.left'] = True
plt.rcParams['axes.spines.bottom'] = True
plt.rcParams['axes.axisbelow'] = True
```

```python
#Defining the geometric an material characteristics as symbolic quantities
L,q,EI,x = sp.symbols('L q_0 EI x')
```

```python
#Analytical solution to deflection
def deflection_analytical():
    a = x/L
    f = q*L**4/(360 * EI)
    return f*(3*a**5 - 10*a**3 + 7*a)
```

```python
deflection_analytical() #check definition
```

$$\frac{L^{4} q_{0}}{360 EI} \left(\frac{7 x}{L} - \frac{10 x^{3}}{L^{3}} + \frac{3 x^{5}}{L^{5}}\right)$$

Now, let's plot the analytical solution. For that purpose, we use some Python magic ("lambdify"). We sample the analytical solution for $x \in [0,L]$ at 100 points and plot the dimensionless deflection over the dimensionless length.

```python
lam_x = sp.lambdify(x, deflection_analytical(), modules=['numpy'])
#For the variable x the function deflection_analytical() will obtain something

x_vals = np.linspace(0, 1, 100)*L #This something is x from 0 to L
analytical = lam_x(x_vals) #We calculate the solution by passing x = [0,...,L] to deflection_analytical

plt.plot(x_vals/L, analytical/(L**4*q)*EI)
plt.xlabel('$x / L$')
plt.ylabel('$w / L^4 q_0  EI^{-1}$')
```

## Trigonometric Ansatz

Let's try the approximation
$$
    \tilde{w} = a_1 \sin \left(\pi \frac{x}{L}\right) + a_2 \sin \left(2\pi\frac{x}{L}\right)
$$

```python
a1, a2 = sp.symbols('a_1 a_2')#Defining the free values as new symbols
```

```python
def deflection_ansatz():
    return a1*sp.sin(sp.pi/L*x) + a2*sp.sin(2*sp.pi/L*x) #defining the approximate solution with the unknown coefficients
```

Now we substitute this solution into the fourth-order ODE for beam bending by differentiating it twice and adding the distributed loading:
$$
EI w^\text{IV} - q_0 \frac{x}{L} = 0 \text{ with } EI = \text{const.}
$$

```python
r = EI * deflection_ansatz().diff(x,4) - q * (x/L) #residual
r
```

$$\frac{\pi^{4} EI}{L^{4}} \left(a_{1} \sin{\left (\frac{\pi x}{L} \right )} + 16 a_{2} \sin{\left (\frac{2 \pi}{L} x \right )}\right) - \frac{q_{0} x}{L}$$

Now we perform the derivatives with respect to the Ansatz free values for obtaining the stationarity of $\int 1/2 r^2 \text{d} x$:

```python
dr_da1 = r.diff(a1)
dr_da2 = r.diff(a2)
dr_da1, dr_da2
```

$$\left ( \frac{\pi^{4} EI}{L^{4}} \sin{\left (\frac{\pi x}{L} \right )}, \quad \frac{16 EI}{L^{4}} \pi^{4} \sin{\left (\frac{2 \pi}{L} x \right )}\right )$$

This yields the two equations to be solved:

```python
stationarity_conditions = (sp.integrate(dr_da1*r,(x,0,L)),sp.integrate(dr_da2*r,(x,0,L)))
```

```python
coefficients = sp.solve(stationarity_conditions,a1,a2)
```

```python
coefficients
```

$$\left \{ a_{1} : \frac{2 L^{4} q_{0}}{\pi^{5} EI}, \quad a_{2} : - \frac{L^{4} q_{0}}{16 \pi^{5} EI}\right \}$$

```python
deflection_ansatz().subs([(a1,coefficients[a1]),(a2,coefficients[a2])]).simplify()
```

$$\frac{L^{4} q_{0}}{16 \pi^{5} EI} \left(32 \sin{\left (\frac{\pi x}{L} \right )} - \sin{\left (\frac{2 \pi}{L} x \right )}\right)$$

Now we're ready to plot the result and compare it to the analytical solution.

```python
#We first substite the now known Ansatz free values into our Ansatz
z = sp.symbols('z')
w_numerical = deflection_ansatz().subs([(a1,coefficients[a1]),(a2,coefficients[a2]),(x,z*L)])
#We also made the coordinate dimensionless (x/L --> z) because of sympy problems

lam_x_num = sp.lambdify(z, w_numerical, modules=['numpy'])
#For the variable x the expression w_numerical will be given something
z_vals = np.linspace(0, 1,100) #This something is z from 0 to 1
numerical = lam_x_num(z_vals) #We calculate the solution by passing x = [0,...,L] to deflection_analytical
plt.plot(x_vals/L, analytical/(L**4*q)*EI,label='analytical')
plt.plot(z_vals, numerical/(L**4*q)*EI,label='numerical')
plt.legend()
plt.xlabel('$x\ /\ L$')
plt.ylabel('$w\ /\ L^4 q_0  EI^{-1}$')
plt.tight_layout()
plt.savefig('beam_least_squares.pdf')
```

```python
print("Maximum absolute error: ", np.max(np.abs(analytical/(L**4*q)*EI - numerical/(L**4*q)*EI)))
```

    Maximum absolute error:  3.41648400857485e-5

We can also plot and compare the bending moment. Let's first find the analytical expression by symbolically differentiating the deflection expression twice to obtain $M(x) = -EI w''(x)$:

```python
#analytical bending moment
moment_analytical = -deflection_analytical().diff(x,2)*EI
#numerical bending moment
moment_numerical = -deflection_ansatz().subs([(a1,coefficients[a1]),(a2,coefficients[a2])]).diff(x,2)*EI

#create lambdas for plotting along dimensionless length z
lam_x_analyt = sp.lambdify(z, moment_analytical.subs(x,z*L), modules=['numpy'])
lam_x_num = sp.lambdify(z, moment_numerical.subs(x,z*L), modules=['numpy'])
z_vals = np.linspace(0, 1,100)
analytical = lam_x_analyt(z_vals)
numerical = lam_x_num(z_vals)

#plot
plt.plot(x_vals/L, analytical/(L**2*q),label='analytical')
plt.plot(z_vals, numerical/(L**2*q),label='numerical')
plt.xlabel('$x\ /\ L$')
plt.ylabel('$M\ /\ L^2 q_0$')
plt.legend()
plt.tight_layout()
plt.savefig('beam_least_squares_moment.pdf')
```

```python
print("Maximum absolute error in bending moment: ", np.max(np.abs(analytical/(L**2*q) - numerical/(L**2*q))))
```

    Maximum absolute error in bending moment:  0.00384875067588690

```python

```


---
# What are the marginal and the conditional probabilities?
---

In this script, we show the 1-D marginal and 1-D conditional probability density functions (PDF) for a 2-D gaussian PDF.

In its matrix-form, the equation for the 2-D gausian PDF reads like this:

<blockquote>  $P(\bf{x}) = \frac{1}{2\pi |\Sigma|^{0.5}} \exp{[-\frac{1}{2}(\bf{x}-\bf{\mu})^\top \Sigma^{-1} (\bf{x}-\bf{\mu})]}$  </blockquote>

where

<blockquote>
$
\begin{align}
\bf{x} &= [x_{1} x_{2}]^\top \\
\bf{\mu} &= [\mu_{1} \mu_{2}]^\top  \\
\Sigma &= \begin{pmatrix} \sigma_{x_{1}}^2 & \rho\sigma_{x_{1}}\sigma_{x_{2}} \\ \rho\sigma_{x_{1}}\sigma_{x_{2}} & \sigma_{x_{2}}^2 \end{pmatrix}
\end{align}
$
</blockquote>

where $\rho$ is the correlation factor between the $x_{1}$ and $x_{2}$ data.

## A useful trick to generate a covariance matrix $\Sigma$ with desired features

Instead of guessing the values of $\sigma_{x_{1}}$, $\sigma_{x_{2}}$ and $\rho$ to create a
covariance matrix $\Sigma$ for visualization purpose, we can design it with some desired characteristics.
Those are the principal axis variances $\sigma_{1}^2$ and $\sigma_{2}^2$, and the rotation angle $\theta$.

First, we generate the covariance matrix of a correlated PDF with its principal axes oriented along the
$x_{1}$ and $x_{2}$ axes.

<blockquote>  $\Sigma_{PA} = \begin{pmatrix} \sigma_{1}^2 & 0 \\ 0 & \sigma_{2}^2 \end{pmatrix}$ </blockquote>

Next, we generate the rotation matrix for the angle $\theta$:

<blockquote>  $R = \begin{pmatrix} \cos{\theta}  & -\sin{\theta} \\ \sin{\theta} & \cos{\theta} \end{pmatrix}$ </blockquote>

The covariance matrix we are looking for is

<blockquote>  $\Sigma = R \Sigma_{PA} R^\top $ </blockquote>

Hence, we only have to specify the values of $\sigma_{1}$ and $\sigma_{2}$ along principal axes and the
rotation angle $\theta$. The correlation coefficient $\rho$ depends on the value of $\theta$:

<blockquote>  $\rho>0$ when $\theta>0$ </blockquote>
<blockquote>  $\rho<0$ when $\theta<0$ </blockquote>

<br>
N.B. For ease of reading, we use below the variables x and y instead of x1 and x2. The final results are shown with x1 and x2.

```python
print(__doc__)

# Author: Pierre Gravel <pierre.gravel@iid.ulaval.ca>
# License: BSD

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm

from scipy import stats
from scipy.stats import multivariate_normal

import seaborn as sns
sns.set(color_codes=True)

# Used for reproductibility of the results
np.random.seed(43)
```

    Automatically created module for IPython interactive environment

Here are the characteristics of the 2-D PDF we want to generate:

```python
# Origin of the PDF
Mu = np.array([0.5, 0.5])

# Individual standard deviations along the principal axes
sigma = np.array([0.25, 0.05])

# Rotation angle for a negative correlation
theta = -45.

```

Generate the covariance matrix $\Sigma$:

```python
theta = np.radians(theta)
c, s = np.cos(theta), np.sin(theta)

# Rotation matrix
R = np.array(((c, -s), (s, c)))

# Covariance matrix for a PDF with its principal axes oriented along the x and y directions
Sigma = np.array([[sigma[0]**2, 0.],[0., sigma[1]**2]])

# Covariance matrix after rotation
Sigma = R.dot( Sigma.dot(R.T) )

```

Generate a spatial grid where the various PDF will be evaluated locally.

```python
x_min, x_max = 0., 1.
y_min, y_max = 0., 1.
nx, ny = 60, 60
x = np.linspace(x_min, x_max, nx)
y = np.linspace(y_min, y_max, ny)
xx, yy = np.meshgrid(x,y)
pos = np.dstack((xx, yy))

```

Get the marginal distributions $P(x)$ and $P(y)$. Each one is the probability of an event irrespective
of the outcome of the other variable.

```python
# Generator for the 2-D gaussian PDF
model = multivariate_normal(Mu, Sigma)
pdf = model.pdf(pos)

# Project P(x,y) on the x axis to get the marginal 1-D distribution P(x)
pdf_x = multivariate_normal.pdf(x, mean=Mu[0], cov=Sigma[0,0])

# Project P(x,y) on the y axis to get the marginal 1-D distribution P(y)
pdf_y = multivariate_normal.pdf(y, mean=Mu[1], cov=Sigma[1,1])

```

Get the conditional 1-D distributions $P(x|y=yc)$ and $P(y|x=xc)$. Each one is the probability of one event occurring in
the presence of a second event.

```python
# Vertical slice position
xc = 0.3

# Horizontal slice position
yc = 0.7

# Make a vertical slice of P(x,y) at x=xc to get the conditional 1-D distribution P(y|x=xc)
P = np.empty([ny,2])
P[:,0] = xc
P[:,1] = y
pdf_xc = multivariate_normal.pdf(P, mean=Mu, cov=Sigma)

# Make an horizontal slice of P(x,y) at y=yc to get the conditional 1-D distribution P(x|y=yc)
P = np.empty([nx,2])
P[:,0] = x
P[:,1] = yc
pdf_yc = multivariate_normal.pdf(P, mean=Mu, cov=Sigma)

```

Show the various PDF. The color of each conditional 1-D distribution corresponds to its slice through the 2-D PDF.

```python
fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2,2,figsize=(10,10))

# Display the 2-D PDF P(x,y)
cset = ax1.contourf(xx, yy, pdf, zdir='z', cmap=cm.viridis, levels=7)
ax1.plot([x_min, x_max], [yc, yc], linewidth=2.0, color='red')
ax1.text(0.79, 0.72,'$x_{2} = 0.7$', fontsize=16, color='white')
ax1.plot([xc, xc], [y_min, y_max], linewidth=2.0, color='green')
ax1.text(0.31, 0.02,'$x_{1} = 0.3$', fontsize=16, color='white')
ax1.set_xlabel('$x_{1}$',fontsize=18)
ax1.set_ylabel('$x_{2}$',rotation=0,fontsize=18)
ax1.xaxis.set_label_coords(0.5, -0.08)
ax1.yaxis.set_label_coords(-0.08, 0.5)

# Display the 1-D conditional PDF P(y|x=xc)
ax2.plot(pdf_y, y, label='$P(x_{2})$', linewidth=3.0, color='black')
ax2.plot(pdf_xc, y, label='$P(x_{2}|x_{1}=0.3)$', linewidth=2.0, color='green')
ax2.set_ylabel('$x_{2}$',rotation=0,fontsize=18)
ax2.set_xlabel('Probability Density',fontsize=18)
ax2.xaxis.set_label_coords(0.5, -0.08)
ax2.yaxis.set_label_coords(-0.08, 0.5)
ax2.legend(loc='best',fontsize=14)

# Display the 1-D conditional PDF P(x|y=yc)
ax3.plot(x, pdf_x, label='$P(x_{1})$', linewidth=3.0, color='black')
ax3.plot(x, pdf_yc, label='$P(x_{1}|x_{2}=0.7)$', linewidth=2.0, color='red')
ax3.set_xlabel('$x_{1}$',fontsize=18)
ax3.set_ylabel('Probability Density',fontsize=18)
ax3.xaxis.set_label_coords(0.5, -0.08)
ax3.yaxis.set_label_coords(-0.08, 0.5)
ax3.legend(loc='best',fontsize=14)

# Hide the unused fouth panel
ax4.axis('off')
fig.tight_layout()

plt.savefig('Example_of_2D_PDF_with_conditional_1D.png')
plt.savefig('Example_of_2D_PDF_with_conditional_1D.pdf')

plt.show()

```

```python

```


# Programovanie

Letná škola FKS 2018

Maťo Gažo, Fero Dráček
(& vykradnuté materiály od Mateja Badina, Feriho Hermana, Kuba, Peťa, Jarných škôl FX a kade-tade po internete)

V tomto kurze si ukážeme základy programovania a naučíme sa programovať matematiku a fyziku.
Takéto vedomosti sú skvelé a budete vďaka nim:
* vedieť efektívnejšie robiť domáce úlohy
* kvalitnejšie riešiť seminárové a olympiádové príklady
* lepšie rozumieť svetu (IT je dnes na trhu najrýchlejšie rozvíjajúcim sa odvetvím)

Počítač je blbý a treba mu všetko povedať a vysvetliť. Komunikovať sa s ním dá na viacerých úrovniach, my budeme používať Python. Python (názov odvodený z Monty Python's Flying Circus) je všeobecný programovací jazyk, ktorým sa dajú vytvárať webové stránky ako aj robiť seriózne vedecké výpočty. To znamená, že naučiť sa ho nie je na škodu a možno vás raz bude živiť.

Rozhranie, v ktorom píšeme kód, sa volá Jupyter Notebook. Je to prostredie navrhnuté tak, aby sa dalo programovať doslova v prehliadači a aby sa kód dal kúskovať. Pre zbehnutie kúskov programu stačí stlačiť Shift+Enter.

# Dátové typy a operátory

### Čísla

 podľa očakávaní, vracia trojku

```python
3
```

    3

```python
2+3 # scitanie
```

    5

```python
6-2 # odcitanie
```

    4

```python
10*2 # nasobenie
```

    20

```python
35/5 # delenie
```

    7.0

```python
5//3 # celociselne delenie TODO je toto treba?
```

    1

```python
7%3 # modulo
```

    1

```python
2**3 # umocnovanie
```

    8

```python
4 * (2 + 3) # poradie dodrzane
```

    20

### Logické výrazy

```python
1 == 1 # logicka rovnost
```

    True

```python
2 != 3 # logicka nerovnost
```

    True

```python
1 < 10
```

    True

```python
1 > 10
```

    False

```python
2 <= 2
```

    True

# Premenné

Toto je premenná.

Po stlačení Shift+Enter program v okienku zbehne a premenná sa uloží do pamäte (RAMky, všetko sa deje na RAMke).

```python
a = 2
```

Teraz s ňou možno pracovať ako s bežným číslom.

```python
2 * a
```

    4

```python
a + a
```

    4

```python
a + a*a
```

    6

Možno ju aj umocniť.

```python
a**3
```

    8

Pridajme druhú premennú.

```python
b = 5
```

Nasledovné výpočty dopadnú podľa očakávaní.

```python
a + b
```

    7

```python
a * b
```

    10

```python
b**a
```

    25

Reálne čísla môžeme zobrazovať aj vo vedeckej forme: $2.3\times 10^{-3}$.

```python
d = 2.3e-3
```

### Priklad [0]

FERO DAJ SEM NIECO

# ---------------------------------------------------------------------------

Teraz sa môžeme posunúť k niečomu viac zmysluplnejšiemu. Môžeme začať počítaču zadávať úlohy, ktoré by sme už sami nezvládli !

# Funkcie

Spravme si jednoduchú funkciu, ktorá za nás sčíta dve čísla, aby sme sa s tým už nemuseli trápiť my:

```python
def scitaj(a, b):
    print("Číslo a je {} a číslo b je {}".format(a, b))
    return a + b
```

```python
scitaj(10, 12) # vypise vetu a vrati sucet
```

    Číslo a je 10 a číslo b je 12

    22

Funkcia funguje na celých aj reálnych číslach.

Naša sčítacia funkcia má __štyri podstatné veci__:
1. `def`: toto slovo definuje funkciu.
2. dvojbodka na konci prvého riadku, odtiaľ začína definícia.
3. Odsadenie kódu vnútri funkcie o štyri medzery.
4. Samotný kód. V ňom sa môže diať čokoľvek, Python ho postupne prechádza.
5. `return`: kľúčová vec. Za toto slovo sa píše, čo je output funkcie.

### Úloha 1
Napíšte funkciu `priemer`, ktorá zoberie dve čísla (výšky dvoch chlapcov) a vypočíta ich priemernú výšku.

Ak máš úlohu hotovú, prihlás sa vedúcemu.

```python
# Tvoje riesenie:
def priemer(prvy, druhy):
    return ((prvy+druhy)/2)

priemer(90,20)

```

    55.0

# Poďme na fyziku

V tomto momente môžeme začať používať Python ako sofistikovanejšiu kalkulačku a počítať ňoz základné fyzikálne problémy.
Predstavme si napríklad, že potrebujeme zistiť, koľko mólov atómov je v dvoch litroch vody.

```python
rho = 1000.0                # hustota
V = 2.0 * 1e-3              # treba premeniť na metre kubické
m = rho * V                 # hmotnosť vody
Mm = (16 + 1 + 1) * 1e-3    # kg/mol

n = m / Mm
print(n)                    # vypiseme
```

    111.1111111111111

A koľko molekúl je v jednom litri vody?

```python
NA = 6.022e23               # Avogadrova konštanta
V = 1e-3
m = rho * V
N = m / Mm * NA             # zamyslite sa nad poradím násobenia a delenia
print(N)
```

    3.345555555555555e+25

Vcelku dosť...

## Úloha 2
Spočítajte objem, ktorý v priemere zaberá jedna molekula ľubovoľnej kvapaliny. Vyjadrite ho v nanometroch kubických.

Urobte to tak, že napíšete funkciu, ktorá bude ako vstup brať:
* objem kvapaliny
* hustotu kvapaliny
* molárnu hmotnosť kvapaliny
a ako výstup to dá objem jednej molekuly kvapaliny.

Spočítajte to pre metanol, etanol a benzén a výsledky potom ukážte vedúcemu. Neváhajte používať Google.

```python
# Tvoje riesenie:

def ObjemMolekuly(objem, hustota, molarna_hmotnost):
    pocet_molekul = (objem*hustota/molarna_hmostnost)*6.022e23
    return (objem/pocet_molekul)
```

# Zoznamy

Zatiaľ sme sa zoznámili s číslami (celé, reálne), stringami a trochu aj logickými hodnotami.
Zo všetkých týchto prvkov vieme vytvárať množiny, v informatickom jazyku `zoznamy`.

Na úvod sa teda pozrieme, ako s vytvára zoznam (po anglicky `list`). Takúto vec všeobecne nazývame dátová štruktúra.

```python
li = [] # prazdny list
```

```python
v = [4, 2, 3] # list s cislami
```

```python
v
```

    [4, 2, 3]

```python
v[0] # indexovat zaciname nulou!
```

    4

```python
v[1]
```

    2

```python
type(v)
```

    list

```python
w = [5, 'ahoj', True]
```

```python
type(w[0])
```

    int

Čo sa sa stane, ak zoznamy sčítame? Spoja sa.

```python
v + w
```

    [4, 2, 3, 5, 'ahoj', True]

Môžeme ich násobiť?

```python
v * v
```

Smola, nemôžeme. Ale všimnime si, aká užitočná je chybová hláška. Jasne nám hovorí, že nemožno násobiť `list`y.

So zoznamami môžeme robiť rôzne iné užitočné veci. Napríklad ich sčítať.

```python
sum(v)
```

    9

Alebo zistiť dĺžku:

```python
len(v)
```

    3

Alebo ich utriediť:

```python
sorted(v)
```

    [2, 3, 4]

Alebo na koniec pridať nový prvok:

```python
v.append(10)
v
```

    [4, 2, 3, 10]

Alebo odobrať:

```python
v.pop()
v
```

    [4, 2, 3]

### Interval
v zozname sa dá hľadať pomocou intervalov, ktoré sú $\langle x,y)$, čiže uzavretý - otvorený.

```python
li = [2, 5, 7, 8, 10, 11, 14, 18, 20, 25]
len(li)
```

    10

```python
li[1:5] # indexovanie na zaciatku nulou + polouzavrety interval
```

    [5, 7, 8, 10]

```python
li[:3] # prve tri prvky
```

    [2, 5, 7]

```python
li[6:] # prvky zacinajuce na 6tom mieste
```

    [14, 18, 20, 25]

```python
li[2:9:2] # zaciatok:koniec:krok
```

    [7, 10, 14, 20]

```python
del li[2]
li
```

    [2, 5, 8, 10, 11, 14, 18, 20, 25]

Zoznam možno zadefinovať aj cez rozsah:

```python
range(10)
type(range(10))
```

    range

```python
list(range(10))
```

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

```python
list(range(3, 9))
```

    [3, 4, 5, 6, 7, 8]

## Úloha 3
Spočítajte:
* súčet všetkých čísel od 1 do 1000.

Vytvorte zoznam `letnaskola`, ktorý bude obsahovať vašich 5 obľúbených celých čísel.
* Pridajte na koniec zoznamu číslo 100
* Vymažte druhé číslo zo zoznamu.
* Prepíšte prvé číslo v zozname tak, aby sa rovnalo poslednému v zozname.
* Vypočítajte súčet prvého čísla, posledného čísla a dĺžky zoznamu.

```python
# Tvoje riesenie:
zoznam = list(range(1,1001))
print(sum(zoznam))

letnaskola = [1,1995,12,6,42]
print(letnaskola)
letnaskola.append(100)
print(letnaskola)
del letnaskola[1]
print(letnaskola)
letnaskola[0] = letnaskola[len(letnaskola)-1]
print(letnaskola)

print(letnaskola[0]+letnaskola[len(letnaskola)-1], len(letnaskola))
```

    500500
    [1, 1995, 12, 6, 42]
    [1, 1995, 12, 6, 42, 100]
    [1, 12, 6, 42, 100]
    [100, 12, 6, 42, 100]
    200 5

# For cyklus

Indexy zoznamu môžeme postupne prechádzať. For cyklus je tzv. `iterátor`, ktorý iteruje cez zoznam.

```python
for i in li:
    print(i)
```

    2
    5
    8
    10
    11
    14
    18
    20
    25

```python
for i in li:
    print(i**2)
```

    4
    25
    64
    100
    121
    196
    324
    400
    625

Ako úspešne vytvoriť For cyklus? Podobne, ako pri funkciách:

* `for`: toto slovo je na začiatku.
* `i`: iterovana velicina
* `in`: pred zoznamom, cez ktorý prechádzame (iterujeme).
* dvojbodka na konci prvého riadku.
* kod, ktory sa cykli sa odsadzuje o štyri medzery.

Za pomoci for cyklu môžeme takisto sčítať čísla. Napr. čísla od 0 do 100:

```python
sum = 0

for i in range(101):   # uvedomme si, preco tam je 101 a nie 100
    sum = sum + i      # skratene sum += i

print(sum)
```

    5050

## Úloha 4
Spočítajte súčet druhých mocnín všetkých nepárnych čísel od 1 do 100 s využitím for cyklu.

```python
# Tvoje riesenie:
sum = 0
for i in range(101):
    if (i%2 == 1): sum = sum + i**2
print(sum)

```

    166650

# Podmienky

Pochopíme ich na príklade. Zmeňte `a` a zistite, čo to spraví.

```python
a = 5

if a == 3:
    print("cislo a je rovne trom.")
elif a == 5:
    print("cislo a je rovne piatim")
else:
    print("cislo a nie je rovne trom ani piatim.")
```

    cislo a je rovne piatim

Za pomoci podmienky teraz môžeme z for cyklu vypísať napr. len párne čísla. Párne číslo identifikujeme ako také, ktoré po delení dvomi dáva zvyšok nula.

Pre zvyšok po delení sa používa percento:

```python
for i in range(10):
    if i % 2 == 0:
        print(i)
```

    0
    2
    4
    6
    8

Cyklus mozeme zastavit, ak sa porusi nejaka podmienka

```python
for i in range(20):
    print(i)
    if i>10:
        print('Koniec.')
        break
```

    0
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    Koniec.

## Úloha 5
Spočítajte:
* súčet všetkých čísel od 1 do 1000 deliteľné jedenástimi.
* súčet tretích mocnín čísel od 1 do 1000 deliteľných dvanástimi.

```python
# Tvoje riesenie:
sum = 0

for i in range(1001):
    if (i%11 == 0): sum = sum + i

print(sum)

sum = 0

for i in range(1001):
    if (i%12 == 0): sum = sum + i**3

print(sum)
```

    45045
    20998994688

## Úloha 6

Teraz, keď už vieme, ako sa zisťuje deliteľnosť, môžeme tiež zistiť, či je zadané číslo prvočíslom. Vymyslite algoritmus, ktorý overí prvočíselnú vlastnosť nejakého čísla.

Výstup by mal byť nasledovný:
```Python
>>> prvocislo(10)
nie
>>> prvocislo(13)
ano
```

```python
# Tvoje riesenie:
from math import *

def prvocislo(cislo):
    vysledok = "ano"
    for i in range(2,int(floor(sqrt(cislo)))+1):
        if (cislo%i == 0):
            vysledok = "nie"
    return vysledok

print(prvocislo(10))
print(prvocislo(13))
```

    nie
    ano

## Úloha 7
Predstavme si, že chceme sčítať nekonečný počet čísel:
$$ \sum_{n=1}^\infty \frac{1}{n^2}.$$
Analytický výsledok takejto sumy je $\pi^2/6$. Koľko členov potrebujeme sčítať, aby presnosť s analytický výsledkov bola určená na tri desatinné miesta?

```python
# Tvoje riesenie:
from math import *
print('Presne ',round(pi**2/6,3))

sum = 0
for i in range(1,2005):
    sum = sum + 1.0/i**2

print(i,' ',sum)
```

    Presne  1.645
    2004   1.6444351893330087

# Konečne poďme na niečo zaujímavé!

V predchádzajúcej časti sme sa zoznámili so základnou syntaxou Pythonu, zoznámili sme sa s Jupyterom a naučili sme sa niečo málo o tom akoby sme mohli sčítať nejaké rady. Nastal však čas pustiť sa do niečo zaujímavejšieho.

V nasledujúcej časti sa pozrieme ako sa dajú pomocou numerickej matematiky a nejakých tých šikovných matematických vzťahov dajú vypočítať konštanty, s ktorými ste sa už stretli.

## Hľadanie hodnoty zlatého rezu $\varphi$

Jednoduché cvičenie na oboznámenie sa s tzv. selfkonzistentným problémom a for cyklom

```python
x = 1;

for i in range (0,20):
    x = 1+1/x
    print (x)
```

## Hľadanie hodnoty Eulerovho čísla $e$

Hoci je to málo známe, Eulerove čislo $e$ sa dá nájsť ako odpoveď na nasledujúcu úlohu (rozdeľuj a ponásob):
Na aké veľké časti treba rozdeliť hocaké číslo tak, aby súčin týchto častí bol maximálny.

```python
import matplotlib.pyplot as plt
%matplotlib inline

import numpy as np
```

```python
Num0 = 25

delitele = np.arange(1, 20 , 1)

casti = []
suciny = []

for i in range (0, 19):

    casti.append(Num0/delitele[i])

    suciny.append(casti[i]**delitele[i])
```

```python
plt.plot(casti, suciny, "ro-")
plt.show()
```

```python
MinDel = delitele[np.argmax(suciny)-1]
MaxDel = delitele[np.argmax(suciny)+1]
```

```python
delitele =  np.arange(MinDel, MaxDel, (MaxDel - MinDel)/10)

casti = []
suciny = []

for i in range (0, 9):

    casti.append(Num0/delitele[i])

    suciny.append(casti[i]**delitele[i])
```

```python
plt.plot(casti, suciny, "ro-")
plt.show()
```

```python
casti[np.argmax(suciny)]
```

# Ako sa numericky derivuje, integruje a riešia difky?

## Derivovanie

Na to, aby sme numericky zderivovali nejakú funkciu si musíme najprv uvedomiť, čo tá derivácia vlastne intuitívne robí.
Predstavme si obrázok a v ňom nakreslenú dotyčnicu a malý trojuholníček. Buď sme sa učili alebo sme sa práve teraz dozvedeli, že derivácia funkcie je smernica jej dotyčnice v danom bode. Odtiaľ by už mohlo byť (po nejakých tých obrázkoch) jasné, že to bude nejako takto:

$$ \frac{df}{dx} = \frac{f(x+h) - f(x-h)}{2h} $$

Zvolíme si malé $h$ a derivujeme.

```python
def func(x):
    return cos(x)

def deriv(f, x, h=0.01):
    return (f(x+h)-f(x-h))/(2*h)

x = np.linspace(0, 2*pi, 101)
#print(x)
y = [func(i) for i in x]
dydx = [deriv(func, i) for i in x]

plt.plot(x, y, label="$f(x)$")
plt.plot(x, dydx, label="$\\frac{df}{dx}$")
plt.xlim([0, 2*pi])
plt.xticks([0, pi, 2*pi])
plt.legend(loc="best")
plt.show()
```

### Išlo by to však aj s lepšou presnosťou?

Odkiaľ spadol magický vzorček vyššie? Tí čo už na podobné akcie chodia dlhšie sa už možno stretli s pojmom Taylorov rozvoj, intuitívne v princípe každá slušná a poslušná funkcia, s ktorou sa stretneme sa dá rozviť v istom okolí bodu $x_0$ do súčtu mocninových funkcií nasledovne:

$$ f(x) \approx f(x_0) + \left.\frac{df}{dx}\right|_{x=x_0}(x-x_0) + \frac{1}{2!}{\left(\left.\frac{df}{dx}\right|_{x=x_0}\right)}^2{(x-x_0)}^2 + \frac{1}{3!}{\left(\left.\frac{df}{dx}\right|_{x=x_0}\right)}^3{(x-x_0)}^3 + ...$$

Ak takto rozvinieme funkcie nielen v okolí bodu $x_0$, ale aj $x_0 + h$ a $x_0 - h$, a následne rozvejo vhodne medzi sebou odčítame, tak sme schopný nájsť hodnotu derivácie funkcie $\frac{df}{dx}$ v bode $x_0$, teda
$$
\left.\frac{df}{dx}\right|_{x=x_0}
$$

Skúste rozviť funkciu aj v bodoch $x_0+2h$ a $x_0-2h$ a získať vzorček, ktorý nám umožní vypočítať deriváciu s lepšou presnosťou.

## Integrovanie

Dve základné metódy ako sa dá niečo numericky integrovať sú:
* Kvadratúra
* Monte Carlo

Dva spôsoby:
* Kvadratúra (dobrá v 1D, ale so zvyšovaním rozmerov presnosť klesá)
* Monte Carlo (presnosť vždy $O(N^{-1/2})$ (N je počet bodov, ktoré použijeme na výpočet inegr), použiť pri viac ako troch rozmeroch)

Pomôžu nám knižnice.
Teraz si ukážeme kvadratúru.

```python
#from scipy.integrate import quad
import scipy.integrate as sp

def func2(x):
    return exp(-x**2)

sp.quad(func2, -20, 20)   # vysledok je 1/4
```

```python
sqrt(pi)
```

## Hľadanie hodnoty Ludolfovho čísla $\pi$
Pomocou Monte Carlo metódy integrovania sa naučíme ako napríklad vypočítať $\pi$.

```python
import random as rnd

NOP = 50000

CoordXList = [];
CoordYList = [];

for j in range (NOP):
    CoordXList.append(rnd.random())
    CoordYList.append(rnd.random())
```

```python
CircPhi = np.arange(0,np.pi/2,0.01)
```

```python
plt.figure(figsize=(7,7))

plt.plot(
    CoordXList,
    CoordYList,
    color = "red",
    linestyle= "none",
    marker = ","
)
plt.plot(np.cos(CircPhi),np.sin(CircPhi))
#plt.axis([0, 1, 0, 1])
#plt.axes().set_aspect('equal', 'datalim')
plt.show()
```
NumIn = 0

for j in range (NOP):

    #if (CoordXList[j] - 0.5)*(CoordXList[j] - 0.5) + (CoordYList[j] - 0.5)*(CoordYList[j] - 0.5) < 0.25:
    if CoordXList[j]*CoordXList[j] + CoordYList[j]*CoordYList[j] <= 1:

        NumIn = NumIn + 1;   NumIn/NOP*4
## Diferenciálne rovnice

Príklad neriešiteľnej difky:
$$ y'(x) = \sqrt{1+xy} $$
Príklad Eulerovej (najjednoduchšej) metódy.

```python
N = 101
x = np.linspace(0, 1, N)
dx = x[1] - x[0]
y_eu = np.zeros(N)
y_eu[0] = 1      # pociatocna podmienka

def func(x, y):
    return sqrt(1.0 + x*y)
#    return sin(x)

for i in range(1, N):
    y_eu[i] = y_eu[i-1] + func(x[i-1], y_eu[i-1])*dx

plt.plot(x, y_eu)
plt.show()
```

## Difky za pomoci knižníc

Použijeme funkciu `ode` z knižnice `scipy`. Znova riešime
$$y'(x) = \sqrt{1+xy} .$$

```python
from scipy.integrate import odeint

N = 101

def func(y, x):
    return sqrt(1 + x*y)

x = np.linspace(0, 1, N)
y0 = 1.0
y = odeint(func, y0, x) ### Pozri Google Scipy.integrate.odeint

plt.plot(x, (y_eu-y.T[0])/y.T[0])
plt.title("Rozdiel medzi odeint a Eulerom")
plt.show()
```

## Iný príklad na Difky: Exponenciálny rozpad

Ešte sa kukneme na numerické riešenie jednoduchých diferenciálnych rovníc. Metódu demonštrujeme na príklade exponenciálneho rozpadu. Znovu pomocou najjednoduchšej - Eulerovej metódy.

$$\frac{d n}{dt}=-\lambda n$$

$$\frac{n(t+dt)-n(t)}{dt}=-\lambda n(t)$$

$$n(t+dt)=n(t)(1 -\lambda dt), n(0)=N_0$$

```python
N0 = 500

#τ=λdt#
τ = 0.5
```

```python
tauka=[]
NumOfPart=[]

for i in range (15):

    if i == 0:

        n = N0

    else:

        n = n*(1 - τ)

    tauka.append(i*τ)
    NumOfPart.append(n)
```

```python
plt.plot(tauka, NumOfPart, "ro-")
plt.show()
```

# Obiehanie Zeme okolo Slnka

Fyziku (dúfam!) všetci poznáme.

* gravitačná sila:
$$ \mathbf F(\mathbf r) = -\frac{G m M}{r^3} \mathbf r $$

### Eulerov algoritmus (zlý)
$$\begin{align}
a(t) &= F(t)/m \\
v(t+dt) &= v(t) + a(t) dt \\
x(t+dt) &= x(t) + v(t) dt \\
\end{align}$$

### Verletov algoritmus (dobrý)
$$ x(t+dt) = 2 x(t) - x(t-dt) + a(t) dt^2 $$

```python
from numpy.linalg import norm

G = 6.67e-11
Ms = 2e30
Mz = 6e24
dt = 86400.0
N = int(365*86400.0/dt)
#print(N)

R0 = 1.5e11
r_list = np.zeros((N, 2))
r_list[0] = [R0, 0.0]     # mozno miesat listy s ndarray

v0 = 29.7e3
v_list = np.zeros((N, 2))
v_list[0] = [0.0, v0]

# sila medzi planetami
def force(A, r):
    return -A / norm(r)**3 * r

# Verletova integracia
def verlet_step(r_n, r_nm1, a, dt):  # r_nm1 -- r n minus 1
    return 2*r_n - r_nm1 + a*dt**2

# prvy krok je specialny
a = force(G*Ms, r_list[0])
r_list[1] = r_list[0] + v_list[0]*dt + a*dt**2/2

# riesenie pohybovych rovnic
for i in range(2, N):
    a = force(G*Ms, r_list[i-1])
    r_list[i] = verlet_step(r_list[i-1], r_list[i-2], a, dt)

plt.plot(r_list[:, 0], r_list[:, 1])
plt.xlim([-2e11, 2e11])
plt.ylim([-2e11, 2e11])
plt.xlabel("$x$", fontsize=20)
plt.ylabel("$y$", fontsize=20)
plt.gca().set_aspect('equal', adjustable='box')
#plt.axis("equal")
plt.show()
```

## Pridajme Mesiac

```python
Mm = 7.3e22
R0m = R0 + 384e6
v0m = v0 + 1e3
rm_list = np.zeros((N, 2))
rm_list[0] = [R0m, 0.0]
vm_list = np.zeros((N, 2))
vm_list[0] = [0.0, v0m]

# prvy Verletov krok
am = force(G*Ms, rm_list[0]) + force(G*Mz, rm_list[0] - r_list[0])
rm_list[1] = rm_list[0] + vm_list[0]*dt + am*dt**2/2

# riesenie pohybovych rovnic
for i in range(2, N):
    a = force(G*Ms, r_list[i-1]) - force(G*Mm, rm_list[i-1]-r_list[i-1])
    am = force(G*Ms, rm_list[i-1]) + force(G*Mz, rm_list[i-1]-r_list[i-1])
    r_list[i] = verlet_step(r_list[i-1], r_list[i-2], a, dt)
    rm_list[i] = verlet_step(rm_list[i-1], rm_list[i-2], am, dt)

plt.plot(r_list[:, 0], r_list[:, 1])
plt.plot(rm_list[:, 0], rm_list[:, 1])
plt.xlabel("$x$", fontsize=20)
plt.ylabel("$y$", fontsize=20)
plt.gca().set_aspect('equal', adjustable='box')
plt.xlim([-2e11, 2e11])
plt.ylim([-2e11, 2e11])
plt.show()         # mesiac moc nevidno, ale vieme, ze tam je
```

## Úloha pre Vás: Treba pridať Mars :)
Pridajte Mars!

## Matematické kyvadlo s odporom
Nasimulujte matematické kyvadlo s odporom $\gamma$,
$$ \ddot \theta = -\frac g l \sin\theta -\gamma \theta^2,$$
za pomoci metódy `odeint`.

Alebo pád telesa v odporovom prostredí:
$$ a = -g - kv^2.$$

```python
from scipy.integrate import odeint

def F(y, t, g, k):
    return [y[1], g -k*y[1]**2]

N = 101
k = 1.0
g = 10.0
t = np.linspace(0, 1, N)
y0 = [0.0, 0.0]
y = odeint(F, y0, t, args=(g, k))

plt.plot(t, y[:, 1])
plt.xlabel("$t$", fontsize=20)
plt.ylabel("$v(t)$", fontsize=20)
plt.show()
```

## Harmonický oscilátor pomocou metódy Leapfrog (modifikácia Verletovho algoritmu)

```python
N = 10000
t = linspace(0,100,N)
dt = t[1] - t[0]

# Funkcie
def integrate(F,x0,v0,gamma):
    x = zeros(N)
    v = zeros(N)
    E = zeros(N)

    # Počiatočné podmienky
    x[0] = x0
    v[0] = v0

    # Integrovanie rovníc pomocou metódy Leapfrog (wiki)
    fac1 = 1.0 - 0.5*gamma*dt
    fac2 = 1.0/(1.0 + 0.5*gamma*dt)

    for i in range(N-1):
         v[i + 1] = fac1*fac2*v[i] - fac2*dt*x[i] + fac2*dt*F[i]
         x[i + 1] = x[i] + dt*v[i + 1]
         E[i] += 0.5*(x[i]**2 + ((v[i] + v[i+1])/2.0)**2)

    E[-1] = 0.5*(x[-1]**2 + v[-1]**2)

    # Vrátime riešenie
    return x,v,E
```

```python
# Pozrime sa na tri rôzne počiatočné podmienky
F = zeros(N)
x1,v1,E1 = integrate(F,0.0,1.0,0.0) # x0 = 0.0, v0 = 1.0, gamma = 0.0
x2,v2,E2 = integrate(F,0.0,1.0,0.05) # x0 = 0.0, v0 = 1.0, gamma = 0.01
x3,v3,E3 = integrate(F,0.0,1.0,0.4) # x0 = 0.0, v0 = 1.0, gamma = 0.5

# Nakreslime si grafy

plt.rcParams["axes.grid"] = True
plt.rcParams['font.size'] = 14
plt.rcParams['axes.labelsize'] = 18
plt.figure()
plt.subplot(211)
plt.plot(t,x1)
plt.plot(t,x2)
plt.plot(t,x3)
plt.ylabel("x(t)")

plt.subplot(212)
plt.plot(t,E1,label=r"$\gamma = 0.0$")
plt.plot(t,E2,label=r"$\gamma = 0.01$")
plt.plot(t,E3,label=r"$\gamma = 0.5$")
plt.ylim(0,0.55)
plt.ylabel("E(t)")

plt.xlabel("Čas")
plt.legend(loc="center right")

plt.tight_layout()
```

A čo ak bude oscilátor aj tlmenný?

```python
def force(f0,t,w,T):
          return f0*cos(w*t)*exp(-t**2/T**2)

F1 = zeros(N)
F2 = zeros(N)
F3 = zeros(N)
for i in range(N-1):
    F1[i] = force(1.0,t[i] - 20.0,1.0,10.0)
    F2[i] = force(1.0,t[i] - 20.0,0.9,10.0)
    F3[i] = force(1.0,t[i] - 20.0,0.8,10.0)
```

```python
x1,v1,E1 = integrate(F1,0.0,0.0,0.0)
x2,v2,E2 = integrate(F1,0.0,0.0,0.01)
x3,v3,E3 = integrate(F1,0.0,0.0,0.1)

plt.figure()
plt.subplot(211)
plt.plot(t,x1)
plt.plot(t,x2)
plt.plot(t,x3)
plt.ylabel("x(t)")

plt.subplot(212)
plt.plot(t,E1,label=r"$\gamma = 0$")
plt.plot(t,E2,label=r"$\gamma = 0.01$")
plt.plot(t,E3,label=r"$\gamma = 0.1$")
pt.ylabel("E(t)")

plt.xlabel("Time")
plt.rcParams['legend.fontsize'] = 14.0
plt.legend(loc="upper left")

plt.show()
```

# Lineárna algebra

## Program na dnes

* Matematické operácie s vektormi a maticami. Hľadanie vlastných čísel.
* Zoznámenia sa s knižnicou `numpy`.

## Vektory

```python
a = np.array([1, 2, 3])
print(a)
print(type(a))

b = np.array([2, 3, 4])
print(a + b)    # spravne scitanie!
```
np.dot(a, b)    # skalarny sucin
a.dot(b)

```python
np.cross(a, b)  # vektorovy sucin
```
np.outer(a, b)  # outer product (jak sa to povie po slovensky?), premyslite si
## Matice
A = np.array([[0, 1], [1, 0]])
print(A)
type(A)AA = np.matrix(A)   # premena na iny datovy typ
type(AA)B = np.array([[1, 0], [0, -1]])
print(B)np.dot(A, B)# pozor!
A*B# Vlastnosti numpy matic
len(A)   # pocet riadkovA.shape   # rozmery# uzitocne vektory/matice, konvencia z Matlabu
N = 3
np.ones(N)      # konstantny vektornp.ones((N, N))    # jednotky
np.eye(N)       # identita
np.zeros((N, N+1))      # nulova matica NxN

```python
# spajanie matic
A = np.ones((3, 3))
B = np.eye(3)
print(A, "\n")
print(B)
np.hstack((A, B))
```

### Prístup k prvkom
# Najprv si pripravime nase vektory
A = np.arange(9)
print(A)
A = A.reshape((3, 3))
A# Pri poliach by sme urobili A[1][1]. Pri vektoroch mozme pouzit aj A[1,1].
# Pri maticiach musime pouzit A[1,1], lebo A[1][1] vrati prekvapivy vysledok A[1, 1]A[1][1]A[-1, -1] = 10   # znovu vieme indexovat aj od zadu
AA[0, 0]A[-3, -2]
## Slicing

Ako vyberať jednotlivé stĺpce a riadky matíc?
A = np.arange(9)
A = A.reshape((3, 3))
AA[:, 0]    # Ak nechceme vyberat nejaky usek, pouzijeme dvojbodku.A[1, :]A[[0, 2], :]   # Mozme pouzit aj pole na indexovanie# zmena riadku
A[:, 1] = 4
A# pripocitanie cisla k stlpcu
for i in range(len(A)):
    A[i, 2] += 100

AA = np.arange(25).reshape((5, 5))
AA[2:, 3]
## Generovanie (pseudo)náhodných čísel
# cisla sa menie, pokial nefixneme seed!
np.random.seed(12)

N = 5
a = 2*np.random.rand(N)   # v intervale [0,1]
aA = np.random.rand(N, N)
Aa = np.random.randn(5)
amu, sigma = 1.0, 2.0
a = np.random.randn(1000)*sigma + mu
plt.hist(a, bins=20)
plt.show()a = np.random.randint(10, size=1000)   # cele cisla, pozor na size!
u = plt.hist(a)
#plt.plot(u[1][1:], u[0])
## Vlastné čísla

$$ A v = \lambda v$$
Použijeme funkciu `eig`.

Okrem nej existujú ešte funkcie na vlastné čísla symetrických alebo hermitovských matíc `eigs` a `eigh`.

```python
from numpy.linalg import eig
```
N = 20
A = np.random.rand(N, N)
vals, vecs = eig(A)
print(vals.real)
## Vlastné čísla náhodných matíc

Vygenerujte (100x100) maticu náhodných čísel, získajte jej vlastné čísla a spravte z nich histogram.

N = 100
A = np.random.rand(N, N)
vals, vecs = eig(A)
vals = np.real(vals)
#print(vals)
plt.hist(np.real(vals))
plt.show()


# Part 1: Optimising functions

In this lab we will play with some of the optimisation methods we learned in the lecture by exploring how they work on some analytic functions (both convex and non-convex).

```python
import torch
import torch.optim as optim
```

## A Simple Function

For this first task, we are going to try to optimise the following using Stochastic Gradient Descent:

\begin{equation}
min_{\textbf{x}} (\textbf{x}[0] - 5)^2 + \textbf{x}[1]^2 + (\textbf{x}[2] - 1)^2\; ,
\end{equation}

Use the following block the write down the analytic minima of the above function:

### Implement the function

First, complete the following code block to implement the above function using PyTorch:

```python
def function(x):
     return ((x[0] - 5)**2 + x[1]**2 + (x[2] - 1)**2)

     raise NotImplementedError()
```

### Optimising

We need two more things before we can start optimising.
We need our initial guess - which we've set to [2.0, 1.0, 10.0] and we need to how many epochs to take.

```python
p = torch.tensor([2.0, 1.0, 10.0], requires_grad=True)
epochs = 5000
```

We define the optimisation loop in the standard way:

```python
opt = optim.SGD([p], lr=0.001)

for i in range(epochs):
    opt.zero_grad()
    output = function(p)
    output.backward()
    opt.step()
```

Use the following block to print out the final value of `p`. Does it match the value you expected?

```python
print(p)

#raise NotImplementedError()
```

    tensor([4.9999e+00, 4.4948e-05, 1.0004e+00], requires_grad=True)

## Visualising Himmelblau's Function

We'll now have a go at a more complex example, which we also visualise, with multiple optima; [Himmelblau's function](https://en.wikipedia.org/wiki/Himmelblau%27s_function). This is defined as:

\begin{equation}
f(x, y) = (x^2 + y - 11)^2 + (x + y^2 - 7)^2\; ,
\end{equation}
and has minima at
\begin{equation}
f(3, 2) = f(-2.805118, 3.131312) = f(-3.779310, -3.283186) = f(3.584428, -1.848126) = 0\; .
\end{equation}

Use the following block to first define the function (the inputs $x, y$ are packed into a vector as for the previous quadratic function above):

```python
def himm(x):
    x, y = x[0], x[1]
    return (x**2 + y - 11)**2 + (x + y**2 - 7)**2

    raise NotImplementedError()
```

The following will plot its surface:

```python
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.colors import LogNorm

xmin, xmax, xstep = -5, 5, .2
ymin, ymax, ystep = -5, 5, .2
x, y = np.meshgrid(np.arange(xmin, xmax + xstep, xstep), np.arange(ymin, ymax + ystep, ystep))
z = himm(torch.tensor([x, y])).numpy()

fig = plt.figure(figsize=(8, 5))
ax = plt.axes(projection='3d', elev=50, azim=-50)
ax.plot_surface(x, y, z, norm=LogNorm(), rstride=1, cstride=1,
                edgecolor='none', alpha=.8, cmap=plt.cm.jet)
ax.set_xlabel('$x$')
ax.set_ylabel('$y$')
ax.set_zlabel('$z$')

ax.set_xlim((xmin, xmax))
ax.set_ylim((ymin, ymax))

plt.show()
```

Check that the above plot looks correct  by comparing to the picture on the [Wikipedia page](https://en.wikipedia.org/wiki/Himmelblau%27s_function).

### Optimising

Let's see how it looks for a few different optimisers from a range of starting points

```python
xmin, xmax, xstep = -5, 5, .2
ymin, ymax, ystep = -5, 5, .2
x, y = np.meshgrid(np.arange(xmin, xmax + xstep, xstep), np.arange(ymin, ymax + ystep, ystep))
z = himm(torch.tensor([x, y])).numpy()

fig, ax = plt.subplots(figsize=(8, 8))
ax.contourf(x, y, z, levels=np.logspace(0, 5, 35), norm=LogNorm(), cmap=plt.cm.gray)

p = torch.tensor([[0.0],[0.0]], requires_grad=True)
opt = optim.SGD([p], lr=0.01)

path = np.empty((2,0))
path = np.append(path, p.data.numpy(), axis=1)

for i in range(50):
    opt.zero_grad()
    output = himm(p)
    output.backward()
    opt.step()
    path = np.append(path, p.data.numpy(), axis=1)

ax.plot(path[0], path[1], color='red', label='SGD', linewidth=2)

ax.legend()
ax.set_xlabel('$x$')
ax.set_ylabel('$y$')

ax.set_xlim((xmin, xmax))
ax.set_ylim((ymin, ymax))
```

Use the following block to run SGD with momentum (lr=0.01, momentum=0.9) from the same initial point, saving the position at each timestep into a variable called `path_mom`.

```python
p = torch.tensor([[0.0],[0.0]], requires_grad=True)
opt = optim.SGD([p], lr=0.01, momentum=0.9)

for i in range(50):
    opt.zero_grad()
    output = himm(p)
    output.backward()
    opt.step()
    path_mom = np.append(path, p.data.numpy(), axis=1)

#raise NotImplementedError()
```

The following will plot the path taken when momentum was used, as well as the original plain SGD path:

```python
ax.plot(path_mom[0], path_mom[1], color='yellow', label='SGDM', linewidth=2)
ax.legend()
fig
```

Now explore what happens when you start from different points. What effect do you get with different optimisers?

```python
p = torch.tensor([[2.0],[4.0]], requires_grad=True)
opt = optim.SGD([p], lr=0.01, momentum=0.9)

for i in range(50):
    opt.zero_grad()
    output = himm(p)
    output.backward()
    opt.step()
    path_mom = np.append(path, p.data.numpy(), axis=1)

ax.plot(path_mom[0], path_mom[1], color='yellow', label='SGDM', linewidth=2)
ax.legend()
fig
```


$\large{\text{Naive Bayes Classifier}}$

Consider the data set as given below.

```python
#First, we import the required packages
import pandas as pd #the pandas library is useful for data processing
import matplotlib.pyplot as plt #the matplotlib library is useful for plotting purposes

#Get the data from the csv file
sample_data = pd.read_csv('dataset.csv', index_col=False)

#print the data
sample_data

```

      Temp gt 100
      Travel to foreign country
      Cough
      Antibodies in blood
      Disease presence

      0
      Yes
      Yes
      No
      Yes
      Yes

      1
      No
      Yes
      Yes
      Yes
      Yes

      2
      Yes
      No
      Yes
      No
      Yes

      3
      Yes
      No
      Yes
      Yes
      Yes

      4
      No
      No
      No
      Yes
      No

      5
      No
      Yes
      Yes
      No
      No

      6
      No
      Yes
      No
      Yes
      No

      7
      Yes
      Yes
      No
      No
      No

Now suppose we have the observation $x$ given by

$(\text{Temp gt 100=Yes, Travel to foreign country=Yes, Cough = Yes, Antibodies in blood= Yes})$,

then how do we classify $x$ into the Disease presence category?

Consider $X$ to be a random variable of which $x$ is an observation.

Suppose $Y$ denotes the random variable associated with the Disease presence category, then the problem of finding the Disease presence category can be cast as finding the conditional probability of class label $P(Y=\text{Yes}|X=x)$ and the conditional probability $P(Y=\text{No}|X=x)$.

To find these conditional probabilities, we shall use the famous $\textbf{Bayes' theorem}$ idea.

$\large{\text{Bayes' Theorem}}$

The conditional probability $P(Y|X)$ can be written as:

$\begin{align}
P(Y|X) = \frac{P(Y,X)}{P(X)}.
\end{align}
$

Since we know that $P(X|Y) = \frac{P(X,Y)}{P(Y)}$, and since the $\textbf{joint probability}$ $P(X,Y)$ is the same as $\text{joint probability}$ $P(Y,X)$, we can write:

$P(Y,X) = P(X|Y) P(Y)$.

Substituting $P(Y,X)$ in the previous conditional probability $P(Y|X)$, we have:

$\begin{align}
P(Y|X) = \frac{P(Y,X)}{P(X)} = \frac{P(X|Y)P(Y)}{P(X)}.
\end{align}
$

Thus the $\textbf{posterior probability}$ $P(Y|X)$ of observing $Y$ after receiving $X$ can be seen to be proportional to the product of the $\textbf{likelihood}$ term $P(X|Y)$ and the $\textbf{prior probability}$ $P(Y)$ of $Y$.

$P(X)$ is a $\textbf{normalization}$ factor and is usually a constant for any value of the observation of $Y$.

Hence we may write $P(Y|X) \propto P(X|Y)P(Y)$.

Hence using Bayes' Theorem we can write:

$P(Y=\text{Yes}|X=x) \propto P(X=x|Y=\text{Yes}) P(Y=\text{Yes})$.

and

$P(Y=\text{No}|X=x) \propto P(X=x|Y=\text{No}) P(Y=\text{No})$.

$\textbf{Question:}$ How do we compute the prior probabilities $P(Y=\text{Yes})$, $P(Y=\text{No})$ and the likelihood terms $P(X=x|Y=\text{Yes})$, $P(X=x|Y=\text{No})$?

$\textbf{Idea:}$

Use the training data to get the required probabilities.

$\textbf{Computing the prior probabilities}$

Let us relook at the data at hand. Then we can use a frequency based computation for prior probabilities $P(Y=\text{Yes})$ and $P(Y=\text{No})$.

```python
#print the labels
sample_data['Disease presence']

```

    0    Yes
    1    Yes
    2    Yes
    3    Yes
    4     No
    5     No
    6     No
    7     No
    Name: Disease presence, dtype: object

We note that out of the $8$ samples, $Y=\text{Yes}$ appears $4$ times and $Y=\text{No}$ appears $4$ times.

Hence we can write:

$P(Y=\text{Yes}) = \frac{4}{8} = 0.5$ and $P(Y=\text{No}) = \frac{4}{8} = 0.5$.

$\textbf{How to compute P(X=x|Y=y)}$?

Note that in our case $X$ is multi-dimensional attribute of the form $X=(X_1,X_2,X_3,X_4)$ where:

*   $X_1$ is the random variable associated with $\texttt{Temp gt 100}$
*   $X_2$ is the random variable associated with $\texttt{Travel to foreign country}$
*   $X_3$ is the random variable associated with $\texttt{Cough}$
*   $X_4$ is the random variable associated with $\texttt{Antibodies in blood}$

Similarly, an observation $x$ of $X$ is also multi-variate and is of the form $x=(x_1,x_2,x_3,x_4)$.

Thus $P(X=x|Y=y)$ can be equivalently written as: $P( (X_1,X_2,X_3,X_4) = (x_1,x_2,x_3,x_4)|Y=y)$.

Hence given an observation $x=(x_1,x_2,x_3,x_4)$ of $X=(X_1,X_2,X_3,X_4)$ how do we find the conditional probability $P((X_1,X_2,X_3,X_4)=(x_1,x_2,x_3,x_4)|Y=y)$?

$\textbf{Conditional Independence Assumption}$

In Naive Bayes classifier, we make an important assumption about the data.

That is the covariates (or) attributes are conditionally independent given the class label.

This assumption is called the conditional independence assumption.

Using the conditional independence assumption, we can write:

$
\begin{align}
P( (X_1,X_2,X_3,X_4) = (x_1,x_2,x_3,x_4) | Y=y) = \prod_{i=1}^{4} P(X_i=x_i|Y=y).
\end{align}
$

Now finding $P( (X_1,X_2,X_3,X_4) = (x_1,x_2,x_3,x_4) | Y=y)$ boils down to finding $P(X_i=x_i|Y=y)$ for $i=1,2,3,4$.

$\textbf{Question:}$ How do we find $P(X_i=x_i|Y=\text{Yes})$ and $P(X_i=x_i|Y=\text{No})$?

```python

```

$\textbf{Idea:}$

Use the training data.

$\textbf{Note:}$ Luckily, each $X_i$ in the data is discrete-valued and hence finding the probabilities would be easy.

$\textbf{Recall:}$

The observation $x$ is given by

$(\text{Temp gt 100=Yes, Travel to foreign country=Yes, Cough = Yes, Antibodies in blood= Yes})$.

Let us now compute $P(X_1 = \text{Yes}|Y=\text{Yes})$.

We shall look into the data.

```python
#print the column 'Temp gt 100' along with label 'Yes'
(sample_data.loc[sample_data['Disease presence'] == 'Yes'])[['Temp gt 100', 'Disease presence']]
```

      Temp gt 100
      Disease presence

      0
      Yes
      Yes

      1
      No
      Yes

      2
      Yes
      Yes

      3
      Yes
      Yes

Thus from the above data we can compute:

$P(X_1=\text{Yes}|Y=\text{Yes}) = \frac{3}{4} = 0.75$.

```python

```

Let us now compute $P(X_2 = \text{Yes}|Y=\text{Yes})$.

We shall look into the data.

```python
#print the column 'Travel to foreign country' along with label 'Yes'
(sample_data.loc[sample_data['Disease presence'] == 'Yes'])[['Travel to foreign country', 'Disease presence']]
```

      Travel to foreign country
      Disease presence

      0
      Yes
      Yes

      1
      Yes
      Yes

      2
      No
      Yes

      3
      No
      Yes

Thus from the above data we can compute:

$P(X_2=\text{Yes}|Y=\text{Yes}) = \frac{2}{4} = 0.5$.

```python

```

Let us now compute $P(X_3 = \text{Yes}|Y=\text{Yes})$.

We shall look into the data.

```python
#print the column 'Cough' along with label 'Yes'
(sample_data.loc[sample_data['Disease presence'] == 'Yes'])[['Cough', 'Disease presence']]
```

      Cough
      Disease presence

      0
      No
      Yes

      1
      Yes
      Yes

      2
      Yes
      Yes

      3
      Yes
      Yes

Thus from the above data we can compute:

$P(X_3=\text{Yes}|Y=\text{Yes}) = \frac{3}{4} = 0.75$.

```python

```

Let us now compute $P(X_4 = \text{Yes}|Y=\text{Yes})$.

We shall look into the data.

```python
#print the column 'Antibodies in blood' along with label 'Yes'
(sample_data.loc[sample_data['Disease presence'] == 'Yes'])[['Antibodies in blood', 'Disease presence']]
```

      Antibodies in blood
      Disease presence

      0
      Yes
      Yes

      1
      Yes
      Yes

      2
      No
      Yes

      3
      Yes
      Yes

Thus from the above data we can compute:

$P(X_4=\text{Yes}|Y=\text{Yes}) = \frac{3}{4} = 0.75$.

Now that we have computed all relevant probabilities, we can compute
$P((X_1,X_2,X_3,X_4)=(\text{Yes,Yes,Yes,Yes})|Y=\text{Yes}) = \prod_{i=1}^{4} P(X_i = \text{Yes}|Y=\text{Yes})$.

Thus $P((X_1,X_2,X_3,X_4)=(\text{Yes,Yes,Yes,Yes})|Y=\text{Yes}) = 0.75 \times 0.5 \times 0.75 \times 0.75 = 0.2109375$.

```python

```

Similarly, we can compute $P((X_1,X_2,X_3,X_4)=(\text{Yes,Yes,Yes,Yes})|Y=\text{No}) = \prod_{i=1}^{4} P(X_i = \text{Yes}|Y=\text{No})$.

$\textbf{Exercise:}$ Find  $P((X_1,X_2,X_3,X_4)=(\text{Yes,Yes,Yes,Yes})|Y=\text{No})$.

Now we can compute $P(Y=\text{Yes}|X=x)$ as a quantity proportional to $P(X=x|Y=\text{Yes}) P(Y=\text{Yes}) = 0.2109375 \times 0.5$.

Also similarly, we can compute $P(Y=\text{No}|X=x)$ as a quantity proportional to $P(X=x|Y=\text{No}) P(Y=\text{No.})$.

$\textbf{Exercise:}$ Compute $P(Y=\text{No}|X=x)$.

Having computed the posterior probabilities for $Y=\text{Yes}$ label and $Y=\text{No}$ label, we can compare them and assign a label which has the maximimum posterior probability.

$\textbf{Exercise:}$ What is the label predicted for observation $x$?

```python

```

$\textbf{Question:}$ How to deal with Continuous attributes?

```python

```

$\textbf{Idea:}$ Assume that the column containing continuous data has a Gaussian distribution.

Hence $P(X_i=x_i|Y=y_j)$ can be characterized as $f(x_i; \mu_{ij},\sigma^2_{ij}) = \frac{1}{\sqrt{2\pi}\sigma_{ij}}  e^{-\frac{(x_i-\mu_{ij})^2}{2\sigma_{ij}^2}}$.

The mean $\mu_{ij}$ and variance $\sigma_{ij}$ are estimated from data corresponding to $X_i=x_i$ and $Y=y_j$.

Ideally we would be computing $P(x_i \leq X_i \leq x_i + \epsilon|Y=y_j)$.

This would yield a quantity $P(x_i \leq X_i \leq x_i + \epsilon|Y=y_j) \approx \epsilon f(x_i; \mu_{ij},\sigma^2_{ij})$.

Since $\epsilon$ is constant for each class, during the comparative analysis of the posterior probabilities, the effect of $\epsilon$ will be effectively ignored.

```python

```

$\large{\text{Some notable features of Naive Bayes Classifier}}$

1.   $\textbf{Smoothing of noisy attributes:}$ Can Naive Bayes classifier deal with noise in attribute?
2.   $\textbf{Dealing with irrelevant attributes:}$ For attribute $X$ which might not be relevant to prediction of class label $Y$, how will $P(Y|X)$ look like? What impact does it have?
3.  $\textbf{Dealing with correlated attributes:}$ Can Naive Bayes classifier handle correlated attributes?

$\large{\text{Exercise}}$

Suppose the attribute column $\texttt{Temp gt 100}$ is replaced with $\texttt{Temperature}$ attribute column with the values $(100.3, 98.6, 100.5, 99.5, 101.01, 98.3, 99.5, 100.2)$. Find the posterior probabilities

*   $P(Y=\text{Yes}|X=(Yes, No, No, Yes))$ and $P(Y=\text{No}|X=(Yes, No, No, Yes))$.
*   $P(Y=\text{Yes}|X=(No, Yes, No, No))$ and $P(Y=\text{No}|X=(No, Yes, No, No))$.

```python

```


# Introduction

Welcome to the Cyborg Math course!

In this course we will learn how to use sympy - a python package for symbolic math. The first step is to import the package

```python
import sympy
```

Sympy's default print mode is plain text, which is very robust but not very aesthetic

```python
x = sympy.Symbol('x')
(x**2+1)/(x**2-1)
```

$\displaystyle \frac{x^{2} + 1}{x^{2} - 1}$

We can use latex rendering to make the output nicer

```python
sympy.init_printing()
```

```python
(x**2+1)/(x**2-1)
```

# Section 1 - Objects

sympy works by manipulating symbolic mathematical expressions. This is different from other packaes you might be familiar with, which manipulate collections of raw data (like numbers or strings). Here's an example of the difference between a regular calculation and a symbolic calculation. Suppose we want to calculate the square root of 8, normally, we would use

```python
import math
math.sqrt(8)
```

```python
sympy.sqrt(8)
```

It seems like the sympy function didn't do anything, but what it actually did was to produce a new sympy object

```python
type(sympy.sqrt(8))
```

    sympy.core.mul.Mul

Also, unlike the regular math function, which truncates the output after about 15 significant digits, sympy expressions are, in principle, accurate to infinite precision. sympy knows how to incorporate integers into symbolic expressions seamlessly

```python
sympy.sqrt(8)+1
```

But fractions can be problematic

```python
sympy.sqrt(8)+1/7
```

To create a fraction, we need to use the Rational object

```python
sympy.sqrt(8) + sympy.Rational(1,7)
```

__Exercise a__

Create a sympy object that represents the fraction 1/3. Use the function below to verify your answer

```python
import verify_1
answer = 0
verify_1.check_1a(answer)
```

    False

We can turn symbolic numbers into floating point numbers using .n()

```python
sympy.sqrt(8).n()
```

Though this expression is still a sympy object

```python
type(sympy.sqrt(8).n())
```

    sympy.core.numbers.Float

To turn it into a regular number, we need to use float()

```python
[float(sympy.sqrt(8).n()),type(float(sympy.sqrt(8).n()))]
```

    [2.8284271247461903, float]

sympy will by default display the last equation

```python
sympy.sqrt(8)+1
sympy.sqrt(8)-1
```

If you want to display multiple equation, or lines before the end, use display

```python
display(sympy.sqrt(8)+1)
sympy.sqrt(8)-1
```

# Section 2 - Symbols

Variables and unknown quantities can be represented using symbols

```python
x = sympy.Symbol('x')
x
```

__Exercise a__

Construct a variable called "y"

```python
answer = 0
verify_1.check_2a(answer)
```

    False

Symbols can also include greek letters

```python
[sympy.Symbol('zeta'), sympy.Symbol('Sigma')]
```

__Exercise b__

Construct the greek variable chi

```python
answer = 0
verify_1.check_2b(answer)
```

    False

Symbols can also contain subscript

```python
sympy.Symbol('a3')
```

__Exercise c__

Construct the variable alpha_3

```python
answer = 0
verify_1.check_2c(answer)
```

    False

Finally, variables can be typeset using raw latex

```python
sympy.Symbol(r'\tilde{\aleph}')
```

__Exercise d__

Construct the variable $\mathcal{L}$

```python
answer = 0
verify_1.check_2d(answer)
```

    False

By default, sympy assumes everything is complex, so this is why it refrains from doing some obvious simplification. However, it is possible to give variables qualifiers that will enable these simplifications

```python
y = sympy.Symbol('y')
z = sympy.Symbol('z', positive=True)
[sympy.sqrt(y**2), sympy.sqrt(z**2)]
```

Here's another example for why assumptions are important for simplification

```python
%%html
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">I will not be taking questions at this time <a href="https://t.co/H7KvemYuhF">pic.twitter.com/H7KvemYuhF</a></p>— Anna Hughes (@AnnaGHughes) <a href="https://twitter.com/AnnaGHughes/status/1222608360450617344?ref_src=twsrc%5Etfw">January 29, 2020</a></blockquote>
```

I will not be taking questions at this time pic.twitter.com/H7KvemYuhF
— Anna Hughes (@AnnaGHughes) January 29, 2020

# Section 3 - Arithmetic

We can combine numbers and variables using the four basic arithmetica operations

```python
(2*x+1)/(2*x-1)
```

__Exercise a__

Construct the [Mobius transformation](https://en.wikipedia.org/wiki/M%C3%B6bius_transformation)

$\frac{a z+b}{c z+d}$

```python
answer = 0
verify_1.check_3a(answer)
```

    False

Raising to a power is done using the operator **

```python
x**2
```

__Exercise b__

Create the general expression for a Mersenne number $2^n-1$

```python
answer = 0
verify_1.check_3b(answer)
```

    False

Fractions can be separated to numerator and denominator using the fraction function

```python
sympy.fraction((2*x+1)/(2*x-1))
```

```python
(x**2-1)/(x-1)
```

__Exercise c__

One can construct the [continued fraction](https://en.wikipedia.org/wiki/Continued_fraction) $1+\frac{1}{1+\frac{1}{1+...}}$ by using the recursion relation $a_1 = 1$, $a_{n+1} = 1+\frac{1}{a_n}$. Find the numerator of $a_{20}$

```python
answer = 0
verify_1.check_3c(answer)
```

    False

# Section 4 - Simplification and Expansion

Sometimes sympy behaves in a strange way. Consider the following example

```python
(1+sympy.sqrt(5))*(1-sympy.sqrt(5))
```

This expression is left as is, even though we can clearly see it can be simplified. We can carry out the product by calling the expand function

```python
sympy.expand((1+sympy.sqrt(5))*(1-sympy.sqrt(5)))
```

Conversely, sometimes we want to do the opposite (i.e. put an expanded back in fractions)

```python
x**2+2*x+1
```

This can be done using the factor function

```python
sympy.factor(x**2+2*x+1)
```

In general, the simplest representation of an expression can be obtained using the simplify function

```python
sympy.simplify((x**3 + x**2 - x - 1)/(x**2 + 2*x + 1))
```

__Exercise a__

Simplify the fraction $\frac{\sqrt{3}+\sqrt{2}}{\sqrt{3}-\sqrt{2}}$ by multiplying and expanding both numerator and denominator by $\sqrt{3}+\sqrt{2}$

```python
answer = 0
verify_1.check_4a(answer)
```

    False

# Section 5 - Complex Numbers

sympy can represent complex numbers using the imaginary number sympy.I

```python
[sympy.I, sympy.I**2]
```

Complex and real parts

```python
dummy = 3+4*sympy.I
[sympy.re(dummy), sympy.im(dummy)]
```

Complex conjugate

```python
dummy.conjugate()
```

Absolute magnitude

```python
sympy.Abs(dummy)
```

Phase

```python
sympy.arg(dummy)
```

__Exercise a__

Find the imaginary part of $(3+4i)^2$

```python
answer = 0
verify_1.check_5a(answer)
```

    False

# Section 6 - Substitution

Substitution replaces one expression with another expression

```python
temp = (2*x+1)/(2*x-1)
[temp, temp.subs(x, x/2)]
```

subs will match an exact expression, and not mathematically equivalent expressions

```python
y = sympy.Symbol('y')
temp = sympy.sqrt(x**2)+x
[temp,temp.subs(x**2,y)]
```

One common pitfall is that identically looking variables might be different, and so will respond differently to subs

```python
not_x = sympy.Symbol('x', complex=True)
temp = x+not_x
[temp, temp.subs(x,1)]
```

__Exercise a__

The Babylonians discovered a [repeated substitution](https://en.wikipedia.org/wiki/Methods_of_computing_square_roots#Babylonian_method) to calculate the square root of a number $S$

$x_{n+1} = \frac{1}{2} \left(\frac{S}{x_n}+x_n\right)$

Suppose we want to use this method to calculate the square root of 3. Start from $x_1 = 1$ and find $x_{10}$

```python
answer = 0
verify_1.check_6a(answer)
```

    False

# Section 7 - Functions

sympy supports all elementary functions, as well as a number of non elementary functions

```python
[sympy.log(x), sympy.sin(x), sympy.exp(x), sympy.gamma(x)]
```

Some functions have their own [simplifying functions](https://docs.sympy.org/1.5.1/tutorial/simplification.html)

```python
[sympy.cos(2*x),sympy.expand_trig(sympy.cos(2*x))]
```

You can also define implicit function

```python
F = sympy.Function('F', positive=True)
F(x)
```

And later substitute a different function

```python
temp = F(2*x) - F(x)
temp = temp.subs(F, sympy.log)
temp
```

# Section 8 - Equations

sympy has an equation object

```python
sympy.Eq(2*x+3,x**2)
```

We can solve this equation and find its roots

```python
sympy.solve(sympy.Eq(2*x+3,x**2),x, dict=True)
```

The equation object is often unnecessary, since if we input $f(x)$, simpy will solve for $f(x) = 0$

```python
sympy.solve(2*x+3-x**2,x)
```

Solve can't do magic, and if you give it something too complicated it will fail.

__Exercise a__

Find the positive root of the polynomial $x^2-x-1$

```python
answer = 0
verify_1.check_8a(answer)
```

    False

You can retrieve the left hand side or right hand side of equations

```python
E = sympy.Symbol('E')
M = sympy.Symbol('M')
c = sympy.Symbol('c')
temp = sympy.Eq(E,M*c**2)
[temp, temp.lhs, temp.rhs]
```

# Section 9 - Conversion

Sometimes we would like to turn a symbolic expression into a normal python function. This can be achieved using lambdify

```python
func = (x+1)/(x-1)
f = sympy.lambdify(x, func)
f(8)
```

Expressions can also be turned directly into latex

```python
print(sympy.latex(func))
```

    \frac{x + 1}{x - 1}

# Section 10 - A Worked Example

In this example we will derive the strong shock conditions in an ideal gas. We begin with the [Rankine Hugoniot conditions](https://en.wikipedia.org/wiki/Rankine%E2%80%93Hugoniot_conditions)

```python
rho_1 = sympy.Symbol('rho1', positive=True) # Upstream density
rho_2 = sympy.Symbol('rho2', positive=True) # Downstream density
v_1 = sympy.Symbol('v1', positive=True) # Upstream velocity
v_2 = sympy.Symbol('v2', positive=True) # Downstream velocity
p_1 = sympy.Symbol('p1', positive=True) # Upstream pressure
p_2 = sympy.Symbol('p2', positive=True) # Downstream pressure
h_1 = sympy.Symbol('h1', positive=True) # Upstream enthalpy
h_2 = sympy.Symbol('h2', positive=True) # Downstream enthalpy
mass_conservation = sympy.Eq(rho_1*v_1, rho_2*v_2)
momentum_conservation = sympy.Eq(p_1+rho_1*v_1**2, p_2+rho_2*v_2**2)
enthalpy_conservation = sympy.Eq(h_1+v_1**2/2, h_2+v_2**2/2)
rankine_hugoniot_conditions = [mass_conservation,
                              momentum_conservation,
                              enthalpy_conservation]
rankine_hugoniot_conditions
```

Adaptation for an ideal gas

```python
gamma = sympy.Symbol('gamma', positive=True) # Adiabatic index
ideal_gas_rankine_hugoniot_conditions = [itm.subs({h_1:gamma*p_1/rho_1/(gamma-1),
                                                   h_2:gamma*p_2/rho_2/(gamma-1)})
                                        for itm in rankine_hugoniot_conditions]
ideal_gas_rankine_hugoniot_conditions
```

sympy can solve these equations, but the result is not very insightful. A more useful approximation is the strong shock assumption, where the upstream pressure is neglected

```python
strong_sohck_rankine_hugoniot = [itm.subs(p_1,0) for itm in ideal_gas_rankine_hugoniot_conditions]
strong_sohck_rankine_hugoniot
```

Solving the system of equations to obtain the downstream values

```python
sympy.solve(strong_sohck_rankine_hugoniot,[p_2,rho_2,v_2],dict=True)[0]
```

And we reproduce one of the interesting features of strong shocks. Even when the shock is extremely strong, the shocked material is only compressed by a constant factor, that only depends on the adiabatic index

```python
temp = rho_2.subs((sympy.solve(strong_sohck_rankine_hugoniot,[p_2,rho_2,v_2],dict=True)[0]))
temp = temp/rho_1
temp = temp.subs(gamma, sympy.Rational(5,3))
temp
```

```python

```