# Visualizing stationary paths of a  functional

Last revised: 02-Feb-2019 by Dick Furnstahl [furnstahl.1@osu.edu]

Consider the functional

$\begin{align}
  S = \int_{x_1}^{x_2} f[y(x), y'(x), x] \, dx
\end{align}$

with $y_1 = y(x_1)$ and $y_2 = y(x_2)$ fixed.  We denote by $y^*(x)$ the path that minimizes $S$ (or, more generally, makes it stationary).  Then we consider the class of candidate paths $y(x)$ given by

$\begin{align}
  y(x) = y^*(x) + \alpha \eta(x)
\end{align}$

where $\eta(x)$ is some function that vanishes at the endpoints: $\eta(x_1) = \eta(x_2) = 0$.  We can derive the Euler-Lagrange equations by minimizing $S(\alpha)$ with respect to $\alpha$.

Here we visualize this problem by considering a particular $S$, choosing among some possible $\eta(x)$ definitions, and seeing how $S$ is minimized with respect to $\alpha$.  We will also allow for an incorrect determination of $y^*(x)$, in which case we expect that the minimum alpha will give us a reasonable reproduction of the true $y^*(x)$.  The variation of $\alpha$ and the choice of functions will be made using widgets from `ipywidgets`.

## Looking at a plot of the functional evaluation versus $\alpha$

\[We'll use `%matplotlib notebook` so that we can modify figures without redrawing them.\]

```python
%matplotlib notebook
```

```python
import numpy as np
import matplotlib.pyplot as plt

import ipywidgets as widgets
from IPython.display import display

```

### Functional from Taylor problem 6.12

This problem states: "Find the equation of the path from the origin $O$ to the point $P(1,1)$ in the $xy$ plane that makes the integral $\int_O^P x (1 + y'^2)$ stationary.  The answer from solving the Euler-Lagrange equation is $y^*(x) = \sinh (x + 1) C$.

```python
def y_star(x):
    """Path that minimizes the functional in Taylor problem 6.9."""
    return np.arcsinh(x + 1)
```

```python
delta_x = 0.001
x_pts = np.arange(0., 1., delta_x)

fig = plt.figure(figsize=(6,3),
                 num='Visualizing stationary paths of a functional')
ax1 = fig.add_subplot(1,2,1)
ax2 = fig.add_subplot(1,2,2)

def setup_figure():
    ax1.set_title('Show paths')
    ax1.plot(x_pts, y_star(x_pts), color='black', lw=2)
    ax1.set_xlabel('x')
    ax1.set_ylabel('y(x)')

    ax2.set_title('Evaluate functional')
    ax2.set_xlabel(r'$\alpha$')
    ax2.set_ylabel('functional')
    #ax2.set_xlim(-0.4, 0.4)
    #ax2.set_ylim(1.5, 3.)
    #ax2.axvline(0., color='black', alpha=0.3)
    ax2.axhline(evaluate_functional(x_pts, y_star(x_pts)),
                                    color='black', alpha=0.3)

    fig.tight_layout()

def evaluate_functional(x_pts, y_pts):
    """Given arrays of x and y points, evaluate the functional from 6.9."""
    # The numpy gradient function takes the derivative of an array y_pts
    #  that is a function of points x in x_pts.
    y_deriv_pts = np.gradient(y_pts, x_pts)
    f = y_deriv_pts**2 + y_pts * y_deriv_pts + y_pts**2

    # Use the numpy trapezoid rule (trapz) to do the integral over f.
    return np.trapz(f, x_pts)

def make_path(alpha, ax1_passed, ax2_passed,
              base_function='exact', eta_function='sine'):
    """Given a base function, which may be the exact y^*(x) or a guess that
       is not correct, generate and plot the path corresponding to adding
       alpha*eta(x) to the base function, with eta(x) chosen among some
       functions that vanish at the endpoints in x.
    """
    # map x_pts to zero to 1 (it may already be there)
    x_mapped_pts = (x_pts - x_pts[0]) / (x_pts[-1] - x_pts[0])

    # Choices for the base function
    if (base_function == 'exact'):
        base = lambda x : y_star(x)
    elif (base_function == 'guess 1'):
        base = lambda x : np.arcsinh(2.*x) / np.arcsinh(2.)
    elif (base_function == 'guess 2'):
        base = lambda x : x**3

    if (eta_function == 'sine'):
        eta = lambda x : np.sin(np.pi * x)
    elif (eta_function == 'parabola'):
        eta = lambda x : 4. * x * (1. - x)

    y_new_pts = base(x_pts) + alpha * eta(x_mapped_pts)

    ax1_passed.plot(x_pts, y_new_pts, color='red', lw=1)
    ax2_passed.plot(alpha, evaluate_functional(x_pts, y_new_pts), '.',
                    color='red')

def reset_graph(event):
    ax1.clear()
    ax2.clear()
    setup_figure()

button = widgets.Button(
    description='reset graph'
)
button.on_click(reset_graph)

widgets.interact(make_path,
                 alpha=widgets.FloatSlider(min=-1., max=1., step=.05,
                                           value=0.0, description=r'$\alpha$',
                                           continuous_update=False),
                 ax1_passed=widgets.fixed(ax1),
                 ax2_passed=widgets.fixed(ax2),
                 base_function=widgets.Dropdown(options=['exact', 'guess 1',
                                                         'guess 2'],
                                                value='exact',
                                                description='base function'),
                 eta_function=widgets.Dropdown(options=['sine', 'parabola'],
                                               value='sine',
                                               description=r'$\eta(x)$')
                )

setup_figure()
button
```

    <IPython.core.display.Javascript object>

    interactive(children=(FloatSlider(value=0.0, continuous_update=False, description='$\\alpha$', max=1.0, min=-1…

    Button(description='reset graph', style=ButtonStyle())

```python

```


# Taylor Expansions

Series expansions are used in a variety of circumstances:
- When we need a tractable approximation to some ugly equation
- To transform between equivalent ways of looking at a problem (e.g. time domain vs frequency domain)
- When they are (part of) a solution to a particular class of differential equation

For approximations, there is an important divide between getting the best fit *near a point* (e.g. [Taylor series](#taylor)) and getting the best fit *over an interval* (e.g. Fourier series). This notebook only deals with the former; there is a separate notebook for Fourier, Bessel, etc.

## Fitting near a point

What is the best (low order) polymomial approximating my function at *this* point? It doesn't matter if it diverges wildly as we get far away from the point, though adding higher-order terms may extend the range of usefulness.

## Taylor and Maclaurin series

A function $f(x)$ can be approximated at a point $x=a$ by the polynomial

$$ f(x) \approx f(a) + f'(a) (x-a) + \frac{1}{2} f''(a) (x-a)^2 \dots \frac{1}{n!} f^n(a) (x-a)^n $$

For this to converge, either $(x-a)$ should be small so we can ignore high powers, or the differentials should become zero.

The general case for $x=a$ is the Taylor Series. The special case where $x=0$ is sometimes called the Maclaurin series.

For the Python sections we will need a substantial range of imports, adding SymPy to the usual set of numpy, matplotlib and widgets.

```python
import numpy as np
import matplotlib.pyplot as plt

from ipywidgets import interact, interactive, fixed, interact_manual, Layout
import ipywidgets as w

plt.rcParams.update({'font.size': 16})

from sympy import * # TODO - narrow this later
init_printing()
from sympy.functions import sin, cos

from sympy.parsing.sympy_parser import parse_expr
from sympy.parsing.sympy_parser import standard_transformations, implicit_multiplication_application
transformations = (standard_transformations + (implicit_multiplication_application,))

```

Define some functions to be used later, plus a SymPy symbol.

```python
# Define the variable for SymPy functions
x = Symbol('x')

# Taylor approximation at x=a of function f, to order n
def taylor(f, a, n):
    # f is a SymPy function
    terms = []
    for i in range(n+1):
        terms.append((f.diff(x, i).subs(x, a))/(factorial(i))*(x - a)**i)
    return terms

# Plot results
def plotTaylor(f_sympy, a, n):

    # get a NumPy-style function from the SymPy version
    f_np = lambdify(x, f_sympy, 'numpy')

    # plot the starting function
    x_lims = [-5,5]
    x1 = np.linspace(x_lims[0], x_lims[1], 500)
    plt.figure(figsize=(9, 9))
    plt.plot(x1, f_np(x1), 'k.', label=f_sympy)

    # get n terms of a Taylor series
    f_taylor_terms = taylor(f_sympy, 0, n) # a list
    f_taylor = sum(f_taylor_terms) # the whole func to order n
    display(f_sympy) # display shows LaTex, print wouldn't
    print('Taylor expansion at x = {:.2f}, n = {:d}:'.format(a, n))
    display(f_taylor)

    # plot the successive approximations
    y = np.zeros(len(x1))
    for i in range(n):
        term = f_taylor_terms[i]
        if term.is_zero:
            # odd or even functions only use alternate terms
            continue
        term_np = lambdify(x, term, 'numpy')
        y += term_np(x1)
        plt.plot(x1, y, label='order ' + str(i+1))

    # graph housekeeping
    plt.xlim(x_lims)
    plt.ylim([-3,3])
    plt.xlabel('x')
    plt.ylabel('y')
    plt.legend()
    plt.grid(True)
    plt.title('Taylor series approximation of ' + str(f_sympy))
```

We will want to enter arbitrary functions into a text box, but these need to be parsed into a form that SymPy (and plotTaylor()) can use.

Note the variety of formats for our function: we start with a text string, parse it to a SymPy object suitable for symbolic math (differentiation, etc), 'lambdify' it to a NumPy function which can handle array input, and finally generate a plottable array of y-values.

```python
def parse_input(f_txt, a, n):
    f_sympy = parse_expr(f_txt, transformations=transformations)
    plotTaylor(f_sympy, a, n)
```

Enter any valid Python function into the text box. Enter or Tab will get trigger a redraw. Don't forget to use `**` for exponentiation rather than `^`!

Implicit multiplication may work, e.g. `x sin(x)` as a synonym for `x*sin(x)`. Adding a space improves your chances of success.

The parser presumably has limits but it's unclear what they are. Experiment...

```python
style = {'description_width': 'initial'} # to avoid the labels getting truncated
interact(parse_input,
             f_txt = w.Text(description='f(x):',
                                            layout=Layout(width='80%'),
                                            continuous_update=False,
                                            value='sin(x)'),
             a = w.FloatSlider(description="Evaluation point $a$", style=style,
                                            layout=Layout(width='80%'),
                                            continuous_update=False,
                                            min=-4, max=4,
                                            value=0),
             n = w.IntSlider(description="Number of terms $n$", style=style,
                                            layout=Layout(width='80%'),
                                            continuous_update=False,
                                            min=1, max=20,
                                            value=6));
```

    interactive(children=(Text(value='sin(x)', continuous_update=False, description='f(x):', layout=Layout(width='…

## References

Boas, "Mathematical methods in the physical sciences"

```python

```


Traçar um esboço do gráfico e obter uma equação da parábola que satisfaça as condições dadas.

21. Vértice: $V(0,-2)$; diretriz: $2x-3=0$

Arrumando a equação da diretriz

$d: x = \frac{3}{2}$

Fazendo um esboço é possivel perceber que a parábola é paralela ao eixo $x$, logo sua équação é dada por $(y-k)^2 = 2p(x-h)$

Substituindo os pontos do vértice por $x=0$ e $y=-2$

$(y-(-2))^2 = 2p(x-0)$

$(y+2)^2 = 2px$

Achando o valor de $p$, utilizando a coordenada da diretriz $D(\frac{3}{2},-2)$

$\frac{p}{2} = \sqrt{(0-\frac{3}{2})^2+(-2-(-2))^2}$

$\frac{p}{2} = \sqrt{(-\frac{3}{2})^2 + 0}$

$\frac{p}{2} = \pm \sqrt{\frac{9}{4}}$

$\frac{p}{2} = -\frac{3}{2}$

$p = -3$

Substituindo $p$ na fórmula

$(y+2)^2 = 2 \cdot -3 \cdot x$

$(y+2)^2 = -6x$

$y^2 + 4y + 4 = -6x$

$y^2 + 4y + 6x + 4 = 0$

Gráfico da parábola

```python
from sympy import *
from sympy.plotting import plot_implicit
x, y = symbols("x y")
plot_implicit(Eq((y+2)**2, -6*(x+0)), (x,-20,20), (y,-20,20),
title=u'Gráfico da parábola', xlabel='x', ylabel='y');
```

Problemas Propostos. Pag. 172 - 175/21.ipynb


# Symbolinen laskenta sympy-kirjastoa käyttäen

Tässä muistikirjassa käydään läpi Pythonin symbolisen laskennan sympy-kirjaston perusulottuvuuksia.

```python
import sympy
```

```python
z**2
```

```python
sympy.var('z') # esitellään muuttuja ensin
```

    z

```python
z**2 # nyt sillä voidaan laskea
```

    z**2

```python
True^True # ^ on varattu Pythonissa XOR-operaatiolle
```

    False

```python
True^False
```

    True

```python
z**(1/2)
```

    z**0.5

Yllä Python laskee suluissa olevan jakolaskun ensin
ja siitä tulee Python-objekti ennen kuin SymPy ehtii muuttaa sen SymPyn olioksi.

Ratkaisu on käyttää sympify() tai Rational-luokkaa.

```python
z**(sympy.sympify(1)/2)
```

    sqrt(z)

```python
z**sympy.Rational(1/2)
```

    sqrt(z)

Esitellään muuttujia:

```python
from sympy import symbols
x, y, z = symbols('x y z')
u, v, w = symbols('u v w')
symbols('j0:7')
```

    (j0, j1, j2, j3, j4, j5, j6)

```python
g, h = symbols('h g')
```

```python
g
```

    h

```python
h
```

    g

Variables Assignment does not Create a Relation Between Expressions

```python
from sympy import Symbol
a=Symbol('a') # muuttujan a arvo a
```

```python
b=a+1 # muuttujan b arvo lauseke a+1
```

```python
print(b)
```

    a + 1

```python
a=4 # nyt a osoittaa kokonaislukuun 4, ei symboliin a
```

```python
print(a)
```

    4

```python
print(b) # b osoittaa lausekkeeseen, jonka arvo a+1
```

    a + 1

```python
from sympy.abc import p
```

```python
sympy.sqrt(p**2) # SymPy ei yksinkertaista tätä, koska vastauksena voi olla 1- tai 1
```

    sqrt(p**2)

```python
p=Symbol('p', positive=True) # nyt ratkaisu on yksikäsitteinen
```

```python
sympy.sqrt(p**2)
```

    p

Yhtäsuuruuden testausta, Eq, ==

```python
from sympy import symbols
x, y = symbols('x y')
sympy.Eq(x,y) # symbolinen yhtäsuuruus ilmaistaan Eq:lla
```

    Eq(x, y)

```python
(x-1)**2 == x**2-2*x+1 # == testaa symbolista samankaltaisuutta
```

    False

```python
(x-1)**2 == (x-1)**2 # == testaa symbolista samankaltaisuutta
```

    True

Semanttisen samankaltaisuuden testaamiseen vähennä lausekkeet toisistaan ja käytä expand, simplify. Muista myös trigsimp, powsimp, trigsimp, logcombine, radsimp, together

```python
from sympy import simplify
simplify((x-1)**2-(x**2-2*x+1))
```

    0

```python
from sympy import sin, cos, simplify, expand, powsimp, trigsimp, logcombine, radsimp, together
sinilauseke1 = sin(2*x)-2*sin(x)*cos(x)

```

```python
simplify(sinilauseke1)
```

    0

```python
expand(sinilauseke1, trig=True)
```

    0

```python
from sympy.abc import a,b
expand((a-b)**4) # lasketaan (a-b)^4
```

    a**4 - 4*a**3*b + 6*a**2*b**2 - 4*a*b**3 + b**4

```python
expand(2*x+3*y, complex=True) # otetaan kompleksiluvut käyttöön
```

    2*re(x) + 3*re(y) + 2*I*im(x) + 3*I*im(y)

```python
expand(cos(x+y)**2, trig=True) # otetaan trigonometriset funktiot käyttöön
```

    sin(x)**2*sin(y)**2 - 2*sin(x)*sin(y)*cos(x)*cos(y) + cos(x)**2*cos(y)**2

```python
expand(cos(x+2*y), trig=True)  # otetaan trigonometriset funktiot käyttöön
```

    -2*sin(x)*sin(y)*cos(y) + 2*cos(x)*cos(y)**2 - cos(x)

```python
simplify((x**3+y**2*x)/x) # lausekkeen sieventämistä
```

    x**2 + y**2

```python
from sympy import I
expand((4+3*I)**(4+3*I)) #lausekkeen sieventämistä
```

    336*I*(4 + 3*I)**(3*I) - 527*(4 + 3*I)**(3*I)

Harjoittele: laske (x-y)^4 ja sievennä sin(x)/cos(x)

```python
powsimp(x**a*x**b) #lasketaan potensseja
```

    x**(a + b)

```python
import pprint #pretty printing
pprint((3-x**(2*x)/(x+1)))
```

```python
together(x**6-17*x**4+8*x**2+5+3*x**5+22*x**3-8*x+6)
```

    x**6 + 3*x**5 - 17*x**4 + 22*x**3 + 8*x**2 - 8*x + 11

```python
together(1/a+1/b)
```

    (a + b)/(a*b)

Kuviot

```python
import matplotlib.pyplot
%matplotlib inline
```

```python
sympy.plot(sympy.sin(x))
```

```python
sympy.plot(sympy.sin(x)/x)
```

```python
sympy.plot(sympy.exp(-x)*sympy.sin(x**2), (x, 0, 1)) #tässä rajoitettiin aluetta
```

```python
sympy.plot(sympy.exp(2*x))
```

```python
sympy.plot(sympy.log(2/x))
```

Derivointi, differentiaalit ja integrointi

```python
from sympy import diff, exp
diff(cos(2*x),x)
```

    -2*sin(2*x)

```python
k = symbols('k', integer=True)
diff(exp(k*x),x)
```

    k*exp(k*x)

```python
from sympy import log
diff(log(1/x),x)
```

    -1/x

Korkeammat derivaatat

```python
diff(cos(2*x),x,3)
```

    8*sin(2*x)

```python
diff(exp(k*x),x,2)
```

    k**2*exp(k*x)

```python
from sympy import erf
diff(erf(x),x)
```

    2*exp(-x**2)/sqrt(pi)

Hyvä tapa on myös määritellä lausekkeita ja derivoida niitä:

```python
expr = x**2*sympy.sin(sympy.log(x))
```

```python
sympy.diff(expr,x)
```

    2*x*sin(log(x)) + x*cos(log(x))

Osittaiderivointi:

```python
expr2 = x*sympy.cos(y**2 + x)
```

```python
sympy.diff(expr2, x, 2, y, 3)
```

    4*y*(-2*x*y**2*sin(x + y**2) + 3*x*cos(x + y**2) + 4*y**2*cos(x + y**2) + 6*sin(x + y**2))

Määräämätön derivaatta:

```python
sympy.Derivative(expr2, x, 2, y, 3)
```

    Derivative(x*cos(x + y**2), (x, 2), (y, 3))

```python
sympy.Derivative(expr2, x, 2, y, 3).doit()
```

    4*y*(-2*x*y**2*sin(x + y**2) + 3*x*cos(x + y**2) + 4*y**2*cos(x + y**2) + 6*sin(x + y**2))

Integrointi:

```python
from sympy import integrate
integrate(5*x**5,x)
```

    5*x**6/6

```python
integrate(3*exp(3*x),x)
```

    exp(3*x)

```python
integrate(cos(x),x)
```

    sin(x)

```python
integrate(log(x),x)
```

    x*log(x) - x

```python
integrate(exp(-x**2)*erf(x), x)
```

    sqrt(pi)*erf(x)**2/4

```python
sympy.integrate(sympy.exp(-(x+y))*sympy.cos(x)*sympy.sin(y), x, y)
```

    -exp(-x)*exp(-y)*sin(x)*sin(y)/4 - exp(-x)*exp(-y)*sin(x)*cos(y)/4 + exp(-x)*exp(-y)*sin(y)*cos(x)/4 + exp(-x)*exp(-y)*cos(x)*cos(y)/4

Määritetään lauseke ja operoidaan sillä:

```python
integrand=sympy.log(x)**2
```

```python
sympy.integrate(integrand, x)
```

    x*log(x)**2 - 2*x*log(x) + 2*x

Sarjat:

```python
from sympy import series
series(cos(x), x)
```

    1 - x**2/2 + x**4/24 + O(x**6)

```python
series(erf(x),x)
```

    2*x/sqrt(pi) - 2*x**3/(3*sqrt(pi)) + x**5/(5*sqrt(pi)) + O(x**6)

```python
series(exp(x),x)
```

    1 + x + x**2/2 + x**3/6 + x**4/24 + x**5/120 + O(x**6)

```python
series(1/cos(x), x)
```

    1 + x**2/2 + 5*x**4/24 + O(x**6)

Sympy laskimena

```python
v, w, u = sympy.symbols('v, w, u')
n = sympy.symbols('n', interger=True)
from sympy import Mul, Pow, Rational, pi, N, oo
```

```python
Mul(3, Rational(2,4))
```

    3/2

```python
Mul(3, Rational(2,4), evaluate=False)
```

    3/2

```python
Pow(x*y,2)
```

    x**2*y**2

```python
Pow(x*y,2, evaluate=False)
```

    (x*y)**2

```python
pi*2
```

    2*pi

```python
exp(1)
```

    E

```python
pi.evalf()
```

    3.14159265358979

```python
pi.evalf(5)
```

    3.1416

```python
N(pi)
```

    3.14159265358979

```python
N(pi, 5)
```

    3.1416

```python
print(exp(1))
```

    E

```python
oo>999999999
```

    True

```python
oo/2
```

    oo

Laske
- luvun kaksi neliöjuuri 50 desimaalilla,
- mikä on piin 26. desimaali,
- paljonko on luvun pi + e likiarvo.

Raja-arvot

```python
from sympy import limit, sin, exp, oo
limit(sin(x)/x,x,0)
```

    1

```python
limit(exp(x)/x, x, oo)
```

    oo

```python
limit((1+1/x)**x,x,oo)
```

    E

Määrätty intergraali:

```python
integrand=sympy.log(x)**2
```

```python
sympy.integrate(integrand, (x, 1, 10))
```

    -20*log(10) + 18 + 10*log(10)**2

```python
sympy.integrate(sympy.exp(-x), (x, 0, sympy.oo))
```

    1

Moninkertainen integraali:

```python
integrate(exp(-x**2-y**2),(x,-oo,oo),(y,-oo,oo))
```

    pi

```python
Voidaan myös luoda määräämätön Integral-objekti. Sen arvo voidaan laskea myöhemmin.
```

```python
sympy.Integral(integrand, (x, 1, 10))
```

    Integral(log(x)**2, (x, 1, 10))

```python
sympy.Integral(integrand, (x, 1, 10)).doit()
```

    -20*log(10) + 18 + 10*log(10)**2

Yhtälönratkaisu

```python
a, b, c, x = sympy.symbols(('a', 'b', 'c', 'x'))
```

```python
quadr_eq = sympy.Eq(a*x**2+b*x+c, 0)
```

```python
sympy.solve(quadr_eq)
```

    [{a: -(b*x + c)/x**2}]

```python
sympy.solve(quadr_eq, x)
```

    [(-b + sqrt(-4*a*c + b**2))/(2*a), -(b + sqrt(-4*a*c + b**2))/(2*a)]

Opetus: Jos ei muuta määritetä, sympy ratkaisee ensimmäisen symbolin suhteen.

```python
from sympy import solve
solve(x**4 - 1, x)
```

    [-1, 1, -I, I]

```python
solve(x**2+2*x-1,x)
```

    [-1 + sqrt(2), -sqrt(2) - 1]

Yllä olevat vastaukset hakasuluissa [] ovat listoja. Listat voivat sisältää erilaisia objekteja ja niitä voi muuttaa.

```python
from sympy import roots
roots(x**2+2*x-1,x)
```

    {-sqrt(2) - 1: 1, -1 + sqrt(2): 1}

```python
solve([x + 5*y - 2, -3*x + 6*y - 15], [x, y])
```

    {x: -3, y: 1}

Edellä olevat vastaukset taas ovat sanakirjoja. Ne ovat järjestämättömiä listoja ilman toistoja.

Huom. solve -> list, roots -> dictionary.

```python
solve(exp(x) + 1, x) #Eulerin lause, Eulerin identiteetti
```

    [I*pi]

Ratkaise yhtälö
(x-1)^4=4^4

Jatketaan yhtälön quadr_eq ratkaisemista:

```python
ratkaisut=sympy.solve(quadr_eq, x)
```

```python
xplus=ratkaisut[0]
xminus=ratkaisut[1]
```

```python
xplus_arvot=xplus.subs([(a,1),(b,2),(c,3)])
```

```python
xplus_arvot
```

    -1 + sqrt(2)*I

```python
Voidaan sijoittaa myös muita muuttujia:
```

```python
sympy.var('z0')
xminus_arvot=xminus.subs([(b,a), (c,a+z0)])
```

```python
xminus_arvot
```

    -(a + sqrt(a**2 - 4*a*(a + z0)))/(2*a)

```python
xminus_arvot.simplify()
```

    -(a + sqrt(-a*(3*a + 4*z0)))/(2*a)

Solveset on suositeltavin yhtälöiden ratkaisuun. Solveset kertoo kunkin ratkaisun vain kerran, moninkertaisuuden selvittämiseksi on hyvä käyttää funktiota roots:

```python
from sympy import solveset, roots
solveset(x**3-6*x**2+9*x,x)
```

    {0, 3}

```python
roots(x**3-6*x**2+9*x,x)
```

    {0: 1, 3: 2}

Huomaa vielä ero kahden seuraavan outputin kanssa.

```python
solveset(exp(x), x) # Yhtälöllä e^x = 0 ei ole ratkaisuja.
```

    EmptySet()

```python
solveset(cos(x)-x,x) # Tälle yhtälölle sympy ei löydä ratkaisua.
```

    ConditionSet(x, Eq(-x + cos(x), 0), Complexes(Reals x Reals, False))

Määritellään vielä funktioita:

```python
from sympy import Function
f, g = symbols('f g', cls=Function)
```

```python
f(x)
```

    f(x)

```python
f(x).diff(x)
```

    Derivative(f(x), x)

Tekijöihin jakoa:

```python
poly = x**4 - 3*x**2 + 1
```

```python
from sympy import factor
factor(poly)
```

    (x**2 - x - 1)*(x**2 + x - 1)

```python
factor(poly,modulus=5)
```

    (x - 2)**2*(x + 2)**2

Boolean lausekkeita

```python
from sympy import satisfiable
satisfiable(x & y)
```

    {x: True, y: True}


## ILAS : Introduction to Programming for Engineers
## Exam 2018

Answer all questions.

Submit your work by email to philamore.hemma.5s@kyoto-u.ac.uk

Format: .ipynb file.

Deadline: 23:59 July 23 (07/23)

Total: 50 marks

### Question 1
###### 1 mark
The gravitational potential, $V$, of a particle of mass $m$ at a distance $r$ from a body of mass $M$, is:

$$
V = \frac{G M m}{r}
$$

In the cell below, find $V$ when:

$G$ = *gravitational constant* = $6.674 \times 10^{-11} \textrm{Nm}^{2}\textrm{kg}^{-2}$.

$M = 1.65 \times 10^{12}\textrm{kg}$

$m = 6.1 \times 10^2\textrm{kg}$

$r = 7.0 \times 10^3\textrm{kg}$

Use variable names `G`, `M`, `m` and `r` to represnt $G, M, m$ and $r$.

Express the numbers using __scientific notation__.

```python

```

### Question 2
#### Part A
###### 1 mark
The volume of a cone is:
$$
V = \frac{1}{3}(base \ area)\times(perpendicular \ height)
$$

Find the internal volume of a cone of internal dimensions:

base radius, $r = 5cm$

perpendicular height, $h = 15cm$

Assign variables for $r$ and $h$ before solving.

```python
pi = 3.142
# Internal volume
```

#### Part B
###### 2 marks

The cone is filled with liquid.

The liquid is transferred to a hollow cylinder (base radius , $r_c = 4cm$).

The __volume__ of liquid in the cylinder is:

$V = (base \ area)\times(height \ of \ liquid)$

Find the __height__ of the liquid in the cylinder?

Create a variable for $r_c$ before solving.

```python

```

#### Part C
###### 1 mark
The total height of the cyclinder, $H_{tot}$ is 10cm.

Use a __comparison operator__ to show if the height of the liquid, $H$, is more than half the total height of the cylinder.

```python

```

### Question 3
#### Part A
###### 2 marks
Use a for loop to print the square roots of the first 25 odd positive integers.

#### Part B
###### 2 marks
If the number generated is greater than 3 and smaller than 5, print "`skip`" and __`continue`__ to the next iteration *without* printing the number.

```python

```

### Question 4

#### Part A
###### 3 marks
Write a function called `is_even` that determines if a number is even by running:

```python
is_even(n)
```

Input: `n` (an integer).

Output:
 - `True` if the argument is even
 - `False` if the argument is not even

Include a __documentation string (docstring)__ to say what your function does.

#### Part B
###### 3 marks
Edit `is_even` to:
- take two input arguments:
 - a numeric variable, `n`
 - a numpy function that returns the sqaure root of the input argument.
- return:
 - `True` if the square root of n is even
 - `False` if the square root of n is not even

#### Part C
###### 1 mark
Make `square_root` the __default__ value of the function argument.

Force the function argument to be input using a named argument.

#### Part D
###### 1 mark
Print the output ofthe function `is_even` for the first 25 natural numbers.

```python

```

### Question 5
###### 2 marks
In the example below, complete the comments with definition ("local variable"/"global variable") describing the scope of variables a-c.

```python
# In the code below:
# a is a local variable / global variable
# b is a ...
# c is a ...
# d is a ...

def my_function(a):
    b = a - 2
    return b

c = 3

if c > 2:
    d = my_function(5)
    print(d)
```

    3

### Question 6
###### 3 marks
Write a function called counter:

Input: A list. e.g. `[7, 3, 5, 8, 50]`

Output: Returns the numer of times a number greater than 5 appears in a list.

```python

```

### Question 7

The magnitude of an $n$ dimensional vector can be written

$$
|\mathbf{x}|= \sqrt{x_1^2 + x_2^2 + ... x_n^2} = \sqrt{\sum_{i = 1}^{n} (x_{n})^2 }
$$

Therefore...

The magnitude of a 2D vector (e.g. $x = [x_1, x_2]$):

$$
|\mathbf{x}|= \sqrt{x_1^2 + x_2^2}
$$

The magnitude of a 3D vector (e.g. $x = [x_1, x_2, x_3]$):

$$
|\mathbf{x}|= \sqrt{x_1^2 + x_2^2 + x_3^2}
$$

#### Part A
###### 3 marks
In the cell below, write a function called `magnitude` that computes the magnitude of an n-dimensional vector.

Input: `list` with n elements (e.g. [x, y]) if vector is 2D, [x, y, z]), if vector is 3D.

Output: Return magnitude of the vector.

Include a doc-string to say what your function does.

#### Part B
###### 1 marks
Print the output of your function to show that it works for both 2D and 3D input vectors.

Check your function, for example use the numpy function `numpy.linalg.norm()` to verify the answer is correct.

```python
# A function that computes the magnitude of an n-dimensional vector
```

## Question 8

The factorial of a positive integer $n$ is:

\begin{align}
n! = \prod_{i=1}^{n} i =1 \cdot 2 \cdot 3 \cdot ... (n - 2) \cdot (n - 1) \cdot n
\end{align}

We can write this *recursively*.

i.e. using the value of $(n-1)!$ to compute the value of $n!$:

$$
n! = (n-1)! n
$$

Note: $0! = 1$

e.g.

 $1! = 1 \cdot 0! = 1 $

 $2! = 2 \cdot 1! = 2 \cdot 1 = 2$

 $3! = 3 \cdot 2! = 3 \cdot 2 \cdot 1 = 6$

 $4! = 4 \cdot 3! = 4 \cdot 3 \cdot 2 \cdot 1 = 24$

#### Part A
###### 4 marks
Write a __recursive function__ called `factorial` to compute $n!$ of an input argument `n`:

- __Input:__ Numerical variable `n`
- __Output:__ `factorial(n) = factorial(n-1)*n`

Include a doc-string to say what your function does.

#### Part B
###### 3 marks
The formula above only applies to __positive integers__(with a specific exception for 0).

Edit your function to raise an error if the input value is a positive integer or zero.

```python

```

## Question 9
###### 6 marks
Create an array, `x` with 100 values from 0 to 20, inclusive.

Compute $y=\sin(x)$.

Plot $y$ vs. $x$ with a blue line.

Replace all values of $y$ that are larger than 0.5 with 0.5.

Replace all values of $y$ that are smaller than $-$0.75 with $-0.75$.

Plot $x$ vs. $y$ using a red line on the same graph.

```python

```

## Question 10

Using the example data set `a`,`b`:

#### Part A
###### 2 marks
Plot a scatter graph of `a` against `b`, with `a` on the horizontal axis and `b` on the vertical axis.

Remember to use `%matplotlib inline` to make figures appear in Jupyter Notebook

#### Part B
###### 5 marks
Try fitting 3 different polynomial functions to the data. Plot each curve as a line of on the graph.

#### Part C
###### 3 marks
Show which polynomial function gives the best fit using the root mean squared error (RMSE). Print the equation of the *optimal* curve i.e. the curve with the *smallest* RMSE.

#### Part D
##### 1 mark
Display the RMSE of each curve as a figure legend.

```python
import numpy as np
a = np.array([88438,45505,75127,115571,89911,87432,100083,85589,73104,86890,78580,70785,41050,57610,107537,59262,73038,87891,75368,111638,74911,71599,96774,79667,90725,93816,75859,64969,205688,71500,53098,71250,89615,94747,50400,63673,78257,72785,83015,150000,84699,67191,86298,117705,88935,89643,106678,97894,132164,59387,60684,96151,68794,74559,29430,88362,111792,57205,83651,87518,80129,86801,110761,63274,66143,110694,52590,59994,80460,103589,68298,59056,40294,161848,103100,86354,37428,43307,80792,77368,109159,71538,84783,86250,82900,74728,48597,75549,106942,102167,62708,60630,70273,84918,88693,74141,46627,119112,88260,97262,86095,110472,82734,84761,91715,103292,86339,147993,77560,100625,68094,78250,75426,86138,112344,115000,98846,90499,80029,61959,76779,68833,81026,66361,92737,76692,64974,103869,51951,108854,61038,75938,75346,40639,73156,80067,82322,52353,
              62832,207262,160106,77740,72011,167094,58458,41639,79528,66583,83993,138082,77366])
b = np.array([1.7,-0.4,0.5,2.6,1.4,1.5,1.5,1.7,-0.5,1.6,0.9,1.1,-1.7,0.3,1.8,0.5,1,1.9,0.1,2,1.7,1,1.2,1.5,1,1.1,1.2,0,2.6,1.4,-0.8,1.6,1.1,1.2,-1.4,-0.5,1.9,0,1.5,2.4,1.5,0.7,1.8,2,2.4,1.6,2,2.3,2,0.1,0.3,2.3,0,0,-1.7,1.9,2,0,0.9,1.3,0.4,1.6,2.3,-0.1,1.7,2.1,-0.9,0.1,1,1.9,0.4,-0.3,-2.4,2.7,1.3,2,-1.3,-1.5,0.7,1.1,2.3,1.1,0.7,0.9,1.1,0.1,-0.9,1.4,2.1,1.2,0.1,0.8,0.3,1.4,1.5,1,-0.5,2.4,0.9,1.5,1.6,1.2,1.3,1.8,0.8,1.8,1.9,2.6,1.5,1.8,1.8,0.6,0.7,1.2,1.5,2.5,1.1,1.6,1.6,1,0,0,1,0.5,1.7,0.6,0.1,1.7,0.2,2.1,0.1,0.9,0.8,-1.3,1.3,0.5,1.5,-0.6,1.2,2.4,2.6,1.1,0.8,2.5,-0.2,-2,0.1,0.1,1.6,2.6,1.2])
```

## Questionnare
This is a question about your experience of the course this year.

Please answer honestly and constructively.

You may give written answers in English or Japanese.

この問題は、この授業の君の経歴を訪ねるアンケートである

ただし、真面目、かつ真剣な答案が望ましい。

英語や日本語で回答できる。勿論、点数に関係ないか、出来れば英語でお願いします。

__a) ENTRY LEVEL__

Please write the title of your major at your home university.

君の大学の専門の名前を書きなさい。

Please write which number best describes your knowledge of this subject before starting the course?

この授業のする前に、授業の内容に対して、自分に最も近い状況を、次の番号の中から選んてください。
1. No previous study.　全然知識無し
1. A basic understanding.　基本の知識が持つ
1. A good fundamental understanding.　ある程度の知識が持つ、深い理解した
1. Comprehensive previous study.　私はプログラムマスター

```python
# Major 専門名:

# Knowledge of this subject before starting the course　この授業の前の知識レベル:
```

__b) ENJOYMENT__

Write what you enjoyed most/found most interesting  about the course.

この授業の最も面白いところを書きなさい。

```python

```

Write what you enjoyed least / found least interesting about the course.

この授業の最もつまらないところを書きなさい。

```python

```

__c) KNOWLEDGE TRANSFER__

Please choose the number indicating how much you feel your understanding improved as a result of taking this course.

この授業した後、どのくらい上昇したが、次の番号の中から選んてください。

1.	__Dramatic improvement.__ I can confidently and independently use and apply my knowledge of programming in Python.
__驚き激増した.__　自力でPythonを使える、思ったままPythonプログラムイング出来る。

2.	__Significant improvement.__ I have a much better understanding of this subject than when starting the course.
__かなり増えた.__　授業の前と比べて、プログラムイングのことよく分かりました。

3.	__Slight improvement.__ My knowledge has improved but I do not feel confident in my understanding of this topic.
__少し増えた.__　ある程度の知識を受けたが、まだ分からいことある。

4.	__No improvement.__ My knowledge of this topic has improved very little as a result of taking this course.
__全然増えてない.__　授業を受けでも、前と比べて、あまり増えることはない。

```python

```

__d) DELIVERY__

Please rate the __use of english__ by writing the number you feel is most appropriate next to each of the communication formats listed below .

先生の英語（文章）は読めやすさ、次の番号の中から選んてください。
1. Easy to understand.　読め易い
1. A few things were unclear / not well explained.　少し読めないところがある
1. Many things were unclear / not explained well.　たくさん読めないところがある
1. I could understand very little. 　読めるところも少ない

```python
# Written instructions in Jupyter notebooks `Jupyter notebooks`の使え方の説明（理解安さは何番）:

# Written instructions for Test-Yourself exercises 毎章の練習問題の説明（理解安さは何番）:

# Written instructions for coursework assignement 最後の宿題の説明（理解安さは何番）:

# Verbal instructions/explanation 全体的の説明（理解安さは何番）:
```

Please rate __how well-explained and understandable__ the course material was by writing the number you feel is most appropriate next to each of the communication formats listed below .

先生の説明（文章）は理解やすかったが、次の番号の中から選んてください。
1. Easy to understand.　理解易い
1. A few things were unclear / not well explained.　少し理解できないところがある
1. Many things were unclear / not explained well.　たくさん理解できないところがある
1. I could understand very little. 　理解できるところも少ない

```python
# Written instructions in Jupyter notebooks `Jupyter notebooks`の使え方の説明（理解安さは何番）:

# Written instructions for Test Yourself exercises 毎章の練習問題の説明（理解安さは何番）:

# Written instructions for coursework assignement 最後の宿題の説明（理解安さは何番）:

# Verbal instructions/explanation 全体的の説明（理解安さは何番）:
```

_e) FURTHER STUDY__

Are you interested to continue studying programming?

プログラミングを引き続き学びたいですか.

```python

```

__f) ADDITIONAL COMMENTS__

__[optional]__

Please include any additional ideas you have about the course.

この授業についてコメントがあれば自由に記載してください

```python

```

```python

```


# 5.1 Explanation Continuity

## Introduction

**This section corresponds to Section 7.1 of the original paper.**

What characterizes a good explanation? A first desirable property of an explanation technique is that if it is given two similar images, then its explanation must also be similar. Explanation continuity (or lack of it) can be quantified by looking for the strongest variation of the explanation $R(x)$ in the input domain:

\begin{equation}
\max_{x \neq x'} \frac{\lVert R(x) - R(x') \lVert_1}{\lVert x - x' \lVert_2}
\end{equation}

When $f(x)$ is a deep ReLU network, both sensitivity analysis and Simple Taylor Decomposition have sharp discontinuities in their explanation function while Deep Taylor Decomposition provides continuous explanations. Let us explore what this means with a simple function $f(\mathbf{x}) = \max (x_1, x_2)$ in $\mathbb{R}_2^+$, here implemented by the two-layer ReLU network

\begin{equation}
f(\mathbf{x}) = \max \left( 0, \frac{\max (0, x_1 - x_2) + \max (0, x_2 - x_1) + \max (0, x_1 + x_2)}{2} \right)
\end{equation}

The network is comprised of three layers (input, hidden and output). The number at each line connecting the neurons indicates the associated weight value and the bent line denotes the use of ReLU activation function. $a_j$ and $b_k$ denote activations of the hidden and output layers respectively. Then, we can define the network as:

\begin{align}
a_1 & = \max (0, x_1 - x_2) \\
a_2 & = \max (0, x_2 - x_1) \\
a_3 & = \max (0, x_1 + x_2) = x_1 + x_2 \\
b_1 & = \max \left( 0, \frac{a_1 + a_2 + a_3}{2} \right) = \frac{a_1 + a_2 + a_3}{2} = \max (x_1, x_2)
\end{align}
We could remove the $\max$ in *Eqs. (5)* and *(6)* because $x_1$ and $x_2$ are guaranteed to be positive.

Now, we are ready to compare the various explanation techniques we convered throughout the tutorials. I will not calculate the relevance scores for Sensitivity Analysis and Simple Taylor Decomposition because they can be easily derived.

### Sensitivity Analysis

\begin{equation}
R(\mathbf{x}) =
\begin{cases}
(1,0) & \text{if} \ x_1 > x_2 \\
(0,1) & \text{if} \ x_1 < x_2
\end{cases}
\end{equation}

### Simple Taylor Decomposition

\begin{equation}
R(\mathbf{x}) =
\begin{cases}
(x_1,0) & \text{if} \ x_1 > x_2 \\
(0,x_2) & \text{if} \ x_1 < x_2
\end{cases}
\end{equation}

### Deep Taylor Decomposition

To derive the relevance at the input layer, we must backpropagate the function output throughout the layers using the Deep Taylor Decomposition rules. Since the input is constrained to be positive ($\mathbb{R}_2^+$) and we are using ReLU activation in rest of the layers, we only need to apply the $z^+$-rule.

#### Relevance propagation from output to hidden layer

The $z^+$-rule is given below:

\begin{equation}
R_{j \leftarrow k}^{(2,3)} = \frac{z_{jk}^+}{\sum_k z_{jk}^+} R_{k}^{(3)}
\end{equation}
where $z_{jk}^+ = a_j w_{jk}^+$ and where $w_{jk}^+$ denotes the positive part of $w_{jk}$.

By the *Layer-wise Relevance Conservation Constraint*, $R_k^{(3)} = b_1$. $\{z_{jk}^+\}$ and $\sum_j \{z_{jk}^+\}$ is given as follows:

\begin{equation}
\{z_{jk}^+\} =
\begin{bmatrix}
\displaystyle \frac{a_1}{2} \\ \displaystyle \frac{a_2}{2} \\ \displaystyle \frac{a_3}{2}
\end{bmatrix}
\end{equation}

\begin{equation}
\sum_j \{z_{jk}^+\} =
\begin{bmatrix}
\displaystyle \frac{\sum_j a_j}{2}
\end{bmatrix}
\end{equation}

Then $R_{j \leftarrow k}^{(2,3)} = R_{j}^{(2)}$ can be calculated with *Eq. (9)*.

\begin{equation}
R_{j}^{(2)} =
\begin{bmatrix}
\displaystyle \frac{a_1}{\sum_j a_j} \cdot b_1 \\ \displaystyle \frac{a_2}{\sum_j a_j} \cdot b_1 \\ \displaystyle \frac{a_3}{\sum_j a_j} \cdot b_1
\end{bmatrix}
\end{equation}

#### Relevance propagation from hidden to input layer

The $z^+$-rule is given below:

\begin{equation}
R_{i \leftarrow j}^{(1,2)} = \frac{z_{ij}^+}{\sum_j z_{ij}^+} R_{j}^{(2)}
\end{equation}
where $z_{ij}^+ = x_i w_{ij}^+$ and where $w_{ij}^+$ denotes the positive part of $w_{ij}$.

$\{z_{ij}^+\}$ and $\sum_i \{z_{ij}^+\}$ is given as follows:

\begin{equation}
\{z_{ij}^+\} =
\begin{bmatrix}
x_1 & 0 & x_1 \\
0 & x_2 & x_2
\end{bmatrix}
\end{equation}

\begin{equation}
\sum_i \{z_{ij}^+\} =
\begin{bmatrix}
x_1 & x_2 & x_1 + x_2 \\
\end{bmatrix}
\end{equation}

Then $R_{i}^{(1)} = \sum_j R_{i \leftarrow j}^{(1,2)}$ can be calculated with *Eq. (13)*.

\begin{equation}
R_{i}^{(1)} =
\begin{bmatrix}
\displaystyle \frac{a_1}{\sum_j a_j} \cdot b_1 + \frac{x_1}{x_1+x_2} \cdot \frac{a_3}{\sum_j a_j} \cdot b_1 \\
\displaystyle \frac{a_2}{\sum_j a_j} \cdot b_1 + \frac{x_2}{x_1+x_2} \cdot \frac{a_3}{\sum_j a_j} \cdot b_1
\end{bmatrix} =
\begin{bmatrix}
\displaystyle \frac{a_1}{2} + \frac{x_1}{x_1+x_2} \cdot \frac{a_3}{2} \\
\displaystyle \frac{a_2}{2} + \frac{x_2}{x_1+x_2} \cdot \frac{a_3}{2}
\end{bmatrix} =
\begin{bmatrix}
\displaystyle \frac{a_1 + x_1}{2} \\
\displaystyle \frac{a_2 + x_2}{2}
\end{bmatrix}
\end{equation}

This gives us the following relevance scores:

\begin{equation}
R(\mathbf{x}) =
\begin{cases}
\displaystyle \left( \frac{2 x_1 - x_2}{2}, \frac{x_2}{2} \right) & \text{if} \ x_1 > x_2 \\[1.5ex]
\displaystyle \left( \frac{x_1}{2}, \frac{- x_1 + 2 x_2}{2} \right) & \text{if} \ x_1 < x_2
\end{cases}
\end{equation}

Visualization of all three techniques gives us the following figure:

It can be seen that despite the continuity of the prediction function, the explanations offered by Sensitivity Analysis and Simple Taylor Decomposition are discontinuous on the line $x_1 = x_2$. Only Deep Taylor Decomposition produces a smooth transition. In the Tensorflow walkthrough, we are going to demonstrate the continuity of explanation techniques in another way. We are first going to train a DNN to recognize MNIST digits. Then, we are going to compare explanations offered by different explanation techniques as we move a handwritten digit from left to right. If the technique satisfies explanation continuity, the explanations produced also must change continuously without sudden jumps.

## Tensorflow Walkthrough

### 1. Import Dependencies

```python
import os

from tensorflow.examples.tutorials.mnist import input_data
from tensorflow.python.ops import nn_ops, gen_nn_ops
import matplotlib.pyplot as plt
import matplotlib
import tensorflow as tf
import numpy as np

from models.models_5_1 import MNIST_CNN, Taylor
from utils import translate

%matplotlib inline

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
images = mnist.train.images
labels = mnist.train.labels

logdir = './tf_logs/5_1_EQ/'
ckptdir = logdir + 'model'

if not os.path.exists(logdir):
    os.mkdir(logdir)
```

    Extracting MNIST_data/train-images-idx3-ubyte.gz
    Extracting MNIST_data/train-labels-idx1-ubyte.gz
    Extracting MNIST_data/t10k-images-idx3-ubyte.gz
    Extracting MNIST_data/t10k-labels-idx1-ubyte.gz

### 2. Building Graph

```python
with tf.name_scope('Classifier'):

    # Initialize neural network
    DNN = MNIST_CNN('CNN')

    # Setup training process
    X = tf.placeholder(tf.float32, [None, 784], name='X')
    Y = tf.placeholder(tf.float32, [None, 10], name='Y')

    activations, logits = DNN(X)

    tf.add_to_collection('EC', X)

    for activation in activations:
        tf.add_to_collection('EC', activation)

    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y))

    optimizer = tf.train.AdamOptimizer().minimize(cost, var_list=DNN.vars)

    correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(Y, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

cost_summary = tf.summary.scalar('Cost', cost)
accuray_summary = tf.summary.scalar('Accuracy', accuracy)
summary = tf.summary.merge_all()
```

### 3. Training Network

This is the step where the DNN is trained to classify the 10 digits of the MNIST images. Summaries are written into the logdir and you can visualize the statistics using tensorboard by typing this command: `tensorboard --lodir=./tf_logs`

```python
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())

saver = tf.train.Saver()
file_writer = tf.summary.FileWriter(logdir, tf.get_default_graph())

# Hyper parameters
training_epochs = 15
batch_size = 100

for epoch in range(training_epochs):
    total_batch = int(mnist.train.num_examples / batch_size)
    avg_cost = 0
    avg_acc = 0

    for i in range(total_batch):
        batch_xs, batch_ys = mnist.train.next_batch(batch_size)
        _, c, a, summary_str = sess.run([optimizer, cost, accuracy, summary], feed_dict={X: batch_xs, Y: batch_ys})
        avg_cost += c / total_batch
        avg_acc += a / total_batch

        file_writer.add_summary(summary_str, epoch * total_batch + i)

    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost), 'accuracy =', '{:.9f}'.format(avg_acc))

    saver.save(sess, ckptdir)

print('Accuracy:', sess.run(accuracy, feed_dict={X: mnist.test.images, Y: mnist.test.labels}))

sess.close()
```

    Epoch: 0001 cost = 0.229207651 accuracy = 0.926581823
    Epoch: 0002 cost = 0.065176102 accuracy = 0.980072739
    Epoch: 0003 cost = 0.044646955 accuracy = 0.986000010
    Epoch: 0004 cost = 0.035834330 accuracy = 0.988909100
    Epoch: 0005 cost = 0.028319894 accuracy = 0.990672736
    Epoch: 0006 cost = 0.023727389 accuracy = 0.992454552
    Epoch: 0007 cost = 0.018540170 accuracy = 0.994145460
    Epoch: 0008 cost = 0.017687961 accuracy = 0.993981824
    Epoch: 0009 cost = 0.014346573 accuracy = 0.995436368
    Epoch: 0010 cost = 0.012755402 accuracy = 0.996072731
    Epoch: 0011 cost = 0.010847591 accuracy = 0.996745458
    Epoch: 0012 cost = 0.009276978 accuracy = 0.996854548
    Epoch: 0013 cost = 0.009246885 accuracy = 0.996890912
    Epoch: 0014 cost = 0.007504274 accuracy = 0.997436366
    Epoch: 0015 cost = 0.008067215 accuracy = 0.997290912
    Accuracy: 0.9935

### 4. Restoring Subgraph

Here we first rebuild the DNN graph from metagraph, restore DNN parameters from the checkpoint and then gather the necessary nodes using the `tf.get_collection()` function.

```python
tf.reset_default_graph()

sess = tf.InteractiveSession()

new_saver = tf.train.import_meta_graph(ckptdir + '.meta')
new_saver.restore(sess, tf.train.latest_checkpoint(logdir))
```

    INFO:tensorflow:Restoring parameters from ./tf_logs/4_1_EQ/model

### 5. Attaching Subgraph for Calculating Relevance Scores

```python
conv_ksize = [1, 3, 3, 1]
pool_ksize = [1, 2, 2, 1]
conv_strides = [1, 1, 1, 1]
pool_strides = [1, 2, 2, 1]

activations = tf.get_collection('EC')
weights = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='CNN')

X = activations[0]
predictions = activations[-1]

activations.reverse()
weights.reverse()

taylor = Taylor(activations, weights, conv_ksize, pool_ksize, conv_strides, pool_strides, 'Taylor')

Rs = [taylor(i) for i in range(10)]
SA_scores = [tf.square(tf.gradients(predictions[:,i], X)) for i in range(10)]
STD_scores = [tf.gradients(predictions[:,i], X) * X for i in range(10)]
```

### 6. Calculating Relevance Scores $R(x_i)$ and Displaying Images

We can see that while Sensitivity Analysis and Simple Taylor Decomposition provides discontinuous explanations with frequent jumps, Deep Taylor Decomposition gives us a continuous explanation.

```python
digit = 2
dist = 41
init = -20

sample_imgs = [images[np.argmax(labels, axis=1) == i][3] for i in range(10)]
translated_imgs = [[translate(img, init + i, 0) for i in range(dist)] for img in sample_imgs]

preds = [[sess.run(predictions[:,i], feed_dict={X:translated_imgs[i][j]}) for j in range(dist)] for i in range(10)]

sa_imgs = [[sess.run(SA_scores[i], feed_dict={X:translated_imgs[i][j]}) for j in range(dist)] for i in range(10)]
sa_imgs = np.square(np.reshape(sa_imgs, [10, dist, -1]))

std_imgs = [[sess.run(STD_scores[i], feed_dict={X:translated_imgs[i][j]}) for j in range(dist)] for i in range(10)]
std_imgs = np.reshape(std_imgs, [10, dist, -1])

dtd_imgs = [[sess.run(Rs[i], feed_dict={X:translated_imgs[i][j]}) for j in range(dist)] for i in range(10)]
dtd_imgs = np.squeeze(dtd_imgs, axis=2)

sa_imgs_q0 = sa_imgs[digit]
sa_imgs_q1 = sa_imgs[digit].reshape(dist,28,28)[:,0:14,0:14].reshape(dist,-1)
sa_imgs_q2 = sa_imgs[digit].reshape(dist,28,28)[:,0:14,14:28].reshape(dist,-1)
sa_imgs_q3 = sa_imgs[digit].reshape(dist,28,28)[:,14:28,14:28].reshape(dist,-1)
sa_imgs_q4 = sa_imgs[digit].reshape(dist,28,28)[:,14:28,0:14].reshape(dist,-1)

std_imgs_q0 = std_imgs[digit]
std_imgs_q1 = std_imgs[digit].reshape(dist,28,28)[:,0:14,0:14].reshape(dist,-1)
std_imgs_q2 = std_imgs[digit].reshape(dist,28,28)[:,0:14,14:28].reshape(dist,-1)
std_imgs_q3 = std_imgs[digit].reshape(dist,28,28)[:,14:28,14:28].reshape(dist,-1)
std_imgs_q4 = std_imgs[digit].reshape(dist,28,28)[:,14:28,0:14].reshape(dist,-1)

dtd_imgs_q0 = dtd_imgs[digit]
dtd_imgs_q1 = dtd_imgs[digit].reshape(dist,28,28)[:,0:14,0:14].reshape(dist,-1)
dtd_imgs_q2 = dtd_imgs[digit].reshape(dist,28,28)[:,0:14,14:28].reshape(dist,-1)
dtd_imgs_q3 = dtd_imgs[digit].reshape(dist,28,28)[:,14:28,14:28].reshape(dist,-1)
dtd_imgs_q4 = dtd_imgs[digit].reshape(dist,28,28)[:,14:28,0:14].reshape(dist,-1)

fig = plt.figure(figsize=[15,15])
for i in range(6):
    if i is 0:
        ax = fig.add_subplot(1, 6, 1)
        ax.imshow(np.zeros([28,28]), cmap='gray_r', vmin=0, vmax=1)
        ax.tick_params(labelbottom='off', labelleft='off', bottom='off', left='off')
        ax.axvline(14, linestyle='--', color='black')
        ax.axhline(14, linestyle='--', color='black')
        ax.text(2, 9, '$R_1$', fontdict={'fontsize': 50})
        ax.text(16, 9, '$R_2$', fontdict={'fontsize': 50})
        ax.text(16, 23, '$R_3$', fontdict={'fontsize': 50})
        ax.text(2, 23, '$R_4$', fontdict={'fontsize': 50})
        continue

    ax = fig.add_subplot(1, 6, i + 1)
    ax.imshow(dtd_imgs_q0[3 * i + 12].reshape([28,28]), cmap='hot_r')
    ax.tick_params(labelbottom='off', labelleft='off', bottom='off', left='off')
plt.tight_layout()

fig = plt.figure(figsize=[20,5])
ax1 = fig.add_subplot(131)
x = list(range(init,init+dist))
y0 = preds[digit]
y1 = np.sum(sa_imgs_q1, axis=1) / 2
y2 = np.sum(sa_imgs_q2, axis=1) / 2
y3 = np.sum(sa_imgs_q3, axis=1) / 2
y4 = np.sum(sa_imgs_q4, axis=1) / 2

ax1.plot(x, y0, marker='.', label='$f(x)$')
ax1.plot(x, y1, marker='.', label='$\propto R_1(x)$')
ax1.plot(x, y2, marker='.', label='$\propto R_2(x)$')
ax1.plot(x, y3, marker='.', label='$\propto R_3(x)$')
ax1.plot(x, y4, marker='.', label='$\propto R_4(x)$')
ax1.legend(loc='upper left', fontsize='x-large')

ax1.set_title('Sensitivity Analysis for Image Translation', fontdict={'fontsize': 14})
ax1.set_xlabel('Digit Position', fontdict={'fontsize': 14})
ax1.set_ylabel('Relevance Scores', fontdict={'fontsize': 14})
ax1.tick_params(labelsize=12)

ax2 = fig.add_subplot(132)
y1 = np.sum(std_imgs_q1, axis=1)
y2 = np.sum(std_imgs_q2, axis=1)
y3 = np.sum(std_imgs_q3, axis=1)
y4 = np.sum(std_imgs_q4, axis=1)

ax2.plot(x, y0, marker='.', label='$f(x)$')
ax2.plot(x, y1, marker='.', label='$R_1(x)$')
ax2.plot(x, y2, marker='.', label='$R_2(x)$')
ax2.plot(x, y3, marker='.', label='$R_3(x)$')
ax2.plot(x, y4, marker='.', label='$R_4(x)$')
ax2.legend(loc='upper left', fontsize='x-large')

ax2.set_title('Simple Taylor Decomposition for Image Translation', fontdict={'fontsize': 14})
ax2.set_xlabel('Digit Position', fontdict={'fontsize': 14})
ax2.set_ylabel('Relevance Scores', fontdict={'fontsize': 14})
ax2.tick_params(labelsize=12)

ax3 = fig.add_subplot(133)
y1 = np.sum(dtd_imgs_q1, axis=1)
y2 = np.sum(dtd_imgs_q2, axis=1)
y3 = np.sum(dtd_imgs_q3, axis=1)
y4 = np.sum(dtd_imgs_q4, axis=1)

ax3.plot(x, y0, marker='.', label='$f(x)$')
ax3.plot(x, y1, marker='.', label='$R_1(x)$')
ax3.plot(x, y2, marker='.', label='$R_2(x)$')
ax3.plot(x, y3, marker='.', label='$R_3(x)$')
ax3.plot(x, y4, marker='.', label='$R_4(x)$')
ax3.legend(loc='upper left', fontsize='x-large')

ax3.set_title('Deep Taylor Decomposition for Image Translation', fontdict={'fontsize': 14})
ax3.set_xlabel('Digit Position', fontdict={'fontsize': 14})
ax3.set_ylabel('Relevance Scores', fontdict={'fontsize': 14})
ax3.tick_params(labelsize=12)
```

```python

```


# differentiation for optimization

### basic formular

- constant

$$ \dfrac{d}{dx}(c) = 0 $$

- exponentiation

$$ \dfrac{d}{dx}(x^n) = nx^{n-1} $$

- log

$$ \dfrac{d}{dx}(\log x) = \dfrac{1}{x} $$

- Euler

$$ \dfrac{d}{dx}(e^x) = e^x $$

### linear combination

$$ \dfrac{d}{dx}(c_1f_1 + c_2f_2) = c1\dfrac{df_1}{dx} + c2\dfrac{df_2}{dx} $$

### multiplication

$$ \dfrac{d}{dx}(f \cdot g) = \dfrac{df}{dx} \cdot g + f \cdot \dfrac{dg}{dx}$$

### chain rule

$$ f(x) = h(g(x)) $$

$$ -> \dfrac{df}{dx} = \dfrac{dh}{dg} \cdot \dfrac{dg}{dx} $$

- ex: log function

$$ \dfrac{d}{dx}\log f(x) = \dfrac{f'(x)}{f(x)} $$

---

## second derivative
- differentiate derivative

$$ f'' = \dfrac{d^2}{dx^2}f = \dfrac{d^2f}{dx^2} = \dfrac{d^2}{dx^2}y $$

- second derivative means the slope of derivative
    - when the slope(derivative) goes up, second derivative value is positive
    - and we call it **convex**
    - the other way, second derivative is negative, it is called **concave**

### partial differentiation
- when indipendant variables are more than one, differentiate one variable at once. which is partial differentiation (for convenient)

$$ f_x(x, y) = \dfrac{\partial f}{\partial x} $$
$$ f_y(x, y) = \dfrac{\partial f}{\partial y} $$

---

## Sympy
for symbolic operation

```python
import sympy
sympy.init_printing(use_latex='mathjax')
```

```python
x, mu, sigma = sympy.symbols('x mu sigma')
f = sympy.exp((x-mu)**2 / sigma **2)
f
```

$$e^{\frac{1}{\sigma^{2}} \left(- \mu + x\right)^{2}}$$

- in case partial, need to nominate the variable

```python
sympy.diff(f, x) #differentiation
```

$$\frac{1}{\sigma^{2}} \left(- 2 \mu + 2 x\right) e^{\frac{1}{\sigma^{2}} \left(- \mu + x\right)^{2}}$$

```python
sympy.simplify(sympy.diff(f, x))
```

$$\frac{2}{\sigma^{2}} \left(- \mu + x\right) e^{\frac{1}{\sigma^{2}} \left(\mu - x\right)^{2}}$$


# Gotchas

Boilerplate to make the doctester work.  Run this cell first.

```
import sys
import os
sys.path.insert(1, os.path.join(os.path.pardir, "ipython_doctester"))
from sympy import *
from ipython_doctester import test
# Work around a bug in IPython. This will disable the ability to paste things with >>>
def notransform(line): return line
from IPython.core import inputsplitter
inputsplitter.transform_classic_prompt = notransform
```

For each exercise, fill in the function according to its docstring. Execute the cell to see if you did it right.

## Symbols

What will be the output of the following code?

    x = 3
    y = symbols('y')
    a = x + y
    y = 5
    print a

Replace `???` in the below code with what you think the value of `a` will be.  Remember to define any Symbols you need!

```
@test
def symbols_exercise():
    """
    (This tests that your output is correct)

    >>> def testfunc():
    ...     x = 3
    ...     y = symbols('y')
    ...     a = x + y
    ...     y = 5
    ...     return a
    >>> symbols_exercise() == testfunc()
    True
    """
    return ??? # Replace ??? with what you think the value of a is
```

## Equality

Write a function that takes two expressions as input, and returns a tuple of two booleans. The first if they are equal symbolically, and the second if they are equal mathematically.

```
@test
def equality_exercise(a, b):
    """
    Determine if a = b symbolically and mathematically.

    Returns a tuple of two booleans. The first is True if a = b symbolically,
    the second is True if a = b mathematically.  Note the second may be False
    but the two still equal if SymPy is not powerful enough.

    Examples
    ========

    >>> x = symbols('x')
    >>> equality_exercise(x, 2)
    (False, False)
    >>> equality_exercise((x + 1)**2, x**2 + 2*x + 1)
    (False, True)
    >>> equality_exercise(2*x, 2*x)
    (True, True)
    """

```

## `^` and `/`

Correct the following functions

```
@test
def operator_exercise1():
    """
    >>> operator_exercise1()
    x**2 + 2*x + 1/2
    """
    x = symbols('x')
    return x^2 + 2*x + 1/2
```

```
@test
def operator_exercise2():
    """
    >>> operator_exercise2()
    (x**2/2 + 2*x + 3/4)**(3/2)
    """
    x = symbols('x')
    return (1/2*x^2 + 2*x + 3/4)^(3/2)
```


# Differential Drive Robotics Platform

## Problem Setup

A common robot platform is that of the [differential drive](http://en.wikipedia.org/wiki/Differential_wheeled_robot). The robot is propelled using two wheels, each with their own motor. Usually a caster is used as a third wheel to provide stability. By varying the speeds of the two motors, the robot can change direction.

Differential drive platforms are highly maneuverable, as they have a very small turn radius. They are also extremely simple to design. In this example, the dynamics of a differential drive platform will be explored. The resulting model is highly accurate, but makes a few assumptions:

**Assumptions**

- Wheels roll without slipping
- Body is symmetric

**The setup is as follows:**

```
from IPython.display import SVG
SVG(filename='Robot.svg')
```

## Equations of Motion

We’ll start by generating the equations of motion for the robot with `sympy.physics.mechanics`. The `mechanics` module provides a clean, intuitive workflow for deriving the equations of motion for multibody dynamic systems using either `KanesMethod` or `LagrangesMethod` objects. In this example we'll make use of the `LagrangesMethod` object, which generates the equations of motion using [Lagrangian Mechanics](http://en.wikipedia.org/wiki/Lagrangian_mechanics) with constraints and generalized forces.

First we import the necessary functionality from SymPy.

```
from sympy.physics.mechanics import *
from sympy import sin, cos, symbols, Matrix, solve
```

Next we need to define an inertial reference frame, as well as a coordinate origin. All coordinates and reference frames defined later will be in reference to the world coordinate frame.

```
# Inertial Reference Frame
N = ReferenceFrame('N')

# Define a world coordinate origin
O = Point('O')
O.set_vel(N, 0)
```

Lagrange's Method requires a state vector $\vec{q}$ that contains all the generalized coordinates required to define the system in space. In the case of the robot:

$$
\vec{q} = [\matrix{x && y && \theta && \phi_1 && \phi_2}]^T
$$

As these are all time varying variables, they need to be created as `dynamicsymbols`:

```
theta = dynamicsymbols('theta')                      # Rotation about N.z
x, y = dynamicsymbols('x, y')                        # Coordinates of robot in World Frame
phi1, phi2 = dynamicsymbols('phi_1, phi_2')          # Angular displacement of wheel

# Create q and dq vectors
q = Matrix([x, y, theta, phi1, phi2])
dq = q.diff()
```

We'll also define some constant parameters that define the specific robot:

```
# Constants for the wheels
r = symbols('r')                                     # Radius of wheel
m_w = symbols('m_w')                                 # Mass of the wheel
t_w = symbols('t_w')                                 # Thickness of the wheel

# Constants for the Robot Body
w = symbols('w')                                     # 2*w is the width of the wheel base
d = symbols('d')                                     # Distance between axel and center of mass
m_b = symbols('m_b')                                 # Mass of the body
Ixx, Iyy, Izz = symbols('Ixx, Iyy, Izz')             # Moments of inertia of body
```

Next we need to define a reference frame relative to the robot, and a point located at the center of the wheel axis. This will define the robot's location in the world coordinate frame. The velocity of this point is just the derivative of its x and y coordinates.

```
# Robot Reference Frame
R = N.orientnew('R', 'Axis', [theta, N.z])

# Center of wheel base
Cw = O.locatenew('Cw', x*N.x + y*N.y)

# Set the velocity of point Cw
Cw.set_vel(N, x.diff()*N.x + y.diff()*N.y)
```

Coordinates of the 2 wheel hubs then need to be specified. To do this we can use the `locatenew` method. The hubs 1 and 2 are located at $-w\vec{R_y}$ and $w\vec{R_y}$ away from the center of the wheel base $C_w$ respectively.

The velocity of the hubs is then found using the `v2pt_theory` method, which calculates the velocity of a point based off the velocity of another point in the same frame, and the angular velocity of that frame ([reference](http://en.wikipedia.org/wiki/Rotating_reference_frame#Relation_between_velocities_in_the_two_frames)).

```
# Points at wheel hubs
H1 = Cw.locatenew('H1', -w*R.y)
H2 = Cw.locatenew('H2', w*R.y)

# Set the velocity of points H1 and H2
H1.v2pt_theory(Cw, N, R)
H2.v2pt_theory(Cw, N, R);
```

Rotating reference frames fixed to each wheel are then created. These rotate about the $\vec{R_y}$ axis with the angular position of the wheels, $\phi_1$ and $\phi_2$.

```
# Create reference frames for wheels 1 and 2
W1 = R.orientnew('W1', 'Axis', [phi1, R.y])
W2 = R.orientnew('W2', 'Axis', [phi2, R.y])
```

`sympy.physics.mechanics` provides classes for rigid bodies. These make calculating the lagrangian of a multibody system easier, as each body can be handled separately. To do this, each body needs the following things specified:

- A name
- A center of mass
- A reference frame
- A mass
- A tuple of (Inertia, Point of rotation)

Modeling the wheels as solid cylinders, the inertia can be calculated about the hubs, and `RigidBody` objects can be created for each wheel.

```
# Calculate inertia of the wheel
Iw = inertia(R, m_w*(3*r**2 + t_w**2)/12, m_w*r**2/2, m_w*(3*r**2 + t_w**2)/12)

# Create rigid bodies for wheels
Wheel1 = RigidBody('Wheel1', H1, W1, m_w, (Iw, H1))
Wheel2 = RigidBody('Wheel2', H2, W2, m_w, (Iw, H2))
```

The same can be done for the body of the robot. In this case we model the body as a symmetric object with inertias $I_{xx}$, $I_{yy}$, and $I_{zz}$. The center of mass of the body is located a distance $d$ in front of the wheel axel.

```
# Calculate inertia of body
Ib = inertia(R, Ixx, Iyy, Izz)

# Center of mass of body
Cm = Cw.locatenew('Cm', d*R.x)
Cm.v2pt_theory(Cw, N, R)

# Create a rigid body object for body
Body = RigidBody('Body', Cm, R, m_b, (Ib, Cm))
```

## Nonholonomic Constraints
At this point, the Lagrangian of the system can be calculated. However, the system still isn't fully defined; the constraints still need to be specificed.

At any instance in time, the velocity of the ground contact points must have a velocity of zero. To specify this in the Lagrange equations, a constraint matrix needs to be created.

First, two points are created where the wheels contact the ground, and their velocities calculated.

```
# Create two points, where the wheels contact the ground
C1 = H1.locatenew('C1', -r*R.z)
C2 = H2.locatenew('C2', -r*R.z)
# Calculate velocity of points
C1.v2pt_theory(H1, N, W1)
C2.v2pt_theory(H2, N, W2);
```

Using these velocities, a system of equations can be created. As the velocities must be zero, this expression = 0.

```
# Express the velocity of points in the inertial frame
con1 = C1.vel(N).express(N).args[0][0]
con2 = C2.vel(N).express(N).args[0][0]
# Create a matrix of constraints
constraints = con1.col_join(con2)
mprint(constraints)
```

    Matrix([
    [(-r*phi_1' + w*theta')*cos(theta) + x'],
    [(-r*phi_1' + w*theta')*sin(theta) + y'],
    [                                     0],
    [(-r*phi_2' - w*theta')*cos(theta) + x'],
    [(-r*phi_2' - w*theta')*sin(theta) + y'],
    [                                     0]])

Since there are 6 equations with 3 unknowns ($\dot{x}$, $\dot{y}$, and $\dot{\theta}$), there are duplicate equations. Using solve, and reconfiguring the expression, this can be reduced to the required set:

```
# Solve for dx, dy, and dtheta in terms of dphi1 and dphi2
sol = solve(constraints, dq[:3])

# Split the resulting dict into a rhs and lhs, that are equivalent
sol_rhs = Matrix(sol.values())
sol_lhs = Matrix(sol.keys())

# Since sol_rhs = sol_lhs --> sol_rhs - sol_lhs = 0
# This forms the basis of our constraint matrix.
# Combining, and solving for a linear representation:
c = (sol_rhs - sol_lhs).jacobian(dq[:5])
mprint(c)
```

    Matrix([
    [-1,  0,  0, r*cos(theta)/2, r*cos(theta)/2],
    [ 0,  0, -1,        r/(2*w),       -r/(2*w)],
    [ 0, -1,  0, r*sin(theta)/2, r*sin(theta)/2]])

This is our constraint matrix. The `LagrangesMethod` class requires that the constraints be specified as

`coneqs` = $ C\dot{\vec{q}} = 0 $

Thus:

```
# Constraint Equations
coneqs = (c*dq)
mprint(coneqs)
```

    Matrix([
    [r*cos(theta)*phi_1'/2 + r*cos(theta)*phi_2'/2 - x'],
    [          r*phi_1'/(2*w) - r*phi_2'/(2*w) - theta'],
    [r*sin(theta)*phi_1'/2 + r*sin(theta)*phi_2'/2 - y']])

## Generalized Forces

The inputs of system are the torques at the wheels. Using the right-hand-rule, it can be seen that a positive torque will exert force on the system in the $R_x$ direction. With a wheel radius of $r$, the force on the wheel hubs is:

$\vec{F_{hub}} = \tau*\vec{R_x}$

```
# Define forces on system:
T1, T2 = symbols('tau_1, tau_2')              # Torques from the wheels
fl = [(H1, r*T1*R.x),
      (H2, r*T2*R.x)]
```

## Solving the System Dynamics

We are now ready to solve the for the equations of motion. First, calculate the Lagrangian of the system $L = T - V$:

```
Lag = Lagrangian(N, Wheel1, Wheel2, Body)
```

Next, a `LagrangesMethod` object is created. This takes the following parameters:

- The system Lagrangian
- The generalized coordinate vector $\vec{q}$
- The constraint equations
- A list force tuples: (Point, Force at point)
- An inertial reference frame

```
lm = LagrangesMethod(Lag, q, nonhol_coneqs=coneqs, forcelist=fl, frame=N)
```

The equations of motion can then be found:

```
le = lm.form_lagranges_equations()
mprint(le)
```

    Matrix([
    [                                                                                m_b*(-2*d*sin(theta)*theta'' - 2*d*cos(theta)*theta'**2 + 2*x'')/2 + m_w*(-2*w*sin(theta)*theta'**2 + 2*w*cos(theta)*theta'' + 2*x'')/2 + m_w*(2*w*sin(theta)*theta'**2 - 2*w*cos(theta)*theta'' + 2*x'')/2 - r*tau_1*cos(theta) - r*tau_2*cos(theta) - lam1],
    [                                                                                m_b*(-2*d*sin(theta)*theta'**2 + 2*d*cos(theta)*theta'' + 2*y'')/2 + m_w*(-2*w*sin(theta)*theta'' - 2*w*cos(theta)*theta'**2 + 2*y'')/2 + m_w*(2*w*sin(theta)*theta'' + 2*w*cos(theta)*theta'**2 + 2*y'')/2 - r*tau_1*sin(theta) - r*tau_2*sin(theta) - lam3],
    [Izz*theta'' - m_b*(d*(-sin(theta)*y' - cos(theta)*x')*theta' - d*sin(theta)*theta'*y' - d*cos(theta)*theta'*x')/2 + m_b*(2*d**2*theta'' + d*(-sin(theta)*theta'*y' - sin(theta)*x'' - cos(theta)*theta'*x' + cos(theta)*y'') - d*sin(theta)*theta'*y' - d*sin(theta)*x'' - d*cos(theta)*theta'*x' + d*cos(theta)*y'')/2 + m_w*(3*r**2 + t_w**2)*theta''/6 - m_w*(-w*(-sin(theta)*x' + cos(theta)*y')*theta' + w*sin(theta)*theta'*x' - w*cos(theta)*theta'*y')/2 - m_w*(w*(-sin(theta)*x' + cos(theta)*y')*theta' - w*sin(theta)*theta'*x' + w*cos(theta)*theta'*y')/2 + m_w*(2*w**2*theta'' - w*(-sin(theta)*theta'*x' + sin(theta)*y'' + cos(theta)*theta'*y' + cos(theta)*x'') + w*sin(theta)*theta'*x' - w*sin(theta)*y'' - w*cos(theta)*theta'*y' - w*cos(theta)*x'')/2 + m_w*(2*w**2*theta'' + w*(-sin(theta)*theta'*x' + sin(theta)*y'' + cos(theta)*theta'*y' + cos(theta)*x'') - w*sin(theta)*theta'*x' + w*sin(theta)*y'' + w*cos(theta)*theta'*y' + w*cos(theta)*x'')/2 - r*tau_1*w + r*tau_2*w - lam2],
    [                                                                                m_w*r**2*phi_1''/2 + r*lam1*cos(theta)/2 + r*lam3*sin(theta)/2 + r*lam2/(2*w)],
    [                                                                                m_w*r**2*phi_2''/2 + r*lam1*cos(theta)/2 + r*lam3*sin(theta)/2 - r*lam2/(2*w)]])

## Simulation

To gain insight into the system dynamics, numerical simulation will be used. First, we need to import a few more functions:

```
from sympy import lambdify, Dummy
from sympy.physics.mechanics import msubs
from scipy import array, hstack, linspace, ones
from scipy import random, interp, vstack
from scipy.integrate import odeint
import matplotlib.pyplot as plt
%matplotlib inline
```

The vector $\ddot{\vec{q}}$ can be calculated in a closed form expression using the `rhs` method. This method takes quite a long time to solve entirely symbolically (it took over an hour on my machine), so a simplifying assumption will be made: $d = 0$. This is actually pretty reasonable, as the motors and battery will be the heaviest part, and should be situated around the axle.

```
# Solve for the rhs:
rhs = lm.rhs()

# Substitute in d = 0
rhs = msubs(rhs, {d: 0})
```

We're almost done, all we need to do is model the dynamics of the motor. In general, we can model the current dynamics of the motor as follows:

$$
\dot{i} = -K/L \dot{\phi} - R/L i + V/L
$$

$$
\tau = K i
$$

where $i$ and $V$ are the current and voltage through the motor, $K$ is the motor constant, $R$ is the coil resistance, and $L$ is the coil inductance. Creating this model in sympy is fairly straightforward:

```
# Create dynamic symbols for current and voltage
i_1, i_2 = dynamicsymbols('i_1, i_2')        # Currents through motor 1 and 2
V_1, V_2 = symbols('V_1, V_2')               # Voltages across the motor terminals

# Define some motor constants.
# Assuming motor 1 and 2 are the same:
R = symbols('R')                             # Coil resistance
L = symbols('L')                             # Coil inductance
K1, K2 = symbols('K1, K2')                   # Motor constant

# Define the motor dynamics
di = Matrix([[-K1/L*phi1.diff() - R/L*i_1 + V_1/L],
             [-K2/L*phi2.diff() - R/L*i_2 + V_2/L]])
```

Now we just need to combine the motor model with the model for the robot:

```
# Define consts:
params = [Izz,  t_w,  m_w,    r,  m_b,   w,    R,      L,   K1,  K2]
values = [  5, 0.15,  2.0, 0.15, 50.0, 0.6, 0.05, 0.0001,  1.0, 1.0]       # Values of constants

# Create a list of dynamic symbols for simulation
dynamics = q.T.tolist()[0] + dq.T.tolist()[0] + lm.lam_vec.T.tolist()[0] + [i_1, i_2]

# Set the inputs to be the motor voltages
inputs = [V_1, V_2]

# Add the motor model to the rhs equations
aug_rhs = rhs.col_join(di)

# Substitute in K*i for T in the rhs equations
aug_rhs = aug_rhs.subs({T1: K1*i_1, T2: K2*i_2})
```

Currently the codegen functionality provided by `PyDy` doesn't support equations with Lagrange Multipliers (this is planned for future releases). For now `lambdify` can be used to convert the expression into a function to solve the rhs based on the current state and inputs.

```
# Create a list of dynamic symbols for simulation
dummys = [Dummy() for i in dynamics]
dummydict = dict(zip(dynamics, dummys))
# Sub in the dummy symbols
rhs_dummy = msubs(aug_rhs, dummydict)
# Lambdify function
rhs_func = lambdify(dummys + inputs + params, rhs_dummy, modules='numpy')
```

```
# Create a function in the required format for odeint
def right_hand_side(x, t, ts, us, values):
    """Calculate the rhs of the integral, at
    time t, state x.
    ts, us are used to get the current input
    values are constants in the integral"""

    # Interp is needed to get u for timestep
    u1 = interp(t, ts, us[:,0])
    u2 = interp(t, ts, us[:,1])

    arguments = hstack((x, u1, u2, values))

    # Need call to array and reshape, as odeint
    # requires state vector to be 1d
    dx = array(rhs_func(*arguments))
    return dx.reshape(dx.size)
```

Now we have everything we need to simulate the motion. First, let's see what happens if we input a constant, unbalanced voltage signal. Motor 1 gets 24 volts, Motor 2 gets 20 volts. Intuitively, this should cause the robot to drive in a circle.

```
ts = linspace(0, 30, 3000)
us = vstack((24*ones(len(ts)), 20*ones(len(ts)))).T
# Run the simulation
x0 = [0,] * 15                         # Start out at origin, unmoving
xs = odeint(right_hand_side, x0, ts, args=(ts, us, values))

plt.plot(xs[:,5], xs[:,6])
plt.title('Robot Position')
plt.xlabel('x (m)')
plt.ylabel('y (m)');
```

As seen by the simulation, the results line up with our intuition. The reason it doesn't immediately start out in a circle is due to the time it takes for it to accelerate from rest.

If we were designing such a platform, we might want to see the effects different design parameters have on the motion. Because the solution is symbolic, it's easy to experiment. Here we'll see what having a heavier body and larger inertia do to the motion:

```
ts = linspace(0, 30, 1000)
us = vstack((24*ones(len(ts)), 20*ones(len(ts)))).T
# Body is twice as heavy, with twice as much rotational inertia
values = [  10, 0.15,  2.0, 0.15, 100.0, 0.6, 0.05, 0.0001,  1.0, 1.0]       # Values of constants
# Run the simulation
x0 = [0,] * 15                         # Start out at origin, unmoving
xs = odeint(right_hand_side, x0, ts, args=(ts, us, values))
plt.plot(xs[:,0], xs[:,1])
plt.title('Robot Position')
plt.xlabel('x (m)')
plt.ylabel('y (m)');
```


# Mínimos Cuadrados
## Ecuaciones normales

```python
import numpy as np
import scipy.linalg as spla
import matplotlib.pyplot as plt
np.random.seed(123)
```

### Modelos
Definición de modelos
* Lineal: $f(t)=c_1 + c_2\, t$
* Cuadrático: $f(t)=c_1 + c_2\, t + c_3\, t^2$
* Exponencial: $f(t)=c_1\,e^{c_2\,t}$
* Periódico: $f(t)=c_1 + c_2\,\cos(2\pi t) + c_3\,\sin(2\pi t) + c_4\, \cos(4\pi t)$*
* Ley de potencia: $f(t)=c_1t^{c_2}$

```python
lm = lambda c_1, c_2, t: c_1 + c_2 * t
qm = lambda c_1, c_2, c_3, t: c_1 + c_2 * t + c_3 * t ** 2
em = lambda c_1, c_2, t: c_1 * np.exp(c_2 * t)
sm = lambda c_1, c_2, c_3, c_4, t: c_1 + c_2 * np.cos(2 * np.pi * t) + c_3 * np.sin(2 * np.pi * t) + c_4 * np.cos(4 * np.pi * t)
pm = lambda c_1, c_2, t: c_1 * t ** c_2
```

### Resolución de ecuaciones normales

\begin{equation}
    A^*A\,\mathbf{x} = A^*\mathbf{b}
\end{equation}

```python
# A^*A x = A^* b
def normalEquations(A, b):
    #return np.dot(np.linalg.inv(np.dot(A.T, A)), np.dot(A.T, b))
    return np.linalg.solve(np.dot(A.T, A), np.dot(A.T, b))
```

```python
def plot(t, y, tt, yy):
    plt.figure(figsize=(12, 6))
    plt.plot(t, y, 'r.')
    plt.plot(tt, yy)
    plt.grid(True)
    plt.xlabel(r'$t$')
    plt.ylabel(r'$y$')
    plt.show()
```

Se define un $t\in[0.1, \pi]$

```python
n = 200
t = np.linspace(0.1, np.pi, n)
tt = np.linspace(0.1, np.pi, 3 * n)
```

Para probar los modelos, se generan datos "cocinados" utilizando de base el modelo + "ruido".

## Lineal

```python
y_l = lm(1, 2, t) + np.random.normal(1.5, .5, n) # Data
A_l = np.ones((n, 2))
A_l[:,1] = t
b_l = y_l
```

```python
p_l = normalEquations(A_l, b_l)
```

```python
print("Parámetros: c1=%f, c2=%f"%(p_l[0], p_l[1]))
```

    Parámetros: c1=2.543600, c2=1.974268

```python
plot(t, y_l, tt, lm(*p_l, tt))
```

## Cuadrático

```python
y_q = qm(1, 2, 3, t) + np.random.normal(5, 3, n)
A_q = np.ones((n, 3))
A_q[:, 1] = t
A_q[:, 2] = t ** 2
b_q = y_q
```

```python
p_q = normalEquations(A_q, b_q)
```

```python
print("Parámetros: c1=%f, c2=%f, c3=%f"%(p_q[0], p_q[1], p_q[2]))
```

    Parámetros: c1=6.018403, c2=1.682171, c3=3.045116

```python
plot(t, y_q, tt, qm(*p_q, tt))
```

## Periódico

```python
y_s = sm(1, 2, 2, 1, t) + np.random.normal(10, 2, n)
A_s = np.ones((n, 4))
A_s[:, 1] = np.cos(2 * np.pi * t)
A_s[:, 2] = np.sin(2 * np.pi * t)
A_s[:, 3] = np.cos(4 * np.pi * t)
b_s = y_s
```

```python
p_s = normalEquations(A_s, b_s)
```

```python
print("Parámetros: c1=%f, c2=%f, c3=%f, c4=%f"%(p_s[0], p_s[1], p_s[2], p_s[3]))
```

    Parámetros: c1=11.137206, c2=2.134255, c3=1.776716, c4=0.983177

```python
plot(t, y_s, tt, sm(*p_s, tt))
```

## Exponencial

```python
y_e = 10 + em(1, 1.5, t) + np.random.normal(5, 5, n)
A_e = np.ones((n, 2))
A_e[:, 1] = t
b_e = np.log(y_e)
```

```python
p_e = normalEquations(A_e, b_e)
```

```python
print("Parámetros: c1=%f, c2=%f"%(p_e[0], p_e[1]))
```

    Parámetros: c1=2.358559, c2=0.672321

```python
plot(t, y_e, tt,  em(np.exp(p_e[0]), p_e[1], tt))
```

### En escala logarítmica

```python
plot(t, b_e, tt, lm(*p_e, tt))
```

## Ley de potencia

```python
y_p = pm(3, 4, t) * np.random.normal(5, 1, n)
A_p = np.ones((n, 2))
A_p[:, 1] = np.log(t)
b_p = np.log(y_p)
```

```python
p_p = normalEquations(A_p, b_p)
```

```python
print("Parámetros: c1=%f, c2=%f"%(p_p[0], p_p[1]))
```

    Parámetros: c1=2.665977, c2=3.992915

```python
plot(t, y_p, tt, pm(np.exp(p_p[0]), p_p[1], tt))
```

### En escala logarítmica

```python
plot(np.log(t), np.log(y_p), np.log(tt), lm(*p_p, np.log(tt)))
```

# Comentarios

* Basta un método de resolución para las ecuaciones normales, la diferencia aparece en la construcción de $A$, $\mathbf{b}$ y $\mathbf{x}$ con la cantidad de parámetros de los modelos.


```python
from sympy import *
init_printing()
```

```python
x, y = symbols('x y', cls=Function)
R = ((x(t).diff(t)**2 + y(t).diff(t)**2)**(3/2) \
     / (x(t).diff(t)*y(t).diff(t, 2) - y(t).diff(t)*x(t).diff(t,2)))
```

```python
a, b, t, t0 = symbols('a b t t0', real=True)
x0 = symbols('x0', real=True)
X = a*sqrt(pi)*fresnelc((t-t0)/sqrt(pi))
Y = a*sqrt(pi)*fresnels((t-t0)/sqrt(pi))

#plotting.plot_parametric(x.subs(d), y.subs(d), (t, 0, 1))

#K = (t-t0)/a
K = 1 / R.subs({x(t): X, y(t): Y}).doit().simplify()
display(K)
L = sqrt(a**2)*(t)
```

```python
L = integrate(sqrt(X.diff(t)**2 + Y.diff(t)**2), (t, 0, b)).simplify()
display(L)
```

```python
k0, k1, l = symbols('k0 k1 l', real=True)

sn = solve([
    Eq(K.subs(t, 0), k0),
    Eq(K.subs(t, b), k1),
    Eq(L, l),
], (a, t0), dict=True)
display(sn)
```

```python
sn[0][b].subs(d)
```

```python
th = symbols('theta')
d = {
    k0: 0.0,
    k1: 5.9114369622925669e-02,
    l: 5.7125709578755304e+00,

    'x': 34.008391843292344e+00,
    'y': -4.3421826556979193e+00,
    th: 7.9998293398872988e-03,

    'x2': 39.707082592335631e+00,
    'y2': -4.6174670591660050e+00,
    'hdg2': -1.6084768620939371e-01,
}

xy = Matrix([
    [(X - X.subs(t, 0)).simplify()],
    [(Y - Y.subs(t, 0)).simplify()]
])

rot = Matrix([[cos(th), -sin(th)], [sin(th), cos(th)]])
curv = (rot * xy)

plotting.plot_parametric(
    curv[0].subs(sn[0]).subs(d),
    curv[1].subs(sn[0]).subs(d),
    (t, 0, sn[0][b].subs(d))
)
```

```python
xy.subs(sn[0]).subs(d)
```


Here we import everything from `sympy`, load the ability to pretty-print all sympy expressions
and also the ability to display external images and LaTeX expressions.

```
from sympy import *
%load_ext sympyprinting
from IPython.core.display import Image, Math
```

# Rayleigh approximations

```
Image(filename='figures/model.png')
```

We need the relative rotation across each one of the 5 hinges in the top part of the figure, so we introduce two fictitious bar so that we can compute the relative rotation as the difference of the rotations of the adjacent bars for all the hinges.  Having introduced the two fictitious bar, to compute their rotations we have to take into account four additional degrees of freedom, as you can see in the bottom part of the figure.

We start defining the symbols that we will use

```
xm, x0, x1, x2, x3, x4, x5 = symbols('x_-1 x_0 x_1 x_2 x_3 x_4 x_5')
K, k, L , m, W, w, Zo, phi, theta = symbols('K k L m Omega omega Z_0 phi theta')
```

We define a vector of displacements

```
disp = (xm, x0, x1, x2, x3, x4, x5)
display(disp)
```

$$\begin{pmatrix}x_{-1}, & x_{0}, & x_{1}, & x_{2}, & x_{3}, & x_{4}, & x_{5}\end{pmatrix}$$

We define a function that takes the difference between two adjacent menbers of a sequence and optionally scales the difference by a factor, note that if the original sequence lenght is $N$, the sequence of differences that's returned by our function has lenght $N-1$.

```
def differences(seq, factor=1):
    return [(b-a)*factor for a, b in zip(seq[:-1],seq[1:])]
```

Now we use our `differences` to compute first the rotation of each of the six bars (the scaling factor being $\displaystyle\frac1L$) and later the relative rotations across each of the five hinges

```
rot = differences(disp, factor=1/L)
print "The rotations of the rods"
display(Math(r',\quad'.join([r'\phi_{%d} = %s'%(i+0,latex(r.simplify())) for i, r in enumerate(rot[:3])])+','))
display(Math(r',\quad'.join([r'\phi_{%d} = %s'%(i+3,latex(r.simplify())) for i, r in enumerate(rot[3:])])+'.'))

rrot = differences(rot)
print "\nThe relative rotations across the hinges:"
display(Math(r',\quad'.join([r'\theta_{%d} = %s'%(i+0,latex(r.simplify())) for i, r in enumerate(rrot[:3])])+','))
display(Math(r',\quad'.join([r'\theta_{%d} = %s'%(i+3,latex(r.simplify())) for i, r in enumerate(rrot[3:])])+'.'))
#dummy = [display(Math(r"\theta_%d = {%s}"%(i,latex(r.simplify())))) for i, r in enumerate(rrot)]
```

    The rotations of the rods

$$\phi_{0} = \frac{- x_{-1} + x_{0}}{L},\quad\phi_{1} = \frac{- x_{0} + x_{1}}{L},\quad\phi_{2} = \frac{- x_{1} + x_{2}}{L},$$

$$\phi_{3} = \frac{- x_{2} + x_{3}}{L},\quad\phi_{4} = \frac{- x_{3} + x_{4}}{L},\quad\phi_{5} = \frac{- x_{4} + x_{5}}{L}.$$

    The relative rotations across the hinges:

$$\theta_{0} = \frac{x_{-1} - 2 x_{0} + x_{1}}{L},\quad\theta_{1} = \frac{x_{0} - 2 x_{1} + x_{2}}{L},\quad\theta_{2} = \frac{x_{1} - 2 x_{2} + x_{3}}{L},$$

$$\theta_{3} = \frac{x_{2} - 2 x_{3} + x_{4}}{L},\quad\theta_{4} = \frac{x_{3} - 2 x_{4} + x_{5}}{L}.$$

Now, for each element of `rrot`, we substitute `0` for each of the fictitious displacements, to obtain the relative rotations across the springs in terms of the 3 free coordinates of our problem.

```
rrot = [rr.subs({xm:0,x0:0,x4:0,x5:0}) for rr in rrot]
print "The relative rotations across the hinges after substitution of 0 for fictitious degrees of freedom:"
display(Math(r',\quad'.join([r'\theta_{%d} = %s'%(i,latex(r.simplify())) for i, r in enumerate(rrot)])+'.'))

```

    The relative rotations across the hinges after substitution of 0 for fictitious degrees of freedom:

$$\theta_{0} = \frac{x_{1}}{L},\quad\theta_{1} = \frac{- 2 x_{1} + x_{2}}{L},\quad\theta_{2} = \frac{x_{1} - 2 x_{2} + x_{3}}{L},\quad\theta_{3} = \frac{x_{2} - 2 x_{3}}{L},\quad\theta_{4} = \frac{x_{3}}{L}.$$

Next, we can compute the (double of the) strain energy in terms of the free coordinates (note that I substituted `k` for `K/L**2`, just to have the expression for $\omega^2$ look more familiar...)

```
V2 = sum([stif*rr**2 for stif, rr in zip((2*K, K, K, K, 2*K), rrot)]).expand()
V2 = V2.subs(K,k*L*L)
display(V2)
```

Here we define the stiffness and the mass matrix

```
k_mat=k*Matrix(((7, -4, 1), (-4, 6, -4), (1, -4, 7)))
m_mat = m*Matrix(((1,0,0),(0,1,0),(0,0,1)))
display(k_mat) ; display(m_mat)
```

$$\left(\begin{smallmatrix}7 k & - 4 k & k\\- 4 k & 6 k & - 4 k\\k & - 4 k & 7 k\end{smallmatrix}\right)$$

$$\left(\begin{smallmatrix}m & 0 & 0\\0 & m & 0\\0 & 0 & m\end{smallmatrix}\right)$$

In the following we will use many times the duoble matrix multiplication, so we define a shorthand function.

```
def dmp(M,v):
    return (v.transpose()*M*v)[0]
```

Our first use of `dmp` is for verifying the correctness of the stiffness matrix

```
V2test = dmp(k_mat,Matrix((x1,x2,x3))).expand()
display(V2test)
```

Having verified the correctness of the stiffness matrix, we define a vector of maximum displacements and a vector of maximum velocities in terms of the unknown $\omega$.

```
d_max = Matrix((1,1,1))*Zo
v_max = w*d_max
display(d_max)
display(v_max)
```

$$\left(\begin{smallmatrix}Z_{0}\\Z_{0}\\Z_{0}\end{smallmatrix}\right)$$

$$\left(\begin{smallmatrix}Z_{0} \omega\\Z_{0} \omega\\Z_{0} \omega\end{smallmatrix}\right)$$

The (double of the) strain and kinetic energies are now computed, using `dmp`.

```
V2 = dmp(k_mat,d_max)
T2 = dmp(m_mat,v_max)
display(Eq(V2, T2))
```

To solve correctly the equation above we have to make the substitution $\omega^2 = \Omega$, so that
the solver is not confused by the unknown being squared

```
R00 = solve(Eq(V2,T2.subs(w,sqrt(W))),W)[0]
display(Eq(w*w, R00))
```

We compute the maximum values of the forces of inertia multiplying by `W`${}=\Omega=\omega^2$, then a better approximation of the strain energy , that equated to the previuous value of the kinetic energy gives the new approximation to $\omega^2$.

```
f_in = -W*m_mat*d_max
V2 = dmp(k_mat.inv(),f_in)
R01 = solve(Eq(V2,T2.subs(w,sqrt(W))),W)[0]
display(Eq(w*w, R01))
```

Computing a better approximation of the kinetic energy gives the third approximation,

```
v_max = -w*k_mat.inv()*f_in
T2 = dmp(m_mat,v_max)
R11 = solve(Eq(V2,T2.subs(w,sqrt(W))),W)[0]
display(Eq(w*w, R11))
```

Finally, the numerical values

```
display([R.evalf() for R in (R00,R01,R11)])
```

$$\begin{bmatrix}2.0 \frac{k}{m}, & 1.33333333333333 \frac{k}{m}, & 1.26315789473684 \frac{k}{m}\end{bmatrix}$$