# Perfect Foresight CRRA Model - Saving Rate

[](https://econ-ark.org/materials/perfforesightcrra-savingrate#launch)

This notebook demonstrates how to obtain the saving rate of a forward-looking consumer using HARK.

```python
# Preliminaries

%matplotlib inline
import matplotlib.pyplot as plt

import numpy as np
import HARK
from copy import deepcopy
mystr = lambda number : "{:.4f}".format(number)
from HARK.utilities import plot_funcs

# These last two will make our charts look nice
plt.style.use('seaborn-darkgrid')
palette = plt.get_cmap('Dark2')
```

## 1. Creating an agent

We now import the HARK class that represents a perfect-foresight consumer with CRRA utility, and create an instance of it with the parameters we'd like.

```python
# Set up a HARK Perfect Foresight Consumer called PFsavrate
from HARK.ConsumptionSaving.ConsIndShockModel import PerfForesightConsumerType # Import the consumer type

# Now we need to "fill" our consumer with parameters that allow us to solve the consumer's problem

# First we need to set out a dictionary
ρ = CRRA = 2.            # Coefficient of relative risk aversion
Rfree = 1.03             # Interest factor on assets
β = DiscFac = 0.97       # Intertemporal discount factor
LivPrb = [1.0]           # Survival probability
Γ = PermGroFac = [1.01]  # Permanent income growth factor
AgentCount = 1           # Number of agents of this type (only matters for simulation# Number of periods in the cycle for this agent type
cycles = 0               # Agent is infinitely lived
T_cycle = 1              # Every period is the same

# Make a dictionary to specify a perfect foresight consumer type
dict_wealth = { 'CRRA': CRRA,
                'Rfree': Rfree,
                'DiscFac': DiscFac,
                'LivPrb': LivPrb,
                'PermGroFac': PermGroFac,
                'AgentCount': AgentCount,
                'cycles' : cycles,
                'T_cycle' : T_cycle,
                }

# Now lets pass our dictionary to our consumer class
PFsavrate = PerfForesightConsumerType(**dict_wealth)
```

## 2. Solving the consumer's problem and finding the saving rate.

The next step is to ask HARK to solve the agent's optimal consumption problem. The result, or what we call "solution", is a function `cFunc` that tells us what is the optimal amount to consume at each level of normalized market resources.

The next cell solves the agent and tests the function

```python
PFsavrate.solve()
test_m = 4
test_c = PFsavrate.solution[0].cFunc(test_m)

print("When normalized market resources are m={}, the agent's normalized consumption should be c={}".format(test_m,test_c))
```

    When normalized market resources are m=4, the agent's normalized consumption should be c=1.611193200855211

We are now ready to think about the saving rate of the consumer.

Remember that the saving rate of an agent at time $t$ is

\begin{equation}
\text{Saving Rate}_t = \frac{\text{Capital Income}_t + \text{Labor Income}_t - \text{Consumption_t}}{\text{Capital Income}_t + \text{Labor Income}_t}
\end{equation}

Dividing both the numerator and denominator by the agent's permanent income $\textbf{P}_t$ and adopting the notation in [the course's lecture notes](https://www.econ2.jhu.edu/people/ccarroll/public/LectureNotes/Consumption/PerfForesightCRRA/) we have

\begin{equation}
\begin{split}
\text{Saving Rate}_t &= \frac{a_{t-1}*r/\Gamma + 1 - c(m_t)}{a_{t-1}*r/\Gamma + 1}\\
&=\frac{a_{t-1}*r/\Gamma + 1 - c(a_{t-1}*R/\Gamma + 1)}{a_{t-1}*r/\Gamma + 1}
\end{split}
\end{equation}

We now have an expression for the saving rate as a function of the previous period's end-of-period normalized assets $a_t$. The cell below calculates and plots this relationship for different levels of $a_t$.

```python
# Create a grid for a
a_tm1 = np.linspace(-1,4)

# Compute income components at every a
cap_income_t = a_tm1*(Rfree-1)/Γ
lab_income_t = 1

# and market resources
m_t = a_tm1 * Rfree/Γ + 1

# Consumption
c_t = PFsavrate.solution[0].cFunc(m_t)

# and finally the saving rate
ς_t = sav_rate_t = (cap_income_t + lab_income_t - c_t) / (cap_income_t + lab_income_t)

# And now the plot
plt.plot(a_tm1, sav_rate_t)
plt.xlabel(r'$a_{t-1}$')
plt.ylabel('Saving Rate')
```


# Price of Anarchy - Barcelona

## 1. Imports and network readin

First, we import the *paminco* and some other packages and download the traffic data for Barcelona from [GitHub](https://github.com/bstabler/TransportationNetworks/tree/master/Barcelona):

```python
import os
import requests

import paminco

URL = "https://raw.githubusercontent.com/bstabler/TransportationNetworks/master/Barcelona/"
netfile = "Barcelona_net.tntp"
tripsfile = "Barcelona_trips.tntp"

# Load from GitHub and save locally
for f in [netfile, tripsfile]:
    url = os.path.join(URL, f)
    response = requests.get(url)
    with open(f, 'wb') as fh:
        fh.write(response.content)
```

Note that the power factor $p_e$ for the link travel time $l_e(x_e) = \text{fft}_e \cdot \left( 1 + B_e \cdot \left(\frac{x}{\text{cap}_e}\right) ^ {p_e} \right)$ is ``not`` an integer for the Barcelona network, where we find power factors such  ``4.446, 4.924, ...``.

However, we can equip a network with custom cost functions by setting ``cost_type=symbolic``:

```python
# Setup network
net = paminco.Network.from_tntp(netfile, tripsfile=tripsfile, cost_type="symbolic")

# Cleanup
for f in [netfile, tripsfile]:
    os.remove(f)
```

## 2. Setting up network cost

Second, we set up SymbolicCost, defining the edge cost $F_e$ and thus the objective function to calculate both user equilibrium (UE) and system optimum (SO).
We find minimum cost flows for user equilibrium and system optimum if we transform the edge cost by:

$$
\begin{align}
    \text{User equilibrium:} \quad F_e &= \int_0^{x_e} l_e(s) ds \\
    \text{System optimum:} \quad F_e &= x_e \cdot l_e \\
\end{align}
$$

Note that [Frank-Wolfe](https://en.wikipedia.org/wiki/Frank%E2%80%93Wolfe_algorithm) requires the first derivative of the objective function.
Thus, we have to specfiy edge cost $F_e$ and marginal edge cost $f_e$.

```python
# User equilibrium cost funcs (egde cost and marginal edge cost)
ue_F = "fft * (x + b * x / (p + 1) * (x / c)**p)"
ue_f = "fft * (1 + b * (x / c)**p)"

# System optimum cost funcs (egde cost and marginal edge cost)
so_F = "x * fft * (1 + b * x**p * c**(-p))"
so_f = "fft + fft * b * (p + 1) * c**(-p) * x**(p)"

net.cost.coeffs["fft"] = net.cost.coeffs["free_flow_time"]
net.cost.coeffs["c"] = net.cost.coeffs["capacity"]
net.cost.coeffs["p"] = net.cost.coeffs["power"]
```

## 3. Find UE and SO flow

Finally, we equip our network with both cost functions instances and run Frank-Wolfe to find a minimum cost flow:

```python
# copy networks
from copy import deepcopy

# user equilibrium
net_ue = deepcopy(net)
net_ue.cost.funcs["F"] = ue_F
net_ue.cost.funcs["f"] = ue_f
fw_ue = paminco.NetworkFW(net_ue)
fw_ue.run(max_iter=10, print=False)

# system optimum
net_so = deepcopy(net)
net_so.cost.funcs["F"] = so_F
net_so.cost.funcs["f"] = so_f
fw_so = paminco.NetworkFW(net_so)
fw_so.run(max_iter=10, print=False)
```

For example, we can easily access the user equilibrium flow by:

```python
fw_ue.x
```

    array([1062.3604087 , 1183.7485913 ,    0.        , ..., 1061.32166004,
              0.        , 4203.83150619])

## 4. Price of Anarchy

The objective function to calculate the system optimum is the total system travel time (TSTT):

$$
\begin{equation*}
    \text{TSTT}(\mathbf{x}) = \sum_{e \in E} x_{e} \cdot l_{e}(x_e)
\end{equation*}
$$

```python
def TTST(x):
    return net_so.cost(x).sum()
```

We find that the total system travel time for Barcelona is increased by about ``0.9 percent`` if users behave selfishly:

```python
poa = TTST(fw_ue.flow) / TTST(fw_so.flow)
poa
```

    1.0088344480937708


# Matrix Formalism of the Newton-Euler equations and Equations of Movement

Renato Naville Watanabe

In this notebook will be shown two examples of how to use a matrix formalism to perform forward and inverse dynamics analysis. It does not consist a comprehensive treatise about the subject. It is rather an introduction based on examples. Nevertheless, the reader of this notebook will have sufficient knowledge to read recent texts on biomechanics and other multibody dynamic analysis.

## Forward Dynamics

For the forward dynamics analysis, we will consider that we know the torques and find the angular accelerations. Naturally, we could begin our analysis from the muscle forces, from the muscle activations or even from the neural commands from the motor cortex.

Adapted from Erdemir et al. (2007)

As an introduction to the matrix formalism used in multibody analysis, we will consider the double-pendulum system. This system could be used as a  model from the arm and forearm of a subject, for example, and is an example of chaotic system.

In the notebook about [free-body diagram](FreeBodyDiagramForRigidBodies.ipynb#doublependulum), we found the following equations of motion to the double-pendulum with actuators.

\begin{align}
\begin{split}
     \left(\frac{m_1l_1^2}{3} +m_2l_1^2\right)\frac{d^2\theta_1}{dt^2} + \frac{m_2l_1l_2}{2} \cos{(\theta_1-\theta_2)\frac{d^2\theta_2}{dt^2}}  &=  -\frac{m_1gl_1}{2}\sin(\theta_1)- m_2l_1g \sin(\theta_1)   - \frac{m_2l_1l_2}{2}\left(\frac{d\theta_2}{dt}\right)^2 \sin(\theta_1-\theta_2)  + M_1 - M_{12} \\
    \frac{m_2l_1l_2}{2}\cos(\theta_1-\theta_2)\frac{d^2\theta_1}{dt^2} + \frac{m_2l_2^2}{3}\frac{d^2\theta_2}{dt^2} &= -\frac{m_2gl_2}{2}\sin(\theta_2) + \frac{m_2l_1l_2}{2}\left(\frac{d\theta_1}{dt}\right)^2 \sin(\theta_1-\theta_2)+ M_{12}
    \end{split}
\end{align}

If we wanted to simulate this double pendulum we still need to isolate the angular accelerations of each of the bars. As can be easily noted, this would be too laborious to do by hand. Luckily, numerical integration is performed by computers. The easiest way to isolate these angular accelerations is to note that we can write the angular accelerations and the numbers multiplying them as a matrix and a vector.

\begin{equation}
     \left[\begin{array}{cc}\frac{m_1l_1^2}{3} +m_2l_1^2&\frac{m_2l_1l_2}{2} \cos(\theta_1-\theta_2)\\\frac{m_2l_1l_2}{2}\cos(\theta_1-\theta_2)&\frac{m_2l_2^2}{3}\end{array}\right]\cdot\left[\begin{array}{c}\frac{d^2\theta_1}{dt^2}\\\frac{d^2\theta_2}{dt^2} \end{array}\right] = \left[\begin{array}{c}- \frac{m_2l_1l_2}{2}\left(\frac{d\theta_2}{dt}\right)^2 \sin(\theta_1-\theta_2)-\frac{m_1gl_1}{2}\sin(\theta_1)- m_2l_1g \sin(\theta_1) + M_1 - M_{12}\\ \frac{m_2l_1l_2}{2}\left(\frac{d\theta_1}{dt}\right)^2 \sin(\theta_1-\theta_2)-\frac{m_2gl_2}{2}\sin(\theta_2) + M_{12} \end{array}\right]
\end{equation}

   Typically the equations of motion are divided into a matrix corresponding to the terms involving the angular velocities (centrifugal and Coriolis forces), a  matrix corresponding to gravitational force and another matrix corresponding to the forces and torques being applied to the system.

\begin{equation}
    M(q)\ddot{q} = C(q,\dot{q}) + G(q) + Q +  E
\end{equation}

where

- $q$ is the vector of the generalized coordinates, like angles and positions;
- $M(q)$ is the matrix containing the inertia elements like mass and moments of inertia;
- $C(q,\dot{q})$  is the vector with the forces and moments dependent from the velocities and angular velocities, like centrifugal and Coriolis forces;
- $G(q)$ is the vector with the forces and torques caused by the gravitational force;
- $Q$ is the vector with forces and torques being applied to the body, like muscular torques and forces due to the constraints.
- $E$ is the vector with the forces and torques due to some external element, like springs or the Ground reaction Force.

We can divide the equation describing the behavior of the double-pendulum in the matrices above:

\begin{equation}
     \underbrace{\left[\begin{array}{cc}\frac{m_1l_1^2}{3} +m_2l_1^2&\frac{m_2l_1l_2}{2} \cos(\theta_1-\theta_2)\\\frac{m_2l_1l_2}{2}\cos(\theta_1-\theta_2)&\frac{m_2l_2^2}{3}\end{array}\right]}_{M}\cdot\underbrace{\left[\begin{array}{c}\frac{d^2\theta_1}{dt^2}\\\frac{d^2\theta_2}{dt^2} \end{array}\right]}_{\ddot{q}} = \underbrace{\left[\begin{array}{c}- \frac{m_2l_1l_2}{2}\left(\frac{d\theta_2}{dt}\right)^2 \sin(\theta_1-\theta_2)\\ \frac{m_2l_1l_2}{2}\left(\frac{d\theta_1}{dt}\right)^2 \sin(\theta_1-\theta_2)\end{array}\right]}_{C} + \underbrace{\left[\begin{array}{c}-\frac{m_1gl_1}{2}\sin(\theta_1)- m_2l_1g \sin(\theta_1)\\ -\frac{m_2gl_2}{2}\sin(\theta_2) \end{array}\right]}_{G} + \underbrace{\left[\begin{array}{c} M_1 - M_{12}\\M_{12} \end{array}\right]}_{Q}
\end{equation}

To solve this differential equation numerically, we must obtain the expression of the angular accelerations. We can perform this by multiplying both sides by the inverse of the matrix $M$.

\begin{equation}
    \left[\begin{array}{c}\frac{d^2\theta_1}{dt^2}\\\frac{d^2\theta_2}{dt^2} \end{array}\right] = \left[\begin{array}{cc}\frac{m_1l_1^2}{3} +m_2l_1^2&\frac{m_2l_1l_2}{2} \cos(\theta_1-\theta_2)\\\frac{m_2l_1l_2}{2}\cos(\theta_1-\theta_2)&\frac{m_2l_2^2}{3}\end{array}\right]^{-1}\cdot\left(\left[\begin{array}{c}- \frac{m_2l_1l_2}{2}\left(\frac{d\theta_2}{dt}\right)^2 \sin(\theta_1-\theta_2)\\ \frac{m_2l_1l_2}{2}\left(\frac{d\theta_1}{dt}\right)^2 \sin(\theta_1-\theta_2)\end{array}\right] + \left[\begin{array}{c}-\frac{m_1gl_1}{2}\sin(\theta_1)- m_2l_1g \sin(\theta_1)\\ -\frac{m_2gl_2}{2}\sin(\theta_2) \end{array}\right] + \left[\begin{array}{c} M_1 - M_{12}\\M_{12} \end{array}\right]\right)
\end{equation}

Generically, having the differential equations in the format:

\begin{equation}
    M(q)\ddot{q} = C(q,\dot{q}) + G(q) + Q +  E
\end{equation}

we can obtain the equation to perform the forward dynamics by:

\begin{equation}
    \ddot{q} = M(q)^{-1}\left(C(q,\dot{q}) + G(q) + Q +  E\right)
\end{equation}

Now that we have the angular accelerations, to solve the equation numerically we must transform the second-order differential equations in first-order differential equations:

\begin{equation}
    \left[\begin{array}{c}\frac{d\omega_1}{dt}\\\frac{d\omega_2}{dt}\\\frac{d\theta_1}{dt}\\\frac{d\theta_2}{dt} \end{array}\right] =  \left[\begin{array}{c}\left[\begin{array}{cc}\frac{m_1l_1^2}{3} +m_2l_1^2&\frac{m_2l_1l_2}{2} \cos(\theta_1-\theta_2)\\\frac{m_2l_1l_2}{2}\cos(\theta_1-\theta_2)&\frac{m_2l_2^2}{3}\end{array}\right]^{-1}\left(\left[\begin{array}{c}- \frac{m_2l_1l_2}{2}\omega_2^2 \sin(\theta_1-\theta_2)\\ \frac{m_2l_1l_2}{2}\omega_1^2 \sin(\theta_1-\theta_2)\end{array}\right]+\left[\begin{array}{c} -\frac{m_1gl_1}{2}\sin(\theta_1)- m_2l_1g \sin(\theta_1) \\-\frac{m_2gl_2}{2}\sin(\theta_2)  \end{array}\right] + \left[ \begin{array}{c}M_1 - M_{12}\\M_{12}\end{array}\right]\right)\\ \omega_1\\ \omega_2\end{array}\right]
\end{equation}

Below is the numerical solution of a double-pendulum with each bar having length equal to 1 m and mass equal to 1 kg.

```python
import numpy as np

g = 9.81

m1 = 1
m2 = 1
l1 = 1
l2 = 0.5

theta10 = np.pi/10
theta20 = np.pi/3
omega10 = 2*np.pi
omega20 = -6*np.pi

dt = 0.0001
t = np.arange(0, 20, dt)
state = np.zeros((4, len(t)))

state[:,0] = np.array([omega10, omega20, theta10, theta20])
#print(state[0,0])
M1 = 0
M12 = 0

for i in range(1,len(t)):

    thetaDiff = state[2,i-1] - state[3,i-1]

    M = np.array([[m1*l1**2/3 + m2*l1**2, m2*l1*l2*np.cos(thetaDiff)/2],
                  [m2*l1*l2*np.cos(thetaDiff)/2, m2*l2**2/3]])

    C = np.array([[-m2*l1*l2*np.sin(thetaDiff)*state[1,i-1]**2/2],
                  [m2*l1*l2*np.sin(thetaDiff)*state[0,i-1]**2/2]])

    G = np.array([[-m1*g*l1/2*np.sin(state[2,i-1]) - m2*g*l2*np.sin(state[2,i-1])],
                  [-m2*g*l2/2*np.sin(state[3,i-1])]])

    #PID control
    #r = np.pi/3
    #M12 = 30*(r-state[3,i-1])- 2*state[1,i-1] + 3*np.trapz(r-state[3,0:i])*dt
    Q = np.array([[M1 - M12],[M12]])

    dstatedt = np.vstack((np.linalg.inv(M)@(C+G+Q),state[0,[i-1]],state[1,[i-1]]))

    state[:,[i]] =  state[:,[i-1]] + dt*dstatedt

```

```python
import matplotlib.pyplot as plt
%matplotlib notebook
plt.figure()
plt.plot(t[0::10], state[3,0::10].T)
#plt.plot(t[0::10], r*np.ones_like(t[0::10]))
plt.show()
```

    <IPython.core.display.Javascript object>

```python
plt.figure()

step = 3000

for i in range(len(state[2,0::step])):

    plt.plot([0, l1*np.sin(state[2,i*step])],
             [0, -l1*np.cos(state[2,i*step])])
    plt.plot([l1*np.sin(state[2,i*step]), l1*np.sin(state[2,i*step])+l2*np.sin(state[3,i*step])],
             [-l1*np.cos(state[2,i*step]), -l1*np.cos(state[2,i*step])-l2*np.cos(state[3,i*step])])
plt.show()
```

    <IPython.core.display.Javascript object>

## Inverse dynamics

For the inverse dynamics analysis, we will obtain the joint torques and forces, from the joint kinematics and external forces.

Adapted from Erdemir et al. (2007)

  As an example, we will consider the problem of estimating the forces and torques in the ankle and knee joints during the gait, considering a 3D movement. At this point, we consider that the  accelerations, angular velocities, angular accelerations, masses, moments of inertia and rotation matrices necessary  to compute the forces and moments are known.

 The free-body diagram of the gait, considering a 3D movement is very similar [to the 2D case](GaitAnalysis2D.ipynb). The equations of forces and moments are described by the Newton-Euler equations (for a revision on Tridimensional Newton-Euler equations click [here](Tridimensional%20rigid%20body%20Kinetics.ipynb)):
\begin{align}
\overrightarrow{F_A} + \overrightarrow{GRF} + m_F\overrightarrow{g} &= m_F\overrightarrow{a_{cm_F}}\\
\overrightarrow{M_A} + \overrightarrow{M_{GRF}}+ \overrightarrow{M_{FA}}&=I_F\overrightarrow{\dot{\omega_F}} + \overrightarrow{\omega_F} \times (I_F\overrightarrow{\omega_F})\\
\overrightarrow{F_K} -\overrightarrow{F_A} + m_S\overrightarrow{g} &= m_S\overrightarrow{a_{cm_S}}\\
    \overrightarrow{M_K} - \overrightarrow{M_A} + \overrightarrow{M_{FA}} + \overrightarrow{M_{FK}} &= I_S\overrightarrow{\dot{\omega_S}} + \overrightarrow{\omega_S} \times (I_S\overrightarrow{\omega_S})
\end{align}
where

- $\overrightarrow{g} = -9.81\hat{j}$;
- $m_F$ and $m_S$ are the masses of the foot and the shank, respectively;
- $\overrightarrow{GRF}$ is the ground reaction force being applied to the foot;
- $\overrightarrow{a_{cm_F}}$ and $\overrightarrow{a_{cm_S}}$ are the accelerations of the center of mass of the foot and the shank, respectively;
- $\overrightarrow{\omega_F}$ and $\overrightarrow{\omega_S}$  are the angular accelerations of the foot and shank, respectively, described at a basis attached to the segment, and $\overrightarrow{\dot{\omega_F}}$ and $\overrightarrow{\dot{\omega_S}}$ are their time-derivatives;
- $I_S$ and $I_F$ are the matrices of inertia of the shank and the foot, respectively;
- $\overrightarrow{F_K}$, $\overrightarrow{F_A}$, $\overrightarrow{M_A}$ and $\overrightarrow{M_A}$ are the forces and moments at the ankle and knee joints, respectively

Note that each of these equations have components at each of the three directions. Additionally, note that the equations of the forces are described in the global basis, and the equations of the moments must be described in the basis attached to the segment relative to that equation. So, it is a good idea to make this clear with a more precise notation. We will denote as a superscript in the vectors the segment where the basis that we are describing the vector is fixed. So for example, $\overrightarrow{M_A^F}$ is the vector of the moment due to the muscle forces of the ankle, described in the basis fixed at the foot. So, the equations can be rewritten as:

\begin{align}
\overrightarrow{F_A^G} + \overrightarrow{GRF^G} + m_F\overrightarrow{g^G} &= m_F\overrightarrow{a_{cm_F}^G}\\
\overrightarrow{M_A^F} + \overrightarrow{M_{GRF}^F}+ \overrightarrow{M_{FA}^F}&=I_F\overrightarrow{\dot{\omega_F^F}} + \overrightarrow{\omega_F^F} \times (I_F\overrightarrow{\omega_F^F})\\
\overrightarrow{F_K^G} -\overrightarrow{F_A^G} + m_S\overrightarrow{g^G} &= m_S\overrightarrow{a_{cm_S}^G}\\
    \overrightarrow{M_K^S} - \overrightarrow{M_A^S} + \overrightarrow{M_{FA}^S} + \overrightarrow{M_{FK}^S} &= I_S\overrightarrow{\dot{\omega_S^S}} + \overrightarrow{\omega_S^S} \times (I_S\overrightarrow{\omega_S^S})
\end{align}
where the superscript $G$ denotes the global frame of reference, the superscript $S$ denotes the frame of reference in the shank and the superscript $F$ denotes the frame of reference at the foot.

The moments due to the ground reaction force, the force at the ankle and the force at the knee are computed by cross-multiplying them by their moment-arms. As the forces and the moment-arms are described in the global basis, we must multiply them by the rotation matrix of the basis corresponding to the segment. So, the equations can be rewritten as:

\begin{align}
\overrightarrow{F_A^G} + \overrightarrow{GRF^G} + m_F\overrightarrow{g^G} &= m_F\overrightarrow{a_{cm_F}^G}\\
\overrightarrow{M_A^F} + R_F(\overrightarrow{r_{cop/cm_F}^G}\times \overrightarrow{GRF^G})+ R_F(\overrightarrow{r_{A/cm_F}^G}\times \overrightarrow{F_A}^G)&=I_F\overrightarrow{\dot{\omega_F^F}} + \overrightarrow{\omega_F^F} \times (I_F\overrightarrow{\omega_F^F})\\
\overrightarrow{F_K^G} -\overrightarrow{F_A^G} + m_S\overrightarrow{g^G} &= m_S\overrightarrow{a_{cm_S}^G}\\
\overrightarrow{M_K^S} - \overrightarrow{M_A^S} - R_S(\overrightarrow{r_{A/cm_S}^G}\times \overrightarrow{F_A^G}) + R_S(\overrightarrow{r_{K/cm_S}^G}\times \overrightarrow{F_K^G}) &= I_S\overrightarrow{\dot{\omega_S^S}} + \overrightarrow{\omega_S^S} \times (I_S\overrightarrow{\omega_S^S})
\end{align}
where $R_S$ is the rotation matrix of the basis attached to the shank and $R_F$ is the rotation matrix of the basis attached to the foot.

Now, we can note that the vectors $\overrightarrow{M_K^S}$ and $\overrightarrow{M_K^F}$ are the same vectors described in different basis. So we could use only one of the descriptions and use rotation matrices to convert from one to another. To pass the vector from the foot coordinates to the shank coordinate, we must first multiply it by the inverted rotation matrix of the foot and then multiply it by the rotation matrix of the shank. So, $\overrightarrow{M_A^S} = R_SR_F^{-1}\overrightarrow{M_A^F}$ and the equations above can be rewritten as:

\begin{align}
\overrightarrow{F_A^G} + \overrightarrow{GRF^G} + m_F\overrightarrow{g^G} &= m_F\overrightarrow{a_{cm_F}^G}\\
\overrightarrow{M_A^F} + R_F(\overrightarrow{r_{cop/cm_F}^G}\times \overrightarrow{GRF^G})+ R_F(\overrightarrow{r_{A/cm_F}^G}\times \overrightarrow{F_A}^G)&=I_F\overrightarrow{\dot{\omega_F^F}} + \overrightarrow{\omega_F^F} \times (I_F\overrightarrow{\omega_F^F})\\
\overrightarrow{F_K^G} -\overrightarrow{F_A^G} + m_S\overrightarrow{g^G} &= m_S\overrightarrow{a_{cm_S}^G}\\
\overrightarrow{M_K^S} - R_SR_F^{-1}\overrightarrow{M_A^F} - R_S(\overrightarrow{r_{A/cm_S}^G}\times \overrightarrow{F_A^G}) + R_S(\overrightarrow{r_{K/cm_S}^G}\times \overrightarrow{F_K^G}) &= I_S\overrightarrow{\dot{\omega_S^S}} + \overrightarrow{\omega_S^S} \times (I_S\overrightarrow{\omega_S^S})
\end{align}

Now, we divide the equations above in the matrices defined previously:
\begin{equation}
\underbrace{\left[\begin{array}{cccc} m_FI_3& [0]& [0]& [0]\\ [0]& I_F  & [0] & [0] \\ [0] &[0] &  m_SI_3& [0] \\ [0] & [0] & [0] & I_S\end{array}\right]}_{M}\cdot\left[\begin{array}{c}\overrightarrow{a_{cm_F}^G}\\\overrightarrow{\dot{\omega_F^F}}\\\overrightarrow{a_{cm_S}^G}\\\overrightarrow{\dot{\omega_S^S}} \\ \end{array}\right] = \underbrace{\left[\begin{array}{c}[0]\\ - \overrightarrow{\omega_F^F} \times (I_F\overrightarrow{\omega_F^F})  \\ [0] \\ - \overrightarrow{\omega_S^S} \times (I_S\overrightarrow{\omega_S^S}) \end{array}\right]}_{C} + \underbrace{\left[\begin{array}{c}  m_F\overrightarrow{g^G}\\ [0]\\  m_S\overrightarrow{g^G} \\  [0] \end{array}\right]}_{G} + \underbrace{\left[\begin{array}{c}  \overrightarrow{F_A^G}\\ \overrightarrow{M_A^F}+R_F(\overrightarrow{r_{A/cm_F}^G}\times \overrightarrow{F_A}^G)\\  \overrightarrow{F_K^G} - \overrightarrow{F_A^G} \\ \overrightarrow{M_K^S} - R_SR_F^{-1}\overrightarrow{M_A^F} - R_S(\overrightarrow{r_{A/cm_S}^G}\times \overrightarrow{F_A^G}) + R_S(\overrightarrow{r_{K/cm_S}^G}\times \overrightarrow{F_K^G})  \end{array}\right]}_{Q} + \underbrace{\left[\begin{array}{c} \overrightarrow{GRF^G}\\  R_F(\overrightarrow{r_{cop/cm_F}^G}\times \overrightarrow{GRF^G})\\  [0] \\  [0] \end{array}\right]}_{E}
\end{equation}

where $I_3$ is the identity matrix 3x3.

To perform the inverse dynamics, we still cannot isolate the vector of forces and moments. As the vector $F$ has cross-products we must define the a new operator that performs the cross-product through a matrix multiplication.

We can note that the cross-product between the vectors $\vec{v}$ and $\vec{w}$ has the following result:

\begin{equation}
    \vec{v} \times \vec{w} = \left[\begin{array}{c}v_x\\v_y\\v_z \end{array}\right] \times \left[\begin{array}{c}w_x\\w_y\\w_z \end{array}\right] = \left[\begin{array}{c}v_yw_z - v_zw_y\\v_zw_x - v_xw_z\\v_xw_y - v_yw_x \end{array}\right] = \left[\begin{array}{ccc}0&-v_z&v_y\\v_z&0&-v_x\\-v_y&v_x&0 \end{array}\right]\cdot\left[\begin{array}{c}w_x\\w_y\\w_z \end{array}\right]
\end{equation}

So we can define a new operator known as skew-symmetric matrix:

\begin{equation}
    S(\vec{v}) \triangleq \left[\begin{array}{ccc}0&-v_z&v_y\\v_z&0&-v_x\\-v_y&v_x&0 \end{array}\right]
\end{equation}

Therefore:

\begin{equation}
    \vec{v} \times \vec{w} = S(\vec{v})\cdot\vec{w}
\end{equation}

Now, we will use this operator in  the equation we found previously:

\begin{equation}
\left[\begin{array}{cccc} m_FI_3& [0]& [0]& [0]\\ [0]& I_F  & [0] & [0] \\ [0] &[0] &  m_SI_3& [0] \\ [0] & [0] & [0] & I_S\end{array}\right]\cdot\left[\begin{array}{c}\overrightarrow{a_{cm_F}^G}\\\overrightarrow{\dot{\omega_F^F}}\\\overrightarrow{a_{cm_S}^G}\\\overrightarrow{\dot{\omega_S^S}} \\ \end{array}\right] = \left[\begin{array}{c}[0]\\ - \overrightarrow{\omega_F^F} \times (I_F\overrightarrow{\omega_F^F})  \\ [0] \\ - \overrightarrow{\omega_S^S} \times (I_S\overrightarrow{\omega_S^S}) \end{array}\right] + \left[\begin{array}{c}  m_F\overrightarrow{g^G}\\ [0]\\  m_S\overrightarrow{g^G} \\  [0] \end{array}\right] + \left[\begin{array}{c}  \overrightarrow{F_A^G}\\ \overrightarrow{M_A^F}+R_F(S(\overrightarrow{r_{A/cm_F}^G})\cdot\overrightarrow{F_A}^G)\\  \overrightarrow{F_K^G} - \overrightarrow{F_A^G} \\ \overrightarrow{M_K^S} - R_SR_F^{-1}\overrightarrow{M_A^F} - R_S(S(\overrightarrow{r_{A/cm_S}^G})\cdot\overrightarrow{F_A^G}) + R_S(S(\overrightarrow{r_{K/cm_S}^G})\cdot\overrightarrow{F_K^G})  \end{array}\right] + \left[\begin{array}{c} \overrightarrow{GRF^G}\\  R_F(\overrightarrow{r_{cop/cm_F}^G}\times \overrightarrow{GRF^G})\\  [0] \\  [0] \end{array}\right]
\end{equation}

Now it is possible to write the vector $F$ as multiplication of a matrix by a vector:

\begin{equation}
\left[\begin{array}{cccc} m_FI_3& [0]& [0]& [0]\\ [0]& I_F  & [0] & [0] \\ [0] &[0] &  m_SI_3& [0] \\ [0] & [0] & [0] & I_S\end{array}\right]\cdot\left[\begin{array}{c}\overrightarrow{a_{cm_F}^G}\\\overrightarrow{\dot{\omega_F^F}}\\\overrightarrow{a_{cm_S}^G}\\\overrightarrow{\dot{\omega_S^S}} \\ \end{array}\right] = \left[\begin{array}{c}[0]\\ - \overrightarrow{\omega_F^F} \times (I_F\overrightarrow{\omega_F^F})  \\ [0] \\ - \overrightarrow{\omega_S^S} \times (I_S\overrightarrow{\omega_S^S}) \end{array}\right] + \left[\begin{array}{c}  m_F\overrightarrow{g^G}\\ [0]\\  m_S\overrightarrow{g^G} \\  [0] \end{array}\right] + \left[\begin{array}{ccc}  I_3& [0]& [0]& [0]\\ R_FS\left(\overrightarrow{r_{A/cm_F}^G}\right)&I_3& [0]& [0]\\  -I_3& [0]& I_3 & [0] \\ -R_SS\left(\overrightarrow{r_{A/cm_S}^G}\right)& - R_SR_F^{-1} & R_SS\left(\overrightarrow{r_{K/cm_S}^G}\right) & I_3      \end{array}\right]\cdot\left[\begin{array}{c}  \overrightarrow{F_A^G}\\ \overrightarrow{M_A^F}\\  \overrightarrow{F_K^G}\\ \overrightarrow{M_K^S}\end{array}\right] + \left[\begin{array}{c} \overrightarrow{GRF^G}\\  R_F(\overrightarrow{r_{cop/cm_F}^G}\times \overrightarrow{GRF^G})\\  [0] \\  [0] \end{array}\right]
\end{equation}

So, the final equation to compute the forces and torques is obtained by multiplying everything by the inverse of the matrix multipliying the vector of forces:

\begin{equation}
\left[\begin{array}{c}  \overrightarrow{F_A^G}\\ \overrightarrow{M_A^F}\\  \overrightarrow{F_K^G}\\ \overrightarrow{M_K^S}\end{array}\right]  = \left[\begin{array}{ccc}  I_3& [0]& [0]& [0]\\ R_FS\left(\overrightarrow{r_{A/cm_F}^G}\right)&I_3& [0]& [0]\\  -I_3& [0]& I_3 & [0] \\ -R_SS\left(\overrightarrow{r_{A/cm_S}^G}\right)& - R_SR_F^{-1} & R_SS\left(\overrightarrow{r_{K/cm_S}^G}\right) & I_3      \end{array}\right]^{-1}\cdot\left(\left[\begin{array}{cccc} m_FI_3& [0]& [0]& [0]\\ [0]& I_F  & [0] & [0] \\ [0] &[0] &  m_SI_3& [0] \\ [0] & [0] & [0] & I_S\end{array}\right]\cdot\left[\begin{array}{c}\overrightarrow{a_{cm_F}^G}\\\overrightarrow{\dot{\omega_F^F}}\\\overrightarrow{a_{cm_S}^G}\\\overrightarrow{\dot{\omega_S^S}} \\ \end{array}\right] - \left[\begin{array}{c}[0]\\ - \overrightarrow{\omega_F^F} \times (I_F\overrightarrow{\omega_F^F})  \\ [0] \\ - \overrightarrow{\omega_S^S} \times (I_S\overrightarrow{\omega_S^S}) \end{array}\right] -\left[\begin{array}{c} \overrightarrow{GRF^G}\\  R_F(\overrightarrow{r_{cop/cm_F}^G}\times \overrightarrow{GRF^G})\\  [0] \\  [0] \end{array}\right] - \left[\begin{array}{c}  m_F\overrightarrow{g^G}\\ [0]\\  m_S\overrightarrow{g^G} \\  [0] \end{array}\right]\right)
\end{equation}

With the last equation, we can obtain all the forces and moments using only one line of code. Computationally, it is less prone to errors and more efficient.

So, generically, the steps to perform the analysis of inverse dynamics is:

- write the equations of Newton-Euler for each segment. Write explicitly the basis at which  each vector is described.

- use the rotation matrices of the basis to pass the description of a vector to another basis. Use it in a way that the same vector is described at just a single frame of reference.

- write the cross-products as a product between the skew-symmetric matrix $S$ of the first vector and the second vector.

- write the equations in the matrix format, repeated here:

\begin{equation}
    M(q)\ddot{q} = C(q,\dot{q}) + G(q) + Q +  E
\end{equation}

- write explicitly the vector containing the unknown forces and moments $Q$, as a multiplication of a matrix and vector containing only the unknown forces.

- isolate the vector containing only the unknown forces by multiplying the whole equation by the inverse of the matrix multiplying the vector with the forces.

## Problems

1) Solve the problems 18.3.20 and 18.3.24 of the Ruina and Rudra's book by using the Lagrangian formalism (it is much easier than use the Newton-Euler formalism) and then use the matrix formalism to obtain the expressions of the angular accelerations.

2) Write the matrices to find the forces and torques in a tridimensional double pendulum, consisted of two cylindrical bars. Consider that you know all the masses, moments of inertia, rotation matrices, accelerations, angular velocities and angular accelerations necessary to solve the problem.

## References

- YAMAGUCHI, G. T. Dynamic modeling of musculoskeletal motion: a vectorized approach for biomechanical analysis in three dimensions., 2001

- CRAIG, J. Introduction to robotics. , 1989

- JAIN, A. Robot and multibody dynamics. , 2011

- SPONG, M. W.; HUTCHINSON, S.; VIDYASAGAR, M. Robot modeling and control., 2006

- ERDEMIR, A. et al. Model-based estimation of muscle forces exerted during movements. Clinical Biomechanics, v. 22, n. 2, p. 131–154, 2007.

- STANEV, D.; MOUSTAKAS, K. Simulation of constrained musculoskeletal systems in task space. IEEE Transactions on Biomedical Engineering, v. 65, n. 2, p. 307–318, 2018.

- ZAJAC FE, GORDON ME , [Determining muscle's force and action in multi-articular movement](https://drive.google.com/open?id=0BxbW72zV7WmUcC1zSGpEOUxhWXM&authuser=0). Exercise and Sport Sciences Reviews, 17, 187-230. , 1989

- RUINA A, RUDRA P. [Introduction to Statics and Dynamics](http://ruina.tam.cornell.edu/Book/index.html). Oxford University Press. , 2015

```python

```

```python

```


# Symbolic Mathematics in Python

There are times when you need to solve a difficult problem symbollically or analytically. If you have ever used Wolfram Alpha, then you have already done this. Sympy is a python library that allows you to do symbolic mathematics in python.

```python
import sympy as sym
```

## 1. Introduction

### Example 1.1

If you try to write the follwing in python by itself, you will get an error telling you x is undefined:
$$x-x$$

```python
x-x
```

(The error above is on purpose). Variables in python need to be defined before you can say something specific about them

```python
x=102
x-x
```

    0

If you are trying to show that $x-x=0$ is true for any $x$, the above answer would not be valid. Instead you can use a symbolic expression to show that it is true

**First we define the variable as a symmbolic expression**

```python
x = sym.symbols('x')
```

**Now we can use the variable in a symbolic expression**

```python
x-x
```

$\displaystyle 0$

### Example 1.2

Sympy can be used to perform algebreic operations (among other things). Consider the following expression: $$(3a-4b)^3$$

We can use symppy to expand the expression algebraically.

**First we need to define the variables as symbolic expressions**

```python
a,b = sym.symbols('a,b')
```

**Side note** Notice that the left hand side of the epression has two variables being defined. Python can define more than one variable at a time:

```python
x1,y1 =10,20
print(x1)
print(y1)
```

    10
    20

**Back to the expression** We can define an expression using the variables $a$ and $b$.

```python
expr = (3*a-4*b)**3
print(expr)
```

    (3*a - 4*b)**3

We can also make it look nicer in our notebook. This doesn't affect the math, but it makes our notebook more readable.

```python
sym.init_printing()
```

```python
expr
```

**Now we expand the function algebreically**

```python
expr.expand()
```

Sympy can also factor the equation

```python
sym.factor(26*a**3-108*a**2*b+144*a*b**2-64*b**3)
```

If you want to copy and paste a result, you print the result.

```python
print(sym.factor(26*a**3-108*a**2*b+144*a*b**2-64*b**3))
```

    2*(a - 2*b)*(13*a**2 - 28*a*b + 16*b**2)

You can also chain together functions

```python
expr.expand().factor()
```

### Exercise 1.1

Show that the following two expressions are true.
$$(2w-3z)(2w+3z)=4w^2-9z^2$$
$$(2w-3z)^2\ne4w^2-9z^2$$

```python
# Your code here
w,z = sym.symbols('w,z')

expr = (2*w-3*z)*(2*w+3*z)
sym.init_printing()
expr
```

```python
expr.expand()
```

```python
expr = (2*w-3*z)**2
sym.init_printing()
expr
```

```python
expr.expand()
```

## 2. Solving Equations

Sympy can be used to symbolilically solve equations. As before, you need to define which variables are symbols

### Example 2.1

Use sympy to solve the following equation
$$ax^3+bx^2+cx+d=0$$

```python
# Define the variables
a,b,c,d,x = sym.symbols('a,b,c,d,x')
```

```python
# Define the expression
expr=a*x**3+b*x**2+c*x+d
expr
```

We can use the `solvset` function to solve this equation

```python
solutions=sym.solveset(expr,x)
```

```python
print(solutions)
```

    {-(-3*c/a + b**2/a**2)/(3*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)) - (sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)/3 - b/(3*a), -(-3*c/a + b**2/a**2)/(3*(-1/2 - sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)) - (-1/2 - sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)/3 - b/(3*a), -(-3*c/a + b**2/a**2)/(3*(-1/2 + sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)) - (-1/2 + sqrt(3)*I/2)*(sqrt(-4*(-3*c/a + b**2/a**2)**3 + (27*d/a - 9*b*c/a**2 + 2*b**3/a**3)**2)/2 + 27*d/(2*a) - 9*b*c/(2*a**2) + b**3/a**3)**(1/3)/3 - b/(3*a)}

```python
solutions
```

What if I need help. You can do this with any python function. `function?`

```python
# Run this command to see a help box
sym.solveset?
```

### Exercise 2.1

Use the `solveset` function to solve the following chemical problem.

Phosgene gas, $\text{COCl}_2$, dissociates at high temperatures according to the following equilibrium:

$$ \text{COCl}_2 \rightleftharpoons  \text{CO} + \text{Cl}_2 $$

At $\text{400 C}$, the equilibrium constant $K_c=8.05$.

If you start with a  $\text{0.250 M}$ phosgene sample at $\text{400 C}$, determine the concentrations of all species at equilibrium.

```python
# Your code here
x = sym.symbols('x')

expr=((x*x)/(0.250-x))-8.05
expr
```

```python
solutions=sym.solveset(expr,x)
print(solutions)
```

    {-8.29268379803378, 0.242683798033777}

Why did you pick your answer?

```python
#concentrations cannot be negative
#[CO]=[Cl2]=0.243 M
#[COCl2]=0.25 M-0.243 M
```

## 3. Calculus

We can use also Sympy to differentiate and integrate. Let us experiment with differentiating the following expression:

$$x ^ 2 - \cos(x)$$

```python
sym.diff(x ** 2 - sym.cos(x), x)
```

Similarly we can integrate:

```python
sym.integrate(x ** 2 - sym.cos(x), x)
```

We can also carry out definite integrals:

```python
sym.integrate(x ** 2 - sym.cos(x), (x, 0, 5))
```

### Exercise 3.1

Use Sympy to calculate the following:

1. $\frac{d(x ^2 + xy - \ln(y))}{dy}$
1. $\int_0^5 e^{2x}\;dx$

```python
print("1.")
x,y,xy = sym.symbols('x,y,xy')

sym.diff(x**2+x*y-sym.ln(y), y)
```

```python
print("2.")

sym.integrate(sym.E**(2*x), (x, 0, 5))
```

### Exercise 3.2
Solve the following definate integral
$$\int\limits_{ - \infty }^\infty  {\frac{1}{{\sigma \sqrt {2\pi } }}{e^{ - \frac{1}{2}{{\left( {\frac{{x - \mu }}{\sigma }} \right)}^2}}}}$$

Hint, the sympy symbol for infinity is `oo`

```python
# Your code here
o,u,oo = sym.symbols('o,u,oo')

sym.integrate((1/(o*sym.sqrt(2*sym.pi)))*sym.E**(-0.5*(x-u/o)**2), (x, -oo, oo))
```

Lookup Gaussian functions: https://en.wikipedia.org/wiki/Gaussian_function
Does your answer maake sense?

## 4. Plotting with Sympy

Finally Sympy can be used to plot functions. Note that this makes use of [matplotlib](http://matplotlib.org/).

Let us plot $x^2$:

```python
expr = x ** 2
p = sym.plot(expr)
```

### Exercise 4.1 Plot the following function:

1. $y=x + cos(x)$
1. ${\frac{1}{{ \sqrt {2\pi } }}{e^{ - \frac{x^2}{2}}}}$

```python
# Your code here
print("1.")
expr = x+sym.cos(x)
p = sym.plot(expr)
```

```python
print("2.")
expr = (1/sym.sqrt(2*sym.pi))*sym.E**(-(x**2)/2)
p = sym.plot(expr)
```

# Lecture

## L1. Hydrogen Atom

Sympy has built in modules for the eigenfunctions of the hydrogen atom.

```python
import sympy.physics.hydrogen
import numpy as np
```

You can caluclate the eigenvalues ($E$) in Hartrees

`sym.physics.hydrogen.E_nl(n,Z)`

```python
sym.physics.hydrogen.E_nl(1,1)
```

We can use a loop to print out many energies

```python
for n in range(1,5):
    print(sym.physics.hydrogen.E_nl(n,1))
```

    -1/2
    -1/8
    -1/18
    -1/32

We can plot the hydrogen radial wavefunction (1s orbital)

```python
r = sympy.symbols('r')
sympy.physics.hydrogen.R_nl(1, 0, r, 1)
```

```python
sym.plot(sympy.physics.hydrogen.R_nl(1, 0, r, 1),(r,0,10.50))
```

And the probablity distribution function

```python
sympy.symbols('r')
prob_1s=sympy.physics.hydrogen.R_nl(1, 0, r, 1)*sympy.physics.hydrogen.R_nl(1, 0, r, 1)
prob_1s
```

```python
sym.plot(prob_1s,(r,0,10))
```

Plot a 2s orbital

```python
sympy.symbols('r')
prob_2s=sympy.physics.hydrogen.R_nl(2, 0, r, 1)*sympy.physics.hydrogen.R_nl(2, 0, r, 1)
prob_2s
```

```python
sym.plot(prob_2s,(r,0,10))
```

We can change the range to see the node better.

```python
sym.plot(prob_2s,(r,1,8))
```

Notice the node!

### Exercise L1.1

Plot the radial distriubution function for a 2p, 3s, 4s, and 3d orbital.

```python
# Your code here
print("2p orbtial")
sympy.symbols('r')
prob_2p=sympy.physics.hydrogen.R_nl(2, 1, r, 1)*sympy.physics.hydrogen.R_nl(2, 1, r, 1)
prob_2p
```

```python
sym.plot(prob_2p,(r,0,10))
```

```python
print("3s orbtial")
sympy.symbols('r')
prob_3s=sympy.physics.hydrogen.R_nl(3, 0, r, 1)*sympy.physics.hydrogen.R_nl(3, 0, r, 1)
prob_3s
```

```python
sym.plot(prob_3s,(r,1,8))
```

```python
print("4s orbtial")
sympy.symbols('r')
prob_4s=sympy.physics.hydrogen.R_nl(4, 0, r, 1)*sympy.physics.hydrogen.R_nl(4, 0, r, 1)
prob_4s
```

```python
sym.plot(prob_4s,(r,1,8))
```

```python
print("3d orbtial")
sympy.symbols('r')
prob_3d=sympy.physics.hydrogen.R_nl(3, 2, r, 1)*sympy.physics.hydrogen.R_nl(3, 2, r, 1)
prob_3d
```

```python
sym.plot(prob_3d,(r,0,20))
```

```python

```


```
%matplotlib inline
```

Given the coin-tossing experiment with unknown parameter $p$. The individual coin-flips are Bernoulli distributed. As we discussed previously, the estimator for $p$ is the following:

$ \hat{p}   = \frac{1}{n} \sum_{i=1}^n X_i$

We have to separate hypotheses: $H_0$ is the so-called null hypothesis. In our case this can be

$$ H_0 : p \lt \frac{1}{2}$$

and the alternative hypothesis is then

$$ H_1 : p \ge \frac{1}{2}$$

To choose between these, we need a statistical test that is a function, $G$, of the sample set
$\mathbf{X}_n=\left\{ X_i \right\}_n $. This test will have a threshold, something like

$$ G(\mathbf{X}_n) < c \implies H_0 $$

and otherwise choose $H_1$. To sum up at this point, we have the observed data $\mathbf{X}_n$ and a function $G$ that will somehow map that data onto the real line. Then, using the constant $c$ as a threshold, the inequality effectively divides the real line into two parts, one corresponding to each of the hypotheses.

Whatever this test $G$ is, it will make mistakes of two types -- false negatives and false positives. The false positives arise from the case where we declare $H_0$ when the test says we should declare $H_1$. Here are the false positives (i.e. false alarm):

$$ \mathbb{P}_{FA} = \mathbb{P}( G(\mathbf{X}_n) \gt c \vert p \le \frac{1}{2})$$

Or, alternatively,

$$ \mathbb{P}_{FA} = \mathbb{P}( G(\mathbf{X}_n) \gt c \vert H_0)$$

Likewise, the other error is a false negative, which we can write analogously as

$$ \mathbb{P}_{FN} = \mathbb{P}( G(\mathbf{X}_n) \lt c \vert H_1)$$

Thus, by choosing some acceptable values for either of these errors, we can solve for the other one. Later we will see that this is because these two quantities are co-dependent. The practice is usually to pick a value of $ \mathbb{P}_{FA}$ and then find the corresponding value of $ \mathbb{P}_{FN}$. Note that it is traditional to speak about *detection probability*, which is defined as

$$ \mathbb{P}_{D} = 1- \mathbb{P}_{FN} = \mathbb{P}( G(\mathbf{X}_n) \gt c \vert H_1)$$

In other words, this is the probability of declaring $H_1$ when the test exceeds the threshold. This is Otherwise known as the *probability of a true detection*.

## Back to the coin flipping example

In our previous problem, we wanted to derive an estimator for the underlying probability for the coin flipping experiment. For this example, we want to ask a softer question: is the underlying probability greater or less than 1/2. So, this leads to the two following hypotheses:

$$ H_0 : p \lt \frac{1}{2}$$

versus,

$$ H_1 : p \ge \frac{1}{2}$$
5
Let's suppose we want to determine is based upon five observations. Now the only ingredient we're missing is the $G$ function and a way to pick between the two hypotheses. Out of the clear blue sky, let's pick the number of heads observed in the sequence of five observations. Thus, we have

$$ G (\mathbf{X}_5) := \sum_{i=1}^5 X_i$$

and, suppose further that we pick $H_1$  only if exactly five out of five observations are heads. We'll call this the *all-heads* test.

Now, because all of the $X_i$ are random variables, so is $G$. Now, to find the corresponding probability mass function for this.  As usual, assuming independence, the probability of five heads is $p^5$. This means that the probability of rejecting the $H_0$ hypothesis (and choosing $H_1$, because there are only two choices here) based on the unknown underlying probability is $p^5$. In the parlance, this is known and the *power function* as in denoted by $\beta$ as in

$$ \beta(\theta) = \theta^5 $$

where I'm using the standard $\theta$ symbol to represent the underlying parameter ($p$ in this case). Let's get a quick plot this below.

```
px = linspace(0,1,50)
plot( px, px**5)
xlabel(r'$\theta$',fontsize=18)
ylabel(r'$\beta$',fontsize=18)
```

Now, to calculate the

$$ \mathbb{P}_{FA} = \mathbb{P}( G(\mathbf{X}_n)= 5 \vert H_0)$$

or, in other words,

$$ \mathbb{P}_{FA}(\theta) = \mathbb{P}( \theta^5 \vert H_0)$$

Notice that this is a function of $\theta$, which means there are many false alarm probability values that correspond to this test. To be on the conservative side, we'll pick the maximum (or, supremum if there are limits involved) of this function, which is known as the *size* of the test, traditionally denoted by $\alpha$.

$$ \alpha = \sup_{\theta \in \Theta_0} \beta(\theta) $$

which in our case is

$$ \alpha = \sup_{\theta < \frac{1}{2}} \theta^5 = (\frac{1}{2})^5 = 0.03125$$

which is a nice, low value. Likewise, for the detection probability,

$$ \mathbb{P}_{D}(\theta) = \mathbb{P}( \theta^5 \vert H_1)$$

which is again a function of the parameter $\theta$. The problem with this test is that the $\mathbb{P}_{D}$ is pretty low for most of the domain of $\theta$. Useful values for $\mathbb{P}_{D}$ are usually in the nineties, which only happens here when $\theta \gt 0.98$. Ideally, we want a test that is zero for the domain corresponding to $H_0$ (i.e. $\Theta_0$) and equal to one otherwise. Unfortunately, even if we increase the length of the observed sequence, we cannot escape this effect with this test. You can try plotting $\theta^n$ for larger and larger values of $n$ to see this.

### Majority Vote Test

Due to the problems with the detection probability we uncovered in the last example, maybe we can think of another test that will have the performance we want. Suppose we reject $H_0$ if the majority (i.e. more than half) of the observations are heads. Then, using the same reasoning as above, we have

$$ \beta(\theta) = \sum_{k=3}^5 \binom{5}{k} \theta^k(1-\theta)^{5-k} $$

Using some tools from `sympy`, we can plot this out and compare it to the previous case as in the cell below.

```
from sympy.abc import p,k # get some variable symbols
import sympy as S

expr=S.Sum(S.binomial(5,k)*p**(k)*(1-p)**(5-k),(k,3,5)).doit()
p0=S.plot(p**5,(p,0,1),xlabel='p',show=False,line_color='b')
p1=S.plot(expr,(p,0,1),xlabel='p',show=False,line_color='r',legend=True,ylim=(0,1.5))
p1.append(p0[0])
p1.show()
```

In this case, the new test has *size*

$$ \alpha = \sup_{\theta < \frac{1}{2}} \theta^{5} + 5 \theta^{4} \left(- \theta + 1\right) + 10 \theta^{3} \left(- \theta + 1\right)^{2}
 = (\frac{1}{2})^5 = 0.5 $$

which is a pretty bad false alarm probability. Values less that one percent are usually considered acceptable. As before we only get to upwards of 90% for detection probability only when the underlying parameter $\theta > 0.75$. As before, this is not so good. Let's run some simulations to see how this plays out.

### All-heads Test

```
# implement 1st test where all five must be heads to declare underlying parameter > 0.5
from scipy import stats
b = stats.bernoulli(0.8) # true parameter is 0.8 (i.e. hypothesis H_1)
samples = b.rvs(1000).reshape(-1,5) # -1 means let numpy figure out the other dimension i.e. (200,5)
print 'prob of detection = %0.3f'%mean(samples.sum(axis=1)==5) # approx 0.8**3
```

    prob of detection = 0.405

```
# here's the false alarm case
b = stats.bernoulli(0.3) # true parameter is 0.3 (i.e. hypothesis H_0)
samples = b.rvs(1000).reshape(-1,5)
print 'prob of false alarm = %0.3f'%mean(samples.sum(axis=1)==5)
```

    prob of false alarm = 0.000

The above two cells shows that the false alarm probability is great, but the detection probability is poor. Let's try the same simulation for the majority vote test.

### Majority Vote Test

```
# implement majority vote test where three of five must be heads to declare underlying parameter > 0.5
b = stats.bernoulli(0.8) # true parameter is 0.8 (i.e. hypothesis H_1)
samples = b.rvs(1000).reshape(-1,5)
print 'prob of detection = %0.3f'%mean(samples.sum(axis=1)>=3)
```

    prob of detection = 0.935

```
# here's the false alarm case
b = stats.bernoulli(0.3) # true parameter is 0.3 which means it's hypothesis H_0
samples = b.rvs(1000).reshape(-1,5)
print 'prob of false alarm = %0.3f'%mean(samples.sum(axis=1)>=3)
```

    prob of false alarm = 0.165

Both of the simulation results follow our earlier analysis. Try tweaking the underlying parameter (i.e. 0.8, 0.3) here and see how the simulation reacts. Not surprisingly, the majority vote test does better when the underlying parameter is much greater than 0.5

## P-Values

 As we have seen there are a lot of moving parts in hypothesis testing. What we need is a simple way to statistically report the findings. The idea is that we want to find the minimum level at which the test rejects $H_0$. Thus, the p-value is the probability, under $H_0$, that the test-statistic is at least as extreme as what was actually observed.  Informally, this means that smaller values imply that $H_0$ should be rejected, although this doesn't mean that large values imply that $H_0$ should be retained. This is because a large p-value can arise from either $H_0$ being true or the test having low statistical power.

If $H_0$ happens to be true, the p-value is like a uniformly random draw from the interval $
(0,1)$. If $H_1$ is true, the distribution of the p-value will concentrate closer to zero. For continuous distributions, this can be proven rigorously and implies that if we reject $H_0$ when the corresponding p-value is less than $\alpha$, then the probability of a false alarm (a.k.a. type I error) is $\alpha$. Perhaps it helps to formalize this a bit before we get to computing it. Suppose $\tau(X)$ is a test statistic that rejects $H_0$ as it gets bigger. Then, for each sample $x$, corresponding to the data we actually have on-hand, we define

$$ p(x) = \sup_{\theta \in \Theta_0} \mathbb{P}_{\theta}(\tau(X) \ge \tau(x))$$

Here's one way to think about this. Suppose you developed a really controversial study and you are ready to report your results, and someone says that you just got *lucky* and somehow just drew data that happened to correspond to a rejection of $H_0$. What do you do? In a perfect world, someone else would replicate your study (thereby obtaining a different draw) and hopefully also reject $H_0$ just as you did. What p-values provide is a way to addressing this by capturing the odds of just a favorable data-draw. Thus, suppose that your p-value is 0.05. Then, what you are showing is that the odds of just drawing that data sample, given $H_0$ is in force, is just 5%. This means that there's a 5% chance that you somehow lucked out and got a favorable draw of data.

Let's make this concrete with an example. Given, the majority-vote rule above, suppose we actually do observe three of five heads. Given the $H_0$, the probability of observing this event is the following:

$$  p(x) =\sup_{\theta \in \Theta_0} \sum_{k=3}^5\binom{5}{k} \theta^k(1-\theta)^{5-k} = \frac{1}{2}$$

For the all-heads test, the corresponding computation is the following:

$$  p(x) =\sup_{\theta \in \Theta_0} \theta^5 = \frac{1}{2^5} = 0.03125$$

From just looking at these p-values, you might get the feeling that the second test is better, but we still have the same detection probability issues we discussed above; so, p-values help in summarizing and addressing aspects of our hypothesis testing, but they do *not* summarize all the salient aspects of the *entire* situation.

## Summary

In this section, we discussed the structure of statistical hypothesis testing and defined the various  terms that are commonly used for this process, along with the illustrations of what they mean in our running coin-flipping example. From an engineering standpoint, hypothesis testing is not as common as confidence-intervals and point estimates, which we will discuss shortly. On the other hand, hypothesis testing is very common in social and medical science, where one must deal with practical constraints that may limit the sample size or other aspects of the hypothesis testing rubric. In engineering, we can usually have much more control over the samples and models we employ because they are typically inanimate objects that can be measured repeatedly and consistently. This is obviously not so with human studies, which generally have many more confounding factors. Nevertheless, as we will see, hypothesis testing can provide another useful tool for examining our own model assumptions within an engineering context.


Index
-----

Python tutorial Ipython / Sympy

- https://arachnoid.com/IPython/equation_processing.html

Continuous Statistical distribution

- https://arachnoid.com/NormalDistribution/index.html

Binomial probability

- https://arachnoid.com/binomial_probability/index.html

polynomial regression

- https://arachnoid.com/sage/polynomial.html
- https://arachnoid.com/polysolve/index.html

Finance

- https://arachnoid.com/finance/index.html#x_ircalc

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sympy import symbols, sqrt, Rational, lambdify, init_printing, solve, symbols, Subs

%matplotlib inline
init_printing(use_latex=True)
```

solve
-----
symbols
-------

```python
a,b,c,d,x  = symbols('a b c d x')
solve(a*x**2+b*x+c,x)
```

Expressions
-----------

- sum
- product

```python
from sympy import(    diff,
    integrate,
    Integral,
    Sum,
    Product,
    Eq )

from IPython.display import *
from sympy.plotting import (
    plot, plot_implicit)

x, y, a, b, c = symbols("x y a b c")

expr = x**2*a + y**2*b
expr = expr.subs(a, 2)
expr = expr.subs(b, 2)

display(expr)

eq1 = Eq(expr, 5)
p1 = plot_implicit(eq1,  (x, -3, 3), (y, -3, 3))

```

```python
display(Sum(expr, (x,1,10)))
display(Product(expr, (x,0,10)))

display (solve(expr))
```

substitution with matrix equation
---------------------------------

```python
from sympy import Matrix

b11, b12, b13, b21, b22, b23, b31, b32, b33 = symbols ('b11 b12 b13 b21 b22 b23 b31 b32 b33')
c11, c21, c31 = symbols('c11 c21 c31')

mat_symbolic_1 = Matrix([[b11, b12, b13],[b21, b22, b23],[b31, b32, b33]])
mat_symbolic_2 = Matrix([[c11],[c21],[c31]])

mat_symbolic_1 * mat_symbolic_2
```

```python
mat = Matrix([[1.0, 2.0],[-(1.0 / 2.0), 1.0]])

print ('get eigenValues')
pprint ( Matrix(mat).eigenvals())

print ('\nget eigenVector')
pprint (Matrix(mat).eigenvects())

mat
```

Diff
----

```python
from sympy import Function, Eq, dsolve,pprint, cos, sin, Lambda, Derivative, var, latex

x,y,z,t = symbols('x y z t')
F = Function('F')

f = Eq(x**2 - x, 0)
f2 = Eq(x**2 + 2, x)
pprint (f)
pprint (f2)
```

     2
    x  - x = 0
     2
    x  + 2 = x

```python
a,b,k = symbols('a b k')

display(cos(x*y) - sin(z*x))
display(Derivative(f, x))
display(Derivative(f, x).doit())
display(Derivative(f, x, 2))

```

```python
display(Integral(f,x))
display(Integral(f, (x, a,b)))
display(Integral(f, (x, a,b)).doit())

```

```python
eq_1 = Eq(F(x).diff(x), x / F(x))
display(eq_1)
eq_solved = dsolve(eq_1)
display(eq_solved)
```

```python
var('a b t k C1')
u = Function("u")(t)
de = Eq(u+u.diff(t) * k)
display(de)

des = dsolve(de,u)
display(des)
des = des.subs(C1,0)
display(des)
```

```python
f = Lambda((t,k,a,b),(a-b) * des.rhs + b)
display(Latex('$f(t,k,a,b) = ' + str(latex(f(t,k,a,b))) + '$'))
x = np.linspace(0,5,100)

```

$f(t,k,a,b) = b + \left(a - b\right) e^{- \frac{t}{k}}$

```python
plt.grid(True)
plt.xlabel('Time t seconds',fontsize=12)
plt.ylabel('$f(t,1,4,8)$',fontsize=16)
plt.plot(x,[f(t,1,4,8) for t in x],color='#008000')
plt.show()
```

```python
var('a b t k C1')
u = Function("u")(t)
de = Eq(u+u.diff(t) * k)
des = dsolve(de,u).subs(C1,0)
f = Lambda((t,k,a,b),(a-b) * des.rhs + b)
f
```

```python
fig = plt.gcf()
fig.set_size_inches(8,5)
x = np.linspace(0,5,100)
plt.grid(True)
plt.xlabel('Time t seconds',fontsize=12)
plt.ylabel('Voltage',fontsize=12)
plt.title('R-C circuit decay profile',fontsize=16)
r = 10000 # resistor value Ω
c = 100e-6 # capacitor value farads
plt.text(2.1,7,'R = %.1f $\Omega$' % r,fontsize=14)
plt.text(2.1,6.3,'C = %.1f $\mu$f' % (c * 1e6),fontsize=14)
plt.plot(x,[f(t,r * c,0,10) for t in x],color='#008000')
plt.show()

```

```python

```

Limit
-----

```python
from sympy import Limit, sin, oo
x = symbols('x')

print('Range approaching: ', Limit(sin(x) / x, x, 0).doit())
Limit(sin(x) / x, x, 0)
```

```python
print('Range approaching: ', Limit(1/x, x, oo).doit())
Limit(1/x, x, oo)
```

```python
print('Range approaching: ', Limit(1/x, x , 0, dir = '+').doit())
Limit(1/x, x , 0, dir = '+')
```

```python
print('Range approaching: ', Limit(1/x, x , 0, dir = '-').doit())
Limit(1/x, x , 0, dir = '-')
```

```python
print('Range approaching: ', Limit(1/x**2, x , - oo).doit())
Limit(1/x**2, x , - oo)
```

building expressions
--------------------
Lambdify
--------
pandas
------

```python
from sympy import symbols, sqrt, Rational, lambdify

x, y = symbols("x y")
list1 = np.arange(1,1000)
list2 = pd.Series(list1)

def sympy_expr(x_val):
    expr = x**2 + sqrt(3)*x - Rational(1,3)
    return expr.subs(x, x_val)

expr_1 = [sympy_expr(item) for item in list2]
#print (expr_1)

%timeit [expr_1]

lambdifyFunction = f = lambdify(x, expr_1)

%timeit lambdifyFunction(list1)
%timeit lambdifyFunction(list2)
```

    10000000 loops, best of 3: 101 ns per loop
    10000 loops, best of 3: 206 µs per loop
    1000 loops, best of 3: 205 µs per loop

```python

```

plotting
--------
- https://arachnoid.com/IPython/tips_tricks.html

class sympy.plotting.plot.BaseSeries[source]

Base class for the data objects containing stuff to be plotted.

The backend should check if it supports the data series that it’s given. (eg TextBackend supports only LineOver1DRange). It’s the backend responsibility to know how to use the class of data series that it’s given.

Some data series classes are grouped (using a class attribute like is_2Dline) according to the api they present (based only on convention). The backend is not obliged to use that api (eg. The LineOver1DRange belongs to the is_2Dline group and presents the get_points method, but the TextBackend does not use the get_points method).

class sympy.plotting.plot.Line2DBaseSeries[source]

A base class for 2D lines.

    adding the label, steps and only_integers options
    making is_2Dline true
    defining get_segments and get_color_array

class sympy.plotting.plot.LineOver1DRangeSeries(expr, var_start_end, **kwargs)[source]

Representation for a line consisting of a SymPy expression over a range.

var
---

```python
from sympy import var,E, pi, exp

var('x')
f = Lambda(x,E**-x**2)
x = np.linspace(-3,3,100)
y = np.array([f(v) for v in x],dtype='float')
a = 40
b = 60
```

```python
import numpy as np
import matplotlib.pyplot as plt

from sympy.plotting import (
    plot,
    plot_parametric,
    plot3d,
    plot3d_parametric_line,
    plot3d_parametric_surface
)

%matplotlib inline

var('i')
plot(sin(i),(i,0, 2*pi))
```

```python
var('i, j')
plot3d((exp(-(i**2+j**2))),(i,-3,3),(j,-3,3))
```

```python
fig = plt.gcf()
fig.set_size_inches(8,5)
plt.grid(True)
plt.fill_between(x[:a+1],y[:a+1],0,color='#ffff40',alpha=.3)
plt.fill_between(x[a:b],y[a:b],0,color='#40ff40',alpha=.3)
plt.fill_between(x[b-1:],y[b-1:],0,color='#ffff40',alpha=.3)
plt.plot(x,y,color='black')
plt.show()
```

substitution and plotting
-------------------------

```python
x, b1, b2 = symbols("i b1 b2")

f = x/(x+exp(b1-b2*i))
res = {b1:29.3930964972769,b2:0.327159886574049}

plot(f.subs(res), (i, 0, 100))
```

```python
t = symbols('t')
x = 0.05*t + 0.2/((t - 5)**2 + 2)
lam_x = lambdify(t, x, modules=['numpy'])

x_vals = np.linspace(0, 10, 100)
y_vals = lam_x(x_vals)

plt.plot(x_vals, y_vals)
plt.ylabel("Speed")
plt.show()
```

Latex
-----

```python
from IPython.display import HTML

init_printing(use_latex=True)
```

```python
def wrap_tag(t,d):
   return '<%s style="border-color:white">%s</%s>' % (t,d,t)
desc = (
  'Population at time t','Population at time zero',
  'Time','Population growth rate'
)

# source: http://mathworld.wolfram.com/PopulationGrowth.html
# problem: for a given multi-variable equation, generate all its forms
# create symbols
vl = (N,N_0,t,r) = symbols('N N_0 t r')
# base equation
b = log(N/N_0) - r*t
```

```python
s = 'Variables:'
ss = ''
for n,x in enumerate(vl):
  ss += wrap_tag('td',' ')
  ss += wrap_tag('td','$%s$' % str(x))
  ss += wrap_tag('td',': %s' % desc[n])
  ss = wrap_tag('tr',ss)
s += wrap_tag('table',ss)
s += 'Equations:'
ss = ''
for v in vl:
  soln = solve(b,v)[0]
  ss += wrap_tag('td',' ')
  ss += wrap_tag('td','$%s$' % str(v))
  ss += wrap_tag('td','$\displaystyle = %s$' % latex(soln))
  ss = wrap_tag('tr',ss)
s += wrap_tag('table',ss)
```

```python
def makeLatexToHTML(latex):
    return HTML(latex)
makeLatexToHTML(s)
```

Variables:

 	$N$	: Population at time t
 	$N_0$	: Population at time zero
 	$t$	: Time
 	$r$	: Population growth rate

Equations:

 	$N$	$\displaystyle = N_{0} e^{r t}$
 	$N_0$	$\displaystyle = N e^{- r t}$
 	$t$	$\displaystyle = \frac{1}{r} \log{\left (\frac{N}{N_{0}} \right )}$
 	$r$	$\displaystyle = \frac{1}{t} \log{\left (\frac{N}{N_{0}} \right )}$

```python
[solve(b,v)[0] for v in vl]
```

```python

```


# Introducción

Tanto o módulo **Matplotlib** como **Sympy** teñen multitude de ferramentas para traballar con funcións dunha ou varias variables e, máis precisamente, para representar as gráficas asociadas sobre un conxunto de valores dado. Neste guión da práctica 3, revisaremos como reprensetar funcións escalares dunha, dúas e tres variables usando tanto superficies, como curvas de nivel (tamén chamadas isoliñas), superficies ou isosuperfícies (no caso tridimensional). Adicionalmente tamén se revisará como representar liñas no plano e no espazo tridimensional. En todos estos casos, veremos como facer estas representacións gráficas tanto en **Matplotlib** como en **Sympy**.

## Obxectivos

- Representación de funciones de una variable
- Representación de funcións escalares de varias variables
- Representación gráfica de liñas no espazo tridimensional
- Empregar Matplotlib e Sympy para representar funcións

# Representación gráfica de funcións con Matplotlib

Como xa fixemos na práctica anterior, para usar os módulos **Numpy** e **Matplotlib**, o primeiro que debemos facer é importalos, e posto que usaremos este módulo usando a interface `pyplot` de **Matplotlib**, tamén haberá que importar esta.

```python
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
```

## Gráficas de funcións dunha variable

As gráficas de funcións dunha variable créanse a partir da avaliación do valor da función nun gran número de puntos almacenados nun vector **Numpy**. Ao usar un número de puntos suficiente, a gráfica da función parecerá suave. Nembargante, hai que entender que o que se está a representar graficamente é unha concatenación de segmentos rectos que unen os puntos que se avaliaron. Por exemplo, para representar as funcións:
$$
f(x) = \sin(x),\qquad g(x)=\cos(x),\qquad x\in[0,4\pi].
$$
o primeiro paso será crear un vector de valores nos que se avalie a función. Para iso, empregase o a función `np.linspace` como segue:

```python
# Creación dos puntos onde se avalía a función
x = np.linspace(0, 4*np.pi, 1000)

# Gráfica de sin(x) e cos(x), cunha etiqueta para cada unha
plt.plot(x, np.sin(x), label='sen(x)')
plt.plot(x, np.cos(x), label='cos(x)')

# Etiquetas dos eixes
plt.xlabel('x')
plt.ylabel('y')

# Engadir a lenda (mostrando as etiquetas dos "plot")
plt.legend()

# Definir os límites dos eixes x e y
plt.xlim(x[0], x[-1])
plt.ylim([-1.,1.])

plt.show()
```

## Representación de curvas no plano
Do mesmo xeito a que unha gráfica dunha función $f:\mathbb{R}\to\mathbb{R}$ é o conxunto de puntos do plano $xy$ que satisfacen $y=f(x)$, isto é,
$$
\text{Gráfica}(f)=\{(x,y)\in\mathbb{R}^2:\ y=f(x)\}
$$

Unha curva no plano pódese definir como un conxunto de puntos que satisface a relación:
$$
\text{Curva}=\{(x,y)\in\mathbb{R}^2:\ x=f(t),\ y=g(t)\text{ con }t\in\mathbb{R}\}=\{(f(t),g(t)):t\in\mathbb{R}\},
$$
onde $t$ é o que se denomina parámetro da representación da curva.

Por exemplo, para representar a curva
$$
x=\sin(3t),\ y=\cos(5t),\ t\in[0,2\pi]
$$
empregaríase o mesmo comando `plot`:

```python
# Creación dos puntos onde se avalía a función
t = np.linspace(0, 2*np.pi, 1000)
x_t = np.sin(3*t)
y_t = np.cos(5*t)
# Gráfica da curva
plt.plot(x_t, y_t)
# Etiquetas dos eixes
plt.xlabel('x')
plt.ylabel('y')
plt.axis('square')
plt.show()
```

### Exercicio 4.1
Define tres funcións dunha variable e represéntaas gráficamente no intervalo $[1,2]$ usando cores e estilos de liña diferentes. Para iso consulta a axuda da función `plt.plot`

```python
## O TEU CÓDIGO AQUÍ
```

## Representación de funcións escalares de varias variables

Consideremos a función de dúas variables:
$$
f(x, y) = \sin(x) \cos\left(\frac{xy^2}{10}\right),\qquad (x,y)\in[-\pi,\pi]\times[-\pi,\pi].
$$
Non é doado visualizar a forma desta función. Unha das maneiras de obter unha imaxe do comportamento da función é crear unha representación cun mapa de cores ou con curvas de nivel. Para facer isto, hai que crear unha nube de puntos tanto no eixe $x$ como no $y$, para posteriormente evaluar a función neses puntos. Os puntos creados neste proceso de avaliación serán os que finalmente se empreguen para visualizar a función.

```python
N = 50  # 50 puntos en cada dirección dos eixes cartesianos
x = np.linspace(-np.pi, np.pi, N)
y = np.linspace(-np.pi, np.pi, N)
```

O seguinte paso é crear unha grella (ou nube) de puntos $(x, y)$, para posteriormente avaliar a función:

```python
X, Y = np.meshgrid(x, y)
f = np.sin(X)*np.cos((X*Y**2)/10)
```

A primeira posibilidade para representar a función é usando un mapa de cores mediante a función de **Matplotlib** `plt.contourf`:

```python
plt.contourf(X, Y, f)
plt.colorbar()  # Engádese a barra de cores cos valores asociados
# Etiquetas dos eixes
plt.xlabel('x')
plt.ylabel('y')
plt.axis('square')
plt.show()
```

Para crear unha representación gráfica coas liñas de nivel, usaríase `plt.contour`:

```python
p = plt.contour(X, Y, f)
plt.clabel(p) # Engádese o valor numérico a cada curva de nivel
# Etiquetas dos eixes
plt.xlabel('x')
plt.ylabel('y')
plt.axis('square')
plt.show()
```

A función `plt.contour` admite o argumento opcional `level` co que se pode indicar o número de curvas de nivel a representar ou fixar os valores das curvas a representar.

Outra maneira de representar as funcións de dúas variables é usar unha gráfica tridimensional con eixes $x$, $y$, e $z=f(x,y)$. Para crear unha gráfica tridimensional en **Matplotlib**, é necesario incluir módulos extra específicos do espazo 3D:

```python
from mpl_toolkits.mplot3d import axes3d
# Creación da figura 3D
fig = plt.figure()
ax = plt.axes(projection="3d")
# Representación da superficie z=f(x,y)
surf = ax.plot_surface(X, Y, f)
# Etiquetas dos eixes
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
# Punto de vista do observador: elevación
ax.view_init(elev=45)
plt.show()
```

>**Nota**: No caso de funcións de tres variables, non é posible representar un mapa de cores. O concepto equivalente de curvas de nivel (isoliñas) son as isosuperficies, coas que xa traballamos na práctica 2 anterior (para representar superficies cuádricas coa función `isosurface`), pero que tamén podería ser empregadas para representar calquera outra isosuperficie dunha función arbitraria.

### Exercicio 4.2
Define unha función de dúas variable e represéntaa gráficamente na rexión do plano $[1,2]\times[0,1]$ empregando mapas de cores, curvas de nivel e superficies.

```python
## O TEU CÓDIGO AQUÍ
```

## Representación de curvas no espazo tridimensional
De forma análoga no espazo tridimensional, unha curva ven definida por tres funcións diferentes:
$$
\text{Curva}=\{(x,y)\in\mathbb{R}^2:\ x=f(t),\ y=g(t),\ z=h(t)\text{ con }t\in\mathbb{R}\}=\{(f(t),g(t),h(t)):\ t\in\mathbb{R}\},
$$
onde $t$ é o que se denomina parámetro da representación da curva.

Por exemplo, para representar graficamente a curva definida
$$
\begin{align*}
f(t)&=(1+0.2\cos(10t))\cos(t),\\
g(t)&=(1+0.2\cos(10t))\sin(t),\\
h(t)&=2t+0.2\sin(10t),
\end{align*}
$$
con $t\in[0,8\pi]$, usaremos a función `plot3D` de **Matplotlib** como segue:

```python
# Creación da figura 3D
fig = plt.figure()
ax = plt.axes(projection="3d")
# Creación dos puntos da curva
t = np.linspace(0., 8.*np.pi, 1000)
x_t = (1+0.2*np.cos(10*t))*np.cos(t)
y_t = (1+0.2*np.cos(10*t))*np.sin(t)
z_t = 2*t+0.2*np.sin(10*t)
# Representación gráfica da curva 3D
p = ax.plot3D(x_t, y_t, z_t)
# Etiquetas dos eixes
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
plt.show()
```

# Representación gráfica con Sympy

Un dos inconvenientes de traballar con **Matplotlib** é que sempre é necesario empregar un conxunto de puntos onde se avalían as funcións que se queren representar e polo tanto, as gráficas non serán máis que aproximacións do que en realidade se quería representar xa foron estas gráficas de funcións, curvas ou superficies.

No que segue, veremos como se pode representar graficamente con **Sympy** os mesmos exemplos revisados anteriormente.

```python
import sympy as sp
```

### Gráficas de funcións dunha variable

A diferenza con respecto ao uso de **Matplotlib**, en vez de traballar con vectores de números **Numpy**, haberá que definir variables simbólicas para escribir a expresión da función que se quere representar:

```python
x = sp.symbols('x', real=True)
p = sp.plot(sp.sin(x), sp.cos(x), (x, 0, 4*sp.pi), show=False)
p[0].line_color='r'
p[1].line_color='b'
p.xlabel='x'
p.ylabel='y'
p.legend=True
p.show()
```

### Representación de curvas no plano

```python
t = sp.symbols('t', real=True)
p = sp.plotting.plot_parametric(sp.sin(3*t), sp.cos(5*t), (t, 0, 2*sp.pi), show=False)
p.xlabel='x'
p.ylabel='y'
p.show()
```

### Representación de funcións escalares de varias variables

Non é posible representar graficamente mapas de cores ou curvas de nivel con **Sympy**. A única opción posible para representar funcións de dúas variables é representar a superficie $z=f(x,y)$ como segue:

```python
x, y = sp.symbols('x y', real=True)
f = sp.sin(x)*sp.cos((x*y**2)/10)
p = sp.plotting.plot3d(f, (x, -sp.pi, sp.pi), (y, -sp.pi, sp.pi), show=False)
p.xlabel='x'
p.ylabel='y'
p.zlabel='z'
p.show()
```

### Representación de curvas no espazo tridimensional

```python
t = sp.symbols('t', real=True)
xt = (1+sp.S(0.2)*sp.cos(10*t))*sp.cos(t)
yt = (1+sp.S(0.2)*sp.cos(10*t))*sp.sin(t)
zt = 2*t+sp.S(0.2)*sp.sin(10*t)
p = sp.plotting.plot3d_parametric_line(xt, yt, zt, (t, 0, 8*sp.pi), show=False)
p.xlabel='x'
p.ylabel='y'
p.zlabel='z'
p.show()
```

### Exercicio 4.3
A partir das funcións representadas nos exercicios 4.1 e 4.2, fai as mesmas representacións gráficas con **Sympy** (naqueles casos onde sexa posible).

```python
## O TEU CÓDIGO AQUÍ
```


# 02 - Kernel methods and SVMs
** APSTA - Ecole Centrale Nantes **

** Diana Mateus **

PARTICIPANTS: **(Fill in your names)**

```python

import matplotlib.pyplot as plt
import pandas
import numpy as np
import os
```

## 1. Kernel Ridge Regression

Ridge regression is an extension to ordinary least squares by adding a regularization
term to the loss function. It is defined as
\begin{equation}
\min_{\mathbf{w}} \sum_{i=1}^n (y_i -  \mathbf{x}_i^T \mathbf{w} )^2
+ \lambda \lVert \mathbf{w} \rVert_2^2 ,
\end{equation}
where the value of $\lambda > 0$ determines the amount of regularization. In this exercise we will rely on the derivation of Support Vector Machines to extend Ridge regression with the ```Kernel Trick```

**a.** Replace $\mathbf{w}$ with $\sum_{i=1}^n \alpha_i \mathbf{x}_i$

**b.** As in support vector machines, we can use the Kernel trick to make ridge regression
non-linear and at the same time avoid explicitly transforming features. Specify $k(\mathbf{x}, \mathbf{x}^\prime) = \phi(\mathbf{x})^T\phi(\mathbf{x}^\prime)$, to derive the objective function of Kernel Ridge Regression.

**c.** Derive the the solution for the $\alpha$

**d.** How would you use the result to make a new prediction?

**e.** What are the main similarities and differences of KRR with the classification SVM derived in class?

## 2. Wine quality prediction

The wine quality dataset comes from the UCI Machine Learning Repository http://archive.ics.uci.edu/ml/index.php, and contains measurements and opinions for different variants of red and white wine. The goal of this part of the exercise  is to build a model capable to _predict the quality of a wine from the measurements_.

To this end, implement your own version of Kernel Ridge Regresssion and compare it with the in-built SVR function from sklearn

**a.** Run the ```Load and process``` block bellow to load the dataset into the ``wines_backup.csv`` file

```python
# a) Load and Process
# Saves the result in a single file in order to avoid downloading many times.
# Shows the first 5 lines of the table

if not os.path.exists("wines_backup.csv"):
    # if not exist, we create wines.csv which combines red and white wines into a single file
    columns = ["facidity", "vacidity", "citric", "sugar", "chlorides", "fsulfur",
               "tsulfur", "density", "pH", "sulphates", "alcohol", "quality"]
    red = pandas.read_csv("http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv",
                         names=columns, sep=";", skiprows=1)
    white = pandas.read_csv("http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv",
                         names=columns, sep=";", skiprows=1)
    red["color"] = "red"
    white["color"] = "white"
    wines = pandas.concat([white, red])
    wines.to_csv("wines_backup.csv", sep="\t", index=False)
else:
    wines = pandas.read_csv("wines_backup.csv", sep="\t")

wines.head(5)
```

**b. Split the dataset into train (80% of samples) and test (20% samples)**. Use the in-built sklearn function
```train_test_split```

``
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=.2,random_state=3)
``

```python
#b) Data split

from sklearn.model_selection import train_test_split

#build the data matrix from a subset of the available variables in the cvs file
X = wines.as_matrix(["facidity", "vacidity", "citric", "sugar", "chlorides", "fsulfur",
               "tsulfur", "density", "pH", "sulphates", "alcohol"])

#make y the target value we want to predict
y = wines.as_matrix(['quality']).ravel()

print(X.shape, y.shape)

#Split the dataset

X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=.2,random_state=3)

print(X_train.shape, y_train.shape)
print(X_test.shape, y_test.shape)

```

**c.  Implement your own version of Kernel Ridge Regressor**. To this end
- fit the model parameters using the training data, with a radial basis function as kernel
- make predictions for the test data

```python
#c) Fitting model and comparing to sklearn KernelRidge
import scipy
from scipy.spatial.distance import cdist, pdist, squareform
from numpy.linalg import inv

#Example values for the regularization and RBF variance hyper-parameters
p_lambda = 0.1
p_gamma = 0.01

#RBF kernel for two matrices of points
def gkernel(X1, X2, gamma):
    pairwise_dists = np.square(cdist(X1,X2))
    K = np.exp(-gamma* pairwise_dists)
    return K

# Build the kernel matrix for the training set
K =
print(K.shape)

# Find the optimal alpha values with the closed form solution from 1.
alpha =
print(alpha.shape)

# Find the kernel values for the test set
K =
print(K.shape)

# Make predictions
y_mine =
print(y_mine.shape)

```

**d.** Compare your results those of the in-built SVR function (in terms of the mean squared error) for the same values of regularization ($\lambda$) and radial-basis function ($\gamma$) hyper-parameters.

```python
# d) Compute the mean squared error errors

from sklearn.metrics import mean_squared_error, r2_score
from sklearn.kernel_ridge import KernelRidge

print("Mean squared error MINE: %.2f"% mean_squared_error(y_mine, y_test))
print(y_mine)

# built-in version with KernelRidge
FILL IN HERE

print("Mean squared error: %.2f"% mean_squared_error(y_kr, y_test))
print(y_kr)

```

```python

```

```python

```

```python

```

```python

```