# The IS-LM-PC model

```python
import numpy as np
import sympy as sm
from scipy import optimize
from IPython.display import display, Markdown
import matplotlib.pyplot as plt
import sympy
import ipywidgets as widgets
```

The IS-LM-PC model is a basic macroeconomic model which has replaced the AS-AD model in some newer macroeconomic textbooks. It describes the medium run by combining the IS-LM model with the Phillips curve, thus creating a relation between inflation, unemployment, the output gap, and interest rate policy.

We define first the **variables**, **parameters** and **equations** of the IS-LM model:
* $Y$ is output (or income)
* $C$ is consumption
* $c_0$ is the part of consumption that is independent of income
* $c_1$ is the marginal propensity to consume
* $T$ is taxes net of transfers
* $t_0$ is the part of taxes that is independent of income
* $t_1$ is the tax rate on income
* $I$ is investment
* $b_0$ is the part of investment that is independent of income and the interest rate
* $b_1$ is the income sensitivity of investment
* $b_2$ is the interest rate sensitivity of investment
* $G$ is government spending
* $r$ is the real interest rate

$$
\begin{aligned}
Y=C+I+G\\
C=c_{0}+c_{1}*(Y-t_{0}-t_{1}Y)\\
T=t_{0}+t_{1}Y\\
I=b_{0}+b_{1}Y-b_{2}r\\
\end{aligned}
$$

We start out by defing variables, parameters and the equations for Y, C, T,I symbolically.

```python
y = sm.symbols('Y')
c = sm.symbols('C')
i = sm.symbols('I')
g = sm.symbols('G')
t = sm.symbols('T')
i_n = sm.symbols('i')
r = sm.symbols('r')
c0 = sm.symbols('c_0')
c1 = sm.symbols('c_1')
t0 = sm.symbols('t_0')
t1 = sm.symbols('t_1')
b0 = sm.symbols('b_0')
b1 = sm.symbols('b_1')
b2 = sm.symbols('b_2')
r_bar = sm.symbols("rbar")

eqC = sm.Eq(c, (c0+c1*(y-t0-t1*y)))
eqT = sm.Eq(t, (t0+t1*y))
eqI = sm.Eq(i, (b0+b1*y-b2*r))

eq_t = t0+t1*y
eq_c = c0+c1*y-eq_t
eq_i = b0+b1*y-b2*r
eq_y = sm.Eq(y, (eq_c+eq_i+g))
```

We now solve the system of equations to find an expression for Y, which expess the IS curve.

```python

c_func = sm.lambdify((c0,c1,t0,t1,y),sm.solve(eqC,c)[0])
t_func = sm.lambdify((t0,t1,y),sm.solve(eqT,t)[0])
i_func = sm.lambdify((b0,b1,b2,r,y),sm.solve(eqI,i)[0])

y_func = sm.lambdify((c0,c1,t0,t1,b0,b1,b2,r,g),sm.solve((eq_y), y)[0])
y_solve = sm.solve((eq_y), y)[0]
display(Markdown('$IS: Y$ = {}'.format(sympy.latex(y_solve, mode='inline'))))
```

$IS: Y$ = $\frac{- G - b_{0} + b_{2} r - c_{0} + t_{0}}{b_{1} + c_{1} - t_{1} - 1}$

The central bank sets the interest rate directly, such that the real interest rate equals $r=\bar{r}$. The LM curve in this model is thus simply:

$$
\begin{aligned}
LM: r=\bar{r}\\
\end{aligned}
$$

Combining the IS curve and the LM curve gives the equilibrium level of output:

Thus, we substitute $\bar{r}$ into the expression we found for the IS curve above:

```python
y_equi = y_func(c0,c1,t0,t1,b0,b1,b2,r_bar,g)
display(Markdown('Equilibrium level of output = {}'.format(sympy.latex(y_equi, mode='inline'))))
```

Equilibrium level of output = $\frac{- G - b_{0} + b_{2} \bar{r} - c_{0} + t_{0}}{b_{1} + c_{1} - t_{1} - 1}$

We then define globals for the parameters (and G) and then solve for the output again

```python
c0 = 100
c1 = 0.8
t0 = 100
t1 = 0.4
b0 = 100
b1 = 0.1
b2 = 10
g= 500

```

```python
y_num = y_func(c0,c1,t0,t1,b0,b1,b2,r_bar,g)
display(Markdown('$Y$ = {}'.format(sympy.latex(y_num, mode='inline'))))
```

$Y$ = $1200.0 - 20.0 \bar{r}$

We plot this in a figure. We also plot the LM curve for $\bar{r}$ = 10. The intersection between the two curves describes the equilibrium.

```python
# Number of values in grid
N = 100

r_vec = np.linspace(0,20,N)
y_vec = np.empty(N)
rbar_vec = np.ones(N)*10 # interest rate = 10

for i,r_bar_plot in enumerate(r_vec):
    r_bar_plot = r_vec[i]
    y_vec[i] = y_func(c0,c1,t0,t1,b0,b1,b2,r_bar_plot,g)

# creates the figures
plt.style.use("seaborn")

#plt.figure()
#plt.subplot(211)
#plt.plot(y_vec,r_vec,'r',y_vec,rbar_vec,'g')

#plt.subplot(212)
#plt.plot(y_vec,pi_vec,'b')

fig = plt.figure(figsize=(10,8))

fig_ISLM = fig.add_subplot(2,1,1)
fig_ISLM.plot(y_vec,r_vec, label='IS')
fig_ISLM.plot(y_vec,rbar_vec, label='LM')
fig_ISLM.set_xlim([800,1200])
plt.legend()

fig_ISLM.set_title('IS-LM')
fig_ISLM.set_xlabel('$Y$')
fig_ISLM.set_ylabel('$r$')
fig_ISLM.grid(True)
fig_ISLM.plot(1000,10, 'go')

```

## Adding the Phillips curve

The Phillips curve describes the relationship between the unemployment rate and inflation. We assume the following relationship:
$$
\begin{aligned}
\pi-\pi^{e}=-\alpha*(u-u_{n})\\
\end{aligned}
$$

where
* $\pi$ is inflation
* $\pi^{e}$ is expected inflation
* $u$ is the unemployment rate
* $u_n$ is the natural rate of unemployment (or NAIRU)
* $\alpha$ is some parameter describing the sensitivity of wages to the unemplyment rate

We further assume that inflation expectations are anchored, such that $\pi^{e}$ = $\pi^{a}$. This corresponds to the *classic* or *original* Phillips curve.

We assume a simple production function with labour as the only input, and one unit of labour producing one unit of output, such that output is equal to employment:

$$
\begin{aligned}
Y=L*(1-u)\\
Y_n=L*(1-u_n)\\
\end{aligned}
$$

where $L$ is the size of the labour force, and $Y_n$ is the *potential* or *natural* level of output.

This lets us describe the evolution of inflation as a function of the output gap:

$$
\begin{aligned}
\pi-\pi^{a}=\frac{\alpha}{L}\left(Y-Y_{n}\right)
\end{aligned}
$$

We can thus plot the error in the inflation forecast as a funtion of $Y$, for given values of $\alpha$, the labour force, and potential output.

We define additional symbols and parameters and write up the Phillips curve

```python
y_n = sm.symbols('Y_n')
inf = sm.symbols('pi')
inf_bar = sm.symbols('pibar')
inf_a = sm.symbols('(\pi-\pi^{a})')
alpha = sm.symbols('alpha')
l = sm.symbols('L')

#Eq for the Phillips curve
eq_PC = sm.Eq((inf_a), ((alpha/l)*(y-y_n)))

PC_func = sm.lambdify((inf_a,alpha,l,y_n),sm.solve((eq_PC), y)[0])
PC_solve = sm.solve((eq_PC), y)[0]

print('Thus the phillups curve is given by: ')
display(Markdown('$Y$ = {}'.format(sm.latex(PC_solve, mode='inline'))))
```

    Thus the phillups curve is given by:

$Y$ = $(\pi-\pi^{a}) L / \alpha + Y_{n}$

we assign alpha=0.5 and the labour force (l) with size 1000, to simplify the expression:

```python
alpha_num=0.5
l_num=1000

PC_num = PC_func(inf_a,alpha_num,l_num,y_n)
display(Markdown('$Y$ = {}'.format(sm.latex(PC_num, mode='inline'))))
```

$Y$ = $2000.0 (\pi-\pi^{a}) + Y_{n}$

### Plotting the Phillips curve

For plotting the Phillips curve we set $Y_n$=950

```python
y_n_num=950
```

```python
N = 100

# def plot_lines()

r_vec = np.linspace(0,20,N)
y_vec = np.empty(N)
rbar_vec = np.ones(N)*10 # interest rate = 10
rbar_vec2 = np.ones(N)*12.5 # interest rate = 12.5

inf_vec = np.linspace(0.10,-0.10,N)
pi_vec = np.zeros(N) #line where the inflation expectation error is zero
PC_vec = np.empty(N)

for i,r_bar_plot in enumerate(r_vec):
    r_bar_plot = r_vec[i]
    y_vec[i] = y_func(c0,c1,t0,t1,b0,b1,b2,r_bar_plot,g)

for i,inf_err_plot in enumerate(inf_vec):
    inf_err_plot = inf_vec[i]
    PC_vec[i] = PC_func(inf_err_plot,alpha_num,l_num,y_n_num)

# creates the figures
plt.style.use("seaborn")

fig = plt.figure(figsize=(10,8))

fig_ISLM = fig.add_subplot(2,1,1)
fig_ISLM.plot(y_vec,r_vec) # Plots the IS curve
fig_ISLM.plot(y_vec,rbar_vec,label='LM curve for r=10') # Plots the LM cuve
fig_ISLM.plot(y_vec,rbar_vec2,label='LM curve for r=12.5')
fig_ISLM.set_xlim([800,1200])
plt.legend()

fig_PC = fig.add_subplot(2,1,2)
fig_PC.plot(y_vec,pi_vec)
fig_PC.plot(PC_vec, inf_vec, label='PC curve')
fig_PC.set_xlim([800,1200])
plt.legend()

fig_ISLM.set_title('IS-LM')
fig_ISLM.set_xlabel('$Y$')
fig_ISLM.set_ylabel('$r$')
fig_ISLM.grid(True)

fig_PC.set_title('PC')
fig_PC.set_xlabel('$Y$')
fig_PC.set_ylabel('$\pi-\pi^e$')

fig_ISLM.plot(1000,10,'yo')
fig_ISLM.plot(950,12.5, 'go')
fig_ISLM.annotate('Eq', xy=(1000,10))
```

The PC curve has been drawn for $Y_n$ = 950. We see from the IS-LM diagram that when the real interest rate is 10 pct., actual output is 1000. This results in a positive output gap, leading to inflation being higher than expected inflation by 2.5 pct.

The central bank can bring down the inflation rate by increasing the real interest rate. This is drawn as the red line in the IS-LM diagram.

## Maximizing the utility of the Central Bank

We assume that the central bank cares not only about inflation, but also about unemployment. Specifically, we assume that the central bank adjusts the real interest to maximize its utility, given the function
$$
\begin{aligned}
v=f\left(\pi-\pi^{*},\,u_{t}-u_{n}\right)
\end{aligned}
$$

where $\pi^{*}$ is the central banks target rate for inflation. (This rate is not necessarily equal to the anchored inflation expectation, $\bar{\pi}$.)

The central bank faces the constraints set by the relations between unemployment, inflation and the real interest rate by the IS-LM-PC model.

To make these contraints more clear, we express inflation and the difference between the actual and natural rates of unemployment as functions of the real interest rate.

**Firstly** we find an expression for unemployment:

```python
u = sm.symbols('u')
u_n = sm.symbols('u_n')

eq_u = sm.Eq(y, (l*(1-u)))

u_func = sm.lambdify((y,l),sm.solve((eq_u), u)[0])
u_solve = u_func(y_equi,l)
display(Markdown('$u$ = {}'.format(sm.latex(u_solve, mode='inline'))))
```

$u$ = $\left(L - \frac{- G - b_{0} + b_{2} \bar{r} - c_{0} + t_{0}}{b_{1} + c_{1} - t_{1} - 1}\right) / L$

Given the labour force size of 1000 and the previously used parameters, the expression simplifies to:

```python
u_num = u_func(y_num,l_num)
display(Markdown('$u$ = {}'.format(sm.latex(u_num, mode='inline'))))
```

$u$ = $0.02 \bar{r} - 0.2$

**Secondly** we find an expression for the inflation

```python
inf_targ = sm.symbols('\pi_{targ}^*')

eq_pi = sm.Eq((inf-inf_targ), ((alpha/l)*(y-y_n)))

pi_func = sm.lambdify((inf_targ,alpha,l,y,y_n),sm.solve((eq_pi), inf)[0])
pi_solve = pi_func(inf_targ,alpha,l,y_equi,y_n)
display(Markdown('$\pi$ = {}'.format(sm.latex(pi_solve, mode='inline'))))
```

$\pi$ = $\left(L \pi_{targ}^* - Y_{n} \alpha + \frac{\alpha \left(- G - b_{0} + b_{2} \bar{r} - c_{0} + t_{0}\right)}{b_{1} + c_{1} - t_{1} - 1}\right) / L$

We assume the inflation target to be an inflation rate of 3%, which implies inf_targ = 0.03. Thus the expression for the inflation rate can be simplified to:

```python
inf_targ_num=0.05
u_n_num=0.5
pi_num = pi_func(inf_targ_num,u_n_num,l_num,y_num,y_n_num)
display(Markdown('$\pi$ = {}'.format(sm.latex(pi_num, mode='inline'))))

```

$\pi$ = $0.175 - 0.01 \bar{r}$

**Thirdly** We assume a specific form for the central bank utility function:

$$
\begin{aligned}
v=-75(\pi-\pi^{*})^2-(u_{t}-u_{n})\\
\end{aligned}
$$

**Now** we can solve the utility function of the Central Bank for the optimal interest rate:

```python
def utility_CB(r_bar, inf_targ, u_n):
    return-(-75*(0.175-0.01*r_bar-inf_targ)**2-(0.02*r_bar-0.2-u_n))

def optimal_r(inf_targ, u_n):

    sol = optimize.minimize_scalar(utility_CB,method='bounded',
    bounds=(0,100),args=(inf_targ,u_n))
    r_star = sol.x
    return r_star

r_opt=optimal_r(0.03,0.05)
print(f'The optimal interest rate is {r_opt:.2f}')

```

    The optimal interest rate is 13.17

We also solve for the values for output, inflation anf unemployment, that follows from this interest rate.

```python
def solve(c0,c1,t0,t1,b0,b1,b2,r,g,inf_targ,alpha,l,y_n,y_equi):

    #solve for the optimal level of output
    y_opt = y_func(c0,c1,t0,t1,b0,b1,b2,r,g)

    #solve for the optimal level of inflation
    pi_opt = pi_func(inf_targ,alpha,l,y_opt,y_n)

    #solve for the optimal level of unemployment
    u_opt = u_func(y_opt,l)

    return y_opt, pi_opt, u_opt

```

```python
inf_targ = 0.03
alpha = 0.05
l = 1000
y_n = 500
r = r_opt

y_opt,pi_opt,u_opt= solve(c0,c1,t0,t1,b0,b1,b2,r,g,inf_targ,alpha,l,y_n,y_equi)

print(f'The level of output is then {y_opt:.2f}, inflation is {pi_opt:.2f} and the unemployment rate is {u_opt:.2f}')
```

    The level of output is then 936.67, inflation is 0.07 and the unemployment rate is 0.06

We see that, in this case, the central bank prefers to raise the interest rate so much that a negative output gap appears. This is done to bring inflation closer to the central banks tagrte, but it also results in the unemployment rate being higher than the natural rate.

We next plot the optimal interest as a function of the central banks inflation target. We then plot central bank utility as a function of the interest rate.

```python
inf_targ_vec = np.linspace(0,0.1,N)
r_opt_vec = np.empty(N)

for i,inf_targ_plot in enumerate(inf_targ_vec):
    inf_targ_plot = inf_targ_vec[i]
    r_opt_vec[i] = optimal_r(inf_targ_plot,0.05)

# creates the figures
plt.style.use("seaborn")
fig = plt.figure(figsize=(10,8))
fig_r_opt = fig.add_subplot(2,1,1)
fig_r_opt.plot(inf_targ_vec,r_opt_vec)
fig_r_opt.set_title('Optimal r')
fig_r_opt.set_xlabel('$\pi_{targ}$')
fig_r_opt.set_ylabel('$r$')

```

```python
def plot_CB_u(inf_targ):
    r_vec = np.linspace(0,20,N)
    CB_u_vec = np.empty(N)

    for i,r_plot in enumerate(r_vec):
        r_plot = r_vec[i]
        CB_u_vec[i] = -utility_CB(r_plot, inf_targ, 0.05)

    # creates the figures
    plt.style.use("seaborn")
    fig = plt.figure(figsize=(10,8))
    fig_CB_u = fig.add_subplot(1,1,1)
    fig_CB_u.plot(r_vec,CB_u_vec)
    fig_CB_u.set_title('CB utility')
    fig_CB_u.set_xlabel('$r$')
    fig_CB_u.set_ylabel('$CB_utility$')

widgets.interact(plot_CB_u,
    inf_targ=widgets.FloatSlider(description="$\pi_{targ}$", min=0, max=0.1, step=0.01, value=0.03),
);
```

    interactive(children=(FloatSlider(value=0.03, description='$\\pi_{targ}$', max=0.1, step=0.01), Output()), _do…

## Conclusion

In this project we have set up and plotted the IS-LM model. We have then added the PC-Curve to the set up to create a relation between inflation, unemployment, the output gap, and interest rate policy. When graphing the IS-LM-PC model for a interest rate of 10 pct. and 12.5 pct we visualize how the level of the real interest rate may affect output gap and the inflation rate. In the case of a positive out gab as we visualize, the Central Bank can bring down the inflation by increasing the real interest rate.

However, we believe that the central bank cares not only about inflation. Therefore we add a uility function of the Central Bank which reflects that the central banks takes into account both inflation and unemployment when setting the real interest rate.

When optimizing the utility function of the Central Bank we find the optimal real interest rate r= 13,17%, while optimal inflation is 7%, the unemployment rate is 6% and the output level is 936,67.

Finally we compute two plots. The first plots the optimal real interest rate in relation to the inflation target while the other plots the utility of the Central Bank. For the latter graph we have made the inflation target adjustable.


# Problem set 6: Solving the Solow model

[](https://mybinder.org/v2/gh/NumEconCopenhagen/exercises-2020/master?urlpath=lab/tree/PS6/problem_set_6.ipynb)

```python
import numpy as np
from scipy import linalg
from scipy import optimize
import sympy as sm
%load_ext autoreload
%autoreload 2
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

# Tasks

## Solving matrix equations I

```python
np.random.seed(1900)
n = 5
A = np.random.uniform(size=(n,n))
b = np.random.uniform(size=n)
c = np.random.uniform(size=n)
d = np.random.uniform(size=n)
```

**Question A:** Find the determinant of $[A \cdot A]^{-1}$

```python
linalg.det(linalg.inv(np.dot(A,A)))
```

    13132.548734460965

**Answer:**

```python
X = linalg.det(linalg.inv(A@A))
print(X)
```

    13132.548734460965

```python
A
```

    array([[0.33224607, 0.71427591, 0.37834749, 0.24908241, 0.83598633],
           [0.02005845, 0.32670359, 0.05606653, 0.4008206 , 0.13288711],
           [0.88711192, 0.15490098, 0.01708181, 0.95781716, 0.58999632],
           [0.83959058, 0.7146372 , 0.58705537, 0.40933648, 0.14603168],
           [0.16407166, 0.65717511, 0.146494  , 0.67717016, 0.47425348]])

```python
b
```

    array([0.78485347, 0.85159023, 0.84757586, 0.42016935, 0.20991113])

```python
linalg.solve(A,b)
```

    array([-15.33189031, -24.00998148,  40.02675108,  15.24193293,
             4.89008792])

**Question B:** Solve the following equation systems directly using **scipy**.

$$
\begin{aligned}
Ax &= b \\
Ax &= c \\
Ax &= d
\end{aligned}
$$

```python
for vec in [b,c,d]:
    print(linalg.solve(A,vec))

```

    [-15.33189031 -24.00998148  40.02675108  15.24193293   4.89008792]
    [ -7.72469812 -11.6354722   20.86796131   7.93343031   1.55909328]
    [-2.57374569 -4.00256301  7.93409587  2.56007481  1.25709881]

**Answer:**

```python
xb = linalg.solve(A,b)
xc = linalg.solve(A,c)
xd = linalg.solve(A,d)
print('b:',xb)
print('c:',xc)
print('d:',xd)
```

    b: [-15.33189031 -24.00998148  40.02675108  15.24193293   4.89008792]
    c: [ -7.72469812 -11.6354722   20.86796131   7.93343031   1.55909328]
    d: [-2.57374569 -4.00256301  7.93409587  2.56007481  1.25709881]

**Question C:** Solve the same equation systems as above using `linalg.lu_factor()` and `linalg.lu_solve()`. What is the benefit of this approach?

```python
LU,piv = linalg.lu_factor(A)
xb = linalg.lu_solve((LU,piv),b)
xc = linalg.lu_solve((LU,piv),c)
xd = linalg.lu_solve((LU,piv),d)
for vec in [xb,xc,xd]:
    print(vec)
```

    [-15.33189031 -24.00998148  40.02675108  15.24193293   4.89008792]
    [ -7.72469812 -11.6354722   20.86796131   7.93343031   1.55909328]
    [-2.57374569 -4.00256301  7.93409587  2.56007481  1.25709881]

**Answer:**

```python
LU,piv = linalg.lu_factor(A) # only done once
xb = linalg.lu_solve((LU,piv),b) # much faster than regular solve
xc = linalg.lu_solve((LU,piv),c)
xd = linalg.lu_solve((LU,piv),d)
print('b:',xb)
print('c:',xc)
print('d:',xd)
```

    b: [-15.33189031 -24.00998148  40.02675108  15.24193293   4.89008792]
    c: [ -7.72469812 -11.6354722   20.86796131   7.93343031   1.55909328]
    d: [-2.57374569 -4.00256301  7.93409587  2.56007481  1.25709881]

## Solving matrix equations II

```python
F = np.array([[2.0, 1.0, -1.0], [-3.0, -1.0, 2], [-2.0, 1.0, 2.0]])
e = np.array([[8.0, -11.0, -3.0]])
print(e.shape)
```

    (1, 3)

```python
F
```

    array([[ 2.,  1., -1.],
           [-3., -1.,  2.],
           [-2.,  1.,  2.]])

```python
e
```

    array([  8., -11.,  -3.])

**Question:** Use the function `gauss_jordan()` in the `numecon_linalg` module located in this folder to solve

$$
Fx = e
$$

```python
import numecon_linalg
```

```python
np.column_stack((F,e))
```

    array([[  2.,   1.,  -1.,   8.],
           [ -3.,  -1.,   2., -11.],
           [ -2.,   1.,   2.,  -3.]])

```python
## Alternative stacking methods:
np.concatenate((F,e[:, np.newaxis]),axis=1)
```

    array([[  2.,   1.,  -1.,   8.],
           [ -3.,  -1.,   2., -11.],
           [ -2.,   1.,   2.,  -3.]])

```python
np.append(F,e[:, np.newaxis],axis=1)
```

    array([[  2.,   1.,  -1.,   8.],
           [ -3.,  -1.,   2., -11.],
           [ -2.,   1.,   2.,  -3.]])

```python
# It's actually easier if e is intitiated as a 2-D matrix of 3x1
# Using [[ ]]:
e2 = np.array([[8.0, -11.0, -3.0]])
print(e2.shape)
test = np.concatenate((F,e2.T),axis=1)
test
```

    (1, 3)

    array([[  2.,   1.,  -1.,   8.],
           [ -3.,  -1.,   2., -11.],
           [ -2.,   1.,   2.,  -3.]])

```python
Y = np.concatenate((F,e[:, np.newaxis]),axis=1)

numecon_linalg.gauss_jordan(Y)
Y
```

    array([[ 1., -0., -0.,  2.],
           [ 0.,  1.,  0.,  3.],
           [ 0.,  0.,  1., -1.]])

**Answer:**

```python
import numecon_linalg
Y = np.column_stack((F,e))
numecon_linalg.gauss_jordan(Y)
print('solution',Y[:,-1])
assert np.allclose(F@Y[:,-1],e)
```

    solution [ 2.  3. -1.]

## Symbolic

**Question A:** Find

$$
\lim_{x \rightarrow 0} \frac{\sin(x)}{x}
$$

and

$$
\frac{\partial\sin(2x)}{\partial x}
$$

```python
x = sm.symbols('x')
```

```python
sm.limit(sm.sin(x)/x,x,0)
```

$\displaystyle 1$

```python
sm.diff(sm.sin(2*x),x)
```

$\displaystyle 2 \cos{\left(2 x \right)}$

**Answer:**

```python
print('the limit is:')
x = sm.symbols('x')
sm.limit(sm.sin(x)/x,x,0)
```

    the limit is:

$\displaystyle 1$

```python
print('the derivative is')
x = sm.symbols('x')
sm.diff(sm.sin(2*x),x)
```

    the derivative is

$\displaystyle 2 \cos{\left(2 x \right)}$

**Question B:** Solve the equation

$$
\frac{\sin(x)}{x} = 0
$$

```python
sm.solve(sm.sin(x)/x,x)
```

    [pi]

**Answer:**

```python
sm.solve(sm.sin(x)/x)
```

    [pi]

# Problem: Solve the Solow model

## Introduction

Consider the **standard Solow-model** where:

1. $K_t$ is capital2
2. $L_t$ is labor (growing with a constant rate of $n$)
3. $A_t$ is technology (growing with a constant rate of $g$)
4. $Y_t = F(K_t,A_tL_t)$ is GDP

**Saving** is a constant fraction of GDP

$$
S_t = sY_t,\,s\in(0,1)
$$

such that **capital accumulates** according to

$$
K_{t+1}=S_{t}+(1-\delta)K_{t}=sF(K_{t},A_{t}L_{t})+(1-\delta)K_{t}, \delta \in (0,1)
$$

The **production function** has **constant-return to scale** such that

$$
\frac{Y_{t}}{A_{t}L_{t}}=\frac{F(K_{t},A_{t}L_{t})}{A_{t}L_{t}}=F(\tilde{k}_{t},1)\equiv f(\tilde{k}_{t})
$$

where $\tilde{k}_t = \frac{K_t}{A_{t}L_{t}}$ is the technology adjusted capital-labor ratio.

The **transition equation** then becomes

$$
\tilde{k}_{t+1}= \frac{1}{(1+n)(1+g)}[sf(\tilde{k}_{t})+(1-\delta)\tilde{k}_{t}]
$$

If the **production function** is **Cobb-Douglas** then

$$
F(K_{t},A_{t}L_{t})=K_{t}^{\alpha}(A_{t}L_{t})^{1-\alpha}\Rightarrow f(\tilde{k}_{t})=\tilde{k}_{t}^{\alpha}
$$

If it is **CES** (with $\beta < 1, \beta \neq 0$) then

$$
F(K_{t},A_{t}L_{t})=(\alpha K_{t}^{\beta}+(1-\alpha)(A_{t}L_{t})^{\beta})^{\frac{1}{\beta}}\Rightarrow f(\tilde{k}_{t})=(\alpha\tilde{k}_{t}^{\beta}+(1-\alpha))^{\frac{1}{\beta}}
$$

## Steady state

Assume the production function is **Cobb-Douglas**.

**Question A:** Use **sympy** to find an analytical expression for the steady state, i.e. solve

$$
\tilde{k}^{\ast}= \frac{1}{(1+n)(1+g)}[sf(\tilde{k}^{\ast})+(1-\delta)\tilde{k}^{\ast}]
$$

```python
k = sm.symbols('k')
alpha = sm.symbols('alpha')
delta = sm.symbols('delta')
s = sm.symbols('s')
g = sm.symbols('g')
n = sm.symbols('n')
```

```python
k, alpha, delta, s , g, n = sm.symbols('k alpha delta s g n')
```

```python
#f = lambda k : k**alpha
f = k**alpha
```

```python
1/((1+n)*(1+g))*(s*f+(1-delta)*k)-k
```

$\displaystyle - k + \frac{k \left(1 - \delta\right) + k^{\alpha} s}{\left(g + 1\right) \left(n + 1\right)}$

```python
k_star = sm.solve(1/((1+n)*(1+g))*(s*f+(1-delta)*k)-k,k)[0]
k_star
```

$\displaystyle \left(\frac{\delta + g n + g + n}{s}\right)^{\frac{1}{\alpha - 1}}$

**Answer:**

```python
f = k**alpha
ss = sm.Eq(k,(s*f+(1-delta)*k)/((1+n)*(1+g)))
kss = sm.solve(ss,k)[0]
kss
```

$\displaystyle \left(\frac{\delta + g n + g + n}{s}\right)^{\frac{1}{\alpha - 1}}$

```python
from IPython.display import display
```

```python
### CES Preferences, Sympy can get overwhelmed by difficult problems
beta = sm.symbols('beta',negative=True)
f_CES = (alpha *k**(beta)+(1-alpha))**(1/beta)

ss_CES = sm.Eq(k,(s*f_CES+(1-delta)*k)/((1+n)*(1+g)))

kss_CES = sm.solve(ss_CES,k)
for eq in kss_CES:
    display(eq)
```

$\displaystyle \left(\frac{\alpha}{\alpha - 1}\right)^{- \frac{1}{\beta}}$

$\displaystyle \left(\frac{\alpha - \left(\frac{s}{\delta + g n + g + n}\right)^{- \beta}}{\alpha - 1}\right)^{- \frac{1}{\beta}}$

We store the solution and test them later when we have a numerical solution. We will find that only one is right (the second). <br>
A further drawback of this solution is that I could not make it work when assuming $k\in\left(0,1\right)$, even though the solution is still correct in the interval.

```python
ss_CES_func = [sm.lambdify((s,g,n,delta,alpha,beta),sol) for sol in kss_CES]
```

**Question B:** Turn you solution into a Python function called as `ss_func(s,g,n,delta,alpha)`.

```python
k_star
```

$\displaystyle \left(\frac{\delta + g n + g + n}{s}\right)^{\frac{1}{\alpha - 1}}$

```python
ss_func = sm.lambdify((s,g,n,delta,alpha),k_star)
```

**Answer:**

```python
ss_func = sm.lambdify((s,g,n,delta,alpha),kss)
```

```python
ss_func(0.2,0.02,0.01,0.1,1/3)
```

    1.903831539231319

**Question C**: Find the steady state numerically using root-finding with `optimize.root_scalar`.

```python
s = 0.2
g = 0.02
n = 0.01
alpha = 1/3
delta = 0.1
f = lambda k : k**alpha

solveit = lambda k: 1/((1+n)*(1+g))*(s*f(k)+(1-delta)*k)-k
```

```python
# Brentq::
optimize.root_scalar(solveit,bracket=[0.1,100],method='brentq')
```

          converged: True
               flag: 'converged'
     function_calls: 11
         iterations: 10
               root: 1.9038315392313183

```python
# Bisect:
optimize.root_scalar(solveit,bracket=[0.1,100],method='bisect')
```

          converged: True
               flag: 'converged'
     function_calls: 48
         iterations: 46
               root: 1.903831539231108

```python
# secant, no brackets (bounds) needed, however two guesses are:
optimize.root_scalar(solveit,x0=50,x1=100,method='secant')
```

          converged: True
               flag: 'converged'
     function_calls: 8
         iterations: 7
               root: 1.9038315392313196

```python
## Finding the first order condition,
k_, alpha_, delta_, s_ , g_, n_ = sm.symbols('k alpha delta s g n')
solveit_ =  1/((1+n_)*(1+g_))*(s_*k_**alpha_+(1-delta_)*k_)-k_

solveit_diff_ = sm.diff(solveit_,k)

solveit_diff_l = sm.lambdify((k_, alpha_, delta_, s_ , g_, n_),solveit_diff_)
solveit_diff_
```

$\displaystyle -1 + \frac{\frac{\alpha k^{\alpha} s}{k} - \delta + 1}{\left(g + 1\right) \left(n + 1\right)}$

```python
# Newton, no brackets (bounds) needed, however the FOC is.
#solveit_diff = lambda k: 1/((1+n)*(1+g))*(s*alpha*k**(alpha-1)+(1-delta))-1

solveit_diff = lambda k: solveit_diff_l(k, alpha, delta, s , g, n)

optimize.root_scalar(solveit,x0=50,method='newton',fprime=solveit_diff)
```

          converged: True
               flag: 'converged'
     function_calls: 12
         iterations: 6
               root: 1.9038315392313192

```python
# Using root

obj_kss = lambda k : solveit(k[0])
optimize.root(obj_kss,x0=[2])
```

        fjac: array([[-1.]])
         fun: 0.0
     message: 'The solution converged.'
        nfev: 6
         qtf: array([3.40638628e-11])
           r: array([0.08425553])
      status: 1
     success: True
           x: array([1.90383154])

```python
# Using root
jac = lambda k : [solveit_diff(k[0])] ## Output needs to be array-like because root expects jacobian

optimize.root(obj_kss,jac=jac,x0=[2], method='hybr')

```

        fjac: array([[-1.]])
         fun: -2.220446049250313e-16
     message: 'The solution converged.'
        nfev: 5
        njev: 1
         qtf: array([3.40629747e-11])
           r: array([0.08425553])
      status: 1
     success: True
           x: array([1.90383154])

```python
import matplotlib.pyplot as plt
```

```python
s = 0.2
g = 0.02
n = 0.01
alpha = 1/3
delta = 0.1

params = {'s':0.2,'g':0.02,'n':0.01,'alpha':1/3,'delta':0.1}

transitioneq = lambda k,parms: (params['s']*k**params['alpha']+(1-params['delta'])*k)/((1+params['n'])*(1+params['g']))

def ss_num(transitioneq,params,guess=2,rtol=1.0000000000000001e-010,
           atol=1e-12,max_iter=2000, printit=True):
    '''
    Finds the steady state(ss) for the transitionseq by simulating the model until equlibrium is reached
    args:
        transitioneq (function) : Transition equation to find ss for
        params           (dict) : A dictionary with parameters values for the transition equation
        guees           (float) : Inital guess
        rtol,atol       (float) : Relative and absolut tolerance for np.isclose
        jump            (float) : relative size of each jump when making a new guess
        max_iter          (int) : Maximum number of iterations
        prinit           (Bool) : Whether or not to print the results

    '''
    k1= transitioneq(guess,params)
    cond = np.isclose(k1,guess,rtol=rtol,atol=atol)
    i = 0
    while not cond:
        i+=1
        if i>max_iter:
            print('max iterations reached')
            break
            return None
    # Make new guess by guessing the way of the evolution of k, so if kt+1>kt the new guess of k becomes larger
        guess = k1
        k1= transitioneq(guess,params)
        cond = np.isclose(k1,guess,rtol=rtol,atol=atol) # Check if kt+1=kt

    if printit:
        print(f'K_star is {guess:.20f}')
        print(f'Check:    {k1:.20f}')

    return guess

kstar = ss_num(transitioneq,params)
```

    K_star is 1.90383154138895749341
    Check:    1.90383154120716469038

```python
#plot solution
ks = np.linspace(0,4,10000)
ks1 = [transitioneq(k,params) for k in ks]

fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(ks,ks1,label='$k_{t+1}(k_{t})$')
ax.plot(ks,ks,ls='--',color='black',label='$k_{t}=k_{t+1}$')
ax.plot(kstar,transitioneq(kstar,params),ls='',marker='o',color='blue',label='$k^{*}$')

ax.grid(True)
plt.legend(loc="upper left")
fig.tight_layout()
```

**Answer:**

```python
f = lambda k: k**alpha
obj_kss = lambda kss: kss - (s*f(kss) + (1-delta)*kss)/((1+g)*(1+n))
result = optimize.root_scalar(obj_kss,bracket=[0.1,100],method='brentq')

print('the steady state for k is',result.root)
```

    the steady state for k is 1.9038315392313185

**Question D:** Now assume the production function is CES. Find the steady state for $k$ for the various values of $\beta$ shown below.

```python

betas = [-0.5,-0.25,-0.1,-0.05,0.05,0.1,0.25,0.5]

f = lambda k,params: (params['alpha']*k**params['beta']+(1-params['alpha']))**(1/params['beta'])
trans_eq = lambda kss,params:(params['s']*f(kss,params) + (1-params['delta'])*kss)/((1+params['g'])*(1+params['n']))

k_stars = np.empty(len(betas))
for i,beta in enumerate(betas):
    params['beta']=beta
    k_stars[i] = ss_num(trans_eq,params, printit=False)

    print(f'When beta= {beta:5.2f} the steady state for k is {k_stars[i]:.15f}')
```

    When beta= -0.50 the steady state for k is 1.847129701923042
    When beta= -0.25 the steady state for k is 1.873383264875734
    When beta= -0.10 the steady state for k is 1.891085641935206
    When beta= -0.05 the steady state for k is 1.897358104639712
    When beta=  0.05 the steady state for k is 1.910515975173142
    When beta=  0.10 the steady state for k is 1.917422135156361
    When beta=  0.25 the steady state for k is 1.939590275709855
    When beta=  0.50 the steady state for k is 1.982233502409577

```python

for beta,k_star in zip(betas,k_stars):
    params['beta']=beta
    ks = np.linspace(1,3,10000)
    ks1 = [trans_eq(k,params) for k in ks]

    fig = plt.figure()
    ax = fig.add_subplot(1,1,1)

    ax.plot(ks,ks1,label='$k_{t+1}(k_{t})$')
    ax.plot(ks,ks,ls='--',color='black',label='$k_{t}=k_{t+1}$')
    ax.plot(k_star,transitioneq(k_star,params),ls='',marker='o',color='blue',label='$k^{*}$')

    ax.grid(True)
    plt.title(r'$\beta=$'+f'{beta}')
    plt.legend(loc="upper left")
    fig.tight_layout()
```

```python
for beta in betas:
    f = lambda k: (alpha*k**beta + (1-alpha))**(1/beta)
    obj_kss = lambda kss: kss - (s*f(kss) + (1-delta)*kss)/((1+g)*(1+n))
    result = optimize.root_scalar(obj_kss,bracket=[0.1,100],method='brentq')
    print(f'for beta = {beta:.3f} the steady state for k is',result.root)
```

    for beta = -0.500 the steady state for k is 1.8471297000972984
    for beta = -0.250 the steady state for k is 1.873383262758588
    for beta = -0.100 the steady state for k is 1.8910856397655083
    for beta = -0.050 the steady state for k is 1.8973581025712734
    for beta = 0.050 the steady state for k is 1.9105159729244352
    for beta = 0.100 the steady state for k is 1.917422132817728
    for beta = 0.250 the steady state for k is 1.9395902733676993
    for beta = 0.500 the steady state for k is 1.9822334997701418

```python
# Test the symbolic solutions found earlier:
for beta in betas:
    sol = [fun(s,g,n,delta,alpha,beta) for fun in ss_CES_func]
    print(f'For beta = {beta:.3f} first solution is {sol[0]:.4f} and second is {sol[1]:.4f}')
```

    For beta = -0.500 first solution is 0.2500 and second is 1.8471
    For beta = -0.250 first solution is 0.0625 and second is 1.8734
    For beta = -0.100 first solution is 0.0010 and second is 1.8911
    For beta = -0.050 first solution is 0.0000 and second is 1.8974
    For beta = 0.050 first solution is 1048576.0000 and second is 1.9105
    For beta = 0.100 first solution is 1024.0000 and second is 1.9174
    For beta = 0.250 first solution is 16.0000 and second is 1.9396
    For beta = 0.500 first solution is 4.0000 and second is 1.9822

```python

```


# 変分ベイズ

```python
import numpy as np
import matplotlib.pyplot as plt
import sympy as sym
from mpl_toolkits.mplot3d import Axes3D
from scipy import genfromtxt
from scipy.special import gamma, digamma
%matplotlib inline

eps = np.finfo(float).eps
```

### ガウス分布の作成

```python
def makeGaussian(mu, sig):
    def func(x):
        y = np.exp(-(x - mu)**2/(2*sig)) / np.sqrt(2*np.pi*sig)
        return y
    return func

def makeMultiGaussian(mu, sig):
    def func(x):
        y = np.exp(-(x-mu).T.dot(np.linalg.inv(sig)).dot(x-mu)/2) / ((2*np.pi)**(2/2)*np.sqrt(np.linalg.det(sig)))
        return y
    return func
```

### ガンマ分布の作成，及び ガウス-ガンマ分布の作成

```python
def makeGammaDist(a, b):
    def gammaDist(x):
        return b**a*x**(a-1)*np.exp(-b*x) / gamma(a)
    return gammaDist

def gaussGammaDist(tau, mu, a=1, b=1, m=0, l=1):
    pgm = makeGammaDist(a, b)
    pga = makeGaussian(m, (l*tau)**-1)

    return pgm(tau)*pga(mu)
```

メッシュの準備

```python
m = 100
x = np.linspace(-10, 10, m)
y = np.linspace(-10, 10, m)
xx, yy = np.meshgrid(x, y)

tau = np.linspace(eps, 5, 100)
mu = np.linspace(-5, 5, 100)
tt, mm = np.meshgrid(tau, mu)
```

---

## 分解による近似(平均場近似)

1次元正規分布の掛け算結果

```python
norm1 = makeGaussian(0, 1)
norm2 = makeGaussian(0, 1)
plt.figure(0)
plt.subplot(121)
plt.contourf(xx, yy, norm1(xx)*norm2(yy), 8, alpha=.75, cmap='jet')
plt.contour(xx, yy, norm1(xx)*norm2(yy), 8, colors='black', linewidth=.5)
plt.gca().set_aspect('equal')
plt.subplot(122)
plt.imshow(norm1(xx)*norm2(yy))
# グラフ作成
ax = Axes3D(plt.figure(1))
ax.plot(xx.reshape(m**2), yy.reshape(m**2), (norm1(xx)*norm2(yy)).reshape(m**2), ".")
```

$q\left( \boldsymbol{z} \right) = \Pi q_i\left( \boldsymbol{z_i} \right)$ として
2次元正規分布を1次元正規分布の同時確率として近似する．

2次元正規分布の作成

```python
mu = np.array([0, 0]).reshape((2, 1))
sig = np.array([[6, 4], [4, 3]])

norm3 = makeMultiGaussian(mu, sig)
X = np.concatenate([xx.reshape(1, m**2), yy.reshape(1, m**2)], axis=0)
Z = np.array([norm3(xi.reshape(2,1)) for xi in X.T])

plt.figure(0)
plt.subplot(121)
plt.contourf(xx, yy, Z.reshape(m,m), 8, alpha=.75)
plt.contour(xx, yy, Z.reshape(m,m), 8, colors='black', linewidth=.5)
plt.gca().set_aspect('equal')
plt.xlim(-5, 5)
plt.ylim(-5, 5)

plt.subplot(122)
Xs = np.random.multivariate_normal(mu.reshape(2), sig, 1000)
plt.plot(Xs[:,0], Xs[:,1], ".")
plt.gca().set_aspect('equal')
plt.xlim(-5, 5)
plt.ylim(-5, 5)

# グラフ作成
ax = Axes3D(plt.figure(1))
ax.plot(xx.reshape(m**2), yy.reshape(m**2), Z.reshape(m**2), ".")
```

1次元正規分布初期値を作成
目標との比較

```python
m1, m2 = np.random.uniform(-5, 5, 2)
norm1 = makeGaussian(m1, 1)
norm2 = makeGaussian(m2, 1)
plt.figure(0)
plt.contourf(xx, yy, norm1(xx)*norm2(yy), 8, alpha=.75)
plt.contour(xx, yy, norm1(xx)*norm2(yy), 8, colors='black', linewidth=.5)
plt.contourf(xx, yy, Z.reshape(m,m), 8, alpha=.75)
plt.contour(xx, yy, Z.reshape(m,m), 8, colors='black', linewidth=.5)
plt.gca().set_aspect('equal')
```

分解した各分布を変分推論の手法によって更新
最終結果を表示する

```python
#mu_s, sig_s = np.mean(Xs.T, axis=1), np.cov(Xs.T)
for i in range(50):
    m1 = mu[0] - 1/sig[0,0]*sig[0,1]*(m2 - mu[1])
    norm1 = makeGaussian(m1, 1/sig[0,0])
    m2 = mu[1] - 1/sig[1,1]*sig[0,1]*(m1 - mu[0])
    norm2 = makeGaussian(m2, 1/sig[1,1])

plt.figure(0)
plt.contourf(xx, yy, norm1(xx)*norm2(yy), 10, alpha=.75)
plt.contour(xx, yy, norm1(xx)*norm2(yy), 10, colors='black', linewidth=.5)
plt.contourf(xx, yy, Z.reshape(m,m), 8, alpha=.75)
plt.contour(xx, yy, Z.reshape(m,m), 8, colors='black', linewidth=.5)
plt.gca().set_aspect('equal')

# グラフ作成
ax = Axes3D(plt.figure(2))
ax.plot(xx.reshape(m**2), yy.reshape(m**2), (norm1(xx)*norm2(yy)).reshape(m**2), "r.")
ax.plot(xx.reshape(m**2), yy.reshape(m**2), Z.reshape(m**2), "b.", alpha=0.2)
```

```python
norm1 = makeGaussian(mu[0,0], sig[0,0])
norm2 = makeGaussian(mu[1,0], sig[1,1])
plt.figure(0)
plt.contourf(xx, yy, norm1(xx)*norm2(yy), 8, alpha=.75)
plt.contour(xx, yy, norm1(xx)*norm2(yy), 8, colors='black', linewidth=.5)
plt.contourf(xx, yy, Z.reshape(m,m), 8, alpha=.75)
plt.contour(xx, yy, Z.reshape(m,m), 8, colors='black', linewidth=.5)
plt.gca().set_aspect('equal')
```

---

## 1変数ガウス

真の事前分布パラメータ

```python
tau = np.linspace(eps, 1, 100)
mu = np.linspace(-2, 2, 100)
tt, mm = np.meshgrid(tau, mu)
```

```python
mu_D, sig_D = 0, 1.5
```

```python
Ns = 10
Xs = np.random.normal(mu_D, sig_D, Ns)
a_t = Ns/2
b_t = (Xs**2).sum() / 2
mu_t = Xs.sum() / Ns
l_t = Ns

plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')
```

事後分布の推定パラメータ

事前分布が幾つかサンプリングをしたあとの事後分布の場合

```python
# 事前分布のパラメータ
Np = 5
Xp = np.random.normal(mu_D, sig_D, Np)
a_prior = Np/2
b_prior = (Xp**2).sum() / 2
mu_prior = Xp.sum() / Np
l_prior = Np
```

```python
# 変分ベイズによる事後分布の計算
Et = 0
for i in range(10):
    if Et == 0: Et = a_prior / b_prior
    else: Et = a_vb1 / b_vb1
    mu_vb1 = (Xs.sum() + mu_prior) / (Ns + l_prior)
    l_vb1 = Et * (Ns + l_prior)

    a_vb1 = a_prior + (Ns+1) / 2
    b_vb1 = b_prior + ((Xs**2).sum() + l_prior*mu_prior**2) / 2
    b_vb1 += (Ns + l_prior) * (mu_vb1**2 + l_vb1**-1) / 2
    b_vb1 += - (Xs.sum() + l_prior*mu_prior) * mu_vb1

print(mu_D, sig_D, "＝＝＞", mu_vb1, (b_vb1/a_vb1)**0.5)
print(a_vb1, "<==>", a_t)
print(b_vb1, "<==>", b_t)
print(mu_vb1, "<==>", mu_t)
print(l_vb1, "<==>", l_t)

```

    0 1.5 ＝＝＞ -0.481029916979 1.94673473582
    8.0 <==> 5.0
    30.3182090532 <==> 16.4204272395
    -0.481029916979 <==> -0.617479875785
    3.95801743397 <==> 10

事前分布が無情報事前分布の場合

```python
# 変分ベイズによる事後分布の計算
Et = 0
for i in range(10):
    if Et == 0: Et = Xs.var()**-1
    else: Et = a_vb2 / b_vb2
    mu_vb2 = Xs.sum()/Ns
    l_vb2 = Et * Ns

    a_vb2 = (Ns+1) / 2
    b_vb2 = (Xs**2).sum() / 2
    b_vb2 += Ns * (mu_vb2**2 + l_vb2**-1) / 2
    b_vb2 += -Xs.sum() * mu_vb2

print(mu_D, sig_D, "＝＝＞", mu_vb2, (b_vb2/a_vb2)**0.5)
print(a_vb2, "<==>", a_t)
print(b_vb2, "<==>", b_t)
print(mu_vb2, "<==>", mu_t)
print(l_vb2, "<==>", l_t)
```

    0 1.5 ＝＝＞ -0.617479875785 1.70376173537
    5.5 <==> 5.0
    15.9654222799 <==> 16.4204272395
    -0.617479875785 <==> -0.617479875785
    3.44494489627 <==> 10

結果の比較
左：事前分布が無情報で無い
右：事前分布が無情報事前分布

```python
plt.subplot(121)
plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_vb1,b_vb1,mu_vb1,l_vb1), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_vb1,b_vb1,mu_vb1,l_vb1), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')

plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')

plt.subplot(122)
plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_vb2,b_vb2,mu_vb2,l_vb2), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_vb2,b_vb2,mu_vb2,l_vb2), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')

plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')
```

事前分布が無情報事前分布でないとき，初期状態から各ステップでの更新の様子をプロット

```python
Et = 0
Np = 5
Xp = np.random.normal(mu_D, sig_D, Np)
a_prior = Np/2
b_prior = (Xp**2).sum() / 2
mu_prior = Xp.sum() / Np
l_prior = Np

#plt.subplot(221)
plt.figure(0)
plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_prior,b_prior,mu_prior,l_prior), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_prior,b_prior,mu_prior,l_prior), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')

plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')

for i in range(10):
    if Et == 0: Et = a_prior / b_prior
    else: Et = a_vb1 / b_vb1
    mu_vb1 = (Xs.sum() + mu_prior) / (Ns + l_prior)
    l_vb1 = Et * (Ns + l_prior)
    if i == 0:
        #plt.subplot(222)
        plt.figure(1)
        plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_prior,b_prior,mu_vb1, l_vb1), 6, alpha=.75)
        plt.contour(mm, tt, gaussGammaDist(tt, mm, a_prior,b_prior,mu_vb1,l_vb1), 6, colors='black', linewidth=.5)
        #plt.gca().set_aspect('equal')

        plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, alpha=.75)
        plt.contour(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, colors='black', linewidth=.5)
        #plt.gca().set_aspect('equal')

    a_vb1 = a_prior + (Ns+1) / 2
    b_vb1 = b_prior + ((Xs**2).sum() + l_prior*mu_prior**2) / 2
    b_vb1 += (Ns + l_prior) * (mu_vb1**2 + l_vb1**-1) / 2
    b_vb1 += - (Xs.sum() + l_prior*mu_prior) * mu_vb1
    if i == 0:
        #plt.subplot(223)
        plt.figure(2)
        plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_vb1, b_vb1,mu_vb1, l_vb1), 6, alpha=.75)
        plt.contour(mm, tt, gaussGammaDist(tt, mm, a_vb1,b_vb1,mu_vb1,l_vb1), 6, colors='black', linewidth=.5)
        #plt.gca().set_aspect('equal')

        plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, alpha=.75)
        plt.contour(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, colors='black', linewidth=.5)
        #plt.gca().set_aspect('equal')

#plt.subplot(224)
plt.figure(3)
plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_vb1, b_vb1,mu_vb1, l_vb1), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_vb1,b_vb1,mu_vb1,l_vb1), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')

plt.contourf(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, alpha=.75)
plt.contour(mm, tt, gaussGammaDist(tt, mm, a_t,b_t,mu_t,l_t), 6, colors='black', linewidth=.5)
#plt.gca().set_aspect('equal')
print(mu_D, sig_D, "＝＝＞", mu_vb1, (b_vb1/a_vb1)**0.5)
print(a_vb1, "<==>", a_t)
print(b_vb1, "<==>", b_t)
print(mu_vb1, "<==>", mu_t)
print(l_vb1, "<==>", l_t)
```

---

## 変分混合ガウス分布

データサンプリング

```python
mu_D = np.array([[-2, -3], [4, 1], [-3, 4]])
lm_D = np.array([[[2, 0.5],[0.5, 1]], [[3, -1],[-1, 2]], [[2, 0],[0, 2]]])
pi_D = np.array([0.4, 0.4, 0.2])
N_D = 100
Xs = np.concatenate([np.random.multivariate_normal(mu_D[i], np.linalg.inv(lm_D[i]), size=int(N_D*pi_D[i])) for i in range(3)])
plt.scatter(Xs[:,0], Xs[:,1])
_=plt.xlim(-7, 7)
_=plt.ylim(-7, 7)
```

変分Eステップ

```python
def VEstep(K, Xs, a, m, b, nu, w):
    N_D, D = Xs.shape
    rho = np.zeros((N_D, K))
    r = np.zeros((N_D, K))
    for k in range(K):
        Epi = digamma(a[k]) - digamma(a.sum())
        Elm = np.sum([digamma((nu[k]+1-d)/2) for d in range(1,D+1)]) + D*np.log(2) + np.log(np.linalg.det(w[k]))
        for n, x in enumerate(Xs):
            Emulm = D/b[k]
            Emulm += nu[k] * np.trace(w[k].dot(np.outer(x-m[k], x-m[k])))
            rho[n, k] = np.exp(Epi + Elm/2. - D/2*np.log(2*np.pi) - Emulm/2.)

    for n in range(N_D): r[n,:] = rho[n, :] / rho[n, :].sum()

    Nk = np.sum(r, axis=0)
    Nk[Nk<1e-10] = 1e-10
    mu_s = np.zeros((K, D))
    var_s = np.zeros((K, D,D))
    for k in range(K):
        if Nk[k] > 1.0e-5:
            mu_s[k] = r[:,k].T.dot(Xs) / Nk[k]
            e = np.array([r[n,k]*np.outer(x-mu_s[k], x-mu_s[k]) for n,x in enumerate(Xs)])
            var_s[k]= e.sum(0)/Nk[k]

    return (Nk, mu_s, var_s)
```

変分Mステップ

```python
def VMstep(K, Xs, Nk, mu_s, var_s):
    global alpha_prior, beta_prior, nu_prior, mu_prior, W_prior

    mu = np.zeros((K, D))
    W = np.array([np.eye(D) for i in range(K)])

    alpha = alpha_prior + Nk
    beta = beta_prior + Nk
    nu = nu_prior + Nk
    for k in range(K):
        if Nk[k] > 1.0e-5:
            mu[k] = (beta_prior[k]*mu_prior[k] + Nk[k]*mu_s[k]) / beta[k]
            W[k] = np.linalg.inv(W_prior[k])
            W[k] += Nk[k]*var_s[k]
            W[k] += beta_prior[k]*Nk[k] / (beta_prior[k]+Nk[k]) * np.outer(mu_s[k]-mu_prior[k], mu_s[k]-mu_prior[k])
            W[k] = np.linalg.inv(W[k])

    return (alpha, mu, beta, nu, W)
```

パラメータの初期値を設定

```python
K = 7
D = 2
alpha_prior = np.zeros(K)
mu_prior = np.zeros((K,D))
beta_prior = np.zeros(K)
nu_prior = np.zeros(K)
W_prior = np.array([np.eye(D) for i in range(K)])

nu_post = np.ones(K)
mu_post = np.random.uniform(-5,5, (K, D))
W_post = np.array([np.eye(D) for i in range(K)])

r_post = 1/K*np.ones((N_D, K))
for k in range(K): r_post[:, k] = np.array([makeMultiGaussian(mu_post[k], np.eye(D))(x) for x in Xs])
for n in range(N_D): r_post[n,:] = r_post[n, :] / r_post[n,:].sum()
Nk = np.zeros(K)
var_s = np.zeros((K, D,D))
mu_s = np.zeros((K, D))
for k in range(K):
    Nk[k] = np.sum(r_post[:,k])
    mu_s[k] = r_post[:,k].T.dot(Xs) / Nk[k]
    e = np.array([r_post[n,k]*np.outer(x-mu_s[k], x-mu_s[k]) for n,x in enumerate(Xs)])
    var_s[k]= e.sum(0)/Nk[k]
```

事後分布の変分近似を計算

```python
m = 100
x = np.linspace(-7, 7, m)
y = np.linspace(-7, 7, m)
xx, yy = np.meshgrid(x, y)
```

```python
fig, axes = plt.subplots(2, 2, figsize=(10, 10))
axes = axes.flatten()
for j in range(4): axes[j].scatter(Xs[:,0], Xs[:,1])
c = ['b', 'g', 'r', 'c', 'm', 'y', 'k']
i = 0
#for k in range(i,i+1):
for k in range(K):
    if Nk[k] > 1.0e-5:
        xy = np.concatenate([xx.reshape(m*m, 1), yy.reshape(m*m,1)], axis=1)
        dist_estimate = makeMultiGaussian(mu_post[k], np.linalg.inv(nu_post[k]*W_post[k]))
        z = np.array([dist_estimate(xy_) for xy_ in xy])
        axes[0].set_aspect('equal', 'datalim')
        axes[0].contour(xx, yy, z.reshape(m,m), 3, colors=c[k], linewidth=.5)
        #plt.contourf(xx, yy, Nk[k]/np.sum(Nk)*z.reshape(m,m), 6, alpha=.75)

step = [1, 5, 20]
for j, s in enumerate(step):
    for i in range(s):
        # M step
        alpha_post, mu_post, beta_post, nu_post, W_post = VMstep(K, Xs, Nk, mu_s, var_s)
        # E step
        Nk, mu_s, var_s = VEstep(K, Xs, alpha_post, mu_post, beta_post, nu_post, W_post)

    for k in range(K):
        if Nk[k] > 1.0e-5:
            xy = np.concatenate([xx.reshape(m*m, 1), yy.reshape(m*m,1)], axis=1)
            dist_estimate = makeMultiGaussian(mu_post[k], np.linalg.inv(nu_post[k]*W_post[k]))
            z = np.array([dist_estimate(xy_) for xy_ in xy])
            axes[j+1].set_aspect('equal', 'datalim')
            axes[j+1].contour(xx, yy, z.reshape(m,m), 3, colors=c[k], linewidth=.5)
            #axes[j+1].set_xlim(-10, 10)
            #axes[j+1].set_ylim(-10, 10)
            #plt.contourf(xx, yy, Nk[k]/np.sum(Nk)*z.reshape(m,m), 6, alpha=.75)
```

```python
z = np.zeros(m*m)
for k in range(K):
    xy = np.concatenate([xx.reshape(m*m, 1), yy.reshape(m*m,1)], axis=1)
    dist_estimate = makeMultiGaussian(mu_post[k], np.linalg.inv(nu_post[k]*W_post[k]))
    z += Nk[k]/np.sum(Nk)*np.array([dist_estimate(xy_) for xy_ in xy])

plt.figure(0, figsize=(10,10))
#plt.axes().set_aspect('equal', 'datalim')
plt.contour(xx, yy, z.reshape(m,m), colors="k", linewidth=.2)
plt.contourf(xx, yy, z.reshape(m,m), cmap=plt.cm.inferno, alpha=0.5)
plt.scatter(Xs[:,0], Xs[:,1])
```

```python
m = 100
x = np.linspace(-7, 7, m)
y = np.linspace(-7, 7, m)
xx, yy = np.meshgrid(x, y)

# 真の分布よりデータをサンプリング
mu_D = np.random.uniform(-2, 2, (4, 2))
lm_D = np.array([[[2, 0.5],[0.5, 1]], [[3, -1],[-1, 2]], [[2, 0],[0, 2]], [[1, -0.2],[-0.2, 2]]])
pi_D = np.array([0.4, 0.3, 0.2, 0.1])
N_D = 100
Xs = np.concatenate([np.random.multivariate_normal(mu_D[i], np.linalg.inv(lm_D[i]), size=int(N_D*pi_D[i])) for i in range(len(mu_D))])
plt.scatter(Xs[:,0], Xs[:,1])
_=plt.xlim(-7, 7)
_=plt.ylim(-7, 7)

##########################################

# パラメータの初期化
K = 7
D = 2
alpha_prior = np.zeros(K)
mu_prior = np.zeros((K,D))
beta_prior = np.zeros(K)
nu_prior = np.zeros(K)
W_prior = np.array([np.eye(D) for i in range(K)])

nu_post = np.ones(K)
mu_post = np.random.uniform(-5,5, (K, D))
W_post = np.array([np.eye(D) for i in range(K)])

r_post = 1/K*np.ones((N_D, K))
for k in range(K): r_post[:, k] = np.array([makeMultiGaussian(mu_post[k], np.eye(D))(x) for x in Xs])
for n in range(N_D): r_post[n,:] = r_post[n, :] / r_post[n,:].sum()
Nk = np.zeros(K)
var_s = np.zeros((K, D,D))
mu_s = np.zeros((K, D))
for k in range(K):
    Nk[k] = np.sum(r_post[:,k])
    mu_s[k] = r_post[:,k].T.dot(Xs) / Nk[k]
    e = np.array([r_post[n,k]*np.outer(x-mu_s[k], x-mu_s[k]) for n,x in enumerate(Xs)])
    var_s[k]= e.sum(0)/Nk[k]

##########################################

# 初期状態をプロット
fig, axes = plt.subplots(2, 2, figsize=(10, 10))
axes = axes.flatten()
for j in range(4): axes[j].scatter(Xs[:,0], Xs[:,1])
c = ['b', 'g', 'r', 'c', 'm', 'y', 'k']
i = 0
for k in range(K):
    if Nk[k] > 1.0e-5:
        xy = np.concatenate([xx.reshape(m*m, 1), yy.reshape(m*m,1)], axis=1)
        dist_estimate = makeMultiGaussian(mu_post[k], np.linalg.inv(nu_post[k]*W_post[k]))
        z = np.array([dist_estimate(xy_) for xy_ in xy])
        axes[0].set_aspect('equal', 'datalim')
        axes[0].contour(xx, yy, z.reshape(m,m), 3, colors=c[k], linewidth=.5)

##########################################

step = [1, 5, 20]
for j, s in enumerate(step):
    # 予測分布の更新
    for i in range(s):
        # M step
        alpha_post, mu_post, beta_post, nu_post, W_post = VMstep(K, Xs, Nk, mu_s, var_s)
        # E step
        Nk, mu_s, var_s = VEstep(K, Xs, alpha_post, mu_post, beta_post, nu_post, W_post)

    # 分布予測をプロット（負担率の高いもののみ）
    for k in range(K):
        if Nk[k] > 1.0e-5:
            xy = np.concatenate([xx.reshape(m*m, 1), yy.reshape(m*m,1)], axis=1)
            dist_estimate = makeMultiGaussian(mu_post[k], np.linalg.inv(nu_post[k]*W_post[k]))
            z = np.array([dist_estimate(xy_) for xy_ in xy])
            axes[j+1].set_aspect('equal', 'datalim')
            axes[j+1].contour(xx, yy, z.reshape(m,m), 3, colors=c[k], linewidth=.5)

##########################################

# 最終的な予測分布をプロット（全要素込）
z = np.zeros(m*m)
for k in range(K):
    xy = np.concatenate([xx.reshape(m*m, 1), yy.reshape(m*m,1)], axis=1)
    dist_estimate = makeMultiGaussian(mu_post[k], np.linalg.inv(nu_post[k]*W_post[k]))
    z += Nk[k]/np.sum(Nk)*np.array([dist_estimate(xy_) for xy_ in xy])

plt.figure(0, figsize=(10,10))
plt.contour(xx, yy, z.reshape(m,m), colors="k", linewidth=.2)
plt.contourf(xx, yy, z.reshape(m,m), cmap=plt.cm.inferno, alpha=0.5)
plt.scatter(Xs[:,0], Xs[:,1])

##########################################
```


# 최적화(Optimization) 문제 풀이를 위한 GUROBI solver 사용하기

 Written by JunPyo Park

* 최적화(Optimization) 문제를 풀기 위한 솔버 중 하나인 GUROBI 사용법을 간단하게 정리해 보았습니다.
* 사용 후기: 규모가 큰 문제를 풀려면 라이센스가 필요합니다. 학교 다니시면 아카데믹 라이센스 무료로 이용가능합니다.

**Reference**
* [GUROBI Python Documentation](https://www.gurobi.com/documentation/9.1/quickstart_windows/cs_python.html#section:Python)
* [GUROBI Tutorial Slide](https://pages.gurobi.com/rs/181-ZYS-005/images/Videos-MIP-v18.pdf)
* [Introduction to Mathematical Optimization Modeling](https://colab.research.google.com/github/Gurobi/modeling-examples/blob/master/intro_to_modeling/introduction_to_modeling_gcl.ipynb)
---

Table of Contents

1  Installation

2  Simple MIP(MIxed Integer Linear Programmin) Example

2.1  Objective (최적 문제의 목적식)

2.2  Constraints (제약식)

2.3  전체 코드

2.4  모델(최적화 문제) 생성

2.5  모델에 변수 추가 하기

2.6  목적식(objective) 세팅

2.7  제약식(constraints) 추가

2.8  모델 최적화

2.9  결과 분석

3  Consulting Company Problem

3.1  Problem Description

3.2  Data

3.3  Decision Variables

3.4  Job constraints

3.5  Resource constraints

3.6  Objective function

## Installation

```
pip install gurobipy
```

## Simple MIP(MIxed Integer Linear Programmin) Example

### Objective (최적 문제의 목적식)

$\text{Maximize}$

        $x +   y + 2 z$

### Constraints (제약식)

$\text{subject to}$

        $x + 2 y + 3 z <= 4$

        $x +   y       >= 1$

        $x, y, z$ $\text{binary}$

x,y,z 가 0,1 값을 가지는 binary 형태이고 목적식과 제약식이 모두 linear 하기에 위 문제는 MIP라고 불립니다.

### 전체 코드

밑에 부분별 설명이 있습니다.

```python
import gurobipy as gp
from gurobipy import GRB
```

```python
try:

    # Create a new model
    m = gp.Model("mip1")

    # Create variables
    x = m.addVar(vtype=GRB.BINARY, name="x")
    y = m.addVar(vtype=GRB.BINARY, name="y")
    z = m.addVar(vtype=GRB.BINARY, name="z")

    # Set objective
    m.setObjective(x + y + 2 * z, GRB.MAXIMIZE)

    # Add constraint: x + 2 y + 3 z <= 4
    # m.addConstr(x + 2 * y + 3 * z <= 4, "c0")
    constr = gp.LinExpr()
    constr += x
    constr += 2 * y
    constr += 3 * z
    constr
    m.addConstr(constr <= 4, "c0")

    # Add constraint: x + y >= 1
    m.addConstr(x + y >= 1, "c1")

    # Optimize model
    m.optimize()

    for v in m.getVars():
        print('%s %g' % (v.varName, v.x))

    print('Obj: %g' % m.objVal)

except gp.GurobiError as e:
    print('Error code ' + str(e.errno) + ': ' + str(e))

except AttributeError:
    print('Encountered an attribute error')
```

    Gurobi Optimizer version 9.1.2 build v9.1.2rc0 (win64)
    Thread count: 8 physical cores, 16 logical processors, using up to 16 threads
    Optimize a model with 2 rows, 3 columns and 5 nonzeros
    Model fingerprint: 0x98886187
    Variable types: 0 continuous, 3 integer (3 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 3e+00]
      Objective range  [1e+00, 2e+00]
      Bounds range     [1e+00, 1e+00]
      RHS range        [1e+00, 4e+00]
    Found heuristic solution: objective 2.0000000
    Presolve removed 2 rows and 3 columns
    Presolve time: 0.00s
    Presolve: All rows and columns removed

    Explored 0 nodes (0 simplex iterations) in 0.00 seconds
    Thread count was 1 (of 16 available processors)

    Solution count 2: 3 2

    Optimal solution found (tolerance 1.00e-04)
    Best objective 3.000000000000e+00, best bound 3.000000000000e+00, gap 0.0000%
    x 1
    y 0
    z 1
    Obj: 3

### 모델(최적화 문제) 생성

```python
# Create a new model
m = gp.Model("mip1")
```

모델에 하나의 최적화 문제 배정. 모델은 변수들과 constraint들 그리고 최적화 문제와 관련된 attribute(변수 범위, 변수 타입, 목적식의 계수 등)들로 구성

### 모델에 변수 추가 하기

```python
# Create binary variable
x = m.addVar(vtype=GRB.BINARY, name="x")
```

`model.addVar()` 메서드를 통해 모델에 변수를 추가 가능합니다. 어레이 같은 구조를 통해 여러 변수를 한번에 추가하려면 `addVars()` 메서드를 사용하면 됩니다.

```python
help(m.addVar)
```

    Help on method addVar in module gurobipy:

    addVar(lb=0.0, ub=1e+100, obj=0.0, vtype='C', name='', column=None) method of gurobipy.Model instance
        ROUTINE:
          addVar(lb, ub, obj, vtype, name, column)

        PURPOSE:
          Add a variable to the model.

        ARGUMENTS:
          lb (float): Lower bound (default is zero)
          ub (float): Upper bound (default is infinite)
          obj (float): Objective coefficient (default is zero)
          vtype (string): Variable type (default is GRB.CONTINUOUS)
          name (string): Variable name (default is no name)
          column (Column): Initial coefficients for column (default is None)

        RETURN VALUE:
          The created Var object.

        EXAMPLE:
          v = model.addVar(ub=2.0, name="NewVar")

### 목적식(objective) 세팅

**Maximize**

        $x +   y + 2 z$

```python
# Set objective
m.setObjective(x + y + 2 * z, GRB.MAXIMIZE)
   ```

식이 복잡할 경우 아래와 같이 단계적으로(incrementally) 수식 표현 가능

```python
obj = gp.LinExpr()
obj += x
obj += y
obj += 2*z
m.setObjective(obj, GRB.MAXIMIZE)
```

### 제약식(constraints) 추가

```python
# Add constraint: x + 2 y + 3 z <= 4
m.addConstr(x + 2 * y + 3 * z <= 4, "c0")
```

* `model.addConstr()` 메서드를 통해 제약식 추가 가능
* 두번째 arg(`"c0"`)는 제약식의 이름
* 식이 복잡할 경우 위의 2.3과 같이 incrementally 추가 가능
* [addConstrs()](https://www.gurobi.com/documentation/9.1/refman/py_model_addconstrs.html) 를 사용해서 여러 제약식 한번에 추가 가능

### 모델 최적화

```python
# Optimize model
m.optimize()
```

추가된 목적식과 제약식을 바탕으로 최적화를 진행합니다.

### 결과 분석

```python
for v in m.getVars():
    print('%s %g' % (v.varName, v.x))
```

* `variable.varName()` 메서드를 통해 사전에 정의한 변수 이름에 접근 가능
* `v.x`: 변수 v의 최적 value

```python
print('Obj: %g' % m.objVal) # optimal value
```

---

## Consulting Company Problem

이거 보면서 풀어보면 감 잡힙니다. 번역은 귀찮아서....

### Problem Description

Consider a consulting company that has three open positions: Tester, Java Developer, and Architect. The three top candidates (resources) for the positions are: Carlos, Joe, and Monika. The consulting company administered competency tests to each candidate in order to assess their ability to perform each of the jobs. The results of these tests are called matching scores. Assume that only one candidate can be assigned to a job, and at most one job can be assigned to a candidate.

The problem is to determine an assignment of resources and jobs such that each job is fulfilled, each resource is assigned to at most one job, and the total matching scores of the assignments is maximized.

### Data
The list $R$ contains the names of the three resources: Carlos, Joe, and Monika.

The list $J$ contains the names of the job positions: Tester, Java Developer, and Architect.

$r \in R$: index and set of resources. The resource $r$ belongs to the set of resources $R$.

$j \in J$: index and set of jobs. The job $j$ belongs to the set of jobs $J$.

For each resource $r$ and job $j$, there is a corresponding matching score $s$. The matching score $s$ can only take values between 0 and 100. That is, $s_{r,j} \in [0, 100]$ for all resources $r \in R$ and jobs $j \in J$.

```python
# Resource and job sets
R = ['Carlos', 'Joe', 'Monika']
J = ['Tester', 'JavaDeveloper', 'Architect']
```

```python
help(gp.multidict)
```

    Help on built-in function multidict in module gurobipy:

    multidict(...)
        ROUTINE:
          multidict(data)
        PURPOSE:
          Split a single dictionary into multiple dictionaries.
        ARGUMENTS:
          data: A dictionary that maps each key to a list of 'n' values.
        RETURN VALUE:
          A list of the shared keys, followed by individual tupledicts.
        EXAMPLE:
          (keys, dict1, dict2) = multidict( {
                   'key1': [1, 2],
                   'key2': [1, 3],
                   'key3': [1, 4] } )

```python
# Matching score data
combinations, scores = gp.multidict({
    ('Carlos', 'Tester'): 53,
    ('Carlos', 'JavaDeveloper'): 27,
    ('Carlos', 'Architect'): 13,
    ('Joe', 'Tester'): 80,
    ('Joe', 'JavaDeveloper'): 47,
    ('Joe', 'Architect'): 67,
    ('Monika', 'Tester'): 53,
    ('Monika', 'JavaDeveloper'): 73,
    ('Monika', 'Architect'): 47
})
```

The following constructor creates an empty ``Model`` object “m”. We specify the model name by passing the string "RAP" as an argument. The ``Model`` object “m” holds a single optimization problem. It consists of a set of variables, a set of constraints, and the objective function.

```python
# Declare and initialize model
m = gp.Model('RAP')
```

### Decision Variables

To solve this assignment problem, we need to identify which resource is assigned to which job. We introduce a decision variable for each possible assignment of resources to jobs. Therefore, we have 9 decision variables.

To simplify the mathematical notation of the model formulation, we define the following indices for resources and jobs:

For example, $x_{2,1}$ is the decision variable associated with assigning the resource Joe to the job Tester. Therefore, decision variable $x_{r,j}$ equals 1 if resource $r \in R$  is assigned to job $j \in J$, and 0 otherwise.

The ``Model.addVars()`` method creates the decision variables for a ``Model`` object.
This method returns a Gurobi ``tupledict`` object that contains the newly created variables. We supply the ``combinations`` object as the first argument to specify the variable indices. The ``name`` keyword is used to specify a name for the newly created decision variables. By default, variables are assumed to be non-negative.

```python
combinations
```

    <gurobi.tuplelist (9 tuples, 2 values each):
     ( Carlos , Tester        )
     ( Carlos , JavaDeveloper )
     ( Carlos , Architect     )
     ( Joe    , Tester        )
     ( Joe    , JavaDeveloper )
     ( Joe    , Architect     )
     ( Monika , Tester        )
     ( Monika , JavaDeveloper )
     ( Monika , Architect     )
    >

```python
# Create decision variables for the RAP model
x = m.addVars(combinations, name="assign")
```

### Job constraints

We now discuss the constraints associated with the jobs. These constraints need to ensure that each job is filled by exactly one resource.

The job constraint for the Tester position requires that resource 1 (Carlos), resource 2 (Joe), or resource 3 (Monika) is assigned to this job. This corresponds to the following constraint.

Constraint (Tester=1)

$$
x_{1,1} + x_{2,1} + x_{3,1} = 1
$$

Similarly, the constraints for the Java Developer and Architect positions can be defined as follows.

Constraint (Java Developer = 2)

$$
x_{1,2} + x_{2,2} + x_{3,2} = 1
$$

Constraint (Architect = 3)

$$
x_{1,3} + x_{2,3} + x_{3,3} = 1
$$

The job constraints are defined by the columns of the following table.

In general, the constraint for the job Tester can defined as follows.

$$
x_{1,1} + x_{2,1} + x_{3,1} = \sum_{r=1}^{3 } x_{r,1} =  \sum_{r \in R} x_{r,1} = 1
$$

All of the job constraints can be defined in a similarly succinct manner. For each job $j \in J$, take the summation of the decision variables over all the resources. We can write the corresponding job constraint as follows.

$$
\sum_{r \in R} x_{r,j} = 1
$$

The ``Model.addConstrs()`` method of the Gurobi/Python API defines the job constraints of the ``Model`` object “m”. This method returns a Gurobi ``tupledict`` object that contains the job constraints.
The first argument of this method, "x.sum(‘*’, j)", is the sum method and defines the LHS of the jobs constraints as follows:
For each job $j$ in the set of jobs $J$, take the summation of the decision variables over all the resources. The $==$  defines an equality constraint, and the number "1" is the RHS of the constraints.
These constraints are saying that exactly one resource should be assigned to each job.
The second argument is the name of this type of constraints.

```python
# Create job constraints
jobs = m.addConstrs((x.sum('*',j) == 1 for j in J), name='job')
```

### Resource constraints

The constraints for the resources need to ensure that at most one job is assigned to each resource. That is, it is possible that not all the resources are assigned.

For example, we want a constraint that requires Carlos to be assigned to at most one of the jobs: either job 1 (Tester), job 2 (Java Developer ), or job 3 (Architect). We can write this constraint as follows.

Constraint (Carlos=1)

$$
x_{1, 1} + x_{1, 2} + x_{1, 3}  \leq 1.
$$

This constraint is less or equal than 1 to allow the possibility that Carlos is not assigned to any job. Similarly, the constraints for the resources Joe and Monika can be defined as follows:

Constraint (Joe=2)

$$
x_{2, 1} + x_{2, 2} + x_{2, 3}  \leq 1.
$$

Constraint (Monika=3)

$$
x_{3, 1} + x_{3, 2} + x_{3, 3}  \leq 1.
$$

Observe that the resource constraints are defined by the rows of the following table.

The constraint for the resource Carlos can be defined as follows.

$$
x_{1, 1} + x_{1, 2} + x_{1, 3} = \sum_{j=1}^{3 } x_{1,j} = \sum_{j \in J} x_{1,j} \leq 1.
$$

Again, each of these constraints can be written in a succinct manner. For each resource $r \in R$, take the summation of the decision variables over all the jobs. We can write the corresponding resource constraint as follows.

$$
\sum_{j \in J} x_{r,j} \leq  1.
$$

The ``Model.addConstrs()`` method of the Gurobi/Python API defines the resource constraints of the ``Model`` object “m”.
The first argument of this method, "x.sum(r, ‘*’)", is the sum method and defines the LHS of the resource constraints as follows: For each resource $r$ in the set of resources $R$, take the summation of the decision variables over all the jobs.
The $<=$  defines a less or equal constraints, and the number “1” is the RHS of the constraints.
These constraints are saying that each resource can be assigned to at most 1 job.
The second argument is the name of this type of constraints.

```python
# Create resource constraints
resources = m.addConstrs((x.sum(r,'*') <= 1 for r in R), name='resource')
```

### Objective function

The objective function is to maximize the total matching score of the assignments that satisfy the job and resource constraints.

For the Tester job, the matching score is $53x_{1,1}$, if resource Carlos is assigned, or $80x_{2,1}$, if resource Joe is assigned, or $53x_{3,1}$, if resource Monika is assigned.
Consequently, the matching score for the Tester job is as follows, where only one term in this summation will be nonzero.

$$
53x_{1,1} + 80x_{2,1} + 53x_{3,1}.
$$

Similarly, the matching scores for the Java Developer and Architect jobs are defined as follows. The matching score for the Java Developer job is:

$$
27x_{1, 2} + 47x_{2, 2} + 73x_{3, 2}.
$$

The matching score for the Architect job is:

$$
13x_{1, 3} + 67x_{2, 3} + 47x_{3, 3}.
$$

The total matching score is the summation of each cell in the following table.

The goal is to  maximize the total matching score of the assignments. Therefore, the objective function is defined as follows.

\begin{equation}
\text{Maximize} \quad (53x_{1,1} + 80x_{2,1} + 53x_{3,1}) \; +
\end{equation}

\begin{equation}
\quad (27x_{1, 2} + 47x_{2, 2} + 73x_{3, 2}) \; +
\end{equation}

\begin{equation}
\quad (13x_{1, 3} + 67x_{2, 3} + 47x_{3, 3}).
\end{equation}

Each term in parenthesis in the objective function can be expressed as follows.

\begin{equation}
(53x_{1,1} + 80x_{2,1} + 53x_{3,1}) = \sum_{r \in R} s_{r,1}x_{r,1}.
\end{equation}

\begin{equation}
(27x_{1, 2} + 47x_{2, 2} + 73x_{3, 2}) = \sum_{r \in R} s_{r,2}x_{r,2}.
\end{equation}

\begin{equation}
(13x_{1, 3} + 67x_{2, 3} + 47x_{3, 3}) = \sum_{r \in R} s_{r,3}x_{r,3}.
\end{equation}

Hence, the objective function can be concisely written as:

\begin{equation}
\text{Maximize} \quad \sum_{j \in J} \sum_{r \in R} s_{r,j}x_{r,j}.
\end{equation}

The ``Model.setObjective()`` method of the Gurobi/Python API defines the objective function of the ``Model`` object “m”. The objective expression is specified in the first argument of this method.
Notice that both the matching score parameters “score” and the assignment decision variables “x” are defined over the “combinations” keys. Therefore, we use the method “x.prod(score)” to obtain the summation of the elementwise multiplication of the "score" matrix and the "x" variable matrix.
The second argument, ``GRB.MAXIMIZE``, is the optimization "sense." In this case, we want to *maximize* the total matching scores of all assignments.

```python
scores
```

    {('Carlos', 'Tester'): 53,
     ('Carlos', 'JavaDeveloper'): 27,
     ('Carlos', 'Architect'): 13,
     ('Joe', 'Tester'): 80,
     ('Joe', 'JavaDeveloper'): 47,
     ('Joe', 'Architect'): 67,
     ('Monika', 'Tester'): 53,
     ('Monika', 'JavaDeveloper'): 73,
     ('Monika', 'Architect'): 47}

```python
x.prod(scores)
```

    <gurobi.LinExpr: 53.0 assign[Carlos,Tester] + 27.0 assign[Carlos,JavaDeveloper] + 13.0 assign[Carlos,Architect] + 80.0 assign[Joe,Tester] + 47.0 assign[Joe,JavaDeveloper] + 67.0 assign[Joe,Architect] + 53.0 assign[Monika,Tester] + 73.0 assign[Monika,JavaDeveloper] + 47.0 assign[Monika,Architect]>

```python
# Objective: maximize total matching score of all assignments
m.setObjective(x.prod(scores), GRB.MAXIMIZE)
```

```python
# Save model for inspection
m.write('RAP.lp')
```

```python
m.display()
```

    Maximize
       <gurobi.LinExpr: 53.0 assign[Carlos,Tester] + 27.0 assign[Carlos,JavaDeveloper] + 13.0 assign[Carlos,Architect] + 80.0 assign[Joe,Tester] + 47.0 assign[Joe,JavaDeveloper] + 67.0 assign[Joe,Architect] + 53.0 assign[Monika,Tester] + 73.0 assign[Monika,JavaDeveloper] + 47.0 assign[Monika,Architect]>
    Subject To
       job[Tester] : <gurobi.LinExpr: assign[Carlos,Tester] + assign[Joe,Tester] + assign[Monika,Tester]> = 1.0
       job[JavaDeveloper] : <gurobi.LinExpr: assign[Carlos,JavaDeveloper] + assign[Joe,JavaDeveloper] + assign[Monika,JavaDeveloper]> = 1.0
       job[Architect] : <gurobi.LinExpr: assign[Carlos,Architect] + assign[Joe,Architect] + assign[Monika,Architect]> = 1.0
       resource[Carlos] : <gurobi.LinExpr: assign[Carlos,Tester] + assign[Carlos,JavaDeveloper] + assign[Carlos,Architect]> <= 1.0
       resource[Joe] : <gurobi.LinExpr: assign[Joe,Tester] + assign[Joe,JavaDeveloper] + assign[Joe,Architect]> <= 1.0
       resource[Monika] : <gurobi.LinExpr: assign[Monika,Tester] + assign[Monika,JavaDeveloper] + assign[Monika,Architect]> <= 1.0

```python
# Run optimization engine
m.optimize()
```

    Gurobi Optimizer version 9.1.2 build v9.1.2rc0 (win64)
    Thread count: 8 physical cores, 16 logical processors, using up to 16 threads
    Optimize a model with 6 rows, 9 columns and 18 nonzeros
    Model fingerprint: 0xb343b6eb
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [1e+01, 8e+01]
      Bounds range     [0e+00, 0e+00]
      RHS range        [1e+00, 1e+00]
    Presolve time: 0.00s
    Presolved: 6 rows, 9 columns, 18 nonzeros

    Iteration    Objective       Primal Inf.    Dual Inf.      Time
           0    4.6000000e+32   1.800000e+31   4.600000e+02      0s
           5    1.9300000e+02   0.000000e+00   0.000000e+00      0s

    Solved in 5 iterations and 0.01 seconds
    Optimal objective  1.930000000e+02

```python
# Display optimal values of decision variables
for v in m.getVars():
    if v.x > 1e-6:
        print(v.varName, v.x)

# Display optimal total matching score
print('Total matching score: ', m.objVal)
```

    assign[Carlos,Tester] 1.0
    assign[Joe,Architect] 1.0
    assign[Monika,JavaDeveloper] 1.0
    Total matching score:  193.0


```python
import sympy

e1, e2, d1, d2, y1, y2, d3 = sympy.symbols("e1 e2 d1 d2 y1 y2 d3")
c11, c22, c33, c44, c55, c66 = sympy.symbols("c11 c22 c33 c44 c55 c66")
c12, c13, c23 = sympy.symbols("c12 c13 c23")

e2 = (c11 - c33)/(2*c33)
d2 = ((c13 + c55)**2 - (c33 - c55)**2) / (2*c33*(c33 - c55))
y2 = (c66 - C44) / (2*c44)

e1 = (c22 - c33)/(2*c33)
d1 = ((c23 + c44)**2 - (c33 - c44)**2) / (2*c33*(c33 - c44))
y2 = (c66 - C55) / (2*c55)

S = sympy.collect(sympy.simplify(B.det()), s3)
print(S)
print('Done Solve')
```


```python
import numpy as np
from sympy import cos,sin
import math
#------------------------------------------------------------------------------------------------------------------------
matriz  = [0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0]
x=0;
Numeros = ['0','0.00001','0.0001','0.001','0.01','0.1','1','10']
#------------------------------------------------------------------------------------------------------------------------
for i in range (8):

    for e in range (5):
        if e == 0:
            matriz[i][e]=cos(x)
        if e == 1:
            matriz[i][e]=cos(0)-x*sin(0)
        if e == 2:
            matriz[i][e]=cos(0)-x*sin(0)-((x**2)/2)*cos(0)
        if e == 3:
            matriz[i][e]=cos(0)-x*sin(0)-((x**2)/2)*cos(0)+((x**3)/6)*sin(0)
        if e == 4:
            matriz[i][e]=cos(0)-x*sin(0)-((x**2)/2)*cos(0)+((x**3)/6)*sin(0)+((x**4)/24)*cos(0)
        if e == 5:
            matriz[i][e]=cos(0)-x*sin(0)-((x**2)/2)*cos(0)+((x**3)/6)*sin(0)+((x**4)/24)*cos(0)-((x**5)/120)*sin(0)

    if x==0:
        x=x+0.00001
    else:
        x=x*10

#------------------------------------------------------------------------------------------------------------------------
print('X\tExacto\t\t\tAprox.1\tAprox.2\tAprox.3\t\tAprox.4\t\tAprox.5')

print('0\t',cos(0),'\t\t\t',matriz[0][0],'\t',matriz[0][1],'\t',matriz[0][2],'\t\t',matriz[0][3],'\t\t',matriz[0][4])
print('0.00001\t',cos(0.00001),'\t\t',matriz[1][0],'\t',matriz[1][1],'\t',matriz[1][2],'\t',matriz[1][3],'\t',matriz[1][4],)
print('0.0001\t',cos(0.0001),'\t\t',matriz[2][0],'\t',matriz[2][1],'\t',matriz[2][2],'\t',matriz[2][3],'\t',matriz[2][4],)
print('0.001\t',cos(0.001),'\t',matriz[3][0],'\t',matriz[3][1],'\t',matriz[3][2],'\t',matriz[3][3],'\t',matriz[3][4],)
print('0.01\t',cos(0.01),'\t',matriz[4][0],'\t',matriz[4][1],'\t',matriz[4][2],'\t',matriz[4][3],'\t',matriz[4][4],)
print('0.1\t',cos(0.1),'\t',matriz[5][0],'\t',matriz[5][1],'\t',matriz[5][2],'\t\t',matriz[5][3],'\t\t',matriz[5][4],)
print('1\t', cos(1),'\t',matriz[6][0],'\t',matriz[6][1],'\t',matriz[6][2],'\t\t',matriz[6][3],'\t\t',matriz[6][4],)
print('10\t', cos(10),'\t',matriz[7][0],'\t',matriz[7][1],'\t',matriz[7][2],'\t\t',matriz[7][3],'\t\t',matriz[7][4],)

```

    X	Exacto			Aprox.1	Aprox.2	Aprox.3		Aprox.4		Aprox.5
    0	 1 			 1 	 1 	 1 		 1 		 1
    0.00001	 0.999999999950000 		 0.999999999950000 	 1 	 0.999999999950000 	 0.999999999950000 	 0.999999999950000
    0.0001	 0.999999995000000 		 0.999999995000000 	 1 	 0.999999995000000 	 0.999999995000000 	 0.999999995000000
    0.001	 0.999999500000042 	 0.999999500000042 	 1 	 0.999999500000000 	 0.999999500000000 	 0.999999500000042
    0.01	 0.999950000416665 	 0.999950000416665 	 1 	 0.999950000000000 	 0.999950000000000 	 0.999950000416667
    0.1	 0.995004165278026 	 0.995004165278026 	 1 	 0.995000000000000 		 0.995000000000000 		 0.995004166666667
    1	 cos(1) 	 0.540302305868140 	 1 	 0.500000000000000 		 0.500000000000000 		 0.541666666666667
    10	 cos(10) 	 -0.839071529076452 	 1 	 -49.0000000000000 		 -49.0000000000000 		 367.666666666667

```python

```


# Double pendulum with torsion damper
> Application of Kane's method assisted with SymPy.

- toc: false
- branch: master
- badges: false
- comments: false
- author: Erik R. Gomez
- categories: [Dynamics, Kane's method]

Let us consider the double pendulum depicted in the figure below. A mass is located at each end of the massless rods concentrated at $\mathcal{P}$ and $\mathcal{Q}$. The massless rods are of length $l$ and the gravity is acting in $-\mathbf{\hat{n}}_{y}$. A torsional spring acts between the first and the second pendulum at point $\mathcal{P}$ with a stiffness of $\sigma$ and a viscous damping $\delta$.

Recall Kane's equations:
$$f_{r}^* = f_{r}$$

$$\sum_{j=1}^{N} \left[ \dot{\mathbf{p}}^{B_j} \cdot \mathbf{v}_{r}^{B_j} ~+ \dot{\mathbf{L}}^{B_j} \cdot \boldsymbol{\omega}_{r}^{B_j}\right] = \sum_{j=1}^{N} \left[ \mathbf{F}^{B_j} ~ \cdot \mathbf{v}_{r}^{B_j} ~ + \mathbf{T}^{B_j} \cdot \boldsymbol{\omega}^{B_j}_r \right]$$

Let's set-up the reference frames and parameters corresponding to the figure in SymPy mechanics:

```python
import sympy as sp
import sympy.physics.mechanics as me
me.init_vprinting()

q1, q2 = me.dynamicsymbols('q1, q2') # Generalized coordinates as function of time
qd1, qd2 = me.dynamicsymbols('q1, q2', level=1) # Time-derivative of generalized coordinates
u1, u2 = me.dynamicsymbols('u1, u2') # Generalized speeds (function of time)

l, m, g = sp.symbols('l, m, g') # Constants
sig, delta = sp.symbols('sigma, delta') # Force parameters
```

Orient the reference frames:

```python
N = me.ReferenceFrame('N') # Inertial frame
A = N.orientnew('A', 'Axis', (q1, N.z)) # Rod A frame
B = A.orientnew('B', 'Axis', (q2, N.z)) # Rod B frame
```

## Kinematics
Rotational motion:
$$\boldsymbol{\omega}^{A} = \dot{q}_1 \mathbf{\hat{n}_z}, \qquad         \boldsymbol{\omega}^{B} = (\dot{q}_1 + \dot{q}_2) \mathbf{\hat{n}_{z}}$$

```python
A.ang_vel_in(N).express(N)
```

```python
B.ang_vel_in(N).express(N)
```

### Choose generalized speeds as:
$$u_1 = \dot{q}_1, \qquad  u_2 = \dot{q}_1 + \dot{q}_2$$
so
$$\boldsymbol{\omega}^{A} = u_1 \mathbf{\hat{n}_z}, \qquad         \boldsymbol{\omega}^{B} = u_2 \mathbf{\hat{n}_{z}}$$.

```python
kde = [u1 - qd1, u2 - (qd1 + qd2)] # Kinematical differential equation
```

### Differentiation of vector:
$$\frac{^{N}d \mathbf{r}}{d t} =  \frac{^{A}d \mathbf{r}}{d t} + ^{N}\omega^{A} \times \mathbf{r} $$

or by the two-point formula
$$^{N}\mathbf{v}^{\mathcal{P}} =  ^{A}\mathbf{v}^{\mathcal{O}} + ^{N}\boldsymbol{\omega}^{A} \times \mathbf{r}_{\mathcal{OP}}.$$

### Calculate the velocities of $\mathcal{P}$ and $\mathcal{Q}$:
Position vectors:

$$\mathbf{r}^{\mathcal{OP}} = - l \mathbf{\hat{a}_y}$$

$$\mathbf{r}^{\mathcal{OQ}} = - l \mathbf{\hat{a}_y} - l \mathbf{\hat{b}_y}$$

Velocities:

$$\mathbf{v}^{\mathcal{P}} = l u_1\mathbf{\hat{a}_x}$$

$$\mathbf{v}^{\mathcal{Q}} = l u_1\mathbf{\hat{a}_x} + l u_2\mathbf{\hat{b}_x}$$

In Sympy Mechanics:

```python
O = me.Point('O') # Origo
O.set_vel(N, 0)

P = O.locatenew('P', ( -l * A.y )) # Point @ end of rod A
Ap = me.Particle('Ap', P, m) # Define mass
P.v2pt_theory(O, N, A) # Set velocity
```

```python
Q = P.locatenew('Q', ( -l * B.y)) # Point @ end of rod B
Bp = me.Particle('Bp', Q, m)
Q.v2pt_theory(P, N, B) # Set velocity
```

Accelerations are needed in Kane's method:

$$\mathbf{a}^{\mathcal{P}} = l \dot{u}_{1}\mathbf{\hat{a}_x} +  l u_{1}^{2}\mathbf{\hat{a}_y}$$

$$\mathbf{a}^{\mathcal{Q}} = l \dot{u}_{1}\mathbf{\hat{a}_x} + l u_{1}^{2}\mathbf{\hat{a}_y} + l \dot{u}_{2}\mathbf{\hat{b}_x} + l u_{2}^{2}\mathbf{\hat{b}_y}$$

```python
P.acc(N)
```

```python
Q.acc(N)
```

## Inertial forces
Let us calculate the inertial forces before applying forces.
$$f^*_r= \sum_{j=1}^{N} \left[ \dot{\mathbf{p}}^{B_j} \cdot \mathbf{v}_{r}^{B_j} ~+ \dot{\mathbf{L}}^{B_j} \cdot \boldsymbol{\omega}_{r}^{B_j}\right]$$
We have
$$\mathbf{v}^{\mathcal{P}} = l u_1\mathbf{\hat{a}_x}, \qquad \mathbf{v}^{\mathcal{Q}} = l u_1\mathbf{\hat{a}_x} + l u_2\mathbf{\hat{b}_x}$$

### Partial velocities become
$$\mathbf{v}^{\mathcal{P}}_1 = \frac{\partial{} \mathbf{v}^{\mathcal{P}}}{\partial u_1} = l \mathbf{\hat{a}_x}, \qquad
\mathbf{v}^{\mathcal{Q}}_1 = \frac{\partial{} \mathbf{v}^{\mathcal{Q}}}{\partial u_1} = l \mathbf{\hat{a}_x},
$$

$$\mathbf{v}^{\mathcal{P}}_2 = \frac{\partial{} \mathbf{v}^{\mathcal{P}}}{\partial u_2} = \mathbf{0}, \qquad
\mathbf{v}^{\mathcal{Q}}_2 = \frac{\partial{} \mathbf{v}^{\mathcal{Q}}}{\partial u_2} = l \mathbf{\hat{b}_x}
$$

### Time derivative of momentum
$$\mathbf{\dot{p}}^{\mathcal{P}} = m \left(l \dot{u}_{1}\mathbf{\hat{a}_x} + l u_{1}^{2}\mathbf{\hat{a}_y} \right), \quad \mathbf{\dot{p}}^{\mathcal{Q}} = m \left(l \dot{u}_{1}\mathbf{\hat{a}_x} + l u_{1}^{2}\mathbf{\hat{a}_y} + l \dot{u}_{2}\mathbf{\hat{b}_x} + l u_{2}^{2}\mathbf{\hat{b}_y}\right)$$

### Evaluate the sum
$$f^*_1 = m \left(l \dot{u}_{1}\mathbf{\hat{a}_x} + l u_{1}^{2}\mathbf{\hat{a}_y} \right) \cdot l \mathbf{\hat{a}_x} + m \left(l \dot{u}_{1}\mathbf{\hat{a}_x} + l u_{1}^{2}\mathbf{\hat{a}_y} + l \dot{u}_{2}\mathbf{\hat{b}_x} + l u_{2}^{2}\mathbf{\hat{b}_y}\right)\cdot l \mathbf{\hat{a}_x} = 2 m l^{2} \dot{u}_{1} +  m  l^{2}\dot{u}_{2} \operatorname{cos}\left(q_{2}\right)- m l^{2}  u^{2}_{2} \operatorname{sin}\left(q_{2}\right)$$

$$f^*_2 = m \left(l \dot{u}_{1}\mathbf{\hat{a}_x} + l u_{1}^{2}\mathbf{\hat{a}_y} \right) \cdot \mathbf{0} + m \left(l \dot{u}_{1}\mathbf{\hat{a}_x} + l u_{1}^{2}\mathbf{\hat{a}_y} + l \dot{u}_{2}\mathbf{\hat{b}_x} + l u_{2}^{2}\mathbf{\hat{b}_y}\right)\cdot l \mathbf{\hat{b}_x} = m  l^{2}\dot{u}_{1} \operatorname{cos}\left(q_{2}\right) + m l^{2}  u^{2}_{1} \operatorname{sin}\left(q_{2}\right) + m l^{2}  \dot{u}_{2}$$

```python
KM = me.KanesMethod(N, [q1, q2], [u1, u2], kd_eqs=kde)
fr, fstar = KM.kanes_equations([Ap, Bp], None)
sp.Eq(-fstar,fr)
```

## Forces
$$ f_{r} =\sum_{j=1}^{N} \left[ \mathbf{F}^{B_j} ~ \cdot \mathbf{v}_{r}^{B_j} ~ + \mathbf{T}^{B_j} \cdot \boldsymbol{\omega}^{B_j}_r \right]$$

Recall
$$\boldsymbol{\omega}^{A} = u_1 \mathbf{\hat{n}_z}, \qquad         \boldsymbol{\omega}^{B} = u_2 \mathbf{\hat{n}_{z}}$$

### Partial angular velocities
$$\boldsymbol{\omega}^{A}_1 = \mathbf{\hat{n}_z}, \quad \boldsymbol{\omega}^{B}_1 = \mathbf{0},$$

$$\boldsymbol{\omega}^{A}_2 = \mathbf{0}, \quad \boldsymbol{\omega}^{B}_2 = \mathbf{\hat{n}_z}.$$

### Torques

$$\mathbf{T}^{A} = \left(\sigma q_2 + \delta \dot{q}_2 \right) \mathbf{\hat{n}_z} = \left(\sigma q_2 + \delta \left(u_2 -u_1\right) \right) \mathbf{\hat{n}_z} $$

$$\mathbf{T}^{B} = -\mathbf{T}^{A} $$

### Gravity
$$\mathbf{F}^{\mathcal{P}} = - g m \mathbf{\hat{n}_y}$$
$$\mathbf{F}^{\mathcal{Q}} = - g m \mathbf{\hat{n}_y}$$

### Evaluate the sum
Torques
$$ f_{1}^T = \mathbf{T}^{A} \cdot \mathbf{\hat{n}_z} + \mathbf{T}^{B} \cdot \mathbf{0} =  \sigma q_2 + \delta \left(u_2 -u_1\right) $$

$$ f_{2}^T = \mathbf{T}^{A} \cdot \mathbf{0} + \mathbf{T}^{B} \cdot \mathbf{\hat{n}_z} = -\sigma q_2 - \delta \left(u_2 -u_1\right) $$

Gravity
$$ f_{1}^F = - g m \mathbf{\hat{n}_y} \cdot l \mathbf{\hat{a}_x} - g m \mathbf{\hat{n}_y}\cdot l \mathbf{\hat{a}_x} = - 2 g l m \operatorname{sin}\left(q_{1}\right)$$

$$ f_{2}^F = - g m \mathbf{\hat{n}_y} \cdot \mathbf{0} - g m \mathbf{\hat{n}_y}\cdot l \mathbf{\hat{b}_x} = - g l m \operatorname{sin}\left(q_{1} + q_{2}\right) $$

```python
# Torque
Ta = (sig * q2 + delta * qd2) * N.z

# Gravity force
Fg = -g*m*N.y*0

# Apply forces on frames and points
forces = [(A, Ta), (B, -Ta), (P, Fg), (Q, Fg)]
```

### Generate EOM

```python
KM = me.KanesMethod(N, [q1, q2], [u1, u2], kd_eqs=kde) # Kane's method instance

fr, fstar = KM.kanes_equations([Ap, Bp], forces) # "Evaluate the sum"

sp.Eq(-fstar,fr) # Output EOM
```

## Different generalized speeds

```python
# Test different kde.
kde1 = [u1 - qd1, u2 - qd2] # Equivalent to Lagrange

KM = me.KanesMethod(N, [q1, q2], [u1, u2], kd_eqs=kde1)
fr, fstar = KM.kanes_equations([Ap, Bp], forces)
sp.Eq(-fstar,fr)
```

```python

```


DeCART Summer School

for

Biomedical Data Science

#  Numerics

There are multiple types of numbers, each arising from different uses that generated them.

>Numbers arose from the formalization of two different human-cognitive conceptions: counting and measurement. Based on fossil records, anthropologists believe that both concepts existed and were used many thousands of years before numbers were introduced. As early as 35,000 years ago, humans put notches into bones (and probably wooden sticks as well, but none have survived and been found) to record things---possibly the cycles of the moon or the seasons---and it seems probable they used sticks or lengths of vine to measure lengths. Numbers themselves, however---abstractions that stand for the number of notches on a bone or the length of a measuring device---appear to have first appeared much later, around 10,000 years ago in the case of counting collections.
>
>These activities resulted in two different kinds of number: the discrete counting numbers and the continuous real numbers. The connection between these two kinds of numbers was not finally put onto a firm footing until the nineteenth century, with the construction of the modern real number system. (Keith Devlin *Introduction to Mathematical Thinking*, p. 75)

There are more than two types of numbers (see [Peter Alfeld's page](http://www.math.utah.edu/~pa/math/numbers.html)) and Python has two distinct types of numbers (five really, but we're going to ignore types 3, 4, and 5). Before we get into Python, let's review differnt types of numbers and talk about how we can represent them in a computer.

### To Simplify

#### Counting Numbers (Integers)
#### Measuring Numbers (Reals)

#  Numerics
## Two Python Numeric Types
* Integers: **roughly corresponding** to the Integers ($\mathbb{Z}$) of mathematics.
* Floating Point Numbers: **roughly corresponding** to the Reals ($\mathbb{R}$) of mathematics.

### Why "roughly corresponding"?
There are an infinitely many integers and real numbers. A computer consisting of a finite number of transitors cannot represent an infinite number of numbers, so there are design decisions that determine how large or small a number can be represented.

### Python Integers

Python integers are exact representations of a finite subset of the integers. The maximum (and minimum) integers that can be represented is hardware dependent. But the value for the computer your code is running on is available in the ``sys`` library.

For the computers used in this course, the value is $$2^{63}-1$$, because we are using a 64-bit system.

#### Why the -1? Why 63 not 64?

```python
import sys
print(sys.maxsize, 2**63 - 1)
```

    9223372036854775807 9223372036854775807

### What happens if we exceed this value?

```python
print(sys.maxsize+1)
```

    9223372036854775808

### Is this what you expected?

#### Compare this behavior to Numpy arrays below.
#### What do we really mean by a maximum integer?

### Floating Point Numbers

Representing approximations of real numbers via floating point types is more complex, but the issue is the same: we only have so many bits to allocate to represent the number. Consequently floating point numbers are inherently **approximations.**

``sys.float_info`` contains information about the limits of floating point numbers.

*  See [Floating Point Arithmetic: Issues and Limitations](http://docs.python.org/3/tutorial/floatingpoint.html)
*  Also [The Perils of Floating Point](http://www.lahey.com/float.htm)
* Floating point numbers are coded by typing them as decimals

```python
import sys
print("maximum float",sys.float_info.max)
print("minimum float",sys.float_info.min)
print("epislon",sys.float_info.epsilon)
```

    maximum float 1.7976931348623157e+308
    minimum float 2.2250738585072014e-308
    epislon 2.220446049250313e-16

#### Because floating point numbers are approximations, you need to be careful with your concepts of equality

* Although is your data likely to differ by less than $2.2\times 10^{-16}$?

### Floating point numbers can also be typed in *exponential form.*

```python
print(4e1)
print(4E1)

```

## Generating Floats

* Typing a number with a decimal
* Dividing two integers.
* Using the [``float``](https://docs.python.org/3/library/functions.html#float) function

```python
print(9/3)
print(float(5))
print(float('5'))

```

```python

```

# Biomedical Data and Python Data Types
### What would be the appropriate data type (Integer, Float, String) for the following quantities?

* Patient pulse as recorded at a clinic visit
* Patient blood pressure
* My medical record number
* Signal measured in an MRI scanner
* Average breaths per minute over an hour
* The length of a tibia
* My weight in kilograms
* Days since last menustral period
* DNA sequence

## What can we do with numbers?

### Create expressions with basic arithmetic operators

* **"+"** addition
* **"-"** subtraction
* **"*"** multiplication
* **"/"** division (also "//")
* **"%"** modulus
* **"\*\*"** exponentiation

### Compare numbers
* **"=="** equal to **(this will be a frequent mistake)**
* **"!="** not equal to
* **"<"** less than
* **"<="** less than or equal to
* **">"** greater than
* **">=** greater than or equal to

```python
print (5+3,type(5+3))
print (5-3,type(5-3))
print (5*3,type(5*3))
print (5/3,type(5/3))
print (5%3 ,type(5%3))
print (5**30 ,type(5**30))
print(4.3%2,type(4.3%2))
print(4.5*3)

print (5== 4)
print(5>4)
print("5>4",5>4)
```

```python
print(5=4)
```

### Python Operators Have Orders of Precedence

If you recall from mathematics, mathematical operators have order of precedence, that is which operator must be applied first. For example, in the equation "$5\times4-3$" would this be calculated as $20-3$ or $5\times1$?

Python's order of precedence is described [here](https://docs.python.org/3/reference/expressions.html). Expressions that have equal precedence are ordered from left to right.

```python
print(5*3/2)
print(5/2*3)
```

    7.5
    7.5

#### What happens when you mix types?

```python
expr = "5+3"
print(eval(expr),type(eval(expr)))

```

#### What is the Rule?

 Python also provides a [``round``](https://docs.python.org/3/library/functions.html#round) function for floats.

#### Note Behavior of *round*.

```python
print(round(5.7),type(round(5.7)))

```

### Comparison Operators for Numbers
* **"=="** equal to **(this will be a frequent mistake)**
* **"!="** not equal to
* **"<"** less than
* **"<="** less than or equal to
* **">"** greater than
* **">=** greater than or equal to

## Exercise

Use Python as a calculator to compute the following value:

\begin{equation}
\frac{(5\times 3^3-2)^2}{4\times 3+1}
\end{equation}

```python
(5*(3**3)-2)**2/(4*3+1)
```

    1360.6923076923076

```bash
%%bash
cp /home/jovyan/work/decart_bootcamp_part2/modules/module1/quizzes/numeric_quizzes.py /home/jovyan/work/decart_bootcamp_part2/modules/module2/quizzes/numeric_quizzes.py

```

```python
from quizzes.numeric_quizzes import *
numeric_quizz1(1360.6923076923076)
```

    'You got the correct answer!'

## Exercise

\begin{equation}
7^{\frac{1}{2}}\times\frac{4}{3} < \frac{3^2\times\frac{3}{4}}{1+2^2}
\end{equation}

```python
7**0.5*4/3 < 3**2*3/4/(1+2**2)
```

    False

```python
numeric_quizz2(False)
```

    'You got the correct answer'

```python
def numeric_quizz2(answer):
    if not isinstance(answer,bool):
        return "Your answer needs to be the Python value True or False. You provided the type %s"%type(answer)
    try:
        assert_equal(answer, 7**(0.5)*(4/3) < (3**2*(3/4))/(1+2**2))
        return "You got the correct answer"
    except Exception as error:
        return "You did not get the correct answer. Pay attention to parentheses and operator precedence: %s"%error

```

## Exercise

Use the ``math`` library to compute the following value

\begin{equation}
e^{-5^2\times cos(\frac{3\pi}{7})}
\end{equation}

```python
import math
```

```python
help(math.cos)
```

    Help on built-in function cos in module math:

    cos(...)
        cos(x)

        Return the cosine of x (measured in radians).

```python
math.e**((-5**2)*math.cos(3*math.pi/7))
```

    0.0038371577761156625

```python
numeric_quizz3(0.0038371577761156625)
```

    'You got the correct answer'

## [Numpy](http://www.numpy.org/)

Numpy arrays are collections of numbers that can be manipulated efficiently. The arrays have a *type* that specifies what the underlying representation of the number is. These types include the following integers:

* uint8 (unsigned integer represented with 8 bits)
* uint16 (unsigned integer represented with 16 bits)
* uint32 (unsigned integer represented with 32 bits)
* uint64 (unsigned integer represented with 64 bits)
* int8 (signed integer represented with 8 bits)
* int16 (signed integer represented with 16 bits)
* int32 (signed integer represented with 32 bits)
* int64 (signed integer represented with 64 bits)

### What is up with all these different types?
* Memory allocation
    * If you have a very big array, memory constraints might come into play
* Computational efficiency
    * What size matches the underlying hardware?

### How do we create numpy arrays?

```Python
import numpy as np # conventional alias for numpy

array1 = np.ones((3,3)) # create a 3x3 array of ones
array2 = np.zeros((4,2)) # create an array of zeros with 4 rows and 2 columns
array3 = np.array([[1,0],[0,1]], dtype=np.uint8)
```

* ``dtype`` stands for "data type". If we don't provide a value, Numpy will choose a "sensible" value based on the array values provided.

* Maximum integer size
    * What is the minimum and maximum integer that can be represented with a uint8 and a int8?

```python
import numpy as np
np.array([-129,], dtype=np.int8) ## maxium number is 128, so -129 will go back to 127
```

    array([127], dtype=int8)

```python
array1 = np.ones((3,3)) ## rows*colomns
print(array1)
array2 = np.zeros((4,2))
print(array2)
```

    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]
    [[ 0.  0.]
     [ 0.  0.]
     [ 0.  0.]
     [ 0.  0.]]

### Operations on Numpy Arrays

*

```python
A= np.random.randint(-50, 50, size=(3,3), dtype=np.int8)
B= np.random.randint(-50, 50, size=(3,3), dtype=np.int16)
C= np.random.randint(-50, 50, size=(2,2))
print(A)
print("*"*42)
print(B)
print("*"*42)
print(C)
```

    [[-22 -12  20]
     [ 44  10 -10]
     [ -7  -6 -18]]
    ******************************************
    [[ 44  12  38]
     [-47 -15 -48]
     [  3  -5  16]]
    ******************************************
    [[-22  -8]
     [-41  46]]

```python
print(A + B)
```

    [[ 22   0  58]
     [ -3  -5 -58]
     [ -4 -11  -2]]

```python
print(A*B)
```

    [[ -968  -144   760]
     [-2068  -150   480]
     [  -21    30  -288]]

```python
np.abs(A)
```

    array([[22, 12, 20],
           [44, 10, 10],
           [ 7,  6, 18]], dtype=int8)

```python
print(A*C)
```

```python
np.exp(A)
```

```python

```