```python
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

import scqubits as qubit

import numpy as np
```

# Defining circuit topology

```python
flux_qubit = qubit.Circuit()
flux_qubit.add_element(qubit.Capacitance('Cg1'), ['g1', '1'])
flux_qubit.add_element(qubit.Capacitance('Cg2'), ['g2', '2'])
flux_qubit.add_element(qubit.Capacitance('CJ1'), ['GND', '1'])
flux_qubit.add_element(qubit.Capacitance('CJ2'), ['GND', '2'])
flux_qubit.add_element(qubit.Capacitance('CJ3'), ['1', '3'])
flux_qubit.add_element(qubit.JosephsonJunction('J1'), ['GND', '1'])
flux_qubit.add_element(qubit.JosephsonJunction('J2'), ['GND', '2'])
flux_qubit.add_element(qubit.JosephsonJunction('J3'), ['1', '3'])
```

# Parametrizing Hamiltonian

```python
ϕ1 = qubit.Variable('ϕ_1')
ϕ2 = qubit.Variable('ϕ_2')
f = qubit.Variable('f')
g1 = qubit.Variable('g_1')
g2 = qubit.Variable('g_2')

flux_qubit.add_variable(ϕ1)
flux_qubit.add_variable(ϕ2)
flux_qubit.add_variable(f)
flux_qubit.add_variable(g1)
flux_qubit.add_variable(g2)
```

# Mapping nodal phases to variables

```python
flux_qubit.map_nodes_linear(['GND', '1', '2', '3', 'g1', 'g2'],
                            ['ϕ_1', 'ϕ_2', 'f', 'g_1', 'g_2'],
                            np.asarray([[0,   0,   0,   0,   0],
                                        [1,   0,   0,   0,   0],
                                        [0,   1,   0,   0,   0],
                                        [0,   1,  -1,   0,   0],
                                        [0,   0,   0,   1,   0],
                                        [0,   0,   0,   0,   1]]))
```

# Separating variables into external controls and degrees of freedom

```python
ϕ1.create_grid(21, 1) # 21 charge states, 2pi wavefunction periodicity
ϕ2.create_grid(21, 1) # 21 charge states, 2pi wavefunction periodicity
f.set_parameter(0.4*2*np.pi, 0) # external flux: 0.4 quantum, external voltage: 0
g1.set_parameter(0, 0) # external flux: 0 quanta, external voltage: 0
g2.set_parameter(0, 0) # external flux: 0 quanta, external voltage: 0
```

# Printing symbolic Hamiltonian (using sympy)
This part is a bit ugly, but in principle we could improve it so it uses better notation

```python
import sympy
C_g1, C_g2, C_J1, C_J2, C_J3, E_J1, E_J2, E_J3 = \
    sympy.symbols('C_g1, C_g2, C_J1, C_J2, C_J3, E_J1, E_J2, E_J3')
flux_qubit.find_element('Cg1').set_capacitance(C_g1)
flux_qubit.find_element('Cg2').set_capacitance(C_g2)
flux_qubit.find_element('CJ1').set_capacitance(C_J1)
flux_qubit.find_element('CJ2').set_capacitance(C_J2)
flux_qubit.find_element('CJ3').set_capacitance(C_J3)
flux_qubit.find_element('J1').set_critical_current(E_J1)
flux_qubit.find_element('J2').set_critical_current(E_J2)
flux_qubit.find_element('J3').set_critical_current(E_J3)
```

## capacitance matrix in terms of the nodal parameterization

```python
flux_qubit.capacitance_matrix(symbolic=True)
```

$\displaystyle \left[\begin{matrix}C_{J1} + C_{J2} & 0.0 & - C_{J1} & 0.0 & - C_{J2} & 0.0\\0.0 & C_{g1} & - C_{g1} & 0.0 & 0.0 & 0.0\\- C_{J1} & - C_{g1} & C_{J1} + C_{J3} + C_{g1} & 0.0 & 0.0 & - C_{J3}\\0.0 & 0.0 & 0.0 & C_{g2} & - C_{g2} & 0.0\\- C_{J2} & 0.0 & 0.0 & - C_{g2} & C_{J2} + C_{g2} & 0.0\\0.0 & 0.0 & - C_{J3} & 0.0 & 0.0 & C_{J3}\end{matrix}\right]$

## Legendre transform of the capacitance matrix (inverse wrt to degrees of freedom)

```python
flux_qubit.capacitance_matrix_legendre_transform(symbolic=True)
```

$\displaystyle \left[\begin{matrix}\frac{C_{J2} + C_{J3} + C_{g2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{g1} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3} C_{g2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\\\frac{C_{J3}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J1} + C_{J3} + C_{g1}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & - \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3} C_{g1}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{g2} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\\\frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & - \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & C_{J3} \left(- \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - C_{J3} \left(\frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) & - C_{g1} \left(- \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) & - C_{g2} \left(\frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right)\\\frac{C_{g1} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3} C_{g1}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3}^{2} C_{g1}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} C_{g1} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{g1}^{2} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3} C_{g1} C_{g2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\\\frac{C_{J3} C_{g2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{g2} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & - \frac{C_{J3}^{2} C_{g2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} C_{g2} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{J3} C_{g1} C_{g2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} & \frac{C_{g2}^{2} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\end{matrix}\right]$

## Symbolic lagrangian

```python
sympy.nsimplify(flux_qubit.symbolic_lagrangian())
```

$\displaystyle - E_{J1} \left(1 - \cos{\left(ϕ_{1} \right)}\right) - E_{J2} \left(1 - \cos{\left(ϕ_{2} \right)}\right) - E_{J3} \left(1 - \cos{\left(f + ϕ_{1} - ϕ_{2} \right)}\right) + \partial_tg_1 \left(\frac{C_{g1} \partial_tg_1}{2} - \frac{C_{g1} \partial_tϕ_1}{2}\right) + \partial_tg_2 \left(\frac{C_{g2} \partial_tg_2}{2} - \frac{C_{g2} \partial_tϕ_2}{2}\right) + \partial_tϕ_1 \left(- C_{J3} \left(- \frac{\partial_tf}{2} + \frac{\partial_tϕ_2}{2}\right) - \frac{C_{g1} \partial_tg_1}{2} + \frac{\partial_tϕ_1 \left(C_{J1} + C_{J3} + C_{g1}\right)}{2}\right) + \partial_tϕ_2 \left(- \frac{C_{g2} \partial_tg_2}{2} + \frac{\partial_tϕ_2 \left(C_{J2} + C_{g2}\right)}{2}\right) + \left(- \partial_tf + \partial_tϕ_2\right) \left(- \frac{C_{J3} \partial_tϕ_1}{2} + C_{J3} \left(- \frac{\partial_tf}{2} + \frac{\partial_tϕ_2}{2}\right)\right)$

## Symbolic hamiltonian

```python
sympy.nsimplify(flux_qubit.symbolic_hamiltonian())
```

$\displaystyle E_{J1} \left(1 - \cos{\left(ϕ_{1} \right)}\right) + E_{J2} \left(1 - \cos{\left(ϕ_{2} \right)}\right) + E_{J3} \left(1 - \cos{\left(f + ϕ_{1} - ϕ_{2} \right)}\right) + \frac{\partial_tf \left(\partial_tf \left(C_{J3} \left(- \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - C_{J3} \left(\frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right)\right) + \partial_tg_1 \left(\frac{C_{J3}^{2} C_{g1}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} C_{g1} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) + \partial_tg_2 \left(- \frac{C_{J3}^{2} C_{g2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} C_{g2} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - i \partial_{ϕ_1} \left(\frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - i \partial_{ϕ_2} \left(- \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right)\right)}{2} + \frac{\partial_tg_1 \left(\frac{C_{J3} C_{g1} C_{g2} \partial_tg_2}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{i C_{J3} C_{g1} \partial_{ϕ_2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{g1}^{2} \partial_tg_1 \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - C_{g1} \partial_tf \left(- \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - \frac{i C_{g1} \partial_{ϕ_1} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right)}{2} + \frac{\partial_tg_2 \left(\frac{C_{J3} C_{g1} C_{g2} \partial_tg_1}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{i C_{J3} C_{g2} \partial_{ϕ_1}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{g2}^{2} \partial_tg_2 \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - C_{g2} \partial_tf \left(\frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - \frac{i C_{g2} \partial_{ϕ_2} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right)}{2} - \frac{i \partial_{ϕ_1} \left(\frac{C_{J3} C_{g2} \partial_tg_2}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{i C_{J3} \partial_{ϕ_2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{g1} \partial_tg_1 \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \partial_tf \left(\frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{C_{J3} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - \frac{i \partial_{ϕ_1} \left(C_{J2} + C_{J3} + C_{g2}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right)}{2} - \frac{i \partial_{ϕ_2} \left(\frac{C_{J3} C_{g1} \partial_tg_1}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} - \frac{i C_{J3} \partial_{ϕ_1}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{g2} \partial_tg_2 \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \partial_tf \left(- \frac{C_{J3}^{2}}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)} + \frac{C_{J3} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right) - \frac{i \partial_{ϕ_2} \left(C_{J1} + C_{J3} + C_{g1}\right)}{- C_{J3}^{2} + \left(C_{J1} + C_{J3} + C_{g1}\right) \left(C_{J2} + C_{J3} + C_{g2}\right)}\right)}{2}$

# Replacing symbolic circuit parameters with some numbers

```python
RATIO = 60.0
ALPHA = 0.8

flux_qubit.find_element('J1').set_critical_current(1.0)
flux_qubit.find_element('J2').set_critical_current(1.0)
flux_qubit.find_element('J3').set_critical_current(1.0*ALPHA)
flux_qubit.find_element('CJ1').set_capacitance(1/(8*1.0/RATIO))
flux_qubit.find_element('CJ2').set_capacitance(1/(8*1.0/RATIO))
flux_qubit.find_element('CJ3').set_capacitance(1/(8*1.0/RATIO/ALPHA))
flux_qubit.find_element('Cg1').set_capacitance(1/(8*50.0/RATIO))
flux_qubit.find_element('Cg2').set_capacitance(1/(8*50.0/RATIO))
```

# Eigenvalues and eigensystem using scqubit tools

```python
flux_qubit.plot_potential()
```

    (<Figure size 750x750 with 2 Axes>,
     <matplotlib.axes._subplots.AxesSubplot at 0x23542ad3490>)

```python
vals, vecs= flux_qubit.eigensys()
```

```python
vals
#vecs_phase = np.conj(unitary).T@vecs
vals - vals[0]
```

    array([0.        , 0.26299666, 0.29339598, 0.50146767, 0.55310417,
           0.57154646])

# Eigenvalues using circuit tools

```python
flux_qubit.calculate_potentials()
vals, vecs = flux_qubit.diagonalize_phase(num_states=6)
```

```python
np.real(vals-vals[0])
```

    array([0.        , 0.26299666, 0.29339598, 0.50146767, 0.55310417,
           0.57154646])

```python
import matplotlib.pyplot as plt
vecs = np.reshape(vecs, (21,21,6))
plt.pcolormesh(ϕ1.get_phase_grid(), ϕ2.get_phase_grid(), np.abs(vecs[:,:,0])**2)
plt.colorbar()
np.sum(np.abs(vecs[:,:,0])**2)
```

    1.0000000000000113

```python

```


# PRÁCTICA 02.
# Perceptrón Simple.

### OBJETIVO:

Que el alumno implemente el perceptrón simple y lo aplique en distintos conjuntos de datos aplicando variaciones en la forma de aprendizaje del perceptrón.

```python
%matplotlib inline

import matplotlib.pyplot as plt
import matplotlib.cm as cm
from mpl_toolkits.mplot3d import Axes3D

import json, matplotlib
from IPython.core.pylabtools import figsize
figsize(11, 5)
colores = ["#348ABD", "#A60628","#06A628"]

from ipywidgets import interact, interact_manual, interactive, fixed
import ipywidgets as widgets
from IPython.display import display
import numpy as np

### Remueve esta línea de código, pues agregarás tus propias funciones
#from perceptron.Perceptron import *
#from perceptron.minimos import *
```

### PERCEPTRÓN

Un perceptrón simple es una unidad computacional con un umbral $\theta$, que al recibir $X \text{ de }n$ entradas reales $X = x_1, x_2, \cdots, x_n$ a través de sus pesos asociados $W = w_1, w_2, \cdots, w_n$, da una salida de valor 1 si la desigualdad $\sum_{i=1}^{n}w_ix_i \geq \theta$ se cumple, en otro caso la salida es 0.

La ley de aprendizaje se define como: $w_i(t+1) = w_i (t) + \alpha · d(x) · x_i$

$$
\
d(x)=
\begin{cases}
1, & \text{si } x \in A;\\
-1, & \text{si } x \in B
\end{cases}
\
$$

y $\alpha$ la tasa de aprendizaje. Los pesos no se modifican si $x$ está bien clasificado.

1.- Implementa la clase Perceptron que contenga lo siguiente:
   * Inicialización del perceptrón dados los pesos iniciales.
   * Función **evalua_escalon** que reciba X y evalúe la salida utilizando la función escalón en el intervalo [0,1].
   * Función **entrena**, debe entrenar al perceptrón: modificación de los pesos en una iteración cada que se llame a esta función dada una tasa de aprendizaje.

2.- Crea un *Perceptron* de pesos iniciales aleatorios y entrénalo con el conjunto de datos datosAND con inicialización de pesos aleatorios.

 * Completa la función **entrenaPerceptronAND** y utiliza tu función **entrena** para que en cada paso de entrenamiento se dibuje el conjunto de datos y el plano que divide al conjunto, tal como se muestra en la figura (utiliza colores para diferenciar al conjunto de datos)

HINT1: Recuerda que el vector de pesos W contiene el vector normal al plano.

HINT2: Puedes utilizar la función scatter y plot_surface

```python
datosAND = np.array([[[1, 0, 0, 0]],
                     [[1, 0 ,0, 1]],
                     [[1, 0, 1, 0]],
                     [[1, 0, 1, 1]],
                     [[1, 1, 0, 0]],
                     [[1, 1, 0, 1]],
                     [[1, 1, 1, 0]],
                     [[1, 1, 1, 1]]])

respuestasAND = np.array([0,0,0,0,0, 0, 0, 1])

indiceX = 0
pasos = 0

@interact_manual()
def entrenaPerceptronAND():
    global indiceX, pasos
    x = datosAND[:,0,1]
    y = datosAND[:,0,2]
    z = datosAND[:,0,3]

    fig = plt.figure()
    ax = fig.add_subplot(111, projection='3d')

    ax.set_xlabel('X Label')
    ax.set_ylabel('Y Label')
    ax.set_zlabel('Z Label')

```

    interactive(children=(Button(description='Run Interact', style=ButtonStyle()), Output()), _dom_classes=('widge…

### MÍNIMOS CUADRADOS

La meta es encontrar los pesos $w$ que mejor mapean los datos de entrada en los datos objetivo. Con ese fin, se plantea la ecuación de error:

\begin{equation}
    E(w) = \frac{1}{2} \| \bar{y} − \bar{t} \|^2
\end{equation}

Con $\bar{t}$ el vector de datos objetivo, $\bar{y}$ el vector de salidas asociadas $ y(x) = X^Tw $

A partir de la ecuación anterior obtenemos:

\begin{equation}
E(w) = \frac{1}{2}(\bar{y}-\bar{t})^T(\bar{y}-\bar{t})
\end{equation}

Derivando respecto a $w$

\begin{equation}
\frac{dE(w)}{dw} = \phi(\bar{y}-\bar{t})
\end{equation}

con $\phi= [x^1  \cdots x^N] \text{ y }x^i$ el i−ésimo vector de datos de entrada a la red. $N$ el número de datos/registros de entrenamiento.

De lo anterior tenemos:

\begin{equation}
w = (\phi \phi^T)^{-1}\phi \bar{t}
\end{equation}

### ENTRENAMIENTO Y PRUEBA

Hay dos fases en el modelado con redes neuronales:
* **Fase de entrenamiento:** Se utiliza un conjunto de datos o patrones de entrenamiento
para determinar los pesos que definen el modelo. Se calculan de manera iterativa con el objeto de minimizar el error cometido entre la salida obtenida por la red neuronal y la salida deseada.

* **Fase de prueba:** Se utiliza para estimar el error de generalización del modelo.

1. Dado el conjunto de datos datos.txt anexo, entrena un perceptrón con el 70% de los datos (elegidos aleatoriamente). El otro 30% será el conjunto de prueba.

2. Para el mismo conjunto de entrenamiento, calcula los pesos $w$ con el método de mínimos cuadrados.

2. Grafica y compara los resultados de $w$ obtenidos del entrenamiento del perceptrón y el de mínimos cuadrados. La salida dece ser similar a la figura siguiente:

3. Utiliza el conjunto de prueba para obtener la exactitud de los modelos.

OBS:  El archivo datos.txt contiene 3 columnas separadas por comas, las dos primeras columnas son el valor del conjunto $X = \{x_1, x_2\}$; la tercer columna son los datos objetivo $\bar{t}$

El criterio de parada tómalo como un número fijo de iteraciones o hasta que el conjunto esté  totalmente clasificado y fija la tasa de aprendizaje a una constante.

```python

#minimos_perceptron()

```

```python

```

```python

```


```python
# 그래프, 수학 기능 추가
# Add graph and math features
import pylab as py
import numpy as np
import numpy.linalg as nl
# 기호 연산 기능 추가
# Add symbolic operation capability
import sympy as sy

```

# 고유치와 고유벡터 : 주응력과 그 방향
Eigenvalue and Eigenvector : Principal stresses and their directions

화살표를 그리는 함수
Function to draw an arrow

```python
def draw_2dvec(x, y, x0=0, y0=0, color='k', name=None):
    py.quiver(x0, y0, x, y, color=color, angles='xy', scale_units='xy', scale=1)
    if name is not None:
        if not name.startswith('$'):
            vec_str = '$\\vec{%s}$' % name
        else:
            vec_str = name
        py.text(0.5 * x + x0, 0.5 * y + y0, vec_str)

```

참고문헌 : Pytel 외 저, 이주성 외 역, 재료역학, 2판, 한티미디어, 2013.
Ref: Pytel, Kiusalaas, Sharma, Mechanics of Materials, 2nd Ed., Cengage Learning, 2013.

다음 2차원 응력 상태의 주응력과 그 방향을 구해 보자.

Let's find the principal stresses and their directions.

$$
\begin{bmatrix}
\sigma_{xx} & \tau_{xy} \\
sym & \sigma_{yy}
\end{bmatrix}
=
\begin{bmatrix}
8000 & 3000 \\
sym & 4000
\end{bmatrix}
$$

```python
stress_Pa_mat = np.array([[8000, 3000],
                          [3000, 4000]])

```

```python
stress_Pa_mat

```

`numpy.linalg.eig()` 함수는 행렬의 고유치와 고유벡터를 계산할 수 있다.
`numpy.linalg.eig()` function can calculate eiganvalues and eigenvectors of a matrix.

```python
result = nl.eig(stress_Pa_mat)

```

행렬의 고유치
Eigenvalues of the matrix

```python
result[0]

```

행렬의 고유벡터
Eigenvectors of the matrix

```python
result[1]

```

첫번째 고유치의 고유벡터
Eigenvector of the first eigenvalue

```python
x0 = result[1][:, 0]

```

```python
x0

```

```python
angle_0_deg = np.rad2deg(np.arctan2(x0[1], x0[0]))

```

첫번째 고유벡터의 방향
Direction of the first eigenvector

```python
angle_0_deg

```

두번째 고유벡터
Second eigenvector

```python
x1 = result[1][:, 1]

```

```python
x1

```

두번째 고유벡터의 방향
Direction of the second eigenvector

```python
angle_1_deg = np.rad2deg(np.arctan2(x1[1], x1[0]))

```

```python
angle_1_deg

```

## Final Bell
마지막 종

```python
# stackoverfow.com/a/24634221
import os
os.system("printf '\a'");

```

```python

```


```python
import numpy as np

%matplotlib inline
from matplotlib import cm
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.ticker import LinearLocator, FormatStrFormatter

import scipy as sp
from scipy import linalg
from scipy import optimize
from scipy import interpolate
import sympy as sm
import seaborn as sns
import ipywidgets as widgets

# Below is for pretty printing
sm.init_printing(use_unicode=True)
```

# 1. Human capital accumulation

Consider a worker living in **two periods**, $t \in \{1,2\}$.

In each period she decides whether to **work ($l_t = 1$) or not ($l_t = 0$)**.

She can *not* borrow or save and thus **consumes all of her income** in each period.

If she **works** her **consumption** becomes:

$$c_t = w h_t l_t\,\,\text{if}\,\,l_t=1$$

where $w$ is **the wage rate** and $h_t$ is her **human capital**.

If she does **not work** her consumption becomes:

$$c_t = b\,\,\text{if}\,\,l_t=0$$

where $b$ is the **unemployment benefits**.

Her **utility of consumption** is:

$$ \frac{c_t^{1-\rho}}{1-\rho} $$

Her **disutility of working** is:

$$ \gamma l_t $$

From period 1 to period 2, she **accumulates human capital** according to:

$$ h_2 = h_1 + l_1 +
\begin{cases}
0 & \text{with prob. }0.5 \\
\Delta & \text{with prob. }0.5
\end{cases} \\
$$

where $\Delta$ is a **stochastic experience gain**.

In the **second period** the worker thus solves:

$$
\begin{eqnarray*}
v_{2}(h_{2}) & = &\max_{l_{2}} \frac{c_2^{1-\rho}}{1-\rho} - \gamma l_2
\\ & \text{s.t.} & \\
c_{2}& = & \begin{cases}
w h_2 &
\text{if }l_2 = 1  \\
b & \text{if }l_2 = 0
\end{cases} \\
l_{2}& \in &\{0,1\}
\end{eqnarray*}
$$

In the **first period** the worker thus solves:

$$
\begin{eqnarray*}
v_{1}(h_{1}) &=& \max_{l_{1}} \frac{c_1^{1-\rho}}{1-\rho} - \gamma l_1 + \beta\mathbb{E}_{1}\left[v_2(h_2)\right]
\\ & \text{s.t.} & \\
c_{1}& = & \begin{cases}
w h_1 &
\text{if }l_1 = 1  \\
b & \text{if }l_1 = 0
\end{cases} \\
h_2 &=& h_1 + l_1 + \begin{cases}
0 & \text{with prob. }0.5\\
\Delta & \text{with prob. }0.5
\end{cases}\\
l_{1} &\in& \{0,1\}\\
\end{eqnarray*}
$$

where $\beta$ is the **discount factor** and $\mathbb{E}_{1}\left[v_2(h_2)\right]$ is the **expected value of living in period two**.

The **parameters** of the model are:

```python
rho = 2
beta = 0.96
gamma = 0.1
w = 2
b = 1
Delta = 0.1
```

The **relevant levels of human capital** are:

```python
h_vec = np.linspace(0.1,1.5,100)
```

First the needed functions are defined below:

```python
# Consumption function for both outcomes of l_t
def con(w,h,l,b):
    if l==1:
        con = w*h*l
    else:
        con = b
    return con

# Utility of consumption
def utility(con,rho):
    return (con**(1-rho))/(1-rho)

# Disutility of working
def disutility(gamma,l):
    return gamma*l

# Model in period 2
def v2(l2,b,rho,gamma,w,h2):
    return utility(con(w,h2,l2,b),rho) - disutility(gamma,l2)

# Model in period 1
def v1(l1,b,rho,gamma,w,h1,Delta,beta,v2_interp):

    # v2 with NO accumulation
    h2_NA = h1 + l1
    v2_NA = v2_interp([h2_NA])[0]

    # v2 if there is accumulation
    h2_A = h1 + l1 + Delta
    v2_A = v2_interp([h2_A])[0]

    # expected value of v2
    v2 = 0.5*v2_A + 0.5*v2_NA

    # Now we can return v1
    return utility(con(w,h1,l1,b),rho) - disutility(gamma,l1) + beta*v2

```

#### **Question 1.1:** Solve the model in period 2 and illustrate the solution (including labor supply as a function of human capital).

```python
# A function to solve the model in period 2 is defined
def solve_p2(rho,gamma,w,b, Delta):

    # a. defining grids
    v2_vec = np.empty(100)
    l2_vec = np.empty(100)

    # b. solve for h
    for i,h2 in enumerate(h_vec):
        # if the utility of working < not working => you do not work, l2=0
        if v2(0,b,rho,gamma,w,h2) > v2(1,b,rho,gamma,w,h2):
            l2_vec[i] = 0
        # if opposite is true => you work, l2=1
        else:
            l2_vec[i] = 1

        v2_vec[i] = v2(l2_vec[i],b,rho,gamma,w,h2)

    return l2_vec, v2_vec

# Use above function to get the solution for period 2
l2_vec, v2_vec = solve_p2(rho,gamma,w,b, Delta)

# Plotting labor supply as a function of human capital
fig1 = plt.figure(figsize=(11,8))
ax = fig1.add_subplot(1,1,1)
ax.plot(h_vec,l2_vec, color='green')

# Customizing plot
ax.set_xlabel('Human Capital, $h_2$', fontsize=12)
ax.set_ylabel('Labor Supply, $l_2$', fontsize=12)
ax.set_title('Figure 1.1: Labor Supply ($l_2$) as a function of human capital ($h_2$) in Period 2', fontsize=17, fontweight='bold')
ax.grid(linestyle='--', linewidth=.8)
plt.xticks(np.linspace(0,1.5,16))
plt.show()
```

Now we want to find the exact level of human capital where the worker chooses to work.

```python
# Below we find the value when l2 goes from 0 to 1. It finds the index, which is then inserted in the h_vec to find the value
ind = np.nonzero(l2_vec == 1)[0][0]
change = h_vec[ind]

# Below is used to print text in bold
BOLD = '\033[1m'
END = '\033[0m'

print('Amount of human capital required for worker to choose working in period 2 is a level greater or equal to ' + BOLD + str(round(change,4)) + END + '.')
```

    Amount of human capital required for worker to choose working in period 2 is a level greater or equal to 0.5667.

#### **Question 1.2:** Solve the model in period 1 and illustrate the solution (including labor supply as a function of human capital)

The interpolated function, v2_interp, defines the expected utility of period 2, namely the term $E_1[v_2(h_2)]$.

```python
# a. interpolate utility from period 2
v2_interp = interpolate.RegularGridInterpolator([h_vec], v2_vec, bounds_error=False, fill_value=None)

# b. create functionto solve problem in period 1
def solve_p1(w, b, rho, gamma, Delta, beta, v2_interp):

    # i. define grids
    v1_vec = np.empty(100)
    l1_vec = np.empty(100)

    # ii. solve for each h in grid
    for i, h1 in enumerate(h_vec):

        # 1. compare v1 for l1 = 0 and l1 = 1 and return the l1 that maximizes v1
        if v1(0,b,rho,gamma,w,h1,Delta,beta,v2_interp) > v1(1,b,rho,gamma,w,h1,Delta,beta,v2_interp):
            l1_vec[i] = 0
        else:
            l1_vec[i] = 1

        # 2. append the v1 value from the optimized l1
        v1_vec[i] = v1(l1_vec[i],b,rho,gamma,w,h1,Delta,beta,v2_interp)

    return v1_vec, l1_vec

v1_vec, l1_vec = solve_p1(w, b, rho, gamma, Delta, beta, v2_interp)

# Plotting labor supply as a function of human capital
fig2 = plt.figure(figsize=(11,8))
ax = fig2.add_subplot(1,1,1)
ax.plot(h_vec,l1_vec, color='green')

# Customizing plot
ax.set_xlabel('Human Capital, $h_1$', fontsize=12)
ax.set_ylabel('Labor Supply, $l_1$', fontsize=12)
ax.set_title('Figure 1.2: Labor Supply ($l_1$) as a function of human capital ($h_2$) in Period 2', fontsize=17, fontweight='bold')
ax.grid(linestyle='--', linewidth=.8)
plt.xticks(np.linspace(0,1.5,16))
plt.show()
```

```python
# Below we find the value when l2 goes from 0 to 1. It finds the index, which is then inserted in the h_vec to find the value
ind = np.nonzero(l1_vec == 1)[0][0]
change = h_vec[ind]

# Below is used to print text in bold
BOLD = '\033[1m'
END = '\033[0m'

print('Amount of human capital required for worker to choose working in period 1 is a level greater or equal to ' + BOLD + str(round(change,4)) + END + '.')
```

    Amount of human capital required for worker to choose working in period 1 is a level greater or equal to 0.3545.

#### **Question 1.3:** Will the worker never work if her potential wage income is lower than the unemployment benefits she can get? Explain and illustrate why or why not.

To explore whether our worker will work at a wage level lower than the unemployment benefit, $b$, we set the wage parameter to be just under the unemployment benefit. Then we plot the figure like in 1.1 and 1.2 to see whether we have any levels of human capital where the worker will choose to work even if the wage is lower than $b$.

```python
# New value of w
w = 0.99

# Getting new values
l2_vec, v2_vec = solve_p2(rho,gamma,w,b, Delta)
v1_vec, l1_vec = solve_p1(w, b, rho, gamma, Delta, beta, v2_interp)

# Plotting
fig = plt.figure(figsize=(11,8))
ax = fig.add_subplot(1,1,1)
ax.plot(h_vec,l2_vec, color='g', label='Period 2')
ax.plot(h_vec,l1_vec, color='r', label='Period 1')
ax.set_xlabel('Human Capital, $h_t$', fontsize=12)
ax.set_ylabel('Labor Supply, $l_t$', fontsize=12)
ax.set_title('Figure 1.3: Labor Supply ($l_t$) as a function of human capital ($h_t$) for $w<b$', fontsize=17, fontweight='bold')
ax.grid()
ax.legend()
plt.show()
```

In the above figure it is shown that the worker will choose to work in both periods if the human capital level is high enough even though the wage is lower than the unemployment benefit.

# 2. AS-AD model

Consider the following **AS-AD model**. The **goods market equilibrium** is given by

$$ y_{t} = -\alpha r_{t} + v_{t} $$

where $y_{t}$ is the **output gap**, $r_{t}$ is the **ex ante real interest** and $v_{t}$ is a **demand disturbance**.

The central bank's **Taylor rule** is

$$ i_{t} = \pi_{t+1}^{e} + h \pi_{t} + b y_{t}$$

where $i_{t}$ is the **nominal interest rate**, $\pi_{t}$ is the **inflation gap**, and $\pi_{t+1}^{e}$ is the **expected inflation gap**.

The **ex ante real interest rate** is given by

$$ r_{t} = i_{t} - \pi_{t+1}^{e} $$

Together, the above implies that the **AD-curve** is

$$ \pi_{t} = \frac{1}{h\alpha}\left[v_{t} - (1+b\alpha)y_{t}\right]$$

Further, assume that the **short-run supply curve (SRAS)** is given by

$$ \pi_{t} = \pi_{t}^{e} + \gamma y_{t} + s_{t}$$

where $s_t$ is a **supply disturbance**.

**Inflation expectations are adaptive** and given by

$$ \pi_{t}^{e} = \phi\pi_{t-1}^{e} + (1-\phi)\pi_{t-1}$$

Together, this implies that the **SRAS-curve** can also be written as

$$ \pi_{t} = \pi_{t-1} + \gamma y_{t} - \phi\gamma y_{t-1} + s_{t} - \phi s_{t-1} $$

The **parameters** of the model are:

```python
par = {}

par['alpha'] = 5.76
par['h'] = 0.5
par['b'] = 0.5
par['phi'] = 0
par['gamma'] = 0.075
```

#### **Question 2.1:** Use the ``sympy`` module to solve for the equilibrium values of output, $y_t$, and inflation, $\pi_t$, (where AD = SRAS) given the parameters ($\alpha$, $h$, $b$, $\alpha$, $\gamma$) and $y_{t-1}$ , $\pi_{t-1}$, $v_t$, $s_t$, and $s_{t-1}$.

First all the relevant symbols and functions are defined below:

```python
# Defining all symbols
alpha = sm.symbols('alpha')
h = sm.symbols('h')
b = sm.symbols('b')
phi = sm.symbols('phi')
gamma = sm.symbols('gamma')

v = sm.symbols('v_t')
s = sm.symbols('s_t')
s_lag = sm.symbols('s_{t-1}')
pi = sm.symbols('pi_t')
pi_lag = sm.symbols('pi_t-1')
y = sm.symbols('y_t')
y_lag = sm.symbols('y_{t-1}')

# Functions
AD = sm.Eq(pi,(1/(h*alpha)*(v-(1+b*alpha)*y)))
display(AD)

SRAS = sm.Eq(pi,pi_lag + gamma*y - phi*gamma*y_lag + s - phi*s_lag )
display(SRAS)
```

Now we make a function that set the above equations equal to each other:

```python
# AD = SRAS
ADSRAS = sm.Eq((1/(h*alpha)*(v-(1+b*alpha)*y)),pi_lag + gamma*y - phi*gamma*y_lag + s - phi*s_lag)
print('The equations are set equal to each other which becomes:')
print()
display(ADSRAS)
```

```python
# Solve for y
y_EQ = sm.solve(ADSRAS,y)
print('The solution for y is:')
print()
display(y_EQ)
```

Now we can substitute this value into the SRAS function to get the expression for $\pi_t$:

```python
SRAS_new = SRAS.subs(y,y_EQ[0])
pi_EQ = sm.solve(SRAS_new,pi)
print('The solution for pi is:')
print()
display(pi_EQ)
```

#### **Question 2.2:** Find and illustrate the equilibrium when $y_{t-1} = \pi_{t-1} = v_t = s_t = s_{t-1} = 0$. Illustrate how the equilibrium changes when instead $v_t = 0.1$.

We want to be able to parameterize the equations and for this we use sympy's lambdify function:

```python
# Lambdify functions
lambd_y = sm.lambdify((alpha,h,b,phi,gamma,pi_lag,y_lag,s_lag,v,s),y_EQ)
lambd_pi = sm.lambdify((alpha,h,b,phi,gamma,pi_lag,y_lag,s_lag,v,s),pi_EQ)

# Now we can find the equilibrium values
y_par0 = lambd_y(par['alpha'],par['h'],par['b'],par['phi'],par['gamma'],0,0,0,0,0)
pi_par0 = lambd_pi(par['alpha'],par['h'],par['b'],par['phi'],par['gamma'],0,0,0,0,0)

print('The equilibrium values of y and pi when parameters are equal to zero are:')
print(y_par0,pi_par0)
```

    The equilibrium values of y and pi when parameters are equal to zero are:
    [0.0] [0.0]

```python
y_parv = lambd_y(par['alpha'],par['h'],par['b'],par['phi'],par['gamma'],0,0,0,0.1,0)
pi_parv = lambd_pi(par['alpha'],par['h'],par['b'],par['phi'],par['gamma'],0,0,0,0.1,0)

print('The equilibrium values of y and pi when v = 0.1 and all other parameters are zero are:')
print(y_parv,pi_parv)
```

    The equilibrium values of y and pi when v = 0.1 and all other parameters are zero are:
    [0.0244140625] [0.0018310546875]

Below the different equilibriums are illustrated

```python
# First we define the AD and SRAS functions
def AD_fig(v,y):
    return (1/(par['h']*par['alpha'])*(v-(1+par['b']*par['alpha'])*y))
def SRAS_fig(pi_lag,y,y_lag,s,s_lag):
    return pi_lag + par['gamma']*y - par['phi']*par['gamma']*y_lag + s - par['phi']*s_lag
```

```python
# Below the needed values for the figure are defined
y = np.linspace(-0.1,0.1,50)

# Values without shock
y_lag = 0
pi_lag = 0
v = 0
s = 0
s_lag = 0

# Only value that changes with shock is v
v_s = 0.1

AD = AD_fig(v,y)
AD_shock = AD_fig(v_s,y)
SRAS = SRAS_fig(pi_lag,y,y_lag,s,s_lag)

# Figure is plotted
fig = plt.figure(figsize=(11,8))
ax = fig.add_subplot(1,1,1)
ax.plot(y,AD, label='AD, $v_t=0$')
ax.plot(y,AD_shock,label='AD, $v_t=0.1$')
ax.plot(y,SRAS, label='SRAS')
ax.set_xlabel('$y_t$',size=12)
ax.set_ylabel('$\pi_t$',size=12)
ax.legend()
ax.grid()
plt.title('Figure 2.1: Equilibrium with and without shock', size=18)
plt.show()

```

**Persistent disturbances:** Now, additionaly, assume that both the demand and the supply disturbances are AR(1) processes

$$ v_{t} = \delta v_{t-1} + x_{t} $$
$$ s_{t} = \omega s_{t-1} + c_{t} $$

where $x_{t}$ is a **demand shock**, and $c_t$ is a **supply shock**. The **autoregressive parameters** are:

```python
par['delta'] = 0.80
par['omega'] = 0.15
```

```python
# First the above parameters and the ones from the persistent disturbances are coded as symbols
delta = sm.symbols('delta')
omega = sm.symbols('omega')
v = sm.symbols('v_t')
v_lag = sm.symbols('v_t-1')
s = sm.symbols('s_t')
s_lag = sm.symbols('s_t-1')
x = sm.symbols('x_t')
c = sm.symbols('c_t')

# Now the persistent disturbances are defined
v_sh = delta*v_lag + x
s_sh = omega*s_lag + c

# Now the above relations are defined they are inserted in the AD and SRAS curve
AD_sh = sm.Eq(pi,(1/(h*alpha)*(v_sh-(1+b*alpha)*y)))
display(AD_sh)

SRAS_sh = sm.Eq(pi,pi_lag + gamma*y - phi*gamma*y_lag + s_sh - phi*s_lag )
display(SRAS_sh)

```

```python

```

#### **Question 2.3:** Starting from $y_{-1} = \pi_{-1} = s_{-1} = 0$, how does the economy evolve for $x_0 = 0.1$, $x_t = 0, \forall t > 0$ and $c_t = 0, \forall t \geq 0$?

```python

```

**Stochastic shocks:** Now, additionally, assume that $x_t$ and $c_t$ are stochastic and normally distributed

$$ x_{t}\sim\mathcal{N}(0,\sigma_{x}^{2}) $$
$$ c_{t}\sim\mathcal{N}(0,\sigma_{c}^{2}) $$

The **standard deviations of the shocks** are:

```python
par['sigma_x'] = 3.492
par['sigma_c'] = 0.2
```

#### **Question 2.4:** Simulate the AS-AD model for 1,000 periods. Calculate the following five statistics:

1. Variance of $y_t$, $var(y_t)$
2. Variance of $\pi_t$, $var(\pi_t)$
3. Correlation between $y_t$ and $\pi_t$, $corr(y_t,\pi_t)$
4. Auto-correlation between $y_t$ and $y_{t-1}$, $corr(y_t,y_{t-1})$
5. Auto-correlation between $\pi_t$ and $\pi_{t-1}$, $corr(\pi_t,\pi_{t-1})$

#### **Question 2.5:** Plot how the correlation between $y_t$ and $\pi_t$ changes with $\phi$. Use a numerical optimizer or root finder to choose $\phi\in(0,1)$ such that the simulated correlation between $y_t$ and $\pi_t$ comes close to 0.31.

#### **Quesiton 2.6:** Use a numerical optimizer to choose $\sigma_x>0$, $\sigma_c>0$ and $\phi\in(0,1)$ to make the simulated statistics as close as possible to US business cycle data where:

1. $var(y_t) = 1.64$
2. $var(\pi_t) = 0.21$
3. $corr(y_t,\pi_t) = 0.31$
4. $corr(y_t,y_{t-1}) = 0.84$
5. $corr(\pi_t,\pi_{t-1}) = 0.48$

# 3. Exchange economy

Consider an **exchange economy** with

1. 3 goods, $(x_1,x_2,x_3)$
2. $N$ consumers indexed by \\( j \in \{1,2,\dots,N\} \\)
3. Preferences are Cobb-Douglas with log-normally distributed coefficients

    $$ \begin{eqnarray*}
    u^{j}(x_{1},x_{2},x_{3}) &=&
    \left(x_{1}^{\beta_{1}^{j}}x_{2}^{\beta_{2}^{j}}x_{3}^{\beta_{3}^{j}}\right)^{\gamma}\\
     &  & \,\,\,\beta_{i}^{j}=\frac{\alpha_{i}^{j}}{\alpha_{1}^{j}+\alpha_{2}^{j}+\alpha_{3}^{j}} \\
     &  & \,\,\,\boldsymbol{\alpha}^{j}=(\alpha_{1}^{j},\alpha_{2}^{j},\alpha_{3}^{j}) \\
     &  & \,\,\,\log(\boldsymbol{\alpha}^j) \sim \mathcal{N}(\mu,\Sigma) \\
    \end{eqnarray*} $$

4. Endowments are exponentially distributed,

$$
\begin{eqnarray*}
\boldsymbol{e}^{j} &=& (e_{1}^{j},e_{2}^{j},e_{3}^{j}) \\
 &  & e_i^j \sim f, f(z;\zeta) =  1/\zeta \exp(-z/\zeta)
\end{eqnarray*}
$$

Let $p_3 = 1$ be the **numeraire**. The implied **demand functions** are:

$$
\begin{eqnarray*}
x_{i}^{\star j}(p_{1},p_{2},\boldsymbol{e}^{j})&=&\beta^{j}_i\frac{I^j}{p_{i}} \\
\end{eqnarray*}
$$

where consumer $j$'s income is

$$I^j = p_1 e_1^j + p_2 e_2^j +p_3 e_3^j$$

The **parameters** and **random preferences and endowments** are given by:

```python
# a. parameters
N = 50000
mu = np.array([3,2,1])
Sigma = np.array([[0.25, 0, 0], [0, 0.25, 0], [0, 0, 0.25]])
gamma = 0.8
zeta = 1

# b. random draws
seed = 1986
np.random.seed(seed)

# preferences
alphas = np.exp(np.random.multivariate_normal(mu, Sigma, size=N))
betas = alphas/np.reshape(np.sum(alphas,axis=1),(N,1))

# endowments
e1 = np.random.exponential(zeta,size=N)
e2 = np.random.exponential(zeta,size=N)
e3 = np.random.exponential(zeta,size=N)
```

#### **Question 1:** Plot the histograms of the budget shares for each good across agents.

Here we have Cobb-Douglas preferences which means that the budget shares are represented by the $\beta$'s. These are plotted in the below diagram.

```python
# betas
b1 = betas[:,0]
b2 = betas[:,1]
b3 = betas[:,2]

# Plotting the histograms of each good
hist, [ax1, ax2, ax3] = plt.subplots(1, 3, figsize=(20,6), sharey=True, sharex=True)
ax1.hist(b1, bins=15)
ax1.set_title('Good 1', size=13, fontweight='bold')
ax2.hist(b2, color='r', bins=15)
ax2.set_title('Good 2', size=13, fontweight='bold')
ax3.hist(b3, color='g', bins=15)
ax3.set_title('Good 3', size=13, fontweight='bold')
hist.suptitle('Figure 3.1: Budget Shares across Agents', size=20)
ax1.set_ylabel('Number of consumers', size=13)
ax2.set_xlabel('Budget share of goods', size=13)

```

From the figure above we see that the highest budget share is that of Good 1, while Good 3 has the smallest budget share. From the

Consider the **excess demand functions:**

$$ z_i(p_1,p_2) = \sum_{j=1}^N x_{i}^{\star j}(p_{1},p_{2},\boldsymbol{e}^{j}) - e_i^j$$

#### **Question 2:** Plot the excess demand functions.

First the functions given are defined

```python
# 1. Demand functions of goods, p3=1
def dem1(p1,p2,e1,e2,e3,b1):
    I = p1*e1 + p2*e2 + e3
    return b1*(I/p1)

def dem2(p1,p2,e1,e2,e3,b2):
    I = p1*e1 + p2*e2 + e3
    return b2*(I/p2)

def dem3(p1,p2,e1,e2,e3,b3):
    I = p1*e1 + p2*e2 + e3
    return b3*I

# 2. Excess demand functions
def ex_dem1(p1,p2,e1,e2,e3):
    return np.sum(dem1(p1,p2,e1,e2,e3,b1) - e1)

def ex_dem2(p1,p2,e1,e2,e3):
    return np.sum(dem2(p1,p2,e1,e2,e3,b2) - e2)

def ex_dem3(p1,p2,e1,e2,e3):
    return np.sum(dem3(p1,p2,e1,e2,e3,b3) - e3)
```

Below the needed vectors and grids are constructed:

```python
# A set of price vectors are defined
p1_vec = np.linspace(0.1,5,100)
p2_vec = np.linspace(0.1,5,100)
p3_vec = 1

# Now grids for the endowments and prices are constructed
e1_grid = np.empty((100,100))
e2_grid = np.empty((100,100))
e3_grid = np.empty((100,100))
p1_grid, p2_grid = np.meshgrid(p1_vec,p2_vec)

# Now we can find the excess demands with a loop
for i,p1 in enumerate(p1_vec):
    for j,p2 in enumerate(p2_vec):
        e1_grid[i,j] = ex_dem1(p1,p2,e1,e2,e3)
        e2_grid[i,j] = ex_dem2(p1,p2,e1,e2,e3)
        e3_grid[i,j] = ex_dem3(p1,p2,e1,e2,e3)
```

Now we can make the plots of the excess demand functions

```python
fig = plt.figure(figsize=(10,19))

ax1 = fig.add_subplot(3,1,1,projection='3d')
fig1 = ax1.plot_surface(p1_grid, p2_grid, e1_grid, color='green', cmap='rainbow')
ax1.set_xlabel('$p_1$')
ax1.set_ylabel('$p_2$')
ax1.invert_xaxis()
ax1.set_title('Excess demand of Good 1', fontweight='bold')
fig.colorbar(fig1, shrink=0.5, aspect=5)

ax2 = fig.add_subplot(3,1,2,projection='3d')
fig2 = ax2.plot_surface(p1_grid, p2_grid, e2_grid, color='blue', cmap='rainbow')
ax2.set_xlabel('$p_1$')
ax2.set_ylabel('$p_2$')
ax2.invert_xaxis()
ax2.set_title('Excess demand of Good 2', fontweight='bold')
fig.colorbar(fig2, shrink=0.5, aspect=5)

ax3 = fig.add_subplot(3,1,3,projection='3d')
fig3 = ax3.plot_surface(p1_grid, p2_grid, e3_grid, color='red', cmap='rainbow')
ax3.set_xlabel('$p_1$')
ax3.set_ylabel('$p_2$')
ax3.invert_xaxis()
ax3.set_title('Excess demand of Good 3', fontweight='bold')
fig.colorbar(fig3, shrink=0.5, aspect=5)
```

The above figures show that the excess demand of the goods increase when the price is decreasing.

#### **Question 3:** Find the Walras-equilibrium prices, $(p_1,p_2)$, where both excess demands are (approximately) zero, e.g. by using the following tâtonnement process:

1. Guess on $p_1 > 0$, $p_2 > 0$ and choose tolerance $\epsilon > 0$ and adjustment aggressivity parameter, $\kappa > 0$.
2. Calculate $z_1(p_1,p_2)$ and $z_2(p_1,p_2)$.
3. If $|z_1| < \epsilon$ and $|z_2| < \epsilon$ then stop.
4. Else set $p_1 = p_1 + \kappa \frac{z_1}{N}$ and $p_2 = p_2 + \kappa \frac{z_2}{N}$ and return to step 2.

To find the Walras equilibrium prices $(p_1,p_2)$ where the excess demands are approximately zero the tâtonnement process described above is used.

```python
# 1. First we set the tolerance and adjustment aggressivity parameter and make initial guesses on the prices
kappa = 0.5
eps = 1e-8
p1 = 2
p2 = 1.2
maxi = 2500 # maximum iterations

def wal_eq(betas,p1,p2,e1,e2,e3,eps,kappa,maxi):
    t=0
    while True:
    # 2. Calculate excess demands of good 1 and 2
        z1 = ex_dem1(p1,p2,e1,e2,e3)
        z2 = ex_dem2(p1,p2,e1,e2,e3)

    # 3. Defining whether to stop
        if np.abs(z1) < eps and np.abs(z2) < eps or t >= maxi:
            print(f'{t:3d}: p1 = {p1:12.6f} --> excess demand good 1 --> {z1:14.6f}')
            print(f'{t:3d}: p2 = {p2:12.6f} --> excess demand good 2 --> {z2:14.6f}')
            break

    # 4. Updating prices and returning to step 2
        p1 = p1 + kappa*(z1/N)
        p2 = p2 + kappa*(z2/N)

        if t < 2 or t%150 == 0:
            print(f'{t:3d}: p1 = {p1:12.6f} --> excess demand good 1 --> {z1:14.6f}')
            print(f'{t:3d}: p2 = {p2:12.6f} --> excess demand good 2 --> {z2:14.6f}')
        elif t == 2:
            print('   ...')

        t += 1
    return p1,p2

p1,p2 = wal_eq(betas,p1,p2,e1,e2,e3,eps,kappa,maxi)
```

      0: p1 =     2.174261 --> excess demand good 1 -->   17426.145090
      0: p2 =     1.153944 --> excess demand good 2 -->   -4605.642486
      1: p1 =     2.313496 --> excess demand good 1 -->   13923.499507
      1: p2 =     1.140200 --> excess demand good 2 -->   -1374.391164
       ...
    150: p1 =     5.877854 --> excess demand good 1 -->     623.395516
    150: p2 =     2.388193 --> excess demand good 2 -->     232.810385
    300: p1 =     6.346131 --> excess demand good 1 -->     135.633734
    300: p2 =     2.562996 --> excess demand good 2 -->      50.610479
    450: p1 =     6.454289 --> excess demand good 1 -->      33.135580
    450: p2 =     2.603351 --> excess demand good 2 -->      12.362149
    600: p1 =     6.481055 --> excess demand good 1 -->       8.303433
    600: p2 =     2.613336 --> excess demand good 2 -->       3.097701
    750: p1 =     6.487783 --> excess demand good 1 -->       2.093723
    750: p2 =     2.615847 --> excess demand good 2 -->       0.781082
    900: p1 =     6.489481 --> excess demand good 1 -->       0.528758
    900: p2 =     2.616480 --> excess demand good 2 -->       0.197258
    1050: p1 =     6.489910 --> excess demand good 1 -->       0.133587
    1050: p2 =     2.616640 --> excess demand good 2 -->       0.049836
    1200: p1 =     6.490019 --> excess demand good 1 -->       0.033753
    1200: p2 =     2.616680 --> excess demand good 2 -->       0.012592
    1350: p1 =     6.490046 --> excess demand good 1 -->       0.008529
    1350: p2 =     2.616691 --> excess demand good 2 -->       0.003182
    1500: p1 =     6.490053 --> excess demand good 1 -->       0.002155
    1500: p2 =     2.616693 --> excess demand good 2 -->       0.000804
    1650: p1 =     6.490055 --> excess demand good 1 -->       0.000545
    1650: p2 =     2.616694 --> excess demand good 2 -->       0.000203
    1800: p1 =     6.490055 --> excess demand good 1 -->       0.000138
    1800: p2 =     2.616694 --> excess demand good 2 -->       0.000051
    1950: p1 =     6.490055 --> excess demand good 1 -->       0.000035
    1950: p2 =     2.616694 --> excess demand good 2 -->       0.000013
    2100: p1 =     6.490055 --> excess demand good 1 -->       0.000009
    2100: p2 =     2.616694 --> excess demand good 2 -->       0.000003
    2250: p1 =     6.490055 --> excess demand good 1 -->       0.000002
    2250: p2 =     2.616694 --> excess demand good 2 -->       0.000001
    2400: p1 =     6.490055 --> excess demand good 1 -->       0.000001
    2400: p2 =     2.616694 --> excess demand good 2 -->       0.000000
    2500: p1 =     6.490055 --> excess demand good 1 -->       0.000000
    2500: p2 =     2.616694 --> excess demand good 2 -->       0.000000

The Walras equilirium prices for good 1 and good 2 are:
$\quad \pmb{p_1 = 6.49} $
$\quad \pmb{p_2 = 2.62}$

#### **Question 4:** Plot the distribution of utility in the Walras-equilibrium and calculate its mean and variance.
First we define the utility function below,

```python
# Utility function
def util_eq(p1,p2,e1,e2,e3,gamma,betas):
    return ((dem1(p1,p2,e1,e2,e3,b1)**b1)*(dem2(p1,p2,e1,e2,e3,b2)**b2)*(dem3(p1,p2,e1,e2,e3,b3)**b3))**gamma

util = util_eq(p1,p2,e1,e2,e3,gamma,betas)

# Mean and variance of Walras-Equilibrium
print(f'The mean of the Walras-Equilibrium is {round(np.mean(util),2)}')
print(f'The variance of the Walras-Equilibrium is {round(np.var(util),2)}')
```

    The mean of the Walras-Equilibrium is 1.01
    The variance of the Walras-Equilibrium is 0.32

```python
# A histogram is used to plot the distribution
hist, ax = plt.subplots(figsize=(10,6))
ax.hist(util, bins=250, color='g', alpha=.5)
ax.set_title('Figure 3.3: Distribution of Utility in the Walras Equilibrium', size=20)
ax.set_xlabel('Utility', size=14)
ax.set_ylabel('Consumers', size=14)
plt.show()
```

#### **Question 5:** Find the Walras-equilibrium prices if instead all endowments were distributed equally. Discuss the implied changes in the distribution of utility. Does the value of $\gamma$ play a role for your conclusions?
To make equally distributed endowments we make a new random draw and set the endowments equal to each other. Hereafter, the prices are calculated again as in question 4 and the mean and variance values are found.

```python
# We make a new draw of e
e = np.random.exponential(zeta,size=N)
e = e1 = e2 = e3

# Now we repeat the function in 3.3 to find the Walras-Equilibrium with endowmens that are equally distributed
p1,p2 = wal_eq(betas,p1,p2,e1,e2,e3,eps,kappa,maxi)
```

      0: p1 =     6.489400 --> excess demand good 1 -->     -65.486179
      0: p2 =     2.616374 --> excess demand good 2 -->     -31.982385
      1: p1 =     6.488748 --> excess demand good 1 -->     -65.262291
      1: p2 =     2.616067 --> excess demand good 2 -->     -30.688036
       ...
    150: p1 =     6.434611 --> excess demand good 1 -->     -16.926318
    150: p2 =     2.595529 --> excess demand good 2 -->      -6.318294
    300: p1 =     6.421020 --> excess demand good 1 -->      -4.162094
    300: p2 =     2.590456 --> excess demand good 2 -->      -1.553668
    450: p1 =     6.417683 --> excess demand good 1 -->      -1.020161
    450: p2 =     2.589210 --> excess demand good 2 -->      -0.380818
    600: p1 =     6.416866 --> excess demand good 1 -->      -0.249852
    600: p2 =     2.588905 --> excess demand good 2 -->      -0.093268
    750: p1 =     6.416665 --> excess demand good 1 -->      -0.061181
    750: p2 =     2.588830 --> excess demand good 2 -->      -0.022838
    900: p1 =     6.416616 --> excess demand good 1 -->      -0.014980
    900: p2 =     2.588812 --> excess demand good 2 -->      -0.005592
    1050: p1 =     6.416604 --> excess demand good 1 -->      -0.003668
    1050: p2 =     2.588808 --> excess demand good 2 -->      -0.001369
    1200: p1 =     6.416601 --> excess demand good 1 -->      -0.000898
    1200: p2 =     2.588807 --> excess demand good 2 -->      -0.000335
    1350: p1 =     6.416601 --> excess demand good 1 -->      -0.000220
    1350: p2 =     2.588806 --> excess demand good 2 -->      -0.000082
    1500: p1 =     6.416600 --> excess demand good 1 -->      -0.000054
    1500: p2 =     2.588806 --> excess demand good 2 -->      -0.000020
    1650: p1 =     6.416600 --> excess demand good 1 -->      -0.000013
    1650: p2 =     2.588806 --> excess demand good 2 -->      -0.000005
    1800: p1 =     6.416600 --> excess demand good 1 -->      -0.000003
    1800: p2 =     2.588806 --> excess demand good 2 -->      -0.000001
    1950: p1 =     6.416600 --> excess demand good 1 -->      -0.000001
    1950: p2 =     2.588806 --> excess demand good 2 -->      -0.000000
    2100: p1 =     6.416600 --> excess demand good 1 -->      -0.000000
    2100: p2 =     2.588806 --> excess demand good 2 -->      -0.000000
    2250: p1 =     6.416600 --> excess demand good 1 -->      -0.000000
    2250: p2 =     2.588806 --> excess demand good 2 -->      -0.000000
    2400: p1 =     6.416600 --> excess demand good 1 -->      -0.000000
    2400: p2 =     2.588806 --> excess demand good 2 -->      -0.000000
    2416: p1 =     6.416600 --> excess demand good 1 -->      -0.000000
    2416: p2 =     2.588806 --> excess demand good 2 -->      -0.000000

The Walras equilirium prices for good 1 and good 2 with equally distributed endowments are:
$\quad \pmb{p_1 = 6.42} $
$\quad\pmb{p_2 = 2.59}$
Thus, both are a little lower compared to the prices in question 3.3 where the endowments are not equally distributed.
Now the two utility distributions are plotted in a histogram to be compared.

```python
# Function for new utilities
def util_eq(p1,p2,e1,e2,e3,gamma,betas):
    return ((dem1(p1,p2,e1,e2,e3,b1)**b1)*(dem2(p1,p2,e1,e2,e3,b2)**b2)*(dem3(p1,p2,e1,e2,e3,b3)**b3))**gamma

# Getting utilities
util_e = util_eq(p1,p2,e1,e2,e3,gamma,betas)

# Mean and variance of Walras-Equilibrium
print(f'The mean of the Walras-Equilibrium with equally distributed endowments is {round(np.mean(util_e),2)}')
print(f'The variance of the Walras-Equilibrium with equally distributed endowments is {round(np.var(util_e),2)}')
```

    The mean of the Walras-Equilibrium with equally distributed endowments is 0.98
    The variance of the Walras-Equilibrium with equally distributed endowments is 0.62

Thus, the mean is now slightly lower while the variance is twice as high with equally distributed endowments.

```python
# Plotting histograms in same figure to compare distributions
hist, ax = plt.subplots(figsize=(10,7))
ax.hist(util_e, bins=250, color='g', alpha=.5, label='Equal Distribution')
ax.hist(util,bins=250,color='b',alpha=.5, label= 'Unequal Distribution')
ax.set_title('Figure 3.4: Distribution Comparison of Utilities in the Walras Equilibrium', size=20)
ax.set_xlabel('Utility', size=14)
ax.set_ylabel('Consumers', size=14)
plt.legend(loc='lower right',edgecolor='black')
plt.show()
```

From the above figure it is seen that having equally distributed endowments draw the distribution towards 0.

To see how gamma influences the distribution of the utilities, an interactive graph where the gamma value can be changed is made.

```python
# Utility function to make figure
def util_sl(p1,p2,e1,e2,e3,gamma,betas):
    util_sl = ((dem1(p1,p2,e1,e2,e3,b1)**b1)*(dem2(p1,p2,e1,e2,e3,b2)**b2)*(dem3(p1,p2,e1,e2,e3,b3)**b3))**gamma
    return util_sl

# Making interactive figure with slider for gamma values
def interactive_figure(p1,p2,e1,e2,e3,gamma,betas):

    # a. calculations
    dist = util_sl(p1,p2,e1,e2,e3,gamma,betas)

    # b. figure
    fig = plt.figure(figsize=(10,7))
    ax = fig.add_subplot(1,1,1)
    ax.hist(dist,bins=250, color='g',alpha=.5)
    ax.set_title('Figure 3.5: Distribution with different values of $\gamma$', size=20)
    ax.set_xlabel('Utility', size=14)
    ax.set_ylabel('Consumers', size=14)

widgets.interact(interactive_figure,
    p1=widgets.fixed(p1),
    p2=widgets.fixed(p2),
    e1=widgets.fixed(e1),
    e2=widgets.fixed(e2),
    e3=widgets.fixed(e3),
    gamma=widgets.FloatSlider(description="$\gamma$", min=0.01, max=2, step=0.01, value=.8),
    betas=widgets.fixed(betas)
);

```

    interactive(children=(FloatSlider(value=0.8, description='$\\gamma$', max=2.0, min=0.01, step=0.01), Output())…

In the above figure it is shown that the gamma value has a big impact on how the utilities in the Walras Equilibrium are distributed. This is no surprise as gamma influences the budget shares in the Cobb-Douglas utility function given. A higher gamma will draw the distribution of utilites towards zero while a small gamma will draw the distribution towards a higher utility.


# Demo of IPython Notebook

## David Pine

```
2+3
```

    5

```
sin(pi/6.)
```

    0.49999999999999994

```
plot([1,2,3,2,3,4,3,4,5])
```

```
# Calculates time, gallons of gas used, and cost of gasoline for
# a trip

distance = float(raw_input("Input distance of trip in miles: "))
mpg = 30.               # car mileage
speed = 60.             # average speed
costPerGallon = 4.10    # price of gas

time = distance/speed
gallons = distance/mpg
cost = gallons*costPerGallon

print("\nDuration of trip = {0:0.1f} hours".format(time))
print("Gasoline used = {0:0.1f} gallons (@ {1:0.0f} mpg)"
	  .format(gallons, mpg))
print("Cost of gasoline = ${0:0.2f} (@ ${1:0.2f}/gallon)"
	  .format(cost, costPerGallon))
```

    Input distance of trip in miles: 450

    Duration of trip = 7.5 hours
    Gasoline used = 15.0 gallons (@ 30 mpg)
    Cost of gasoline = $61.50 (@ $4.10/gallon)

 The total distance $x$ traveled during a trip can be obtained by integrating the velocity $v(t)$ over the duration $T$ of the trip:
    \begin{align}
        x = \int_0^T v(t)\, dt
    \end{align}

```
!cat LiamSelinaData.txt
```

    Date: 2013-09-16
    Data taken by Liam and Selena
    frequency (Hz) amplitude (mm)  amp error (mm)
        0.7500        13.52         0.32
        1.7885        12.11         0.92
        2.8269        14.27         0.73
        3.8654        16.60         2.06
        4.9038        22.91         1.75
        5.9423        35.28         0.91
        6.9808        60.99         0.99
        8.0192        33.38         0.36
        9.0577        17.78         2.32
       10.0962        10.99         0.21
       11.1346         7.47         0.48
       12.1731         6.72         0.51
       13.2115         4.40         0.58
       14.2500         4.07         0.63

```

```


# Quantization of Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Oversampling

[Oversampling](https://en.wikipedia.org/wiki/Oversampling) is a technique which is applied in [analog-to-digital converters](https://en.wikipedia.org/wiki/Analog-to-digital_converter) to lower the average power of the quantization error. It requires a joint consideration of sampling and quantization.

### Ideal Analog-to-Digital Conversion

Let's consider the ideal sampling of a signal followed by its quantization, as given by the following block diagram

Ideal sampling is modeled by multiplying the continuous signal $x(t)$ with a series of equidistant Dirac functions, resulting in the discrete signal $x[k] = x(k T)$ where $T$ denotes the sampling interval. The discrete signal $x[k]$ is then quantized. The output of the ideal analog-to-digital converter is the quantized discrete signal $x_\text{Q}[k]$.

### Nyquist Sampling

Sampling of the continuous signal $x(t)$ leads to repetitions of the spectrum $X(j \omega) = \mathcal{F} \{ x(t) \}$ at multiples of $\omega_\text{S} = \frac{2 \pi}{T}$. We limit ourselves to a continuous real-valued $x(t) \in \mathbb{R}$, band-limted signal $| X(j \omega) | = 0$ for $|\omega| > \omega_\text{C}$ where $\omega_\text{C}$ denotes its cut-off frequency. The spectral repetitions due to sampling do not overlap if the [sampling theorem](https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem) $\omega_\text{S} \geq  2 \cdot \omega_\text{C}$ is fulfilled. In the case of Nyquist (critical) sampling, the sampling frequency is chosen as $\omega_\text{S} =  2 \cdot \omega_\text{C}$.

### Oversampling

The basic idea of oversampling is to sample the input signal at frequencies which are significantly higher than the Nyquist criterion dictates. After quantization, the signal is low-pass filtered by a discrete filter $H_\text{LP}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ and resampled back to the Nyquist rate. In order to avoid aliasing due to the resampling this filter has to be chosen as an ideal low-pass

\begin{equation}
H_\text{LP}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = \text{rect} \left( \frac{\Omega}{2 \, \Omega_\text{C}} \right)
\end{equation}

where $\Omega_\text{C} = \omega_\text{C} \cdot T$. For an oversampling of factor $L \in \mathbb{Z}$ we have $\omega_\text{S} = L \cdot 2 \omega_\text{C}$. For this case, the resampling can be realized by keeping only every $L$-th sample which is known as decimation. The following block diagram illustrates the building blocks of oversampled digital-to-analog conversion, $\downarrow L$ denotes decimation by a factor of $L$

In order the conclude on the benefits of oversampling we have to derive the average power of the overall quantization error. According to our [model of the quantization error](linear_uniform_quantization_error.ipynb#Model-for-the-Quantization-Error), the quantization error $e[k]$ can be modeled as uniformly distributed white noise. Its power spectral density (PSD) is given as

\begin{equation}
\Phi_{ee}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = \frac{Q^2}{12}
\end{equation}

where $Q$ denotes the quantization step. Before the discrete low-pass filter $H_\text{LP}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$, the power of the quantization error is uniformly distributed over the entire frequency range $-\pi < \Omega \leq \pi$. However, after the ideal low-pass filter the frequency range is limited to $- \frac{\pi}{L} < \Omega \leq \frac{\pi}{L}$. The average power of the quantization error is then given as

\begin{equation}
\sigma_{e, \text{LP}}^2 = \frac{1}{2 \pi} \int\limits_{- \frac{\pi}{L}}^{\frac{\pi}{L}} \Phi_{ee}(\mathrm{e}^{\,\mathrm{j}\,\Omega})  \; \mathrm{d}\Omega = \frac{1}{L} \cdot \frac{Q^2}{12}
\end{equation}

The average power $\sigma_x^2$ of the sampled signal $x[k]$ is not affected, since the cutoff frequency of the low-pass filter has been chosen as the upper frequency limit $\omega_\text{C}$ of the input signal $x(t)$.

In order to calculate the SNR of the oversampled analog-to-digital converter we assume that the input signal is drawn from a uniformly distributed zero-mean random process with $|x[k]| < x_\text{max}$. With the results from our discussion of [linear uniform quantization](linear_uniform_quantization_error.ipynb#Uniformly-Distributed-Signal) and $\sigma_{e, \text{LP}}^2$ from above we get

\begin{equation}
SNR = 10 \cdot \log_{10} \left( 2^{2 w} \right) + 10 \cdot \log_{10} \left( L \right) \approx 6.02 \, w + 10 \cdot \log_{10} \left( L \right) \quad \text{in dB}
\end{equation}

where $w$ denotes the number of bits used for a binary representation of the quantization index. Hence, oversampling by a factor of $L$ brings a plus of $10 \cdot \log_{10} \left( L \right)$ dB in terms of SNR. For instance, an oversampling by a factor of $L = 4$ results in a SNR which is approximately 6 dB higher. For equal SNR the quantization step $Q$ can be chosen larger. In terms the of wordlength of a quantizer this accounts to a reduction by one bit. Consequently, there is a trade-off between accuracy of the quantizer and its sampling frequency.

### Example

The following numerical simulation illustrates the benefit in terms of SNR for an oversampled linear uniform quantizer with $w = 16$ for the quantization of the harmonic signal $x[k] = \cos[\Omega_0 k]$.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import scipy.signal as sig

w = 16  # wordlength of the quantized signal
L = 2**np.arange(1,10)  # oversampling factors

N = 8192  # length of signals
Om0 = 100*2*np.pi/N  # frequency of harmonic signal
Q = 1/(2**(w-1))  # quantization step

def uniform_midtread_quantizer(x, Q):
    # limiter
    x = np.copy(x)
    idx = np.where(x <= -1)
    x[idx] = -1
    idx = np.where(x > 1 - Q)
    x[idx] = 1 - Q
    # linear uniform quantization
    xQ = Q * np.floor(x/Q + 1/2)

    return xQ

def SNR_oversampled_ADC(L):
    x = (1-Q)*np.cos(Om0*np.arange(N))
    xu = (1-Q)*np.cos(Om0*np.arange(N*L)/L)
    # quantize signal
    xQu = uniform_midtread_quantizer(xu, Q)
    # low-pass filtering and decimation
    xQ = sig.resample(xQu, N)
    # estimate SNR
    e = xQ - x

    return 10*np.log10((np.var(x)/np.var(e)))

# compute SNR for oversampled ADC
SNR = [SNR_oversampled_ADC(l) for l in L]

# plot result
plt.figure(figsize=(10, 4))
plt.semilogx(L, SNR, label='with oversampling')
plt.plot(L, (6.02*w+1.76)*np.ones(L.shape), label='without oversampling' )
plt.xlabel(r'oversampling factor $L$')
plt.ylabel(r'SNR in dB')
plt.legend(loc='upper left')
plt.grid()
```

**Exercise**

* What SNR can be achieved for an oversampling factor of $L=16$?
* By how many bits could the word length $w$ be reduced in order to gain the same SNR as without oversampling?

### Anti-Aliasing Filter

Besides an increased SNR, oversampling has also another benefit. In order to ensure that the input signal $x(t)$ is band-limited before sampling, a low-pass filter $H_\text{LP}(\mathrm{j}\,\omega)$ is applied in typical analog-to-digital converters. This is illustrated in the following

The filter $H_\text{LP}(\mathrm{j}\,\omega)$ is also known as [anti-aliasing filter](https://en.wikipedia.org/wiki/Anti-aliasing_filter). The ideal low-pass filter is given as $H_\text{LP}(\mathrm{j}\,\omega) = \text{rect}\left( \frac{\omega}{\omega_\text{S}}  \right)$. The ideal $H_\text{LP}(\mathrm{j}\,\omega)$ can only be approximated in the analog domain. Since the sampling rate is higher than the Nyquist rate, there is no need for a steep slope of the filter in order to avoid aliasing. However, the pass-band of the filter within $|\omega| < |\omega_\text{C}|$ has to be flat.

Before decimation, the discrete filter $H_\text{LP}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ has to remove the spectral contributions that may lead to aliasing. However, a discrete filter $H_\text{LP}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ with steep slope can be realized much easier than in the analog domain.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2017*.


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

import pandas_datareader.data as web
import datetime as dt
from statsmodels.stats.diagnostic import acorr_ljungbox
from statsmodels.tsa.stattools import acf, pacf, adfuller
```

```python
start_time, end_time = dt.datetime(2016,1,1), dt.datetime(2017,4,1)
TOPIX_df = web.DataReader("INDEXTOPIX:TOPIX", 'google', start_time, end_time)['Close'].rename(columns={'Close': 'P(t)'})
TOPIX_price_df = pd.concat([TOPIX_df, TOPIX_df.shift(1)], axis = 1).rename(columns={0:'P(t)', 1: 'P(t-1)'})[1:]
TOPIX_price_df['price_return'] = TOPIX_price_df['P(t)'] / TOPIX_price_df['P(t-1)'] - 1
TOPIX_price_df[:2]
```

      P(t)
      P(t-1)
      price_return

      Date

      2016-01-05
      1504.71
      1509.67
      -0.003285

      2016-01-06
      1488.84
      1504.71
      -0.010547

```python
plt.figure(figsize=(15, 6))
TOPIX_price_df['P(t)'].plot()
plt.title('price'); plt.show();
```

```python
plt.figure(figsize=(15, 6))
TOPIX_price_df['price_return'].plot()
plt.title('price return'); plt.show();
```

# ACF(autocorrelation function)

```python
close_acf = acf(TOPIX_price_df['price_return'])
print(close_acf[:5])
plt.figure(figsize=(15, 6))
plt.bar(range(len(close_acf)), close_acf, width = 0.01)
plt.show()
```

# pACF (partial autocorrelation function)

```python
close_acf = pacf(TOPIX_price_df['price_return'])
print(close_acf[:5])
plt.figure(figsize=(15, 6))
plt.bar(range(len(close_acf)), close_acf, width = 0.01)
plt.show()
```

# Ljung–Box test
Reference: https://en.wikipedia.org/wiki/Ljung%E2%80%93Box_test

The Ljung–Box test (named for Greta M. Ljung and George E. P. Box) is a type of statistical test of whether any of a group of autocorrelations of a time series are different from zero.

## The Ljung–Box test may be defined as
- H0: The data are **independently distributed** (i.e. the correlations in the population from which the sample is taken are 0, so that any observed correlations in the data result from randomness of the sampling process).
- Ha: The data are **not independently distributed**; they **exhibit serial correlation**.

## Statistics

\begin{equation}
Q = n\left(n+2\right)\sum_{k=1}^h\frac{\hat{\rho}^2_k}{n-k}
\end{equation}

For significance level $\alpha$, the critical region for rejection of the hypothesis of randomness is

\begin{equation}
Q > \chi_{1-\alpha,h}^2
\end{equation}

```python
def test_ljungbox(price_df, lag): _, pvalue = acorr_ljungbox(price_df['price_return'], lags=lag); print("P-value = %s on lag = %s"%(pvalue, lag) )
test_ljungbox(TOPIX_price_df, 1)
test_ljungbox(TOPIX_price_df, 2)
test_ljungbox(TOPIX_price_df, 3)
```

    P-value = [ 0.31528089] on lag = 1
    P-value = [ 0.31528089  0.57674997] on lag = 2
    P-value = [ 0.31528089  0.57674997  0.4448136 ] on lag = 3

# Dickey-Fuller test
Reference: https://en.wikipedia.org/wiki/Dickey%E2%80%93Fuller_test

the Dickey–Fuller test tests the null hypothesis of whether a unit root is present in an autoregressive model.

### 単位根の存在を確かめる検定
1. 統計量、もしくはディッキー–フラー検定（DF検定）
2. 根が1以上であるかの有意性検定（F検定）
3. フィリップス–ペロン検定（PP検定）
4. Dickey–Pantula 検定

```python
_, p_value, _, _, _, _ = adfuller(TOPIX_price_df['price_return']); print(p_value)
```

    2.19916281361e-30

p valueは、有意水準1%のもとで、帰無仮説は棄却される。従って、TOPIXの収益率は、単位根を有していない。

** 発散する動きに見えないため、収益率は定常性を満たしている **

と本文では表現していた。ホント？


### Probability and Distributions

Self-study :)

```python
import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline

from scipy.stats import multivariate_normal
```

```python
%%html
<style>
table {float:left}
</style>
```

<style>
table {float:left}
</style>

#### Bivariate distribution - Discrete random variables X and Y

Considering the following bivariate distribution $p(x,y)$ of two discrete random variables $X$ and $Y$:

|       |$x_{1}$|$x_{2}$|$x_{3}$|$x_{4}$|$x_{5}$|
|-------|-------|-------|-------|-------|-------|
|$y_{1}$|0.05   |0.1    |0.05   |0.07   |0.2    |
|$y_{2}$|0.1    |0.05   |0.03   |0.05   |0.04   |
|$y_{3}$|0.01   |0.02   |0.03   |0.1    |0.1    |

Let's compute the marginal distributions $p(x)$ and $p(y)$ and the conditional distributions $p(x | Y = y_{1})$ and $p(y | X = x_{3})$

We denote by $n_{ij}$ the number of events with state $X = x_{i}$ and $Y = y_{j}$. The probability distribution of each random variable, the marginal probability, can be seen as the sum over a row or column:

>$P(X = x_{i}) = \frac{c_{i}}{N}$

or

>$P(Y = y_{j}) = \frac{r_{j}}{N}$

Where $c_{i}$ is the sum of the individual frequencies for the *i*th columns, that is $\sum_{j=0}^{3} n_{ij}$ and $r_{j}$ is the row sum, that is, $r_{j} = \sum_{i=0}^{5} n_{ij}$

```python
bivariate_pmf = np.array([[0.05, 0.1, 0.05, 0.07, 0.2],
                          [0.1, 0.05, 0.03, 0.05, 0.05],
                          [0.01, 0.02, 0.03, 0.1, 0.1]])

N = np.multiply(*bivariate_pmf.shape)

c_i = np.sum(bivariate_pmf, axis=0)
r_j = np.sum(bivariate_pmf, axis=1)

P_X = c_i/N
P_Y = r_j/N
```

The marginal probability of each $X$ and $Y$ random variables are:

```python
(c_i, r_j,
 P_X, P_Y)
```

    (array([0.16, 0.17, 0.11, 0.22, 0.35]),
     array([0.47, 0.28, 0.26]),
     array([0.01066667, 0.01133333, 0.00733333, 0.01466667, 0.02333333]),
     array([0.03133333, 0.01866667, 0.01733333]))

The conditional probaility is the fraction of a row column in a particular cell. For example, the conditional probability of $Y$ given $X$ is:

>$P(Y = y_{j} | X = x_{i}) = \frac{n_{ij}}{c_{i}}$

and the conditional probability of $X$ given $Y$ is:

>$P(X = x_{i} | Y = y_{j}) = \frac{n_{ij}}{r_j}$

```python
P_Y_X = []
P_X_Y = []

for i in range(3):
    for j in range(5):
        n_i_j = bivariate_pmf[i,j]
        p_y_x_tmp = n_i_j / c_i[j]
        p_x_y_tmp = n_i_j / r_j[i]

        P_Y_X = np.hstack([P_Y_X, p_y_x_tmp])
        P_X_Y = np.hstack([P_X_Y, p_x_y_tmp])
```

```python
P_X_Y, P_Y_X
```

    (array([0.10638298, 0.21276596, 0.10638298, 0.14893617, 0.42553191,
            0.35714286, 0.17857143, 0.10714286, 0.17857143, 0.17857143,
            0.03846154, 0.07692308, 0.11538462, 0.38461538, 0.38461538]),
     array([0.3125    , 0.58823529, 0.45454545, 0.31818182, 0.57142857,
            0.625     , 0.29411765, 0.27272727, 0.22727273, 0.14285714,
            0.0625    , 0.11764706, 0.27272727, 0.45454545, 0.28571429]))

#### Two Gaussian Distributions

Consider a mixture of two Gaussian distributions, in the form:

$$p(\boldsymbol x) = \sum_{k=1}^K \pi_k \mathcal{N}(\boldsymbol x|\boldsymbol \mu_k, \boldsymbol \Sigma_k)\,,\quad \pi_k \geq 0\,,\quad \sum_{k=1}^K\pi_k = 1$$

for
$$\mu_{1} =
\begin{bmatrix}
10 \\
2 \\
\end{bmatrix},
\mu_{2} =
\begin{bmatrix}
0 \\
0 \\
\end{bmatrix},
\Sigma_{1} =
\begin{bmatrix}
1 & 0 \\
0 & 1 \\
\end{bmatrix},
\Sigma_{2} =
\begin{bmatrix}
8.4 & 2.0 \\
2.0 & 1.7 \\
\end{bmatrix},
$$
and $\pi_{1} = 0.4$, $\pi_{2} = 0.6$

Let's compute:

(a) The marginal distributions for each dimension

The marginal distribution $p(\boldsymbol{x})$ of a joint Gaussian distribution $p(\boldsymbol{x}, \boldsymbol{y})$ is itself Gaussian and computed by applying the sum rule:

>$$
\begin{equation}
p(\boldsymbol{x}) = \begin{cases}
     \sum_{y\in\mathcal{Y}} p(\boldsymbol{x}, \boldsymbol{y}) & \text{ if } \boldsymbol{y} \text{ is discrete}\\
     \int_{\mathcal{Y}} p(\boldsymbol{x}, \boldsymbol{y})d\boldsymbol{y} & \text{ if } \boldsymbol{y} \text{ is continous}\\
  \end{cases}
 \end{equation}
$$

So, applying the sum rule to the Gaussian distribution, we can compute the marginal distribution for one dimesion:

>$p(\boldsymbol{x}) = \int p(\boldsymbol{x}, \boldsymbol{y}) d\boldsymbol{y} = \mathcal{N}(\boldsymbol{x}|\boldsymbol\mu_{x}, \boldsymbol{\Sigma_{xx}})$

The same holds for $p(\boldsymbol{y})$, which is obtained by marginalizing with respect to $\boldsymbol{x}$. Thus, the marginals of the distributions of interest are:

>$p(\boldsymbol{x}) = 0.4 \mathcal{N}(\boldsymbol{x}|10, 1) + 0.6 \mathcal{N}(\boldsymbol{x}|0, 8.4)$

>$p(\boldsymbol{y}) = 0.4 \mathcal{N}(\boldsymbol{x}|2, 1) + 0.6 \mathcal{N}(\boldsymbol{x}|0, 1.7)$

Generate some data:

```python
# means
m = np.zeros((2,2))
m[0] = np.array([10, 0])
m[1] = np.array([2, 0])

# covariances
S = np.zeros((2, 2, 2))
S[0] = np.array([[1, 0],
                 [0, 1]])
S[1] = np.array([[8.4, 2.0],
                 [2.0, 1.7]])

# weights
w = np.array([0.4, 0.6])
```

```python
N_split = 200
N = N_split * 2

x = []
y = []

for k in range(2):
    x_tmp, y_tmp = np.random.multivariate_normal(m[k], S[k], N_split).T
    x = np.hstack([x, x_tmp])
    y = np.hstack([y, y_tmp])

data = np.vstack([x,y])
```

Visualization of the dataset:

```python
grid = sns.JointGrid(x=x, y=y)

g = grid.plot_joint(sns.scatterplot)
g.plot_joint(sns.kdeplot)
g.plot_marginals(sns.kdeplot)
```

(b) The mean, mode and median for each marginal distribution

The mean of the mixture density $p(x)$ is given by the weighted sum of the means of each random variable:

>$\mathbb{E}[x] = \alpha\mu_1 + (1 - \alpha)\mu_2$

```python
(m[:, :] * w)[:, 0]
```

    array([4. , 0.8])

Thus, we have:

$\mathbb{E}[x] = 4$

$\mathbb{E}[y] = 0.8$

The mode can be defined as the value of $x$ having the highest frequency of occurence. For a continous random variable, the mode is defined as a peak in the density $p(\boldsymbol{x})$. A particular density $p(\boldsymbol{x})$ may have more than one mode, and furthermore there may be a very large number of modes in high-dimensional distributions. Therefore, due to this computational challenge, finding the mode of a continous distribution can be computationally challenging.

One way though, is to compute the extreme condition:

>$\frac{\mathcal{d}p(\boldsymbol{x})}{\mathcal{d}x} = 0$ or $\frac{\mathcal{d}p(\boldsymbol{y})}{\mathcal{d}y} = 0$

And check if it is not an inflection point:

>$\frac{\mathcal{d}^2p(\boldsymbol{x})}{{\mathcal{d}x}^2} < 0$ or $\frac{\mathcal{d}p(\boldsymbol{y})}{{\mathcal{d}y}^2} < 0$

(c) The mean and mode for the two-dimensional distribution

The mean of a random variable $X$ with states $\boldsymbol{x}\in\mathbb{R}^D$is an average and is defined as:

$$\mathbb{E}=
\begin{bmatrix}
\mathbb{E}_{x_{1}}[x_{1}] \\
\vdots \\
\mathbb{E}_{x_{d}}[x_{d}] \\
\end{bmatrix}
$$

where

$$\mathbb{E}_{x_{d}}[x_{d}]=
\int_{\mathcal{X}} x_{d}p(x_{d})\, dx_{d} \text{ for d = 1, 2, ..., D}
$$

Since we already have the expectation values, the mean for the two-dimensional distribution is:

$$\mu=
\begin{bmatrix}
4.0 \\
0.8 \\
\end{bmatrix}
$$

The mode of two dimensional distribution is obtained first by solving the extremum conditions:

$\frac{\partial(x,y)}{\partial x} = 0$ and $\frac{\partial(x,y)}{\partial y} = 0$

and then verifying for every solution that is indeed a peak, i.e.:

$\frac{\partial^{2}(x,y)}{\partial x^{2}} < 0$ and $\frac{\partial^{2}(x,y)}{\partial y^{2}} < 0$

$$\text{det}
\begin{pmatrix}
\begin{bmatrix}
\frac{\partial^{2}(x,y)}{\partial x^{2}} & \frac{\partial^{2}(x,y)}{\partial y\partial x} & \\
\frac{\partial^{2}(x,y)}{\partial x\partial y} & \frac{\partial^{2}(x,y)}{\partial y^{2}} & \\
\end{bmatrix}
\end{pmatrix} > 0
$$

Again, these equations can be solved only numerically.

#### Bernoulli Distribution w/ Beta conjugation

Considering a Bernoulli distribution with a parameter $\theta$:

>$p(x|\theta) = \theta^{x} (1-\theta)^{1 - x}, x\in$ {0,1}

Let $x\in$ {0,1} be distributed according to the Bernoulli distribution with parameter $\theta\in$ [0,1], that is,

>$p(x = 1|\theta) = \theta$.

This can also be expressed as

>$p(x|\theta) = \theta^{x} (1-\theta)^{1-x}$

```python
X = np.random.binomial(1, 0.4, 1000)

plt.hist(X)
plt.title('Bernoulli Distribution')
plt.ylabel(r'p(x | $\theta$  = 0.4)')
plt.xlabel('x')
```

Let $\theta$ be distributed according to a Beta distribution with parameters $\alpha, \beta$, that is,

>$p(\theta|\alpha, \beta) \propto \theta^{\alpha - 1} (1 - \theta)^{\beta -1}$

```python
T = np.random.beta(4, 10, 1000)

plt.hist(T)
plt.title('Beta Distribution')
plt.ylabel(r'p($\theta$ | $\alpha$  = 4, $\beta$ = 10)')
plt.xlabel('x')
```

Now, multiplying the Beta and the Bernoulli distributions, we get:

>$$
\begin{equation}
\begin{split}
p(\theta|x, \alpha, \beta) & =  p(x|\theta) p(\theta|\alpha, \beta) \\
&\propto \theta^{x - 1} (1 - \theta)^{x - 1}\theta^{\alpha - 1} (1 - \theta)^{\beta -1} \\
& = \theta^{\alpha - x - 1} (1 - \theta)^{\beta - x} \\
& \propto p(\theta|\alpha + x + 1, \beta + (1 - x))
\end{split}
\end{equation}
$$

The last line is the Beta distribution with parameters $(\alpha + x + 1, \beta + (1 - x))$

Generallizing, the posterior distribution $p(\theta|x_{i}, \alpha, \beta)$ have the form of:

>$p(\theta |x_{1}, \dots, x_{N}) = \frac{1}{\mathcal{B}(\alpha+x_{i}, \beta + 1 - x_{i})} \theta^{\alpha - x - 1} (1 - \theta)^{\beta - x} $

#### Bayes' theorem

There are two bags. The first bag contains four mangos and two apples; the second bag contains four mangos and four apples. We also have a biased coin, which shows “heads” with probability 0.6 and
“tails” with probability 0.4. If the coin shows “heads”. we pick a fruit at random from bag 1; otherwise we pick a fruit at random from bag 2.

Your friend flips the coin (you cannot see the result), picks a fruit at random from the corresponding bag, and presents you a mango.

What is the probability that the mango was picked from bag 2?

The probability for each bag is given by:
$$
p(Bag=2) = 0.6\\
p(Bag=1) = 0.4\\
$$

The probabilities of being a mango given bags one and two are:
$$
p(Fruit=Mango | Bag=2) = \frac{4}{8} = \frac{1}{2}\\
p(Fruit=Mango | Bag=1) = \frac{4}{6} = \frac{2}{3}\\
$$

And the probability of being a mango:

$$
\begin{equation}
\begin{split}
p(Fruit=Mango) & = p(Fruit=Mango | Bag=1) * p(Bag=1) \\
& + p(Fruit=Mango | Bag=2) * p(Bag=2) \\
& = \frac{2}{3} * 0.6 + \frac{1}{2} * 0.4 =  0.6
\end{split}
\end{equation}
$$

Now, using the Bayes Theorem, we can calculate the probability of a mango was picked from bag two as:

$$
\begin{equation}
\begin{split}
p(Bag=2 | Fruit=Mango) &= \frac{p(Fruit=Mango|Bag=2) * p(Bag=2)}{p(Fruit=Mango)}\\
\end{split}
\end{equation}
$$

Where the prior is $p(Bag=2)$ and the evidence $p(Fruit=Mango)$. Thus:

$$
p(Bag=2 | Fruit=Mango) = \frac{\frac{1}{2} * 0.4}{0.6} = \frac{1}{3}
$$

```python
# Coin flip
C = np.array([0.6,  # Bag 1
              0.4]) # Bag 2
# Fruit
F = np.array([0.6*4/6 + 0.4*4/8,  # Mango
              0.6*2/6 + 0.4*4/8]) # Apple

# Coin Flip and Fruit (bag)
B = np.array([[4/6, 2/6],  # Mango Bag 1, Mango Bag 2
              [4/8, 4/8]]) # Apple Bag 1, Apple Bag 2
```

```python
(B * C / F)[0,1]
```

    0.3333333333333333


## Preface
I found the [topic](https://www.kaggle.com/c/porto-seguro-safe-driver-prediction/discussion/43384) started by [Yang Yu](https://www.kaggle.com/popyy0101) doubting about the significance of small improvement in Gini score. **Yang Yu** wrote that:

> Several years ago, one of my tutors told me that "any improvement within 0.001 level is just noise", which means it was just randomly switching 1 label out of 1000 samples in evaluation set (roughly).

The question is very interesting at least for **Yang Yu** and me so I decided to investigate that. Since I'm not a statistician but a software engineer I decided to use my programming skills to make Gini scoring on simulated data and answer the next very specific questions:
- How much samples it is needed to be correct to improve Gini from *0.286* to *0.287*?
- What is the probability to get from *0.286* to *0.287* by just random guessing?

Also I got a bit excited what I can do and played with public/private scoring. It's cool when you can simulate 0.290! :D

I would like to repeat that I'm not a statistician and I do not pretend on ultimate truth. The experiment I provide may be wrong and contain some things I am missing. Any criticism, suggestions, hints are very welcomed. The more I'm writing the more I'm doubting whether it's not a trash :)

## Assumptions
We assume the next things:
- Imbalance of positive and negative classes in the test set is the same as in the training set.
- Our model outputs uniformly distributed probabilities.
- Public and private leaderboard scores are independent from each other.

```python
import pandas as pd
train = pd.read_csv("../input/train.csv")
counts = train.target.value_counts()
print("Class distribution:")
counts / counts.sum()
```

    Class distribution:

    0    0.963552
    1    0.036448
    Name: target, dtype: float64

Appoximately 96.4% of the data are negative class labels. Let's create perfect submission(oh, here we can!) and target vector that correspond to the test data size.

```python
import numpy as np
test = pd.read_csv("../input/train.csv")
length = len(test)
np.random.seed(287)
perfect_sub = np.random.rand(length)
target = (perfect_sub > 0.963552).astype(dtype=int)
print("Perfect submission looks like: ", perfect_sub)
print("Target vector looks like: ", target)
print("Target vector class distibution: ")
counts = pd.Series(target).value_counts()
counts / counts.sum()
```

    Perfect submission looks like:  [ 0.14448523  0.71075312  0.87616033 ...,  0.3776089   0.76920165
      0.64545703]
    Target vector looks like:  [0 0 0 ..., 0 0 0]
    Target vector class distibution:

    0    0.963845
    1    0.036155
    dtype: float64

Almost the same imbalance we have in the training set.
Now let's define Normalized Gini Score. [Sklearn](http://scikit-learn.org/stable/index.html) package has [roc_auc_score](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html) and we are going to compute Normalized Gini with *AUC* using the next formula:
\begin{align}
Gini & = 2 * AUC - 1
\end{align}
Let's see how our perfect submission is evaluated by Gini. Hold your breath!

```python
from sklearn.metrics import roc_auc_score
def gini(y_target, y_score):
    return 2 * roc_auc_score(y_target, y_score) - 1
print(f"Gini on perfect submission: {gini(target, perfect_sub):0.5f}")
```

    Gini on perfect submission: 1.00000

Amazing! This is the highest score we can get from Gini and probably will never see on the leaderboard. Let's  also define completely random submission and test it.

```python
np.random.seed(8888)
random_sub = np.random.rand(length)
print(f"Gini on random submission: {gini(target, random_sub):0.5f}")
```

    Gini on random submission: 0.00078

Almost zero. That's what we expect from Gini when we have no idea about the data and use guessing.

We know that the public leaderboard is calculated on approximately 30% of the test data. So we should create private and public sets. Our generated answers are randomly distributed and we can just subset first 30% for public dataset and the rest 70% for private dataset. We also create function that evaluates public and private score.

```python
first30percent = int(length * 0.3)
target_pub = target[:first30percent]
target_prv = target[first30percent:]

def evaluate(submission):
    return gini(target_pub, submission[:first30percent]),\
        gini(target_prv, submission[first30percent:])
```

Let's define **spoiler** function. It fills our perfect submission with *n* randomly distributed random answers i.e. imputes the error in the result. Also we will spoil our perfect submission an look how Gini changes. The calculations take some time, let's respect Kaggle for calculating our sometimes trashy submissions.

```python
def spoiler(n, seed=None):
    if seed is not None:
        np.random.seed(seed)
    tospoil = np.random.choice(range(length), size=n, replace=False)
    submission = perfect_sub.copy()
    submission[tospoil] = np.random.rand()
    return submission

submissions = []
for spoil_n in range(0, length, 5000):
    score_pub, score_priv = evaluate(spoiler(spoil_n, spoil_n))
    submissions.append((spoil_n, score_pub, score_priv))
submissions = pd.DataFrame(submissions, columns = ["n", "public_score", "private_score"])
submissions.head()
```

      n
      public_score
      private_score

      0
      0
      1.000000
      1.000000

      1
      5000
      0.982766
      0.986919

      2
      10000
      0.970603
      0.970982

      3
      15000
      0.951957
      0.946309

      4
      20000
      0.995440
      0.995725

The chart showing dependence between number of spoiled samples and Gini is the following:

```python
import matplotlib.pyplot as plt
plt.figure(figsize=(11,6))
plt.plot(submissions["n"], submissions["public_score"], label="Public")
plt.plot(submissions["n"], submissions["private_score"], label = "Private")
plt.xlabel("Samples spoiled")
plt.ylabel("Gini Score")
_ = plt.legend()
```

Let's plot the distribution of difference between public score and private score.

```python
import seaborn as sns
pub_prv_diff = submissions["public_score"] - submissions["private_score"]
_ = sns.distplot(pub_prv_diff, hist=True)
_ = plt.xlabel("Public-Private Difference")
_ = plt.ylabel("Density")
pub_prv_diff.describe()
```

Public score seems to be usually higher than a private score(slightly left-skewed distribution).
Let's see inverse cumulative plot of absolute difference.

```python
_ = sns.distplot(abs(pub_prv_diff), bins=20,
             hist_kws=dict(cumulative=-1),
             kde_kws=dict(cumulative=True), kde=False, norm_hist=True)
_ = plt.xlabel("Public-Private Absolute Difference")
_ = plt.ylabel("Frequency")
```

It says the absolute difference more than *0.005* appears in every third case. It is a lot!
But this has nothing to do with guessing. It just tells us we should expect difference in public-private outcome.
I manually found the submission where the public Gini score is close to *0.286*. It required predicting more that *170000* samples like in the perfect submission.

```python
np.random.seed(123)
correct = 172560
index = np.random.choice(range(length), size=(length-correct), replace=False)
submission286 = perfect_sub.copy()
spoiled_samples = np.random.rand(length-correct)
submission286[index] = spoiled_samples
public, private = evaluate(submission286)
print(f"Public score: {public:0.4f}\nPrivate score: {private:0.4f}")
```

    Public score: 0.2862
    Private score: 0.2896

We should steadily add correct samples to our *submission286* in order to get 0.287. Sounds easy, doesn't it?
We will steadily increase number of correct answers from 5 to infinity with step 5 and give 30 tries for each addition because our correct samples are selected in a random order. We will stop when we reach 0.287.

```python
tries, fix = 30, 0
found = False
np.random.seed(10)
while not found:
    fix += 5
    print(f"Fixing {fix} samples")
    for t in range(tries):
        new_submission = submission286.copy()
        improve_index = np.random.choice(index, size=fix, replace=False)
        new_submission[improve_index] = perfect_sub[improve_index]
        public, _ = evaluate(new_submission)
        if public >= 0.287:
            print("0.287 reached!")
            found = True
            break
```

    Fixing 5 samples
    Fixing 10 samples
    Fixing 15 samples
    Fixing 20 samples
    Fixing 25 samples
    Fixing 30 samples
    Fixing 35 samples
    Fixing 40 samples
    Fixing 45 samples
    Fixing 50 samples
    Fixing 55 samples
    Fixing 60 samples
    Fixing 65 samples
    Fixing 70 samples
    Fixing 75 samples
    Fixing 80 samples
    Fixing 85 samples
    0.287 reached!

**It takes at least 85 samples to predict correctly to reach 0.287 in my dataset**. Now let's see if it is hard to predict 85 samples correctly by simple guessing. However guessing exactly *that* 85 samples may be a hard work if we don't know indexes. So it would be more fair to make guesses with variable number of samples. Since it is a time-consuming operation I decided to check only some certain number of samples to guess.

```python
# fix_samples contains number of samples to guess
fix_samples = [85, 100, 150, 200, 250, 500, 1000, 2000, 3000, 4000,
               6000, 7000, 10000, 20000, 30000, 40000, 50000, 60000,
              100000, 200000, 300000, 500000]
# Number of tries for each group of samples
tries = 300
scores, types = [], []
np.random.seed(888)
for fix in fix_samples:
    goal_counter = 0
    # Let's guess and repeat!
    for i in range(tries):
        new_submission = submission286.copy()
        guess = np.random.choice(range(length), size=fix, replace=False)
        new_submission[guess] = np.random.rand()
        public, _ = evaluate(new_submission)
        if public >= 0.287:
            goal_counter += 1
        scores.append(public)
        types.append(fix)
    print(f"Frequency(score>=0.287 | Guessed={fix} samples, Tries={tries} times) = " +
          f"{goal_counter/tries:0.3f}")
try_history = pd.DataFrame({"type": types, "score": scores})
print("Done!")
```

    Frequency(score>=0.287 | Guessed=85 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=100 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=150 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=200 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=250 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=500 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=1000 samples, Tries=300 times) = 0.003
    Frequency(score>=0.287 | Guessed=2000 samples, Tries=300 times) = 0.030
    Frequency(score>=0.287 | Guessed=3000 samples, Tries=300 times) = 0.010
    Frequency(score>=0.287 | Guessed=4000 samples, Tries=300 times) = 0.020
    Frequency(score>=0.287 | Guessed=6000 samples, Tries=300 times) = 0.033
    Frequency(score>=0.287 | Guessed=7000 samples, Tries=300 times) = 0.020
    Frequency(score>=0.287 | Guessed=10000 samples, Tries=300 times) = 0.017
    Frequency(score>=0.287 | Guessed=20000 samples, Tries=300 times) = 0.013
    Frequency(score>=0.287 | Guessed=30000 samples, Tries=300 times) = 0.013
    Frequency(score>=0.287 | Guessed=40000 samples, Tries=300 times) = 0.003
    Frequency(score>=0.287 | Guessed=50000 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=60000 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=100000 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=200000 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=300000 samples, Tries=300 times) = 0.000
    Frequency(score>=0.287 | Guessed=500000 samples, Tries=300 times) = 0.000
    Done!

Well, we see that **guessing sometimes works in order to get +0.001**! But it works extremely rarely. The best performance on guessing we achieved when our guesses touched ~2K-6K samples. For example, when we guessed 6K samples - we got 10 hits of 0.287 out of 300 tries. Is it feasable to achieve this when probing the leaderboard by the crowd? I believe yes.

Now let's see the results on the plot.

```python
sns.set(font_scale=0.9)
plt.figure(figsize=(15, 10))
ax = sns.stripplot(x="type", y="score", data=try_history, jitter=True, size=9, color="blue", alpha=.5)
ax.set_ylim(0.280, 0.288)
plt.axhline(0.2862, label="baseline", color="blue", linestyle="dashed")
plt.axhline(0.287, label="goal", color="orange", linestyle="dashed")
# Draw means for each group
mean = try_history.groupby("type")["score"].agg(["mean"])["mean"]
plt.plot(range(len(fix_samples)), mean, color="red")
plt.legend()
plt.grid()
```

The more samples we guess - the less our expected score and the larger the variance of our results. I would recommend to stay at 6K - the optimal number :-)

# Instead of a Smart Conclusion
Please do not consider the analysis as exactly applicable to the current leaderboard. This is just a testing of the particular random example.

I learned something... and you?

10 poer sugero safe driver prediction/is-your-small-gini-significant.ipynb