This notebook is part of the `clifford` documentation: https://clifford.readthedocs.io/.

# Application to Robotic Manipulators

This notebook is intended to expand upon the ideas in part of the presentation [Robots, Ganja & Screw Theory](https://slides.com/hugohadfield/game2020)

## Serial manipulator

[(slides)](https://slides.com/hugohadfield/game2020#/serial)

Let's consider a 2-link 3 DOF arm. We'll model the links within the robot with rotors, which transform to the coordinate frame of the end of each link. This is very similar to the approach that would classically be taken with 4×4 matrices.

We're going to define our class piecewise as we go along here.
To aid that, we'll write a simple base class to let us do just that.
In your own code, there's no need to do this.

```python
class AddMethodsAsWeGo:
    @classmethod
    def _add_method(cls, m):
        if isinstance(m, property):
            name = (m.fget or m.fset).__name__
        else:
            name = m.__name__
        setattr(cls, name, m)
```

Let's start by defining some names for the links, and a place to store our parameters:

```python
from enum import Enum

class Links(Enum):
    BASE = 'b'
    SHOULDER = 's'
    UPPER = 'u'
    ELBOW = 'e'
    FOREARM = 'f'
    ENDPOINT = 'n'

class SerialRobot(AddMethodsAsWeGo):
    def __init__(self, rho, l):
        self.l = l
        self.rho = rho
        self._thetas = (0, 0, 0)

    @property
    def thetas(self):
        return self._thetas
```

### Forward kinematics
[(slides)](https://slides.com/hugohadfield/game2020#/serial-forward-rotors)

As a reminder, we can construct rotation and translation motors as:
$$
\begin{align}
T(a)
  &= \exp \left(\frac{1}{2} n_{\infty} \wedge a \right) \\
  &= 1 + \frac{1}{2}n_{\infty} \wedge a \\
R(\theta, \hat B)
  &= \exp (\frac{1}{2} \theta \hat B) \\
  &= \cos \frac{\theta}{2} + \sin \frac{\theta}{2} \hat B
\end{align}
$$

Applying these to our geometry, we get

$$
\begin{align}
R_{\text{base} \gets \text{shoulder}} &= R(\theta_0, e_1 \wedge e_3) \\
R_{\text{shoulder} \gets \text{upper arm}} &= R(\theta_1, e_1 \wedge e_2) \\
R_{\text{upper arm} \gets \text{elbow}} &= T(\rho e_1) \\
R_{\text{elbow} \gets \text{forearm}} &= R(\theta_2, e_1 \wedge e_2) \\
R_{\text{forearm} \gets \text{endpoint}} &= T(-l e_1)\\
\end{align}
$$

From which we can get the overall rotor to the frame of the endpoint, and the positions $X$ and $Y$:

$$
\begin{align}
R_{\text{base} \gets \text{elbow}}
  &= R_{\text{base} \gets \text{shoulder}} R_{\text{shoulder} \gets \text{upper arm}} R_{\text{upper arm} \gets \text{elbow}} \\
X &= R_{\text{base} \gets \text{elbow}} n_0 \tilde R_{\text{base} \gets \text{elbow}} \\
R_{\text{base} \gets \text{endpoint}}
  &= R_{\text{base} \gets \text{shoulder}} R_{\text{shoulder} \gets \text{upper arm}} R_{\text{upper arm} \gets \text{elbow}} R_{\text{elbow} \gets \text{forearm}} R_{\text{forearm} \gets \text{endpoint}} \\
Y &= R_{\text{base} \gets \text{endpoint}} n_0 \tilde R_{\text{base} \gets \text{endpoint}} \\
\end{align}
$$

We can write this as:

```python
from clifford.g3c import *
from clifford.tools.g3c import generate_translation_rotor, apply_rotor
from clifford.tools.g3 import generate_rotation_rotor

def _update_chain(rotors, a, b, c):
    rotors[a, c] = rotors[a, b] * rotors[b, c]

@SerialRobot._add_method
@SerialRobot.thetas.setter
def thetas(self, value):
    theta0, theta1, theta2 = self._thetas = value
    # shorthands for brevity
    R = generate_rotation_rotor
    T = generate_translation_rotor

    rotors = {}
    rotors[Links.BASE, Links.SHOULDER] = R(theta0, e1, e3)
    rotors[Links.SHOULDER, Links.UPPER] = R(theta1, e1, e2)
    rotors[Links.UPPER, Links.ELBOW] = T(self.rho * e1)
    rotors[Links.ELBOW, Links.FOREARM] = R(theta2, e1, e2)
    rotors[Links.FOREARM, Links.ENDPOINT] = T(-self.l * e1)

    _update_chain(rotors, Links.BASE, Links.SHOULDER, Links.UPPER)
    _update_chain(rotors, Links.BASE, Links.UPPER, Links.ELBOW)
    _update_chain(rotors, Links.BASE, Links.ELBOW, Links.FOREARM)
    _update_chain(rotors, Links.BASE, Links.FOREARM, Links.ENDPOINT)
    self.rotors = rotors

@SerialRobot._add_method
@property
def y_pos(self):
    return apply_rotor(eo, self.rotors[Links.BASE, Links.ENDPOINT])

@SerialRobot._add_method
@property
def x_pos(self):
    return apply_rotor(eo, self.rotors[Links.BASE, Links.ELBOW])
```

Let's write a renderer so we can check this all works

```python
from pyganja import GanjaScene

def add_rotor(sc: GanjaScene, r, *, label=None, color=None, scale=0.1):
    """ show how a rotor transforms the axes at the origin """
    y = apply_rotor(eo, r)
    y_frame = [
        apply_rotor(d, r)
        for d in [up(scale*e1), up(scale*e2), up(scale*e3)]
    ]
    sc.add_object(y, label=label, color=color)
    sc.add_facet([y, y_frame[0]], color=(255, 0, 0))
    sc.add_facet([y, y_frame[1]], color=(0, 255, 0))
    sc.add_facet([y, y_frame[2]], color=(0, 0, 255))

@SerialRobot._add_method
def to_scene(self):
    sc = GanjaScene()
    axis_scale = 0.1
    link_scale = 0.05
    arm_color = (192, 192, 192)

    base_obj = (up(0.2*e1)^up(0.2*e3)^up(-0.2*e1)).normal()
    sc.add_object(base_obj, color=0)

    shoulder_axis = [
        apply_rotor(p, self.rotors[Links.BASE, Links.UPPER])
        for p in [up(axis_scale*e3), up(-axis_scale*e3)]
    ]
    sc.add_facet(shoulder_axis, color=(0, 0, 128))
    shoulder_angle = [
        apply_rotor(eo, self.rotors[Links.BASE, Links.SHOULDER]),
        apply_rotor(up(axis_scale*e1), self.rotors[Links.BASE, Links.SHOULDER]),
        apply_rotor(up(axis_scale*e1), self.rotors[Links.BASE, Links.UPPER]),
    ]
    sc.add_facet(shoulder_angle, color=(0, 0, 128))

    upper_arm_points = [
        apply_rotor(up(link_scale*e3), self.rotors[Links.BASE, Links.UPPER]),
        apply_rotor(up(-link_scale*e3), self.rotors[Links.BASE, Links.UPPER]),
        apply_rotor(up(link_scale*e3), self.rotors[Links.BASE, Links.ELBOW]),
        apply_rotor(up(-link_scale*e3), self.rotors[Links.BASE, Links.ELBOW])
    ]
    sc.add_facet(upper_arm_points[:3], color=arm_color)
    sc.add_facet(upper_arm_points[1:], color=arm_color)

    elbow_axis = [
        apply_rotor(p, self.rotors[Links.BASE, Links.ELBOW])
        for p in [up(axis_scale*e3), up(-axis_scale*e3)]
    ]
    sc.add_facet(elbow_axis, color=(0, 0, 128))

    forearm_points = [
        apply_rotor(up(link_scale*e3), self.rotors[Links.BASE, Links.FOREARM]),
        apply_rotor(up(-link_scale*e3), self.rotors[Links.BASE, Links.FOREARM]),
        apply_rotor(up(link_scale*e3), self.rotors[Links.BASE, Links.ENDPOINT]),
        apply_rotor(up(-link_scale*e3), self.rotors[Links.BASE, Links.ENDPOINT])
    ]
    sc.add_facet(forearm_points[:3], color=arm_color)
    sc.add_facet(forearm_points[1:], color=arm_color)

    add_rotor(sc, self.rotors[Links.BASE, Links.ELBOW], label='x', color=(128, 128, 128))
    add_rotor(sc, self.rotors[Links.BASE, Links.ENDPOINT], label='y', color=(128, 128, 128))

    return sc
```

We can now instantiate our robot

```python
serial_robot = SerialRobot(rho=1, l=0.5)
```

Choose a trajectory

```python
import math
theta_traj = [
    (math.pi/6 + i*math.pi/12, math.pi/3 - math.pi/12*i, 3*math.pi/4)
    for i in range(3)
]
```

And plot the robot in each state, using `ipywidgets` ([docs](https://ipywidgets.readthedocs.io/)) to let us plot ganja side-by-side.
Unfortunately, `pyganja` provides no mechanism to animate these plots from python.

<div class="alert alert-info">
This will not render side-by-side in the online clifford documentation, but will in a local notebook.
</div>

```python
import ipywidgets
from IPython.display import Latex, display
from pyganja import draw

outputs = [
    ipywidgets.Output(layout=ipywidgets.Layout(flex='1'))
    for i in range(len(theta_traj))
]
for output, thetas in zip(outputs, theta_traj):
    with output:
        # interesting part here - run the forward kinematics, print the angles we used
        serial_robot.thetas = thetas
        display(Latex(r"$\theta_i = {:.2f}, {:.2f}, {:.2f}$".format(*thetas)))
        draw(serial_robot.to_scene(), scale=1.5)
ipywidgets.HBox(outputs)
```

### Inverse kinematics

[(slides)](https://slides.com/hugohadfield/game2020#/serial-reverse)

For the forward kinematics, we didn't actually need conformal geometric algebra at all—PGA would have done just fine, as all we needed were rotations and translations.
The inverse kinematics of a serial manipulator is where CGA provide some nice tricks.

There are three facts we know about the position $X$, each of which describes a constraint surface

* $X$ must lie on a sphere with radius $l$ centered at $Y$, which can be written
  $$S^* = Y - \frac{1}{2}l^2n_\infty$$

* $X$ must lie on a sphere with radius $\rho$ centered at $n_o$, which can be written
  $$S_\text{base}^* = n_0 - \frac{1}{2}\rho^2n_\infty$$

* $X$ must lie on a plane through $n_o$, $e_3$, and $Y$, which can be written
  $$\Pi = n_0\wedge \operatorname{up}(e_3)\wedge Y\wedge n_\infty$$

  Note that $\Pi = 0$ is possible iff $Y = \operatorname{up}(ke_3)$.

For $X$ to satisfy all three constraints. we have

\begin{align}
S \wedge X = S_\text{base} \wedge X = \Pi \wedge X &= 0 \\
X \wedge (\underbrace{S \vee S_\text{base} \vee \Pi}_P) &= 0 \quad\text{If $\Pi \ne 0$} \\
X \wedge (\underbrace{S \vee S_\text{base}}_C) &= 0 \quad\text{otherwise} \\
\end{align}

By looking at the grade of the term labelled $P$, we conclude it must be a point-pair—which tells us $X$ must lie in one of two locations. Similarly, $C$ must be a circle.

```python
@SerialRobot._add_method
def _get_x_constraints_for(self, Y):
    """ Get the space containing all possible elbow positions """
    # strictly should be undual, but we don't have that in clifford
    S = (Y - 0.5*self.l**2*einf).dual()
    S_base = (eo - 0.5*self.rho**2*einf).dual()
    Pi = eo ^ up(e2) ^ Y ^ einf
    return S, S_base, Pi

@SerialRobot._add_method
def _get_x_positions_for(self, Y):
    """ Get the space containing all possible elbow positions """
    S, S_base, Pi = self._get_x_constraints_for(Y)
    if Pi == 0:
        # any solution on the circle is OK
        return S & S_base
    else:
        # there are just two solutions
        return S & S_base & Pi
```

From the pointpair $P$ we can extract the two possible $X$ locations with:
$$
X = \left[1 \pm \frac{P}{\sqrt{P\tilde{P}}}\right](P\cdot n_\infty)
$$

To be considered a full solution to the inverse kinematics problem, we need to produce the angles $\theta_0, \theta_1, \theta_2$.
We can do this as follows

```python
@SerialRobot._add_method
@SerialRobot.y_pos.setter
def y_pos(self, Y):
    R = generate_rotation_rotor
    T = generate_translation_rotor

    rotors = {}
    rotors[Links.UPPER, Links.ELBOW] = T(self.rho * e1)
    rotors[Links.FOREARM, Links.ENDPOINT] = T(-self.l * e1)

    x_options = self._get_x_positions_for(Y)
    if x_options.grades == {3}:
        # no need to adjust the base angle
        theta_0 = self.thetas[0]
        rotors[Links.BASE, Links.SHOULDER] = self.rotors[Links.BASE, Links.SHOULDER]
        # remove the rotation from x, intersect it with the plane of the links
        x_options = x_options & (eo ^ up(e3) ^ up(e1) ^ einf)
    else:
        y_down = down(Y)
        theta0 = math.atan2(y_down[(3,)], y_down[(1,)])
        rotors[Links.BASE, Links.SHOULDER] = R(theta0, e1, e3)

        # remove the first rotor from x
        x_options = apply_rotor(x_options, ~rotors[Links.BASE, Links.SHOULDER])

    # project out one end of the point-pair
    x = (1 - x_options.normal()) * (x_options | einf)

    x_down = down(x)
    theta1 = math.atan2(x_down[(2,)], x_down[(1,)])
    rotors[Links.SHOULDER, Links.UPPER] = R(theta1, e1, e2)

    _update_chain(rotors, Links.BASE, Links.SHOULDER, Links.UPPER)
    _update_chain(rotors, Links.BASE, Links.UPPER, Links.ELBOW)

    # remove the second rotor
    Y = apply_rotor(Y, ~rotors[Links.BASE, Links.ELBOW])
    y_down = down(Y)

    theta2 = math.atan2(-y_down[(2,)], -y_down[(1,)])
    rotors[Links.ELBOW, Links.FOREARM] = R(theta2, e1, e2)
    _update_chain(rotors, Links.BASE, Links.ELBOW, Links.FOREARM)
    _update_chain(rotors, Links.BASE, Links.FOREARM, Links.ENDPOINT)

    self._thetas = (theta0, theta1, theta2)
    self.rotors = rotors
```

Define a trajectory again, this time with a scene to render it:

```python
y_traj = [
    up(0.3*e3 + 0.8*e2 - 0.25*e1),
    up(0.6*e3 + 0.8*e2),
    up(0.9*e3 + 0.8*e2 + 0.25*e1)
]

expected_scene = GanjaScene()
expected_scene.add_facet(y_traj[0:2], color=(255, 128, 128))
expected_scene.add_facet(y_traj[1:3], color=(255, 128, 128))
```

And we can run the inverse kinematics by setting `serial_robot.y_pos`:

```python
outputs = [
    ipywidgets.Output(layout=ipywidgets.Layout(flex='1'))
    for i in range(len(y_traj))
]
first = True
for output, y in zip(outputs, y_traj):
    with output:
        # interesting part here - run the reverse kinematics, print the angles we used
        serial_robot.y_pos = y
        display(Latex(r"$\theta_i = {:.2f}, {:.2f}, {:.2f}$".format(*serial_robot.thetas)))
        sc = serial_robot.to_scene()

        # Show the spheres we used to construct the solution
        sc += expected_scene
        if first:
            extra_scene = GanjaScene()
            S, S_base, Pi = serial_robot._get_x_constraints_for(y)
            extra_scene.add_object(S_base, label='S_base', color=(255, 255, 128))
            extra_scene.add_object(S, label='S', color=(255, 128, 128))
            extra_scene.add_object(Pi, label='Pi', color=(128, 255, 192, 128))
            sc += extra_scene
        draw(sc, scale=1.5)
    first = False
ipywidgets.HBox(outputs)
```

## Parallel manipulators

For now, refer to the presentation

[(slides)](https://slides.com/hugohadfield/game2020#/parallel)

### Inverse kinematics

[(slides)](https://slides.com/hugohadfield/game2020#/agile-3dof-inverse)

For now, refer to the presentation

### Forward kinematics

[(slides)](https://slides.com/hugohadfield/game2020#/agile-2dof-forward)

For now, refer to the presentation


# Section 6: Advanced Simulation Methods

---

## Table of Contents
* [cadCAD Simulation Methods](#cadCAD-Simulation-Methods)
  1. [Monte Carlo Method](#1.-Monte-Carlo-Method)
  2. [Parameter Sweeps](#2.-Parameter-Sweeps)
  3. [A/B Testing](#3.-A/B-Testing)
* [cadCAD Simulation Methods Overview](#cadCAD-Simulation-Methods-Overview)
* [Data Manipulation & Analysis](#Data-Manipulation-&-Analysis)

---

\begin{align}
\large population_t &\large= population_{t-1} + {\Delta population} \quad \textrm{(sheep)} \tag{1} \\
\large food_t &\large= food_{t-1} + {\Delta food} \quad \textrm{(tons of grass)} \tag{2}
\end{align}

where the rate of change ($\Delta$) is:
\begin{align}
\large {\Delta population} &\large= \alpha * food_{t-1} - \epsilon * population_{t-1} \quad \textrm{(sheep/month)} \\
\large {\Delta food} &\large= -\beta * population_{t-1} + \gamma \quad \textrm{(tons of grass/month)}
\end{align}

where:

$
\begin{align}
\alpha: \quad &\textrm{'reproduction_rate'}\\
\epsilon: \quad &\textrm{'death_rate'}\\
\beta: \quad &\textrm{'consumption_rate'}\\
\gamma: \quad &\textrm{'growth_rate'}\\
\end{align}
$

* A population consumes a food source, and reproduces at a rate proportional to the food source $\alpha$ (alpha), and dies at a rate proportional to the population size $\epsilon$ (epsilon).
* The food source is consumed at a rate proportional to the population $\beta$ (beta), and grows at a constant rate $\gamma$ (gamma).

Import and run the existing ecosystem model code from section 5:

```python
import math
import pandas as pd
import plotly

pd.options.plotting.backend = "plotly"

from cadCAD.configuration.utils import config_sim

from cadCAD.engine import ExecutionMode, ExecutionContext
from cadCAD.engine import Executor
```

```python
from cadCAD.configuration import Experiment

from cadCAD import configs
del configs[:] # Clear any prior configs

experiment = Experiment()
```

# cadCAD Simulation Methods

1. [Monte Carlo Method](#1.-Monte-Carlo-Method)
2. [Parameter Sweeps](#2.-Parameter-Sweeps)
3. [A/B testing](#3.-A/B-Testing)

## 1. Monte Carlo Method

> Monte Carlo methods, or Monte Carlo experiments, are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. The underlying concept is to use randomness to solve problems that might be deterministic in principle.

Source: https://en.wikipedia.org/wiki/Monte_Carlo_method

### Non-determinism

> In computer science, a nondeterministic algorithm is an algorithm that, even for the same input, can exhibit different behaviors on different runs, as opposed to a deterministic algorithm.

Source: https://en.wikipedia.org/wiki/Nondeterministic_algorithm

\begin{align}
\large {\Delta food} &\large= -\beta * population_{t-1} + \gamma * rand() \quad \textrm{(tons of grass/month)} \\
\end{align}

* Addition of a random food growth rate, using `rand()`.

```python
from numpy import random
```

```python
random.rand()
```

```python
# Setting a seed so that we can reproduce the experiment
random.seed(1234)
```

```python
# With a given seed, generates a random number from a given set
[
    random.rand(),
    random.rand()
]
```

```python
random.seed(1234)
[
    random.rand(),
    random.rand()
]
```

```python
random.seed(None)
```

```python
seeds = [
    random.RandomState(1),
    random.RandomState(2),
    random.RandomState(3),
]
```

```python
[
    seeds[0].rand(),
    seeds[1].rand(),
    seeds[2].rand(),
]
```

```python
MONTE_CARLO_RUNS = 3

seeds = [random.RandomState(i) for i in range(MONTE_CARLO_RUNS)]
seeds
```

```python
initial_state = {
    'population': 50, # number of sheep
    'food': 1000 # tons of grass
}

system_params = {
    'reproduction_rate': [0.3],
    'death_rate': [0.03],
    'consumption_rate': [0.03],
    'growth_rate': [30.0],
}
```

```python
def s_population(params, substep, state_history, previous_state, policy_input):
    population = previous_state['population'] + policy_input['delta_population']
    return 'population', max(math.ceil(population), 0)

def s_food(params, substep, state_history, previous_state, policy_input):
    food = previous_state['food'] + policy_input['delta_food']
    return 'food', max(food, 0)
```

```python
def p_reproduction(params, substep, state_history, previous_state):
    population_reproduction = params['reproduction_rate'] * previous_state['food']
    return {'delta_population': population_reproduction}

def p_death(params, substep, state_history, previous_state):
    population_death = params['death_rate'] * previous_state['population']
    return {'delta_population': -population_death}

def p_growth(params, substep, state_history, previous_state):
    run = previous_state['run']
    food_growth = params['growth_rate'] * seeds[run - 1].rand()
    return {'delta_food': food_growth}

def p_consumption(params, substep, state_history, previous_state):
    food_consumption = params['consumption_rate'] * previous_state['population']
    return {'delta_food': -food_consumption}
```

```python
partial_state_update_blocks = [
    {
        'policies': {
            'reproduction': p_reproduction,
            'death': p_death,
            'consumption': p_consumption,
            'growth': p_growth
        },
        'variables': {
            'population': s_population,
            'food': s_food
        }
    }
]
```

```python
SIMULATION_TIMESTEPS = 500
```

```python
sim_config = config_sim({
    'N': MONTE_CARLO_RUNS,
    'T': range(SIMULATION_TIMESTEPS),
    'M': system_params
})

experiment.append_configs(
    initial_state = initial_state,
    partial_state_update_blocks = partial_state_update_blocks,
    sim_configs = sim_config,
)
```

```python
exec_mode = ExecutionMode()
exec_context = ExecutionContext()

simulation = Executor(exec_context=exec_context, configs=configs)
raw_result, tensor_field, sessions = simulation.execute()
```

```python
simulation_result = pd.DataFrame(raw_result)
simulation_result
```

```python
df = simulation_result.copy()
df = df[df.simulation == 0]
df
```

```python
df[df.run == 1].head()
```

```python
df[df.run == 2].head()
```

```python
df[df.run == 3].head()
```

### Simulation Analysis

```python
import plotly.express as px

px.line(df, x='timestep', y=['population', 'food'], facet_row='run', height=800, width=1000)
```

## 2. Parameter Sweeps

```python
dictionary = {
    'reproduction_rate': [0.1, 0.2],
    'death_rate': [0.01, 0.02]
}
dictionary['reproduction_rate'][0]
```

```python
nested_dictionary = {
    'population': [
        {'reproduction_rate': 0.1, 'death_rate': 0.01},
        {'reproduction_rate': 0.2, 'death_rate': 0.02},
    ]
}
nested_dictionary['population'][0]['reproduction_rate']
```

```python
initial_state = {
    'population': 50, # number of sheep
    'food': 1000 # tons of grass
}

system_params = {
    'population': [
        {'reproduction_rate': 0.1, 'death_rate': 0.01},
        {'reproduction_rate': 0.2, 'death_rate': 0.02},
        {'reproduction_rate': 0.3, 'death_rate': 0.03}
    ],
    'food': [
        {'consumption_rate': 0.01, 'growth_rate': 10.0},
        {'consumption_rate': 0.02, 'growth_rate': 20.0},
        {'consumption_rate': 0.03, 'growth_rate': 30.0}
    ]
}
```

```python
{
    'population': system_params['population'][0],
    'food': system_params['food'][0]
}
```

```python
{
    'population': system_params['population'][1],
    'food': system_params['food'][1]
}
```

```python
def s_population(params, substep, state_history, previous_state, policy_input):
    population = previous_state['population'] + policy_input['delta_population']
    return 'population', max(math.ceil(population), 0)

def s_food(params, substep, state_history, previous_state, policy_input):
    food = previous_state['food'] + policy_input['delta_food']
    return 'food', max(food, 0)
```

```python
def p_reproduction(params, substep, state_history, previous_state):
    population_reproduction = params['population']['reproduction_rate'] * previous_state['food']
    return {'delta_population': population_reproduction}

def p_death(params, substep, state_history, previous_state):
    population_death = params['population']['death_rate'] * previous_state['population']
    return {'delta_population': -population_death}

def p_consumption(params, substep, state_history, previous_state):
    food_consumption = params['food']['consumption_rate'] * previous_state['population']
    return {'delta_food': -food_consumption}

def p_growth(params, substep, state_history, previous_state):
    food_growth = params['food']['growth_rate']
    return {'delta_food': food_growth}
```

```python
partial_state_update_blocks = [
    {
        'policies': {
            'reproduction': p_reproduction,
            'death': p_death,
            'consumption': p_consumption,
            'growth': p_growth
        },
        'variables': {
            'population': s_population,
            'food': s_food
        }
    }
]
```

```python
MONTE_CARLO_RUNS = 1
SIMULATION_TIMESTEPS = 500

sim_config = config_sim({
    'N': MONTE_CARLO_RUNS,
    'T': range(SIMULATION_TIMESTEPS),
    'M': system_params
})
```

```python
from pprint import pprint

print('system_params')
pprint(system_params)
print(' ')
print('sim_config')
pprint(sim_config)
```

```python
experiment.append_configs(
    initial_state = initial_state,
    partial_state_update_blocks = partial_state_update_blocks,
    sim_configs = sim_config
)
```

```python
exec_mode = ExecutionMode()
exec_context = ExecutionContext()

simulation = Executor(exec_context=exec_context, configs=configs)
raw_result, tensor_field, sessions = simulation.execute()
```

```python
simulation_result = pd.DataFrame(raw_result)
simulation_result
```

```python
df = simulation_result.copy()
df = df[df.simulation == 1]
df
```

### Simulation Analysis

```python
import plotly.express as px

px.line(df, x='timestep', y=['population', 'food'], facet_row='subset', height=800, width=1000)
```

## 3. A/B Testing

```python
df = simulation_result.copy()
df
```

```python
import plotly.express as px

fig = px.line(
    df,
    x='timestep',
    y=['population', 'food'],
    facet_row='simulation',
    facet_col='run',
    height=800,
    template='seaborn'
)

fig.update_layout(
    margin=dict(l=20, r=20, t=20, b=20),
)

fig.show()
```

# cadCAD Simulation Methods Overview

# Data Manipulation & Analysis

```python
df = simulation_result.copy()
df
```

```python
df = df.query("simulation == 1 and run == 2")
df
```

```python
fig = px.scatter(
    df[df.timestep >= 80],
    x='population',
    y='food',
    color='timestep'
)
fig.show()
```

# Well done!
You're now well on your way to being a cadCAD system modeller!


# Elman

Elman网络大致可分为四层：输入层、隐藏层、承接层和输出层

* 输入层：接受信号输入
* 隐藏层：线性以及非线性变换
* 输出层：加权输出
* 承接层：承接上一时刻的信息

相较于简单的BP网络，Elman网络具有承接层，从而对序列数据具有记忆能力，实现动态建模

## 符号定义

|符号|含义|
|:-:|:-:|
|$\bm{x(t)}$|t时刻的输入信号|
|$\bm{h}$|隐藏层输出|
|$\bm{\hat{y}}$|模型预测输出|
|$\bm{y}$|输入信号对应的真实输出|
|$\_i$|索引为i的值|
|$^j\_i$|第j个向量索引为i的值|
|$\bm{V}$|输入层到隐藏层的连接权矩阵|
|$\bm{b_1}$|输入层到隐藏层的偏置|
|$\bm{W}$|隐藏层到输出层的连接权矩阵|
|$\bm{b_2}$|隐藏层到输出层的偏置|
|$\bm{U}$|承接层到隐藏层的连接权矩阵|
|$f$|隐藏层激活函数|
|$g$|输出层激活函数|
|$n$|输入信号维度|
|$k$|隐藏层维度|
|$o$|输出层维度|
|$d$|训练集总数|
|$T$|总时间步|

## 正向计算

1. **输入层与承接层信号到隐藏层**
$$
\begin{equation}
    \begin{split}
        \bm{h(t)}
        &= f(\bm{V}, \bm{U}, \bm{x(t)}, \bm{h(t-1)}, \bm{b_1}) \\
        &= f(\bm{V}\bm{x(t)} + \bm{U}\bm{h(t-1)} + \bm{b_1})
    \end{split}
\end{equation}
$$
2. **隐藏层到输出层**
$$
\begin{equation}
    \begin{split}
        \bm{\hat{y}(t)}
        &= g(\bm{W}, \bm{h(t)}, \bm{b_2}) \\
        &= g(\bm{W} \bm{h(t)} + \bm{b_2})
    \end{split}
\end{equation}
$$
3. **损失计算**
* 单个训练样本的损失
Elman要求t个时刻的损失均计入总损失，在这里使用$l_2$损失，当然，随着任务类型的变换可以换为交叉熵等
$$
\begin{equation}
    \mathcal{L_j(\bm{V}, \bm{W}, \bm{U}, \bm{b_1}, \bm{b_2})} = \sum_{t=1}^T||\bm{\hat{y(t)}^j}-\bm{y(t)^j}||_2^2
\end{equation}
$$
* 训练集总损失
$$
\begin{equation}
    \mathcal{L(\bm{V}, \bm{W}, \bm{U}, \bm{b_1}, \bm{b_2})} = \sum_{j=1}^d\sum_{t=1}^T ||\bm{\hat{y(t)}^j}-\bm{y(t)^j}||_2^2
\end{equation}
$$

## 误差传递与参数更新

上述正向推理和BP网络基本一致，唯一不同的是在输入层到隐藏层的时候需要同时考虑当前时刻的输入和上一时刻隐藏层的输出。

这里值得注意的是：

上述流程虽然看起来存在环，但是实际上是不存在的。Elman依然是**有向无环**的结构。一般Elman示意图会将上一时刻和当前时刻绘制在同一张图中，但是其结构可以展开为下图的结构。显然，对于当前时刻，整个模型并不存在环，虽然误差会在不同时刻的参数中传递，但是并不会形成环。

值得注意的是，Elman展开后虽然为一个有向无环图，但是其中的权重在不同时刻是**共享**的。

说回来，如果真的是有向有环图，Elman网络也不可能使用梯度下降法进行优化了

Elman网络的优化方法有两大类，分别为**实时循环学习(Real-Time Recurrent Learning, RTRL)**和**时序反向传播(BackPropagation Through Time, BPTT)**。这两类方法均基于BP算法，不同在于，RTRL在前向计算时就计算梯度并在每一个时间步更新参数，BPTT则在完成所有时间步的运算后进行梯度反向传播和参数更新。显然，BPTT需要保存大量的中间结果，其存储开销大；而RTRL则不需要存储多余的中间结果，其存储开销小，但是运算量大。

在这里，主要是对BPTT进行推导

### BPTT

* 单时间步(单样本)
首先考虑仅有一个时间步，并将偏执并入权值，此时不存在沿时间的误差传递
    * $\bm{W}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{W}}
        &= \frac{\partial \mathcal{L}}{\partial \bm{W}\bm{h(1)}}\frac{\partial \bm{W}\bm{h(1)}}{\partial \bm{W}}
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{W}\bm{h}(1)}
        &= 2(\bm{\hat{y}(1)}-\bm{y(1)})^Tdiag(g^{\prime}(\bm{W}\bm{h}(1))) \\
        &= \bm{\delta_1(1)}
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{W}}
        &= \bm{\delta_1(1)}^T\bm{h}(1)^T
    \end{split}
\end{equation}
$$
    * $\bm{h}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{h}(1)}
        &= \frac{\partial \mathcal{L}}{\partial \bm{W}\bm{h}(1)}\frac{\partial \bm{W}\bm{h}(1)}{\partial \bm{h}(1)} \\
        &= \bm{\delta_1(1)}\bm{W}
    \end{split}
\end{equation}
$$

    * $\bm{V}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{V}}
        &= \frac{\partial \mathcal{L}}{\partial \bm{V}\bm{x}(1)}\frac{\partial \bm{V}\bm{x}(1)}{\partial \bm{V}}
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{V}\bm{x}(1)}
        &= \frac{\partial \mathcal{L}}{\partial \bm{h}(1)}\frac{\partial \bm{h}(1)}{\partial \bm{V}\bm{x}(1)} \\
        &= \bm{\delta_1(1)}\bm{W}diag(f^{\prime}(\bm{V\bm{x}(1)})) \\
        &= \bm{\delta_2}(1)
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{V}}
        &= \bm{\delta_2}(1)^T\bm{x}(1)^T
    \end{split}
\end{equation}
$$

    * $\bm{U}$
    由于仅有一个时间步，$\bm{h}(0)=\bm{0}$，因此损失对$\bm{U}$的偏导为0

* 2时间步
同理可得
    * $\bm{W}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{W}}
        &= \bm{\delta_1(2)}^T\bm{h}(2)^T
    \end{split}
\end{equation}
$$
    * $\bm{h}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{h}(2)}
        &= \frac{\partial \mathcal{L}}{\partial \bm{W}\bm{h}(2)}\frac{\partial \bm{W}\bm{h}(2)}{\partial \bm{h}(2)} \\
        &= \bm{\delta_1(2)}\bm{W}
    \end{split}
\end{equation}
$$
    * $\bm{V}$
$$
\begin{equation}
    \frac{\partial \mathcal{L}}{\partial \bm{V}} = \frac{\partial \mathcal{L}}{\partial \bm{h}(2)}(\frac{\partial \bm{h}(2)}{\partial \bm{V}} + \frac{\partial \bm{h}(2)}{\partial \bm{h}(1)}\frac{\partial \bm{h(1)}}{\partial \bm{V}})
\end{equation}
$$
上式和仅有一个时间步的计算显然不同，这是由于对于权重矩阵$\bm{V}$，其不仅用于计算当前时刻的隐藏层输出，还应用于计算上一时刻的隐藏层输出，因此在计算梯度时，应当沿着时序进行反向传播
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L}}{\partial \bm{h}(2)}\frac{\partial \bm{h}(2)}{\partial \bm{h}(1)}
    &= \frac{\partial \mathcal{L}}{\partial \bm{h}(2)}\frac{\partial \bm{h}(2)}{\partial \bm{U}\bm{h}(1)}\frac{\partial \bm{U}\bm{h}(1)}{\partial \bm{h}(1)}
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L}}{\partial \bm{U}\bm{h}(1)}
    &= \bm{\delta_1(2)}\bm{W}diag(f^{\prime}(\bm{V}\bm{x}(2)+\bm{U}\bm{h}(1))) \\
    &= \bm{\delta_3(2)}
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L}}{\partial \bm{h}(1)}
    &= \frac{\partial \mathcal{L}}{\partial \bm{U}\bm{h}(1)}\frac{\partial \bm{U}\bm{h}(1)}{\bm{h}(1)} \\
    &= \bm{\delta_3(2)}\bm{U}
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L}}{\partial \bm{V}}
    &= \bm{\delta_2}(2)^T\bm{x}(2)^T + \frac{\partial \mathcal{L}}{\partial \bm{h}(1)}\frac{\partial \bm{h}(1)}{\partial \bm{V}} \\
    &= \bm{\delta_2}(2)^T\bm{x}(2)^T + \frac{\partial \mathcal{L}}{\partial \bm{h}(1)}(\frac{\partial \bm{h}(1)}{\partial \bm{V}\bm{x(1)}}\frac{\partial \bm{V}\bm{x(1)}}{\partial \bm{V}} + \frac{\partial \bm{h}(1)}{\partial \bm{U}\bm{h(0)}}\frac{\partial \bm{U}\bm{h(0)}}{\partial \bm{V}}) \\
    &= \bm{\delta_2}(2)^T\bm{x}(2)^T + (\bm{\delta_3(2)}\bm{U}diag(f^{\prime}(\bm{V}\bm{x(1)})))^T\bm{x(1)}^T
    \end{split}
\end{equation}
$$
    * $\bm{U}$
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L}}{\partial \bm{U}}
    &= \frac{\partial \mathcal{L}}{\partial \bm{h(2)}}(\frac{\partial \bm{h(2)}}{\partial \bm{U}}+\frac{\partial \bm{h(2)}}{\partial \bm{h(1)}}\frac{\partial \bm{h(1)}}{\partial \bm{U}}) \\
    &= \frac{\partial \mathcal{L}}{\partial \bm{U}\bm{h}(1)}\frac{\partial \bm{U}\bm{h}(1)}{\partial \bm{U}} + \frac{\partial \mathcal{L}}{\partial \bm{h}(1)}\frac{\partial \bm{h}(1)}{\partial \bm{U}} \\
    &= \bm{\delta_3(2)}^T\bm{h}(1)^T
    \end{split}
\end{equation}
$$

* t时间步
同上有
    * $\bm{W}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{W}}
        &= \bm{\delta_1(t)}^T\bm{h}(t)^T
    \end{split}
\end{equation}
$$
    * $\bm{h}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{h}(t)}
        &= \frac{\partial \mathcal{L}}{\partial \bm{W}\bm{h}(t)}\frac{\partial \bm{W}\bm{h}(t)}{\partial \bm{h}(t)} \\
        &= \bm{\delta_1(t)}\bm{W}
    \end{split}
\end{equation}
$$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{h}(t-1)}
        &= \frac{\partial \mathcal{L}}{\partial \bm{U}\bm{h}(t-1)}\frac{\partial \bm{U}\bm{h}(t-1)}{\partial \bm{h}(t-1)} \\
        &= \bm{\delta_3(t)}\bm{U}
    \end{split}
\end{equation}
$$
    * $\bm{V}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{V}}
        &= \frac{\partial \mathcal{L}}{\partial \bm{V}\bm{x(t)}}\frac{\partial \bm{V}\bm{x(t)}}{\partial \bm{V}} + \frac{\partial \mathcal{L}}{\partial \bm{h(t-1)}}\frac{\partial \bm{h(t-1)}}{\partial \bm{V}} \\
        &= \bm{\delta_2(t)}^T\bm{x(t)}^T + \bm{\delta_3(t)}\bm{U}\frac{\partial \bm{h(t-1)}}{\partial \bm{V}}
    \end{split}
\end{equation}
$$
    * $\bm{U}$
$$
\begin{equation}
    \begin{split}
        \frac{\partial \mathcal{L}}{\partial \bm{U}}
        &= \frac{\partial \mathcal{L}}{\partial \bm{U}\bm{h}(t-1)}\frac{\partial \bm{U}\bm{h}(t-1)}{\partial \bm{U}} + \frac{\partial \mathcal{L}}{\partial \bm{h}(t-1)}\frac{\partial \bm{h}(t-1)}{\partial \bm{U}} \\
        &= \bm{\delta_3(t)}^T\bm{h}(t-1)^T + \bm{\delta_3(t)}\bm{U}\frac{\partial \bm{h}(t-1)}{\partial \bm{U}}
    \end{split}
\end{equation}
$$

式-23和式-24中存在两个偏导数：$\frac{\partial \bm{h(t-1)}}{\partial \bm{V}}$和$\frac{\partial \bm{h}(t-1)}{\partial \bm{U}}$

显然，这两个偏导数是沿时间的递归公式，需要沿时间进行递归计算，而且由于是向量对矩阵的偏导，计算较为复杂

下面将使用另外一种方式进行推导

* $\bm{W}$
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L(t)}}{\partial \bm{W}}
    &= \frac{\partial \mathcal{L(t)}}{\partial \bm{\hat{y(t)}}}\frac{\partial \bm{\hat{y(t)}}}{\partial \bm{W}\bm{h(t)}}\frac{\partial \bm{W}\bm{h(t)}}{\partial \bm{W}} \\
    &= \bm{\delta_1(t)}^T\bm{h}(t)^T
    \end{split}
\end{equation}
$$

* $\bm{V}$
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L(t)}}{\partial \bm{V}}
    &= \frac{\partial \mathcal{L(t)}}{\partial \bm{\hat{y(t)}}}\frac{\bm{\hat{y(t)}}}{\partial \bm{h(t)}}\frac{\partial \bm{h(t)}}{\partial \bm{V}} \\
    &= \sum_{q=1}^t\frac{\partial \mathcal{L(t)}}{\partial \bm{\hat{y(t)}}}\frac{\bm{\hat{y(t)}}}{\partial \bm{h(t)}}\frac{\partial \bm{h(t)}}{\partial \bm{h(q)}}\frac{\partial \bm{h(q)}}{\partial \bm{V}} \\
    &= \sum_{q=1}^t\frac{\partial \mathcal{L(t)}}{\partial \bm{\hat{y(t)}}}\frac{\bm{\hat{y(t)}}}{\partial \bm{h(t)}}(\prod_{j=q+1}^{t}\frac{\partial \bm{h(j)}}{\partial \bm{h(j-1)}})\frac{\partial \bm{h(q)}}{\partial \bm{V}} \\
    \end{split}
\end{equation}
$$

* $\bm{U}$
$$
\begin{equation}
    \begin{split}
    \frac{\partial \mathcal{L(t)}}{\partial \bm{U}}
    &= \frac{\partial \mathcal{L(t)}}{\partial \bm{\hat{y(t)}}}\frac{\bm{\hat{y(t)}}}{\partial \bm{h(t)}}\frac{\partial \bm{h(t)}}{\partial \bm{U}} \\
    &= \sum_{q=1}^t\frac{\partial \mathcal{L(t)}}{\partial \bm{\hat{y(t)}}}\frac{\bm{\hat{y(t)}}}{\partial \bm{h(t)}}\frac{\partial \bm{h(t)}}{\partial \bm{h(q)}}\frac{\partial \bm{h(q)}}{\partial \bm{U}} \\
    &= \sum_{q=1}^t\frac{\partial \mathcal{L(t)}}{\partial \bm{\hat{y(t)}}}\frac{\bm{\hat{y(t)}}}{\partial \bm{h(t)}}(\prod_{j=q+1}^{t}\frac{\partial \bm{h(j)}}{\partial \bm{h(j-1)}})\frac{\partial \bm{h(q)}}{\partial \bm{U}} \\
    \end{split}
\end{equation}
$$

在式26、27中，将沿时间的误差反向传播完全展开，每一个$\frac{\partial \bm{h(q)}}{\partial \bm{U}}$或是$\frac{\partial \bm{h(q)}}{\partial \bm{V}}$仅要对当前层的权重进行求解，而无需沿时间迭代计算，在一定程度上降低了计算难度

```python
import numpy as np
import matplotlib.pyplot as plt
import imageio
import os
import random

# seed
random.seed(1024)
np.random.seed(1024)
%matplotlib inline
```

```python
# 定义单隐藏层Elman神经网络

class MyElman(object):

    def __init__(self, input_dims, hidden_dims, output_dims, max_back_time_step=5):
        """
        input_dims: 输入维度
        hidden_dims: 隐藏层神经元数量
        output_dims: 输出维度
        max_back_time_step: 反向传播最大时间步数
        """

        self.input_dims = input_dims
        self.hidden_dims = hidden_dims
        self.output_dims = output_dims
        self.max_back_time_step = max_back_time_step

        # 模型权重
        random.seed(1024)
        np.random.seed(1024)
        self.weight_input2hidden = np.random.randn(hidden_dims, input_dims+1)
        self.weight_lasthidden2hidden = np.random.randn(hidden_dims, hidden_dims)
        self.weight_hidden2output = np.random.randn(output_dims, hidden_dims)

        # 激活函数
        self.activate = self.activate_func
        # 激活函数微分，事先准备好
        self.activate_dif = self.activate_dif_func

        # 中间结果

        # 输入数据 (input_dims, 1)
        self.input_vector = None
        # 输入数据记录 (t, input_dims, 1)
        self.input_vectors = list()

        # 隐藏层输出 (hidden_dims, 1)
        self.hidden_output_vector = None
        # 隐藏层输出记录 (t, hidden_dims, 1)
        self.hidden_output_vectors = list()

        # 未激活隐藏层输出 (batch, hidden_dims)
        self.hidden_output_vector_no_activate = None
        # 未激活隐藏层输出记录 (t, hidden_dims, 1)
        self.hidden_output_vectors_no_activate = list()

        # 输出 (output_dims, 1)
        self.output_vector = None
        # 输出记录 (t, output_dims, 1)
        self.output_vectors = list()

        # 梯度结果
        self.grad_hidden2output = None
        self.grad_input2hidden = None
        self.grad_lasthidden2hidden = None

    def cla(self):
        # 中间结果

        # 输入数据 (input_dims, 1)
        self.input_vector = None
        # 输入数据记录 (t, input_dims, 1)
        self.input_vectors = list()

        # 隐藏层输出 (hidden_dims, 1)
        self.hidden_output_vector = None
        # 隐藏层输出记录 (t, hidden_dims, 1)
        self.hidden_output_vectors = list()

        # 未激活隐藏层输出 (batch, hidden_dims)
        self.hidden_output_vector_no_activate = None
        # 未激活隐藏层输出记录 (t, hidden_dims, 1)
        self.hidden_output_vectors_no_activate = list()

        # 输出 (output_dims, 1)
        self.output_vector = None
        # 输出记录 (t, output_dims, 1)
        self.output_vectors = list()

        # 梯度结果
        self.grad_hidden2output = None
        self.grad_input2hidden = None
        self.grad_lasthidden2hidden = None

    def forward(self, input_data):
        """
        正向计算
        input_data: 输入数据，其长度可以任意长度
        """

        # 首先判断输入能切分为多少长度
        total_t = len(input_data)-self.input_dims
        self.cla()
        self.hidden_output_vector = np.zeros((self.hidden_dims, 1))
        for t in range(total_t):

            # 输入向量
            self.input_vector = input_data[t: t + self.input_dims].reshape((-1, 1))

            # 隐藏层输出
            expend_input_vector = np.concatenate((self.input_vector, [[1]]), axis=0)
            self.hidden_output_vector_no_activate = np.matmul(self.weight_input2hidden, expend_input_vector) + np.matmul(self.weight_lasthidden2hidden, self.hidden_output_vector)
            self.hidden_output_vector = self.activate(self.hidden_output_vector_no_activate)

            # 输出层输出
            self.output_vector = np.matmul(self.weight_hidden2output, self.hidden_output_vector)

            # 数据记录
            self.input_vectors.append(expend_input_vector)
            self.hidden_output_vectors.append(self.hidden_output_vector_no_activate)
            self.hidden_output_vectors_no_activate.append(self.hidden_output_vector)
            self.output_vectors.append(self.output_vector)

        return np.array(self.output_vectors)

    def BPTT(self, train_seq, label_list):
        """
        BPTT更新参数
        label_list: 标签序列
        """

        # 正向推理
        self.forward(train_seq)

        max_t = len(label_list)

        self.grad_input2hidden = np.zeros(self.weight_input2hidden.shape)
        self.grad_hidden2output = np.zeros(self.weight_hidden2output.shape)
        self.grad_lasthidden2hidden = np.zeros(self.weight_lasthidden2hidden.shape)

        delta_1 = np.zeros((max_t, self.output_dims))
        for t in np.arange(max_t)[::-1]:
            delta_1[t] = 2 * (self.output_vectors[t]-label_list[t]).reshape(-1)

            self.grad_hidden2output += np.outer(delta_1[t], self.hidden_output_vectors[t])

            # 初始化dL/d(Vx(t)+Uh(t-1))
            delta_t = self.weight_hidden2output.T.dot(delta_1[t].reshape(-1, 1)) * self.activate_dif_func(self.hidden_output_vectors_no_activate[t])
            for bptt_step in np.arange(max(0, t-self.max_back_time_step), t+1)[::-1]:
                self.grad_lasthidden2hidden += np.outer(delta_t, self.hidden_output_vectors[bptt_step-1])

                self.grad_input2hidden += np.outer(delta_t, self.input_vectors[bptt_step])

                # 更新dL/d(Vx(t)+Uh(t-1))至前一个时刻
                delta_t = self.weight_lasthidden2hidden.T.dot(delta_t) * self.activate_dif_func(self.hidden_output_vectors_no_activate[bptt_step-1])

        self.grad_input2hidden /= max_t
        self.grad_hidden2output /= max_t
        self.grad_lasthidden2hidden /= max_t

    def fit(self, train_seq, label_list, lr, iteration=1000, draw_fig=False, x_list=None):
        """
        优化
        train_seq: 训练序列
        label_list: 标签序列
        lr: 学习率
        """
        fig_count = 1
        print_frequency = 1
        plt.figure(figsize=(20, 10))
        for i in range(iteration):
            # BPTT
            self.BPTT(train_seq, label_list)

            # 参数更新
            self.weight_hidden2output -= self.grad_hidden2output * lr
            self.weight_input2hidden -= self.grad_input2hidden * lr
            self.weight_lasthidden2hidden -= self.grad_lasthidden2hidden * lr

            if i % print_frequency == 0:
                pred_list = self.forward(train_seq).reshape(-1)

                if draw_fig:
                    self.draw_func(x_list, train_seq, label_list, fig_count, iter_=i)

                fig_count += 1
                print_frequency = int(pow(1.1, fig_count))

                train_loss = np.mean((pred_list-label_list)**2)
                print("[{}/{}], train_loss:{:.5f}".format(i, iteration, train_loss))

        pred_list = self.forward(train_seq).reshape(-1)
        train_loss = np.mean((pred_list-label_list)**2)
        print("[{}/{}], train_loss:{:.5f}".format(iteration, iteration, train_loss))

        self.draw_func(x_list, train_seq, label_list, fig_count, iter_=iteration, if_show=True)

    def draw_func(self, x_list, train_seq, label_list, fig_count, iter_=0, if_show=False):

        pred_list = self.forward(train_seq)

        plt.cla()
        plt.clf()
        plt.title("example & iter:{}".format(iter_), fontsize=20)
        plt.xticks(fontsize=20)
        plt.yticks(fontsize=20)
        plt.plot(x_list[self.input_dims:], label_list, label="Ground True", linewidth=3)
        plt.plot(x_list[self.input_dims:], pred_list.reshape(-1), label="Pred", linewidth=3)
        plt.legend(loc=1, fontsize=15)
        plt.savefig("./images/temp/{}.png".format(fig_count), dpi=100)
        if if_show:
            plt.show()

    @staticmethod
    def activate_func(x):
        """
        sigmoid
        """
        return 1/(1+np.exp(-x))

    @staticmethod
    def activate_dif_func(activate_result):
        return activate_result*(1-activate_result)
```

```python
# ----------------------- 测试数据 ---------------------

# 输入维度
input_dims = 10
# 输出维度
output_dims = 1
# 隐藏层维度
hidden_dims = 15
# 序列总长度
seq_len = 200

# 待拟合函数 cos(x)sin(x)
def arm_func(input_x):
    return np.cos(input_x) * np.sin(input_x)

# 定义序列
x_list = np.linspace(0, 10, seq_len)
seq_data = arm_func(x_list)
# 目标输出
seq_output = seq_data[input_dims:]

# ---------------- 定义模型 --------------------
elman_nn = MyElman(input_dims, hidden_dims, output_dims, max_back_time_step=3)

# ---------------- 正向推理 --------------------
plt.figure(figsize=(20, 10))
elman_nn.draw_func(x_list, seq_data, seq_output, fig_count=0, if_show=True)

# ---------------- BPTT训练 ---------------------
elman_nn.fit(seq_data, seq_data[input_dims:], lr=0.005, iteration=6000, draw_fig=True, x_list=x_list)

# 保存gif
with imageio.get_writer("./images/{}.gif".format("ELman_regression_example_1"), mode="I", fps=5) as Writer:
    fig_num = len(os.listdir("./images/temp"))
    for ind in range(fig_num):
        image = imageio.imread("./images/temp/{}.png".format(ind))
        if ind != fig_num-1:
            os.remove("./images/temp/{}.png".format(ind))
        Writer.append_data(image)
    for i in range(10):Writer.append_data(image)
    os.remove("./images/temp/{}.png".format(fig_num-1))
```

```python
# ----------------------- 测试数据 ---------------------

# 输入维度
input_dims = 10
# 输出维度
output_dims = 1
# 隐藏层维度
hidden_dims = 15
# 序列总长度
seq_len = 400

# 待拟合函数
def arm_func(input_x):
    return np.cos(input_x) * np.sin(input_x * 10) * np.sin(input_x * 5) * np.exp(np.sin(input_x)*2)

# 定义序列
x_list = np.linspace(0, 10, seq_len)
seq_data = arm_func(x_list)
# 目标输出
seq_output = seq_data[input_dims:]

# ---------------- 定义模型 --------------------
elman_nn = MyElman(input_dims, hidden_dims, output_dims, max_back_time_step=3)

# ---------------- 正向推理 --------------------
plt.figure(figsize=(20, 10))
elman_nn.draw_func(x_list, seq_data, seq_output, fig_count=0, if_show=True)

# ---------------- BPTT训练 ---------------------
elman_nn.fit(seq_data, seq_data[input_dims:], lr=0.005, iteration=4000, draw_fig=True, x_list=x_list)

# 保存gif
with imageio.get_writer("./images/{}.gif".format("ELman_regression_example_2"), mode="I", fps=5) as Writer:
    fig_num = len(os.listdir("./images/temp"))
    for ind in range(fig_num):
        image = imageio.imread("./images/temp/{}.png".format(ind))
        if ind != fig_num-1:
            os.remove("./images/temp/{}.png".format(ind))
        Writer.append_data(image)
    for i in range(10):Writer.append_data(image)
    os.remove("./images/temp/{}.png".format(fig_num-1))
```


```python
from datascience import *
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
from utils import *
plt.style.use('seaborn-muted')
import sympy
```

# Elasticity

Suppose you own a business but you find yourself struggling to make a profit lately. How would you resolve this? You might consider raising your prices. But surely if you were to raise your prices too much you might lose some of your customers. To relate to our discussions above regarding surplus, maybe some of your customers are already at their highest willingness to pay, and raising prices would push them away from your business. Depending on the situation, you might lose so many customers that the increased revenue from raising your prices are completely offset by a decline in customers. This is a problem. To answer the question of how much to raise prices without losing too many customers and therefore losing revenue, we would like a way to measure this concept of customers' responsiveness to prices. We call this concept *elasticity*.

How would we measure elasticity? In the above context, we want to see how much quantity demanded of your business's goods or services changes in response to a change in price. It seems like a good place to start would be to define elasticity as the percent change in quantity over percent change in price. Let's use the demand curve, assuming linear non-log demand curves, depicted below as an example.

```python
def plot_demand1():
    p = sympy.Symbol("p")
    demand_equation = 10 - p
    prices = [x for x in range(0,11)]
    demand_Q = [demand_equation.subs(p,x) for x in prices]
    plt.figure(figsize = [9,6])
    plt.plot(prices,demand_Q)
    plt.xlabel("Demand Quantity")
    plt.ylabel("Price")
    plt.title("Demand Curve")

plot_demand1()
plt.show()
```

Now let's say our current price is 5, and we would like to increase the price to 6. We can see from the curve that this causes demand to drop from 5 to 4.

```python
plot_demand1()
plt.plot(np.full(6,5), np.arange(0,6,1), 'r--')
plt.plot(np.full(7,4), np.arange(0,7,1), 'r--')
plt.plot(np.arange(0,6,1), np.full(6,5), 'r--')
plt.plot(np.arange(0,5,1), np.full(5,6), 'r--')
plt.plot(5,5,'ro')
plt.text(5.1,5.1,"(5,5)")
plt.plot(4,6,'ro')
plt.text(4.1,6.1,"(4,6)")
plt.arrow(4.9,5.1,-0.01,0.01,head_width=0.2)
plt.arrow(4.6,5.4,-0.01,0.01,head_width=0.2)
plt.arrow(4.3,5.7,-0.01,0.01,head_width=0.2)
plt.show()
```

Using our definition from above, we can calculate the elasticity.

$$
\begin{align}
\frac{\%\Delta Q}{\%\Delta P} &= \frac{\frac{4-5}{5}\times100\%}{\frac{6-5}{5}\times100\%} \\
&= \frac{-20\%}{20\%} \\
&= 1
\end{align}
$$

*Note that we take the absolute value for elasticity.*

This tells us that in this specific example, the percent change in price is met with an equal percent change in quantity. But what happens if we start at a price of 6 and go down to a price of 5? This movement is just the opposite of what we did above, so is the elasticity also 1? Let's see.

```python
plot_demand1()
plt.plot(np.full(6,5), np.arange(0,6,1), 'r--')
plt.plot(np.full(7,4), np.arange(0,7,1), 'r--')
plt.plot(np.arange(0,6,1), np.full(6,5), 'r--')
plt.plot(np.arange(0,5,1), np.full(5,6), 'r--')
plt.plot(5,5,'ro')
plt.text(5.1,5.1,"(5,5)")
plt.plot(4,6,'ro')
plt.text(4.1,6.1,"(4,6)")
plt.arrow(4.7,5.3,0.01,-0.01,head_width=0.2)
plt.arrow(4.4,5.6,0.01,-0.01,head_width=0.2)
plt.arrow(4.1,5.9,0.01,-0.01,head_width=0.2)
plt.show()
```

$$
\begin{align}
\frac{\%\Delta Q}{\%\Delta P} &= \frac{\frac{5-4}{4}\times100\%}{\frac{5-6}{6}\times100\%} \\
&= \frac{25\%}{-16.67\%} \\
&\approx 1.5
\end{align}
$$

We get a different number! This is of course a property of percentages. In both scenarios we moved price by 1, but we started at different prices, and increasing price by 1 starting from 5 is different percentage-wise from starting at 6 and decreasing by 1.

If we want to find a unique value for elasticity between the price points of 5 and 6, independent of price movement, we can use something called the **mid-point method**. It looks like this:

$$\frac{\frac{Q_2 - Q_1}{(Q_2 + Q_1)\times0.5}\times100\%}{\frac{P_2 - P_1}{(P_2 + P_1)\times0.5}\times100\%}$$

What this does is it essentially finds the elasticity of the point between the two given points.

It is also useful, however, to think of elasticity as a property of a single point. For example, we might want the price point of 5 in the example above to have a set elasticity of 1 irrespective of what other point we are going to or coming from. For this, we can use the **point-slope formula**.

$$\frac{\Delta Q}{\Delta P}\frac{P}{Q}$$

or

$$\frac{dQ}{dP}\Bigr|_{P,Q}\times\frac{P}{Q}$$

where the left fraction is the inverse slope at that particular point.

It turns out that in our example above, we can think of the very first formula for elasticity (percent change in quantity over percent change in price) as the point-slope elasticity of the starting point. To verify this, find the elasticity from a price of 5 to 6, from a price of 5 to 4, and at the price of 5 using the point-slope formula. You should get the same answer all three times. This does not hold in general for all curves, but using the base formula that was first introduced should be fine for our examples and other situations.

## Other Elasticities

We can measure elasticity in different contexts as well. Naturally, if we have an elasticity of demand, it follows that we should have an elasticity of supply. Just like consumers make adjustments to their quantity demanded based on changes in the market price, producers respond to market prices as well. Elasticity of supply can be calculated similarly to elasticity of demand.

Imagine that there are two fast food locations next to each other on campus competing for students' business. Let's call them Railway and Bear Express. Initially, their food is priced such that they get equal amounts of business from the students, and students on average don't prefer one over the other; they just want a good and affordable place to eat. Now, imagine that Bear Express raises the prices of their food, while Railway holds their prices steady. In this scenario, one can imagine that students would begin eating more frequently at Railway as opposed to Bear Express due to it being relatively cheaper out of the two options. Now, if Railway in turn increases their prices, the flow of students would shift back to Bear Express by a certain amount. These two restaurants are said to offer goods that are *substitutes* to each other. That is, a consumer will want one or the other, but usually not both. Furthermore, it would seem that just like the price of a good affects its quantity demanded, for substitutes the price of one good can affect the quantity demanded of the other. This is called *cross-price elasticity of demand* (as opposed to *own-price elasticity* that we just learned earlier). It can be defined as follows:

$$\frac{\%\Delta Q_{Good A}}{\%\Delta P_{Good B}}$$

Notice that here there is a notion of positive and negative elasticity. Substitutes will have positive cross-price elasticity, because we expect that an increase in the price of one good will lead to an increase in the quantity demanded of its substitute. But what about when cross-price elasticity is negative? In other words, are there goods where an increase in the price of one leads to a decrease in the quantity demanded of the other? Of course; these are called *complements*. Examples of complementary goods could be cars and gasoline, hot dogs and hot dog buns, or peanut butter and jelly.

There is one more type of elasticity that we will briefly discuss, and that is *income elasticity of demand*. For most goods, the more disposable income consumers have the more they will demand these goods. Almost all goods follow this pattern, and thus there are countless examples. These goods are called *normal goods*. More interestingly, there are some goods, called *inferior goods*, for which we see quantity demanded decrease as income increases. An example of this might be frozen, microwaveable food. As consumers grow wealthier and their palates grow more discerning, their consumption of frozen food might decrease in place of more cooked meals. To calculate this elasticity we would use the following:

$$\frac{\%\Delta Q}{\%\Delta Income}$$

## Elasticity and Revenue

Let's take a closer look at own-price elasticity of demand. Below we've defined a function using our first definition. Let's see what happens as we increase price. Note that is this percent change in quantity over % change in price. While this is negative quantity usually, we still denote it as a positive number.

```python
def elasticity(P1,P2,Q1,Q2):
    return ((Q2-Q1)/Q1)/((P2-P1)/P1)

p = sympy.Symbol("p")
demand_equation = 10-p
prices = [i for i in range(1,10)]

print("Price: Elasticity")
print()

for x in prices:
    print(str(x)+":", abs(elasticity(x,x+1,demand_equation.subs(p,x),demand_equation.subs(p,x+1))))
```

    Price: Elasticity

    1: 0.111111111111111
    2: 0.250000000000000
    3: 0.428571428571429
    4: 0.666666666666667
    5: 1.00000000000000
    6: 1.50000000000000
    7: 2.33333333333333
    8: 4.00000000000000
    9: 9.00000000000000

Notice how a price increase will increase revenue until a certain point, at which point revenue decreases again. Intuitively this makes sense. A firm can only increase its prices so much until it starts losing too many of its customers for the increase to be worthwhile. Let's try and find when this happens in general. First let's plot revenue as a function of price.

```python
elast = [abs(elasticity(x,x+1,demand_equation.subs(p,x),demand_equation.subs(p,x+1))) for x in range(1,10)]
revenue = [demand_equation.subs(p,x)*x for x in range(1,10)]
prices = [x for x in range(1,10)]
plt.figure(figsize = [9,6])
plt.plot(prices,revenue)
plt.xlabel("Price")
plt.ylabel("Revenue")
plt.title("Revenue vs Price for Given Demand Curve")
plt.show()
```

We notice that revenue reaches a maximun at a price of 5. What elasticity does this price point have?

```python
plt.figure(figsize = [9,6])
plt.plot(prices,elast)
plt.plot(np.arange(1,6,1), np.full(5,1), 'r--')
plt.plot(np.full(2,5), np.arange(0,2,1), 'r--')
plt.plot(5,1,'ro')
plt.text(5,1.5,"(5,1)")
plt.xlabel("Price")
plt.ylabel("Elasticity")
plt.title("Elasticity vs Price for Given Demand Curve")
plt.show()
```

Revenue is maximized when elasticity is 1! To further reinforce this, let's plot revenue vs elasticity.

```python
plt.figure(figsize = [9,6])
plt.plot(elast,revenue)
plt.plot(np.full(17,1), np.arange(9,26,1), 'r--')
plt.plot(1,25,'ro')
plt.text(1.3,25,"(1,25)")
plt.xlabel("Elasticity")
plt.ylabel("Revenue")
plt.title("Revenue vs Elasticity for Given Demand Curve")
plt.show()
```

Before we continue to the final section let's make sure we have a solid intuitive grasp of the significance of our findings. Near low price points, a firm deciding how to raise prices will tend to face inelastic demand. You can imagine that if there were some market where let's say laptops cost only a few dollars, doubling the price of laptops would proabalby not decrease demand for laptops at all, since the price of laptops is still very low. Here, demand is inelastic, or in other words, it is less than 1, because a unit change in price is not met by the same change in quantity. Alternatively, if laptops were to cost as much as a house, and then the price for laptops doubled, the few people who were still in the market for laptops would probably be very discouraged from continuing to buy laptops. It wouldn't be unreasonable to assume that nearly all of the demand would be lost. Here, demand is elastic, or greater than 1, because a unit change in price is met by an even greater change in quantity. The point of the above exercise is to illustrate that there is some middle ground between these two extremes, where a firm with a good pulse on market demand should be pricing its products.

It's important to remember that while we have so far been discussing elasticity from a demand perspective, supply also has the property of elasticity.

## Relative Elasticity

Suppose that you are hiking in the mountains when a venomous snake bites you. Fortunately there is a hospital nearby. Unfortunately, it seems that their antivenom is very expensive. If the antivenom costs \\$1000, would you pay? What if the price doubled to \\$2000? What is your max willingness to pay for this antivenom? In the moment, you would likely be willing to pay anything and everything for the antivenom, as any money you refuse to pay won't be of much use to you if you're dead.

Contrast this with your demand for pizza. Imagine a pizza chain close to where you live doubles their prices from \\$3 a slice to \\$6 a slice. Most people would be disuaded from buying pizza by this price increase, assuming that there are many alternatives to pizza close to where you live. Since pizza is not a very essential good, and there are plenty of other places to eat close to where you live, most wouldn't see the point in getting pizza anymore.

We can easily see that any change in price is unlikely to change the demand for antivenom in our example, and thus we say the demand for antivenom is inelastic. But this is a different kind of elasticity. We are not measuring the marginal change in quantity demanded at a given price point or interval, we are saying that in general, on the demand curve for antivenom the elasticity at any price is more inelastic than some other typical demand curve. Similarly, in our example we can say that the demand for pizza is relatively elastic, as the pizza chain cannot change their prices too much without significantly altering the quantity demanded for their pizza. The keyword in this concept is *relative*. It doesn't really make much sense to say that the demand for antivenom is inelastic without there being at least one other kind of demand that is comparatively elastic.

More specifically, our example of antivenom likely exhibits *perfect inelasticity*, meaning that demand for antivenom is the same at absolutely any price; the hospital can charge any price it wants for antivenom. Contrast this with our nearby pizza chain. If we make some further assumptions and say that the market for food around your house or apartment is perfectly competitive, such that any marginal increase in price of one food chain will cause them to lose all of their customers to other food chains, then the demand for the specific food of that chain is *perfectly elastic*. The chain cannot increase its prices or else it will lose all demand. These two extreme scenarios are depicted below.

```python
plt.figure(figsize = [9,6])
plt.plot(np.full(20,1), np.arange(0,20,1))
plt.xlabel("Quantity Demanded")
plt.ylabel("Price")
plt.title("Perfectly Inelastic Demand Curve")
plt.xticks([], [])
plt.yticks([], [])
plt.show()
```

```python
plt.figure(figsize = [9,6])
plt.plot(np.arange(0,20,1), np.full(20,1))
plt.xlabel("Quantity Demanded")
plt.ylabel("Price")
plt.title("Perfectly Elastic Demand Curve")
plt.xticks([], [])
plt.yticks([], [])
plt.show()
```