# The Jupyter Notebook, Markdown and Python

This is an introduction to Jupyter IPython Notebooks, highlighting some of their key functionality. Please read through and execute cells with **`Shift + Enter`** to get an understanding of what Jupyter Notebooks can do.

Please do not be concerned if this is a little confusing at this stage, try returning to this notebook at the end of the course when you have more experience using the notebooks.

Take a quick look at the Keyboard shortcuts in the help section of Jupyter.

## What is a Jupyter Notebook?

An Ipython Jupyter Notebook is:

- an interactive environment for writing and running code
- a notebooke that weaves code, data, prose, equations, analysis, and visualization
- a tool for prototyping new code and analysis
- a method for creating a reproducible workflow for scientific research

A *live* notebook is composed of an interactive web page (the front end), a running IPython session (the kernel or back end), and a web server responsible for handling communication between the two (the, err..., middle-end)

A *static* notebook, as for example seen on NBViewer, is a static view of the notebook's content.  The default format is HTML, but a notebook can also be output in PDF or other formats.

The centerpiece of an IPython Notebook is the "kernel", the IPython instance responsible for executing all code.  Your IPython kernel maintains its state between executed cells.

# Writing and Running Code

The Jupyter Notebook consists of an ordered list of cells, there are four main cell types:
* **Code**
* **Markdown**
* **Heading**
* **Raw**

Here we briefly introduce how Code Cells work. We will return to the other three cell types later.

### Code Cells

A notebook is a list of cells. Cells contain either explanatory text or executable code and its output. Click a cell to select it.

Below is a **code cell**. Once the toolbar button indicates CONNECTED, click in the cell to select it and execute the contents in the following ways:

* Click the **Play icon** in the left gutter of the cell;
* Type **Cmd/Ctrl+Enter** to run the cell in place;
* Type **Shift+Enter** to run the cell and move focus to the next cell (adding one if none exists); or
* Type **Alt+Enter** to run the cell and insert a new code cell immediately below it.

There are additional options for running some or all cells in the **Runtime** menu.

```python
# This is a code cell made up of Python comments
# We can execute it by clicking on it with the mouse
# then clicking the "Run Cell" button
# or if this does not work try pressing "Ctrl + Enter" or "Shift + Enter"
```

```python
# A comment is a pretty boring piece of code
# This code cell generates "Hello, World" when executed

print("Hello, World")
```

```python
print("Enter your name:")
name = input()
print("Hello {}, welcome to the Data Curation Course!".format(name))
```

```python
# Code cells can also generate graphical output
%matplotlib inline
from matplotlib import pyplot
pyplot.hist([0, 1, 2, 2, 3, 3, 3, 4, 4, 4, 10])
```

### Text cells
This is a **text cell**. You can **double-click** to edit this cell. Text cells
use markdown syntax. To learn more, see our [markdown
guide](/notebooks/markdown_guide.ipynb).

You can also add math to text cells using [LaTeX](http://www.latex-project.org/)
to be rendered by [MathJax](https://www.mathjax.org). Just place the statement
within a pair of **\$** signs. For example `$\sqrt{3x-1}+(1+x)^2$` becomes
$\sqrt{3x-1}+(1+x)^2.$

## Working with python
Colaboratory is built on top of [Jupyter Notebook](https://jupyter.org/). Below are some examples of convenience functions provided.

There are two important actions for interacting with the kernel.  The first is to interrupt it.  This is the same as sending a Control-C from the command line.  The second is to restart it.  This completely terminates the kernel and starts it anew.  None of the kernel state is saved across a restart.

Long running python processes can be interrupted. Run the following cell and select **Runtime -> Interrupt execution** (*hotkey: Cmd/Ctrl-M I*) to stop execution.

```python
import time
print("Sleeping")
time.sleep(30) # sleep for a while; interrupt me!
print("Done Sleeping")
```

### Rich, interactive outputs
Until now all of the generated outputs have been text, but they can be more interesting, like the chart below.

```python
import numpy as np
from matplotlib import pyplot as plt

ys = 200 + np.random.randn(100)
x = [x for x in range(len(ys))]

plt.plot(x, ys, '-')
plt.fill_between(x, ys, 195, where=(ys > 195), facecolor='g', alpha=0.6)

plt.title("Fills and Alpha Example")
plt.show()
```

## Integration with Drive

Colaboratory is integrated with Google Drive. It allows you to share, comment, and collaborate on the same document with multiple people:

* The **SHARE** button (top-right of the toolbar) allows you to share the notebook and control permissions set on it.

* **File->Make a Copy** creates a copy of the notebook in Drive.

* **File->Save** saves the File to Drive. **File->Save and checkpoint** pins the version so it doesn't get deleted from the revision history.

* **File->Revision history** shows the notebook's revision history.

* Multiple people can **collaboratively edit** the same notebook at the same time. Like Google Docs, you can see collaborators both within the document (top right, left of the comments button) and within a cell (right of the cell).

## Markdown cells

Text can be added to IPython Notebooks using Markdown cells.  Markdown is a popular markup language that is a superset of HTML.  Its specification can be found here:

<http://daringfireball.net/projects/markdown/>

## Markdown basics

### Text formatting

You can make text *italic* or **bold** or `monospace`

### Itemized Lists

* One
    - Sublist
        - This
  - Sublist
        - That
        - The other thing
* Two
  - Sublist
* Three
  - Sublist

### Enumerated Lists

1. Here we go
    1. Sublist
    2. Sublist
2. There we go
3. Now this

### Horizontal Rules

---

---

---

### Blockquotes

> To me programming is more than an important practical art. It is also a gigantic undertaking in the foundations of knowledge. -- Rear Admiral Grace Hopper

### Links

[IPython's website](http://ipython.org)

### Code

This is a code snippet:

```python
def f(x):
    """a docstring"""
    return x**2
```

This is an example of a **Python** function

You can also use triple-backticks to denote code blocks.
This also allows you to choose the appropriate syntax highlighter.

```C
if (i=0; i<n; i++) {
  printf("hello %d\n", i);
  x += 4;
}
```

### Tables

Time (s) | Audience Interest
---------|------------------
 0       | High
 1       | Medium
 5       | Facebook

### Images

### YouTube

```python
from IPython.display import YouTubeVideo
YouTubeVideo('jFNgjQoEigY')
```

### Other HTML

<strong> Be Bold! </strong>

## Mathematical Equations

Courtesy of MathJax, you can beautifully render mathematical expressions, both inline:
$e^{i\pi} + 1 = 0$, and displayed:

$$e^x=\sum_{i=0}^\infty \frac{1}{i!}x^i$$

### Equation Environments

You can also use a number of equation environments, such as `align`:

\begin{align}
  x &= 4 \\
y+z &= x
\end{align}

### Other Useful MathJax Notes

## Magics

IPython kernels execute a superset of the Python language.  The extension functions, commonly referred to as *magics*, come in two variants.

### Line Magics

* A *line magic* looks like a command line call.  The most important of these is `%matplotlib inline`, which embeds all matplotlib plot output as images in the notebook itself.

```python
%matplotlib inline
```

```python
%whos
```

### Cell Magics

* A *cell magic* takes its entire cell as an argument.  Although there are a number of useful cell magics, you may find `%%timeit` to be useful for exploring code performance.

```python
%%timeit

import numpy as np
np.sum(np.random.rand(1000))
```

### Execute Code as Python 2

Just need to type `%%python2`

```Python
%%python2

i = 10**60
print type(i)
```

### Interacting with the Command Line

IPython supports one final trick, the ability to interact directly with your  shell by using the `!` operator.

```python
!ls
```

```python
x = !ls
```

```python
print(x)
```

## Exercises

- Create a new notebook
- Write both a numerated and a bullet list in Markdown
- Create a new code cell and what happens if you try the following code?
```python
import antigravity
```
- As we know that the following code may get the current time:
```python
from datetime import datetime
print(datetime.now())
```
Create another new code cell and print "The current time is: 2018-09-06 11:10:11.708220"
- Insert a photo of the new campus in the notebook.


```python
%matplotlib inline
```

Reinforcement Learning (DQN) Tutorial
=====================================
**Author**: `Adam Paszke <https://github.com/apaszke>`_

This tutorial shows how to use PyTorch to train a Deep Q Learning (DQN) agent
on the CartPole-v0 task from the `OpenAI Gym <https://gym.openai.com/>`__.

**Task**

The agent has to decide between two actions - moving the cart left or
right - so that the pole attached to it stays upright. You can find an
official leaderboard with various algorithms and visualizations at the
`Gym website <https://gym.openai.com/envs/CartPole-v0>`__.

.. figure:: /_static/img/cartpole.gif
   :alt: cartpole

   cartpole

As the agent observes the current state of the environment and chooses
an action, the environment *transitions* to a new state, and also
returns a reward that indicates the consequences of the action. In this
task, rewards are +1 for every incremental timestep and the environment
terminates if the pole falls over too far or the cart moves more then 2.4
units away from center. This means better performing scenarios will run
for longer duration, accumulating larger return.

The CartPole task is designed so that the inputs to the agent are 4 real
values representing the environment state (position, velocity, etc.).
However, neural networks can solve the task purely by looking at the
scene, so we'll use a patch of the screen centered on the cart as an
input. Because of this, our results aren't directly comparable to the
ones from the official leaderboard - our task is much harder.
Unfortunately this does slow down the training, because we have to
render all the frames.

Strictly speaking, we will present the state as the difference between
the current screen patch and the previous one. This will allow the agent
to take the velocity of the pole into account from one image.

**Packages**

First, let's import needed packages. Firstly, we need
`gym <https://gym.openai.com/docs>`__ for the environment
(Install using `pip install gym`).
We'll also use the following from PyTorch:

-  neural networks (``torch.nn``)
-  optimization (``torch.optim``)
-  automatic differentiation (``torch.autograd``)
-  utilities for vision tasks (``torchvision`` - `a separate
   package <https://github.com/pytorch/vision>`__).

```python
import gym
import math
import random
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from collections import namedtuple
from itertools import count
from PIL import Image

import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torchvision.transforms as T

env = gym.make('CartPole-v0').unwrapped

# set up matplotlib
is_ipython = 'inline' in matplotlib.get_backend()
if is_ipython:
    from IPython import display

plt.ion()

# if gpu is to be used
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
```

Replay Memory
-------------

We'll be using experience replay memory for training our DQN. It stores
the transitions that the agent observes, allowing us to reuse this data
later. By sampling from it randomly, the transitions that build up a
batch are decorrelated. It has been shown that this greatly stabilizes
and improves the DQN training procedure.

For this, we're going to need two classses:

-  ``Transition`` - a named tuple representing a single transition in
   our environment. It essentially maps (state, action) pairs
   to their (next_state, reward) result, with the state being the
   screen difference image as described later on.
-  ``ReplayMemory`` - a cyclic buffer of bounded size that holds the
   transitions observed recently. It also implements a ``.sample()``
   method for selecting a random batch of transitions for training.

```python
Transition = namedtuple('Transition',
                        ('state', 'action', 'next_state', 'reward'))

class ReplayMemory(object):

    def __init__(self, capacity):
        self.capacity = capacity
        self.memory = []
        self.position = 0

    def push(self, *args):
        """Saves a transition."""
        if len(self.memory) < self.capacity:
            self.memory.append(None)
        self.memory[self.position] = Transition(*args)
        self.position = (self.position + 1) % self.capacity

    def sample(self, batch_size):
        return random.sample(self.memory, batch_size)

    def __len__(self):
        return len(self.memory)
```

Now, let's define our model. But first, let quickly recap what a DQN is.

DQN algorithm
-------------

Our environment is deterministic, so all equations presented here are
also formulated deterministically for the sake of simplicity. In the
reinforcement learning literature, they would also contain expectations
over stochastic transitions in the environment.

Our aim will be to train a policy that tries to maximize the discounted,
cumulative reward
$R_{t_0} = \sum_{t=t_0}^{\infty} \gamma^{t - t_0} r_t$, where
$R_{t_0}$ is also known as the *return*. The discount,
$\gamma$, should be a constant between $0$ and $1$
that ensures the sum converges. It makes rewards from the uncertain far
future less important for our agent than the ones in the near future
that it can be fairly confident about.

The main idea behind Q-learning is that if we had a function
$Q^*: State \times Action \rightarrow \mathbb{R}$, that could tell
us what our return would be, if we were to take an action in a given
state, then we could easily construct a policy that maximizes our
rewards:

\begin{align}\pi^*(s) = \arg\!\max_a \ Q^*(s, a)\end{align}

However, we don't know everything about the world, so we don't have
access to $Q^*$. But, since neural networks are universal function
approximators, we can simply create one and train it to resemble
$Q^*$.

For our training update rule, we'll use a fact that every $Q$
function for some policy obeys the Bellman equation:

\begin{align}Q^{\pi}(s, a) = r + \gamma Q^{\pi}(s', \pi(s'))\end{align}

The difference between the two sides of the equality is known as the
temporal difference error, $\delta$:

\begin{align}\delta = Q(s, a) - (r + \gamma \max_a Q(s', a))\end{align}

To minimise this error, we will use the `Huber
loss <https://en.wikipedia.org/wiki/Huber_loss>`__. The Huber loss acts
like the mean squared error when the error is small, but like the mean
absolute error when the error is large - this makes it more robust to
outliers when the estimates of $Q$ are very noisy. We calculate
this over a batch of transitions, $B$, sampled from the replay
memory:

\begin{align}\mathcal{L} = \frac{1}{|B|}\sum_{(s, a, s', r) \ \in \ B} \mathcal{L}(\delta)\end{align}

\begin{align}\text{where} \quad \mathcal{L}(\delta) = \begin{cases}
     \frac{1}{2}{\delta^2}  & \text{for } |\delta| \le 1, \\
     |\delta| - \frac{1}{2} & \text{otherwise.}
   \end{cases}\end{align}

Q-network
^^^^^^^^^

Our model will be a convolutional neural network that takes in the
difference between the current and previous screen patches. It has two
outputs, representing $Q(s, \mathrm{left})$ and
$Q(s, \mathrm{right})$ (where $s$ is the input to the
network). In effect, the network is trying to predict the *expected return* of
taking each action given the current input.

```python
class DQN(nn.Module):

    def __init__(self, h, w, outputs):
        super(DQN, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, kernel_size=5, stride=2)
        self.bn1 = nn.BatchNorm2d(16)
        self.conv2 = nn.Conv2d(16, 32, kernel_size=5, stride=2)
        self.bn2 = nn.BatchNorm2d(32)
        self.conv3 = nn.Conv2d(32, 32, kernel_size=5, stride=2)
        self.bn3 = nn.BatchNorm2d(32)

        # Number of Linear input connections depends on output of conv2d layers
        # and therefore the input image size, so compute it.
        def conv2d_size_out(size, kernel_size = 5, stride = 2):
            return (size - (kernel_size - 1) - 1) // stride  + 1
        convw = conv2d_size_out(conv2d_size_out(conv2d_size_out(w)))
        convh = conv2d_size_out(conv2d_size_out(conv2d_size_out(h)))
        linear_input_size = convw * convh * 32
        self.head = nn.Linear(linear_input_size, outputs)

    # Called with either one element to determine next action, or a batch
    # during optimization. Returns tensor([[left0exp,right0exp]...]).
    def forward(self, x):
        x = F.relu(self.bn1(self.conv1(x)))
        x = F.relu(self.bn2(self.conv2(x)))
        x = F.relu(self.bn3(self.conv3(x)))
        return self.head(x.view(x.size(0), -1))
```

Input extraction
^^^^^^^^^^^^^^^^

The code below are utilities for extracting and processing rendered
images from the environment. It uses the ``torchvision`` package, which
makes it easy to compose image transforms. Once you run the cell it will
display an example patch that it extracted.

```python
resize = T.Compose([T.ToPILImage(),
                    T.Resize(40, interpolation=Image.CUBIC),
                    T.ToTensor()])

def get_cart_location(screen_width):
    world_width = env.x_threshold * 2
    scale = screen_width / world_width
    return int(env.state[0] * scale + screen_width / 2.0)  # MIDDLE OF CART

def get_screen():
    # Returned screen requested by gym is 400x600x3, but is sometimes larger
    # such as 800x1200x3. Transpose it into torch order (CHW).
    screen = env.render(mode='rgb_array').transpose((2, 0, 1))
    # Cart is in the lower half, so strip off the top and bottom of the screen
    _, screen_height, screen_width = screen.shape
    screen = screen[:, int(screen_height*0.4):int(screen_height * 0.8)]
    view_width = int(screen_width * 0.6)
    cart_location = get_cart_location(screen_width)
    if cart_location < view_width // 2:
        slice_range = slice(view_width)
    elif cart_location > (screen_width - view_width // 2):
        slice_range = slice(-view_width, None)
    else:
        slice_range = slice(cart_location - view_width // 2,
                            cart_location + view_width // 2)
    # Strip off the edges, so that we have a square image centered on a cart
    screen = screen[:, :, slice_range]
    # Convert to float, rescale, convert to torch tensor
    # (this doesn't require a copy)
    screen = np.ascontiguousarray(screen, dtype=np.float32) / 255
    screen = torch.from_numpy(screen)
    # Resize, and add a batch dimension (BCHW)
    return resize(screen).unsqueeze(0).to(device)

env.reset()
plt.figure()
plt.imshow(get_screen().cpu().squeeze(0).permute(1, 2, 0).numpy(),
           interpolation='none')
plt.title('Example extracted screen')
plt.show()
```

Training
--------

Hyperparameters and utilities
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This cell instantiates our model and its optimizer, and defines some
utilities:

-  ``select_action`` - will select an action accordingly to an epsilon
   greedy policy. Simply put, we'll sometimes use our model for choosing
   the action, and sometimes we'll just sample one uniformly. The
   probability of choosing a random action will start at ``EPS_START``
   and will decay exponentially towards ``EPS_END``. ``EPS_DECAY``
   controls the rate of the decay.
-  ``plot_durations`` - a helper for plotting the durations of episodes,
   along with an average over the last 100 episodes (the measure used in
   the official evaluations). The plot will be underneath the cell
   containing the main training loop, and will update after every
   episode.

```python
BATCH_SIZE = 128
GAMMA = 0.999
EPS_START = 0.9
EPS_END = 0.05
EPS_DECAY = 200
TARGET_UPDATE = 10

# Get screen size so that we can initialize layers correctly based on shape
# returned from AI gym. Typical dimensions at this point are close to 3x40x90
# which is the result of a clamped and down-scaled render buffer in get_screen()
init_screen = get_screen()
_, _, screen_height, screen_width = init_screen.shape

# Get number of actions from gym action space
n_actions = env.action_space.n

policy_net = DQN(screen_height, screen_width, n_actions).to(device)
target_net = DQN(screen_height, screen_width, n_actions).to(device)
target_net.load_state_dict(policy_net.state_dict())
target_net.eval()

optimizer = optim.RMSprop(policy_net.parameters())
memory = ReplayMemory(10000)

steps_done = 0

def select_action(state):
    global steps_done
    sample = random.random()
    eps_threshold = EPS_END + (EPS_START - EPS_END) * \
        math.exp(-1. * steps_done / EPS_DECAY)
    steps_done += 1
    if sample > eps_threshold:
        with torch.no_grad():
            # t.max(1) will return largest column value of each row.
            # second column on max result is index of where max element was
            # found, so we pick action with the larger expected reward.
            return policy_net(state).max(1)[1].view(1, 1)
    else:
        return torch.tensor([[random.randrange(n_actions)]], device=device, dtype=torch.long)

episode_durations = []

def plot_durations():
    plt.figure(2)
    plt.clf()
    durations_t = torch.tensor(episode_durations, dtype=torch.float)
    plt.title('Training...')
    plt.xlabel('Episode')
    plt.ylabel('Duration')
    plt.plot(durations_t.numpy())
    # Take 100 episode averages and plot them too
    if len(durations_t) >= 100:
        means = durations_t.unfold(0, 100, 1).mean(1).view(-1)
        means = torch.cat((torch.zeros(99), means))
        plt.plot(means.numpy())

    plt.pause(0.001)  # pause a bit so that plots are updated
    if is_ipython:
        display.clear_output(wait=True)
        display.display(plt.gcf())
```

Training loop
^^^^^^^^^^^^^

Finally, the code for training our model.

Here, you can find an ``optimize_model`` function that performs a
single step of the optimization. It first samples a batch, concatenates
all the tensors into a single one, computes $Q(s_t, a_t)$ and
$V(s_{t+1}) = \max_a Q(s_{t+1}, a)$, and combines them into our
loss. By defition we set $V(s) = 0$ if $s$ is a terminal
state. We also use a target network to compute $V(s_{t+1})$ for
added stability. The target network has its weights kept frozen most of
the time, but is updated with the policy network's weights every so often.
This is usually a set number of steps but we shall use episodes for
simplicity.

```python
def optimize_model():
    if len(memory) < BATCH_SIZE:
        return
    transitions = memory.sample(BATCH_SIZE)
    # Transpose the batch (see https://stackoverflow.com/a/19343/3343043 for
    # detailed explanation). This converts batch-array of Transitions
    # to Transition of batch-arrays.
    batch = Transition(*zip(*transitions))

    # Compute a mask of non-final states and concatenate the batch elements
    # (a final state would've been the one after which simulation ended)
    non_final_mask = torch.tensor(tuple(map(lambda s: s is not None,
                                          batch.next_state)), device=device, dtype=torch.bool)
    non_final_next_states = torch.cat([s for s in batch.next_state
                                                if s is not None])
    state_batch = torch.cat(batch.state)
    action_batch = torch.cat(batch.action)
    reward_batch = torch.cat(batch.reward)

    # Compute Q(s_t, a) - the model computes Q(s_t), then we select the
    # columns of actions taken. These are the actions which would've been taken
    # for each batch state according to policy_net
    state_action_values = policy_net(state_batch).gather(1, action_batch)

    # Compute V(s_{t+1}) for all next states.
    # Expected values of actions for non_final_next_states are computed based
    # on the "older" target_net; selecting their best reward with max(1)[0].
    # This is merged based on the mask, such that we'll have either the expected
    # state value or 0 in case the state was final.
    next_state_values = torch.zeros(BATCH_SIZE, device=device)
    next_state_values[non_final_mask] = target_net(non_final_next_states).max(1)[0].detach()
    # Compute the expected Q values
    expected_state_action_values = (next_state_values * GAMMA) + reward_batch

    # Compute Huber loss
    loss = F.smooth_l1_loss(state_action_values, expected_state_action_values.unsqueeze(1))

    # Optimize the model
    optimizer.zero_grad()
    loss.backward()
    for param in policy_net.parameters():
        param.grad.data.clamp_(-1, 1)
    optimizer.step()
```

Below, you can find the main training loop. At the beginning we reset
the environment and initialize the ``state`` Tensor. Then, we sample
an action, execute it, observe the next screen and the reward (always
1), and optimize our model once. When the episode ends (our model
fails), we restart the loop.

Below, `num_episodes` is set small. You should download
the notebook and run lot more epsiodes, such as 300+ for meaningful
duration improvements.

```python
num_episodes = 50
for i_episode in range(num_episodes):
    # Initialize the environment and state
    env.reset()
    last_screen = get_screen()
    current_screen = get_screen()
    state = current_screen - last_screen
    for t in count():
        # Select and perform an action
        action = select_action(state)
        _, reward, done, _ = env.step(action.item())
        reward = torch.tensor([reward], device=device)

        # Observe new state
        last_screen = current_screen
        current_screen = get_screen()
        if not done:
            next_state = current_screen - last_screen
        else:
            next_state = None

        # Store the transition in memory
        memory.push(state, action, next_state, reward)

        # Move to the next state
        state = next_state

        # Perform one step of the optimization (on the target network)
        optimize_model()
        if done:
            episode_durations.append(t + 1)
            plot_durations()
            break
    # Update the target network, copying all weights and biases in DQN
    if i_episode % TARGET_UPDATE == 0:
        target_net.load_state_dict(policy_net.state_dict())

print('Complete')
env.render()
env.close()
plt.ioff()
plt.show()
```

Here is the diagram that illustrates the overall resulting data flow.

.. figure:: /_static/img/reinforcement_learning_diagram.jpg

Actions are chosen either randomly or based on a policy, getting the next
step sample from the gym environment. We record the results in the
replay memory and also run optimization step on every iteration.
Optimization picks a random batch from the replay memory to do training of the
new policy. "Older" target_net is also used in optimization to compute the
expected Q values; it is updated occasionally to keep it current.


```python
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
import sympy as sp
from plots import *

sp.init_printing()
freqs = [f for f in np.random.standard_cauchy(11) if abs(f) < 10]
omega = [2+ f for f in freqs] + [1 - f for f in freqs] + [1]

from BondGraphTools import version
import BondGraphTools as bgt
assert version == "0.3.7"
scale = 2

from matplotlib.font_manager import FontProperties

def plot_graph(t, x):
    fontP = FontProperties()
    fontP.set_size('small')
    fig = plt.figure(figsize=(scale*4,scale*4))
    plt.plot(t,x)
    ax = fig.gca()
    ax.set_xlabel('t')
    ax.set_title(f"System response to {impulse}")
    ax.legend(
        [f"$x_{i}$" for i in range(len(x))],
        bbox_to_anchor=(1.,1.),
        loc=1,
        borderaxespad=0.,
        prop=fontP
    )
    return fig

def print_tree(bond_graph, pre=""):
    print(f"{pre}{bond_graph}")
    try:
        for component in reversed(bond_graph.components):
            if pre == "": print_tree(component, pre +"|-" )
            else: print_tree(component, pre +"-" )
    except AttributeError:
        pass
```

```python

```

TODO:
2. Add critical values for Kuramoto model.
3. Add critical values for Nonlinear oscillator model.

- Introduction to complex systems
- Basic research question

Lit Points:
- Emergence is low dimension dynamics in high dimensional space. (Should be a geometric description; but we're not there yet.)
- Collective motion requires tradeoffs between coulping strength and population heterogeneity. (Framed in terms of some kind of averaging)

# On Emergence in Complex Physical Systems

https://github.com/peter-cudmore

    Dr. Peter Cudmore.
    Systems Biology Labratory,
    The School of Chemical and Biomedical Engineering,
    The University of Melbourne.

Many problems in biology, physics and engineering involve predicting and controlling complex systems, loosely defined as interconnected system-of-systems. Such systems can exhibit a variety of interesting non-equilibrium features such as emergence and phase transitions, which result from mutual interactions between nonlinear subsystems.

Modelling these systems is a task in-and-of itself, as systems can span many physical domains and evolve on multiple time scales. Nonetheless, one wishes to analyse the geometry of these models and relate both qualitative and quantitative insights back to the physical system.

Beginning with the modelling and analysis of a coupled optomechanical systems, this talk presents some recent results concerning the existence and stability of emergent oscillations. This forms the basis for a discussion of new directions in symbolic computational techniques for complex physical systems as a means to discuss emergence more generally.

## The problem with big systems is that they're _big_...

## Example: Many-body Quantum Optomechanics

(Image courtesy of Knap, M. https://users.ph.tum.de/ga32pex/ )

(Image courtesy of Marquardt, F.
https://photons-and-matter.org/research/ )

## Example: Human Metabolism

(Image courtesy of Human Metabolism map https://www.vmh.life )

# Example: Ecosystems

## Complex Physical Systems

A dynamical system is said to be a _complex physical system_ when:
* It is made up of many _interacting_ parts, or subsystems (High-dimensional).
* The subsystems are not all of the same (Heterogenenous).
* The subsystems are complicated (Nonlinear and/or Noisy).
* There are well defined boundaries between the subsystems (Network Topology).
* **Coupling takes place via resource exchange (Conservation Laws).**

## Complex Systems can exhibit _emergence_.

- _Emergence_ is a phenomenom where the system displays novel new behaviour that could not be produced by individuals alone.
- _Synchronisation_ is the most studied example of emergence, and can occur in systems of coupled oscillator.

 How can one predict and control emergent phenomenon?

How can nonlinear dynamics be "scaled up"?

## Outline of this talk

In this talk we will:

Briefly discuss synchronisation as it's the best example of emergence.

Discuss how we might generalise this for energetic systems.

## Part 1: Synchronised Oscillators

## The Kuramoto Model

_Self-entrainment of a population of coupled non-linear oscillators_ Kuramoto, Y. (1975).

The phase $\theta_j$ of each oscillator with a natural frequency $\omega_j$ is given by

\begin{equation}
\dot{\theta}_j =  \omega_j + \frac{K}{n}\sum_{k=1}^n\sin(\theta_k - \theta_j),\qquad j=1,\ldots n
\end{equation}
- When $0\le K <K_c $ each oscillator rotates at their own frequency.

- A Hopf bifurcation occurs at $K=K_c$ creating collective motion.

- When $K>K_c$ more oscillator are recruited to collective.

The value of $K_c$ depends upon the distribution of $\{\omega_j\}$. For symmetric distribtuions we have
$$K_c = \frac{2}{\pi g(0)}$$

```python
# Omega = Cauchy(2,1) so that K_c = 2
p = KuramotoModel(omega=omega, scale=scale)
plt.show()
```

    <IPython.core.display.Javascript object>

Points:
- Wiener -> Winfree -> Kuramoto
- Comes from studying BZ reaction
- Motion on a strongly attracive limit cycle (invariant manifold) such that coupling
- All-to-all coupling on a complete graph.
- sinusoidal in phase -> linear in complex co-ordinates.
- Kuramoto showed that at $K_c=2$ a Hopf bifurcation creates a synchronised state, that becomes progressive more stable as $K_c$ increases.

## The Kuramoto Model (Cont.)

Kuramoto introduced an 'order parameter' $r$ to measure phase coherence
\begin{equation}
z = r\mathrm{e}^{\mathrm{i}\Theta} = \frac{1}{n}\sum_{k=1}^n \exp{\mathrm{i}\theta_k} \implies r = \frac{1}{n}\sum_{k=1}^n \exp\mathrm{i}(\theta_k-\Theta)
\end{equation}
It follows that
$$
\Im\left[\frac{1}{n}\sum_{k=1}^n\exp i(\theta_k - \theta_j)\right] =
\Im\left[r\exp i(\Theta - \theta_j)\right]
$$

Hence
$$
\dot{\theta}_j =  \omega_j + \frac{K}{n}\sum_{k=1}^n\sin(\theta_k - \theta_j)$$

becomes
$$
\dot{\theta}_j =  \omega_j + rK\sin(\Theta - \theta_j).
$$

```python
p = KuramotoOrderModel(omega,scale=scale)
plt.show()
```

    <IPython.core.display.Javascript object>

Points:
- Mean phase is a kind of coordinate for the synchronous manifold.
- Weak interactions with entire populations <=> strong coupling to collective statistics
- Feedback look means that if coupling increases coherence, then $r$ increases asymptotically to $r_\infty = \sqrt{1-K_c/K}$$.

## The Status of the Kuramoto Model

$$
\dot{\theta}_j =  \omega_j + rK\sin(\Theta - \theta_j),\qquad j = 1,\ldots n.\qquad r = \frac{1}{n}\sum_{k=1}^n \exp i (\theta_k - \Theta).
$$

- Identical oscillators evolve on a 3 dimensional manifold (Watanabe and Strogatz, Physica D 1994. Ott and Antonsen, Chaos 2008).
- Heterogenous oscillator dynamics represented in terms of collective co-ordinates in the thermodynamic limit (Pikovsky and Rosenblum, Physica D, 2011) and for finite-n (Gottwald, Chaos 2015).
- Active research into applications in biology (particuarly neuroscience), physics and chemsitry.
- Extensions to noisy, graph coupled and with various different coupling mechanisms.
- Very few global results for heterogenous oscillators (Dietert, J. Math. Pures Appl. 2016).
- _No results as yet for geometrical interpretation of transtion to synchrony._

## Some Takeaways

$$
\dot{\theta}_j =  \omega_j + rK\sin(\Theta - \theta_j),\qquad j = 1,\ldots n.\qquad r = \frac{1}{n}\sum_{k=1}^n \exp i (\theta_k - \Theta).
$$

1. When thinking about emergence, we want to think about mutual coupling between population statistics and individuals.

2. This means that, for a given system, we need to understand both the individual dynamics _and_ population level dynamics.

# Part 2: Coupled Optomechanical Systems

## Physics of Many-Body Quantum Optomechanics

The Quantum Hamiltonain for the system is
$$
H = H_\text{cavity}+H_\text{drive}+H_\text{beams}
+H_\text{coupling} + \mathcal{H}_\text{diss}
$$
where
$$
\begin{align}
H_\text{cavity} =&\ \frac{\omega_c}{2} a^\dagger a\\
H_\text{drive} =&\ u^\dagger a + a^\dagger u\\
H_\text{beams} =&\ \sum_j \frac{\omega_j}{2}b_j^\dagger b\\
H_\text{coupling} =&\ \sum_j \frac{G}{4}
(b_j+b^\dagger_j)a^\dagger a\\
\end{align}
$$
where $a^\dagger,a ,b_j^\dagger,b_j,u^\dagger,u$ are the creation and annihilaton operators of the
optical mode, vibrational modes and optical forcing respectively.

Also:
- $\omega_c$ is the resonant frequency of the optical cavity
- $\omega_j$ is the mechanical resonance of the $j$th beam
- $G$ is the strengh of opto-mechanical coupling
- $\mathcal{H}_\text{diss}$ is the 'dissipative structure'...

## From Quantum Physics to Semiclassical Dynamics

1. Derive/produce the (generalised) Langevin equations.
2. Use the semiclassical approximation to replace operators with averages, so that $a \rightarrow \alpha$ and $a^\dagger \rightarrow \alpha^*$
3. Assume the optical forcing is $\left = \nu\exp[i\omega_ft]$ and define $\delta =\omega_c - \omega_f$
4. Perform some rescaling and non-dimensionlisation to get
$$
\begin{align}
\dot{\alpha}&= - (1+i\delta)\alpha - i\alpha \frac{1}{n}\sum_{j=1}^n\Re \left[\beta_j\right] -i\nu \\
\dot{\beta_j} +  i\bar{\omega}\beta_j &= -\gamma \left[(1+i\omega'_j)\beta_j + i\frac{G}{2}|\alpha|^2\right], \quad j=1,\ldots, n
\end{align}
$$
where $\gamma \ll1$ and $\omega_j = \bar{\omega} + \gamma\omega_j'$
5. Apply the 'method of multiple scales' with $\tau = \gamma t$ to get $\beta(t,
\tau) = z(\tau)\exp[-i\bar{\omega}t]$.
6. Notice that $\Re[\beta] = \cos(\omega t +\phi)$ makes $\alpha$ look like a F.M oscillator, so has a basis in terms of Bessel functions.
7. More algebra...

...how much of this can be automated?

## On the other side

The fast time system (for the cavity $\alpha$):

$$
\frac{d\alpha}{dt} =
-\alpha  - i(\delta + r(\tau)\cos[\bar{\omega}t + \Theta(\tau)])\alpha - i\nu
$$

The slow time system (coupling induced frequency shift $z_j$):

$$
\frac{d z_j}{d\tau}
= -(1- i\omega'_j)z_j + zF(|z|), \qquad z = re^{i\Theta} = \frac{1}{n}\sum_{j=1}^n z_j
$$

With $F:[0,\infty) \rightarrow \mathbb{C}$ a complicated but known and computable function.

## Collective motion on the slow time scale

**Theorem (PC & Holmes. 2015)**
Define the _dispersion function_
$$
f(\mu) = \int_{-\infty}^\infty \frac{g(\omega)d\omega}{\mu - i\omega}.
$$

(1) For sufficiently large $n$, the system
$$\dot{z}_j
= -(1- i\omega_j)z_j + zF(|z|), \qquad z = \frac{1}{n}\sum_{j=1}^n z_j$$
has coherent motion on (possibly many multi-stable) invariant manifold(s) characterised by
$$
z = re^{i\Theta},\quad \dot{r} = 0, \qquad \dot{\Theta} = \Omega
$$
and
$$r\left[F(r) - \frac{1}{f(1+i\Omega)}\right] = 0.$$

(2) For symmetric $g$ solutions are strictly unsable iff

$$
\left\langle\frac{dF}{dr},\frac{d}{d\mu}\frac{1}{f(\mu)}\bigg|_{1+i\Omega}\right\rangle >0,
$$
where $\langle x,y\rangle = \Re[xy^*]$ is the geometric inner product for $x,y\in \mathbb{C}$.

```python
# F(r) = K/(1+r^2) is stable iff K > 2
m = DampedCoupledOsc(omega=omega, extent=4, scale=scale)
plt.show()
```

    <IPython.core.display.Javascript object>

Notes:
- Only collective motion: S^1.
- Multistability which can be created and destroyed via Hopf's.
- Results for general coupling function $F$.
- Proof involves treating $1/f(\mu)$ as a conformal map from the RHP.

## Synch corresponds to frequency modulation on the fast time scale

On the fast time scale:

$$\frac{d\alpha}{dt} =
-\alpha  - i(\delta + r(\tau)\cos[\bar{\omega}t + \Omega\tau])\alpha - i\nu
$$

If we can measure $(r,\Omega)$, we can determine the internal state.

Here $\alpha$ falls well into the realm of existing signal processing theory (in particular frequency modulation). Hence, we can associate spectral properties to internal state.

(The maths is hideous though)

## Some Observations

# Object Oriented Modelling for Complex Physical Systems

### Inheritance, Composition and Encapsulation

## Ad-hoc modelling.

1. Use descriptions of physical processes plus network topology to generate some odes.
2. Do a whole bunch of algebra.
3. Work out the appropriate coordinates to use based on geometric features (here, a non-standard slow fast system)
4. More algebra to reduce model.
5. Investigate the dynamics of the reduced models.
6. Relate results in reduced model to observables in the original system

In the case of emergent phenomenon, the 'reduced' subspace involves the whole (or at least a large part of) system. E.g. mean fields.

## Ad-hoc approaches won't scale.

As an example:
- individual processes are far more heterogenous
- network topolgy is complicated
- many parameters are unknown
- almost guaranteed to be a differential-algebraic system
- **too big for one person, or even one lab**

We must have:
- Ways to respresent and manipulate such systems,
- Ways to manage congnitive complexity,
- Ways to automate model capture and reduction,
- Ways to effective share work between researchers

## Energy provides an interface.

## The Structure of Complex Physical Systems

An approach based on 'bond graph' modelling, and port-Hamiltonian systems.

- Energy is stored in 'state variables' $q,p$
- Power is distributed via 'power variables' $e,f$
- Formally describes the hyrdo-mechanical-electrical analogies.

For example;
- Dissipation relates $e,f$ variables. (eg. Ohm's law, friction, etc)
- Potential storage $q$ to $e$ (eg, capacitors, gravity)

    Domain
    $q$
     $f = \dot{q}$
     $p$
    $e = \dot{p}$

     Translational Mechanics
     position
     velocity
     momentum
     force

Rotational Mechanics
     angle
     angular velocity
     angular momentum
     torque

Electronics
     charge
     current
     flux linkage
     voltage

Hydraulics
     volume
     flow
     pressure momentum
     pressure

     Thermodynamics
     entropy
     entropy flow
     temperature momentum
     temperature

    Chemistry
     moles
     molar flow

     chemical potential

## An Object Oriented Representation of Energetic Systems

Object Oriented Programming (OOP) is a software development paradigm that seeks to manage large, complicated projects by breaking problems into _data_ plus _methods_ that act on the data.

Three big ideas in OOP are:
1. _Inheritance_ or is-a relationships.
2. _Composition_ or has-a relationships.
3. _Encapsulation_ or infomation hiding.

This allows for _hierarchical_ and _modular_ design which reduces model complexity.

'Energetic systems' draws from:
- Network based analysis from engineering; in particular circuit electrical analysis and the more general (and less well known) bond graph methodology,
- Classical mechanics, and in particular recent advances in port based Hamiltonian mechanics,
- Modern nonlinear dynamics,
- The effective was of managing complexity within software engineering.

## Inheritance

For networked dynamic systems, _inheritance_ means we have:
- conditions on the dynamical sub-systems.
- a description of the interface between nodes.

### Definition (Energetic System)

An energetic system is a tuple $(M, \mathcal{D}, U,\Phi)$
where the
* *state space* $M$ is a manifold of $\dim(M) = m\ge 0$
* *port space* $\mathcal{D} \subset \mathcal{F} \times \mathcal{E}$ where, $\mathcal{E} = \mathcal{F}^*$ and $ \dim{\mathcal{D}} = \mathcal{F}|_\mathcal{D} =n$.
* *control space* $U \subset C^r:\mathbb{R}_+ \rightarrow \mathbb{R^k}$ with $k\ge 0$
* *constitutive relation* is a smooth map $\Phi: TM \times \mathcal{D} \times U\times\mathbb{R}_+ \rightarrow
  \mathbb{R}^{m+n}$
    such that
  $$\Phi\left(\frac{dx}{dt},x,f,e,u,t\right)=0.$$

$\Phi$ relates the _internal state_ $M$ and the _external environment_ (via $\mathcal{D}$).

One can show that for conservative systems, one can define a storage function $H(x)$, choose $$\Phi(\dot{x}, x,f,e,t) =
\left(\begin{matrix}
\dot{x} - f\\
e - \nabla_x H(x)
\end{matrix}\right) = 0$$
and encode the appropriate sympectic structure in a linear subspace of $\mathcal{D}$.

The incoming *power* is $P_\text{in} = \left<e,f\right>$ for $(f,e)\in \mathcal{D}$

## Inheritance

For energetic systems-of-systems:

### Nodes are particular _energetic systems_
Each node is described by a set of differential-algebraic equations $\Phi(\dot{x},x,e,f) = 0$.

### Edges are constraints on port variables.

An edge represents how state is shared between systems.

## Composition

For networked dynamic systems _composition_ means that we can replace nodes with subgraphs and vice-versa.

## Corollary (Composition)
If $\Psi_1 = (M_1, \mathcal{D}_1, U_1,\Phi_1)$ and $\Psi_2 = (M_2, \mathcal{D}_2, U_2,\Phi_2)$  are energetic systems, then

$$\begin{eqnarray}\Psi_0 &=& \Psi_1 \oplus\Psi_2\\
&=&
\left(M_1\oplus M_2,\mathcal{D}_1 \oplus\mathcal{D}_2,U_1\oplus U_2, \Phi_1\oplus\Phi_2\right)
\end{eqnarray}$$
is also an energetic system.

Suppose (abusing notation) $\Psi_0 = (\Psi_1,\Psi_2)$ is an energetic system with ports

$$(e_i, f_i) \in \mathcal{D}_1, \quad (e_j,f_j)  \in \mathcal{D}_2$$

Then $\Phi_0$ with the additional power conserving constraint

$$e_i - e_j = 0\qquad f_i+f_j=0$$

is also a energetic system

## Encapsulation

For a networked dynamical system _encapsulation_ means that we can apply simplification methods to a subgraph so that the replacement system is less complicated, while representing the same behaviour.

One can also go the other way by replacing a node with a more complicated subgraph.

## Object Oriented Modelling and Energetic Systems

Energetic systems provide:
- _Inheritance_; an abstract base representation of energetic systems.
- _Composition_; a way to hierarchically compose systems of systems.
- _Encapsulation_; a framework inside which simplifications can occur.

#  Introducing `BondGraphTools`

## `BondGraphTools` a `python` library for energetic systems.

`BondGraphTools` (https://github.com/BondGraphTools) a framework for modelling energetic systems.
* Based upon an extension of bond graph and port-Hamiltonian modelling.
* Provies a simple, *minimal* object-oriented interface for constructing models.
* Implemented in `python` and uses the standard `scipy` stack.
* Performs symbolic model reduction and simulation.
* Simulations with DAE solvers in `julia`.
* Developed with sustainable software practices.
* Intended to be used in _conjunction_ with other tools.

'Bond Graphs' are a multi-domain port-based graphical modelling technique used predominantly in mechatronics.
Port-Hamiltonian systems integrate geometric approaches from classical mechanics and control theory with port based modelling.

## Example: Linear Oscillator

```python
class Linear_Osc(bgt.BondGraph):
    damping_rate = 0.1 #D amping rate common across oscillator array

    def __init__(self, freq, index):
        """Linear Oscillator Class

        Args:
            freq:  Natural (undamped) frequency of this oscillator
            index: Oscillator number (used for naming).

        Instances of this class are bond graph models of externally forced
        damped harmonic oscillators.
        In the electrical analogy, these is simply an open loop series RLC
        circuit."""

        # Create the components
        r = bgt.new("R", name="R", value=self.damping_rate)
        l = bgt.new("I", name="L", value=1/freq)
        c = bgt.new("C", name="C", value=1/freq)
        port = bgt.new("SS")
        conservation_law = bgt.new("1")

        # Create the composite model and add the components
        super().__init__(
            name=f"Osc_{index}",
            components=(r, l, c, port, conservation_law)
        )

        # Wire the model up
        for component in (r,l,c):
            bgt.connect(conservation_law, component)
        bgt.connect(port, conservation_law)

        # Expose the SS component as an external port
        bgt.expose(port, label="P_in")
```

`Linear_Osc`
- _inherits_ from BondGraph, which is a base 'class' containing much of functionality
- is _composed_ of a variety of subcomponents
- _encapsulates_ a one port RLC component.

```python
example_osc = Linear_Osc(1000,1)
example_osc.constitutive_relations
```

# Example: Using Linear oscillators for Coupled Cavity

```python
def coupled_cavity():
    model = bgt.new(name="Cavity Model")

    # Define the interaction Hamiltonain
    coupling_args = {
        "hamiltonian":"(w + G*x_1)*(x_2^2 + x_3^2)/2",
        "params": {"G": 1, "w": 6}
    }

    port_hamiltonian = bgt.new("PH", value=coupling_args)

    # Define the symplectic junction structure
    symplectic_gyrator = bgt.new("GY", value=-1)
    em_field = bgt.new("1")

    bgt.add(model, port_hamiltonian, symplectic_gyrator, em_field)
    bgt.connect(em_field, (port_hamiltonian, 1))
    bgt.connect(em_field, (symplectic_gyrator, 1))
    bgt.connect((port_hamiltonian, 2), (symplectic_gyrator, 0))

    # Construct the open part of the system
    dissipation = bgt.new("R", value=1)
    photon_source = bgt.new('SS')

    bgt.add(model, dissipation, photon_source)
    bgt.connect(em_field, dissipation)
    bgt.connect(photon_source, em_field)
    bgt.expose(photon_source)

    # Build the oscillator array
    frequencies = [2 + f for f in (-0.3, -0.1, 0, 0.1, 0.3)]
    osc_mean_field = bgt.new("0")

    bgt.add(model, osc_mean_field)
    bgt.connect(osc_mean_field, (port_hamiltonian, 0))
    osc_array = [Linear_Osc(freq, index)
        for index, freq in enumerate(frequencies)]

    for osc in osc_array:
        bgt.add(model, osc)
        bgt.connect(osc_mean_field, (osc, "P_in"))

    return model
```

## Running a simulation

```python
def experiment(cavity, signal, duration=50):
    laser = bgt.new("Se")
    experiment = bgt.new()

    experiment.add(cavity, laser)
    bgt.connect(cavity, laser)

    t,x = bgt.simulate(
        experiment,
        timespan=[0, duration],
        x0=[0 for _ in cavity.state_vars],
        control_vars=[signal]
    )

    # do post-processing or data shaping here

    # This just selects the oscillator
    indicies = [i for i, (_,  name)
                in enumerate(cavity.state_vars.values())
                if name == 'x_0']
    return t, x[:, indicies]
```

```python
cavity = coupled_cavity()

forcing = "cos(0.9t)"

t, x = experiment(cavity, signal=forcing)

fig = plot_graph(t, x) ## Just makes things pretty.
```

## Getting the equations

```python
cavity.constitutive_relations
```

```python
print_tree(cavity)
```

## Under the Hood

Each energetic systems is represented in local co-ordinates $\mathcal{X}_1 \in \{(\dot{x}_1,e_1,f_1,x_1,u_1)\}$

Motion on the subspace satisfies $\Phi_1(X) = L_1 X + V_1(X) =0$, where $L$ are matricies and $V$ is a strictly nonlinear vector field.

Suppose $\Phi_2$ is a similarly defined subsystem, then the column vector
$X =[X_1;X_2]$ satisfies

$$0 =LX+V(X)  =  \left(
\begin{matrix}
L_1 & 0 \\
0 & L_2 \end{matrix}
\right) X +
\left(\begin{matrix}V_1\circ \pi_1 \\ V_\beta\circ \pi_\beta \end{matrix}\right)(X)
$$

which gives a way of 'stitching togehtor' systems using power conserving flows via

$$e^j_1 = e^k_2,\qquad f^j_1 =-f^k_2$$

Then, we apply a bunch of symnbolic linear algebra, and projections/substitutions.

## State of `BondGraphTools`

Current Status:
- In active development (v.0.3.7) and active use within the lab.
- Documentation at https://bondgraphtools.readthedocs.io/en/latest/
- Available on PyPI https://pypi.org/project/BondGraphTools/
- Source on GitHub https://github.com/BondGraphTools/BondGraphTools
- Manuscript in preparation.

### Planned Future Developments
- Robust parameter and control value network.
- Interface for measuring port space.
- Algorithmic model reduction (particularly manifold reductions).
- Bifurcation analysis (particularly fixed point tracking).

# In Summary

- Emergence is an interesting and relevant phenomenom that is not well understood.
- Emergent phenomenom occur due to interactions between subsystems within the context of a larger system.
- Even in the case of synch, still much is unknowm.

- Emergence can be thought of in terms of low dimensional invariant manifolds.
- So, it would be useful to be able to "scale up" nonlinear analysis tools.
- But there are problems here particularly in how one represent systems

- `BondGraphTools` provides a way to build and recude big model in symbolic form.
- It is hoped that this can feed into algorithmic approaches via GSP to begin to answer these questions.

# Thank You!

Thanks to
- Eduardo and Robbie
- The University of Syndey
- Prof. Edmund Crampin
- The Systems Biology Lab at The University of Melbourne

```python
from BondGraphTools.reaction_builder import Reaction_Network

TCA_reactions = {
    "Citrate synthase":
        ["acetyl-CoA + oxaloacetate + H2O = citrate + CoA-SH"],
    "Aconitase":
        ["Citrate = cis-Aconitate + H2O", "cis-Aconitate + H2O = Isocitrate"],
    "Isocitrate dehydrogenase":
        ["Isocitrate + NAD = Oxalosuccinate + NADH + H",
         "Oxalosuccinate = a-Ketoglutarate + CO2" ],
    "a-Ketoglutarate dehydrogenase":
        ["a-Ketoglutarate + NAD + CoA-SH = Succinyl-CoA + NADH + H + CO2"],
     "Succinyl-CoA synthetase":
        ["Succinyl-CoA + ADP + Pi = Succinate + CoA-SH + ATP"],
     "Succinate dehydrogenase":
        ["Succinate + Q = Fumarate + QH2"],
     "Fumarase":
        ["Fumarate + H2O = L-Malate"],
     "Malate dehydrogenase":
        ["L-Malate + NAD = Oxaloacetate + NADH + H"]
}

def TCA_Cycle():
    reaction_net = Reaction_Network(name="TCA_Cycle")
    for enzyme in TCA_reactions:
        for index, reaction in enumerate(TCA_reactions[enzyme]):
            reaction_name = f"{enzyme} - {index}"
            reaction_net.add_reaction(reaction,  name=reaction_name)
    return reaction_net
```

```python
tca_bg = TCA_Cycle().as_network_model()
tca_bg.constitutive_relations
```

# Please check out `BondGraphTools`

# https://github.com/BondGraphTools/

```python

```


---
Project for the course in Microeconometrics | Summer 2021, M.Sc. Economics, Bonn University | Gözde Özden

# Replication of Lee, D. S., Moretti, E., & Butler, M. J. (2004): Do Voters Affect or Elect Policies? Evidence from the US House.
---

This notebook contains my replication of the results from the following paper:

> [Lee, D. S., Moretti, E., & Butler, M. J. (2004). Do Voters Affect or Elect Policies? Evidence from the US House. The Quarterly Journal of Economics, 119(3), 807-859](https://academic.oup.com/qje/article-abstract/119/3/807/1938834?redirectedFrom=fulltext).

##### Downloading and viewing this notebook:

* The best way to view this notebook is to visit repository which is located in from [GitHub](https://github.com/OpenSourceEconomics/ose-data-science-course-project-ozdengo). In the case of Latex rendering issue, the best way to view notebook properly is by downloading it. Also, it can be viewed by MyBinder or NBViewer. If some images are missing, please visit [files folder](https://github.com/OpenSourceEconomics/ose-data-science-course-project-ozdengo/tree/master/files).

##### About this notebook:

* I have three main purposes:

    * First, to understand and investigate concepts and theories of the close election design using the paper of Lee et al.(2004).
    * To replicate selected key parts and results of Lee et al.(2004).
    * To extend the results by checking validity of the design and bandwidth sensivity across different polynomial of order.

* I try to follow the original structure of the paper; however, the numbers of some titles may differ due to the fact that I do not replicate all parts of the paper.

Table of Contents

1. Introduction

2. Identification

3. Theoretical Framework

4. Estimating Framework

5. Replication of Lee, D. S., Moretti, E., & Butler, M. J. (2004)

5.1. Data & Descriptive Statistics

5.2. Results

5.2.1. Main Empirical Results

5.2.2. Test for Quasi-Random Assignment

5.2.3. Sensitivity to Alternative Measures of Voting Records

5.2.4. Heterogeneity

6. Extensions

6.1.  Check for the Validity of the Design

6.2. Bandwidth Sensitivity and Polynomial Models

7. Conclusion

8. References

```python
%matplotlib inline
import numpy as np
import pandas as pd
import pyreadstat as pread
import pandas.io.formats.style
import statsmodels as sm
import statsmodels.formula.api as smf
import matplotlib.pyplot as plt
from scipy.ndimage import gaussian_filter
from IPython.display import HTML, display
from stargazer.stargazer import Stargazer
import warnings
warnings.filterwarnings('ignore')

```

```python
from auxiliary.auxiliary_plots import *
from auxiliary.auxiliary_predictions import *
from auxiliary.auxiliary_tables import *
```

---
# 1. Introduction
---
The goal of Lee et al.(2004) is to examine how voters affect policy. The power of voters to affect or elect policies is depending on candidates' desire to make credible promises to execute moderate policies. There are two main theories about how political competition affects policies. The first theory, the converge theory, argues that political competition enables candidates to be more moderate about their policies. The most well-known component of this is the Median Voter Theorem. Politicians who only care about winning choose the same policy in extreme cases as "full policy convergence" (Downs, 1957). When opposing candidates are concerned not only about winning the election, but also about the policies that are adopted, a similar result emerges. Although opposing candidates may not have identical policies, electoral competition will force them to pursue policies that are more moderate than their primary goals as "partial policy convergence" (Wittman,1938;Calvert,1985). In this theory, voters "affect" policies by promoting the candidates to modarate their policies toward median voter.

Downs' theorem may be compromised in the case that main concerns of the politicians are policies and there is no chance for binding about policies. The second theory, the divergence theory, indicates that the elected politicians cannot be credible about the policy promises they make when they are candidates (Alesina, 1988). They simply adopt their fixed policies once they got the office. In this theory, voters merely "elect" policies rather than affecting policies as in converge theory.

On the one side, politicians moderate their policies when electoral promises are credible as in "partial policy converge". On the other side, elections are only a means of determining whether candidate's fixed policy will be adopted when politicans are not credible about their policy promises. The real question is that which of these opposite perspectives is more empirically valid. The paper of Lee et al. evaluates the relative importance of these different perspectives by regression discontiunity design and roll-call voting records data from the US House between 1946 and 1995.

This paper first discusses the identification strategy, then establishes theoretical and emprical frameworks. The results. The fifth parth of this paper presents the replication results of original paper. The fourth part contains my extensions about banwidth sensivity and polynomial order. The sixth part of this paper concludes.

---
# 2. Identification
---
Lee, Moretti and Butler indicates that the best way to understand the role of voters is to compare most preferred policies ("bliss points") of candidates with the policies they will actually adopt. More modarate voting records than candidates' bliss points show that voting behavior influences candidates' policy decisions. If the bliss points and voting records are same, the role of voters is electing the policies. However, because of the fact that there are no accurate assessments of candidates' bliss points, such an analysis is impossible.

Therefore, they adapted very clever structure. First of all, they focus on Democrat wins to examine whether voter affect or elect policies. Because it is important to eliminate any difficulties that occurs due to differences in ideological preferences of voters and politicians, demographic structures, resources available for candidates; they aim to randomize the Democrat win. To do so, they focus on wins with very narrow margin. For instance, they begin their analysis with less than a 2 percent vote share. Therefore, they try to create a virtual random environment in which it is ambigious who would get the office. It is obvious that the winner of 1992 election will have relatively stronger position in 1994 election. However, the main point of this structure is that creating randomized environment for 1992 elections provides randomized environment in which party's candidate has greater electoral strength for 1994 election. They try to use this difference in electoral strength to understand the effect of fully divergence and partially convergence.

They compare the voting scores of the winners in 1995-1996 elections where the Democrats got the office in 1994 elections and the voting scores of the winners where the Republicans got the office. This effect called as "overal effect" and consists of two factors.

The first factor is the effect of voting scores in 1995-1996 where the incumbent is Democrat in 1994 elections. Since the winner party in 1994 election will have greater electoral strength in the next election and Democrats are more liberal, it is expected that the voting scores would be more liberal in 1995-1996 election. Therefore, first factor indicates how policies elected by voters. It can be computed by analyzing the probability that Democrat will be winner where the office is gotten by Democrats and the difference between the votes of Democrats and Republicans.

The second factor shows that what candidates reaction would be to an exogenous shock of winning probability in 1994 election. In the case that politicians need to follow their promises, the votes of a Democrat candidate running against a Republican who is winner in 1994 election are expected to be less liberal than a Democrat incumbent. Therefore, this effect indicates how voters "affect" policies.

It is essential to compute relative magnitudes of these two factors to understand the relative importance of these effects: full policy divergence and partial policy convergence. This computation is possible thanks to randomization of Democrat wins.

---
# 3. Theoretical Framework
---
The underlying framework utilized for the analysis is a framework developed by Alesina (2008). Consider that there are two parties: D (Democrats) and R (Republicans). Assume that the policy space is unidimensional and policy preferences of Democrats and Republicans are represented by quadratic loss functions, respectively: $u(l) = -(1/2)(l-c)^2$ and $v(l) = -(1/2)l^2$ where $l$ is policy variable and $c(>0)$ bliss points.

The expected policy preferences of D and R before election are, respectively, $x^e$ and $y^e$ and rational expectations denoted by $x = x^e$ and $y=y^e$. The expected probability of win for a candidate is $P(x^e,y^e)$.

When $x^e>y^e$, the best strategy to get more votes is to moderate policy preferences such that:

\begin{equation}
    \frac{\partial P}{\partial x^e} > 0 , \frac{\partial P}{\partial y^e} > 0
\end{equation}

This game repeats for every period where period $t$ includes the election and the subsequent Congressional session, and same for next periods. For example, period $t = 1992$ consists of 1992 election and roll-call votes ($RC_{t}$) for Congressional session in 1993-1994.

There are 3 Nash equilibria for the solution of this game:

 1. **Full Policy Convergence**: $x^* = y^* = \lambda^*c$
     * The key result:
      \begin{equation}
         \frac{\partial P}{\partial x^e} = \frac{\partial P}{\partial y^e} = (\frac{\partial \lambda}{\partial P^*})c > 0
      \end{equation}

     * An increase in $P^*$ means that an exogenous increase in popularity of Democrats (like a helicopter drop of Democrats) and an increase in bargaining power of Democrats. Since the candidates change their policy preferences and adopt the same policies as $\frac{\partial x^*}{\partial P^*} > 0$, voters affect policies.

 2. **Partial Policy Convergence**: $0≤y^*≤x^*≤c$
     * The key result:
     \begin{equation}
         \frac{\partial x^*}{\partial P^*},\frac{\partial y^*}{\partial P^*} > 0
     \end{equation}

     * Voters can effect the policy promises of candidates although they cannot force them to adopt the same policies.

 3. **Full Policy Divergence**: $x^*=c, y^*=0$
     * The key result:
     \begin{equation}
         \frac{\partial x^*}{\partial P^*}  , \frac{\partial y^*}{\partial P^*} = 0
     \end{equation}

     * Exogenous increase or decrease in popularity does not affect the equilibrium. Therefore, voters can merely elect the policies.

There are two main assumption about voters that they are forward-looking and rational. In real life, candidates do not need to be credible about their policy promises. But in this game, it is assumed that candidates are credible due to prevent any lose on their reputation (Alesina, 1988).

---
# 4. Estimating Framework
---
The record of the roll-call voting outcomes for the candidates is computed as:

\begin{equation}
RC_{t} = (1 - D_{t})y_{t} + D_{t}x_{t}
\end{equation}

where $D_{t}$ represents whether Democrat wins the election in period t. Note that, the only observable policy is the winner's policy.

This equilibrium is converted into regression equilibrium as :

\begin{equation}
RC_{t} = constant + π_{0}P_{t}^* + π_{1}D_{t} + \epsilon_{t}
\end{equation}

\begin{equation}
RC_{t+1} = constant + π_{0}P_{t+1}^* + π_{1}D_{t+1} + \epsilon_{t+1}
\end{equation}

where $P^*$ indicates the popularity of D, and $\epsilon$ indicates the heterogeneity in bliss points across districts. If $π > 0$, then voters can affect policies as partial policy convergence. If $π = 0$, then voters merely effect policies as full policy divergence.

This regression equation cannot be computed directly due to the fact that $P^*$ is unobservable. However, we can let $D_{t}$ to be independent from $P_{t}^*$ and $\epsilon_{t}^*$ by randomizing the $D_{t}$. It follows that:

\begin{equation} \label{eq1}
\begin{split}
\underbrace{E[RC_{t+1} | D_{t} = 1] - E[RC_{t+1} | D_{t} = 0]}_\text{observable} & = \underbrace{π_{0}[P_{t+1}^*D - P_{t+1}^*R]}_\text{"affect" component (unobservable)} \\
& + \underbrace{π_{1}[P_{t+1}^D - P_{t+1}^R]}_\text{"elect" component (observable)} \\
& =  \underbrace{\gamma}_\text{total effect (observable)}
\end{split}
\end{equation}

where superscripts of D and R indicate the winner.

$\gamma$ indicates the overall effect of Democrat win in t on $RC_{t+1}$ and it is sum of elect and affect components. It can be computed by the simple difference in $RC_{t+1}$ between the regions where D and R got the office in $t$.

\begin{equation}
E[RC_{t} | D_{t} = 1] - E[RC_{t} | D_{t} = 0] = π_1
\end{equation}

Basically, $π_1$ can be computed by the difference in $RC_t$ between the regions won by D and R.

\begin{equation}
E[D_{t+1} | D_{t} = 1] - E[D_{t+1} | D_{t} = 0] = P_{t+1}^D - P_{t+1}^R
\end{equation}

The rest of the elect component can be computed by the difference in fraction of districts won by D in $t+1$.

Since the affect component cannot be estimated directly, it can be estimated by substracting the elect component from the total effect. The essential point is the randomization of $D_t$. Without it, there would be bias for $\gamma$ and $π_1$ and it will show that how policiticans can have more liberal bliss points in the districts where Democrats won.

---
# 5. Replication of Lee, D. S., Moretti, E., & Butler, M. J. (2004)
---

## 5.1. Data & Descriptive Statistics

The result of Lee et al. based on the data from U.S. House of Representatives election results from 1946 to 1996. There are several ways to measure the voting records of Representatives. They use the data of Americans for Democratic Action (ADA) provided by the liberal political organization for voting records since it is commonly used index in the literature. Twenty high-profile roll-call votes are selected by ADA for each congress, and an index between 0 and 100 is created for each Representative. Higher ADA score means that the voting record of Representative is more liberal.
They also calculate the Democrat share of votes by dividing the number of Democrat votes to the number of votes for both the Democrat and the Republican.

---
**NOTE**: The original data used in this paper by authors are available at [here](https://eml.berkeley.edu/~moretti/papers.html).

---

#### Table 1- Summary statistics

```python
pd.set_option("display.precision",2)
data = pread.read_dta("data/enricoall2.dta")[0]
data[["realada","demvoteshare","lagdemvoteshare"]].describe()

```

      realada
      demvoteshare
      lagdemvoteshare

      count
      18842.00
      20238.00
      18972.00

      mean
      40.74
      0.58
      0.58

      std
      32.69
      0.23
      0.23

      min
      -27.92
      0.00
      0.00

      25%
      10.22
      0.41
      0.41

      50%
      36.58
      0.55
      0.55

      75%
      71.63
      0.71
      0.72

      max
      128.85
      1.00
      1.00

##  5.2. Results

### 5.2.1. Main Empirical Results

It is important to examine the effect of exogenous change in roll-call votes to understand the difference between the effects of full policy divergence and partial policy divergence.
The total effect ( $\gamma$ ) contains of two factors. These factors are the elect component ( $π_1[P^{D}_{t+1} - P^{R}_{t+1}]$ ) and the effect component ( $π_0[P^{*D}_{t+1} - P^{*R}_{t+1}]$ ).

The elect component is expected to be higher than the effect component and candidates do not make big adjusments in their policy promises if voters merely elect the policies. The affect component will be higher than the elect component and candidates adjust their bliss points to increase their winning probability if voters affect policies.

The first graph shows the ADA scores at $t+1$ against the Democrat vote share at $t$. Each point on the graph indicates the average ADA score at $t+1$ with 0.01 intervals of the vote share at $t$. The verdical line shows the cutoff, 50% of the vote share. The right side of the line indicates the Democrat win districts while the left side is for the Republican win districts. The continuous line is the predicted ADA score at $t+1$. It is calculated by a regression that contains fourth-order polynomial in vote share and a dummy for the vote share of above the cutoff and an interaction between dummy and polnomial component. The dotted lines shows the 95% confidence interval.

There is an obvious discontinuous jump, about twenty points, in ADA score at the cutoff line on the graph below. This discontinuity indicates the total effect ($\gamma$). It is expected to that the probability of Democrat win at $t+1$ is higher if the winner is Democrat at $t$ in the same district. Therefore, the district that run by a Democrat at $t$ would be more liberal at $t+1$ due to advantage of winning. This first effect called as "elect" component as I discussed above.

The second component that affects $\gamma$ is the affect component. In order to understand this component, we should compare a Democrat in a district where a Republican held the seat (left side of the graph) and other Democrat who is incumbent in her district (right side of the graph). In this case, a democrat who is challenging a Republican should adjust her policies more than an incumbent Democrat. This is another affect to that the district would be more liberal, where a Democrat held the seat.

#### Figure 1 - Total Effect of Initial Win on Future ADA Scores: $\gamma$

```python
ada_score(data, lag=True)
```

The real question is that which component dominates another altough e found the $\gamma$. However; due to the fact that it is impossible to compute the affect component, we should analyze the elect component first in order to compare both effects. To analyze the elect component, Lee et al. compute $π_1$, the expected difference in votes between two parties, and [ $P^{D}_{t+1} - P^{R}_{t+1}$ ], the advantage of winning party, separately. Then, we can substract the elect component from $\gamma$ to analyze the affect component.

In Figure 2a, it is a jump at the cutoff about 45 points. In order to understand this jump, we should compare the disctricts where a Democrat barely won and lost. Therefore, we can understand voting of Democrats in district where barely won by a Republican thanks to average voting records of Democrats who barely won the election.

#### Figure 2a - Effect of Party Affiliation: $π_1$

```python
ada_score(data, lag=False)
```

The figure 2b indicates that the probability of winning for a Democrat is higher at $t+1$ in disctricts where a Democrat barely won at $t$. In this case, the jump is about 5 point.

#### Figure 2b - Effect of Initial Win on Winning Next Election: ( $P^{D}_{t+1} - P^{R}_{t+1}$ )

```python
probability_democrat(data)
```

For all graphs above, it is important to take account that the RD design is valid and the the winner at $t$ is generated by a random assignment.

Figure 1 shows that the total effect $\gamma$ is about twenty ADA points. Figure 2a indicates that $π_1$ is about 45 points and [ $P^{D}_{t+1} - P^{R}_{t+1}$ ] is about 0.5 points. Therefore the elect component, $π_1[ P^{D}_{t+1} - P^{R}_{t+1} ]$, is about $ 45 x 0.5 = 22.5 $. The difference between the total effect and elect component is $ 20 - 22.5 = - 2.5$. We understand that the exogenous shock in ADA score is not determined by the change in policy promises of candidates to increase her the probability of winning. This effect only works by changing the relative probability that a party that will get the office. Therefore, these graphs show that voters elect policies instead of affecting them.

Table 2 is created to understand the results deeply. Column 1 corresponds to the jump in Figure 1. It determined by the difference in average ADA scores at $t+1$ between the districts where Democrat vote share between 50% and 52% and the districts where Democrat vote share between 48% and 50%. The result of this difference is $21.4$.

Column 2 corresponds to the jump in Figure 2a, which is equal to $π_1$. It shows the difference in ADA score at $t$ between the disricts where Democrat vote share between 50% and 52% and the districts where Democrat vote share between 48% and 50%. The estimated result of difference is $47.7$.

Column 3 corresponds to [ $P^{D}_{t+1} - P^{R}_{t+1}$ ] which is graphed in Figure 2b. The estimated result of this jump is $0.46$. It means that in a district where the winner was Democrat at time $t$, the probability of being winner for a Democrat. is $0.46$ higher.

In column 4, to compute the elect component ( $π_1[P^{D}_{t+1} - P^{R}_{t+1}]$ ), I multiply column 2 and column 3 as Lee et al.
The result is $22.08$ and it is similar with the result in column 1. Therefore, it is possible to conclude that the affect component is small than the elect component.

#### Table 2 - Results Basen on ADA Scores - Close Elections Sample

```python
table_vscore(data, 'realada', bandwidth = True)
```

	$\gamma$	$π_1$	($P^{D}_{t+1} - P^{R}_{t+1}$)

Intercept	31.28***	18.90***	0.24***

	(1.36)	(0.92)	(0.02)

democrat		47.70***

		(1.32)

lagdemocrat	21.46***		0.46***

	(1.91)		(0.03)

Observations	950	950	1,019

R2	0.12	0.58	0.21

Adjusted R2	0.12	0.58	0.21

Residual Std. Error	29.50 (df=948)	20.40 (df=948)	0.44 (df=1017)

F Statistic	125.65*** (df=1; 948)	1297.90*** (df=1; 948)	278.07*** (df=1; 1017)

Note:

  *p<0.1;
  **p<0.05;
  ***p<0.01

#### Extension | Table - Results Basen on ADA Scores - Full Elections Sample

In the case that we estimate the global regression, the coefficients are larger than close election as in extension table below. The distance at $c_0$ spreads widely due to the fact that strong outliers exist in the data. Therefore, we have to restrict data as above to eliminate these outliers.

```python
table_vscore(data, 'realada', bandwidth = False)
```

	$\gamma$	$π_1$	($P^{D}_{t+1} - P^{R}_{t+1}$)

Intercept	23.62***	16.27***	0.12***

	(0.38)	(0.32)	(0.00)

democrat		41.83***

		(0.42)

lagdemocrat	31.43***		0.82***

	(0.49)		(0.00)

Observations	13,983	14,973	14,970

R2	0.23	0.40	0.67

Adjusted R2	0.23	0.40	0.67

Residual Std. Error	28.70 (df=13981)	25.45 (df=14971)	0.28 (df=14968)

F Statistic	4079.97*** (df=1; 13981)	9802.25*** (df=1; 14971)	30778.71*** (df=1; 14968)

Note:

  *p<0.1;
  **p<0.05;
  ***p<0.01

### 5.2.2. Test for Quasi-Random Assignment

It is crucial to remember that the randomization of the winner is key point for all estimation. Without it, estimation would be biased. The closer elections means that the difference in features of Democrats and Republicans become smaller.

For instance, we can say that the probability of winning for a Democrat is higher than a Republican in South districts based on the comparison geographical locations. However, if we restrict the elections around a cutoff, this difference decreases.

Differences occur due to characteristics is graphed in Figure 3 and 4. Real income, percentage of high school degree, percentage of black population, percentage of eligible for voting are analyzed to compare the differences at the 50% threshold. It is found that these differences are statistically insignificant and small.

#### Figure 3 - Similarity of Constituents’ Characteristics in Bare Democrat and Republican Districts—Part 1

```python
variables_1 = ["realincome", "pctblack", "pcthighschl", "pcteligible"]
variables_2 = ["votingpop", "North", "South", "West"]
all_variables = [variables_1, variables_2]
```

```python
subplot2_2(variables_1)
```

#### Figure 4 - Similarity of Constituents’ Characteristics in Bare Democrat and Republican Districts—Part 2

```python
subplot2_2(variables_2)
```

In Table 3, more characteristics used in order to understand the difference between Democrat and Republican. The results for entire sample can be seen in Column 1. Column 2 to Column 5, there are results for Democrat vote share with different restriction intervals In column 6, fourth order polynomial Democrat vote share is illustrated. It can be concluded that any significant difference is not observed around 50% threshold. Also, Figure 5 proves the continuity visually as there is no significant gap (3.8).

A valid RD design is proved thanks to these results. Therefore, we can safely say that the winning is randomized in close election design.

#### Table 3 - Difference in the District Characteristics between Democrat and Republican

```python
pd.set_option('display.float_format','{:.0f}'.format)
coef, err = table_char(data)
coef['type'] = ['coefficient']*len(coef)
err['type'] = ['err']*len(err)
table_2 = coef.append(err)
table_2 = table_2.groupby([table_2.index, 'type']).mean()
table_2.columns = ['All', '+/- 25', '+/- 10', '+/- 5','+/- 2','Polynomial']
table_2.style.set_precision(2)
```

        	        	        All	        +/- 25	        +/- 10	        +/- 5	        +/- 2	        Polynomial

        	        type

                        North
                        coefficient
                        -0.22
                        -0.16
                        -0.10
                        -0.05
                        -0.06
                        3.64

                                err
                        0.01
                        0.01
                        0.01
                        0.02
                        0.03
                        0.28

                        South
                        coefficient
                        0.25
                        0.14
                        0.09
                        0.05
                        0.01
                        -3.18

                                err
                        0.01
                        0.01
                        0.01
                        0.01
                        0.02
                        0.24

                        West
                        coefficient
                        -0.03
                        0.01
                        0.00
                        -0.00
                        0.06
                        0.40

                                err
                        0.01
                        0.01
                        0.01
                        0.01
                        0.02
                        0.22

                        pctblack
                        coefficient
                        0.08
                        0.04
                        0.01
                        0.00
                        -0.00
                        0.67

                                err
                        0.00
                        0.00
                        0.00
                        0.00
                        0.01
                        0.09

                        pcthighschl
                        coefficient
                        -0.04
                        -0.02
                        -0.01
                        -0.00
                        0.00
                        0.12

                                err
                        0.00
                        0.00
                        0.00
                        0.00
                        0.01
                        0.05

                        pcturban
                        coefficient
                        0.07
                        0.07
                        0.05
                        0.05
                        0.06
                        1.88

                                err
                        0.00
                        0.01
                        0.01
                        0.01
                        0.02
                        0.16

                        realincome
                        coefficient
                        -2588.50
                        -1238.05
                        384.58
                        754.65
                        841.99
                        38534.64

                                err
                        144.53
                        160.01
                        251.26
                        369.33
                        596.37
                        5407.22

#### Figure 5 - Specification Test: Similarity of Historical Voting Patterns between Bare Democrat and Republican Districts

```python
lagada_score(df)
```

### 5.2.3. Sensitivity to Alternative Measures of Voting Records

Lee, Moretti and Butler used the ADA voting score for all estimations. They try to find an answer to the question of whether there would be any change in results if other voting scores are used.

In Table 4, two different voting scores are analyzed to understand the answer of this question. The results based on DW-Nominate scores that created by McCarty, Poole and Rosenthal are illustrated in Table 4a. In table 4b, the results for the percent of the roll-call votes like Democrat leadership.

If DW-Nominate is used instead of ADA score, $\gamma$ is $-0.26$. The elect component, ( $π_1[P^{D}_{t+1} - P^{R}_{t+1}]$ ), is computed as $-0.27$ which is extremely close with $\gamma$. These estimations are negative due to the fact that DW-Nominate measures the voting scores more conservative.
If "percent voted like Democrat leadership" is used, $\gamma$ is computed as $0.14$. It is almost same with the elect component,( $π_1[P^{D}_{t+1} - P^{R}_{t+1}]$ ), which is $0.14$. Also, I show the graphical illustration of these results in Figure 6a and Figure 6b.

These results are compatible with the results in Table 2. Therefore, we can conclude that these results are independent from which index is chosen for voting score. With these alternative voting score measures, we can again argue that elect component dominates affect component.

```python
data2 = pread.read_dta("data/enricoall4.dta")[0]
```

#### Table 4a - Results Based on Nominate Scores - Close Election Sample

```python
table_vscore(data2, 'dwnom1', bandwidth = True)
```

	$\gamma$	$π_1$	($P^{D}_{t+1} - P^{R}_{t+1}$)

Intercept	0.14***	0.29***	0.24***

	(0.01)	(0.00)	(0.01)

democrat		-0.59***

		(0.01)

lagdemocrat	-0.26***		0.47***

	(0.01)		(0.01)

Observations	3,767	3,767	3,896

R2	0.14	0.69	0.22

Adjusted R2	0.14	0.69	0.22

Residual Std. Error	0.33 (df=3765)	0.20 (df=3765)	0.44 (df=3894)

F Statistic	595.11*** (df=1; 3765)	8197.56*** (df=1; 3765)	1103.76*** (df=1; 3894)

Note:

  *p<0.1;
  **p<0.05;
  ***p<0.01

#### Table 4b - Results Based on Percent Voted Like Democrat Leadership - Close Election Sample

```python
table_vscore(data2, 'eq_Dlead', bandwidth = True)
```

	$\gamma$	$π_1$	($P^{D}_{t+1} - P^{R}_{t+1}$)

Intercept	0.62***	0.54***	0.24***

	(0.00)	(0.00)	(0.01)

democrat		0.30***

		(0.00)

lagdemocrat	0.14***		0.47***

	(0.01)		(0.01)

Observations	3,865	3,865	3,896

R2	0.13	0.61	0.22

Adjusted R2	0.13	0.61	0.22

Residual Std. Error	0.18 (df=3863)	0.12 (df=3863)	0.44 (df=3894)

F Statistic	568.53*** (df=1; 3863)	5985.07*** (df=1; 3863)	1103.76*** (df=1; 3894)

Note:

  *p<0.1;
  **p<0.05;
  ***p<0.01

#### Figure 6a - Nominate Scores, by Democrat Vote Share

```python
dwscore(data2, lag=False)
```

#### Figure 6b - Percent Voted with Democrat Leader, by Democrat Vote Share

```python
Dlead(data2, lag=False)
```

### 5.2.4. Heterogeneity

The bliss points of candidates can vary over time and across districts. This called as heterogeneity. We assume that candidates' bliss points do not change across districts and over time. However; this assumption can be refuted if there are any differences in the gap policy promises between D and R across districts or over time.

The results for heterogeneity over time can be found at Table 5. The estimated discontinuity is smaller in 1970s and larger in 1990s as seen in Column 1. Column 3 shows that the advantage of winning increases over time. It is important to highlight that the difference between efect component and affect component are stable over time.

#### Table 5 - Results Based on ADA Scores, by Decade - Close Election Sample

```python
year_list = [[1946, 1958], [1960, 1968], [1970, 1978], [1980, 1996]]
all_year = [df.year.min(), df.year.max()]
table_multiyears(data, years = year_list)
```

      $ADA_{t+1}$
      $ADA_t$
      $DEM_{t+1}$

      1946-1958

      14.58
      41.91
      0.42

      (3.19)
      (2.29)
      (0.05)

      1960-1968

      22.79
      49.7
      0.47

      (3.73)
      (2.47)
      (0.05)

      1970-1978

      11.55
      46.63
      0.41

      (4.72)
      (3.36)
      (0.07)

      1980-1996

      44.86
      54.9
      0.66

      (3.52)
      (2.71)
      (0.05)

---
# 6. Extensions
---

## 6.1.  Check for the Validity of the Design

It is important to check the validity of the RD design by using several approaches. Therefore, we can understand that the possibility of manipulations on running variable. There are 3 main techniques to check validity of the design. Lee et al. conduct two of them : falsification test based on previous ADA scores and continiuty covariates. The third way is to use density test, which I investigate the distribution of Democrat Vote share. McCrary (2008) argues that a jump in the density or distribution means that the running variable manipulable which invalidates the design.

In order to prove the validity of the design, there should be no significant change in the density of the frequency at the threshold. As it can be seen in Figure 7, there was no manipulation on Democrat vote share.

#### Figure 7 - Distribution of Democrat Vote Share

```python
plot_hist(data, 'demvoteshare', bins = 20)
```

## 6.2. Bandwidth Sensitivity and Polynomial Models

Lee et. al use parametric methods in their research although non-parametric methods are usually preferred in the literature. I extend my analysis in order to ensure that results are non-sensitive to the specification choices. Table 6 represents the estimations for $\gamma$ using different bandwidth size and polynomial of orders. Increase in the bandwidth size due to decrease the bias, causes necessity of higher polynomial of orders. The scope of bias decreases with smaller bandwidth size. Therefore, higher polynomial of orders causes overfitting with smaller bandwidth size. Table 7 represents the same analysis for $π_1$. Columns and rows represent respectively bandwidth and polynomial orders.

#### Table 6 - Total Effect Estimation ($\gamma$)  by Different Bandwidth and Polynomial Order

```python
bw_list = [0, 0.375, 0.25, 0.2, 0.15, 0.1, 0.05, 0.02]
table_df = table_poly_and_bw(df,"score","lagdemocrat",1,bw_list)
for ord in range(2,5):
    a = table_poly_and_bw(df,"score","lagdemocrat",ord,bw_list)
    table_df = table_df.append(a)
table_df
```

      0
      0.375
      0.25
      0.2
      0.15
      0.1
      0.05
      0.02

      one

      31.47
      37.07
      36.16
      34.9
      33.31
      30.8
      24.8
      21.41

      (0.44)
      (0.44)
      (0.47)
      (0.51)
      (0.58)
      (0.72)
      (1.03)
      (1.72)

      two

      15.74
      18.53
      18.08
      17.45
      16.66
      15.4
      12.4
      10.71

      (0.22)
      (0.22)
      (0.23)
      (0.25)
      (0.29)
      (0.36)
      (0.52)
      (0.86)

      three

      10.49
      12.36
      12.05
      11.63
      11.1
      10.27
      8.27
      7.14

      (0.15)
      (0.15)
      (0.16)
      (0.17)
      (0.19)
      (0.24)
      (0.34)
      (0.57)

      four

      7.87
      9.27
      9.04
      8.73
      8.33
      7.7
      6.2
      5.35

      (0.11)
      (0.11)
      (0.12)
      (0.13)
      (0.14)
      (0.18)
      (0.26)
      (0.43)

#### Table 7 - Party Affiliation Effect Estimation ($π_1$)  by Different Bandwidth and Polynomial Order

```python
table_df = table_poly_and_bw(df,"score","democrat",1,bw_list)
for ord in range(2,5):
    a = table_poly_and_bw(df,"score","democrat",ord,bw_list)
    table_df = table_df.append(a)
table_df
```

      0
      0.375
      0.25
      0.2
      0.15
      0.1
      0.05
      0.02

      one

      42.59
      48.82
      48.91
      49.04
      49.37
      49.55
      49.25
      48.74

      (0.37)
      (0.36)
      (0.36)
      (0.39)
      (0.43)
      (0.51)
      (0.71)
      (1.11)

      two

      21.29
      24.41
      24.45
      24.52
      24.68
      24.78
      24.63
      24.37

      (0.19)
      (0.18)
      (0.18)
      (0.19)
      (0.22)
      (0.26)
      (0.36)
      (0.56)

      three

      14.2
      16.27
      16.3
      16.35
      16.46
      16.52
      16.42
      16.25

      (0.12)
      (0.12)
      (0.12)
      (0.13)
      (0.14)
      (0.17)
      (0.24)
      (0.37)

      four

      10.65
      12.2
      12.23
      12.26
      12.34
      12.39
      12.31
      12.18

      (0.09)
      (0.09)
      (0.09)
      (0.1)
      (0.11)
      (0.13)
      (0.18)
      (0.28)

---
# 7. Conclusion
---

In conclusion, the paper of Lee, Moretti an Butler (2004) is replicated in this paper. They find that the candidates do not shift their policy promises to the median voter due to incumbency effect. This supports the view of full policy divergence. They argued that voters merely elect policies rather than affect policies. The findings in this replication project are compatible with the results in original paper. Additionaly, I discussed the identification strategy of the paper and analyzed the results with using different bandwidth size and order polynomial. The results are same with different approaches.

The finding about full policy divergence do not represent overall degree of representativeness which can be defined as the degree of represented policies of supporters by politician. Neither Lee et al. nor I cannot find a suitable solution to analyze the most preferred policy choice of median voters.  These results can be represent the representativeness of median voter within the party. Instead of this, we try to analyze the deviations on candidates' bliss points due to competition.

Full policy divergence based on very strong assumptions and definitions that candidates do not change their policy promises due to exogenous shock in vote shares. The analysis conducted in this paper is compatible with this view. At least, we can conclude that candidates do not behave credible about their policy promises. However, it is beneficial to explore how voters can affect which candidates policy promises in which way.

---
# 8. References
---

* **Alesina, Alberto,** “Credibility and Policy Convergence in a Two-Party System with Rational Voters,” American Economic Review, LXXVIII (1988), 796–805.

* **Alesina, Alberto, and Stephen E. Spear,** “An Overlapping Generations Model of Electoral Competition,” Journal of Public Economics, XXXVII (1988), 359–379.

* **Downs, Anthony,** An Economic Theory of Democracy (New York, NY: Harper and Row, 1957).

* **Lee, D. S., & Lemieux, T. (2010)**. Regression Discontinuity Designs in Economics. *Journal of Economic Literature*, 48(2), 281-355.

* **McCrary, J. (2008)**. Manipulation of the Running Variable in the Regression Discontinuity Design: A Design Test. *Journal of Econometrics*, 142: 698–714.

* **Wittman, Donald,** “Candidate Motivation: A Synthesis of Alternative Theories,” American Political Science Review, LXXVII (1983), 142–157.

-------
Notebook by Gözde Özden

---