## Configurations for Colab

```python
import sys
IN_COLAB = 'google.colab' in sys.modules

if IN_COLAB:
    !apt-get install -y xvfb python-opengl > /dev/null 2>&1
    !pip install gym pyvirtualdisplay > /dev/null 2>&1
    !pip install JSAnimation==0.1

    from pyvirtualdisplay import Display

    # Start virtual display
    dis = Display(visible=0, size=(400, 400))
    dis.start()
```

# 05. Noisy Networks for Exploration

[M. Fortunato et al., "Noisy Networks for Exploration." arXiv preprint arXiv:1706.10295, 2017.](https://arxiv.org/pdf/1706.10295.pdf)

NoisyNet is an exploration method that learns perturbations of the network weights to drive exploration. The key insight is that a single change to the weight vector can induce a consistent, and potentially very complex, state-dependent change in policy over multiple time steps.

Firstly, let's take a look into a linear layer of a neural network with $p$ inputs and $q$ outputs, represented by

$$
y = wx + b,
$$

where $x \in \mathbb{R}^p$ is the layer input, $w \in \mathbb{R}^{q \times p}$, and $b \in \mathbb{R}$ the bias.

The corresponding noisy linear layer is defined as:

$$
y = (\mu^w + \sigma^w \odot \epsilon^w) x + \mu^b + \sigma^b \odot \epsilon^b,
$$

where $\mu^w + \sigma^w \odot \epsilon^w$ and $\mu^b + \sigma^b \odot \epsilon^b$ replace $w$ and $b$ in the first linear layer equation. The parameters $\mu^w \in \mathbb{R}^{q \times p}, \mu^b \in \mathbb{R}^q, \sigma^w \in \mathbb{R}^{q \times p}$ and $\sigma^b \in \mathbb{R}^q$ are learnable, whereas $\epsilon^w \in \mathbb{R}^{q \times p}$ and $\epsilon^b \in \mathbb{R}^q$ are noise random variables which can be generated by one of the following two ways:

1. **Independent Gaussian noise**: the noise applied to each weight and bias is independent, where each random noise entry is drawn from a unit Gaussian distribution. This means that for each noisy linear layer, there are $pq + q$ noise variables (for $p$ inputs to the layer and $q$ outputs).
2. **Factorised Gaussian noise:** This is a more computationally efficient way. It produces 2 random Gaussian noise vectors ($p, q$) and makes $pq + q$ noise entries by outer product as follows:

$$
\begin{align}
\epsilon_{i,j}^w &= f(\epsilon_i) f(\epsilon_j),\\
\epsilon_{j}^b &= f(\epsilon_i),\\
\text{where } f(x) &= sgn(x) \sqrt{|x|}.
\end{align}
$$

In all experiements of the paper, the authors used Factorised Gaussian noise, so we will go for it as well.

```python
import math
import os
from typing import Dict, List, Tuple

import gym
import matplotlib.pyplot as plt
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from IPython.display import clear_output
from torch.nn.utils import clip_grad_norm_
```

## Replay buffer

Please see *01.dqn.ipynb* for detailed description.

```python
class ReplayBuffer:
    """A simple numpy replay buffer."""

    def __init__(self, obs_dim: int, size: int, batch_size: int = 32):
        self.obs_buf = np.zeros([size, obs_dim], dtype=np.float32)
        self.next_obs_buf = np.zeros([size, obs_dim], dtype=np.float32)
        self.acts_buf = np.zeros([size], dtype=np.float32)
        self.rews_buf = np.zeros([size], dtype=np.float32)
        self.done_buf = np.zeros(size, dtype=np.float32)
        self.max_size, self.batch_size = size, batch_size
        self.ptr, self.size, = 0, 0

    def store(
        self,
        obs: np.ndarray,
        act: np.ndarray,
        rew: float,
        next_obs: np.ndarray,
        done: bool,
    ):
        self.obs_buf[self.ptr] = obs
        self.next_obs_buf[self.ptr] = next_obs
        self.acts_buf[self.ptr] = act
        self.rews_buf[self.ptr] = rew
        self.done_buf[self.ptr] = done
        self.ptr = (self.ptr + 1) % self.max_size
        self.size = min(self.size + 1, self.max_size)

    def sample_batch(self) -> Dict[str, np.ndarray]:
        idxs = np.random.choice(self.size, size=self.batch_size, replace=False)
        return dict(obs=self.obs_buf[idxs],
                    next_obs=self.next_obs_buf[idxs],
                    acts=self.acts_buf[idxs],
                    rews=self.rews_buf[idxs],
                    done=self.done_buf[idxs])

    def __len__(self) -> int:
        return self.size
```

## Noisy Layer

**References:**
- https://github.com/higgsfield/RL-Adventure/blob/master/5.noisy%20dqn.ipynb
- https://github.com/Kaixhin/Rainbow/blob/master/model.py

```python
class NoisyLinear(nn.Module):
    """Noisy linear module for NoisyNet.

    Attributes:
        in_features (int): input size of linear module
        out_features (int): output size of linear module
        std_init (float): initial std value
        weight_mu (nn.Parameter): mean value weight parameter
        weight_sigma (nn.Parameter): std value weight parameter
        bias_mu (nn.Parameter): mean value bias parameter
        bias_sigma (nn.Parameter): std value bias parameter

    """

    def __init__(self, in_features: int, out_features: int, std_init: float = 0.5):
        """Initialization."""
        super(NoisyLinear, self).__init__()

        self.in_features = in_features
        self.out_features = out_features
        self.std_init = std_init

        self.weight_mu = nn.Parameter(torch.Tensor(out_features, in_features))
        self.weight_sigma = nn.Parameter(
            torch.Tensor(out_features, in_features)
        )
        self.register_buffer(
            "weight_epsilon", torch.Tensor(out_features, in_features)
        )

        self.bias_mu = nn.Parameter(torch.Tensor(out_features))
        self.bias_sigma = nn.Parameter(torch.Tensor(out_features))
        self.register_buffer("bias_epsilon", torch.Tensor(out_features))

        self.reset_parameters()
        self.reset_noise()

    def reset_parameters(self):
        """Reset trainable network parameters (factorized gaussian noise)."""
        mu_range = 1 / math.sqrt(self.in_features)
        self.weight_mu.data.uniform_(-mu_range, mu_range)
        self.weight_sigma.data.fill_(
            self.std_init / math.sqrt(self.in_features)
        )
        self.bias_mu.data.uniform_(-mu_range, mu_range)
        self.bias_sigma.data.fill_(
            self.std_init / math.sqrt(self.out_features)
        )

    def reset_noise(self):
        """Make new noise."""
        epsilon_in = self.scale_noise(self.in_features)
        epsilon_out = self.scale_noise(self.out_features)

        # outer product
        self.weight_epsilon.copy_(epsilon_out.ger(epsilon_in))
        self.bias_epsilon.copy_(epsilon_out)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """Forward method implementation.

        We don't use separate statements on train / eval mode.
        It doesn't show remarkable difference of performance.
        """
        return F.linear(
            x,
            self.weight_mu + self.weight_sigma * self.weight_epsilon,
            self.bias_mu + self.bias_sigma * self.bias_epsilon,
        )

    @staticmethod
    def scale_noise(size: int) -> torch.Tensor:
        """Set scale to make noise (factorized gaussian noise)."""
        x = torch.FloatTensor(np.random.normal(loc=0.0, scale=1.0, size=size))

        return x.sign().mul(x.abs().sqrt())
```

## Noisy Network

We use NoisyLinear for the last two FC layers, and there is a method to reset noise at every step.
These are the only differences from the example of *01.dqn.ipynb*.

```python
class Network(nn.Module):
    def __init__(self, in_dim: int, out_dim: int):
        """Initialization."""
        super(Network, self).__init__()

        self.feature = nn.Linear(in_dim, 128)
        self.noisy_layer1 = NoisyLinear(128, 128)
        self.noisy_layer2 = NoisyLinear(128, out_dim)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """Forward method implementation."""
        feature = F.relu(self.feature(x))
        hidden = F.relu(self.noisy_layer1(feature))
        out = self.noisy_layer2(hidden)

        return out

    def reset_noise(self):
        """Reset all noisy layers."""
        self.noisy_layer1.reset_noise()
        self.noisy_layer2.reset_noise()
```

## DQN + NoisyNet Agent (w/o DuelingNet)

Here is a summary of DQNAgent class.

| Method           | Note                                                 |
| ---              | ---                                                  |
|select_action     | select an action from the input state.               |
|step              | take an action and return the response of the env.   |
|compute_dqn_loss  | return dqn loss.                                     |
|update_model      | update the model by gradient descent.                |
|target_hard_update| hard update from the local model to the target model.|
|train             | train the agent during num_frames.                   |
|test              | test the agent (1 episode).                          |
|plot              | plot the training progresses.                        |

In the paper, NoisyNet is used as a component of the Dueling Network Architecture, which includes Double-DQN and Prioritized Experience Replay. However, we don't implement them to simplify the tutorial. One thing to note is that NoisyNet is an alternertive to $\epsilon$-greedy method, so all $\epsilon$ related lines are removed. Please check all comments with *NoisyNet*.

```python
class DQNAgent:
    """DQN Agent interacting with environment.

    Attribute:
        env (gym.Env): openAI Gym environment
        memory (ReplayBuffer): replay memory to store transitions
        batch_size (int): batch size for sampling
        target_update (int): period for target model's hard update
        gamma (float): discount factor
        dqn (Network): model to train and select actions
        dqn_target (Network): target model to update
        optimizer (torch.optim): optimizer for training dqn
        transition (list): transition information including
                           state, action, reward, next_state, done
    """

    def __init__(
        self,
        env: gym.Env,
        memory_size: int,
        batch_size: int,
        target_update: int,
        gamma: float = 0.99,
    ):
        """Initialization.

        Args:
            env (gym.Env): openAI Gym environment
            memory_size (int): length of memory
            batch_size (int): batch size for sampling
            target_update (int): period for target model's hard update
            gamma (float): discount factor
        """
        # NoisyNet: All attributes related to epsilon are removed
        obs_dim = env.observation_space.shape[0]
        action_dim = env.action_space.n

        self.env = env
        self.memory = ReplayBuffer(obs_dim, memory_size, batch_size)
        self.batch_size = batch_size
        self.target_update = target_update
        self.gamma = gamma

        # device: cpu / gpu
        self.device = torch.device(
            "cuda" if torch.cuda.is_available() else "cpu"
        )
        print(self.device)

        # networks: dqn, dqn_target
        self.dqn = Network(obs_dim, action_dim).to(self.device)
        self.dqn_target = Network(obs_dim, action_dim).to(self.device)
        self.dqn_target.load_state_dict(self.dqn.state_dict())
        self.dqn_target.eval()

        # optimizer
        self.optimizer = optim.Adam(self.dqn.parameters())

        # transition to store in memory
        self.transition = list()

        # mode: train / test
        self.is_test = False

    def select_action(self, state: np.ndarray) -> np.ndarray:
        """Select an action from the input state."""
        # NoisyNet: no epsilon greedy action selection
        selected_action = self.dqn(
            torch.FloatTensor(state).to(self.device)
        ).argmax()
        selected_action = selected_action.detach().cpu().numpy()

        if not self.is_test:
            self.transition = [state, selected_action]

        return selected_action

    def step(self, action: np.ndarray) -> Tuple[np.ndarray, np.float64, bool]:
        """Take an action and return the response of the env."""
        next_state, reward, done, _ = self.env.step(action)

        if not self.is_test:
            self.transition += [reward, next_state, done]
            self.memory.store(*self.transition)

        return next_state, reward, done

    def update_model(self) -> torch.Tensor:
        """Update the model by gradient descent."""
        samples = self.memory.sample_batch()

        loss = self._compute_dqn_loss(samples)

        self.optimizer.zero_grad()
        loss.backward()
        # gradient clipping
        # https://pytorch.org/docs/stable/nn.html#torch.nn.utils.clip_grad_norm_
        clip_grad_norm_(self.dqn.parameters(), 1.0, norm_type=1)
        self.optimizer.step()

        # NoisyNet: reset noise
        self.dqn.reset_noise()
        self.dqn_target.reset_noise()

        return loss.item()

    def train(self, num_frames: int, plotting_interval: int = 200):
        """Train the agent."""
        self.is_test = False

        state = self.env.reset()
        update_cnt = 0
        losses = []
        scores = []
        score = 0

        for frame_idx in range(1, num_frames + 1):
            action = self.select_action(state)
            next_state, reward, done = self.step(action)

            state = next_state
            score += reward

            # NoisyNet: removed decrease of epsilon

            # if episode ends
            if done:
                state = env.reset()
                scores.append(score)
                score = 0

            # if training is ready
            if len(self.memory) >= self.batch_size:
                loss = self.update_model()
                losses.append(loss)
                update_cnt += 1

                # if hard update is needed
                if update_cnt % self.target_update == 0:
                    self._target_hard_update()

            # plotting
            if frame_idx % plotting_interval == 0:
                self._plot(frame_idx, scores, losses)

        self.env.close()

    def test(self) -> List[np.ndarray]:
        """Test the agent."""
        self.is_test = True

        state = self.env.reset()
        done = False
        score = 0

        frames = []
        while not done:
            frames.append(self.env.render(mode="rgb_array"))
            action = self.select_action(state)
            next_state, reward, done = self.step(action)

            state = next_state
            score += reward

        print("score: ", score)
        self.env.close()

        return frames

    def _compute_dqn_loss(self, samples: Dict[str, np.ndarray]) -> torch.Tensor:
        """Return dqn loss."""
        device = self.device  # for shortening the following lines
        state = torch.FloatTensor(samples["obs"]).to(device)
        next_state = torch.FloatTensor(samples["next_obs"]).to(device)
        action = torch.LongTensor(samples["acts"].reshape(-1, 1)).to(device)
        reward = torch.FloatTensor(samples["rews"].reshape(-1, 1)).to(device)
        done = torch.FloatTensor(samples["done"].reshape(-1, 1)).to(device)

        # G_t   = r + gamma * v(s_{t+1})  if state != Terminal
        #       = r                       otherwise
        curr_q_value = self.dqn(state).gather(1, action)
        next_q_value = self.dqn_target(next_state).max(
            dim=1, keepdim=True
        )[0].detach()
        mask = 1 - done
        target = (reward + self.gamma * next_q_value * mask).to(self.device)

        # calculate dqn loss
        loss = ((target - curr_q_value).pow(2)).mean()

        return loss

    def _target_hard_update(self):
        """Hard update: target <- local."""
        self.dqn_target.load_state_dict(self.dqn.state_dict())

    def _plot(
        self,
        frame_idx: int,
        scores: List[float],
        losses: List[float],
    ):
        """Plot the training progresses."""
        clear_output(True)
        plt.figure(figsize=(20, 5))
        plt.subplot(131)
        plt.title('frame %s. score: %s' % (frame_idx, np.mean(scores[-10:])))
        plt.plot(scores)
        plt.subplot(132)
        plt.title('loss')
        plt.plot(losses)
        plt.show()
```

## Environment

You can see the [code](https://github.com/openai/gym/blob/master/gym/envs/classic_control/cartpole.py) and [configurations](https://github.com/openai/gym/blob/master/gym/envs/__init__.py#L53) of CartPole-v0 from OpenAI's repository.

```python
# environment
env_id = "CartPole-v0"
env = gym.make(env_id)
```

## Set random seed

```python
seed = 777

def seed_torch(seed):
    torch.manual_seed(seed)
    if torch.backends.cudnn.enabled:
        torch.backends.cudnn.benchmark = False
        torch.backends.cudnn.deterministic = True

np.random.seed(seed)
seed_torch(seed)
env.seed(seed)
```

    [777]

## Initialize

```python
# parameters
num_frames = 10000
memory_size = 1000
batch_size = 32
target_update = 200

# train
agent = DQNAgent(env, memory_size, batch_size, target_update)
```

    cuda

## Train

```python
agent.train(num_frames)
```

## Test

Run the trained agent (1 episode).

```python
frames = agent.test()
```

    score:  116.0

## Render

```python
# Imports specifically so we can render outputs in Colab.
from matplotlib import animation
from JSAnimation.IPython_display import display_animation
from IPython.display import display

def display_frames_as_gif(frames):
    """Displays a list of frames as a gif, with controls."""
    patch = plt.imshow(frames[0])
    plt.axis('off')

    def animate(i):
        patch.set_data(frames[i])

    anim = animation.FuncAnimation(
        plt.gcf(), animate, frames = len(frames), interval=50
    )
    display(display_animation(anim, default_mode='loop'))

# display
display_frames_as_gif(frames)
```

    <input id="_anim_sliderDIJSPYUVAAIVCOHT" type="range" style="width:350px" name="points" min="0" max="1" step="1" value="0" onchange="animDIJSPYUVAAIVCOHT.set_frame(parseInt(this.value));"></input>

    <button onclick="animDIJSPYUVAAIVCOHT.slower()">–</button>
    <button onclick="animDIJSPYUVAAIVCOHT.first_frame()"></button>
    <button onclick="animDIJSPYUVAAIVCOHT.previous_frame()"></button>
    <button onclick="animDIJSPYUVAAIVCOHT.reverse_animation()"></button>
    <button onclick="animDIJSPYUVAAIVCOHT.pause_animation()"></button>
    <button onclick="animDIJSPYUVAAIVCOHT.play_animation()"></button>
    <button onclick="animDIJSPYUVAAIVCOHT.next_frame()"></button>
    <button onclick="animDIJSPYUVAAIVCOHT.last_frame()"></button>
    <button onclick="animDIJSPYUVAAIVCOHT.faster()">+</button>
  <form action="#n" name="_anim_loop_selectDIJSPYUVAAIVCOHT" class="anim_control">
    <input type="radio" name="state" value="once" > Once </input>
    <input type="radio" name="state" value="loop" checked> Loop </input>
    <input type="radio" name="state" value="reflect" > Reflect </input>
  </form>


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Promijeni vidljivost <a href="javascript:code_toggle()">ovdje</a>.''')
display(tag)

# Hide the code completely

# from IPython.display import HTML
# tag = HTML('''<style>
# div.input {
#     display:none;
# }
# </style>''')
# display(tag)
```

Promijeni vidljivost ovdje.

```python
%matplotlib notebook
import numpy as np
import control as control
import matplotlib.pyplot as plt
import ipywidgets as widgets
import scipy.signal as signal
continuous_update=False
```

## Vremenski odziv sustava prvog reda

Prijenosna funkcija odabranog sustava prvog reda definirana je kao

\begin{equation}
    \frac{K_p}{\tau_p s+1},
\end{equation}

gdje su $K_p$ i $\tau_p$ parametri sustava.

Odziv sustava ovisi o ulaznom signalu. U ovom primjeru se kao mogući ulazni signali koriste: jedinična step funkcija (s Laplaceovom transformacijom $\frac{1}{s}$), jedinična impulsna funkcija (s Laplaceovom transformacijom $1$), jedinična rampa funkcija (s Laplaceovom transformacijom $\frac{1}{s^2}$) i sinusni signal (s Laplaceovom transformacijom $\frac{1}{s^2+1}$).

Grafikon (dolje) prikazuje ulazni signal i odgovarajući izlazni signal za odabrane vrijednosti parametara $K_p$ i $\tau_p$.

### Kako koristiti ovaj interaktivni primjer?

Mijenjajte različite ulazne funkcije (step, impuls, rampa ili sinus). Pomičite klizače za promjenu vrijednosti $K_p$ i $\tau_p$.

```python
# sinus, step, ramp, x^2, sqrt(x)
functionSelect = widgets.ToggleButtons(
    options=[('step funkcija', 0), ('impulsna funkcija', 1), ('rampa funkcija', 2), ('funkcija sinus', 3)],
    description='Odabir: ')

fig = plt.figure(num='Vremenski odziv sustava prvog reda')
fig.set_size_inches((9.8, 3))
fig.set_tight_layout(True)
f1 = fig.add_subplot(1, 1, 1)

f1.grid(which='both', axis='both', color='lightgray')

f1.set_xlabel('$t$ [s]')
f1.set_ylabel('ulaz, izlaz')

f1.axhline(0,Color='black',linewidth=0.5)
f1.axvline(0,Color='black',linewidth=0.5)

inputf, = f1.plot([],[])
responsef, = f1.plot([],[])
arrowf, = f1.plot([],[])

num_samples=2041

def create_draw_functions(Kp,taup,index):
    t=np.linspace(-0.1,5,num_samples)

    num=[Kp]
    den=[taup,1]
    Wsys=control.tf(num,den)

    global inputf,responsef, arrowf

    if index==0:
        yin=np.zeros(2041)
        yin[40:num_samples]=1
        tnew=np.linspace(0,5,2001)
        tout,yout=control.step_response(Wsys, T=tnew)
    elif index==1:
        yin=signal.unit_impulse(2001, 0)
        tnew=np.linspace(0,5,2001)
        t=tnew
        tout,yout=control.impulse_response(Wsys,tnew,X0=0)
    elif index==2:
        yin=np.zeros(num_samples)
        yin[40:num_samples]=np.linspace(0,5,2001)
        tnew=np.linspace(0,5,2001)
        tout,yout,xx=control.forced_response(Wsys,tnew,yin[40:])
    elif index==3:
        yin=np.sin(np.linspace(0,30,2001))
        tnew=np.linspace(0,30,2001)
        t=tnew
        tout,yout,xx=control.forced_response(Wsys,tnew,yin)

    f1.lines.remove(inputf)
    f1.lines.remove(responsef)
    f1.lines.remove(arrowf)

    inputf, = f1.plot(t,yin,color='C0',label='ulaz')
    responsef, = f1.plot(tout,yout,color='C1',label='izlaz')

    if index == 1:
        arrowf, = f1.plot([-0.1,0,0.1],[0.95,1,0.95],color='C0')
    else:
        arrowf, = f1.plot([],[])

    f1.legend()

    f1.relim()
    f1.relim()
    f1.autoscale_view()
    f1.autoscale_view()

Kp_slider = widgets.FloatSlider(value=1, min=0, max=2, step=0.1, description='$K_p$',
                          continuous_update=True, layout=widgets.Layout(width='auto', flex='5 5 auto'),readout_format='.1f')

taup_slider = widgets.FloatSlider(value=1, min=0, max=2, step=0.1, description='$\\tau_p$',
                          continuous_update=True, layout=widgets.Layout(width='auto', flex='5 5 auto'),readout_format='.1f')

input_data = widgets.interactive_output(create_draw_functions, {'Kp':Kp_slider,
                                                                'taup':taup_slider,
                                                                'index':functionSelect})

def update_sliders(index):
    global x0_slider

    Kpval = [1,1,1,1]
    Kp_slider.value = Kpval[index]
    taupval = [1,1,1,1]
    taup_slider.value = taupval[index]

input_data2 = widgets.interactive_output(update_sliders, {'index':functionSelect})

display(functionSelect)

display(Kp_slider,taup_slider,input_data)

# display(Markdown("The system can be represented as $f(x)=5$ for small excursions of x about x0."))
```

    <IPython.core.display.Javascript object>

    ToggleButtons(description='Odabir: ', options=(('step funkcija', 0), ('impulsna funkcija', 1), ('rampa funkcij…

    FloatSlider(value=1.0, description='$K_p$', layout=Layout(flex='5 5 auto', width='auto'), max=2.0, readout_for…

    FloatSlider(value=1.0, description='$\\tau_p$', layout=Layout(flex='5 5 auto', width='auto'), max=2.0, readout…

    Output()

```python

```


# Lambda School Data Science Module 142
## Sampling, Confidence Intervals, and Hypothesis Testing

## Prepare - examine other available hypothesis tests

If you had to pick a single hypothesis test in your toolbox, t-test would probably be the best choice - but the good news is you don't have to pick just one! Here's some of the others to be aware of:

```
import numpy as np
from scipy.stats import chisquare  # One-way chi square test

# Chi square can take any crosstab/table and test the independence of rows/cols
# The null hypothesis is that the rows/cols are independent -> low chi square
# The alternative is that there is a dependence -> high chi square
# Be aware! Chi square does *not* tell you direction/causation

ind_obs = np.array([[1, 1], [2, 2]]).T
print(ind_obs)
print(chisquare(ind_obs, axis=None))

dep_obs = np.array([[16, 18, 16, 14, 12, 12], [32, 24, 16, 28, 20, 24]]).T
print(dep_obs)
print(chisquare(dep_obs, axis=None))
```

    [[1 2]
     [1 2]]
    Power_divergenceResult(statistic=0.6666666666666666, pvalue=0.8810148425137847)
    [[16 32]
     [18 24]
     [16 16]
     [14 28]
     [12 20]
     [12 24]]
    Power_divergenceResult(statistic=23.31034482758621, pvalue=0.015975692534127565)

```
# Distribution tests:
# We often assume that something is normal, but it can be important to *check*

# For example, later on with predictive modeling, a typical assumption is that
# residuals (prediction errors) are normal - checking is a good diagnostic

from scipy.stats import normaltest
# Poisson models arrival times and is related to the binomial (coinflip)
sample = np.random.poisson(5, 1000)
print(normaltest(sample))  # Pretty clearly not normal
```

    NormaltestResult(statistic=39.52271588422112, pvalue=2.6166840720314508e-09)

```
# Kruskal-Wallis H-test - compare the median rank between 2+ groups
# Can be applied to ranking decisions/outcomes/recommendations
# The underlying math comes from chi-square distribution, and is best for n>5
from scipy.stats import kruskal

x1 = [1, 3, 5, 7, 9]
y1 = [2, 4, 6, 8, 10]
print(kruskal(x1, y1))  # x1 is a little better, but not "significantly" so

x2 = [1, 1, 1]
y2 = [2, 2, 2]
z = [2, 2]  # Hey, a third group, and of different size!
print(kruskal(x2, y2, z))  # x clearly dominates
```

    KruskalResult(statistic=0.2727272727272734, pvalue=0.6015081344405895)
    KruskalResult(statistic=7.0, pvalue=0.0301973834223185)

And there's many more! `scipy.stats` is fairly comprehensive, though there are even more available if you delve into the extended world of statistics packages. As tests get increasingly obscure and specialized, the importance of knowing them by heart becomes small - but being able to look them up and figure them out when they *are* relevant is still important.

## Live Lecture - let's explore some more of scipy.stats

Candidate topics to explore:

- `scipy.stats.chi2` - the Chi-squared distribution, which we can use to reproduce the Chi-squared test
- Calculate the Chi-Squared test statistic "by hand" (with code), and feed it into `chi2`
- Build a confidence interval with `stats.t.ppf`, the t-distribution percentile point function (the inverse of the CDF) - we can write a function to return a tuple of `(mean, lower bound, upper bound)` that you can then use for the assignment (visualizing confidence intervals)

```
# Taking requests! Come to lecture with a topic or problem and we'll try it.

import pandas as pd
import numpy as np
from scipy import stats

df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/adult.csv', na_values=" ?")
print(df.shape)
df.head()
```

    (32561, 15)

      age
      workclass
      fnlwgt
      education
      education-num
      marital-status
      occupation
      relationship
      race
      sex
      capital-gain
      capital-loss
      hours-per-week
      country
      salary

      0
      39
      State-gov
      77516
      Bachelors
      13
      Never-married
      Adm-clerical
      Not-in-family
      White
      Male
      2174
      0
      40
      United-States
      <=50K

      1
      50
      Self-emp-not-inc
      83311
      Bachelors
      13
      Married-civ-spouse
      Exec-managerial
      Husband
      White
      Male
      0
      0
      13
      United-States
      <=50K

      2
      38
      Private
      215646
      HS-grad
      9
      Divorced
      Handlers-cleaners
      Not-in-family
      White
      Male
      0
      0
      40
      United-States
      <=50K

      3
      53
      Private
      234721
      11th
      7
      Married-civ-spouse
      Handlers-cleaners
      Husband
      Black
      Male
      0
      0
      40
      United-States
      <=50K

      4
      28
      Private
      338409
      Bachelors
      13
      Married-civ-spouse
      Prof-specialty
      Wife
      Black
      Female
      0
      0
      40
      Cuba
      <=50K

```
df.isnull().sum()
```

    age                  0
    workclass         1836
    fnlwgt               0
    education            0
    education-num        0
    marital-status       0
    occupation        1843
    relationship         0
    race                 0
    sex                  0
    capital-gain         0
    capital-loss         0
    hours-per-week       0
    country            583
    salary               0
    dtype: int64

```
cut_points = [0, 9, 19, 29, 39, 49, 1000]
label_names = ['0-9', '10-19', '20-29', '30-39', '40-49', '50+']
df['hours_per_week_categories'] = pd.cut(df['hours-per-week'], cut_points, labels=label_names)

df.head()
```

      age
      workclass
      fnlwgt
      education
      education-num
      marital-status
      occupation
      relationship
      race
      sex
      capital-gain
      capital-loss
      hours-per-week
      country
      salary
      hours_per_week_categories

      0
      39
      State-gov
      77516
      Bachelors
      13
      Never-married
      Adm-clerical
      Not-in-family
      White
      Male
      2174
      0
      40
      United-States
      <=50K
      40-49

      1
      50
      Self-emp-not-inc
      83311
      Bachelors
      13
      Married-civ-spouse
      Exec-managerial
      Husband
      White
      Male
      0
      0
      13
      United-States
      <=50K
      10-19

      2
      38
      Private
      215646
      HS-grad
      9
      Divorced
      Handlers-cleaners
      Not-in-family
      White
      Male
      0
      0
      40
      United-States
      <=50K
      40-49

      3
      53
      Private
      234721
      11th
      7
      Married-civ-spouse
      Handlers-cleaners
      Husband
      Black
      Male
      0
      0
      40
      United-States
      <=50K
      40-49

      4
      28
      Private
      338409
      Bachelors
      13
      Married-civ-spouse
      Prof-specialty
      Wife
      Black
      Female
      0
      0
      40
      Cuba
      <=50K
      40-49

```
df['sex'].iloc[0]
```

    ' Male'

```
df = df.sort_values(by='hours_per_week_categories', ascending=True)
df.head()
```

      age
      workclass
      fnlwgt
      education
      education-num
      marital-status
      occupation
      relationship
      race
      sex
      capital-gain
      capital-loss
      hours-per-week
      country
      salary
      hours_per_week_categories

      31290
      55
      Self-emp-not-inc
      41938
      Bachelors
      13
      Married-civ-spouse
      Prof-specialty
      Wife
      White
      Female
      0
      0
      8
      United-States
      <=50K
      0-9

      5172
      32
      NaN
      134886
      HS-grad
      9
      Married-civ-spouse
      NaN
      Wife
      White
      Female
      0
      0
      2
      United-States
      >50K
      0-9

      22928
      17
      NaN
      332666
      10th
      6
      Never-married
      NaN
      Own-child
      White
      Female
      0
      0
      4
      United-States
      <=50K
      0-9

      7902
      35
      Private
      359131
      Bachelors
      13
      Married-civ-spouse
      Prof-specialty
      Wife
      White
      Female
      7298
      0
      8
      NaN
      >50K
      0-9

      6604
      41
      Private
      406603
      HS-grad
      9
      Never-married
      Other-service
      Not-in-family
      White
      Male
      0
      0
      6
      Iran
      <=50K
      0-9

```
contingency_table = pd.crosstab(df['sex'], df['hours_per_week_categories'], margins=True)

contingency_table
```

      hours_per_week_categories
      0-9
      10-19
      20-29
      30-39
      40-49
      50+
      All

      sex

      Female
      235
      671
      1287
      1914
      5636
      1028
      10771

      Male
      223
      575
      1105
      1753
      12700
      5434
      21790

      All
      458
      1246
      2392
      3667
      18336
      6462
      32561

```
female_count = contingency_table.iloc[0][0:6].values
female_count
```

    array([ 235,  671, 1287, 1914, 5636, 1028])

```
male_count = contingency_table.iloc[1][0:6].values
male_count
```

    array([  223,   575,  1105,  1753, 12700,  5434])

```
import matplotlib.pyplot as plt
import seaborn as sns

#Plots the bar chart
fig = plt.figure(figsize=(10, 5))
sns.set(font_scale=1.8)
categories = ["0-9","10-19","20-29","30-39","40-49","50+"]
p1 = plt.bar(categories, male_count, 0.55, color='#d62728')
p2 = plt.bar(categories, female_count, 0.55, bottom=male_count)
plt.legend((p2[0], p1[0]), ('Male', 'Female'))
plt.xlabel('Hours per Week Worked')
plt.ylabel('Count')
plt.show()
```

## Expected Value Calculation
\begin{align}
expected_{i,j} =\frac{(row_{i} \text{total})(column_{j} \text{total}) }{(\text{total observations})}
\end{align}

```
# Get row sums
row_sums = contingency_table.iloc[0:2,6].values
col_sums = contingency_table.iloc[2,0:6].values
print(row_sums)
print(col_sums)
```

    [10771 21790]
    [  458  1246  2392  3667 18336  6462]

```
total_obs = contingency_table.loc['All','All']
total_obs
```

    32561

```
expected = []

for i in range(len(row_sums)):
  expected_row = []
  for column in col_sums:
    expected_val = row_sums[i] * column / total_obs
    expected_row.append(expected_val)
  expected.append(expected_row)

print(np.array(expected))
```

    [[  151.50388502   412.16995793   791.26046497  1213.02346365
       6065.44811277  2137.59411566]
     [  306.49611498   833.83004207  1600.73953503  2453.97653635
      12270.55188723  4324.40588434]]

```
observed = pd.crosstab(df['sex'], df['hours_per_week_categories'])
observed
```

      hours_per_week_categories
      0-9
      10-19
      20-29
      30-39
      40-49
      50+

      sex

      Female
      235
      671
      1287
      1914
      5636
      1028

      Male
      223
      575
      1105
      1753
      12700
      5434

```
observed = observed.values
observed
```

    array([[  235,   671,  1287,  1914,  5636,  1028],
           [  223,   575,  1105,  1753, 12700,  5434]])

## Chi-Squared Statistic with Numpy

\begin{align}
\chi^2 = \sum \frac{(observed_{i}-expected_{i})^2}{(expected_{i})}
\end{align}

For the $observed$ values we will just use a version of our contingency table without the margins as a numpy array. In this way, if our observed values array and our expected values array are the same shape, then we can subtract them and divide them directly which makes the calculations a lot cleaner. No for loops!

```
chi_squ = ((observed-expected)**2/expected).sum()
print(f'Chi-Squared: {chi_squ}')
```

    Chi-Squared: 2287.190943926107

```
# calculate degress of freedom

dof = (len(row_sums)-1)*(len(col_sums)-1)
print(f'Degrees of Freedom: {dof}')
```

    Degrees of Freedom: 5

```
chi_squ, p_value, dof, expected = stats.chi2_contingency(observed)

print(f'Chi-Squared: {chi_squ}')
print(f'P-Value: {p_value}')
print(f'Degrees of Freedom: {dof}')
print('Expected: \n', np.array(expected))
```

    Chi-Squared: 2287.190943926107
    P-Value: 0.0
    Degrees of Freedom: 5
    Expected:
     [[  151.50388502   412.16995793   791.26046497  1213.02346365
       6065.44811277  2137.59411566]
     [  306.49611498   833.83004207  1600.73953503  2453.97653635
      12270.55188723  4324.40588434]]

## Confidence Interval Example

```
# confidence_interval = [lower_bound, upper_bound]

coinflips = np.random.binomial(n=1, p=.5, size=100)
print(coinflips)
```

    [1 1 0 0 0 1 0 0 0 0 0 0 1 0 1 1 0 1 0 1 1 1 0 0 1 0 0 1 0 0 0 1 0 0 0 1 1
     0 1 0 1 0 0 1 0 1 1 0 0 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 0 0 0 0 1 0 1 1 1 1
     1 0 1 1 0 1 0 1 0 0 0 1 1 0 1 1 0 1 0 1 1 1 0 0 1 1]

```
coinflips.mean()
```

    0.48

```
stats.ttest_1samp(coinflips, .5)
```

    Ttest_1sampResult(statistic=-0.39831375340784614, pvalue=0.6912566363051549)

```
coinflips_100 = np.random.binomial(n=1, p=.5, size=100)
coinflips_1000 = np.random.binomial(n=1, p=.5, size=1000)

print('100 coinflips mean:', coinflips_100.mean())
print('1000 coinflips mean:', coinflips_1000.mean())

print('100 coinflips std:', np.std(coinflips_100))
print('1000 coinflips std:', np.std(coinflips_1000))
```

    100 coinflips mean: 0.49
    1000 coinflips mean: 0.487
    100 coinflips std: 0.4998999899979995
    1000 coinflips std: 0.49983097142934235

```
print('100 coinflips standard error:', stats.sem(coinflips_100))
print('1000 coinflips standard error:', stats.sem(coinflips_1000))
```

    100 coinflips standard error: 0.05024183937956914
    1000 coinflips standard error: 0.01581395210189664

## Confidence Interval Equation

```
import numpy as np
from scipy import stats

def confidence_interval(data, confidence=0.95):
  """
  Calculate a confidence interval around a sample mean for given data.
  Using t-distribution and two-tailed test, default 95% confidence.

  Arguments:
    data - iterable (list or numpy array) of sample observations
    confidence - level of confidence for the interval

  Returns:
    tuple of (mean, lower bound, upper bound)
  """

  data = np.array(data)
  mean = np.mean(data)
  n = len(data)
  stderr = stats.sem(data)
  interval = stderr * stats.t.ppf((1 + confidence) / 2.0, n - 1)
  return (mean, mean - interval, mean + interval)

def report_confidence_interval(confidence_interval):
  """
  Return a string with a pretty report of a confidence interval.

  Arguments:
    confidence_interval - tuple of (mean, lower bound, upper bound)

  Returns:
    None, but prints to screeen the report
  """

  s = 'Our mean lies in the interval [{:.5}, {:.5}]'.format(
      confidence_interval[1], confidence_interval[2])
  return s
```

```
report_confidence_interval(confidence_interval(coinflips_100))
```

    'Our mean lies in the interval [0.39031, 0.58969]'

```
report_confidence_interval(confidence_interval(coinflips_1000))
```

    'Our mean lies in the interval [0.45597, 0.51803]'

```
stats.t.ppf(0.05, 5) # p-value, dof
```

    -2.0150483726691575

## Assignment - Build a confidence interval

A confidence interval refers to a neighborhood around some point estimate, the size of which is determined by the desired p-value. For instance, we might say that 52% of Americans prefer tacos to burritos, with a 95% confidence interval of +/- 5%.

52% (0.52) is the point estimate, and +/- 5% (the interval $[0.47, 0.57]$) is the confidence interval. "95% confidence" means a p-value $\leq 1 - 0.95 = 0.05$.

In this case, the confidence interval includes $0.5$ - which is the natural null hypothesis (that half of Americans prefer tacos and half burritos, thus there is no clear favorite). So in this case, we could use the confidence interval to report that we've failed to reject the null hypothesis.

But providing the full analysis with a confidence interval, including a graphical representation of it, can be a helpful and powerful way to tell your story. Done well, it is also more intuitive to a layperson than simply saying "fail to reject the null hypothesis" - it shows that in fact the data does *not* give a single clear result (the point estimate) but a whole range of possibilities.

How is a confidence interval built, and how should it be interpreted? It does *not* mean that 95% of the data lies in that interval - instead, the frequentist interpretation is "if we were to repeat this experiment 100 times, we would expect the average result to lie in this interval ~95 times."

For a 95% confidence interval and a normal(-ish) distribution, you can simply remember that +/-2 standard deviations contains 95% of the probability mass, and so the 95% confidence interval based on a given sample is centered at the mean (point estimate) and has a range of +/- 2 (or technically 1.96) standard deviations.

Different distributions/assumptions (90% confidence, 99% confidence) will require different math, but the overall process and interpretation (with a frequentist approach) will be the same.

Your assignment - using the data from the prior module ([congressional voting records](https://archive.ics.uci.edu/ml/datasets/Congressional+Voting+Records)):

1. Generate and numerically represent a confidence interval
2. Graphically (with a plot) represent the confidence interval
3. Interpret the confidence interval - what does it tell you about the data and its distribution?

Stretch goals:

1. Write a summary of your findings, mixing prose and math/code/results. *Note* - yes, this is by definition a political topic. It is challenging but important to keep your writing voice *neutral* and stick to the facts of the data. Data science often involves considering controversial issues, so it's important to be sensitive about them (especially if you want to publish).
2. Apply the techniques you learned today to your project data or other data of your choice, and write/discuss your findings here.
3. Refactor your code so it is elegant, readable, and can be easily run for all issues.

```
# TODO - your code!
# histogram w/ error bars(whiskers)

import pandas as pd
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.stats import ttest_ind, ttest_ind_from_stats, ttest_rel, ttest_1samp

# ttest_ind - 2 function T test
# ttest_1samp - 1 function(sample) T test
# chi-squares - categorical comparison
# t-test - comparing means, samples
```

```
# load and clean data
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/voting-records/house-votes-84.data', header=None)
df.columns = ["party", "handicapped-infants", "water-project-cost-sharing"
              , "adoption-of-the-budget-resolution", "physician-fee-freeze"
              , "el-salvador-aid", "religious-groups-in-schools"
              , "anti-satellite-test-ban", "aid-to-nicaraguan-contras"
              , "mx-missile", "immigration", "synfuels-corporation-cutback"
              , "education-spending", "superfund-right-to-sue", "crime"
              , "duty-free-exports", "export-administration-act-south-africa"
             ]
df = df.replace({'n':0.0,'y':1.0,'?':np.nan})

rep = df[df['party']=='republican']
dem = df[df['party']=='democrat']

print(df.shape)
df.head()
```

    (435, 17)

      party
      handicapped-infants
      water-project-cost-sharing
      adoption-of-the-budget-resolution
      physician-fee-freeze
      el-salvador-aid
      religious-groups-in-schools
      anti-satellite-test-ban
      aid-to-nicaraguan-contras
      mx-missile
      immigration
      synfuels-corporation-cutback
      education-spending
      superfund-right-to-sue
      crime
      duty-free-exports
      export-administration-act-south-africa

      0
      republican
      0.0
      1.0
      0.0
      1.0
      1.0
      1.0
      0.0
      0.0
      0.0
      1.0
      NaN
      1.0
      1.0
      1.0
      0.0
      1.0

      1
      republican
      0.0
      1.0
      0.0
      1.0
      1.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      1.0
      1.0
      0.0
      NaN

      2
      democrat
      NaN
      1.0
      1.0
      NaN
      1.0
      1.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      1.0
      1.0
      0.0
      0.0

      3
      democrat
      0.0
      1.0
      1.0
      0.0
      NaN
      1.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      1.0
      0.0
      0.0
      1.0

      4
      democrat
      1.0
      1.0
      1.0
      0.0
      1.0
      1.0
      0.0
      0.0
      0.0
      0.0
      1.0
      NaN
      1.0
      1.0
      1.0
      1.0

```
rep_plot = rep.drop(['party'], axis=1)
dem_plot = dem.drop(['party'], axis=1)
```

```
sns.set(style="whitegrid")

ax = sns.violinplot(x=rep_plot.values, y=rep_plot.columns, data=rep_plot)
```

```
def confidence_interval(data, confidence=0.95):
  """
  Calculate a confidence interval around a sample mean for given data.
  Using t-distribution and two-tailed test, default 95% confidence.

  Arguments:
    data - iterable (list or numpy array) of sample observations
    confidence - level of confidence for the interval

  Returns:
    tuple of (mean, lower bound, upper bound)
  """

  data = np.array(data)
  mean = np.mean(data)
  n = len(data)
  stderr = stats.sem(data)
  interval = stderr * stats.t.ppf((1 + confidence) / 2.0, n - 1)
  return (mean, mean - interval, mean + interval, interval)

def say_confidence_interval(confidence_interval):
  """
  Return a string with a pretty report of a confidence interval.

  Arguments:
    confidence_interval - tuple of (mean, lower bound, upper bound)

  Returns:
    None, but prints to screeen the report
  """

  s = 'With 95% confidence interval within [{:.4}, {:.4}]'.format(
      confidence_interval[1]*100, confidence_interval[2]*100)
  return s
```

```
df_plot = pd.DataFrame({'party': ['democrat','republican']})
df_plot
```

      party

      0
      democrat

      1
      republican

```
# df_plot = df_plot.drop(['handicapped-infants'], axis=1)
df_plot
```

      party
      handicapped-infants
      water-project-cost-sharing
      adoption-of-the-budget-resolution
      physician-fee-freeze
      el-salvador-aid
      religious-groups-in-schools
      anti-satellite-test-ban
      aid-to-nicaraguan-contras
      mx-missile
      immigration
      synfuels-corporation-cutback
      education-spending
      superfund-right-to-sue
      crime
      duty-free-exports
      export-administration-act-south-africa

      0
      democrat
      60.465116
      50.209205
      88.846154
      5.405405
      21.568627
      47.674419
      77.220077
      82.889734
      75.806452
      47.148289
      50.588235
      14.457831
      28.968254
      35.019455
      63.745020
      93.513514

      1
      republican
      18.787879
      50.675676
      13.414634
      98.787879
      95.151515
      89.759036
      24.074074
      15.286624
      11.515152
      55.757576
      13.207547
      87.096774
      86.075949
      98.136646
      8.974359
      65.753425

```
dem_means, rep_means, dem_moe, rep_moe
```

```
dem_means = []
rep_means = []
dem_moe = []
rep_moe = []

for col in range(len(df.columns)-1):
  issue = df.columns[col+1]
  print('Voting Issue: ' + issue)

  # 2 sample test, removing NaN
  t, p = ttest_ind(dem[issue], rep[issue], nan_policy='omit')

  # converting to array, removing NaN
  dem_array = np.array(dem[issue])[~np.isnan(np.array(dem[issue]))]
  rep_array = np.array(rep[issue])[~np.isnan(np.array(rep[issue]))]

  # evaluating which party supports the issue
  if p < 0.01 and dem[issue].mean() > rep[issue].mean():
    print('Significant Democrat Support: p < 0.01')
  elif p < 0.01 and dem[issue].mean() < rep[issue].mean():
    print('Significant Republican Support: p < 0.01')
  elif p > 0.1:
    print('There may not be much of a difference: p > 0.1')
    print('p-value = %0.3f' % p)
  else:
    print('Not enough certainty level.')
    print('p-value = %0.3f' % p)

  # reporting support %, confidence interval
  d_per = dem[issue].mean()*100
  r_per = rep[issue].mean()*100
  print("Democrat Support: %0.2f%%" % d_per)
  print(say_confidence_interval(confidence_interval(dem_array)))
  print("Republican Support: %0.2f%%" % r_per)
  print(say_confidence_interval(confidence_interval(rep_array)))
  print()

#   df_plot[issue] = dem[issue].mean()*100, rep[issue].mean()*100
  dem_means.append(d_per)
  rep_means.append(r_per)
  dem_moe.append(confidence_interval(dem_array)[3]*100)
  rep_moe.append(confidence_interval(rep_array)[3]*100)
```

    Voting Issue: handicapped-infants
    Significant Democrat Support: p < 0.01
    Democrat Support: 60.47%
    With 95% confidence interval within [54.46, 66.47]
    Republican Support: 18.79%
    With 95% confidence interval within [12.77, 24.81]

    Voting Issue: water-project-cost-sharing
    There may not be much of a difference: p > 0.1
    p-value = 0.929
    Democrat Support: 50.21%
    With 95% confidence interval within [43.82, 56.59]
    Republican Support: 50.68%
    With 95% confidence interval within [42.53, 58.82]

    Voting Issue: adoption-of-the-budget-resolution
    Significant Democrat Support: p < 0.01
    Democrat Support: 88.85%
    With 95% confidence interval within [84.99, 92.7]
    Republican Support: 13.41%
    With 95% confidence interval within [8.144, 18.69]

    Voting Issue: physician-fee-freeze
    Significant Republican Support: p < 0.01
    Democrat Support: 5.41%
    With 95% confidence interval within [2.633, 8.178]
    Republican Support: 98.79%
    With 95% confidence interval within [97.1, 100.5]

    Voting Issue: el-salvador-aid
    Significant Republican Support: p < 0.01
    Democrat Support: 21.57%
    With 95% confidence interval within [16.49, 26.65]
    Republican Support: 95.15%
    With 95% confidence interval within [91.84, 98.46]

    Voting Issue: religious-groups-in-schools
    Significant Republican Support: p < 0.01
    Democrat Support: 47.67%
    With 95% confidence interval within [41.54, 53.81]
    Republican Support: 89.76%
    With 95% confidence interval within [85.1, 94.42]

    Voting Issue: anti-satellite-test-ban
    Significant Democrat Support: p < 0.01
    Democrat Support: 77.22%
    With 95% confidence interval within [72.08, 82.36]
    Republican Support: 24.07%
    With 95% confidence interval within [17.42, 30.73]

    Voting Issue: aid-to-nicaraguan-contras
    Significant Democrat Support: p < 0.01
    Democrat Support: 82.89%
    With 95% confidence interval within [78.31, 87.47]
    Republican Support: 15.29%
    With 95% confidence interval within [9.595, 20.98]

    Voting Issue: mx-missile
    Significant Democrat Support: p < 0.01
    Democrat Support: 75.81%
    With 95% confidence interval within [70.44, 81.17]
    Republican Support: 11.52%
    With 95% confidence interval within [6.593, 16.44]

    Voting Issue: immigration
    Not enough certainty level.
    p-value = 0.083
    Democrat Support: 47.15%
    With 95% confidence interval within [41.08, 53.22]
    Republican Support: 55.76%
    With 95% confidence interval within [48.1, 63.42]

    Voting Issue: synfuels-corporation-cutback
    Significant Democrat Support: p < 0.01
    Democrat Support: 50.59%
    With 95% confidence interval within [44.41, 56.77]
    Republican Support: 13.21%
    With 95% confidence interval within [7.888, 18.53]

    Voting Issue: education-spending
    Significant Republican Support: p < 0.01
    Democrat Support: 14.46%
    With 95% confidence interval within [10.06, 18.86]
    Republican Support: 87.10%
    With 95% confidence interval within [81.76, 92.43]

    Voting Issue: superfund-right-to-sue
    Significant Republican Support: p < 0.01
    Democrat Support: 28.97%
    With 95% confidence interval within [23.33, 34.61]
    Republican Support: 86.08%
    With 95% confidence interval within [80.62, 91.53]

    Voting Issue: crime
    Significant Republican Support: p < 0.01
    Democrat Support: 35.02%
    With 95% confidence interval within [29.15, 40.89]
    Republican Support: 98.14%
    With 95% confidence interval within [96.03, 100.2]

    Voting Issue: duty-free-exports
    Significant Democrat Support: p < 0.01
    Democrat Support: 63.75%
    With 95% confidence interval within [57.76, 69.73]
    Republican Support: 8.97%
    With 95% confidence interval within [4.439, 13.51]

    Voting Issue: export-administration-act-south-africa
    Significant Democrat Support: p < 0.01
    Democrat Support: 93.51%
    With 95% confidence interval within [89.93, 97.1]
    Republican Support: 65.75%
    With 95% confidence interval within [57.96, 73.54]

```
# plot a bar graph
plt.style.use('fivethirtyeight')
fig, ax = plt.subplots(figsize=(8,6))

barWidth = 0.4

r1 = np.arange(len(dem_means))
r2 = [x + barWidth for x in r1]

plt.bar(r1, dem_means, width = barWidth, color = 'blue', edgecolor = 'black'
        , yerr = dem_moe, capsize = 4, label = 'Democrats')
plt.bar(r2, rep_means, width = barWidth, color = 'red', edgecolor = 'black'
        , yerr = rep_moe, capsize = 4, label = 'Republicans')

# title text
plt.title('Support for bills by party')

# modify y labels
ax.set_yticks([0, 25, 50, 75, 100])
ax.set_yticklabels(labels=['0','25','50','75','100%'],
                   fontsize=14, color='#3C3C3C')

# bold y axis at 0
ax.axhline(y=-0.5, color='black', linewidth=1.5, alpha=1)
plt.ylim(-1,105)

ax.legend()

plt.xticks([r + barWidth for r in range(len(dem_means))], rep_plot.columns, rotation = 45, ha="right");
```

```
# plot a bar graph
plt.style.use('fivethirtyeight')
fig, ax = plt.subplots(figsize=(8,6))

barWidth = 0.5

r1 = np.arange(len(dem_means))
r2 = [x + barWidth for x in r1]

ax1= ax.barh(r1, dem_means, height = barWidth, color = 'blue', edgecolor = 'black'
        , xerr = dem_moe, capsize = 3, label = 'Democrats', alpha=.8)
ax2= ax.barh(r2, rep_means, height = barWidth, color = 'red', edgecolor = 'black'
        , xerr = rep_moe, capsize = 3, label = 'Republicans', alpha=.8)

# title text
plt.title('Support for bills by party')

# modify y labels
ax.set_xticks([0, 25, 50, 75, 100])
ax.set_xticklabels(labels=['0','25','50','75','100%'],
                   fontsize=14, color='#3C3C3C')

# bold y axis at 0
ax.axvline(x=0, color='black', linewidth=1.5, alpha=1)
plt.xlim(-2,105)

ax.legend()

plt.yticks([r + barWidth for r in range(len(dem_means))], rep_plot.columns, ha="right");
```

```
df_col = df.columns[1:]
ct = pd.crosstab(df['party'], df.columns[1:])
ct
```

```
df.columns[1:]
```

```
value = df.mean()
# margin of error

plt.bar(range(len(df.columns)), value+np.abs(df.values.min()), bottom=df.values.min(),
        align='center', alpha=0.5)

plt.xticks(range(len(df.columns)), df.columns)
plt.ylabel('Stock price')
plt.title('Something')
plt.show()
```

```
# set style to fivethirtyeight
plt.style.use('fivethirtyeight')

# set plot size
fig, ax = plt.subplots(figsize=(8,6))

ax.bar(x, y, width=0.5, color='b', align='center')
ax.bar(x, z, width=0.5, color='g', align='center')
ax.bar(x, k, width=0.5, color='r', align='center')
```

```
# set style to fivethirtyeight
plt.style.use('fivethirtyeight')

# set plot size
fig, ax = plt.subplots(figsize=(8,6))

# plot category by occupation
# ax = sns.barplot(dem.values, dem.columns, alpha=0.8)
ax = sns.violinplot(x=df_plot.values, y=df_plot.columns, hue="party", data=df_plot)
# ax = sns.violinplot(x="orbital_period", y="method"
#                     , data=planets[planets.orbital_period < 1000]
#                     , scale="width", palette="Set3")

# title text
# ax.text(x=1996.7, y=120, s="Who Got to Be On 'The Daily Show'?", fontsize=20,
#         fontweight='semibold', family='Arial', color='#3C3C3C')

# # subtitle text
# ax.text(x=1996.7, y=114, s='Occupation of guests, by year', fontsize=17,
#         color='#3C3C3C')

# text labels for each line
# ax.text(x=2000.5, y=80, s="Acting, Comedy & Music", fontsize=14,
#         fontweight='semibold', color='#008FD5')
# ax.text(x=2008.3, y=6, s="Government and Politics", fontsize=14,
#         fontweight='semibold', color='#FF2700')
# ax.text(x=2007.4, y=52.5, s="Media", fontsize=14,
#         fontweight='semibold', color='#810F7C')
# plt.text(x = 2000.5, y = 80, s = 'Acting, Comedy & Music', color = '#008FD5',
#            weight = 'semibold', backgroundcolor = '#f0f0f0')
# plt.text(x = 2008.3, y = 6, s = 'Government and Politics', color = '#FF2700',
#            weight = 'semibold', backgroundcolor = '#f0f0f0');
# plt.text(x = 2007.5, y = 52.5, s = 'Media', color = '#810F7C',
#            weight = 'semibold', backgroundcolor = '#f0f0f0')

# set y axis past 0 and 100 for extra grid line
# plt.ylim(-5,110)
# ax.tick_params(axis = 'y', labelsize = 10, length = 8,
#                labelcolor = '#3C3C3C')

# bold y axis at 0
# ax1.axhline(y=0, color='black', linewidth=1.5, alpha=0.5)

# modify y labels
# ax.set_yticks([0, 25, 50, 75, 100])
# ax.set_yticklabels(labels=['0','25','50','75','100%'],
#                    fontsize=14, color='#3C3C3C')

# modify x labels
# ax.set_xticks([2000, 2004, 2008, 2012])
# ax.set_xticklabels(labels=['2000',"'04","'08","'12"],
#                    fontsize=14, color='#3C3C3C')

# signature line at bottom
# ax1.text(x = 1996.5, y = -17,
#     s = '  GEORGE                                       SOURCE: UCI Congressional Voting Records',
#     fontsize = 9, family='monospace', color = '#F0F0F0',
#          backgroundcolor = '#5B5E5F');
```

## Resources

- [Interactive visualize the Chi-Squared test](https://homepage.divms.uiowa.edu/~mbognar/applets/chisq.html)
- [Calculation of Chi-Squared test statistic](https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test)
- [Visualization of a confidence interval generated by R code](https://commons.wikimedia.org/wiki/File:Confidence-interval.svg)
- [Expected value of a squared standard normal](https://math.stackexchange.com/questions/264061/expected-value-calculation-for-squared-normal-distribution) (it's 1 - which is why the expected value of a Chi-Squared with $n$ degrees of freedom is $n$, as it's the sum of $n$ squared standard normals)


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)

# Hide the code completely

# from IPython.display import HTML
# tag = HTML('''<style>
# div.input {
#     display:none;
# }
# </style>''')
# display(tag)
```

Toggle cell visibility here.

```python
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
import sympy as sym
import scipy.signal as signal
from ipywidgets import widgets, interact
```

## PID controller - closed-loop model

A proportional–integral–derivative (PID) control algorithm is by far the most common control algorithm. Its transfer function is equal to:

\begin{equation}
    P(s)=K_p \cdot \left( 1 + \frac{1}{T_i s} + T_d s \right).
\end{equation}

It is made as a sum of proportional, integral and derivative channels. Not all of them have to be present, so PI or PD control algorithms are also used. In this example the response of a P, PI, PD or PID controller is shown for unit step,  unit impulse, unit ramp or sine input. The controller is in this case part of a feedback control system. The object can either be a proportional of the zeroth, first or second order, or an integral of zeroth or first order.

The plots below show:
1. The response of the closed-loop system for the selected input, object type and the selected controller (left figure).
2. The position of the zeros and poles of the transfer function of the resulting closed-loop system.

---

### How to use this notebook?
1. Toggle between *unit step function*, *unit impulse function*, *unit ramp function*, and *sine* to select the input signal.
2. Click on *P0*, *P1*, *I0* or *I1* to toggle between the following objects: proportional of the zeroth, first or second order, or an integral one of zeroth or first order. The transfer function of P0 object is $k_p$ (in this example $k_p=2$), of PI object $\frac{k_p}{\tau s+1}$ (in this example $k_p=1$ and $\tau=2$), of IO object $\frac{k_i}{s}$ (in this example $k_i=\frac{1}{10}$) and of I1 object $\frac{k_i}{s(\tau s +1) }$ (in this example $k_i=1$ and $\tau=10$).
3. Click on the *P*, *PI*, *PD* and *PID* buttons to toogle between proportional, proportional-integral, proportional-derivative and proportional–integral–derivative control algorithm type.
4. Move the sliders to change the values of proportional ($K_p$), integral ($T_i$) and derivative ($T_d$) PID tunning coefficients.
5. Move the slider $t_{max}$ to change the maximum value of the time on x axis of the Time response plot.

```python
A = 10
a=0.1
s, P, I, D = sym.symbols('s, P, I, D')

obj = 1/(A*s)
PID = P + P/(I*s) + P*D*s#/(a*D*s+1)
system = obj*PID/(1+obj*PID)
num = [sym.fraction(system.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system.factor())[0], gen=s)))]
den = [sym.fraction(system.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system.factor())[1], gen=s)))]

# make figure
fig = plt.figure(figsize=(9.8, 4),num='PID controller - closed-loop system')
plt.subplots_adjust(wspace=0.3)

# add axes
ax = fig.add_subplot(121)
ax.grid(which='both', axis='both', color='lightgray')
ax.set_title('Time response')
ax.set_xlabel('$t$ [s]')
ax.set_ylabel('input, output')
ax.axhline(linewidth=.5, color='k')
ax.axvline(linewidth=.5, color='k')

rlocus = fig.add_subplot(122)

input_type = 'unit step function'

# plot step function and responses (initalisation)
input_plot, = ax.plot([],[],'C0', lw=1, label='input')
response_plot, = ax.plot([],[], 'C1', lw=2, label='output')
ax.legend()

rlocus_plot, = rlocus.plot([], [], 'r')

plt.show()

def update_plot(KP, TI, TD, Time_span):
    global num, den, input_type

    num_temp = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in num]
    den_temp = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in den]
    system = signal.TransferFunction(num_temp, den_temp)
    zeros = np.roots(num_temp)
    poles = np.roots(den_temp)

    rlocus.clear()
    rlocus.scatter([np.real(i) for i in poles], [np.imag(i) for i in poles], marker='x', color='g', label='pole')
    rlocus.scatter([np.real(i) for i in zeros], [np.imag(i) for i in zeros], marker='o', color='g', label='zero')
    rlocus.set_title('Pole-zero plot')
    rlocus.set_xlabel('Re')
    rlocus.set_ylabel('Im')
    rlocus.grid(which='both', axis='both', color='lightgray')

    time = np.linspace(0, Time_span, 300)

    if input_type == 'unit step function':
        u = np.ones_like(time)
        u[0] = 0
        time, response = signal.step(system, T=time)
    elif input_type == 'unit impulse function':
        u = np.zeros_like(time)
        u[0] = 10
        time, response = signal.impulse(system, T=time)
    elif input_type == 'sine function':
        u = np.sin(time*2*np.pi)
        time, response, _ = signal.lsim(system, U=u, T=time)
    elif input_type == 'unit ramp function':
        u = time
        time, response, _ = signal.lsim(system, U=u, T=time)
    else:
        raise Exception("Error in the program. Please restart simulation.")

    response_plot.set_data(time, response)
    input_plot.set_data(time, u)

    rlocus.axhline(linewidth=.3, color='k')
    rlocus.axvline(linewidth=.3, color='k')
    rlocus.legend()

    ax.set_ylim([min([np.min(u), min(response),-.1]),min(100,max([max(response)*1.05, 1, 1.05*np.max(u)]))])
    ax.set_xlim([-0.1,max(time)])

    plt.show()

controller_ = PID
object_ = obj

def calc_tf():
    global num, den, controller_, object_
    system_func = object_*controller_/(1+object_*controller_)

    num = [sym.fraction(system_func.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func.factor())[0], gen=s)))]
    den = [sym.fraction(system_func.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func.factor())[1], gen=s)))]
    update_plot(Kp_widget.value, Ti_widget.value, Td_widget.value, time_span_widget.value)

def transfer_func(controller_type):
    global controller_
    proportional = P
    integral = P/(I*s)
    differential = P*D*s/(a*D*s+1)
    if controller_type =='P':
        controller_func = proportional
        Kp_widget.disabled=False
        Ti_widget.disabled=True
        Td_widget.disabled=True
    elif controller_type =='PI':
        controller_func = proportional+integral
        Kp_widget.disabled=False
        Ti_widget.disabled=False
        Td_widget.disabled=True
    elif controller_type == 'PD':
        controller_func = proportional+differential
        Kp_widget.disabled=False
        Ti_widget.disabled=True
        Td_widget.disabled=False
    else:
        controller_func = proportional+integral+differential
        Kp_widget.disabled=False
        Ti_widget.disabled=False
        Td_widget.disabled=False

    controller_ = controller_func
    calc_tf()

def transfer_func_obj(object_type):
    global object_
    if object_type == 'P0':
        object_ = 2
    elif object_type == 'P1':
        object_ = 1/(2*s+1)
    elif object_type == 'I0':
        object_ = 1/(10*s)
    elif object_type == 'I1':
        object_ = 1/(s*(10*s+1))
    calc_tf()

style = {'description_width': 'initial'}

def buttons_controller_clicked(event):
    controller = buttons_controller.options[buttons_controller.index]
    transfer_func(controller)
buttons_controller = widgets.ToggleButtons(
    options=['P', 'PI', 'PD', 'PID'],
    description='Select control algorithm type:',
    disabled=False,
    style=style)
buttons_controller.observe(buttons_controller_clicked)

def buttons_object_clicked(event):
    object_ = buttons_object.options[buttons_object.index]
    transfer_func_obj(object_)
buttons_object = widgets.ToggleButtons(
    options=['P0', 'P1', 'I0', 'I1'],
    description='Select object:',
    disabled=False,
    style=style)
buttons_object.observe(buttons_object_clicked)

def buttons_input_clicked(event):

    global input_type
    input_type = buttons_input.options[buttons_input.index]
    update_plot(Kp_widget.value, Ti_widget.value, Td_widget.value, time_span_widget.value)
buttons_input = widgets.ToggleButtons(
    options=['unit step function','unit impulse function', 'unit ramp function', 'sine function'],
    description='Select input signal:',
    disabled=False,
    style=style)
buttons_input.observe(buttons_input_clicked)

Kp_widget = widgets.IntSlider(value=10,min=1,max=50,step=1,description=r'\(K_p\)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.1d')
Ti_widget = widgets.FloatLogSlider(value=1.,min=-3,max=1.1,step=.001,description=r'\(T_{i} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')
Td_widget = widgets.FloatLogSlider(value=1.,min=-3,max=1.1,step=.001,description=r'\(T_{d} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')

time_span_widget = widgets.FloatSlider(value=10.,min=.5,max=50.,step=0.1,description=r'\(t_{max} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.1f')

transfer_func(buttons_controller.options[buttons_controller.index])
transfer_func_obj(buttons_object.options[buttons_object.index])

display(buttons_input)
display(buttons_object)
display(buttons_controller)

interact(update_plot, KP=Kp_widget, TI=Ti_widget, TD=Td_widget, Time_span=time_span_widget);
```

    <IPython.core.display.Javascript object>

    ToggleButtons(description='Select input signal:', options=('unit step function', 'unit impulse function', 'uni…

    ToggleButtons(description='Select object:', options=('P0', 'P1', 'I0', 'I1'), style=ToggleButtonsStyle(descrip…

    ToggleButtons(description='Select control algorithm type:', options=('P', 'PI', 'PD', 'PID'), style=ToggleButt…

    interactive(children=(IntSlider(value=10, description='\\(K_p\\)', max=50, min=1, readout_format='.1d'), Float…