# Encoding Schemes

This is not a computer science class, but in your practical work, you will frequently have to deal with text that is encoded in a variety of styles. Understanding the difference between them is key.

### Implications for Data Science

Reasons I've encountered in my own work with NLP why it is beneficial to understand text encoding schemes:
* If you are not using the right encoding, you cannot perform adequate feature engineering
* Some data-scientists have simply "thrown away" samples of tweets, social media comments that seem "mal-formed" but are actually simply just using a different encoding scheme

## World Languages, in Context

## Bits and Bytes

- Computers, at its lowest level, store everything in the form of bits (either a 0 or a 1). The amount of information that can be represented in a computer is determined by the number of bits.

For instance, a using only 4 bits, you can store **$2^4$ = 16** different values.

How $101010$ is converted to decimal (human-readable numbers): each of the green numbers is summed up to equal 42.

It is not physically efficient for a computer to try to read one bit at a time, so typically data is stored in **8-bit** groups called **bytes**.

### Exercises
1. How many bits (0s and 1s) does it take to represent **4 different characters**?

\begin{equation}
2^{n} = 4
\end{equation}

\begin{equation}
log(2^{n}) = log(4)
\end{equation}

\begin{equation}
n \cdot log(2) = log(4)
\end{equation}

\begin{equation}
n = \frac{log(4)}{log(2)}
\end{equation}

\begin{equation}
n = \frac{log(4)}{log(2)}
\end{equation}

\begin{equation}
n = 2
\end{equation}

It takes **2** bits to represent **4** distinct characters.

2. How many bits (0s and 1s) does it take to represent **128 different characters**?

\begin{equation}
2^{n} = 128
\end{equation}

\begin{equation}
log(2^{n}) = log(128)
\end{equation}

\begin{equation}
n \cdot log(2) = log(128)
\end{equation}

\begin{equation}
n = \frac{log(128)}{log(2)}
\end{equation}

\begin{equation}
n = \frac{log(128)}{log(2)}
\end{equation}

\begin{equation}
n = 7
\end{equation}

It takes **7** bits to represent **128** distinct characters.

## ASCII

The oldest, yet still relevant encoding style to be aware of is **ASCII**, where computers represent text (**every character on a keyboard**) initially as a number between 0 and 127 (question: how many bits does it take to do this?)

ASCII table converting numbers to characters.(Wikipedia)

*If the smallest amount of data a computer can realistically read in is a byte (**8-bits**), why is ASCII only **7-bits**?* The last bit was a **parity bit** is used for **error checking** - to ensure that the data wasn't corrected or unintentionally altered.

### Encoding/Decoding Words

How would you write the word `Data` using ASCII encoding?

#### Steps:

1. Look up the "codepoint" for the first character (`D`).

When you look up the character map value for `D`, its corresponding codepoint is `68`. Note that this is a different codepoint than lowercase `d`.

2. Write out that number in binary.

The ASCII binary encoding for `D` is `1000100`.

3. Repeat for `a`, `t`, and `a`.

Use [this website to check your answer](https://www.rapidtables.com/convert/number/ascii-to-binary.html).

### Python Code

Do not worry about understanding what is happening inside the `get_binary_for_char` and `get_binary` functions. Just know that they take in a string and produce the 0s and 1s that the string is encoded in:

```python
# to find out your computer's default encoding system
import sys
sys.getdefaultencoding()
```

```python
import re
from typing import List

def get_binary_for_char(char: str, encoding="utf-8") -> str:
    """
    Encodes a character using the desired encoding into its corresponding hex, then converts the
    hex code into binary, formatted with tab spaces between byte marks.
    """

    hex_code = char.encode(encoding).hex()
    code_point = hex(ord(char))[2:].upper()

    binary: str = f"{int(hex_code, 16):08b}"

    byte_list: List[str] = re.findall('[01]{8}', binary)
    formatted_binary: str = "\t".join(byte_list)  # for variable length encoding, tab space between byte marks.
    print(f"{char} (U+{code_point.zfill(4)}, hex:{hex_code}) - {encoding}: {formatted_binary}")
    return formatted_binary

def get_binary(text: str, encoding="utf-8"):
    return " ".join([get_binary_for_char(char, encoding) for char in text])

get_binary("Data")
```

    D (U+0044, hex:44) - utf-8: 01000100
    a (U+0061, hex:61) - utf-8: 01100001
    t (U+0074, hex:74) - utf-8: 01110100
    a (U+0061, hex:61) - utf-8: 01100001

    '01000100 01100001 01110100 01100001'

```python
get_binary("Data", encoding="ascii")
```

### Extended ASCII

The dominant language in earlier eras of computing was English. People began to realize that ASCII was relatively limited, and even other European languages could not be properly supported. At the same time, transmission technology evolved to a standard of reliability such that the parity bit used for checking for errors was no longer needed.

As a result, people began using the last (eighth) bit to extend the number of characters represented by ASCII from 128 characters to 256 characters.

#### Latin-1

Character map [available here](https://www.htmlhelp.com/reference/charset/latin1.gif).

Characters such as `Ç` (pronounced `ch` in Turkish, for instance), is represented by the number `199`. The Spanish word `año` (year) includes a character `ñ` that would be represented by the code point `241`.

#### Excel on Macs

Macs commonly use [Mac OS Roman encoding](https://en.wikipedia.org/wiki/Mac_OS_Roman).

```python
get_binary("cat", encoding="latin1")
```

```python
get_binary("¿Cuántas?", encoding="ascii")
get_binary("¿Cuántas?", encoding="latin1")
```

### Exercises

#### Encode the word **`más`** in ASCII.

Find the codepoints for each of the characters:

* `m` $\rightarrow$ 109
* `á` $\rightarrow$ **None**
* `s` $\rightarrow$ 115

It is impossible to correctly encode `más` in ASCII because there is no corresponding code point for `á`.

#### Encode the word **`más`** in **`latin1`**.

Find the codepoints for each of the characters:

* `m` $\rightarrow$ 109
* `á` $\rightarrow$ 225
* `s` $\rightarrow$ 115

Next, encode each of the codepoints into binary:

* `m` $\rightarrow$ 109: `01101101`
* `á` $\rightarrow$ 225: `11100001`
* `s` $\rightarrow$ 115: `01110011`

The `latin1` encoding for `más` is `01101101 11100001 01110011`.

```python
# using Python to check
get_binary("más", encoding="latin1")
```

    m (U+006D, hex:6d) - latin1: 01101101
    á (U+00E1, hex:e1) - latin1: 11100001
    s (U+0073, hex:73) - latin1: 01110011

    '01101101 11100001 01110011'

#### Decode the binary stream **`01100011 01100001 01110100`**. Assume that it is using **`latin1`** encoding.

* `01100011` $\rightarrow$ 99 $\rightarrow$ `c`
* `01100001` $\rightarrow$ 97 $\rightarrow$ `a`
* `01110100` $\rightarrow$ 116 $\rightarrow$ `t`

The binary stream `01100011 01100001 01110100`, decoded using **`latin1`**, represents the string `cat`.

## Unicode

128 characters is not enough to represent the characters in other Languages, like **Greek, Turkish, Cyrillic**, etc., or newer social media phenomenons like **emojis**. Unicode stores text as either 8, 16, or 32 bits (1, 2, or 4 bytes). This means there's significantly more characters that can be encoded (approximately 1 billion characters).

As a point of reference, there's a total of **50,000** characters in the Chinese language (but only around **15-20,000** that are used commonly).

If you don't specify the right encoding to read in text, you'll end up with something like this:

Malformed characters because of incorrect encoding.(Wikipedia)

### UTF-8

The default encoding scheme of the internet today is `UTF-8`.

Share of web pages with different encodings.(Wikipedia)

There is another encoding schema very similar to `UTF-8` called `UTF-16`. You'll typically find it being used on Windows systems and within Java applications.

### Variable Length Encoding /Digitalization and Internationalization

UTF-8 is the default encoding schema of the internet. Whenever you save files to disk, or read files in, your first choice should be to try using UTF-8. UTF-8 is an example of **variable-length encoding**. This means sometimes a character will take 8 bits to encode (represent), sometimes 16 bits, sometimes 24 bits, and sometimes 32 bits.

On the other hand, another encoding scheme is `UTF-32`- it always takes **32 bits**.

#### Data Science Implications
UTF-8 should be your default encoding of choice when working with Big Data. Because the # of bits it takes to encode a character changes, it can be more "storage-efficient" on disk, and more "memory-efficient" when representing this text in memory.

Many machine-learning algorithms (like **batch and mini-batch gradient descent**) will perform updates using batches of samples. If you choose the wrong encoding, you will not be able to fit as many samples into your batch for training as you'd like - this means your model may require significantly more training time and perform worse.

```python
#get_binary("I 😍 DSO 560", encoding="ascii")
#get_binary("I 😍 DSO 560", encoding="latin1")
get_binary("I 😍 DSO 560", encoding="utf8")
```

week2/Text Encoding Schemes (Completed).ipynb


# Chap2: An Asset Selling Problem

The chapter to answer basic question: **When to sell an asset**

In this case, we use a simple class of policies known as:  *policy function approximations*.

**Mathematical Model**

* **State Variable**: $S_t$ that captures all the information we need ...($R_t$, $I_t$, $B_t$)
* **Decsion Variable** : $x_t$ where $x_0$ is the design decision, while $x_t$ for t>0 represents control varilables.
* **Exogenous Information**: The variable $W_t$ captures information that first becomes available between $t-1$ and $t$ from outside of our system.
* **Transition Fucntion**:
$$
S_{t+1} = S^M(S_t, x_t,W_{t+1})
$$
* **Objective Function**: When there are elements of the *problem* that depend on the state variables, we represent costs/contribution as:

$$
C(S_{t},x_{t}) \ or \ C(S_t,x_t, W_{t+1})
$$

**Set of learning problems** : where we are trying to learn about a function which is not itself a function of state variable, it means:

This means the state variable consists purely of belief state $B_t$.

## Narrative

We are holding a block of shares of stock, looking for oppurtunity to sell.If we sell at time t, we receive a price that varies according to some random process over time. Once we sell the stock, the process stops.

## Basic Model

### State Variable:

Two state variables:
* The **Physical State** which says whether or not we are still holding the assest
* The **Information State** the price of stock

* The physical State:

$$
  R_t=\left\{
  \begin{array}{@{}ll@{}}
    1, & \text{if we are holding the stock at time t}\  \\
    0, & \text{if we are not longer holding the stock at time t}
  \end{array}\right.
$$

If we sell the stock, we receive the price per share $p_t$. This means our state variable is:
$$
S_t=(R_t, p_t)
$$

### Decision Variable:

$$
\begin{equation}
  x_t=\left\{
  \begin{array}{@{}ll@{}}
    1, & \text{if we sell the stock at time t}\  \\
    0, & \text{if we do not sell the stock at time t}
  \end{array}\right.
\end{equation}
$$
we only can sell stock if we hold it,
$$
x_t < R_t
$$

#### Policy:

What is the policy:
 **which is going to define how we make decision**

$$
X^{\pi}(S_t)
$$

Example policy is we might be to sell if the price drops the below some limit point, thus we could write:

$$
\begin{equation}
  X^{sell-low}(S_t|\theta^{low})=\left\{
  \begin{array}{@{}ll@{}}
    1, & \text{if}\ p_t<\theta^{low}  \ \text{and}\ R_t=1  \\
    1, & \text{if}\ t= T\ \text{and} \ R_t = 1 \\
    0, & \text{otherwise}
  \end{array}\right.
\end{equation}
$$

### Exogenous Function

The only random process in our basic model is the change in price. Alternatively, naturally we simply write $W_t$ be the new price, in which we would write:

$$
W_{t+1} = (p_{t+1})
$$

### Transition Function

Is the equation that describe how the states evolves. The transition equation is:

$$
R_{t+1} = R_{t} - x_{t}
$$

How the price evolves over time.

$$p_{t+1} = p_t  + \hat{p}_{t+1}$$

**Transition Function**

$$
S_{t+1} = S^M(S_t, X^{\pi}(S_t), W_{t+1})
$$

If we use our policy $X^{\pi}(S_t)$, and if we choose sample path $\omega$, that determine the sequence $W_1,W_2,W_3,...,W_T$...than the simulation of process can be written as:

$$
(S_0,x_0 = X^{\pi}(S_0)), W_1(\omega), S_1, x_1 = X^{\pi}(S_1), W_(\omega),..., x_{T-1}, W_T(\omega),S_T)
$$

### Objective Function
The performance metric is how much we earn from selling our stock.

$$
C(S_t,x_t) = p_tx_t
$$

We now can formulate the optimizatio problem:

$$
max \ \sum_{t=0}^{T-1} p_tx_t \\
x_0,x_1,...,x_{T-1}
$$

Constraints of the Optimization:

$$
\sum_{t=0}^{T-1} x_t = 1, \\
x_t \leq 1, \\
x_t \geq 0,
$$

**Including Uncertainty**:

We are simulating a policy following a sample path $\omega$ of price $p_1(\omega), p_2(\omega),...$

$$
S_{t+1}(\omega) = S^{M}(S_t(\omega), X^{\pi}(S_t(\omega)), W_{t+1}(\omega))
$$

If we follow policy $\pi$ along this sample path, we can compute the performance:

$$
\hat{F}^{\pi}(\omega) = \sum_{t=0}^{T-1} p_t(\omega)X^{\pi}(S_t(\omega))
$$

This is for one sample path . We can simulate over sample of $N$ sampels ($\omega^{1},...,\omega^n,...\omega^N$), and take average:

$$
\overline{F}^{\pi} = \frac{1}{N} \sum_{n=1}^{N}\hat{F}^{\pi}(\omega^n)
$$

Finally, we write out optimization problem in terms of finding the best policy, which we can write:

$$
max \ \ \overline{F}^{\pi} \\
x \in \Pi
$$

## Designing Policy

### Sell_low Policy:
In this policy we *sell* at that the price lower than, $\theta^{sell-low}$.

$$
  X^{sell-low}(S_t|\theta^{low}) =\left\{
  \begin{array}{@{}ll@{}}
    1  & if \ p_t < \theta^{low} \ and \ R_t=1  \\
    1, & t=T \ R_t=1 \\
    0, & otherwise
  \end{array}\right.
$$

### High_Low policy:

In this policy , we will sell the asset if the price jumps *too high** or ** too low**:

$$
  X^{high-low}(S_t|\theta^{high-low}) =\left\{
  \begin{array}{@{}ll@{}}
    1  & if \ p_t < \theta^{low} \ or \ p_t > \theta^{high}   \\
    1, & t=T \ and \ R_t=1 \\
    0, & otherwise
  \end{array}\right.
$$

### Track Policy:

Perhaps we just want to sell when the stock rises above a tracking signal. Tod this, first create a smoothed estimate of price:

$$\overline{p}_t = (1- \alpha)\overline{p}_{t-1} + \alpha \hat{p}_t$$

Now consider a tracking policy that we might write:

$$
  X^{track}(S_t | \theta^{track}) =\left\{
  \begin{array}{@{}ll@{}}
    1  & if \ p_t > \overline{p}_t + \theta^{track} \    \\
    1, & t=T \ and \ R_t=1 \\
    0, & otherwise
  \end{array}\right.
$$

For this policy, we are going to need to tweak our model, because we now $\overline{p}_{t}$ in order to make decision. This means we would now write our state as:

$$
S_t = (R_t, p_t, \overline{p}_t)
$$

**Classes of Policies**:

$F = \{"sell-low","high-low","track"\}$

**Parameter for each policy**:

$\theta^f \in F$

**Search over policies**:

$\pi \in \Pi$

## Coding Part:

### Import libraries

```python
import pandas as pd
import numpy as np
from collections import namedtuple
import matplotlib.pyplot as plt
from copy import copy
import math
import seaborn as sns
sns.set_theme(style="darkgrid")
sns.set(rc={'figure.figsize':(12,8)})
```

### Change the directory to to get module

```python
import os
os.chdir("/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/function")
os.getcwd()
```

    '/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/function'

### Import modules from function

```python
from AssetSellingModel import AssetSellingModel
from AssetSellingPolicy import AssetSellingPolicy
```

### Change the directory to to get parameters

```python
import os
os.chdir("/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/data")
os.getcwd()
```

    '/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/data'

### Policy Parameters:

```python
sheet1= pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet1", usecols=["policy", "param1","param2"])
```

```python
sheet1
```

      policy
      param1
      param2

      0
      sell_low
      2
      NaN

      1
      high_low
      4
      10.0

      2
      track
      0
      4.0

```python
params = zip(sheet1["param1"], sheet1["param2"])
param_list = list(params)
param_list
```

    [(2, nan), (4, 10.0), (0, 4.0)]

### Full grid Policy Parameters:

```python
sheet2 = pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet2")
```

```python
sheet2.dtypes
```

    low_min             int64
    low_max             int64
    high_min            int64
    high_max            int64
    increment_size    float64
    dtype: object

```python
dtype(sheet2)
```

### Parameters of the Dynamic Model:

```python
sheet3 = pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet3")
sheet3
```

      Policy
      TimeHorizon
      DiscountFactor
      InitialPrice
      InitialBias
      UpStep
      DownStep
      Variance
      Iterations
      PrintStep

      0
      track
      40
      0.99
      16
      Up
      1
      -1
      2
      10
      40

### Bias Term:

```python
biasdf = pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet4")
biasdf_edit=biasdf.set_index('additional')
biasdf_edit
```

      Up
      Neutral
      Down

      additional

      Up
      0.9
      0.1
      0.0

      Neutral
      0.2
      0.6
      0.2

      Down
      0.0
      0.1
      0.9

### Track Policy:

```python
policy_selected = sheet3['Policy'][0]
print("The selected policy is: {}".format(policy_selected))

T = sheet3['TimeHorizon'][0]
print("The T value is: {}".format(T))

initPrice = sheet3['InitialPrice'][0]
print("The initial price is: {}".format(initPrice))

initBias = sheet3['InitialBias'][0]
print("The initial Bias is: {}".format(initBias))

exog_params = {'UpStep':sheet3['UpStep'][0],'DownStep':sheet3['DownStep'][0],'Variance':sheet3['Variance'][0],'biasdf'
               :biasdf_edit}
print("The exog_params is: {}".format(exog_params))

nIterations = sheet3['Iterations'][0]
print("The nIteration is: {}".format(nIterations))

printStep = sheet3['PrintStep'][0]
print("The printStep is: {}".format(printStep))

printIterations = [0]
print("The printIterations is: {}".format(printIterations))

printIterations.extend(list(reversed(range(nIterations-1,0,-printStep))))
```

    The selected policy is: track
    The T value is: 40
    The initial price is: 16
    The initial Bias is: Up
    The exog_params is: {'UpStep': 1, 'DownStep': -1, 'Variance': 2, 'biasdf':              Up  Neutral  Down
    additional
    Up          0.9      0.1   0.0
    Neutral     0.2      0.6   0.2
    Down        0.0      0.1   0.9}
    The nIteration is: 10
    The printStep is: 40
    The printIterations is: [0]

```python
policy_names = ['sell_low', 'high_low', 'track']
print("policy_names are: {}".format(policy_names))

state_names = ['price', 'resource','bias']
print("state_names are: {}".format(state_names))

init_state = {'price': initPrice, 'resource': 1,'bias':initBias}
print("init_state are: {}".format(init_state))

decision_names = ['sell', 'hold']
print("decision_names are: {}".format(decision_names))

```

    policy_names are: ['sell_low', 'high_low', 'track']
    state_names are: ['price', 'resource', 'bias']
    init_state are: {'price': 16, 'resource': 1, 'bias': 'Up'}
    decision_names are: ['sell', 'hold']

```python
biasdf = exog_params["biasdf"]
print("the biasdf_edit is \n .{}".format(biasdf))
print(type(biasdf))
biasdf= biasdf.cumsum(axis =1)
print("the biasdf_edit is \n .{}".format(biasdf))

```

    the biasdf_edit is
     .             Up  Neutral  Down
    additional
    Up          0.9      0.1   0.0
    Neutral     0.2      0.6   0.2
    Down        0.0      0.1   0.9
    <class 'pandas.core.frame.DataFrame'>
    the biasdf_edit is
     .             Up  Neutral  Down
    additional
    Up          0.9      1.0   1.0
    Neutral     0.2      0.8   1.0
    Down        0.0      0.1   1.0

```python
M = AssetSellingModel(state_names, decision_names, init_state,exog_params,T)
```

```python
P = AssetSellingPolicy(M, policy_names)
```

### Policy Evaluation

```python
t = 0
prev_price = init_state['price']
```

```python
policy_info = {'sell_low': param_list[0],
                   'high_low': param_list[1],
                   'track': param_list[2] + (prev_price,)}
```

```python
policy_info
```

    {'sell_low': (2, nan), 'high_low': (4, 10.0), 'track': (0, 4.0, 16)}

```python
policy_selected
```

    'track'

```python
#if (not policy_selected =='full_grid'):
print("Selected policy {}, time horizon {}, initial price {} and number of iterations {}".format(policy_selected,T,initPrice,nIterations))
contribution_iterations=[P.run_policy(param_list, policy_info, policy_selected, t) for ite in list(range(nIterations))]

contribution_iterations = pd.Series(contribution_iterations)
print("Contribution per iteration: ")
print(contribution_iterations)
cum_avg_contrib = contribution_iterations.expanding().mean()
print("Cumulative average contribution per iteration: ")
print(cum_avg_contrib)

#plotting the results

#fig, axsubs = plt.subplots(1,2,sharex=True,sharey=True)
#fig.suptitle("Asset selling using policy {} with parameters {} and T {}".format(policy_selected,policy_info[policy_selected],T) )
#i = np.arange(0, nIterations, 1)

#axsubs[0].plot(i, cum_avg_contrib, 'g')
#axsubs[0].set_title('Cumulative average contribution')

#axsubs[1].plot(i, contribution_iterations, 'g')
#axsubs[1].set_title('Contribution per iteration')

# Create a big subplot
#ax = fig.add_subplot(111, frameon=False)
# hide tick and tick label of the big axes
#plt.tick_params(labelcolor='none', top=False, bottom=False, left=False, right=False)

#ax.set_ylabel('USD', labelpad=0) # Use argument `labelpad` to move label downwards.
#ax.set_xlabel('Iterations', labelpad=10)

#plt.show()

```

    Selected policy track, time horizon 40, initial price 16 and number of iterations 10
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.4139840589481404  curr_bias  Up  new_bias  Up
    smoothed_price = 19.275851851363697
    time=1, obj=0.0, s.resource=1, s.price=16.818962962840924, x=Decision(sell=0, hold=1)
    coin  0.8487800961190239  curr_bias  Up  new_bias  Up
    smoothed_price = 12.919965111987779
    time=2, obj=0.0, s.resource=1, s.price=15.844213500127637, x=Decision(sell=1, hold=0)
    coin  0.45203620488154816  curr_bias  Up  new_bias  Up
    obj=15.844213500127637, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.2720800519673423  curr_bias  Up  new_bias  Up
    smoothed_price = 29.300677434463253
    time=1, obj=0.0, s.resource=1, s.price=19.325169358615813, x=Decision(sell=0, hold=1)
    coin  0.905534061916374  curr_bias  Up  new_bias  Neutral
    smoothed_price = 11.678950072957633
    time=2, obj=0.0, s.resource=1, s.price=17.41361453720127, x=Decision(sell=1, hold=0)
    coin  0.3755268704762992  curr_bias  Neutral  new_bias  Neutral
    obj=17.41361453720127, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.09168318648059992  curr_bias  Up  new_bias  Up
    smoothed_price = 30.17968207409804
    time=1, obj=0.0, s.resource=1, s.price=19.54492051852451, x=Decision(sell=0, hold=1)
    coin  0.9746533545172668  curr_bias  Up  new_bias  Neutral
    smoothed_price = 9.10715798582386
    time=2, obj=0.0, s.resource=1, s.price=16.93547988534935, x=Decision(sell=1, hold=0)
    coin  0.845463715621411  curr_bias  Neutral  new_bias  Down
    obj=16.93547988534935, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.7718855570743978  curr_bias  Up  new_bias  Up
    smoothed_price = 13.590645854568095
    time=1, obj=0.0, s.resource=1, s.price=15.397661463642024, x=Decision(sell=1, hold=0)
    coin  0.058599417274914245  curr_bias  Up  new_bias  Up
    obj=15.397661463642024, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.5630457710972723  curr_bias  Up  new_bias  Up
    smoothed_price = 14.404641398215574
    time=1, obj=0.0, s.resource=1, s.price=15.601160349553894, x=Decision(sell=1, hold=0)
    coin  0.6337684851009756  curr_bias  Up  new_bias  Up
    obj=15.601160349553894, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.32637382885748367  curr_bias  Up  new_bias  Up
    smoothed_price = 29.23727426734112
    time=1, obj=0.0, s.resource=1, s.price=19.30931856683528, x=Decision(sell=0, hold=1)
    coin  0.010789716774533442  curr_bias  Up  new_bias  Up
    smoothed_price = 24.356969142318995
    time=2, obj=0.0, s.resource=1, s.price=20.57123121070621, x=Decision(sell=0, hold=1)
    coin  0.10655404328873319  curr_bias  Up  new_bias  Up
    smoothed_price = 32.917325150498336
    time=3, obj=0.0, s.resource=1, s.price=23.65775469565424, x=Decision(sell=0, hold=1)
    coin  0.2744609047476718  curr_bias  Up  new_bias  Up
    smoothed_price = 20.209678891873452
    time=4, obj=0.0, s.resource=1, s.price=22.795735744709045, x=Decision(sell=1, hold=0)
    coin  0.7876352836985036  curr_bias  Up  new_bias  Up
    obj=22.795735744709045, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.5732916894272274  curr_bias  Up  new_bias  Up
    smoothed_price = 27.318028871373315
    time=1, obj=0.0, s.resource=1, s.price=18.82950721784333, x=Decision(sell=0, hold=1)
    coin  0.25230738226248894  curr_bias  Up  new_bias  Up
    smoothed_price = 32.205300235020715
    time=2, obj=0.0, s.resource=1, s.price=22.173455472137675, x=Decision(sell=0, hold=1)
    coin  0.34703831077049807  curr_bias  Up  new_bias  Up
    smoothed_price = 22.184929865392462
    time=3, obj=0.0, s.resource=1, s.price=22.176324070451372, x=Decision(sell=0, hold=1)
    coin  0.020311694923608736  curr_bias  Up  new_bias  Up
    smoothed_price = 36.75498055259952
    time=4, obj=0.0, s.resource=1, s.price=25.82098819098841, x=Decision(sell=0, hold=1)
    coin  0.5773741485181105  curr_bias  Up  new_bias  Up
    smoothed_price = 19.512647218984497
    time=5, obj=0.0, s.resource=1, s.price=24.243902947987433, x=Decision(sell=1, hold=0)
    coin  0.9296106478220897  curr_bias  Up  new_bias  Neutral
    obj=24.243902947987433, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.4326408307147147  curr_bias  Up  new_bias  Up
    smoothed_price = 29.78638009471284
    time=1, obj=0.0, s.resource=1, s.price=19.44659502367821, x=Decision(sell=0, hold=1)
    coin  0.9844689697230611  curr_bias  Up  new_bias  Neutral
    smoothed_price = 9.047636235056132
    time=2, obj=0.0, s.resource=1, s.price=16.84685532652269, x=Decision(sell=1, hold=0)
    coin  0.8262653217404742  curr_bias  Neutral  new_bias  Down
    obj=16.84685532652269, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.4433318705856356  curr_bias  Up  new_bias  Up
    smoothed_price = 4.415644002958089
    time=1, obj=0.0, s.resource=1, s.price=13.103911000739522, x=Decision(sell=1, hold=0)
    coin  0.6753561026946597  curr_bias  Up  new_bias  Up
    obj=13.103911000739522, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.8662383958568906  curr_bias  Up  new_bias  Up
    smoothed_price = 41.35203449737756
    time=1, obj=0.0, s.resource=1, s.price=22.33800862434439, x=Decision(sell=0, hold=1)
    coin  0.9282091767826981  curr_bias  Up  new_bias  Neutral
    smoothed_price = 24.822358317891954
    time=2, obj=0.0, s.resource=1, s.price=22.959096047731283, x=Decision(sell=0, hold=1)
    coin  0.3886605993398763  curr_bias  Neutral  new_bias  Neutral
    smoothed_price = 28.44497845663615
    time=3, obj=0.0, s.resource=1, s.price=24.3305666499575, x=Decision(sell=0, hold=1)
    coin  0.7922183625790814  curr_bias  Neutral  new_bias  Neutral
    smoothed_price = 8.076789104517587
    time=4, obj=0.0, s.resource=1, s.price=20.26712226359752, x=Decision(sell=1, hold=0)
    coin  0.7101424377623707  curr_bias  Neutral  new_bias  Neutral
    obj=20.26712226359752, state.resource=0
    Contribution per iteration:
    0    15.844214
    1    17.413615
    2    16.935480
    3    15.397661
    4    15.601160
    5    22.795736
    6    24.243903
    7    16.846855
    8    13.103911
    9    20.267122
    dtype: float64
    Cumulative average contribution per iteration:
    0    15.844214
    1    16.628914
    2    16.731103
    3    16.397742
    4    16.238426
    5    17.331311
    6    18.318824
    7    18.134828
    8    17.575837
    9    17.844966
    dtype: float64

## Plot Results

```python
df_cont_per_ite = pd.DataFrame({"iteration_number": list(range(nIterations)), "contribution_iterations":contribution_iterations})
sns.lineplot(data=df_cont_per_ite, x= "iteration_number", y="contribution_iterations")
```

## Full grid Search

```python
policy_selected =='full_grid'
# obtain the theta values to carry out a full grid search
grid_search_theta_values = P.grid_search_theta_values(sheet2['low_min'], sheet2['low_max'], sheet2['high_min'], sheet2['high_max'], sheet2['increment_size'])
# use those theta values to calculate corresponding contribution values
```

```python
len(grid_search_theta_values[0])
```

    25

```python
%%capture
policy_selected =='full_grid'
# obtain the theta values to carry out a full grid search
grid_search_theta_values = P.grid_search_theta_values(sheet2['low_min'], sheet2['low_max'], sheet2['high_min'], sheet2['high_max'], sheet2['increment_size'])
# use those theta values to calculate corresponding contribution values

contribution_iterations = [P.vary_theta(param_list, policy_info, "high_low", t, grid_search_theta_values[0]) for ite in list(range(nIterations))]

contribution_iterations_arr = np.array(contribution_iterations)
cum_sum_contrib = contribution_iterations_arr.cumsum(axis=0)
nElem = np.arange(1,cum_sum_contrib.shape[0]+1).reshape((cum_sum_contrib.shape[0],1))
cum_avg_contrib=cum_sum_contrib/nElem
#print("cum_avg_contrib")
#print(cum_avg_contrib)

        # plot those contribution values on a heat map
P.plot_heat_map_many(cum_avg_contrib, grid_search_theta_values[1], grid_search_theta_values[2], printIterations)
```

```python
print("cum_avg_contrib")
print(cum_avg_contrib)
```

    cum_avg_contrib
    [[18.47367058 20.02044278 23.71558344 24.04911983 25.64664546 18.46354228
      19.9971443  23.4944788   8.82900535 27.39145963 18.41058556 20.46051754
      21.80163486 25.30872441 24.51162131 18.97575447 13.24130896 13.83837746
      14.65914641 24.51858946 22.05931859 19.54746608 23.29884974 13.64768737
      26.03581103]
     [19.69100318 20.37305553 23.23858313 23.76467669 25.84529101 18.23380197
      16.47730129 17.92318102 10.78275225 26.04158751 15.6912722  20.2448214
      17.87449738 19.47026892 24.74667952 16.78197067 16.76903764 14.10385934
      18.63771033 24.42103201 22.06444139 17.12797049 23.73444868 14.77986095
      20.09595631]
     [20.69994569 16.83950464 22.74294447 19.56126436 20.52459656 18.47919587
      15.11744387 16.20921065 15.00940661 25.51281981 13.80659895 20.25218014
      19.14019475 17.30568466 21.05307615 17.63382212 18.0583466  13.60198246
      20.07523909 21.21275022 20.9849638  16.32976729 20.95614325 15.05129774
      17.81028782]
     [20.18013059 17.80921563 22.83681797 20.66946341 21.67762855 18.64562133
      16.51153134 15.06668557 13.78071309 25.56287196 13.84389485 20.76422187
      19.73595235 18.96932916 21.87421549 18.05258783 18.52434386 15.50891778
      18.55859893 22.63139844 19.56249686 16.15022656 19.50527058 15.13432515
      17.22395803]
     [20.20517103 18.45097802 22.52135779 18.84335523 19.48817699 18.84645425
      17.12043862 16.58986652 15.60289554 25.29226768 14.71355315 19.09216357
      20.18432852 17.72401005 20.26431727 18.53066645 18.82637145 15.34413496
      19.59040908 21.06536502 19.48544474 16.04541057 20.42913257 14.70689882
      16.94170926]
     [19.9262659  18.67286269 20.75302403 19.71022848 18.21367716 19.19238006
      17.51795722 17.68624384 16.78180395 22.78350727 15.48395009 19.30591146
      19.01314863 17.08034736 20.95328276 18.66106818 18.10879285 14.98659596
      20.39322653 20.0443103  19.9037171  16.94296788 19.57782471 14.52283041
      18.59472501]
     [19.8137429  18.9086271  20.84268636 18.39076378 17.23701861 19.36730219
      18.20099241 18.23276394 17.66079438 21.17920167 16.1442936  18.43430402
      19.43508475 17.9552651  21.50933229 18.57660873 18.42222031 16.25785103
      21.01294484 19.29514058 19.22370624 16.51033466 18.79126479 14.48088114
      17.95088124]
     [19.62653733 17.82007762 20.94883455 17.34353318 18.63876886 19.81500259
      17.46284492 18.58482107 16.70445351 20.04914113 16.41550734 18.8476955
      19.63813038 17.28721213 20.53508951 18.99281685 17.86891367 17.53757442
      20.159934   20.00713271 18.72468824 16.88411792 18.1474659  14.48123896
      18.91527785]
     [19.63648073 18.1478174  21.24734214 16.45992462 19.69289125 19.82953391
      16.79495559 19.30543893 16.28062686 19.00743174 16.61822087 18.12487532
      19.85294926 16.80365162 19.69257256 19.14099563 18.57933051 18.03082425
      20.63793609 19.24262662 18.34680726 16.33745161 17.85696122 14.5706945
      18.48843037]
     [19.63661504 18.36680492 21.42992756 17.12582509 20.30051109 19.84620636
      17.08335876 19.55338491 15.79131993 18.31310674 17.06211989 18.52285707
      20.03049054 17.48311031 20.27369434 19.18396337 18.20648271 18.56249637
      19.88403096 18.75795701 18.08166846 16.30207016 18.1781855  14.65780872
      19.2417764 ]]

```python
P.plot_heat_map_many(cum_avg_contrib, grid_search_theta_values[1], grid_search_theta_values[2], printIterations)
```

```python
type(cum_avg_contrib)
```

    numpy.ndarray

```python
grid_search_theta_values[1]
```

    array([[12.],
           [13.],
           [14.],
           [15.],
           [16.]])

```python
ax = sns.heatmap(cum_avg_contrib)
```

```python
len(cum_avg_contrib)
```

    10


# Manual definition of Feynman-Kac models

It is not particularly difficult to define manually your own `FeynmanKac` classes. Consider the following problem: we would like to approximate the probability that $X_t \in [a,b]$ for all $0\leq t < T$, where $(X_t)$ is a random walk: $X_0\sim N(0,1)$, and
$$ X_t | X_{t-1} = x_{t-1} \sim N(x_{t-1}, 1).$$

This probability, at time $t$, equals $L_t$, the normalising constant of the following Feynman-Kac sequence of distributions:

\begin{equation}
\mathbb{Q}_t(dx_{0:t}) = \frac{1}{L_t} M_0(dx_0)\prod_{s=1}^t M_s(x_{s-1},dx_s)
\prod_{s=0}^t G_s(x_{s-1}, x_s)
\end{equation}

where:

* $M_0(dx_0)$ is the $N(0,1)$ distribution;
* $M_s(x_{s-1},dx_s)$ is the $N(x_{s-1}, 1)$ distribution;
* $G_s(x_{s-1}, x_s)= \mathbb{1}_{[0,\epsilon]}(x_s)$

Let's define the corresponding `FeymanKac` object:

```python
from matplotlib import pyplot as plt
import seaborn as sb
import numpy as np
from scipy import stats

import particles

class GaussianProb(particles.FeynmanKac):
    def __init__(self, a=0., b=1., T=10):
        self.a, self.b, self.T = a, b, T

    def M0(self, N):
        return stats.norm.rvs(size=N)

    def M(self, t, xp):
        return stats.norm.rvs(loc=xp, size=xp.shape)

    def logG(self, t, xp, x):
        return np.where((x < self.b) & (x > self.a), 0., -np.inf)
```

The class above defines:

* the initial distribution, $M_0(dx_0)$ and the kernels $M_t(x_{t-1}, dx_t)$, through methods `M0(self, N)` and `M(self, t, xp)`. In fact, these methods simulate $N$ random variables from the corresponding distributions.
* Function `logG(self, t, xp, x)` returns the log of function $G_t(x_{t-1}, x_t)$.

Methods `M0` and `M` also define implicitly how the $N$ particles should be represented internally: as a (N,) numpy array. Indeed, at time $0$, method `M0` generates a (N,) numpy array, and at times $t\geq 1$, method `M` takes as an input (`xp`) and returns as an output arrays of shape (N,). We could use another type of object to represent our $N$ particles; for instance, the `smc_samplers` module defines a `ThetaParticles` class for storing $N$ particles representing $N$ parameter values (and associated information).

Now let's run the corresponding SMC algorithm:

```python
fk_gp = GaussianProb(a=0., b=1., T=30)
alg = particles.SMC(fk=fk_gp, N=100)
alg.run()

plt.style.use('ggplot')
plt.plot(alg.summaries.logLts)
plt.xlabel('t')
plt.ylabel(r'log-probability');
```

That was not so hard. However our implementation suffers from several limitations:

1. The SMC sampler we ran may be quite inefficient when interval $[a,b]$ is small; in that case many particles should get a zero weight at each iteration.
2. We cannot currently run the SQMC algorithm (the quasi Monte Carlo version of SMC); to do so, we need to specify the Markov kernels $M_t$ in a different way: not as simulators, but as deterministic functions that take as inputs uniform variates (see below).

Let's address the second point:

```python
class GaussianProb(particles.FeynmanKac):
    du = 1  # dimension of uniform variates

    def __init__(self, a=0., b=1., T=10):
        self.a, self.b, self.T = a, b, T

    def M0(self, N):
        return stats.norm.rvs(size=N)

    def M(self, t, xp):
        return stats.norm.rvs(loc=xp, size=xp.shape)

    def Gamma0(self, u):
        return stats.norm.ppf(u)

    def Gamma(self, t, xp, u):
        return stats.norm.ppf(u, loc=xp)

    def logG(self, t, xp, x):
        return np.where((x < self.b) & (x > self.a), 0., -np.inf)

fk_gp = GaussianProb(a=0., b=1., T=30)
```

We have added:

* methods `Gamma0` and `Gamma`, which define the deterministic functions $\Gamma_0$ and $\Gamma$ we mentioned above. Mathematically, for $U\sim \mathcal{U}([0,1]^{d_u})$, then $\Gamma_0(U)$ is distributed according to $M_0(dx_0)$, and $\Gamma_t(x_{t-1}, U)$ is distributed according to $M_t(x_{t-1}, dx_t)$.
* class attribute `du`, i.e. $d_u$, the dimension of the $u$-argument of functions $\Gamma_0$ and $\Gamma_t$.

We are now able to run both the SMC and the SQMC algorithms that corresponds to the Feyman-Kac model of interest; let's compare their respective performance.  (Recall that function ``multiSMC`` runs several algorithms multiple times,  possibly with varying parameters; here we vary parameter `qmc`, which determines whether we run SMC or SMQC.)

```python
results = particles.multiSMC(fk=fk_gp, qmc={'smc': False, 'sqmc': True}, N=100, nruns=10)

sb.boxplot(x=[r['qmc'] for r in results], y=[r['output'].logLt for r in results]);
```

We do get some variance reduction, but not so much. Let's see if we can do better by addressing point 1 above.

The considered problem has the structure of a state-space model, where process $(X_t)$ is a random walk, $Y_t = \mathbb{1}_{[a,b]}(X_t)$, and $y_t=1$ for all $t$'s. This remark leads us to define alternative Feynman-Kac models, that would correspond to *guided* and *auxiliary* formalisms of that state-space model. In particular, for the guided filter, the  optimal proposal distribution, i.e. the distribution of $X_t|X_{t-1}, Y_t$, is simply a Gaussian distribution truncated to interval $[a, b]$; let's implement the corresponding Feynman-Kac class.

```python
def logprobint(a, b, x):
    """ returns log probability that X_t\in[a,b] conditional on X_{t-1}=x
    """
    return np.log(stats.norm.cdf(b - x) - stats.norm.cdf(a - x))

class Guided_GP(GaussianProb):

    def Gamma(self, t, xp, u):
        au = stats.norm.cdf(self.a - xp)
        bu = stats.norm.cdf(self.b - xp)
        return xp + stats.norm.ppf(au + u * (bu - au))

    def Gamma0(self, u):
        return self.Gamma(0, 0., u)

    def M(self, t, xp):
        return self.Gamma(t, xp, stats.uniform.rvs(size=xp.shape))

    def M0(self, N):
        return self.Gamma0(stats.uniform.rvs(size=N))

    def logG(self, t, xp, x):
        if t == 0:
            return np.full(x.shape, logprobint(self.a, self.b, 0.))
        else:
            return logprobint(self.a, self.b, xp)

fk_guided = Guided_GP(a=0., b=1., T=30)
```

In this particular case, it is a bit more convenient to define methods `Gamma0` and `Gamma` first, and then define methods `M0` and `M`.

To derive the APF version, we must define the auxiliary functions (functions $\eta_t$ in Chapter 10 of the book) that modify the resampling probabilities; in practice, we define the log of these functions,
as follows:

```python
class APF_GP(Guided_GP):
    def logeta(self, t, x):
        return logprobint(self.a, self.b, x)

fk_apf = APF_GP(a=0., b=1., T=30)
```

Ok, now everything is set! We can do a 3x2 comparison of SMC versus SQMC, for the 3 considered Feynman-Kac models.

```python
results = particles.multiSMC(fk={'boot':fk_gp, 'guided':fk_guided, 'apf': fk_apf},
                             N=100, qmc={'smc': False, 'sqmc': True}, nruns=200)

sb.boxplot(x=['%s-%s'%(r['fk'], r['qmc'])  for r in results], y=[r['output'].logLt for r in results]);
```

Let's discard the bootstrap algorithms to better visualise the results for the other algorithms:

```python
res_noboot = [r for r in results if r['fk']!='boot']
sb.boxplot(x=['%s-%s'%(r['fk'], r['qmc'])  for r in res_noboot], y=[r['output'].logLt for r in res_noboot]);
```

Voilà!


## CCNSS 2018 Module 1: Neurons, synapses and networks
# Tutorial 2: Eigenvalues and stability
[source](https://colab.research.google.com/drive/1AjAsADoPZXHw4HDkmYMCjQbeuNujKtol)

Please execute the cell below to initialise the notebook environment.

```python
%autosave 0

import matplotlib.pyplot as plt    # import matplotlib
import numpy as np                 # import numpy
import random                      # import basic random number generator functions
import scipy.optimize as opt       # import root-finding algorithm

fig_w, fig_h = (6, 4)
plt.rcParams.update({'figure.figsize': (fig_w, fig_h)})
```

    Autosave disabled

## Objectives
In this notebook, we return to the Wilson-Cowan equations and learn about eigenvalues and stability. First execute the cell below to initialise the simulation parameters.

**Background paper:**
*  Wilson H and Cowan J (1972) Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal 12.

```python
dt = 0.1

# Connection weights
wEE = 12
wEI = 4
wIE = 13
wII = 11

# Refractory parameter
r = 1

# External input
I_ext = 0

# Excitatory parameters
tau_E = 1       # Timescale of excitatory population
a_E = 1.2       # Gain of excitatory population
theta_E = 2.8   # Threshold of excitatory population

# Inhibitory parameters
tau_I = 1       # Timescale of inhibitory population
a_I = 1         # Gain of inhibitory population
theta_I = 4     # Threshold of inhibitory population
```

Recall from Tutorial 1 that the Wilson-Cowan equations are given by:

\begin{align}
& \frac{dE}{dt} = -E + (1 - r E) F(w_{EE}E -w_{EI}I + I_{ext};a,\theta)\\
& \frac{dI}{dt} = -I + (1 - r I) F(w_{IE}E -w_{II}I;a,\theta)
\end{align}

Note that, in the cell above, we set $\tau_E = \tau_I = 1$, so we have dropped the timescales from the equations above. Let's now define the right-hand side of the equation using the functions $G_E(E,I)$ and $G_I(E,I)$:

\begin{align}
&\frac{dE}{dt} =G_E(E,I)\\
&\frac{dI}{dt} = G_I(E,I)
\end{align}

Previously, we studied a bifurcation in the Wilson-Cowan equations in which the system changed from one stable steady state to two (bistability). In this tutorial, we will use linear stability analysis to analyse the stability of the fixed points during this bifurcation. We will need to borrow some functions from the previous tutorial to simulate the equations, as well as to plot the E and I nullclines.

Execute the cell below to define the relevant functions (you may also copy paste the ones you have already created).

```python
def F(x,a,theta):
    """Population activation function.

    Arguments:
    x -- the population input
    a -- the gain of the function
    theta -- the threshold of the function

    Returns:
    y -- the population activation response
    """

    y = (1+np.exp(-a*(x-theta)))**-1 - (1+np.exp(a*theta))**-1

    return y

def simulate_wc(t,E0,I0):
    """Simulate the Wilson-Cowan equations.

    Arguments:
    t -- time (vector)
    E0 -- initial condition of the excitatory population
    I0 -- initial condition of the inhibitory population

    Returns:
    E -- Activity of excitatory population (vector)
    I -- Activity of inhibitory population (vector)
    """

    dt = t[1]-t[0]
    E = np.zeros(len(t))
    I = np.zeros(len(t))
    E[0] = E0
    I[0] = I0
    for k in range(len(t)-1):
        E[k+1] = E[k] + dt/tau_E * (-E[k]+(1-r*E[k])*F(wEE*E[k]-wEI*I[k]+I_ext,a_E,theta_E))
        I[k+1] = I[k] + dt/tau_I * (-I[k]+(1-r*I[k])*F(wIE*E[k]-wII*I[k],a_I,theta_I))

    return E,I

def F_inv(x,a,theta):
    """Define the inverse of the population activation function.

    Arguments:
    x -- the population input
    a -- the gain of the function
    theta -- the threshold of the function

    Returns:
    y -- value of the inverse function
    """

    y = -1/a * np.log((x+(1+np.exp(a*theta))**-1)**-1 -1) + theta

    return y

def get_E_nullcline(E):
    """Solve for I along the E nullcline (dE/dt = 0).

    Arguments:
    E -- values of E over which the nullcline is computed

    Returns:
    I -- values of I along the nullcline for each E
    """
    I = 1./wEI*(wEE*E+I_ext-F_inv(E/(1-r*E),a_E,theta_E))

    return I

def get_I_nullcline(I):
    """Solve for E along the I nullcline (dI/dt = 0).

    Arguments:
    I -- values of I over which the nullcline is computed

    Returns:
    E -- values of E along the nullcline for each I
    """
    E = 1./wIE*(wII*I+F_inv(I/(1-r*I),a_I,theta_I))

    return E
```

**Exercise 1:**  To find the exact fixed points of the system we will need to numerically solving for the fixed point values as the point where $G_E = G_I = 0$. Fill in the function ``wilson_cowan`` below to define $G_E$ and $G_I$. Then, use ``opt.root`` to find the zeros of the Wilson-Cowan equations. Because there are multiple fixed points, you will need to input a different initial guess for each one: use the nullclines from Tutorial 1 Exercise 5 to find a nearby value by eye. Print the values of each fixed point, and save them as `x_fp_1`, etc.

**Hint:** To find the fixed point, use \\
``opt.root(fun,x0).x`` \\
where ``fun`` is the function whose zeros you want to solve numerically. Since there are many potential zeros, you also need to guess ``x0``an initial point in [E,I] space that is near the fixed point. Note that since ``fun`` is a vector functions, note that ``x0`` will also have to be a vector of the right size. In order to guess ``x0``, use the intersection of the nullclines below:

```python
def wilson_cowan(x):
    """Compute dE/dt and dI/dt at a single point in time.
    Used to numerically find fixed points.

    Arguments:
    x -- 2x1 state vector, of which element 0 is E activity and element 1 is I activity

    Returns:
    y -- 2x1 vector of the right hand side of the WC equations (i.e., F_x,F_I)
    """
    # insert your code here

    return y

# insert your code here
```

**EXPECTED OUTPUT**

```
[0. 0.]
[0.17803726 0.06280118]
[0.46246487 0.24336411]
```

**Exercise 2:** Now we will. First differentiate F (the sigmoidal activation function) with respect to x. Then fill out the first function to define the derivative of F. Next fill in the second function to calculate the eigenvalues of the Jacobian matrix using ``np.linalg.eig``. Recall that the Jacobian is defined as:

 $$
   J=
  \left[ {\begin{array}{cc}
   \frac{dG_E}{dE} & \frac{dG_E}{dI} \\
   \frac{dG_I}{dE} & \frac{dG_I}{dI} \\
  \end{array} } \right]
$$

To calculate these derivatives we will need to differentiate the population activation function. Recall that:
$$ F(x;a,\theta) = \frac{1}{1+\exp\{-a(x-\theta)\}} - \frac{1}{1+\exp\{a\theta\}}$$
Therefore, the derivative is given by:
\begin{align}
\frac{dF}{dx} & = \frac{d}{dx} (1+\exp\{-a(x-\theta)\})^{-1}  \\
& = a\exp\{-a(x-\theta)\} (1+\exp\{-a(x-\theta)\})^{-2}
\end{align}
Define a function to calculate $\frac{dF}{dx}$ by executing the cell below.

```python
def DF(x,a,theta):
    """Calculate the derivative of the population activation function, and evaluate at x.

    Arguments:
    x -- the population input
    a -- the gain of the function
    theta -- the threshold of the function

    Returns:
    DF -- derivative of activation function
    """

    DF = a*np.exp(-a*(x-theta))*(1+np.exp(-a*(x-theta)))**-2

    return DF
```

Using the chain and product rules, the derivatives for the excitatory population are given by:
\begin{align}
&\frac{dG_E}{dE} =  -1 - r F(w_{EE}E -w_{EI}I + I_{ext}) + w_{EE} (1 - r E) F'(w_{EE}E -w_{EI}I + I_{ext}) \\
&\frac{dG_E}{dI} =  -w_{EI}(1 - rE) F'(w_{EE}E -w_{EI}I + I_{ext})
\end{align}

And similarly for the inhibitory population. Therefore the rest of the equations are given by:
```
dGE_dE = -1 - r*F(wEE*E-wEI*I+I_ext,a_E,theta_E) + wEE*(1-r*E)*DF(wEE*E-wEI*I+I_ext,a_E,theta_E);
dGI_dI = -1 - r*F(wIE*E-wII*I,a_I,theta_I) - wII*(1-r*I)*DF(wIE*E-wII*I,a_I,theta_I);
dGE_dI = -wEI*(1-r*E)*DF(wEE*E-wEI*I+I_ext,a_E,theta_E);
dGI_dE = wIE*(1-r*I)*DF(wIE*E-wII*I,a_I,theta_I);
```
Use these equations to fill in the function below to defined the Jacobian function at a given fixed point and calculate the eigenvalues. Use this function to calculate the eigenvalues at each of the three fixed points. What does this tell you about the stability? Compare with your phase portrait from Tutorial 1.

```python
def get_eig_Jacobian(fp):
    """Calculate the Jacobian, and then return its eigenvalues.

    Arguments:
    fp -- 2x1 vector for the fixed point, where fp[0] is E, fp[1] is I

    Returns:
    evals -- 2x1 vector of eigenvalues of the Jacobian matrix
    """
    # insert your code here

    return evals

# insert your code here
```

**EXPECTED OUTPUT**

```
[-0.59221954 -1.13490067]
[ 0.73429984 -1.59936786]
[-1.43398007 -2.88063222]
```

**Exercise 3:** Vary $w_{EE}$ for 35 points between 5 and 15 while plotting the maximum real part of the eigenvalue of each fixed point. What happens in this bifurcation? Also simulate and plot the model for several different values of $w_{EE}$ to see how fast the solution decays to the fixed point [0,0]. Use the initial condition $E_0=I_0=0.05$. How does the timescale of the dynamics change with the eigenvalue?

```python
# insert your code here
```

**EXPECTED OUTPUT**

**Exercise 4:** Now change the connectivity and external input parameters by executing the cell below.

```python
# New recurrent connection strengths
wEE = 16
wEI = 12
wIE = 15
wII = 3

# New external input
I_ext = 1
```

Use ``get_E_nullcline`` and ``get_I_nullcline`` to plot the nullclines. How many fixed points do you see? Now plot some example trajectories with different initial conditions (in the 'expected output' I used $E_0 = 0,0.2,0.4$ and $I_0 = 0,0.2,0.4$, and an extra initial condition at $E_0=I_0= 0.25$). What attractors do you see? Test the stability of the limit cycle by plotting E and I vs t for an example initial condition that converges to the limit cycle (up to $t_{max}=50$). Finally, test the stability of the fixed point by calculating the eigenvalues.

```python
# insert your code here
```

**EXPECTED OUTPUT**

```
[0.10551462+1.63867543j 0.10551462-1.63867543j]
```

**Exercise 5:** Now plot the phase plane for $w_{II} =2,4,6$. This is the same kind of bifurcation that we saw with the FitzHugh Nagumo model (a 'Hopf' bifurcation). Plot the real part of the eigenvalues against $w_{11}$ between $0 \leq w_{11} \leq 6$ to see the transition from.

```python
# insert your code here
```

**EXPECTED OUTPUT**

**Bonus Exercise:** Ask Alex.


# Mixture Models and, specifically, Gaussian Mixture Models

* Thus far, we have primarily discussed relatively simple models consisting of only one peak in the probability distribution function (pdf) when representing data using pdfs.
* For example, when we introduced the probabilistic generative classifier, our examples focused on representing each class using a single Gaussian distribuion.
* Consider the following data set, would a multivariate Gaussian be able to represent each of these clustering data set well?

```python
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
%matplotlib inline

n_samples = 1500
n_clusters = 1;

#generate data
transformation = [[ 0.60834549, -0.63667341], [-0.40887718, 0.85253229]]
X, y = datasets.make_blobs(n_samples=n_samples , centers = n_clusters)
X = np.dot(X, transformation)

#Plot Results
plt.figure(figsize=(12, 12))
plt.subplot(221)
plt.scatter(X[:, 0], X[:, 1])
```

* Would a single multivariate Gaussian be able to represent this data set well?

```python
n_clusters = 10;

#generate data
X, y = datasets.make_blobs(n_samples=n_samples , centers = n_clusters)

#Plot Results
plt.figure(figsize=(12, 12))
plt.subplot(221)
plt.scatter(X[:, 0], X[:, 1])
```

* The second data set would be better represented by a *mixture model*

$p(x) = \sum_{k=1}^K \pi_k f(x | \theta_k)$

where

$0 \le \pi_k \le 1$, $\sum_k \pi_k =1$

* If each $f(x | \theta_k)$ is assumed to be a Gaussian distribution, then the above mixture model would be a *Gaussian Mixture Model*

$p(x) = \sum_{k=1}^K \pi_k N(x | \mu_k, \Sigma_k)$

where

$0 \le \pi_k \le 1$, $\sum_k \pi_k =1$

* *How would you draw from samples from a Gaussian Mixture Model? from a mixture model in general?*

* Gaussian mixture models (GMMs) can be used to learn a complex distribution that represents a data set.  Thus, it can be used within the probabilistic generative classifier framework to model complex classes.
* GMMs are also commonly used for clustering where a GMM is fit to a data set to be clustered and each estimated Gaussian component is a resulting cluster.

* *If you were given a data set, how would you estimate the parameters of a GMM to fit the data?*
* A common approach for estimating the parameters of a GMM given data is *expectation maximization* (EM)

# Expectation Maximization

* EM is a general algorithm that can be applied to a variety of problems (not just mixture model clustering).

* With MLE, we define a likelihood and maximize it to find parameters of interest.

* With MAP, we maximize the posterior to find parameters of interest.

* The goal of EM is to also find the parameters that maximize your likelihood function.

* *The 1st step* is to define your likelihood function (defines your objective)

* Originally introduced by Dempster, Laird, and Rubin in 1977 - ``Maximum Likelihood from Incomplete Data via the EM Algorithm''

* EM is a method to simplify difficult maximum likelihood problems.

* Suppose we observe $\mathbf{x}_1, \ldots, \mathbf{x}_N$ i.i.d. from $g(\mathbf{x}_i | \Theta)$

* We want: $\hat\Theta = argmax  L(\Theta|X) = argmax \prod_{i=1}^N g(\mathbf{x}_i | \Theta)$

* But suppose this maximization is very difficult.  EM simplifies it by expanding the problem to a bigger easier problem - ``demarginalization''

\begin{equation}
g(x|\Theta) = \int_z f(x, z | \Theta) dz
\end{equation}

Main Idea: Do all of your analysis on $f$ and then integrate over the unknown z's.

### Censored Data Example
* Suppose we observe $\mathbf{y}_1, \ldots, \mathbf{y}_N$ i.i.d. from $f(\mathbf{y} | \Theta)$
* Lets say that we know that values are censored at $\ge a$
* So, we see: $\mathbf{y}_1, \ldots, \mathbf{y}_m$ (less than $a$) and we do not see $\mathbf{y}_{m+1}, \ldots, \mathbf{y}_N$  which are censored and set to $a$.
* Given this censored data, suppose we want to estimate the mean if the data was uncensored.
* Our observed data likelihood in this case would be:
\begin{eqnarray}
L &=& \prod_{i=1}^m \left[ 1 - F(a |\theta)\right]^{N-m}f(\mathbf{y}_i | \theta)\\
&=& \prod_{i=1}^m f(\mathbf{y}_i | \theta) \prod_{j=m+1}^N \int_a^\infty f(\mathbf{y}_j | \theta) dy_j
\end{eqnarray}
where $F(\cdot)$ is the cumulative distribution function and $f(y|\theta) = N(y|\theta)$, for example.

* So, the observed data likelihood would be very difficult to maximize to solve for $\theta$
* In EM, we introduce *latent variables* (i.e., ``hidden variables'') to simplify the problem
* *The second step*: Define the *complete likelihood* by introducing variables that simplify the problem.

* Going back to the censored data example, if we had observed the missing data, the problem would be easy to solve! It would simplify to a standard MLE.  For this example, the complete data likelihood is:
\begin{equation}
L^c = \prod_{i=1}^m f(y_i | \theta) \prod_{i=m+1}^N f(z_i | \theta)
\end{equation}
where $z_i$ are the latent, hidden variables.
* Note: you cannot just use $a$ for the censored data, it would skew the results!
* The complete data likelihood would be much much simplier to optimize for $\theta$ if we had the $z$s...

# Reading Assignment

* Reading on K-Means: Section 9.1
* Reading on GMMs and EM: Section 9.2-9.4

```python

```

Lecture09_GMM/Lecture 09 Gaussian Mixture Models.ipynb


# Quantization of Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Introduction

[Digital signal processors](https://en.wikipedia.org/wiki/Digital_signal_processor) and general purpose processors can only perform arithmetic operations within a limited number range. So far we considered discrete signals with continuous amplitude values. These cannot be handled by processors in a straightforward manner. [Quantization](https://en.wikipedia.org/wiki/Quantization_%28signal_processing%29) is the process of mapping a continuous amplitude to a countable set of amplitude values. This refers also to the *requantization* of a signal from a large set of countable amplitude values to a smaller set. Scalar quantization is an instantaneous and memoryless operation. It can be applied to the continuous amplitude signal, also referred to as *analog signal* or to the (time-)discrete signal. The quantized discrete signal is termed as *digital signal*. The connections between the different domains are illustrated in the following.

### Model of the Quantization Process

In order to discuss the effects of quantizing a continuous amplitude signal, a mathematical model of the quantization process is required. We restrict our considerations to a discrete real-valued signal $x[k]$. The following mapping is used in order to quantize the continuous amplitude signal $x[k]$

\begin{equation}
x_Q[k] = g( \; \lfloor \, f(x[k]) \, \rfloor \; )
\end{equation}

where $g(\cdot)$ and $f(\cdot)$ denote real-valued mapping functions, and $\lfloor \cdot \rfloor$ a rounding operation. The quantization process can be split into two stages

1. **Forward quantization**
    The mapping $f(x[k])$ maps the signal $x[k]$ such that it is suitable for the rounding operation. This may be a scaling of the signal or a non-linear mapping. The result of the rounding operation is an integer number $\lfloor \, f(x[k]) \, \rfloor \in \mathbb{Z}$, which is termed as *quantization index*.

2. **Inverse quantization**
    The mapping $g(\cdot)$, maps the quantization index to the quantized value $x_Q[k]$ such that it constitutes an approximation of $x[k]$. This may be a simple scaling or non-linear operation.

The quantization error (quantization noise) $e[k]$ is defined as

\begin{equation}
e[k] = x_Q[k] - x[k]
\end{equation}

Rearranging yields that the quantization process can be modeled by adding the quantization error to the discrete signal

#### Example - Quantization of a sine signal

In order to illustrate the introduced model, the quantization of one period of a sine signal is considered

\begin{equation}
x[k] = \sin[\Omega_0 k]
\end{equation}

using

\begin{align}
f(x[k]) &= 3 \cdot x[k] \\
i &= \lfloor \, f(x[k]) \, \rfloor \\
g(i) &= \frac{1}{3} \cdot i
\end{align}

where $\lfloor \cdot \rfloor$ denotes the [nearest integer function](https://en.wikipedia.org/wiki/Nearest_integer_function) and $i$ the quantization index. The quantized signal is then given as

\begin{equation}
x_Q[k] = \frac{1}{3} \cdot \lfloor \, 3 \cdot \sin[\Omega_0 k] \, \rfloor
\end{equation}

The discrete signals are not shown by stem plots for ease of illustration.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

N = 1024  # length of signal

# generate signal
x = np.sin(2*np.pi/N * np.arange(N))
# quantize signal
xi = np.round(3 * x)
xQ = 1/3 * xi
e = xQ - x

# plot (quantized) signals
fig, ax1 = plt.subplots(figsize=(10,4))
ax2 = ax1.twinx()

ax1.plot(x, 'r', label=r'signal $x[k]$')
ax1.plot(xQ, 'b', label=r'quantized signal $x_Q[k]$')
ax1.plot(e, 'g', label=r'quantization error $e[k]$')
ax1.set_xlabel('k')
ax1.set_ylabel(r'$x[k]$, $x_Q[k]$, $e[k]$')
ax1.axis([0, N, -1.2, 1.2])
ax1.legend()

ax2.set_ylim([-3.6, 3.6])
ax2.set_ylabel('quantization index')
ax2.grid()
```

**Exercise**

* Investigate the quantization error $e[k]$. Is its amplitude bounded?
* If you would represent the quantization index (shown on the right side) by a binary number, how much bits would you need?
* Try out other rounding operations like `np.floor()` and `np.ceil()` instead of `np.round()`. What changes?

Solution: It can be concluded from the illustration that the quantization error is bounded as $|e[k]| < \frac{1}{3}$. There are in total 7 quantization indexes needing 3 bits in a binary representation. The properties of the quantization error are different for different rounding operations.

### Properties

Without knowledge of the quantization error $e[k]$, the signal $x[k]$ cannot be reconstructed exactly from its quantization index or quantized representation $x_Q[k]$. The quantization error $e[k]$ itself depends on the signal $x[k]$. Therefore, quantization is in general an irreversible process. The mapping from $x[k]$ to $x_Q[k]$ is furthermore non-linear, since the superposition principle does not hold in general. Summarizing, quantization is an inherently irreversible and non-linear process. It potentially removes information from the signal.

### Applications

Quantization has widespread applications in Digital Signal Processing. For instance

* [Analog-to-Digital conversion](https://en.wikipedia.org/wiki/Analog-to-digital_converter)
* [Lossy compression](https://en.wikipedia.org/wiki/Lossy_compression) of signals (speech, music, video, ...)
* Storage and transmission ([Pulse-Code Modulation](https://en.wikipedia.org/wiki/Pulse-code_modulation), ...)

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2018*.


# Neurons and Neural Networks in the Human Brain

* The brain is a complex, nonlinear information-processing system made up of a huge number of interconnected of *neurons*
* The neuron in the human brain is five to six orders of magnitude slower than silicon logic gates ( $10^{-3}$s/millisecond vs. $10^{-9}$s/nanosecond )
* See http://biomedicalengineering.yolasite.com/neurons.php for an illustration of neurons in the brain.

* *Synapse:* units that control interaction between neurons. A *Chemical synapse* is the most common kind of synapse.  It converts presynaptic electrical signal into a chemical signal and then back.  Traditional descriptions of assume that a synapse can either impose *excitation* or *inhibition* on the receptive neuron.
* See: https://science.education.nih.gov/supplements/webversions/BrainAddiction/guide/lesson2-1.html

# Brief ANN History

* 1943 McCulloch and Pitts.  McCulloch was a psychatrist and neuroanatomist.  Pitts was a mathematician.  They published a widley read article that introduced the idea of neural networks as computing machines. Their goal was to develop a model/understand how neurons in the brain might work.  They showed a range of arithmetic and logical functions their neuron could compute.
* 1949 Hebb wrote "The Organization of Behavior" which postulated (among many other things): "When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased"  or "when neurons fire together they wire together"  This has come to be known as Hebbian learning.
* 1954 Minsky wrote a "neural network" doctoral thesis at Princeton.
* 1958 Rosenblatt introduced his work in the perceptron and he came up with the perceptron convergence theorem. First model for learning with a teacher (i.e., "supervised learning")
* 1969 Minsky and Papert demonstrated the limits of the perceptron. They introduce multi-layer perceptrons but the published limits had the biggest influence - and interest dropped away. The AI winter began.
* 1974 Werbos' Ph.D. thesis at Harvard developed back-propagation
* 1986, the book "Parallel Distributed Processing: Explorations in the Microstructures of Cognition" was published and it covered back-propagation.  This made NN popular again.
* Early 90's, SVMs overtook ANNs in popularity due to a number of challenges/downsides to ANNs in comparison to SVMs.  This included that SVMs were less likely to overtrain and easier to get good results on. Also, ANNs were very slow to train and had issues when they became "deep".
* 2012 ImageNet challenge won by Hinton's team using a deep CNN (based on top 5 error rate, given an image, the model does not output the correct label within its top 5 predictions). They had an error rate of 15.4\% (which was way better than 2nd place as 26.6\%) This started the current DL/ANN resurgance. Now its HUGE.
* *What do you think will happen next?*

# Rosenblatt's Perceptron

* In an artificial neural network, we attempt to model the human neural network and neurons using programming constructs
* A basic model for a neuron consists of the following:
     * A set of *synapses* each of which is characterized by a *weight* (which includes a *bias*).
     * An *adder*
     * An *activiation function* (e.g. Rectified Linear Unit, Heaviside function, Piece-wise linear function, Sigmoid function)

* We can write this mathematically as:
\begin{equation}
y_k = \phi\left( \sum_{j=1}^m w_{kj}x_j + b_k\right)
\end{equation}
* *What does this look like graphically?*

# Perceptron Learning Algorithm
* Suppose we have a ``neural network" made of only one neuron - i.e., Rosenblatt's perceptron and we would like to train it to distinguish between two classes.
* For Rosenblatt's perceptron to be effective, the classes must be linearly separable.  Mathematically, the classes must satisfy:
\begin{equation}
\exists \mathbf{w} \mid \mathbf{w}^T\mathbf{x} > 0 \quad \forall \mathbf{x} \in C_1;  \quad \mathbf{w}^T\mathbf{x} \le 0 \quad \forall \mathbf{x} \in C_2
\end{equation}
* The Perceptron Learning Algorithm will converge to the correct solution if the classes are linearly separable.

```python
import numpy as np
import matplotlib.pyplot as plt
import math
import textwrap
import time
%matplotlib inline

def generateMVNRandData(Npts, mu, sigma):
	data  = np.random.multivariate_normal(mu, sigma*np.eye(len(mu)), Npts)
	return data

def plotLine(weights, range):
	x = np.array(range)
	y = -(weights[0]/weights[1])-(weights[2]/weights[1])*x
	plt.plot(y,x)
	plt.pause(2)

def perceptronLearningAlg(data,labels,eta,nIterations):
	nPts = data.shape[0]
	weights = np.random.rand(data.shape[1])
	print('Initial weights:', weights)

	error = 1;
	iter = 0;
	while(error > 0 & iter < nIterations):
		print('Iteration: ', iter,'; Error: ', error)
		error = 0
		iter += 1
		for i in range(nPts):
			activation =  data[i,:]@weights
			activation = (activation>0)
			if (activation-labels[i])!=0:
				weights-=eta*data[i,:]*(activation-labels[i])
				error += 1
				plotLine(weights, [-2,2])

	plotLine(weights, [-2,2])
	print('Final Iteration: ', iter,'; Final Error: ', error)
	return weights

Npts  = 100
mu1   = [2,2]
mu2   = [0,0]
var   = .1
eta   = 10
nIterations = 10;

fig   = plt.figure()

data1 = np.array(generateMVNRandData(Npts, mu1, .1))
data1 = np.hstack((np.ones((Npts,1)),data1))
data2 = np.array(generateMVNRandData(Npts, mu2, .1))
data2 = np.hstack((np.ones((Npts,1)),data2))

data  = np.vstack(( data1, data2))
labels= np.hstack((np.ones(Npts), np.zeros(Npts)))

plt.scatter(data[:,1],data[:,2], c=labels, linewidth=0)
plt.pause(2)

perceptronLearningAlg(data,labels,eta,nIterations)
```

Questions to consider:
* How does the learning behavior change with changes in $\eta$? as $\eta$ increases? as $\eta$ decreases?
* How would you generate overlapping classes using the provided code? Explain your answer. (Only change parameters.  You do not need to change code. )
* What happens to the learning behavior when you have overlapping classes?
* The implementation provided uses \{0,1\} labels whereas the psuedo-code above uses \{-1,1\}.  How does the code need to change to account for this difference (i.e., suppose you want to use \{-1,1\} labels.  What would you need to change in the code?) Why?
* In the provided code, there is not a separate line for learning the bias $b$as in the psuedo-code above.  How is it being estimated and represented it in the code (... the code IS still learning the bias value).

```python

```


# Appendix A: Gradients Review

## Introduction

This short appendix provides a refresher on gradients and calculus. Some of you may never have seen gradients before. Multivariate calculus is not a required prerequisite for the MDS program but it will be helpful to know for this course, so we'll cover the basics in this appendix. This material has been modified after material originally created by [Mike Gelbart](https://www.mikegelbart.com/).

## Imports

```python
import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt
plt.style.use('ggplot')
plt.rcParams.update({'font.size': 16,
                     'axes.labelweight': 'bold',
                     'axes.grid': False,
                     'figure.figsize': (8,6)})
```

## 1. Ingredient 1: Functions of Multiple Variables

- We saw this in DSCI 551 with joint PDFs.
- We can write such a function as $f(x,y,z)$ (for 3 inputs) or $f(x)$ if $x$ is a vector.
- Example: $f(x,y,z) = x^2 + y^2 + e^z + x^z + xyz$.

```python
def f(x, y, z):
    return x**2 + y**2 + np.exp(z) + np.power(x,z) + x*y*z

f(1,2,3)
```

    32.08553692318767

- Another example: $f(x,y) = \sin(xy)$
- We can visualize functions of two variables, but it gets much harder after that.

```python
f = lambda x, y: np.sin(x * y)

x = np.linspace(0, 4, 1000)
y = np.linspace(0, 4, 1000)
xx, yy = np.meshgrid(x, y)
zz = f(xx, yy)

plt.imshow(zz, extent=(np.min(x), np.max(x), np.min(y), np.max(y)), origin="lower")
plt.xlabel("x")
plt.ylabel("y")
plt.title("$\sin(xy)$")
plt.colorbar();
```

## 2. Ingredient 2: Vector-valued Functions

- You may not have encountered these yet in MDS.
- These are functions with multiple _outputs_ (and may or may not have multiple _inputs_).
- Example with 1 input and 3 outputs:

$$f(x)=\begin{bmatrix} x^2 \\ 2x \\ -x\end{bmatrix}$$

- Example with 3 inputs and 4 outputs:

$$f(x,y,z)=\begin{bmatrix} yz^2 \\ 0 \\ xyz \\ x-y\end{bmatrix}$$

- Example with 2 inputs and 2 outputs:

$$f(x,y)=\begin{bmatrix} x \\ \sin(y) \end{bmatrix}$$

```python
def f(x, y):
    return np.array([x, np.sin(y)])

f(2, 10)
```

    array([ 2.        , -0.54402111])

We can visualize functions with two outputs (and two inputs), but it gets much harder after that.

```python
x = np.linspace(-5, 5, 20)
y = np.linspace(-5, 5, 20)
xx, yy = np.meshgrid(x, y)
zz = f(xx, yy)

plt.quiver(xx, yy, zz[0], zz[1])
# plt.axis('square');
plt.xlabel("x")
plt.ylabel("y")
plt.title("$f(x,y) = [x; \; \sin(y)]$")
plt.show()
```

Notes:

- For a fixed $y$, when $x$ grows, the $x$-component of the output grows (horizontal length of the arrows)
- A similar argument can be made for $y$.
- It's not always the case that the number of inputs equals the number of outputs - this is a special case!
  - But it's a very important special case, as we'll see below.
  - What it means is that the "input space" and the "output space" are the same.
  - Which allows for this kind of visualization.
- (optional) It's not always the case that the $i$th component of the output depends on the $i$th component of the inputs - this is also a special case!

## 3. Ingredient 3: Partial Derivatives

- A partial derivative is just a derivative of a multivariable function with respect to one of the input variables.
- When taking this derivative, we treat all the other variables as constants.
  - See [partial derivative slide](partial-deriv.pdf).
- Example: let $f(x,y,z) = x^2 + y^2 + e^x + x^z + xyz$, let's compute $\frac{\partial}{\partial x} f(x,y,z)$

$$\begin{align}\frac{\partial}{\partial x} \quad &x^2 &+ \quad &y^2 &+ \quad &e^x &+ \quad &x^z &+ \quad &xyz\\=\quad &2x &+ \quad &0 &+\quad  &e^x &+ \quad &zx^{z-1} &+ \quad &yz\end{align}$$

- **Important note**: $\frac{\partial f}{\partial x} $ is itself a function of $x,y,z$, not just a function of $x$. Think about the picture from the PDF slide above: the slope depends on your position in all coordinates.
- (optional) Thus, the partial derivative operator $\frac{\partial}{\partial x}$ maps from multivariate functions to multivariable functions.

## 4. Ingredient 4: Gradients

- This is the easy part: a gradient is just a box holding all the $d$ partial derivatives (assuming you have a function of $d$ variables). For example, when $d=3$:

$$\nabla f(x,y,z)=\begin{bmatrix}\frac{\partial f}{\partial x}(x,y,z)\\ \frac{\partial f}{\partial y}(x,y,z) \\\frac{\partial f}{\partial z}(x,y,z)\end{bmatrix}$$

Or, more generally, if $x$ is a vector then

$$\nabla f(x)=\begin{bmatrix}\frac{\partial f}{\partial x_1}(x)\\ \frac{\partial f}{\partial x_2}(x) \\ \vdots \\ \frac{\partial f}{\partial x_d}(x)\end{bmatrix}$$

- (optional) Thus, a partial derivative is a function that has the same mapping as the original, e.g. $\mathbb{R}^3\rightarrow \mathbb{R}$ ("R three to R").
- (optional) a gradient is a function that maps from the original input space to the same space, e.g. $\mathbb{R}^3\rightarrow \mathbb{R}^3$ ("R three to R three").

**Notation warning**: we use the term "derivative" or "gradient" to mean three different things:

1. Operator (written $\frac{d}{dx}$ or $\nabla$), which maps **functions to functions**; "now we take the gradient".
2. Function (written $\frac{df}{dx}$ or $\nabla f$), which maps **vectors to vectors**; "the gradient is $2x+5$"
    - This is what you get after applying the operator to a function.
3. Value (written as a number or vector), which is **just a number or vector**; "the gradient is  $\begin{bmatrix}-2.34\\6.43\end{bmatrix}$"
    - This is what you get after applying the function to an input.

This is extremely confusing!

Here's a table summarizing the situation, assuming 3 variables (in general it could be any number)

|  Name   | Operator | Function | Maps  | Example Value  |
|---------|----------|-----------|------|----------|
| Derivative | $\frac{d}{dx}$ | $\frac{df}{dx}(x)$ | $\mathbb{R}\rightarrow \mathbb{R}$ | $2.5$ |
| Partial Derivative |  ${\frac{\partial}{\partial x}}$ |  ${\frac{\partial f}{\partial x}}(x,y,z)$ | ${\mathbb{R}^3\rightarrow \mathbb{R}}$ | $2.5$ |
| Gradient |  $\nabla$ | $\nabla f(x,y,z)$ |$\mathbb{R}^3\rightarrow \mathbb{R}^3$ | $\begin{bmatrix}2.5\\0\\-1\end{bmatrix}$ |

### 4.1 Gradients intuition

Since a gradient is a vector, we can talk about its magnitude and direction.
  - The magnitude is $\|\nabla f\|$ and tells us **how fast things are changing**.
  - The direction is $\frac{\nabla f}{\|\nabla f \|}$ and tells us **the direction of fastest change** or the steepest direction.

```python
# gradient vector field
f = lambda x, y: np.sin(x * y)

x = np.linspace(0, 4, 1000)
y = np.linspace(0, 4, 1000)
xx, yy = np.meshgrid(x, y)
zz = f(xx, yy)

plt.imshow(zz, extent=(np.min(x), np.max(x), np.min(y), np.max(y)), origin="lower")
plt.xlabel("x")
plt.ylabel("y")
plt.title("$\sin(xy)$")
plt.colorbar();
```

$$\nabla \sin(xy) = \begin{bmatrix} y \cos(xy)\\x\cos(xy)\end{bmatrix}$$

```python
gradf = lambda x, y: (y * np.cos(x * y), x * np.cos(x * y))

xsmall = np.linspace(0, 4, 15)
ysmall = np.linspace(0, 4, 15)
xxsmall, yysmall = np.meshgrid(xsmall, ysmall)
gradx, grady = gradf(xxsmall, yysmall)
```

```python
plt.quiver(xxsmall, yysmall, gradx, grady)
plt.axis("square")
plt.xlabel("x")
plt.ylabel("y")
plt.title("∇$ \sin(xy)$");
```

```python
plt.figure(figsize=(8,8))
plt.imshow(zz,extent=(np.min(x), np.max(x), np.min(y), np.max(y)), origin='lower')
plt.colorbar();
plt.quiver(xxsmall,yysmall,gradx,grady);
plt.xlabel("x")
plt.ylabel("y")
plt.title("$\sin(xy)$ and its gradient");
```

### 4.2 Why is it the direction of fastest increase?

For example if the gradient is:

$$\begin{bmatrix} 5 \\ 2 \\ -1 \\ 0 \end{bmatrix}\, ,$$

why go in the gradient direction instead of the $x_1$ direction, since that first component has the biggest partial derivative. Doesn't it seem wasteful to go partly in those other directions?

First, a proof that the gradient is the best direction. Let's say we are at position $x$ and we move by an infinitesimal (i.e. extremely tiny) $v$, which is a vector having components $v_1, v_2, \ldots v_d$. The change in $f$ from moving from $x$ to $x+v$ is $\frac{\partial f}{dx_1} v_1 + \frac{\partial f}{dx_2} v_2 + \ldots \frac{\partial f}{dx_d} v_d$, where all the partial derivatives are evaluated at $x$ (I believe this is related to the "total derivative"). In other words, the change in $f$ is the dot product $\nabla f \cdot v$. So now the question is, what vector $v$ of fixed length maximizes $\nabla f \cdot v$. The answer is a vector that points in the same direction as $\nabla f$. (That's a property of the dot product, and is evident by the definition: $a \cdot b = \| a \| \|b \| \cos(\theta)$. Since $\| \nabla f \|$ and $\|v\|$ are fixed in our case, to maximize this we want to maximize $\cos(\theta)$, which means we want $\cos(\theta)=1$ meaning $\theta=0$, or the angle between the vectors is $0$).

Second, the intuition. I think the "paradox" comes from over-privileging the coordinate axes. They are not special in any way! For example if you rotate the coordinate system by 45 degrees, the direction of steepest ascent should also rotate by 45 degrees. Under the suggested system, this would not happen. Why? Well, there is always going to be one element of the gradient that is largest. Does that mean the direction of steepest ascent is always one of the coordinate axis directions? No. That doesn't make sense and also fails the "rotate by 45 degrees test" because the direction will have rotated by 0, 90, 180 or 270 degrees.


# Fourier methods
> Fourier methods using Python

- toc: true
- badges: true
- comments: true
- categories: [jupyter]

The Fourier transform (FT) for a well-behaved functions $f$ is defined as:

$$f(k) = \int e^{-ikx} f(x) ~dx$$

The inverse FT is then

$$f(x) = \frac{1}{2\pi} \int e^{ikx} f(k) ~dk$$

## Discrete Fourier transforms (DFTs)

If the function is periodic in real space, $f(x+L) = f(x)$, then the Fourier space is discrete with spacing $\frac{2\pi}{L}$. Moreover, if the real space is periodic as well as discrete with the spacing $h$, then the Fourier space is discrete as well as bounded.

$$f(x) = \sum e^{ikx} f(k) ~dk~~~~~~      \text{where } k = \Bigg[ -\frac{\pi}{h}, \frac{\pi}{h}\Bigg];~~ \text{with interval} \frac{2\pi}{L} $$

This is very much in line with crystallography with $ [ -\frac{\pi}{h}, \frac{\pi}{h} ]$ being the first Brillouin zone. So we see that there is a concept of the maximum wavenumber $ k_{max}=\frac{\pi}{h} $, we will get back to this later in the notes. Usually in computations we need to find FT of discrete function rather than of a well defined analytic function. Since the real space is discrete and periodic, the Fourier space is also discrete and periodic or bounded. Also, the Fourier space is continuous if the real space is unbounded. If the function is defined at $N$ points in real space and one wants to calculate the function at $N$ points in Fourier space, then **DFT** is defined as

$$f_k = \sum_{n=0}^{N-1} f_n ~ e^{-i\frac{2\pi~n~k}{N}}$$

while the inverse transform of this is

$$f_n = \frac1N \sum_{n=0}^{N-1} f_k ~ e^{~i\frac{2\pi~n~k}{N}}$$

To calculate each $f_n$ one needs $N$ computations and it has to be done $N$ times, i.e, the algorithm is simply $\mathcal{O}(N^2)$. This can be implemented numerically as a matrix multiplication, $f_k = M\cdot f_n$, where $M$ is a $N\times N$ matrix.

## Fast fourier tranforms (FFTs)

The discussion here is based on the Cooley-Tukey algorithm. FFTs improves on DFTs by exploiting their symmetries.

$$ \begin{align}
f_k &= \sum_{n=0}^{N-1} f_n  e^{-i~\frac{2\pi~k~n}{N}} \\
&= \sum_{n=0}^{N/2-1} f_{2n} e^{-i~\frac{2\pi~k~2n}{N}} &+ \sum_{n=0}^{N/2-1} f_{2n + 1} e^{-i~\frac{2\pi~k~(n+1)}{N}}\\
&= \sum_{n=0}^{N/2 - 1} f_{2n} e^{-i~\frac{2\pi k~n}{N/2}} &+ e^{-i\frac{2\pi k}{N}} \sum_{n=0}^{N/2 - 1} f_{2n + 1} e^{-i~\frac{2\pi~k~n~}{N/2}}\\
&=\vdots &\vdots
\end{align}$$

We can use the symmetry property, from the definition, $f_{N+k} = f_k$. Notice that, because of the tree structure, there are $\ln_2 N$ stages of the calculation. By applying the method of splitting the computation in two halves recursively, the complexity of the problem becomes  $\mathcal{O}(N \ln N)$  while the naive algorithm is $\mathcal{O}(N^2)$. Below we will look at a simple implementation of FFT.

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
def testFFT(x):
    ''' FFT in 1d  '''
    N = x.shape[0]
    if N % 2 > 0:
        raise Exception('x must have even size')
    elif N <= 16:
        ''' this is the naive implementation using matrix multiplication'''
        n = np.arange(N)
        k = n.reshape((N, 1))
        M = np.exp(-2j * np.pi * k * n / N)
        return np.dot(M, x)
    else:
        M1 = testFFT(x[::2])
        M2 = testFFT(x[1::2])
        fac = np.exp(-2j*np.pi*np.arange(N)/N)
        return np.concatenate([M1+fac[:N/2]*M2, M1+fac[N/2:]*M2])
```

```python
x = np.random.random(1024)
#np.allclose(np.fft.fftn(x), testFFT(x))
```

```python
# TRANSLATION by e^{ikr}!
L, N = 64, 128
ll =  np.linspace(0, L, N)
x, y = np.meshgrid(ll, ll)

# Fourier grid.
kx = 2 * np.pi / L * np.concatenate((np.arange(0, N/2+1,1),np.arange(-N/2+1, 0, 1)))  # k = (2\pi)/L
ky = 2 * np.pi / L * np.concatenate((np.arange(0, N/2+1,1),np.arange(-N/2+1, 0, 1)))
kx, ky = np.meshgrid(kx, ky)

def plotFirst(x, y, sig, n_):
    sp =  f.add_subplot(1, 3, n_ )
    plt.pcolormesh(x, y, np.real(sig), cmap=plt.cm.gist_heat_r)
    plt.axis('off');

f = plt.figure(figsize=(20, 5), dpi=80);
rr = np.sqrt( (x - L/2)*(x - L/2) + (y - L/2)*(y - L/2) )
sig = np.fft.fftn(np.exp(-0.1*rr))

xx = ([-L/4, L/2, -L/2,])
yy = ([-L/8, -L/9, L/2])
for i in range(3):
    kdotr = kx*xx[i] + ky *yy[i]
    sig = sig*np.exp(-1j*kdotr)
    plotFirst(x, y, np.fft.ifftn(sig), i+1)
```

## More examples
### Sampling: Aliasing error

We saw that because of the smallest length scale, $h$, in the real space there is a corresponding largest wave-vector, $k_{max}$ in the Fourier space. The error is because of this $k_{max}$ and a signal which has $k>k_{max}$ can not be distinguished on this grid. In the given example, below, we see that if the real space has 10 points that one can not distinguish between $sin(2\pi x/L)$ and $sin(34 \pi x/L)$. In general, $sin(k_1 x)$ and $sin(k_2 x)$ can not be distinguished if $k_1 -k_2$ is a multiple of $\frac{2\pi}{h}$. This is a manifestation of the gem called the sampling theorem which is defined, as on wikipedia:

If a function x(t) contains no frequencies higher than B hertz, it is completely determined by giving its ordinates at a series of points spaced 1/(2B) seconds apart.

```python
L, N = 1, 16
x = np.arange(0, L, L/512)
xx = np.arange(0, L, L/N)

def ff(k, x):
    return sin(k*x)
```

```python
f = plt.figure(figsize=(17, 6), dpi=80);  pi=np.pi; sin=np.sin; cos=np.cos

plt.plot(x, ff(x, 2*pi), color="#A60628", linewidth=2);
plt.plot(x, ff(x, 34*pi), color="#348ABD", linewidth=2);
plt.plot(xx, ff(xx, 2*pi), 'o', color="#020e3e", markersize=8)
plt.xlabel('x', fontsize=15); plt.ylabel('y(x)', fontsize=15);
plt.title('Aliasing in sampling of $sin(2\pi x/L)$ and $sin(34 \pi x/L)$', fontsize=15);
```

 ### Differentiation

 In this section we will use the in-built FFT modules of numpy and then perform differentiation. Differentiation in Fourier space is trivial, $\mathbf{\nabla}_{\alpha}$ gets replaced by $ik_{\alpha}$. Steps involved are

* FFT the function to be differentiated.
* multiply by suitable numbers of $ik$
* IFFT on the resulting thing to get the differentiated function in real space.

```python
def f1(kk, x):
    return cos(kk*x)

```

```python
f = plt.figure(figsize=(10, 5), dpi=80);

L, N = 1, 32
kk   = 2*pi/L
x = np.arange(0, N)*(L/N)
k = np.concatenate(( np.arange(0, N/2+1,1), np.arange(-N/2+1, 0, 1) ))*(2*pi/L)

fk = np.fft.fft(f1(kk, x))
f1_kk = -k*k*fk
f1_xx = np.fft.ifft(f1_kk)

plt.plot(x, -f1(kk, x)*kk*kk, color="#348ABD", label = 'analytical', linewidth=2)
plt.plot(x, f1_xx, 'o', color="#A60628", label = 'numerical', markersize=6)
plt.legend(loc = 'best')
plt.xlabel('x', fontsize=15)
plt.title('derivatives using FFT', fontsize=15)
plt.xlim([0, max(x)]);
```

There are additional symmetry properties for a real function. Wavenumbers corresponding to $j$ and $N-j$ are same and hence we only need to consider wavenumber till N/2!

```python
# since we know that function is real, then we could save some time and memory by using real FFTs.
f = plt.figure(figsize=(10, 5), dpi=80);

L, N = 1, 64
x = np.arange(0, L, L/N)
kk   = 4*pi/L
k =  np.arange(0, N/2 + 1)*(2*pi/L)

# k[2] = 0 # all the pattern is dead at a particular k mode which coooresponds to the signal!!

fk = np.fft.rfft(f1(kk, x))
fk = -k*k*fk
fx = np.fft.irfft(fk)

plt.plot(x, -kk*kk*f1(kk, x), color="#348ABD", label = 'analytical', linewidth=2)
plt.plot(x, fx, 'o', color="#A60628", label = 'numerical')
plt.legend(loc = 'best')
plt.xlabel('x', fontsize=15)
plt.title('derivatives using FFT', fontsize=15)
plt.xlim([0, max(x)]);
```


# Piecewise linear regression

By Sang woo Ham (ecosang@gmail.com), Last edited on 09/09/2021

## Table of Contents

* [Introduction](#Introduction)
* [Piecewise linear regression](#Piecewise-linear-regression)
* [Parameter estimation](#Parameter-estimation)
* [Conclusion and discussion](#Conclusion-and-discussion)
* [References](#References)

# Introduction

This notebook is a simple Python demonstration of [piecewise linear regression](https://online.stat.psu.edu/stat501/lesson/8/8.8). Piecewise linear regression can be used to represent the relationship between outdoor air temperature and whole building energy consumption. This can be viewed as a simple data-driven model of a whole building energy consumption, and it has been widely used for retrofitting application [1,2,3].

In this document, We provide a reproducible Python notebook how to make a piecewise linear regression model so that one can apply the model to the building energy consumption data. First, we generate a toy dataset from the known piecewise linear regression model. Then, we provide an optimization based approach to estimate model parameters of the piecewise linear regression model.

# Piecewise linear regression

[Piecewise linear regression](https://online.stat.psu.edu/stat501/lesson/8/8.8) is a model that is composed of several linear regressions for different intervals of indepedent variables. This is useful when the behavior of depedent variable radically change in different intervals of indepdent variables (e.g., heating and cooling energy consumption with respect to outdoor air temperature).

To generate the toy dataset, we assume we know the true model parameters of a piecewise linear model. The true model is a combination of two linear lines intersecting at $x=10$:

$$
\begin{align}
y &= -x+5 &\text{ if } &x<10 \\
y &= -x+5+3(x-10) &\text{ if } &x\geq10.
\end{align}
$$

This can be written in this format as shown in those references [1,2,3]:

$$y=5-x+3(x-10)^{+},$$

where $()^{+}$ is $0$ when the value in $()$ is less than $0$. Likewise, $()^{-}$ is $0$ when the value in $()$ is greater than $0$.

In real world, we don't have 100% accurate measurement. For the simplicity, all sources of uncertainties (e.g., measurement errors, noise in data generation process, etc.) are modeled as a single noise $\varepsilon$. We assume this noise follows a normal distribution of $\varepsilon_i \sim \text{N}(\mu_{\varepsilon}=0,\sigma_{\varepsilon}=2)$.

With data specific index $i=1,2,...n$:

$$y_{i}=5-x_{i}+3(x_{i}-10)^{+}+\varepsilon_{i}.$$

Below code snippet shows how to generate the data.

```python
# loading required packages.
import numpy as np # numpy for numerical calculation
import matplotlib.pyplot as plt # visualization
%matplotlib inline
```

```python
# fix seed to have consistent random number generation to make this example reproducible.
np.random.seed(1234)

# Data generation
n_data=100
x_data=np.random.uniform(-5,30,n_data) # Create x data in [-5,30]
e_data=np.random.normal(0,2.0,n_data) # e~Normal(0,2)
y_data=np.where(x_data<10,-1*(x_data)+5,-1*x_data+5+3*(x_data-10))+e_data # y data from model(x_data)+e

# Data to visualize the true model
x_grid=np.linspace(-5,30,36)
y_grid=np.where(x_grid<10,-1*(x_grid)+5,-1*x_grid+5+3*(x_grid-10))
```

Below figure shows the generated data and true model in blue x and red line, respectively.

```python
fig, ax =plt.subplots(nrows=1, ncols=1,  figsize=(8,6))
ax.plot(x_data,y_data,'bx',label="Generated data",markersize=5)
ax.plot(x_grid,y_grid,'r-',label="True model",linewidth=0.5)
ax.set_xlabel("$x$",fontsize=12)
ax.set_ylabel("$y$",fontsize=12)
ax.legend(loc="best",fontsize=12)
```

# Parameter estimation

Now, let's have a different view. We have data set (i.e., `x_data` and `y_data` in the above code, which are $\{x_i, y_i\}$ for $i=1,2,...n$). We want to find a piecewise linear regression model that explains the data.

Let's parameterizes a piecewise linear regression in the following format:

$$y_{i}=\beta_0+\beta_{1}x_{i}+\beta_{2}(x_{i}-\beta_{3})^{+}+\varepsilon_{i},$$
where $\varepsilon_{i}\sim \text{N}(0,\sigma_{\varepsilon})$, and the unknown parameters are $\beta0$, $\beta1$, $\beta2$, $\beta3$, and $\sigma_{\varepsilon}$.

There are two types of methods to estimate the parameters: (1) [least-squares method](https://en.wikipedia.org/wiki/Least_squares) and (2) [maximum likelihood estimation (MLE)](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation). MLE method requires the understanding of probabilistic modeling, and both methods give same results for linear regression models with gaussian noise (i.e., $\varepsilon_{i}\sim \text{N}(0,\sigma_{\varepsilon})$). Therefore, we use least-square method in this notebook to estimate parameters. In this case, we don't need to explicitly estimate $\sigma_{\varepsilon}.$

Least-squares method is to find parameters that minimizes the sum of squares of prediction errors. When collecting all the parameters as $\theta=\{\beta_0, \beta_1, \beta_2, \text{ and } \beta_3 \}$, the least-squares soltuion of $\theta^{*}$ is estimated by solving:

$$\theta^{*}=\underset{\theta}{\arg\min} \sum_{i=1}^n \left(y_i-\hat{y}_i \right)^2,$$

where $\hat{y}_{i}=\beta_0+\beta_{1}x_{i}+\beta_{2}(x_{i}-\beta_{3})^{+}$.

This problem can be solved by using least-squares method for typical types of curves (linear or non-linear) implemented in [scipy package](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html). In addition, [numpy package](https://numpy.org/doc/stable/reference/generated/numpy.piecewise.html) provides a function to model the piecewise defined function. Please refer the comments in below code snippet.

```python
# refer these sites
# https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html
# https://numpy.org/doc/stable/reference/generated/numpy.piecewise.html

# loading package
from scipy import optimize

def piecewise_linear(x, beta0, beta1, beta2, beta3):
    condlist = [x < beta3,  x >= beta3] # x<beta3 applies to lambda x: beta0+beta1*x.
    funclist = [lambda x: beta0+beta1*x, lambda x: beta0+beta1*x+beta2*(x-beta3)]
    return np.piecewise(x, condlist, funclist)

# estimate theta* and covariance of theta*
theta , theta_cov = optimize.curve_fit(piecewise_linear, x_data, y_data) #least square
```

Print the results. The least-squares method correctly estimate the parameters. One can notice the results are not exactly identical. This is we have limited number of data. When you increase `n_data=100` in data generation process, the estimation will be more accurate.

```python

# beta0, beta1, beta2, beta3
print("Print results")
print(f'The estimated       beta0*:{round(theta[0],2)}, beta1*: {round(theta[1],2)}, beta2*: {round(theta[2],2)},  and  beta3*: {round(theta[3],2)}.')
print(f'The true parameters beta0: 5.00, beta1 : -1.00, beta2 : 3.00, and  beta3 : 10.00')
```

    Print results
    The estimated       beta0*:5.57, beta1*: -1.11, beta2*: 3.2,  and  beta3*: 10.12.
    The true parameters beta0: 5.00, beta1 : -1.00, beta2 : 3.00, and  beta3 : 10.00

Visualization. Although there is little difference, the method successfully identifies the piecewise linear regression model.

```python
fig, ax =plt.subplots(nrows=1, ncols=1,  figsize=(8,6))
ax.plot(x_data, y_data, "bx",label="Generated data", markersize=5)
ax.plot(x_grid, piecewise_linear(x_grid, *theta),'g-',label='Estimated model',linewidth=1.0)
ax.plot(x_grid,y_grid,'r-',label='True model',linewidth=1.0)
ax.legend(fontsize=12,loc="best")
ax.set_xlabel("$x$",fontsize=12)
ax.set_ylabel("$y$",fontsize=12)
```

# Conclusion and discussion

In this notebook, we provide an introduction of a piecewise linear regression and a way to estimate model parameters from the data. The parameter estimation is achieved by an optimization-based method using `scipy` and `numpy` packages, and this method successfully estimated the model parameters.

The formulation of the piecewise can be flexible. For example, if we want to have 0 slop below $\beta_{3}$, you can simply delete $\beta_{1}$ in the above example. Also, you can add one more change-point (or break point) in the above model. This can be simply achieved by adding `condlist` and `funclist` in the `piecewise_linear` function.

# References

[1] Kissock, J. K., Haberl, J. S., & Claridge, D. E. (2003). Inverse modeling toolkit: numerical algorithms. ASHRAE transactions, 109, 425. [(link)](http://www.eeperformance.org/uploads/8/6/5/0/8650231/ashrae_-_inverse_modeling_toolkit_-_numerical_algorithms.pdf)

[2] ASHRAE. (2014). ASHRAE Guideline 14–2014, Measurement of Energy, Demand, and Water Savings. [(link)](https://www.techstreet.com/standards/guideline-14-2014-measurement-of-energy-demand-and-water-savings?product_id=1888937)

[3] Server, F., Kissock, J. K., Brown, D., & Mulqueen, S. (2011). Estimating industrial building energy savings using inverse simulation. [(link)](https://ecommons.udayton.edu/cgi/viewcontent.cgi?article=1155&context=mee_fac_pub)

```python

```