# Meeting 1 : Formalizing The Problem
Environment : Simple Tic-tac-toe game.\
Goal : Winning the game.\
Actions : choosing a grid from unchoosed grids.\
Reward : For each win, draw, loose, the agent will get 1, 0, -1, respectively.\
States : Each grid in gridworld of our problem has 3 possibilities, chose by enemy or us or It's not chosen yet, hense we have 3 ^ 9 states.

# Meeting 2 : Choosing The Learning Algorithm.
1. Can we represent the value function with a table? No.
2. Are we using average reward? No.
3. Will we learn on each timestep? Yes.
4. Is this a control problem? Yes.\
So we are now down to choose between 3 algorithms : Expected SARSA, Q Learning and SARSA. We use SARSA in this problem.

# Meeting 3 : Agent Architecture Meeting : Overview of Design Choices
## Metaparameter Choices
### Which Function Approximator Will We Use?
Neural Network.
#### Which Activation Function Will We Use?
\begin{align}
        \text{f}(x) = \left\{
        \begin{array}{cl}
        x & x > 0 \\
        0
        \end{array}
        \right.
    \end{align}
#### How Are We Going to Train The Neural Network?
We are going to use the ADAM optimizer.
### Which Exploration Method Will We Use?
Softmax Policy.

```python
from google.colab import files

uploaded = files.upload()

for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn])))
```

<input type="file" id="files-d6ab8997-665c-4573-aaf6-311068361440" name="files[]" multiple disabled
   style="border:none" />
<output id="result-d6ab8997-665c-4573-aaf6-311068361440">
 Upload widget is only available when the cell has been executed in the
 current browser session. Please rerun this cell to enable.
 </output>

    Saving agent.py to agent.py
    Saving environment.py to environment.py
    User uploaded file "agent.py" with length 2041 bytes
    User uploaded file "environment.py" with length 1830 bytes

```python
# Required Packages

import numpy as np
import matplotlib.pyplot as plt
from environment import BaseEnvironment
from BaseOptimizer import BaseOptimizer
from agent import BaseAgent
from copy import deepcopy
import random
```

```python
class Adam(BaseOptimizer):
    def __init__(self):
        pass

    def optimizer_init(self, optimizer_info):
        """Setup for the optimizer.

        Set parameters needed to setup the Adam algorithm.

        Assume optimizer_info dict contains:
        {
            num_states: integer,
            num_hidden_layer: integer,
            num_hidden_units: integer,
            step_size: float,
            self.beta_m: float
            self.beta_v: float
            self.epsilon: float
        }
        """

        self.num_states = optimizer_info.get("num_states")
        self.num_hidden_layer = optimizer_info.get("num_hidden_layer")
        self.num_hidden_units = optimizer_info.get("num_hidden_units")

        # Specify Adam algorithm's hyper parameters
        self.step_size = optimizer_info.get("step_size")
        self.beta_m = optimizer_info.get("beta_m")
        self.beta_v = optimizer_info.get("beta_v")
        self.epsilon = optimizer_info.get("epsilon")

        self.layer_size = np.array([self.num_states, self.num_hidden_units, 1])

        # Initialize Adam algorithm's m and v
        self.m = [dict() for i in range(self.num_hidden_layer + 1)]
        self.v = [dict() for i in range(self.num_hidden_layer + 1)]

        for i in range(self.num_hidden_layer + 1):
            # Initialize self.m[i]["W"], self.m[i]["b"], self.v[i]["W"], self.v[i]["b"] to zero
            self.m[i]["W"] = np.zeros((self.layer_size[i], self.layer_size[i + 1]))
            self.m[i]["b"] = np.zeros((1, self.layer_size[i + 1]))
            self.v[i]["W"] = np.zeros((self.layer_size[i], self.layer_size[i + 1]))
            self.v[i]["b"] = np.zeros((1, self.layer_size[i + 1]))

        # Initialize beta_m_product and beta_v_product to be later used for computing m_hat and v_hat
        self.beta_m_product = self.beta_m
        self.beta_v_product = self.beta_v

    def update_weights(self, weights, g):
        """
        Given weights and update g, return updated weights
        """

        for i in range(len(weights)):
            for param in weights[i].keys():
                # update self.m and self.v
                self.m[i][param] = self.beta_m * self.m[i][param] + (1 - self.beta_m) * g[i][param]
                self.v[i][param] = self.beta_v * self.v[i][param] + (1 - self.beta_v) * (g[i][param] * g[i][param])

                # compute m_hat and v_hat
                m_hat = self.m[i][param] / (1 - self.beta_m_product)
                v_hat = self.v[i][param] / (1 - self.beta_v_product)

                # update weights
                weights[i][param] += self.step_size * m_hat / (np.sqrt(v_hat) + self.epsilon)

        # update self.beta_m_product and self.beta_v_product
        self.beta_m_product *= self.beta_m
        self.beta_v_product *= self.beta_v

        return weights

```

```python
# Tic-Tac-Toe Environment

class TicTacToeEnvironment(BaseEnvironment):
    def __init__(self):
        pass

    def env_init(self, env_info={}):
        """
        Setup for the environment called when the experiment first starts.
        """
        # initialize board
        self.board = np.zeros((3, 3))

    def env_start(self):
        """
        The first method called when the experiment starts, called before the
        agent starts.

        Returns:
            The first state observation from the environment.
        """

        reward = 0.0
        self.board = np.zeros((3, 3))
        state = np.zeros((3, 3))
        actions = self.find_actions(state)
        observation = (state, actions)
        is_terminal = False

        self.reward_obs_term = (reward, observation, is_terminal)

        # return first state observation from the environment
        return self.reward_obs_term[1]

    def env_step(self, action):
        """A step taken by the environment.

        Args:
            action: The action taken by the agent

        Returns:
            (float, state, Boolean): a tuple of the reward, state observation,
                and boolean indicating if it's terminal.
        """

        self.take_action(action)
        self.take_action_opp()
        state = np.copy(self.board)
        actions = self.find_actions(state)
        observation = (state, actions)
        reward, is_terminal = self.get_reward()

        self.reward_obs_term = (reward, observation, is_terminal)

        return self.reward_obs_term

    def find_actions(self, state):
        actions = []
        for i in range(0, 3, 1):
            for j in range(0, 3, 1):
                if state[i, j] == 0:
                    actions.append((i, j))
        return actions

    def get_board(self):
        return self.board

    def set_board(self, board):
        self.board = board

    def take_action(self, action):
        """
            0 represents none chosen grid.
            1 represents agent action.
            2 represents opponent action.
        """
        self.board[int(action[0]), int(action[1])] = 1

    def take_action_opp(self):
        possible_actions = self.find_actions(np.copy(self.board))
        if len(possible_actions) != 0:
            action = random.choice(possible_actions)
            self.board[int(action[0]), int(action[1])] = 2

    def get_reward(self):
        """
            Win : 10
            Draw : 0
            Loose : -10
            Each time step : -1
        """
        reward = -1
        is_terminal = False
        if np.count_nonzero(self.board) == 9:
            is_terminal = True
        for x in self.board:
            if np.array_equal(x, np.ones(3)):
                reward += 10
                is_terminal = True
                return reward, is_terminal
            elif np.array_equal(x, np.full(3, 2)):
                reward += -10
                is_terminal = True
                return reward, is_terminal
        for x in self.board.T:
            if np.array_equal(x, np.ones(3)):
                reward += 10
                is_terminal = True
                return reward, is_terminal
            elif np.array_equal(x, np.full(3, 2)):
                reward += -10
                is_terminal = True
                return reward, is_terminal
        if np.array_equal(np.diagonal(self.board), np.full(3, 1)) or np.array_equal(np.fliplr(self.board).diagonal(),
                                                                                np.full(3, 1)):
            reward += 10
            is_terminal = True
            return reward, is_terminal
        elif np.array_equal(np.diagonal(self.board), np.full(3, 2)) or np.array_equal(np.fliplr(self.board).diagonal(),
                                                                                np.full(3, 2)):
            reward += -10
            is_terminal = True
            return reward, is_terminal
        return reward, is_terminal
```

```python
class StateValueNetwork:
    def __init__(self, network_config):
        # number of states
        self.num_states = network_config.get("num_states")
        # number of hidden layers
        self.num_hidden_layer = network_config.get("num_hidden_layer")
        # number of hidden units
        self.num_hidden_units = network_config.get("num_hidden_units")
        # discount
        self.discount = network_config['gamma']

        self.rand_generator = np.random.RandomState(network_config.get("seed"))

        # layer size : NN
        self.layer_size = np.array([self.num_states, self.num_hidden_units, 1])

        # Initialize the neural network's parameter
        self.weights = [dict() for i in range(self.num_hidden_layer + 1)]
        for i in range(self.num_hidden_layer + 1):
            ins, outs = self.layer_size[i], self.layer_size[i + 1]
            self.weights[i]['W'] = self.rand_generator.normal(0, np.sqrt(2 / ins), (ins, outs))
            self.weights[i]['b'] = self.rand_generator.normal(0, np.sqrt(2 / ins), (1, outs))

    def get_value(self, s):
        """
        Compute value of input s given the weights of a neural network
        """

        # DONE!

        psi = self.my_matmul(s, self.weights[0]["W"]) + self.weights[0]["b"]
        x = np.maximum(psi, 0)
        v = self.my_matmul(x, self.weights[1]["W"]) + self.weights[1]["b"]

        return v

    def get_gradient(self, s):
        """
        Given inputs s and weights, return the gradient of v with respect to the weights
        """

        # DONE!

        grads = [dict() for i in range(len(self.weights))]
        x = np.maximum(self.my_matmul(s, self.weights[0]["W"]) + self.weights[0]["b"], 0)
        grads[0]["W"] = self.my_matmul(s.T, (self.weights[1]["W"].T * (x > 0)))
        grads[0]["b"] = self.weights[1]["W"].T * (x > 0)
        grads[1]["W"] = x.T
        grads[1]["b"] = 1

        return grads

    def get_action_values(self, observation):
        # find values of each next state
        state = observation[0]
        possible_actions = observation[1]

        q_values = np.zeros((len(possible_actions), 2))

        for i in range(len(possible_actions)):
            temp = state
            temp[possible_actions[i][0], possible_actions[i][1]] = 1
            state_vec = self.one_hot(self.generate_hash(temp), self.num_states)
            v_s = self.get_value(state_vec)
            action = 3 * possible_actions[i][0] + possible_actions[i][1]

            # POSSIBLE FAULT
            q_values[i] = (action, -1 + self.discount * v_s)

        return q_values

    def one_hot(self, state, num_states):
        """
        Given num_state and a state, return the one-hot encoding of the state
        """

        # DONE!

        one_hot_vector = np.zeros((1, num_states))
        one_hot_vector[0, int((state - 1))] = 1

        return one_hot_vector

    def generate_hash(self, m):
        my_hash = 0
        for i in range(0, 3, 1):
            for j in range(0, 3, 1):
                my_hash = my_hash * 3 + int(m[i][j])
        return my_hash

    def my_matmul(self, x1, x2):
        """
        Given matrices x1 and x2, return the multiplication of them
        """

        result = np.zeros((x1.shape[0], x2.shape[1]))
        x1_non_zero_indices = x1.nonzero()
        if x1.shape[0] == 1 and len(x1_non_zero_indices[1]) == 1:
            result = x2[x1_non_zero_indices[1], :]
        elif x1.shape[1] == 1 and len(x1_non_zero_indices[0]) == 1:
            result[x1_non_zero_indices[0], :] = x2 * x1[x1_non_zero_indices[0], 0]
        else:
            result = np.matmul(x1, x2)
        return result

    def get_weights(self):
        """
        Returns:
            A copy of the current weights of this network.
        """
        return deepcopy(self.weights)

    def set_weights(self, weights):
        """
        Args:
            weights (list of dictionaries): Consists of weights that this network will set as its own weights.
        """
        self.weights = deepcopy(weights)
```

```python
class Agent(BaseAgent):
    def __init__(self):
        self.name = "expected_sarsa_agent"

    def agent_init(self, agent_config):

        # DONE!

        """Setup for the agent called when the experiment first starts.

        Set parameters needed to setup the agent.

        Assume agent_config dict contains:
        {
            optimizer_config: dictionary : {
                "num_states",
                "num_hidden_layer",
                "num_hidden_units",
                "step_size",
                "beta_m",
                "beta_v",
                "epsilon"},
            network_config : dictionary : {
                "num_states",
                "num_hidden_layer",
                "num_hidden_units",
                "discount_factor"
            }
            discount_factor: float,
        }
        """
        self.rand_generator = np.random.RandomState(agent_config.get("seed"))

        # optimizer
        self.optimizer = Adam()
        self.optimizer.optimizer_init(agent_config["optimizer_config"])
        # network
        self.network = StateValueNetwork(agent_config["network_config"])
        # discount
        self.discount = agent_config['gamma']
        # tau
        self.tau = agent_config['tau']
        # number of states
        self.num_states = agent_config.get("num_states")
        # number of hidden layers : NN
        self.num_hidden_layer = agent_config.get("num_hidden_layer")
        # number of hidden units : NN
        self.num_hidden_units = agent_config.get("num_hidden_units")

        # layer size : NN
        self.layer_size = np.array([self.num_states, self.num_hidden_units, 1])

        # Initialize the neural network's parameter
        self.weights = [dict() for i in range(self.num_hidden_layer + 1)]
        for i in range(self.num_hidden_layer + 1):
            ins, outs = self.layer_size[i], self.layer_size[i + 1]
            self.weights[i]['W'] = self.rand_generator.normal(0, np.sqrt(2 / ins), (ins, outs))
            self.weights[i]['b'] = self.rand_generator.normal(0, np.sqrt(2 / ins), (1, outs))

        self.last_observation = None
        self.last_action = None

        self.sum_rewards = 0
        self.episode_steps = 0

    def policy(self, observation):

        # DONE!

        """
        Args:
            state (Numpy array): the state.
        Returns:
            the action.
        """
        action_values = self.network.get_action_values(observation)
        probs_batch = self.softmax(action_values, self.tau)
        try:
            index = self.rand_generator.choice(len(observation[1]), p=probs_batch[:, 1].squeeze())
        except:
            index = 0
        action_num = action_values[index, 0]
        y = action_num % 3
        x = action_num // 3
        return (x, y)

    def agent_start(self, observation):

        # DONE!

        """The first method called when the experiment starts, called after
        the environment starts.
        Args:
            state (Numpy array): the state from the
                environment's evn_start function.
        Returns:
            The first action the agent takes.
        """
        self.sum_rewards = 0
        self.episode_steps = 0
        self.last_observation = (np.copy(observation[0]), observation[1])
        self.last_action = self.policy(self.last_observation)
        return self.last_action

    def agent_step(self, reward, observation):
        """A step taken by the agent.
        Args:
            reward (float): the reward received for taking the last action taken
            state (Numpy array): the state from the
                environment's step based, where the agent ended up after the
                last step
        Returns:
            The action the agent is taking.
        """
        self.sum_rewards += reward
        self.episode_steps += 1

        # Delta for expected sarsa
        last_state_hash = self.network.generate_hash(self.last_observation[0])
        last_state_vec = self.network.one_hot(last_state_hash, self.num_states)
        last_value = self.network.get_value(last_state_vec)

        state_hash = self.network.generate_hash(np.copy(observation[0]))
        state_vec = self.network.one_hot(state_hash, self.num_states)
        value = self.network.get_value(state_vec)

        delta = reward + self.discount * value - last_value

        grads = self.network.get_gradient(last_state_vec)

        g = [dict() for i in range(self.num_hidden_layer + 1)]
        for i in range(self.num_hidden_layer + 1):
            for param in self.weights[i].keys():
                g[i][param] = delta * grads[i][param]

        weights = self.optimizer.update_weights(self.weights, g)

        self.network.set_weights(weights)

        # Select action
        action = self.policy((np.copy(observation[0]), observation[1]))

        # Update the last state and last action.
        self.last_observation = (np.copy(observation[0]), observation[1])
        self.last_action = action

        return (int(action[0]), int(action[1]))

    def agent_end(self, reward):
        """Run when the agent terminates.
        Args:
            reward (float): the reward the agent received for entering the
                terminal state.
        """
        self.sum_rewards += reward
        self.episode_steps += 1

        last_state_hash = self.network.generate_hash(self.last_observation[0])
        last_state_vec = self.network.one_hot(last_state_hash, self.num_states)
        last_value = self.network.get_value(last_state_vec)
        delta = reward - last_value

        grads = self.network.get_gradient(last_state_vec)

        g = [dict() for i in range(self.num_hidden_layer + 1)]
        for i in range(self.num_hidden_layer + 1):
            for param in self.weights[i].keys():
                g[i][param] = delta * grads[i][param]

        weights = self.optimizer.update_weights(self.weights, g)

        self.network.set_weights(weights)

    def agent_message(self, message):
        if message == "get_sum_reward":
            return self.sum_rewards
        else:
            raise Exception("Unrecognized Message!")

    def softmax(self, action_values, tau=1.0):
        """
        Args:
            action_values (Numpy array): A 2D array of shape (batch_size, num_actions).
                        The action-values computed by an action-value network.
            tau (float): The temperature parameter scalar.
        Returns:
            A 2D array of shape (batch_size, num_actions). Where each column is a probability distribution over
            the actions representing the policy.
        """

        # Compute the preferences by dividing the action-values by the temperature parameter tau
        preferences = action_values[:, 1] / tau
        # Compute the maximum preference across the actions
        try:
            max_preference = np.max(preferences)
        except:
            print(action_values)
            raise Error

        # Reshape max_preference array which has shape [Batch,] to [Batch, 1]. This allows NumPy broadcasting
        # when subtracting the maximum preference from the preference of each action.
        reshaped_max_preference = max_preference.reshape((-1, 1))

        # Compute the numerator, i.e., the exponential of the preference - the max preference.
        exp_preferences = np.exp(preferences - reshaped_max_preference)
        # Compute the denominator, i.e., the sum over the numerator along the actions axis.
        sum_of_exp_preferences = np.sum(exp_preferences)

        # Reshape sum_of_exp_preferences array which has shape [Batch,] to [Batch, 1] to  allow for NumPy broadcasting
        # when dividing the numerator by the denominator.
        reshaped_sum_of_exp_preferences = sum_of_exp_preferences.reshape((-1, 1))

        # Compute the action probabilities according to the equation in the previous cell.
        probs = exp_preferences / reshaped_sum_of_exp_preferences

        # squeeze() removes any singleton dimensions. It is used here because this function is used in the
        # agent policy when selecting an action (for which the batch dimension is 1.) As np.random.choice is used in
        # the agent policy and it expects 1D arrays, we need to remove this singleton batch dimension.
        probs = probs.squeeze()

        action_probs = np.zeros((len(action_values), 2))
        for i in range(len(action_probs)):
            action_probs[i, 0] = action_values[i, 0]
            try:
                action_probs[i, 1] = probs[i]
            except:
                action_probs[i, 1] = 1.0

        return action_probs
```

```python
# Train

agent_config = {
    "optimizer_config" : {
        "num_states" : 3 ** 9,
        "num_hidden_layer" : 1,
        "num_hidden_units" : 3 ** 6,
        "step_size" : 3e-5,
        "beta_m": 0.9,
        "beta_v": 0.999,
        "epsilon": 1e-8
    },
    "network_config" : {
        "num_states" : 3 ** 9,
        "num_hidden_layer" : 1,
        "num_hidden_units" : 3 ** 6,
        'gamma': 0.99
    },
    "num_states" : 3 ** 9,
    "num_hidden_layer" : 1,
    "num_hidden_units" : 3 ** 6,
    'replay_buffer_size': 32,
    'minibatch_sz': 32,
    'num_replay_updates_per_step': 4,
    'gamma': 0.99,
    'tau': 1000.0,
    'seed': 0
}

avg_rewards = np.zeros(10)

agent = Agent()
agent.agent_init(agent_config)

env = TicTacToeEnvironment()
env.env_init()
for j in range(len(avg_rewards)):
    print("Progress : " + str(j * 10) + "%.")
    rewards = np.zeros(300)
    for i in range(len(rewards)):
        obs = env.env_start()
        action = agent.agent_start(obs)
        reward_obs_term = env.env_step(action)
        while reward_obs_term[2] != True:
            action = agent.agent_step(reward_obs_term[0], reward_obs_term[1])
            reward_obs_term = env.env_step(action)
        agent.agent_end(reward_obs_term[0])
        rewards[i] = agent.sum_rewards
    avg_rewards[j] = np.average(rewards)

x_axis = np.zeros(10)
for i in range(len(x_axis)):
    x_axis[i] = i + 1
plt.plot(x_axis, avg_rewards)
plt.show()
```

```python
plt.plot(x_axis, avg_rewards)
plt.xlabel("Runs (Each run = 300 episodes)")
plt.ylabel("Average Reward")
plt.show()

print("AVERAGE POINTS : " + str(np.average(avg_rewards)))
```

Results after 3000 episodes.


# Evapotranspiration - exercises

- toc: true
- badges: true
- comments: false
- categories: [jupyter]

We will calculate the evapotranspiration using two methods: Thornthwaite and Penman.

## Download data from the IMS

Go to the Israel Meteorological Service website, and download the following data:
1. [hourly data](https://ims.data.gov.il/he/ims/1)
 * on the first page, choose all options and press continue.
 * on the next page, choose the following date range: 01/01/2020 to 01/01/2021, then press continue.
 * Choose station Bet Dagan (בית דגן 2523), then select (בחר), then continue.
 * Choose option "by station" (לפי תחנות), then produce report.
 * Download report as csv, call it "bet-dagan-3h.csv".
2. [daily data](https://ims.data.gov.il/he/ims/2)
 * on the first page, choose all options and press continue.
 * on the next page, choose the following date range: 01/01/2020 to 01/01/2021, then press continue.
 * Choose station Bet Dagan Meuyeshet (בית דגן מאוישת 2520), then select (בחר), then continue.
 * Choose option "by station" (לפי תחנות), then produce report.
 * Download report as csv, call it "bet-dagan-day-pan.csv".
3. [radiation data](https://ims.data.gov.il/he/ims/6)
 * on the first page, choose all options, **then on the bottom right option "radiation" (קרינה), choose kJ/m2**, and then press continue.
 * on the next page, choose the following date range: 01/01/2020 to 01/01/2021, then press continue.
 * Choose station Bet Dagan Krina (בית דגן קרינה 2524), then select (בחר), then continue.
 * Choose option "by station" (לפי תחנות), then produce report.
 * Download report as csv, call it "bet-dagan-radiation.csv".

## Import relevant packages

```python
#collapse-hide

import matplotlib.pyplot as plt
import matplotlib
import numpy as np
import pandas as pd
from pandas.plotting import register_matplotlib_converters
register_matplotlib_converters()  # datetime converter for a matplotlib
import seaborn as sns
sns.set(style="ticks", font_scale=1.5)
```

## import hourly data

```python
#collapse-hide

df = pd.read_csv('bet-dagan-3h.csv', encoding = 'unicode_escape', na_values=["-"])
# find out what hebrew gibberish means: http://www.pixiesoft.com/flip/
name_conversion_dictionary = {"ùí úçðä": "station name",
                              "îñôø úçðä": "station number",
                              "úàøéê": "Date",
                              "ùòä-LST": "LST time",
                              "èîôøèåøä(C°)": "T",
                              "èîôøèåøä ìçä(C°)": "wet-bulb temperature (°C)",
                              "èîôøèåøú ð÷åãú äèì(C°)": "dew_point_T",
                              "ìçåú éçñéú(%)": "relative humidity (%)",
                              "îäéøåú äøåç(m/s)": "wind_speed",
                              "ëéååï äøåç(îòìåú)": "wind direction (degrees)",
                              "ìçõ áâåáä äúçðä(hPa)": "Pressure",
                              "ìçõ áâåáä ôðé äéí(hPa)": "pressure at sea level (hPa)",
                              """äúàãåú éåîéú îâéâéú ñåâ à'(î"î)""": "pan evaporation (mm)",
                              "ñåâ ÷øéðä()": "radiation type",
                             }
# units
# T = temperature (°C)
# dew_point_T = dew point temperature (°C)
# wind_speed = wind speed (m/s)
# Pressure = pressure at station height (hPa = 0.1 kPa)

df = df.rename(columns=name_conversion_dictionary)
df['timestamp'] = df['Date'] + ' ' + df['LST time']
df['timestamp'] = pd.to_datetime(df['timestamp'], dayfirst=True)
df = df.set_index('timestamp')
df
```

      station name
      station number
      Date
      LST time
      T
      wet-bulb temperature (°C)
      dew_point_T
      relative humidity (%)
      wind_speed
      wind direction (degrees)
      ...
      pressure at sea level (hPa)
      ëîåú òððéí ëåììú(÷åã)
      ëîåú òððéí ðîåëéí(÷åã)
      âåáä áñéñ òððéí ðîåëéí(÷åã)
      ñåâ äòððéí äðîåëéí(÷åã)
      ñåâ äòððéí äáéðåðééí(÷åã)
      ñåâ äòððéí äâáåäéí(÷åã)
      îæâ àååéø ðåëçé(÷åã)
      îæâ àååéø ùçìó(÷åã)
      øàåú àô÷éú(÷åã)

      timestamp

      2020-01-01 02:00:00
      áéú ãâï                                       ...
      2523
      01-01-2020
      02:00
      7.9
      7.2
      6.4
      90
      1.7
      117.0
      ...
      1018.8
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2020-01-01 05:00:00
      áéú ãâï                                       ...
      2523
      01-01-2020
      05:00
      7.5
      7.0
      6.4
      93
      1.2
      116.0
      ...
      1018.1
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2020-01-01 08:00:00
      áéú ãâï                                       ...
      2523
      01-01-2020
      08:00
      8.6
      8.3
      8.0
      96
      1.1
      107.0
      ...
      1018.2
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2020-01-01 11:00:00
      áéú ãâï                                       ...
      2523
      01-01-2020
      11:00
      15.9
      13.1
      10.6
      71
      2.4
      196.0
      ...
      1017.4
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2020-01-01 14:00:00
      áéú ãâï                                       ...
      2523
      01-01-2020
      14:00
      18.1
      14.0
      10.4
      61
      2.8
      264.0
      ...
      1015.3
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      2021-01-31 11:00:00
      áéú ãâï                                       ...
      2523
      31-01-2021
      11:00
      19.0
      13.7
      8.9
      52
      5.6
      235.0
      ...
      1017.3
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2021-01-31 14:00:00
      áéú ãâï                                       ...
      2523
      31-01-2021
      14:00
      19.2
      14.7
      11.0
      59
      4.6
      252.0
      ...
      1016.7
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2021-01-31 17:00:00
      áéú ãâï                                       ...
      2523
      31-01-2021
      17:00
      18.2
      14.8
      12.2
      68
      0.8
      203.0
      ...
      1017.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2021-01-31 20:00:00
      áéú ãâï                                       ...
      2523
      31-01-2021
      20:00
      13.1
      12.3
      11.7
      91
      1.2
      79.0
      ...
      1018.2
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2021-01-31 23:00:00
      áéú ãâï                                       ...
      2523
      31-01-2021
      23:00
      10.8
      10.6
      10.3
      97
      1.7
      111.0
      ...
      1018.9
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

3172 rows × 21 columns

## import daily data with pan evaporation

```python
#collapse-hide

df2 = pd.read_csv('bet-dagan-day-pan.csv', encoding = 'unicode_escape', na_values=["-"])
df2 = df2.rename(columns=name_conversion_dictionary)
df2['Date'] = pd.to_datetime(df2['Date'], dayfirst=True)
df2 = df2.set_index('Date')
df2
```

      station name
      station number
      èîôøèåøú î÷ñéîåí(C°)
      èîôøèåøú îéðéîåí(C°)
      èîôøèåøú îéðéîåí ìéã ä÷ø÷ò(C°)
      îùê æäéøú ùîù(ã÷åú)
      pan evaporation (mm)
      ÷åã äúàãåú éåîéú()

      Date

      2020-01-01
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      0.8
      0.0

      2020-01-02
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2020-01-03
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2020-01-04
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2020-01-05
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      2.4
      0.0

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      2021-01-27
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      2.5
      0.0

      2021-01-28
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      1.2
      0.0

      2021-01-29
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2021-01-30
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2021-01-31
      áéú ãâï îàåéùú                                ...
      2520
      NaN
      NaN
      NaN
      NaN
      2.6
      0.0

397 rows × 8 columns

## import daily data with radiation

```python
#collapse-hide

df3 = pd.read_csv('bet-dagan-radiation.csv', encoding = 'unicode_escape', na_values=["-"])
df3 = df3.rename(columns=name_conversion_dictionary)
df3['Date'] = pd.to_datetime(df3['Date'], dayfirst=True)
df3 = df3.set_index('Date')
df3 = df3.replace({"éùéøä": "direct",
                   "îôåæøú": "diffuse",
                   "âìåáàìéú": "global"})
df3['daily_radiation_MJ_per_m2_per_day'] = df3.iloc[:, 3:].sum(axis=1)/1000
df_radiation = df3.loc[df3["radiation type"] == "global", "daily_radiation_MJ_per_m2_per_day"].to_frame()
df_radiation
```

      daily_radiation_MJ_per_m2_per_day

      Date

      2020-01-01
      10.0296

      2020-01-02
      4.3128

      2020-01-03
      11.6748

      2020-01-04
      1.6452

      2020-01-05
      6.8544

      ...
      ...

      2021-01-27
      12.2652

      2021-01-28
      7.1640

      2021-01-29
      7.2936

      2021-01-30
      9.3276

      2021-01-31
      13.5468

396 rows × 1 columns

## Part 1: Thornthwaite estimation

$$
\begin{equation}
E = 16\left[ \frac{10\,T^\text{monthly mean}}{I} \right]^a,
\end{equation}
$$
where
$$
\begin{equation}
I = \sum_{i=1}^{12} \left[ \frac{T_i^\text{monthly mean}}{5} \right]^{1.514},
\end{equation}
$$
and
$$
\begin{align}
a &= 6.75\times 10^{-7}I^3 \\
   &- 7.71\times 10^{-5}I^2 \nonumber\\
   &+ 1.792\times 10^{-2}I \nonumber\\
   &+ 0.49239 \nonumber
\end{align}
$$

 - $E$ is the monthly potential ET (mm)
 - $T_\text{monthly mean}$ is the mean monthly temperature in °C
 - $I$ is a heat index
 - $a$ is a location-dependent coefficient

From df, make a new dataframe, df_th, that stores monthly temperatures means. Use `resample` function.

```python
#collapse-hide

# monthly data
df_th = (df['T'].resample('MS')  # MS assigns mean to first day in the month
                .mean()
                .to_frame()
        )
# we now add 14 days to the index, so that all monthly data is in the middle of the month
# not really necessary, makes plot look better
df_th.index = df_th.index + pd.DateOffset(days=14)
df_th
```

      T

      timestamp

      2020-01-15
      12.484274

      2020-02-15
      14.046983

      2020-03-15
      16.439113

      2020-04-15
      18.512500

      2020-05-15
      23.166532

      2020-06-15
      24.600000

      2020-07-15
      27.353226

      2020-08-15
      28.090323

      2020-09-15
      28.462500

      2020-10-15
      25.120161

      2020-11-15
      19.308475

      2020-12-15
      15.916129

      2021-01-15
      14.123790

Calculate $I$, then $a$, and finally $E_p$. Add $E_p$ as a new column in df_th.

```python
#collapse-hide

# Preparing "I" for the Thornthwaite equation
I = np.sum( (df_th['T']/5)**(1.514) )

# Preparing "a" for the Thornthwaite equation
a = (+6.75e-7 * I**3
     -7.71e-5 * I**2
     +1.792e-2 * I
     + 0.49239)

# The final Thornthwaite model for monthly potential ET (mm)
df_th['Ep'] = 16*((10*df_th['T']/I)**a)
df_th
```

      T
      Ep

      timestamp

      2020-01-15
      12.484274
      20.163427

      2020-02-15
      14.046983
      27.179636

      2020-03-15
      16.439113
      40.472053

      2020-04-15
      18.512500
      54.671821

      2020-05-15
      23.166532
      96.461219

      2020-06-15
      24.600000
      112.296873

      2020-07-15
      27.353226
      146.898516

      2020-08-15
      28.090323
      157.128632

      2020-09-15
      28.462500
      162.453109

      2020-10-15
      25.120161
      118.406386

      2020-11-15
      19.308475
      60.820862

      2020-12-15
      15.916129
      37.291178

      2021-01-15
      14.123790
      27.557481

Plot the Thornthwaite ET that you calculated.

```python
#collapse-hide

fig, ax = plt.subplots(1, figsize=(10,7))
ax.plot(df_th['Ep'])
ax.set(xlabel="date",
       ylabel=r"$E_p$ (mm)",
       title="Thornthwaite potential evapotranspiration");
```

## Part 2: Penman

The Penman model is almost entirely a theory based formula for predicting evaporative flux. It can run on a much finer timescale, and requires a much wider variety of data than most models. In addition to temperature, the Penman functions on measurements of radiation, wind speed, elevation above sea level, vapour-pressure deficit, and heat flux density to the ground.

$$
\begin{equation}
E = \frac{1}{\lambda}\left[ \frac{\Delta}{\Delta+\gamma}Q_{ne}+ \frac{\gamma}{\Delta+\gamma}E_A \right],
\end{equation}
$$

where $Q_n$ is the available energy flux density

$$
\begin{equation}
Q_n = R_n - G,
\end{equation}
$$

and $E_A$ is the drying power of the air

$$
\begin{equation}
E_A = 6.43\cdot f(u)\cdot\text{VPD}.
\end{equation}
$$

$$
  \begin{equation}
    \gamma = \frac{c_p\, P}{\lambda\cdot MW_\text{ratio}}
  \end{equation}
$$

  $$
  \begin{equation}
    P = 101.3-0.01055 H
  \end{equation}
  $$

  $$
  \begin{equation}
    \lambda = 2.501 - 2.361\times 10^{-3}\,T
  \end{equation}
  $$

* $MW_\text{ratio}=0.622$: ratio molecular weight of water vapor/dry air
* $P$: atmospheric pressure (kPa). Can be either measured or inferred from station height above sea level (m).
* $\lambda$: latent heat of water vaporization (MJ kg$^{-1}$)

$$R_n = (1-\alpha)R_s\!\! \downarrow -R_b \!\! \uparrow,$$

where $\alpha$ (dimensionless) is the albedo.
The net outgoing thermal radiation $R_b$ is given by

$$R_b = \left( a\frac{R_s}{R_{so}+b} \right)R_{bo},$$

where $R_{so}$ is the solar radiation on a cloudless day, and it depends on latitude and day of the year.
$R_{bo}$ is given by

$$R_{bo} = \epsilon\, \sigma\, T^4_{Kelvin},$$

where $\sigma=4.903\times 10^{-9}$ MJ m$^{-2}$ d$^{-1}$ K$^{-4}$, and $\epsilon$ is net net emissivity:

$$\epsilon=-0.02+0.261 \exp\left(-7.77\times10^{-4}T_{Celcius}^2\right).$$

The parameters $a$ and $b$ are determined for the climate of the area:
* $a=1.0$, $b=0.0$ for humid areas,
* $a=1.2$, $b=-0.2$ for arid areas,
* $a=1.1$, $b=-0.1$ for semihumid areas.

  $$
  \begin{equation}
    G = 4.2\frac{T_{i+1}-T_{i-1}}{\Delta t}
  \end{equation}
  $$

$$\text{VPD} = e_s - e_d.$$

For temperatures ranging from 0 to 50 °C, the saturation vapor pressure can be calculated with

$$
  \begin{equation}
    e_s = \exp \left[ \frac{16.78\, T -116.9}{T+237.3} \right],
  \end{equation}
$$

and the actual vapor pressure is given by

$$
  \begin{equation}
    e_d = e_s \frac{RH}{100},
  \end{equation}
$$

$$
  \begin{equation}
    \Delta = \frac{\text{d} e_s}{\text{d}T} = e_s(T)\cdot \frac{4098.79}{(T+237.3)^2}.
  \end{equation}
$$

$$
  \begin{equation}
    f(u) = 0.26(1.0 + 0.54\, u_2)
  \end{equation}
$$

The various components of the equations above are:

  $$
  \begin{equation}
    \Delta = 0.200 \cdot (0.00738\,T + 0.8072)^7 - 0.000116
  \end{equation}
  $$

  $$
  \begin{equation}
    \gamma = \frac{c_p\, P}{0.622 \lambda}
  \end{equation}
  $$

  $$
  \begin{equation}
    P = 101.3-0.01055 H
  \end{equation}
  $$

  $$
  \begin{equation}
    \lambda = 2.501 - 2.361\times 10^{-3}\,T
  \end{equation}
  $$

  $$
  \begin{equation}
    f_e(u) = 1.0 + 0.53\, u_2
  \end{equation}
  $$

  $$
  \begin{equation}
    G = 4.2\frac{T_{i+1}-T_{i-1}}{\Delta t}
  \end{equation}
  $$

  $$
  \begin{equation}
    e_s = \exp \left[ \frac{16.78\, T -116.9}{T+237.3} \right]
  \end{equation}
  $$

  $$
  \begin{equation}
    e_d = e_s \frac{RH}{100}
  \end{equation}
  $$
where $\Delta t$ is the time *in days* between midpoints of time periods $i+1$ and $i−1$, and $T$ is the air temperature (°C).

 - $\Delta$: slope of the saturation water vapor pressure curve (kPa °C$^{-1}$)
 - $\gamma$: psychrometric constant (kPA °C$^{-1}$)
 - $c_p=0.001013$: specific heat of water at constant pressure (MJ kg$^{-1}$ °C$^{-1}$)
 - $P$: atmospheric pressure (kPa)
 - $H$: elevation above sea level (m)
 - $\lambda$: latent heat of vaporization (MJ kg$^{-1}$)
 - $R_n$: net radiation (MJ m$^{-2} d^{-1}$)
 - $G$: heat flux density to the ground (MJ m$^{-2} d^{-1}$)
 - $u_{2}$: wind speed measured 2 m above ground (m s$^{-1}$)
 - $e_{s} - e_{d}$: vapor pressure deficit (kPa)
 - $e_{s}$: saturation vapor pressure (kPa)
 - $e_{d}$: actual vapor pressure (kPa)

Calculate daily means for the following columns: temperature `T`, wind speed `wind_speed`, atmospheric pressure `Pressure`, and relative humidity `relative humidity (%)`. Remember that pressure data was given in hectopascal, 1 hPa = 0.1 kPa. Store all the calculated values in a new dataframe, called `df_pen`.

```python
#collapse-hide

# Resampling hourly data over same day and taking mean, to obtain daily averages
df_pen = (df['T'].resample('D')
                 .mean()
                 .to_frame()
         )
df_pen['dew_point'] = (df['dew_point_T'].resample('D')
                                        .mean()
                      )
df_pen['u'] = (df['wind_speed'].resample('D')
                               .mean()
              )
df_pen['P'] = (df['Pressure'].resample('D')
                             .mean()
              )/10
df_pen['RH'] = (df['relative humidity (%)'].resample('D')
                                           .mean()
               )
df_pen
```

      T
      dew_point
      u
      P
      RH

      timestamp

      2020-01-01
      12.3625
      9.0625
      1.5250
      101.30875
      81.500

      2020-01-02
      11.9750
      9.8250
      1.9250
      101.20125
      87.000

      2020-01-03
      13.0500
      4.9750
      5.1750
      101.37125
      58.500

      2020-01-04
      10.8625
      6.6875
      5.5625
      101.15500
      78.375

      2020-01-05
      12.9375
      9.2125
      4.5625
      101.23625
      79.125

      ...
      ...
      ...
      ...
      ...
      ...

      2021-01-27
      13.8125
      8.2375
      1.8875
      100.83750
      72.375

      2021-01-28
      14.4000
      10.2250
      3.5250
      101.12750
      76.750

      2021-01-29
      12.3500
      7.9125
      5.0250
      101.22125
      75.250

      2021-01-30
      12.9625
      7.6500
      4.4250
      101.49500
      71.375

      2021-01-31
      15.0625
      8.3125
      3.7500
      101.32500
      66.000

397 rows × 5 columns

With average $T$ for every day of the year, we can now calculate daily latent heat of vaporization $\lambda$, the slope of the saturation-vapor pressure-temperature curve $\Delta$, and the heat flux density to the ground $G$. Add each of these to dataframe `df_pen`.

Calculate also the wind function using the data for wind speed, and add this to `df_pen`.

```python
#collapse-hide

def lambda_latent_heat(T):
    """daily latent heat of vaporization (MJ/kg)"""
    return 2.501 - 2.361e-3*T

def Delta(T):
    """slope of saturation-vapor curve (kPa/°C)"""
    return 0.2000*(0.00738*T + 0.8072)**7 - 0.000116

def G(T):
    """heat flux density to the ground, G (MJ/m2/d)"""
    return 4.2*np.gradient(T.values)

cp = 0.001013  # (MJ kg−1 °C−1)
df_pen['lambda'] = lambda_latent_heat(df_pen['T'])
df_pen['Delta'] = Delta(df_pen['T'])
df_pen['G'] = G(df_pen['T'])
df_pen['gamma'] = (cp*df_pen['P'])/(0.622*df_pen['lambda'])
df_pen['f_wind'] = 1.0 + 0.53 * df_pen['u']
df_pen
```

      T
      dew_point
      u
      P
      RH
      lambda
      Delta
      G
      gamma
      f_wind

      timestamp

      2020-01-01
      12.3625
      9.0625
      1.5250
      101.30875
      81.500
      2.471812
      0.094385
      -1.62750
      0.066750
      1.808250

      2020-01-02
      11.9750
      9.8250
      1.9250
      101.20125
      87.000
      2.472727
      0.092300
      1.44375
      0.066654
      2.020250

      2020-01-03
      13.0500
      4.9750
      5.1750
      101.37125
      58.500
      2.470189
      0.098185
      -2.33625
      0.066835
      3.742750

      2020-01-04
      10.8625
      6.6875
      5.5625
      101.15500
      78.375
      2.475354
      0.086530
      -0.23625
      0.066553
      3.948125

      2020-01-05
      12.9375
      9.2125
      4.5625
      101.23625
      79.125
      2.470455
      0.097554
      4.25250
      0.066739
      3.418125

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      2021-01-27
      13.8125
      8.2375
      1.8875
      100.83750
      72.375
      2.468389
      0.102551
      4.77750
      0.066532
      2.000375

      2021-01-28
      14.4000
      10.2250
      3.5250
      101.12750
      76.750
      2.467002
      0.106028
      -3.07125
      0.066760
      2.868250

      2021-01-29
      12.3500
      7.9125
      5.0250
      101.22125
      75.250
      2.471842
      0.094317
      -3.01875
      0.066691
      3.663250

      2021-01-30
      12.9625
      7.6500
      4.4250
      101.49500
      71.375
      2.470396
      0.097694
      5.69625
      0.066911
      3.345250

      2021-01-31
      15.0625
      8.3125
      3.7500
      101.32500
      66.000
      2.465437
      0.110070
      8.82000
      0.066933
      2.987500

397 rows × 10 columns

It's time to calculate net radiation $R_n$. The monthly mean solar radiation $R_{so}$ for latitude 30 degrees is
`[17.46, 21.65, 25.96, 29.85, 32.11, 33.20, 32.66, 30.44, 26.67, 22.48, 18.30, 16.04]` (MJ m$^{-2}$ d$^{-1}$)
(Israel's latitude is ~ 31 degrees).

* Add a new column `Rso_monthly` to `df_pen`, where each day has the appropriate $R_{so}$ given by the data above.
* Add a new columns `Rs` with the global radiation data imported in the 3rd file.

```python
#collapse-hide

# Rso: mean solar radiation from a cloudless sky (based on latitude)
# MJ/m2/d
Rso_monthly = np.array([17.46, 21.65, 25.96, 29.85,
                        32.11, 33.20, 32.66, 30.44,
                        26.67, 22.48, 18.30, 16.04])

# create empty columns
df_pen["Rso_monthly"] = ""

# every day in the month will have the same values for Rso
for i in range(12):
    df_pen.loc[df_pen.index.month==(i+1), "Rso_monthly"] = Rso_monthly[i]

df_pen["Rs"] = df_radiation["daily_radiation_MJ_per_m2_per_day"]

fig, ax = plt.subplots(1, figsize=(10,7))
ax.plot(df_pen['Rso_monthly'])
plt.gcf().autofmt_xdate()
ax.set_ylabel(r"$R_{so}$ (MJ m$^{-2} d^{-1}$)")
```

```python
#collapse-hide

middle = pd.date_range(start='1/1/2020', periods=13, freq='MS') + pd.DateOffset(days=14)
new = df_pen.loc[middle, 'Rso_monthly'].astype('float')
new
df_i = (pd.DataFrame(data=new, index=new.index) #create the dataframe
           .resample("D") #resample daily
           .interpolate(method='time') #interpolate by time
       )

fig, ax = plt.subplots(1, figsize=(10,7))
ax.plot(df_i, 'o')
ax.plot(df_pen['Rso_monthly'])
plt.gcf().autofmt_xdate()
ax.set_title("time interpolation")
ax.set_ylabel(r"$R_{so}$ (MJ m$^{-2} d^{-1}$)")
```

from: Ward & Trimble, "Environmental Hydrology", 2nd Edition, page 99.

* Calculate
  $$R_{bo} = \epsilon\, \sigma\, T^4_{Kelvin},$$
  where
  $$\epsilon=-0.02+0.261 \exp\left(-7.77\times10^{-4}T_{Celcius}^2\right),$$
  $$\sigma=4.903\times 10^{-9} \text{  MJ m$^{-2}$ d$^{-1}$ K$^{-4}$},$$
  and
  $$T_{Kelvin}=T_{Celcius}+273.15$$
* Calculate
  $$R_b = \left( a\frac{R_s}{R_{so}+b} \right)R_{bo},$$
  where
  - for humid areas, $a=1.0$ and $b=0$,
  - for arid areas, $a=1.2$ and $b=-0.2$,
  - for semihumid areas, $a=1.1$ and $b=-0.1$
* Finally, calculate
  $$R_n = (1-\alpha)R_s\!\! \downarrow -R_b \!\! \uparrow,$$
  where
  - $\alpha= 0.23$ for most green crops with a full cover
  - $\alpha= 0.04$ for fresh asphalt
  - $\alpha= 0.12$ for worn-out asphalt
  - $\alpha= 0.55$ for fresh concrete

Add a new column `Rn` to `df_pen` dataframe.

```python
#collapse-hide

# Stefan-Boltzmann constant
sigma = 4.903e-9
emissivity = -0.02 + 0.261 * np.exp(-7.77e-4 * df_pen['T']**2)

# Rbo: net longwave radiation for clear skies, otherwise known as diffuse radiation or emitted radiation from the
# atmosphere - 'how hot is it?'
Rbo = emissivity*sigma*((df_pen['T']+273.15)**4)

# net outgoing long-wave radiation (note: Rs/Rso = proportion of how clear the day is)
# for humid areas, a=1.0 and b=0
# for arid areas, a=1.2 and b=-0.2
# for semihumid areas, a=1.1 and b=-0.1
a = 1.2
b = -0.2
Rb = (a*df_pen['Rs']/df_pen['Rso_monthly'] + b)*Rbo

# α is the albedo, or short-wave reflectance (dimensionless)
alpha = 0.23

# net radiation
Rn = (1 - alpha) * df_pen['Rs'] - Rb   # (MJ/m2/d)
df_pen['Rn'] = Rn
df_pen
```

      T
      dew_point
      u
      P
      RH
      lambda
      Delta
      G
      gamma
      f_wind
      Rso_monthly
      Rs
      Rn

      timestamp

      2020-01-01
      12.3625
      9.0625
      1.5250
      101.30875
      81.500
      2.471812
      0.094385
      -1.62750
      0.066750
      1.808250
      17.46
      10.0296
      4.346568

      2020-01-02
      11.9750
      9.8250
      1.9250
      101.20125
      87.000
      2.472727
      0.092300
      1.44375
      0.066654
      2.020250
      17.46
      4.3128
      2.653905

      2020-01-03
      13.0500
      4.9750
      5.1750
      101.37125
      58.500
      2.470189
      0.098185
      -2.33625
      0.066835
      3.742750
      17.46
      11.6748
      4.854942

      2020-01-04
      10.8625
      6.6875
      5.5625
      101.15500
      78.375
      2.475354
      0.086530
      -0.23625
      0.066553
      3.948125
      17.46
      1.6452
      1.871722

      2020-01-05
      12.9375
      9.2125
      4.5625
      101.23625
      79.125
      2.470455
      0.097554
      4.25250
      0.066739
      3.418125
      17.46
      6.8544
      3.415474

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      2021-01-27
      13.8125
      8.2375
      1.8875
      100.83750
      72.375
      2.468389
      0.102551
      4.77750
      0.066532
      2.000375
      17.46
      12.2652
      5.060995

      2021-01-28
      14.4000
      10.2250
      3.5250
      101.12750
      76.750
      2.467002
      0.106028
      -3.07125
      0.066760
      2.868250
      17.46
      7.1640
      3.534995

      2021-01-29
      12.3500
      7.9125
      5.0250
      101.22125
      75.250
      2.471842
      0.094317
      -3.01875
      0.066691
      3.663250
      17.46
      7.2936
      3.537119

      2021-01-30
      12.9625
      7.6500
      4.4250
      101.49500
      71.375
      2.470396
      0.097694
      5.69625
      0.066911
      3.345250
      17.46
      9.3276
      4.152687

      2021-01-31
      15.0625
      8.3125
      3.7500
      101.32500
      66.000
      2.465437
      0.110070
      8.82000
      0.066933
      2.987500
      17.46
      13.5468
      5.513874

397 rows × 13 columns

```python

```

Calculate the vapor pressure deficit, VPD, add a new column to `df_pen`.

$$e_d = e_s\cdot \frac{RH}{100}$$

$$e_s = \exp\left(\frac{16.78\,T-116.9}{T+237.3}\right)$$

```python
#collapse-hide

# vapor pressure deficit = VPD
def vp_sat(T):
    return np.exp((16.78*T - 116.9)/(T + 237.3))
df_pen['es'] = vp_sat(df_pen['T'])
df_pen['ed'] = df_pen['es'] * df_pen['RH'] / 100
df_pen['VPD'] = df_pen['es'] - df_pen['ed']
df_pen
```

      T
      dew_point
      u
      P
      RH
      lambda
      Delta
      G
      gamma
      f_wind
      Rso_monthly
      Rs
      Rn
      es
      ed
      VPD

      timestamp

      2020-01-01
      12.3625
      9.0625
      1.5250
      101.30875
      81.500
      2.471812
      0.094385
      -1.62750
      0.066750
      1.808250
      17.46
      10.0296
      4.346568
      1.437148
      1.171276
      0.265872

      2020-01-02
      11.9750
      9.8250
      1.9250
      101.20125
      87.000
      2.472727
      0.092300
      1.44375
      0.066654
      2.020250
      17.46
      4.3128
      2.653905
      1.400935
      1.218813
      0.182122

      2020-01-03
      13.0500
      4.9750
      5.1750
      101.37125
      58.500
      2.470189
      0.098185
      -2.33625
      0.066835
      3.742750
      17.46
      11.6748
      4.854942
      1.503424
      0.879503
      0.623921

      2020-01-04
      10.8625
      6.6875
      5.5625
      101.15500
      78.375
      2.475354
      0.086530
      -0.23625
      0.066553
      3.948125
      17.46
      1.6452
      1.871722
      1.301383
      1.019959
      0.281424

      2020-01-05
      12.9375
      9.2125
      4.5625
      101.23625
      79.125
      2.470455
      0.097554
      4.25250
      0.066739
      3.418125
      17.46
      6.8544
      3.415474
      1.492399
      1.180860
      0.311538

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      2021-01-27
      13.8125
      8.2375
      1.8875
      100.83750
      72.375
      2.468389
      0.102551
      4.77750
      0.066532
      2.000375
      17.46
      12.2652
      5.060995
      1.580054
      1.143564
      0.436490

      2021-01-28
      14.4000
      10.2250
      3.5250
      101.12750
      76.750
      2.467002
      0.106028
      -3.07125
      0.066760
      2.868250
      17.46
      7.1640
      3.534995
      1.641414
      1.259785
      0.381629

      2021-01-29
      12.3500
      7.9125
      5.0250
      101.22125
      75.250
      2.471842
      0.094317
      -3.01875
      0.066691
      3.663250
      17.46
      7.2936
      3.537119
      1.435967
      1.080565
      0.355402

      2021-01-30
      12.9625
      7.6500
      4.4250
      101.49500
      71.375
      2.470396
      0.097694
      5.69625
      0.066911
      3.345250
      17.46
      9.3276
      4.152687
      1.494843
      1.066944
      0.427899

      2021-01-31
      15.0625
      8.3125
      3.7500
      101.32500
      66.000
      2.465437
      0.110070
      8.82000
      0.066933
      2.987500
      17.46
      13.5468
      5.513874
      1.713106
      1.130650
      0.582456

397 rows × 16 columns

```python

```

Now that all variables have been defined, daily E_penman can be calculated.

$$
\begin{equation}
E_{tp} = \frac{\Delta}{\Delta+\gamma}Q_{ne}+ \frac{\gamma}{\Delta+\gamma}E_A
\end{equation}
$$

$Q_n$ is the available energy flux density:

$$
\begin{equation}
Q_n = R_n - G,
\end{equation}
$$

and $E_A$ is the drying power of the air:

$$
\begin{equation}
E_A = f_e(u)\cdot\text{VPD}
\end{equation}
$$

Add a new column `E_penman` to `df_pen`.

```python
#collapse-hide

def E_penman(df):
    T = df['T']
    Delta = df['Delta']
    gamma = df['gamma']
    Rn = df['Rn']
    G = df['G']
    EA = 6.43*df['f_wind'] * df['VPD']
    lambd = df['lambda']
    return ((Delta / (Delta + gamma))*(Rn - G) + ((gamma / (Delta + gamma))*EA)) / lambd

# daily_data
df_pen['E_penman'] = E_penman(df_pen)

fig, ax = plt.subplots(1, figsize=(10,7))
ax.plot(df_pen['E_penman'])
plt.gcf().autofmt_xdate()
ax.set_ylabel(r"$ET_{penman}$ (mm d$^{-1}$)")
```

```python

```

Make a plot with the following:
1. the Penman (daily) estimate of the potential evapotranspiration.
2. the Thornthwaite (monthly) estimate of the potential ET.
3. daily evaporation pan data.

```python
#collapse-hide

fig, ax = plt.subplots(1, 1, figsize=(10,7))
ax.plot(df_pen['E_penman'], color="tab:red", label="Penman", linewidth=2)
ax.plot(df_th['Ep']/30, color="tab:blue", label="Thornthwaite", linewidth=2)
ax.plot(1*df2['pan evaporation (mm)'], color="black", label="pan", linewidth=2)

ax.set(xlabel="date",
       ylabel="evaporation (mm)")
ax.legend();
```

```python

```

Plot the mean temperatures used in the Penman calculation (daily mean) and in the Thornthwaite calculation (monthly mean).

```python
#collapse-hide

fig, ax = plt.subplots(1, 1, figsize=(10,7))
ax.plot(df_pen['T'], color="tab:blue", label="Penman", linewidth=2)
ax.plot(df_th['T'], color="tab:orange", label="Thornthwaite", linewidth=2)
ax.set(xlabel="date",
       ylabel="temperature (°C)")
ax.legend();
```

```python

```


```python
import numpy as np
import sympy as sym
import json
import sympy as sym
import pydae.build as db
```

```python
file_path = 'cigre_eur_lv_res_bpu_iso.json'
grid = bal_pu(file_path)
```

    AVR added
    AVR added

```python
params_dict = grid['params']

g_list = grid['g']
h_dict = grid['h']
f_list = grid['f']
x_list = grid['x']

sys = {'name':'cigre_eur_lv_res_bpu_iso',
       'params_dict':params_dict,
       'f_list':f_list,
       'g_list':g_list,
       'x_list':x_list,
       'y_ini_list':grid['y'],
       'y_run_list':grid['y'],
       'u_run_dict':grid['u'],
       'u_ini_dict':grid['u'],
       'h_dict':h_dict}

sys = db.system(sys)
db.sys2num(sys)
```

```python
# -*- coding: utf-8 -*-
"""
Created on Thu Feb 25 23:52:55 2021

@author: jmmau
"""

import numpy as np
import sympy as sym
import json

def bal_pu(data_input):
    '''

    Parameters
    ----------
    data_input : string or dict
        File path to the system data information or dictionar y with the information.

    Returns
    -------
    dict
        Dictionary with the equations for pydae.

    {
     'sys':{'name':'pf_1','S_base':100e6},
     'buses':[{'name':'GRI','P_W':0.0,'Q_var':0.0,'U_kV':66.0, 'type':'slack'},
              {'name':'POI','P_W':0.0,'Q_var':0.0,'U_kV':66.0},
              {'name':'PMV','P_W':0.0,'Q_var':0.0,'U_kV':20.0}],
     'lines':[{'bus_j':'GRI','bus_k':'POI','X_km':0.4,'R_km':0.12,'km':20},
              {'bus_j':'POI','bus_k':'PMV','X_pu':0.04,'R_pu':0.01, 'S_mva':50.0}]
    }

    '''

    if type(data_input) == str:
        with open(file_path,'r') as fobj:
            data = json.loads(fobj.read().replace("'",'"'))
    elif type(data_input) == dict:
        data = data_input

    sys = data['sys']
    buses = data['buses']
    lines = data['lines']
    x_grid = []
    f_grid = []

    params_grid = {'S_base':sys['S_base']}
    S_base = sym.Symbol("S_base", real=True)
    N_bus = len(buses)
    N_branch = len(lines)
    A = sym.zeros(3*N_branch,N_bus)
    G_primitive = sym.zeros(3*N_branch,3*N_branch)
    B_primitive = sym.zeros(3*N_branch,3*N_branch)
    buses_list = [bus['name'] for bus in buses]
    it = 0
    for line in lines:

        bus_j = line['bus_j']
        bus_k = line['bus_k']

        idx_j = buses_list.index(bus_j)
        idx_k = buses_list.index(bus_k)

        A[3*it,idx_j] = 1
        A[3*it,idx_k] =-1
        A[3*it+1,idx_j] = 1
        A[3*it+2,idx_k] = 1

        line_name = f"{bus_j}_{bus_k}"
        g_jk = sym.Symbol(f"g_{line_name}", real=True)
        b_jk = sym.Symbol(f"b_{line_name}", real=True)
        bs_jk = sym.Symbol(f"bs_{line_name}", real=True)
        G_primitive[3*it,3*it] = g_jk
        B_primitive[3*it,3*it] = b_jk
        B_primitive[3*it+1,3*it+1] = bs_jk/2
        B_primitive[3*it+2,3*it+2] = bs_jk/2

        if 'X_pu' in line:
            if 'S_mva' in line: S_line = 1e6*line['S_mva']
            R = line['R_pu']*sys['S_base']/S_line  # in pu of the system base
            X = line['X_pu']*sys['S_base']/S_line  # in pu of the system base
            G =  R/(R**2+X**2)
            B = -X/(R**2+X**2)
            params_grid.update({f"g_{line_name}":G})
            params_grid.update({f'b_{line_name}':B})

        if 'X' in line:
            bus_idx = buses_list.index(line['bus_j'])
            U_base = buses[bus_idx]['U_kV']
            Z_base = U_base**2/sys['S_base']
            R = line['R']/Z_base  # in pu of the system base
            X = line['X']/Z_base  # in pu of the system base
            G =  R/(R**2+X**2)
            B = -X/(R**2+X**2)
            params_grid.update({f"g_{line_name}":G})
            params_grid.update({f'b_{line_name}':B})

        if 'X_km' in line:
            bus_idx = buses_list.index(line['bus_j'])
            U_base = buses[bus_idx]['U_kV']*1000
            Z_base = U_base**2/sys['S_base']
            R = line['R_km']*line['km']/Z_base  # in pu of the system base
            X = line['X_km']*line['km']/Z_base  # in pu of the system base
            G =  R/(R**2+X**2)
            B = -X/(R**2+X**2)
            params_grid.update({f"g_{line_name}":G})
            params_grid.update({f'b_{line_name}':B})

        params_grid.update({f'bs_{line_name}':0.0})
        if 'Bs_pu' in line:
            if 'S_mva' in line: S_line = 1e6*line['S_mva']
            Bs = line['Bs_pu']*S_line/sys['S_base']  # in pu of the system base
            bs = -Bs/2.0
            params_grid[f'bs_{line_name}'] = bs

        if 'Bs_km' in line:
            bus_idx = buses_list.index(line['bus_j'])
            U_base = buses[bus_idx]['U_kV']*1000
            Z_base = U_base**2/sys['S_base']
            Y_base = 1.0/Z_base
            Bs = line['Bs_km']*line['km']/Y_base # in pu of the system base
            bs = Bs
            params_grid[f'bs_{line_name}'] = bs

        it += 1

    G = A.T * G_primitive * A
    B = A.T * B_primitive * A

    sin = sym.sin
    cos = sym.cos
    y_grid = []
    g = sym.zeros(2*N_bus,1)
    u_grid = {}
    h_grid = {}
    for j in range(N_bus):
        bus_j_name = buses_list[j]
        P_j = sym.Symbol(f"P_{bus_j_name}", real=True)
        Q_j = sym.Symbol(f"Q_{bus_j_name}", real=True)
        g[2*j]   = -P_j/S_base
        g[2*j+1] = -Q_j/S_base
        for k in range(N_bus):

            bus_k_name = buses_list[k]
            V_j = sym.Symbol(f"V_{bus_j_name}", real=True)
            V_k = sym.Symbol(f"V_{bus_k_name}", real=True)
            theta_j = sym.Symbol(f"theta_{bus_j_name}", real=True)
            theta_k = sym.Symbol(f"theta_{bus_k_name}", real=True)
            g[2*j]   += V_j*V_k*(G[j,k]*cos(theta_j - theta_k) + B[j,k]*sin(theta_j - theta_k))
            g[2*j+1] += V_j*V_k*(G[j,k]*sin(theta_j - theta_k) - B[j,k]*cos(theta_j - theta_k))
            h_grid.update({f"V_{bus_j_name}":V_j})
        bus = buses[j]
        bus_name = bus['name']
        if 'type' in bus:
            if bus['type'] == 'slack':
                y_grid += [P_j]
                y_grid += [Q_j]
                u_grid.update({f"V_{bus_name}":1.0})
                u_grid.update({f"theta_{bus_name}":0.0})
        else:
            y_grid += [V_j]
            y_grid += [theta_j]
            u_grid.update({f"P_{bus_name}":bus['P_W']})
            u_grid.update({f"Q_{bus_name}":bus['Q_var']})

        params_grid.update({f'U_{bus_name}_n':bus['U_kV']*1000})
    g_grid = list(g)

    if False:
        v_sym_list = []
        for bus in buses_list:
            V_m = sym.Symbol(f'V_{bus}',real=True)
            V_a = sym.Symbol(f'theta_{bus}',real=True)
            v_sym_list += [V_m*sym.exp(sym.I*V_a)]

        sym.Matrix(v_sym_list)

        I_lines = (G_primitive+1j*B_primitive) * A * sym.Matrix(v_sym_list)

        it = 0
        for line in lines:
            I_jk_r = sym.Symbol(f"I_{line['bus_j']}_{line['bus_k']}_r", real=True)
            I_jk_i = sym.Symbol(f"I_{line['bus_j']}_{line['bus_k']}_i", real=True)
            g_grid += [-I_jk_r + sym.re(I_lines[it])]
            g_grid += [-I_jk_i + sym.im(I_lines[it])]
            y_grid += [I_jk_r]
            y_grid += [I_jk_i]
            it += 1

    grid = {'g':g_grid,'y':y_grid,'u':u_grid,'h':h_grid, 'x':x_grid, 'f':f_grid,
            'params':params_grid, 'data':data, 'A':A, 'B_primitive':B_primitive}

    if 'syns' in data:
        syns_add(grid)

        # omega COI
        omega_coi_n = 0
        N_syn = 0
        for syn in data['syns']:
            bus_name = syn['bus']
            omega = sym.Symbol(f"omega_{bus_name}", real=True)
            omega_coi_n += omega
            N_syn += 1
        omega_coi = sym.Symbol("omega_coi", real=True)
        y_grid += [ omega_coi]
        g_grid += [-omega_coi + omega_coi_n/N_syn]

        # secondary frequency control
        omega_coi_n = 0
        xi_freq = sym.Symbol("xi_freq", real=True)
        for syn in data['syns']:
            bus_name = syn['bus']
            p_r = sym.Symbol(f"p_r_{bus_name}", real=True)
            K_sec = sym.Symbol(f"K_sec_{bus_name}", real=True)

            y_grid += [  p_r]
            g_grid += [ -p_r + K_sec*xi_freq/N_syn]
            params_grid.update({str(K_sec):syn['K_sec']})
        x_grid += [ xi_freq]
        f_grid += [ 1-omega_coi]

    return grid

def syns_add(grid):
    sin = sym.sin
    cos = sym.cos
    buses = grid['data']['buses']
    buses_list = [bus['name'] for bus in buses]
    for syn in grid['data']['syns']:

        bus_name = syn['bus']
        idx_bus = buses_list.index(bus_name) # get the number of the bus where the syn is connected
        if not 'idx_powers' in buses[idx_bus]: buses[idx_bus].update({'idx_powers':0})
        buses[idx_bus]['idx_powers'] += 1

        p_g = sym.Symbol(f"p_g_{bus_name}_{buses[idx_bus]['idx_powers']}", real=True)
        q_g = sym.Symbol(f"q_g_{bus_name}_{buses[idx_bus]['idx_powers']}", real=True)
        V = sym.Symbol(f"V_{bus_name}", real=True)
        theta = sym.Symbol(f"theta_{bus_name}", real=True)
        i_d = sym.Symbol(f"i_d_{bus_name}", real=True)
        i_q = sym.Symbol(f"i_q_{bus_name}", real=True)
        delta = sym.Symbol(f"delta_{bus_name}", real=True)
        omega = sym.Symbol(f"omega_{bus_name}", real=True)
        p_m = sym.Symbol(f"p_m_{bus_name}", real=True)
        e1q = sym.Symbol(f"e1q_{bus_name}", real=True)
        e1d = sym.Symbol(f"e1d_{bus_name}", real=True)
        v_f = sym.Symbol(f"v_f_{bus_name}", real=True)

        omega_coi = sym.Symbol("omega_coi", real=True)

        p_m_ref  = sym.Symbol(f"p_m_ref_{bus_name}", real=True)
        v_ref  = sym.Symbol(f"v_ref_{bus_name}", real=True)
        xi_m = sym.Symbol(f"xi_m_{bus_name}", real=True)

        v_d = V*sin(delta - theta)
        v_q = V*cos(delta - theta)

        for item in ['S_n','H','Omega_b','T1d0','T1q0','X_d','X_q','X1d','X1q',
                     'D','R_a','K_delta']:
            string = f"{item}=sym.Symbol('{item}_{bus_name}', real = True)"
            exec(string,globals())
            grid['params'].update({f'{item}_{bus_name}':syn[item]})

        p_e = i_d*(v_d + R_a*i_d) + i_q*(v_q + R_a*i_q)

        omega_s = omega_coi

        ddelta = Omega_b*(omega - omega_s) - K_delta*delta
        domega = 1/(2*H)*(p_m - p_e - D*(omega - omega_s))
        de1q = 1/T1d0*(-e1q - (X_d - X1d)*i_d + v_f)
        de1d = 1/T1q0*(-e1d + (X_q - X1q)*i_q)

        dxi_m =   omega - 1

        g_id  = v_q + R_a*i_q + X1d*i_d - e1q
        g_iq  = v_d + R_a*i_d - X1q*i_q - e1d
        g_p_g  = i_d*v_d + i_q*v_q - p_g
        g_q_g  = i_d*v_q - i_q*v_d - q_g

        f_syn = [ddelta,domega,de1q,de1d]
        x_syn = [ delta, omega, e1q, e1d]
        g_syn = [g_id,g_iq,g_p_g,g_q_g]
        y_syn = [i_d,i_q,p_g,q_g]

        if 'f' not in grid: grid.update({'f':[]})
        if 'x' not in grid: grid.update({'x':[]})
        grid['f'] += f_syn
        grid['x'] += x_syn
        grid['g'] += g_syn
        grid['y'] += y_syn

        S_base = sym.Symbol('S_base', real = True)
        grid['g'][idx_bus*2]   += -p_g*S_n/S_base
        grid['g'][idx_bus*2+1] += -q_g*S_n/S_base

        #for item in syn:
        #    grid['params'].update({f"{item}_{bus_name}":syn[item]})
        #grid['params'].pop(f"bus_{bus_name}")
        #

        if 'avr' in syn:
            add_avr(grid,syn)
        if 'gov' in syn:
            add_gov(grid,syn)

def add_avr(grid,syn_data):

    if syn_data['avr']['type'] == 'sexs':
        sexs(grid,syn_data)

def add_gov(grid,syn_data):

    if syn_data['gov']['type'] == 'tgov1':
        tgov1(grid,syn_data)

def tgov1(grid,syn_data):
    bus_name = syn_data['bus']
    gov_data = syn_data['gov']

    omega = sym.Symbol(f"omega_{bus_name}", real=True)
    p_r = sym.Symbol(f"p_r_{bus_name}", real=True)
    p_c = sym.Symbol(f"p_c_{bus_name}", real=True)
    p_m = sym.Symbol(f"p_m_{bus_name}", real=True)
    p_m_ref = sym.Symbol(f"p_m_ref_{bus_name}", real=True)
    T_m = sym.Symbol(f"T_m_{bus_name}", real=True)
    Droop = sym.Symbol(f"Droop_{bus_name}", real=True)

    dp_m =   (p_m_ref - p_m)/T_m
    g_p_m_ref  = -p_m_ref + p_c + p_r - 1/Droop*(omega - 1)

    grid['f'] += [dp_m]
    grid['x'] += [ p_m]
    grid['g'] += [g_p_m_ref]
    grid['y'] += [  p_m_ref]
    grid['params'].update({str(Droop):gov_data['Droop']})
    grid['params'].update({str(T_m):gov_data['T_m']})
    grid['u'].update({str(p_c):gov_data['p_c']})

def sexs(grid,syn_data):
    bus_name = syn_data['bus']
    avr_data = syn_data['avr']

    v_t = sym.Symbol(f"V_{bus_name}", real=True)
    v_c = sym.Symbol(f"v_c_{bus_name}", real=True)
    xi_v  = sym.Symbol(f"xi_v_{bus_name}", real=True)
    v_f = sym.Symbol(f"v_f_{bus_name}", real=True)
    T_r = sym.Symbol(f"T_r_{bus_name}", real=True)
    K_a = sym.Symbol(f"K_a_{bus_name}", real=True)
    K_ai = sym.Symbol(f"K_ai_{bus_name}", real=True)
    v_ref = sym.Symbol(f"v_ref_{bus_name}", real=True)
    v_pss = sym.Symbol(f"v_pss_{bus_name}", real=True)

    dv_c =   (v_t - v_c)/T_r
    dxi_v =   v_ref - v_t
    g_v_f  = K_a*(v_ref - v_c + v_pss) + K_ai*xi_v  - v_f

    print('AVR added')
    grid['f'] += [dv_c,dxi_v]
    grid['x'] += [ v_c, xi_v]
    grid['g'] += [g_v_f]
    grid['y'] += [v_f]
    grid['params'].update({str(K_a):avr_data['K_a']})
    grid['params'].update({str(K_ai):avr_data['K_ai']})
    grid['params'].update({str(T_r):avr_data['T_r']})
    grid['u'].update({str(v_ref):avr_data['v_ref']})
    grid['u'].update({str(v_pss):avr_data['v_pss']})
```

```python
p = 0
q = 0
for bus in grid['data']['buses']:
    p+=bus['P_W']
    q+=bus['Q_var']
```

```python
p
```

    -193800.0

```python
q/150e3/2
```

    -0.21232992666666667

```python

```


##**Chapter 10 – Introduction to Neural Networks with Keras**

 Photo Credits: Galaxy's Edge by Rod Long licensed under the Unsplash License

> *The defnition of AI is a highly contested concept. It often refers to technologies that demonstrate levels of independent intelligence from humans. By its very
defnition, it is an intelligence that is differentiated from natural intelligence; it is
a constructed, artificial, or machine intelligence.*

$\quad$Ryan, M. (2020). In AI we trust: ethics, artificial intelligence, and reliability. Science and Engineering Ethics, 26(5), 2749-2767.

This notebook will be used in the lab session for week 5 of the course, covers Chapters 10 of Géron, and builds on the [notebooks made available on _Github_](https://github.com/ageron/handson-ml2).

Need a reminder of last week's labs? Click [_here_](https://colab.research.google.com/github/tbeucler/2022_ML_Earth_Env_Sci/blob/main/Lab_Notebooks/Week_4_Dimensionality_Reduction_Clustering.ipynb) to go to notebook for week 4 of the course.

## Notebook Setup

First, let's import a few common modules, ensure MatplotLib plots figures inline and prepare a function to save the figures. We also check that Python 3.5 or later is installed (although Python 2.x may work, it is deprecated so we strongly recommend you use Python 3 instead), as well as Scikit-Learn ≥0.20 and TensorFlow ≥2.0.

```python
# Python ≥3.5 is required
import sys
assert sys.version_info >= (3, 5)

# Scikit-Learn ≥0.20 is required
import sklearn
assert sklearn.__version__ >= "0.20"

try:
    # %tensorflow_version only exists in Colab.
    %tensorflow_version 2.x
except Exception:
    pass

# TensorFlow ≥2.0 is required
import tensorflow as tf
assert tf.__version__ >= "2.0"

# Common imports
import numpy as np
import os

# to make this notebook's output stable across runs
rnd_seed = 42
rnd_gen = np.random.default_rng(rnd_seed)

# To plot pretty figures
%matplotlib inline
import matplotlib as mpl
import matplotlib.pyplot as plt
mpl.rc('axes', labelsize=14)
mpl.rc('xtick', labelsize=12)
mpl.rc('ytick', labelsize=12)

# Where to save the figures
PROJECT_ROOT_DIR = "."
CHAPTER_ID = "ann"
IMAGES_PATH = os.path.join(PROJECT_ROOT_DIR, "images", CHAPTER_ID)
os.makedirs(IMAGES_PATH, exist_ok=True)

def save_fig(fig_id, tight_layout=True, fig_extension="png", resolution=300):
    path = os.path.join(IMAGES_PATH, fig_id + "." + fig_extension)
    print("Saving figure", fig_id)
    if tight_layout:
        plt.tight_layout()
    plt.savefig(path, format=fig_extension, dpi=resolution)

# Initialize the run_index
run_index = None

# Loading Tensorboard
%load_ext tensorboard
```

## Data Setup

Today, we'll once again be working on the MNIST handwritten digit database - we're becoming experts in typography! ✍

Let's begin by importing the dataset from the keras dataset library.

### **Q1) Load the MNIST dataset from Keras. Divide it into a training, validation, and test dataset**

*Hint 1: To access the Keras library, you can either reimport keras (e.g., `import tensorflow.keras as keras`), or you can access it from the instance of tensorflow we imported during setup (i.e., using `tf.keras`)*

*Hint 2: [Here is the documentation](https://keras.io/api/datasets/mnist/) for the Keras implementation of the MNIST dataset*

*Hint 3: If you use the `mnist.load_data()` method, what will be returned will be a set of tuples: (training_data, testing_data), where training_data and testing_data are tuples of inputs and labels (X, y)*

*Hint 4: You can break down the training dataset from the `.load()` method into a training and validation dataset. Since the full training dataset includes 60 000 samples, try using 50 000 samples as training data and 10 000 samples as validation data.*

```python
# Load the keras dataset data
( (X_train_full, y_train_full) , (_____, _____) ) = _____.mnist._____()
```

```python
# Split the data
X_train =
X_valid =
_______ =
_______ =
```

What does our data look like? Let's get an idea of the values and figure out what kind of preprocessing we should do before training our neural network.

### **Q2) Print the shape of the training, validation, and test sets. Then, print the maximum and minimum input values.**

*Hint 1: You loaded the data as numpy arrays. Thus, you can rely on the built-in methods for finding the shape and min/max values.*

*Hint 2: Click for the documentation on [`ndarray.max()`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.max.html), [`ndarray.min()`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.min.html), and [`ndarray.shape`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.shape.html)*

```python
#Write your code here
```

If you used the same train/validation split as we did, you should have 50k samples in the training set, 10k in the validation set, and 10k in the test set.

Since the data represents grayscale image values, data values should vary between 0 and 255; Normalize the data by dividing it by 255.
### **Q3) Normalize the input data for the training, validation, and testing sets**

*Hint 1: The datasets are stored as simple numpy arrays, so you can perform arithmetic operations on them!*

```python
X_train = _____ / 255
_____ =
_____ =
```

We now have the normalized training, validation, and testing data that we'll use to train our neural network. Before moving on, it might be worth it to make a small visualiation of samples in our data to ensure that everything worked out correctly.

### **Q4) Write a function that:

1) Takes in an input dataset and its labels, a number of rows, and a number of columns
 2) Prints out a random n_rows by n_columns sample of images with their labels
**

*Hint 1: You can use the `rnd_seed.integers()` generator to generate a set of integers between 0 and the number of samples, with a size of (rows,columns). [Here is some documentation that can help](https://numpy.org/doc/stable/reference/random/generator.html#simple-random-data). It's best practice to take in the random generator as an argument for your function.*

*Hint 2: You can use matplotlib's `fig, axes = plt.subplots()` to make a grid of axes and call the `imshow()` method on each ax in order to plot the digit. It is recommended that you use the `cmap='binary'` argument in imshow to print the digits in black and white*. Click on the links for the documentation to [`plt.sublopts()`](https://matplotlib.org/3.5.0/api/_as_gen/matplotlib.pyplot.subplots.html), [`plt.imshow()`](https://matplotlib.org/3.5.0/api/_as_gen/matplotlib.pyplot.imshow.html), and [the colormaps (i.e., cmap values)](https://matplotlib.org/stable/gallery/color/colormap_reference.html) available in matplotlib.

*Hint 3: You can iterate using numpy `ndenumerate()` method, which will return the n-dimensional index of the array and the element located there. This will be useful when iterating through the indices you generated and plotting the corresponding digit and label*

*Hint 4: Feeling stuck? [Here is sample code for a way this function can be implemented.](https://unils-my.sharepoint.com/:t:/g/personal/tom_beucler_unil_ch/ERT6Sl_NHp5Nt2YjzkDPzLwB0RQ7_rVAl3RDx4BfHI047g?download=1)*

```python
def sample_plotter(___, ___, ___, ___):

    # Create a set of indices to access the sample images/labels

    # Create a figure with n_rows and n_columns

    # Plot each selected digit
    for in :

    return None
```

Now that our function is defined, let's go ahead and print out a 4 row by 8 column sample from each dataset.

### **Q5) Grab a 4x8 sample of digits from each dataset and print out the image and labels**

```python
#Write your code here!
```

We're now ready to start developing our neural network. The first thing that we want to do is figure out an appropriate learning rate for our model - after all, we want to choose one that converges to a solution *and* is the least computationally expensive possible.

Let's start by setting up a keras *callback* [(click here for the documentation)](https://keras.io/api/callbacks/), a type of object that will allow us to change the learning rate after every iteration (i.e., after every batch of data). We will set up what is called an exponential learning rate (that is, the learning will increase by a factor of $k$ after each iteration). Expressed mathematically,
\begin{align}
\eta_{\scriptsize{t}} = \eta_{\scriptsize{0}} \, \cdot \, k^{\scriptsize{t}}
\end{align}
where $t$ is the current iteration.

As a reminder, an epoch is an iteration through the entire training dataset, while a batch is an iteration through a predefined subset of . It's important to make this distinction, as ML algorithms are often trained in batches when dealing with large datasets, and we *normally* do not want to change the learning rate in between batches during model training. However, we will do so during this evaluation phase in order to determine an adequate learning rate.

We will therefore set a callback that will do two things after the end of each batch:

> 1) Keep a track of the losses
 2) Adjust the learning rate by multiplying it by a predefined factor

### **Q5) Set up an *Exponential_Learning_Rate* callback that, after each batch, logs the value of the loss function and learning rate, and then multiplies the learning rate by a factor of $k$**

*Hint 1: Multiple backend options are available with Keras. We will be using tensorflow, but the code is thought to be written in such a way that a different backend **could** be used. `tf.keras.backend` has a `.backend()` method that allows you to check what backend is being used.*

*Hint 2: You should extend the `tf.keras.callbacks.Callback` class. (Confused about extending classes? [Here is a question on stack overflow](https://stackoverflow.com/questions/15526858/how-to-extend-a-class-in-python) that could provide some context) *

*Hint 3: The ExponentialLearningRate callback we will implement will need to take in the $k$ factor durint its initialization ([here's a quick overview](https://stackoverflow.com/questions/625083/what-do-init-and-self-do-in-python) on the __init__ contructor method and **self** arguments in classes, with a focus on python.). You will also need to save an empty list as an attribute for both the losses and the learning rates*

*Hint 4: Keras model optimizers have an attribute where the learning rate is stored: `model.optimizer.learning_rate`. In order to read the value, you will have to use the keras backend's `.get_value()` method with the model's learning rate as an argument*

*Hint 5: the on_train_batch_end method pass the `logs` argument into the function. You can access the loss function by using `logs['loss']`*

*Hint 6: In order to set the learning rate to a different value, you will have to depend on the keras backend's `.set_value()` method. This method takes in two arguments: the first is the value that will be set (e.g., the learning rate in the model's optimizer) and the value that it will be set to (e.g., the learning rate multiplied by the k factor).*

*Hint 7: Unlike in other documentations we've seen, `backend.get_value()` and `backend.set_value()` don't yet have their own page. However, [here is the link](https://www.tensorflow.org/guide/keras/custom_callback#learning_rate_scheduling) to an example where both methods are used in a learning rate scheduler.*

```python
# We'll start by making it easier to access the keras backend. See hint #1 for
# more details
K = tf.keras.backend

# Use the .backend() method to determine what backend we're running
___.___
```

```python
# Remember that you can access the keras.backend using K, which we defined in
# the code cell above!

class _____(____.____.____.Callback): #define the ExponentialLearningRate class
    # Start
    def __init__(self, factor):
        self.____ = ____ # set the factor
        self.____ = ____ # initialize the losses list
        self.____ = ____ # initialize the learning rates list

    def on_batch_end(self, batch, logs):
        # Add the value of the learning rate to the list
        self.___.append(__.___(self.model.___.___))

        # Add the value of the loss
        self.___.append(___[___])

        # Set the value of the
        ___.___(self.model.___.___, self.model.___.___ * self.___)
```

Now that we've defined out callback, we can go ahead and start thinking about our neural network. For consistency's sake, let's start by clearing the Keras backend and setting our random state.

```python
# Run this cell
K.clear_session()
np.random.seed(rnd_seed)
tf.random.set_seed(rnd_seed)
```

Let's make a simple neural network model using Keras. For this, we will rely on a [*Sequential model*](https://keras.io/guides/sequential_model/), since we will want all of the inputs of one layer to be fed into the next layer. We recommend using the architecture described in the diagram below, but feel free to define your own architecture!

### **Q6) Write a sequential Keras model that will predict the digit class.**

*Hint 1: You can add the layers in the sequential model when initializing the model. It expects the layers in a list. Alternatively, you can add them one by one using the model's `.add()` method. [Check out the documentation here](https://keras.io/guides/sequential_model/#creating-a-sequential-model).*

*Hint 2: The input images should be flattened before feeding them into any densely connected layers. [Here is the documentation](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten) for the flatten layer.*

*Hint 3: You want to use simple, densely connected layers for this exercise. [Here is the documentation](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense) for the dense layer.*

*Hint 4: Using a dense layer with the number of units set to the number of classes (e.g., the number of different digits in the MNIST dataset: 10) using a softmax activation unit can be interpreted as a probability of the input belonging to a given class. [Here is the documentation](https://keras.io/api/layers/activations/#softmax-function) for the softmax activation function in Keras*

```python
# Create your model! Feel free to use our outline, or make your own from scratch

model = tf.___.____.sequential([  # call the keras sequential model class
                            ___,  # 1st Layer
                            ___,  # 2nd Layer
                            ___,  # 3rd Layer
                            ___]) # 4th Layer
```

Now that we have a model defined, we need to run its `.compile()' method, in which we will give the model the following hyper-parameters:
> 1) Loss function will be set to sparse categorical cross entropy
 2) The optimizer will be set to Stochastic Gradient Descent with a learning rate of 1e-3
 3) The model metrics will include the accuracy score

### **Q7) Compile the model with the given hyperparameters (i.e., loss function, optimizer, and metrics) and instantiate the callback we defined previously using a $k$ factor of 1.005 (i.e., a 0.5% increase in learning rate per batch)**

*Hint 1: [Here is the documentation](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/sparse_categorical_crossentropy) for the sparse categorical cross entropy loss function in keras. You can simply reference the function using `loss='sparse_categorical_crossentropy'` when compiling.*

*Hint 2: [Here is the documentation](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/SGD) for the Stochastic Gradient Descent optimizer in keras*

*Hint 3: [Here is the documentation](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/Accuracy) for the accuracy score implementation in keras. Like with the sparse_categorical_cross_entropy loss, you can reference the accuracy score in the metrics list, e.g. by setting `metrics=['accuracy']` when compiling.*

```python
____.compile(___=___, # Set the loss function
              ___=___.____.___(___=___), # Set the optimizer and learning rate
              ___=[___]) # Set the metrics
```

```python
exponential_lr_callback = _____(factor=____)
```

Let's go ahead and train the compiled model for a single epoch.

### **Q8) Fit the model for a single epoch, using the exponential learning rate callback we defined in the previous code cell. Then, plot the Loss vs Learning rate.**

*Hint 1: Just like in scikit-learn, the keras model includes a `.fit()` method to train the algorithm! [Here is the documentation](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit).*

*Hint 2: After training, you can access the recorded losses and corresponding learning rates using the attributes we defined when we defined the class in Q5!*

```python

history = model.___(____, # set the training inputs
                    ____, # set the training labels
                    ____=__, # set the number of epochs
                    validation_data=(____, ____), # set validation input/labels
                    callbacks=[_____]) # Set the callback
```

```python
# Plotting
fig, ax = plt.subplots()

ax.plot(___.___, # learning rates
        ___.___) # losses

# Define a tuple with (min_learning_rate, max_learn_rate)
x_limits = ( min(___.___), max(___.___) )

# Set the xscale to logarithmic
ax.set_xscale('log')

# Draw a horizontal line at the minimum loss value
ax.hlines(min(____.___), #Find the minimum loss value to draw a horizontal line
          *x_limits, # the star unpacks x_limits to the expected num of args
          'g')

# Set the limits for drawing the curves
ax.set_xlim(x_limits)
ax.set_ylim(0, ____) # use the initial loss as the top y boundary

# Display gridlines to see better
ax.grid(which='both')

ax.set_xlabel("Learning rate")
ax.set_ylabel("Loss")
```

If you used the architecture we defined above with the learning rate we defined above, you should produce a graph that looks like this:

In this graph, you can see that the loss reaches a minimum at around 6e-1 and then begins to shoot up violently. Let's avoid that by using half that value (e.g., 3e-1).

If you have a different curve, try setting your learning rate to half of the learning rate with the minimum loss! 😃

Now that we have an idea of what the learning rate should be, let's go ahead and start from scratch once more.

```python
# Run this cell - let's go back to a clean slate!
K.clear_session()
np.random.seed(rnd_seed)
tf.random.set_seed(rnd_seed)
```

We also want to instantiate the model again - the weights in our current model are quite bad and if we use it as is it won't be able to learn since the weights are too far away from the solution. There are other ways to do this, but since our model is quite simple it's worth it to just redefine and recompile it.

### **Q9) Redefine and re-compile the model with the learning rate you found in Q8.**

```python
# redefine the model
model = tf.keras.___.___([ # call the sequential model class
    tf.keras.layers.___(), # flatten the data
    tf.keras.layers.___(), # densely connected ReLU layer, 300 units
    tf.keras.layers.___(), # densely connected ReLU layer, 100 units
    tf.keras.layers.___())] # densely connected Softmax layer, 10 units

```

```python
____.compile(___=___, # Set the loss function
              ___=___.____.___(___=___), # Set the optimizer and learning rate
              ___=[___]) # Set the metrics
```

We're now going to set up a saving directory in case you want to try running the model with different learning rates or other hyper-parameters!

```python
#Change this number and rerun this cell whenever you want to change runs
run_index = 1

run_logdir = os.path.join(os.curdir, "my_mnist_logs", "run_{:03d}".format(run_index))

print(run_logdir)
```

We'll also set up some additional callbacks.
> 1) An early stopping callback ([documentation here](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping)). This callback will stop the training if no improvement is found after a `patience` number of epochs.
 2) A model checkpoint callback ([documentation here](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ModelCheckpoint)). This callback will ensure that only the best version of the model is kept (in case your model's performance reaches a maximum and then deteriorates after a certain number of epochs)
 3) A tensorboard callback ([documentation here](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/TensorBoard)). This callback will enable using Tensorboard to visualize learning curves, metrics, etc. Handy 🙌!

```python
early_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=20)
checkpoint_cb = tf.keras.callbacks.ModelCheckpoint("my_mnist_model.h5", save_best_only=True)
tensorboard_cb = tf.keras.callbacks.TensorBoard(run_logdir)
```

Let's go ahead and fit the model again!

### **Q10) Fit the updated model for 100 epochs**

```python
history = model.fit(____, # inputs
                    ____, # labels
                    ____=___, #epochs
                    validation_data=(___, ___),
                    callbacks=[checkpoint_cb, early_stopping_cb, tensorboard_cb])
```

Finally, we need to evaluate the performance of our model. Go ahead and try it out on the test set!

### **Q11) Evaluate the model on the test set.**

*Hint 1: Keras models include an `evaluate()` method that takes in the test set inputs/labels. [Here is the documentation](https://www.tensorflow.org/api_docs/python/tf/keras/Model#evaluate).*

```python
# Rollback to best model, which was saved by the callback
model = tf.keras.models.load_model("my_mnist_model.h5") # rollback to best model

# Evaluate the model
model.____(____, ____)
```

Finally, we can use tensorboard to check out our model's performance! Note that the tensorboard extension was loaded in the notebook setup cell.

```python
%tensorboard --logdir=./my_mnist_logs --port=6006
```

An enthusiastic (albeit somewhat sick 😷) TA noted that during the development of the notebook the accuracy reached on the test dataset was 97.84%. Additionally, the tensorboard curves from the test run is given below: