# Loops and numerical integration in Python

Last week we saw how to make `quiver` plots with `matplotlib` in a Python notebook. Today, we will see how to conduct numerical integration in Python.

***HEY YOU! PAY ATTENTION! There are several exercises for you to complete within this notebook. Don't miss 'em.***

## Loops

To program a numerical integration scheme (or most things), we need to review the concept of a loop. Loops are one of the most common ideas in programming. The basic idea is that instead of entering a command lots of times with slight variations, let the programming language loop over it a certain number of times while making slight changes each time on your behalf. For example, let's use the most common loop: a `for` loop. The idea is that `for` each thing in some list that you specify, Python runs some code for you. This is easier to understand with an example:

```python
fruits = ["apple", "banana", "cherry"] # A list of fruit words
counter=0 #a counter to help understand loops

# Now we start a for loop. It must:
#    - use the for command to initiate it
#    - identify a variable that changes with each iteration (i in this case)
#    - identify a list-like variable to loop through (fruits in this case)
# Then we indent our code to indicate the code that is getting looped.
for i in fruits: # start a for loop
    counter=counter+1 # Add 1 to our counter, so we know we started a new loop
    print('I am a for loop. This is loop number {}.'.format(counter)) # Print some info
    print('My favorite fruit during this loop is {}.'.format(i)) # Use our loop variable

```

    I am a for loop. This is loop number 1.
    My favorite fruit during this loop is apple.
    I am a for loop. This is loop number 2.
    My favorite fruit during this loop is banana.
    I am a for loop. This is loop number 3.
    My favorite fruit during this loop is cherry.

A very useful function for loops is `range`. It makes a range of numbers that you can loop over. Suppose I want to square the integers from 0 to 5 and print the results. I could do this:

```python
for i in range(0,6): # Loop i over the range 0 through 6.
    print(i**2)
```

    0
    1
    4
    9
    16
    25

Notice that my `range` command was `range(0,6)`, not `range(0,5)`. I need to do this because of how Python counts. I am telling it to give me 0 ***up to but not including*** 6.

`range` also can change the increment between values (for example, you can count by twos or threes instead of by ones), which is often very handy. Here's a quick example,

```python
for i in range(0,10,3): # Loop i over the range 0 through 10 in increments of 3.
    print(i)
```

    0
    3
    6
    9

#### ***An exercise for you:*** Use the code cell below this one. By using a `for` loop and the `range` command, find a way to print the result of raising each integer to the power of the integer above it, and do so for 0 through 15. For example, the first line printed should give the result of $0^1$, the next should be $1^2$, then $2^3$, and so on, ending with $15^{16}$. Do ***not*** manually type these commands; start a loop and use a single line of code within it. Make sure your code gives the expected results.

```python
# Your loop code goes here!
```

## Numerical integrating functions of one variable
Now we will learn how to approximately determine the integral of a functions of a single variable using the trapezoidal rule and Simpson's rule. You will compare your results to the simplest built-in integrator from the ``scipy.integrate`` library, ``quad``. In this notebook, we have started by walking through the results of a computing numerically the integral of a function with known anti-derivative, but have left developing the Simpson alogorithm to you.

### Import all the libraries

We will start by importing all the libarries and functions that we need. These can be called upon in any cell, but it's often useful to call them up in the first cell, so you know what's available. Also, to avoid conflicts in namespaces (i.e., functions and methods that appear in multiple libraries), we will use the ``as`` method of importing libraries.

```python
import sympy as sp                 ## Import symbolic python library
import numpy as np                 ## Import basic scientific computing library
import matplotlib.pyplot as plt    ## Import plotting library
from scipy.integrate import quad   ## Import a specific integrator from scipy.intergrate library

## Make sure our plots show up inline
%matplotlib inline

## Add some style to the printing of sympy functions
sp.init_printing(use_latex='mathjax')

# If equations aren't displaying right in Google Colab, uncommenting
# the following lines might fix it:
#def custom_latex_printer(exp,**options):
#    from google.colab.output._publish import javascript
#    url = "https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.3/latest.js?config=default"
#    javascript(url=url)
#    return printing.latex(exp,**options) #this command may need to be modified depending on how you import sympy
#init_printing(use_latex="mathjax",latex_printer=custom_latex_printer)
```

### An example

We will work with a function with a known anti-derivative,

$$ g(x) = 10~e^{-x},$$

which are planning to integrate from 0 to 10,

$$I(0,10) = \int_0^{10} 10~e^{-x} dx.$$

Let's start by plotting this function to get an idea of what we'll be integrating. This bit of code also serves as a good example of making a basic plot in `matplotlib`, a skill which you will need later in this notebook.

We want to plot the function $10~e^{-x}$ from $x=0$ to $x=10$. Mathematically, this seems simple: just draw the part of the parabola over the infinite number of points from $x=0$ to $x=5$. But computers can't work in infinities; they break everything up into discrete, separate points. We need to think a little differently to use a computer to make such a plot. We need to make an array of $x$ values, evaluate them with $x^2$ to get the $y$ values, and then put all of those points on the screen. We'll use `numpy` to make an array of $x$ values.

```python
t = np.arange(0,10,0.01) # Create a numpy array over the range of interest to serve as our array of independent variables
g = 10*np.exp(-t)

fig_example_plot = plt.figure(figsize=(10,6)) # Make figure object

axes = fig_example_plot.add_axes([0.1, 0.1, 0.8, 0.8]) # Place axes
axes.plot(t, g, label="$g(x) = 10e^{-x}$")
axes.legend(loc=1,fontsize=14); # Make legend in upper left corner. Location codes listed at http://matplotlib.org/users/legend_guide.html#legend-location
axes.set_xlabel(r"$x$",fontsize=18)
axes.set_ylabel(r"$g(x)$",fontsize=18);
```

Ok, great, we know what the function looks like. I said that this function has a known anti-derivative, letting us check our numerical integration. Let's find that anti-derivative. We could try to figure it out ourselves, or look it up, but we can just have Python find it with the ``sympy`` library.

```python
x=sp.Symbol('x') # Tell sympy to treat x as an algebraic symbol, not a normal Python variable
sp.integrate(10*sp.exp(-x),x) # do the integral
```

$\displaystyle - 10 e^{- x}$

In this case, the function has a known anti-derivative, so numerically finding the integral from 0 to 10 is not necessary. We can simply use the antiderivative to evaluate it at the end points and compute the integral,

$$I = -10~e^{-x}$$

So that,

$$I(0,10) = -10({e^{-10}-1}) \approx 9.9995460007$$

But let's see what happens when we numerically integrate it. We start by defining our function:

```python
def g(t):
    return 10*np.exp(-t)
```

Now let's use the trapezoidal integration scheme that we implemented in class, this time with ten steps:

```python
N = 10
a = 0
b = 10
h = (b-a)/N
s = 0       # this just ensures we don't add more to it if we run it again!

s = 0.5*g(a) + 0.5*g(b)
for k in range(1,N):
    s += g(a+k*h)

I = s*h
print(I)
```

    10.819275852028298

That's, unsurprisingly, pretty bad, but it gets better with more steps:

```python
N = 1000
a = 0
b = 10
h = (b-a)/N
s = 0       # this just ensures we don't add more to it if we run it again!

s = 0.5*g(a) + 0.5*g(b)
for k in range(1,N):
    s += g(a+k*h)

I = s*h
print(I)
```

    9.999629330113484

That's only off by 0.0007%. We can keep doing this for a while, but at some point we are running too many iterations for the gain in precision. We will also run into the issue of machine precision, which is error inherent in the storing of floating point numbers in a computer.

There are better methods for numerical integration, which are inherently faster (fewer iterations for same or better precision) and/or adaptive to the problem (get you the best answer for your required precision). One of these is Gaussian quadrature, which the `quad` function from `scipy` performs:

```python
I = quad(g, 0, 10)
print(I)
```

    (9.999546000702377, 2.83275284251709e-13)

The first result is the integral result.

## A better integration scheme

For most problems, we will want to use a better integration scheme than the trapezoidal approached. As I introduced in class, a simple choice is Simpson's rule, which uses a quadratic shape to approximate the function curvature at each integration step, as in the figure below. Notice that we want to think about **pairs** of integration slices, each of which will be fit to a quadratic function.

Let's figure out how to do this mathematically. Suppose we have three points we want to lie on a quadratic, and they lie at $x=-h, 0, +h$, as in this figure:

A quadratic function is of the form $Ax^2+Bx+C$, where $A$, $B$, and $C$ are constants, so, by simply plugging in the three $x$ values, it must be that: $$f(-h)=Ah^2-Bh+C ~~~~~~~ f(0)=C ~~~~~~~ f(+h)=Ah^2+Bh+C.$$ These are three equations with three unknown constants, so we can solve the system of equations algebraically. Skipping through the algebra to the result, one can obtain:
$$ C=f(0) ~~~~~ A=\frac{1}{h^2}\left[ \frac{1}{2}f(-h)-f(0)+\frac{1}{2}f(h)\right] ~~~~~ B=\frac{1}{2h}\left[f(h)-f(-h)\right].$$
Now we can analytically integrate this result to obtain the area under this quadratic approximation: $$\int_{-h}^h\left( Ax^2+Bx+c\right) dx=\frac{2}{3}Ah^3+2Ch=\frac{1}{3}h\left[ f(-h)+4f(0)+f(h)\right]$$
This result is Simpson's rule. Notice it depends on **only** the function at 3 equally-spaced points, so it is easy to evaluate computationally.

If we are doing a numerical integral with $N$ bins numbered as $k=1,2,...,N$, the approximate area of a **pair** of adjacent bins is therefore $${\rm Area}\approx\frac{1}{3}h\left[ f(-x_k)+4f(x_{k+1})+f(x_{k+2})\right],$$ and the total integral is the sum of these pairs:
$$I(a,b)\approx\frac{1}{3}h\left[ f(a)+4f(a+h)+f(a+2h)\right]+\frac{1}{3}h\left[ f(a+2h)+4f(a+3h)+f(a+4h)\right]+...+\frac{1}{3}h\left[ f(a+(N-2)h)+4f(a+(N-1)h)+f(b)\right].$$
We can clearn this up by collecting like terms:
$$I(a,b)\approx\frac{1}{3}h\left[ f(a)+4f(a+h)+2f(a+2h)+4f(a+3h)+...+f(b)\right].$$
Notice something **important** here. All of the function terms that are calculated at a location depending on an odd number of $h$s are multiplied by 4, and all of the even ones are multplied by 2. Therefore,
$$I(a,b)\approx\frac{1}{3}h\left[ f(a)+f(b)+4\sum^{N-1}_{k_{\rm odd}}f(a+kh)+2\sum^{N-2}_{k_{\rm even}}f(a+kh)\right].$$

This is now in a form that is easy to implement in Python. However, the loop (or loops?) have that odd alternating behaviour. There are a variety of ways of dealing with that, but it is sometimes helpful to remember a property of the `range()` function: it can take steps of sizes that are different from 1, like so:

```python
N=10
for k in range(1,N,2): # starting at 1, take two steps at a time until N
    print(k, end = '')
print()
for k in range(2,N,2): # starting at 2, take two steps at a time until N
    print(k, end = '')
```

    13579
    2468

## An exercise: Numerically integrating functions without simple antiderivatives

Up till now, most of your experience with integration has likely been integrating continuous functions that have anti-derivatives. In the real world, many continuous functions do not have anti-derivatives, including raw, discrete data (e.g., using individual laboratory measurements of position to determine potential energy). Let's therefore turn our attention to functions for which there are no simple anti-derivatives.

Consider a line of charge that lives on the x-axis. It exists from $x=0$ to $x=3$, and distribution of that charge is given by,

$$\lambda(x) = 2e^{-x^2}.$$

In this problem, you will determine the total charge one this line by performing the integral,

$$Q = \int_0^3 2e^{-x^2} dx.$$

Complete the exercises below, either within this notebook or in another Python file or files. **You may not already know how to do these things. My explanations in this notebook will get you most of the way there, but you may find yourself stuck. You need to realize that much of learning to implement problems in code is the seeking out of the necessary information and tools. You should begin to get used to Googling for Python documentation, Wikipedia descriptions of algorithms, and helpful questions on StackExchange. Your assignment is to figure out how to do these things.**

1. Plot the distribution of charge. Does it look familiar to you in some way? Maybe you've seen it in other classes?
2. Use ``sympy`` to compute the integral of the function (i.e., the total charge). What does it return? Discuss the function. Is it analytic?
3. Apply the trapezoidal rule to compute the integral of this function. Use 10 equal width steps. To receive full credit, you must write the code to do this and not use built-in integration functions of ``scipy``.
4. Apply Simpson's rule to compute the integral of this function. Again, use 10 equal with steps. Compare the result to part 3. To receive full credit, you must write the code to do this and not use built-in integration functions of ``scipy``.
5. Compute both integrals with increasingly more steps (i.e., from 10 to 100) and plot the results. That is, produce a figure to compare the results of your integration schemes. Your plot(s) should have number of steps on the horizontal axis and the value of the integral on the vertical axis.
6. Use the ``scipy.integrate`` built-in quadrature function, ``quad.`` Use it to compute the same integral and compare its result to what your code produced. What are the two variables that ``quad`` gives you by default?
7. Explain in plain English how the trapezoidal rule and Simpson's rule are different. How do they compute the integrals? Explain how the [Gaussian quadrature](https://en.wikipedia.org/wiki/Gaussian_quadrature) method, which is used in ``quad`` from the ``scipy.integrate`` library, is different from both of them.


# Thompson Sampling

Thompson sampling is an ingenious algorithm that implicitly balances exploration and exploitation based on quality and uncertainty. Let's say we sample a 3-armed bandit and model the probability that each arm gives us a positive reward. The goal is of course to maximize our rewards by pulling the most promising arm. Assume at the current timestep arm-3 has mean reward of 0.9 over 800 pulls, arm-2 has mean reward of 0.8 over 800 pulls, and arm-1 has mean reward of 0.78 over 10 pulls. So far, arm-3 is clearly the best. But if we were to explore, would we choose arm-2 or arm-1? An $\epsilon$-greedy algorithm would, with probability $\epsilon$, just as likely choose arm-3, arm-2, or arm-1. However, arm-2 has been examined many times, as many as arm-1, and has a mean reward lower than arm-1. Selecting arm-2 seems like a wasteful exploratory action. Arm-1 however, has a lower mean reward than either arm-2 or arm-3, but has only been pulled a few times. In other words, arm-1 has a higher chance of being a better action than arm-3 when compared to arm-2, since we are more uncertain about its true value. The $\epsilon$-greedy algorithm completely misses this point. Thompson sampling, on the other hand, incorporates uncertainty by modelling the bandit's Bernouilli parameter with a prior beta distribution.

The beauty of the algorithm is that it always chooses the action with the highest expected reward, with the twist that this reward is weighted by uncertainty. It is in fact a Bayesian approach to the bandit problem. In our Bernouilli bandit setup, each action $k$ returns reward of 1 with probability $\theta_k$, and 0 with probability $1-\theta_k$. At the beginning of a simulation, each $\theta_k$ is sampled from a uniform distribution $\theta_k \sim Uniform(0,1)$ with $\theta_k$ held constant for the rest of that simulation (in the stationary case). The agent begins with a prior belief of the reward of each arm $k$ with a beta distribution, where $\alpha = \beta = 1$. The prior probability density of each $\theta_k$ is:

$$
p(\theta_k) = \frac{\Gamma(\alpha_k + \beta_k)}{\Gamma(\alpha_k)\Gamma(\beta_k)} \theta_k^{\alpha_k -1} (1-\theta_k)^{\beta_k-1}
$$
An action is chosen by first sampling from the beta distribution, followed by choosing the action with highest mean reward:$$
x_t = \text{argmax}_k (\hat{\theta}_k), \quad \hat{\theta}_k \sim \text{beta}(\alpha_k, \beta_k)
$$

According to Bayes' rule, an action's posterior distribution is updated depending on the reward $r_t$ received:$$
(\alpha_k, \beta_k) = (\alpha_k, \beta_k) + (r_t, 1-r_t)
$$

Thus the actions' posterior distribution are constantly updated throughout the simulation. We will measure the Thompson algorithm by comparing it with the $\epsilon$-greedy and Upper Confidence Bound (UCB) algorithms using regret. The per-period regret for the Bernouilli bandit problem is the difference between the mean reward of the optimal action minus the mean reward of the selected action:$$
\text{regret}_t(\theta) = \max_k \theta_k - \theta_{x_t}
$$

First we setup the necessary imports and the standard k-armed bandit. The get_reward_regret samples the reward for the given action, and returns the regret based on the true best action.

```python
import numpy as np
import matplotlib.pyplot as plt
from pdb import set_trace

stationary = True
class Bandit():
    def __init__(self, arm_count):
        """
        Multi-armed bandit with rewards 1 or 0
        At initialization, multiple arms are created. The probability of each arm returning reward 1
        if pulled is sample from Bernoulli(p), where randomly chosen from Uniform(0, 1) at initization
        """
        self.arm_count = arm_count
        self.generate_thetas()
        self.timestep = 0
        global stationary
        self.stationary=stationary

    def generate_thetas(self):
        self.thetas = np.random.uniform(0, 1, self.arm_count)

    def get_reward_regret(self, arm):
        """
        Returns random reward for arm action. Assument action are zero-indexed
        Args:
            arg is an int
        """
        self.timestep += 1
        if (self.stationary==False) and (self.timestep%100 == 0) :
          self.generate_thetas()
        # Simulate bernouilli sampling
        sim = np.random.uniform(0,1,self.arm_count)
        rewards = (sim < self.thetas).astype(int)
        reward = rewards[arm]
        regret = self.thetas.max() - self.thetas[arm]
        # print('reward', arm, reward)
        return reward, regret
```

We implement the two beta algorithms from [1], although we focus only on the Thompson algorithm. For the Bernouilli-greedy algorithm, the Bernouilli parameters are the expected values of the Beta distribution, i.e.:$$
\mathbb{E}(x_k) = \frac{\alpha_k}{(\alpha_k + \beta_k)}
$$

The Thompson algorithm follows the pseudocode below, based on [1]

Algorithm: Thompson($K$,$\alpha$, $\beta$)

for $t$ = 1,2, ..., do

    // sample action parameter from beta distribution

    for $k = 1, \dots, K$ do

        Sample $\hat{\theta}_k \sim \text{beta}(\alpha_k, \beta_k)$

    end for

    // select action, get reward

    $x_t \leftarrow \text{argmax}_k \hat{\theta}_k$

    $r_t \leftarrow \text{observe}(x_t)$

    // update beta parameters

    $(\alpha_{x_t}, \beta_{x_t}) \leftarrow (\alpha_{x_t}, \beta_{x_t})+(r_t, 1-r_t)$

end for

```python
class BetaAlgo():
  """
  The algos try to learn which Bandit arm is the best to maximize reward.

  It does this by modelling the distribution of the Bandit arms with a Beta,
  assuming the true probability of success of an arm is Bernouilli distributed.
  """
  def __init__(self, bandit):
    """
    Args:
      bandit: the bandit class the algo is trying to model
    """
    self.bandit = bandit
    self.arm_count = bandit.arm_count
    self.alpha = np.ones(self.arm_count)
    self.beta = np.ones(self.arm_count)

  def get_reward_regret(self, arm):
    reward, regret = self.bandit.get_reward_regret(arm)
    self._update_params(arm, reward)
    return reward, regret

  def _update_params(self, arm, reward):
    self.alpha[arm] += reward
    self.beta[arm] += 1 - reward

class BernGreedy(BetaAlgo):
  def __init__(self, bandit):
    super().__init__(bandit)

  @staticmethod
  def name():
    return 'beta-greedy'

  def get_action(self):
    """ Bernouilli parameters are the expected values of the beta"""
    theta = self.alpha / (self.alpha + self.beta)
    return theta.argmax()

class BernThompson(BetaAlgo):
  def __init__(self, bandit):
    super().__init__(bandit)

  @staticmethod
  def name():
    return 'thompson'

  def get_action(self):
    """ Bernouilli parameters are sampled from the beta"""
    theta = np.random.beta(self.alpha, self.beta)
    return theta.argmax()
```

For comparison, we also implement the $\epsilon$-greedy algorithm and Upper Confidence Bound (UBC) algorithm. The implementations are based on [2] (pages 24-28). The $\epsilon$-greedy algorithm is straightforward and explained briefly above. Note in this implementation we make use of the incremental update rule. That is, to update the $Q$-value of each action, we maintain a count of each action. For action $k$ taken at time $t$:$$
\begin{align}
r_t &\leftarrow \text{observe}(k) \\
N(k) &\leftarrow N(k) + 1 \\
Q(k) &\leftarrow Q(k) + \frac{1}{N(k)}[r_t-Q(k)] \\
\end{align}
$$

```python
epsilon = 0.1
class EpsilonGreedy():
  """
  Epsilon Greedy with incremental update.
  Based on Sutton and Barto pseudo-code, page. 24
  """
  def __init__(self, bandit):
    global epsilon
    self.epsilon = epsilon
    self.bandit = bandit
    self.arm_count = bandit.arm_count
    self.Q = np.zeros(self.arm_count) # q-value of actions
    self.N = np.zeros(self.arm_count) # action count

  @staticmethod
  def name():
    return 'epsilon-greedy'

  def get_action(self):
    if np.random.uniform(0,1) > self.epsilon:
      action = self.Q.argmax()
    else:
      action = np.random.randint(0, self.arm_count)
    return action

  def get_reward_regret(self, arm):
    reward, regret = self.bandit.get_reward_regret(arm)
    self._update_params(arm, reward)
    return reward, regret

  def _update_params(self, arm, reward):
    self.N[arm] += 1 # increment action count
    self.Q[arm] += 1/self.N[arm] * (reward - self.Q[arm]) # inc. update rule
```

The UCB action selection is different to the $\epsilon$-greedy. Like the Thompson algorithm, it includes a measure of uncertainty. The selected action follows the rule:$$
A_t = \text{argmax}_a \left[ Q_t(a) + c \sqrt{\frac{\ln t}{N_t (a)}} \right]
$$

Where $N_t (a)$ is the number of times action $a$ has been selected up to time $t$. As the denominator grows in the square root expression, the added effect on $Q_t(a)$ diminishes. This uncertainty measure is weighed by the hyperparameter $c$. The disadvantage is that, unlike the Thompson algorithm, this uncertainty hyperparameter requires tuning. Fundamentally, the UCB uncertainty is deterministic and beneficial, whereas in the Thompson case, uncertainty increases the expected reward variance. Since the Thompson algorithm samples the mean rewards from a beta distribution, the actions with high variance may not only have a higher chance of being chosen, but may also have a lower chance.

```python
ucb_c = 2
class UCB():
  """
  Epsilon Greedy with incremental update.
  Based on Sutton and Barto pseudo-code, page. 24
  """
  def __init__(self, bandit):
    global ucb_c
    self.ucb_c = ucb_c
    self.bandit = bandit
    self.arm_count = bandit.arm_count
    self.Q = np.zeros(self.arm_count) # q-value of actions
    self.N = np.zeros(self.arm_count) + 0.0001 # action count
    self.timestep = 1

  @staticmethod
  def name():
    return 'ucb'

  def get_action(self):
    ln_timestep = np.log(np.full(self.arm_count, self.timestep))
    confidence = self.ucb_c * np.sqrt(ln_timestep/self.N)
    action = np.argmax(self.Q + confidence)
    self.timestep += 1
    return action

  def get_reward_regret(self, arm):
    reward, regret = self.bandit.get_reward_regret(arm)
    self._update_params(arm, reward)
    return reward, regret

  def _update_params(self, arm, reward):
    self.N[arm] += 1 # increment action count
    self.Q[arm] += 1/self.N[arm] * (reward - self.Q[arm]) # inc. update rule
```

Below are some helper functions. The function simulate will simulate the learning for a single algorithm and return the mean regrets over a number of trials. The experiment function runs the simulations over all algorithms and plots their mean regrets

```python
def plot_data(y):
  """ y is a 1D vector """
  x = np.arange(y.size)
  _ = plt.plot(x, y, 'o')

def multi_plot_data(data, names):
  """ data, names are lists of vectors """
  x = np.arange(data[0].size)
  for i, y in enumerate(data):
    plt.plot(x, y, 'o', markersize=2, label=names[i])
  plt.legend(loc='upper right', prop={'size': 16}, numpoints=10)
  plt.show()

def simulate(simulations, timesteps, arm_count, Algorithm):
  """ Simulates the algorithm over 'simulations' epochs """
  sum_regrets = np.zeros(timesteps)
  for e in range(simulations):
    bandit = Bandit(arm_count)
    algo = Algorithm(bandit)
    regrets = np.zeros(timesteps)
    for i in range(timesteps):
      action = algo.get_action()
      reward, regret = algo.get_reward_regret(action)
      regrets[i] = regret
    sum_regrets += regrets
  mean_regrets = sum_regrets / simulations
  return mean_regrets

def experiment(arm_count, timesteps=1000, simulations=1000):
  """
  Standard setup across all experiments
  Args:
    timesteps: (int) how many steps for the algo to learn the bandit
    simulations: (int) number of epochs
  """
  algos = [EpsilonGreedy, UCB, BernThompson]
  regrets = []
  names = []
  for algo in algos:
    regrets.append(simulate(simulations, timesteps, arm_count, algo))
    names.append(algo.name())
  multi_plot_data(regrets, names)
```

## Experiments
For all experiments, in each trial the agents are allowed 1000 timesteps to maximize reward. We perform 5000 trials for each experiment.

### Baseline
In this first experiment, we aim for a standard setup, inspired by the bandit testbed in Chapter 2 of [2]. We set $\epsilon=0.1$ for the $\epsilon$-greedy algorithm, and $c=2$ for UCB. As can be seen in the chart below, the Thompson and $\epsilon$-greedy agents quickly converge to a steady regret value after only 200 steps. The UCB agent on the other hand very slowly decreases, lagging behind the two other agents, and continues in its downward trend even at step 1000. This suggests the non-Thompson agents could benefit from parameter tuning, whereas the Thompson agent works well right off the bat.

```python
# Experiment 1
arm_count = 10 # number of arms in bandit
epsilon = 0.1
ucb_c = 2
stationary=True
experiment(arm_count)
```

### Being greedier
After fiddling with UCB's $c$ parameter, we can see that if it is set to 0.1, it quickly converges to a low regret value, after only a few dozen steps and now outperforms $\epsilon$-greedy with very little variance. Note that at first it outperforms Thompson, but not in the long run. Decreasing UCB's $c$ basically decreased the "premium" the agent adds to uncertainty. This encourages UCB to be more "greedy", hence why it converges quickly. Yet it still does not outperform Thompson. Given this intuition, we also make $\epsilon$-greedy greedier, decreasing its hyperparameter to 0.03. We can see that after 1000 steps, it does outperform the previous $\epsilon$-greedy, but only marginally. Also, this comes at the cost of converging much more slowly. We do not observe the massive swing as observed with UCB, even if we continue to lower $\epsilon$. It's possible that given only 10 actions, being greedy is a better solution, and exploring is costly and unnecessary, leading us to the next experiment.

```python
# Experiment 2
arm_count = 10 # number of arms in bandit
epsilon = 0.03
ucb_c = 0.1
stationary=True
experiment(arm_count)
```

### Large action space
In this setting, we increase the number of actions from 10 to 200, leaving the other parameters equal to our baseline. The $\epsilon$-greedy agent barely budges in its learning pattern when compared to the baseline. The Thompson agent very slowly, but surely, decreases regret to eventually outperform $\epsilon$-greedy around step 700, another vote of confidence for Thompson. On the other hand, UCB is drastically unstable and diverges. This behaviour is not only due to the increase in action space, but due to the fact that the Bernouilli reward probabilities are so close to each other. Recall that regret is calculated as the difference between the optimal $\theta_k$ and the action $\hat{\theta}_k$. Since the true $\theta_k$ are sampled from a uniform distribution, they are evenly spread between 0 and 1. The greedy algorithm need only sample a few values and has a high probability of its most favourable action's estimated reward probability being close to the true optimal reward probability, ensuring it a low regret, even if it never chooses the optimal action. Thompson and UCB, on the other hand, with their emphasis on exploration, give a high premium to exploring actions not yet seen. Since there are so many actions, and since the mean rewards are evenly distributed between 0 and 1, there is a high cost of constantly exploring actions with rewards close to 0. Noticeably, the Thompson algorithm is incredibly stable across time, despite the large action space. Due to its sampling nature, it has a lower probability than UCB of constantly taking suboptimal actions.

```python

# Experiment 3
arm_count = 200 # number of arms in bandit
epsilon = 0.1
ucb_c = 2
stationary=True
experiment(arm_count)
```

We now lower UCB's $c$ variable and observe a dramatic increase in stability, which even leads it to outperform all other algorithms. However, the Thompson agent still seems to be on a decreasing regret trend around 1000 steps without having converged, in between $\epsilon$-greedy and UCB.

```python
# Experiment 4
arm_count = 200 # number of arms in bandit
epsilon = 0.1
ucb_c = 0.01
stationary=True
experiment(arm_count)
```

In the graph below, increasing the timesteps to 3000 confirms that Thompson and UCB closely converge.

```python

# Experiment 5
arm_count = 200 # number of arms in bandit
epsilon = 0.1
ucb_c = 0.01
stationary=True
experiment(arm_count, timesteps=3000, simulations=1000)
```

### Non-stationarity
We now reset our parameters to the baseline settings: 10-arm bandit, $\epsilon=0.1$ and UCB-c value of 2. However, we now set the environment to being non-stationary. Actually, after every 100 steps, the bandit resamples the action Bernouilli parameters. This places special emphasis on exploration. An algorithm that converges to an action and does not emphasize exploration will perform quite poorly as the true reward distribution changes over time. Below we see a clear jump in regret every time the true bandit parameters are resampled (every 100 steps). While the $\epsilon$-greedy and Thompson's regret quickly decrease, on average they increase over time. On the other hand, UCB reacts much more quickly to the bandit non-stationarity, and actually continues to improve in regret over time, no doubt due to its higher emphasis on exploration. We confirm this in the next experiment.

```python

# Experiment 6
arm_count = 10 # number of arms in bandit
epsilon = 0.1
ucb_c = 2
stationary=False
experiment(arm_count)
```

We now set UCB's $c$ parameter to 0.1 as we did in Experiment 2. Due to Experiment 2's setup, we concluded a greedier version of UCB would perform better in that setup, which it did. In the current setup, we can observe that the greedy UCB performs quite poorly and much like the other algorithms. Since we prefer a more exploratory algorithm, we fine tune $\epsilon$-greedy to be much less greedy and observe that it does not perform well. In fact, after trying for $\epsilon \in \{0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9\}$, $\epsilon$-greedy performs the same as in the chart below.

```python
# Experiment 7
arm_count = 10 # number of arms in bandit
epsilon = 0.6
ucb_c = 0.1
stationary=False
experiment(arm_count)
```

We have seen the advantages and disadvantages of various algorithms. The Thompson algorithm has been shown to perform quite well in various settings without any necessary parameter tuning to boot. In some cases, UCB performed close to Thompson, but required fine tuning. Exceptionally, UCB dealt better than Thompson in the non-stationary setup, but only after tuning its hyperparameter. Notably, in no cases was $\epsilon$-greedy the best performing agent.


# Simulating a Predator and Prey Relationship

Without a predator, rabbits will reproduce until they reach the carrying capacity of the land. When coyotes show up, they will eat the rabbits and reproduce until they can't find enough rabbits. We will explore the fluctuations in the two populations over time.

# Using Lotka-Volterra Model

## Part 1: Rabbits without predators

According to [Mother Earth News](https://www.motherearthnews.com/homesteading-and-livestock/rabbits-on-pasture-intensive-grazing-with-bunnies-zbcz1504), a rabbit eats six square feet of pasture per day. Let's assume that our rabbits live in a five acre clearing in a forest: 217,800 square feet/6 square feet = 36,300 rabbit-days worth of food. For simplicity, let's assume the grass grows back in two months. Thus, the carrying capacity of five acres is 36,300/60 = 605 rabbits.

Female rabbits reproduce about six to seven times per year. They have six to ten children in a litter.  According to [Wikipedia](https://en.wikipedia.org/wiki/Rabbit), a wild rabbit reaches sexual maturity when it is about six months old and typically lives one to two years. For simplicity, let's assume that in the presence of unlimited food, a rabbit lives forever, is immediately sexually mature, and has 1.5 children every month.

For our purposes, then, let $x_t$ be the number of rabbits in our five acre clearing on month $t$.
$$
\begin{equation*}
  R_t = R_{t-1} + 1.5\frac{605 - R_{t-1}}{605} R_{t-1}
\end{equation*}
$$

The formula could be put into general form
$$
\begin{equation*}
  R_t = R_{t-1} + growth_{R} \times \big( \frac{capacity_{R} - R_{t-1}}{capacity_{R}} \big) R_{t-1}
\end{equation*}
$$

By doing this, we allow users to interact with growth rate and the capacity value visualize different interaction

```python
from __future__ import print_function
from ipywidgets import interact, interactive, fixed, interact_manual
from IPython.display import display, clear_output
import ipywidgets as widgets
import matplotlib.pyplot as plt
import numpy as np
```

```python
%matplotlib inline
style = {'description_width': 'initial'}
capacity_R = widgets.FloatText(description="Capacity", value=605)
growth_rate_R = widgets.FloatText(description="Growth rate", value=1.5)
initial_R = widgets.FloatText(description="Initial population",style=style, value=1)
button_R = widgets.Button(description="Plot Graph")
display(initial_R, capacity_R, growth_rate_R, button_R)

def plot_graph_r(b):
    print("helo")
    clear_output()
    display(initial_R, capacity_R, growth_rate_R, button_R)
    fig = plt.figure()
    ax = fig.add_subplot(111)
    t = np.arange(0, 20, 1)
    s = np.zeros(t.shape)
    R = initial_R.value
    for i in range(t.shape[0]):
        s[i] = R
        R = R + growth_rate_R.value * (capacity_R.value - R)/(capacity_R.value) * R
        if R < 0.0:
            R = 0.0

    ax.plot(t, s)
    ax.set(xlabel='time (months)', ylabel='number of rabbits',
       title='Rabbits Without Predators')
    ax.grid()

button_R.on_click(plot_graph_r)
```

    FloatText(value=1.0, description='Initial population', style=DescriptionStyle(description_width='initial'))

    FloatText(value=605.0, description='Capacity')

    FloatText(value=1.5, description='Growth rate')

    Button(description='Plot Graph', style=ButtonStyle())

## Tweaking the Growth Function
The growth is regulated by this part of the formula:
$$
\begin{equation*}
  \frac{capacity_{R} - R_{t-1}}{capacity_{R}}
\end{equation*}
$$
That is, this fraction (and thus growth) goes to zero when the land is at capacity. As the number of rabbits goes to zero, this fraction goes to 1.0, so growth is at its highest speed.  We could substitute in another function that has the same values at zero and at capacity, but has a different shape. For example,
$$
\begin{equation*}
  \left( \frac{capacity_{R} - R_{t-1}}{capacity_{R}} \right)^{\beta}
\end{equation*}
$$
where $\beta$ is a positive number. For example, if $\beta$ is 1.3, it indicates that the rabbits can sense that food supplies are dwindling and pre-emptively slow their reproduction.

```python
%matplotlib inline
import math
style = {'description_width': 'initial'}
capacity_R_2 = widgets.FloatText(description="Capacity", value=605)
growth_rate_R_2 = widgets.FloatText(description="Growth rate", value=1.5)
initial_R_2 = widgets.FloatText(description="Initial population",style=style, value=1)
shaping_R_2 = widgets.FloatText(description="Shaping", value=1.3)
button_R_2 = widgets.Button(description="Plot Graph")
display(initial_R_2, capacity_R_2, growth_rate_R_2, shaping_R_2, button_R_2)

def plot_graph_r(b):
    clear_output()
    display(initial_R_2, capacity_R_2, growth_rate_R_2, shaping_R_2, button_R_2)
    fig = plt.figure()
    ax = fig.add_subplot(111)
    t = np.arange(0, 20, 1)
    s = np.zeros(t.shape)
    R = initial_R_2.value
    beta = float(shaping_R_2.value)
    for i in range(t.shape[0]):
        s[i] = R
        reserve_ratio = (capacity_R_2.value - R)/capacity_R_2.value
        if reserve_ratio > 0.0:
            R = R + R * growth_rate_R_2.value * reserve_ratio**beta
        else:
            R = R - R * growth_rate_R_2.value * (-1.0 * reserve_ratio)**beta
        if R < 0.0:
            R = 0

    ax.plot(t, s)
    ax.set(xlabel='time (months)', ylabel='number of rabbits',
       title='Rabbits Without Predators (Shaped)')
    ax.grid()

button_R_2.on_click(plot_graph_r)
```

## Part 2: Coyotes without Prey
According to [Huntwise](https://www.besthuntingtimes.com/blog/2020/2/3/why-you-should-coyote-hunt-how-to-get-started), coyotes need to consume about 2-3 pounds of food per day. Their diet is 90 percent mammalian. The perfect adult cottontail rabbits weigh 2.6 pounds on average. Thus, we assume the coyote eats one rabbit per day.

For coyotes, the breeding season is in February and March. According to [Wikipedia](https://en.wikipedia.org/wiki/Coyote#Social_and_reproductive_behaviors), females have a gestation period of 63 days, with an average litter size of 6, though the number fluctuates depending on coyote population density and the abundance of food. By fall, the pups are old enough to hunt for themselves.

In the absence of rabbits, the number of coyotes will drop, as their food supply is scarce.
The formula could be put into general form:

$$
\begin{align*}
  C_t & \sim (1 - death_{C}) \times C_{t-1}\\
  &= C_{t-1} - death_{C} \times C_{t-1}
\end{align*}
$$

```python
%matplotlib inline
style = {'description_width': 'initial'}
initial_C=widgets.FloatText(description="Initial Population",style=style,value=200.0)
declining_rate_C=widgets.FloatText(description="Death rate",value=0.5)
button_C=widgets.Button(description="Plot Graph")
display(initial_C, declining_rate_C, button_C)

def plot_graph_c(b):
    clear_output()
    display(initial_C, declining_rate_C, button_C)
    fig = plt.figure()
    ax = fig.add_subplot(111)
    t1 = np.arange(0, 20, 1)
    s1 = np.zeros(t1.shape)
    C = initial_C.value
    for i in range(t1.shape[0]):
        s1[i] = C
        C = (1 - declining_rate_C.value)*C

    ax.plot(t1, s1)
    ax.set(xlabel='time (months)', ylabel='number of coyotes',
       title='Coyotes Without Prey')
    ax.grid()

button_C.on_click(plot_graph_c)

```

    FloatText(value=200.0, description='Initial Population', style=DescriptionStyle(description_width='initial'))

    FloatText(value=0.5, description='Death rate')

    Button(description='Plot Graph', style=ButtonStyle())

## Part 3: Interaction Between Coyotes and Rabbit
With the simple interaction from the first two parts, now we can combine both interaction and come out with simple interaction.
$$
\begin{align*}
  R_t &= R_{t-1} + growth_{R} \times \big( \frac{capacity_{R} - R_{t-1}}{capacity_{R}} \big) R_{t-1} - death_{R}(C_{t-1})\times R_{t-1}\\\\
  C_t &= C_{t-1} - death_{C} \times C_{t-1} + growth_{C}(R_{t-1}) \times C_{t-1}
\end{align*}
$$

In equations above, death rate of rabbit is a function parameterized by the amount of coyote. Similarly, the growth rate of coyotes is a function parameterized by the amount of the rabbit.

The death rate of the rabbit should be $0$ if there are no coyotes, while it should approach $1$ if there are many coyotes. One of the formula fulfilling this characteristics is hyperbolic function.

$$
\begin{equation}
death_R(C) = 1 - \frac{1}{xC + 1}
\end{equation}
$$

where $x$ determines how quickly $death_R$ increases as the number of coyotes ($C$) increases. Similarly, the growth rate of the coyotes should be $0$ if there are no rabbits, while it should approach infinity if there are many rabbits. One of the formula fulfilling this characteristics is a linear function.

$$
\begin{equation}
growth_C(R) = yC
\end{equation}
$$

where $y$ determines how quickly $growth_C$ increases as number of rabbit ($R$) increases.

Putting all together, the final equtions are

$$
\begin{align*}
  R_t &= R_{t-1} + growth_{R} \times \big( \frac{capacity_{R} - R_{t-1}}{capacity_{R}} \big) R_{t-1} - \big( 1 - \frac{1}{xC_{t-1} + 1} \big)\times R_{t-1}\\\\
  C_t &= C_{t-1} - death_{C} \times C_{t-1} + yR_{t-1}C_{t-1}
\end{align*}
$$

```python
%matplotlib inline
initial_rabbit = widgets.FloatText(description="Initial Rabbit",style=style, value=1)
initial_coyote = widgets.FloatText(description="Initial Coyote",style=style, value=1)
capacity = widgets.FloatText(description="Capacity rabbits", style=style,value=5)
growth_rate = widgets.FloatText(description="Growth rate rabbits", style=style,value=1)
death_rate = widgets.FloatText(description="Death rate coyotes", style=style,value=1)
x = widgets.FloatText(description="Death rate ratio due to coyote",style=style, value=1)
y = widgets.FloatText(description="Growth rate ratio due to rabbit",style=style, value=1)
button = widgets.Button(description="Plot Graph")
display(initial_rabbit, initial_coyote, capacity, growth_rate, death_rate, x, y, button)
def plot_graph(b):
    clear_output()
    display(initial_rabbit, initial_coyote, capacity, growth_rate, death_rate, x, y, button)
    fig = plt.figure()
    ax = fig.add_subplot(111)
    t = np.arange(0, 20, 0.5)
    s = np.zeros(t.shape)
    p = np.zeros(t.shape)
    R = initial_rabbit.value
    C = initial_coyote.value
    for i in range(t.shape[0]):
        s[i] = R
        p[i] = C
        R = R + growth_rate.value * (capacity.value - R)/(capacity.value) * R - (1 - 1/(x.value*C + 1))*R
        C = C - death_rate.value * C  + y.value*s[i]*C

    ax.plot(t, s, label="rabit")
    ax.plot(t, p, label="coyote")
    ax.set(xlabel='time (months)', ylabel='population size',
       title='Coyotes-Rabbit (Predator-Prey) Relationship')
    ax.grid()
    ax.legend()

button.on_click(plot_graph)
```

    FloatText(value=1.0, description='Initial Rabbit', style=DescriptionStyle(description_width='initial'))

    FloatText(value=1.0, description='Initial Coyote', style=DescriptionStyle(description_width='initial'))

    FloatText(value=5.0, description='Capacity rabbits', style=DescriptionStyle(description_width='initial'))

    FloatText(value=1.0, description='Growth rate rabbits', style=DescriptionStyle(description_width='initial'))

    FloatText(value=1.0, description='Death rate coyotes', style=DescriptionStyle(description_width='initial'))

    FloatText(value=1.0, description='Death rate ratio due to coyote', style=DescriptionStyle(description_width='i…

    FloatText(value=1.0, description='Growth rate ratio due to rabbit', style=DescriptionStyle(description_width='…

    Button(description='Plot Graph', style=ButtonStyle())

### Trajectories and Direction Fields for a system of equations

To further demonstrate the predator numbers rise and fall cyclically with their preferred prey, we will be using the Lotka-Volterra equations, which is based on differential equations. The Lotka-Volterra Prey-Predator model involves two equations, one describes the changes in number of preys and the second one decribes the changes in number of predators. The dynamics of the interaction between a rabbit population $R_t$ and a coyotes $C_t$ is described by the following differential equations:
$$
\begin{align*}
\frac{dR}{dt} = aR_t - bR_tC_t
\end{align*}
$$

$$
\begin{align*}
\frac{dC}{dt} = bdR_tC_t - cC_t
\end{align*}
$$

with the following notations:

x: number of preys(rabbits)\
y: number of predators(coyotes)\
a: natural growing rate of rabbits, when there is no coyotes\
b: natural dying rate of rabbits, which is killed by coyotes per unit of time\
c: natural dying rate of coyotes, when ther is not rabbits\
d: natural growing rate of coyotes with which consumed prey is converted to predator

We start from defining the system of ordinary differential equations, and then find the equilibrium points for our system. Equilibrium occurs when the frowth rate is 0, and we can see that we have two equilibrium points in our example, the first one happens when theres no preys or predators, which represents the extinction of both species, the second equilibrium happens when $x=\frac{c}{b d}$ $y=\frac{a}{b}$. Move on, we will use the scipy to help us integrate the differential equations, and generate the plot of evolution for both species:

```python
from scipy import integrate

#using the same input number from the previous example
a = 1
b = 1
c = 1
d = 1
# Define the system of ODEs
# P[0] is prey, P[1] is predator
def dP_dt(P,t=0):
    return np.array([a*P[0]-b*P[0]*P[1], d*b*P[0]*P[1]-c*P[1]])
# Define the Equilibrium points
eq1 = np.array([0. , 0.])
eq2 = np.array([c/(d*b),a/b])
```

```python
values = np.linspace(0.1, 3, 10)
# Colors for each trajectory
vcolors = plt.cm.autumn_r(np.linspace(0.1, 1., len(values)))
f = plt.figure(figsize=(10,6))
t = np.linspace(0, 150, 1000)
for v, col in zip(values, vcolors):
    # Starting point
    P0 = v*eq2
    P = integrate.odeint(dP_dt, P0, t)
    plt.plot(P[:,0], P[:,1],
             lw= 1.5*v, # Different line width for different trajectories
             color=col, label='P0=(%.f, %.f)' % ( P0[0], P0[1]) )
ymax = plt.ylim(bottom=0)[1]
xmax = plt.xlim(left=0)[1]
nb_points   = 20
x = np.linspace(0, xmax, nb_points)
y = np.linspace(0, ymax, nb_points)
X1,Y1  = np.meshgrid(x, y)
DX1, DY1 = dP_dt([X1, Y1])
M = (np.hypot(DX1, DY1))
M[M == 0] = 1.
DX1 /= M
DY1 /= M
plt.title('Trajectories and direction fields')
Q = plt.quiver(X1, Y1, DX1, DY1, M, pivot='mid', cmap=plt.cm.plasma)
plt.xlabel('Number of rabbits')
plt.ylabel('Number of coyotes')
plt.legend()
plt.grid()
plt.xlim(0, xmax)
plt.ylim(0, ymax)
plt.show()

```

## Multiple Predators and Preys Relationship

The previous relationship could be extended to multiple predators and preys relationship

```python
%matplotlib inline
initial_rabbit2 = widgets.FloatText(description="Initial Rabbit", style=style,value=2)
initial_coyote2 = widgets.FloatText(description="Initial Coyote",style=style, value=2)
initial_deer2 = widgets.FloatText(description="Initial Deer", style=style,value=1)
initial_wolf2 = widgets.FloatText(description="Initial Wolf", style=style,value=1)
population_capacity = widgets.FloatText(description="capacity",style=style, value=10)
population_capacity_rabbit = widgets.FloatText(description="capacity rabbit",style=style, value=3)
growth_rate_rabbit = widgets.FloatText(description="growth rate rabbit",style=style, value=1)
death_rate_coyote = widgets.FloatText(description="death rate coyote",style=style, value=1)
growth_rate_deer = widgets.FloatText(description="growth rate deer",style=style, value=1)
death_rate_wolf = widgets.FloatText(description="death rate wolf",style=style, value=1)
x1 = widgets.FloatText(description="death rate ratio due to coyote",style=style, value=1)
y1 = widgets.FloatText(description="growth rate ratio due to rabbit", style=style,value=1)
x2 = widgets.FloatText(description="death rate ratio due to wolf",style=style, value=1)
y2 = widgets.FloatText(description="growth rate ratio due to deer", style=style,value=1)
plot2 = widgets.Button(description="Plot Graph")
display(initial_rabbit2, initial_coyote2,initial_deer2, initial_wolf2, population_capacity,
        population_capacity_rabbit, growth_rate_rabbit, growth_rate_deer, death_rate_coyote,death_rate_wolf,
        x1, y1,x2, y2, plot2)
def plot_graph(b):
    clear_output()
    display(initial_rabbit2, initial_coyote2,initial_deer2, initial_wolf2, population_capacity,
        population_capacity_rabbit, growth_rate_rabbit, growth_rate_deer, death_rate_coyote,death_rate_wolf,
        x1, y1,x2, y2, plot2)
    fig = plt.figure()
    ax = fig.add_subplot(111)
    t_m = np.arange(0, 20, 0.5)
    r_m = np.zeros(t_m.shape)
    c_m = np.zeros(t_m.shape)
    d_m = np.zeros(t_m.shape)
    w_m = np.zeros(t_m.shape)
    R_m = initial_rabbit2.value
    C_m = initial_coyote2.value
    D_m = initial_deer2.value
    W_m = initial_wolf2.value
    population_capacity_deer = population_capacity.value - population_capacity_rabbit.value
    for i in range(t_m.shape[0]):
        r_m[i] = R_m
        c_m[i] = C_m
        d_m[i] = D_m
        w_m[i] = W_m

        R_m = R_m + growth_rate_rabbit.value * (population_capacity_rabbit.value - R_m)\
            /(population_capacity_rabbit.value) * R_m - (1 - 1/(x1.value*C_m + 1))*R_m - (1 - 1/(x2.value*W_m + 1))*R_m
        D_m = D_m + growth_rate_deer.value * (population_capacity_deer - D_m) \
            /(population_capacity_deer) * D_m - (1 - 1/(x1.value*C_m + 1))*D_m - (1 - 1/(x2.value*W_m + 1))*D_m

        C_m = C_m - death_rate_coyote.value * C_m  + y1.value*r_m[i]*C_m + y2.value*d_m[i]*C_m
        W_m = W_m - death_rate_wolf.value * W_m  + y1.value*r_m[i]*W_m + y2.value*d_m[i]*W_m

    ax.plot(t_m, r_m, label="rabit")
    ax.plot(t_m, c_m, label="coyote")
    ax.plot(t_m, d_m, label="deer")
    ax.plot(t_m, w_m, label="wolf")
    ax.set(xlabel='time (months)', ylabel='population',
       title='Multiple Predator Prey Relationship')
    ax.grid()
    ax.legend()

plot2.on_click(plot_graph)
```

```python

```


```python
#SIR EPIDEMIC MODEL
```

```python
import matplotlib.pyplot as plt
import numpy as np
import sympy as sp
import scipy.linalg as sl
from sympy.interactive import printing
printing.init_printing(use_latex=True)
```

```python
#Defining the symbols.
S,I,V,= sp.symbols('S,I,V')
rho,mu,beta,alpha,gamma,N=sp.symbols('rho,mu,beta,alpha,gamma,N')
```

```python
#Equations representing our system
dot_S_t=(1-rho)*mu*N-beta*S*I/N-(mu+alpha)*S
display(dot_S_t)
dot_I_t=beta*S*I/N-(mu+gamma)*I
display(dot_I_t)
dot_V_t=rho*mu*N+gamma*I-mu*V+alpha*S
display(dot_V_t)
```

```python
#Creating equations from the expressions above.
SEq = sp.Eq(dot_S_t,0)
display(SEq)
IEq = sp.Eq(dot_I_t,0)
display(IEq)
VEq = sp.Eq(dot_V_t,0)
display(VEq)
```

```python
#Determining the equilibrium points.
EqPnts = sp.solve( (SEq,IEq,VEq), S,I,V )
display(EqPnts)
```

```python
#Linearization of the system.
Matrix=sp.Matrix([dot_S_t,dot_I_t,dot_V_t])
Matrix
```

```python
Mat=sp.Matrix([S,I,V])
Mat
```

```python
JacobianMat=Matrix.jacobian(Mat)
JacobianMat
```

```python
EV1=JacobianMat.eigenvals()
display(EV1)
```

```python
p=EqPnts[1][0]
q=EqPnts[1][1]
r=EqPnts[1][2]
Jacsub=JacobianMat.subs([(S,p),(I,q),(V,r)])
Jacsub
```

```python
EV1=Jacsub.eigenvals()
display(EV1)
```

```python
p=EqPnts[0][0]
q=EqPnts[0][1]
r=EqPnts[0][2]
Jacsub1=JacobianMat.subs([(S,p),(I,q),(V,r)])
Jacsub1
```

```python
EV1=Jacsub1.eigenvals()
display(EV1)
```

```python

```


# 04. Números de punto flotante y redondeo

### 1. Números de punto flotante y la norma IEEE 754

Los números de punto flotante son números reales que tienen la forma:

\begin{equation}
x = (-1)^\sigma m \times \beta^e,
\end{equation}
donde $\beta\geq 2$ es un entero mayor que define la *base*, $m = (b_0.b_1\, b_2\, b_3  \cdots)_\beta$ es la *mantisa* o *significante* escrita en la base $\beta$, donde $b_i = {0, 1, \dots, \beta-1}$, y $\sigma$ sirve para definir el signo. Finalmente, $e$, el exponente, es un número entero. Para evitar *las colas infinitas de nueves*, que en binario son las de unos, uno además pide que $0\leq b_i \leq \beta-2$ para un número infinito de $i$'s.

Este conjunto es *no numerable*, como los reales. Esta representación *no es única*.

Llevar los reales a la computadora obviamente involucra hacer dos concesiones:  en primer lugar, uno debe considerar conjuntos *finitos* de números como modelo de los reales, y en segundo término, que la representación sea *única*.

En particular, pedimos que el primer dígito $b_0$ no sea cero para todo $x$ distinto de cero; los números de punto flotante que satisfacen esta condición se llaman *normales* o *normalizados*.

Hacer que el número de *bits* (o sea, la base es $\beta=2$) sea finito no es suficiente considerar
que la mantisa consista de un número finito de entradas, sino además que el rango de $e$ sea *finito*. Esto es:

\begin{eqnarray}
m & = & (b_0.b_1\,b_2\,\cdots \, b_p),\\
e & \in & [ e_{min}, e_{max} ].\\
\end{eqnarray}

Aquí, $p$ se conoce como el número de *dígitos significativos* (*precision* en inglés), que **no** hay que confundir con la precisión (*accuracy*).

En el caso de la representación binaria, dado que $b_0=1$, entonces uno *puede* obviar esto, lo que significa que el bit $b_0$ **no** se guarda. A esta convención se le llama la *regla del bit escondido* (*hidden bit rule*).

-----

**Ejercicio 1**

Dados $p$, $e_{min}$ y $e_{max}$, usando el sistema binario, ¿de cuántos números *normales* consiste el conjunto? ¿Y si se trata de otra base $\beta$? ¿Están uniformemente distribuidos los números normales? ¿Cuál es la distancia entre el 0 y el primer número positivo distinto de cero?
*NOTA* Para guiar la respuesta, uno puede jugar con $\beta=2$, $p=2$,  $e_{min} = -1$ y $e_{max} = 2$, pero la idea es obtener expresiones generales.

(Entre paréntesis, a estas alturas del partido deben haber escrito funciones que permiten pasar de un entero a una cadena binaria equivalente, y viceversa, y de un número de punto flotante en base 10 al equivalente en binario, y viceversa. No sólo se trata de que estas funciones estén escritas, sino de que nos sean *útiles*, o sea, de que den resultados que podemos confiar y usar.)

```

```

---

**Ejercicio 2**

¿Cómo se representa al $x=0$ en el conjunto anterior? ¿Funciona la regla del bit escondido para una base $\beta>2$?

```

```

Del ejercicio 1 arriba, uno se da cuenta de que hay una distancia anómalamente grande entre 0 y el primer número positivo. Esto trae problemas prácticos relacionados con la pérdida de precisión cuando se tienen cantidades cercanas a cero. La manera de evitar esto es permitir que haya ciertos números que no son normales; a éstos se les llama *subnormales*. Un número distinto de cero es subnormal si $b_0=0$. Esto permite llegar a cero de manera gradual al incorporar estos números al conjunto de números representables.

---

**Ejercicio 3**

En el ejemplo de juguete anterior ($\beta=2$, $p=2$,  $e_{min} = -1$ y $e_{max} = 2$) enumera los posibles números subnormales. ¿Cómo cambia la distancia de 0 al número positivo más próximo? ¿El espaciamiento entre los números subnormales, es uniforme?

```

```

---

**Ejercicio 4**

¿Qué expresiones definen a $N_{min}$ y $N_{max}$, los números de punto flotante (positivos) mínimo y máximo? Obtén los resultados para los números normales y los subnormales.

La norma [IEEE 754](http://en.wikipedia.org/wiki/IEEE_floating_point) define el estándar de la representación de los números de punto flotante en las computadoras. En particular, aquí nos interesará la representación *en doble precisión* o de 64-bits, que consiste en el primer bit que contiene el signo $\sigma$, de un exponente $e$ de 11-bits que tiene un *sesgo* (*bias*) $B_*$ y de la mantisa que consta de $p=52$ bits (de facto, $p=53$, si se incluye la regla del bit escondido). El resultado claramente son los 64 bits de almacenamiento.

El sesgo se introduce por la siguiente razón. Sin sesgo, el exponente $e$ varía entre 0 y $2^{E_*} - 1$, donde $E_*$ son el número de bits asignados al exponente. Entonces, el sesgo se introduce *restando* $B_* = 2^{E_*-1} - 1$ del exponente, lo que de facto hace que el rango del exponente sea $e \in [-2^{E_*-1} + 1, 2^{E_*-1}]$.

---

**Ejercicio 5**

Usando `bits` y un poco de imaginación, cuáles son las reglas para representar
- $x=$`NaN`

- $x=$`Inf`

- un número $x>0$ *normal*

- un número $x$ *subnormal*

- $x=0.0$ y $x = -0.0$

```

```

---

**Ejercicio 6**

- Dado que los números de punto flotante son *finitos*, construye la representación decimal de dos números normales de punto flotante *consecutivos*, y dos subnormales consecutivos.

- ¿Cuál es la representación binaria del promedio de esos dos números? ¿Qué obtienes a partir de tus programas?

- ¿Cuál es la representación binaria de la diferencia?

```

```

---

**Ejercicio 7** Donde *todo esto*, y mucho más, puede ser importante...

Escribe una función que evalúe

\begin{equation}
f(x,y) = 333.75 y^6 + x^2 (11 x^2 y^2 - y^6 - 121 y^4 - 2) + 5.5 y^8 + \frac{x}{2y}
\end{equation}

Usando esta función, *demuestra* que $f(x=77617, y=33096) = -0.8273960599468214$.

**NOTA** Los números han sido verificados y *son correctos* :-)

```

```

---

**Ejercicio 8**

Pensando en que vas a generar datos para hacer una gráfica (con suficientes puntos), genera dichos datos para los *polinomios* $f(x) = (1-x)^6$ y $g(x) = x^6 -6x^5 + 15x^4 -20 x^3 + 15x^2 - 6 x + 1$, en el dominio $0.995 \leq x \leq 1.005$. ¿Hay algo sorprendente en esto?

```

```

---

**Ejercicio 9**

Explica por qué es mala idea usar como paso de integración $0.1$ o $0.01$, u otros valores de este estilo. ¿Qué pasa con el tiempo que vas calculando en cada paso de la integración?

```

```

notebooks/04. Numeros de punto flotante.ipynb


### (a): Graphically show that the function is unimodal on the interval [0,3]. (You may use software to create the graph.)

```python
import numpy as np
import pandas as pd
```

```python
#generate a list of x values with end points 0 and 3
start = 0
stop = 3
x_values = np.linspace(start, stop, num = 50, endpoint = True, retstep = False,   dtype = float)

x_list, f_x = [], [] #create lists to store data
for value in x_values:
    temp = (0.5/np.sqrt(1+value**2))-(np.sqrt(1+value**2)*(1-(0.5/(1+value**2))))+value
    x_list.append(value)
    f_x.append(temp)

fx_max_value = max(f_x) #Return the max value of the list.
fx_max_index = f_x. index(fx_max_value) #return index of max value
x_at_max_value = x_list[fx_max_index] #get x value corresponding to fx_max
```

```python
fx_max_value, x_at_max_value
```

    (0.3002635410437066, 0.7959183673469388)

```python
# importing the required module
import matplotlib.pyplot as plt

plt.figure(figsize=(8, 4))
plt.plot(x_list, f_x)
plt.axvline(x = x_at_max_value, ymin = min(f_x), ymax=0.95, linestyle = ":", color="firebrick", label="Maximizer")
plt.xlabel('x', fontsize = 18)
plt.ylabel('f(x)', fontsize = 18)
plt.xticks(fontsize = 16)
plt.yticks(fontsize = 16)
plt.legend(loc="center", fontsize = 16)
plt.title('Grapgh of f(x)', fontsize = 20)
plt.show()
```

A function is unimodal on an interval if it has a single minimizer or single maximizer on that interval, and not both. The displayed graph has only one maximizer

### (b) Use the bisection method to find a MAXIMUM for this function with a tolerance of 10^(-5)

The bisection method is part of family of elimination methods which are numerical methods for nonlinear optimization (single variable). In this method, the size of the region containing the optimal solution is reduced in each round of iteration.

__1.__ The bisection method assumes that the function f(x) to be minimized is continuously differentiable in the given closed interval [a, b]

__2.__ $\Delta$f(a)$\Delta$f(b)<0

**Steps:**

1. Let C = 0.5(a+b). If b-c >

```python
import sympy as sympy
import math
```

```python
def logistic_funtcion(x):
    return 1/(1 + sympy.exp(-x))

def differential():
    x = sympy.symbols('x')
    f_x = logistic_funtcion(x)
    return f_x.diff(x)

f_x_prime = differential()
print(f_x_prime)
```

    exp(-x)/(1 + exp(-x))**2

#### Bisection method

```python
def bisection(fx, fxprime,a,b,Threshold):
    """
    proximate solution of f(x)=0 on interval [a,b] by bisection method.

    Parameters
    ----------
    fx : Function
        The function to be maximized
    fxprime: differential of the function to be maximized
    a,b : Numbers
        The interval to search for a solution.
    Threshold: float
        The threshold of approximation error to terminate iteration
    @eturns
    -------
    df:
       pandas dataframe of a,b, midpoint values, function evalauted at each iteration, slope of function at each iteration
    """

    #create list to store data
    a_list, b_list, c_list, diff_list, fprime_b_list, fprime_c_list, mult_list, fx_c_list= [],[],[],[],[],[],[],[]

    #calculate midpoint
    c = (a + b)/2

    #calculate difference
    diff = b-c

    #if the difference is smaller or equal to threshold, return
    if diff <= Threshold:
        return c

    #find optimal point by iteration
    while diff >Threshold:
        fprime_b = fxprime(a) #evaluate differential at lower limit
        fprime_c = fxprime(b) #evaluate differential at upperit
        mult = fprime_b*fprime_c

        #evaluate function at current optimal
        fx_c = fx(c)
        a_list.append(a), b_list.append(b), c_list.append(c), diff_list.append(diff)

        fprime_b_list.append(fprime_b), fprime_c_list.append(fprime_c), mult_list.append(mult), fx_c_list.append(fx_c)

        if mult <=0:
            a = c
        else:
            b = c

        #recalculate the medpoint
        c = (a + b)/2

        #recalculate the diff
        diff = b-c

    #save data in a pandas dataframe
    df = pd.DataFrame(zip(a_list, b_list, c_list, diff_list, fprime_b_list, fprime_c_list, mult_list, fx_c_list),
                     columns = ["a", "b", "c", "b-c", "f'(b)", "f'(c)", "f'(b)*f'(c)", "f(c)"]
                     )
    return df
```

```python
def class_example(x):
    return x**5 - 5*x**3 - 20*x + 5

def differential():
    x = sympy.symbols('x')
    f_x = class_example(x)
    return f_x.diff(x)

examp_prime = differential()
examp_prime
```

$\displaystyle 5 x^{4} - 15 x^{2} - 20$

```python
fx = lambda x: x**5 - 5*x**3 - 20*x + 5
#differential of function
fxprime = lambda x: 5*x**4 - 15*x**2 - 20
x_optimal = bisection(fx,fxprime, 0,3,0.00001)
```

```python
x_optimal
```

      a
      b
      c
      b-c
      f'(b)
      f'(c)
      f'(b)*f'(c)
      f(c)

      0
      0.00
      3.000000
      1.500000
      1.500000
      -20.000000
      250.000000
      -5000.000000
      -34.281250

      1
      1.50
      3.000000
      2.250000
      0.750000
      -28.437500
      250.000000
      -7109.375000
      -39.288086

      2
      2.25
      3.000000
      2.625000
      0.375000
      32.207031
      250.000000
      8051.757812
      -13.302582

      3
      2.25
      2.625000
      2.437500
      0.187500
      32.207031
      114.044189
      3673.024774
      -30.116387

      4
      2.25
      2.437500
      2.343750
      0.093750
      32.207031
      67.380447
      2170.124175
      -35.525708

      5
      2.25
      2.343750
      2.296875
      0.046875
      32.207031
      48.476796
      1561.293689
      -37.597503

      6
      2.25
      2.296875
      2.273438
      0.023438
      32.207031
      40.027089
      1289.153720
      -38.488612

      7
      2.25
      2.273438
      2.261719
      0.011719
      32.207031
      36.040124
      1160.745400
      -38.899578

      8
      2.25
      2.261719
      2.255859
      0.005859
      32.207031
      34.104563
      1098.406720
      -39.096612

      9
      2.25
      2.255859
      2.252930
      0.002930
      32.207031
      33.151071
      1067.697567
      -39.193040

      10
      2.25
      2.252930
      2.251465
      0.001465
      32.207031
      32.677873
      1052.457268
      -39.240736

      11
      2.25
      2.251465
      2.250732
      0.000732
      32.207031
      32.442158
      1044.865592
      -39.264454

      12
      2.25
      2.250732
      2.250366
      0.000366
      32.207031
      32.324521
      1041.076860
      -39.276281

      13
      2.25
      2.250366
      2.250183
      0.000183
      32.207031
      32.265758
      1039.184270
      -39.282186

      14
      2.25
      2.250183
      2.250092
      0.000092
      32.207031
      32.236390
      1038.238418
      -39.285137

      15
      2.25
      2.250092
      2.250046
      0.000046
      32.207031
      32.221709
      1037.765603
      -39.286611

      16
      2.25
      2.250046
      2.250023
      0.000023
      32.207031
      32.214370
      1037.529223
      -39.287349

      17
      2.25
      2.250023
      2.250011
      0.000011
      32.207031
      32.210701
      1037.411040
      -39.287717

```python
def function(x):
    return (0.5*((1 + x**2)**-0.5))-(((1+ x**2)**0.5)*(1-(0.5*(1 + x**2)**-1))) + x

def differential():
    x = sympy.symbols('x')
    f_x = function(x)
    return f_x.diff(x)

f_x_prime = differential()
f_x_prime
```

$\displaystyle - \frac{1.0 x \left(1 - \frac{0.5}{x^{2} + 1}\right)}{\left(x^{2} + 1\right)^{0.5}} - \frac{1.5 x}{\left(x^{2} + 1\right)^{1.5}} + 1$

```python
#function to optimize
fx = lambda x: (0.5*((1 + x**2)**-0.5))-(((1+ x**2)**0.5)*(1-(0.5*(1 + x**2)**-1))) + x

#differential of function
fxprime = lambda x: (-x*(1 - 0.5/(1+x**2))/(1+x**2)**0.5) - (1.5*x/(x**2+1)**1.5) + 1
#fxprime = lambda x: 1 - x*((1+x**2)**-1.5+(1+x**2)**-0.5)

x_optimal = bisection(fx,fxprime, 0,3,0.00001)
```

```python
x_optimal
```

      a
      b
      c
      b-c
      f'(b)
      f'(c)
      f'(b)*f'(c)
      f(c)

      0
      0.0
      3.000000
      1.500000
      1.500000
      1.000000
      -0.043552
      -0.043552
      0.251925

      1
      1.5
      3.000000
      2.250000
      0.750000
      -0.088066
      -0.043552
      0.003835
      0.193924

      2
      1.5
      2.250000
      1.875000
      0.375000
      -0.088066
      -0.064543
      0.005684
      0.220588

      3
      1.5
      1.875000
      1.687500
      0.187500
      -0.088066
      -0.077753
      0.006847
      0.235758

      4
      1.5
      1.687500
      1.593750
      0.093750
      -0.088066
      -0.083880
      0.007387
      0.243742

      5
      1.5
      1.593750
      1.546875
      0.046875
      -0.088066
      -0.086345
      0.007604
      0.247813

      6
      1.5
      1.546875
      1.523438
      0.023438
      -0.088066
      -0.087320
      0.007690
      0.249864

      7
      1.5
      1.523438
      1.511719
      0.011719
      -0.088066
      -0.087724
      0.007726
      0.250893

      8
      1.5
      1.511719
      1.505859
      0.005859
      -0.088066
      -0.087903
      0.007741
      0.251409

      9
      1.5
      1.505859
      1.502930
      0.002930
      -0.088066
      -0.087987
      0.007749
      0.251667

      10
      1.5
      1.502930
      1.501465
      0.001465
      -0.088066
      -0.088027
      0.007752
      0.251796

      11
      1.5
      1.501465
      1.500732
      0.000732
      -0.088066
      -0.088046
      0.007754
      0.251860

      12
      1.5
      1.500732
      1.500366
      0.000366
      -0.088066
      -0.088056
      0.007755
      0.251892

      13
      1.5
      1.500366
      1.500183
      0.000183
      -0.088066
      -0.088061
      0.007755
      0.251908

      14
      1.5
      1.500183
      1.500092
      0.000092
      -0.088066
      -0.088063
      0.007755
      0.251916

      15
      1.5
      1.500092
      1.500046
      0.000046
      -0.088066
      -0.088065
      0.007755
      0.251921

      16
      1.5
      1.500046
      1.500023
      0.000023
      -0.088066
      -0.088065
      0.007756
      0.251923

      17
      1.5
      1.500023
      1.500011
      0.000011
      -0.088066
      -0.088065
      0.007756
      0.251924

### Golden section search method

```python
def golden_section_search_method(fx,a,b,Threshold):
    """
    proximate solution of f(x)=0 on interval [a,b] by bisection method.

    Parameters
    ----------
    fx : Function
        The function to be maximized
    a,b : Numbers
        The interval to search for a solution.
    Threshold: float
        The threshold of approximation error to terminate iteration
    @eturns
    -------
    df:
       pandas dataframe of a,b, midpoint values, function evalauted at each iteration, slope of function at each iteration
    """

    #create list to store data
    a_list, b_list, diff_list, x1_list, x2_list, fx_x1_list, fx_x2_list= [],[],[],[],[],[],[]

    #Golden ration
    gamma = (-1+ math.sqrt(5))/2
    #calculate difference
    diff = b-a

    #if the difference is smaller or equal to threshold, return
    if diff <= Threshold:
        if fx(a) < fx(b):
            return a #minimizer
        else:
            return b #as minimizer

    #find optimal point by iteration
    while diff >Threshold:
        x1 = b - gamma*(b-a)
        x2 = a + gamma*(b-a)
        fx_x1 = fx(x1) #evaluate differential at lower limit
        fx_x2 = fx(x2) #evaluate differential at upperit

        a_list.append(a), b_list.append(b), diff_list.append(diff)

        x1_list.append(x1), x2_list.append(x2), fx_x1_list.append(fx_x1), fx_x2_list.append(fx_x2)

        if fx_x1 < fx_x2:
            b = x2
        else:
            a = x1
            #if the difference is smaller or equal to threshold, return
        if diff <= Threshold:
            if fx(a) < fx(b):
                return a #minimizer
            else:
                return b #as minimizer

        diff = b-a
    #save data in a pandas dataframe
    df = pd.DataFrame(zip(a_list, b_list, diff_list, x1_list, x2_list, fx_x1_list, fx_x2_list),
                     columns = ["a", "b", "b-a", "x1", "x2", "f(x1)", "f(x2)"])
    return df
```

```python
#function to optimize
fx = lambda x: (0.5*((1 + x**2)**-0.5))-(((1+ x**2)**0.5)*(1-(0.5*(1 + x**2)**-1))) + x
x_optimal = golden_section_search_method(fx,0,3,0.00001)
x_optimal
```

      a
      b
      b-a
      x1
      x2
      f(x1)
      f(x2)

      0
      0.000000
      3
      3.000000
      1.145898
      1.854102
      0.282529
      0.222221

      1
      1.145898
      3
      1.854102
      1.854102
      2.291796
      0.222221
      0.191256

      2
      1.854102
      3
      1.145898
      2.291796
      2.562306
      0.191256
      0.175343

      3
      2.291796
      3
      0.708204
      2.562306
      2.729490
      0.175343
      0.166590

      4
      2.562306
      3
      0.437694
      2.729490
      2.832816
      0.166590
      0.161552

      5
      2.729490
      3
      0.270510
      2.832816
      2.896674
      0.161552
      0.158571

      6
      2.832816
      3
      0.167184
      2.896674
      2.936141
      0.158571
      0.156777

      7
      2.896674
      3
      0.103326
      2.936141
      2.960533
      0.156777
      0.155686

      8
      2.936141
      3
      0.063859
      2.960533
      2.975608
      0.155686
      0.155019

      9
      2.960533
      3
      0.039467
      2.975608
      2.984925
      0.155019
      0.154609

      10
      2.975608
      3
      0.024392
      2.984925
      2.990683
      0.154609
      0.154357

      11
      2.984925
      3
      0.015075
      2.990683
      2.994242
      0.154357
      0.154201

      12
      2.990683
      3
      0.009317
      2.994242
      2.996441
      0.154201
      0.154105

      13
      2.994242
      3
      0.005758
      2.996441
      2.997801
      0.154105
      0.154046

      14
      2.996441
      3
      0.003559
      2.997801
      2.998641
      0.154046
      0.154009

      15
      2.997801
      3
      0.002199
      2.998641
      2.999160
      0.154009
      0.153987

      16
      2.998641
      3
      0.001359
      2.999160
      2.999481
      0.153987
      0.153973

      17
      2.999160
      3
      0.000840
      2.999481
      2.999679
      0.153973
      0.153964

      18
      2.999481
      3
      0.000519
      2.999679
      2.999802
      0.153964
      0.153959

      19
      2.999679
      3
      0.000321
      2.999802
      2.999877
      0.153959
      0.153955

      20
      2.999802
      3
      0.000198
      2.999877
      2.999924
      0.153955
      0.153953

      21
      2.999877
      3
      0.000123
      2.999924
      2.999953
      0.153953
      0.153952

      22
      2.999924
      3
      0.000076
      2.999953
      2.999971
      0.153952
      0.153951

      23
      2.999953
      3
      0.000047
      2.999971
      2.999982
      0.153951
      0.153951

      24
      2.999971
      3
      0.000029
      2.999982
      2.999989
      0.153951
      0.153951

      25
      2.999982
      3
      0.000018
      2.999989
      2.999993
      0.153951
      0.153950

      26
      2.999989
      3
      0.000011
      2.999993
      2.999996
      0.153950
      0.153950

```python
#class example
fx = lambda x: x**5 - 5*x**3 - 20*x + 5
x_optimal = golden_section_search_method(fx,0,3,0.00001)
x_optimal
```

      a
      b
      b-a
      x1
      x2
      f(x1)
      f(x2)

      0
      0.000000
      3.000000
      3.000000
      1.145898
      1.854102
      -23.465512
      -42.039920

      1
      1.145898
      3.000000
      1.854102
      1.854102
      2.291796
      -42.039920
      -37.798567

      2
      1.145898
      2.291796
      1.145898
      1.583592
      1.854102
      -36.569198
      -42.039920

      3
      1.583592
      2.291796
      0.708204
      1.854102
      2.021286
      -42.039920
      -42.977005

      4
      1.854102
      2.291796
      0.437694
      2.021286
      2.124612
      -42.977005
      -42.153429

      5
      1.854102
      2.124612
      0.270510
      1.957428
      2.021286
      -42.912047
      -42.977005

      6
      1.957428
      2.124612
      0.167184
      2.021286
      2.060753
      -42.977005
      -42.807468

      7
      1.957428
      2.060753
      0.103326
      1.996894
      2.021286
      -42.999519
      -42.977005

      8
      1.957428
      2.021286
      0.063859
      1.981819
      1.996894
      -42.983682
      -42.999519

      9
      1.981819
      2.021286
      0.039467
      1.996894
      2.006211
      -42.999519
      -42.998063

      10
      1.981819
      2.006211
      0.024392
      1.991136
      1.996894
      -42.996096
      -42.999519

      11
      1.991136
      2.006211
      0.015075
      1.996894
      2.000453
      -42.999519
      -42.999990

      12
      1.996894
      2.006211
      0.009317
      2.000453
      2.002653
      -42.999990
      -42.999648

      13
      1.996894
      2.002653
      0.005758
      1.999094
      2.000453
      -42.999959
      -42.999990

      14
      1.999094
      2.002653
      0.003559
      2.000453
      2.001293
      -42.999990
      -42.999916

      15
      1.999094
      2.001293
      0.002199
      1.999934
      2.000453
      -43.000000
      -42.999990

      16
      1.999094
      2.000453
      0.001359
      1.999613
      1.999934
      -42.999993
      -43.000000

      17
      1.999613
      2.000453
      0.000840
      1.999934
      2.000132
      -43.000000
      -42.999999

      18
      1.999613
      2.000132
      0.000519
      1.999811
      1.999934
      -42.999998
      -43.000000

      19
      1.999811
      2.000132
      0.000321
      1.999934
      2.000010
      -43.000000
      -43.000000

      20
      1.999934
      2.000132
      0.000198
      2.000010
      2.000056
      -43.000000
      -43.000000

      21
      1.999934
      2.000056
      0.000123
      1.999981
      2.000010
      -43.000000
      -43.000000

      22
      1.999981
      2.000056
      0.000076
      2.000010
      2.000028
      -43.000000
      -43.000000

      23
      1.999981
      2.000028
      0.000047
      1.999999
      2.000010
      -43.000000
      -43.000000

      24
      1.999981
      2.000010
      0.000029
      1.999992
      1.999999
      -43.000000
      -43.000000

      25
      1.999992
      2.000010
      0.000018
      1.999999
      2.000003
      -43.000000
      -43.000000

      26
      1.999992
      2.000003
      0.000011
      1.999996
      1.999999
      -43.000000
      -43.000000

```python

```

```python

```