```python
import numpy as np
import matplotlib.pyplot as plt
from IPython.display import display, Math, Latex, HTML, clear_output, Markdown, Javascript
import ipywidgets as widgets
from ipywidgets import interact, interact_manual, FloatSlider, IntSlider, interactive, Layout
from traitlets import traitlets
import warnings

import plotly as py
import plotly.graph_objs as go
from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
init_notebook_mode(connected=True)
```

```python
HTML('''
<form action="javascript:code_toggle()"><input type="submit" id="toggleButton" value="Show Code"></form>''')

```

 Root Finding

# Introduction
---
In this notebook you will gain an understanding of basic root-finding algorithms and their implementation. This topic is introduced with the goal of easing you into numerical solutions to mathematical problems. We will outline the advantages and disadvantages of solving a problem numerically rather than analytically. At the end of the notebook there will be example problems, in which you have the opportunity to implement these methods, yielding a broader understanding of their applicability and power.

**Necessary background**:
- Be able to factor quadratic polynomials
- Understand basic python syntax
- Read, graph, and analyze functions
- Rudimentary algebra

In this Notebook you will see examples of code to perform some basic root-finding algorithms. You will not be required to know how the code works or need to write any yourself. Nevertheless, you will be encouraged to manipulate some of the lines in order to input a function of your choice, but that is all. The pieces of code and algorithms can be simply thought of as tools to execute the task you want them to perform, and only running them will be a necessary to accomplish this task. Although the coding isn't required, I strongly encourage you to try to understand how and why it works.

For a warm-up, we have created an exercise for you to determine the intervals on which a polynomial is $>$ 0. A first approach to this problem would be to find the roots of the polynomial and then analyze the behaviour in between these roots. We recommend converting the function into a form that you can graph in order to determine the behaviour on either side of the roots. Think about the possible forms of the graph, i.e. concavity, convexity, and what this tells you about the intervals where the function is positive. This type of analytical thinking will help further along in the notebook.

Given a polynomial of order $\leq$ 3, find where the function is $>$ 0.
Below is simply **a** method of approaching this problem, not necessarily the best or most effective. There are many ways one can go about this, each offering a different insight or understanding. Find what works best for you and try to understand why  you used this method (is it visual? is it solely algebraic? did you manipulate the function?).

**Example**:

Find the interval on which $f(x) = x^2 - 3x +2 > 0$.

Solve for roots:

$f(x) = (x-1)(x-2)$

Using an inequality approach, if I picked a point, $c$ in the interval $(1,2)$, then (c-1)> 0 and (c-2)< 0. Therefore, our function evaluated at this point:

$f(c)=(c-1)(c-2) < 0$.

We can deduce, by knowing the shape of a parabola, that the interval on which $f(x)>0$ is $x = (-\infty,1)\cup(2,\infty)$

By picking a point in between the roots we quickly found whether the function was concave up or down. Knowing what the graph of a parabola looks like, this quickly told us where the function was positive.

Given a polynomial of order 2 or 3, find where the function $>$ 0. Try polynomials of 2nd order first and then 3rd for more of a challenge. Below, we will provide some algorithms that will make finding the roots of a third (or higher) order polynomial much less tedious.

- Please input your answer in interval notation, using "U" for the union of intervals. Press **Enter** after typing your response.
- For $\infty$ type infinity
- If the function is nowhere $ > 0$, then type "Nowhere"

I recommend trying the exercise for a polynomial of order two, reviewing the content of the notebook and then trying for polynomials of order 3. Input answer in interval notation (i.e. (-4,-1)U(5,infinity) ):

```python
def find_interval(poly_order):
    poly_order = int(poly_order)
    if poly_order > 3:
        display(Latex('Order of polynomial must be less than or equal to 3'))
        display(Latex('Provide order of polynomial:'))

    if poly_order == 3:
        C = np.random.randint(-5,5,poly_order)
        C1 = -1*np.sum(C)
        C2 = C[0]*C[1] + C[2]*(C[0]+C[1])
        C3 = -1*C[0]*C[1]*C[2]
        C11=C1
        C22=C2
        C33=C3
        display(Latex("Find the interval where $P(x) > 0$ :"))
        if C1>0:
            str1 = '+' + str(C11) + 'x^2'
        elif C1== 0:
            str1 = ''
        else:
            str1=  str(C11) + 'x^2'
        if C2>0:
            str2 = '+' + str(C22) + 'x'
        elif C2== 0:
            str2=''
        else:
            str2= str(C22) + 'x'
        if C3>0:
            str3 = '+' + str(C33)
        elif C3== 0:
            str3=''
        else:
            str3= str(C33)

        a = "P(x)= x^3" + str1 + str2 + str3

        display(Math(a))

        def poly(x):
            return x**3 + C1*x**2 + C2*x + C3

    if poly_order == 2:
        C = np.random.randint(-5,5,poly_order)
        C1 = -1*np.sum(C)
        C2 = C[0]*C[1]
        C11=C1
        C22=C2
        if C1>0:
            str1 = '+' + str(C11) + 'x'
        elif C1== 0:
            str1 = ''
        else:
            str1=  str(C11) + 'x'
        if C2>0:
            str2 = '+' + str(C22)
        elif C2== 0:
            str2=''
        else:
            str2= str(C22)
        display(Latex('Find the interval where $P(x) > 0 $ :'))
        a = 'P(x) = x^2 ' + str1 + str2
        display(Math(a))
        def poly(x):
            return x**2 + C1*x + C2

    Max = max(C)
    Min = min(C)
    M = [Min, Max]
    V = np.sort(C)
    eps = 0.1

    if poly_order ==3:
        v = V[1]
        if Max == Min and poly(Max +eps) > 0:
                          interval = '('+str(Max)+',infinity)' #One single root, increasing
        if Max == Min and poly(Max +eps) < 0:
                          interval = '(-infinity,' + str( Max)+')' #One single root, decreasing

        if poly(Max + eps) >0:
#         interval = '(' + str(Max) + ', infinity)'
            if v !=  Max and v!= Min:
                          interval = '('+str(Min) + ',' + str(v) + ')U(' + str(Max) + ',infinity)'
            if v == Max:
                          interval = '(' + str(Min) + ', infinity)'
            if v== Min:
                          interval = '(' + str(Max) + ', infinity)'

        if poly(Max + eps) <0:
#         interval = '(-infinty,' + str(Min) + ')'
            if v != Max and v != Min:
                          interval = '(-inifinity,' + str(Min) + 'U('+str(v) + ','+str(Max) + ')'
            if v == Max:
                          interval = '(-infinity,' + str( Max) + ')'
            if v == Min:
                          interval = '(-infinity,' + str(Min) + ')'

    if poly_order == 2:
            if Max == Min and poly(Max+eps)>0:
                interval = '(-infinity,'+str(Min)+')U('+str(Min)+',infinity)' #one root, convex
            elif poly(Max+eps)<0:
                interval = '('+str(Min)+','+str(Max)+')' #Two distinct roots, Concave
            elif poly(Max + eps)>0:
                interval = '(-infinity,'+str(Min)+')U(' + str(Max)+',infinity)' #Two distinct roots, convex
            else:
                interval = 'Nowhere' #one root, concave

    x = np.linspace(-100,100,10000)
    p = poly(x)

    return interval, p

def check_answer(answer,interval,y):

    answer = answer.replace(" ","")

    if answer == "":
        display(Latex("Please be sure to press enter after submitting your answer."))
    else:
        if answer != interval:
            display(Latex("That's not quite right, try again."))
        if answer == interval:

            display(Latex("Correct! Here's a visualization of the solution:"))
            x1=np.linspace(-100,100,10000)
            trace = go.Scatter(
                x = x1,
                y = y
            )
            layout = go.Layout(
                xaxis = dict(
                    title='x',
                    titlefont=dict(
                    family='Arial, serif',
                    size=18,
                    color='black'),
                    range = [-10,10],
                    ticks='inside',
                    tick0=0,
                    dtick=1
                ),
                yaxis = dict(
                    title='P(x)',
                    titlefont=dict(
                    family='Arial, serif',
                    size=18,
                    color='black'),
                    range = [-10,10],
                    ticks='inside',
                    tick0=0,
                    dtick=1
                ),
            )

            data = [trace]

            fig = go.Figure(data=data, layout=layout)
            iplot(fig)

    return

text2 = widgets.Text(disabled = False, placeholder = 'Enter Interval')
out2 = widgets.Text(disabled  = True, value = None)
text3 = widgets.Text(disabled = False, placeholder = 'Enter Interval')
out3 = widgets.Text(disabled  = True, value = None)

def bind_to_out2(sender):
    out2.value = text2.value
    return out2.value

text2.on_submit(bind_to_out2)

def bind_to_out3(sender):
    out3.value = text3.value
    return out3.value

text3.on_submit(bind_to_out3)

button2 = widgets.Button(description="Attempt Exercise (order 2)",
                         layout = Layout(width='30%', height='60px'),
                         button_style = 'info'
                        )
button3 = widgets.Button(description="Attempt Exercise (order 3)",
                         layout = Layout(width='30%', height='60px'),
                         button_style = 'info'
                        )
check_button2 = widgets.Button(description="Check your answer",
                              layout = Layout(width='30%', height='60px'),
                              button_style = 'info'
                             )
check_button3 = widgets.Button(description="Check your answer",
                              layout = Layout(width='30%', height='60px'),
                              button_style = 'info'
                             )

H1 = widgets.HBox([button2, button3])
display(H1)

def on_button_clicked2(b):
    text2.value = ""
    clear_output()
    display(H1)
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+0,IPython.notebook.get_selected_index()+1)'))
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+1,IPython.notebook.get_selected_index()+2)'))

button2.on_click(on_button_clicked2)

def on_button_clicked3(b):
    text3.value = ""
    clear_output()
    display(H1)
    #Rerun this cell to avoid the multiple outputs
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+0,IPython.notebook.get_selected_index()+1)'))
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+2, IPython.notebook.get_selected_index()+3)'))

button3.on_click(on_button_clicked3)

```

```python
clear_output()
interval2, y2 = find_interval(2)

display(text2)
display(check_button2)

def check2(b):
    check_answer(out2.value,interval2,y2)

check_button2.on_click(check2)

```

```python
interval3, y3 = find_interval(3)
display(text3)
display(check_button3)

def check3(b):
    check_answer(out3.value,interval3,y3)

check_button3.on_click(check3)
```

## Analytic vs. Numerical Solutions
-----------------------------------------------------

   Below we will outline the differences, pros and cons and methods of analytic vs. numerical solutions within the context of root finding. By solving for the roots of the polynomial and analyzing the graph of the function, you were able to explicitly determine an answer to the question posed. This is the benefit of these analytic expressions; they give you a nice and clear explicit answer. It is often the case that we can derive analytic solutions to simpler, well-posed problems. Now, what if the problem is not so well-posed? What if our analytic approach becomes way too complex or tedious? How would you approach this problem without finding zeroes? What if the polynomial is of $n^{\rm th}$ order? For these cases we turn to a numerical approach, alleviating the work load and attaining the same end goal, but often with less accuracy.

   Next we will walk you through different ways to approach this problem, and more complex problems of the same flavour, numerically. You will gain some insight into the implementation and benefit of numerical solutions while developing some basic skills in Python and numerical analysis.

## Inspection
-----------------------

In order to answer the question, "On what interval(s) is $ f(x) > 0?$", you probably found the roots of the polynomial and looked at the behaviour of the graph in between these roots. This can be done for nice polynomials with integer coefficients and a low-order, but becomes increasingly difficult the more terms there are and the less simple the polynomials starts looking. Nevertheless, your initial approach to this problem can still be taken, i.e. find the roots and analyze the behaviour in between these roots in order to determine where $f(x)>$ 0. Below we will discuss some algorithms used to determine roots of a polynomial.

Take, for instance, the polynomial:
$f(x) = x^3 -\frac{7}{9}x^2 - \frac{1}{4}x+\frac{7}{36}$

A great advantage of utilizing a numerical approach is that it allows for quick ways to approximate a solution. This is useful if the amount of error is negligible and all one is looking for is an estimation. A quick way to get an estimation for our solution in question is simply to plot $f(x)$. This is similar to what you would do on your graphing calculator, now just on the computer using code! If you would like to get started plotting with python, the best place to start is with matplotlib. It's quick to learn, if you know basic python syntax, and there are tons of internet resources to explore its capabilities and debug problems. For extensive documentation and functionality of the matplotlib library see https://matplotlib.org/. Click on plot function to inspect $f(x)$ above.

```python
button = widgets.Button(description="Plot Function",
                        layout = Layout(width='30%', height='60px'),
                        button_style = 'info'
                       )
display(button)
def on_button_clicked(b):

    x = np.linspace(-50,50,1000)
    y = x**3 - (7/9)*x**2 - (1/4)*x + 7/36
    trace = go.Scatter(
        x = x,
        y = y
    )
    layout = go.Layout(
        xaxis = dict(
            title='x',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            showticklabels=True,
            tickangle=0,
            range = [-1,1],
            ticks='outside',
            tick0=0,
            dtick=1
        ),
        yaxis = dict(
            title='P(x)',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            range = [-1,1],
            ticks='outside',
            tick0=0,
            dtick=1
        ),
    )

    data = [trace]

    fig = go.Figure(data=data, layout=layout)

    iplot(fig)

button.on_click(on_button_clicked)
```

---
Purely by inspection, we were able to observe where the roots of this function lie: $x \approx -\frac{1}{2},\frac{1}{2},\frac{3}{4}$. This was a nice and clean numerical solution to a problem that would've been much more difficult to solve analytically. Although this may seem convenient and albeit simple, this estimation approach is primitive. It is rare that a problem would ever require this simple of a solution, but nonetheless we found the roots of the polynomial. By simply graphing the solution, we were also able to see easily where the function was greater than zero.

Here we will provide an exercise to determine roots of a function based on arbitrary parameters and visualize how a graph of the function changes for different values of these parameters.

Learning outcomes:
- You will understand how arbitrary parameters change the graph of a quadratic polynomial function
- Understand how to determine the roots given arbitrary values
- See how the roots change under changing the values of the parameter

First we provide the steps to derive the famous "Quadratic Formula", try following them on your own, click **Show Answer** to see the answer.

Starting with an arbitrary polynomial, $P(x) = ax^2 + bx + c$, where $a \ne 0$:

1. Set $P(x) = 0$.
2. Complete the square.
3. Rearrange the equation, isolating "$x$".
---

```python
button = widgets.Button(description="Show Answer",
                        layout = Layout(width='30%', height='60px'),
                         button_style = 'info'
                       )
display(button)

def on_button_clicked(b):
    display(Markdown('$ P(x) = ax^2 + bx + c = 0$ </br> </br> \
    Complete the square, first dividing by a:</br> </br> \
    $\\frac{P(x)}{a} = (x+\\frac{b}{2a})^2 + \\frac{c}{a} - \\frac{b^2}{4a}= 0$ </br> </br>\
    Rearrange for $x$: </br> </br> \
    $(x+\\frac{b}{2a})^2 = \\frac{b^2}{4a^2} -\\frac{c}{a} = \\frac{b^2 - 4ac}{4a^2}$ \
    $\Rightarrow x+\\frac{b}{2a} = \pm \\frac{\sqrt{b^2-4ac}}{\sqrt{4a^2}}$ </br> \
    $\Rightarrow x = -\\frac{b}{2a} \pm \\frac{\sqrt{b^2-4ac}}{2a}$'))
    button_clicked = 1
button.on_click(on_button_clicked)

```

---
A potential use of inspection would be our initial exercise: "Find the intervals where $f(x) > 0$ ". The next cell has been left for you to input the values $(a,b,c,d)$ for a third order polynomial in the form $f(x) = ax^3 + bx^2 + cx + d$ the function that our first exercise output, see if you can use inspection to obtain the correct interval. Also, by setting $a$ to zero, see observe how the roots change for a quadratic function. Think about how changing these parameters changes the roots obtained from the "Quadratic formula" (e.g. what does increasing the value of $c$ do?). Press the "Plot Function" to start this exercise.

```python
#function to graph third order polynomial with given inputs:
def plot3(a,b,c,d):
    x = np.linspace(-50,50,1000)
    y = a*x**3 + b*x**2 + c*x + d
    trace = go.Scatter(
        x = x,
        y = y
    )
    layout = go.Layout(
        xaxis = dict(
            title='x',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            showticklabels=True,
            tickangle=0,
            range = [-15,15],
            ticks='outside',
            tick0=0,
            dtick=1
        ),
        yaxis = dict(
            title='P(x)',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            range = [-25,25],
            ticks='outside',
            tick0=0,
            dtick=5
        ),
    )

    data = [trace]

    fig = go.Figure(data=data, layout=layout)

    iplot(fig)

    return

a = widgets.IntSlider(
    value=1,
    min=-10,
    max=10,
    step=1.0,
    description=r'$a$:',
    disabled=False
)
b = widgets.IntSlider(
    value=1,
    min=-10,
    max=10,
    description=r'$b$:',
    disabled=False
)
c = widgets.IntSlider(
    value=1,
    min=-10,
    max=10,
    description=r'$c$:',
    disabled=False
)
d = widgets.IntSlider(
    value=1,
    min=-10,
    max=10,
    description=r'$d$:',
    disabled=False
)
H = widgets.HBox([a,b,c,d])

abc = widgets.interactive_output(plot3, {'a': a, 'b': b, 'c': c, 'd': d})

button = widgets.Button(description="Plot Function",
                        layout = Layout(width='30%', height='60px'),
                         button_style = 'info'
                       )
display(button)
def on_button_clicked(b):

    button_clicked = display(abc,H)
    d.value = 0 #this is so it plots, widget has to change value in order to observe change

button.on_click(on_button_clicked)
```

We will now proceed into some algorithmic numerical methods for root finding. These methods are useful as they can be easily integrated into a computer program. These are some of the more basic methods but they are often combined with each other, or with other methods, for more robust applications that require greater accuracy and less processing time. We first introduce how to use these methods analytically, then introduce a sketch of the code in python syntax. The analytic approach provides a means of solving the problem, while the numerical approach speeds the calculations up. Combining the two approaches provides insight into potential problems that arise in the computations and how to remedy them.

# Bisection Method
---

The bisection method is a simple algorithm to quickly find an approximation for a root. The basic idea of the method is to initially take an interval, $[a,b]$ over which a function $f$ is continuous, such that $f(a)$ and $f(b)$ have different signs (so you know that there is a root in between). We then split the interval into two and check the new endpoints of each subinterval. Let $c = (a+b)/2$ be the midpoint of the interval. If the function values at the endpoints of one of these two subintervals have identical signs (that is, if $f(a)$ and $f(c)$ have the same signs or $f(b)$ and $f(c)$ have the same signs) then this subinterval is discarded, knowing that a root must lie within the other subinterval. Note that a root could also exist within the other interval but it is not guaranteed. We then divide this subinterval into two and repeat the same analysis on it. Eventually, up to an accuracy that we are happy with, we will have a small interval within which a root is found.

## Analytical Approach

Try this method, analytically, for $f(x)= x^2-4$ and the initial interval $[0,5]$. Try three iterations of the algorithm, and see the accuracy you obtain. Would you be happy with this accuracy? Do you feel this was quick enough? Click "Show Answer" to see the calculations, after you have done so yourself of course...

```python
button = widgets.Button(description="Show Answer",
                        layout = Layout(width='30%', height='60px'),
                         button_style = 'info'
                       )
display(button)

def on_button_clicked(b):
    display(Markdown('Firstly, $f(0) = -4$ and $\\rm f(5) = 21$, which have opposite signs, so this is a valid interval to start with. </br> </br> \
    Split the interval in two: $c_1 = \\frac{0+5}{2} = 2.5$. </br> </br> \
    Check the sign of the function for on each subinterval endpoint (i.e. check the sign of $f(c)$):  $f(2.5) = 2.25$, therefore our new interval is $[0,2.5]$.</br> </br> \
    Split interval again and check signs of function at the middle: $c_2 = \\frac{2.5+0}{2} = 1.75$ , $f(1.75) = -0.9375$ </br> </br> \
    $\Rightarrow$ new interval is $[1.75,2.5]$, $c_3 = \\frac{1.75+2.5}{2}$, $f(c_3) = 0.515625$, new interval = $[1.75,2.125]$ </br> </br> \
    root is approximately $c_4 = \\frac{1.75 + 2.125}{2} = 1.9375$ </br> </br> \
    Percent error: $\\frac{2-1.9375}{2} x 100\% = 3.125 \%$'))
    button_clicked = 1
button.on_click(on_button_clicked)

```

***
We can clearly see that the method is somewhat slow and *really* good accuracy is hard to achieve. Nevertheless, it narrowed down our intervals and gave us a fairly good approximation to the root. What about the other root at $x = -2$, how could we have obtained this root? Well, with a different set of initial conditions (our initial interval). The initial conditions play a very important role for this method as they can yield different results for a root. Try this same procedure, now instead with $[-5,0]$ as your starting point. Your answer after each iteration should be the same, except with a different sign.

This serves to illustrate a flaw in a lot of root finding algorithms; the sensitivity to the initial guess. Many of these algorithms require a good educated guess to work properly. Moreover, the algorithm can only determine one root for each initial guess, which is another disadvantage.

***

## Numerical Approach

Specify the initial interval and the tolerance (accuracy) that you would like to achieve:
```python
x0 = -1
x1 = 1
tol = 0.01
```

Define the function you would like to find the root for:
```python
def f(x):
    return x**3 - (7/9)*x**2 -(1/4)*x + 7/36
```
This function can accept a single number or a whole list of numbers: it returns an output that applies a function to the given number or every number in the given list. Now we can begin the algorithm. First we set up the initial conditions:
```python
c =(x1 + x0)/2  #where c is the mid-point, as defined in the analytical solution
if (f(x0) > 0 and f(x1) > 0) or (f(x0) < 0 and f(x1) < 0):
    print('Pick new initial conditions') #The Bisection method does not apply for these cases
else:
    while f(c) > tol:  #run the loop until your tolerance is satisfied

        if f(c) == 0:
            break   # you have found a root; break out of the while loop
        if sign(f(c)) == sign(f(x0)):
            x0 = c #Interval [x0,c] does not necessarily have a root; x0 is set to c
        if sign(f(c)) == sign(f(x1)):
            x1 = c #Interval [c,x1] does not necessarily have root; x1 is set to c

        c = (x1+x0)/2  # update the value of c, based on the current values of x0 and x1
    print(c)
```

### What does the `while` statement do?

The statement
```python
while f(c) > tol:
```
deserves some unpacking. This is what is known as a while loop, and is an essential tool used in programming. The indented code written after the colon will be executed until the conditional statement $f(c) > \rm tol$ doesn't hold anymore. Here is a visual interpretation of this process:

**Common errors that occur**:

- Infinite loop: this is probably the most common error that occurs with a while loop. This happens if the statement is always true, i.e. the code block never makes the conditional statement false.
- Loop is not entered: This occurs if the conditional statement is not initially satisfied. This depends on our initial guess for $[x_0,x_1]$.

The next cell is an illustration of how one could write a piece of code to execute this algorithm. The default initial conditions are $x_0 = -1$, $x_1 = 1$. The same code is included in the cell afterwards, if you would like to change the inital conditions or the function itself, press the "show code" to manipulate the algorithm.

```python
#Bisection Method algorithm

#Specify the initial interval
x0 = -1
x1 = 1

#Specify tolerance (accuracy) you want:
tol = 0.01

#Function (of your choice) we will use our function from before:
def f(x):
    return x**3 - (7/9)*x**2 -(1/4)*x + 7/36

#Defining a sign function
def sign(x):
    if x < 0:
        sign = 0
    if x >= 0:
        sign = 1
    return sign

#Bisection method algorithm
c = (x1+x0)/2  #Midpoint
if (f(x0) > 0 and f(x1) > 0) or (f(x0) < 0 and f(x1) < 0):
    print('Pick new initial conditions') #The Bisection method does not apply for these cases
else:
    while f(c) > tol:  #run the loop until your tolerance is satisfied
        c = (x1+x0)/2
        if f(c) == 0:
            break   # you have found a root; break out of the while loop
        if sign(f(c)) == sign(f(x0)):
            x0 = c #Interval [x0,c] does not necessarily have a root; x0 is set to c
        if sign(f(c))== sign(f(x1)):
            x1 = c #Interval [c,x1] does not necessarily have root; x1 is set to c
        root = (x1+x0)/2
    display(Latex('$\\rm Root =  $' + str(round(root,4))))

```

# Secant Method
____

Here we will walk you through a root-finding algorithm known as the *secant method* which provides quick approximations of roots. For a quick review or another reference of this method and other root finding algorithms, please see https://en.wikipedia.org/wiki/Root-finding_algorithm.

In this section of the tutorial we will demonstrate the basics of the method and walk you through some of the elementary steps you would need to write a root-finding algorithm implementing the secant method yourself.

## Analytical approach
---
The secant method employs successive operations, known as iterations, of finding the root of a *secant line* of a function $f(x)$.

A secant line can thought of as a line which intersects a curve at two points. Given the domain $[a,b]$ and a continuous function $ f:[a,b] \mapsto \mathbb{R}$, take two points $ x_0, x_1 \in [a,b]$. The secant line between these two points, in slope-intercept form is given as:

\begin{equation}
s(x) = \frac{f(x_1)-f(x_0)}{x_1-x_0}(x-x_1) + f(x_1)
\end{equation}

The next step in the algorithm is to find the $x$ intercept of this line:

\begin{equation}
\begin{split}
s(x) &= 0 = \frac{f(x_1)-f(x_0)}{x_1-x_0}(x-x_1) + f(x_1)
\\& \Rightarrow x = x_1 - f(x_1)\frac{x_1-x_0}{f(x_1)-f(x_0)}
\end{split}
\end{equation}

This point now becomes our next endpoint, i.e.:

\begin{equation}
x_2 = x_1 - f(x_1)\frac{x_1-x_0}{f(x_1)-f(x_0)}
\end{equation}

We now *iterate* this algorithm, applying the same process successively, until our algorithm yields a value $f(x_n) \approx 0$ after *n* iterations.

\begin{equation}
\begin{split}
x_3 &= x_2 - f(x_2)\frac{x_2-x_1}{f(x_2)-f(x_1)}
\\ x_4 &= x_3 - f(x_3)\frac{x_3-x_2}{f(x_3)-f(x_2)}
\\& .
\\& .
\\& .
\\ x_n &= x_{n-1} - f(x_{n-1}) \frac{x_{n-1}-x_{n-2}}{f(x_{n-1})-f(x_{n-2})}
\end{split}
\end{equation}

## Numerical Approach

The algorithm to implement the secant method is given as follows.
```python
#same function as for the bisection method
def f(x):
    return x**3 - (7/9)*x**2 -(1/4)*x + 7/36  #Function to employ algorithm on

#initial conditions
n = 20 #number of iterations you want to use

for i in range(1,n):
    x2 = x1-f(x1)*((x1-x0)/(f(x1)-f(x0))) #equivalent to what we had defined analytically
    x0 = x1  #update the values
    x1 = x2  #update

print(x1)
    break
```

### What does the `for` statement do?
The statement
```python
for i in range(1,n):
```
also deserves some explanation. This is another essential programming tool known as the *for loop*. Within the `for` statement, the indented code block is executed, then the next iteration of the loop is performed, in our case $i \mapsto i+1$, and this continues until all iterations are done ($i = n$). It provides a means of counting or iterating through a process. In our case these are the iterations of the secant method.

**Note**: The `print(x1)` statement is outside of the indented block. This means that x1 is printed after the loop has finished. If the statement were included within the block, then x1 would be every time (how many times would x1 be printed?).

Below is a visualization of the algorithm at work. Input the initial values of $x_0$ and $x_1$ below and press "Iterate" to see how the algorithm searches for the roots of the same function used in the "Inspection" example. This will give you a sense how this algorithm works and the problems it may run into. Reset the cell by pressing restart and playing around with different initial conditions to gain a better sense of how the algorithm works. Does it seem faster than the bisection method? How many iterations are required until it converges to the solution with the accuracy you wanted? Did it even converge? Notice how different initial guesses can make the algorithm converge to different roots.

Try working through the algorithm analytically, on paper with a function of your choice, beforehand. See how fast you can obtain a root and to what accuracy.

```python
# Secant method visualization
#First we define our arbitrary function
x = np.linspace(-10,10,10000)
f = x**3 - (7/9)*x**2 -(1/4)*x + 7/36 #Underlying plot

def func(x):
    return x**3 - (7/9)*x**2 -(1/4)*x + 7/36  #Function to employ algorithm on

class LoadedButton(widgets.Button):
    #A button that can holds a value as a attribute.

    def __init__(self, value=None, *args, **kwargs):
        super(LoadedButton, self).__init__(*args, **kwargs)
        # Create the value attribute.
        self.add_traits(value=traitlets.Any(value))

def secantmethod(x0,x1,n,tol):

    if str.isdigit(x0) == False or str.isdigit(x1) == False or str.isdigit(tol) == False:
        print("Please input a number")
    else:
        if x0 and x1 != None:
            x0.replace(" ","")
            x1.replace(" ","")
            tol.replace(" ","")
            x0 = np.array(x0).astype(np.float)
            x1 = np.array(x1).astype(np.float)
            tol = np.array(tol).astype(np.float)

            for i in range (1,n):
                x2 = x1-func(x1)*((x1-x0)/(func(x1)-func(x0)))
                x0 = x1
                x1 = x2
                if abs(x1-x0)< tol:

                    display(Markdown("Secant method has provided the root to within the specified tolerance"))
                    display(Markdown('Root = '+ str(round(x1,4))))
                    break

    return x0, x1

x0s_in = widgets.Text(placeholder = "Enter a Value")
x0s_out = widgets.Text(value = None, disabled = True)
x1s_in = widgets.Text(placeholder = "Enter a Value")
x1s_out = widgets.Text(value = None, disabled = True)
tols_in = widgets.Text(placeholder = "Enter a Value")
tols_out = widgets.Text(value = None, disabled = True)

def bind_x0(sender):
    x0s_out.value = x0s_in.value
    return x0s_out.value
def bind_x1(sender):
    x1s_out.value = x1s_in.value
    return x1s_out.value
def bind_tol(sender):
    tols_out.value = tols_in.value
    return tols_out.value

x1s_in.on_submit(bind_x1)
x0s_in.on_submit(bind_x0)
tols_in.on_submit(bind_tol)

lb = LoadedButton(description="Iterate", value=0,
                  layout = Layout(width='30%', height='60px'),
                  button_style = 'info'
                 )

restart_button = widgets.Button(description="Restart",
                        layout = Layout(width='30%', height='60px'),
                        button_style = 'info'
                       )

def on_button_clicked(b):
    x0s_in.value = ""
    x1s_in.value = ""
    tols_in.value = ""
    lb.value = 0
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index(),IPython.notebook.get_selected_index()+1)'))

restart_button.on_click(on_button_clicked)

def add_num(n):
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+1,IPython.notebook.get_selected_index()+2)'))
    lb.value = lb.value + 1
    return lb.value

lb.on_click(add_num)
```

```python
clear_output()
display(restart_button)
display(Latex('Input a value for $x_0$'))
display(x0s_in)
display(Latex("Input a value for $x_1$"))
display(x1s_in)
display(Latex("Specify an accuracy"))
display(tols_in)
display(lb)
```

```python
if lb.value == 0:
    clear_output()

if lb.value != 0:
    if x0s_out.value == '' or x1s_out.value == '' or tols_out.value == '':
        display(Latex("Please be sure to press enter after submitting values"))
    else:
        x_0, x_1 = secantmethod(x0s_out.value,x1s_out.value,lb.value,tols_out.value)
        x_0 = float(x_0)
        x_1 = float(x_1)

        trace1 = go.Scatter(
            x = [x_0,x_1],
            y = [func(x_0),func(x_1)]
        )
        trace2 = go.Scatter(
            x = x,
            y = f
        )
        layout = go.Layout(
            showlegend = False,
            xaxis = dict(
                title='x',
                titlefont=dict(
                family='Arial, sans-serif',
                size=18,
                color='black'),
                showticklabels=True,
                tickangle=0,
                range = [-15,15],
               # autotick=True,
                ticks='outside',
            ),
            yaxis = dict(
                title='P(x)',
                titlefont=dict(
                family='Arial, sans-serif',
                size=18,
                color='black'),
                range = [-10,10],
               # autotick=True,
                ticks='outside',
            ),
        )

        data = [trace1,trace2]
        fig = go.Figure(data = data, layout = layout)
        iplot(fig)

```

# Transcendental equations
---

A nice application of root finding occurs when one is faced with a transcendental equation. Below we will walk you through how to solve these equations numerically using the methods we have previously discussed.

### What is a transcendental equation?

A *transcendental equation* is defined as an equation which contains a transcendental function of the variable being solved for.

Examples:
- $x = e^x$
- $\tan(x) = x$
- $\ln(x) = e^x$

These equations may look intimidating, but by formulating their solution as a root finding problem and applying the methods we learned in this notebook, they become solvable. Developing these tools and being exposed to these types of problems will aid you if you are ever faced with an equation of this type in the future.

Firstly, we illustrate how to use these techniques to solve the problem $\cos(x) = x$ and then provide some practice problems where you get the opportunity to manipulate the code from our example.

***
### Problem Statement

Find all solutions of the equation: $\cos(x) = x$.

Firstly, the problem can be formulated as: find the roots of $f(x) = \cos(x) - x$ (why are these equivalent?). Let's first apply Inspection in order to get a sense of what this function looks like and where it may have roots/solutions. The next cell plots the function on a grid so we can visually "inspect" where the root is approximately. Use the interactive function 'zoom' to see where the function crosses the $x$ axis more precisely.

```python
button = widgets.Button(description="Analyze Graph",
                        layout = Layout(width='30%', height='60px'),
                        button_style = 'info'
                       )
display(button)
def on_button_clicked(b):
    x = np.linspace(-50,50,1000)
    f = np.cos(x) -x
    trace = go.Scatter(
        x = x,
        y = f,
        name = 'P(x) = cos(x) - x',
        line = dict(
        color = ('black')
        )
    )
    layout = go.Layout(
        title = 'Transcendental Equation Inspection',
        xaxis = dict(
            title='x',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            showticklabels=True,
            tickangle= 270,
            range = [-20,20],
           # autotick=False,
            ticks='outside',
            tick0=0,
            dtick=5

        ),
        yaxis = dict(
            title='P(x)',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            range = [-20,20],
         #   autotick=False,
            ticks='outside',
            tick0=0,
            dtick=5
        ),
    )

    data = [trace]

    fig = go.Figure(data=data, layout=layout)

    iplot(fig)
button.on_click(on_button_clicked)
```

***
We can see via inspection that the root is $x \approx 0.75$. As we can see this method is effective for our purposes, especially when you can zoom into the root. In the next cell we will locate the value of the root using the bisection method. Please specify the initial interval and the accuracy you would like. Additionally, try to find the root beforehand, on paper, for the interval you choose. See how many iterations it takes.
***

```python
x0_in = widgets.Text(placeholder = 'Enter a Value')
x0_out = widgets.Text(value = None)
x1_in = widgets.Text(placeholder = 'Enter a Value')
x1_out = widgets.Text(value = None)
tol_in = widgets.Text(placeholder = 'Enter a Value')
tol_out = widgets.Text(value = None)

def bisection_method(x0,x1,tol):
    clear_output()

    if x0 == None or x1 == None or tol == None:
        print("Remember to press enter after submitting an answer")

    if tol == None:
        display(Latex('Please specify a tolerance'))
#     if str.isdigit(x0) == False or str.isdigit(x1) == False or str.isdigit(tol) == False:
#         print("Please input a number")

    try:
        x0 = float(x0)
    except ValueError:
        print("Please input only numbers")

    try:
        x1 = float(x1)
    except ValueError:
        print("Please enter only a number for x1")
    try:
        tol = float(tol)
    except ValueError:
        print("Please enter only a number for tol")

    else:

#         x0.replace(" ","")
#         x1.replace(" ","")
#         tol.replace(" ","")
        tol = float(tol)
        x0 = float(x0)
        x1 = float(x1)

        #Function (of your choice) we will use our function from before:
        #Change this return statement for other functions of your choice.
        def f(x):
            return np.cos(x)-x

        #Bisection method algorithm
        c = (x1+x0)/2  #First define the formula for subdiving our interval
        n = 0
        if f(x0) > 0 and f(x1) > 0 or f(x0) < 0 and f(x1) < 0:
            print('Pick new initial conditions') #The Bisection method does not apply for theses two cases (as previously discussed)
        else:
            while f(c)>tol:
                c = (x1+x0)/2
                if f(c) == 0:
                    break   # you have found the root exactly
                if np.sign(f(c)) == np.sign(f(x0)):
                    x0 = c #Interval [x0,c] does not necessarily have a root; x0 is set to c. Think about why this is the case.
                if np.sign(f(c))== np.sign(f(x1)):
                    x1 = c #Interval [c,x1] does not necessarily have a root; x1 is set to c. Think about why this is the case.
                c = (x1+x0)/2
                n += 1
            display(Latex('Root = ' +str(round(c,4))))
            display(Latex('Number of iterations ='+str(n)))
    return

def bind0(sender):
    x0_out.value = x0_in.value
    return x0_out.value

def bind1(sender):
    x1_out.value = x1_in.value
    return x1_out.value

def bindtol(sender):
    tol_out.value = tol_in.value
    return tol_out.value

x0_in.on_submit(bind0)
x1_in.on_submit(bind1)
tol_in.on_submit(bindtol)

display(Latex("Enter a value for $x_0$"))
display(x0_in)

display(Latex("Enter a value for $x_1$"))
display(x1_in)

display(Latex("Please specify a tolerance"))
display(tol_in)

show_root = widgets.Button(description="Show Root",
                           layout = Layout(width='30%', height='60px'),
                           button_style = 'info'
                          )

display(show_root)

def button_Click(b):
    display(Javascript('IPython.notebook.execute_cell_range(IPython.notebook.get_selected_index()+1,IPython.notebook.get_selected_index()+2)'))
    return

show_root.on_click(button_Click)

```

```python

```

```python

x0 = x0_out.value
x1 = x1_out.value
tol = tol_out.value
bisection_method(x0,x1,tol)

```

### Analysis
---
Our bisection algorithm gave us a reasonably precise value of the root; this value could be passed into another function. Due to this reason, and the fact that the function does not need to be plotted, the bisection method is more robust and more applicable.

What do you think? Would you rather just see a graph of the function and compromise on accuracy?
Below we will outline a clear disadvantage of the bisection method.

Let's now try the same problem for $\tan(x) = x$:

- If we run the algorithm above for the function $f(x) = \tan(x)-x$, with the initial conditions $x_0 = -1, x_1 = 1$, we obtain the     answer: $x = 0$.

- If again we run the algorithm with the initial conditions $x_0 = 1, x_1 = 3$,
  we obtain the answer: $x = 1.5708$

Why is this the case?
Let's plot the function to inspect the cause of this.

```python
button = widgets.Button(description="Analyze Graph",
                        layout = Layout(width='30%', height='60px'),
                         button_style = 'info'
                       )
display(button)

def on_button_clicked(b):
    x = np.linspace(-25,25,10000) # Notice we changes the range of x values, in order to visualize the solution better
    f = np.tan(x)-x
    utol = 250
    ltol = -250
    f[f>utol] = np.inf
    f[f<ltol] = -np.inf

    trace = go.Scatter(
        x = x,
        y = f,
        name = 'P(x) = tan(x) - x',
        line = dict(
        color = ('black')
        )
    )
    layout = go.Layout(
        title = 'Transcendental equation Inspection',
        xaxis = dict(
            title='x',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            showticklabels=True,
            tickangle= 270,
            range = [np.min(x),np.max(x)],
         #   autotick=False,
            ticks='outside',
            tick0=0,
            dtick=5

        ),
        yaxis = dict(
            title='P(x)',
            titlefont=dict(
            family='Arial, serif',
            size=18,
            color='black'),
            range = [-50,50],
      #      autotick=False,
            ticks='outside',
            tick0=0,
            dtick=5
        ),
    )

    data = [trace]

    fig = go.Figure(data=data, layout=layout)

    iplot(fig)
button.on_click(on_button_clicked)

```

### Analysis
---
We see that this function crosses the $x$ axis many times, actually infinitely many times (if we were to extend the plot). Our bisection algorithm can only narrow down on one root at a time. In fact, since the function contains asymptotes, the bisection method may not even converge (remember the function must be continuous on the interval). With appropriate guesses for the initial conditions, we would be able to obtain all roots (in theory) but this would be a tedious task. It took a combination of inspection and the bisection method to recognize that there were multiple solutions and obtain their exact values.

### Practice Problems
---

For extra practice, I strongly encourage you to try out the algorithms in order to solve the first exercise in this notebook for second- and third-order polynomials. The answers are all integer values so you do not need to worry about accuracy too much. Additionally, consider the following transcendental equations. You may run into some problems (technical and theoretical) solving them, that is OK and all part of the learning process.

Find all solutions to:

1. $\ln(x) = x-5$  (think about the domain of definition of ln(x))
2. $\cos(x) = x/4$ (plot the function)
3. $\sin(x) = x-1$ (Try this one with the secant method for fun)
4. $\cos(x) = x^2$
5. $\ln(x) = \cos(x)$

# Conclusion
---

In this Jupyter notebook you learned the basics of root-finding algorithms. Namely, we studied the process of finding and approximating the roots of a function by inspection, the bisection method,and the secant method. Equally as important, you gained some experience and exposure (if you haven't seen some already) to scientific computing.

Additionally, we applied our methods within the context of solving transcendental equations. It is likely that this is the first time you have been exposed to transcendental equations and the methods we have discussed here. Hopefully, working through the notebook helped you understand the different ways in which one can break down a problem like this. Which did you like best? Being able to adapt and apply the mathematical tools that you have acquired in class is an invaluable skill to have. These skills often end up transferring to general problem-solving skills. I encourage you to reflect upon which of the methods you enjoyed and why. It will offer insight into your own personal learning process.

[](https://github.com/callysto/curriculum-notebooks/blob/master/LICENSE.md)


### Imports

```python
#imports
from __future__ import print_function
from ortools.constraint_solver import routing_enums_pb2
from ortools.constraint_solver import pywrapcp

import sys
import os
import pandas as pd
import time
import math
import random
```

### Initialisation Functions

```python
def calculate_minimum_trucks_from_demand(demand_from_nodes, avg_vehicle_capacity):
    total = 0
    with open(demand_from_nodes, 'r') as f:
        for line in f:
            total = total + int(line)
        f.close()

    minimum_num = math.floor(total / avg_vehicle_capacity)
    return minimum_num

def vehicle_demand_initialiser(total_number_of_nodes, demand_from_nodes):
    with open(demand_from_nodes, 'w') as f:
        f.write("0" + '\n')
        for i in range(total_number_of_nodes - 1):
            demand = random.randint(1, 10)
            f.write(str(demand) + '\n')
        f.close()

def vehicle_time_window_initialiser(total_number_of_nodes, nodes_time_window):

    with open(nodes_time_window, 'w') as f:
        f.write("1"+ ' ' + "300" + '\n')
        for i in range(total_number_of_nodes - 1):
            begin = random.randint(1,10)
            end   = random.randint(begin+1, 100)
            f.write(str(begin) + ' ' + str(end) + '\n')
        f.close()
```

```python
def format_demand_data():
    demands = []
    with open('data/Node_Demands.txt', 'r') as f:
        for line in f:
            demands.append(int(line))
        f.close()
    demands = demands[1:] #removing depot node having zero demand

    with open('data/demand_delivery_contraint.csv', 'w') as f:
        f.write("Store" + "," + "Demand" + "," + "Acceptance constraint" + "\n")
        for i in range(len(demands)):
            f.write("S" + str(i) + "," + str(demands[i]) + "," + str(float(demands[i]/2)) + "\n")
```

```python
def format_obtained_routes(number_of_stations, obtained_routes = 'output/route.txt'):

    #Maximum number of stops in header of table
    max_num_of_stops = 10

    with open('data/final_routing_table.csv', 'w') as f:
        #creating the header of table
        f.write("Routes" + "," + "Stop_1" + ",")
        for i in range(2, max_num_of_stops):
            f.write("Stop_" + str(i) + ",")
        f.write("\n")

        for i in range(number_of_stations):
            f.write(str(i) + "," + "S" + str(i) + "\n")

        f.close()

    with open(obtained_routes, 'r') as f:
        current_station = number_of_stations #just a counter

        for line in f:
            # Ignore rest, select only the route from the output
            if(line.split()[0][0] != '0'):
                continue

            split = line.split('->')
            route = [int(i) for i in split[1:-1]]

            #creating a string to append to the final routing csv file
            route_string = ''
            for stop in route:
                route_string += 'S' + str(stop) + ','
            route_string += '\n'

            with open('data/final_routing_table.csv', 'a') as f2:
                f2.write(str(current_station) + "," + route_string)
                current_station += 1
                f2.close()
        f.close()
```

### Data Model

```python
def create_data_model(demand_from_nodes, nodes_time_window):
    """Stores the data for the problem."""

    ### INITIALISE DATA STRUCTURE
    data = {}
    data['demands'] = []
    data['time_matrix']  = []
    data['time_windows'] = []
    data['num_vehicles'] = 2
    data['depot'] = 0
    data['vehicle_capacities'] = [50] * data['num_vehicles'] #Modified later on, shouldn't be a concern here.

    ### READ DEMAND FOR EACH NODE FROM FILE
    with open(demand_from_nodes, 'r') as f:
        for line in f:
            data['demands'].append(int(line.rstrip()))
        f.close()

    number_of_nodes = len(data['demands'])
    ### READ TRAVEL TIME DATA FROM FILE
    time_matrix_data = pd.read_csv('data/site_time.csv')
    time_matrix_data = time_matrix_data.iloc[:number_of_nodes, 1:number_of_nodes+1]
    time_matrix_data[time_matrix_data.columns[0]] = 0
    time_matrix      = time_matrix_data.values.tolist()
    for site_time_data_row in time_matrix:
        data['time_matrix'].append(site_time_data_row)

    ### READ TIME WINDOW DATA FROM FILE
    with open(nodes_time_window, 'r') as f:
        for line in f:
            window_start, window_end = map(int,line.rstrip().split())
            data['time_windows'].append((window_start, window_end))
        f.close()

    return data

```

### Solution Printing Fn

```python
def print_solution(data, manager, routing, assignment):
    """Prints assignment on console."""
    total_distance = 0
    for vehicle_id in range(data['num_vehicles']):
        index = routing.Start(vehicle_id)
        plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
        route_distance = 0
        while not routing.IsEnd(index):
            plan_output += '{}->'.format(manager.IndexToNode(index))
            previous_index = index
            index = assignment.Value(routing.NextVar(index))
            route_distance += routing.GetArcCostForVehicle(
                previous_index, index, vehicle_id)
        plan_output += '{}\n'.format(manager.IndexToNode(index))
        plan_output += 'Time of the route: {}m\n'.format(route_distance)
        print(plan_output)
        with open('output/route.txt', 'a') as f:
            f.write(plan_output)

        total_distance += route_distance
    print('Total Time of all routes: {}m'.format(total_distance))

```

### Methodolgy Wrapper Function

```python
def wrapper(number_of_vehicles, vehicle_capacity, demand_from_nodes, nodes_time_window):

    # Instantiate the data problem.
    data = create_data_model(demand_from_nodes, nodes_time_window)
    data['num_vehicles'] = number_of_vehicles
    data['vehicle_capacities'] = [vehicle_capacity] * number_of_vehicles

    # Create the routing index manager.
    manager = pywrapcp.RoutingIndexManager(len(data['time_matrix']), data['num_vehicles'], data['depot'])

    # Create Routing Model.
    routing = pywrapcp.RoutingModel(manager)

    # Time Callback and constraints
    def time_callback(from_index, to_index):
        from_node = manager.IndexToNode(from_index)
        to_node = manager.IndexToNode(to_index)
        return data['time_matrix'][from_node][to_node]

    time_callback_index = routing.RegisterTransitCallback(time_callback)

    # Using time transit callback as optimisation parameter
    routing.SetArcCostEvaluatorOfAllVehicles(time_callback_index)

    ### Smaller values limit the travel of vehicles, the following values have no affect on time windows
    ### Large values makes sure that we get the best solution for now
    '''Time Window Constraint'''
    time = 'Time'
    routing.AddDimension(
        time_callback_index,
        10000,  # allow waiting time
        10000,  # maximum time per vehicle
        False,  # Don't force start cumul to zero.
        time)
    time_dimension = routing.GetDimensionOrDie(time)

    '''
    Limiting the number of stops travelled by each vehicle
    We set an upper bound of one of the routing dimensions (here, time dimension)
    No vehicle is allowed to travel more than the specified units for that dimension

    We can use distance, time, or any other dimension for setting the upper bound.

    Please note: minimum value of upper bound = max(distance(depot, node))
    Otherwise solution will not exist, as at least one node would become unreachable
    '''
    for vehicle_id in range(data['num_vehicles']):
        time_dimension.SetSpanUpperBoundForVehicle(10, vehicle_id)

    # Add time window constraints for each location except depot.
    for location_idx, time_window in enumerate(data['time_windows']):
        if location_idx == 0:
            continue
        index = manager.NodeToIndex(location_idx)
        time_dimension.CumulVar(index).SetRange(time_window[0], time_window[1])

    # Add time window constraints for each vehicle start node.
    for vehicle_id in range(data['num_vehicles']):
        index = routing.Start(vehicle_id)
        # Require that a vehicle must visit a location during the location's time window.
        time_dimension.CumulVar(index).SetRange(data['time_windows'][0][0],
                                                data['time_windows'][0][1])

    # Instantiate route start and end times to produce feasible times.
    for i in range(data['num_vehicles']):
        routing.AddVariableMinimizedByFinalizer(time_dimension.CumulVar(routing.Start(i)))
        routing.AddVariableMinimizedByFinalizer(time_dimension.CumulVar(routing.End(i)))

    # Demand callback and constaints
    def demand_callback(from_index):
        from_node = manager.IndexToNode(from_index)
        return data['demands'][from_node]

    demand_callback_index = routing.RegisterUnaryTransitCallback(demand_callback)

    routing.AddDimensionWithVehicleCapacity(
        demand_callback_index,
        0,  # null capacity slack
        data['vehicle_capacities'],  # vehicle maximum capacities
        True,  # start cumul to zero
        'Capacity')

    # Setting first solution heuristic.
    search_parameters = pywrapcp.DefaultRoutingSearchParameters()
    search_parameters.first_solution_strategy = (routing_enums_pb2.FirstSolutionStrategy.PATH_CHEAPEST_ARC)
    search_parameters.local_search_metaheuristic = (routing_enums_pb2.LocalSearchMetaheuristic.GUIDED_LOCAL_SEARCH)
    search_parameters.time_limit.FromSeconds(1)

    # Solve the problem.
    solution = routing.SolveWithParameters(search_parameters)

    # Print solution on console.
    if solution:
        if(os.path.exists('output/route.txt')):
            os.remove('output/route.txt')

        print_solution(data, manager, routing, solution)
        print("Vehicles Required: {}".format(number_of_vehicles))
        print("-"*40)
        return 1
    else:
        print("Current Number of Vehicles = {}".format(number_of_vehicles))
        print("No Solution Yet")
        print("-"*40)
        print("Incrementing number of vehicles")
        return 0
```

### Main Function

```python
def main():

    """ Files Required
            - site_time.csv (time to travel between any two nodes)
            - Node_Demands.txt (can be generated)
            - nodes_time_window.txt (can be generated)

            To change number of nodes, just edit the variable below and regenerate demand and time windows.
            Solution isn't guaranteed when new time windows are created

            Preferably change the name of text file so that previous data isn't lost.
    """

    demand_from_nodes  = "data/Node_Demands.txt"
    nodes_time_window  = "data/nodes_time_window.txt" # Filename containing time window for each node
    number_of_nodes    = 20

#     vehicle_demand_initialiser(number_of_nodes, demand_from_nodes)
#     vehicle_time_window_initialiser(number_of_nodes, nodes_time_window)

    vehicle_capacity   = 30
    current_num_of_trucks = calculate_minimum_trucks_from_demand(demand_from_nodes, vehicle_capacity)
    print("minimum_num_of_vehicles = {}".format(current_num_of_trucks) + '\n' + '-'*40)
    solution = wrapper(current_num_of_trucks, vehicle_capacity, demand_from_nodes, nodes_time_window)

    cntr = 0
    while True:
        current_num_of_trucks += 1
        cntr += 1
        solution = wrapper(current_num_of_trucks, vehicle_capacity, demand_from_nodes, nodes_time_window)
        if(current_num_of_trucks > 100):
            break
        if(cntr == 10):
            break

    #TODO: Add binary search instead of linear search.

    '''
    Optimisation Inventory Routing Problem Part:

     Stations = Total nodes - depot

    '''
    format_demand_data()
    format_obtained_routes(number_of_nodes - 1, obtained_routes = 'output/route.txt')
```

```python
if __name__ == '__main__':
    main()
```

    minimum_num_of_vehicles = 4
    ----------------------------------------
    Current Number of Vehicles = 4
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 5
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 6
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 7
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 8
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 9
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 10
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 11
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 12
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 13
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles
    Current Number of Vehicles = 14
    No Solution Yet
    ----------------------------------------
    Incrementing number of vehicles

#### Some Results
Upper Bound = 12m | Min Vehicles 5 | Min time = 46m

Upper Bound = 11m | Min Vehicles 6 | Min time = 49m

```python

```

```python

```

# Inventory Routing Formulation

Decision Variable:

\begin{equation}
\begin{array}{11}
\ Y_{i,k} & \forall i \in R, k \in WD
\end{array}
\end{equation}
\begin{equation}
\begin{array}{11}
\ X_{i,j,k} & \forall i \in R, j \in S, k \in WD
\end{array}
\end{equation}

Objective Function:

\begin{equation}
\begin{array}{ll}
\text{minimise} &  \sum_{i \in R}  \sum_{k \in W} Y_{i,k}  c_{i,k}\\
\end{array}
\end{equation}

Subject to:
\
Respecting Demand Constraints

\begin{equation}
\begin{array}{11}
\sum_{i \in R} \sum_{k \in W}  X_{i,j,k} m_{i,j} = d_j & \forall j \in S
\end{array}
\end{equation}

\
Respecting Delivery Acceptance Constraints

\begin{equation}
\begin{array}{11}
\sum_{i \in R} X_{i,j,k} m_{i,j} \le a_j & \forall j \in S, k \in W
\end{array}
\end{equation}

\
Respecting Truck Constraints

\begin{equation}
\begin{array}{11}
\sum_{j \in S} X_{i,j,k}m_{i,j} \le C_iY_{i,k} & \forall i \in R, k \in W
\end{array}
\end{equation}

\
General Representations

\
Sets
* R: Set of routes
* S: Set of stores
* W: Set of days in a week

\
Parameters
\
$
\begin{equation}
\begin{array}{11}
d_j & \text{: Weekly demand of store j,} & j \in S
\end{array}
\end{equation}
$

$
\begin{equation}
\begin{array}{11}
a_j & \text{: Delivery acceptance limits for store j,} & j \in S
\end{array}
\end{equation}
$

$
\begin{equation}
\begin{array}{11}
m_{i,j} & \text{: Dummy variable to indicate 1 if store j is along route i and 0 otherwise,} & i \in R, j \in S
\end{array}
\end{equation}
$

$
\begin{equation}
\begin{array}{11}
c_{i,k} & \text{: Cost of  truck on each route i on day k }
\end{array}
\end{equation}
$

$
\begin{equation}
\begin{array}{11}
C_{i} & \text{: Full capacity of truck on route i on each day  }
\end{array}
\end{equation}
$

\
Variables
$
\begin{equation}
\begin{array}{11}
X_{i,j,k} & \text{: Units on each route i to be delivered to a store j on day k} \\
Y_{i,k} & \text{: If route i is used on day k}
\end{array}
\end{equation}
$

\
Entities
* i: Route in R
* j: Store in S
* k: Day in W

```python
#Importing Packages

from gurobipy import *
from math import sqrt
import pandas as pd
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import xlrd
```

```python
#Importing Data

routes = pd.read_csv("data/final_routing_table.csv").to_numpy()
print(routes)

demand = pd.read_csv("data/demand_delivery_contraint.csv").to_numpy()
print(demand)

```

    [[0 'S0' nan nan nan nan nan nan nan nan nan]
     [1 'S1' nan nan nan nan nan nan nan nan nan]
     [2 'S2' nan nan nan nan nan nan nan nan nan]
     [3 'S3' nan nan nan nan nan nan nan nan nan]
     [4 'S4' nan nan nan nan nan nan nan nan nan]
     [5 'S5' nan nan nan nan nan nan nan nan nan]
     [6 'S6' nan nan nan nan nan nan nan nan nan]
     [7 'S7' nan nan nan nan nan nan nan nan nan]
     [8 'S8' nan nan nan nan nan nan nan nan nan]
     [9 'S9' nan nan nan nan nan nan nan nan nan]
     [10 'S10' nan nan nan nan nan nan nan nan nan]
     [11 'S11' nan nan nan nan nan nan nan nan nan]
     [12 'S12' nan nan nan nan nan nan nan nan nan]
     [13 'S13' nan nan nan nan nan nan nan nan nan]
     [14 'S14' nan nan nan nan nan nan nan nan nan]
     [15 'S15' nan nan nan nan nan nan nan nan nan]
     [16 'S16' nan nan nan nan nan nan nan nan nan]
     [17 'S17' nan nan nan nan nan nan nan nan nan]
     [18 'S18' nan nan nan nan nan nan nan nan nan]
     [19 'S5' 'S3' 'S7' 'S9' 'S16' nan nan nan nan nan]
     [20 'S2' 'S4' 'S12' 'S14' nan nan nan nan nan nan]
     [21 'S8' 'S6' 'S1' 'S10' 'S19' nan nan nan nan nan]
     [22 'S11' 'S17' 'S13' nan nan nan nan nan nan nan]
     [23 'S18' 'S15' nan nan nan nan nan nan nan nan]]
    [['S0' 1 0.5]
     ['S1' 7 3.5]
     ['S2' 6 3.0]
     ['S3' 7 3.5]
     ['S4' 3 1.5]
     ['S5' 10 5.0]
     ['S6' 6 3.0]
     ['S7' 10 5.0]
     ['S8' 10 5.0]
     ['S9' 6 3.0]
     ['S10' 5 2.5]
     ['S11' 10 5.0]
     ['S12' 10 5.0]
     ['S13' 6 3.0]
     ['S14' 10 5.0]
     ['S15' 5 2.5]
     ['S16' 7 3.5]
     ['S17' 1 0.5]
     ['S18' 1 0.5]]

```python
#Demand
d = demand[:,1]

#Delivery constraints
dc = demand[:,2]

#Route matrix
#Store names
snames = demand[:,0]

count = 1
num_routes = len(routes)
num_stores = len(demand)
mat = np.zeros(num_routes)
for i in snames:
    routestore1 = (routes[:,1] == str(i)) * 1
    routestore2 = (routes[:,2] == str(i)) * 1
    mainroutestore = routestore1 + routestore2
    mat = np.row_stack((mat, mainroutestore))
    #mat = np.concatenate((mat, mainroutestore),axis=1)

mat = np.array(mat, dtype='int16')[1:,:].transpose((1,0))

print(mat)

```

    [[1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1]
     [0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0]
     [0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0]
     [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1]]

```python
#Shipping costs
c = np.full((num_routes, num_stores), 1000)
c

#Supply constraints (truck capacity)
t = np.full((num_routes,7), 30)
t
```

    array([[30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30],
           [30, 30, 30, 30, 30, 30, 30]])

```python
#Transportation Problem
m4 = Model('transportation')

#Variables
#Variables are in proportion
var = m4.addVars(num_routes,num_stores,7)
yvar = m4.addVars(num_routes,7,vtype = GRB.BINARY)

#Objective
m4.setObjective(sum(yvar[i,k] for i in range(num_routes) for k in range(7)), GRB.MINIMIZE)

#Weekly Demand
for j in range(num_stores):
    m4.addConstr(sum(var[i,j,k]*mat[i,j] for i in range(num_routes) for k in range(7)) == d[j])

#Delivery Constraints
for j in range(num_stores):
    for k in range(7):
        m4.addConstr(sum(var[i,j,k]*mat[i,j] for i in range(num_routes)) <= 0.6*dc[j])

#Supply constraint
for i in range(num_routes):
    for k in range(7):
        m4.addConstr(sum(var[i,j,k]*mat[i,j] for j in range(num_stores)) <= t[i,k]*yvar[i,k])

#Solving the optimization problem
m4.optimize()

#Printing the optimal solutions obtained
print("Optimal Solutions:")
for i, val in var.items():
    if val.getAttr("x") != 0:
        print("Number of units from route %g to store %g on day %g:\t %g " %(i[0]+1, i[1]+1, i[2]+1, val.getAttr("x")))

#Printing y
for i, val in yvar.items():
    print("Run route %g on day %g:\t %g " %(i[0]+1, i[1]+1, val.getAttr("x")))

print(yvar)

```

    Academic license - for non-commercial use only - expires 2021-05-10
    Using license file /Users/adityagoel/gurobi.lic
    Gurobi Optimizer version 9.1.1 build v9.1.1rc0 (mac64)
    Thread count: 2 physical cores, 4 logical processors, using up to 4 threads
    Optimize a model with 320 rows, 3360 columns and 777 nonzeros
    Model fingerprint: 0x9cfca9f0
    Variable types: 3192 continuous, 168 integer (168 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 3e+01]
      Objective range  [1e+00, 1e+00]
      Bounds range     [1e+00, 1e+00]
      RHS range        [3e-01, 1e+01]
    Found heuristic solution: objective 75.0000000
    Presolve removed 283 rows and 3311 columns
    Presolve time: 0.04s
    Presolved: 37 rows, 49 columns, 105 nonzeros
    Found heuristic solution: objective 60.0000000
    Variable types: 28 continuous, 21 integer (21 binary)

    Root relaxation: objective 5.533333e+01, 35 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0   55.33333    0    4   60.00000   55.33333  7.78%     -    0s
    H    0     0                      58.0000000   55.33333  4.60%     -    0s
    H    0     0                      57.0000000   55.33333  2.92%     -    0s
    H    0     0                      56.0000000   55.33333  1.19%     -    0s
         0     0   55.33333    0    4   56.00000   55.33333  1.19%     -    0s

    Explored 1 nodes (35 simplex iterations) in 0.14 seconds
    Thread count was 4 (of 4 available processors)

    Solution count 5: 56 57 58 ... 75

    Optimal solution found (tolerance 1.00e-04)
    Best objective 5.600000000000e+01, best bound 5.600000000000e+01, gap 0.0000%
    Optimal Solutions:
    Number of units from route 1 to store 1 on day 2:	 0.3
    Number of units from route 1 to store 1 on day 3:	 0.1
    Number of units from route 1 to store 1 on day 4:	 0.3
    Number of units from route 1 to store 1 on day 5:	 0.3
    Number of units from route 2 to store 2 on day 2:	 2.1
    Number of units from route 2 to store 2 on day 4:	 2.1
    Number of units from route 2 to store 2 on day 5:	 0.7
    Number of units from route 2 to store 2 on day 7:	 2.1
    Number of units from route 8 to store 8 on day 2:	 3
    Number of units from route 8 to store 8 on day 4:	 3
    Number of units from route 8 to store 8 on day 5:	 1
    Number of units from route 8 to store 8 on day 7:	 3
    Number of units from route 10 to store 10 on day 2:	 1.8
    Number of units from route 10 to store 10 on day 3:	 0.6
    Number of units from route 10 to store 10 on day 4:	 1.8
    Number of units from route 10 to store 10 on day 5:	 1.8
    Number of units from route 11 to store 11 on day 2:	 1.5
    Number of units from route 11 to store 11 on day 3:	 0.5
    Number of units from route 11 to store 11 on day 4:	 1.5
    Number of units from route 11 to store 11 on day 5:	 1.5
    Number of units from route 13 to store 13 on day 2:	 3
    Number of units from route 13 to store 13 on day 4:	 3
    Number of units from route 13 to store 13 on day 5:	 1
    Number of units from route 13 to store 13 on day 7:	 3
    Number of units from route 14 to store 14 on day 2:	 1.8
    Number of units from route 14 to store 14 on day 3:	 0.6
    Number of units from route 14 to store 14 on day 4:	 1.8
    Number of units from route 14 to store 14 on day 5:	 1.8
    Number of units from route 15 to store 15 on day 2:	 3
    Number of units from route 15 to store 15 on day 4:	 3
    Number of units from route 15 to store 15 on day 5:	 1
    Number of units from route 15 to store 15 on day 7:	 3
    Number of units from route 17 to store 17 on day 2:	 2.1
    Number of units from route 17 to store 17 on day 4:	 2.1
    Number of units from route 17 to store 17 on day 5:	 0.7
    Number of units from route 17 to store 17 on day 7:	 2.1
    Number of units from route 20 to store 4 on day 1:	 2.1
    Number of units from route 20 to store 4 on day 2:	 0.7
    Number of units from route 20 to store 4 on day 3:	 2.1
    Number of units from route 20 to store 4 on day 6:	 2.1
    Number of units from route 20 to store 6 on day 1:	 3
    Number of units from route 20 to store 6 on day 2:	 1
    Number of units from route 20 to store 6 on day 3:	 3
    Number of units from route 20 to store 6 on day 6:	 3
    Number of units from route 21 to store 3 on day 3:	 1.8
    Number of units from route 21 to store 3 on day 4:	 1.8
    Number of units from route 21 to store 3 on day 5:	 0.6
    Number of units from route 21 to store 3 on day 7:	 1.8
    Number of units from route 21 to store 5 on day 3:	 0.9
    Number of units from route 21 to store 5 on day 4:	 0.9
    Number of units from route 21 to store 5 on day 5:	 0.3
    Number of units from route 21 to store 5 on day 7:	 0.9
    Number of units from route 22 to store 7 on day 1:	 1.8
    Number of units from route 22 to store 7 on day 2:	 0.6
    Number of units from route 22 to store 7 on day 3:	 1.8
    Number of units from route 22 to store 7 on day 6:	 1.8
    Number of units from route 22 to store 9 on day 1:	 3
    Number of units from route 22 to store 9 on day 2:	 3
    Number of units from route 22 to store 9 on day 3:	 3
    Number of units from route 22 to store 9 on day 6:	 1
    Number of units from route 23 to store 12 on day 4:	 3
    Number of units from route 23 to store 12 on day 5:	 3
    Number of units from route 23 to store 12 on day 6:	 1
    Number of units from route 23 to store 12 on day 7:	 3
    Number of units from route 23 to store 18 on day 4:	 0.3
    Number of units from route 23 to store 18 on day 5:	 0.3
    Number of units from route 23 to store 18 on day 6:	 0.1
    Number of units from route 23 to store 18 on day 7:	 0.3
    Number of units from route 24 to store 16 on day 1:	 1.5
    Number of units from route 24 to store 16 on day 4:	 0.5
    Number of units from route 24 to store 16 on day 6:	 1.5
    Number of units from route 24 to store 16 on day 7:	 1.5
    Number of units from route 24 to store 19 on day 1:	 0.3
    Number of units from route 24 to store 19 on day 4:	 0.3
    Number of units from route 24 to store 19 on day 6:	 0.3
    Number of units from route 24 to store 19 on day 7:	 0.1
    Run route 1 on day 1:	 -0
    Run route 1 on day 2:	 1
    Run route 1 on day 3:	 1
    Run route 1 on day 4:	 1
    Run route 1 on day 5:	 1
    Run route 1 on day 6:	 -0
    Run route 1 on day 7:	 -0
    Run route 2 on day 1:	 -0
    Run route 2 on day 2:	 1
    Run route 2 on day 3:	 -0
    Run route 2 on day 4:	 1
    Run route 2 on day 5:	 1
    Run route 2 on day 6:	 -0
    Run route 2 on day 7:	 1
    Run route 3 on day 1:	 -0
    Run route 3 on day 2:	 -0
    Run route 3 on day 3:	 -0
    Run route 3 on day 4:	 -0
    Run route 3 on day 5:	 -0
    Run route 3 on day 6:	 -0
    Run route 3 on day 7:	 -0
    Run route 4 on day 1:	 -0
    Run route 4 on day 2:	 -0
    Run route 4 on day 3:	 -0
    Run route 4 on day 4:	 -0
    Run route 4 on day 5:	 -0
    Run route 4 on day 6:	 -0
    Run route 4 on day 7:	 -0
    Run route 5 on day 1:	 -0
    Run route 5 on day 2:	 -0
    Run route 5 on day 3:	 -0
    Run route 5 on day 4:	 -0
    Run route 5 on day 5:	 -0
    Run route 5 on day 6:	 -0
    Run route 5 on day 7:	 -0
    Run route 6 on day 1:	 -0
    Run route 6 on day 2:	 -0
    Run route 6 on day 3:	 -0
    Run route 6 on day 4:	 -0
    Run route 6 on day 5:	 -0
    Run route 6 on day 6:	 -0
    Run route 6 on day 7:	 -0
    Run route 7 on day 1:	 -0
    Run route 7 on day 2:	 -0
    Run route 7 on day 3:	 -0
    Run route 7 on day 4:	 -0
    Run route 7 on day 5:	 -0
    Run route 7 on day 6:	 -0
    Run route 7 on day 7:	 -0
    Run route 8 on day 1:	 -0
    Run route 8 on day 2:	 1
    Run route 8 on day 3:	 -0
    Run route 8 on day 4:	 1
    Run route 8 on day 5:	 1
    Run route 8 on day 6:	 -0
    Run route 8 on day 7:	 1
    Run route 9 on day 1:	 -0
    Run route 9 on day 2:	 -0
    Run route 9 on day 3:	 -0
    Run route 9 on day 4:	 -0
    Run route 9 on day 5:	 -0
    Run route 9 on day 6:	 -0
    Run route 9 on day 7:	 -0
    Run route 10 on day 1:	 -0
    Run route 10 on day 2:	 1
    Run route 10 on day 3:	 1
    Run route 10 on day 4:	 1
    Run route 10 on day 5:	 1
    Run route 10 on day 6:	 -0
    Run route 10 on day 7:	 -0
    Run route 11 on day 1:	 -0
    Run route 11 on day 2:	 1
    Run route 11 on day 3:	 1
    Run route 11 on day 4:	 1
    Run route 11 on day 5:	 1
    Run route 11 on day 6:	 -0
    Run route 11 on day 7:	 -0
    Run route 12 on day 1:	 -0
    Run route 12 on day 2:	 -0
    Run route 12 on day 3:	 -0
    Run route 12 on day 4:	 -0
    Run route 12 on day 5:	 -0
    Run route 12 on day 6:	 -0
    Run route 12 on day 7:	 -0
    Run route 13 on day 1:	 -0
    Run route 13 on day 2:	 1
    Run route 13 on day 3:	 -0
    Run route 13 on day 4:	 1
    Run route 13 on day 5:	 1
    Run route 13 on day 6:	 -0
    Run route 13 on day 7:	 1
    Run route 14 on day 1:	 -0
    Run route 14 on day 2:	 1
    Run route 14 on day 3:	 1
    Run route 14 on day 4:	 1
    Run route 14 on day 5:	 1
    Run route 14 on day 6:	 -0
    Run route 14 on day 7:	 -0
    Run route 15 on day 1:	 -0
    Run route 15 on day 2:	 1
    Run route 15 on day 3:	 -0
    Run route 15 on day 4:	 1
    Run route 15 on day 5:	 1
    Run route 15 on day 6:	 -0
    Run route 15 on day 7:	 1
    Run route 16 on day 1:	 -0
    Run route 16 on day 2:	 -0
    Run route 16 on day 3:	 -0
    Run route 16 on day 4:	 -0
    Run route 16 on day 5:	 -0
    Run route 16 on day 6:	 -0
    Run route 16 on day 7:	 -0
    Run route 17 on day 1:	 -0
    Run route 17 on day 2:	 1
    Run route 17 on day 3:	 -0
    Run route 17 on day 4:	 1
    Run route 17 on day 5:	 1
    Run route 17 on day 6:	 -0
    Run route 17 on day 7:	 1
    Run route 18 on day 1:	 -0
    Run route 18 on day 2:	 -0
    Run route 18 on day 3:	 -0
    Run route 18 on day 4:	 -0
    Run route 18 on day 5:	 -0
    Run route 18 on day 6:	 -0
    Run route 18 on day 7:	 -0
    Run route 19 on day 1:	 -0
    Run route 19 on day 2:	 -0
    Run route 19 on day 3:	 -0
    Run route 19 on day 4:	 -0
    Run route 19 on day 5:	 -0
    Run route 19 on day 6:	 -0
    Run route 19 on day 7:	 -0
    Run route 20 on day 1:	 1
    Run route 20 on day 2:	 1
    Run route 20 on day 3:	 1
    Run route 20 on day 4:	 -0
    Run route 20 on day 5:	 -0
    Run route 20 on day 6:	 1
    Run route 20 on day 7:	 -0
    Run route 21 on day 1:	 -0
    Run route 21 on day 2:	 -0
    Run route 21 on day 3:	 1
    Run route 21 on day 4:	 1
    Run route 21 on day 5:	 1
    Run route 21 on day 6:	 -0
    Run route 21 on day 7:	 1
    Run route 22 on day 1:	 1
    Run route 22 on day 2:	 1
    Run route 22 on day 3:	 1
    Run route 22 on day 4:	 -0
    Run route 22 on day 5:	 -0
    Run route 22 on day 6:	 1
    Run route 22 on day 7:	 -0
    Run route 23 on day 1:	 -0
    Run route 23 on day 2:	 -0
    Run route 23 on day 3:	 -0
    Run route 23 on day 4:	 1
    Run route 23 on day 5:	 1
    Run route 23 on day 6:	 1
    Run route 23 on day 7:	 1
    Run route 24 on day 1:	 1
    Run route 24 on day 2:	 -0
    Run route 24 on day 3:	 -0
    Run route 24 on day 4:	 1
    Run route 24 on day 5:	 -0
    Run route 24 on day 6:	 1
    Run route 24 on day 7:	 1
    {(0, 0): <gurobi.Var C3192 (value -0.0)>, (0, 1): <gurobi.Var C3193 (value 1.0)>, (0, 2): <gurobi.Var C3194 (value 1.0)>, (0, 3): <gurobi.Var C3195 (value 1.0)>, (0, 4): <gurobi.Var C3196 (value 1.0)>, (0, 5): <gurobi.Var C3197 (value -0.0)>, (0, 6): <gurobi.Var C3198 (value -0.0)>, (1, 0): <gurobi.Var C3199 (value -0.0)>, (1, 1): <gurobi.Var C3200 (value 1.0)>, (1, 2): <gurobi.Var C3201 (value -0.0)>, (1, 3): <gurobi.Var C3202 (value 1.0)>, (1, 4): <gurobi.Var C3203 (value 1.0)>, (1, 5): <gurobi.Var C3204 (value -0.0)>, (1, 6): <gurobi.Var C3205 (value 1.0)>, (2, 0): <gurobi.Var C3206 (value -0.0)>, (2, 1): <gurobi.Var C3207 (value -0.0)>, (2, 2): <gurobi.Var C3208 (value -0.0)>, (2, 3): <gurobi.Var C3209 (value -0.0)>, (2, 4): <gurobi.Var C3210 (value -0.0)>, (2, 5): <gurobi.Var C3211 (value -0.0)>, (2, 6): <gurobi.Var C3212 (value -0.0)>, (3, 0): <gurobi.Var C3213 (value -0.0)>, (3, 1): <gurobi.Var C3214 (value -0.0)>, (3, 2): <gurobi.Var C3215 (value -0.0)>, (3, 3): <gurobi.Var C3216 (value -0.0)>, (3, 4): <gurobi.Var C3217 (value -0.0)>, (3, 5): <gurobi.Var C3218 (value -0.0)>, (3, 6): <gurobi.Var C3219 (value -0.0)>, (4, 0): <gurobi.Var C3220 (value -0.0)>, (4, 1): <gurobi.Var C3221 (value -0.0)>, (4, 2): <gurobi.Var C3222 (value -0.0)>, (4, 3): <gurobi.Var C3223 (value -0.0)>, (4, 4): <gurobi.Var C3224 (value -0.0)>, (4, 5): <gurobi.Var C3225 (value -0.0)>, (4, 6): <gurobi.Var C3226 (value -0.0)>, (5, 0): <gurobi.Var C3227 (value -0.0)>, (5, 1): <gurobi.Var C3228 (value -0.0)>, (5, 2): <gurobi.Var C3229 (value -0.0)>, (5, 3): <gurobi.Var C3230 (value -0.0)>, (5, 4): <gurobi.Var C3231 (value -0.0)>, (5, 5): <gurobi.Var C3232 (value -0.0)>, (5, 6): <gurobi.Var C3233 (value -0.0)>, (6, 0): <gurobi.Var C3234 (value -0.0)>, (6, 1): <gurobi.Var C3235 (value -0.0)>, (6, 2): <gurobi.Var C3236 (value -0.0)>, (6, 3): <gurobi.Var C3237 (value -0.0)>, (6, 4): <gurobi.Var C3238 (value -0.0)>, (6, 5): <gurobi.Var C3239 (value -0.0)>, (6, 6): <gurobi.Var C3240 (value -0.0)>, (7, 0): <gurobi.Var C3241 (value -0.0)>, (7, 1): <gurobi.Var C3242 (value 1.0)>, (7, 2): <gurobi.Var C3243 (value -0.0)>, (7, 3): <gurobi.Var C3244 (value 1.0)>, (7, 4): <gurobi.Var C3245 (value 1.0)>, (7, 5): <gurobi.Var C3246 (value -0.0)>, (7, 6): <gurobi.Var C3247 (value 1.0)>, (8, 0): <gurobi.Var C3248 (value -0.0)>, (8, 1): <gurobi.Var C3249 (value -0.0)>, (8, 2): <gurobi.Var C3250 (value -0.0)>, (8, 3): <gurobi.Var C3251 (value -0.0)>, (8, 4): <gurobi.Var C3252 (value -0.0)>, (8, 5): <gurobi.Var C3253 (value -0.0)>, (8, 6): <gurobi.Var C3254 (value -0.0)>, (9, 0): <gurobi.Var C3255 (value -0.0)>, (9, 1): <gurobi.Var C3256 (value 1.0)>, (9, 2): <gurobi.Var C3257 (value 1.0)>, (9, 3): <gurobi.Var C3258 (value 1.0)>, (9, 4): <gurobi.Var C3259 (value 1.0)>, (9, 5): <gurobi.Var C3260 (value -0.0)>, (9, 6): <gurobi.Var C3261 (value -0.0)>, (10, 0): <gurobi.Var C3262 (value -0.0)>, (10, 1): <gurobi.Var C3263 (value 1.0)>, (10, 2): <gurobi.Var C3264 (value 1.0)>, (10, 3): <gurobi.Var C3265 (value 1.0)>, (10, 4): <gurobi.Var C3266 (value 1.0)>, (10, 5): <gurobi.Var C3267 (value -0.0)>, (10, 6): <gurobi.Var C3268 (value -0.0)>, (11, 0): <gurobi.Var C3269 (value -0.0)>, (11, 1): <gurobi.Var C3270 (value -0.0)>, (11, 2): <gurobi.Var C3271 (value -0.0)>, (11, 3): <gurobi.Var C3272 (value -0.0)>, (11, 4): <gurobi.Var C3273 (value -0.0)>, (11, 5): <gurobi.Var C3274 (value -0.0)>, (11, 6): <gurobi.Var C3275 (value -0.0)>, (12, 0): <gurobi.Var C3276 (value -0.0)>, (12, 1): <gurobi.Var C3277 (value 1.0)>, (12, 2): <gurobi.Var C3278 (value -0.0)>, (12, 3): <gurobi.Var C3279 (value 1.0)>, (12, 4): <gurobi.Var C3280 (value 1.0)>, (12, 5): <gurobi.Var C3281 (value -0.0)>, (12, 6): <gurobi.Var C3282 (value 1.0)>, (13, 0): <gurobi.Var C3283 (value -0.0)>, (13, 1): <gurobi.Var C3284 (value 1.0)>, (13, 2): <gurobi.Var C3285 (value 1.0)>, (13, 3): <gurobi.Var C3286 (value 1.0)>, (13, 4): <gurobi.Var C3287 (value 1.0)>, (13, 5): <gurobi.Var C3288 (value -0.0)>, (13, 6): <gurobi.Var C3289 (value -0.0)>, (14, 0): <gurobi.Var C3290 (value -0.0)>, (14, 1): <gurobi.Var C3291 (value 1.0)>, (14, 2): <gurobi.Var C3292 (value -0.0)>, (14, 3): <gurobi.Var C3293 (value 1.0)>, (14, 4): <gurobi.Var C3294 (value 1.0)>, (14, 5): <gurobi.Var C3295 (value -0.0)>, (14, 6): <gurobi.Var C3296 (value 1.0)>, (15, 0): <gurobi.Var C3297 (value -0.0)>, (15, 1): <gurobi.Var C3298 (value -0.0)>, (15, 2): <gurobi.Var C3299 (value -0.0)>, (15, 3): <gurobi.Var C3300 (value -0.0)>, (15, 4): <gurobi.Var C3301 (value -0.0)>, (15, 5): <gurobi.Var C3302 (value -0.0)>, (15, 6): <gurobi.Var C3303 (value -0.0)>, (16, 0): <gurobi.Var C3304 (value -0.0)>, (16, 1): <gurobi.Var C3305 (value 1.0)>, (16, 2): <gurobi.Var C3306 (value -0.0)>, (16, 3): <gurobi.Var C3307 (value 1.0)>, (16, 4): <gurobi.Var C3308 (value 1.0)>, (16, 5): <gurobi.Var C3309 (value -0.0)>, (16, 6): <gurobi.Var C3310 (value 1.0)>, (17, 0): <gurobi.Var C3311 (value -0.0)>, (17, 1): <gurobi.Var C3312 (value -0.0)>, (17, 2): <gurobi.Var C3313 (value -0.0)>, (17, 3): <gurobi.Var C3314 (value -0.0)>, (17, 4): <gurobi.Var C3315 (value -0.0)>, (17, 5): <gurobi.Var C3316 (value -0.0)>, (17, 6): <gurobi.Var C3317 (value -0.0)>, (18, 0): <gurobi.Var C3318 (value -0.0)>, (18, 1): <gurobi.Var C3319 (value -0.0)>, (18, 2): <gurobi.Var C3320 (value -0.0)>, (18, 3): <gurobi.Var C3321 (value -0.0)>, (18, 4): <gurobi.Var C3322 (value -0.0)>, (18, 5): <gurobi.Var C3323 (value -0.0)>, (18, 6): <gurobi.Var C3324 (value -0.0)>, (19, 0): <gurobi.Var C3325 (value 1.0)>, (19, 1): <gurobi.Var C3326 (value 1.0)>, (19, 2): <gurobi.Var C3327 (value 1.0)>, (19, 3): <gurobi.Var C3328 (value -0.0)>, (19, 4): <gurobi.Var C3329 (value -0.0)>, (19, 5): <gurobi.Var C3330 (value 1.0)>, (19, 6): <gurobi.Var C3331 (value -0.0)>, (20, 0): <gurobi.Var C3332 (value -0.0)>, (20, 1): <gurobi.Var C3333 (value -0.0)>, (20, 2): <gurobi.Var C3334 (value 1.0)>, (20, 3): <gurobi.Var C3335 (value 1.0)>, (20, 4): <gurobi.Var C3336 (value 1.0)>, (20, 5): <gurobi.Var C3337 (value -0.0)>, (20, 6): <gurobi.Var C3338 (value 1.0)>, (21, 0): <gurobi.Var C3339 (value 1.0)>, (21, 1): <gurobi.Var C3340 (value 1.0)>, (21, 2): <gurobi.Var C3341 (value 1.0)>, (21, 3): <gurobi.Var C3342 (value -0.0)>, (21, 4): <gurobi.Var C3343 (value -0.0)>, (21, 5): <gurobi.Var C3344 (value 1.0)>, (21, 6): <gurobi.Var C3345 (value -0.0)>, (22, 0): <gurobi.Var C3346 (value -0.0)>, (22, 1): <gurobi.Var C3347 (value -0.0)>, (22, 2): <gurobi.Var C3348 (value -0.0)>, (22, 3): <gurobi.Var C3349 (value 1.0)>, (22, 4): <gurobi.Var C3350 (value 1.0)>, (22, 5): <gurobi.Var C3351 (value 1.0)>, (22, 6): <gurobi.Var C3352 (value 1.0)>, (23, 0): <gurobi.Var C3353 (value 1.0)>, (23, 1): <gurobi.Var C3354 (value -0.0)>, (23, 2): <gurobi.Var C3355 (value -0.0)>, (23, 3): <gurobi.Var C3356 (value 1.0)>, (23, 4): <gurobi.Var C3357 (value -0.0)>, (23, 5): <gurobi.Var C3358 (value 1.0)>, (23, 6): <gurobi.Var C3359 (value 1.0)>}

```python
for i in range(num_routes):
    for k in range(7):
        print(yvar[i,k].getAttr("x"))
```

    -0.0
    1.0
    1.0
    1.0
    1.0
    -0.0
    -0.0
    -0.0
    1.0
    -0.0
    1.0
    1.0
    -0.0
    1.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    1.0
    -0.0
    1.0
    1.0
    -0.0
    1.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    1.0
    1.0
    1.0
    1.0
    -0.0
    -0.0
    -0.0
    1.0
    1.0
    1.0
    1.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    1.0
    -0.0
    1.0
    1.0
    -0.0
    1.0
    -0.0
    1.0
    1.0
    1.0
    1.0
    -0.0
    -0.0
    -0.0
    1.0
    -0.0
    1.0
    1.0
    -0.0
    1.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    1.0
    -0.0
    1.0
    1.0
    -0.0
    1.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    -0.0
    1.0
    1.0
    1.0
    -0.0
    -0.0
    1.0
    -0.0
    -0.0
    -0.0
    1.0
    1.0
    1.0
    -0.0
    1.0
    1.0
    1.0
    1.0
    -0.0
    -0.0
    1.0
    -0.0
    -0.0
    -0.0
    -0.0
    1.0
    1.0
    1.0
    1.0
    1.0
    -0.0
    -0.0
    1.0
    -0.0
    1.0
    1.0

```python

```

```python
print("Finish")
```

    Finish

```python

```


```python
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Class 14: Prescott's Real Business Cycle Model I

In this notebook, we'll consider a centralized version of the model from pages 11-17 in Edward Prescott's article "Theory Ahead of Business Cycle Measurement in the Fall 1986 of the Federal Reserve Bank of Minneapolis' *Quarterly Review* (link to article: https://www.minneapolisfed.org/research/qr/qr1042.pdf). The model is just like the RBC model that we studying in the previous lecture, except that now we include an endogenous labor supply.

## Prescott's RBC Model with Labor

The equilibrium conditions for Prescott's RBC model with labor are:

\begin{align}
\frac{1}{C_t} & = \beta E_t \left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1}L_{t+1}^{1-\alpha} +1-\delta }{C_{t+1}}\right]\\
\frac{\varphi}{1-L_t} & = \frac{(1-\alpha)A_tK_t^{\alpha}L_t^{-\alpha}}{C_t} \\
Y_t & = A_t K_t^{\alpha}L_t^{1-\alpha}\\
K_{t+1} & = I_t + (1-\delta) K_t\\
Y_t & = C_t + I_t\\
\log A_{t+1} & = \rho \log A_t + \epsilon_{t+1}
\end{align}

where $\epsilon_{t+1} \sim \mathcal{N}(0,\sigma^2)$.

The objective is use `linearsolve` to simulate impulse responses to a TFP shock using the following parameter values for the simulation:

| $\rho$ | $\sigma$ | $\beta$ | $\varphi$ | $\alpha$ | $\delta $ |
|--------|----------|---------|-----------|----------|-----------|
| 0.75   | 0.006    | 0.99    | 1.7317    | 0.35     |  0.025    |

The value for $\beta$ implies a steady state (annualized) real interest rate of about 4 percent:

\begin{align}
4 \cdot \left(\beta^{-1} - 1\right)  & \approx 0.04040
\end{align}

$\rho = 0.75$ and $\sigma = 0.006$ are consistent with the statistical properties of the cyclical component of TFP in the US. $\alpha$ is set so that, consistent with the long-run average of the US, the labor share of income is about 65 percent of GDP. The deprecation rate of capital is calibrated to be about 10 percent annually. Finally, $\varphi$ was chosen last to ensure that in the steady state households allocate about 33 percent of their available time to labor.

## Model Preparation

Before proceding, let's recast the model in the form required for `linearsolve`. Write the model with all variables moved to the left-hand side of the equations and dropping the expecations operator $E_t$ and the exogenous shock $\epsilon_{t+1}$:

\begin{align}
0 & = \beta\left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1}L_{t+1}^{1-\alpha} +1-\delta }{C_{t+1}}\right] - \frac{1}{C_t}\\
0 & = \frac{(1-\alpha)A_tK_t^{\alpha}L_t^{-\alpha}}{C_t} - \frac{\varphi}{1-L_t}\\
0 & = A_t K_t^{\alpha}L_t^{1-\alpha} - Y_t\\
0 & = I_t + (1-\delta) K_t - K_{t+1}\\
0 & = C_t + I_t - Y_t\\
0 & = \rho \log A_t - \log A_{t+1}
\end{align}

Remember, capital and TFP are called *state variables* because they're $t+1$ values are predetermined. Output, consumption, and investment are called a *costate* or *control* variables. Note that the model as 5 equations in 5 endogenous variables.

## Initialization, Approximation, and Solution

The next several cells initialize the model in `linearsolve` and then approximate and solve it.

```python
# Create a variable called 'parameters' that stores the model parameter values in a Pandas Series

# Print the model's parameters

```

```python
# Create variable called 'varNames' that stores the variable names in a list with state variables ordered first

# Create variable called 'shockNames' that stores an exogenous shock name for each state variable.

```

```python
# Define a function that evaluates the equilibrium conditions of the model solved for zero. PROVIDED
def equilibrium_equations(variables_forward,variables_current,parameters):

    # Parameters. PROVIDED
    p = parameters

    # Current variables. PROVIDED
    cur = variables_current

    # Forward variables. PROVIDED
    fwd = variables_forward

    # Euler equation

    # Labor-labor choice

    # Production function

    # Capital evolution

    # Market clearing

    # Exogenous tfp

    # Stack equilibrium conditions into a numpy array

```

Next, initialize the model using `ls.model` which takes the following required arguments:

* `equations`
* `nstates`
* `varNames`
* `shockNames`
* `parameters`

```python
# Initialize the model into a variable named 'rbc_model'

```

```python
# Compute the steady state numerically using .compute_ss() method of rbc_model

# Print the computed steady state

```

```python
# Find the log-linear approximation around the non-stochastic steady state and solve using .approximate_and_solve() method of rbc_model

```

## Impulse Responses

Compute a 26 period impulse responses of the model's variables to a 0.01 unit shock to TFP in period 5.

```python
# Compute impulse responses

# Print the first 10 rows of the computed impulse responses to the TFP shock

```

Construct a $2\times3$ grid of plots of simulated TFP, output, labor, consumption, investment, and capital. Be sure to multiply simulated values by 100 so that vertical axis units are in "percent deviation from steady state."

```python
# Create figure. PROVIDED
fig = plt.figure(figsize=(18,8))

# Create upper-left axis. PROVIDED
ax = fig.add_subplot(2,3,1)

# Create upper-center axis. PROVIDED
ax = fig.add_subplot(2,3,2)

# Create upper-right axis. PROVIDED
ax = fig.add_subplot(2,3,3)

# Create lower-left axis. PROVIDED
ax = fig.add_subplot(2,3,4)

# Create lower-center axis. PROVIDED
ax = fig.add_subplot(2,3,5)

# Create lower-right axis. PROVIDED
ax = fig.add_subplot(2,3,6)

```