## First Assignment

#### 1) Apply the appropriate string methods to the **x** variable (as '.upper') to change it exactly to: "$Dichlorodiphenyltrichloroethane$".

```python
x = "DiClOrod  IFeNi  lTRicLOr oETaNo  DiChlorod iPHeny lTrichL oroEThaNe"
```

```python
x1 = str.lower(x)
x1 = x1.replace(" ", "")
x2 = x1[27:]
x2 = x2.capitalize()

print(x2)# I know to complicated, but I´m learning
```

    Dichlorodiphenyltrichloroethane

#### 2) Assign respectively the values: 'word', 15, 3.14 and 'list' to variables A, B, C and D in a single line of code. Then, print them in that same order on a single line separated by a space, using only one print statement.

```python
A, B, C, D = "word", 15, 3.14, 'list'
print(A,B,C,D)
print(A, "", B, "", C, "", D)
```

    word 15 3.14 list
    word  15  3.14  list

#### 3) Use the **input()** function to receive an input in the form **'68.4 1.71'**, that is, two floating point numbers in a line separated by space. Then, assign these numbers to the variables ** p ** and ** h ** respectively, which represent an individual's weight and height (hint: take a look at the '.split ()' method). With this data, calculate the individual's Body Mass Index (BMI) from the following relationship:

\begin{equation}
BMI = \dfrac{weight}{height^2}
\end{equation}

```python
h = float(input("Height in m: ")) # 1.78    # just testing input & bmi function before adressing split issue
p = float(input("Weight in kg: ")) # 77.4

bmi = p/(h**2)

print(bmi)
```

    Height in m: 1.78
    Weight in kg: 77.4
    24.428733745739176

```python
init = input('initinput: ') # split issue with not separated numbers 77.4 1.78
p,h = [float(init) for init in init.split(" ")]

print(p)
print(h)

bmi = p/(h**2)
print(bmi)
```

    initinput: 77.4 1.78
    77.4
    1.78
    24.428733745739176

#### This value can also be classified according to ranges of values, following to the table below. Use conditional structures to classify and print the classification assigned to the individual.

<\center>

```python
print(bmi)

```

    24.428733745739176

```python
if (bmi < 18.5):
    print("Underweight")
elif (18.5 <= bmi):
    print("Healthy weight")
elif (25 <= bmi):
    print("Overweight")
elif (30 <= bmi):
    print("Obese")
else:
    print("Not working")
```

    Healthy weight

#### 4) Receive an integer as an input and, using a loop, calculate the factorial of this number, that is, the product of all the integers from one to the number provided.

```python
n = int(input("use number:")) # 7
print(type(init))

init = 1

for i in range(1, n+1):
    init = init*i

print("factorial of ", n, ' is ', init)

```

    use number:7
    <class 'int'>
    factorial of  7  is  5040

#### 5) Using a while loop and the input function, read an indefinite number of integers until the number read is -1. Present the sum of all these numbers in the form of a print, excluding the -1 read at the end.

```python
init_list = [] # start with empty list
summe = 0

while True:
    init = int(input('Input Number '))
    summe += init
    if (init <= (-1)):    # couldn´t add a statement in the loop that the -1 is ignored
        break

# return summe  wrong

print(summe)
init_list.append(summe)
print(init_list)
```

```python
summe = 0

while True:
    init = int(input('Input Number '))
    summe += init
    if (init <= (-1)):
        break

print(summe+1) # simple solution in print, even through it´s not excluding the -1, but just cancelling the effect of it
```

    Input Number 2
    Input Number 3
    Input Number 4
    Input Number 5
    Input Number -1
    14

#### 6) Read the **name** of an employee, his **number of hours worked** and the **sallary per hour** in a single line separated by space. Next, calculate the **total salary** for this employee and show it to two decimal places.

```python
# understood that the info should be stored in a string
import math

employ = "Martin 13.20223 15.62231"
name, hour, salary = [employ for employ in employ.split(" ")]
hour, salary = float(hour), float(salary)

income = hour * salary

print(round(income, 2))

```

    206.25

#### 7) Read three floating point values **A**, **B** and **C** respectively. Then calculate and show:

```python
A, B, C = 2.14, 3.24, 5.66  #  testing with assigning
```

```python
# reading from a file solution

file = open("own_data/Data_float_values.txt", 'r')

with open("own_data/Data_float_values.txt", 'r') as file:
    read_data = file.read()
    file.closed
```

    a) the area of the triangle rectangle with A as the base and C as the height.

```python
area = (A*C)/2
print(area)
```

    6.0562000000000005

    b) the area of the circle of radius C. (pi = 3.14159)

```python
import math # To be honest I had to google the function

pi = math.pi

area = (C*2)*pi

print(area)
```

    35.562828838636456

    c) the area of the trapezoid that has A and B for bases and C for height.

```python
area = 0.5*(A+B)*C
print(area)
```

    15.225400000000002

    d) the area of the square that has side B.

```python
area = B**2
print(area)
```

    10.497600000000002

    e) the area of the rectangle that has sides A and B.

```python
area = A*B
print(area)
```

    6.933600000000001

#### 8) Read **3 floating point values** and calculate the **roots of the second degree equation** formed by these coefficients (using [this formula](https://en.wikipedia.org/wiki/Quadratic_equation)). If it is not possible to calculate the roots, display the message **“There are no real roots”**.

```python
import math
#  (a*x^2) + b*x + c = 0  #quadratic function

a, b, c = 2.24, 6.0, 1.6 # testing with assignment

w = b**2 - 4 * a *c  # diskriminante quadratische gleichung

if w>0: # is r positiv there are two possible solutions for x
    x1 = (((-b) + (math.sqrt(w)))/(2*a))
    x2 = (((-b) - (math.sqrt(w)))/(2*a))
    print('There are two solutions', (x1,x2))
elif w == 0:
    x = (-b)/2*a
    print('One Solution')
elif w < 0:
        print('There are no real roots')

```

    There are two solutions (-0.3003436994436531, -2.3782277291277754)

```python
#  (a*x^2) + b*x + c = 0

# free input

a = float(input("a is:"))
b = float(input("b is:"))
c = float(input("c is:"))

w = b**2 - 4 * a *c  # diskriminante quadratische gleichung

if w>0: # is r positiv there are two possible solutions for x
    x1 = (((-b) + (math.sqrt(w)))/(2*a))
    x2 = (((-b) - (math.sqrt(w)))/(2*a))
    print('There are two solutions', (x1,x2))
elif w == 0:
    x = (-b)/2*a
    print('One solution')
elif w < 0:
        print('There are no real roots')

```

    a is:0.1
    b is:0.2
    c is:5
    There are no real roots

#### 9) Read four floating point numerical values corresponding to the coordinates of two geographical coordinates in the cartesian plane. Each point will come in a line with its coordinates separated by space. Then calculate and show the distance between these two points.

(obs: $d=\sqrt{(x_1-x_2)^2 + (y_1-y_2)^2}$)

```python
lat1, lon1, lat2, lon2 = 40.4165000, -3.7025600, 48.2084900, 16.3720800  # coordinaten madridd und wien through assignement
mad = [40.4165000, -3.7025600]
wie = [48.2084900, 16.3720800]
```

write values to csv file (not after priviples of tidy data):  # still have to work on that

fields = ["lat1",'lon1',"lat2", "lon2"]
rows =[["40.4165000"],
       ["-3.7025600"],
       ["48.2084900"],
       ["16.3720800"],
      ]

file_3 = open("own_data/coordinates_ass_1.csv", "w")

with open("own_data/coordinates_ass_1.csv" , "w") as csvfile:
    csvwriter = csvwriter(csvfile)

```python
# calculation
import math

distance =  math.sqrt(((mad[0]-wie[0])**2)+((mad[1]-wie[1])**2))
print(distance)
```

    21.533840328415646

#### 10) Read ** two floating point numbers ** on a line that represent **coordinates of a cartesian point**. With this, use **conditional structures** to determine if you are at the origin, printing the message **'origin'**, in one of the axes, printing **'x axis'** or **'y axis'**; or in one of the four quadrants, printing **'q1'**, **'q2**', **'q3'** or **'q4'**.

```python
# only digits as input allowed, also practicing while loop

while True:
    try:
        x = float(input('Enter the Value for X'))
        y = float(input('Enter the Value for Y'))
    except ValueError:
        print('x and y have to be digits')
        continue
    else:
        print('Both ', x, ' and ', y, ' are digits')
        break

if x == 0 and y == 0:
    print('Point is at the origin')

elif x > 0 and y > 0:
    print('q1')

elif x < 0 and y > 0:
    print('q2')

elif x < 0 and y < 0:
    print('q3')

elif x > 0 and y < 0:
    print ('q4')

else:
    print('not working what so ever')
```

    Enter the Value for Xsfd
    x and y have to be digits
    Enter the Value for Xsd
    x and y have to be digits
    Enter the Value for X7
    Enter the Value for Y5
    Both  7.0  and  5.0  are digits
    q1

#### 11) Read an integer that represents a phone code for international dialing. Then, inform to which city the code belongs to, considering the picture below [(from this table)](https://en.wikipedia.org/wiki/Telephone_numbers_in_Europe).

<\center>

```python
# make a dictionaire with countrycodes and match the code e.g. dictionaire with 5 keys
# not for every point in data table

country_phone = {   # test dictionaire
   'Austria' : 43,
   'Germany' : 49,
   'Spain' : 34,
   'France' : 33,
   'Portugal' : 351,
   'Italy' : 39,
   'Norway': 47
}

phone = int(input('Enter international calling code'))

for (key, value) in country_phone.items():
    if value == phone:
        print(key,value)

# def info_country_code(country_phone, phone):   # functions not working
#    for (key, value) in country_phone.items():
#        if value == phone:
#            return(key,value)

# info_country_code(key, value)

#def info_on_phone_code(country_phone, phone):  # not working, don´t know why...
#    for country in country_phone:
#        if country == phone:
#            return country_phone[country]

#print(info_on_phone_code(country_phone, phone))
```

    Enter international calling code47
    Norway 47

```python
# still to do import with data table from wikipedia
```

#### 12) Write a piece of code that reads 6 values in a row. Next, show the number of positive values entered. On the next line, print the average of the values to one decimal place.

```python
N = int(input('Enter 6 numbers'))                 # Enter 6 to enter 6 values
init = []
count_pos = 0

for num in range(0,6):
    n = int(input())
    if (n>0):
        init.append(n)

        count_pos += 1
        avg = sum(init)/(count_pos)

print(count_pos)
print(round(avg, 1))
```

    Enter multiple numbers4
    3
    2
    3
    4
    5
    6
    6
    3.8

#### 13) Read an integer **N**. Then print the **square of each of the even values**, from 1 to N, including N, if applicable, arranged one per line.

```python
N = int(input('Use a number')) # I´don´t know how to include N when N = odd?

for num in range(1, N+1):     # seems to work now, I don´t know why
    if (num % 2 == 0):
        squared_and_even = (num**2)
        print("The square of ", num, " is", squared_and_even)

    else:
        continue
        # print("wrong, odd or both")

```

    Use a number10
    The square of  2  is 4
    The square of  4  is 16
    The square of  6  is 36
    The square of  8  is 64
    The square of  10  is 100

#### 14) Using **input()**, read an integer and print its classification as **'even / odd'** and **'positive / negative'** . The two classes for the number must be printed on the same line separated by a space. In the case of zero, print only **'null'**.

```python

def even_odd(n):
    if(N==0):
        return "null"
    elif(N % 2)==0:
        return "even"
    elif(N % 2)==1:
        return "odd"

def pos_neg(n):
    if(n>0):
        return "positive"
    elif(n<0):
        return "negative"
    elif(n==0):
        return "neither positive or negative"

N = int(input('Enter the digit'))

print("The class of this number is defined as", even_odd(N), "and", pos_neg(N))

```

    Enter the digit-12
    The class of this number is defined as even and negative

## Challenge
#### 15) Ordering problems are recurrent in the history of programming. Over time, several algorithms have been developed to fulfill this function. The simplest of these algorithms is the [**Bubble Sort**](https://en.wikipedia.org/wiki/Bubble_sort), which is based on comparisons of elements two by two in a loop of passes through the elements. Your mission, if you decide to accept it, will be to input six whole numbers ramdonly ordered. Then implement the **Bubble Sort** principle to order these six numbers **using only loop and conditional structure**. At the end, print the six numbers in ascending order on a single line separated by space.

```python
# haven´t tried yet
```


# Where are the diamonds? Using Northern Lights

```python
import SimPEG as simpeg
from SimPEG import NSEM
import MT_poster_utils
from IPython.display import HTML
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
```

## Scipy 2016 Poster:
https://drive.google.com/open?id=0B4oB1gnNZnWNcmdncHh3WVZqa1U

## What is Magnetotelluric?

Short answer: The magnetotelluric method is a geophysical technique that relays on Natrual Eletromagnetic (EM) waves as a source of energy to image the subsurface. The method is widely used in geophysical exploration, specially imaging deep structures such as in geothermal, kimberlite or techtonic exploration.

Long answer: Keep on reading...

### What is our goal in this notebook?

In this notebook with are sharing the work we did for our SciPy 2016 poster, where we used NSEM code that are built with the SimPEG package (http://simpeg.xyz) to make data from a geological model (forward model) and to image the model again (invert the data).

This poster is a part of a community effort, using a diamond exploration example as motivation to use the large spectrum of geophysical methods within SimPEG for the same geological setting. We want to demonstrate the advantages of having a shared language between the governing physics of all these problems, the written codes, and other utilitices used in the work. We will show

- Explain the background theory's
- Show and share the code used
- Visulize the data and models

#### Conceptual idea

The northern lights are generated by interaction of charged particles and the Earth’s magnetic field, and along with lightning strikes around the world, generate electromagnetic energy that induces electrical currents in the ground. These currents are affected by electrical conductivity contrasts and by measuring electromagnetic fields at the surface, we can image the subsurface structures using geophysical numerical simulation and optimization.

#### Animation of the physical phenomenon

```python
HTML("Figures/Magnetotelluric_Movie_ThibautAstic.html")
```

Details of the physics at:
http://em.geosci.xyz/content/maxwell3_fdem/MT/MT_N_layered_Earth.html

### Physics (quasi-static approximation)

The Maxwell's equations govern the propagation of EM fields in the Earth. In frequency domain (and using the quasi-static approximation and a $e^{i\omega t}$ time relationship), they are given as

$$
\begin{align}
\nabla \times \textbf{E} + i \omega \textbf{B} &= 0   \\
\nabla \times \frac{1}{\mu} \textbf{B} - \sigma \textbf{E} &=   \textbf{s}_E
\end{align}
$$

The system can be reduced to a single equation

$$
\begin{align}
(\nabla \times \frac{1}{\mu} \nabla \times + i \omega \sigma) \textbf{E}_s &=   - i \omega \Delta \sigma \textbf{E}_p
\end{align}
$$

which is then solved numerically using finite volume methods, in SimPEG.

## Load the discrezited geological model

#### Original isosurfaces model

We start with a geological model which is based on a surfaces derived from diamond exploration.

We need to discritize the Earth and correlate the geological units with physical property (conductivity) need to generate the data.

Below we use built in functions in SimPEG to read VTK (http://www.vtk.org) files of the physical property model in to mesh and model of the conductivity.

```python
# Load the geological discretized model
mesh, modelDict = simpeg.Mesh.TensorMesh.readVTK('./datafiles/nsmesh_model.vtr')
sigma = modelDict['S/m']
```

```python
# Print model information
print mesh.nC," cells"
print mesh
```

```python
# Define the area of interest in UTM (meters)
bw, be = 557100, 557580
bs, bn = 7133340, 7133960
bb, bt = 0,480
```

```python
# View the model
slice=20
fig,ax=plt.subplots(figsize=(10,8))
modelplot = mesh.plotSlice(np.log10(sigma),normal='Y',ind=slice,ax=ax,grid=True, pcolorOpts={"cmap":"viridis"})
ax.set_xlim([bw,be])
ax.set_ylim([0,bt])
ax.set_aspect('equal')
plt.colorbar(modelplot[0])
ax.set_title("Discretized Model",fontsize=24)
```

#### Paraview view

## Load stations locations and frequency range

Below with load in the information needed for solving the problem

```python
# Load stations locations and frequency range
locs = np.load('./datafiles/MTlocations.npy')
freqList = np.load('./datafiles/MTfrequencies.npy')
```

```python
# View a scatter plot of the locations at the surface
plt.scatter(locs[:,0],locs[:,1])
```

```python
# List of the frequencies used for the problem
print freqList
```

### Types of data

The natural source of the MT method is unknown and cannot be seporated from the induced signal due to the Earth's conductivity or explicitly modelled numercially. To address this the ratio's of the meaured fields are used, and assuming that the source is the same, the source cancels out, leaving only the secondary signal in the data.

#### Impedance

Is the ratio of the horizontal electrical over magnetic fields. It is a complex tensor, defined as

$$\begin{align}
\textbf{Z}(\omega) =
\begin{bmatrix}
Z_{xx}(\omega) & Z_{xy}(\omega)\\
Z_{yx}(\omega) & Z_{yy}(\omega)
\end{bmatrix} =
\begin{bmatrix}
\frac{E_x(\omega)}{H_x(\omega)} & \frac{E_x(\omega)}{H_y(\omega)}\\
\frac{E_y(\omega)}{H_x(\omega)} & \frac{E_y(\omega)}{H_y(\omega)}
\end{bmatrix} \label{eq:ImpFull}
\end{align}$$

#### Tipper

Is the ratio of the vertical magnetic field over horizontal. It is a complex vector.

$$\begin{align}
\textbf{T}(\omega) =
\begin{bmatrix}
T_{zx}(\omega) \\
T_{zy}(\omega)
\end{bmatrix} =
\begin{bmatrix}
\frac{H_z(\omega)}{H_x(\omega)} \\
\frac{H_z(\omega)}{H_y(\omega)}
\end{bmatrix} \label{eq:TipFull}
\end{align}$$

## Run Forward modeling script on a cluster

Running the numerical codes to solve the proble are resource heavy. The file below is the script we used to run the forward modelling on a cluster.

[Forward Modeling](datafiles/findDiam_MTforward_HKPK1.py)

## Data Visualization

The data are visulized as sections.

```python
# Load the data - stored as numpy.recArray
mtData = np.load('./datafiles/MTdata.npy')
```

```python
# Plot data
fig, axes, csList = MT_poster_utils.pseudoSect_OffDiagTip_RealImag(mtData,{'y':7133627.5},colBarMode='each')
```

```python
# Make the plot
fig, axes, csList = MT_poster_utils.pseudoSect_FullImpTip_RealImag(mtData,{'y':7133627.5},colBarMode='each')
```

## Run the inversions on a cluster

To image the Earth, based on the data we forward modelled above, we use inversion. As with the forward modelling, the inversion is a computationally intensive procedure, and as before we ran yours on a cluster. A single iteration takes at the order of 4 hours to solve using ~ 20Gb of ram.

We present 2 inversion results, for the impedance and the tipper.

Links to the files used to run the inversions.

[Off-diagonal impedance inversion](datafiles/findDiam_inversion_offdiagonal.py)

[Tipper inversion](datafiles/findDiam_inversion_tipper.py)

## Load Inversion results

```python
#Load Model Off-diagonal
mesh,inv= simpeg.Mesh.TensorMesh.readVTK('./datafiles/recoveredMod_off_it18.vtr')
siginvoff=inv['S/m']
```

```python
#Load Model Tipper
mesh,invtip= simpeg.Mesh.TensorMesh.readVTK('./datafiles/recoveredMod_tip_it36.vtr')
siginvtip=invtip['S/m']
```

```python
MT_poster_utils.CompareInversion(mesh,sigma,siginvoff,siginvtip,slice_ver=20,slice_hor=40)
```

```python

```

SciPy2016/MTwork/CleanedNotebook/Example MT forward and inverse modeling.ipynb


```python
import numpy as np
import matplotlib.pyplot as plt
```

# Funciones para que funcionen las demas

```python
# Funcion de la que me tomo mas tiempo hacerlo
def gauss(a,b):
    '''Funcion que retorna un array "x" despues de hacer una eliminacion de Gauss con pivoteo'''

    n = a.shape[0]
    # https://numpy.org/doc/1.18/reference/generated/numpy.ndarray.html
    # estamos creando un matriz columna
    x = np.ndarray( shape=(n,1), dtype = np.float64 )
    a_copy = a.copy() # con el fin de hacer segumiento a la matriz al final .
    s = [] # lista para alojar los valores maximos (es parte del pivoteo)
    l = [] # usada para reordenar como se hara la eliminacion

    # #### LLENAMOS LA LISTA "S" CON LOS VALORES MAXIMOS  #######
    for i in range(0,n):
        l.append(i) # vamos agregando .Establecemos esta matriz
        smax = 0 # el maximo de una fila es importante para el pivote
        for j in range(0,n):
            # se escoje el maximo entre todos los elemnos de una fila
            smax = max(smax,abs(a[i][j]))

        s.append(smax) # alojamos el maximo de la fila

    # ---------- print('l inicial ' ,l) (SEGUIMIENTO)
    # --------- print('s inicial ', s) (SEGUIMIENTO)

    # ##### PIVOTEO(ESCOJEMOS LA FILAS PIVOTES Y ELIMINAMOS) #########
    # k hace referencia a la columna donde se hara ceros seran creados en el array a_ij
    # Pero recuerda que los ceros no se crean realmente . Por que en esos espacios almacenamos
    # otras cosas(los multiplicadorese para descomposicion LU) .
    for k in range(0,n-1):

        rmax = 0
        # ##### SE ESCOJE LA FILA PIVOTE EN ESTE BLOQUE #########
        for i in range (k,n):

            # esto es para escoger la correcta fila pivote
            # se esta dividiendo (elemnto de columna k y fila [k-n][k])/los Smax
            r = abs( a[l[i]][k] / s[l[i]] )
            if (r > rmax):
                # j = i es para escoger el correcto pivote
                # rmax = r es para escoger la maxima proporcion(ratio)
                rmax,j = r,i
                # al final no usaremos rmax , solo es necesario para detener el if cuando sea necesario

        l[j],l[k] = l[k],l[j] # luego se cambia el lugar donde ocurre el maximo proporcion

        # ####### ELIMINACION GAUSSIANA PERO LA FILA PIVOTE LO DETERMINA "l" ##################
        for i in range(k+1,n):
            xmult = a[l[i]][k]/a[l[k]][k]
            a[l[i]][k] = xmult # los guardo para fines de LU y para eliminar "b"

            a_copy[l[i]][k] -= xmult*a[l[k]][k] # para hacer segumiento(se puede borrar)

            # este for hara segumiento de que se haga operaciones en toda la fila que no es pivot
            for j in range(k+1,n):
                a[l[i]][j] -= xmult*a[l[k]][j]
                a_copy[l[i]][j] -= xmult*a[l[k]][j] # para hacer seguimiento(se puede borrar)

    # --------- print('l final ' ,l) (SEGUIMIENTO)
    # --------- print('a final copy \n' , a_copy) (SEGUIMIENTO)
    # ##### SEGIMOS PIVOTEANDO PERO PARA LA MATRIZ b###
    for k in range(0,n-1):

        # recuerda el que ahora maneja el orden sera la lista "l"
        # debemos hacer las operaciones en el mismo orden que hemos hecho para "a"
        for i in range(k+1,n):
            b[l[i]] -= a[l[i]][k]*b[l[k]]

    #------ print('b final \n ' , b) (SEGUIMIENTO)

    # ######### AHORA HACEMOS LA SUSTITUCION BACKWARD ##########

    # espero se entienda por que -1 . Es por la cuenta por cero
    x[n-1] = b[l[n-1]]/a[l[n-1]][n-1]

    for i in range(n-2,-1,-1):
        summ = b[l[i]]
        for j in range(i+1,n):
            summ -= a[l[i]][j]*x[j]

        x[i] = summ/a[l[i]][i]

    return x

def RK4_system(F=None,t0=0,X0=0,limsup=0,nstep=10):
    '''
    Funcion que recibe como argumentos
    - F = funcion de Rn+1-> Rn F(X,t)
    - t0,X0 = condicion inicial (valor inicial) X0 es una matriz columna de orden "n"
    - limsup = hasta donde se quiere hacer la prediccion
    - nstep = numero de pasos
    Y restorna un array de n+1 dimensiones , donde en cada fila el primer elemento es el tiempo y los restantes son los valores predichos en ese tiempo
    Si se desea tener por columnas podrias usar retornar la transpuesta de la matriz (OJO) .
    '''
    h = (limsup-t0)/nstep
    t = t0
    X = np.array(X0,dtype=np.float64)
    # https://numpy.org/doc/stable/reference/generated/numpy.insert.html ( documentacion )
    valor_inicio = np.insert(X,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
    cal_valores = []
    cal_valores.append(valor_inicio) # agregamos la parte inicial
    for i in range(0,nstep):
        K1 = F(t,X)
        K2 = F(t+1/2*h,X+1/2*K1*h)
        K3 = F(t+1/2*h,X+1/2*K2*h)
        K4 = F(t+1/2*h,X+1/2*K3*h)
        t = t0 + (i+1)*h
        X = X + 1/6*(K1+ 2*K2+ 2*K3 + K4)*h
        valor_calculado = np.insert(X,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
        cal_valores.append(valor_calculado) # agregamos la parte inicial

    lista_resultados = np.array(cal_valores,dtype=np.float64)
    return lista_resultados # aca puedes retornar esto o su transpuesta
```

# Aca intentare compactar la condicion de dirichlet con newman

```python
# solo copiar ,pegar y modificar ciertos detalles . Que esta facil hacerlo
def sol_EDO_Diri_or_New(u=None,v=None,w=None,ta=1,alpha=0,tb=2,beta=0,n=99,tipo="dirichlet",analitic_solution=None):
    '''
    Se quiere resolver la ecuacion diff2x = u(t) + v(t)x + w(t)diff1x para eso se le pasa
    - u,v,t son funciones de R-> R que deben ser pasadas si o si
    - n = es el numero de pasos entre , pero debes tener en mente que en el paso 0 se tiene "ta" y en el paso "n" se tiene "tb"
     Asi por ejemplo si quieres 100 puntos para dividir el dominio de "ta" a "tb" debes poner n=99.
    - Se debe saber que las condicioones depende ,
        - dirichlet : x(ta) = alpha , x(tb) = beta
        - newman_left : x'(ta) = alpha , x(tb) = beta
        - newman_right : x(ta) = alpha , x'(tb) = beta
        - newman_both : x'(ta) = alpha , x'(tb) = beta
    - analitic_solution = Es la solucion analitica (optional) si se conoce

    Esta funcion retornara si se tiene la solucion analitica , Una lista de 4 fila donde la primera fila
    se muestra los puntos donde se predice , la segunda fila los valores predichos , tercero los valores analiticos
    y cuarto el error . Si no se tiene la solucion analitica , es decir "analitic_solution=None", solo se pasa dos filas
    '''
    h = (tb-ta)/n # el tamaño del paso esta determinado por los extremos el numero de pasos "n"

    ########## armamos el espacio para los elementos la matriz (n+1) x (n+1)
    # recuerda que queremos en general predecir los 100 puntos repartidos entre ta y tb ( a veces no se sabe x(ta) y x(tb) )
    a = np.ndarray( shape=(n+1), dtype = np.float64 )
    b = np.ndarray( shape=(n+1), dtype = np.float64 )
    c = np.ndarray( shape=(n+1), dtype = np.float64 )
    d = np.ndarray( shape=(n+1), dtype = np.float64 )
    A = np.zeros( shape=(n+1,n+1), dtype = np.float64 )
    # esto se usara para almacenar los resultados / primera fila sera los ti donde se predice
    # / segunda fila sera los x(ti) predichos / tercera fila sera los reales / cuarta fila el error
    resultado = []

    ############# LLenamos los elemenos de nuestra matrices
    # estos elementos salen de forma natural cuando se forma las matrices a mano
    for i in range(0,n+1):
        t = ta + (i)*h # el i+1 es necesario por el contador de range que toma desde el cero
        a[i] = -(1+(h/2)*w(t))
        b[i] = -h**2*u(t)
        c[i] = -(1-(h/2)*w(t))
        d[i] = 2+h**2*v(t)

    # llenamos la matriz
    if tipo=="dirichlet" or tipo == "newman_right":
        A[0][0] = 1
        b[0] = alpha
    elif tipo == "newman_left" or tipo == "newman_both":
        A[0][0] = d[0]
        A[0][1] = c[0] + a[0]
        b[0] = b[0] + 2*h*alpha*a[0]

    for k in range(1,n):
        A[k][k] = d[k] # llenamos la diagonal
    for k in range(1,n):
        A[k][k+1] = c[k] # llenamos la parte diagonal superiror de la banda tridiagonal
    for k in range(1,n):
        A[k][k-1] = a[k] # llenamos la parte diagonal inferior de la banda tridiagonal

    if tipo=="dirichlet" or tipo == "newman_left":
        A[n][n] = 1
        b[n] = beta
    elif tipo == "newman_right" or tipo == "newman_both":
        A[n][n] = d[n]
        A[n][n-1] = c[n] + a[n]
        b[n] = b[n] - 2*h*beta*c[n]
    #### Resolvemos la matriz
    X = (gauss(A,b)).T # va a retornar una matriz columna de orden 1xn+1 / el .T es para devolver transpuesta

    # agregamso a la lista resultado los puntos evaluados
    puntos = []
    for i in range(0,n+1):
        puntos.append(ta+i*h)
    resultado.append(puntos)

    resultado.append(X[0]) # agregamos los resultados predichos , el [0] es debido a que es una matriz 1x(n+1)

    # ahora si tenemos la solucion analitica entonces es conveniente retornar no solo el resultado sino tambien
    # su comparacion con los valores analiticos y su error
    if analitic_solution != None:
        analitic_solucion = [] # para almacenar las soluciones analiticas
        lista_error = [] # Para almacenar las soluciones analiticas

        # puntos inicial e intermedios
        for i in range(0,n+1):
            t = ta + (i)*h
            analitico = analitic_solution(t)
            error = analitico - X[0][i] # hallamos el error
            analitic_solucion.append(analitico)
            lista_error.append(error)

        resultado.append(analitic_solucion)
        resultado.append(lista_error)

    # retornamos
    return np.array(resultado,dtype=np.float64)
```

### Probamos con condiciones de dirichlet

```python
# Probemos si nuesta funcion trabaja
'''
Queremos resolver el problema diff2x = e^{t} - 3sin(t) + diff1x - x
con x(1) = 1.09737 491 y x(2) = 8.63749 661 y tomar 100 puntos incluyendo a y b
--- Por lo tanto estos son nuestros parametros
- u(t) = e^(t) - 3sin(t)
- v(t) = -1
- w(t) = 1
- n=99
- ta = 1 , alpha = 1.09737 491 , tb = 2 , beta = 8.63749 661
'''
def upara(t):
    return np.float64(np.e**(t)-3*np.sin(t))
def vpara(t):
    return np.float64(-1)
def wpara(t):
    return np.float64(1)
def analitico(t):
    return np.float64(np.e**(t)-3*np.cos(t))

resultado1 = sol_EDO_Diri_or_New(u=upara,v=vpara,w=wpara,ta=1,alpha=1.09737491,tb=2,beta=8.63749661,n=4,tipo="dirichlet",
                           analitic_solution=analitico)
resultado1[0],resultado1[1] ,resultado1[2]
```

    (array([1.  , 1.25, 1.5 , 1.75, 2.  ]),
     array([1.09737491, 2.54494201, 4.2702745 , 6.28991125, 8.63749661]),
     array([1.09737491, 2.54437587, 4.26947747, 6.28934084, 8.63749661]))

### Ahora probare con condiciones de newman - right

```python
# Probemos si nuesta funcion trabaja
'''
Queremos resolver el problema diff2x = 4*t - 4*diff1x
con x(0) = 0 y x(\pi/2) = 0 y tomar 11 puntos incluyendo a y b
--- Por lo tanto estos son nuestros parametros
- u(t) = 4*t
- v(t) = -4
- w(t) = ''
- n=10
- ta = 0 , alpha = 0 , tb = pi/2 , beta = 0
'''
def upara(t):
    return np.float64(4*t)
def vpara(t):
    return np.float64(-4)
def wpara(t):
    return np.float64(0)
def analitico(t):
    return np.float64(t+(1/2)*np.sin(2*t))

resultado1 = sol_EDO_Diri_or_New(u=upara,v=vpara,w=wpara,ta=0,alpha=0,tb=np.pi/2,beta=0,n=5,tipo="newman_right",
                           analitic_solution=analitico)
resultado1[0],resultado1[1] ,resultado1[2]
```

    (array([0.        , 0.31415927, 0.62831853, 0.9424778 , 1.25663706,
            1.57079633]),
     array([0.        , 0.62877912, 1.1333513 , 1.43854454, 1.54789847,
            1.54226701]),
     array([0.        , 0.60805189, 1.10384679, 1.41800605, 1.55052969,
            1.57079633]))

#### Ahora probare con  condiciones de newman - left

```python
# Probemos si nuesta funcion trabaja
'''
Queremos resolver el problema diff2x = x
con x(0) = 1 y x(\pi) = 0 y tomar 100 puntos incluyendo a y b
--- Por lo tanto estos son nuestros parametros
- u(t) = 0
- v(t) = -1
- w(t) = 0
- n=99
- ta = 0 , alpha = 1 , tb = pi , beta = 0
'''
def upara(t):
    return np.float64(0)
def vpara(t):
    return np.float64(1)
def wpara(X):
    return np.float64(0)
def analitico2(t):
    return np.float64(np.e**(t))

resultado1 = sol_EDO_Diri_or_New(u=upara,v=vpara,w=wpara,ta=0,alpha=1,tb=2,beta=np.e**(2),n=3,tipo="newman_left",
                           analitic_solution=analitico2)
resultado1[0],resultado1[1] ,resultado1[2] , np.sin(0.52359878)
```

    (array([0.        , 0.66666667, 1.33333333, 2.        ]),
     array([1.11981211, 2.03532591, 3.855429  , 7.3890561 ]),
     array([1.        , 1.94773404, 3.79366789, 7.3890561 ]),
     0.5000000038119851)

#### Ahora probare con condiciones de newman - both //// confio en que funcione , ya estoy cansado de probar :(

# Aca aplicare el shooting method

```python
# esta funcion no se si esta bien , pero lo he probado y me resulta cosas raras pero muy raras
def shooting_method(Func=None,ta=0,alpha=0,tb=0,beta=0,npaso=20,supos1=0,supos2=0):
    '''
    La siguiente funcion requiere tener el runge kutta que resuelve multiple ( OJO )
    Resuelve un tipo de ecuacion diferenicial , la lineal de "diff2x = f(t,x,x')" .
    ya que esas funciones se pueden desdoblar en ecuaciones 2 ecuaciones de primer grado .
    haciendo x = x1 y x'=x2

    Esta funcion recibe como parametros
    - F = Funcion que se refiere al sistema de ecuaciones desdoblado
    - ta ,alpha,tb , beta = se refiere a las condiciones de dirichlet y(ta) = alpha , y(tb) = beta
    - npaso = esto se refiere al numero de pasos para hacer la evaluacion con runge kuta
    - supos1,supos2 = las dos suposiciones de y'(ta) = supos1 or supos2 .

    Esta funcion retorna un array con 4 filas , la primera fila es los puntos donde se hace las predicciones
    la ssegunda fila es lo hallado con la primera suposicion
    la tercera fila es lo hallado con la suposicion 2 y la cuarta fila es luego de hacer la interpolacion lineal y resolver
    el sistema con la suposicion

    '''
    # recordar que RK4_system devuelve un array de 3 columnas , primera columna los valores evaluados en el tiempo
    # segunda columna los valores predichos para x1 en esos tiempos
    # tercera columan los valores predichos para x2 en esos tiempos

    # resolvemos para la primera suposicion
    respuesta1 = RK4_system(F=Func,t0=ta,X0=np.array([alpha,supos1]),limsup=tb,nstep=npaso).T
    # resolvemos para la segunda suposicion
    respuesta2 = RK4_system(F=Func,t0=ta,X0=np.array([alpha,supos2]),limsup=tb,nstep=npaso).T

    # los valores finales que se obtienen para comparar con beta son
    beta1 = respuesta1[1][-1]
    beta2 = respuesta2[1][-1]

    supos3 = supos1 + (supos2-supos1)/(beta2-beta1) * (beta-beta1)

    # finalmente resolvemos para ese suposicion 3
    respuesta3 = RK4_system(F=Func,t0=ta,X0=np.array([alpha,supos3]),limsup=tb,nstep=npaso).T

    # finalmente retornamos lo que prometimos en la descripcion inicial
    return np.array([respuesta1[0],respuesta1[1],respuesta2[1],respuesta3[1]] , dtype=np.float64)

# Primero definimos nuestra funcion diff2x = f(x,y,y')
# diff2x = np.e**(t) - 3*np.sin(t) + diff1x - x

# desdoblamos la funcion de la siguiente manera
# funcion retorna un array
def funcion1(t,X):
    x1 = X[0]
    x2 = X[1]
    diffx1 = x2
    diffx2 = np.e**(t)-3*np.sin(t)+x2-x1
    return np.array([diffx1,diffx2],dtype=np.float64)

def analitico(t):
    return np.float64(np.e**(t)-3*np.cos(t))

# vamos a resolver el mismo problema que hicimos para las condiciones de dirichlet visto arriba
resultado2 = shooting_method(Func=funcion1,ta=1,alpha=1.09737491,tb=2,beta=8.63749661,npaso=4,supos1=2,supos2=7)
plt.plot(resultado2[0],resultado2[1],'-*c',label="suposicion 1")
plt.plot(resultado2[0],resultado2[2],'-*r',label="suposicion 2")
plt.plot(resultado2[0],resultado2[3],'-*b',label="interpolando")
plt.plot(np.linspace(1,2,50),analitico(np.linspace(1,2,50)),'g',label="teorico")
plt.plot(2,8.63749661,'o')
plt.legend( loc='upper left', shadow=True)
print(resultado2[3][-1],"resultado en tb" )
```

# Resolvere el ejercicio 1 de la tarea

```python
# solucion analitica la hallamos usando sympy
from sympy import *
x , h,T_a = symbols("x,h,T_a")
T = symbols('T', cls=Function)
h = 0.01
T_a = 20
eq = Eq(Derivative(T(x),x,x) , -h*(T_a-T(x) ) )
ics = {T(0): 40, T(10): 200}
resuelto = dsolve(eq,ics=ics)
resuelto
```

```python
def analitico(x):
    return np.float64(73.4522786865455*np.e**(0.1*𝑥)+20.0-53.4522786865455*np.e**(-0.1*𝑥) )

analitico(0),analitico(10) # para comprobar
```

```python
print("//////////////// Pregunta 1 resuelto por Dirichlet//////////////////")
'''
Queremos resolver el problema diff2T = -h'Ta + h'T
con T(0) = 40 y T(10) = 200 y tomar 11 puntos incluyendo a y b , para que mi paso sea igual a "1" , Asi que escojo n = 10
Tambien h'= 0.01 , Ta = 20
--- Por lo tanto estos son nuestros parametros
- u(t) = -h'Ta
- v(t) = h'
- w(t) = 0
- n=10
- ta = 0 , alpha = 40 , tb = 10 , beta = 200
'''
Ta = 20
hprima = 0.01
def upara(x):
    return np.float64(-hprima*Ta)
def vpara(x):
    return np.float64(hprima)
def wpara(x):
    return np.float64(0)
def analitico(x):
    return np.float64(73.4522786865455*np.e**(0.1*x)+20.0-53.4522786865455*np.e**(-0.1*x) )

# resolvemos por dirichlet
resultado1 = sol_EDO_Diri_or_New(u=upara,v=vpara,w=wpara,ta=0,alpha=40,tb=10,beta=200,n=10,tipo="dirichlet",
                           analitic_solution=analitico)

print("//////////////// Pregunta 1 resuelto por shooting//////////////////")

# Primero definimos nuestra funcion diff2T = f(x,T,T')
# diff2T = -h'Ta + h'T
# ta = 0 , alpha = 40 , tb = 10 , beta = 200
# desdoblamos la funcion de la siguiente manera
# funcion retorna un array

def funcion_desdoblado(x,T):
    T1 = T[0]
    T2 = T[1]
    diffT1 = T2
    diffT2 = -hprima*Ta + hprima*T1
    return np.array([diffT1,diffT2],dtype=np.float64)

def analitico(x):
    return np.float64(73.4522786865455*np.e**(0.1*𝑥)+20.0-53.4522786865455*np.e**(-0.1*𝑥) )

# vamos a resolver con el metodo de shooting
resultado2 = shooting_method(Func=funcion_desdoblado,ta=0,alpha=40,tb=10,beta=200,npaso=3,supos1=100,supos2=200)

# graficaremos nuestros resultados
fig, axes = plt.subplots(nrows=1,ncols=2 , figsize = (15,6))
axes[0].plot(resultado1[0],resultado1[1],'-*c',label="Predecido_dirichlet")
axes[0].plot(np.linspace(0,10,50),analitico(np.linspace(0,10,50)),'r',label="Analitico")
axes[0].legend( loc='upper left', shadow=True)

axes[1].plot(resultado2[0],resultado2[1],'-*c',label="suposicion 1")
axes[1].plot(resultado2[0],resultado2[2],'-*r',label="suposicion 2")
axes[1].plot(resultado2[0],resultado2[3],'-*b',label="interpolando")
axes[1].plot(np.linspace(0,10,50),analitico(np.linspace(0,10,50)),'g',label="teorico")
axes[1].annotate(f"({10},{200})", xy=(10,200), xytext=(4, 350),
            arrowprops=dict(facecolor='black', shrink=0.05),fontsize=16)
axes[1].legend( loc='upper left', shadow=True)
plt.show()
```

# Problema numero 2 dejado por el profesor

```python
# solucion analitica la hallamos usando sympy
from sympy import *
x , h,T_infi = symbols("x,h,T_infi")
T = symbols('T', cls=Function)
h = 0.01
T_infi = 40
eq = Eq(Derivative(T(x),x,x) , -h*(T_infi-T(x) ) )
ics = {T(x).diff(x).subs(x, 0): 10, T(10): 200}
resuelto = dsolve(eq,ics=ics)
resuelto
```

```python
# comprobamos
def analitico(x):
    return np.float64(63.7646340953225*np.e**(0.1*x)+40.0-36.2353659046775*np.e**(-0.1*x) )

def analitico_deriv(x):
    return np.float64(63.7646340953225*(0.1)*np.e**(0.1*x)-36.2353659046775*(-0.1)*np.e**(-0.1*x) )

analitico_deriv(0),analitico(10) # para comprobar
```

    (10.0, 199.99999999999972)

```python
print("//////////////// Pregunta 2 resuelto por newman left//////////////////")
'''
Queremos resolver el problema diff2T = -h'Tinfi + h'T
con T'(0) = 10 y T(10) = 200 y tomar 11 puntos incluyendo a y b , para que mi paso sea igual a "1" , Asi que escojo n = 10
Tambien h'= 0.01 , Tinfi = 40
--- Por lo tanto estos son nuestros parametros
- u(t) = -h'Tinfi
- v(t) = h'
- w(t) = 0
- n=10
- ta = 0 , alpha = 10 , tb = 10 , beta = 200
'''
Tinfi = 40
hprima = 0.01
def upara(x):
    return np.float64(-hprima*Tinfi)
def vpara(x):
    return np.float64(hprima)
def wpara(x):
    return np.float64(0)
def analitico(x):
    return np.float64(63.7646340953225*np.e**(0.1*𝑥)+40.0-36.2353659046775*np.e**(-0.1*𝑥) )

resultado3 = sol_EDO_Diri_or_New(u=upara,v=vpara,w=wpara,ta=0,alpha=10,tb=10,beta=200,n=10,tipo="newman_left",
                           analitic_solution=analitico)

# graficaremos nuestros resultados
fig, axes = plt.subplots(nrows=1,ncols=2 , figsize = (15,6))
axes[0].plot(resultado3[0],resultado3[1],'-*c',label="Predecido_newman")
axes[0].plot(np.linspace(0,10,50),analitico(np.linspace(0,10,50)),'r',label="Analitico")
axes[0].legend( loc='upper left', shadow=True)

axes[1].plot(resultado3[0],abs(resultado3[3]),'-*c',label="error_por_resta")
axes[1].legend( loc='lower left', shadow=True)
axes[1].set_yscale("log")

plt.show()

resultado3[0],resultado3[1] ,resultado3[2]
```

# Problema de autovalores con metodo de potencias

```python
def eigevalores_potencias(A=None,X=None,kmax=100):
    '''
    Funcion que usa el metodo de potencias para hallar el autovalor dominante y su respectivo eingevalor de una matriz
    - A = matriz de nxn
    - X = vector inicial de nx1 , (Ojo debes mandar copias si no quieres modificar tu X)
    - kmax = cuantas iteraciones se quiere hacer ,por defecto es 100
    Esta funcion retorna el autovalor y el respectivo autovecto
    '''
    for k in range(0,kmax):
        Y = A.dot(X)
        autovalor = Y[0] / X[0]
        X = Y/Y[1] # esto lo hago con el fin de que mi matriz no diverja
    return X , autovalor
```

#### Tarea

# Primer matriz lo resolvemos analiticamente(usando metodos alternativos )

```python
# resolvemos analiticamente usando sympy
from sympy import *
init_printing(use_unicode=True) # La salida sera en unicode / en latex
```

```python
# Primero
M1 = Matrix([[-4,10],[7,5]])
M1
```

```python
M1.eigenvects()
```

```python

```

### Probamos nuestro codigo para la matriz

```python
print("//////////////// Pregunta de autovalores //////////////////")
A1 = np.array([[-4,10],[7,5]],dtype=np.float64)
X1 = np.array([[1/2],[1]],dtype=np.float64)
matriz , autovalor = eigevalores_potencias(A=A1,X=X1,kmax=100)
print(f"Para la primera matriz tenemos el autovector \n{matriz} \n con el autovalor dominante {autovalor}")
```

    //////////////// Pregunta de autovalores //////////////////
    Para la primera matriz tenemos el autovector
    [[0.71427953]
     [1.        ]]
     con el autovalor dominante [9.99986542]

# Segunda matriz resolvemos analiticamente(usando metodos alternativos )

```python
# Primero
M2 = Matrix([[1,2,-2],[-2,5,-2],[-6,6,-3]])
M2
```

```python
M2.eigenvects()
```

```python
A2 = np.array([[1,2,-2],[-2,5,-2],[-6,6,-3]],dtype=np.float64)
X2 = np.array([[2],[1/2],[1]],dtype=np.float64)
matriz , autovalor = eigevalores_potencias(A=A2,X=X2,kmax=100)
matriz , autovalor
print(f"Para la segunda matriz tenemos el autovector \n{matriz} \n con el autovalor dominante {autovalor}")
```

    Para la segunda matriz tenemos el autovector
    [[4.]
     [1.]
     [2.]]
     con el autovalor dominante [18.]


+++
title = "Causality Tools"
subtitle = ""
date = 2019-04-15T00:00:00
lastmod = 2019-04-15T00:00:00
draft = false

# Authors. Comma separated list, e.g. `["Bob Smith", "David Jones"]`.
authors = ["Kristian A. Haaga", "David Diego", "Tor Einar Møller"]

tags = []
summary = "A Julia package for computing causal estimators from time series data."

# Projects (optional).
#   Associate this post with one or more of your projects.
#   Simply enter your project's folder or file name without extension.
#   E.g. `projects = ["deep-learning"]` references
#   `content/project/deep-learning/index.md`.
#   Otherwise, set `projects = []`.
# projects = ["causalitytools"]
+++

``CausalityTools.jl`` is a Julia package providing algorithms for detecting causal relations in complex systems based on time series data. The main goals are:

* Provide a comprehensive, easy-to-use framework for the detection of directional causal influences in complex dynamical systems from time series.

* Functional and efficient implementations of causality detection algorithms, with thorough documentation and references to primary literature.

* Integration with ``UncertainDatasets.jl``, which greatly simplifies working with uncertain data.

* Integration with ``DynamicalSystems.jl``, for quick analysis of time series from systems where the governing equations are known.

* Library of example dynamical systems for testing algorithm performance.

* Surrogate data methods for null-hypothesis testing. In the future, surrogate methods will be provided as part of resampling schemes.

* Worked examples for the algorithms.

Source code can be found at https://github.com/kahaaga/CausalityTools.jl.

For more information, see the package documentation at https://kahaaga.github.io/CausalityTools.jl/dev/.

# A brief tutorial

Here's an example of how to use some of the functionality in the ``CausalityTools.jl`` package.  The ``jupyter`` notebook was run using Julia 1.2.

```julia
using CausalityTools
using DynamicalSystemsBase
using Distributions
using Measures
using LaTeXStrings
using Plots; pyplot()
```

    Plots.PyPlotBackend()

### Defining a dynamical system and generating time series
Let's define a system composed of two unidirectionally coupled logistic maps:

```julia
function eom_logistic2(dx, x, p, n)
    c_xy, r₁, r₂, σ = (p...,)
    ξ = rand()
    x, y = x[1], x[2]
    f_xy = (y +  (c_xy*(x + σ*ξ)/2) ) / (1 + (c_xy/2)*(1+σ))

    dx[1] = r₁ * x * (1 - x)
    dx[2] = r₂ * (f_xy) * (1 - f_xy)
    return
end
```

    eom_logistic2 (generic function with 1 method)

Create a function that returns an instance of a `DynamicalSystems.DiscreteDynamicalSystem`.

```julia
function logistic2(u₀, c_xy, r₁, r₂, σ)
    p = [c_xy, r₁, r₂, σ]
    DiscreteDynamicalSystem(eom_logistic2_unidir, u₀, p)
end

logistic2(;u₀ = rand(2), c_xy = 0.1, r₁ = 3.78, r₂ = 3.66, σ = 0.05) =
    logistic2(u₀, c_xy, r₁, r₂, σ)
```

    logistic2 (generic function with 2 methods)

Initialise a system

```julia
system = logistic2(c_xy = 0.5, r₁ = 3.7, r₂ = 3.8)
```

    2-dimensional discrete dynamical system
     state:       [0.422895, 0.736883]
     e.o.m.:      eom_logistic2_unidir
     in-place?    true
     jacobian:    ForwardDiff
     parameters:  [0.5, 3.7, 3.8, 0.05]

Generate orbit (time series).

```julia
npts = 300
orbit = trajectory(system, 300, Ttr = 1000); x, y = columns(orbit);
```

Plot it

```julia
plot(size = (700, 400), bottom_margin = 5mm, left_margin = 5mm,
    tickfont = font(12), guidefont = font(12), legendfont = font(12),
    xlabel = "Time", ylabel = "Value", legend = :topleft)
plot!(0:npts, x, c = :black, label = "x", lw = .5)
plot!(0:npts, y, c = :red, label = "y", lw = .5)
```

Okay, so we have some time series

## PredictiveAsymmetryTest

In this example, we will use a new test for causality from time series. Let's first investigate the documentation for the test:

```julia
?PredictiveAsymmetryTest
```

    search: PredictiveAsymmetryTest predictive_asymmetry

```
PredictiveAsymmetryTest(predictive_test::CausalityTest)
```

The parameters for a predictive asymmetry causality test [1].

## Mandatory keywords

  * **`predictive_test`**: An instance of a predictive causality test that explicitly    uses prediction lags (e.g. [`VisitationFrequencyTest`](@ref) or    [`TransferOperatorGridTest`](@ref)).

## About the prediction lags

The prediction lags in the predictive causality test must consist of `n` negative  integers and `n` positive integers that are symmetric around zero.

In other words, negative lags  must exactly match the positive lags but with opposite  sign. The zero lag can be included, but will be ignored, so it is possible to give  ranges too.

## Examples

```julia
test_visitfreq = VisitationFrequencyTest(ηs = [-5, -4, -2, -1, 0, 1, 2, 4, 5])
test_transferoperator = TransferOperatorGridTest(ηs = -3:3)

# Note that `predictive_test` is a *mandatory* keyword.
PredictiveAsymmetryTest(predictive_test = test_visitfreq)
PredictiveAsymmetryTest(predictive_test = test_transferoperator)
```

## References

1. Diego, David, Kristian Agasøster Haaga, Jo Brendryen, and Bjarte Hannisdal.   A simple test for causal asymmetry in complex systems. In prep.

## A Transfer entropy test

Okay, so we first need to define a transfer entropy test to use for the prediction. For this, we have several choices, but for the demonstration, we'll stick to the `TransferOperatorGridTest`, which estimates probabilities from an approximation to the transfer operator (Diego et al., 2019).

```julia
?TransferOperatorGridTest
```

    search: TransferOperatorGridTest TransferOperatorGrid

```
TransferOperatorGridTest(k::Int = 1, l::Int = 1, m::Int = 1, n::Int = 1,
    τ::Int = 1, b = 2, estimator::TransferOperatorGrid = TransferOperatorGrid(),
    binning_summary_statistic::Function = StatsBase.mean,
    binning::RectangularBinning, ηs)
```

The parameters for a transfer entropy test using the `TransferOperatorGrid` estimator [1].

## Mandatory keyword arguments

  * **`binning::RectangularBinning`**: An instance of a [`RectangularBinning`](@ref) that dictates    how the delay embedding is discretized.
  * **`ηs`**: The prediction lags (that gos into the $T_{f}$ component of the embedding).

## Optional keyword arguments

  * **`k::Int`**: The dimension of the $T_{f}$ component of the embedding.
  * **`l::Int`**: The dimension of the $T_{pp}$ component of the embedding.
  * **`m::Int`**: The dimension of the $S_{pp}$ component of the embedding.
  * **`n::Int`**: The dimension of the $C_{pp}$ component of the embedding.
  * **`τ::Int`**: The embedding lag. Default is `τ = 1`.
  * **`b`**: Base of the logarithm. The default (`b = 2`) gives the TE in bits.
  * **`estimator::TransferOperatorGrid`**: A `TransferOperatorGrid` estimator instance.
  * **`binning_summary_statistic::Function`**: A summary statistic to summarise the    transfer entropy values if multiple binnings are provided.

## Estimation of the invariant measure

With a `TransferOperatorGridTest`, the first step is to compute an approximation to the  [transfer operator](@ref transfer_operator_rectangular) over the partition elements of a rectangular [discretization](@ref discretization)  of an appropriate [delay reconstruction](@ref custom_delay_reconstruction) of the time  series to be analysed. Transfer entropy is then computed from the [invariant  distribution](@ref invariant_measure_rectangular) arising from the transfer operator.

## About the delay reconstruction for transfer entropy analysis

Denote the time series for the source process $S$ as $S(t)$, and the time series for  the target process $T$ as $T(t)$, and $C_i(t)$ as the time series for any conditional  processes $C_i$ that also may influence $T$. To compute (conditional) TE, we need a  generalised embedding [3, 4] incorporating all of these processes.

For convenience, define the state vectors

$$
\begin{align}
T_f^{(k)} &= \{(T(t+\eta_k), \ldots, T(t+\eta_2), T(t+\eta_1))\}, \label{eq:Tf} \\
T_{pp}^{(l)} &= \{ (T(t), T(t-\tau_1), T(t-\tau_2), \ldots, T(t - \tau_{l - 1})) \\
S_{pp}^{(m)} &= \{(S(t), S(t-\tau_1), S(t-\tau_2), \ldots, S(t-\tau_{m - 1}))\},\\
C_{pp}^{(n)} &= \{ (C_1(t), C_1(t-\tau_1), \ldots,  C_2(t), C_2(t-\tau_1) \},
\end{align}
$$

where the state vectors $T_f^{(k)}$ contain $k$ future values of the target  variable, $T_{pp}^{(l)}$ contain $l$ present and past values of the target  variable, $S_{pp}^{(m)}$ contain $m$ present and past values of the source  variable, $C_{pp}^{(n)}$ contain a total of $n$ present and past values of any  conditional variable(s).  Combining all variables, we have the generalised embedding

$$
\begin{align}
\mathbb{E} = (T_f^{(k)}, T_{pp}^{(l)}, S_{pp}^{(m)}, C_{pp}^{(n)})
\end{align}
$$

with a total embedding dimension of $k + l + m + n$.  Here, $\tau$ indicates the [embedding](@ref custom_delay_reconstruction) lag  (in the `VisitationFrequencyTest`, we set $\tau_1 = \tau_2 = \tau_3 = \ldots$), and $\eta$ indicates the prediction lag (the lag of the influence the source  has on the target).

Hence, in the generalised embedding, only $T_f$ depends on the prediction lag $\eta$,  which is to be determined by the analyst. For transfer entropy analysis, $\eta$ is chosen  to be some positive integer, while for  [predictive asymmetry analysis](@ref predictive_asymmetry),  symmetric negative and positive $\eta$s are used for computing $\mathbb{A}$.

In terms of this generalised embedding, transfer entropy from a source variable $S$ to a  target variable $T$ with conditioning on variable(s) $C$ is thus defined as

$$
\begin{align}
TE_{S \rightarrow T|C} = \int_{\mathbb{E}} P(T_f, T_{pp}, S_{pp}, C_{pp}) \log_{2}{\left( \frac{P(T_f | T_{pp}, S_{pp}, C_{pp})}{P(T_f | T_{pp}, C_{pp})}\right)}
\end{align}
$$

Without conditioning, we have

$$
\begin{align}
TE_{S \rightarrow T} = \int_{\mathbb{E}} P(T_f, T_{pp}, S_{pp}) \log_{2}{\left(\frac{P(T_f | T_{pp}, S_{pp})}{P(T_f | T_{pp})}\right)}
\end{align}
$$

## Low-level estimator

This test uses the `TransferOperatorGrid` estimator on the following low-level method  under the hood.

  * [`transferentropy(::Any, ::TEVars, ::RectangularBinning, ::TransferEntropyEstimator)`](@ref)

In this estimator, the mapping between variables of the  [generalised embedding](@ref custom_delay_reconstruction) and the marginals during  transfer entropy computation is controlled using a [`TEVars`](@ref)  instance. It is *highly* recommended that you check the documentation for this  method, because it describes the transfer entropy estimation procedure in detail.

## Notes:

  * Use `causality(source, target, params::TransferOperatorGridTest)` for regular    transfer entropy analysis. This method uses only the `k`, `l`, `m` and ignores `n`    when constructing the delay reconstruction.
  * Use `causality(source, target, cond, params::TransferOperatorGridTest)` for conditional    transfer entropy analysis. This method uses the `k`, `l`, `m` *and* `n` when constructing    the delay reconstruction.

## Example

```julia
# Prediction lags
ηs = 1:10
binning = RectangularBinning(10)

# Use defaults, binning and prediction lags are required.
# Note that `binning` and `ηs` are *mandatory* keyword arguments.
TransferOperatorGridTest(binning = binning, ηs = ηs)

# The other keywords can also be adjusted
TransferOperatorGridTest(k = 1, l = 2, binning = binning, ηs = ηs)
```

## References

1. Diego, David, Kristian Agasøster Haaga, and Bjarte Hannisdal. "Transfer entropy computation   using the Perron-Frobenius operator." Physical Review E 99.4 (2019): 042212.  [https://journals.aps.org/pre/abstract/10.1103/PhysRevE.99.042212](https://journals.aps.org/pre/abstract/10.1103/PhysRevE.99.042212)

Now we have the information we need!

#### Define test

```julia
# Maximum prediction
η_max = 8

# Embedding parameters (3-dimensional embedding)
k, l, m = 1, 1, 1

# Partition
n_subdivisions = floor(Int, npts^(1/(k + l + m + 1)))
bin_scheme = RectangularBinning(n_subdivisions)

#Transfer entropy test with base-2 logarithm,
te_test = TransferOperatorGridTest(binning = bin_scheme,
    ηs = -η_max:η_max, b = 2, k = k, l = l, m = m)

# Predictive asymmetry test based on the transfer entropy test
test = PredictiveAsymmetryTest(predictive_test = te_test)
```

    PredictiveAsymmetryTest{TransferOperatorGridTest}(predictive_test = TransferOperatorGridTest(k = 1, l = 1, m = 1, n = 1, τ = 1, b = 2, estimator = TransferOperatorGrid(), binning_summary_statistic = mean, binning = RectangularBinning(4), ηs = -8:8))

#### Apply test to time series

```julia
influence_xtoy = causality(x, y, test)
```

    8-element Array{Float64,1}:
     0.07353044320389923
     0.2048533816659921
     0.3420588893868155
     0.5687136636138481
     0.6351432545620197
     0.7562916061278022
     0.7939027156800291
     0.7966558227653682

```julia
influence_ytox = causality(y, x, test)
```

    8-element Array{Float64,1}:
     -0.03844661921591097
     -0.11252642048669959
     -0.20612641064467407
     -0.32656831712281686
     -0.3982976779871139
     -0.4919700965986813
     -0.535133253437186
     -0.6047774983739265

Plot the predictive asymmetries against the prediction lags.

```julia
plot(size = (600, 300), bottom_margin = 5mm, left_margin = 5mm,
    tickfont = font(12), guidefont = font(12), legendfont = font(12),
    xlabel = "Prediction lag", ylabel = L"\mathbb{A} (bits)",
    legend = :topleft)
plot!(1:η_max, influence_xtoy, m = :circle, c = :black, label = L"x \to y")
plot!(1:η_max, influence_ytox,  m = :circle, c = :red, label = L"y \to x")
hline!([0], ls = :dash, lc = :grey, label = "")
```

We find that the predictive asymmetry in the causal direction ($\mathbb{A}_{x \to y}$) is positive, whereas the asymmetry in the non-causal direction ($\mathbb{A}_{y \to x}$) is negative. We thus conclude that there is a *unidirectional* causal relationship $x \to y$, which is the simple truth.

### An example of bi-directional coupling
But can we also detect if the causal influence goes both ways? To illustrate this, let's define a system of two bi-directionally coupled logistic maps:

```julia
function eom_logistic2_bidir(dx, x, p, n)

    # c_xy is the coupling from x to y
    # c_yx is the coupling from y to x
    # σ_yx is the dynamical noise from y to x
    # σ_xy is the dynamical noise from y to x
    c_xy, c_yx, r₁, r₂, σ_xy, σ_yx = (p...,)

    ξ₁ = rand() # random number from flat distribution on [0, 1]
    ξ₂ = rand() # random number from flat distribution on [0, 1]
    x, y = x[1], x[2]

    f_xy = (y +  c_xy*(x + σ_xy*ξ₁) ) / (1 + c_xy*(1+σ_xy))
    f_yx = (x +  c_yx*(y + σ_yx*ξ₂) ) / (1 + c_yx*(1+σ_yx))

    dx[1] = r₁ * (f_yx) * (1 - f_yx)
    dx[2] = r₂ * (f_xy) * (1 - f_xy)
    return
end

function logistic2_bidir(u₀, c_xy, c_yx, r₁, r₂, σ_xy, σ_yx)
    p = [c_xy, c_yx, r₁, r₂, σ_xy, σ_yx]
    DiscreteDynamicalSystem(eom_logistic2_bidir, u₀, p)
end

logistic2_bidir(;u₀ = rand(2), c_xy = 0.1, c_yx = 0.1,
    r₁ = 3.78, r₂ = 3.66, σ_xy = 0.05, σ_yx = 0.05) =
    logistic2_bidir(u₀, c_xy, c_yx, r₁, r₂, σ_xy, σ_yx)
```

    logistic2_bidir (generic function with 2 methods)

```julia
# realize the system with coupling x -> y stronger than coupling y -> x
system = logistic2_bidir(c_xy = 0.5, c_yx = 0.2)
npts = 300
orbit = trajectory(system, 300, Ttr = 1000)
x, y = columns(orbit);
```

```julia
influence_xtoy = causality(x, y, test)
influence_ytox = causality(y, x, test)

plot(size = (600, 300), bottom_margin = 5mm, left_margin = 5mm,
    tickfont = font(12), guidefont = font(12), legendfont = font(12),
    xlabel = "Prediction lag", ylabel = L"\mathbb{A} (bits)",
    legend = :topleft)
plot!(1:η_max, influence_xtoy, m = :circle, c = :black, label = L"x \to y")
plot!(1:η_max, influence_ytox,  m = :circle, c = :red, label = L"y \to x")
hline!([0], ls = :dash, lc = :grey, label = "")
```

In this case, we find that the predictive asymmetry $\mathbb{A}$ is positive in both directions, and the asymmetry is greater in the direction $x \to y$, as expected. We provide a more comprehensive discussion of the predictive asymmetry test in a forthcoming paper by Haaga *et al.*

For more examples from ``CausalityTools.jl``, see the package documentation.

```julia

```


# Technician Routing and Scheduling Problem

## Objective and Prerequisites

Try this modeling example to discover how mathematical optimization can help telecommunications firms automate and improve their technician assignment, scheduling, and routing decisions in order to ensure the highest levels of customer satisfaction.

This modeling example is at the intermediate level, where we assume that you know Python and are familiar with the Gurobi Python API. In addition, you have some knowledge about building mathematical optimization models. To fully understand the content of this notebook, you should be familiar with object-oriented-programming.

**Download the Repository**

You can download the repository containing this and other examples by clicking [here](https://github.com/Gurobi/modeling-examples/archive/master.zip).

**Gurobi License**

In order to run this Jupyter Notebook properly, you must have a Gurobi license. If you do not have one, you can request an [evaluation license](https://www.gurobi.com/downloads/request-an-evaluation-license/?utm_source=3PW&utm_medium=OT&utm_campaign=WW-MU-TRA-OR-O_LEA-PR_NO-Q3_FY20_WW_JPME_technician-routing-sche-problem_COM_EVAL_GITHUB_&utm_term=technician-routing-scheduling-problem&utm_content=C_JPM) as a *commercial user*, or download a [free license](https://www.gurobi.com/academia/academic-program-and-licenses/?utm_source=3PW&utm_medium=OT&utm_campaign=WW-MU-TRA-OR-O_LEA-PR_NO-Q3_FY20_WW_JPME_technician-routing-sch-problem_ACADEMIC_EVAL_GITHUB_&utm_term=technician-routing-scheduling-problem&utm_content=C_JPM) as an *academic user*.

## Motivation
The Technician Routing and Scheduling (TRS) is a common problem faced by telecommunication firms, which must be able to provide services for a large number of customers. With its limited resources, these firms must try to deliver timely, affordable and reliable services to maximize customer satisfaction. The TRS problem involves assignment, scheduling, and routing of multi-skilled technicians to serve customers with unique priorities, service time windows, processing times, skills requirements, and geographical locations.
In the current practice, these decisions are often being made by operators via heuristics, sometimes manually -and this method
can not only be time-consuming,
but may also deviate from optima significantly.
Using mathematical optimization to solve the
TRS problem  will
enable
telecom operators and dispatchers to automate and improve their assignment, scheduling, and routing decisions, while achieving high customer satisfaction.

## Problem Description
A telecom firm operates multiple service centers to serve its customers.
Each service center has
its technicians who are dispatched from the service center to work on their assigned jobs, and return to the center after all the jobs are completed.
A technician has multiple skills and available working capacity that cannot be exceeded during the scheduling horizon.
A service order/job has
a known processing time,
a customer-specified time window for starting the service,
a deadline of completing the service,
and its skill requirements.
Depending on the nature of the job (routine maintenance or emergency), and relationship with the customer (existing, new), the firm assesses the importance of the job and assigns a priority score to it.
A job is assigned to at most one technician who possesses the required
skills,
but the technician's available capacity during the scheduling horizon cannot be exceeded.
To solve the  TRS problem, telecommunications companies must be able to
simultaneously makes three types of decisions: (i) the assignment of jobs to a technician at all the service centers;
(ii) the routing of each technician, i.e. the sequence/order of customers for a technician to visit;
(iii) and the scheduling of jobs, i.e. the timing for a technician to arrive at a customer location and complete the corresponding job.
The firm’s goal is to minimize the total weighted lateness of all  jobs, with their priority being the weights.

The following constraints must be satisfied:
* A technician, if utilized, departs from the service center where he/she is based and returns to the same service center after his/her assigned jobs are completed.
* A technician’s available capacity during the scheduling horizon cannot be exceeded.
* A job, if selected, is assigned to at most one technician who  possesses the required skills.
* A technician must arrive at a customer location during an interval (time window) specified by the customer, and must complete a job before the deadline required by the customer. This an important constraint for guaranteeing customer satisfaction.

The above basic TRS is a variant of the multi-depot vehicle routing problem with time windows, known as the MDVRPTW in the literature [1].

## Solution Approach

Mathematical programming is a declarative approach where the modeler formulates a mathematical optimization model that captures the key aspects of a complex decision problem. The Gurobi Optimizer solves such models using state-of-the-art mathematics and computer science.

A mathematical optimization model has five components, namely:

* Sets and indices.
* Parameters.
* Decision variables.
* Objective function(s).
* Constraints.

To simplify the MIP formulation, we consider that for each job's skill requirements, we can determine the subset of technicians that are qualified to perform the job based on the skills that the technicians have.

We now present a MIP formulation for the basic TRS problem.

## Model Formulation

### Sets and Indices
$j \in J = \{1,2,...,n\}$: Index and set of jobs.

$k \in K$: Index and set of technicians.

$K(j) \subset K$: Subset of technicians qualified to perform job $j \in J$

$d \in D = \{n+1, n+2, ...,n+m \}$: Index and set of depots (service centers), where $m$ is the number of depots.

$l, i, j \in L = J \cup D = \{1,2,..., n+m \}$: index and set of locations to visit.

### Parameters
$\beta_{k,d} \in \{0,1\}$: This parameter is equal 1 if technician $k \in K$ must depart from and return to depot
$d \in D$; and 0 otherwise.

$p_{j} \in \mathbb{R^{+}}$; Processing time (duration) of job $j \in J$.

$\tau_{i,j} \in \mathbb{R^{+}}$: Travel time, in minutes, from location $i \in L$ to location $j \in L$.

$W_{k} \in \mathbb{N}$: Workload limit of technician $k \in K$ during the planning horizon. That is, the number of hours, in minutes, that the technician is available for the next work day.

$\pi_{j} \in \{1,2,3,4\}$: Priority weight of job $j \in J$, a larger number means higher priority.

$a_{j} \in \mathbb{R^{+}} $: Earliest time to start the service at location of job $j \in J$.

$b_{j} \in \mathbb{R^{+}} $: Latest time to start the service at location of job $j \in J$.

$dd_{j} \in \mathbb{R^{+}} $: Due date of job $j \in J$, i.e. the latest time to complete the service for job $j \in J$.

$M \in \mathbb{N}$: This is a very large number. This number is determined as follows: the planning horizon is of 10 working hours (i.e. 600 min). We want the value of $M$ to be an order of magnitude larger than the planning horizon. Therefore, we set M = 6100.

### Decision Variables

$x_{j,k} \in \{0,1\}$: This variable is equal 1 if job $j \in J$ is assigned to technician $k \in K$, and 0 otherwise.

$u_{k} \in \{0,1\}$: This variable is equal 1 if technician $k \in K$  is used to perform a job, and 0 otherwise.

$y_{i,j,k} \in \{0,1\}$: This variable is equal 1 if technician $k \in K$  travels from location $i \in L$ to location
$j \in L$; and 0 otherwise.

$t_{j} \geq 0$: This variable determines the time to arrive or start the service at location $j \in J$.

$z_{j} \geq 0$: This variable determines the lateness of completing job $j \in J$.

$xa_{j}, xb_{j} \geq 0$: Correction to earliest and latest time to start the service for job $j \in J$.

$g_{j}$: This variable is equal 1 if job $j \in J$ cannot be filled, and 0 otherwise.

### Objective Function

- **Minimize lateness:** The Objective function is to minimize the total weighted lateness of all the jobs.

\begin{equation}
\sum_{j \in J} \pi_{j} \cdot z_{j} + \sum_{j \in J} 0.01 \cdot \pi_{j} \cdot M(xa_{j} + xb_{j}) +
\sum_{j \in J} \pi_{j} \cdot M \cdot g_{j}
\tag{0}
\end{equation}

**Note**: We want to treat the constraints of filling the jobs demand and the constraints about starting the jobs within a time window as soft constraints. Soft constraints can be violated, but the violation will incur in a huge penalty. To consider that each job has different priority, we use the priority penalty for the calculation of the penalty associated with a violation of soft constraints. We assume that not filling jobs demand greatly deteriorate customer satisfaction, consequently should incur the largest penalties associated with the soft constraints. Recall that the value of the paramter $M$ is a large number, then the penalty associated with not filling a job is determined as follows: $\pi_{j} \cdot M$. The time windows constraints violation also deteriorates customer satisfaction but to a lesser degree than the penalty associated with violating the time windows constraints. The time window constraints are determined as follows: $0.01 \cdot \pi_{j} \cdot M$.

### Constraints

- **Assign qualified technicians:** For each job, we  assign one technician who is qualified for the job, or a gap is declared.

\begin{equation}
\sum_{k \in K(j)} x_{j,k} + g_{j} = 1 \quad \forall j \in J
\tag{1}
\end{equation}

**Note**: The penalty of the gap variable $g_{j}$ is ($0.1 \cdot \pi_{j} \cdot M$) which is a large number to discourage
not being able to satisfy demand.

- **Only one technician:** For each job, we only allow one technician to be assigned.

\begin{equation}
\sum_{k \in K} x_{j,k} \leq 1 \quad \forall j \in J
\tag{2}
\end{equation}

- **Technician capacity:** For each technician, we ensure that the available capacity of the technician is not exceeded.

\begin{equation}
\sum_{j \in J}  p_{j} \cdot x_{j,k} + \sum_{i \in L} \sum_{j \in L} \tau_{i,j} \cdot y_{i,j,k} \leq W_{k} \cdot u_{k} \quad \forall k \in K
\tag{3}
\end{equation}

- **Technician tour:** For each technician and job, we ensure that if the technician is assigned to the job, then the technician must travel to another location (to form a tour).

\begin{equation}
\sum_{j \in L}  y_{i,j,k} = x_{i,k} \quad \forall i \in J, k \in K
\tag{4}
\end{equation}

- For each technician and job, we ensure that if a technician is assigned to the job, then the technician must travel from another location to the location of the job (to form a tour).

\begin{equation}
\sum_{i \in L}  y_{i,j,k} = x_{j,k} \quad \forall j \in J, k \in K
\tag{5}
\end{equation}

- **Same depot:** For each technician and depot, we ensure that a technician, if assigned to any job, must depart from and return to the service center (depot) where the technician is based.

\begin{equation}
\sum_{j \in J}  y_{d,j,k} = \beta_{k,d} \cdot u_{k} \quad \forall  k \in K, d \in D
\tag{6}
\end{equation}

\begin{equation}
\sum_{i \in J}  y_{i,d,k} = \beta_{k,d} \cdot u_{k} \quad \forall  k \in K, d \in D
\tag{7}
\end{equation}

- **Temporal relationship:** For each location and job, we ensure the temporal relationship between two consecutive jobs served by the same technician. That is, if a technician $k$ travels from job $i$ to job $j$, then the start of the service time at job $j$ must be no less than the completion time of job $i$ plus the travel time from job $i$ to job $j$.

\begin{equation}
t_{j} \geq t_{i} + p_{i} + \tau_{i,j} - M \cdot (1 - \sum_{k \in K}  y_{i,j,k}) \quad \forall i \in L, j \in J
\tag{8}
\end{equation}

**Note**: Observe that if the technician $k$ travels from the location of job $i$ to the location of job $j$, then
$\sum_{k \in K}  y_{i,j,k} = 1$. Therefore, $M \cdot (1 - \sum_{k \in K}  y_{i,j,k}) = 0$, and the constraint
$t_{j} \geq t_{i} + p_{i} + \tau_{i,j}$ would be properly enforced. Now consider the case where the technician $k$ does not travel  from the location of job $i$ to the location of job $j$. Hence, $\sum_{k \in K}  y_{i,j,k} = 0$ and
$M \cdot (1 - \sum_{k \in K}  y_{i,j,k}) = M$. In this case, this constraint becomes
$t_{j} \geq t_{i} + p_{i} + \tau_{i,j} - M$. But $M$ is a very large number, then $t_{i} + p_{i} + \tau_{i,j} - M < 0$ and
since $t_{j} \geq 0$, this constraint is redundant.

- **Time window:** For each job $j \in J$ ensure that the time window for the job is satisfied.

\begin{equation}
t_{j} \geq a_{j} - xa_{j} \quad \forall j \in J
\tag{9}
\end{equation}

\begin{equation}
t_{j} \leq b_{j} + xb_{j} \quad \forall j \in J
\tag{10}
\end{equation}

**Note**: To discourage that the time window of a job is violated, we associate the  penalty of
($0.01 \cdot \pi_{j} \cdot M$) to the correction variables $xa_{j}, xb_{j}$.

- **Lateness constraint:** For each job $j \in J$ calculate the lateness of the job.

\begin{equation}
z_{j} \geq t_{j} + p_{j} - dd_{j} \quad \forall j \in J
\tag{11}
\end{equation}

- Note that since the lateness decision variable $z_{j}$ is non-negative, there is no benefit to complete a job before its due date; on the other hand, since the objective function minimizes the total weighted lateness, Constraint (11) should always be binding.

## Problem Instance

In this scenario, we consider the problem of routing and scheduling technicians for the next work-day in such a way that the delay of customer appointments is minimized. The telecom firm has seven technicians: Albert, Bob, Carlos, Doris, Ed, Flor, and Gina. There are two service centers: Heidelberg and Freiburg im Breisgau. Technicians are based at only one of these service centers. The number of hours available and the service center base (depot) for each technician is described in the following table.

|  | Albert | Bob | Carlos | Doris | Ed | Flor | Gina |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Minutes | 480 | 480 | 480 | 480 | 480 | 360 | 360 |
| Depot | Heidelberg | Heidelberg | Freiburg im Breisgau | Freiburg im Breisgau | Heidelberg | Freiburg im Breisgau | Heidelberg |

The telecom company has different type of jobs.
The following table shows the priority (4 for the highest priority and 1 for the least important) and duration (in hours) of a job type.

|  | Priority | Duration (min) |
| --- | --- | --- |
| Equipment Installation | 2 | 60 |
| Equipment Setup | 3 | 30 |
| Inspect/Service Equipment | 1 | 60 |
| Repair - Regular | 1 | 60 |
| Repair - Important | 2 | 120 |
| Repair - Urgent | 3 | 90 |
| Repair - Critical | 4 | 60 |

The following table shows the jobs that each technician is qualified for.

|  | Albert | Bob | Carlos | Doris | Ed | Flor | Gina |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Equipment Installation | - | - | - | 1 | - | 1 | 1 |
| Equipment Setup | 1 | 1 | 1 | 1 | - | - | 1 |
| Inspect/Service Equipment | 1 | - | 1 | - | 1 | - | - |
| Repair - Regular | - | 1 | 1 | - | 1 | 1 | 1 |
| Repair - Important | - | - | - | 1 | - | 1 | 1 |
| Repair - Urgent | - | 1 | 1 | - | 1 | 1 | 1 |
| Repair - Critical | - | - | - | 1 | - | - | 1 |

The telecom company receives customers' requests for a specific job type, appointment (due) time, and the service time window where the technician can arrive. The following table shows the customers' orders and their requirements. For each customer, the location is specified.

|  | C1:Mannheim  | C2: Karlsruhe  | C3: Baden-Baden  | C4: Bühl  | C5: Offenburg | C6: Lahr/Schwarzwald | C7: Lörrach |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Job type | Equipment Setup | Equipment Setup | Repair - Regular | Equipment Installation | Equipment Installation | Repair - Critical | Inspect/Service Equipment |
| Due time | 8:00 | 10:00 | 11:00  | 12:00 | 14:00 | 15:00 | 16:00 |
| Time Window | 7:00-7:30 | 7:30-9:30 | 8:00-10:00 | 9:00-11:00 | 11:00-13:00 | 12:00-14:00 | 13:00-15:00 |

The planning horizon is from 7:00 to 17:00, or 10 hours. The time period is in minutes, then the due time and time windows will be translated into minutes starting at 0 minutes and ending at 600 minutes. For example, for customer C2 the due time is at 10:00  (180 min), and the time window is from 7:30 to 9:30  (30 min to  150 min).

The following table shows the travel time (in minutes) from any depot or customer location to any depot or customer location.

|  | Heidelberg | Freiburg im Breisgau | Mannheim  | Karlsruhe  | Baden-Baden  | Bühl  | Offenburg | Lahr/Schwarzwald | Lörrach |
| --- | --- | --- | --- | --- | --- | --- | --- |--- |--- |
| Heidelberg | - | 120 | 24 | 50 | 67 | 71 | 88 | 98 | 150 |
| Freiburg im Breisgau | - | - | 125 | 85 | 68 | 62 | 45 | 39 | 48 |
| Mannheim | - | - | - | 53 | 74 | 77 | 95 | 106 | 160 |
| Karlsruhe | - | - | - | - | 31 | 35 | 51 | 61 | 115 |
| Baden-Baden | - | - | - | - | - | 16 | 36 | 46 | 98 |
| Bühl | - | - | - | - | - | - | 30 | 40 | 92 |
| Offenburg | - | - | - | - | - | - | - | 26 | 80 |
| Lahr/Schwarzwald | - | - | - | - | - | - | - | - | 70 |
| Lörrach | - | - | - | - | - | - | - | - | - |

## Python Implementation

In this Jupyter Notebook, we use the following libraries:

* `sys` to access system-specific parameters and functions.
* `defaultdict` to supply missing values for arbitrary keys.
* `xlrd` to read data from Excel files.
* `gurobipy` Gurobi Optimizer library.

This implementation is based on object-oriented-programming.

### Helper Classes

The following classes properly organize the input data of the MIP model.

* `Technician`.
* `Job`.
* `Customer`.

### Helper Functions

* `solve_trs0` builds and solves MIP model.
* `printScen` prints headings for output reports.

```python
import sys
from collections import defaultdict
import xlrd
import gurobipy as gp
from gurobipy import GRB

# tested with Gurobi v9.1.0 and Python 3.7.0
```

```python
class Technician():
    def __init__(self, name, cap, depot):
        self.name = name
        self.cap = cap
        self.depot = depot

    def __str__(self):
        return f"Technician: {self.name}\n  Capacity: {self.cap}\n  Depot: {self.depot}"
```

```python
class Job():
    def __init__(self, name, priority, duration, coveredBy):
        self.name = name
        self.priority = priority
        self.duration = duration
        self.coveredBy = coveredBy

    def __str__(self):
        about = f"Job: {self.name}\n  Priority: {self.priority}\n  Duration: {self.duration}\n  Covered by: "
        about += ", ".join([t.name for t in self.coveredBy])
        return about
```

```python
class Customer():
    def __init__(self, name, loc, job, tStart, tEnd, tDue):
        self.name = name
        self.loc = loc
        self.job = job
        self.tStart = tStart
        self.tEnd = tEnd
        self.tDue = tDue

    def __str__(self):
        coveredBy = ", ".join([t.name for t in self.job.coveredBy])
        return f"Customer: {self.name}\n  Location: {self.loc}\n  Job: {self.job.name}\n  Priority: {self.job.priority}\n  Duration: {self.job.duration}\n  Covered by: {coveredBy}\n  Start time: {self.tStart}\n  End time: {self.tEnd}\n  Due time: {self.tDue}"

```

## Base Scenario

For the base scenario, there is enough technician capacity to fill all customers demand. To run the base scenario, consider the spreadsheet file `data-Sce0.xlsx` and insert this name when you open the Excel workbook.

```python
# Open Excel workbook
wb = xlrd.open_workbook('data-Sce0.xlsx')

# Read technician data
ws = wb.sheet_by_name('Technicians')
technicians = []
for i,t in enumerate(ws.col_values(0)[3:]):
    # Create Technician object
    thisTech = Technician(*ws.row_values(3+i)[:3])
    technicians.append(thisTech)
```

```python
# Read job data
jobs = []
for j,b in enumerate(ws.row_values(0)[3:]):
    coveredBy = [t for i,t in enumerate(technicians) if ws.cell_value(3+i,3+j) == 1]
    # Create Job object
    thisJob = Job(*ws.col_values(3+j)[:3], coveredBy)
    jobs.append(thisJob)
```

```python
# Read location data
ws = wb.sheet_by_name('Locations')
locations = ws.col_values(0)[1:]
dist = {(l, l) : 0 for l in locations}
for i,l1 in enumerate(locations):
    for j,l2 in enumerate(locations):
        if i < j:
            dist[l1,l2] = ws.cell_value(1+i, 1+j)
            dist[l2,l1] = dist[l1,l2]
```

```python
# Read customer data
ws = wb.sheet_by_name('Customers')
customers = []
for i,c in enumerate(ws.col_values(0)[1:]):
    for b in jobs:
        if b.name == ws.cell_value(1+i, 2):
            # Create Customer object using corresponding Job object
            rowVals = ws.row_values(1+i)
            #print(rowVals)
            thisCustomer = Customer(*rowVals[:2], b, *rowVals[3:])
            customers.append(thisCustomer)
            break

```

```python
def solve_trs0(technicians, customers, dist):
    # Build useful data structures
    K = [k.name for k in technicians]
    C = [j.name for j in customers]
    J = [j.loc for j in customers]
    L = list(set([l[0] for l in dist.keys()]))
    D = list(set([t.depot for t in technicians]))
    cap = {k.name : k.cap for k in technicians}
    loc = {j.name : j.loc for j in customers}
    depot = {k.name : k.depot for k in technicians}
    canCover = {j.name : [k.name for k in j.job.coveredBy] for j in customers}
    dur = {j.name : j.job.duration for j in customers}
    tStart = {j.name : j.tStart for j in customers}
    tEnd = {j.name : j.tEnd for j in customers}
    tDue = {j.name : j.tDue for j in customers}
    priority = {j.name : j.job.priority for j in customers}

        ### Create model
    m = gp.Model("trs0")

    ### Decision variables
    # Customer-technician assignment
    x = m.addVars(C, K, vtype=GRB.BINARY, name="x")

    # Technician assignment
    u = m.addVars(K, vtype=GRB.BINARY, name="u")

    # Edge-route assignment to technician
    y = m.addVars(L, L, K, vtype=GRB.BINARY, name="y")

    # Technician cannot leave or return to a depot that is not its base
    for k in technicians:
        for d in D:
            if k.depot != d:
                for i in L:
                    y[i,d,k.name].ub = 0
                    y[d,i,k.name].ub = 0

    # Start time of service
    t = m.addVars(L, ub=600, name="t")

    # Lateness of service
    z = m.addVars(C, name="z")

    # Artificial variables to correct time window upper and lower limits
    xa = m.addVars(C, name="xa")
    xb = m.addVars(C, name="xb")

    # Unfilled jobs
    g = m.addVars(C, vtype=GRB.BINARY, name="g")

        ### Constraints

    # A technician must be assigned to a job, or a gap is declared (1)
    m.addConstrs((gp.quicksum(x[j, k] for k in canCover[j]) + g[j] == 1 for j in C), name="assignToJob")

    # At most one technician can be assigned to a job (2)
    m.addConstrs((x.sum(j, '*') <= 1 for j in C), name="assignOne")

    # Technician capacity constraints (3)
    capLHS = {k : gp.quicksum(dur[j]*x[j,k] for j in C) +\
        gp.quicksum(dist[i,j]*y[i,j,k] for i in L for j in L) for k in K}
    m.addConstrs((capLHS[k] <= cap[k]*u[k] for k in K), name="techCapacity")

    # Technician tour constraints (4 and 5)
    m.addConstrs((y.sum('*', loc[j], k) == x[j,k] for k in K for j in C),\
        name="techTour1")
    m.addConstrs((y.sum(loc[j], '*', k) == x[j,k] for k in K for j in C),\
        name="techTour2")

    # Same depot constraints (6 and 7)
    m.addConstrs((gp.quicksum(y[j,depot[k],k] for j in J) == u[k] for k in K),\
        name="sameDepot1")
    m.addConstrs((gp.quicksum(y[depot[k],j,k] for j in J) == u[k] for k in K),\
        name="sameDepot2")

    # Temporal constraints (8) for customer locations
    M = {(i,j) : 600 + dur[i] + dist[loc[i], loc[j]] for i in C for j in C}
    m.addConstrs((t[loc[j]] >= t[loc[i]] + dur[i] + dist[loc[i], loc[j]]\
        - M[i,j]*(1 - gp.quicksum(y[loc[i],loc[j],k] for k in K))\
        for i in C for j in C), name="tempoCustomer")

    # Temporal constraints (8) for depot locations
    M = {(i,j) : 600 + dist[i, loc[j]] for i in D for j in C}
    m.addConstrs((t[loc[j]] >= t[i] + dist[i, loc[j]]\
        - M[i,j]*(1 - y.sum(i,loc[j],'*')) for i in D for j in C),\
        name="tempoDepot")

    # Time window constraints (9 and 10)
    m.addConstrs((t[loc[j]] + xa[j] >= tStart[j] for j in C), name="timeWinA")
    m.addConstrs((t[loc[j]] - xb[j] <= tEnd[j] for j in C), name="timeWinB")

    # Lateness constraint (11)
    m.addConstrs((z[j] >= t[loc[j]] + dur[j] - tDue[j] for j in C),\
        name="lateness")

    ### Objective function
    M = 6100

    m.setObjective(z.prod(priority) + gp.quicksum( 0.01 * M * priority[j] * (xa[j] + xb[j]) for j in C) +
                   gp.quicksum( M * priority[j] * g[j] for j in C) , GRB.MINIMIZE)

    m.write("TRS0.lp")
    m.optimize()

    status = m.Status
    if status in [GRB.INF_OR_UNBD, GRB.INFEASIBLE, GRB.UNBOUNDED]:
        print("Model is either infeasible or unbounded.")
        sys.exit(0)
    elif status != GRB.OPTIMAL:
        print("Optimization terminated with status {}".format(status))
        sys.exit(0)

    ### Print results
    # Assignments
    print("")
    for j in customers:
        if g[j.name].X > 0.5:
            jobStr = "Nobody assigned to {} ({}) in {}".format(j.name,j.job.name,j.loc)
        else:
            for k in K:
                if x[j.name,k].X > 0.5:
                    jobStr = "{} assigned to {} ({}) in {}. Start at t={:.2f}.".format(k,j.name,j.job.name,j.loc,t[j.loc].X)
                    if z[j.name].X > 1e-6:
                        jobStr += " {:.2f} minutes late.".format(z[j.name].X)
                    if xa[j.name].X > 1e-6:
                        jobStr += " Start time corrected by {:.2f} minutes.".format(xa[j.name].X)
                    if xb[j.name].X > 1e-6:
                        jobStr += " End time corrected by {:.2f} minutes.".format(xb[j.name].X)
        print(jobStr)

    # Technicians
    print("")
    for k in technicians:
        if u[k.name].X > 0.5:
            cur = k.depot
            route = k.depot
            while True:
                for j in customers:
                    if y[cur,j.loc,k.name].X > 0.5:
                        route += " -> {} (dist={}, t={:.2f}, proc={})".format(j.loc, dist[cur,j.loc], t[j.loc].X, j.job.duration)
                        cur = j.loc
                for i in D:
                    if y[cur,i,k.name].X > 0.5:
                        route += " -> {} (dist={})".format(i, dist[cur,i])
                        cur = i
                        break
                if cur == k.depot:
                    break
            print("{}'s route: {}".format(k.name, route))
        else:
            print("{} is not used".format(k.name))

    # Utilization
    print("")
    for k in K:
        used = capLHS[k].getValue()
        total = cap[k]
        util = used / cap[k] if cap[k] > 0 else 0
        print("{}'s utilization is {:.2%} ({:.2f}/{:.2f})".format(k, util,\
            used, cap[k]))
    totUsed = sum(capLHS[k].getValue() for k in K)
    totCap = sum(cap[k] for k in K)
    totUtil = totUsed / totCap if totCap > 0 else 0
    print("Total technician utilization is {:.2%} ({:.2f}/{:.2f})".format(totUtil, totUsed, totCap))

```

```python
def printScen(scenStr):
    sLen = len(scenStr)
    print("\n" + "*"*sLen + "\n" + scenStr + "\n" + "*"*sLen + "\n")

```

```python
if __name__ == "__main__":
    # Base model
    printScen("Solving base scenario model")
    solve_trs0(technicians, customers, dist)

```

    ***************************
    Solving base scenario model
    ***************************

    Using license file c:\gurobi\gurobi.lic
    Warning: variable name "y[Loerrach,Freiburg im Breisgau,Albert]" has a space
    Warning: constraint name "tempoDepot[Freiburg im Breisgau,Customer1]" has a space
    Warning: to let Gurobi read it back, use rlp format
    Gurobi Optimizer version 9.1.0 build v9.1.0rc0 (win64)
    Thread count: 4 physical cores, 8 logical processors, using up to 8 threads
    Optimize a model with 217 rows, 660 columns and 2329 nonzeros
    Model fingerprint: 0x0f3b53fc
    Variable types: 30 continuous, 630 integer (630 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 8e+02]
      Objective range  [1e+00, 2e+04]
      Bounds range     [1e+00, 6e+02]
      RHS range        [1e+00, 6e+02]
    Found heuristic solution: objective 245830.00000
    Presolve removed 28 rows and 188 columns
    Presolve time: 0.04s
    Presolved: 189 rows, 472 columns, 2116 nonzeros
    Variable types: 26 continuous, 446 integer (446 binary)

    Root relaxation: objective 0.000000e+00, 141 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

    H    0     0                       0.0000000 -256200.00      -     -    0s
         0     0          -    0         0.00000    0.00000  0.00%     -    0s

    Explored 0 nodes (308 simplex iterations) in 0.09 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 2: 0 245830

    Optimal solution found (tolerance 1.00e-04)
    Best objective 0.000000000000e+00, best bound 0.000000000000e+00, gap 0.0000%

    Bob assigned to Customer1 (Equipment Setup) in Mannheim. Start at t=30.00.
    Doris assigned to Customer2 (Equipment Setup) in Karlsruhe. Start at t=113.00.
    Gina assigned to Customer3 (Repair - Regular) in Baden-Baden. Start at t=67.00.
    Gina assigned to Customer4 (Equipment Installation) in Buehl. Start at t=143.00.
    Doris assigned to Customer5 (Equipment Installation) in Offenburg. Start at t=240.00.
    Doris assigned to Customer6 (Repair - Critical) in Lahr/Schwarzwald. Start at t=326.00.
    Ed assigned to Customer7 (Inspect/Service Equipment) in Loerrach. Start at t=480.00.

    Albert is not used
    Bob's route: Heidelberg -> Mannheim (dist=24.0, t=30.00, proc=30.0) -> Heidelberg (dist=24.0)
    Carlos is not used
    Doris's route: Freiburg im Breisgau -> Karlsruhe (dist=85.0, t=113.00, proc=30.0) -> Offenburg (dist=51.0, t=240.00, proc=60.0) -> Lahr/Schwarzwald (dist=26.0, t=326.00, proc=60.0) -> Freiburg im Breisgau (dist=39.0)
    Ed's route: Heidelberg -> Loerrach (dist=150.0, t=480.00, proc=60.0) -> Heidelberg (dist=150.0)
    Flor is not used
    Gina's route: Heidelberg -> Baden-Baden (dist=67.0, t=67.00, proc=60.0) -> Buehl (dist=16.0, t=143.00, proc=60.0) -> Heidelberg (dist=71.0)

    Albert's utilization is 0.00% (0.00/480.00)
    Bob's utilization is 16.25% (78.00/480.00)
    Carlos's utilization is 0.00% (0.00/480.00)
    Doris's utilization is 73.12% (351.00/480.00)
    Ed's utilization is 75.00% (360.00/480.00)
    Flor's utilization is 0.00% (0.00/360.00)
    Gina's utilization is 76.11% (274.00/360.00)
    Total technician utilization is 34.07% (1063.00/3120.00)

## Base Scenario Analysis
For the base scenario, we have enough technician capacity to satisfy customers demand. Notice that the optimal objective function value is 0.00, this means that all jobs were completed, they were completed on time, and there were no corrections to the time window limits.

The first report describes which technician was assigned to each customer, the location of the customer, and the start time of the job. The second report describes the route of each assigned technician. The route defines the *from-location* and the
*to-location*. The parameter *dist*  defines the number of minutes that it takes to drive from the *from-location* to the *to-location*. The parameter t determines the *start-time* of the job at the *to-location*. The parameter *proc*  shows the number of minutes that it takes the technician to complete the job.
The third report describes the capacity utilization of each technician and the overall capacity utilization of all technicians. The technician capacity utilization is the number of minutes the technician spends driving or serving customers, divided by the technician capacity. The overall capacity utilization of all technicians is the total number of minutes that all technicians spend driving or serving customers, divided by the total capacity of all the technicians available.

## Scenario 1

We assume that technicians are working at half of their capacity. Also, we assume that customer 5 requires an urgent service and its due time is at 00 min. Since the due time has changed, we remove the time window limits, that is the time window is any time within the planning horizon. The new time window is [0, 600].

To run this scenario, consider the spreadsheet file `data-Sce3.xlsx` and insert this name when you open the Excel workbook.

```python
# Open Excel workbook
wb = xlrd.open_workbook('data-Sce3.xlsx')

# Read technician data
ws = wb.sheet_by_name('Technicians')
technicians = []
for i,t in enumerate(ws.col_values(0)[3:]):
    # Create Technician object
    thisTech = Technician(*ws.row_values(3+i)[:3])
    technicians.append(thisTech)
```

```python
# Read job data
jobs = []
for j,b in enumerate(ws.row_values(0)[3:]):
    coveredBy = [t for i,t in enumerate(technicians) if ws.cell_value(3+i,3+j) == 1]
    # Create Job object
    thisJob = Job(*ws.col_values(3+j)[:3], coveredBy)
    jobs.append(thisJob)
```

```python
# Read location data
ws = wb.sheet_by_name('Locations')
locations = ws.col_values(0)[1:]
dist = {(l, l) : 0 for l in locations}
for i,l1 in enumerate(locations):
    for j,l2 in enumerate(locations):
        if i < j:
            dist[l1,l2] = ws.cell_value(1+i, 1+j)
            dist[l2,l1] = dist[l1,l2]
```

```python
# Read customer data
ws = wb.sheet_by_name('Customers')
customers = []
for i,c in enumerate(ws.col_values(0)[1:]):
    for b in jobs:
        if b.name == ws.cell_value(1+i, 2):
            # Create Customer object using corresponding Job object
            rowVals = ws.row_values(1+i)
            #print(rowVals)
            thisCustomer = Customer(*rowVals[:2], b, *rowVals[3:])
            customers.append(thisCustomer)
            break
```

```python
if __name__ == "__main__":
    # Base model
    printScen("Solving scenario 1 model")
    solve_trs0(technicians, customers, dist)
```

    ************************
    Solving scenario 1 model
    ************************

    Warning: variable name "y[Loerrach,Freiburg im Breisgau,Albert]" has a space
    Warning: constraint name "tempoDepot[Freiburg im Breisgau,Customer1]" has a space
    Warning: to let Gurobi read it back, use rlp format
    Gurobi Optimizer version 9.1.0 build v9.1.0rc0 (win64)
    Thread count: 4 physical cores, 8 logical processors, using up to 8 threads
    Optimize a model with 217 rows, 660 columns and 2329 nonzeros
    Model fingerprint: 0x58dea2f1
    Variable types: 30 continuous, 630 integer (630 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 8e+02]
      Objective range  [1e+00, 2e+04]
      Bounds range     [1e+00, 6e+02]
      RHS range        [1e+00, 6e+02]
    Found heuristic solution: objective 216670.00000
    Presolve removed 168 rows and 593 columns
    Presolve time: 0.02s
    Presolved: 49 rows, 67 columns, 279 nonzeros
    Found heuristic solution: objective 24610.000000
    Variable types: 15 continuous, 52 integer (52 binary)

    Root relaxation: objective 8.214242e+03, 16 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

    *    0     0               0    12410.000000 12410.0000  0.00%     -    0s

    Explored 0 nodes (17 simplex iterations) in 0.04 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 3: 12410 24610 216670

    Optimal solution found (tolerance 1.00e-04)
    Best objective 1.241000000000e+04, best bound 1.241000000000e+04, gap 0.0000%

    Gina assigned to Customer1 (Equipment Setup) in Mannheim. Start at t=24.00.
    Bob assigned to Customer2 (Equipment Setup) in Karlsruhe. Start at t=50.00.
    Bob assigned to Customer3 (Repair - Regular) in Baden-Baden. Start at t=111.00.
    Nobody assigned to Customer4 (Equipment Installation) in Buehl
    Flor assigned to Customer5 (Equipment Installation) in Offenburg. Start at t=45.00. 105.00 minutes late.
    Doris assigned to Customer6 (Repair - Critical) in Lahr/Schwarzwald. Start at t=300.00.
    Carlos assigned to Customer7 (Inspect/Service Equipment) in Loerrach. Start at t=480.00.

    Albert is not used
    Bob's route: Heidelberg -> Karlsruhe (dist=50.0, t=50.00, proc=30.0) -> Baden-Baden (dist=31.0, t=111.00, proc=60.0) -> Heidelberg (dist=67.0)
    Carlos's route: Freiburg im Breisgau -> Loerrach (dist=48.0, t=480.00, proc=60.0) -> Freiburg im Breisgau (dist=48.0)
    Doris's route: Freiburg im Breisgau -> Lahr/Schwarzwald (dist=39.0, t=300.00, proc=60.0) -> Freiburg im Breisgau (dist=39.0)
    Ed is not used
    Flor's route: Freiburg im Breisgau -> Offenburg (dist=45.0, t=45.00, proc=60.0) -> Freiburg im Breisgau (dist=45.0)
    Gina's route: Heidelberg -> Mannheim (dist=24.0, t=24.00, proc=30.0) -> Heidelberg (dist=24.0)

    Albert's utilization is 0.00% (0.00/240.00)
    Bob's utilization is 99.17% (238.00/240.00)
    Carlos's utilization is 65.00% (156.00/240.00)
    Doris's utilization is 57.50% (138.00/240.00)
    Ed's utilization is 0.00% (0.00/240.00)
    Flor's utilization is 83.33% (150.00/180.00)
    Gina's utilization is 43.33% (78.00/180.00)
    Total technician utilization is 48.72% (760.00/1560.00)

## Scenario 1 Analysis

We assume that technicians are working at half of their capacity. Also, we assume that customer 5 requires an urgent service and its due time is at 00 min. Since the due time has changed, we remove the time window limits, that is the time window is any time within the planning horizon. The new time window is [0, 600].

* Six jobs out of seven are filled.
  * The service of customer 5 will be 105 min late.
  * The start time is at min 45 which is within the time window of [0, 600].
* Service requirement of Customer 4 could not be filled.
  * Observe that the priority of this customer is low.
* Six technicians were assigned to fill the requirements of 6 customers.
* Ed was not assigned to any customer.
  * Notice that Ed is not qualified for the job of Customer 4 (Equipment Installation).
* The overall technician utilization is 54.23%.

## Conclusion

In this Jupyter Notebook, we formulated and solved a multi-depot vehicle routing problem with time windows constraints using the Gurobi Python API. This MIP formulation is an optimization application for the telecommunication industry; however, the formulation is quite general and can easily be adapted to any kind of vehicle routing problems in the transportation and logistics industries.

The routing and scheduling problem that we tackled considers a telecom firm that operates multiple service centers to serve its customers. We solved this routing and scheduling problem by simultaneously making three types of decisions:
- (i) the assignment of jobs to a technician at all the service centers
- (ii) the routing of each technician, i.e. the sequence/order of customers for a technician to visit
- (iii) the scheduling of jobs, i.e. the timing for a technician to arrive at a customer location and complete the corresponding job.

The objective of the telecom firm is to minimize the total weighted lateness of all the jobs.

In this notebook, we discussed two scenarios, the base scenario has enough technicians' capacity to fill all customers demand on time. In scenario 1, we reduced the capacity of all technicians by half, and one customer has an emergency that requires service first thing in the morning. The scenario captures various trade-offs that the MIP model needs to consider given the limited technician capacity and customers' job priority and requirements.

## References
[1] S. Salhi, A. Imran, N. A. Wassan. *The multi-depot vehicle routing problem with heterogeneous vehicle fleet: Formulation and a variable neighborhood search implementation*. Computers & Operations Research 52 (2014) 315-325.

Copyright © 2020 Gurobi Optimization, LLC

```python

```