# Lecture 7: Optimization Using Python - ORTools

In this lecture / tutorial, we will learn how to solve some simple optimization problems using Python, focusing on the specific optimization library ```ortools```.

***

## Learning goals
- Obtain an overview of optimization problems that can be easily solved using ORTools.
- Learn the syntax to solve some simple optimization problems using ORTools.
- Test your understanding by solving a few of the practice problems in each section.

***
# Prerequisites for running this notebook

You should have Python 3.6 installed on your computer, with all necessary packages installed.

We recommend that you install Anaconda (Python 3.6 version) from the following links depending on your OS:
- For Windows: https://www.anaconda.com/download/#windows
- For macOS: https://www.anaconda.com/download/#macos
- For Linux: https://www.anaconda.com/download/#linux

**If you are not using Anaconda, it is your responsibility to make sure that Python and all necessary packages are correctly installed and configured to be able to run this notebook.**

***

Once Anaconda is installed, open a **Terminal** (if you are using macOS / Linux), or **Anaconda Prompt** (if you are using Windows), and then create a new Python environment called **cme193**, by running the following command:

> ```conda create -n cme193 python=3.6```

Next, change to the newly created virtual environment by running the command:

On Windows
> ```activate cme193```

On macOS or Linux
> ```source activate cme193```

Next install all the necessary packages by running the following commands:

> ```conda install nb_conda```

> ```conda install -c anaconda scipy```

> ```conda install -c conda-forge matplotlib```

> ```conda install -c anaconda networkx```

> ```pip install ortools```

Now navigate to the directory containing this .ipynb file, from inside the terminal, and start jupyter notebook by typing the following command:
> ```jupyter notebook```

You should now be able to launch the .ipynb file from the browser. For more information on jupyter notebooks, read the user documentation.

***
# 1. Introduction to OR-Tools

In this section we will learn how to solve some simple optimization problems using the ```OR-Tools``` package. ```OR-Tools``` is an open source software suite for optimization, available from Google. It is possible to configure ```OR-Tools``` to use commercial solvers like ```CPLEX``` or ```Gurobi```, or open-source solvers like ```SCIP``` or ```GLPK```, but this involves building ```OR-Tools``` from source, and we will not discuss this here as it is an advanced topic that is not suited for an introductory course on Python. Instead we will focus on using Google's ```GLOP``` and ```CP-SAT``` solver which is available upon following the installation instructions, as described above. More information on ```OR-Tools``` can be found at the OR-Tools homepage. The user guide can be found here, which contains extensive documentation and lots of examples.

**Note: Detailed documentation only exists for C++ interface. The documentation for the Python interface is mostly work in progress. But the examples provided by ```OR-Tools``` are good enough to do many sophisticated tasks at an introductory level!**

The main tools provided by ```OR-Tools```, that we need to be aware of are solvers for the following broad category of problems:
- ```Constraint Programming```: The specialized ```CP-SAT``` solver (or the old ```original CP solver```) has been designed specifically to solve these kind of problems. The current recommendation is to always use the ```CP-SAT``` solver whenever possible. We will mostly stick to this guideline in this tutorial, with a few possible exceptions.
- ```Linear and Mixed Integer Linear Programming```: These are the kind of problems that the specialized library ```GLOP``` is designed to solve. For solving Mixed Integer Linear Programming (MILP) problems, the default installer uses the Coin-or branch and cut (CBC) open-source solver.
- ```Vehicle Routing```: This is a specialized library designed specifically for solving routing problems.
- ```Graph Algorithms```: Specialized library for finding shortest paths, max flows, min-cost flows and linear sum assignment.
- ```Bin Packing```: Specialized library for bin packing problems such as knapsack.

We will learn to use the ```OR-Tools``` library by solving a few examples in each of the above categories.

We can import the ```OR-Tools``` library as follows (henceforth to be referred to as ```ortools```). We also import some other modules we will use in this notebook.

```python
import ortools
import scipy.optimize as sciopt
import numpy as np
import networkx as nx
import matplotlib.pyplot as plt
```

***
## 1.1 Linear programming

We have already seen how to solve linear programming (LP) examples using ```scipy.optimize```. So we will keep this discussion concise, and reuse a couple of the examples from there, and solve it using ```ortools```. More information on solving LPs using ```ortools``` can be found here. Google's open-source linear solver library ```GLOP``` is specifically designed for solving linear programs.

```GLOP``` requires that the LP be expressed in the following form

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & c^{T}x  \\
\text{subject to} \;\; & b_{lb} \leq Ax \leq b_{ub}
\end{split}
\end{equation}
$$

where $c, x \in \mathbb{R}^n$, $A \in \mathbb{R}^{m \times n}$, and $b_{ub}, b_{lb} \in \mathbb{R}^{m}$. It should be noted that all LP can be put in this form, as for equality constraints we can set upper and lower bounds to be the same. If either upper or lower bound is not present, then one can set it to $-\infty$ and $\infty$ respectively, as shown in the examples below.

Let us first import the python wrapper ```pywraplp``` for the underlying C++ solver using the following Python code.

```python
from ortools.linear_solver import pywraplp
```

***
### Example 1.1.1
We consider an example that we have encountered previously on solving LPs using ```scipy.optimize```. The example is

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & x_1 + 2 x_2 - 3 x_3  \\
\text{subject to} \;\; & |x_1| \leq 1 \\
& |x_2| \leq 2 \\
& |x_3| \leq 1 \\
& x_1 + x_2 + x_3 = 1,
\end{split}
\end{equation}
$$

which we saw is equivalent to the following optimization problem

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & x_1 + 2 x_2 - 3 x_3  \\
\text{subject to} \;\; & -1 \leq x_1 \leq 1 \\
& -2 \leq x_2 \leq 2 \\
& -1 \leq x_3 \leq 1 \\
& x_1 + x_2 + x_3 = 1.
\end{split}
\end{equation}
$$

The basic steps involved in solving the LP with ```pywraplp``` are:
- Declare the solver - the algorithm that solves the problem
- Create the variables in the LP
- Define the constraints
- Define the objective function
- Invoke the solver to solve the problem
- Extract information about the solved problem

We demonstrate basic usage and implementation of these steps below using Python code.

**Note: For each of the object handles we obtain below, there are a lot of methods for the object which can be accessed but not discussed in this tutorial. For example to access ```solver``` object's methods, just type ```solver.``` and hit ```tab``` on your keyboard in a Jupyter Notebook ```code cell```.**

#### Declare the solver
Notice that the argument ```pywraplp.Solver.GLOP_LINEAR_PROGRAMMING``` tells the solver to use ```GLOP```.

```python
# Instantiate a Glop solver, naming it Example1
solver = pywraplp.Solver('Example1', pywraplp.Solver.GLOP_LINEAR_PROGRAMMING)
```

#### Create the variables in the LP
The basic syntax is to call the ```solver``` object's method ```NumVar``` as ```solver.NumVar(lower bound, upper bound, name)```.

```python
# Create the variables and put bounds on them thus incorporating the inequality constraints
x1 = solver.NumVar(-1, 1, 'x1')
x2 = solver.NumVar(-2, 2, 'x2')
x3 = solver.NumVar(-1, 1, 'x3')
```

#### Define the constraints
This is done in two steps for each constraint:
- Set the bounds on the constraints using the syntax ```constraint = solver.Constraint(lower bound, upper bound)```.
- Set the coefficients of the variables using the created ```constraint``` object's ```SetCoefficient``` method as ```constraint.SetCoefficient(variable, coefficient)```.

```python
# Constraint 1: x1 + x2 + x3 = 1
constraint1 = solver.Constraint(1, 1)
constraint1.SetCoefficient(x1, 1)
constraint1.SetCoefficient(x2, 1)
constraint1.SetCoefficient(x3, 1)
```

#### Define the objective
This is done in two steps for the obejctive:
- Create the object ```objective``` by calling the ```Objective``` method of the ```solver``` object as ```objective = solver.Objective()```.
- Set the coefficients of each variable in the objective function using the created ```objective``` object's method ```SetCoefficient``` using the syntax ```constraint.SetCoefficient(variable, coefficient)```.
- Set whether to maximize or minimize the objective as ```objective.SetMaximization()``` or ```objective.SetMinimization()
``` respectively.

```python
# Objective function: x1 + 2 * x2 - 3 * x3
objective = solver.Objective()
objective.SetCoefficient(x1, 1)
objective.SetCoefficient(x2, 2)
objective.SetCoefficient(x3, -3)
objective.SetMinimization()
```

#### Invoke the solver to solve the problem
Call the ```Solve``` method of the ```solver``` object as ```solver.Solve()```.

```python
# Solve the problem and verify the problem has an optimal solution
status = solver.Solve()
assert status == pywraplp.Solver.OPTIMAL
```

#### Extract information about the solved problem
The following Python code shows how to extract information from the ```solver``` object.

```python
# Print information of the problem
print('Number of variables =', solver.NumVariables())
print('Number of constraints =', solver.NumConstraints())

# The value of each variable in the solution
print('Solution:')
print('x1 = ', x1.solution_value())
print('x2 = ', x2.solution_value())
print('x3 = ', x3.solution_value())

# The objective value of the solution
print('Optimal objective value =', objective.Value())
```

    Number of variables = 3
    Number of constraints = 1
    Solution:
    x1 =  1.0
    x2 =  -1.0
    x3 =  1.0
    Optimal objective value = -4.0

***
### Example 1.1.2
Consider the example from before in the LP tutorial section using ```scipy.optimize```
$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & x_1 + 2 x_2  \\
\text{subject to} \;\; & x_1 \leq 1 \\
& 5 x_1 + x_2 \geq 0 \\
& x_1 + x_2 = 3.
\end{split}
\end{equation}
$$

We demonstrate a full Python program to solve it below, especially how to handle the constraint $5 x_1 + x_2 \geq 0$.

```python
"""
Python code for Example 2
"""
def lp_example2():

    # Instantiate a Glop solver, naming it Example 2
    solver = pywraplp.Solver('Example1', pywraplp.Solver.GLOP_LINEAR_PROGRAMMING)

    # Create the variables and put bounds on them
    x1 = solver.NumVar(-solver.infinity(), 1, 'x1')
    x2 = solver.NumVar(-solver.infinity(), solver.infinity(), 'x2')

    # Constraint 1: x1 + x2 = 3
    constraint1 = solver.Constraint(3, 3)
    constraint1.SetCoefficient(x1, 1)
    constraint1.SetCoefficient(x2, 1)

    # Constraint 2: 5 * x1 + x2 >= 0
    constraint2 = solver.Constraint(0, solver.infinity())
    constraint2.SetCoefficient(x1, 5)
    constraint2.SetCoefficient(x2, 1)

    # Objective function: x1 + 2 * x2
    objective = solver.Objective()
    objective.SetCoefficient(x1, 1)
    objective.SetCoefficient(x2, 2)
    objective.SetMinimization()

    # Solve the system
    status = solver.Solve()

    # Print information of the problem
    print('Number of variables =', solver.NumVariables())
    print('Number of constraints =', solver.NumConstraints())

    # The value of each variable in the solution
    print('Solution:')
    print('x1 = ', x1.solution_value())
    print('x2 = ', x2.solution_value())

    # The objective value of the solution
    print('Optimal objective value =', objective.Value())

if __name__ == "__main__":
    lp_example2()
```

    Number of variables = 2
    Number of constraints = 2
    Solution:
    x1 =  1.0
    x2 =  2.0
    Optimal objective value = 5.0

***
### Exercise 1.1.1
Study the Stigler diet example solved using ```GLOP``` on the documentation page. Change the problem in your own way, and modify the code to solve your modified problem.

```python
# Write your code here
```

***
## 1.2 Mixed-integer linear programming
While solving combinatorial optimization problems, one often encounters situations where some of the variables are only allowed to be integers. If such a problem can be represented as an optimization problem with a cost function that is linear in the variables of the problem, and some (but not all) of the variables are constrained to be integers, then it is called a **Mixed Integer Linear Program (MILP)**. If all of the variables are constrained to be integers then it is called an **Integer Linear Program (ILP)**.

```ortools``` provides us several options to solve these kinds of problems:
- Mixed integer programming (MIP) solver
- Constraint programming (CP) solver
- Min cost flow solver

Of these, the first two are very general and can be used to solve many different MILP problems, while the min cost flow solver can only solve structured problems representable as network flow problems. There are some key differences between all three of them. In this section we focus on the MIP solver, while the other two are discussed in later sections.

The MIP solver that is provided by ```ortools``` is just an interface to the Coin-or branch and cut (CBC) open-source solver. While CBC allows the capability to also solve **Mixed Integer Quadratic Programming (MIQP)** problems, currently this capability is not wrapped by ```ortools```.

The basic MILP problem type that we can solve using ```ortools``` is

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & c^{T}x  \\
\text{subject to} \;\; & b_{lb} \leq Ax \leq b_{ub}
\end{split}
\end{equation}
$$

where $x$ can be partitioned into two sets of variables $x = (x_1, x_2)$, with $x_1$ constrained to be integers, and $x_2$ not constrained to be integers. As in the case of LPs, note that any MILP can be put in this form; in particular for equality constraints we just set the upper and lower bounds to be the same. More information on solving MILPs with ```ortools``` can be found here.

We illustrate the process of solving such problems using ```ortools``` with a few examples. The python wrapper ```pywraplp``` that we will use was already imported before.

***
### Example 1.2.1
Consider the following optimization problem over the variables $x_1, x_2, x_3, x_4, x_5$

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & x_1 + 2 x_2 - 3 x_3 + x_4  \\
\text{subject to} \;\; & 3 x_2 + x_4 + x_5 \leq 2 \\
& -1 \leq x_1 + x_3 + x_4 \leq 1 \\
& x_1 + 2 x_2 + x_3 = 10 \\
& x_1, x_2 \in \{1,2\} \\
& x_5 \in \{0,1,2\}.
\end{split}
\end{equation}
$$

The basic steps involved in solving this MILP with ```pywraplp``` are analogous to the LP case:
- Declare the solver - the algorithm that solves the problem
- Create the variables in the MILP
- Define the constraints
- Define the objective function
- Invoke the solver to solve the problem
- Extract information about the solved problem

We demonstrate basic usage and implementation of these steps below using Python code.

#### Declare the solver
Notice that the argument ```pywraplp.Solver.CBC_MIXED_INTEGER_PROGRAMMING``` tells the solver to use the MIP solver.

```python
# Instantiate a mixed-integer solver, naming it Example1
solver = pywraplp.Solver('Example1', pywraplp.Solver.CBC_MIXED_INTEGER_PROGRAMMING)
```

#### Create the variables in the MILP
The basic syntax is to call the ```solver``` object's method ```NumVar``` as ```solver.NumVar(lower bound, upper bound, name)``` for defining non-integer variables, while for integer variables we need to call the ```solver``` object's method ```IntVar``` as ```solver.IntVar(lower bound, upper bound, name)```.

```python
# Create the non-integer variables
x3 = solver.NumVar(-solver.infinity(), solver.infinity(), 'x3')
x4 = solver.NumVar(-solver.infinity(), solver.infinity(), 'x4')

# Create the integer variables and put bounds for the ones applicable
x1 = solver.IntVar(1, 2, 'x1')
x2 = solver.IntVar(1, 2, 'x2')
x5 = solver.IntVar(0, 2, 'x5')
```

#### Define the constraints
This is done exactly as in the case of LP.

```python
# Constraint 1: 3 * x2 + x4 + x5 <= 2
constraint1 = solver.Constraint(-solver.infinity(), 2)
constraint1.SetCoefficient(x2, 3)
constraint1.SetCoefficient(x4, 1)
constraint1.SetCoefficient(x5, 1)

# Constraint 2: -1 <= x1 + x3 + x4 <= 1
constraint2 = solver.Constraint(-1, 1)
constraint2.SetCoefficient(x1, 1)
constraint2.SetCoefficient(x3, 1)
constraint2.SetCoefficient(x4, 1)

# Constraint 3: x1 + 2 * x2 + x3 = 10
constraint3 = solver.Constraint(10, 10)
constraint3.SetCoefficient(x1, 1)
constraint3.SetCoefficient(x2, 2)
constraint3.SetCoefficient(x3, 1)
```

#### Define the objective
This is done exactly as in the case of LP.

```python
# Objective function: x1 + 2 * x2 - 3 * x3 + x4
objective = solver.Objective()
objective.SetCoefficient(x1, 1)
objective.SetCoefficient(x2, 2)
objective.SetCoefficient(x3, -3)
objective.SetCoefficient(x4, 1)
objective.SetMinimization()
```

#### Invoke the solver to solve the problem
Call the ```Solve``` method of the ```solver``` object as ```solver.Solve()```.

```python
# Solve the problem and verify that an optimal solution has been found
status = solver.Solve()
assert status == pywraplp.Solver.OPTIMAL
```

#### Extract information about the solved problem
The following Python code shows how to extract information from the ```solver``` object.

```python
# Print information of the problem
print('Number of variables =', solver.NumVariables())
print('Number of constraints =', solver.NumConstraints())

# The value of each variable in the solution
print('Solution:')
print('x1 = ', x1.solution_value())
print('x2 = ', x2.solution_value())
print('x3 = ', x3.solution_value())
print('x4 = ', x4.solution_value())
print('x5 = ', x5.solution_value())

# The objective value of the solution
print('Optimal objective value =', objective.Value())
```

    Number of variables = 5
    Number of constraints = 3
    Solution:
    x1 =  1.0
    x2 =  1.0
    x3 =  7.000000000000001
    x4 =  -9.000000000000002
    x5 =  0.0
    Optimal objective value = -27.0

***
### Example 1.2.2: Weighted Vertex Cover
The **weighted vertex cover** is a classic problem in combinatorial optimization. The basic setting is that we have a simple graph $G(V,E)$, which means that is it is an undirected graph with no multiple edges and with no loops, and is equipped with a cost function defined on the set of vertices $c : V \rightarrow \mathbb{R}$. The goal is to find a subset of vertices $S \subset V$ that **cover** all the edges in $E$, such that the total sum of the cost function for the selected vertices is minimized. An edge $e \in E$ is said to be covered by $S$ if and only if there exists a vertex $v \in S$ that is an end point of $e$. Clearly this problem is feasible, as choosing $S=V$ covers all the edges in $E$.

The goals of the weighted vertex cover problem can be expressed by an integer (binary) optimization problem. Let us assign a binary variable $x_v \in \{0,1\}$ for every vertex $v \in V$, with $x_v = 1$ if and only if $v \in S$, and $0$ otherwise. Then the goals of the weighted vertex cover problem can be expressed as the following ILP:

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & \sum_{v \in V} c(v) \; x_v  \\
\text{subject to} \;\; & x_u + x_v \geq 1, \;\; \forall \;\; \{u,v\} \in E  \\
& x_v \in \{0,1\}, \;\; \forall \;\; v \in V.
\end{split}
\end{equation}
$$

The first constraint says that if $\{u,v\}$ is an edge, then it must be covered, while the second constraint says that each vertex is either selected in the set $S$ or not.

Let us take a concrete example. Let $V = \{1, 2, 3, 4, 5\}$, and $E = \{ \{1, 2\}, \{1, 3\}, \{2, 3\}, \{3, 4\}, \{1, 5\} \}$. Let the cost function be $c(1) = 1, \; c(2) = 20, \; c(3) = -2.5, \; c(4) = 0, \; \text{and} \; c(5) = 2$.

We first visualize the graph using the ```NetworkX``` package which we have already imported before. More information on ```NetworkX``` can be found on its documentation page.

```python
%matplotlib inline

# Function for visualizing graph
def graph_visualize(V, E, valmin=0, valmax=1, values=None):
    """
    V: list of vertices
    E: list of edges (each edge is a tuple of vertices)
    """

    # Create an empty graph object
    G = nx.Graph()

    # Add the vertices to G
    G.add_nodes_from(V)

    # Add the edges to G
    G.add_edges_from(E)

    # Draw the graph
    if values is None:
        values = len(G.nodes()) * [0.5]
        nx.draw_circular(G, with_labels=True, cmap=plt.get_cmap('Reds'), node_color=values, vmin=valmin, vmax=valmax)
    else:
        nx.draw_circular(G, with_labels=True, cmap=plt.get_cmap('Reds'), node_color=values, vmin=valmin, vmax=valmax)

if __name__ == "__main__":

    # Create vertex list
    V = [1, 2, 3, 4, 5]

    # Create edge list
    E = [(1, 2), (1, 3), (2, 3), (3, 4), (1, 5)]

    # Create list of node values
    values = [1, 20, -2.5, 0, 2]

    # Print vertex and edge information
    print("List of vertices:", V)
    print("List of edges:", E)
    print("List of node values:", values)

    # Visualize the graph
    print("\nDrawing the graph")
    graph_visualize(V, E)
```

The following Python code solves the weighted vertex cover problem using ```ortools```.

```python
from ortools.linear_solver import pywraplp

def weighted_vertex_cover():

    # Represent the problem data
    V = [1, 2, 3, 4, 5]
    E = [(1, 2), (1, 3), (2, 3), (3, 4), (1, 5)]

    # Print the problem data
    print("List of vertices in the graph:", V)
    print("List of edges in the graph:", E)

    # Instantiate a mixed-integer solver, naming it Weighted-Set-Cover
    solver = pywraplp.Solver('Weighted-Vertex-Cover', pywraplp.Solver.CBC_MIXED_INTEGER_PROGRAMMING)

    # Define integer binary variables.
    x1 = solver.IntVar(0, 1, '1')
    x2 = solver.IntVar(0, 1, '2')
    x3 = solver.IntVar(0, 1, '3')
    x4 = solver.IntVar(0, 1, '4')
    x5 = solver.IntVar(0, 1, '5')

    # Constraint 1 (edge (1,2) is covered): x1 + x2 >= 1
    constraint1 = solver.Constraint(1, solver.infinity())
    constraint1.SetCoefficient(x1, 1)
    constraint1.SetCoefficient(x2, 1)

    # Constraint 2 (edge (1,3) is covered): x1 + x3 >= 1
    constraint2 = solver.Constraint(1, solver.infinity())
    constraint2.SetCoefficient(x1, 1)
    constraint2.SetCoefficient(x3, 1)

    # Constraint 3 (edge (2,3) is covered): x2 + x3 >= 1
    constraint3 = solver.Constraint(1, solver.infinity())
    constraint3.SetCoefficient(x2, 1)
    constraint3.SetCoefficient(x3, 1)

    # Constraint 4 (edge (3,4) is covered): x3 + x4 >= 1
    constraint4 = solver.Constraint(1, solver.infinity())
    constraint4.SetCoefficient(x3, 1)
    constraint4.SetCoefficient(x4, 1)

    # Constraint 5 (edge (1,5) is covered): x1 + x5 >= 1
    constraint5 = solver.Constraint(1, solver.infinity())
    constraint5.SetCoefficient(x1, 1)
    constraint5.SetCoefficient(x5, 1)

    # Minimize 1 * x1 + 20 * x2 - 2.5 * x3 + 0 * x4 + 2 * x5
    objective = solver.Objective()
    objective.SetCoefficient(x1, 1)
    objective.SetCoefficient(x2, 20)
    objective.SetCoefficient(x3, -2.5)
    objective.SetCoefficient(x4, 0)
    objective.SetCoefficient(x5, 2)
    objective.SetMinimization()

    # Solve the problem and verify the problem has an optimal solution
    result_status = solver.Solve()
    assert result_status == pywraplp.Solver.OPTIMAL

    # Print the selected subsets in the optimal solution, and extract the optimal value of all variables
    print("\n")
    print("The selected vertices are:")
    values_opt = []
    for item in ['1', '2', '3', '4', '5']:
        var = solver.LookupVariable(item)
        values_opt.append(var.solution_value())
        if var.solution_value() == 1:
            print(item)

    # Display solution
    graph_visualize(V, E)
    plt.title("Original Graph", fontsize=16)
    plt.show()

    graph_visualize(V, E, 0, 2, values_opt)
    plt.title("Vertex Cover", fontsize=16)
    plt.show()

if __name__ == "__main__":
    weighted_vertex_cover()
```

***
### Example 1.2.3: Weighted Set Cover
The **weighted set cover** problem is another classic problem in combinatorial optimization. Suppose that we are given a finite set $\mathcal{S}$ of elements, and another subset $\mathcal{T}$ of the power set of $\mathcal{S}$, i.e. $\mathcal{T} \subset 2^{\mathcal{S}}$, with the property that $\bigcup\limits_{t \in \mathcal{T}} t = \mathcal{S}$. There is also a cost function $w : \mathcal{T} \rightarrow \mathbb{R}$. The goal is to find a subset of $\mathcal{T}$ that covers all the elements in $\mathcal{S}$, such that the total sum of the costs of the selected elements of $\mathcal{T}$ is minimized.

Formally our goals can be expressed as an integer (binary) optimization problem. Assign a binary variable $x_t \in \{0,1\}$ for every element $t \in \mathcal{T}$, which will be referred to as **subset indicator variables**. Also for all $t \in \mathcal{T}$, and $s \in \mathcal{S}$, we define $c_{ts} = 1$ if $s \in t$, and $c_{ts} = 0$ if $s \notin t$. Then our weighted set cover problem goals can be expressed by the following ILP:

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & \sum_{t \in \mathcal{T}} w(t) \; x_t  \\
\text{subject to} \;\; & \sum_{t \in \mathcal{T}} c_{ts} x_t \geq 1, \;\; \forall \;\; s \in \mathcal{S}  \\
& x_t \in \{0,1\}, \;\; \forall \;\; t \in \mathcal{T}.
\end{split}
\end{equation}
$$

The first constraint expresses the fact that each element $s \in \mathcal{S}$ is covered by at least one element $t \in \mathcal{T}$, which is the **set cover** constraint, from which the problem derives its name.

Let us take a concrete example. Suppose $\mathcal{S} = \{1,2,3,4,5,6,7\}$, and let $\mathcal{T} = \{a,b,c,d\}$, where

$$
\begin{equation}
\begin{split}
a &= \{1,2,3\} \\
b &= \{3,4,6\} \\
c &= \{4,5\} \\
d &= \{2,5,6,7\}.
\end{split}
\end{equation}
$$

We will represent $c_{ts}$ using a cost matrix $C$ defined below, with rows indexing elements of $\mathcal{T}$, and columns indexing elements of $\mathcal{S}$,

$$
C =
\begin{bmatrix}
1 & 1 & 1 & 0 & 0 & 0 & 0 \\
0 & 0 & 1 & 1 & 0 & 1 & 0 \\
0 & 0 & 0 & 1 & 1 & 0 & 0 \\
0 & 1 & 0 & 0 & 1 & 1 & 1
\end{bmatrix}
\;\;.
$$

Also let the cost function $w$ be the constant function $w(t) = 1$, for all $t \in \mathcal{T}$, which corresponds to the **original set cover** problem, that seeks to minimize the number of selected subsets that cover the set $\mathcal{S}$.

Here is the full Python code that solves the problem using ```ortools```.

```python
from ortools.linear_solver import pywraplp

def weighted_set_cover():

    # Represent the problem data
    S = [1, 2, 3, 4, 5, 6, 7]
    T = {'a':{1, 2, 3}, 'b':{3, 4, 6}, 'c':{4, 5}, 'd':{2, 5, 6, 7}}

    # Print the problem
    print("The set S of elements:")
    for item in S:
        print(item)

    print("\n")
    print("The set T of subsets of S:")
    for key, val in T.items():
        print(key, ":", val)

    # Instantiate a mixed-integer solver, naming it Weighted-Set-Cover
    solver = pywraplp.Solver('Weighted-Set-Cover', pywraplp.Solver.CBC_MIXED_INTEGER_PROGRAMMING)

    # Define integer binary variables.
    xa = solver.IntVar(0, 1, 'a')
    xb = solver.IntVar(0, 1, 'b')
    xc = solver.IntVar(0, 1, 'c')
    xd = solver.IntVar(0, 1, 'd')

    # Constraint 1: xa >= 1
    constraint1 = solver.Constraint(1, solver.infinity())
    constraint1.SetCoefficient(xa, 1)

    # Constraint 2: xa + xd >= 1
    constraint2 = solver.Constraint(1, solver.infinity())
    constraint2.SetCoefficient(xa, 1)
    constraint2.SetCoefficient(xd, 1)

    # Constraint 3: xa + xb >= 1
    constraint3 = solver.Constraint(1, solver.infinity())
    constraint3.SetCoefficient(xa, 1)
    constraint3.SetCoefficient(xb, 1)

    # Constraint 4: xb + xc >= 1
    constraint4 = solver.Constraint(1, solver.infinity())
    constraint4.SetCoefficient(xb, 1)
    constraint4.SetCoefficient(xc, 1)

    # Constraint 5: xc + xd >= 1
    constraint5 = solver.Constraint(1, solver.infinity())
    constraint5.SetCoefficient(xc, 1)
    constraint5.SetCoefficient(xd, 1)

    # Constraint 6: xb + xd >= 1
    constraint6 = solver.Constraint(1, solver.infinity())
    constraint6.SetCoefficient(xb, 1)
    constraint6.SetCoefficient(xd, 1)

    # Constraint 7: xd >= 1
    constraint6 = solver.Constraint(1, solver.infinity())
    constraint6.SetCoefficient(xd, 1)

    # Minimize xa + xb + xc + xd
    objective = solver.Objective()
    objective.SetCoefficient(xa, 1)
    objective.SetCoefficient(xb, 1)
    objective.SetCoefficient(xc, 1)
    objective.SetCoefficient(xd, 1)
    objective.SetMinimization()

    # Solve the problem and verify the problem has an optimal solution
    result_status = solver.Solve()
    assert result_status == pywraplp.Solver.OPTIMAL

    # Print the selected subsets in the optimal solution
    print("\n")
    print("The selected subsets are:")
    for item in ['a', 'b', 'c', 'd']:
        var = solver.LookupVariable(item)
        if var.solution_value() == 1:
            print(item, ":", T[item])

if __name__ == "__main__":
    weighted_set_cover()
```

    The set S of elements:
    1
    2
    3
    4
    5
    6
    7

    The set T of subsets of S:
    a : {1, 2, 3}
    b : {3, 4, 6}
    c : {4, 5}
    d : {2, 5, 6, 7}

    The selected subsets are:
    a : {1, 2, 3}
    b : {3, 4, 6}
    d : {2, 5, 6, 7}

***
## 1.3 Bin packing: multidimensional knapsack
Bin packing refers to the problem of finding a set of objects to pack into bins. The objects have **volumes**, and each bin has a **capacity**, which is the total volume the container can hold. We discuss the multidimensional knapsack problem here, which is arguably the most famous bin packing problem. More information on solving bin packing problems using ```ortools``` can be found here.

### 1.3.1 Multidimensional knapsack
The setting involves a finite set of objects $\mathcal{S}$, each with $n + 1$ attributes. The first $n$ attributes are **volumes** (or some other property) of each object along $n$ different dimensions, and the last attribute is the **value** of each object. There is a **knapsack** (or container) which also has $n$ attributes associated with it (called **capacities**), and correspond to the total volume of objects that can fit along each dimension in the knapsack. The objective of the problem is to choose objects from $\mathcal{S}$ to put into the knapsack, such that the total value of all the objects is as large as possible, and the total volume of the selected objects do not exceed the capacity of the knapsack along any dimension.

Mathematically the knapsack problem is equivalent to an ILP. We briefly mention this formulation here, as it shows the combinatorial structure of the problem. Assign a binary variable $x_s \in \{0,1\}$ for each element $s \in \mathcal{S}$. Let $v_s$ denote the value, and $c_{d,s}$ denote the volume along dimension $d$, of each element $s \in \mathcal{S}$. Also let $C_d$ denote the capacity of the knapsack along dimension $d$. Then the goals of multidimensional knapsack are expressed by the following optimization problem:

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & \sum_{s \in \mathcal{S}} x_s v_s  \\
\text{subject to} \;\; & \sum_{s \in \mathcal{S}} c_{d,s} x_s \leq C_d, \;\; \forall \;\; 1 \leq d \leq n  \\
& x_s \in \{0,1\}, \;\; \forall \;\; s \in \mathcal{S}.
\end{split}
\end{equation}
$$

While this problem can be certainly solved using the techniques developed in the last section on MILP, ```ortools``` provides a specialized solver called ```KnapsackSolver``` to solve this problem. The reader can find more details about using the solver on the documentation page. One thing to note is that ```KnapsackSolver``` only accepts **non-negative integer values** for values, volumes and capacities.

We demonstrate how to use the solver using a simple example. But let us first import the python wrapper ```pywrapknapsack_solver``` for the underlying C++ solver using the following Python code.

```python
from ortools.algorithms import pywrapknapsack_solver
```

***
### Example 1.3.1
Consider an instance of multidimensional knapsack in 2 dimensions ($d = 2$), where $\mathcal{S} = \{a, b, c, d, e\}$, and the knapsack capacities are $C_1 = 10, C_2 = 15$. Let the values of the objects be given by the following table:

|  $s$ | $v_s$ |
|------|-------|
|  $a$ |  $2$  |
|  $b$ |  $10$ |
|  $c$ |  $5$  |
|  $d$ |  $4$  |
|  $e$ |  $3$  |

Let the volumes of the objects be given by the following table:

|  $s$ | $c_{1,s}$ | $c_{2,s}$ |
|------|-------|-------|
|  $a$ |  $1$  |  $3$  |
|  $b$ |  $6$  |  $6$  |
|  $c$ |  $3$  |  $8$  |
|  $d$ |  $2$  |  $1$  |
|  $e$ |  $5$  |  $4$  |

The problem can then be solved using ```ortools``` by following the steps as shown below.

**Declare the values, volumes, and capacities**

The ```KnapsackSolver``` accepts the data to be in a certain format. The values should be a list of the same length as the number of objects, while the capacities should be a list of length equal to the number of dimensions. The volumes of the objects should be a list of lists. The outer list need to have the same length as the number of dimensions, while each inner list must have the same length as the number of objects.

```python
# Store the name of elements (this is not needed for the solver, but useful to display results)
objects = ['a', 'b', 'c', 'd', 'e']

# Declare the values, volumes and capacities
values = [2, 10, 5, 4, 3]
volumes = [[1, 6, 3, 2, 5], [3, 6, 8, 1, 4]]
capacities = [10, 15]
```

**Create an instance of ```KnapsackSolver```**

The next step is to create an instance of ```KnapsackSolver```. It is important to use ```KNAPSACK_MULTIDIMENSION_BRANCH_AND_BOUND_SOLVER``` as shown below. Other options include ```KNAPSACK_DYNAMIC_PROGRAMMING_SOLVER```, but it can only solve 1 dimensional knapsacks.

```python
# Create the solver, name it Example1
solver = pywrapknapsack_solver.KnapsackSolver(
    pywrapknapsack_solver.KnapsackSolver.KNAPSACK_MULTIDIMENSION_BRANCH_AND_BOUND_SOLVER,
    'Example1'
)
```

**Initialize the solver with the data**

The next step feeds the problem data into the solver.

```python
# Initialize the solver
solver.Init(values, volumes, capacities)
```

**Solve the problem**

```python
# Solve the problem
computed_value = solver.Solve()
```

**Display the results**

We can display the results as follows.

```python
# Display results
packed_items = [objects[x] for x in range(0, len(objects)) if solver.BestSolutionContains(x)]
packed_volumes = [
    [volumes[0][x] for x in range(0, len(objects)) if solver.BestSolutionContains(x)],
    [volumes[1][x] for x in range(0, len(objects)) if solver.BestSolutionContains(x)]
]
total_volumes = [sum(packed_volumes[0]), sum(packed_volumes[1])]

print("The maximum possible knapsack value is", computed_value)
print("Packed items: ", packed_items)
print("Total volumes: ", total_volumes)
```

    The maximum possible knapsack value is 16
    Packed items:  ['a', 'b', 'd']
    Total volumes:  [9, 10]

Here is the full Python code in one place.

```python
from ortools.algorithms import pywrapknapsack_solver

def multiknapsack(objects, values, volumes, capacities, name="multiknapsack"):

    # Create the solver, name it Example1
    solver = pywrapknapsack_solver.KnapsackSolver(
        pywrapknapsack_solver.KnapsackSolver.KNAPSACK_MULTIDIMENSION_BRANCH_AND_BOUND_SOLVER,
        name
    )

    # Initialize the solver
    solver.Init(values, volumes, capacities)

    # Solve the problem
    computed_value = solver.Solve()

    # Display results
    packed_items = [objects[x] for x in range(0, len(objects)) if solver.BestSolutionContains(x)]
    packed_volumes = [
        [volumes[0][x] for x in range(0, len(objects)) if solver.BestSolutionContains(x)],
        [volumes[1][x] for x in range(0, len(objects)) if solver.BestSolutionContains(x)]
    ]
    total_volumes = [sum(packed_volumes[0]), sum(packed_volumes[1])]

    print("The maximum possible knapsack value is", computed_value)
    print("Packed items: ", packed_items)
    print("Total volumes: ", total_volumes)

if __name__ == '__main__':

    # Store the name of elements (this is not needed for the solver, but useful to display results)
    objects = ['a', 'b', 'c', 'd', 'e']

    # Declare the values, volumes and capacities
    values = [2, 10, 5, 4, 3]
    volumes = [[1, 6, 3, 2, 5], [3, 6, 8, 1, 4]]
    capacities = [10, 15]

    # Solve
    multiknapsack(objects=objects, values=values, volumes=volumes, capacities=capacities, name="Example1")
```

    The maximum possible knapsack value is 16
    Packed items:  ['a', 'b', 'd']
    Total volumes:  [9, 10]

***
### Exercise 1.3.2
Consider the 1 dimensional knapsack problem with the following data.

```python
# Store the name of elements
objects = ['a', 'b', 'c', 'd', 'e']

# Declare the values, volumes and capacities
values = [2, 10, 5, 4, 3]
volumes = [[1, 6, 3, 2, 5]]
capacities = [10]
```

Solve the problem in three different ways:
- Using ```pywrapknapsack_solver.KnapsackSolver.KNAPSACK_MULTIDIMENSION_BRANCH_AND_BOUND_SOLVER```.
- Using ```pywrapknapsack_solver.KnapsackSolver.KNAPSACK_DYNAMIC_PROGRAMMING_SOLVER```.
- Using ```pywraplp.Solver.CBC_MIXED_INTEGER_PROGRAMMING```.

Time the different solvers.

```python
# Write your code here
```

***
## 1.4 Constraint programming
**Constraint programming (CP)** or **constraint optimization** refers to the task of finding feasible solutions to a set of arbitrary constraints, and such problems arise in many science and engineering applications. Thus CP is distinctly different from optimization problems; in fact in most cases, a CP may not even have an objective function, and the goal is to simply narrow down a large set of possible solutions to a more manageable subset by adding constraints to the problem. In fact, CP may arise as a subproblem in the solution process of an optimization problem. It should be noted however that any optimization problem can be solved this way by simply checking the objective function value at all the feasible solutions, and choosing the one that is the best. However this may be highly inefficient and hence is not recommended in most cases.

```ortools``` provides two libraries for solving CP problems:
- ```CP-SAT``` solver (SAT stands for **satisfiability**)
- ```original CP``` solver.

The recommended CP solver from Google is the ```CP-SAT``` solver, as it is much faster than the ```original CP``` solver, and we will strictly focus on the former in this lecture. More information on the two solvers, and some solved examples using each of them can be found by starting on the documentation page of the solvers. We will demonstrate the usage and syntax for ```CP-SAT``` using some examples. Most of the examples that we have chosen to illustrate are slight variants of the examples provided by ```ortools```, so that the reader can find more extensive discussion of these problems from online resources. This reference page also contains extensive documentation.

It should be noted that the ```CP-SAT``` solver only works on integer data. However in most cases CP problems with non-integer data can be converted to CP problems with integer data using the techniques described for example here.

The python wrappers ```cp_model``` and ```pywrapcp``` provide access to the underlying C++ solver for the ```CP-SAT``` solver and the ```original CP``` solver respectively. Let us import them, although we will not be using ```pywrapcp```.

```python
from ortools.sat.python import cp_model
from ortools.constraint_solver import pywrapcp
```

***
### Exercise 1.4.1
It is very instructive to read through the code implementing the Python interface ```cp_model```, as described here:

https://github.com/google/or-tools/blob/master/ortools/sat/python/cp_model.py.

***
### Example 1.4.1
We work through the first example in detail to understand the basic syntax of ```CP-SAT```.

Consider the following feasibility problem:

$$
\begin{equation}
\begin{split}
\text{find} \;\; & x, y  \\
\text{subject to} \;\; & x \neq y  \\
& x + y \leq 4  \\
& 1 \leq 2x + y \leq 5  \\
& x, y \in \{0,1,2,3\}.
\end{split}
\end{equation}
$$

The steps to model this problem using ```CP-SAT``` and solve it are explained below.

#### Instantiate the solver
We need to create two objects - the ```model``` and the ```solver```, the first of which is used to model the problem, such as all the data and the constraints, while the second one solves the problem.

```python
# Create the model and solver
model = cp_model.CpModel()
solver = cp_model.CpSolver()
```

#### Create the variables
We then create the variables involved in the problem. Here we only need ```NewIntVar``` for the problem.

**Note: Many other kinds of variables are available. You can see them by browsing the list after typing ```model.``` and pressing ```tab```.**

```python
# Create the variables
num_values = 4
x = model.NewIntVar(0, num_values - 1, "x")
y = model.NewIntVar(0, num_values - 1, "y")
```

#### Create the constraints
The next step is to create the constraints of the problem.

```python
# Create the constraints

# Constraint 1: x != y
constraint1 = model.Add(x != y)

# Constraint 2: x + y <= 4
constraint2 = model.Add(x + y <= 4)

# Constraint 3: 1 <= 2x + y <= 5
constraint3 = model.AddLinearConstraint(terms=[(x, 2), (y, 1)], lb=1, ub=5)
```

#### Create the solution printer
The ```CP-SAT``` solver displays the results using a **solution printer**. The solution printer is a callback defined in a Python class, which we pass to the solver as shown below, and the callback is executed each time a new solution is found. It needs to be implemented as a class inherited from ```CpSolverSolutionCallback```. It is highly recommended that you check the code here. The method ```NewSolution``` must be implemented which gets called everytime the solver finds a new solution.

```python
# Create the SolutionPrinter class
class SolutionPrinter(cp_model.CpSolverSolutionCallback):

    """
    Print intermediate solutions.
    """

    def __init__(self, variables):
        self.__variables = variables
        self.__solution_count = 0

    def NewSolution(self):
        self.__solution_count += 1
        for v in self.__variables:
            print('%s = %i,' % (v, self.Value(v)), end = ' ')
        print()

    def SolutionCount(self):
        return self.__solution_count

# Create a solution printer
solution_printer = SolutionPrinter([x, y])
```

#### Call the solver
We can finally solve the problem by calling the solver. Here we will search for all solutions by using the method ```SearchForAllSolutions```.

```python
# Call the solver, verify solution and pront results
print("Solving the CP problem...\n")
print("Printing all solutions...")
status = solver.SearchForAllSolutions(model, solution_printer)
assert status == cp_model.FEASIBLE
print('\nNumber of solutions found: %i' % solution_printer.SolutionCount())
```

    Solving the CP problem...

    Printing all solutions...
    x = 1, y = 0,
    x = 2, y = 0,
    x = 2, y = 1,
    x = 0, y = 1,
    x = 0, y = 2,
    x = 0, y = 3,
    x = 1, y = 2,
    x = 1, y = 3,

    Number of solutions found: 8

***
### Example 1.4.2
This example illustrates how to implement ```AND``` and ```OR``` constraints. Consider the following feasibility problem:

$$
\begin{equation}
\begin{split}
\text{find} \;\; & x, y, z  \\
\text{subject to} \;\; & (x \neq y) \;\&\; (y \neq z) \;\&\; (z \neq x)  \\
& (x + y + z \leq 4) \text{ or } (1 \leq 2x + y \leq 5) \\
& x, y, z \in \{0,1,2,3\}.
\end{split}
\end{equation}
$$

The following Python code then solves this problem using **channeling constraints**, as described here.

```python
# Solution to Example 2

def cp_example2():

    ###############################################
    # Create the model and solver
    model = cp_model.CpModel()
    solver = cp_model.CpSolver()

    ###############################################
    # Create the variables
    num_values = 4
    x = model.NewIntVar(0, num_values - 1, "x")
    y = model.NewIntVar(0, num_values - 1, "y")
    z = model.NewIntVar(0, num_values - 1, "z")

    # Create boolean variable needed to implement the OR constraint
    b = model.NewBoolVar("b")

    ###############################################
    # Create the constraints
    #----------------------------------------------
    # Constraint 1: (x != y) & (y != z) & (z != x)
    model.AddAllDifferent([x, y, z])

    #----------------------------------------------
    # Constraint 2: (x + y + z <= 4) or (1 <= 2x + y <= 5)
    model.Add(x + y + z <= 4).OnlyEnforceIf(b)
    model.Add(x + y + z > 4).OnlyEnforceIf(b.Not())
    model.AddLinearConstraint(terms=[(x, 2), (y, 1)], lb=1, ub=5).OnlyEnforceIf(b.Not())

    ###############################################
    # Create a solution printer
    solution_printer = SolutionPrinter([x, y, z, b])

    # Call the solver, verify solution and print results
    print("Solving the CP problem...\n")
    print("Printing all solutions...")
    status = solver.SearchForAllSolutions(model, solution_printer)
    assert status == cp_model.FEASIBLE
    print('\nNumber of solutions found: %i' % solution_printer.SolutionCount())

if __name__ == "__main__":
    cp_example2()
```

    Solving the CP problem...

    Printing all solutions...
    x = 0, y = 2, z = 3, b = 0,
    x = 1, y = 2, z = 3, b = 0,
    x = 2, y = 0, z = 3, b = 0,
    x = 2, y = 1, z = 3, b = 0,
    x = 2, y = 1, z = 0, b = 1,
    x = 3, y = 1, z = 0, b = 1,
    x = 1, y = 3, z = 2, b = 0,
    x = 0, y = 3, z = 2, b = 0,
    x = 0, y = 3, z = 1, b = 1,
    x = 1, y = 3, z = 0, b = 1,
    x = 1, y = 2, z = 0, b = 1,
    x = 0, y = 1, z = 2, b = 1,
    x = 0, y = 2, z = 1, b = 1,
    x = 0, y = 1, z = 3, b = 1,
    x = 1, y = 0, z = 3, b = 1,
    x = 1, y = 0, z = 2, b = 1,
    x = 2, y = 0, z = 1, b = 1,
    x = 3, y = 0, z = 1, b = 1,

    Number of solutions found: 18

***
### Example 1.4.3: SAT problems with constraints
Find a solution to the following **conjunctive normal form** (CNF) involving binary $\{0,1\}$ variables:

$$
(x_1 \lor x_2 \lor x_4) \land (\neg x_3 \lor x_5 \lor x_4) \land (x_2 \lor \neg x_4 \lor x_6) \land (x_1 \lor x_4 \lor x_5)
$$

subject to the additional constraint that

$$
x_2 \implies (x_5 \lor x_3) \land x_6.
$$

This is a specific instance of a 3-SAT problem with constraints. To solve this problem we need to use **reified constraints**. The Python code is given below.

```python
# Solution to Example 3

def cp_example3():

    ###############################################
    # Create the model and solver
    model = cp_model.CpModel()
    solver = cp_model.CpSolver()

    ###############################################
    # Create the boolean variables
    x1 = model.NewBoolVar("x1")
    x2 = model.NewBoolVar("x2")
    x3 = model.NewBoolVar("x3")
    x4 = model.NewBoolVar("x4")
    x5 = model.NewBoolVar("x5")
    x6 = model.NewBoolVar("x6")

    ###############################################
    # Create the constraints
    #----------------------------------------------
    # Constraint 1: 3-SAT clause
    model.AddBoolOr([x1, x2, x4])
    model.AddBoolOr([x3.Not(), x5, x4])
    model.AddBoolOr([x2, x4.Not(), x6])
    model.AddBoolOr([x1, x4, x5])

    #----------------------------------------------
    # Constraint 2: x2 => (x5 OR x3) & x6

    # Create extra boolean variables to implement constraints
    y1 = model.NewBoolVar("y1")
    y2 = model.NewBoolVar("y2")

    model.AddBoolOr([x5, x3]).OnlyEnforceIf(y1)
    model.AddBoolAnd([x5.Not(), x3.Not()]).OnlyEnforceIf(y1.Not())
    model.AddBoolAnd([y1, x6]).OnlyEnforceIf(y2)
    model.AddBoolOr([y1.Not(), x6.Not()]).OnlyEnforceIf(y2.Not())
    model.AddImplication(x2, y2)

    """
    #---------------DIFFERENT WAY------------------
    # Constraint 2: x2 => (x5 OR x3) & x6

    # Create extra boolean variables to implement constraints
    y1 = model.NewBoolVar("y1")

    model.AddBoolOr([x5, x3]).OnlyEnforceIf(y1)
    model.AddBoolAnd([x5.Not(), x3.Not()]).OnlyEnforceIf(y1.Not())
    model.AddImplication(x2, y1)
    model.AddImplication(x2, x6)
    """

    ###############################################
    # Create a solution printer
    solution_printer = SolutionPrinter([x1, x2, x3, x4, x5, x6])

    # Call the solver, verify solution and pront results
    print("Solving the CP problem...\n")
    print("Printing all solutions...")
    status = solver.SearchForAllSolutions(model, solution_printer)
    assert status == cp_model.FEASIBLE
    print('\nNumber of solutions found: %i' % solution_printer.SolutionCount())

if __name__ == "__main__":
    cp_example3()
```

    Solving the CP problem...

    Printing all solutions...
    x1 = 0, x2 = 0, x3 = 0, x4 = 1, x5 = 0, x6 = 1,
    x1 = 1, x2 = 0, x3 = 0, x4 = 1, x5 = 0, x6 = 1,
    x1 = 1, x2 = 0, x3 = 1, x4 = 1, x5 = 0, x6 = 1,
    x1 = 0, x2 = 0, x3 = 1, x4 = 1, x5 = 0, x6 = 1,
    x1 = 0, x2 = 0, x3 = 1, x4 = 1, x5 = 1, x6 = 1,
    x1 = 1, x2 = 0, x3 = 1, x4 = 1, x5 = 1, x6 = 1,
    x1 = 1, x2 = 0, x3 = 0, x4 = 1, x5 = 1, x6 = 1,
    x1 = 0, x2 = 0, x3 = 0, x4 = 1, x5 = 1, x6 = 1,
    x1 = 0, x2 = 1, x3 = 0, x4 = 1, x5 = 1, x6 = 1,
    x1 = 1, x2 = 1, x3 = 0, x4 = 1, x5 = 1, x6 = 1,
    x1 = 1, x2 = 1, x3 = 1, x4 = 1, x5 = 1, x6 = 1,
    x1 = 0, x2 = 1, x3 = 1, x4 = 1, x5 = 1, x6 = 1,
    x1 = 0, x2 = 1, x3 = 1, x4 = 1, x5 = 0, x6 = 1,
    x1 = 1, x2 = 1, x3 = 1, x4 = 1, x5 = 0, x6 = 1,
    x1 = 1, x2 = 0, x3 = 1, x4 = 0, x5 = 1, x6 = 1,
    x1 = 1, x2 = 0, x3 = 0, x4 = 0, x5 = 1, x6 = 1,
    x1 = 1, x2 = 0, x3 = 0, x4 = 0, x5 = 0, x6 = 1,
    x1 = 1, x2 = 1, x3 = 0, x4 = 0, x5 = 1, x6 = 1,
    x1 = 0, x2 = 1, x3 = 0, x4 = 0, x5 = 1, x6 = 1,
    x1 = 0, x2 = 1, x3 = 1, x4 = 0, x5 = 1, x6 = 1,
    x1 = 1, x2 = 1, x3 = 1, x4 = 0, x5 = 1, x6 = 1,
    x1 = 1, x2 = 0, x3 = 1, x4 = 0, x5 = 1, x6 = 0,
    x1 = 1, x2 = 0, x3 = 0, x4 = 0, x5 = 1, x6 = 0,
    x1 = 1, x2 = 0, x3 = 0, x4 = 0, x5 = 0, x6 = 0,

    Number of solutions found: 24

***
### Example 1.4.4: Integer optimization
CP can also be used to solve integer optimization problems in many cases. Consider the ILP:

$$
\begin{equation}
\begin{split}
\text{maximize} \;\; & x_1 + 2 x_2 - 3 x_3 + x_4  \\
\text{subject to} \;\; & 3 x_2 + x_4 + x_5 \leq 10 \\
& x_1 + x_3 + x_4 \leq 15 \\
& x_1, x_2, x_3 \in \{1,2,3,4\} \\
& x_4, x_5 \in \{0,1,2,3,4\}.
\end{split}
\end{equation}
$$

Here is the Python code that solves this problem using the ```CP-SAT``` solver.

```python
# Solution to Example 4

def cp_example4():

    # Create the model and solver
    model = cp_model.CpModel()
    solver = cp_model.CpSolver()

    # Create the variables
    x1 = model.NewIntVar(1, 4, "x1")
    x2 = model.NewIntVar(1, 4, "x2")
    x3 = model.NewIntVar(1, 4, "x3")
    x4 = model.NewIntVar(0, 4, "x4")
    x5 = model.NewIntVar(0, 4, "x5")

    # Create the constraints

    # Constraint 1: 3 * x2 + x4 + x5 <= 10
    model.AddLinearConstraint(terms=[(x2, 3), (x4, 1), (x5, 1)], lb=0, ub=10)

    # Constraint 2: x1 + x3 + x4 <= 15
    model.AddSumConstraint([x1, x3, x4], lb=0, ub=15)

    # Create the objective: x1 + 2 * x2 - 3 * x3 + x4
    model.Maximize(x1 + 2 * x2 - 3 * x3 + x4)

    # Call the solver
    print("Solving the CP problem...\n")
    status = solver.Solve(model)

    # Verify solution and print result
    assert status == cp_model.OPTIMAL
    print("Optimal objective value:", solver.ObjectiveValue())
    for var in [x1, x2, x3, x4, x5]:
        print(var.name(), "=", solver.Value(var))

if __name__ == "__main__":
    cp_example4()
```

    Solving the CP problem...

    Optimal objective value: 9.0
    x1 = 4
    x2 = 2
    x3 = 1
    x4 = 4
    x5 = 0

***
### Exercise 1.4.2: N-queens problem
Study the N-queen's problem as described on the documentation page. The problem is solved there using the ```original CP solver```. Solve it using ```CP-SAT``` solver.

```python
# Write your code here
```

***
## 1.5 Scheduling problems
Many optimization problems involving assigning resources to perform a set of specific tasks, and at different times, frequently arise in the manufacturing industries, as well in the transportation and delivery sector. These problems are broadly classified under the umbrella of **scheduling problems**. Typically, the goal of such problems is to find a schedule that minimizes the total amount of time (or cost) required to complete all the tasks.

The ```CP-SAT``` solver is capable of solving many such problems. Two specific problems that is somewhat widely applciable are:
- The job shop problem.
- The employee scheduling problem.

The documentation of ```ortools``` guides you on how to solve these problems using the ```original CP solver```. In this tutorial, we will cover how to solve the **job shop problem** using ```CP-SAT```. You will be asked to do the same for the **employee scheduling problem** as an exercise. It is good to know both these problems, as they are extremely common.

***
### Example 1.5.1: The job shop problem
Let us first describe the basic setup of the job shop problem. We have a finite set of jobs $J_1, \dots, J_N$, and each job has associated to it a finite set of tasks. So for the $i^{\text{th}}$ job $J_i$, we have the tasks $T_i = \{t_{i,1}, \dots, t_{i,m_i}\}$, and all the $m_i$ need not be the same, for all $1 \leq i \leq N$. We also have a finite set $P$ of processors on which the tasks can be executed, $P = \{p_1,\dots,p_M\}$. Next for each $i$, $1 \leq i \leq N$, we have a map $f_i: T_i \rightarrow P$, from the set of tasks to the set of processors and another function $g_i: T_i \rightarrow \mathbb{N}$, which signifies the amount of time taken by each task to complete.

The basic task in the job shop scheduling problem is to create a plan of execution of all the tasks on the given processors, subject to the following constraints:
- For each job $J_i$, if $1 \leq j_1 < j_2 \leq m_i$, then task $t_{i,j_2}$ can only start after task $t_{i,j_1}$ has finished completely.
- Every task $t_{i,j}$ must necessarily execute on the processor $f_i(t_{i,j})$.
- Only one task can be executed on a processor at any given time.

The goal then is to minimize the total time of completion for all the jobs.

**Note: In general the times taken by the tasks need not be integers, but we need this to be able to solve this problem using ```CP-SAT```.**

In order to solve this problem, let us assign a non-negative integer variable $x_{i,j}$ that denotes the start time for task $t_{i,j}$, for all $i,j$. We will assume that each task has been assigned to the correct processor, and so there will not be a need to implement this constraint explicitly. The goals of the job shop scheduling problem can be expressed as the following combinatorial **minimax** problem:

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & \max \{x_{i,j} + g_i(t_{i,j}) : 1 \leq i \leq N, 1 \leq j \leq m_i \}  \\
\text{subject to} \;\; & x_{i,j} + g_i(t_{i,j}) \leq x_{i,j+1}, \; \forall \; 1 \leq i \leq N, 1 \leq j \leq m_i - 1 \\
& \left( x_{i_1,j_1} + g_{i_1}(t_{i_1,j_1}) \leq x_{i_2,j_2} \right) \; \text{or} \; \left( x_{i_2,j_2} + g_{i_2}(t_{i_2,j_2}) \leq x_{i_1,j_1} \right), \; \forall \; i_1, i_2, j_1, j_2, \text{ such that } \; f_{i_1}(t_{i_1,j_1}) = f_{i_2}(t_{i_2,j_2}) \\
& x_{i,j} \in \mathbb{N}, \; \forall \; 1 \leq i \leq N, 1 \leq j \leq m_i.
\end{split}
\end{equation}
$$

We take the specific instance of the job shop scheduling problem described here. The following Python code then solves this problem using ```CP-SAT```.

```python
# Solution to the job shop problem

def job_shop_cpsat(machines, processing_times):

    """
    Machines and processing times need to have the same shape
    """

    ###############################################
    # Get number of jobs
    num_jobs = len(machines)

    # Get processor ids and number of different processors
    procs = []
    for job in machines:
        for task_proc in job:
            if task_proc not in procs:
                procs.append(task_proc)

    procs.sort()
    num_procs = len(procs)

    ###############################################
    # Create the model and solver
    model = cp_model.CpModel()
    solver = cp_model.CpSolver()

    ###############################################
    # Get an upper bound on maximum time needed to solve the problem
    ub = 0
    for job in processing_times:
        for task_time in job:
            ub += task_time

    ###############################################
    # Create start, end, and interval variables (one for each task)
    variables_start = [[] for _ in range(num_jobs)]
    variables_end = [[] for _ in range(num_jobs)]
    variables_interval = [[] for _ in range(num_jobs)]

    for i, job in enumerate(processing_times):
        for j, task_time in enumerate(job):

            start = model.NewIntVar(0, ub, "x" + str(i) + str(j))
            end = model.NewIntVar(0, ub, "y" + str(i) + str(j))
            interval = model.NewIntervalVar(start, task_time, end, "i" + str(i) + str(j))

            variables_start[i].append(start)
            variables_end[i].append(end)
            variables_interval[i].append(interval)

    # Create a list of interval variables by processors
    variables_proc = [[] for _ in range(num_procs)]
    for i, job in enumerate(machines):
        for j, task_proc in enumerate(job):
            variables_proc[task_proc].append(variables_interval[i][j])

    ###############################################
    # Create the constraints

    # Constraint 1 (no task for a job can start before all preceding tasks of the same job finish)
    for i, job in enumerate(variables_start):
        num_tasks = len(job)

        for j in range(1, num_tasks):
            model.Add(variables_start[i][j] >= variables_end[i][j - 1])

    # Constraint 2 (each processor runs one task at any given time)
    for interval_variables in variables_proc:
        model.AddNoOverlap(interval_variables)

    ###############################################
    # Create the objective
    obj = model.NewIntVar(0, ub, "obj")
    model.AddMaxEquality(obj, [var_end for job in variables_end for var_end in job])
    model.Minimize(obj)

    ###############################################
    # Call the solver
    print("Solving the CP problem...\n")
    status = solver.Solve(model)

    ###############################################
    # Verify solution and print schedule
    assert status == cp_model.OPTIMAL
    print("Time needed to finish all jobs in the optimal schedule:", solver.ObjectiveValue())
    print("\n")

    print("Job schedule:")
    for i, job in enumerate(variables_start):
        print("\nJob " + str(i))
        for j, _ in enumerate(job):
            print(
                "Task " + str(j) + ": start =",
                solver.Value(variables_start[i][j]),
                ", end =",
                solver.Value(variables_end[i][j]),
                ", proc =",
                machines[i][j]
            )

if __name__ == "__main__":

    # Create data for the problem (machines and processing times need to have the same shape)
    machines = [[0, 1, 2], [0, 2, 1], [1, 2]]
    processing_times = [[3, 2, 2], [2, 1, 4], [4, 3]]

    # Solve the problem
    job_shop_cpsat(machines=machines, processing_times=processing_times)
```

    Solving the CP problem...

    Time needed to finish all jobs in the optimal schedule: 11.0

    Job schedule:

    Job 0
    Task 0: start = 0 , end = 3 , proc = 0
    Task 1: start = 4 , end = 6 , proc = 1
    Task 2: start = 6 , end = 8 , proc = 2

    Job 1
    Task 0: start = 3 , end = 5 , proc = 0
    Task 1: start = 5 , end = 6 , proc = 2
    Task 2: start = 6 , end = 10 , proc = 1

    Job 2
    Task 0: start = 0 , end = 4 , proc = 1
    Task 1: start = 8 , end = 11 , proc = 2

***
### Exercise 1.5.1: Employee scheduling
Study the employee scheduling problem as described here. Solve it using the ```CP-SAT``` solver.

```python
# Write your code here
```

***
## 1.6 Graph algorithms
Many problems in combinatorial optimization arise from graph theory; some examples are network flow problems, finding hamiltonian paths, finding shortest paths, and the traveling salesman problem, just to name a few. ```ortools``` provides two libraries - the ```algorithms``` library, and the ```graph``` library, that solves a great majority of these problems. The reader is encouraged to look up these libraries:
- ```algorithms```: https://developers.google.com/optimization/reference/algorithms/.
- ```graph```: https://developers.google.com/optimization/reference/graph/.

In this tutorial we will look at the **network flow** class of problems. Generally speaking network flow problems involve transporting goods or materials across a network. The network could for example consist of cities, and roads or railways connecting them. In that case, the network can be represented as a graph, with the cities being represented by **vertices** and road / railway connection between cities being represented by **edges** or **arcs**. Each arc also comes with a capacity constraint representing the maximum amount of good that can be transported across it in unit time.

We will look at two flow problems that arise quite frequently - the **maximum flow** problem, and the **minimum cost flow** problem, and will solve them using ```ortools```. More information on network flows and how to solve them using ```ortools``` can be found here.

But first we import the graph library in Python.

```python
from ortools.graph import pywrapgraph
```

***
### 1.6.1 Maximum flow problem
The maximum flow problem is described by a **directed** graph $G(V,E)$. An edge $e := (u,v), \; e \in E$, will denote a directed edge starting at the vertex $u \in V$ and ending at the vertex $v \in V$, and in addition each edge also has a capacity constraint, which are only required to be positive for the maximum flow problem, but in addition we will also need them to be postive integers for us to be able to solve them using ```ortools``` - thus we will assume that this is the case going forward. In addition, there are two special vertices in the graph called the **source** and the **sink**, which are denoted $s$ and $t$ respectively. A **valid flow** is an assignment of non-negative integers to the directed edges that satisfy the following constraints:
- For every edge, the assigned flow does not exceed the capacity of the edge.
- At every vertex, except $s$ and $t$, the net flow of the incident edges, i.e. the sum of flows of incoming edges minus the sum of flows of outgoing edges, must be zero.

The objective of the maximum flow problem is to find a valid flow assignment that maximizes the net outflow from $s$, or alternatively the net inflow into $t$. Both of them are equivalent, and a proof of this fact can be found in any introductory graph theory textbook.

Let us take a specific problem - in fact we will use the example problem described in the documentation page.

The data for the problem is given by the list of tuples: ```(start_node, end_node, capacity)```. The first two entities in each tuple denote the start and end vertices respectively of a directed edge of a graph, and the third entity denotes the capacity.

```python
# Data for the problem
data = [(0, 1, 20), (0, 2, 30), (0, 3, 10), (1, 2, 40), (1, 4, 30), (2, 3, 10), (2, 4, 20), (3, 2, 5), (3, 4, 20)]

# Declare source and sink
s = 0
t = 4
```

```ortools``` provides the method ```pywrapgraph.SimpleMaxFlow``` to solve this problem. The following Python code illustrates how to use it.

```python
# Create lists for start, end, and capacities
start_nodes = []
end_nodes = []
capacities = []

for item in data:
    start_nodes.append(item[0])
    end_nodes.append(item[1])
    capacities.append(item[2])

# Instantiate a SimpleMaxFlow solver
max_flow = pywrapgraph.SimpleMaxFlow()

# Add each arc
for i in range(0, len(start_nodes)):
    max_flow.AddArcWithCapacity(start_nodes[i], end_nodes[i], capacities[i])

# Solve the maximum flow problem and check for optimality
status = max_flow.Solve(s, t)
assert status == max_flow.OPTIMAL

# Display results
print('Max flow:', max_flow.OptimalFlow())
print('')
print('  Arc    Flow / Capacity')
for i in range(max_flow.NumArcs()):
    print('%1s -> %1s   %3s  / %3s' % (max_flow.Tail(i), max_flow.Head(i), max_flow.Flow(i), max_flow.Capacity(i)))

print('Source side min-cut:', max_flow.GetSourceSideMinCut())
print('Sink side min-cut:', max_flow.GetSinkSideMinCut())
```

    Max flow: 60

      Arc    Flow / Capacity
    0 -> 1    20  /  20
    0 -> 2    30  /  30
    0 -> 3    10  /  10
    1 -> 2     0  /  40
    1 -> 4    20  /  30
    2 -> 3    10  /  10
    2 -> 4    20  /  20
    3 -> 2     0  /   5
    3 -> 4    20  /  20
    Source side min-cut: [0]
    Sink side min-cut: [4, 1]

***
### Exercise 1.6.1.1
- Run some simple experiments by choosing different nodes as $s$ and $t$ in the above example.
- Change the problem data as you wish and find the maximum flow solution.

***
### 1.6.2 Minimum cost flow problem
The minimum cost flow problem is an optimization problem that is encountered very frequently in logistics planning, and supply chain management. The basic idea is that there is a network, just like in the maximum flow problem, and there are some nodes where resources are produced, while there are other nodes where resources are consumed. The goal is to transport the resources from the supply nodes to the demand nodes at the minimum cost.

The problem is closely related to the maximum flow problem, but there are some key differences. We again model the network using a **directed** graph $G(V,E)$. An edge (arc) $e := (u,v), \; e \in E$, denotes a directed edge starting at the vertex $u \in V$ and ending at the vertex $v \in V$, and as before has a capacity which is a postive integer (due to ```ortools``` requirement). In addition, there are special vertices in the graph called **supply** and **demand** nodes, where resources (flow) are either created or consumed respectively. In fact we will model all vertices as **supply** nodes, with the convention that a node is a supply node if and only if it has a positive integral supply of resources, it is a demand node if and only if it has a negative integral supply of resources (i.e. positive integral demand for resources), and a normal vertex if and only if it has exactly zero supply of resources. The supplies must be integers. Another difference as compared to the maximum flow problem is that there is also a unit cost (again non-negative integers) associated with transporting resources across each arc, and so if the flow value through an arc is $f$, and the unit cost for the arc is $c$, then the total cost incurred for that arc is $cf$.

A **valid flow** is an assignment of non-negative integers to the directed edges that satisfy the following constraints:
- For every edge, the assigned flow does not exceed the capacity of the edge.
- At every vertex that is not a supply or demand node, the net flow of the incident edges, i.e. the sum of flows of outgoing edges minus the sum of flows of incoming edges, must be zero.
- At a supply node, the net flow of the incident edges should equal the supply.
- At a demand node, the net flow of the incident edges should equal the negative of the demand.

It should be clear from the description above that the only way this could possibly work is if the supply at the **supply** nodes equal the demand at the **demand** nodes, i.e. in our language above the total sum of the supplies at all the vertices must be exactly zero!

The goal of the minimum cost flow problem is then to design a valid flow which achieves the minimum cost.

We demonstrate this using the specific example described in the documentation page.

The data for the problem is given by the list of tuples: ```(start_node, end_node, capacity, unit_cost)```. The first two entities in each tuple denote the start and end vertices respectively of a directed edge of a graph, the third entity denotes its capacity, and the last element of the tuple denotes the cost of unit flow through the edge. The supplies for each node of the graph is also input.

```python
# Data for the problem
data = [
    (0, 1, 15, 4),
    (0, 2, 8, 4),
    (1, 2, 20, 2),
    (1, 3, 4, 2),
    (1, 4, 10, 6),
    (2, 3, 15, 1),
    (2, 4, 4, 3),
    (3, 4, 20, 2),
    (4, 2, 5, 3)
]

# Define an array of supplies at each node
supplies = [20, 0, 0, -5, -15]
```

```ortools``` provides the method ```pywrapgraph.SimpleMinCostFlow``` to solve this problem. The following Python code illustrates how to use it.

```python
# Create lists for start, end, and capacities
start_nodes = []
end_nodes = []
capacities = []
unit_costs = []

for item in data:
    start_nodes.append(item[0])
    end_nodes.append(item[1])
    capacities.append(item[2])
    unit_costs.append(item[3])

# Instantiate a SimpleMinCostFlow solver
min_cost_flow = pywrapgraph.SimpleMinCostFlow()

# Add each arc.
for i in range(0, len(start_nodes)):
    min_cost_flow.AddArcWithCapacityAndUnitCost(start_nodes[i], end_nodes[i], capacities[i], unit_costs[i])

# Add node supplies.
for i in range(0, len(supplies)):
    min_cost_flow.SetNodeSupply(i, supplies[i])

# Solve the maximum flow problem and check for optimality
status = min_cost_flow.Solve()
assert status == min_cost_flow.OPTIMAL

# Display results
print('Minimum cost:', min_cost_flow.OptimalCost())
print('')
print('  Arc    Flow / Capacity  Cost')
for i in range(min_cost_flow.NumArcs()):
    cost = min_cost_flow.Flow(i) * min_cost_flow.UnitCost(i)
    print(
        '%1s -> %1s   %3s  / %3s       %3s' % (
            min_cost_flow.Tail(i),
            min_cost_flow.Head(i),
            min_cost_flow.Flow(i),
            min_cost_flow.Capacity(i),
            cost
        )
    )
```

    Minimum cost: 150

      Arc    Flow / Capacity  Cost
    0 -> 1    12  /  15        48
    0 -> 2     8  /   8        32
    1 -> 2     8  /  20        16
    1 -> 3     4  /   4         8
    1 -> 4     0  /  10         0
    2 -> 3    12  /  15        12
    2 -> 4     4  /   4        12
    3 -> 4    11  /  20        22
    4 -> 2     0  /   5         0

***
### Exercise 1.6.2: Assignment as a min cost flow, MIP and CP problem.
If you are interested, you should read up the sections that describe how to pose assignment as a min cost flow problem, as a MIP problem, and as a CP problem [link1, link2, link3].

***
### Exercise 1.6.3 (Bonus, Self-Study): Routing problems
In this tutorial we did not get the time to discuss routing problems. If you want to learn about them you can start on the documentation page.

```python

```


# 1. $\pi$-Model and $T$-Model equivalent equations

- Equivalent capacitive model for 4 plates

```python
# Libraries
from sympy import*
import numpy as np

# To define automatic printing mode
init_printing()
```

## $\pi$-Model:

```python
# Define the symbolic variables

# Capacitance
c_1, c_2, c_M = symbols('c_1 c_2 c_M', real = True)

# Angular frequency
w = symbols('w', real = True)
```

```python
# Common denominator
com_D = (1/(w*I))*(1/c_1 + 1/c_2 + 1/c_M)
```

```python
# T-model equivalent c'_1
Zp_1 = ((1/(w*I))**2)*((1/c_1)*(1/c_M))/com_D
simplify(Zp_1)
```

```python
# T-model equivalent c'_2
Zp_2 = ((1/(w*I))**2)*((1/c_2)*(1/c_M))/com_D
simplify(Zp_2)
```

```python
# T-model equivalent c'_M
Zp_M = ((1/(w*I))**2)*((1/c_1)*(1/c_2))/com_D
simplify(Zp_M)
```

## $T$-Model:

```python
# Define the symbolic variables

# Capacitance
cp_1, cp_2, cp_M = symbols('cp_1 cp_2 cp_M', real = True)
```

```python
# Common numerator
com_N = ((1/(w*I))**2)*((1/(cp_1*cp_M)) + (1/(cp_1*cp_2)) + (1/(cp_M*cp_2)))
```

```python
# Pi-model equivalent c_1
Z_1 = (com_N)/(1/(w*cp_2*I))
simplify(Z_1)
```

```python
# Pi-model equivalent c_2
Z_2 = (com_N)/(1/(w*cp_1*I))
simplify(Z_2)
```

```python
# Pi-model equivalent c_M
Z_M = (com_N)/(1/(w*cp_M*I))
simplify(Z_M)
```


# Linear Modelling - Least Squares

_Linear modelling_ is the process of learning a linear relationship between attributes and responces. Consider the following data set where $x_n$ denotes the nth olympic year and $t_n$ denotes the nth time of the mens 100m.

```python
%matplotlib inline

x = [1896, 1900, 1904, 1906, 1908, 1912, 1920, 1924, 1928, 1932, 1936, 1948, 1952, 1956, 1960, 1964,
     1968, 1972, 1976, 1980, 1984, 1988, 1992, 1996, 2000, 2004, 2008]
t = [12.00, 11.00, 11.00, 11.20, 10.80, 10.80, 10.80, 10.60, 10.80, 10.30, 10.30, 10.30, 10.40, 10.50,
     10.20, 10.00, 9.95, 10.14, 10.06, 10.25, 9.99, 9.92, 9.96, 9.84, 9.87, 9.85, 9.69]

import matplotlib.pyplot as plt
plt.plot(x, t, 'ro')
plt.show()
```

## Defining the model

We can clearly see a statistical dependence between year and time (note this is not clausal).

We will define are model as a function that maps the input attributes (olympic year) to the target values (winning time). Clearly this function will involves terms other than $x$ called _parameters_, these are the values that we learn from the data set. A function $f$ that maps input attributes $x_0, ..., x_n$ with parameters $w_0, ..., w_k$ is denoted as $f(x_0, ..., x_n; w_0, ... w_k)$.

To pick which model we are going to use, we make the assumtion that the data can be modelled linearly, thus the function will be of the form.

$$
t = f(x; w_0, w_1) = w_0 + w_1 x
$$

## Loss function

To find the _best_ model we must define what the _best_ means. In our case we a looking for the line which is as close to all data points as possible. A common way to measure this is the squared diffrence known as the squared loss function:

$$
\mathcal{L}_n(t_n, f(x_n; w_0, w_1)) = (t_n - f(x_n; w_0, w_1))^2
$$

The smaller the loss for year $n$, the closer the model at $x_n$ is to $t_n$. We need a low loss for all years, thus we find the average loss:

$$
\mathcal{L} = \frac{1}{N} \sum^{N}_{n = 1} L_n(t_n, f(x_n; w_0, w_1))
$$

Thus we will tune $w_0$ and $w_1$ such that the average loss is minimilized, expressed mathmatically:

$$
\operatorname*{argmin}_{w_0, w_1} = \frac{1}{N} \sum^N_{n=1} L_n(t_n, f(x_n; w_0, w_1))
$$

We have choosen this loss function because we can find values for $w_0$ and $w_1$ analytically however other loss functions could be used such as the absolute loss:

$$
\mathcal{L}_n = \left| t_n - f(x_n; w_0, w_1) \right|
$$

## Finding a solution

To find the $\operatorname*{argmin}_{w_0, w_1} = \frac{1}{N} \sum^N_{n=1} L_n(t_n, f(x_n; w_0, w_1))$ we can differentiate to find the point in which the gradient is zero (the minima of the function).

First we substiture the linear model into the expression for average loss:
$$
\begin{align}
\mathcal{L} &= \frac{1}{N} \sum^{N}_{n = 1} L_n(t_n, f(x_n; w_0, w_1)) \\
&= \frac{1}{N} \sum^{N}_{n = 1} (t_n - f(x_n; w_0, w_1))^2 \\
&= \frac{1}{N} \sum^{N}_{n = 1} (t_n - (w_0 x_n + w_1))^2 \\
&= \frac{1}{N} \sum^{N}_{n = 1} (w_1^2 x_n^2 + 2 w_1 x_n w_0 - 2 w_1 x_n t_n + w_0^2 - 2 w_0 t_n + t_n^2) \\
&= \frac{1}{N} \sum^{N}_{n = 1} (w_1^2 x_n^2 + 2 w_1 x_n (w_0 - t_n) + w_0^2 - 2 w_0 t_n + t_n^2) \\
\end{align}
$$

Differentiate with respect to $w_0$

$$
\begin{align}
\frac{1}{N} \sum^{N}_{n = 1} (w_0^2 + 2 w_1 x_n w_0 - 2 w_0 t_n) && \text{Remove terms not including $w_0$} \\
w_0^2 + 2 w_1 w_0 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) - 2 w_0 \frac{1}{N} \left( \sum^{N}_{n = 1} t_n \right) && \text{Rearrange} \\
\frac{\partial{\mathcal{L}}}{\partial{w_0}} = 2 w_0 + 2 w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) - \frac{2}{N} \left( \sum^{N}_{n = 1} t_n \right) && \text{Differentiate}
\end{align}
$$

Differentiate with respect to $w_1$

$$
\begin{align}
\frac{1}{N} \sum^{N}_{n = 1} (w_1^2 x_n^2 + 2 w_1 x_n w_0 - 2 w_1 x_n t_n)) && \text{Remove terms not including $w_1$} \\
w_1^2\frac{1}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) + 2 w_1 w_0 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) - 2 w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) && \text{Rearrange} \\
\frac{\partial{\mathcal{L}}}{\partial{w_1}} = 2 w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) + 2 w_0 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) - \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) && \text{Differentiate}
\end{align}
$$

Differentiate again, to check if the turning point is a minima
$$
\begin{align}
\frac{\partial^2{\mathcal{L}}}{\partial{w_0}} &= 2 \\
\frac{\partial^2{\mathcal{L}}}{\partial{w_1}} &= \frac{2}{N} \sum^N_{n=1}{x^2_n} \\
\end{align}
$$

Set $\frac{\partial{\mathcal{L}}}{\partial{w_0}}$ to $0$

$$
\begin{align}
2 w_0 + 2 w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) - \frac{2}{N} \left( \sum^{N}_{n = 1} t_n \right) &= 0 \\
2 w_0 &= \frac{2}{N} \left( \sum^{N}_{n = 1} t_n \right) - 2 w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) \\
w_0 &= \frac{1}{N} \left( \sum^{N}_{n = 1} t_n \right) - w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) \\
\widehat{w_0} &= \bar{t} - w_1 \bar{x} \\
\end{align}
$$

Subing in the new expression for $w_0$ into $\frac{\partial{\mathcal{L}}}{\partial{w_1}}$ we get:

$$
\begin{align}
\frac{\partial{\mathcal{L}}}{\partial{w_1}} &= 2 w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) + 2 \hat{w_0} \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) - \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) \\
&= 2 w_1 \frac{1}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) + 2 (\bar{t} - w_1 \bar{x}) \frac{1}{N} \left( \sum^{N}_{n = 1} x_n \right) - \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) \\
&= w_1 \frac{2}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) + \bar{t} \frac{2}{N} \left( \sum^{N}_{n = 1} x_n \right) - w_1 \bar{x} \frac{2}{N} \left( \sum^{N}_{n = 1} x_n \right) - \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right)
\end{align}
$$

Simplify using $\bar{x} = \frac{1}{N} \sum^{N}_{n = 1} x_n$

$$
\frac{\partial{\mathcal{L}}}{\partial{w_1}} = w_1 \frac{2}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) + 2 \bar{t} \bar{x} - 2 w_1 (\bar{x})^2 - \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right)
$$

Set $\frac{\partial{\mathcal{L}}}{\partial{w_1}}$ to $0$

$$
\begin{align}
w_1 \frac{2}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) + 2 \bar{t} \bar{x} - 2 w_1 (\bar{x})^2 - \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) &= 0 \\
w_1 \frac{2}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) - 2 w_1 (\bar{x})^2 &= \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) - 2 \bar{t} \bar{x} \\
w_1 \left[\frac{2}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) - 2 (\bar{x})^2 \right] &= \frac{2}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) - 2 \bar{t} \bar{x} \\
w_1 &= \frac{\frac{1}{N} \left( \sum^{N}_{n = 1} x_n t_n \right) - \bar{t} \bar{x}}{\frac{1}{N} \left( \sum^{N}_{n = 1} x_n^2 \right) - (\bar{x})^2} \\
\end{align}
$$

To simplify we will define a few more average quantitys, $\overline{x^2} = \frac{1}{N} \sum^{N}_{n = 1} x_n^2$ and $\overline{xt} = \frac{1}{N} \sum^{N}_{n = 1} x_n t_n$, thus we arrive at:

$$
\widehat{w_1} = \frac{\overline{xt} - \bar{x} \bar{t}}{\overline{x^2} - (\bar{x})^2}
$$

Thus all that we need to compute the best parameter values are:

$$
\begin{align}
\widehat{w_0} &= \bar{t} - w_1 \bar{x}\\
\widehat{w_1} &= \frac{\overline{xt} - \bar{x} \bar{t}}{\overline{x^2} - (\bar{x})^2}\\
\end{align}
$$

## Least squares fit

```python
import numpy as np

t_bar = np.average(t)
x_bar = np.average(x)
xt_bar = np.average([x[i] * t[i] for i in range(len(x))])
x2_bar = np.average(list(map(lambda xi: xi**2, x)))

w1 = (xt_bar - x_bar * t_bar) / (x2_bar - x_bar ** 2)
w0 = t_bar - w1 * x_bar

print("w0 = {}, w1 = {}".format(w0, w1))
```

    w0 = 36.41645590250286, w1 = -0.013330885710960602

```python
model_x = np.linspace(min(x), max(x), 100)
model_t = list(map(lambda xi: w0 + w1 * xi, model_x))

plt.plot(x, t, 'ro')
plt.plot(model_x, model_t)
plt.show()
```

## Extending to vectors

Clearly their are lots of attributes which affect mens 100m times. We can extend are model by using vectors, consider the following:

$$
\mathbf{X} = \begin{bmatrix}1 & x_1\\1 & x_2\\\vdots & \vdots\\1 & x_n\end{bmatrix}
\quad
\mathbf{t} = \begin{bmatrix}t_1\\t_2\\\vdots\\t_n\end{bmatrix}
\quad
\mathbf{w} = \begin{bmatrix}w_0\\w_1\end{bmatrix}
$$

The loss can be written as:

$$
\mathcal{L} = \frac{1}{N}(\mathbf{t}-\mathbf{Xw})^T (\mathbf{t}-\mathbf{Xw})
$$

Which we can see by expanding out:

$$
\begin{align}
\mathcal{L} &= \frac{1}{N} \times \left( \begin{bmatrix}t_1 - w_0 - w_1 x_1\\t_2 - w_0 - w_1 x_2\\ \vdots \\ t_n - w_0 - w_1 x_n \end{bmatrix} \right)^T \times \left( \begin{bmatrix}t_1 - w_0 - w_1 x_1\\t_2 - w_0 - w_1 x_2\\ \vdots \\ t_n - w_0 - w_1 x_n \end{bmatrix} \right)\\
&= \frac{1}{N} \left[ (t_1 - w_0 + w_1 x_1)^2 + ... + (t_n - w_0 + w_1 x_n)^2 \right] \\
&= \frac{1}{N} \sum^{N}_{n = 1} L_n(t_n, f(x_n; w_0, w_1))
\end{align}
$$

As before we differentiate with respect to $\mathbf{w}$ (the parameters in vector form), to do this we can use the following rules:

| $f(\mathbf{w})$ | $\frac{\partial f}{\partial \mathbf{w}}$ |
| :-------------: |:-------------:|
| $\mathbf{w}^T \mathbf{x}$ | $\mathbf{x}$ |
| $\mathbf{x}^T \mathbf{w}$ | $\mathbf{x}$ |
| $\mathbf{w}^T \mathbf{w}$ | $2 \mathbf{w}$ |
| $\mathbf{w}^T \mathbf{Cw}$ | $2 \mathbf{Cw}$ |

Thus we get the following:

$$
\frac{\partial \mathcal{L}}{\partial \mathbf{w}} = \frac{2}{N} \mathbf{X}^T \mathbf{Xw} - \frac{2}{N} \mathbf{X}^T \mathbf{t}
$$

To find $\widehat{\mathbf{w}}$ (the optimum parameters) we set $\frac{\partial \mathcal{L}}{\partial \mathbf{w}}$ and use inverses since division isnt defined for matricies.

$$
\begin{align}
\frac{2}{N} \mathbf{X}^T \mathbf{Xw} - \frac{2}{N} \mathbf{X}^T \mathbf{t} &= 0 \\
\frac{2}{N} \mathbf{X}^T \mathbf{Xw} &= \frac{2}{N} \mathbf{X}^T \mathbf{t} \\
\mathbf{X}^T \mathbf{Xw} &= \mathbf{X}^T \mathbf{t} \\
\mathbf{I} \mathbf{w} &= (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{t} \\
\widehat{\mathbf{w}} &= (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{t} \\
\end{align}
$$

Now we can define linear models with any number of parameters.

## Non-linear responces

So far the model takes the form $f(x; \mathbf{w}) = w_0 + w_1 x$, however by extending matrix $\mathbf{X}$ we get a polynomial function of any order:

$$
\mathbf{X} = \begin{bmatrix}
x_1^0 & x_1^1 & \cdots & x_1^K \\
x_2^0 & x_2^1 & \cdots & x_2^K \\
\vdots & \vdots & \ddots & \vdots \\
x_N^0 & x_N^1 & \cdots & x_N^K \\
\end{bmatrix}
$$

Once more we can extend this past polynomial function with $K$ functions of $x$, $h_k(X)$:

$$
\mathbf{X} = \begin{bmatrix}
h_1(x_1) & h_2(x_1) & \cdots & h_K(x_1) \\
h_1(x_2) & h_2(x_2) & \cdots & h_K(x_2) \\
\vdots & \vdots & \ddots & \vdots \\
h_1(x_N) & h_2(x_N) & \cdots & h_K(x_N) \\
\end{bmatrix}
$$

```python
from scipy.optimize import curve_fit

# Original linear model
def f1(x, w0, w1):
    return w0 + np.multiply(w1, x)

# Second order polinomial
def f2(x, w0, w1, w2):
    return w0 + np.multiply(w1, x) + np.multiply(w2, np.power(x, 2))

# 5-th order polinomial
def f3(x, w0, w1, w2, w3, w4, w5):
    return w0 + np.multiply(w1, x) + np.multiply(w2, np.power(x, 2)) + np.multiply(w3, np.power(x, 3)) \
         + np.multiply(w4, np.power(x, 4)) + np.multiply(w5, np.power(x, 5))

# With sin
def f4(x, w0, w1, w2, a, b):
    return w0 + np.multiply(w1, x) + np.multiply(w2, np.power(x, 2)) + np.multiply(w1, np.sin((x - a)/b))

functions = [
    (f1, 'r', "First order polynomial (linear)"),
    (f2, 'g', "Second order polynomial"),
    (f3, 'y', "5th order polynomial"),
    (f4, 'b', "Function with sin term")
]

for (f, color, title) in functions:
    popt, pcov = curve_fit(f, x, t)
    plt.plot(x, t, 'ro')
    plt.plot(model_x, f(model_x, *popt), color)
    plt.title(title)
    plt.show()
```

## Generalization and overfitting

The question becomes, what is the best model. We want a model that _generalize_ beyond the training data to make future predictions. Clearly the 5th order polynomial has a lower loss that lower order polynomials, however it doesnt make good future predictions since it pays to much attention to the training data, and is _overfitted_.

```python
plt.plot(x, t, 'ro')

# training range
popt, pcov = curve_fit(f3, x, t)
plt.plot(model_x, f3(model_x, *popt), 'y')

# prediction range
pred_x = np.linspace(max(x), max(x) + 40, 40)
popt, pcov = curve_fit(f3, x, t)
plt.plot(pred_x, f3(pred_x, *popt), 'y--')

plt.title("5th order polynomial")
plt.show()
```

Determining the optimum model complexity such that it generalizes well without overfitting is refered to as the bias-variance tradeoff and is very challenging.

## Validation data

A common way to overcome this problem is splitting the dataset into training data and validation data. Several models are trained with the training data, then the loss is computed for each against the validation data. Models that generalize well will have a lower loss against the validation data, where as models that are overfitted will have higher losses.

```python
training_x = x[:-10]
training_t = t[:-10]

validation_x = x[-10:]
validation_t = t[-10:]

for (f, color, title) in functions:
    popt, pcov = curve_fit(f, training_x, training_t)
    plt.plot(training_x, training_t, 'ro')
    plt.plot(validation_x, validation_t, 'rs')
    plt.plot(model_x, f(model_x, *popt), color)
    plt.title(title)
    plt.show()
```

## K-Fold cross validation

The loss for the validation data is sensitive to how we split the dataset, cross validation is a tecnique that allows us to make more efficent use of the data.

K-Fold cross validation splits the dataset into $K$ blocks, each block in turn is used as validation data and the other $K -1$ blocks as training data. The final loss value is the average of all $K$ loss values.

The extream case where $K = N$ is known as leave one out cross validation (LOOCV), which can be expressed as the following:
$$
\mathcal{L}^{CV} = \frac{1}{N} \sum_{n=1}^N (t_n - \widehat{\mathbf{w}}_{-n}^T x_n)^2
$$
Where $\widehat{\mathbf{w}}_{-n}^T$ is the estimate of the parameters without the $n$th datapoint.

## Regularized least squares

Both types of validation prevents models from becoming too complex, another solution is regularization. In general the higher the values of the parameters, the more complex the model becomes, thus instead of average squared loss, we can consider the regularized loss:

$$
\mathcal{L}' = \mathcal{L} + \lambda \mathbf{w}^T \mathbf{w}
$$

The term $\lambda \mathbf{w}^T \mathbf{w}$ penalises complexity with parameter $\lambda$ controlling the trade off between not fitting the data well and penalising complexity. To determine a value for $\lambda$ that gives the best predictive performance we can use cross validation.


```python
!pip install sympy tornado
```

# Symbolic Computation with Python

## Declaring Symbols, Printing Functions

```python
import sympy as sp
sp.init_printing()
```

```python
x=sp.Symbol('x')
y,z=sp.symbols('y,z')
f=sp.Function('f')
g=x**2 + y**2 + z**2
```

```python
print(g)
```

```python
g
```

```python
sp.pprint(g)
```

## Math Expressions and Evaluating at certain values

```python
h=x**2+2*x-5
h
```

```python
print(h.subs(x,2))

```

```python
print(h.subs(x,y**2.5))
print(h.subs(x,z**2))
```

### Simplifying, Expanding and Factorizing Expressions

```python
f=(x**2-x-6)/(x**2-3*x)
sp.simplify(f)
```

```python
f=(x+1)**3*(x+2)**2
sp.expand(f)
```

```python
f=3*x**4-36*x**3+99*x**2-6*x-144
sp.factor(f)
```

## Differentiating and Intergrating Functions

```python
y=(sp.sin(x))**2 *sp.exp(2*x)
y
```

```python
z=sp.diff(y,x)
z
```

```python
z.subs(x,3.2)
```

```python
f=x**2*sp.sin(x**2)
f

```

```python
g=sp.integrate(f,(x,0,5))
g
```

```python
g.evalf()
```

## Solving Equations and Groups of Equations

```python
y1=sp.Eq(x**3+15*x**2,3*x-10)
y1

```

```python
z=sp.solve(y1,x)
z
```

```python
z[0].evalf()
```

```python
z[1].evalf()
```

```python
z[2].evalf()
```

```python
for w in z:
    sp.pprint(w.evalf())
```

```python
x,y,z=sp.symbols('x,y,z')
eq1=sp.Eq(x+y+z,0)
eq1
```

```python
eq2=sp.Eq(2*x-y-z,10)
eq2

```

```python
eq3=sp.Eq(y+2*z,5)
eq3
```

```python
sp.solve([eq1,eq2,eq3],[x,y,z])
```

### Verifying answer using scipy methods of solving equations

```python
from scipy.optimize import fsolve
def f(w):
    x=w[0]
    y=w[1]
    z=w[2]
    f1=x+y+z
    f2=2*x-y-z-10
    f3=y+2*z-5
    return [f1,f2,f3]

result=fsolve(f,[0,0,0])
sp.pprint(result)
```

## Solving differential equations symbolically using the ```dsolve``` function:
$$\frac{df(x)}{dx} = x\cos(x).$$

```python
pf=sp.Function('pf')
y=sp.dsolve(sp.Derivative(pf(x),x)-x*sp.cos(x),pf(x))
y
```

```python
z=sp.integrate(x*sp.cos(x))
z
```

## Matrices, Vectors and Solving Equations of the form `Ax=b` :

```python
from sympy import Matrix
A=sp.Matrix([[1,2,5],[3,4,6],[-1,0,3]])
b=sp.Matrix([1,0,-2])
sp.pprint(A)
sp.pprint(b)
```

```python
sp.pprint(A.inv()*b)
sp.pprint(A.LUsolve(b))
```

```python
sp.pprint(A[1:2,:])
```

```python
sp.pprint(A[:,1:2])
```

```python
M=sp.zeros(2,2)
M[1,1]=3
M[1,0]=x**2
sp.pprint(M)
```

```python
M=sp.ones(2,2)
M[1,1]=0
sp.pprint(M.inv())
```

```python
!pip install keras
```

```python
!pip install rise
```

```python
!jupyter-nbextension install rise --py --sys-prefix
```

```python
!jupyter-nbextension enable rise --py --sys-prefix
```

```python
!pip install tensorflow
```

```python
!pip install plotly
```

```python

```


Monte Carlo integration is a technique for numerical integration using random numbers. It can be useful, for example, when evaluating integrals over domains of high dimension. Meshing in high dimensions suffers from the [curse of dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality): 100 evenly spaced sample points suffice to sample the unit interval $[0,1]$ with no more than 0.01 distance between points, whereas sampling of the 10-dimensional unit hypercube $[0,1]^{10}$ with a lattice that has a spacing of 0.01 between adjacent points would require $10^{20}$ sample points.

Let $\Omega \subset \mathbb R^n$ and let $f : \Omega \to \mathbb R$ be piecewise continuous.
Let $p$ be such a probability density function on $\Omega$ that $p(x) > 0$ for all $x \in \Omega$.
The Monte Carlo approach to approximate the integral $\int_\Omega f(x) dx$
works as follows:

1. Generate a large number of independent samples $x_1,\dots,x_N \in \Omega$
from the probablity distribution with the density $p$
2. Compute the quantity

$$
I_N = \frac 1 N \sum_{i=1}^N \frac{f(x_i)}{p(x_i)}.
$$

The [law of large numbers](https://en.wikipedia.org/wiki/Law_of_large_numbers) implies that

$$
I_N \to \int_\Omega f(x) dx \quad \text{as $n \to \infty$}.
$$

Choosing $p$ cleverly is the basic idea behind [importance sampling](https://en.wikipedia.org/wiki/Importance_sampling).

[Buffon's needle](https://en.wikipedia.org/wiki/Buffon%27s_needle_problem) experiment is a classical pedagogical example of Monte Carlo integration.
Suppose we have a floor made of parallel strips of wood, each the same width $t$, and we drop a needle of length $l < t$ onto the floor. Buffon showed that the  probability $P$ that the needle will lie across a line between two strips is

\begin{equation}\tag{1}
P=\frac{2}{\pi}\frac{l}{t}.
\end{equation}

Let $s$ be the distance from the center of the needle to the closest parallel line, and let $\theta$ be the acute angle between the needle and one of the parallel lines.
Here $s$ and $\theta$ are random variables with uniform distributions over $[0, t/2]$ and $[0,\pi/2]$, respectively. We write $p(s,\theta)$ for their joint probability density function and $A$ for the event that the needle lies across a line between two strips. Then

$$
P = \int_A p(s,\theta) ds d\theta.
$$

Writing $\Omega = [0, t/2] \times [0,\pi/2]$ and $f(x) = 1_A(x) p(x)$
where $x = (s,\theta)$ and

$$
1_A(x) =
\begin{cases}
1 & x \in A,
\\
0 & x \notin A,
\end{cases}
$$

we have, using the notation above, that $I_N \to P$ as $N \to \infty$.
Observe that, in this case,

$$
I_N = \frac 1 N \sum_{i=1}^N 1_A(x_i),
$$

and evaluating $I_N$ boils down to counting the needles that lie across a line between two strips.

Take $l = 5/6$ and $t = 1$. The goal of this homework is to approximate
$\pi$ via the formula

$$
\pi=\frac{2l}{t P} \approx \frac{2l}{t I_N},
$$

that follows from (1).

Let us give some remarks on the history of Monte Carlo integration. Formula (1) was first derived in

> Buffon, Georges L. L., comte de. _Histoire naturelle, générale et particulière, Supplément 4_. Imprimerie royale, Paris, 1777. (scan in [Google Books](https://books.google.fi/books?id=AjhYD1vsVAIC&hl=fi&pg=PA100#v=onepage&q&f=false))

Now my French is not very strong, but as claimed [here](https://en.wikipedia.org/wiki/Buffon's_needle_problem), it seems that approximating $\pi$ was not the original motivation for Buffon's question. You can have a look at his book, see pp. 100-104, and while you are at it, you can also try figure out if there is an error in his derivation, as claimed [here](https://mathworld.wolfram.com/Buffon-LaplaceNeedleProblem.html).

The idea of using Buffon's formula to design a method for approximating the number $\pi$ goes back at least to Laplace. _"Si l'on projette un grand nombre de fois ce cylindre [...] ce qui fera connaître la valeur de la circonférence $2 \pi$"_, see p. 360 of

> Laplace, Pierre S., marquis de. _Théorie analytique des probabilités_. Veuve Courcier, Paris, 1812. (scan in [Internet Archive](https://archive.org/details/thorieanalytiqu01laplgoog/page/n464/mode/2up))

The first computerized Monte Carlo simulations were run on [ENIAC](https://en.wikipedia.org/wiki/ENIAC) in 1948 by a team including John and Klara von Neumann and Nick Metropolis. It can be argued that the simulations were also the first code written in the modern paradigm, associated with the "stored program concept," ever to be executed, see

> Haigh, Thomas, Priestley, Mark, and Rope, Crispin. _Los Alamos Bets on ENIAC: Nuclear Monte Carlo Simulations, 1947-1948_. IEEE Annals of the History of Computing 36, no. 3, 42-63, 2014. <https://doi.org/10.1109/MAHC.2014.40> (in [Helka](https://helka.helsinki.fi/permalink/358UOH_INST/qn0n39/cdi_ieee_primary_6880250))

```python
import numpy as np
rng = np.random.default_rng()

t = 1
l = 5/6

def sample():
    '''Returns s and theta generated using the random number generator rng'''
    # Draw samples from uniform distributions using the function rng.uniform, see
    # https://numpy.org/doc/stable/reference/random/generated/numpy.random.Generator.uniform.html
    raise NotImplementedError() # a placeholder, your implementation goes here

def intersects(s, theta):
    '''Returns True iff the needle lies across a line between two strips'''
    raise NotImplementedError() # a placeholder, your implementation goes here

# We will plot I_N for several N so let's save every nth approximation
def I(n, K):
    '''Return I_n, I_{2n}, ..., I_{Kn}'''
    out = np.zeros(K)
    raise NotImplementedError() # a placeholder, your implementation goes here
    return out
```

```python
import matplotlib.pyplot as plt
```

```python
# Plot one sample
s, theta = sample()
c = np.array([0, s]) # Center of needle
d = np.array([np.cos(theta), np.sin(theta)]) # Direction of needle
# End points of needle
end1 = c - l/2*d
end2 = c + l/2*d
ends = np.stack((end1, end2))
xs = ends[:,0]
ys = ends[:,1]

plt.plot(xs, ys, 'r') # needle in red
plt.plot([-1,1],[0, 0], 'b') # closest line in blue
ax = plt.gca()
ax.set_ylim(-0.5, 1)
ax.set_aspect(1)
print(f'Needle intersects the closest line: {intersects(s, theta)}')
```

```python
# Plot convergence to pi
n = 100
K = 40
Is = I(n, K)
Ns = n*np.arange(1,K+1)
plt.plot(Ns, 2*l/(t*Is), 'b') # approximation in blue
plt.plot([n,n*K],[np.pi, np.pi],'r') # pi in red
ax = plt.gca()
ax.set_ylim(2.4, 3.6);
```

**How to hand in your solution**

1. Run the whole notebook by choosing _Restart Kernel and Run All Cells_ in the _Run_ menu
    - Alternatively you can click the ⏩ icon in the toolbar
2. Click the link below to check that the piece of code containing your solution was uploaded to pastebin
    - If you have changed the order of cells in the notebook, you may need to change the number in the below cell to the one in the left margin of the cell containing your solution
3. Copy the link and submit it in Moodle
    - You can copy the link easily by right-clicking it and choosing _Copy Output to Clipboard_

```python
# Upload the code in the first input cell to pastebin
%pastebin 1
```


```python
import numpy as np
import matplotlib.pyplot as plt
import plotutils as pu
from sympy.mpmath import nsum, inf, mp
from sympy import *
init_printing()
%matplotlib inline
```

The **Leibniz formula** states that:

$$
1 - \frac{1}{3} + \frac{1}{5} - \frac{1}{7} + \frac{1}{9} - \ldots = \frac{\pi}{4}
$$

So in other words:

$$
\sum^{\infty}_{n = 0}\frac{(-1)^n}{2n + 1} = \frac{\pi}{4}
$$

Let's see if we can understand the sum formula first. What we'll do is just take the top part of the fraction $(-1)^n$ and the bottom part $2n + 1$ and plot them seperately for some values of $n$:

```python
with plt.xkcd():
    x = np.arange(0, 10, 1)
    fig, axes = plt.subplots(1, 2, figsize=(10, 4))
    for ax in axes: pu.setup_axes(ax)
    axes[0].plot(x, (-1)**x, 'bo', zorder=10)
    axes[0].set_xlim(0, 9)
    axes[0].set_ylim(-5, 5)
    axes[1].plot(x, (2*x) + 1)
    axes[1].set_xlim(0, 9)
    axes[1].set_ylim(0, 10)
```

So as $n$ gets bigger and bigger we have two things. One flips between $1$ and $-1$ and the other is just a linear value $y = 2n + 1$ that just keeps on getting bigger and bigger. Now the equation above tells us that if we take an near infinite sum of these values we will get closer and closer to the value of $\frac{\pi}{4}$ so let's see if that's true.

Below are two lines, one line represents $y = \frac{(-1)^n}{2n + 1}$ and the other line is the sum of all the values of that equation for $y$ at $n = 0, 1, 2, \ldots, n$. You can see that it (slowly) converges to some value, namely the value $\frac{4}{\pi}$.

```python
n = np.arange(0, 10, 1)
f = lambda x: ((-1)**x) / (2*x + 1)
with plt.xkcd():
    fig, axes = plt.subplots(1, figsize=(8, 8))
    pu.setup_axes(axes, xlim=(-1, 9), ylim=(-0.5, 1.2), yticks=[1], yticklabels=[1], xticks=[1,2,3,4,5,6,7,8])
    plt.plot(n, f(n), zorder=10, label='THE INFINITE SERIES')
    plt.plot(n, [nsum(f, [0, n]) for n in n], label='SUMMATION OF THE SERIES')
    plt.annotate('THE LEIBNIZ FORMULA FOR PI', (1, 1))
    axes.set_aspect(4.0)
    axes.legend(loc=4)
```

Now if we sum up all the terms of that line above for $x = 0, 1, 2, 3, \ldots, n$ we'll get closer and closer to $\frac{4}{\pi}$. Using `mpmath` we can calculate $\pi$ with pretty good detail using the `mp.dps` setting to control the precision.

```python
leibniz = lambda n: ((-1)**n) / (2 * n + 1)
mp.dps = 50
nsum(leibniz, [0, inf]) * 4
```

    mpf('3.1415926535897932384626433832795028841971693993751068')

Of course we can compute it symbolically as well. These fractions get pretty crazy real quickly.

```python
leibniz = S('((-1)^n)/(2*n+1)')
n = S('n')
sum([leibniz.subs(n, i) for i in range(100)])
```

```python

```