# Standard Pooling Problem

## Objective and Prerequisites

One of the new features of Gurobi 9.0 is the addition of a bilinear solver, which enables finding the optimal solution of non-convex quadratic programming  problems (i.e. QPs, QCQPs, MIQPs, and MIQCQPs). This notebook will show you how to use this feature by tackling an instance of the Standard Pooling Problem, one of the most well-known Bilinear Programs. Moreover, we will present two alternative formulations, casted as Quadratically Constrained Quadratic Problems, to contrast their performance when calling the optimizer.

This modeling example is at the advanced level. To fully understand the content of this notebook, the reader should be familiar with the following:

- Python.
- Gurobi Python interface.
- Advanced knowledge of building mathematical optimization models. Typically, the constraints of these Jupyter Notebooks are complex or require advanced features of the Gurobi Python Interface.

**Download the Repository**

You can download the repository containing this and other examples by clicking [here](https://github.com/Gurobi/modeling-examples/archive/master.zip).

**Gurobi License**

In order to run this Jupyter Notebook properly, you must have a Gurobi license. If you do not have one, you can request an [evaluation license](https://www.gurobi.com/downloads/request-an-evaluation-license/?utm_source=3PW&utm_medium=OT&utm_campaign=WW-MU-OGS-OR-O_LEA-PR_NO-Q3_FY20_WW_JPME_standard-pooling_COM_EVAL_GITHUB_&utm_term=standard-pooling-problem&utm_content=C_JPM) as a *commercial user*, or download a [free license](https://www.gurobi.com/academia/academic-program-and-licenses/?utm_source=3PW&utm_medium=OT&utm_campaign=WW-MU-OGS-OR-O_LEA-PR_NO-Q3_FY20_WW_JPME_standard-pooling_ACADEMIC_EVAL_GITHUB_&utm_term=standard-pooling-problem&utm_content=C_JPM) as an *academic user*.

---
## Motivation

The pooling problem is a challenging problem in the petrochemical refining, wastewater treatment and mining industries. This problem can be regarded as a generalization of the minimum-cost flow problem and the blending problem. It is indeed important because of the significant savings it can generate, so it comes at no surprise that it has been studied extensively since Haverly pointed out the non-linear structure of this problem in 1978 [5].

---
## Problem Description

The Minimum-Cost Flow Problem (MCFP) seeks to find the cheapest way of sending a certain amount of flow from a set of source nodes to a set of target nodes, possibily via transshipment nodes, in a directed capacitated network. The Blending Problem is a type of MCFP with only source and target nodes, where raw materials with different attribute qualities are blended together to create end products in such a way that their attribute qualities are within tolerances.

The Pooling Problem combines features of both problems, as flow streams from different sources are mixed at intermediate pools and blended again at the target nodes. The non-linearity is in fact the direct result of considering pools, as the quality of a given attribute at a pool —defined as the weighted average of the qualities of the incoming streams— is an unknown quantity and thus needs to be captured by a decision variable. We refer to this problem as the Standard Pooling Problem when the network can be represented by a tripartite graph, i.e. three disjoint sets of nodes such that no nodes within the same set are adjacent. In a nutshell, it can be stated as follows: Given a list of source nodes with raw materials containing known attribute qualities, what is the cheapest way of mixing these materials at intermediate pools so as to meet the demand and tolerances at multiple target nodes? (Gupte et al., 2017) [4]. Several different formulations for the Standard Pooling Problem and its extensions exist in the literature, which can be classified into two main categories: one that consists of flow and quality variables, and the other that uses flow proportions instead of quality variables. Both categories will be considered in this notebook.

---
## Problem Instance

As an illustrative example, we will solve the second Pooling Problem posed by Rehfeldt and Tisljar in 1997 and cited by Audet et al. in 2004:

To that end, let's declare the required data structures to represent this problem instance:

```python
import numpy as np
import pandas as pd
from itertools import product

import gurobipy as gp
from gurobipy import GRB

attrs = {'den', 'bnz', 'roz', 'moz'}

sources, cost, supply, content = gp.multidict({
    "s1": [49.2, 6097.56, {'den': 0.82, 'bnz':3, 'roz':99.2,'moz':90.5}],
    "s2": [62.0, 16129, {'den': 0.62, 'bnz':0, 'roz':87.9,'moz':83.5}],
    "s3": [300.0, 500, {'den': 0.75, 'bnz':0, 'roz':114,'moz':98.7}]
})

targets, price, demand, min_tol, max_tol = gp.multidict({
    "t1": [190, 500, {'den': 0.74, 'roz':95,'moz':85}, {'den': 0.79}],
    "t2": [230, 500, {'den': 0.74, 'roz':96,'moz':88}, {'den': 0.79, 'bnz':0.9}],
    "t3": [150, 500, {'den': 0.74, 'roz':91}, {'den': 0.79}]
})

pools, cap = gp.multidict({
    "p1": 1250,
    "p2": 1750
})

# The function `product` deploys the Cartesian product of elements in sets A and B
s2p = set(product(sources, pools))
p2t = set(product(pools, targets))
s2t = {("s1", "t2"),
       ("s2", "t1"),
       ("s2", "t3"),
       ("s3", "t1")}
```

---
## Solution Approach

Mathematical programming is a declarative approach where the modeler formulates a mathematical optimization model that captures the key aspects of a complex decision problem. The Gurobi Optimizer solves such models using state-of-the-art mathematics and computer science.

A mathematical optimization model has five components, namely:

- Sets and indices.
- Parameters.
- Decision variables.
- Objective function(s).
- Constraints.

A quadratic constraint that involves only products of disjoint pairs of variables is often called a bilinear constraint, and a model that contains bilinear constraints is often called a Bilinear Program. Bilinear constraints are a special case of non-convex quadratic constraints. This type of problems is typically solved using spatial Branch and Bound (sB&B). This algorithm explores the entire search space, so it provides a globally valid lower bound on the optimal objective value and —given enough time— it will find a globally optimal solution (subject to tolerances). The interested reader is referred to [references](#references) [3], [6] and [7].

We now present two alternative Bilinear Programs for the Standard Pooling Problem:

### P-formulation (Concentration)

#### Sets and Indices

$G=(V,E)$: Directed graph.

$i,j \in V$: Set of nodes.

$(i,j) \in E \subset  V \times V$: Set of edges.

$N(i)^+ = \{j \in V \mid (i,j) \in E \}$: Set of successor nodes receiving outflow from node $i$.

$N(j)^- = \{i \in V \mid (i,j) \in E \}$: Set of predecessor nodes sending inflow to node $i$.

$k \in \text{Attrs}$: Set of attributes.

$s \in \text{Sources} \subset V$: Set of source nodes, i.e. $N(s)^-= \emptyset$.

$t \in \text{Targets} \subset V$: Set of target nodes, i.e. $N(t)^+= \emptyset$.

$p \in \text{Pools} = V \setminus (\text{Sources} \cup \text{Targets})$: Set of pools.

#### Parameters

$\text{Cost}_s \in \mathbb{R}^+$: Cost of acquiring one unit of raw material at source node $s$.

$\text{Supply}_s \in \mathbb{R}^+$: Maximum number of units of raw material available at source node $s$.

$\text{Content}_{s,k} \in \mathbb{R}^+$: Content of attribute $k$ in raw material at source node $s$.

$\text{Price}_t \in \mathbb{R}^+$: Price for selling one unit of final blend at target node $t$.

$\text{Demand}_t \in \mathbb{R}^+$: Minimum number of units required of final blend at target node $t$.

$\text{Min_tol}_{t,k} \in \mathbb{R}^+$: Minimum tolerance for attribute $k$ in final blend at target node $t$.

$\text{Max_tol}_{t,k} \in \mathbb{R}^+$: Maximum tolerance for attribute $k$ in final blend at target node $t$.

$\text{Cap}_p \in \mathbb{R}^+$: Maximum Capacity to store intermediate blend at pool $p$.

$\text{UB}_{i,j}\in \mathbb{R}^+$: Maximum flow from node $i$ to node $j$.

#### Decision Variables

$\text{flow}_{i,j} \in [0, \text{UB}_{i,j}]$: Flow from node $i$ to node $j$.

$\text{quality}_{p,k} \in \mathbb{R}^+$: Concentration of attribute $k$ at pool $p$.

#### Objective Function

- **Profit**: Maximize total profits.

\begin{equation}
\text{Max} \quad Z = \sum_{t \in \text{Targets}}{\sum_{i \in N(t)^-}{\text{Price}_t \cdot \text{flow}_{i,t}}} - \sum_{s \in \text{Sources}}{\sum_{j \in N(s)^+}{\text{Cost}_s \cdot \text{flow}_{s,j}}}
\tag{0}
\end{equation}

#### Constraints

- **Flow conservation**: Total inflow of pool $p$ must be equal to its total outflow (nothing is stored in them).

\begin{equation}
\sum_{t \in N(p)^+}{\text{flow}_{p,t}} - \sum_{s \in N(p)^-}{\text{flow}_{s,p}} = 0 \quad \forall p \in \text{Pools}
\tag{1}
\end{equation}

- **Source capacity**: Total outflow of source $s$ cannot exceed its capacity.

\begin{equation}
\sum_{j \in N(s)^+}{\text{flow}_{s,j}} \leq \text{Supply}_s \quad \forall s \in \text{Sources}
\tag{2}
\end{equation}

- **Pool capacity**: Total outflow of pool $p$ cannot exceed its capacity.

\begin{equation}
\sum_{t \in N(p)^+}{\text{flow}_{p,t}} \leq \text{Cap}_p \quad \forall p \in \text{Pools}
\tag{3}
\end{equation}

- **Target demand**: Total inflow of target $t$ must at least meet its minimum demand.

\begin{equation}
\sum_{i \in N(t)^-}{\text{flow}_{i,t}} \geq \text{Demand}_t \quad \forall t \in \text{Targets}
\tag{4}
\end{equation}

- **Pool concentration**: Concentration of attribute $k$ at pool $p$ is expressed as the weighted average (linear blending) of the concentrations associated to the incoming flows (notice the bilinear terms on the right-hand side).

\begin{equation}
\sum_{s \in N(p)^-}{\text{Content}_{s,k} \cdot \text{flow}_{s,p}} = \text{quality}_{p,k} \cdot \sum_{t \in N(p)^+}{\text{flow}_{p,t}} \quad \forall (p,k) \in \text{Pools} \times \text{Attrs}
\tag{5}
\end{equation}

- **Target tolerances**: Concentration of attribute $k$ at target $t$ is also the result of linear blending, and must be within tolerances (notice the bilinear terms on the second expression of the left-hand side).

\begin{equation}
\sum_{s \in N(t)^- \cap \text{Sources}}{\text{Content}_{s,k} \cdot \text{flow}_{s,t}}+ \sum_{p \in N(t)^- \cap \text{Pools}}{\text{quality}_{p,k} \cdot \text{flow}_{p,t}} \geq \text{Min_tol}_{t,k} \cdot \sum_{i \in N(t)^-}{\text{flow}_{i,t}} \quad \forall (t,k) \in \text{Targets} \times \text{Attrs}
\tag{6.1}
\end{equation}

\begin{equation}
\sum_{s \in N(t)^- \cap \text{Sources}}{\text{Content}_{s,k} \cdot \text{flow}_{s,t}}+ \sum_{p \in N(t)^- \cap \text{Pools}}{\text{quality}_{p,k} \cdot \text{flow}_{p,t}} \leq \text{Max_tol}_{t,k} \cdot \sum_{i \in N(t)^-}{\text{flow}_{i,t}} \quad \forall (t,k) \in \text{Targets} \times \text{Attrs}
\tag{6.2}
\end{equation}

The number of bilinear terms in this formulation is proportional to the number of attributes. An alternative formulation relies on decision variables that represent fractions of flow instead of concentrations, so that the bilinear terms are no longer associated to the number of attributes. Two types of decision variables may be used:

- fraction of total inflow at pool $p$, coming from source $s$.
- fraction of total outflow at pool $p$, going to terminal $t$.

A model based on the first option is now presented:

### Q-formulation (Proportion)

#### Decision Variables

$\text{flow}_{i,j} \in [0, \text{UB}_{i,j}]$: Flow from node $i$ to node $j$.

$\text{prop}_{s,p} \in \mathbb{R}^+$: fraction of total inflow at pool $p$, coming from source $s$.

**Note:** The $\text{flow}$ variables from sources to pools are replaced by the $\text{prop}$ variables.

#### Objective Function

- **Profit**: Maximize total profits (notice the bilinear terms on the second expression).

\begin{equation}
\text{Max} \quad Z = \sum_{t \in \text{Targets}}{\sum_{i \in N(t)^-}{\text{Price}_t \cdot \text{flow}_{i,t}}} - \sum_{s \in \text{Sources}}{\text{cost}_s \cdot \left( \sum_{t \in N(s)^+ \cap \text{Targets}}{\text{flow}_{s,t}} + \sum_{p \in N(s)^+ \cap \text{Pools}}{\text{prop}_{s,p} \cdot \sum_{t \in N(p)^+}{\text{flow}_{p,t}}} \right) }
\tag{0}
\end{equation}

#### Constraints

- **Source capacity**: Total outflow of source $s$ cannot exceed its capacity (notice the bilinear terms on the first expression of the left-hand side).

\begin{equation}
\sum_{p \in N(s)^+ \cap \text{Pools}}{\text{prop}_{s,p} \cdot \sum_{t \in N(p)^+}{\text{flow}_{p,t}}} + \sum_{t \in N(s)^+ \cap \text{Targets}}{\text{flow}_{s,t}} \leq \text{Supply}_s \quad \forall s \in \text{Sources}
\tag{1}
\end{equation}

- **Pool capacity**: Total outflow of pool $p$ cannot exceed its capacity.

\begin{equation}
\sum_{t \in N(p)^+}{\text{flow}_{p,t}} \leq \text{Cap}_p \quad \forall p \in \text{Pools}
\tag{2}
\end{equation}

- **Target demand**: Total inflow of target $t$ must at least meet its minimum demand.

\begin{equation}
\sum_{i \in N(t)^-}{\text{flow}_{i,t}} \geq \text{Demand}_t \quad \forall t \in \text{Targets}
\tag{3}
\end{equation}

- **Pool inflow**: The sum of the contributions of all incoming sources to pool $p$ must equal one.

\begin{equation}
\sum_{s \in N(p)^-}{\text{prop}_{s,p}} = 1 \quad \forall p \in \text{Pools}
\tag{4}
\end{equation}

- **Target tolerances**: Concentration of attribute $k$ at target $t$ is also the result of linear blending, and must be within tolerances (notice the bilinear terms on the second expression of the left-hand side).

\begin{equation}
\sum_{s \in N(t)^- \cap \text{Sources}}{\text{Content}_{s,k} \cdot \text{flow}_{s,t}} + \sum_{p \in N(t)^- \cap \text{Pools}}{\text{flow}_{p,t} \cdot \sum_{s \in N(p)^-}{\text{content}_{s,k} \cdot \text{prop}_{s,p}}}
\geq \text{Min_tol}_{t,k} \cdot \sum_{i \in N(t)^-}{\text{flow}_{i,t}} \\
\forall (t,k) \in \text{Targets} \times \text{Attrs}
\tag{5.1}
\end{equation}

\begin{equation}
\sum_{s \in N(t)^- \cap \text{Sources}}{\text{Content}_{s,k} \cdot \text{flow}_{s,t}} + \sum_{p \in N(t)^- \cap \text{Pools}}{\text{flow}_{p,t} \cdot \sum_{s \in N(p)^-}{\text{content}_{s,k} \cdot \text{prop}_{s,p}}}
\leq \text{Max_tol}_{t,k} \cdot \sum_{i \in N(t)^-}{\text{flow}_{i,t}} \\
\forall (t,k) \in \text{Targets} \times \text{Attrs}
\tag{5.2}
\end{equation}

One drawback is that if some of the source-to-pool edges have flow capacity, we need to define additional constraints instead of just specifying upper bounds on the associated decision variables. Such constraints can be defined with bilinear terms as follows:

- **Flow limit**: Flow from source $s$ to pool $p$ cannot exceed the installed capacity (notice the bilinear terms on the left-hand side). In the P-formulation, declaring this is as easy as setting the upper bound of the associated $\text{flow}$ variable. However, this variable no longer exists in the Q-formulation so we need to model the capacity as a constraint.

\begin{equation}
\text{prop}_{s,p} \cdot \sum_{t \in N(p)^+}{\text{flow}_{p,t}} \leq \text{UB}_{s,p} \quad \forall (i,j) \in E \cap \left( \text{Sources} \times \text{Pools} \right) \mid \text{UB}_{i,j} < \infty
\tag{6}
\end{equation}

---
## Python Implementation

Solving Bilinear Programs with Gurobi is as easy as configuring the global parameter `nonConvex`. When setting this parameter to a value of 2, non-convex quadratic problems are solved by means of translating them into bilinear form and applying sB&B. We will first deploy the P-formulation model, and then compare it with the Q-formulation model:

### P-formulation (Concentration)

```python
p_pooling = gp.Model("Pooling")

# Set global parameters
p_pooling.params.nonConvex = 2
p_pooling.params.timelimit = 5*60 # time limit of 5 minutes

# Declare decision variables

# flow
ik = p_pooling.addVars(s2t, name="Source2Target")
ij = p_pooling.addVars(s2p, name="Source2Pool")
jk = p_pooling.addVars(p2t, name="Pool2Target")
ik["s1","t2"].ub = 750
ik["s3","t1"].ub = 750
# quality
prop = p_pooling.addVars(pools, attrs, name="Proportion")

# Deploy constraint sets

# 1. Flow conservation
p_pooling.addConstrs((ij.sum('*',j) == jk.sum(j,'*') for j in pools),
                     name="Flow_conservation")
# 2. Source capacity
p_pooling.addConstrs((ij.sum(i,'*') + ik.sum(i,'*') <= supply[i] for i in sources),
                     name="Source_capacity")
# 3. Pool capacity
p_pooling.addConstrs((jk.sum(j,'*') <= cap[j] for j in pools),
                     name="Pool_capacity")
# 4. Target demand
p_pooling.addConstrs((ik.sum('*',k) + jk.sum('*',k) >= demand[k] for k in targets),
                     name="Target_demand")
# 5. Pool concentration
p_pooling.addConstrs((gp.quicksum(content[i][attr]*ij[i,j]
                               for i in sources if (i,j) in s2p)
                      == prop[j,attr]*jk.sum(j,'*') for j in pools for attr in attrs),
                     name="Pool_concentration")
# 6.1 Target (min) tolerances
p_pooling.addConstrs((gp.quicksum(content[i][attr]*ik[i,k]
                               for i in sources if (i,k) in s2t)
                      + gp.quicksum(prop[j,attr]*jk[j,k]
                                 for j in pools if (j,k) in p2t)
                      >= min_tol[k][attr]*(ik.sum('*',k) + jk.sum('*',k))
                      for k in targets for attr in min_tol[k].keys()),
                     name="Target_min_tolerances")
# 6.2 Target (max) tolerances
p_pooling.addConstrs((gp.quicksum(content[i][attr]*ik[i,k]
                               for i in sources if (i,k) in s2t)
                      + gp.quicksum(prop[j,attr]*jk[j,k]
                                 for j in pools if (j,k) in p2t)
                      <= max_tol[k][attr]*(ik.sum('*',k) + jk.sum('*',k))
                      for k in targets for attr in max_tol[k].keys()),
                     name="Target_max_tolerances")

# Deploy Objective Function

# 0. Total profit
obj = gp.quicksum(price[k]*(ik.sum('*',k) + jk.sum('*',k))
               for k in targets) \
- gp.quicksum(cost[i]*(ij.sum(i,'*') + ik.sum(i,'*'))
           for i in sources)
p_pooling.setObjective(obj, GRB.MAXIMIZE)

# Find the optimal solution
p_pooling.optimize()
```

    Using license file c:\gurobi\gurobi.lic
    Set parameter TokenServer to value SANTOS-SURFACE-
    Changed value of parameter nonConvex to 2
       Prev: -1  Min: -1  Max: 2  Default: -1
    Changed value of parameter timelimit to 300.0
       Prev: inf  Min: 0.0  Max: inf  Default: inf
    Gurobi Optimizer version 9.0.0 build v9.0.0rc2 (win64)
    Optimize a model with 10 rows, 24 columns and 38 nonzeros
    Model fingerprint: 0x67ac2e0d
    Model has 20 quadratic constraints
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      QMatrix range    [1e+00, 1e+00]
      QLMatrix range   [1e-02, 1e+02]
      Objective range  [5e+01, 3e+02]
      Bounds range     [8e+02, 8e+02]
      RHS range        [5e+02, 2e+04]

    Continuous model is non-convex -- solving as a MIP.

    Presolve removed 1 rows and 0 columns
    Presolve time: 0.00s
    Presolved: 125 rows, 49 columns, 311 nonzeros
    Presolved model has 24 bilinear constraint(s)
    Variable types: 49 continuous, 0 integer (0 binary)

    Root relaxation: objective 9.577646e+05, 56 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0 957764.599    0   10          - 957764.599      -     -    0s
         0     0 957764.599    0   13          - 957764.599      -     -    0s
         0     0 878230.977    0   22          - 878230.977      -     -    0s
         0     0 878230.977    0   21          - 878230.977      -     -    0s
         0     0 878230.977    0   20          - 878230.977      -     -    0s
         0     2 878230.977    0   20          - 878230.977      -     -    0s
    * 5617  3367             187    305285.85400 710014.759   133%  10.2    0s
    * 6518  3356             213    322989.75963 710014.759   120%   9.9    0s
    * 6616  3196             203    420278.24685 709866.667  68.9%   9.9    0s
    * 6617  3196             203    420278.24715 709866.667  68.9%   9.9    0s
    * 6625  3189             203    426921.63138 709866.667  66.3%   9.9    0s
    * 6626  3189             203    426921.63168 709866.667  66.3%   9.9    0s
    *14156  3644              96    428861.92345 703808.676  64.1%  13.0    1s
    *14187  3644             115    428861.93092 703808.676  64.1%  13.0    1s
    *16305  4417              47    438810.28381 695701.543  58.5%  13.3    1s
    *16315  4417              52    438810.72501 695701.543  58.5%  13.3    1s
    *16330  4417              60    438810.72562 695701.543  58.5%  13.3    1s
     54433  8788 infeasible   81      438810.726 688169.500  56.8%  17.2    5s
     114427 11887 496879.815   93    9 438810.726 688070.658  56.8%  19.3   10s
     166621 15541     cutoff   74      438810.726 687859.137  56.8%  19.7   15s
     222977 19206     cutoff   90      438810.726 687811.479  56.7%  20.0   20s
     281824 22913     cutoff   90      438810.726 687779.194  56.7%  20.2   25s
     334154 25696 687760.058  101    8 438810.726 687760.308  56.7%  20.2   30s
     391810 28356     cutoff   99      438810.726 687741.091  56.7%  20.3   35s
     445981 31429     cutoff  121      438810.726 687717.662  56.7%  20.3   40s
     494259 34608 635236.868   88   12 438810.726 687695.472  56.7%  20.3   45s
     549373 38162 677414.463   73   10 438810.726 687672.453  56.7%  20.4   50s
     602929 40885 infeasible  107      438810.726 687637.030  56.7%  20.4   55s
     658374 43671     cutoff   76      438810.726 687604.866  56.7%  20.4   60s
     710810 46988 infeasible   93      438810.726 687577.720  56.7%  20.5   65s
     764370 49682 684094.585   89    7 438810.726 687548.600  56.7%  20.5   70s
     821374 53365 496231.780  135    9 438810.726 687520.315  56.7%  20.5   75s
     876252 56852 635009.634   83   12 438810.726 687471.163  56.7%  20.5   80s
     931323 60658 496045.907   80    9 438810.726 687359.522  56.6%  20.5   85s
     979057 63653     cutoff   81      438810.726 687310.783  56.6%  20.5   90s
     1022144 65727 infeasible   95      438810.726 687280.723  56.6%  20.6   95s
     1068692 68147     cutoff   67      438810.726 687242.213  56.6%  20.6  100s
     1117024 71080     cutoff   84      438810.726 687210.158  56.6%  20.6  105s
     1161191 73277 687178.210  109    7 438810.726 687179.855  56.6%  20.6  110s
     1205549 75483     cutoff   89      438810.726 687148.344  56.6%  20.6  115s
     1253740 78261     cutoff   91      438810.726 687092.429  56.6%  20.6  120s
     1296426 80702 infeasible   93      438810.726 687031.431  56.6%  20.6  125s
     1345545 83035 686949.508   75    9 438810.726 686973.984  56.6%  20.6  130s
     1389245 85822 infeasible   86      438810.726 686939.110  56.5%  20.6  135s
     1433028 87643     cutoff  101      438810.726 686910.298  56.5%  20.6  140s
     1478083 89950 686880.632   79    8 438810.726 686880.974  56.5%  20.6  145s
     1520105 92214 infeasible   83      438810.726 686857.739  56.5%  20.6  150s
     1560703 94390     cutoff   94      438810.726 686839.042  56.5%  20.6  155s
     1606093 96639 infeasible   68      438810.726 686819.533  56.5%  20.6  160s
     1648233 99350 495382.847   93   10 438810.726 686799.761  56.5%  20.6  165s
     1690016 100901     cutoff  107      438810.726 686780.261  56.5%  20.6  170s
     1732467 103101     cutoff  112      438810.726 686751.318  56.5%  20.6  175s
     1777823 105709     cutoff  115      438810.726 686717.340  56.5%  20.6  180s
     1822528 108116     cutoff  116      438810.726 686687.047  56.5%  20.6  185s
     1866932 109716     cutoff  105      438810.726 686649.859  56.5%  20.6  190s
     1907178 112344 infeasible  109      438810.726 686613.582  56.5%  20.6  195s
     1948968 114240     cutoff  107      438810.726 686577.186  56.5%  20.7  200s
     1989241 116469 630915.020   85   14 438810.726 686540.908  56.5%  20.7  205s
     2034163 119446     cutoff   96      438810.726 686512.239  56.4%  20.7  210s
     2075959 121668 683084.471   86   10 438810.726 686489.065  56.4%  20.7  215s
     2117969 123642 infeasible  133      438810.726 686470.044  56.4%  20.6  220s
     2161396 126104     cutoff  101      438810.726 686445.024  56.4%  20.6  225s
     2205472 129075     cutoff  101      438810.726 686418.933  56.4%  20.6  230s
     2248478 131480 infeasible  109      438810.726 686399.097  56.4%  20.6  235s
     2291736 134074 494885.927   98    9 438810.726 686382.753  56.4%  20.6  240s
     2333492 136457 686363.747   99   10 438810.726 686365.310  56.4%  20.7  245s
     2381579 138806 infeasible  117      438810.726 686338.696  56.4%  20.6  250s
     2426662 141501     cutoff  102      438810.726 686312.277  56.4%  20.6  255s
     2466556 143946 686286.831  112    8 438810.726 686288.574  56.4%  20.6  260s
     2512626 146125 infeasible  105      438810.726 686257.950  56.4%  20.7  265s
     2552737 148328 infeasible   97      438810.726 686231.814  56.4%  20.7  270s
     2596263 151017     cutoff  109      438810.726 686203.718  56.4%  20.7  275s
     2637996 153267     cutoff  105      438810.726 686182.253  56.4%  20.7  280s
     2675094 155614 642666.374  114   15 438810.726 686157.874  56.4%  20.7  285s
     2715248 158512 infeasible   70      438810.726 686130.761  56.4%  20.7  290s
     2758734 160601     cutoff  107      438810.726 686111.133  56.4%  20.7  295s
     2796119 162698 494550.663  106    9 438810.726 686098.107  56.4%  20.7  300s

    Explored 2797241 nodes (57909738 simplex iterations) in 300.01 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 10: 438811 438811 438810 ... 322990

    Time limit reached
    Best objective 4.388107256162e+05, best bound 6.860980277975e+05, gap 56.3540%

The P-formulation for this instance has:

- 24 decision variables.
- 10 linear constraints.
- 20 bilinear constraints.
- a linear objective function.

As can be seen, we still observe a gap of 56.35% after reaching the time limit of five minutes (at this point, the incumbent solution induces a total profit of 438,810.73 USD). In fact, even after 20 minutes the solver does not make much progress in closing the gap.

### Q-Formulation (proportion)

Let's now see how the q-formulation model performs:

```python
q_pooling = gp.Model("Pooling")

# Set global parameters
q_pooling.params.nonConvex = 2
q_pooling.params.timelimit = 5*60

# Declare decision variables

# flow
ik = q_pooling.addVars(s2t, name="Source2Target")
jk = q_pooling.addVars(p2t, name="Pool2Target")
ik["s1","t2"].ub = 750
ik["s3","t1"].ub = 750
# proportion
p_ij = q_pooling.addVars(s2p, ub=1.0, name="Prop_Source2Pool")

# Deploy constraint sets

# 1. Source capacity
q_pooling.addConstrs((gp.quicksum(p_ij[i,j]*jk.sum(j,'*')
                               for j in pools if (i,j) in s2p)
                      + ik.sum(i,'*') <= supply[i] for i in sources),
                     name="Source_capacity")
# 2. Pool capacity
q_pooling.addConstrs((jk.sum(j,'*') <= cap[j] for j in pools),
                     name="Pool_capacity")
# 3. Target demand
q_pooling.addConstrs((ik.sum('*',k) + jk.sum('*',k) >= demand[k] for k in targets),
                     name="Target_demand")
# 4. Pool inflow
q_pooling.addConstrs((p_ij.sum('*',j) == 1 for j in pools),
                     name="Pool_inflow")
# 5.1 Target (min) tolerances
q_pooling.addConstrs((gp.quicksum(content[i][attr]*ik[i,k]
                               for i in sources if (i,k) in s2t)
                      + gp.quicksum(content[i][attr]*p_ij[i,j]*jk[j,k]
                                 for (i,j) in s2p if (j,k) in p2t)
                      >= min_tol[k][attr]*(ik.sum('*',k) + jk.sum('*',k))
                      for k in targets for attr in min_tol[k].keys()),
                     name="Target_min_tolerances")
# 5.2 Target (max) tolerances
q_pooling.addConstrs((gp.quicksum(content[i][attr]*ik[i,k]
                               for i in sources if (i,k) in s2t)
                      + gp.quicksum(content[i][attr]*p_ij[i,j]*jk[j,k]
                                 for (i,j) in s2p if (j,k) in p2t)
                      <= max_tol[k][attr]*(ik.sum('*',k) + jk.sum('*',k))
                      for k in targets for attr in max_tol[k].keys()),
                     name="Target_max_tolerances")

# Deploy Objective Function

# 0. Total profit
obj = gp.quicksum(price[k]*(ik.sum('*',k) + jk.sum('*',k))
               for k in targets) \
- gp.quicksum(cost[i]*(gp.quicksum(p_ij[i,j]*jk.sum(j,'*')
                             for j in pools if (i,j) in s2p)
                    + ik.sum(i,'*'))
           for i in sources)
q_pooling.setObjective(obj, GRB.MAXIMIZE)

# Find the optimal solution
q_pooling.optimize()
```

    Changed value of parameter nonConvex to 2
       Prev: -1  Min: -1  Max: 2  Default: -1
    Changed value of parameter timelimit to 300.0
       Prev: inf  Min: 0.0  Max: inf  Default: inf
    Gurobi Optimizer version 9.0.0 build v9.0.0rc2 (win64)
    Optimize a model with 7 rows, 16 columns and 22 nonzeros
    Model fingerprint: 0xeac904e0
    Model has 18 quadratic objective terms
    Model has 15 quadratic constraints
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      QMatrix range    [6e-01, 1e+02]
      QLMatrix range   [1e-02, 1e+02]
      Objective range  [9e+01, 2e+02]
      QObjective range [1e+02, 6e+02]
      Bounds range     [1e+00, 8e+02]
      RHS range        [1e+00, 2e+03]
      QRHS range       [5e+02, 2e+04]

    Continuous model is non-convex -- solving as a MIP.

    Presolve time: 0.00s
    Presolved: 95 rows, 35 columns, 315 nonzeros
    Presolved model has 18 bilinear constraint(s)
    Variable types: 35 continuous, 0 integer (0 binary)

    Root relaxation: objective 1.810743e+06, 38 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0 1810743.26    0   13          - 1810743.26      -     -    0s
         0     0 865587.763    0   13          - 865587.763      -     -    0s
         0     0 595135.123    0   13          - 595135.123      -     -    0s
         0     0 564193.525    0   15          - 564193.525      -     -    0s
         0     0 556780.563    0   15          - 556780.563      -     -    0s
         0     0 549889.932    0   15          - 549889.932      -     -    0s
         0     0 549889.932    0   18          - 549889.932      -     -    0s
         0     0 549889.932    0   18          - 549889.932      -     -    0s
         0     2 549889.932    0   18          - 549889.932      -     -    0s
    *   47    54               8    404852.41910 488385.774  20.6%   7.4    0s
    H   62    52                    433077.56632 488385.774  12.8%   6.9    0s
    H   65    52                    439182.58928 488385.774  11.2%   6.6    0s

    Cutting planes:
      RLT: 18

    Explored 131 nodes (656 simplex iterations) in 0.10 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 3: 439183 433078 404852

    Optimal solution found (tolerance 1.00e-04)
    Best objective 4.391825892775e+05, best bound 4.392166623994e+05, gap 0.0078%

The Q-formulation for this instance has:

- 16 decision variables.
- 7 linear constraints.
- 15 bilinear constraints.
- a bilinear objective function.

Notice it has fewer decision variables and also fewer bilinear constraints. Now Gurobi was able to find the optimal solution of 439,182.59 USD in less than one second.

---
## Analysis

Let's see the optimal flows found:

### Flow from Sources to Targets
The following table determines the flows from source nodes to target nodes. For example, from source node s2 to target node t1 there is a flow of 966.7$\times 10^{2}$ bbl.

```python
rows = sources.copy()
columns = targets.copy()
s2t_plan = pd.DataFrame(columns=columns, index=rows, data=0.0)

for source, target in ik.keys():
    if (abs(ik[source, target].x) > 1e-6):
        s2t_plan.loc[source, target] = np.round(ik[source, target].x, 1)
s2t_plan
```

      t1
      t2
      t3

      s1
      0.0
      0.0
      0.0

      s2
      966.7
      0.0
      200.0

      s3
      0.0
      0.0
      0.0

### Flow from Pools to Targets
The following table defines the flows from pool nodes to target nodes. For example, from pool node p1 to target node t1 there is a flow of 92.8$\times 10^{2}$ bbl.

```python
rows = pools.copy()
columns = targets.copy()
p2t_plan = pd.DataFrame(columns=columns, index=rows, data=0.0)

for pool, target in jk.keys():
    if (abs(jk[pool, target].x) > 1e-6):
        p2t_plan.loc[pool, target] = np.round(jk[pool, target].x, 1)
p2t_plan
```

      t1
      t2
      t3

      p1
      92.8
      990.6
      0.0

      p2
      1450.0
      0.0
      300.0

### Flow from Sources to Pools
The following table shows the flows from source nodes to pool nodes. For example, from source node s2 to pool node p1  there is a flow of 258.3$\times 10^{2}$ bbl.

```python
rows = sources.copy()
columns = pools.copy()
s2p_plan = pd.DataFrame(columns=columns, index=rows, data=0.0)

for source, pool in p_ij.keys():
    if (abs(p_ij[source, pool].x) > 1e-6):
        flow = p_ij[source, pool].x * p2t_plan.loc[pool,:].sum()
        s2p_plan.loc[source, pool] = np.round(flow, 1)
s2p_plan
```

      p1
      p2

      s1
      325.0
      1750.0

      s2
      258.3
      0.0

      s3
      500.0
      0.0

---
## Conclusions

This notebook showed how easy it is to solve Bilinear Programs using Gurobi. It also highlighted the dramatic difference in performance of alternative formulations when solving challenging problems, such as the Standard Pooling Problem. It is thus of utmost importance to analyze carefully the context of the problem at hand, and to weigh the pros and cons of alternative models.

---

## References

1. Alfaki, M. (2012). Models and solution methods for the pooling problem.
2. Audet, C., Brimberg, J., Hansen, P., Digabel, S. L., & Mladenović, N. (2004). Pooling problem: Alternate formulations and solution methods. Management science, 50(6), 761-776.
3. Dombrowski, J. (2015, June 07). McCormick envelopes. Retrieved from https://optimization.mccormick.northwestern.edu/index.php/McCormick_envelopes
4. Gupte, A., Ahmed, S., Dey, S. S., & Cheon, M. S. (2017). Relaxations and discretizations for the pooling problem. Journal of Global Optimization, 67(3), 631-669.
5. Haverly, C. A. (1978). Studies of the behavior of recursion for the pooling problem. Acm sigmap bulletin, (25), 19-28.
6. Liberti, L. (2008). Introduction to global optimization. Ecole Polytechnique.
7. Zhuang E. (2015, June 06). Spatial branch and bound method. Retrieved from
https://optimization.mccormick.northwestern.edu/index.php/Spatial_branch_and_bound_method

Copyright © 2020 Gurobi Optimization, LLC

```python

```


# CONTROL LIBRARY EXAMPLES

# Preliminaries

```python
from controlSBML.control_extensions.state_space_tf import StateSpaceTF

import control
import numpy as np
import sympy
import matplotlib.pyplot as plt
```

# Predatory Prey

```python

def predprey_rhs(t, x, u, params):
    # Parameter setup
    a = params.get('a', 3.2)
    b = params.get('b', 0.6)
    c = params.get('c', 50.)
    d = params.get('d', 0.56)
    k = params.get('k', 125)
    r = params.get('r', 1.6)

    # Map the states into local variable names
    H = x[0]
    L = x[1]

    # Compute the control action (only allow addition of food)
    u_0 = u if u > 0 else 0

    # Compute the discrete updates
    dH = (r + u_0) * H * (1 - H/k) - (a * H * L)/(c + H)
    dL = b * (a * H *  L)/(c + H) - d * L

    return [dH, dL]
```

```python
io_predprey = control.NonlinearIOSystem(
    predprey_rhs, None, inputs=('u'), outputs=('H', 'L'),
    states=('H', 'L'), name='predprey')
```

```python
# Simulate the system
X0 = [25, 20]                 # Initial H, L
T = np.linspace(0, 70, 500)   # Simulation 70 years of time

# Simulate the system
t, y = control.input_output_response(io_predprey, T, 0, X0)

# Plot the response
plt.figure(1)
plt.plot(t, y[0])
plt.plot(t, y[1])
plt.legend(['Hare', 'Lynx'])
plt.show(block=False)
```

```python
# Create a feedback controller. Step 1: Find an equalibrium point
eqpt = control.find_eqpt(io_predprey, X0, 0)
xeq = eqpt[0]                         # choose the nonzero equilibrium point
lin_predprey = control.linearize(io_predprey, xeq, 0)
```

    /home/ubuntu/controlSBML/ctl/lib/python3.6/site-packages/control/iosys.py:1007: VisibleDeprecationWarning:

    Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray

```python
# Step 2: Construct the controller via pole placement
K = control.place(lin_predprey.A, lin_predprey.B, [-0.1, -0.2])
A, B = lin_predprey.A, lin_predprey.B
C = np.array([[0, 1]])                # regulated output = number of lynxes
kf = -1/(C @ np.linalg.inv(A - B @ K) @ B)
```

```python
# Construct the controller
io_controller = control.NonlinearIOSystem(
  None,
  lambda t, x, u, params: -K @ (u[1:] - xeq) + kf * (u[0] - xeq[1]),
  inputs=('Ld', 'u1', 'u2'), outputs=1, name='control')
```

```python
# Create the closed loop system
io_closed = control.interconnect(
  [io_predprey, io_controller],       # systems
  connections=[
    ['predprey.u', 'control.y[0]'],
    ['control.u1',  'predprey.H'],
    ['control.u2',  'predprey.L']
  ],
  inplist=['control.Ld'],
  outlist=['predprey.H', 'predprey.L', 'control.y[0]']
)
```

```python
# Simulate the system
t, y = control.input_output_response(io_closed, T, 30, [15, 20])

# Plot the response
plt.figure(2)
plt.subplot(2, 1, 1)
plt.plot(t, y[0])
plt.plot(t, y[1])
plt.legend(['Hare', 'Lynx'])
plt.subplot(2, 1, 2)
plt.plot(t, y[2])
plt.legend(['input'])
plt.show(block=False)
```

# Vertial Takeoff and Landing: TF + Bode plots

```python
# pvtol-nested.py - inner/outer design for vectored thrust aircraft
# RMM, 5 Sep 09
#
# This file works through a fairly complicated control design and
# analysis, corresponding to the planar vertical takeoff and landing
# (PVTOL) aircraft in Astrom and Murray, Chapter 11.  It is intended
# to demonstrate the basic functionality of the python-control
# package.
#

import os
import matplotlib.pyplot as plt  # MATLAB plotting functions
from control.matlab import *    # MATLAB-like functions
import numpy as np

# System parameters
m = 4               # mass of aircraft
J = 0.0475          # inertia around pitch axis
r = 0.25            # distance to center of force
g = 9.8             # gravitational constant
c = 0.05            # damping factor (estimated)

# Transfer functions for dynamics
Pi = tf([r], [J, 0, 0])  # inner loop (roll)
Po = tf([1], [m, c, 0])  # outer loop (position)

#
# Inner loop control design
#
# This is the controller for the pitch dynamics.  Goal is to have
# fast response for the pitch dynamics so that we can use this as a
# control for the lateral dynamics
#

# Design a simple lead controller for the system
k, a, b = 200, 2, 50
Ci = k*tf([1, a], [1, b])  # lead compensator
Li = Pi*Ci

# Bode plot for the open loop process
plt.figure(1)
bode(Pi)

# Bode plot for the loop transfer function, with margins
plt.figure(2)
bode(Li)

# Compute out the gain and phase margins
#! Not implemented
# gm, pm, wcg, wcp = margin(Li)

# Compute the sensitivity and complementary sensitivity functions
Si = feedback(1, Li)
Ti = Li*Si

# Check to make sure that the specification is met
plt.figure(3)
gangof4(Pi, Ci)

# Compute out the actual transfer function from u1 to v1 (see L8.2 notes)
# Hi = Ci*(1-m*g*Pi)/(1+Ci*Pi)
Hi = parallel(feedback(Ci, Pi), -m*g*feedback(Ci*Pi, 1))

plt.figure(4)
plt.clf()
plt.subplot(221)
bode(Hi)

# Now design the lateral control system
a, b, K = 0.02, 5, 2
Co = -K*tf([1, 0.3], [1, 10])  # another lead compensator
Lo = -m*g*Po*Co
```

# State Space to Transfer Function

``ss2tf`` does the conversion from state space model to transfer function for a single input, single output system.
Conceptually, this is done by a similarity transformation in which the ${\bf A}$ matrix is restructured into the "controlability" form: (a) the first row has a leading 1; (b) row $n > 1$ has its only non-zero column as $n + 1$. That is, the new state variables are $x_{n+1} = \dot{x}_n$. If the matrix is singular, then some rows are zero. From this, it is easy to construct a transfer function.

```python
A_mat = [[1., -2], [3, -4]]
B_mat = np.identity(2)
C_mat = np.identity(2)
D_mat = 0*np.identity(2)
sys_mimo = StateSpace(A_mat, B_mat, C_mat, D_mat)
sys1 = ss2tf(sys_mimo)
sys1
```

$$\begin{bmatrix}\frac{s + 4}{s^2 + 3 s + 2}&\frac{-2}{s^2 + 3 s + 2}\\\frac{3}{s^2 + 3 s + 2}&\frac{s - 1}{s^2 + 3 s + 2}\\ \end{bmatrix}$$

```python
s = sympy.symbols("s")
```

```python
#A_mat = np.random.randint(0, 10, (size, size))
size= np.shape(A_mat)[0]
smat = sympy.eye(size) *s - sympy.Matrix(A_mat)
smat.inv()
```

$\displaystyle \left[\begin{matrix}\frac{3.0 s + 12.0}{3.0 s^{2} + 9.0 s + 6.0} & - \frac{6.0}{3.0 s^{2} + 9.0 s + 6.0}\\- \frac{3}{- s^{2} - 3.0 s - 2.0} & \frac{1.0 s - 1.0}{1.0 s^{2} + 3.0 s + 2.0}\end{matrix}\right]$

```python
ss_tf = StateSpaceTF(sys_mimo)
print(ss_tf)
```

    (input, output)

    (1, 1):      s + 4
                -------------
                s^2 + 3 s + 2

    (1, 2):       -2
                -------------
                s^2 + 3 s + 2

    (2, 1):        3
                -------------
                s^2 + 3 s + 2

    (2, 2):      s - 1
                -------------
                s^2 + 3 s + 2

```python
ss_tf.plotBode(legend_crd=(1.2, 1))
```

```python
func11 = lambda s: 20*np.log10(np.abs( (s+4)/(s**2 + 3*s + 2)))
func12 = lambda s: 20*np.log10(np.abs( (-2)/(s**2 + 3*s + 2)))
func21 = lambda s: 20*np.log10(np.abs( (3)/(s**2 + 3*s + 2)))
func22 = lambda s: 20*np.log10(np.abs( (s - 1)/(s**2 + 3*s + 2)))
for f in [func11, func12, func21, func22]:
    print(f(0.1))
```

    4.983437536551822
    -1.2516396845632625
    2.2701854965503623
    -8.187389409056388

```python
_ = control.bode(ss_tf.dataframe.loc["2", "2"])
```

```python
A = [[1., -2], [3, -4]]
B = [[5.], [7]]
C = [[6., 8]]
D = [[9.]]
sys2 = ss2tf(A, B, C, D)
sys2
```

$$\frac{9 s^2 + 113 s + 118}{s^2 + 3 s + 2}$$

```python
sys = control.StateSpace(A, B, C, D)
sys
```

\[
\left(\begin{array}{rllrll|rll}
1\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&-2\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&5\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}\\
3\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&-4\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&7\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}\\
\hline
6\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&8\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}&9\phantom{.}&\hspace{-1em}&\hspace{-1em}\phantom{\cdot}\\
\end{array}\right)
\]

```python
_ = bode(sys[0, 0])
```

```python
tf = ss2tf(sys)
```

One description of a MIMO system is the matrix of input, output transfer functions. Another is to diagonlize the system.

```python
tf.frequency_response(np.pi)

```

    (array([29.01021596]), array([-0.77771055]), array([3.14159265]))

```python
mag, phase, angle = control.freqresp(sys1, [0.1*v for v in range(1, 101)])
plt.plot(np.log10(angle), 20*np.log10(mag[0][0,:]))
plt.plot(np.log10(angle), 20*np.log10(mag[0][1,:]))
```

```python
plt.plot(angle, mag[0][1,:])
```

```python
sys = control.TransferFunction(tf[0, 0])
ax = plt.subplots(1)
mag, phase, angle = bode(sys, [0.1*v for v in range(1, 10000)])
```

```python
20*np.log10(mag)
```

    array([35.39606105, 35.33435066, 35.235321  , ..., 19.08539952,
           19.08539941, 19.0853993 ])

```python
angle
```

    array([1.000e-01, 2.000e-01, 3.000e-01, ..., 9.997e+02, 9.998e+02,
           9.999e+02])


# Difference Target Propagation
---
In this notebook, we will use an interesting neural network that learns via **target propagation** to classify images from the CIFAR-10 database. Note that we provide a number of instructions (including code and descriptions) for the ease of your learning. Be free to change the provided code if you want but in this case you should explain your motivation in the submitted report.

Our aim is to reproduce the results of the paper entitled "Difference Target Propagation" （ https://arxiv.org/abs/1412.7525 ） on the CIFAR-10 database. Before completing this work, you are required to carefully read the paper and understand its basic idea about the proposed learning strategy.

Figure 1. The Schematic overview of the target propagation

### Test for CUDA

Since these are larger (32x32x3) images, it may prove useful to speed up your training time by using a GPU. CUDA is a parallel computing platform and CUDA Tensors are the same as typical Tensors, only they utilize GPU's for computation.

```python
import torch
import numpy as np

# check if CUDA is available
train_on_gpu = torch.cuda.is_available()
device=torch.device('cuda' if torch.cuda.is_available() else 'cpu')
if not train_on_gpu:
    print('CUDA is not available.  Training on CPU ...')
else:
    print('CUDA is available!  Training on GPU ...')
```

    CUDA is available!  Training on GPU ...

---
## Load the Data

If you're not familiar with the Cifar-10, you may find it useful to look at: http://www.cs.toronto.edu/~kriz/cifar.html .

A copy of the data is also placed on the class website.

#### TODO: Load the data

```python
#############################################################################
# TODO: load the data   #
#############################################################################
import torchvision
import torchvision.transforms as transforms

batch_size=1024

transform=transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])

full_dataset = torchvision.datasets.CIFAR10(
    root='./data',
    train=True,
    download=True,
    transform=transform)
train_length = int(full_dataset.__len__()*0.8)
val_length = full_dataset.__len__() - train_length
train_dataset, val_dataset = torch.utils.data.random_split(full_dataset, [train_length, val_length])
test_dataset = torchvision.datasets.CIFAR10(
    root='./data',
    train=False,
    download=True,
    transform=transform)

trainloader = torch.utils.data.DataLoader(
    train_dataset,
    batch_size=batch_size,
    shuffle=True,
    num_workers=0)
valloader = torch.utils.data.DataLoader(
    val_dataset,
    batch_size=batch_size,
    shuffle=True,
    num_workers=0)
testloader = torch.utils.data.DataLoader(
    test_dataset,
    batch_size=1,
    shuffle=False,
    num_workers=0)
classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
#############################################################################
#                          END OF YOUR CODE               #
#############################################################################
```

    Files already downloaded and verified
    Files already downloaded and verified

---
## Define the Network Architecture

Here, you'll define a neural network named DTPNet, whose architecture resembles multiple layer perceptron (MLP) but adopts target propagation for parameter updating. You may use the following Pytorch functions to build it.

* [Linear transformation layer](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html)
* [Non-linear activations](https://pytorch.org/docs/stable/generated/torch.tanh.html)
* [Gradient computing operation](https://pytorch.org/docs/stable/autograd.html?highlight=torch%20autograd%20grad#torch.autograd.grad)

### TODO:
#### 1)  Define DTPNet: Completing the __init__ function
**Sizes of the hidden layers**: As the MLP takes a vector as input (ignoring the dimension of the batch), we should transform the image to a vector, whose dimension should be 3072 ($ 32\times 32\times 3 $)。 We suggest that the network architecture was 3072-1000-1000-1000-10. In each layer, the network uses the hyperbolic tangent as the activation function. You can also design these hyperparameters on your own.

 HINT: because we will compute the loss and gradient for each layer instead of chain rule, you might want to build a separate computational graph for each layer. Think of how to do this.
#### 2)  Define the forward path of DTPNet: Completing the "forward" function
The forward path involves computing unit values for all layers, that is,

\begin{align}
\text{for } i&=  \text{1 to } M \\
 & h_i \leftarrow f_i(h_{i-1})
\end{align}
where $f_i$ stands for the transformation layer of the DTPNet. Please refer to the original paper for more information.

#### 3)  Define the backward path of DTPNet: Completing the "backward" function
The computational details of the backward path are described in Algorithm 1 of the paper. It involves computing the targets, calculating the loss and calculate the gradients for each of the layers in the neural network in a top-down manner.

To make the code more readable, you need first define the "backward" function will call ``compute_target`` and ``reconstruction`` function.

**Important:** The basic idea of the strategy to update the parameter for each layer are presented in the following section. You may need it for completing this function.

##### 3.1)  Calculate the targets for each layer
For the target of the highest layer, it is computed by
$\hat{\mathbf{h}}_{M-1} \leftarrow \mathbf{h}_{M-1} - \hat{\eta} \frac {\partial L} {\partial \mathbf{h}_{M-1}}$, \; ($L$ is the global loss)

For the targets of the lower layers, they are computed by
\begin{align}
\text{for } i&=  M-1 \text{ to } 2 \\
& \hat{\mathbf{h}}_{i-1} \leftarrow \mathbf{h}_{i-1} - g_i(\mathbf{h}_{i}) +  g_i(\hat{\mathbf{h}}_{i})
\end{align}

##### 3.2) Implement the reconstruction function
The reconstruction function involves $g_i(f_i(\mathbf{h}_{i-1}))$.

```python
import torch
import torch.nn as nn
import torch.optim as optim

#Define a MLP
class DTPNet(nn.Module):
    def __init__(self,hidden_sizes=None):
        super(DTPNet, self).__init__()
        # construct the network
        #############################################################################
        self.num_layers=len(hidden_sizes)-1
        self.criterion = torch.nn.MSELoss()

        self.forward_model=dict()
        self.forward_optim=dict()
        for i in range(self.num_layers):
            self.forward_model['F{}'.format(i+1)]=nn.Linear(in_features=hidden_sizes[i],out_features=hidden_sizes[i+1]).to(device)
            self.forward_optim['F{}'.format(i+1)]=optim.RMSprop(params=self.forward_model['F{}'.format(i+1)].parameters(), lr=0.01)

        self.backward_model=dict()
        self.backward_optim=dict()
        for i in range(1,self.num_layers-1):
            self.backward_model['G{}'.format(i+1)]=nn.Linear(in_features=hidden_sizes[i+1],out_features=hidden_sizes[i]).to(device)
            self.backward_optim['G{}'.format(i+1)]=optim.RMSprop(params=self.backward_model['G{}'.format(i+1)].parameters(), lr=0.01)
        #############################################################################

    def forward(self, x):
        #Input: a batch of images, the size is [batchsize, 3072]
        #Output:the values of each layer in the network
        values = dict()
        #############################################################################
        for i,k in enumerate(self.forward_model):
            if i==self.num_layers-1:
                x=torch.softmax(self.forward_model[k](x),dim=-1)
            else:
                x=torch.tanh(self.forward_model[k](x))
            values['H{}'.format(i+1)]=x
        #############################################################################
        return values

    #以下函数定义有修改，详见实验报告
    def backward(self, values, global_loss):
        targets=self.compute_target(values, global_loss)

        loss_inv=self.compute_loss_inv(values)
        loss=self.compute_loss(values,targets,global_loss)

        for key in loss_inv.keys():
            grad=torch.autograd.grad(loss_inv[key], self.backward_model[key].parameters(), retain_graph = True)
            self.backward_model[key].weight.grad=grad[0]
            self.backward_model[key].bias.grad=grad[1]
        for key in loss.keys():
            grad=torch.autograd.grad(loss[key], self.forward_model[key].parameters(), retain_graph = True)
            self.forward_model[key].weight.grad=grad[0]
            self.forward_model[key].bias.grad=grad[1]

        for key in loss_inv.keys():
            self.backward_optim[key].step()
        for key in loss.keys():
            self.forward_optim[key].step()

    def compute_target(self, values, global_loss):
       #Input: values=[value_1,value_2,...,value_N]: the values of each layer (totally N layers) in the network
        #Output: loss=[target_1,target_2,...,target_N]: the targets of each layer (totally N layers) in the network
        targets  = dict()
        lr0=0.327736332653
        targets['H{}_'.format(self.num_layers-1)]=values['H{}'.format(self.num_layers-1)]-lr0*torch.autograd.grad(global_loss, values['H{}'.format(self.num_layers-1)], retain_graph = True)[0]
        for i in range(self.num_layers-1,1,-1):
            targets['H{}_'.format(i-1)]=values['H{}'.format(i-1)]-self.backward_model['G{}'.format(i)](values['H{}'.format(i)])+self.backward_model['G{}'.format(i)](targets['H{}_'.format(i)])

        return targets

    def compute_loss(self, values, targets, global_loss):
        loss=dict()

        for i in range(self.num_layers-1):
            loss['F{}'.format(i+1)]=self.criterion(values['H{}'.format(i+1)],targets['H{}_'.format(i+1)])
        loss['F{}'.format(self.num_layers)]=global_loss
        return loss

    def compute_loss_inv(self, values):
        #Input: values=[value_1,value_2,...,value_N]: the values of each layer (totally N layers) in the network
        #output: loss=[value_2,...,value_N-1]: the reconstructed values of each layer (totally N layers) in the network
        loss_inv = dict()
        c=0.359829566008
        for i in range(self.num_layers-1,1,-1):
            temp=torch.randn(values['H{}'.format(i-1)].shape).to(device)*(c**1/2)+values['H{}'.format(i-1)]
            loss_inv['G{}'.format(i)]=self.criterion(self.backward_model['G{}'.format(i)](self.forward_model['F{}'.format(i)](temp)),temp)

        return loss_inv

    def set_train(self):
        for key in self.forward_model.keys():
            self.forward_model[key].train()
        for key in self.backward_model.keys():
            self.backward_model[key].train()

    def set_eval(self):
        for key in self.forward_model.keys():
            self.forward_model[key].eval()
        for key in self.backward_model.keys():
            self.backward_model[key].eval()

model = DTPNet([3072,1000,1000,1000,10])
print('forward_model:',model.forward_model)#用字典定义模型层，打印模型层
print('backward_model:',model.backward_model)#用字典定义模型层，打印模型层
```

    forward_model: {'F1': Linear(in_features=3072, out_features=1000, bias=True), 'F2': Linear(in_features=1000, out_features=1000, bias=True), 'F3': Linear(in_features=1000, out_features=1000, bias=True), 'F4': Linear(in_features=1000, out_features=10, bias=True)}
    backward_model: {'G2': Linear(in_features=1000, out_features=1000, bias=True), 'G3': Linear(in_features=1000, out_features=1000, bias=True)}

### Specify [Loss Function](http://pytorch.org/docs/stable/nn.html#loss-functions)

Before the training process, you need to first specify your the loss function. For example, we set the negative log-likelihood as the the global loss of the image classification task.

#### TODO: Define the loss

```python
#############################################################################
# TODO: define the loss function           #
#############################################################################
global_loss_fn=torch.nn.NLLLoss()
#############################################################################
#                          END OF YOUR CODE               #
#############################################################################
```

---
## Train the Network
The details of the training phrase of DTPNet can be found in Algorithm 1 of the paper, which is also posted here for your convenience.  Please note that the training process of DTPNet is dramatically different from the traditional neural network. The main difference is that back-propagation uses chain rule to update parameters and the DTP calculates the targets and updates parameters layer by layer. Thus, so you can use the "detach" function in Pytorch to truncate the gradients to avoid auto-differetiation.

Hint: Instead using the backpropagation algorithm, we calculate the loss and then calculate the gradients for each layer, then we can use the gradients for each layer to update the parameters for each layer.
## Examples to show how to update parameters of a single layer by Pytorch
There are two approaches to derive the gradients for the parameters of a neural network, namely, the optimizer module and autograd.grad() function. You can choose one of them or both of them to accomplish this work. Below, we will elaborate them in detail.
### 1)  Optimizer
The ``optimizer`` module controls the parameter updating for the neural network in Pytorch. We take a single layer of the multiple layer perceptron (MLP) as an example, which is defined by
```
single_layer = torch.nn.Linear(2,3)
```
We can build an optimizer via Pytorch.
```
optimizer = torch.optim.RMSprop([{'params':single_layer.parameters(), 'lr': 1}])
```
Then we can update the parameters of single_layer by calling the ``step`` function of the optimizer, for example,
```
x = torch.randn(10,2)
y = torch.randn(10,3)
predict = single_layer(x)
loss = ((predict-y)**2).sum()
loss.backward()
optimizer.step()
```

### 2) autograd.grad()
This function provides a way for the users to manually obtain the gradients for each of the parameters.
```
x = torch.randn(10,2)
y = torch.randn(10,3)
predict = MLP(x)
loss = ((predict-y)**2).sum()
grad_weight = torch.autograd.grad(loss, MLP.weight, retain_graph = True)[0]

```

Remember to look at how the training and validation loss decreases over time and print them.

```python
#############################################################################
# TODO: train and validation              #
#############################################################################
for epoch in range(30):
    train_loss = 0.0
    train_acc = 0.0
    train_correct_count = 0
    model.set_train()
    for i, data in enumerate(trainloader, 0):
        inputs, labels = data
        inputs=inputs.to(device)
        inputs=inputs.view([inputs.shape[0],3072])
        labels=labels.to(device)

        values = model(inputs)
        outputs = values['H{}'.format(model.num_layers)]
        train_correct_count += (torch.argmax(outputs,dim=1)==labels).sum().cpu().item()
        global_loss = global_loss_fn(outputs, labels)
        model.backward(values,global_loss)
        train_loss += global_loss.cpu().item()

    train_loss=train_loss/(i+1)
    train_acc=train_correct_count/(i+1)/batch_size

    val_loss = 0.0
    val_acc = 0.0
    val_correct_count = 0
    model.set_eval()
    for i, data in enumerate(valloader, 0):
        inputs, labels = data
        inputs=inputs.to(device).view(inputs.shape[0],3072)
        labels=labels.to(device)

        outputs = model(inputs)['H{}'.format(model.num_layers)]
        val_correct_count += (torch.argmax(outputs,dim=1)==labels).sum().cpu().item()
        global_loss = global_loss_fn(outputs, labels)
        val_loss += global_loss.cpu().item()
    val_loss=val_loss/(i+1)
    val_acc=val_correct_count/(i+1)/batch_size

    print('Epoch %d|Train_loss:%.3f Eval_loss:%.3f Train_acc:%.3f Eval_acc:%.3f'%(epoch+1,train_loss,val_loss,train_acc,val_acc))
#############################################################################
#                          END OF YOUR CODE               #
#############################################################################
```

    Epoch 1|Train_loss:-0.331 Eval_loss:-0.339 Train_acc:0.335 Eval_acc:0.341
    Epoch 2|Train_loss:-0.373 Eval_loss:-0.350 Train_acc:0.378 Eval_acc:0.352
    Epoch 3|Train_loss:-0.384 Eval_loss:-0.355 Train_acc:0.388 Eval_acc:0.356
    Epoch 4|Train_loss:-0.391 Eval_loss:-0.360 Train_acc:0.396 Eval_acc:0.362
    Epoch 5|Train_loss:-0.396 Eval_loss:-0.375 Train_acc:0.402 Eval_acc:0.378
    Epoch 6|Train_loss:-0.400 Eval_loss:-0.375 Train_acc:0.406 Eval_acc:0.375
    Epoch 7|Train_loss:-0.405 Eval_loss:-0.363 Train_acc:0.411 Eval_acc:0.364
    Epoch 8|Train_loss:-0.407 Eval_loss:-0.371 Train_acc:0.413 Eval_acc:0.372
    Epoch 9|Train_loss:-0.409 Eval_loss:-0.381 Train_acc:0.416 Eval_acc:0.383
    Epoch 10|Train_loss:-0.413 Eval_loss:-0.380 Train_acc:0.419 Eval_acc:0.378
    Epoch 11|Train_loss:-0.414 Eval_loss:-0.383 Train_acc:0.421 Eval_acc:0.381
    Epoch 12|Train_loss:-0.417 Eval_loss:-0.376 Train_acc:0.422 Eval_acc:0.374
    Epoch 13|Train_loss:-0.418 Eval_loss:-0.380 Train_acc:0.424 Eval_acc:0.381
    Epoch 14|Train_loss:-0.419 Eval_loss:-0.383 Train_acc:0.426 Eval_acc:0.384
    Epoch 15|Train_loss:-0.423 Eval_loss:-0.384 Train_acc:0.427 Eval_acc:0.383
    Epoch 16|Train_loss:-0.424 Eval_loss:-0.384 Train_acc:0.429 Eval_acc:0.384
    Epoch 17|Train_loss:-0.425 Eval_loss:-0.386 Train_acc:0.431 Eval_acc:0.384
    Epoch 18|Train_loss:-0.427 Eval_loss:-0.387 Train_acc:0.433 Eval_acc:0.385
    Epoch 19|Train_loss:-0.429 Eval_loss:-0.381 Train_acc:0.433 Eval_acc:0.378
    Epoch 20|Train_loss:-0.427 Eval_loss:-0.381 Train_acc:0.434 Eval_acc:0.378
    Epoch 21|Train_loss:-0.427 Eval_loss:-0.380 Train_acc:0.436 Eval_acc:0.378
    Epoch 22|Train_loss:-0.431 Eval_loss:-0.385 Train_acc:0.436 Eval_acc:0.385
    Epoch 23|Train_loss:-0.432 Eval_loss:-0.378 Train_acc:0.437 Eval_acc:0.374
    Epoch 24|Train_loss:-0.433 Eval_loss:-0.388 Train_acc:0.438 Eval_acc:0.385
    Epoch 25|Train_loss:-0.435 Eval_loss:-0.376 Train_acc:0.439 Eval_acc:0.371
    Epoch 26|Train_loss:-0.432 Eval_loss:-0.386 Train_acc:0.440 Eval_acc:0.383
    Epoch 27|Train_loss:-0.435 Eval_loss:-0.385 Train_acc:0.441 Eval_acc:0.382
    Epoch 28|Train_loss:-0.435 Eval_loss:-0.385 Train_acc:0.443 Eval_acc:0.381
    Epoch 29|Train_loss:-0.440 Eval_loss:-0.384 Train_acc:0.443 Eval_acc:0.382
    Epoch 30|Train_loss:-0.437 Eval_loss:-0.384 Train_acc:0.445 Eval_acc:0.381

---
## Test the Trained Network

Test your trained model on previously unseen data and print the test accuracy of each class and the whole! Try your best to get a better accuracy.

```python
#############################################################################
# TODO: test the trained network             #
#############################################################################
class_count=np.zeros(10,dtype=int)
correct_count=np.zeros(10,dtype=int)
model.set_eval()
for i, data in enumerate(testloader, 0):
    inputs, labels = data
    inputs=inputs.to(device)
    inputs=inputs.view([inputs.shape[0],3072])
    outputs = torch.argmax(model(inputs)['H{}'.format(model.num_layers)],dim=1).cpu().item()
    class_count[labels]+=1
    if outputs==labels:
        correct_count[labels]+=1
for i in range(10):
    print('Test|Class{}('.format(i+1)+classes[i]+')-acc={}'.format(correct_count[i]/class_count[i]))
print('\nTest|Overall-acc={}'.format(np.sum(correct_count/10000)))
#############################################################################
#                          END OF YOUR CODE               #
#############################################################################
```

    Test|Class1(plane)-acc=0.48
    Test|Class2(car)-acc=0.453
    Test|Class3(bird)-acc=0.246
    Test|Class4(cat)-acc=0.171
    Test|Class5(deer)-acc=0.311
    Test|Class6(dog)-acc=0.339
    Test|Class7(frog)-acc=0.464
    Test|Class8(horse)-acc=0.403
    Test|Class9(ship)-acc=0.561
    Test|Class10(truck)-acc=0.496

    Test|Overall-acc=0.39239999999999997

### Question: What are your model's weaknesses during your experiment and how might they be improved?
+ 本实验只使用了较为简单的MLP网络，由于网络结构过于简单，最终分类结果较差，仅用作DTP算法是示意。实际上，MLP网络输入时先将图像展开为列向量，这种操作不易获得图像上邻近位置的信息，因而分类效果较差。使用更加复杂的卷积神经网络应当可以达到更好的效果，将卷积神经网络中的一个卷积层或一个卷积块看做一个广义的前向计算函数${f_i}$即可使用DTP算法进行训练。由于任务一中已经验证了ResNet的分类表现，此处不再重复实现。
+ DTP算法中涉及较多的超参数选择，如各层局部参数更新的学习率、随机高斯噪声的功率谱密度以及全局损失函数、优化器的选择。在这些超参数上进行进一步调优可能在相同的网络结构上实现更好的测试表现。


# Exercise 4
* Due: 4.2.2016 at noon
* Max points: 8

## General rules
* Send the source code or the written notes of your answers as an attachment to markus.hartikainen@jyu.fi before the due time
* You will get feedback about your answers a week from the due date
* From the second week on, the exercises will be given on the previous Wednesday's lecture

## Exercises

For exercises 1-3, we study optimization problem
$$
\begin{align}
\min \qquad & x_1^2+x_2^2 + x_1+2x_2\\
\text{s.t.}\qquad &x_1+x_2=1\\
    &x_1,x_2\in\mathbb R
\end{align}
$$

1. **(2 points)** Solve the problem using the penalty function method
2. **(2 points)** Solve the problem (i.e., approximate the optimal solution) using the barrier function method. Note that you need to do something a bit clever.
3. **(2 points)** Solve the problem using projected gradient method.
4. **(2 points)** Check the *necessary first order KKT conditions* for the solution that you found.


Massimo Nocentini

March 2, 2018: compositional inverse

February 26, 2018: splitting from generic nb

Abstract

Ctors for (symbolic) Riordan arrays.

```python
from sympy import *
from sympy.abc import n, i, N, x, lamda, phi, z, j, r, k, a, t, alpha

from sequences import *

init_printing()
```

```python
m = 8

d_fn, h_fn = Function('d'), Function('h')
d, h = IndexedBase('d'), IndexedBase('h')
```

---

```python
from commons import define
```

```python
%run ../../src/commons.py
```

```python
at = define(a,t+1)
at
```

```python
at(return_eq=True)
```

```python
D = define(d_fn(t),sin(t)**2+cos(t)**2)
D
```

```python
D(pi/2, return_eq=False)
```

```python
d1 = D(pi/2, return_eq=True)
d1
```

```python
d1.substitution()
```

```python
callable(d1)
```

    False

```python
s = Subs(t+1,[t,a],[3,4])
s
```

```python
Lambda([],t+1)
```

```python
Subs?
```

---

## By series convolution

```python
d_series = Eq(d_fn(t), sum(d[i]*t**i for i in range(m))).subs({d[0]:1})
h_series = Eq(h_fn(t), t*sum(h[i]*t**i for i in range(m-1))).expand().subs({h[0]:1})

d_series, h_series
```

```python
R = Matrix(m, m, riordan_matrix_by_convolution(m, d_series, h_series))
R
```

```python
production_matrix(R) # too verbose to show
```

```python
d_series = Eq(d_fn(t), 1/(1-t))
h_series = Eq(h_fn(t), t*d_series.rhs)

d_series, h_series
```

```python
R = Matrix(m, m, riordan_matrix_by_convolution(m, d_series, h_series))
R
```

## By recurrence relation

```python
dim = m
a, b, b_bar, c = symbols(r'a b \bar{b} c')
M = Matrix(dim, dim, riordan_matrix_by_recurrence(dim, lambda n, k: {(n-1, k-1):a,
                                                                     (n-1, k): b if k else b_bar,
                                                                     (n-1, k+1):c}))
M
```

```python
production_matrix(M)
```

## By $A, Z$ sequences

### $\mathcal{P}$

```python
A = Function('A')
A_one_t = Eq(A(t), 1 + t)
A_one = Eq(A(t),1)
A_one, A_one_t
```

```python
R = Matrix(m, m, riordan_matrix_by_AZ_sequences(dim, (A_one, A_one_t)))
R
```

```python
production_matrix(R)
```

### $\mathcal{C}$

```python
A = Function('A')
A_ones = Eq(A(t), 1/(1-t))
R = Matrix(m, m, riordan_matrix_by_AZ_sequences(dim, (A_ones, A_ones)))
R
```

```python
production_matrix(R)
```

### $\mathcal{R}$

```python
A = Function('A')
a = IndexedBase('a')
A_gen = Eq(A(t), sum((a[j] if j else 1)*t**j for j in range(dim)))
R = Matrix(m, m, riordan_matrix_by_AZ_sequences(dim, (A_gen, A_gen)))
R
```

```python
production_matrix(R)
```

```python
A, Z = Function('A'), Function('Z')
a, z = IndexedBase('a'), IndexedBase('z')
A_gen = Eq(A(t), sum((a[j] if j else 1)*t**j for j in range(dim)))
Z_gen = Eq(Z(t), sum((z[j] if j else 1)*t**j for j in range(dim)))
R = Matrix(m, m, riordan_matrix_by_AZ_sequences(dim, (Z_gen, A_gen)))
R
```

```python
production_matrix(R)
```

## Compositional inverse

```python
H = Function('h')
C_eq = Eq(H(t), (1-sqrt(1-4*t))/2)
C_eq, compositional_inverse(C_eq)
```

```python
P_eq = Eq(H(t), t/(1-t))
P_eq, compositional_inverse(P_eq), compositional_inverse(compositional_inverse(P_eq), y=t)
```

## Exponential RA

```python
d_series = Eq(d_fn(t), 1)
h_series = Eq(h_fn(t), exp(t)-1)
d_series, h_series # build the triangle of Stirling numbers of the II kind
```

```python
R = Matrix(m, m, riordan_matrix_exponential(
            riordan_matrix_by_convolution(m, d_series, h_series)))
R
```

---

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.


# Tutorial 1: Part 2

Objectives:
- Learn how to define a simple lattice and compute the TWISS functions using MAD-X.
- Thick vs thin lense approximation TWISS comparison for a lattice with only quadrupoles.
- Tune and $\beta$-function dependence on K1.

**My first accelerator: a FODO cell**

1. Make a simple lattice cell with $L_{cell}$= 100 m made of focusing and defocusing qudrupoles (so called FODO cell). Each quadrupole is 5 m long ($L_{quad}$). Put the start of the first qudrupole at the start of the sequence. Each quadrupole has a focal length f=200 m (k1 x $L_{quad}$ = 1/f in thin lense approximation).

2. Define a proton beam at $E_{tot}$= 2 GeV. Activate the sequence and try to find the periodic solution and plot the $\beta$-functions. If you found $\beta_{max}$= 460 m you succeded.

3. Using the plot you obtained can you estimate the phase advance of the cell. Compare with the tunes obtained with the TWISS.

4. Try with $E_{tot}$ = 0.7 GeV: what is the MAD-X error message?

5. Try with f= 20 m: what is the MAD-x error message?

**Matching the FODO cell using a parametric plot**

6. Try to twiss it powering the quds to obtain a $\Delta \mu \approx 90^\circ$ in the cell using the thin lense approximation (using the figures from Tutorial 1: Part 1). What is the actual phase advance computed by MAD-X?

7. What is the $\beta_{mx}$? Compare with the thin lense approximation (using the figures from Tutorial 1: Part 1).

8. Compute the maximum beam size $\sigma$ assuming a normalized emittance of 3 mrad mm and $E_{tot}= 7 TeV$.

9. Halve the focusing strength of the quadrupoles, what is the effect of it on the $\beta_{max}$ and $\beta_{min}$ and on the $\Delta \mu$? Compare with the parametric plots.

```python
#Import the needed libraries
from matplotlib import pyplot as plt
import numpy as np
import pandas as pd
from cpymad.madx import Madx
```

# Launching MAD-X

```python
myMad = Madx(stdout=True)
```

1. Make a simple lattice cell with 𝐿𝑐𝑒𝑙𝑙= 100 m made of focusing and defocusing qudrupoles (so called FODO cell). Each quad is 5 m long (𝐿𝑞𝑢𝑎𝑑). Put the start of the first qudrupole at the start of the sequence. Each quadrupole has a focal length f=200 m (k1 x 𝐿𝑞𝑢𝑎𝑑 = 1/f in thin lense approximation).

2. Define a proton beam at $E_{tot}$= 2 GeV. Activate the sequence and try to find the periodic solution and plot the $\beta$-functions. If you found $\beta_{max} \approx$ 460 m you succeded.

```python
myMad = Madx(stdout=True)
myString='''
! *********************************************************************
! Definition of parameters
! *********************************************************************

l_cell=100;
quadrupoleLenght=5;
f=200;
myK:=1/f/quadrupoleLenght;// m^-2

! *********************************************************************
! Definition of magnets
! *********************************************************************
QF: quadrupole, L=quadrupoleLenght, K1:=myK;
QD: quadrupole, L=quadrupoleLenght, K1:=-myK;

! *********************************************************************
! Definition of sequence
! *********************************************************************
myCell:sequence, refer=entry, L=L_CELL;
quadrupole1: QF, at=0;
marker1: marker, at=25;
quadrupole2: QD, at=50;
endsequence;

! *********************************************************************
! Definition of beam
! *********************************************************************
beam, particle=proton, energy=2;

! *********************************************************************
! Use of the sequence
! *********************************************************************
use, sequence=myCell;

! *********************************************************************
! TWISS
! *********************************************************************
title, 'My first twiss';
twiss, file=/Users/nuria/Desktop/MADX/MyfirstFODO.madx;
plot, haxis=s, vaxis=betx,bety,dx,colour=100, title="test",file=/Users/nuria/Desktop/MADX/MyfirstFODO;
'''
myMad.input(myString);
```

```python
########################
# Using MAD-X commands #
########################

myString='''
value, table(SUMM,Q1);
value, table(SUMM,betymax);
'''
myMad.input(myString);
```

    table( summ q1 )   =      0.03853349451 ;
    table( summ betymax ) =        463.6232883 ;

```python
#########################
# Using python commands #
#########################
# SUMM table
myDF=myMad.table.summ.dframe()
myDF
myDF["q1"]
```

    #e    0.038533
    Name: q1, dtype: float64

```python
#########################
# Using python commands #
#########################
# TWISS table
myDF=myMad.table.twiss.dframe()
myDF[['name','s','betx','bety','alfx','alfy','mux','muy']]
```

      name
      s
      betx
      bety
      alfx
      alfy
      mux
      muy

      #s
      mycell$start:1
      0.0
      463.623288
      369.779162
      -1.156109
      0.929316
      0.000000
      0.000000

      quadrupole1
      quadrupole1:1
      5.0
      463.623288
      369.779162
      1.156109
      -0.929316
      0.001709
      0.002161

      drift_0[0]
      drift_0:0
      25.0
      419.394867
      408.967742
      1.055312
      -1.030113
      0.008930
      0.010350

      marker1
      marker1:1
      25.0
      419.394867
      408.967742
      1.055312
      -1.030113
      0.008930
      0.010350

      drift_1[0]
      drift_1:0
      50.0
      369.779162
      463.623288
      0.929316
      -1.156109
      0.019041
      0.019493

      quadrupole2
      quadrupole2:1
      55.0
      369.779162
      463.623288
      -0.929316
      1.156109
      0.021202
      0.021202

      drift_2[0]
      drift_2:0
      100.0
      463.623288
      369.779162
      -1.156109
      0.929316
      0.038533
      0.038533

      #e
      mycell$end:1
      100.0
      463.623288
      369.779162
      -1.156109
      0.929316
      0.038533
      0.038533

```python
%matplotlib notebook
# Plot
plt.plot(myDF['s'],myDF['betx'],'.-b',label='$\\beta_x$')
plt.plot(myDF['s'],myDF['bety'],'.-r',label='$\\beta_y$')
plt.legend()
plt.grid()
plt.xlabel('s [m]')
plt.ylabel('[m]')
plt.title('My first FODO cell')
```

    <IPython.core.display.Javascript object>

    Text(0.5, 1.0, 'My first FODO cell')

If you found $\beta_{max}$= 463.6 m you succeded!

# Tune from the plot
3. Using the plot you obtained can you estimate the phase advance of the cell. Compare with the tunes obtained with the TWISS.

For the phase advance one can consider the definition

\begin{equation}
\mu=\int\frac{1}{\beta(s)}ds.
\end{equation}

Remember that the unit of phase in MAD-X is [2$\pi$].

```python
# a very basic approximation
1/400.*100/2/np.pi
```

    0.039788735772973836

```python
# computing the integral
np.trapz(1/myDF['betx'],myDF['s'])/2/np.pi
```

    0.03862786461917964

```python
# correct values from MAD-X TWISS commands
myDF.iloc[-1]['mux']
```

    0.03853349450910022

# Varying the energy
4. Try with $E_{tot}$ = 0.7 GeV: what is the MAD-X error message?

```python
# with this simple wrapper in case of error the code enter in an infinite loop you have to stop manually.
myString='''
beam, particle=proton, energy=0.7;
title, 'My third twiss';
twiss, file=./secondpart/thirdTwiss.txt;
'''
myMad.input(myString);
```

There is an error due to the fact that the total energy is lower than the rest proton mass.

# With f=20 m
5. Try with f= 20 m: what is the MAD-x error message?

```python
myMad = Madx(stdout=True)
```

      ++++++++++++++++++++++++++++++++++++++++++++
      +     MAD-X 5.07.00  (64 bit, Darwin)      +
      + Support: mad@cern.ch, http://cern.ch/mad +
      + Release   date: 2021.05.03               +
      + Execution date: 2021.12.03 18:21:07      +
      ++++++++++++++++++++++++++++++++++++++++++++

```python
myString='''
! *********************************************************************
! Definition of parameters
! *********************************************************************

l_cell=100;
quadrupoleLenght=5;
f=20;
myK:=1/f/quadrupoleLenght;// m^-2

! *********************************************************************
! Definition of magnet
! *********************************************************************
QF: quadrupole, L=quadrupoleLenght, K1:=myK;
QD: quadrupole, L=quadrupoleLenght, K1:=-myK;

! *********************************************************************
! Definition of sequence
! *********************************************************************
myCell:sequence, refer=entry, L=L_CELL;
quadrupole1: QF, at=0;
marker1: marker, at=25;
quadrupole2: QD, at=50;
endsequence;

! *********************************************************************
! Definition of beam
! *********************************************************************
beam, particle=proton, energy=2;

! *********************************************************************
! Use of the sequence
! *********************************************************************
use, sequence=myCell;

! *********************************************************************
! TWISS
! *********************************************************************
select, flag=twiss, clear;
select, flag=twiss, column=name, keyword, s, betx, alfx, mux, bety, alfy, muy, x, px, y, py, dx, dy, dx, dpx, dy, dpy;
twiss, file=f20.txt;
'''
myMad.input(myString);
```

    enter Twiss module

    iteration:   1 error:   0.000000E+00 deltap:   0.000000E+00
    orbit:   0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00
    ++++++ warning: TWCLORB:  Horizontal plane might be unstable More information with the debug flag on.
    ++++++ warning: TWCLORB:  Vertical plane might be unstable More information with the debug flag on.
    ++++++ warning: TWCPIN:  Both modes are unstable for delta(p)/p =     0.000000: cosmux =    -1.914796, cosmuy =    -1.914796
    ++++++ warning: Twiss failed:  MAD-X continues

**INTERPRETATION**: The cell is unstable due to the fact that the focal length is too short. Please note the value of the cosmux and cosmuy. **REMEMBER** |Trace(M)|< 2; -1 <= cos $\mu$ <= 1

# **Matching the FODO cell using a parametric plot**
6. Try to twiss it powering the quadrupoles to obtain a $\Delta \mu \approx 90^\circ$ in the cell using the thin lense approximation (using the figures from Tutorial 1: Part 1). What is the actual phase advance computed by MAD-X?

```python
myMad = Madx(stdout=True)
```

      ++++++++++++++++++++++++++++++++++++++++++++
      +     MAD-X 5.07.00  (64 bit, Darwin)      +
      + Support: mad@cern.ch, http://cern.ch/mad +
      + Release   date: 2021.05.03               +
      + Execution date: 2021.12.03 18:21:15      +
      ++++++++++++++++++++++++++++++++++++++++++++

```python
# From the plot from Tutoria 1 - part2
quadrupoleLenght=5
cellLength=100
myK=2.8/cellLength/quadrupoleLenght
print(myK)
```

    0.005599999999999999

```python
myString='''
! Definition of parameters
quadrupoleLenght=5;
cellLength=100;
myK=2.8/cellLength/quadrupoleLenght;// m^-2

! Definition of magnet
QF: quadrupole, L=quadrupoleLenght, K1:=myK;
QD: quadrupole, L=quadrupoleLenght, K1:=-myK;

! Definition of sequence
myCell:sequence, refer=entry, L=cellLength;
q1: QF, at=0;
q2: QD, at=cellLength/2;
endsequence;

! Definition of beam
beam, particle=proton, energy=7000;
use, sequence=myCell;

! TWISS
select, flag=twiss, clear;
select, flag=twiss, column=name, keyword, s, betx, alfx, mux, bety, alfy, muy;
twiss;
value, myK;
'''
myMad.input(myString);
```

    enter Twiss module

    iteration:   1 error:   0.000000E+00 deltap:   0.000000E+00
    orbit:   0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00

    ++++++ table: summ

                length             orbit5               alfa            gammatr
                   100                 -0                  0                  0

                    q1                dq1            betxmax              dxmax
          0.2363728498      -0.2921003657        160.6036546                  0

                 dxrms             xcomax             xcorms                 q2
                     0                  0                  0       0.2363728498

                   dq2            betymax              dymax              dyrms
         -0.2921003657        160.6036546                 -0                  0

                ycomax             ycorms             deltap            synch_1
                     0                  0                  0                  0

               synch_2            synch_3            synch_4            synch_5
                     0                  0                  0                  0

               synch_6            synch_8             nflips
                     0                  0                  0
    myk                =             0.0056 ;

```python
myDFTable=myMad.table.twiss.dframe()
myDFTable[["name", "keyword","betx","bety","alfx","alfy", "mux", "muy" ]]
```

      name
      keyword
      betx
      bety
      alfx
      alfy
      mux
      muy

      #s
      mycell$start:1
      marker
      160.603655
      34.217492
      -2.259847
      0.548735
      0.000000
      0.000000

      q1
      q1:1
      quadrupole
      160.603655
      34.217492
      2.259847
      -0.548735
      0.004841
      0.023892

      drift_0[0]
      drift_0:0
      drift
      34.217492
      160.603655
      0.548735
      -2.259847
      0.108661
      0.127712

      q2
      q2:1
      quadrupole
      34.217492
      160.603655
      -0.548735
      2.259847
      0.132553
      0.132553

      drift_1[0]
      drift_1:0
      drift
      160.603655
      34.217492
      -2.259847
      0.548735
      0.236373
      0.236373

      #e
      mycell$end:1
      marker
      160.603655
      34.217492
      -2.259847
      0.548735
      0.236373
      0.236373

```python
myString='''
value, table(SUMM,Q1);
value, table(SUMM,Q2);
'''
myMad.input(myString);
```

    table( summ q1 )   =       0.2363728498 ;
    table( summ q2 )   =       0.2363728498 ;

```python
#rad
0.236*2*np.pi
```

    1.4828317324943823

```python
#deg
1.4828317324943823*180/np.pi
```

    84.96

7. What is the $\beta_{max}$? Compare with the thin lense approximation (using the figures from Tutorial 1: Part 1).

```python
# From the MAD-X calculation
myDFTable['betx'].max()
```

    160.60365457633432

```python
myDFTable['bety'].max()
```

    160.6036545763342

```python
#From the parametric plot
#K1*Lcell*Lq
0.0056*100*5
```

    2.8

```python
1.697*100
```

    169.70000000000002

8. Compute the maximum beam size $\sigma$ assuming a normalized emittance of 3 mrad mm and $E_{tot}= 7 TeV$.

One has to remember
\begin{equation} \sigma=\sqrt{\frac{\beta \epsilon_n}{\beta_r \gamma_r}} \end{equation}

```python
import numpy as np
emittance_n=3e-6 #m*rad
beta_gamma=7000/.938 # this is an approximation
np.sqrt(myDFTable['betx'].max()*emittance_n/beta_gamma)
```

    0.00025409185177743577

9. Halve the focusing strength of the quadrupoles, what is the effect of it on the $\beta_{max}$ and $\beta_{min}$ and on the $\Delta \mu$? Compare with the parametric plots.

```python
myString='''
myK=1.4/cellLength/quadrupoleLenght;// m^-2
twiss, file=firstTwiss.txt;
'''
myMad.input(myString);
```

    ++++++ info: myk redefined
    enter Twiss module

    iteration:   1 error:   0.000000E+00 deltap:   0.000000E+00
    orbit:   0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00

    ++++++ table: summ

                length             orbit5               alfa            gammatr
                   100                 -0                  0                  0

                    q1                dq1            betxmax              dxmax
          0.1097922006      -0.1143581814        205.4604013                  0

                 dxrms             xcomax             xcorms                 q2
                     0                  0                  0       0.1097922006

                   dq2            betymax              dymax              dyrms
         -0.1143581814        205.4604013                  0                  0

                ycomax             ycorms             deltap            synch_1
                     0                  0                  0                  0

               synch_2            synch_3            synch_4            synch_5
                     0                  0                  0                  0

               synch_6            synch_8             nflips
                     0                  0                  0

```python
myString='''
value, table(SUMM,Q1);
value, table(SUMM,Q2);
value, table(SUMM,betxmax);
value, table(SUMM,betymax);
'''
myMad.input(myString);
```

    table( summ q1 )   =       0.1097922006 ;
    table( summ q2 )   =       0.1097922006 ;
    table( summ betxmax ) =        205.4604013 ;
    table( summ betymax ) =        205.4604013 ;

```python
myDFTable=myMad.table.twiss.dframe()
myDFTable.head()
```

      name
      keyword
      s
      betx
      alfx
      mux
      bety
      alfy
      muy
      x
      ...
      sig54
      sig55
      sig56
      sig61
      sig62
      sig63
      sig64
      sig65
      sig66
      n1

      #s
      mycell$start:1
      marker
      0.0
      205.460401
      -1.434644
      0.000000
      106.483860
      0.764835
      0.000000
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      q1
      q1:1
      quadrupole
      5.0
      205.460401
      1.434644
      0.003828
      106.483860
      -0.764835
      0.007564
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      drift_0[0]
      drift_0:0
      drift
      50.0
      106.483860
      0.764835
      0.053029
      205.460401
      -1.434644
      0.056764
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      q2
      q2:1
      quadrupole
      55.0
      106.483860
      -0.764835
      0.060592
      205.460401
      1.434644
      0.060592
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      drift_1[0]
      drift_1:0
      drift
      100.0
      205.460401
      -1.434644
      0.109792
      106.483860
      0.764835
      0.109792
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

5 rows × 256 columns

```python
# From the plot
#K1*Lcell*Lq
0.0028*100*5
```

    1.4

```python
#Max. betax
2.042*100
```

    204.2

Better agreement is observed as we move to the left on the parametric plot (smaller K or smaller Lq for a fixed value of the cell length) as the thin lense approximation condition is better satisfied.

# EXTRA

# Adding markers

This is an example to add markers in the sequence using a macros.

```python
myString='''
! *********************************************************************
! Tutorial 1, second part
! *********************************************************************

! *********************************************************************
! Definition of parameters
! *********************************************************************

option, echo=false, info=false, warn=false;
l_cell=100;
quadrupoleLenght=5;
f=200;
myK:=1/f/quadrupoleLenght;// m^-2

! *********************************************************************
! Definition of magnet
! *********************************************************************
QF: quadrupole, L=quadrupoleLenght, K1:=myK;
QD: quadrupole, L=quadrupoleLenght, K1:=-myK;

installMarkers(nn): macro ={
markernn: marker, at=nn;
!value,f;
};
N=6;

! *********************************************************************
! Definition of sequence
! *********************************************************************
myCell:sequence, REFER=centre, L=L_CELL;
quadrupole1: QF, at=2.5;
while (N<50) {
exec, installMarkers($N);
N=N+1;
}
quadrupole2: QD, at=52.5;
N=56;
while (N<100) {
exec, installMarkers($N);
N=N+1;
}
endsequence;

! *********************************************************************
! Definition of beam
! *********************************************************************
beam, particle=proton, energy=2;

! *********************************************************************
! Use of the sequence
! *********************************************************************
use, sequence=myCell;

! *********************************************************************
! TWISS
! *********************************************************************
select, flag=twiss, clear;
select, flag=twiss, column=name, keyword, L, s, betx, alfx, mux, bety, alfy, muy, x, px, y, py, dx, dy, dx, dpx, dy, dpy;

title, 'My fourth twiss';
twiss, file=./secondpart/fifthTwiss.txt;
'''
myMad.input(myString);
```

```python
myDF=myMad.table.twiss.dframe()
myDF.head()
```

```python
plt.plot(myDF['s'],myDF['betx'],'.-b',label='$\\beta_x$')
plt.plot(myDF['s'],myDF['bety'],'.-r',label='$\\beta_y$')
plt.xlabel('s [m]')
plt.ylabel('[m]')
plt.legend(loc='best')
plt.grid()
```

```python
np.trapz(1/myDF['betx'],myDF['s'])/2/np.pi
```

# Thin lense approximation

```python
myString='''
select, flag=makethin, class=quadrupole, slice=5;
makethin,sequence=myCell;
use,sequence=myCell;
twiss,sequence=myCell,file=./secondpart/sixthTwiss.txt;
'''
myMad.input(myString);
```

```python
myDF=myMad.table.twiss.dframe()
myDF
```

```python
plt.plot(myDF['s'],myDF['betx'],'.-b',label='$\\beta_x$')
plt.plot(myDF['s'],myDF['bety'],'.-r',label='$\\beta_y$')
plt.xlabel('s [m]')
plt.ylabel('[m]')
plt.legend(loc='best')
plt.grid()
```

```python
np.trapz(1/myDF['betx'],myDF['s'])/2/np.pi
```


# Basic Operations

```python
from sympy import *
init_printing()
```

For each exercise, fill in the function according to its docstring.

## Substitution

Write a function that takes a list of expressions, a variable, and a point, and evaluates each expression in the list at that point.

```python
def evaluate(exprs, x, x0):
    """
    Evaluate each expression in exprs at the point x = x0.

    >>> x, y = symbols('x y')
    >>> exprs = [x**2, cos(x), x*y]
    >>> evaluate(exprs, x, 1)
    [1, cos(1), y]
    >>> evaluate(exprs, y, 0)
    [x**2, cos(x), 0]
    """
    ??? <- Write the answer here
```

```python
x, y = symbols('x y')
```

```python
exprs = [x**2, cos(x), x*y]
```

```python
evaluate(exprs, x, 1)
```

```python
evaluate(exprs, y, 0)
```

Write a function that computes

$$
{\underbrace{x^{{}^{.^{.^{.^x}}}}}_\text{n copies of x}}
$$

That is, `x**(x**(...x))`, with `n` copies of `x`.  In [Knuth up-arrow notation](http://en.wikipedia.org/wiki/Up_arrow_notation), $x\uparrow\uparrow n$.

```python
def uparrow(x, n):
    """
    Computes x**(x**(...x)), with n copies of x.

    >>> x = symbols('x')
    >>> uparrow(x, 3)
    x**(x**x)
    >>> uparrow(x, 1)
    x
    >>> uparrow(x**x, 3)
    (x**x)**((x**x)**(x**x))
    """

```

```python
x = symbols('x')
```

```python
uparrow(x, 3)
```

```python
uparrow(x, 1)
```

```python
uparrow(x**x, 3)
```

Write a function that takes a function and nests it within itself n times.

For example, if we started with $x^x$, and $n=3$, we would end up with

$$\left(\left(x^{x}\right)^{\left(x^{x}\right)}\right)^\left({\left(x^{x}\right)^{\left(x^{x}\right)}}\right)$$

```python
def nest(expr, x, n):
    """
    Nests expr into itself (in the variable x) n times.

    >>> x, y = symbols('x y')
    >>> nest(x**x, x, 3)
    ((x**x)**(x**x))**((x**x)**(x**x))
    >>> nest(sin(x)*cos(y), x, 2)
    sin(sin(x)*cos(y))*cos(y)
    >>> nest(sin(x)*cos(y), y, 2)
    sin(x)*cos(sin(x)*cos(y))
    >>> nest(x**2, x, 1)
    x**2
    """

```

```python
x, y = symbols('x y')
```

```python
nest(x**x, x, 3)
```

```python
nest(sin(x)*cos(y), x, 2)
```

```python
nest(sin(x)*cos(y), y, 2)
```

```python
nest(x**2, x, 1)
```

Write a function that replaces all trig functions in terms of $\sin(x)$ and $\cos(x)$. You can assume that the trig function will be of the form $t(x)$, where $x$ is the variable.  The trig functions implemented in SymPy are

$$\tan(x) = \frac{\sin(x)}{\cos(x)}$$
$$\cot(x) = \frac{\cos(x)}{\sin(x)}$$
$$\sec(x) = \frac{1}{\cos(x)}$$
$$\csc(x) = \frac{1}{\sin(x)}$$

```python
def trig_rewrite(expr, x):
    """
    Rewrite all trig functions t(x) in terms of sin(x) and cos(x)

    >>> x, y = symbols('x y')
    >>> trig_rewrite(tan(x), x)
    sin(x)/cos(x)
    >>> trig_rewrite(tan(x) + cos(x)*sec(x), x)
    sin(x)/cos(x) + 1
    >>> trig_rewrite(cot(x) + sin(x)*csc(x), x)
    1 + cos(x)/sin(x)
    >>> trig_rewrite(tan(x)*tan(y), x)
    sin(x)*tan(y)/cos(x)
    >>> trig_rewrite(tan(x)*tan(y), y)
    sin(y)*tan(x)/cos(y)
    """

```

```python
x, y = symbols('x y')
```

```python
trig_rewrite(tan(x), x)
```

```python
trig_rewrite(tan(x) + cos(x)*sec(x), x)
```

```python
trig_rewrite(cot(x) + sin(x)*csc(x), x)
```

```python
trig_rewrite(tan(x)*tan(y), x)
```

```python
trig_rewrite(tan(x)*tan(y), y)
```

Suppose we are working with the series expansion of a function at $x=0$, like $a_0 + a_1x + a_2x^2 + \cdots$.  In this expansion, the terms with higher powers are less significant to the calculation.  For example, if we had

$$1 + x + \frac{x^{2}}{2} + \frac{x^{3}}{6} + \frac{x^{4}}{24} + \frac{x^{5}}{120} + \frac{x^{6}}{720} + \frac{x^{7}}{5040} + \frac{x^{8}}{40320} + \frac{x^{9}}{362880}$$

We might only care about the terms with powers less than 5

$$1 + x + \frac{x^{2}}{2} + \frac{x^{3}}{6} + \frac{x^{4}}{24}$$

We will see later that this is can be done automatically using the `O` class, but it can also be done using `subs`.

```python
def series_reduce(expr, x, p):
    """
    Remove all powers of x in expr with power greater than p.

    You may assume that there are no powers of x greater than 10.

    Bonus: which functions are represented by the series expansions below (you
    can use expr.series(x, 0, 10) to check if you are right)?

    >>> x, y = symbols('x y')
    >>> series_reduce(1 - x**2/2 + x**4/24 - x**6/720 + x**8/40320, x, 5)
    x**4/24 - x**2/2 + 1
    >>> series_reduce(1 + x + x**2 + x**3 + x**4 + x**5 + x**6 + x**7 + x**8 + x**9 + x**10, x, 0)
    1
    >>> series_reduce(x*y + x**3*y**3/3 + 2*x**5*y**5/15 + 17*x**7*y**7/315 + 62*x**9*y**9/2835, x, 5)
    2*x**5*y**5/15 + x**3*y**3/3 + x*y
    """

```

```python
x, y = symbols('x y')
```

```python
series_reduce(1 - x**2/2 + x**4/24 - x**6/720 + x**8/40320, x, 5)
```

```python
series_reduce(1 + x + x**2 + x**3 + x**4 + x**5 + x**6 + x**7 + x**8 + x**9 + x**10, x, 0)
```

```python
series_reduce(x*y + x**3*y**3/3 + 2*x**5*y**5/15 + 17*x**7*y**7/315 + 62*x**9*y**9/2835, x, 5)
```

## Evalf

At the 762nd place in the decimal expansion of $\pi$, there is `999999` (see the below comic ref: http://www.qwantz.com/index.php?comic=1013).  Note T-Rex is counting the digits like

        π: 3 . 1 4 1 5 9

    digit:     1 2 3 4 5 6

```python
from IPython.core.display import Image
Image(filename='../imgs/comic2-1040.png')
```

Write a function that takes a symbolic expression (like `pi`), and determines the first place where `999999` appears.

Tip: Use the string representation of the number. You can take the part after the `.` by using `split('.', 2)[1]`.

```python
'1.2345'.split('.', 2)[1]
```

And remember that Python starts counting at `0`, so if you use `str.find`, you'll need to add one.

```python
str(1.2345).split('.', 2)[1].find('345')
```

```python
str(1.2345).split('.', 2)[1].find('345') + 1
```

And remember that `str.find` returns `-1` if it cannot find the string.

```python
def find_999999(expr, limit=100000):
    """
    Find the first place in the decimal expr where 999999 appears.

    Only checks up to limit digits.

    Returns False when 999999 does not appear.

    >>> find_999999(pi)
    763
    >>> find_999999(E)
    False
    >>> find_999999(E, 1000000) # This one will take a few seconds to compute
    384341
    """

```

```python
find_999999(pi)
```

```python
find_999999(E)
```

```python
find_999999(E, 1000000)
```

```python

```