# Recitation 2 - Formulations

In today's recitation, we will build different formulations of the same problem, and explore the computational implications.

First, we load packages.

```julia
using JuMP, Gurobi

using DataFrames, CSV, Combinatorics
using LinearAlgebra
```

```julia
import Pkg; Pkg.add("Combinatorics")
```

       Updating registry at `~/.julia/registries/General`
    ######################################################################### 100.0%
      Resolving package versions...
      Installed Combinatorics ─ v1.0.2
    Updating `~/.julia/environments/v1.5/Project.toml`
      [861a8166] + Combinatorics v1.0.2
    Updating `~/.julia/environments/v1.5/Manifest.toml`
      [861a8166] + Combinatorics v1.0.2

## Part 1: Facility location

### 1.1 Problem setup

We are now ready to formulate our first integer optimization problem. The facility location problem takes as inputs two sets and two parameters:
- Sets:
    - facilities $j\in \mathcal{J}=\{1,\ldots,n\}$
    - customers $i\in \mathcal{I}=\{1, \ldots, m\}$
- Parameters:
    - $d_{ij}$: distance from customer $i$ to facility $j$
    - $c_j$: cost of facility $j$

Let's define the size of the problem.

```julia
n = 50
m = 500
```

In real-life, we would have data on the location of facilities and customers. For simplicity, we assume that the facilities and customers are uniformly sampled over the unit square $[0,1]^2$. The first column of the following arrays denotes the x coordinate, and the second column denotes the y coordinate.

```julia
facilities = rand(n,2);
customers = rand(m,2); # notice we add a semicolon to suppress Jupyter output
```

We can now define the distance matrix:

```julia
dist = [LinearAlgebra.norm(customers[i, :] .- facilities[j, :]) for i=1:m, j=1:n];
@show size(dist);
```

Finally, we sample a vector of facility costs uniformly between 10 and 20:

```julia
c = rand(n)*10 .+ 10;
```

### 1.2 Formulations

We covered two formulations of the problem in class, one with many constraints but a tight formulation, the other with fewer constraints but a less tight formulation. Time to implement!

#### Formulation 1

$$\min \sum_{j=1}^nc_j y_j + \sum_{i=1}^m\sum_{j=1}^nd_{ij}x_{ij}$$
$$\text{subject to}$$
$$\sum_{j=1}^n x_{ij}=1 \quad \forall i\in[m]$$
$$x_{ij}\le y_j \quad\forall i\in[m], j\in[n]$$
$$x_{ij}\in\{0,1\}\quad\forall i\in[m], j\in[n]$$
$$y_j \in \{0,1\}\quad \forall j\in[n]$$

```julia
"Build facility location model 1"
function facility_model_1(distance::Matrix, cost::Vector)
    # extract problem dimensions from distance matrix and verify coherence of input data
    m, n = size(distance)
    @assert length(cost) == n
    model = Model(Gurobi.Optimizer)
    set_optimizer_attribute(model, "TimeLimit", 1800)
    # VARIABLES
    # Whether to open each facility
    @variable(model, y[1:n], Bin)
    # Whether to solve a particular customer from a particular facility
    @variable(model, x[1:m, 1:n], Bin)
    # CONSTRAINTS
    @constraint(
        model, serve_every_customer[i = 1:m],
        sum(x[i, j] for j = 1:n) == 1
    )
    @constraint(
        model, only_serve_from_open_facility[i = 1:m, j=1:n],
        x[i, j] <= y[j]
    )
    # OBJECTIVE
    @objective(
        model, Min, sum(cost[j] * y[j] for j = 1:n) + sum(distance[i, j] * x[i, j] for i=1:m, j=1:n)
    )
    return model, x, y
end
```

Now we can call our function to build the model with our sampled data, then solve and report the elapsed time.

```julia
buildtime1 = @elapsed model1, x1, y1 = facility_model_1(dist, c);
solvetime1 = @elapsed optimize!(model1)
@show buildtime1
@show solvetime1
```

#### Formulation 2

$$\min \sum_{j=1}^nc_j y_j + \sum_{i=1}^m\sum_{j=1}^nd_{ij}x_{ij}$$
$$\text{subject to}$$
$$\sum_{j=1}^n x_{ij}=1 \quad \forall i\in[m]$$
$$\sum_{i=1}^m x_{ij}\le my_j \quad\forall j\in[n]$$
$$x_{ij}\in\{0,1\}\quad\forall i\in[m], j\in[n]$$
$$y_j \in \{0,1\}\quad \forall j\in[n]$$

```julia
"Build facility location model 1"
function facility_model_2(distance::Matrix, cost::Vector)
    # extract problem dimensions from distance matrix and verify coherence of input data
    m, n = size(distance)
    @assert length(cost) == n
    model = Model(Gurobi.Optimizer)
    set_optimizer_attribute(model, "TimeLimit", 1800)
    # VARIABLES
    # Whether to open each facility
    @variable(model, y[1:n], Bin)
    # Whether to solve a particular customer from a particular facility
    @variable(model, x[1:m, 1:n], Bin)
    # CONSTRAINTS
    @constraint(
        model, serve_every_customer[i = 1:m],
        sum(x[i, j] for j = 1:n) == 1
    )
    @constraint(
        model, only_serve_from_open_facility[j=1:n],
        sum(x[i, j] for i=1:m) <= m * y[j]
    )
    # OBJECTIVE
    @objective(
        model, Min, sum(cost[j] * y[j] for j = 1:n) + sum(distance[i, j] * x[i, j] for i=1:m, j=1:n)
    )
    return model, x, y
end
```

```julia
buildtime2 = @elapsed model2, x2, y2 = facility_model_2(dist, c);
solvetime2 = @elapsed optimize!(model2)
@show buildtime2
@show solvetime2
```

## Part 2: TSP

In the traveling salesman problem, we have $n$ locations, or "cities", indexed by $[n]=\{1,\ldots,n\}$ with $d_{ij}$ denoting the distance between location $i$ and location $j$.

The goal is to find a tour that visits each location exactly once, while minimizing total distance traveled.

### 2.1 Problem setup

We have a few TSP instances saved in the `tsp` directory. The data are stored as ASCII text files, which we can read as follows.

```julia
file = open("tsp/berlin52.tsp");
data = readlines(file)
data[1:10]
```

    10-element Array{String,1}:
     "NAME: berlin52"
     "TYPE: TSP"
     "COMMENT: 52 locations in Berlin (Groetschel)"
     "DIMENSION: 52"
     "EDGE_WEIGHT_TYPE: EUC_2D"
     "NODE_COORD_SECTION"
     "1 565.0 575.0"
     "2 25.0 185.0"
     "3 345.0 750.0"
     "4 945.0 685.0"

The first few lines describe the file and the problem type, and the following lines list the (x,y) coordinates of each point in a grid. We would to extract the coordinates and define a distance matrix. The function below is one way to do this, but there are many others.

```julia
function get_distances(filename)
    file = open(filename)
    data = readlines(file)
    close(file)
    # get useful lines
    dimension_line = findfirst(x -> occursin("DIMENSION", x), data)
    metric_line = findfirst(x -> occursin("EDGE_WEIGHT_TYPE", x), data)
    first_data_line = findfirst(x -> occursin("NODE_COORD_SECTION", x), data) + 1
    last_data_line = findfirst(x -> occursin(r"EOF", x), data) - 1
    # extract dimension
    n = parse(Int64, match(r"[0-9]+", data[dimension_line]).match)
    # check metric is Euclidean 2D
    occursin("EUC_2D", data[metric_line]) || error("Unsupported metric type")
    # Create coordinates
    coords = zeros(n, 2)
    for line in data[first_data_line:last_data_line]
        temp = split(line)
        parse(Int64,temp[1])
        coords[parse(Int64,temp[1]),:] = [parse(Float64,temp[2]) parse(Float64,temp[3])]
    end
    # Create distance matrix
    distances = [norm(coords[i, :] .- coords[j, :]) for i=1:n, j=1:n]
    return distances
end
```

    get_distances (generic function with 1 method)

We can now use our shiny new `get_distances` function to convert a TSP text file to a distance matrix. We can visually check the matrix is 52x52, is symmetric, has nonnegative values, and zeros on the diagonal.

```julia
get_distances("tsp/berlin52.tsp")
```

### 2.2 Formulations

#### Basic formulation

$$\begin{align}
\min\quad & \sum_{i=1}^n\sum_{j=1}^n d_{ij}x_{ij}\\
\text{s.t.}\quad & \sum_{i=1}^n x_{ij}=1 & \forall j\in[n]\\
&\sum_{j=1}^nx_{ij} =1 & \forall i\in[n]\\
&x_{ii}=0 & \forall i\in[n]\\
& x_{ij}\in\{0,1\}&\forall i, j \in [n]
\end{align}$$

_Note: even though our distance matrix is symmetric, it is slightly easier to implement a more general formulation where we do not assume this property. If we know our distance matrix is symmetric, we can cut the number of variables in half by only defining $x_{ij}$ for $i < j$ (i.e. not defining both $x_{12}$ and $x_{21}$), but this is beyond the scope of this recitation._

```julia
"Construct TSP pre-model, without cycle elimination mechanism"
function prebuild_tsp(dist::Matrix)
    n = size(dist, 1)
    # Definition of model
    model = Model(Gurobi.Optimizer)
    # Main variable: x_ij=1 if the tour visits i and j in that order, 0 otherwise
    @variable(model, x[1:n, 1:n], Bin)
    # Objective: minimizing the total cost (distance) of the tour
    @objective(model, Min, sum(dist[i, j] * x[i, j] for i = 1:n, j = 1:n))
    # SHARED CONSTRAINTS
    @constraint(
        model, no_self_edges[i = 1:n], x[i,i] == 0
    )
    @constraint(
        model, exactly_one_successor[i = 1:n], sum(x[i, j] for j = 1:n) == 1
    )
    @constraint(
        model, exactly_one_predecessor[j = 1:n], sum(x[i, j] for i = 1:n) == 1
    )
    return model, x
end
```

    prebuild_tsp

#### Issues with the formulation

The formulation we've built so far is nice, but it has one big flaw! It doesn't ensure that every city must be part of the same tour.

For instance, the following solution would be feasible:

We need to somehow eliminate these "subtours".

#### Attempt 1: MTZ formulation (compact)

This formulation eliminates subtours using a $u_i$ variable for each node $i$, which defines the "order" of the visit starting from node 1 ($u_1=1$). More precisely, we impose the following constraints:

$u_1=1$

$2\le u_i \le n \quad\forall i=2, \ldots, n$

$u_j \ge u_i + 1 - (n-1)(1-x_{ij}) \quad \forall i, j \in [n]$

The first two are straightforward. What does the third one mean?

- If $x_{ij}=1$, then we impose that $u_j\ge u_i +1$ (the order of $j$ is at least 1 + the order of $i$). Since all $u_i$ are upper-bounded by $n$, this leads to the order of $j$ being exactly 1 + the order of $i$).
- If $x_{ij}=0$, then we impose that $u_j\ge u_i + 2 - n$. Since $u_i$ is at most $n$, in the worst case we impose $u_j\ge 2$ which holds as long as $j>1$ (recall our convention that $u_1=1$). So $x_{ij}=0$ effectively "turns off" the constraint.

We implement the MTZ formulation by appending these constraints to the core assignment model.

```julia
"Solve TSP using MTZ formulation, return runtime and objective"
function solveMTZ(dist::Matrix; time_limit_seconds::Real=1800)
    model, x = prebuild_tsp(dist, time_limit_seconds);
    set_optimizer_attribute(model, "TimeLimit", time_limit_seconds)
    n = size(dist,1);
    # Lower bound: 1 for node 1, 2 for all other nodes
    lb = [1 ; 2*ones(n-1)]
    # Upper bound: 1 for node 1, n for all other nodes
    ub = [1 ; n*ones(n-1)]
    # We define the u variable with lower and upper bounds, ensuring in particular that u_1=1
    @variable(model, lb[i] <= u[i = 1:n] <= ub[i])
    # Constraint
    @constraint(
        model, [i = 2:n, j = 2:n], u[i] - u[j] + 1 <= (n-1) * (1 - x[i, j])
    )
    # We then solve the model and store the runtime
    start = time()
    optimize!(model)
    solvetime = time() - start
    return solvetime, objective_value(model)
end
```

Let's try it!

```julia
dist52 = getDistances("tsp/berlin52.tsp");
solvetime52_MTZ, obj52_MTZ = solveMTZ(dist52);
println("Runtime: $solvetime52_MTZ seconds")
println("Objective: $obj52_MTZ")
```

Let's try it on a larger instance now.

```julia
dist76 = getDistances("tsp/pr76.tsp");
solvetime76_MTZ, obj76_MTZ = solveMTZ(dist76, time_limit_seconds=60)
println("Runtime: $solvetime76_MTZ seconds")
println("Objective: $obj76_MTZ")
```

On 52 cities, we solve in about one second. On 76 cities, we are much slower! Even after 60 seconds, our objective gap is still 6.9%. Clearly this will not scale to hundreds of cities.

#### Attempt 2: combinatorial formulation (cutset)

OK, so MTZ is a bit of a dud (no offense). Can we do better? Yes, provided we're not afraid of large numbers.

*Intuition:* since we seem to dislike subtours so much, let's write a specific constraint for every possible subtour, preventing it from existing.

One way to do this is the following pair of "cutset" constraints:

$$\sum_{i\in S}\sum_{j\notin S} x_{ij}\ge 1 \quad\forall S\subset V, S\neq\emptyset, V$$

$$\sum_{i\notin S}\sum_{j\in S} x_{ij}\ge 1 \quad\forall S\subset V, S\neq\emptyset, V$$

_What's going on here?_ For a subset of nodes $S$, the constraints above ensure that there is at least one edge leaving the subset $S$, and at least another edge entering the subset $S$.

Unfortunately, a direct implementation is simply impossible, even for small-scale instances. Indeed, the number of subsets of $\{1,n\}$ is equal to $2^n$. For $n=52$, this means $4.5\cdot 10^15$ -- clearly impossible with our computer's memory.

Instead, we will add constraints *only when we need them*. We have two options:
1. We will solve the model naively, without subtour elimination constraints, and add constraints that are violated by the incumbent solution.

2. We can also define *lazy constraints* in a callback function. This is a bit harder, but the idea is to give the constraints to the solver, and the solver will use them as needed through the branch-and-cut process. Specifically, the solver will keep the constraints in a "pool". As solutions are generated (in the branching tree), the solver checks which constraints are violated and adds them to the active formulation. Hence, the name "lazy" constraints because we let the solver use them as needed.

Either way, we need to write a function to find a subtour in a candidate solution. If we were trying to make our code as efficient as possible, our best bet would be to write this function ourselves. In this recitation, we're going to pick an easier route, using the `LightGraphs` package.

```julia
using LightGraphs
```

The `LightGraphs` package is a way to easily work with graphs in Julia.

Given a solution $\boldsymbol{x}^*$, we can define the _induced_ graph $G(\boldsymbol{x}^*)$ as the graph with one node per TSP node, and an edge between each pair of nodes $(i,j)$ for which $x^*_{ij}=1$.

We know that a graph induced by a feasible solution will have exactly one edge into each node, and one edge out of each node. There is therefore a one-to-one mapping between subtours in the induced TSP graph and connected components of the induced graph.

```julia
"""
    Given the induced graph as an adjacency list (i.e., next[i] is the next node to visit after node i),
        compute all subtours.
    Return them as a list of lists of nodes in the same component
"""
function find_subtours(next::Vector{Int})
    n = length(next)
    g = DiGraph(n)
    for i = 1:n
        add_edge!(g, i, next[i])
    end
    components = strongly_connected_components(g)
    return sort(components, by=length)
end
```

Let's try using this function

```julia
dist52 = getDistances("tsp/berlin52.tsp")
model, x = prebuild_tsp(dist52)
optimize!(model)
next = [findfirst(x -> x > 0.5, value.(x[i, :])) for i = 1:size(x, 1)]
find_subtours(next)
```

We notice two things here:

1. We have a lot of subtours of length 2 - this is due to the directed formulation, different from the undirected formulation in lecture.

2. We have lost some information by using connected components, namely the order of the cycle. But it's ok, because the cutset constraints doesn't care about the order of the cycle anyway.

We also notice we have quite a few subtours to eliminate. For the subtours of length 2, we could decide to eliminate all of them at once, or to eliminate them as they come up. We can even see which approach is faster!

```julia
"Solve the TSP using an iterative approach"
function solve_iterative(dist::Matrix; time_limit_seconds::Real = 1800,
                         eliminate_length_2::Bool=false,
                         verbose::Bool = true)
    # We first solve the model without any subtour elimination consideration
    model, x = prebuild_tsp(dist)
    n = size(dist,1)
    if eliminate_length_2
        @constraint(model, no_length_2[i = 1:n, j = 1:n], x[i, j] + x[j, i] <= 1)
    end
    verbose || set_optimizer_attribute(model, "OutputFlag", 0)
    start=time()
    optimize!(model)

    while true
        # We store the incumbent solution
        next = [findfirst(x -> x > 0.5, value.(x[i, :])) for i = 1:n]
        # Note: checking for >0.5 is conservative (x is binary!) but it avoids numerical errors
        subtours = find_subtours(next)
        println("Found $(length(subtours)) subtours after $(time() - start) seconds")
        if length(subtours) == 1 # only one cycle, the TSP solution
            solvetime = time() - start
            return solvetime, objective_value(model)
        else
            # eliminate subtours
            for subtour in subtours
                @constraint(model, sum(x[i, j] for i=subtour, j=setdiff(1:n, subtour)) >= 1)
                @constraint(model, sum(x[i, j] for i=setdiff(1:n, subtour), j=subtour) >= 1)
            end
        end
        optimize!(model)
        time() - start > time_limit_seconds && return solvetime, objective_value(model)
    end
end
```

```julia
solvetime52_iterative, obj52_iterative = solve_iterative(dist52, verbose=false, eliminate_length_2=false)
println("Runtime: $solvetime52_iterative seconds")
println("Objective: $obj52_iterative")
```

Now let's try that 76-city instance. Remember it didn't solve in one minute using the MTZ formulation.

```julia
solvetime76_iterative, obj76_iterative = solve_iterative(dist76, verbose=false,
                                                         eliminate_length_2=false)
println("Runtime: $solvetime76_iterative seconds")
println("Objective: $obj76_iterative")
```

Let's try to solve even larger instances! This one has 280 locations.

```julia
dist280 = getDistances("tsp/a280.tsp")
solvetime280_iterative, obj280_iterative = solve_iterative(dist280, verbose=false);
println("Runtime: $solvetime280_iterative seconds")
println("Objective: $obj280_iterative")
```

```julia

```


# SymPy Tutorial

*Arthur Ryman* <br/>
*Last Updated: 2020-04-13*

This notebook contains the examples from the
[SymPy Tutorial](https://docs.sympy.org/latest/tutorial/index.html).

## Preliminaries

### Installation

The following example confirms that SymPy is installed correctly.

```python
from sympy import *
x = symbols('x')
a = Integral(cos(x)*exp(x), x)
Eq(a, a.doit())
```

$\displaystyle \int e^{x} \cos{\left(x \right)}\, dx = \frac{e^{x} \sin{\left(x \right)}}{2} + \frac{e^{x} \cos{\left(x \right)}}{2}$

The above example shows that SymPy is very nicely integrated with Jupyter.
The expression is printed as a properly typeset mathematical equation.
I assume that Jupyter sets the default pretty printer to LaTeX.

## Introduction

### What is Symbolic Computation?

```python
import math
math.sqrt(9)
```

    3.0

```python
math.sqrt(8)
```

    2.8284271247461903

```python
import sympy
sympy.sqrt(3)
```

$\displaystyle \sqrt{3}$

```python
sympy.sqrt(8)
```

$\displaystyle 2 \sqrt{2}$

```python
from sympy import symbols
x, y = symbols('x y')
expr = x + 2*y
expr
```

$\displaystyle x + 2 y$

```python
expr + 1
```

$\displaystyle x + 2 y + 1$

```python
expr - x
```

$\displaystyle 2 y$

```python
x*expr
```

$\displaystyle x \left(x + 2 y\right)$

```python
from sympy import expand, factor
expanded_expr = expand(x*expr)
expanded_expr
```

$\displaystyle x^{2} + 2 x y$

```python
factor(expanded_expr)
```

$\displaystyle x \left(x + 2 y\right)$

### The Power of Symbolic Computation

```python
from sympy import *
x, t, z, nu = symbols('x t z nu')
```

```python
init_printing(use_unicode=True)
```

```python
diff(sin(x)*exp(x), x)
```

```python
integrate(exp(x)*sin(x) + exp(x)*cos(x), x)
```

```python
integrate(sin(x**2), (x, -oo, oo))
```

```python
limit(sin(x)/x, x, 0)
```

```python
solve(x**2 -2, x)
```

```python
y = Function('y')
dsolve(Eq(y(t).diff(t, t) - y(t), exp(t)), y(t))
```

```python
Matrix([[1, 2], [2, 2]]).eigenvals()
```

```python
besselj(nu, z).rewrite(jn)
```

```python
latex(Integral(cos(x)**2, (x, 0, pi)))
```

    '\\int\\limits_{0}^{\\pi} \\cos^{2}{\\left(x \\right)}\\, dx'

```python
Integral(cos(x)**2, (x, 0, pi))
```

```python
latex_1 = latex(Integral(cos(x)**2, (x, 0, pi)))
print(latex_1)
```

    \int\limits_{0}^{\pi} \cos^{2}{\left(x \right)}\, dx

### Why SymPy?

## Gotchas

### Symbols

```python
from sympy import *
```

```python
x + 1
```

```python
x = symbols('x')
x + 1
```

```python
x, y, z = symbols('x y z')
```

```python
a, b = symbols('b a')
a
```

```python
b
```

```python
crazy = symbols('unrelated')
crazy + 1
```

```python
print(latex(crazy))
```

    unrelated

```python
x = symbols('x')
expr = x + 1
x = 2
print(expr)
```

    x + 1

```python
x = 'abc'
expr = x + 'def'
expr
```

    'abcdef'

```python
x = 'ABC'
expr
```

    'abcdef'

```python
x = symbols('x')
expr = x + 1
expr.subs(x, 2)
```

### Equal signs

```python
x + 1 == 4
```

    False

```python
Eq(x + 1, 4)
```

```python
(x + 1)**2 == x**2 + 2*x + 1
```

    False

```python
a = (x + 1)**2
b = x**2 + 2*x + 1
simplify(a - b)
```

```python
c = x**2 - 2*x + 1
simplify(a - c)
```

```python
a = cos(x)**2 - sin(x)**2
b = cos(2*x)
a.equals(b)
```

    True

```python
simplify(a - b)
```

### Two Final Notes: ^ and /

```python
True ^ False
```

    True

```python
True ^ True
```

    False

```python
Xor(x, y)
```

```python
type(Integer(1) + 1)
```

    sympy.core.numbers.Integer

```python
type(1 + 1)
```

    int

```python
Integer(1)/Integer(3)
```

```python
type(Integer(1)/Integer(3))
```

    sympy.core.numbers.Rational

```python
1/3
```

```python
from __future__ import division
1/2
```

```python
1//2
```

```python
Rational(1, 2)
```

```python
x + 1/2
```

```python
x + Rational(1,2)
```

 ### Further Reading

## Basic Operations

```python
from sympy import *
x, y, z = symbols('x y z')
```

 ### Substitution

```python
expr = cos(x) + 1
expr.subs(x, y)
```

```python
expr.subs(x, 0)
```

```python
expr = x**y
expr
```

```python
expr = expr.subs(y, x**y)
expr
```

```python
expr = expr.subs(y, x**x)
expr
```

```python
expr = sin(2*x) + cos(2*x)
expand_trig(expr)
```

```python
expr.subs(sin(2*x), 2*sin(x)*cos(x))
```

```python
expr = cos(x)
expr.subs(x, 0)
```

```python
expr
```

```python
x
```

```python
expr = x**3 + 4*x*y -z
expr.subs([(x, 2), (y, 4), (z, 0)])
```

```python
expr = x**4 - 4*x**3 + 4*x**2 - 2*x + 3
replacements = [(x**i, y**i) for i in range(5) if i % 2 == 0]
expr.subs(replacements)
```

### Converting Strings to SymPy Expression

```python
str_expr = 'x**2 + 3*x - 1/2'
expr = sympify(str_expr)
expr
```

```python
expr.subs(x, 2)
```

### evalf

```python
expr = sqrt(8)
expr.evalf()
```

```python
pi.evalf(100)
```

```python
expr = cos(2*x)
expr.evalf(subs={x: 2.4})
```

```python
one = cos(1)**2 + sin(1)**2
(one -1).evalf()
```

```python
(one -1).evalf(chop=True)
```

### lambdify

```python
import numpy
a = numpy.arange(10)
expr = sin(x)
f = lambdify(x, expr, 'numpy')
f(a)
```

    array([ 0.        ,  0.84147098,  0.90929743,  0.14112001, -0.7568025 ,
           -0.95892427, -0.2794155 ,  0.6569866 ,  0.98935825,  0.41211849])

```python
f = lambdify(x, expr, 'math')
f(0.1)
```

```python
def mysin(x):
    """
    My sine. Note that this is only accurate for small x.
    """
    return x
f = lambdify(x, expr, {'sin': mysin})
f(0.1)
```

## Printing

### Printers

### Setting up Pretty Printing

```python
from sympy import init_printing
init_printing()
```

```python
from sympy import init_session
init_session()
```

    IPython console for SymPy 1.5.1 (Python 3.7.6-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at https://docs.sympy.org/1.5.1/

```python
from sympy import *
x, y, z = symbols('x y z')
init_printing()
```

```python
Integral(sqrt(1/x), x)
```

### Printing Functions

#### str

```python
from sympy import *
x, y, z = symbols('x y z')
str(Integral(sqrt(1/x), x))
```

    'Integral(sqrt(1/x), x)'

```python
print(Integral(sqrt(1/x), x))
```

    Integral(sqrt(1/x), x)

#### srepr

```python
srepr(Integral(sqrt(1/x), x))
```

    "Integral(Pow(Pow(Symbol('x'), Integer(-1)), Rational(1, 2)), Tuple(Symbol('x')))"

#### ASCII Pretty Printer

```python
pprint(Integral(sqrt(1/x), x), use_unicode=False)
```

      /
     |
     |     ___
     |    / 1
     |   /  -  dx
     | \/   x
     |
    /

```python
pretty(Integral(sqrt(1/x), x), use_unicode=False)
```

    '  /          \n |           \n |     ___   \n |    / 1    \n |   /  -  dx\n | \\/   x    \n |           \n/            '

```python
print(pretty(Integral(sqrt(1/x), x), use_unicode=False))
```

      /
     |
     |     ___
     |    / 1
     |   /  -  dx
     | \/   x
     |
    /

#### Unicode Pretty Printer

```python
pprint(Integral(sqrt(1/x), x), use_unicode=True)
```

    ⌠
    ⎮     ___
    ⎮    ╱ 1
    ⎮   ╱  ─  dx
    ⎮ ╲╱   x
    ⌡

#### Latex

```python
print(latex(Integral(sqrt(1/x), x)))
```

    \int \sqrt{\frac{1}{x}}\, dx

#### MathML

```python
from sympy.printing.mathml import print_mathml
print_mathml(Integral(sqrt(1/x), x))
```

    <apply>
    	<int/>
    	<bvar>
    		<ci>x</ci>
    	</bvar>
    	<apply>
    		<root/>
    		<apply>
    			<power/>
    			<ci>x</ci>
    			<cn>-1</cn>
    		</apply>
    	</apply>
    </apply>

#### Dot

```python
from sympy.printing.dot import dotprint
from sympy.abc import x
print(dotprint(x+2))
```

    digraph{

    # Graph style
    "ordering"="out"
    "rankdir"="TD"

    #########
    # Nodes #
    #########

    "Add(Integer(2), Symbol('x'))_()" ["color"="black", "label"="Add", "shape"="ellipse"];
    "Integer(2)_(0,)" ["color"="black", "label"="2", "shape"="ellipse"];
    "Symbol('x')_(1,)" ["color"="black", "label"="x", "shape"="ellipse"];

    #########
    # Edges #
    #########

    "Add(Integer(2), Symbol('x'))_()" -> "Integer(2)_(0,)";
    "Add(Integer(2), Symbol('x'))_()" -> "Symbol('x')_(1,)";
    }


# Example 2: One-dimensional heat flow (exs2.py)

This example is from the CALFEM manual.

**Purpose:**

Analysis of one-dimensional heat flow.

**Description:**

Consider a wall built up of concrete and thermal insulation. The outdoor
temperature is −17 ◦C and the temperature inside is 20 ◦C. At the inside of
the thermal insulation there is a heat source yielding $10 ~W/m^2$.

The wall is subdivided into five elements and the one-dimensional spring
(analogy) element `spring1e` is used. Equivalent spring stiffnesses are
$k_i = λ A/L$ for thermal conductivity and $k_i = A/R$ for thermal
surface resistance. Corresponding spring stiffnesses per $m^2$ of the wall
are:

\begin{align}
k_1 &= 1/0.04 = 25.0 ~W/K  \\
k_2 &= 1.7/0.070 = 24.3 ~W/K  \\
k_3 &= 0.040/0.100 = 0.4 ~W/K  \\
k_4 &= 1.7/0.100 = 17.0 ~W/K  \\
k_5 &= 1/0.13 = 7.7 ~W/K
\end{align}

A global system matrix K and a heat flow vector f are defined. The heat source
inside the wall is considered by setting $f_4 = 10$. The element matrices
`Ke` are computed using `spring1e`, and the function `assem` assembles the
global stiffness matrix.

The system of equations is solved using `solveq` with considerations to the
boundary conditions in `bc` and `bcVal`. The prescribed temperatures are
$T_1 = −17 ~^{\circ}C$ and $T_2 = 20~^{\circ}C$.

Necessary modules are first imported.

```python
import numpy as np
import calfem.core as cfc
```

Next, the element topology is defined

```python
Edof = np.array([
    [1,2],
    [2,3],
    [3,4],
    [4,5],
    [5,6]
])
```

Create stiffness matrix K and load vector f

```python
K = np.mat(np.zeros((6,6)))
f = np.mat(np.zeros((6,1)))
f[3] = 10.0
```

Define element properties (ep) and create element matrices for the different material layers.

```python
ep1 = 25.0
ep2 = 24.3
ep3 = 0.4
ep4 = 17.0
ep5 = 7.7
```

Element stiffness matrices

```python
Ke1 = cfc.spring1e(ep1)
Ke2 = cfc.spring1e(ep2)
Ke3 = cfc.spring1e(ep3)
Ke4 = cfc.spring1e(ep4)
Ke5 = cfc.spring1e(ep5)
```

Assemble all element matrices into the global stiffness matrix

```python
cfc.assem(Edof[0,:], K, Ke1)
cfc.assem(Edof[1,:], K, Ke2)
cfc.assem(Edof[2,:], K, Ke3)
cfc.assem(Edof[3,:], K, Ke4)
cfc.assem(Edof[4,:], K, Ke5)

print("Stiffness matrix K:")
print(K)
```

    Stiffness matrix K:
    [[ 25.  -25.    0.    0.    0.    0. ]
     [-25.   49.3 -24.3   0.    0.    0. ]
     [  0.  -24.3  24.7  -0.4   0.    0. ]
     [  0.    0.   -0.4  17.4 -17.    0. ]
     [  0.    0.    0.  -17.   24.7  -7.7]
     [  0.    0.    0.    0.   -7.7   7.7]]

Define the boundary conditions and solve the system of equations

```python
bc = np.array([1,6])
bcVal = np.array([-17.0, 20.0])
a,r = cfc.solveq(K, f, bc, bcVal)

print("Displacements a:")
print(a)

print("Reaction forces r:")
print(r)
```

    Displacements a:
    [[-17.        ]
     [-16.43842455]
     [-15.86067203]
     [ 19.23779344]
     [ 19.47540439]
     [ 20.        ]]
    Reaction forces r:
    [[-1.40393862e+01]
     [ 0.00000000e+00]
     [ 0.00000000e+00]
     [ 0.00000000e+00]
     [ 5.68434189e-14]
     [ 4.03938619e+00]]


##  Analytical definition of Heaviside Kv profile

Reviewer 2: "You are using the opposite sign convention".

Fix: For z that increases upwards (positive above surface) the smoothed Heaviside function should be:

$Kv(z) = \begin{cases}
\kappa_{bg} &\mbox{if } z > H_r+\epsilon \\
\kappa_{bg}+ \kappa_{can}\left(0.5 + \frac{H_r-z}{2\epsilon} + \frac{1}{2\pi}\sin{\frac{\pi(H_r-z)}{\epsilon}}\right) & \mbox{if } H_r+\epsilon > z > H_r-\epsilon \\
\kappa_{can}+\kappa_{bg} & \mbox{if }  H_r-\epsilon > z
 \end{cases} $

 $H_{r}$ is rim depth ($<0$)

 $z$ < 0

```python
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
import seaborn as sns
import sympy as sym
sym.init_printing() # enable fancy printing
```

```python
# Set appearance options seaborn

sns.set_style('darkgrid')
sns.set_context('notebook')
```

```python
def Kv(kbg,kcan,Hr,eps,z):

    k1 = np.where(z > Hr+eps ,kbg, 0.)
    k2 = np.where((z <= Hr+eps) & (z >= Hr-eps),
                  kbg + kcan*(0.5 + ((Hr-z)/(2*eps)) + (1/(2*np.pi))*(np.sin(np.pi*(Hr-z)/eps))),
                  0.)
    k3 = np.where(z < Hr-eps ,kcan+kbg, 0.)

    return(k1+k2+k3)

```

```python
kbg = 1E-5
kcan = 1E-3
Hr = -100
eps = 40
z = np.linspace(-300,0,200)
diffusi = Kv(kbg,kcan,Hr,eps,z)

plt.plot(diffusi,Hr*np.ones(np.shape(diffusi)),color='0.4',label='Hr')
plt.plot(diffusi,Hr*np.ones(np.shape(diffusi))+eps,'--',color='grey',label='Hr+$\epsilon$')
plt.plot(diffusi,Hr*np.ones(np.shape(diffusi))-eps,'--',color='grey',label='Hr-$\epsilon$')
plt.plot(diffusi,z, 'g', linewidth=2, label='$Kv$')
plt.xlabel(r'Kv / m$^2$s$^{-1}$')
plt.ylabel('Z')
plt.legend(loc=0)
```

```python

```


```python
from sympy import stats
```

```python
X = stats.Normal('x',0,10)
from sympy.abc import x
#probability density function
stats.density(X)(x)
```

$\displaystyle \frac{\sqrt{2} e^{- \frac{x^{2}}{200}}}{20 \sqrt{\pi}}$

```python
# evaluate cdf
stats.cdf(X)(0)
```

$\displaystyle \frac{1}{2}$

```python
# evaluate probability
stats.P(X>0).evalf()
```

$\displaystyle 0.5$

```python
stats.E(X)

```

$\displaystyle 0$


#### Zápis do souboru
Vyhodnoťte funkci sin(x) ve 100 bodech na intervalu [0, 10] a uložte hodnoty x a sin(x) do souboru (např csv).

a) čistý Python

```python
xmin = 0
xmax = 10
N = 100
```

```python
import math

f = open("data.txt", "w")
for i in range(0, N):
    x = xmin + i*(xmax-xmin)/(N-1)
    y = math.sin(x)
    f.write("%f %f\n"%(x,y))
f.close()
```

Alternativní způsob práce se souborem

```python
import math

with open("data.csv", "w") as f:
    for i in range(0, 100):
        x = xmin + i*(xmax-xmin)/(N-1)
        y = math.sin(x)
        f.write(f"{x:f} {y:f}\n")
```

b) s využitím numpy je to o poznání jednodušší

```python
import numpy as np

x = np.linspace(0, 10, 100)
y = np.sin(x)
np.savetxt("data.txt", np.vstack((x, y)).T)
```

#### Čtení ze souboru do pole

Načtěte data uložená v předchozí úloze a vykreslete funkci graficky na obrazovce

a) čistý Python

```python
data = []

f = open("data.csv")
for line in f:
    data.append(list(map(float, line.strip().split(" "))))
```

b) s využitím numpy

```python
data = np.loadtxt("data.csv")
```

Vykresleni

```python
import matplotlib.pyplot as plt
plt.plot(data[:,0], data[:,1])
```

### Floating point aritmetika

Reprezentace des. čísel ve dvojkové soustavě: vytiskněte číslo 0.1 na 19 desetinných míst

```python
"%.19f"%0.1
```

    '0.1000000000000000056'

Sčítání a odčítání.

Vypočtěte $1+10^{-15}$ a $1+10^{-16}$

```python
1+1e-15
```

    1.000000000000001

```python
1+1e-16
```

    1.0

Kolik desítkových platných míst má float64 s 53-bitovou mantisou?

```python
np.log10(2**53)
```

    15.954589770191003

Výpočty tedy probíhají přibližně s přesností na 16 platných míst. Relativní zaokrouhlovací chyba roste, pokud například počítáme rozdíl dvou podobně velkých čísel:

Vypočtěte $1.001-1$

```python
1.001-1
```

    0.0009999999999998899

Z důvodu zaokrouhlovací chyby také záleží na pořadí aritmetických operací (jsou vyhodnocovány zleva doprava):

Vypočtěte $10^{-15}+1-1$ a $1-1+10^{-15}$

```python
1-1+1e-15
```

    1e-15

```python
1e-15+1-1
```

    1.1102230246251565e-15

Nevhodně navržený algoritmus může vést k "zesilování" chyby. Algoritmus potom je tzv. numericky nestabilní. Viz následující příklad:

Spočítejte členy následující posloupnosti pro $j=1\ldots10$:

$$x_0 = 1$$
$$x_j = (1.01 - x_{j-1})\cdot 100$$

a porovnejte s hodnotami vypočtenými "na papíře"

```python
# ukol 1
```

#### Sčítání číselných řad

Spočítejte

$$y = \sum_{i=0}^{1000} \frac{1}{10^{-6} + i}$$

 - "popořadě"
 - v obráceném pořadí, tedy od nejmenších členů
 - s využitím knihovních funkce `sum` a `math.fsum`

a porovnejte výsledky

```python
x = np.arange(1001)
y = 1/(1e-6 + x)
```

popořadě:

```python
s = 0
for i in y: s += i
s
```

    1000007.4854692194

pozpátku:

```python
s = 0
for i in y[::-1]: s += i
s
```

    1000007.4854692166

Vestavěná obecná funkce sum pracuje popořadě. Umí sčítat jakýkoliv objekt s definovanou operací sčítání:

```python
sum(y)
```

    1000007.4854692194

Funkce `math.fsum` je optimální pro přesnost výpočtu. V tomto případě se shoduje se součtem od nejmenších členů:

```python
math.fsum(y)
```

    1000007.4854692166

Funkce `np.sum` je optimalizovaná pro rychlost výpočtu, relativně přesná:

```python
np.sum(y)
```

    1000007.4854692165

Poznámka: exaktní výsledek můžeme snadno ověřit symbolickým výpočtem v knihovně sympy, která umožňuje numerické vyhodnocení sumy s libovolnou přesností. Nejdříve symbolicky definujeme náš výraz:

```python
import sympy as sp
si = sp.Symbol("si")
series = sp.Sum(1/(sp.S(10)**(-6) + si), (si, 0, 1000))
series
```

$\displaystyle \sum_{si=0}^{1000} \frac{1}{si + \frac{1}{1000000}}$

a vyhodnotíme s přesností na 30 platných míst. Vidíme, že v tomto případě je sumace "od nejmenších členů" dostatečně přesná.

```python
sp.Sum(1/(sp.S(10)**(-6) + si), (si, 0, 1000)).evalf(30)
```

$\displaystyle 1000007.48546921661698028641805$

### Porovnávání čísel

Zjistěte, jestli se rovnají čísla:

$1+ 10^{-16}$ a $1$;

$(1.01-1)\cdot100$ a $1$

```python
1+1e-16 == 1
```

    True

```python
1 == (1.01 - 1)*100
```

    False

Vidíme, že z důvodu zaokrouhlovací chyby se mohou shodná čísla jevit jako různá a naopak. Je nutné toto mít na paměti a v případě potřeby použít porovnávání s tolerancí (absolutní a relativní, viz např funkce `np.isclose` a její dokumentace)

```python
def isclose(a, b, rel_tol=1e-9, abs_tol=0.0):
    return abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)
```

```python
isclose(1, (1.01 - 1)*100)
```

    True

Vhodnější je ale použití knihovních funkcí `np.isclose` nebo `math.isclose`, které zaručují korektní chování s nekonečny a NaN...

```python
np.isclose(1, (1.01 - 1)*100)
```

    True

#### "Speciální" čísla: nula, nekonečno NaN

Poznámka bokem k typům v Pythonu: Python je dynamicky typovaný, takže jméno proměnné není nijak spjato s datovým typem, jako třeba v C/C++. Typ proměnné nebo výrazu můžeme zjistit příkazem `type`

```python
x = 0
type(x)
```

    int

```python
x = 0.0
type(x)
```

    float

A ještě jedna trochu záludná poznámka k typům čísel v Pythonu: spočtěte `1/0` a `1/np.sin(0)`. Proč se výsledky liší?

```python
1/0
```

```python
1/np.sin(0)
```

    <ipython-input-62-21b4b538a0f5>:1: RuntimeWarning: divide by zero encountered in double_scalars
      1/np.sin(0)

    inf

Je to typem čísel, zkuste type(0), type(0.0) a type(np.sin(0)

```python
type(0)
```

    int

```python
type(0.0)
```

    float

```python
type(np.sin(0))
```

    numpy.float64

Problém je v různých typech float čísel. Numpy float umožňuje dělení nulou. Pouze generuje varování a korektně vrací hodnotu inf. Toto chování numpy lze nastavit.

64-bitový nulový numpy float můžete vytvořit příkazem `np.float64(0)` a tento datový typ dokáže reprezentovat i speciální hodnoty NaN (not a number, nedefinovaný výraz), +inf, -inf (nekonečna), +0, -0 (nula se znaménkem)

```python
-np.float64(0), np.nan, -np.inf, np.inf
```

    (-0.0, nan, -inf, inf)

Ukázka některých výpočtů s NaN a inf hodnotami:

```python
1/np.float64(0), -1/np.float64(0)
```

    <ipython-input-52-18dbe72427c2>:1: RuntimeWarning: divide by zero encountered in double_scalars
      1/np.float64(0), -1/np.float64(0)

    (inf, -inf)

```python
0/np.float64(0)
```

    <ipython-input-53-f117ac9e30c2>:1: RuntimeWarning: invalid value encountered in double_scalars
      0/np.float64(0)

    nan

NaN dle definice není větší, menší, ani rovno žádnému číslu:

```python
np.nan > 3, np.nan <= 3, np.nan == np.nan
```

    (False, False, False)

```python
np.inf == np.inf
```

    True

```python
-np.float64(0) == np.float64(0)
```

    True

```python
np.arctan(np.inf)/np.pi
```

    0.5

### Hledání kořenů

Naprogramujte hledání kořenů metodou půlení intervalu.

Najděte kořen sin(x) mezi 3 a 4 metodou půlení intervalu

```python
a, b = 3, 4
f = np.sin
```

```python
def bisect(f, a, b, maxiter=53):
    # ukol 2: definujte tělo této funkce
```

```python
bisect(f, 3, 4)
```

    3.141592653589793

Newtonova metoda

```python
def newton(f, df, a):
    for i in range(10):
        a_new = a - f(a)/df(a)
        if a_new == a:
            return a
        a = a_new
```

```python
df = np.cos
```

```python
newton(f, df, 4.8)
```

    15.707963267948966

### Minimalizace

Newtonova metoda

```python
def newton_min(f, df, ddf, a):
    for i in range(10):
        a_new = a - df(a)/ddf(a)
        if a_new == a:
            if ddf(a) > 0:
                return a
            else:
                raise(RuntimeError("Method did not converge to minimum"))
        a = a_new
```

```python
def f(x): return x**2 + x
def df(x): return 2*x + 1
def ddf(x): return 2
```

```python
newton_min(f, df, ddf, 1)
```

    -0.5

```python
x = np.linspace(-2, 2)
plt.plot(x, f(x))
plt.ylim(ymax=2)
plt.grid()
```

```python

```


# 4. Introduction to Sympy for Mathematicians

## Table of Contents
- __Introduction to Sympy__
- __Plots__
- __Polynomials__
- Series
    - Taylor Series
    - Summing Series
- Linear Algebra
- Differential Equations
- __Limits, Derivatives, Integrals__
    - __Limits__
    - __Derivatives__
    - __Integrals__

Next week? Choose among:

- Stochastics/statistics: `random`, `scipy.statistics` , `statsmodels`
- Machine Learning 2/Deep Learning: `scikit-learn`, `keras`, `tensorflow`
- SAGE
- Other: ___________?

## 4.1 Introduction to SymPy

`sympy` is a package for symbolic calculations in Python, similar to *Wolfram Alpha/Mathematica*. It works with expressions containing symbols. Its core symbolic capabilities include:

- Elementary algebra
- Pattern matching and substitution
- Symbolic transformations and simplifications
- All the common mathematical functions and many special ones
- Calculus: limits, derivatives, integrals, Taylor series, and so on
- Solving many kinds of equations, including differential equations
- Linear algebra
- Polynomials
- Infinite sums and products
- Planar geometry
- Physical constants and units
- Numeric computation in arbitrary precision

Today we will import everything contained in the package:

```python
# import sympy as sym
from sympy import *
init_printing()
```

SymPy is also available at http://live.sympy.org, an online Python shell, with `sympy` already imported, which
allows you to perform simple calculations. The full documentation of this package can be found here: https://docs.sympy.org/latest/index.html. We can also watch a YouTube video:

```python
from IPython.display import YouTubeVideo
YouTubeVideo('kx2GzBeGPco', width=533, height=300)
```

Symbols are basic bricks used to construct expressions.
Each symbol has a name used for printing expressions. Objects of the class `Symbol` should be created and assigned to Python variables in order to be used in expressions. The symbol name and the name of the variable to which this symbol is assigned are two independent things, and one may write `abc=Symbol('xyz')`. But then one has to write `abc` in input expressions, while `SymPy` will write `xyz` in output ones, producing unnecessary confusion. The Python variable name should better be the same as the symbol name.

In languages specifically designed for symbolic calculations, such as *Mathematica*, if a variable to which nothing has been assigned is used, it automatically means a symbol with the same name. Python has not been designed for symbolic calculations. If you use a variable to which nothing has been assigned, you will get an error message. Symbol objects have to be created explicitly.

```python
x = Symbol('x')
x
```

```python
a = x**2-1
a
```

However, we can also use the command `sympify`, then we don't need to define $x$ manually:

```python
sympify('x^2 - 1')
```

$\displaystyle x^{2} - 1$

Internally expressions are trees. The function `srepr` returns a string representing this tree.

```python
srepr(a)
```

    "Add(Pow(Symbol('x'), Integer(2)), Integer(-1))"

One may use the functions `Add`, `Mul`, `Pow`, etc. instead of the binary operations `+`, `*`, `**`, etc.

```python
Add(Pow(Symbol('x'), Integer(2)), Integer(-1))
```

Some packages are available that provide Greek letters:

```python
from sympy.abc import _clash2

f = sympify('beta^alpha * x^(alpha - 1) * e^(-beta * x)/ Gamma(alpha)', locals=_clash2)
f
```

```python
srepr(f)
```

    "Mul(Pow(Symbol('beta'), Symbol('alpha')), Pow(Symbol('e'), Mul(Integer(-1), Symbol('beta'), Symbol('x'))), Pow(Symbol('x'), Add(Symbol('alpha'), Integer(-1))), Pow(Function('Gamma')(Symbol('alpha')), Integer(-1)))"

Several symbols can be defined at once. The string is split at spaces.

```python
y,z = symbols('y z')
```

Let's substitute $y+1$ for $x$.

```python
b = a.subs(x,y+1)
b
```

`sympy` automatically applies simplifications of elementary functions which are correct everywhere.

```python
cos(-x)
```

```python
cos(pi/4)
```

The method `rewrite` tries to rewrite an expression in terms of a given function.

```python
cos(x).rewrite(exp)
```

Be careful that some functions cannot always be simplified by default:

```python
sqrt(x)**4, sqrt(x**4)
```

Why not $x^2$? Try $x=i$.

```python
x = symbols('x',real=True)
sqrt(x**4)
```

## 4.2 Plots

Let's plot `a = x**2-1` and `b = a.subs(x,y+1)`. To do this, we first change the variable $y$ to $x$:

```python
a = x**2-1
b = b.subs(y,x)
b
```

`sympy` uses `matplotlib`. However, it distributes $x$ points adaptively, not uniformly.

```python
%matplotlib inline
```

```python
p = plot(a, b, show=False)
p[1].line_color = "r"
p.legend = True
p.show()
```

We can also create plots in which functions are defined implicitly:

```python
equation = Eq(x**2,y)
equation
```

```python
plot_implicit(equation,(x,-1,1),(y,0,1))
```

```python
plot_implicit(x**2+y**2>1,(x,-1,1),(y,-1,1))
```

Some additional plotting functions can be imported from `sympy.plotting`.

```python
from sympy.plotting import plot_parametric, plot3d, plot3d_parametric_line, plot3d_parametric_surface
```

```python
t = Symbol('t')
plot_parametric(sin(2*t),cos(3*t),(t,0,2*pi),
                title='Lissajous',xlabel='x',ylabel='y')
```

We can also create some 3d plots using `matplotlib` via `sympy`:

```python
# show plots in new window
%matplotlib qt
```

First we plot the surface $f(x,y) = x^2 + y^2$:

```python
plot3d(x**2+y**2,(x,-2,2),(y,-2,2))
```

    <sympy.plotting.plot.Plot at 0x20cf7d1c5c8>

Now a parametric space curve - a spiral.

```python
plot3d_parametric_line(cos(t),sin(t),0.1*t,(t,0,4*pi))
```

    <sympy.plotting.plot.Plot at 0x20cf859c708>

And finally a torus:

```python
u,v = symbols('u v')
a = .5
plot3d_parametric_surface((1+a*cos(u))*cos(v),(1+a*cos(u))*sin(v),a*sin(u),(u,0,2*pi),(v,0,2*pi))
```

    <sympy.plotting.plot.Plot at 0x20cf7ea4f08>

## 4.3 Polynomials

`SymPy` does not expand brackets automatically. The function `expand` is used for this.

```python
a = (x+y-z)**6
a
```

```python
a = expand(a)
a
```

Degree of the polynomial $a$ in $x$:

```python
degree(a,x)
```

Roots of the polynomial $a$:

```python
roots(a,x)
```

Let's collect terms with same power of $x$ together.

```python
collect(a,y)
```

Any polynomial with integer coefficients can be factorized into polynomials with integer coefficients (which cannot be factorized further). There exist efficient algorithms to do this.

```python
a = factor(a)
a
```

`sympy` does not automatically cancel ratios of polynomials by their greatest common divisor. The function `cancel` is used for this.

```python
a = (x**3-y**3)/(x**2-y**2)
a
```

```python
cancel(a)
```

`sympy` does not automatically bring sums of rational expressions to common denominator. The function `together` is used for this.

```python
a = y/(x-y)+x/(x+y)
a
```

```python
together(a)
```

The function `simplify` tries to rewrite an expression *in a simplest form*. This concept is not well defined (different forms may be considered simplest in different contexts), and there exists no algorithm for such simplification. The function `simplify` works heuristically, and it is not possible to guess in advance what simplifications it will try. It is very convenient in interactive sessions in order to check if it will succeed in rewriting an expression in some reasonably good form. But it is not desirable to use it in programs. There one should better use more specialized functions which perform well defined expression transformations.

```python
simplify(a)
```

We can solve the equation $a = 0$ for $x$:

```python
solve(Eq(a,0),x)[1] # solve(Eq(a,0),x)
```

Partial fraction decomposition with respect to $x$:

```python
apart(a,x)
```

Let's substitute some values for the symbols $x$ and $y$.

```python
evaluation = a.subs({x:1,y:2})
evaluation
```

And how much is it numerically?

```python
evaluation.n()
```

`subs` can also be used to interchange two variables. However, we need to be careful. Compare the difference:

```python
a.subs([(x,y),(y,x)])
# a.subs([(y,x),(x,y)])
```

```python
a.subs([(x,y),(y,x)],simultaneous=True)
```

Homework

Read Section 4.4-4.6.

## 4.4 Series

### 4.4.1 Taylor Series

The Taylor series can be computed using the function `series`:

```python
exp(x).series(x,0,n=5)
```

Let's prepare 3 series.

```python
sinx = series(sin(x),x,0,n=8)
sinx
```

```python
cosx = series(cos(x),x,n=8)
cosx
```

```python
tanx = series(tan(x),x,n=8)
tanx
```

Products and ratios of series are not calculated automatically, the function `series` should be applied to them.

```python
series(sinx / cosx,n=8) # sinx / cosx
```

The following series should be equal to 1. But since `sinx` and `cosx` are known only with a limited accuracy, we obtain 1 with the same accuracy.

```python
series(sinx**2+cosx**2,n=8)
```

Series can be differentiated and integrated using the functions `diff` and `integrate`, respectively.

```python
diff(cosx,x)
```

```python
integrate(cosx,x)
```

It is not possible to substitute a numerical value for the expansion variable in a series (and hence to plot it). To this end one has to remove the $\mathcal{O}$ term first, transforming a series into a polynomial.

```python
sinx.removeO()
```

### 4.4.2 Summing Series

In `scipy`, `oo` means $\infty$.

```python
n = Symbol('n')
summation(1/n**2,(n,1,oo))
```

```python
summation(1/n**4,(n,1,oo))
```

An unevaluated sum is denoted as `Sum`.

```python
a = Sum(x**n/factorial(n),(n,0,oo))
Eq(a,a.doit())
```

## 4.5 Linear Algebra

```python
a,b,c,d,e,f = symbols('a b c d e f')
```

A matrix can be constructed from a list of lists.

```python
M=Matrix([[a,b,c],[d,e,f]])
M
```

$\displaystyle \left[\begin{matrix}a & b & c\\d & e & f\end{matrix}\right]$

```python
M.shape
```

A matrix can be constructed from a function.

```python
def g(i,j):
    return Rational(1,i+j+1)
Matrix(3,3,g)
```

$\displaystyle \left[\begin{matrix}1 & \frac{1}{2} & \frac{1}{3}\\\frac{1}{2} & \frac{1}{3} & \frac{1}{4}\\\frac{1}{3} & \frac{1}{4} & \frac{1}{5}\end{matrix}\right]$

Or from an undefined function.

```python
g = Function('g')
M = Matrix(3,3,g)
M
```

$\displaystyle \left[\begin{matrix}g{\left(0,0 \right)} & g{\left(0,1 \right)} & g{\left(0,2 \right)}\\g{\left(1,0 \right)} & g{\left(1,1 \right)} & g{\left(1,2 \right)}\\g{\left(2,0 \right)} & g{\left(2,1 \right)} & g{\left(2,2 \right)}\end{matrix}\right]$

```python
M[1,2] = 0
M
```

$\displaystyle \left[\begin{matrix}g{\left(0,0 \right)} & g{\left(0,1 \right)} & g{\left(0,2 \right)}\\g{\left(1,0 \right)} & g{\left(1,1 \right)} & 0\\g{\left(2,0 \right)} & g{\left(2,1 \right)} & g{\left(2,2 \right)}\end{matrix}\right]$

```python
M[2,:]
```

$\displaystyle \left[\begin{matrix}g{\left(2,0 \right)} & g{\left(2,1 \right)} & g{\left(2,2 \right)}\end{matrix}\right]$

Diagonal matrices can be constructed using the function `diag`:

```python
M = Matrix([[a,1],[0,a]])
diag(1,M,2)
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & a & 1 & 0\\0 & 0 & a & 0\\0 & 0 & 0 & 2\end{matrix}\right]$

With `sympy`, we can just use `*` for matrix multiplication:

```python
A = Matrix([[a,b],[c,d]])
B = Matrix([[1,2],[3,4]])
A*B
```

$\displaystyle \left[\begin{matrix}a + 3 b & 2 a + 4 b\\c + 3 d & 2 c + 4 d\end{matrix}\right]$

```python
B*A
```

$\displaystyle \left[\begin{matrix}a + 2 c & b + 2 d\\3 a + 4 c & 3 b + 4 d\end{matrix}\right]$

```python
simplify(A**(-1))
```

$\displaystyle \left[\begin{matrix}\frac{d}{a d - b c} & - \frac{b}{a d - b c}\\- \frac{c}{a d - b c} & \frac{a}{a d - b c}\end{matrix}\right]$

Consider the following matrix:

```python
x = Symbol('x',real=True)
M = Matrix([[(1-x)**3*(3+x),4*x*(1-x**2),-2*(1-x**2)*(3-x)],
          [4*x*(1-x**2),-(1+x)**3*(3-x),2*(1-x**2)*(3+x)],
          [-2*(1-x**2)*(3-x),2*(1-x**2)*(3+x),16*x]])
M
```

$\displaystyle \left[\begin{matrix}\left(1 - x\right)^{3} \left(x + 3\right) & 4 x \left(1 - x^{2}\right) & \left(3 - x\right) \left(2 x^{2} - 2\right)\\4 x \left(1 - x^{2}\right) & - \left(3 - x\right) \left(x + 1\right)^{3} & \left(2 - 2 x^{2}\right) \left(x + 3\right)\\\left(3 - x\right) \left(2 x^{2} - 2\right) & \left(2 - 2 x^{2}\right) \left(x + 3\right) & 16 x\end{matrix}\right]$

The determinant.

```python
det(M)
```

The eigenvalues and their multiplicities.

```python
M.eigenvals()
```

The eigenvector corresponding to the second eigenvalue.

```python
M.eigenvects()[1][2][0]
```

$\displaystyle \left[\begin{matrix}\frac{x}{2} + \frac{1}{2}\\\frac{x + 1}{x - 1}\\1\end{matrix}\right]$

## 4.6 Differential Equations

```python
t = Symbol('t')
x = Function('x')
p = Function('p')
```

First we solve the first-order ODE $\frac{\mathrm{d}}{\mathrm{d}t}x(t) = -x(t)$.

```python
dsolve(Eq(diff(x(t),t),-x(t)),x(t))
```

Now the second-order ODE $\frac{\mathrm{d}^2}{\mathrm{d}t^2}x(t) = -x(t)$.

```python
dsolve(Eq(diff(x(t),t,2),x(t)),x(t))
```

Finally a system of first-order equations:
\begin{align*}
\begin{cases}
\frac{\mathrm{d}}{\mathrm{d}t}x(t) = p(t), \\
\frac{\mathrm{d}}{\mathrm{d}t}p(t) = -x(t).
\end{cases}
\end{align*}

```python
dsolve((diff(x(t),t)-p(t),diff(p(t),t)+x(t)))
```

Done reading?

Then read the final section:

## 4.7 Limits, Derivatives, Integrals

### 4.7.1 Limits

Limits can be computed by using the function `limit`. We will do this for the function

\begin{align*}
f(x) = \frac{\tan(\sin x)-\sin(\tan x)}{x^7}.
\end{align*}

```python
%matplotlib inline
x = Symbol('x')
f = (tan(sin(x))-sin(tan(x)))/x**7
plot(f, (x,-2,2))
```

```python
limit(f,x,0)
```

This limit is easy to compute as it is a two-sided limit. Now let's calculate one-sided limits for the function

\begin{align*}
f(x) = \frac{\tan(\sin x)-\sin(\tan x)}{x^7 + \exp\left(-\frac{1}{x}\right)}.
\end{align*}

```python
a = (tan(sin(x))-sin(tan(x)))/(x**7 + exp(-1/x))
plot(a, xlim=[-.25,.25], ylim=[-0.5,0.5]) # does not always work
```

```python
limit(a,x,0,'+')
```

```python
limit(a,x,0,'-')
```

### 4.7.2 Derivatives

We compute derivatives using the function `diff`.

```python
a = x*sin(x+y)
diff(a,x,1)
```

```python
diff(a,y)
```

Let's take the second derivative in $x$ and the first one in $y$.

```python
diff(a,x,2,y)
```

Expressions with undefined functions can also be differentiated.

```python
f = Function('f')
b = diff(x*f(x**2),x)
```

What's this?

```python
b
```

$\displaystyle 2 x^{2} \left. \frac{d}{d \xi_{1}} f{\left(\xi_{1} \right)} \right|_{\substack{ \xi_{1}=x^{2} }} + f{\left(x^{2} \right)}$

The function `Derivative` represents an unevaluated derivative. It can be evaluated by the method `doit`.

```python
a = Derivative(sin(x),x)
Eq(a,a.doit())
# diff(sin(x),x)
```

### 4.7.3 Integrals

Integrals can be computed using the function `integrate`.

```python
a = 1/(x*(x**2-2)**2)
integrate(a,x)
```

```python
integrate(1/(exp(x)+1),x)
```

```python
integrate(x*sin(x),x)
```

```python
a = integrate(x**x,x)
a.doit()
```

This is an unevaluated integral (so it is written as `Integral`).

```python
print(a)
```

    Integral(x**x, x)

```python
a = Integral(sin(x),x)
Eq(a,a.doit())
```

### What's next? SAGE.

### 🔴 *Next Week:*

```python
import numpy as np
np.random.choice(['Statistics','Something else'], p=[0.99,0.01])
```

    'Statistics'

Notebooks/4 - Sympy for Mathematicians.ipynb


$$\newcommand{\ket}[1]{\left|{#1}\right\rangle}$$
$$\newcommand{\bra}[1]{\left\langle{#1}\right|}$$
$$\newcommand{\braket}[2]{\left\langle{#1}\middle|{#2}\right\rangle}$$

## Search on a cube cloud
The SKW search algorithm is based on discrete quantum walk on a hypercube and it is a convenient algorithm for searching a disordered database. In this algorithm, walk is performed on an n-dimensional hypercube and the number of data problem is $N = 2^n$.
We place the specified $N$ data on the vertices of the cube cloud and represent each vertex with an n-bit string. The two vertices x and y are connected only if the Hamming weight difference is one, i.e.

\begin{equation}
    |\bar{x} - \bar{y}| = 1.
\end{equation}

Figure 3.1 The data mapping on the hypercube grid is displayed in $d = 3$. Three-component bit strings represent the vertices, and the edges are labeled based on which bit must be flipped to reach the adjacent vertex. For the $n$ qubit string $\textbf{x} = (0010000010) $, Hamming weight is defined as follows [[S07]](https://arxiv.org/abs/0712.0625):
\begin{equation}
    |x = \sum_{i=1}^n x_i|.
\end{equation}

Since the cube cloud is symmetric and each vertex is connected to n edges, a quantum walk is done with the n-dimensional coin. Hilbert space and system state are defined as $H = H^n \otimes H^{2n}$ and $\ket{d,\bar{x}}$ respectively, where $d$ denotes the state of the coin. The conditional displacement operator also maps the state $\ket{d,\bar{x}}$ to the state $\ket{d,\bar{x} \oplus \bar{e}_d}$, representing the -th vector of the base of the hypercube (a vector whose all components are zero except the n-th component. Therefore, the unitary displacement operator is defined as follows [[S08]](https://arxiv.org/abs/quant-ph/0205083):

\begin{equation}
    S = \sum_{d,\bar{x}}^{n-1} \ket{d,\bar{x} \oplus \bar{e}_d} \bra{d,\bar{x}}
\end{equation}

According to this equation, the point's position will move to an adjacent vertex in the direction of the edge $d$, and the coin operator must be unitary transformation. In standard quantum spinning, only one coin operator is used, which is applied to all vertices of the graph (i.e., the coin operator does not change from one vertex to another), so the coin operator can be written as a separable:

\begin{equation}
    C = C. \otimes I
\end{equation}

Choosing the suitable coin operator is important in advancing the algorithm, since we tend to consider all the edges leading to the vertex of the target to be of equal value, so the probability of the point remaining in the current state $\ket{d,\bar{x}}$ for all values $d$ must be the same. Also, for all $d$ the range of transitions from $\ket{d,\bar{x}}$ to $\ket{d^{'},\bar{x}}$ must be the same. The only unit matrices that satisfy these conditions are: G, -G, I, -I. An appropriate and efficient choice is the Grover diffusion operator, which is a suitable unitary operator, and with this choice, the amplitudes of probability are propagated by quantum walk at the maximum possible speed [[S09]](http://stephanhoyer.com/pubs/thesis.pdf). This operator is defined as follows:

\begin{equation}
    C. = G = -I_n + \ket{s^c} \bra{s^c},
\end{equation}

Fig. 2) Quantum walk mapping on a cube cloud for one-dimensional walk, in n = 3.

Or in the matrix form:

\begin{equation}
    G = \begin{pmatrix}
\frac{2}{n} - 1 & ... & \frac{2}{n}\\
: & ... & : \\
\frac{2}{n} & ... & \frac{2}{n} -1
\end{pmatrix},
\end{equation}

In which

\begin{equation}
    \ket{s^c} = \frac{1}{\sqrt{n}} \sum_{d=1}^n \ket{d},
\end{equation}

where $\sum_{d=1}^n \ket{d}\bra{d} = I_n$.

A classical walk on a Hypercube has excellent symmetry. For example, if we start from the vertex, all vertices with the same Hamming weight can be moved together without correcting the quantum walk. This permutation applies to other vertices with the same Hamming weight. Slowly This symmetry requires that all vertices with the same Hamming weight in the probability distribution fall.

Classically, they have the same weight, so we can add vertices of the same weight at the same vertex and reduce the random walk on the cube to a balanced spin on the line [[S10]](https://arxiv.org/abs/quant-ph/0303081). The probability of transition from vertex $i$ to vertex $i+1$ is equal to $P_{i,i+1} = \frac{d - i}{d}$. In addition to being uniform and highly diffused, the Graver coin also maintains permutation symmetry. The Gravure coin is inaccurate concerning the simultaneous permutation of rows and columns. In addition, it is not a balanced coin, meaning that the probability that the coin will not change direction is not equal to the probability that the coin will change direction [[S10]](https://arxiv.org/abs/quant-ph/0303081). To analyze the dynamics of the quantum walk, we need to consider the transformation operator: $U = S.(C.\otimes I)$. For this purpose, the general state of the system after  steps can be written as follows,

\begin{equation}
    \ket{\psi (t)} = \sum_{d=1}^n \sum_{\bar{x} = 1}^{2^n - 1} \psi_{d,\bar{x}} (t) \ket{d,\bar{x}},
\end{equation}

where $\psi_{d,\bar{x}} (t)$ is the probability of amplitude and normal condition would be accrue. In general, this equation is very difficult to solve, but the same efficient Fourier transform method can be used to diagonalize this operator. In computational foundations, the discrete Fourier transform on space is as follows:

\begin{equation}
    \ket{\bar{k}} = \frac{1}{\sqrt{2^n}} \sum_{\bar{x} = 1}^{2^n - 1} (-1)^{\bar{K},\bar{x}} \ket{\bar{x}},
\end{equation}

Where $\ket{\bar{k}}$ comes from Fourie transformation. Now, by diagonalizing the transformation operator, the following equation can be obtained,

\begin{equation}
    e^{\pm i \omega_k} = 1 - \frac{2k}{n} \pm 2i \sqrt{\frac{k}{n} (1 - (k/n))}.
\end{equation}

Therefore, the specifics of normal transformations vector operators can be written as follows

\begin{equation}
    \ket{v_k},\ket{v_k}^* = \sum_{\bar{x},d} (-1)^{\bar{k}.\bar{x}} \frac{2^{-\frac{n}{2}}}{\sqrt{2}} \ket{d,\bar{x}} \times \begin{matrix}
1/\sqrt{k} && k_d = 1 \\
\mp i / \sqrt{n -  k} && k_d = 0
\end{matrix}
\end{equation}

Because of the symmetry in the hypercube, vertices can be labeled so that the vertex is always marked $\bar{x}_{tg} = \bar{0}$, so without diminishing the totality of the problem, we always assume that the vertex sought is $\bar{0}$. Therefore, the location of this vertex is not important. As a result, the disturbance in the coin operator, which indicates the spatial dependence of the new operator, is described as follows

\begin{equation}
     C^{'} = C.\otimes (I_{2^n} - \ket{\bar{x}_{tg}} \bra{\bar{x}_{tg}} + C_1 \otimes \ket{\bar{x}_{tg}} \bra{\bar{x}_{tg}}),
\end{equation}

In this phrase, the first sentence, the coin $C_0$, is applied to all vertices except the vertex of the target. With these descriptions, the unit transformation operator is disrupted as follows

\begin{align*}
    U^{'} &= S.C^{'} \\
    &= S.(G \otimes I - (G - I) \otimes \ket{\textbf{0}}\bra{\textbf{0}})\\
    &= U - 2S.(\ket{s^C}\bra{s^C} \otimes \ket{\textbf{0}}\bra{\textbf{0}})
\end{align*}

Analysis of the effect of this disorder will lead to the definition of a quantum walk search algorithm. The conditions are created for the particle that initially starts from a uniform distribution in the whole space to eventually converge to the vertex of the search by applying a small perturbation. This process is contrary to the standard quantum walk, in which the particle starts moving from a point and spreads throughout space.

Fig. 3) Simulation of the search algorithm on a cube, in n = 2 mode, when the searched vertex is $\ket{0}$ [[S09]](http://stephanhoyer.com/pubs/thesis.pdf).

Fig. 4) Quantum circuit of quantum spin search algorithm on n-dimensional transduce network.

.ipynb_checkpoints/Search on a cube cloud-checkpoint.ipynb


### Sverdrup Balance in observations

```python
import numpy as np
import xarray as xr
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import matplotlib.pyplot as plt
```

```python
%matplotlib inline
```

In this assignment, we are going to compute the Sverdrup streamfunction from the wind stress. As a reminder, the Sverdrup balance is
\begin{equation}
    \beta V = \frac{\vec{\nabla} \times \vec{\tau}}{\rho}
\end{equation}

Download the zonal and meridional GODAS momentum flux (=wind stress in [N/m$^2$]) files for one year from the GODAS website at https://psl.noaa.gov/data/gridded/data.godas.html (via 'List of *.nc files'>'See list'>'Save Link as') and use their time average from here on.

```python
# loading and inspecting
data = '/Users/Andre/Downloads'
uflx = xr.open_dataset(f'{data}/uflx.2020.nc')
vflx = xr.open_dataset(f'{data}/vflx.2020.nc')
taux = uflx.uflx.mean('time')
tauv = vflx.vflx.mean('time')
uflx
```

#### (a) Determine the length (in m) of the grid cells in the x- and y-directions and plot them as a function of latitude. Also compute $\beta$ as a function of latitude and plot it in the same figure with a secondary y-axis (using e.g. `ax.twinx()`).

#### (b) Plot maps of (1) the zonal and (2) meridional wind stress components, (3) the total wind stress with streamlines (using `plt.streamplot()`), and (4) wind stress curl with its (5) zonal and (6) meridional components.

#### (c) Integrate the wind curl  to arrive at the Sverdrup streamfunction.
$$ \Psi = \frac{1}{\beta \rho} \int_{E}^{W} \left( \frac{d\tau_{wy}}{dx}-\frac{d\tau_{wx}}{dy} \right) dx$$

There are two step:
1. integrate East to West with the proper longitudinal grid size.
2. fix discontinuity at the null meridian in the Atlantic.

#### Why do you integrate East to West and not the other way around?

Hints:
- You should arrive at a plot similar to the one on slide 19 of lecture 6.
- As always in pogramming, there are many roads to Rome. We would recommend keeping the data within the xarray DataArrays/Datasets as much as possible, but you could also use numpy arrays (and then have to keep track of your own indices).
- the following code gives an example of how you can fix the null meridian issue in the Atlantic (in this case for a single latitude, this code easily extends to 2D):
```
example = xr.DataArray(dims=['lon'],
                       coords={'lon':[1,2,3,4,256,257,258,359,360]},
                       data=[5,4,2,np.nan,1,0,np.nan,3,0])
# data representating:     (East-Atl.)..(S.Amer.)..(West-Atl.)
example.plot()  # here the West-Atlantic starts at 0 in the null meridian
da_add = xr.zeros_like(example)
da_add[-1] = example[0]
for i in np.arange(2,10):  # fixing zero meridian
    da_add[-i] += xr.where(np.isnan(example[-i]), np.nan, da_add[-i+1])
fixed = (example+xr.where(np.isnan(da_add),0,da_add))
fixed.plot()  # now the values continue smoothly across the Atlantic null meridian from East to West
```

#### (d) Explain the features that you see. How is the Sverdrup flow and what happens at the Western boundaries, the Aghulhas Retroflection, and in the Antarctic Circumpolar Current regions?

```python

```


Copyright 2021 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

# Bayes Rule for Covid

Heavily influenced by Chapter 2 of [Kevin Murphy's *Probabalistic Machine Learning*](https://github.com/probml/pyprobml/blob/master/book1/chapters/chap2.md). Here we define and examine the relationship between sensitivity, specificity, prevalence, accuracy, precision, recall, and the posterior probability estimate.

1. A Bayesian approach to toy examples for the Covid PCR test.
   - define sensitivity, specificity
   - define precision and recall
2. Derive identities relating precision and accuracy to the others statistics.
3. Use that derivation to see how varying these measures affects the posterior.

In this analysis we assume that the statistics of a particular test are fixed but unknown; we allow the statistics to vary to understand its affect on the posterior. For an approach which does not assume these rates to be fixed but varying, see [Gelman and Carpenter (2020)](https://www.medrxiv.org/content/10.1101/2020.05.22.20108944v2.abstract).

# The Posterior

```python
import numpy as np

def normalize(x):
  return x / np.sum(x)

def posterior_covid(pcr_result,       # observation
                    prevalence=.1,    # prior
                    sensitivity=.875, # hyper-parameter
                    specificity=.975  # hyper-parameter
                    ):
  # observed pcr_result = 0 for negative test, 1 for positive test
  # hidden state = 0 if no-covid, 1 if have-covid
  TPR = sensitivity;
  FNR = 1-TPR
  TNR = specificity
  FPR = 1-TNR

  # likelihood(hidden, obs)
  likelihood_fn = np.array([[TNR, FPR],
                            [FNR, TPR]])

  # prior(hidden)
  prior = np.array([1-prevalence, prevalence])
  likelihood = likelihood_fn[:, pcr_result].T
  posterior = normalize(prior * likelihood)
  return posterior[1]
```

```python
def prevalence_report(prevalence):
  print("prevalence = %.0f%%" % (prevalence*100))
  for i, label in enumerate(("negative", "positive")):
    print(f"PCR:{label} -> p(have covid) = "
          f"{100*posterior_covid(i, prevalence):4.1f}%")
```

```python
prevalence_report(.1)
```

    prevalence = 10%
    PCR:negative -> p(have covid) =  1.4%
    PCR:positive -> p(have covid) = 79.5%

```python
prevalence_report(.01)
```

    prevalence = 1%
    PCR:negative -> p(have covid) =  0.1%
    PCR:positive -> p(have covid) = 26.1%

# Priors
How do they affect our model?

```python
from matplotlib import pyplot as plt
from matplotlib.collections import PatchCollection
from matplotlib.patches import Rectangle

min_prior, max_prior = 0,.2

X = np.linspace(min_prior, max_prior, 100)

fig, axes = plt.subplots(1, 2, sharex=True, sharey=True)
fig.suptitle('How does posterior probability vary '
             'as prior (i.e. prevalence) varies?')

for i, (ax,label) in enumerate(zip(axes, "false true".split())):
  ax.plot(X, [posterior_covid(i, x) for x in X])
  ax.set_title("PCR:%s" % label)
  # ax.add_collection(
  #     PatchCollection(
  #         [Rectangle((min_prior, posterior_covid(1, min_prior)),
  #                     max_prior, posterior_covid(1, max_prior))],
  #                     edgecolor="black", facecolor="white"))
  if not i:
    ax.set_ylabel("probability")
    ax.set_xlabel("prevalence")
  else:
    prior = .1
    plt.plot(prior, posterior_covid(i, prior), marker="o", color='black')
    prior = .01
    plt.plot(prior, posterior_covid(i, prior), marker="o", color='black')

```

What does this graph mean? If 1% of the population has covid, a positive PCR test means you have a 26% chance of having Covid. "Oh no!" you might say, that means I probably don't have it even if I test positive. This is true, though we might ask how much more likely you are to have Covid with a positive test vs having Covid with a negative test. A 26% chance of having Covid vs a .1% chance of having covid is a pretty big difference: your chances of having Covid are now 260 times higher because of the positive test result.

```python
np.log(.261/.001), np.log(.795/.014)
```

    (5.564520407322694, 4.039284785039073)

It is all well and good that we examine how our modeled probabilities vary with our prior. But it is easy to see that our prior is not the only assumption of this model. We are also making assumptions (supported with evidence) about the sensitivity and specificity of the PCR test. How do perturbations here affect our modeled probability? This is a different analysis to do conceptually but it is straightforward to see.

# Sensitivity
How does it affect our model?

```python
X = np.linspace(.6, 1, 100)

fig, axes = plt.subplots(1, 2, sharex=True, sharey=True)
fig.suptitle('How does posterior probability vary as sensitivity varies?')

for i, (ax,label) in enumerate(zip(axes, "false true".split())):
  ax.plot(X, [posterior_covid(i, sensitivity=x) for x in X])
  if not i:
    ax.set_ylabel("probability")
    ax.set_xlabel("sensitivity")
  ax.set_title("PCR:%s" % label)
```

Answer: if sensitivity varies, our modeled probabilities do not change a great deal.

# Specificity
How does it affect our model?

```python
X = np.linspace(.6, 1, 100)

fig, axes = plt.subplots(1, 2, sharex=True, sharey=True)
fig.suptitle('How does posterior probability vary as specificity varies?')

for i, (ax,label) in enumerate(zip(axes, "false true".split())):
  ax.plot(X, [posterior_covid(i, prevalence=.01, specificity=x) for x in X])
  if not i:
    ax.set_ylabel("probability")
    ax.set_xlabel("specificity")
  ax.set_title("PCR:%s" % label)
```

```python
# WOW
print(
    "*If* 10% of the population was infected and you test positive with a\n"
    "90%-specificity test, you would only have a "
    f"{100*posterior_covid(1, specificity=.9):.0f}% chance of actually having it.\n")
```

    *If* 10% of the population was infected and you test positive with a
    90%-specificity test, you would only have a 49% chance of actually having it.

# Priors and Specificity

Wow, the posterior probability looks to vary **extremely dramatically** as specificity varies.

Even a specificity of 90% wouldn't tell you much: if 10% of the population was infected, you would only have a 49% chance of actually having it if you tested positive.

Let's look at those curves for a few priors.

```python
num_rows = 5
priors = 10**np.linspace((np.log(.5) / np.log(10)), -4, num_rows)

fig, axes = plt.subplots(len(priors), 2, sharex=True, sharey=True)
for j, (prior, row) in enumerate(zip(priors, axes)):
  for i, (ax,label) in enumerate(zip(row, "false true".split())):
    ax.plot(X, [posterior_covid(i, prevalence=prior, specificity=x) for x in X])
    if not i and not j:
      ax.set_ylabel("probability")
    if not i:
      ax.set_ylabel("prior=%.2f%%" % (100*prior))
    if not j:
      ax.set_title("PCR:%s" % label)
      ax.set_xlabel("specificity")

fig.set_figheight(10)
```

### Upshot

If prevalence is low (under 8%) then specificity needs to be increasingly close to 100% in order to be able to conclude anything meaningful from `PCR=True`. This is less true for sensitivity/recall.

# Other Quality Statistics

There are several motivating questions:
1. How to measure how good a classifier is?
1. How can we quantify how good a test is?

If we were a toy manufacturer and some toys were defective, we might want to issue a recall of them. If we wanted be really sure we are really getting almost all of them, we would want a classification test with **high recall**. For the Covid PCR test, though, the point of the test is to empower individuals to know if they probably have it (i.e. have it with high posterior probability, given priors). This is better captured by a specificity measure, as we saw above. We will see below this is also well captured by a precision measure.

Another useful recall mnemonic is a memorization task. Given some items to remember (actual positives), what fraction of them can you *recall* (i.e. test positive).

Above, we defined sensitivity (aka recall), specificity, and prevalence. Now we define two other measures.

- Precision := $\frac {TP}{TP+FP}$.
- Accuracy := $\frac {TP+TN}{TP+FP+TN+FN}$.

In practice, different fields use 2 from that list of 4 and [everyone looks up](https://en.wikipedia.org/wiki/Confusion_matrix) the definition again if one of the measures is not listed. Just because different fields use different statistics, we don't need to confine ourselves to just a few because, thankfully, we can translate between each of them -- which is precisely what we will do below.

# Toy Example (aka a unit test)

We go through a toy example to test our understanding of each of the statistics and at the same time check our solutions to the exercises below.

Now let us suppose we have a PCR test but we don't actually know how effective it is. We have 17 sampled patients and we know some of them really do have Covid (say,  for the moment, we had some other way to *know* this) and some do not. We can use the actual prevalence of Covid in our population as a proxy for the prevalence in general, so from this small sample we can compute all the statistics we could care to know. And we will.

```python
#@title Solutions (hidden)

def hidden_solution_precision2specificity(recall, prevalence, precision):
  r = recall
  w = prevalence
  x = precision
  return (x * (1 - w * (1 - r)) - w*r) / (x * (1-w))

def hidden_solution_accuracy2specificity(sensitivity, prevalence, accuracy):
  r = sensitivity
  v = prevalence
  a = accuracy
  return (r*v-a)/(v-1)
```

```python
from collections import Counter
import math

def precision2specificity(recall, prevalence, precision):
  # Hey this parameterization is possible! See exercises on how to derive it.
  return hidden_solution_precision2specificity(recall, prevalence, precision)

def accuracy2specificity(sensitivity, prevalence, accuracy):
  # Hey this parameterization is possible! See exercises on how to derive it.
  return hidden_solution_accuracy2specificity(sensitivity, prevalence, accuracy)

def test():
  predicted = [1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,]
  actual    = [1,1,1,1,1,1,1,1,0,0,1,0,0,0,0,0,0,]
  labels    = "a a a a a a a a b b c d d d d d d"
  """ Here we think of the confusion matrix
        TP FP    ==    a b
        FN TN          c d
        """
  true_positive  = sum(    a for a,p in zip(actual, predicted) if     p)
  false_negative = sum(    a for a,p in zip(actual, predicted) if not p)
  false_positive = sum(not a for a,p in zip(actual, predicted) if     p)
  true_negative  = sum(not a for a,p in zip(actual, predicted) if not p)

  # Eesh that math just now was hard. Better double-check that:
  counts = Counter(labels.split())
  assert counts["a"] == true_positive
  assert counts["b"] == false_positive
  assert counts["c"] == false_negative
  assert counts["d"] == true_negative

  print(f'Of {len(actual)} patients, {sum(actual)} really had covid and '
        f'{sum(predicted)} tested positive.\n')
  print(f'true_positive:  {true_positive:2d}')
  print(f'false_positive: {false_positive:2d}')
  print(f'false_negative: {false_negative:2d}')
  print(f'true_negative:  {true_negative:2d}')

  sensitivity = recall = true_positive / float(sum(actual))
  specificity = true_negative / float(sum(not(a) for a in actual))
  precision = true_positive / float(true_positive + false_positive)
  accuracy = (true_positive + true_negative) / float(len(actual))
  prevalence = float(sum(actual)) / float(len(actual))

  print(f'sensitivity (AKA recall): {sensitivity*100:.2f}%')
  print(f'specificity :             {specificity*100:.2f}%')
  print(f'precision :               {precision*100:.2f}%')
  print(f'accuracy :                {accuracy*100:.2f}%')
  print(f'prevalence :              {prevalence*100:.2f}%')

  # Eesh changing variables is hard, better double-check them:
  assert math.isclose(precision2specificity(recall, prevalence, precision),
                      specificity)
  assert math.isclose(accuracy2specificity(sensitivity, prevalence, accuracy),
                      specificity)

test()
```

    Of 17 patients, 9 really had covid and 10 tested positive.

    true_positive:   8
    false_positive:  2
    false_negative:  1
    true_negative:   6
    sensitivity (AKA recall): 88.89%
    specificity :             75.00%
    precision :               80.00%
    accuracy :                82.35%
    prevalence :              52.94%

### The Confusion Matrix

This information is conveniently displayed in a table which is so important it has its own name: **the Confusion Matrix**.

```
      Actual Class           Covid    No Covid
Predicted
  Covid                      8       2
  No Covid                   1       6
```

```
      Actual Class           Covid    No Covid
Predicted
  Covid                      TP       FP
  No Covid                   FN       TN
```

## Discussion

With the above definitions we can re-do the above analysis but for each of these other measures in addition to sensitivity and specificity.

To do that, we need to solve for specificity (and/or accuracy) in terms of recall, precision, and prevalence.

How do we do that?

All we need to do is see that TP,FP,FN,TN are rates so without loss of generality we can let FP=1, and we can parameterize each of TP, FN, and TN by recall, precision, and prevalence.

Here is one handy parameterization that you can verify. We can succinctly describe it with an array of whole numbers:

\begin{align}\begin{pmatrix}
TP & FP \\
FN & TN \\
\end{pmatrix} = \begin{pmatrix}
a & b \\
c & d \\
\end{pmatrix}\end{align}

Then, by their definitions:
- prevalence=:$v=\frac{a+c}{a+b+c+d}$
- sensitivity=recall=:$r=\frac{a}{a+c}$
- specificity=:$s=\frac{d}{b+d}$
- precision=:$\frac{a}{a+b}$
- accuracy=:$\frac{a+d}{a+b+c+d}$.

Solving the first three equations for a,c,d yields:
\begin{align}
\begin{pmatrix}
a & b \\
c & d \\
\end{pmatrix} = \begin{pmatrix}
b\frac{v}{1-v}\frac{r}{1-s} & b \\
b\frac{v}{1-v}\frac{1-r}{1-s} & b\frac{s}{1-s} \\
\end{pmatrix}\end{align}

We can use this parameterization to solve for specificity given precision or solve for specificity given accuracy.

If $v = \frac{a+c}{a+b+c+d}, r = \frac {a}{a+c}, s = \frac{d}{b+d}$, solve for a,c,d (which [pretty much works verbatim in wolfram alpha](https://www.wolframalpha.com/input/?i=If+v%3D%28a%2Bc%29%2F%28a%2Bb%2Bc%2Bd%29%2C+r%3Da%2F%28a%2Bc%29%2C+s%3Dd%2F%28d%2Bb%29%2C+solve+for+a%2C+c%2C+d)), yielding several useful nondegenerate parameterizations, in particular:

\begin{align}
a &=& b\cdot\frac {v} {1-v} \frac r{1-s} \\
c &=& b\cdot\frac v{1-v}\frac{1-r}{1-s} \\
d &=& b\cdot\frac {s}{1 - s}
\end{align}

Note, the sensible nondegeneracy, i.e. whenever $v,s \neq 1$ and $b r v\neq 0$. This is satisfied by our implicit assumpition that $s,r,v$ are between 0 and 1.

Note also that since we are only interested in rates, we can parameterize setting any of $a,b,c,d$ to 1, and in this case we do so with $b$.

# Exercises

1. Derive specificity in terms of sensitivity, prevalence, and *precision*.
1. Derive specificity in terms of sensitivity, prevalence, and *accuracy*.

# Solutions

1. Let
  - sensitivity $=r$
  - prevalence $=v$
  - precision $=c$

  Then:
\begin{align}
\mathrm{specificity} = \frac{c-v c +crv - rv} {c - cv}
\end{align}

2.  Let
  - sensitivity $=r$
  - prevalence $=v$
  - accuracy $=a$

  [Then](https://www.wolframalpha.com/input/?i=If+x%3D%28vr%2F%28%281-v%29%281-s%29%29+%2B+%28s%2F%281-s%29%29%29+%2F+%28vr%2F%28%281-v%29%281-s%29%29+%2B+1+%2B+v%281-r%29%2F%28%281-v%29%281-s%29%29+++%2B%28s%2F%281-s%29%29%29%2C+solve+for+s):
\begin{align}
\mathrm{specificity} = \frac{rv-a} {v-1}
\end{align}

# Precision and Accuracy

How would differently precise or differently accurate tests affect our model?

The nice thing about the solutions to the above exercises is they allow us to re-use verbatim `posterior_covid`.

```python
def posterior_covid_precision(
    pcr_result,
    recall=.875,
    prevalence=.1,
    precision=.975):
  specificity = precision2specificity(recall, prevalence, precision)
  return posterior_covid(pcr_result,
                         prevalence=prevalence,
                         sensitivity=recall,
                         specificity=specificity)

def posterior_covid_accuracy(
    pcr_result,
    sensitivity=.875,
    prevalence=.1,
    accuracy=.9):
  specificity = accuracy2specificity(sensitivity, prevalence, accuracy)
  return posterior_covid(pcr_result,
                         prevalence=prevalence,
                         sensitivity=sensitivity,
                         specificity=specificity)
```

```python
X = np.linspace(.6, 1, 100)

fig, axes = plt.subplots(1, 2, sharex=True, sharey=True)
fig.suptitle('How does posterior probability vary as precision varies?')

for i, (ax,label) in enumerate(zip(axes, "false true".split())):
  ax.plot(X, [posterior_covid_precision(i, precision=x) for x in X])
  if not i:
    ax.set_ylabel("probability")
    ax.set_xlabel("precision")
  ax.set_title("PCR:%s" % label)
```

Precision seems to varies linearly with the posterior probability. It looks quite similar to the posterior in fact. This is nice. We can assert this:

```python
# Look! It's true! The precision is the posterior probabilty.
assert all(math.isclose(posterior_covid_precision(i, precision=p), p)
           for p in np.linspace(0.001,.999,100))
```

```python
X = np.linspace(.6, 1, 100)

fig, axes = plt.subplots(1, 2, sharex=True, sharey=True)
fig.suptitle('How does posterior probability vary as accuracy varies?')

for i, (ax,label) in enumerate(zip(axes, "false true".split())):
  ax.plot(X, [posterior_covid_accuracy(i, accuracy=x) for x in X])
  if not i:
    ax.set_ylabel("probability")
    ax.set_xlabel("accuracy")
  ax.set_title("PCR:%s" % label)
```

As accuracy improves, the posterior probability of a positive value goes up. We can now do the same thing we did with the prior&specificity investigation and see how varying priors affect how much accuracy impacts our posterior estimate.

```python
priors = 10**np.linspace((np.log(.5) / np.log(10)), -4, 5)

fig, axes = plt.subplots(len(priors), 2, sharex=True, sharey=True)
for j, (prior, row) in enumerate(zip(priors, axes)):
  for i, (ax,label) in enumerate(zip(row, "false true".split())):
    ax.plot(X, [posterior_covid_accuracy(i, prevalence=prior, accuracy=x) for x in X])
    if not i and not j:
      ax.set_ylabel("probability")
    if not i:
      ax.set_ylabel("prior=%.2f%%" % (100*prior))
    if not j:
      ax.set_title("PCR:%s" % label)
      ax.set_xlabel("accuracy")

fig.set_figheight(10)
```


##Practica 1 Ejercicio 5
Para cada uno de los siguientes sistemas encontrar todos los puntos de equilibrio y determinar el tipo de cada punto de equilibio aislado.

 $$\left\{ \begin{array}{lcc}
            \dot{x}_{1}=a x_{1}-x_{1}x_{2} \\
             \\ \dot{x}_{2}=bx_{1}^{2}-cx_{2}
             \end{array}
\right.$$

```python
import sympy as sym
```

```python
#Con esto las salidas van a ser en LaTeX
sym.init_printing(use_latex=True)
```

```python
x_1, x_2, a, b, c= sym.symbols('x_1 x_2 a b c')
```

```python
sym.Symbol("a",positive=True)
sym.Symbol("b",positive=True)
sym.Symbol("c",positive=True)
```

```python
X = sym.Matrix([x_1, x_2])
X
```

```python
f_1 = a * x_1 - x_1 * x_2
f_1
```

```python
f_2 = b * x_1 ** 2 - c * x_2
f_2
```

```python
F = sym.Matrix([f_1,f_2])
F
```

```python
# puntos de equilibrio del sistema
pes = sym.solve([f_1,f_2],[x_1,x_2])
pes
```

```python
A = F.jacobian(X)
A
```

```python
A_1 = A.subs({x_1:pes[0][0],x_2:pes[0][0]})
A_1
```

```python
A_2 = A.subs({x_1:pes[1][0],x_2:pes[1][1]})
A_2
```

```python
A_2.eigenvals()
```

```python
A_3 = A.subs({x_1:pes[2][0],x_2:pes[2][1]})
A_3
```

```python
A_3.eigenvals()
```

```python

```