# Scalars, vectors, matrices, and tensors
>Correspodence between mathematics and PyTorch.

- toc: true
- badges: true
- comments: true
- categories: [mathematics]

---

tags: scalars vectors matrices tensors mathematics linear algebra pytorch

---

# tl;dr
We establish the following correspondence between entities commonly encountered in linear algebra and their counterparts in `PyTorch`.

|mathematical name|mathematical notation|`tensor` shape|`tensor` dimension|
|---|---|---|---|
|scalar|$x$|`()`|`0`|
|vector|$(x_1, \dots, x_n)$|`(n,)`|`1`|
|linear transform| $w_{i,j}$
 $i=1, \dots, m$
$j=1, \dots, n$|N/A|N/A|
|matrix|$\begin{bmatrix}w_{1,1}&\dots&w_{1,n}\\\vdots&\ddots&\vdots\\w_{m,1}&\dots&w_{n,m}\end{bmatrix}$|`(m,n)`| `2`|
|column vector|$\begin{bmatrix}x_1\\\vdots\\x_n\end{bmatrix}$|`(n,1)`|`2`|
|row vector|$\begin{bmatrix}x_1&\dots&x_n\end{bmatrix}$|`(1,n)`|`2`|

>Note:
When we write `tensor`, we mean a `PyTorch` `tensor`.  Otherwise, "tensor" refers to the mathematical notion.

>Important:
The dimension of a _vector_ is the number of its entries.  The dimension of a _tensor_ is the number of indices needed to label its entries, in other words the length of its shape.

>Note: As a consequence, it is enough to refer to a `(m,n)`-`tensor` rather than to a `2`-dimensional `tensor` with shape `(m,n)`, but at times we will want to emphasize the dimension and make the information explicit.

# Scalars
A scalar is a real number.  It corresponds to a `0`-dimensional `tensor` with empty shape:

```python
from torch import tensor
x = tensor(3.14)
print("Example of a 0-dimensional tensor in PyTorch:")
print("-"*45)
print(x)
print(f"\nDimension:   {x.ndim}")
print(f"Shape:       {tuple(x.shape)}")
```

    Example of a 0-dimensional tensor in PyTorch:
    ---------------------------------------------
    tensor(3.1400)

    Dimension:   0
    Shape:       ()

# Vectors
An $n$-dimensional vector is a **list** of $n$ scalars:
\begin{equation}(x_1, \dots, x_n)\,.\end{equation}
It corresponds to a `0`-dimensional `tensor` of shape `(n,)`.

```python
# collapse-show
from torch import tensor
x = tensor([-1., 5., 7.])
print("Example of a 1-dimensional tensor in PyTorch:")
print("-"*45)
print(x)
print(f"\nDimension:   {x.ndim}")
print(f"Shape:       {tuple(x.shape)}")
```

    Example of a 1-dimensional tensor in PyTorch:
    ---------------------------------------------
    tensor([-1.,  5.,  7.])

    Dimension:   1
    Shape:       (3,)

# Linear transformations
A linear transformation which maps $n$-dimensional vectors to $m$-dimensional vectors can be identified with a doubly-indexed list of scalars $w_{i,j}$ where $i$ runs from $1$ to $m$ and $j$ from $1$ to $n$, in that they act on input vectors according to the familiar formula:
$$y_i = \sum_{j=1}^nw_{i,j}x_j\quad \textrm{for}\quad i=1, \dots, m\,.$$

>Tip:
The indices in $w_{i,j}x_j$ appear in the order $(i, j, j)$.  The dummy index $j$ is repeated next to itself and "disappears" upon summation, while the index $i$ remains.

# Column vectors
For computational purposes, instead of a vector $(x_1, \dots, x_n)$, it is convenient to work with its **column vector** counterpart, which is the array
$$\begin{bmatrix}x_1\\\vdots\\x_n\end{bmatrix}\,.
$$
>Important:
It is important to distinguish a vector and its corresponding column vector:
$$(x_1, \dots, x_n) \quad \ne\quad \begin{bmatrix}x_1\\\vdots\\x_n\end{bmatrix}\,.$$

Indeed, while the vector corresponds to a `1`-dimensional `tensor`, its column vector counterpart corresponds to a `2`-dimensional `tensor` with shape `(n,1)`:

```python
# collapse-show
from torch import tensor
x = tensor([-1., 5., 7.]).reshape(-1, 1)
print("Example of a 1-dimensional tensor in PyTorch:")
print("-"*45)
print(x)
print(f"\nDimension:   {x.ndim}")
print(f"Shape:       {tuple(x.shape)}")
```

    Example of a 1-dimensional tensor in PyTorch:
    ---------------------------------------------
    tensor([[-1.],
            [ 5.],
            [ 7.]])

    Dimension:   2
    Shape:       (3, 1)

# Matrices
An **$m$-by-$n$ matrix** is a doubly-indexed list $w_{i,j}$ where $i$ ranges from $1$ to $m$ and $j$ from $1$ to $n$.  It is usually written as an array with $m$ rows and $n$ columns:
$$\begin{bmatrix}
w_{1,1}&\dots&w_{1,j}&\dots&w_{1,n}\\
\vdots&&\vdots&&\vdots\\
w_{i,1}&\dots&w_{i,j}&\dots&w_{i,n}\\
\vdots&&\vdots&&\vdots\\
w_{m,1}&\dots&w_{m,j}&\dots&w_{m,n}
\end{bmatrix}
$$

>Important:
The first index in $w_{i,j}$ labels rows and the second index labels columns.

An $m$-by-$n$ matrix corresponds to a `2`-dimensional`tensor` with shape `(m,n)`:

```python
# collapse-show
import torch
n = 3
m = 4
x = torch.randn(n, m)
print("Example of a 2-dimensional tensor in PyTorch:")
print("-"*45)
print(x)
print(f"\nDimension:   {x.ndim}")
print(f"Shape:       {tuple(x.shape)}")
```

    Example of a 2-dimensional tensor in PyTorch:
    ---------------------------------------------
    tensor([[-0.3536,  1.9439, -0.0042, -0.8287],
            [-0.4937,  0.4641, -1.0579,  0.2006],
            [-0.4996, -1.2149, -0.9282,  0.1443]])

    Dimension:   2
    Shape:       (3, 4)

# Computation and compact representation of linear transformations
The _collection_ of $m$ formulas $y_i = \sum_{j=1}^nw_{i,j}x_j$, where $i$ runs from $1$ to $m$, is compactly represented as
$$
\begin{bmatrix}y_1\\\vdots\\y_m\end{bmatrix}
\,=\,\begin{bmatrix}w_{1,1}&\dots&w_{1,n}\\\vdots&\ddots&\vdots\\w_{m,1}&\dots&w_{m,n}\end{bmatrix}
\begin{bmatrix}x_1\\\vdots\\x_n\end{bmatrix}
$$
where the matrix is written to the left of the input column vector.

>Note:
The number of columns in the matrix matches the number of rows in the column vector.

# Row vectors
Identifying a vector with its column vector is the more common convention.  Alternatively, the vector $(x_1, \dots, x_n)$ can be identified with its **row vector** counterpart, which is the array
$$\begin{bmatrix}x_1&\dots&x_n\end{bmatrix}\,.$$

>Important:
Again, a vector, its column and row vector counterparts are all different objects:
$$(x_1, \dots, x_n) \quad \ne \quad \begin{bmatrix}x_1\\\vdots\\x_n\end{bmatrix}
\quad\ne\quad
\begin{bmatrix}x_1&\dots&x_n\end{bmatrix}
\quad\ne\quad
(x_1, \dots, x_n)\,.$$

Indeed, a row vector of length $n$ corresponds to a `2`-dimensional `tensor` with shape `(1,n)`:

```python
# collapse-show
from torch import tensor
x = tensor([-1., 5., 7.]).reshape(1, -1)
print("Example of a 1-dimensional tensor in PyTorch:")
print("-"*45)
print(x)
print(f"\nDimension:   {x.ndim}")
print(f"Shape:       {tuple(x.shape)}")
```

    Example of a 1-dimensional tensor in PyTorch:
    ---------------------------------------------
    tensor([[-1.,  5.,  7.]])

    Dimension:   2
    Shape:       (1, 3)

> Note:
Vectors and row vectors are all represented horizontally.  However, vectors are written with parentheses while row vectors are written with square brackets.

# Computing with row vectors
In terms of row vectors, the output of a linear transformation is more conveniently rewritten with the similar but different formulas
$$y_j = \sum_{i=1}^nx_i\tilde w_{i,j}\qquad \textrm{for}\quad j=1, \dots, m$$
where now the matrix with coefficients $\tilde w_{i,j}$ where $i$ runs from $1$ to $n$ and $j$ from $1$ to $m$, has $n$ rows and $m$ columns:
$$
\begin{bmatrix}
\tilde w_{1,1}&\dots&\tilde w_{1,j}&\dots&\tilde w_{1,m}\\
\vdots&&\vdots&&\vdots\\
\tilde w_{i,1}&\dots&\tilde w_{i,j}&\dots&\tilde w_{i,m}\\
\vdots&&\vdots&&\vdots\\
\tilde w_{n,1}&\dots&\tilde w_{n,i}&\dots&\tilde w_{n,m}
\end{bmatrix}\,.$$

>Tip:
The indices in $x_i\tilde w_{i,j}$ appear in the order $(i, i, j)$ (note the difference from earlier).  The dummy index, which is now $i$, is repeated next to itself.  The remaining index, which is now $j$, is the same as that in the quantity $y_j$ on the left of the equation.

The collection of $m$ formulas $y_j = \sum_{i=1}^nx_i\tilde w_{i,j}$, for $j$ from $1$ to $m$, is compactly represented as
$$
\begin{bmatrix}y_1&\dots&y_m\end{bmatrix}
\quad=\quad
\begin{bmatrix}x_1&\dots&x_n\end{bmatrix}
\begin{bmatrix}\tilde w_{1,1}&\dots&\tilde w_{1,m}\\\vdots&\ddots&\vdots\\\tilde w_{n,1}&\dots&\tilde w_{n,m}
\end{bmatrix}
$$
where the matrix is written to the right of the input row vector.

>Note:
The number of columns in the input row vector matches the number of rows in the matrix.

>Important:
The coefficients $\tilde w_{i,j}$ share the same values as the coefficients $w_{i,j}$ in the representation with column vectors, but they are coefficients of distinct matrices: the first is $n$-by-$m$ while the second is $m$-by-$n$.  In fact, these matrices are transposes of each other:
$$
\begin{bmatrix}\tilde w_{1,1}&\dots&\tilde w_{1,m}\\\vdots&\ddots&\vdots\\\tilde w_{n,1}&\dots&\tilde w_{n,m}
\end{bmatrix}
\quad=\quad
\begin{bmatrix}w_{1,1}&\dots&w_{1,n}\\\vdots&\ddots&\vdots\\w_{m,1}&\dots&w_{m,n}\end{bmatrix}^\top
\quad=\quad
\begin{bmatrix}w_{1,1}&\dots&w_{m,1}\\\vdots&\ddots&\vdots\\w_{1,n}&\dots&w_{m,n}\end{bmatrix}
\,.$$
Equivalently,
$$\tilde w_{i,j} = w_{j,i}\qquad \textrm{for}\quad i=1, \dots, n\,,\quad j=1, \dots, m\,.$$


April 2015, J. Slavič in L. Knez

Vprašanje 1: V simbolni obliki definirajte izraz:
$$f(x) = x^4 - 6.4\,x^3 + 6.45\,x^2 + 20.538\,x - 31.752$$

```python
from sympy import *
init_printing()
import matplotlib.pyplot as plt
```

```python
x = symbols('x')
f = x**4 - 6.4*x**3 + 6.45*x**2 + 20.538*x - 31.752
f
```

Vprašanje 2: Narišite funkcijo $f(x)$. Koliko ničel pričakujemo za funkcijo $f(x)$?

```python
%matplotlib inline
plot(f, (x, -2, 5));
```

Glede na sliko pričakujemo 3 ničle.

Vprašanje 3: V simbolni obliki določite ničle polinoma $f(x)$.

```python
sol = solve(f, x)
sol
```

Preverimo, če so rešitve ustrezne. Če x-e vstavimo v funkcijo, potem mora biti njena vrednost enaka 0.

```python
[f.subs({x: _}).evalf() for _ in sol]
```

Vprašanje 4: Kako preverimo ali je ničla ekstrem?

Ekstreme funkcije je točka, kjer funkcija doseže minimum ali pa maksimum. Posledično je tangenta na premico v tej točki vodoravna (njen odvod je enak 0).

```python
[f.diff(x).subs({x: _}).evalf() for _ in sol]
```

Vidimo, da je druga ničla ekstrem.

Vprašanje 5: Numerično najdite vse ničle z bisekcijsko metodo (uporabite ``scipy``).

```python
from scipy import optimize   # Uvozimo vse metode za iskanje ničel
```

```python
def f(x):   # Pripravimo funkcijo, ki jo bomo uporabili za numerični izračun
    return x**4 - 6.4*x**3 + 6.45*x**2 + 20.538*x - 31.752
```

```python
optimize.bisect(f, -2, 5, xtol=1e-3)
```

Kot vidimo, nam bisekcijska metoda javi napako. Predznak funkcije je namreč enak pri f(-2) in pri f(5). Zmanjšajmo interval iskanja in poiščimo vsako ničlo posebej. Intervale iskanja določimo na podlagi izrisane funkcije.

```python
b0_1 = optimize.bisect(f, -2, 1, xtol=1e-3)
b0_1
```

```python
b0_2 = optimize.bisect(f, 1, 3, xtol=1e-3)   # Ta je problematična, saj je predznak funkcije na obeh straneh isti
b0_2
```

```python
b0_3 = optimize.bisect(f, 3, 5, xtol=1e-3)
b0_3
```

Vidimo, da smo z bisekcijsko metodo uspešno lahko poiskali le dve ničli.

Vprašanje 6: Numerično najdite vse ničle z sekantno metodo (uporabite ``scipy``).

Katero ničlo bo metoda vrnila je odvisno od začetnega približka. Metoda navadno konvertira k najbližji ničli (ni pa to vedno nujno).

```python
prva_nicla = optimize.newton(f, -2)
prva_nicla
```

```python
druga_nicla = optimize.newton(f, 1)
druga_nicla
```

```python
druga_nicla = optimize.newton(f, 3.5)   # Poskusite tu z začetnim približkom 3
druga_nicla
```

pypinm-master/vprasanja za razmislek/Vaja 9 - polovica.ipynb


#  Taylor problem 3.23

last revised: 04-Jan-2020 by Dick Furnstahl [furnstahl.1@osu.edu]

**This notebook is almost ready to go, except that the initial conditions and $\Delta v$ are different from the problem statement and there is no statement to print the figure.  Fix these and you're done!**

This is a conservation of momentum problem, which in the end lets us determine the trajectories of the two masses before and after the explosion.  How should we visualize that the center-of-mass of the pieces continues to follow the original parabolic path?

Plan:
1. Plot the original trajectory, also continued past the explosion time.
2. Plot the two trajectories after the explosion.
3. For some specified times of the latter two trajectories, connect the points and indicate the center of mass.

The implementation here could certainly be improved!  Please make suggestions (and develop improved versions).

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
```

First define some functions we think we will need.  The formulas are based on our paper-and-pencil work.

The trajectory starting from $t=0$ is:

$
\begin{align}
  x(t) &= x_0 + v_{x0} t  \\
  y(t) &= y_0 + v_{y0} t - \frac{1}{2} g t^2
\end{align}
$

```python
def trajectory(x0, y0, vx0, vy0, t_pts, g=9.8):
    """Calculate the x(t) and y(t) trajectories for an array of times,
        which must start with t=0.
    """
    return x0 + vx0*t_pts, y0 + vy0*t_pts - g*t_pts**2/2.
```

The velocity at the final time $t_f$ is:

$
\begin{align}
  v_{x}(t) &= v_{x0}  \\
  v_{y}(t) &= v_{y0}  - g t_f
\end{align}
$

```python
def final_velocity(vx0, vy0, t_pts, g=9.8):
    """Calculate the vx(t) and vy(t) at the end of an array of times t_pts"""
    return vx0, vy0 - g*t_pts[-1]  # -1 gives the last element
```

The center of mass of two particles at $(x_1, y_1)$ and $(x_2, y_2)$ is:

$
\begin{align}
 x_{cm} &= \frac{1}{2}(x_1 + x_2)  \\
 y_{cm} &= \frac{1}{2}(y_1 + y_2)
\end{align}
$

```python
def com_position(x1, y1, x2, y2):
    """Find the center-of-mass (com) position given two positions (x,y)."""
    return (x1 + x2)/2., (y1 + y2)/2.
```

**1. Calculate and plot the original trajectory up to the explosion.**

```python
# initial conditions
x0_before, y0_before = [0., 0.]    # put the origin at the starting point
vx0_before, vy0_before = [6., 3.]  # given in the problem statement
g = 1.  # as recommended

# Array of times to calculate the trajectory up to the explosion at t=4
t_pts_before = np.array([0., 1., 2., 3., 4.])
x_before, y_before = trajectory(x0_before, y0_before,
                                vx0_before, vy0_before,
                                t_pts_before, g)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(x_before, y_before, 'ro-')

ax.set_xlabel('x')
ax.set_ylabel('y')
```

Does it make sense so far?  Note that we could use more intermediate points to make a more correct curve (rather than the piecewise straight lines) but this is fine at least for a first pass.

**2. Calculate and plot the two trajectories after the explosion.**

For the second part of the trajectory, we reset our clock to $t=0$ because that is how our trajectory function is constructed.  We'll need initial positions and velocities of the pieces just after the explosion.  These are the final position of the combined piece before the explosion and the final velocity plus and minus $\Delta \mathbf{v}$.  We are told $\Delta \mathbf{v}$.   We have to figure out the final velocity before the explosion.

```python
delta_v = np.array([2., 1.])   # change in velociy of one piece

# reset time to 0 for calculating trajectories
t_pts_after = np.array([0., 1., 2., 3., 4., 5.])
# Also could have used np.arange(0.,6.,1.)

x0_after = x_before[-1]   # -1 here means the last element of the array
y0_after = y_before[-1]
vxcm0_after, vycm0_after = final_velocity(vx0_before, vy0_before,
                                          t_pts_before, g)
# The _1 and _2 refer to the two pieces after the explosinon
vx0_after_1 = vxcm0_after + delta_v[0]
vy0_after_1 = vycm0_after + delta_v[1]
vx0_after_2 = vxcm0_after - delta_v[0]
vy0_after_2 = vycm0_after - delta_v[1]

# Given the initial conditions after the explosion, we calculate trajectories
x_after_1, y_after_1 = trajectory(x0_after, y0_after,
                                  vx0_after_1, vy0_after_1,
                                  t_pts_after, g)
x_after_2, y_after_2 = trajectory(x0_after, y0_after,
                                  vx0_after_2, vy0_after_2,
                                  t_pts_after, g)
# This is the center-of-mass trajectory
xcm_after, ycm_after = trajectory(x0_after, y0_after,
                                  vxcm0_after, vycm0_after,
                                  t_pts_after, g)

# These are calculated points of the center-of-mass
xcm_pts, ycm_pts = com_position(x_after_1, y_after_1, x_after_2, y_after_2)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(x_before, y_before, 'ro-', label='before explosion')
ax.plot(x_after_1, y_after_1, 'go-', label='piece 1 after')
ax.plot(x_after_2, y_after_2, 'bo-', label='piece 2 after')
ax.plot(xcm_after, ycm_after, 'r--', label='original trajectory')
ax.plot(xcm_pts, ycm_pts, 'o', color='black', label='center-of-mass of 1 and 2')
for i in range(len(t_pts_after)):
    ax.plot([x_after_1[i], x_after_2[i]],
            [y_after_1[i], y_after_2[i]],
            'k--'
           )
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.legend();
```

```python

```

```python

```


# The Cournot Model

### Imports:

```python
import numpy as np
import scipy as sp
import sympy as sm
import matplotlib.pyplot as plt
import ipywidgets as widgets
import pylab
from scipy import optimize
#We initiate our printing as nice equations, using the sympy package
sm.init_printing(use_unicode=True)

```

## Model Description

In the model of Cournot Competition, two firms compete for the amount they produce, by simulataneously setting quantities.

**We use the following parameters:**

1. q1 is the quantity produced by firm 1
2. q2 is the quantity produced by firm 2
3. c is the cost, which is equal for both firms
4. a,b are parameters of the price functions of the firms
5. pi1 and pi2 are profits of firms 1 and 2

**Preliminaries of the model:**

1. Both firms produce a homogenous product
2. There is no collusion between the firms
3. Both firms have identical costs and the same price functions
4. The firms are economically rational

**Prices evolve according to:**

p1 = (a-b*(q1+q2))

p2 = (a-b*(q1+q2))

**Profits are derived through:**

pi1 =(p1*q1-c*q1)

pi2 = (p2*q2-c*q2)

## Solving the model:

First we define all symbols:

```python
#Here we define the symbols as sympy objects. This lets us use sympy operations on them
#The qs are the quantities that firm 1 and firm 2 produce

q1 = sm.symbols('q_1')
q2 = sm.symbols('q_2')
#c is the production cost, this is assumed to be constant
c = sm.symbols('c')

# a is a parameter used to calculate the prices.
a = sm.symbols('a')
#b is a substiotion parameter, that describes the substitution elasticity between firm 1 and firm 2's product. If b = 1
# the products are perfect substitues, anything under 1 and above 1 is non-perfect.
b = sm.symbols("b")

#Pi are the profits for the two firms
pi1 = sm.symbols('pi_1')
pi2 = sm.symbols('pi_2')
```

We use the price and profit equations, to then derive the first order conditions wrt. the quantity produced by the respective firm:

```python
# The prices are defined as the parameter a minus the substitution parameter times the quantities of firm 1 and 2
p1 = (a-b*(q1+q2))

p2 = (a-b*(q1+q2))

# The profit of the firm is given by the prices times quantity, minus the cost times the quantity
pi1 =(p1*q1-c*q1)

pi2 = (p2*q2-c*q2)
```

```python
# In order to find the Cournot equilibrium, we need to begin by differentiating the two profit functions
# Here we use the sympy fuctionality to differentiante the profit function with regards to the qunatity
foc1=sm.diff(pi1,q1)
foc1
```

```python
# Same as above, for firm 2
foc2=sm.diff(pi2,q2)
foc2
```

Setting first order conditions to 0 and solving for the respective firms produced quantitiy (q1 and q2), give us the optimal qunatities the two firms should produce, given the other firms output (reaction functions).

```python
#Here we again use the sympy functionality to set the function = 0 by using sm.Eq, and thereafter solving it for the quantity
sol1= sm.solve(sm.Eq(foc1,0),q1)[0]
sol1
```

```python
#Same as above for firm 2
sol2= sm.solve(sm.Eq(foc2,0),q2)[0]
sol2
```

The Cournot equlibrium is achieved, when substituting one firm's reaction function into the other firm's reaction function. This equlibrium constitutes the optimal quantity that both firms should produce, given that the other firm is rational and is aiming to produce the profit maximizing amount.

We first substitute q2 in firm 1's fist order condition:

```python
#Here we use sympy to substitute the optimizal quantity for firm 2, into the reaction curve for firm 1
sol1_subs = foc1.subs(q2,sol2)
sol1_subs
```

We then solve the equation for q1, to get the Cournot equilibrium, which is dependent on a,b and c.

```python
#Lastly we use the sympy functionality to solve the equilibrium for the quantity of q1
sol1_subs_solve = sm.solve(sm.Eq(sol1_subs,0),q1)
sol1_subs_solve
```

## Numeric optimization

An alternative approach to the symbolic analysis of the Cournot equilibrium, is a numeric solution. The SciPy package has multiple good tools, to optimize a function numerically. This way of solving the Cournot model is redundant when it is easy to solve it symbolically, but the numeric optimization is good for more complex problems.

```python
#We begin by defining a demand function
```

```python
#Here the demand function is a liniar function, determined by the quantity that the two firms produce, and a substitution
# elasticity parameter (b).

def demand(q1,q2,b=1,a=100,):
    """This function returns the demand based on the substitution parameter and the parameter a.

    args:
        q1 & q2 = the quantities for form 1 and firm 2
        a = a is a paramter that determines the price. The default is 100
        b = is a parameter for the substitution elasticity between q1 and q2. Should be between 0 and 1.
            1 is perfect susbtitution
        """

    return a-q1-b*q2

#in order to maximixe profits, we still need a cost function. This is simply given by the cost times the quantity
def cost(q,c):
    """This function return the total costs of a firm

    args:
        q = the quantity for the firm
        c = the cost of each product

    """

    cost = c*q
    return cost
```

```python
#Based on this demand function we can define a profit function.
#This function returns the profit (demand * revenue), for firm 1, given firm 2 produces a given amount of product,
#and given equal prices
def profit(q1,q2,c,b):
    """This function takes the quantities, cost and substitution paramter and returns the profit, as the demand - the costs

    args:
        q1 & q2 = the quantities of the two firms
        c = the cost of the production of one unit, assumed constant in this function
        b = the substitution paramter

        """
    return demand(q1,q2,b)*q1-cost(q1,c)

```

```python
#Armed with this profit function, it is possible to optimize the profit of firm 1, given a production of firm 2
```

```python
def num_reaction(q2,c,b):
    """This function minimizes the profit for firm 1, given the production of firm 2, the cost and the substitution parameter.
        The function returns -min profit, as minimazation, is preffered to maximazation. The function uses an annonymous
        function, which is just minus the profit function, to minimize over.

    args:
        q2 = The quantity of firm 2 - should be between 0-100 as a is 100 as default
        c  = The cost associated with producing one unit
        b  = The substitution parameter
    """

    q1_opt = optimize.minimize(lambda x: -profit(x,q2,c,b), x0=1,)

    return q1_opt

```

```python
#We calculate the optimal quantity for firm 1, given that firm 2 produces 50, that the cost is 1 and that they are
# producing perfect substitutes
num_reaction(50,1,1)
```

          fun: -600.249999999997
     hess_inv: array([[0.50000008]])
          jac: array([7.62939453e-06])
      message: 'Optimization terminated successfully.'
         nfev: 15
          nit: 3
         njev: 5
       status: 0
      success: True
            x: array([24.50000178])

The optimazation shows that the optimal production for firm 1 given that firm 2 produces 50, is 24,5

This numeric analysis could be extented to solving the equilibrium. However, in this project we will analyse the
equilibrium using a symbolic approach, as it is easy to do in the case of the Counnot equilibrium

## Gettting to equilibrium

In this secton we will first define and show the reaction functions. Hereafter we will calculate Cournot equilibrum. Afterwards we will visualize the reaction curves, showing the Cournot quilibrium.

```python
#To begin we define a function for the reaction of one firms quantity to the other firms quantity
def reaction(q_other, a=100, b=1, cost=1,print_it=False):
    """This function calculates the quantity that a firm will produce, given the other firms production, and some parameters.
        It can also do some prettified printing

        args:
            q_other = the production of the other firm. should not exceed a
            a = a parameter in the price function.
            b = a parameter determing the substitution elasticity between firm 1 and firm 2s product
            print_it = Boolean argument. If True print the results as a sentence, and does not retun the quantity

        """
    #I define the quantity of firm 1 as a function of the other firm production
    quantity = (a-b*q_other - cost)/2*b

    #If print_it is True the function will return a readeble sentence instead of the quantity. Use this only for presentation
    if print_it:
        print("Given the other firm produces",q_other,"The firm will produce", quantity, sep=" ")
        return
    else:
        return quantity

```

Using the reaction function, we can now calculate the quatity that firm 1 produces, given firm 2's quantity

```python
#I define a widgets slider object, that i will use to make the reaction function interactive
qslider = widgets.IntSlider(min=0,max=100,step=1,value=50)

#I use the interact function on the reaction function, fixing a to 100, b and cost to 1, and print_it to True.
#This means that the only thing that can be interacted with is the qunatity of the other firm.
widgets.interact(reaction,q_other = qslider,a=widgets.fixed(100),b=widgets.fixed(1),cost=widgets.fixed(1),print_it=widgets.fixed(True))
```

    interactive(children=(IntSlider(value=50, description='q_other'), Output()), _dom_classes=('widget-interact',)…

    <function __main__.reaction(q_other, a=100, b=1, cost=1, print_it=False)>

The reader will remember that the numerical optimazation estimated the equilibrium to 24.5, given the other firm produces 50. This is the the correct value, as can be seen in the interactive slider above.

While this is interesting, theory tells us that firms will always end in the Cournout equilibrium, given fixed costs and prices, as this maximises profits.

```python
print("The equilibrium we derived earlier as:")
sol1_subs_solve

```

We now define a function that expreses the equilibrium, so that we can calculate the equilibrium given any values of the parameters

```python
def optimal(a, b, cost, text=True):

    """This function calculates the equilibrium that we have derived using the sympy functionality. It can also print it
        in a nice way

    args:
        a = a parameter in the price function.
        b = a paramter determening the substitution elasticity between firm 1 and firm 2. Should be between 0 and 1
        cost = the cost of the firms of produces a unit, here assumed equal
        text = Boolean, if True prints the equilibrium and the parameters in a readeble way,
                if false just return equilibrium value. True by default.

    """
    #defining the equilibrium as derived
    equilibrium = (a-cost)/3*b
    #Then i round the equilibrium value, as we assume that the product they procuce is discrete (no half cars allowed)
    equilibrium = round(equilibrium,0)

    #if text is false the function returns the equilibrium value
    if text==False:
        return equilibrium
    #if text is true (default), then the function does not return the equilibrium value, but instead prints a readeble line of text
    else:
        print("The Cournot equilibrium, given that","a =",a,", b =",b,"and that cost =",cost,",","is:",equilibrium)

```

```python
print("Given a = 100, b= 1 and cost = 1, is:",optimal(100,1,1,text=False))
```

    Given a = 100, b= 1 and cost = 1, is: 33.0

This function can calculate the value of the equilibrium. We use this interactivly below.

```python
#I define 3 sliders with default values. This allows us to calculate the equilibrium interactively
a_slider = widgets.IntSlider(min=10,max=150,step=1,value=100)
b_slider = widgets.FloatSlider(min=0,max=1,step=0.1,value=1)
cost_slider = widgets.IntSlider(min=0,max=50,step=1,value=1)

#I use the inteact function on the optimal function, and use the slideres defined as inputs.
widgets.interact(optimal,a=a_slider,b=b_slider,cost=cost_slider)
```

    interactive(children=(IntSlider(value=100, description='a', max=150, min=10), FloatSlider(value=1.0, descripti…

    <function __main__.optimal(a, b, cost, text=True)>

## Visualization

The relation between two firms output quantity decision can also be shown visualy. To do this, I begin by defining a function that creates a plot.

```python
def reaction_plot(q_other, a=100, b=1, cost=1,scatter=True, price = False):

    """This function calculates the two firms reaction functions, and plots these as lines.

        Args:
            q_other = The quantity of firm 2
            a = a parameter from the price function, default is 100
            b = a parameter determening the substituttion elsticity between firm 1 and firm 2s goods
            cost = the cost of producing a unit for the firms
            scatter = boolean. If true returns a scatter plot, else retusn a line plot
            price = boolean. If true return the price in the equilibrium

    """

    #Begin by calculating the quantity firm 1 will produce given firm 2s production
    quantity = reaction(q_other,a, b, cost)

    #I create two plots, either scatter or line plots.
    if scatter == True :
        plt.scatter(quantity,q_other,label = "Firm 1's reaction curve")
        plt.scatter(q_other,quantity,label = "Firm 2's reaction curve")
    else :
        plt.plot(quantity,q_other,label = "Firm 1's reaction curve")
        plt.plot(q_other,quantity,label = "Firm 2's reaction curve")

    #I create labels, legens and limits
    plt.xlabel("Firm 1 quantity")
    plt.ylabel("Firm 2 quantity")
    plt.legend()
    plt.xlim(0,100)
    plt.ylim(0,100)

    #Estimate the price if price == True
    if price :
        price = a-b*(q_other+quantity)
        return price
    plt.show()
    return
```

```python
#I choose 1000 values between 0 and 100 as x
x = np.linspace(0, 100, 1000);

#Then I plot these as a line plot
reaction_plot(q_other=x,scatter=False)
```

This plot shows us visually that the conclusion that the equilibrium can be found where both firms produce 33 holds up. This is given a set of parameters though, and it might be nice to see how the equilibrium behaves, when we change these. In peticualar, changes to b can show what happens when the firms goods are not perfect substitutes. Therefore we make the plot interactive below.

```python
#We define some sliders for the parameters
ai_slider = widgets.IntSlider(min=10,max=150,step=1,value=100)
bi_slider = widgets.FloatSlider(min=0,max=1,step=0.1,value=1)
costi_slider = widgets.IntSlider(min=1,max=50,step=1,value=1)

#Here we call the intercat function on the reaction_plot function, to get an interactive plot. We fix the input of the other
# firm is to be x again, and the plot is to be a line.
widgets.interact(reaction_plot,q_other=widgets.fixed(x),a=ai_slider,b=bi_slider,cost=costi_slider, scatter = widgets.fixed(False), price = widgets.fixed(False))

```

    interactive(children=(IntSlider(value=100, description='a', max=150, min=10), FloatSlider(value=1.0, descripti…

    <function __main__.reaction_plot(q_other, a=100, b=1, cost=1, scatter=True, price=False)>

# Prices

The Cournot equilibrium is mostly concerned with figuring out the qunatity of product that the firms should produce. But it is also possible to calculate the price of the product in equilibrium, given a set of parameters.

```python
#Here we put in the price and the symbollic estimation of it. We use sympy to substitute q1 for the optimal q1 and q2
# into the price functions
price_1 = p1.subs([(q1,sol1_subs_solve[0]), (q2, sol1_subs_solve[0])])
price_1

```

```python
#Then we substitute in the prices in the profit function
pi = pi1.subs([(p1,price_1), (q1,sol1_subs_solve[0])])
pi
```

The price in equilibrium only depends on the 3 paramters a, b and c. We therefore write a function that takes these arguments and returns the equilibrium price.

```python
def price_eq(q_other, a = 100, b = 1, cost = 1):
    """This function calculates the price, and the quantity in the cournot equilibrum, given firm 2s quantity and
        some parameter. It prints the results nicely. There is also a check of whether or not prices are above 0, which they
        should be.

    args:
        q_other = the quantity of the other firm
        a = a parameter from the pricefunction, default is 100
        b = a parameter determening the substituttion elsticity between firm 1 and firm 2s goods
        cost = the two firms cost of producing a unit

    """

    #We begin by calculating firm 1s quantity given firm 2s produciton
    q = reaction(q_other, a, b, cost)

    #We then calculate the price
    price = round(a/3 + (2*cost)/3,0)

    #Then we calculate the counot equilibrium given a, b and cost
    opt_q = optimal(a, b, cost, text = False)

    #If prices are above 0 we print the price and q in equilibrium, and the quantity that the firms will produce, given
    # the current production of firm 2 (q_other)
    if price > 0:
        print("The price in the Cournot equilibrium is",price)
        print("The quantity that the two firms will produce in the Cournot equilibrium is",opt_q)
        print("When firm 2 produces",q_other,"then firm 1 will produce",q)
    else:
        print("Price can't be negative")

```

```python
#I define some slideres for the interactive calculator
qp_slider = widgets.IntSlider(min=10,max=101,step=1,value=50)
ap_slider = widgets.IntSlider(min=10,max=100,step=1,value=100)
bp_slider = widgets.FloatSlider(min=0,max=1,step=0.1,value=1)
costp_slider = widgets.IntSlider(min=1,max=50,step=1,value=1)

#I use the interact function on the price_eq function, using the sliders previously defined
widgets.interact(price_eq,q_other=qp_slider, a = ap_slider, b= bp_slider, cost = costp_slider)
```

    interactive(children=(IntSlider(value=50, description='q_other', max=101, min=10), IntSlider(value=100, descri…

    <function __main__.price_eq(q_other, a=100, b=1, cost=1)>

The calcultor can be used to se how prices and the equilibrium will react to changes in the parameters.

Another way to find we equilibrium regarding the parameters :

```python
#definition of the cost which is a function of the quantity
#here n is the quantity and C the cost. The letters are not the same as before to avoid conflict
#here we do not need the if condition, it would have been usefull if we wanted to keep a cost function with a fixe cost
#constant such as c*n + f to avoid a cost even if the firm do not produce anything.

def cost(n,C) :
    """ This function calculates the cost for the firm given the quantity.

    args:
     n (float) : the quantity produce
     C (float) : the cost for the production of a unit

    return: The production cost for the firm.
    """
    if n == 0:
        cost = 0
    else:
        cost = C*n
    return cost
```

```python
#definition of the demand function
#we take our price function (a-b*(q1+q2)) and set a=1 in this exercice. b is a coefficient which represent the impact of the quantity on the price
#Indeed less a production is rare less the price will be. We can also add a coefficient "k" which can represent the heterogeneity
#between q1 and q2 but we will juste assume the perfect substitution in our problem set
def demand(n1,n2,B):
    return 1 - B*(n1+n2)
```

```python
#we define the profit as the demand function minus the cost function given the quantity n1, n2, the cost C and B.
def profit1(n1,n2,C,B):
    return demand(n1,n2,B)*n1 - cost(n1,C)
```

```python
#As priviously, in a cournot model the firms set their quantities (production) given the quantity of the other firm.
#Which is why we need the reaction function and to get it we have to maximize (here minimize -f(x)) the profit function.
def reacfunction(n1,n2,C,B):
    x1 = optimize.brute(lambda n1: -profit1(n1,n2,C,B), ((0,1,),)) # brute minimizes the function;
    return x1[0]
```

```python
#we define a vector which take the vector first guess and the vector parameters (that we will define after) as input for our model
def vector_reaction(x,param):
    return np.array(x)-np.array([reacfunction(x[0],x[1],param[1],param[0]),reacfunction(x[1],x[0],param[2],param[0])])
```

Now let's create a vector with the value of the three parameters, B, C1 and C2 ; and a vector with the initial guesses for the quantity of firm1 and quantity of firm2.
We input these two vector in the function "vector_reaction" which we optimize to solve the model:

```python
#it has to be [B, C1, C2] and [guess1,guess2]
param = [0.01,0.0,0.0]
x0 = [0.3, 0.3]

#solve the model
answer = optimize.fsolve(vector_reaction, x0, args = (param))
answer
```

    array([33.33330078, 33.33330078])

The result is pretty closed of the equilibrium which is 33/33. Now we will define a clear function which will return the estimate quantities given the parameters. It is just applying what we've just before in a function.

```python
#Define the function which return the estimate equilibrum given the parameters
def equilibriumf(param, guess):
    """
    This function take the three parameters and the two guess to solve the cournot model and estimate the equilibrium
            quantity level.

        args : param (list, three floats)   B : cost for one units of production
                                            C1 : cost for the firm one for the production of one unit
                                            C2 : cost for the firm two for the production of one unit
                guess (list, two floats)    guess 1: The first guess of equilibrium quantity for the firm one
                                            guess 2: The first guess of equilibrium quantity for the firm two

        return : It returns the estimate quantity equilibrium given the parameters for the cournot model
    """
    answer = optimize.fsolve(vector_reaction, x0=guess, args = (param))
    if param[1] == param[2]:
            return "The equilibrium is reach, the firms produce both aound " + str(answer[0]) + " units, given a cost of " + str(param[1]) + "."
    else :
            return "Firm 1 produces around " + str(answer[0]) + " units given a cost of " + str(param[1]) + " while firm 2 produces " + str(answer[1]) + " units whith a cost of " + str(param[2]) + "."
```

```python
equilibriumf([0.01, 0, 0], [0.3, 0.3])
```

    'The equilibrium is reach, the firms produce both aound 33.33330078125009 units, given a cost of 0.'

Now we have a function which estimate the quantities for both firms given the parameters. In the previous result the equilibrium is reach because of a common cost. If cost are not equal the firm with the cheapest price produce more :

```python
equilibriumf([0.01, 0, 0.1], [0.3,0.3])
```

    'Firm 1 produce around 36.66660156250006 units given a cost of 0 while firm 2 produces 26.666699218750047 units whith a cost of 0.1.'


 Tomography Reconstruction using CIL

### In this demo, we reconstruct simulated tomographic data using:

* the Filtered Back Projection (FBP) algorithm,
* Total variation (TV) regularisation under a non-negativity constraint

    $$\begin{equation}
    \underset{u}{\operatorname{argmin}} \frac{1}{2} \| A u - g\|^{2}_{2} + \alpha\,\mathrm{TV}(u) + \mathbb{I}_{\{u\geq0\}}(u)
    \tag{1}
    \end{equation}$$

    where,

    * g is a noisy data corrupted with Gaussian noise and
    * $A$ is the Projection operator.

```python
# Import libraries

from cil.framework import  AcquisitionGeometry

from cil.optimisation.functions import L2NormSquared, BlockFunction, MixedL21Norm, IndicatorBox
from cil.optimisation.operators import GradientOperator, BlockOperator
from cil.optimisation.algorithms import PDHG
from cil.plugins.astra.operators import ProjectionOperator
from cil.plugins.astra.processors import FBP
from cil.plugins import TomoPhantom
from cil.utilities.display import show2D, show_geometry
from cil.utilities import noise

import matplotlib.pyplot as plt

import numpy as np
```

We first configure our acquisition geometry, e.g., **2D parallel geometry**. Then, the image geometry is extracted and used to configure our phantom. To create our simulated phantoms, we use the [Tomophantom](https://github.com/dkazanc/TomoPhantom) library.

```python
# Detectors
N = 256
detectors =  N

# Angles
angles = np.linspace(0,180,180, dtype='float32')

# Setup acquisition geometry
ag = AcquisitionGeometry.create_Parallel2D()\
                        .set_angles(angles)\
                        .set_panel(detectors, pixel_size=0.1)
# Get image geometry
ig = ag.get_ImageGeometry()

# Get phantom
phantom = TomoPhantom.get_ImageData(12, ig)
```

Next, we create our simulated tomographic data by projecting our noiseless `phantom` to the acquisition space. Using the image geometry `ig` and acquisition geometry `ag`, we define the `ProjectionOperator` with `device=cpu` or `device=gpu`. Finally,  Gaussian noise is added to the noiseless sinogram.

```python
# Create projection operator using Astra-Toolbox. Available CPU/CPU
A = ProjectionOperator(ig, ag, device = 'cpu')

# Create an acqusition data (numerically)
sino = A.direct(phantom)

# Simulate Gaussian noise for the sinogram
gaussian_var = 0.5
gaussian_mean = 0

n1 = np.random.normal(gaussian_mean, gaussian_var, size = ag.shape)

noisy_sino = ag.allocate()
noisy_sino.fill(n1 + sino.array)
noisy_sino.array[noisy_sino.array<0]=0

# Show numerical and noisy sinograms
show2D([phantom, sino, noisy_sino], title = ['Ground Truth','Sinogram','Noisy Sinogram'], num_cols=3, cmap = 'inferno')

```

For our first reconstruction, we use the Filtered BackProjection algorithm, i.e., `FBP` applied to our noisy sinogram `noisy_sino`.

```python
# Setup and run the FBP algorithm
fbp_recon = FBP(ig, ag,  device = 'cpu')(noisy_sino)

# Show reconstructions
show2D([phantom, fbp_recon],
       title = ['Ground Truth','FBP reconstruction'],
       cmap = 'inferno', fix_range=(0,1.), size=(10,10))
```

In the above reconstruction noise is not penalised. In order to remove noise artifacts, we will use the TotalVariation regularisation as shown in the minimisation problem above.

We solve [(1)](#tv_reg), using the **Primal-Dual Hybrid Gradient (PDHG)** algorithm introduced in [ChambollePock](https://link.springer.com/article/10.1007/s10851-010-0251-1). We need to write [(1)](#tgv_reg) in the following general form $$\underset{x\in \mathbb{X} }{\operatorname{argmin}} f(Kx) + g(x).$$

We let $x=u$ and

* define an operator $K:\mathbb{X}\rightarrow\mathbb{Y}$ as$\\[10pt]$

    $$\begin{equation}
    K =
    \begin{bmatrix}
    A \\
    D
    \end{bmatrix} \quad\Rightarrow\quad
    Kx = Ku =
    \begin{bmatrix}
    \mathcal{A}u\\
    Du\\
    \end{bmatrix} =
    \begin{bmatrix}
    y_{1}\\
    y_{2}
    \end{bmatrix} = y\in \mathbb{Y},
    \label{def_K}
    \end{equation}$$

* define a function $f:\mathbb{Y}\rightarrow\mathbb{R}$ as$\\[10pt]$

    $$\begin{equation}
    \begin{aligned}
    & f(y)  := f(y_{1}, y_{2}) = f_{1}(y_{1}) +  f_{2}(y_{2}) , \mbox{ where},\\[10pt]
    & f_{1}(y_{1}) :=  \frac{1}{2}\| y_{1} - g\|_{2}^{2},\, f_{2}(y_{2}) :=  \alpha \|y_{2}\|_{2,1}
    \end{aligned}
    \label{def_f}
    \end{equation}\\[10pt]$$

* and define a function $g(x) = g(u) = \mathbb{I}_{\{u\geq0\}}(u)$ for the non-negativity constraint.

```python
# Define BlockOperator K
Grad = GradientOperator(ig)
K = BlockOperator(A, Grad)

# Define BlockFunction f
alpha = 0.1
f1 = 0.5 * L2NormSquared(b=noisy_sino)
f2 = alpha * MixedL21Norm()
f = BlockFunction(f1, f2)

# Define Function g
g = IndicatorBox(lower=0)

# Primal/Dual stepsizes
normK = K.norm()
sigma = 1./normK
tau = 1./normK

# Setup and run PDHG
pdhg = PDHG(f = f, g = g, operator = K, sigma = sigma, tau = tau,
            max_iteration = 200,
            update_objective_interval = 50)
pdhg.run(verbose=2)
```

Finally, we compare the PDHG and FBP reconstructions and plot the middle line profiles.

```python
show2D([pdhg.solution,fbp_recon, phantom], title = ['TV regularisation','FBP','Ground Truth'], cmap = 'inferno', num_cols=3, fix_range=(0,1.))

# Plot middle line profile
plt.figure(figsize=(30,15))
plt.plot(phantom.subset(horizontal_y = int(N/2)).as_array(), label = 'Ground Truth', linewidth=5)
plt.plot(fbp_recon.subset(horizontal_y = int(N/2)).as_array(), label = 'FBP', linewidth=5, linestyle='dashed')
plt.plot(pdhg.solution.subset(horizontal_y = int(N/2)).as_array(), label = 'TV', linewidth=5)
plt.legend()
plt.title('Middle Line Profiles')
plt.show()
```

```python

```


# Objective: To filter a textured image without loosing its texture

```python
# Import the required libs
from torch_pdegraph.pdes import pdeanisodiff
from torch_pdegraph.utilities import *
import numpy as np
from matplotlib import image as mpimg
import faiss
import matplotlib.pyplot as plt
import torch
```

```python
# load the texture based image, add some noise
# and extract patches
x = mpimg.imread("../data/barbara.png")
x = addNoise(5,x)

# feature that shall be used to create the knn-graph
patches = imgPatches(x,(5,5))
dispImg(x)
```

# Create the graph
- In order to simply show the effectiveness of PDEs on graph,  I am only creating a simple K-NN based graphs. This may or maynot be the best graph for a given problem at hand.

- One can create graph using whatsoever apt approach or one can even use third-party network datasets and run a PDE on that graph.  PDEs are extensible to any given graph/network at hand as long as that graph has edges and weights( edge_index and edge_attr).

Although torch_cluster comes with a knn-graph method. I found it to be limited and  slow when the node-features have high dimensions. We shall be using facebook's faiss library which is blazingly fast for a KNN-graph construction.

```python
# Create a Knn graph of the pacthes of the texture based image
# and assign weights to the edges
res = faiss.StandardGpuResources()
index = faiss.IndexFlatL2(patches.shape[1])
gpu_index_flat = faiss.index_cpu_to_gpu(res,0,index)
gpu_index_flat.add(patches.astype(np.float32))
k = 30
D, I = gpu_index_flat.search(patches.astype(np.float32),k+1)

#Graph
edge_index = np.vstack((I[:,1:].flatten(), np.repeat(I[:,0].flatten(),k)))
#edge_attr = np.exp(-(D[:,1:].flatten()/np.inf))
edge_attr = np.ones(edge_index.shape[1]) # Let the weights be equal to 1 !

edge_index = torch.tensor(edge_index, dtype=torch.long).to('cuda:0')
edge_attr = torch.tensor(edge_attr, dtype=torch.float32).to('cuda:0')

edge_attr = edge_attr.view(-1,1)
graph = Graph(edge_index, edge_attr)
```

# Run a predefined PDE on graph :
We shall be using the following adaptive $dt$  (time step) iterative scheme for anisotropic diffusion

\begin{equation}
\mathbf{x}^{n+1}_{i} = \frac{\lambda \mathbf{x}^{0}_{i} + \sum_{j \in N(i)} w_{i,j}^{p/2}|\mathbf{x}^{n}_{j}-\mathbf{x}^{n}_{i}|^{p-2}\mathbf{x}^{n}_{j}}{\lambda+ \sum_{j \in N(i)} w_{i,j}^{p/2}|\mathbf{x}^{n}_{j}-\mathbf{x}^{n}_{i}|^{p-2}}
\end{equation}

- $\mathbf{x}_{i}$ is the node feature/signal at the $i^{th}$ node
- $w_{i,j}$ is the scalar edge_attr on the edge $e(i\to j)$

**Example:**

```python
from torch_pdegraph.pdes import pdeanisodiff
# create a pde instance
pde = pdeanisodiff.PDE(graph, **hp)

# run the pde on the features on nodes for some iterations
new_features = pde(features, itr)

"""
graph is a structure which has graph.edge_index and graph.edge_attr(weights)
hp is hyperparams in the equation (\lambda and p)
"""
```

The above equation is basically a gauss-jacobi based iterative scheme on graphs for this Energy:

\begin{equation}
  \mathcal{E}_{w,p}(\mathbf{x, x^{0}, \lambda}) = \frac{\lambda}{2}\|\mathbf{x} - \mathbf{x}^{0}\|_{2}^{2} + \frac{1}{p}\sum_{i \in V}\left( \sum_{j \in N(i)} w_{i,j}^{p/2}\bigl |\mathbf{x}_{j}{-}\mathbf{x}_{i}\bigr|^p \right)
\end{equation}

To know more about it ref [Elmoataz](https://hal.archives-ouvertes.fr/hal-00163573/document)

```python
#hyperparams
hp = dict(x0=torch.tensor(toFmat(x),dtype=torch.float32).cuda(), lamb=0, epsilon=5e-07, p_val=1)

# Run the implicit PDE on graph
itr = 30
features = torch.tensor(toFmat(x),dtype=torch.float32).cuda()

pde = pdeanisodiff.PDE(graph, **hp)
new_features = pde(features, itr)
```

    100%|██████████| 30/30 [00:00<00:00, 468.53it/s]

```python
# Now reshape the new_features to the sqaure shape
y = new_features.cpu().numpy()
y = toImg(y,x.shape)

#compare the noisy and filterd image
f = plt.figure(figsize=(8,8))
f.add_subplot(1,2, 1)
plt.imshow(y, cmap="gray")
f.add_subplot(1,2, 2)
plt.imshow(x, cmap="gray")
plt.show(block=True)
```

# Filter an image over a surface with same pde

```python
# Filter an image over a surface
p = np.load("../data/position_lena.npz")
t = np.load("../data/texture_lena.npz")
p = p["position"].astype(np.float32)
signal = t["texture"]

#param = dict(position=p, texture=signal)
#displaySur(**param)

#add noise
signal = addNoise(10, signal)

# Create a knn graph
res = faiss.StandardGpuResources()
index = faiss.IndexFlatL2(p.shape[1])
gpu_index_flat = faiss.index_cpu_to_gpu(res,0,index)
gpu_index_flat.add(p)
k = 10
_, I = gpu_index_flat.search(p,k+1)
edge_index = np.vstack((I[:,1:].flatten(), np.repeat(I[:,0].flatten(),k)))
edge_attr = np.exp(-np.sum(((signal[I]-signal[I][:,0,None])**2),axis=2)/(0.5)**2)[:,1:].flatten()
edge_index = torch.tensor(edge_index, dtype=torch.long).to('cuda:0')
edge_attr = torch.tensor(edge_attr, dtype=torch.float32).to('cuda:0')
edge_attr = edge_attr.view(-1,1)
graph = Graph(edge_index, edge_attr)

#hyperparams
hp = dict(x0=torch.tensor(signal,dtype=torch.float32).cuda(), lamb=0, epsilon=5e-07, p_val=1)

# Run the implicit PDE on graph
itr = 100
features = torch.tensor(signal,dtype=torch.float32).cuda()

pde = pdeanisodiff.PDE(graph, **hp)
new_sig = pde(features, itr)
```

    100%|██████████| 100/100 [00:00<00:00, 1439.50it/s]

```python

displayJSur(**dict(position=p, texture=new_sig.cpu().numpy()))
```

    JVisualizer with 1 geometries

```python
displayJSur(**dict(position=p, texture=signal))
```

    JVisualizer with 1 geometries

```python
# Unlike an image, a pcd display cannot be saved in ipython notebook, hence the notebook must be running to
# to view the pcd. Expect to see the surface filtered like shown in the following image.
from matplotlib import image as mpimg
from matplotlib import pyplot as plt
res = mpimg.imread("../gallery/lena_sur.png")
f = plt.figure(figsize=(12,12))
plt.imshow(res)
```


##Practica 2 Ejercicio 1
Mostrar que cada uno de los siguientes sistemas tiene una orbita periódica:

  * a)

 $$\left\{ \begin{array}{lcc}
            \dot{x}_{1}=x_{2} \\
             \\ \dot{x}_{2}=-x_{1}+x_{2}(1-3x_{1}^{2}-2x_{2}^{2})
             \end{array}
\right.$$

  * b)

$$
\ddot{y}+y=\epsilon \dot{y}(1-y^{2}-(\dot{y})^{2}) \:; \epsilon > 0
$$

```python
import sympy as sym
```

```python
#Con esto las salidas van a ser en LaTeX
sym.init_printing(use_latex=True)
```

```python
x_1, x_2 = sym.symbols('x_1 x_2')
```

```python
X = sym.Matrix([x_1, x_2])
X
```

```python
f_1 = x_2
```

```python
f_2 = -x_1 + x_2 * (1 - 3 * x_1 ** 2 - 2 * x_2 ** 2)
f_2
```

```python
F = sym.Matrix([f_1,f_2])
F
```

```python
A = F.jacobian(X)
#A.simplify()
A
```

```python
# puntos de equilibrio del sistema
pes = sym.solve([f_1,f_2],[x_1,x_2])
pes
```

```python
A_1 = A.subs({x_1:0,x_2:0})
A_1
```

```python
A_1.eigenvals()
```

```python
eq =2 * x_2 ** 2 - 6 * x_1 * x_2 ** 2 - 4 * x_2 ** 4
```

```python
eq.factor()
```

```python
sym.plot_implicit(eq)
```

    <sympy.plotting.plot.Plot at 0x7fca80f266d0>

```python

```


# Separation of Variables in spherical coordinates

with the Laplace equation in spherical coordiantes:

$$ \frac{1}{r^2}\frac{\partial}{\partial r}\left (r^2 \frac{\partial V}{\partial r}\right )+
   \frac{1}{r^2\sin \theta}\frac{\partial}{\partial \theta} \left (\sin \theta \frac{\partial V}{\partial \theta}\right )+
   \frac{1}{r^2\sin^2 \theta}\frac{\partial^2 V}{\partial \phi^2}
   =0
$$

we are assuming azimuthal symmetry here and ths V is independent of $\phi$, even though the general care is solveable as well and it simplifies to:

$$ \frac{\partial}{\partial r}\left (r^2 \frac{\partial V}{\partial r}\right )+
   \frac{1}{\sin \theta}\frac{\partial}{\partial \theta} \left (\sin \theta \frac{\partial V}{\partial \theta}\right )
   =0
$$

Again we are looking for solutions as products:

$$ V(r,\theta) = R(r)\Theta(\theta)$$

Separating of variables gives us:

$$ \frac 1 R \frac{\partial}{\partial r}\left (r^2 \frac{\partial R}{\partial r}\right )=
   -\frac{1}{\Theta\sin \theta}\frac{\partial}{\partial \theta} \left (\sin \theta \frac{\partial \Theta}{\partial \theta}\right ) $$

which can only be equal if the left side and the right side are constants (since they each only depend on $r$ or $\theta$ respectively)

So we are setting constants for the left side:

$$ \frac 1 R \frac{\partial}{\partial r}\left (r^2 \frac{\partial R}{\partial r}\right )=
   l(l+1)$$

and for the right side:
$$ \frac{1}{\Theta\sin \theta}\frac{\partial}{\partial \theta} \left (\sin \theta \frac{\partial \Theta}{\partial \theta}\right ) = - l(l+1) $$

and thus have converted a PDE in two ODEs.

### Solving the left side:

```python
from sympy.interactive import printing # use latex for printing
printing.init_printing(use_latex=True)
from sympy import Function, dsolve, Eq, Derivative, sin, cos, symbols # import necessary methods
from sympy.abc import theta # import variables
```

```python
r, l = symbols('r l',positive=True)
R = Function('R',real=True)
s1_ode = Eq(Derivative((r**2*Derivative(R(r),r,1)), r, 1)-l*(l+1)*R(r))
s1_ode
```

```python
sol1 = dsolve(s1_ode, R(r))
sol1
```

### Solving the right side

```python
t, l = symbols('theta l')
T = Function('Theta',real=True)
s2_ode = Eq(Derivative(sin(t)*Derivative(T(t),t,1), t, 1)+l*(l+1)*sin(t)*T(t))
s2_ode
```

```python
sol2 = dsolve(s2_ode, T(t))
sol2
```

### Legendre Polynomials

```python
from sympy.plotting import plot
from sympy import legendre, pi
%matplotlib inline
```

```python
t = symbols('theta',positive=True)
P0 = lambda t: legendre(0,t)
P1 = lambda t: legendre(1,t)
P2 = lambda t: legendre(2,t)
P3 = lambda t: legendre(3,t)
P4 = lambda t: legendre(4,t)
```

```python
P0(cos(t))
```

```python
P1(cos(t))
```

```python
P2(cos(t))
```

```python
P3(cos(t))
```

```python
P4(cos(t))
```

```python
plot(P0(cos(t)),P1(cos(t)),P2(cos(t)),P3(cos(t)),P4(cos(t)),(t,0,pi))
```

Legendre polynomials with the argument $\cos\theta$ are the solutions for the right side:

$$ \Theta(\theta) = P_l(\cos\theta) $$

Which gives the most general separabe solution to Laplace equation (for physics):

$$ V(r,\theta)=\sum_{l=0}^{\infty}\left (A_l r^l+\frac{B_l}{r^{l+1}}\right ) P_l(\cos\theta) $$

```python

```

Separation of Variable, spherical coordinates.ipynb


## Network Flow Example

We have a network of $m$ directed edges that support $n$ flows, each of which goes over a fixed path in the graph.  We denote the flow values as $f \in \mathbb{R}_+^n$.  The resulting traffic on the $m$ edges is given by $R f$, where $R \in \{0,1\}^{m \times n}$ is the routing matrix, with $R_{ij}=1$ if flow $j$ goes over edge $i$, and 0 otherwise.

The edges have capacity $c \in \mathbb{R}_+^m$, so we have $Rf \leq c$. The objective is to maximize the total utility, which is $U(f) = \sum_i U_i(f_i)$, with

\begin{equation}
U_i(f_i) = \begin{cases}
-\infty \quad &\text{if} \quad f_i < f^\mathrm{min}_i \\
w_i f_i \quad &\text{if} \quad f^\mathrm{min}_i \leq f_i \leq f_i^\mathrm{max} \\
w_i f_i^\mathrm{max} \quad &\text{otherwise},
\end{cases}
\end{equation}

where the first case is the implicit version of the constraint $f^\mathrm{min} \leq f$. Except for the case of degenerate networks, the third condition will never hold at optimality. Intuitively, when $f_i > f_i^\mathrm{max}$, reducing $f_i$ at not cost, to free capacity for other flows that run through the edges of flow $i$, improves the objective function. We encode this information in the constraint $f \leq f^\mathrm{max}$ and rewrite $U$ in vector form to arrive at the optimization problem

\begin{equation}
\begin{array}{ll}
\text{maximize} \quad &w^T f \\
\text{subject to} \quad &R f \leq c \\
&f^\mathrm{min} \leq f \leq f^\mathrm{max},
\end{array}
\end{equation}

with variable $f \in \mathbb{R}_+^n$ and parameters $w\geq 0$, $c \geq 0$, $R$, and $0 \leq f^\mathrm{min} \leq f^\mathrm{max}$.

Let's define the corresponding CVXPY problem.

```python
import cvxpy as cp
import numpy as np

# define dimensions
n, m = 4, 5

# define variable
f = cp.Variable(n, name='f')

# define parameters
R = cp.Parameter((m, n), name='R')
c = cp.Parameter(m, nonneg=True, name='c')
w = cp.Parameter(n, nonneg=True, name='w')
f_min = cp.Parameter(n, nonneg=True, name='f_min')
f_max = cp.Parameter(n, nonneg=True, name='f_max')

# define objective
objective = cp.Maximize(w@f)

# define constraints
constraints = [R@f <= c, f_min <= f, f <= f_max]

# define problem
problem = cp.Problem(objective, constraints)
```

Assign parameter values and solve the problem.

```python
import numpy as np

np.random.seed(0)
R.value = np.round(np.random.rand(m, n))
c.value = n*(0.1+0.1*np.random.rand(m))
w.value = 0.1+np.random.rand(n)
f_min.value = np.zeros(n)
f_max.value = np.ones(n)

val = problem.solve()
```

Generating C source for the problem is as easy as:

```python
from cvxpygen import cpg

cpg.generate_code(problem, code_dir='network_code')
```

Now, you can use a python wrapper around the generated code as a custom CVXPY solve method.

```python
from network_code.cpg_solver import cpg_solve
import numpy as np
import pickle
import time

# load the serialized problem formulation
with open('network_code/problem.pickle', 'rb') as f:
    prob = pickle.load(f)

# assign parameter values
np.random.seed(0)
prob.param_dict['R'].value = np.round(np.random.rand(m, n))
prob.param_dict['c'].value = n*(0.1+0.1*np.random.rand(m))
prob.param_dict['w'].value = np.random.rand(n)
prob.param_dict['f_min'].value = np.zeros(n)
prob.param_dict['f_max'].value = np.ones(n)

# solve problem conventionally
t0 = time.time()
# CVXPY chooses eps_abs=eps_rel=1e-5, max_iter=10000, polish=True by default,
# however, we choose the OSQP default values here, as they are used for code generation as well
val = prob.solve()
t1 = time.time()
print('\nCVXPY\nSolve time: %.3f ms' % (1000 * (t1 - t0)))
print('Objective function value: %.6f\n' % val)

# solve problem with C code via python wrapper
prob.register_solve('CPG', cpg_solve)
t0 = time.time()
val = prob.solve(method='CPG')
t1 = time.time()
print('\nCVXPYgen\nSolve time: %.3f ms' % (1000 * (t1 - t0)))
print('Objective function value: %.6f\n' % val)
```

```python
from visualization.network import create_animation
from IPython.display import Image

create_animation(prob, 'network_animation')

with open('network_animation.gif', 'rb') as f:
    display(Image(f.read()))
```


# Derivación numérica

Aunque la derivada de una función se puede obtener algorítmicamente de manera analítica, los algoritmos numéricos que usemos pueden depender de muchas derivadas o no pueden acceder a otra cosa más que la función original.

Así, comúnmente utilizaremos una aproximación numérica de la derivada en lugar de el valor real.

## Preludio matemático: series de Taylor

Sea $f:\mathbb{R} \to \mathbb{R}$ diferenciable $k$ veces en un punto $a$. Entonces :

$$
f(x) = f(a) + f'(a)(x-a)+ \frac{f''(a)}{2!} (x-a)^2 + \ldots + \frac{f^{(k)}(a)}{k!}(x-a)^k + R_{k}(x)
$$

Y $R_{k}(x)$ cumple que

\begin{equation}
\lim_{x\to a} \frac{R_{k}(x)}{(x-a)^k} = 0 \quad \left( R_k(x) \sim (x-a)^{k+1}\right)
\label{eq1}
\end{equation}

En forma aproximada, se cumple que:
$$
f(x) \approx f(a) + f'(a)(x-a)+ \frac{f''(a)}{2!} (x-a)^2 + \ldots + \frac{f^{(k)}(a)}{k!}(x-a)^k
$$

### Forma explícita de $R_k(x)$

Utilizando el teorema del punto medio, podemos expresar a $R_k(x)$ como

$$
R_k(x) = \frac{f^{(k+1)}(\xi)}{(k+1)!} (x-a)^{k+1}
$$

Con $\xi$ un número real **fijo** entre $x$ y $a$. Con esta expresión, es claro que se la condición de $\lim_{x\to a} \frac{R_{k}(x)}{(x-a)^k} = 0 $ se cumple.

### Reescribir el teorema:

Sustitución: $x-a = h \implies x = a+h$

$$
f(a+h) = f(a) + f'(a)(h)+ \frac{f''(a)}{2!} (h)^2 + \ldots + \frac{f^{(k)}(a)}{k!}(h)^k + R_{k}(a+h)
$$

Y $R_{k}(a+h)$ cumple que

$$
\lim_{h\to 0} \frac{R_{k}(a+h)}{h^k} = 0 \quad \left( R_k(a+h) \sim h^{k+1}\right)
$$

## ¿Cómo aproximar la derivada?

$$
f'(a) = \lim_{h\to 0 } \frac{f(a+h) - f(a)}{h}
$$

**No podemos hacer limites en la computadora**. ¿Qué hacemos? Fijamos un valor de $h$ muy pequeño y aproximamos el valor. Una aproximación de  **diferencias finitas**

## Preludio computacional: funciones como objetos

Aunque en clases pasadas hemos entendido a las funciones como un objeto abstracto, podemos tratarlas como un objeto con un tipo definido arbitrario. Por ahora no podemos explicar profundamente cuál es el tipo de una función, así que solo procederemos a utilizarlas como un objeto: asignar una función a una variable, dársela a otra función como argumento, etc.

```julia
# ejemplo de asignación
function f(x)
    return sin(3*x^2)
end
```

    f (generic function with 1 method)

```julia
println(f(45))
# asigno a la variable `g` la función representada por `f`
g = f
# `g` me imprime el mismo valor que `f` al evaluarla
println(g(45))
```

    -0.7447712875753175
    -0.7447712875753175

```julia
# ejemplo de utilizar una función como argumento de otra función
function evaluarEn3(func)
    return func(3)
end
```

    evaluarEn3 (generic function with 1 method)

```julia
println(evaluarEn3(sin))
println(sin(3))
```

    0.1411200080598672
    0.1411200080598672

```julia
println(evaluarEn3(exp))
println(exp(3))
```

    20.085536923187668
    20.085536923187668

## Funciones unidimensionales

### Primera aproximación: diferencia hacia adelante (forward difference)

Sea $h > 0$, $h << a$

$$
f(a+h) = f(a) + f'(a)h + R_{1}(a+h)
$$

Despejo la derivada:

$$
f'(a) = \frac{f(a+h)-f(a)}{h} - \frac{R_{1}(a+h)}{h} \approx \frac{f(a+h)-f(a)}{h}
$$

$$
\lim_{h \to 0} \frac{R_1(a+h)}{h} = 0
$$

Notemos que el error absoluto de nuestra aproximación ($f'(a) - \frac{f(a+h) - f(a)}{h}$) está dado por el término $ \frac{R_1 (a+h)}{h}$, el cuál sabemos que es proporcional a $h$ (ya que $R_1(a+h) \sim h^2$)

## Notación:

En la clase de complejidad, introducimos la notación $\mathcal{O}(f(n))$ para decir que la complejidad temporal o espacial de un algoritmo está acotada superiormente por una función de la forma $C \cdot f(n)$.

En **análisis numérico**, esa misma notación se utiliza para los errores numéricos. En el caso de la diferencia hacia adelante, por la forma del error absoluto, sabemos que este es $\mathcal{O}(h)$

En general, cuando el **error de aproximación** es proporcional a $h^p$, decimos que nuestro algoritmo numérico es de clase $\mathcal{O}(h^p)$

Pueden consultar más información sobre estas notaciones [en este link](https://courses.engr.illinois.edu/cs357/fa2019/references/ref-2-error/)

```julia
using Plots
```

```julia
function primera(x)
    return sin(2*pi*x)
end
```

    primera (generic function with 1 method)

```julia
xs = range(0,stop=1,length=100)
ys = [primera(x) for x in xs]
plot(xs,ys,title="funcion a derivar")
```

Quiero implementar la aproximación
$$
f'(a) = \frac{f(a+h) - f(a)}{h}
$$

```julia
# esta función me calcula la derivada de f en un punto a usando la diferencia hacia adelante con un valor de h
function difAdelante(f,a,h)
    return (f(a+h) - f(a)) / h
end
```

    difAdelante (generic function with 1 method)

Sabemos que

$$
primera'(x) = (\sin{2\pi x})' = 2 \pi \cos{2 \pi x}
$$

Podemos comparar el resultado numérico con el analítico

```julia
# derivada analítica de la funcion `primera`
function dprimera(x)
    return 2*pi*cos(2*pi*x)
end
```

    dprimera (generic function with 1 method)

```julia
println(difAdelante(primera,0,0.01))
println(dprimera(0))
# ver el error absoluto
println(abs(dprimera(0)-difAdelante(primera,0,0.01)))
```

    6.279051952931337
    6.283185307179586
    0.004133354248248899

```julia
# cambiando el valor de $h$ la aproximación cambia
h = 0.001
println(difAdelante(primera,0,h))
println(dprimera(0))
# ver el error absoluto
println(abs(dprimera(0)-difAdelante(primera,0,h)))
```

    6.283143965558951
    6.283185307179586
    4.134162063529345e-5

```julia
# ver la derivada como función
xs = range(0,stop=1,length=101)
ys1 = [dprimera(x) for x in xs]
h = 0.001
ys2 = [difAdelante(primera,x,h) for x in xs]
plot(xs,ys1,label="Valor analítico",title="Derivada")
plot!(xs,ys2,label="Diferencia hacia adelante")
```

## Ejercicios

1. Realiza una gráfica en la que compares la derivada analítica de la función $f(x) = \sin{x} \cdot \cos{x}$ con la obtenida por diferencia hacia adelante. Toma como dominio el intervalo $[-\pi,\pi]$

2. Para la función del inciso anterior, fija un valor de $x$, y gráfica, el error absoluto entre la derivada analítica y la numérica como función de $h$ ¿Qué observas? ¿Es lo esperado? Utiliza la escala log-log si es necesario.

#### ¿Qué otras aproximaciones hay?

### Segunda aproximación: diferencia hacia atrás (backwards difference)

Sea $h > 0$, $h << a$

$$
f(a-h) = f(a) + f'(a)(-h) + R_{1}(a-h)
$$

Despejo la derivada:

$$
f'(a) = \frac{f(a-h)-f(a)}{-h} - \frac{R_{1}(a-h)}{-h} = \frac{f(a)-f(a-h)}{h} + \frac{R_{1}(a-h)}{h}
$$

Obtenemos entonces:

$$
f'(a) \approx \frac{f(a)-f(a-h)}{h}
$$

$$
\lim_{h \to 0} \frac{R_1(a-h)}{h} = 0
$$

## Ejercicios
3. Define una función llamada `diferenciaAtras(f,a,h)` que aproxime la derivada de $f$ en a, $f'(a)$, utilizando  una diferencia hacia atras de orden $h$

### Tercera aproximación: diferencia centrada (central difference)

Primero expandimos $f(a+h)$ y $f(a-h)$ en polinomio de taylor de orden 2:

$$
f(a+h) = f(a) + f'(a) h + \frac{f''(a)}{2!} h^2 + R_2 (a+h)
$$

$$
\begin{split}
f(a-h) &= f(a) + f'(a)(- h) + \frac{f''(a)}{2!} (-h)^2 + R_2 (a-h) \\
&= f(a) - f'(a)h + \frac{f''(a)}{2!} h^2 + R_2 (a-h)
\end{split}
$$

Podemos restar ambas aproximaciones:

$$
\begin{split}
f(a+h) - f(a-h) &= (f(a) - f(a)) + (f'(a)h - (-f'(a) h)) + (\frac{f''(a)}{2!} h^2 - \frac{f''(a)}{2!} h^2) +  (R_2(a+h) - R_2(a-h)) \\
 &=  2 f'(a) h + (R_2(a+h) - R_2(a-h))
\end{split}
$$

Despejo la derivada:

$$
f'(a) = \frac{f(a+h) - f(a-h)}{2h} - \frac{R_2(a+h) - R_2(a-h)}{2h} \approx \frac{f(a+h) - f(a-h)}{2h}
$$

$$
\lim_{h \to 0} \frac{R_2(a+h) - R_2(a-h)}{2h} = 0
$$

## Ejercicios:

4. Define una función llamada `diferenciaCentrada(f,a,h)` que aproxime la derivada de $f$ en a, $f'(a)$, utilizando  una diferencia centrada de orden $h$

Para las siguientes funciones, realiza dos gráficas: una en la que compares las derivadas obtenidas analíticamente, con diferencia hacia adelante, diferencia hacia atrás y diferencia centrada, Y  otra gráfica en la que analices el error absoluto como función de $h$ para un punto fijo. Puedes escoger el dominio de la primera gráfica

5. $f_1(x) = 10x^2 + 6x - 1$

6. $f_2(x) = \sin{\left( \cos{(6x+2)} \right)}$

7. $f_3(x) = 2^{x \cdot \sin{x}}$

```julia
function ejer7(x)
    return 2^(x*sin(x))
end
```

```julia
using Plots
```

```julia
function difCentrada(f,a,h)
    return (f(a+h) - f(a-h))/(2*h)
end
```

## Derivadas de orden superior

Podemos seguir el mismo esquema para obtener aproximaciones de las derivadas de orden más grande. Esto no es un proceso simple, pero para orden 2 podemos hacerlo de manera muy sencilla.

## Ejercicios

8. Suma las expansiones en serie de taylor a orden 2 de $f(a+h)$ y $f(a-h) $ para obtener una expresión para la segunda derivada $f''(a)$. ¿Cuál es su error de aproximación?

9. Comprueba el error de aproximación obtenido al analizar el error absoluto en la segunda derivada de la función $\sin{(6x^2+1)}$ en un punto fijo del intervalo $[1,3]$

## Funciones multidimensionales

Podemos también utilizar diferencias finitas para aproximar las derivadas parciales de una función $f:\mathbb{R^n} \to \mathbb{R}$. Hacer expresiones de las derivadas de orden superior se vuelve complicado ya que hay que hacer expansiones de Taylor multidimensionales, pero para las primeras derivadas parciales basta recordar la definición

$$
\frac{\partial f}{\partial x_k} (a_1, \ldots, a_n) = \lim_{h \to 0} \frac{f(a_1, \ldots, a_k + h, \ldots, a_n) - f(a_1, \ldots, a_k, \ldots, a_n)}{h}
$$

## Ejercicios

10. Define una función `devParcial(f,i,A,h)`, donde $f:\mathbb{R^n} \to \mathbb{R}$ es una función multivariada, que toma como argumento un solo arreglo, $A$ es un arreglo de longitud $n$ e $1\leq i \leq n$ es un natural, que te regrese la derivdad parcial $\frac{\partial f}{\partial x_i} (A)$. Pruebala con alguna función de tu elección.

11. Sea $g:\mathbb{R}^2 \to \mathbb{R}$, obtén una expresión de diferencias finitas para la derivada curzada $\frac{\partial^2 g}{\partial x \partial y}$. ¿Cuál es el orden de su error?

## La visión general:

Existe una manera de derivar aproximaciones de diferencias finitas arbitrarias que sean $\mathcal{O}(h^p)$ para el $p \in \mathbb{N}$ que yo quiera. Ese método escapa al alcance del curso, pero lo pueden consultar en el libro de LeVeque.

Existe también una manera de calcular las derivadas de manera exacta, llamada **diferenciación automática**. No veremos ese tema pues requiere de conocimientos más avanzados de programación.

```julia

```


---
author: Nathan Carter (ncarter@bentley.edu)
---

This answer assumes you have imported SymPy as follows.

```python
from sympy import *                   # load all math functions
init_printing( use_latex='mathjax' )  # use pretty math output
```

Let's consider the example of the unit circle, $x^2+y^2=1$.

To plot it, SymPy first expects us to move everything to the left-hand side
of the equation, so in this case, we would have $x^2+y^2-1=0$.

We then use that left hand side to represent the equation as a single formula,
and computue $\frac{dy}{dx}$ using the `idiff` function (standing for
"implicit differentiation").

```python
var( 'x y' )
formula = x**2 + y**2 - 1   # to represent x^2+y^2=1
idiff( formula, y, x )
```

$\displaystyle - \frac{x}{y}$

So in this case, $\frac{dy}{dx}=-\frac xy$.

database/tasks/How to do implicit differentiation/Python, using SymPy.ipynb


```python
%matplotlib inline
import numpy as np
import pylab as plt
import pandas as pd
from sklearn import svm
from sklearn.metrics import classification_report,confusion_matrix,accuracy_score
```

```python
np.random.seed(0)
X = np.r_[np.random.randn(20, 2) - [2, 2], np.random.randn(20, 2) + [2, 2]]
Y = [0] * 20 + [1] * 20
plt.scatter(X[:, 0], X[:, 1], c=Y, cmap=plt.cm.coolwarm, s=25)
```

```python
fig, ax = plt.subplots()
clf2 = svm.LinearSVC(C=1).fit(X, Y)

# get the separating hyperplane
w = clf2.coef_[0]
a = -w[0] / w[1]
xx = np.linspace(-5, 5)
yy = a * xx - (clf2.intercept_[0]) / w[1]

# create a mesh to plot in
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx2, yy2 = np.meshgrid(np.arange(x_min, x_max, .2),
                     np.arange(y_min, y_max, .2))
Z = clf2.predict(np.c_[xx2.ravel(), yy2.ravel()])

Z = Z.reshape(xx2.shape)
ax.contourf(xx2, yy2, Z, cmap=plt.cm.coolwarm, alpha=0.3)
ax.scatter(X[:, 0], X[:, 1], c=Y, cmap=plt.cm.coolwarm, s=25)
ax.plot(xx,yy)

ax.axis([x_min, x_max,y_min, y_max])
plt.show()
```

```python
def make_region(X,Y,clf,ax):
    fig, ax = plt.subplots()
    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
    xx2, yy2 = np.meshgrid(np.arange(x_min, x_max, .2),
                         np.arange(y_min, y_max, .2))
    Z = clf.predict(np.c_[xx2.ravel(), yy2.ravel()])

    Z = Z.reshape(xx2.shape)
    ax.contourf(xx2, yy2, Z, cmap=plt.cm.coolwarm, alpha=0.3)
    ax.scatter(X[:, 0], X[:, 1], c=Y, cmap=plt.cm.coolwarm, s=25)
    ax.axis([x_min, x_max,y_min, y_max])
```

**Supervised learning**:

  - Linear models (Ridge, Lasso, Elastic Net, ...)
  - Support Vector Machines
  - Tree-based methods (Random Forests, Bagging, GBRT, ...)
  - Nearest neighbors
  - Neural networks (basics)

**Unsupervised learning**:

  - Clustering (KMeans, Ward, ...)
  - Outlier detection

## Accuracy and precision

1. precision:
The fraction of relevant instances among the retrieved instances,
1. recall:
The fraction of relevant instances that have been retrieved over the total amount of relevant instances.
1. F-score

* The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative.

* The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples.

* The F-beta score can be interpreted as a weighted harmonic mean of the precision and recall, where an F-beta score reaches its best value at 1 and worst score at 0. The F-beta score weights recall more than precision by a factor of beta. beta == 1.0 means recall and precision are equally important.

* The support is the number of occurrences of each class in y_true.

Take a look at [HERE](https://en.wikipedia.org/wiki/F1_score) or [HERE](https://en.wikipedia.org/wiki/Precision_and_recall).

```python
# Generate data
from sklearn.datasets import make_blobs,make_circles,make_moons
X, y = make_blobs(n_samples=1000, centers=2,
                  cluster_std=1.5,
                  center_box=(-4.0, 4.0))
# X, y = make_circles(n_samples=1000, noise=.1, factor=.5)
# X,y = make_moons(n_samples=1000, noise=.2)

plt.scatter(X[:,0],X[:,1],c=y,)

```

```python
print(X[:3])
```

    [[ 3.3168855   0.8283272 ]
     [-2.64062115 -0.47053114]
     [ 3.10295584  0.86800108]]

```python
print(y[:3])
```

    [1 0 1]

```python
# X is a 2 dimensional array, with 1000 rows and 2 columns
print(X.shape)

# y is a vector of 1000 elements
print(y.shape)
```

    (1000, 2)
    (1000,)

```python
X_train, y_train = X[:700], y[:700]
X_test, y_test = X[700:], y[700:]
```

```python
fig,(ax1,ax2) = plt.subplots(1,2,figsize=(14,7))

ax1.scatter(X_train[:,0],X_train[:,1],c=y_train,)
ax2.scatter(X_test[:,0],X_test[:,1],c=y_test,)

```

## K-Nearest Neighbours

```python
# K-Nearest Neighbours
from sklearn.neighbors import KNeighborsClassifier

Model = KNeighborsClassifier(n_neighbors=8)
Model.fit(X_train, y_train)

y_pred = Model.predict(X_test)

# Summary of the predictions made by the classifier
print(classification_report(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
# Accuracy score

print('accuracy is',accuracy_score(y_pred,y_test))
```

                  precision    recall  f1-score   support

               0       0.98      0.99      0.98       155
               1       0.99      0.98      0.98       145

       micro avg       0.98      0.98      0.98       300
       macro avg       0.98      0.98      0.98       300
    weighted avg       0.98      0.98      0.98       300

    [[153   2]
     [  3 142]]
    accuracy is 0.9833333333333333

```python
# Compute (approximate) class probabilities
print(Model.predict_proba(X_test[:5]))
```

    [[1. 0.]
     [0. 1.]
     [1. 0.]
     [0. 1.]
     [0. 1.]]

```python
make_region(X_test,y_test,Model,ax)
```

##  Radius Neighbors Classifier

```python
from sklearn.neighbors import  RadiusNeighborsClassifier
Model=RadiusNeighborsClassifier(radius=8.0)
Model.fit(X_train,y_train)
y_pred=Model.predict(X_test)
#summary of the predictions made by the classifier
print(classification_report(y_test,y_pred))
print(confusion_matrix(y_test,y_pred))
#Accouracy score
print('accuracy is ', accuracy_score(y_test,y_pred))
```

                  precision    recall  f1-score   support

               0       1.00      0.91      0.95       155
               1       0.91      1.00      0.95       145

       micro avg       0.95      0.95      0.95       300
       macro avg       0.96      0.95      0.95       300
    weighted avg       0.96      0.95      0.95       300

    [[141  14]
     [  0 145]]
    accuracy is  0.9533333333333334

```python
make_region(X_test,y_test,Model,ax)
```

## Naive Bayes

\begin{align}\begin{aligned}P(y \mid x_1, \dots, x_n) \propto P(y) \prod_{i=1}^{n} P(x_i \mid y)\\\Downarrow\\\hat{y} = \arg\max_y P(y) \prod_{i=1}^{n} P(x_i \mid y),\end{aligned}\end{align}

```python
# Naive Bayes
from sklearn.naive_bayes import GaussianNB
Model = GaussianNB()
Model.fit(X_train, y_train)

y_pred = Model.predict(X_test)

# Summary of the predictions made by the classifier
print(classification_report(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
# Accuracy score
print('accuracy is',accuracy_score(y_pred,y_test))
```

                  precision    recall  f1-score   support

               0       0.99      0.99      0.99       155
               1       0.99      0.99      0.99       145

       micro avg       0.99      0.99      0.99       300
       macro avg       0.99      0.99      0.99       300
    weighted avg       0.99      0.99      0.99       300

    [[153   2]
     [  2 143]]
    accuracy is 0.9866666666666667

```python
make_region(X_test,y_test,Model,ax)
```

## SVM

```python
# Support Vector Machine
from sklearn.svm import SVC

Model = SVC(kernel='linear')
# Model = svm.LinearSVC(C=1)

Model.fit(X_train, y_train)

y_pred = Model.predict(X_test)

# Summary of the predictions made by the classifier
print(classification_report(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
# Accuracy score

print('accuracy is',accuracy_score(y_pred,y_test))
```

                  precision    recall  f1-score   support

               0       0.99      0.99      0.99       155
               1       0.99      0.99      0.99       145

       micro avg       0.99      0.99      0.99       300
       macro avg       0.99      0.99      0.99       300
    weighted avg       0.99      0.99      0.99       300

    [[153   2]
     [  2 143]]
    accuracy is 0.9866666666666667

```python
make_region(X_test,y_test,Model,ax)
```

## Decision Tree

```python
# Decision Tree's
from sklearn.tree import DecisionTreeClassifier

Model = DecisionTreeClassifier()

Model.fit(X_train, y_train)

y_pred = Model.predict(X_test)

# Summary of the predictions made by the classifier
print(classification_report(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
# Accuracy score
print('accuracy is',accuracy_score(y_pred,y_test))
```

                  precision    recall  f1-score   support

               0       0.98      0.98      0.98       155
               1       0.98      0.98      0.98       145

       micro avg       0.98      0.98      0.98       300
       macro avg       0.98      0.98      0.98       300
    weighted avg       0.98      0.98      0.98       300

    [[152   3]
     [  3 142]]
    accuracy is 0.98

```python
y_pred = Model.predict(X_train)

# Summary of the predictions made by the classifier
print(classification_report(y_train, y_pred))
```

                  precision    recall  f1-score   support

               0       1.00      1.00      1.00       345
               1       1.00      1.00      1.00       355

       micro avg       1.00      1.00      1.00       700
       macro avg       1.00      1.00      1.00       700
    weighted avg       1.00      1.00      1.00       700

```python
make_region(X_train,y_train,Model,ax)
```

```python
make_region(X_test,y_test,Model,ax)
```

## RandomForest

```python
from sklearn.ensemble import RandomForestClassifier
Model=RandomForestClassifier(max_depth=2)
Model.fit(X_train,y_train)
y_pred=Model.predict(X_test)
print(classification_report(y_test,y_pred))
print(confusion_matrix(y_pred,y_test))
#Accuracy Score
print('accuracy is ',accuracy_score(y_pred,y_test))
```

                  precision    recall  f1-score   support

               0       0.99      0.98      0.99       155
               1       0.98      0.99      0.99       145

       micro avg       0.99      0.99      0.99       300
       macro avg       0.99      0.99      0.99       300
    weighted avg       0.99      0.99      0.99       300

    [[152   1]
     [  3 144]]
    accuracy is  0.9866666666666667

    /home/gf/packages/anaconda3/lib/python3.6/site-packages/sklearn/ensemble/forest.py:246: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.
      "10 in version 0.20 to 100 in 0.22.", FutureWarning)

```python
# def visualize_classifier(model, X, y, ax=None, cmap='rainbow'):
#     ax = ax or plt.gca()

#     # Plot the training points
#     ax.scatter(X[:, 0], X[:, 1], c=y, s=30, cmap=cmap,
#                clim=(y.min(), y.max()), zorder=3)
#     ax.axis('tight')
#     ax.axis('off')
#     xlim = ax.get_xlim()
#     ylim = ax.get_ylim()

#     # fit the estimator
#     model.fit(X, y)
#     xx, yy = np.meshgrid(np.linspace(*xlim, num=200),
#                          np.linspace(*ylim, num=200))
#     Z = model.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)

#     # Create a color plot with the results
#     n_classes = len(np.unique(y))
#     contours = ax.contourf(xx, yy, Z, alpha=0.3,
#                            levels=np.arange(n_classes + 1) - 0.5,
#                            cmap=cmap,
#                            zorder=1)

#     ax.set(xlim=xlim, ylim=ylim)
```

```python
make_region(X_test,y_test,Model,ax)
```

```python
Model.feature_importances_
```

    array([0.76430017, 0.23569983])

## Neural network

```python
from sklearn.neural_network import MLPClassifier
Model=MLPClassifier()
Model.fit(X_train,y_train)
y_pred=Model.predict(X_test)
# Summary of the predictions
print(classification_report(y_test,y_pred))
print(confusion_matrix(y_test,y_pred))
#Accuracy Score
print('accuracy is ',accuracy_score(y_pred,y_test))
```

```python
make_region(X_test,y_test,Model,ax)
```

```python

```

```python

```

```python

```