# Introduction to PyRational ADG models

**author:Alessio Benavoli**
 alessiobenavoli.com

We will see in this tutorial the basics for building a PyRational model.

We will consider  **ADG: Almost Desirable Gambles** belief model.

To understand this model, we consider a coin tossing *experiment* whose outcome  $\omega$ belongs to the  space of possibilities $\Omega$ (Head or Tail). We can model agent's beliefs (we call our agent Alice) about $\omega$ by asking her whether she accepts engaging in certain **risky transactions**, called **gambles**, whose outcome depends on the actual
outcome of the experiment.

Mathematically, a gamble is a bounded real-valued function on $\Omega$, $g:\Omega
\rightarrow \mathbb{R}$. If Alice accepts a gamble $g$, this means that she commits herself to
receive $g(\omega)$ euros if the outcome of the experiment eventually happens
to be the event $\omega \in \Omega$. Since $g(\omega)$ can be negative, Alice can also lose euros. Therefore Alice's acceptability of a gamble depends on her knowledge about the experiment.

The  set of gambles that Alice accepts is called her set of *desirable gambles*.
One such set is said to be **coherent** when it satisfies the following criteria:

* A.1: Any gamble $g$ such that $g(\omega)\geq0$ for each $\omega \in \Omega$ must be desirable for Alice, given that it may increase Alice's capital without ever decreasing it  (**accepting partial gain**).
* A.2: Any gamble $g$ such that $\max g<0$  must not be desirable for Alice, given
that it may only decrease Alice's capital without ever increasing it  (**avoiding sure loss**).
* A.3: If Alice finds $g$  desirable,  then also $\lambda g$ must be desirable for her for any $0<\lambda \in \mathbb{R}$ (**positive homogeneity**).
* A.4: If Alice finds $g_1$ and $g_2$ desirable, then she also must accept $g_1+g_2$ (**additivity**).
* A.5: If Alice finds $g+\delta$  desirable for every $\delta\geq0$, then also $g$ should be desirable to her (**closure**).

If the set of desirable gamble $G$ satisfies these property we say that it is *coherent*, or, equivalently, that **Alice is rational**.

Note how the first four axioms express some truly minimal requirements: the first means that Alice likes to increase her wealth; the second that she does not like to decrease it; the third and fourth together simply rephrase the assumption that Alice's utility scale is linear. The last axiom is a continuity property.

We will now build our first ADG model. We will first import the libraries we will need:

```python
%load_ext autoreload
%autoreload 2
from __future__ import absolute_import
from PyRational.models.ADG import  ADG
from PyRational.special.gambles import indicator
import numpy as np
from sympy import symbols,  FiniteSet, Piecewise, Eq
```

*PyRational* requires numerical input variables. We therefore encode categorical symbols (Head and Tail) into integer data (1, 0).

*PyRational* uses *Sympy* for symbolic mathematics. We need to define in *Sympy*  a `symbol` associated to the coin toss outcome and its relative domain {0,1} (we use Sympy `FiniteSet` for the latter).

```python
x  = symbols('x');
domain_x=FiniteSet(0,1)
```

We are now ready to build our first *PyRational* model: **ADG**.

```python
ListSymbols=[x]
ListDomains=[domain_x]
model = ADG(ListSymbols,ListDomains)
model
```

ADG model

  List of Symbols 	[x]

  Domain 	Ω={0, 1}

  List of desirable gambles 	G=[]

  Avoiding sure loss? 	  to be verified

We will clarify later the meaning of "avoiding sure loss".
The set $G$, the gambles that Alice finds desirable, is empty at the moment.

What is a gamble in the coin experiment? We can model any gamble as a combination of `indicator` functions.

```python
g = -0.1*indicator('Eq(x,1)') + 1*indicator('Eq(x,0)')
```

In this example Alice receives 1 euro if Tails and loses -0.1 euro if Heads. Note that, in Sympy, the == must be replaced by Eq(.,.). The indicator is implemented as a `Piecewise` Sympy function:

```python
g
```

    Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True))

Therefore, you can write the same gamble as

```python
1*Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True))
```

    Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True))

We can evaluate this gamble as follows:

```python
print('if Tails: ', g.subs(x,0),', if Heads:',g.subs(x,1))
```

    if Tails:  1 , if Heads: -0.100000000000000

Which gambles does Alice find desirable?

Alice is obviously willing to accept any gamble $g$ that, no matter the result of the experiment,
may increase her wealth without ever decreasing it, that is with $g \geq 0$.
For instance, the following gambles:

```python
g1 = 1.0*indicator('Eq(x,1)') + 1.0*indicator('Eq(x,0)')
g2 = 0.1*indicator('Eq(x,1)') + 1.0*indicator('Eq(x,0)')
```

Similarly.  Alice does not accept  any gamble $g$ that will surely decrease her wealth, that is with  $\max g<0$. For the other types of gambles, their desirability depends on Alice's beliefs about the coin.

Let's assume Alice thinks that the coin is slightly biased.  So she decides to accept a gamble, only if the ratio between its positive part and negative part is greater than $1.1$:

```python
g3 =    1*indicator('Eq(x,1)') - 0.1*indicator('Eq(x,0)')
g4 = -0.1*indicator('Eq(x,1)') +   1*indicator('Eq(x,0)')
g5 =    2*indicator('Eq(x,1)') - 0.5*indicator('Eq(x,0)')
g6 =  0.5*indicator('Eq(x,1)') - 0.4*indicator('Eq(x,0)')
g7 = -0.4*indicator('Eq(x,1)') + 0.5*indicator('Eq(x,0)')
g8 =  1.1*indicator('Eq(x,1)') -   1*indicator('Eq(x,0)')
g9 =   -1*indicator('Eq(x,1)') + 1.1*indicator('Eq(x,0)')
```

We add all these gambles to `model` as follows:

```python
model.add_gambleList([g1,g2,g3,g4,g5,g6,g7,g8,g9])
model
```

ADG model

  List of Symbols 	[x]

  Domain 	Ω={0, 1}

  List of desirable gambles 	G=[1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 1.0*Piecewise((1, Eq(x, 1)), (0, True)), 1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 0.1*Piecewise((1, Eq(x, 1)), (0, True)), -0.1*Piecewise((1, Eq(x, 0)), (0, True)) + Piecewise((1, Eq(x, 1)), (0, True)), Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True)), -0.5*Piecewise((1, Eq(x, 0)), (0, True)) + 2*Piecewise((1, Eq(x, 1)), (0, True)), -0.4*Piecewise((1, Eq(x, 0)), (0, True)) + 0.5*Piecewise((1, Eq(x, 1)), (0, True)), 0.5*Piecewise((1, Eq(x, 0)), (0, True)) - 0.4*Piecewise((1, Eq(x, 1)), (0, True)), -Piecewise((1, Eq(x, 0)), (0, True)) + 1.1*Piecewise((1, Eq(x, 1)), (0, True)), 1.1*Piecewise((1, Eq(x, 0)), (0, True)) - Piecewise((1, Eq(x, 1)), (0, True))]

  Avoiding sure loss? 	  to be verified

Note that $G$ is a list that includes all Alice's desirable gambles. We now `build` the belief model.

```python
model.buildModel()
model
```

ADG model

  List of Symbols 	[x]

  Domain 	Ω={0, 1}

  List of desirable gambles 	G=posi(ℒ(Ω)+      ∪    [1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 1.0*Piecewise((1, Eq(x, 1)), (0, True)), 1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 0.1*Piecewise((1, Eq(x, 1)), (0, True)), -0.1*Piecewise((1, Eq(x, 0)), (0, True)) + Piecewise((1, Eq(x, 1)), (0, True)), Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True)), -0.5*Piecewise((1, Eq(x, 0)), (0, True)) + 2*Piecewise((1, Eq(x, 1)), (0, True)), -0.4*Piecewise((1, Eq(x, 0)), (0, True)) + 0.5*Piecewise((1, Eq(x, 1)), (0, True)), 0.5*Piecewise((1, Eq(x, 0)), (0, True)) - 0.4*Piecewise((1, Eq(x, 1)), (0, True)), -Piecewise((1, Eq(x, 0)), (0, True)) + 1.1*Piecewise((1, Eq(x, 1)), (0, True)), 1.1*Piecewise((1, Eq(x, 0)), (0, True)) - Piecewise((1, Eq(x, 1)), (0, True))])

  Avoiding sure loss? 	 Yes

You can think about "model building" as a compiling phase for the belief model.

It adds to $G$ all gambles that are implied by axioms A.1, A.3, A.4, A.5.

This means that Alice doesn't need to say that $g_1,g_2$ (in the above example) are desirable, because this is implied by A.1. Similarly, she doesn't need to say that $10*g_3$ is also desirable to her, because this is implied by A.3 and so on.

Given that A.1, A.3, A.4, A.5 are satisfied, to check that Alice is rational, it remains to verify that A.2 is also satisfied. We use `check_avs` for that.
Before doing that, we will set some options for the optimiser. I will explain the meaning of these options in another notebook.

```python
optimoptions={ 'method_LISP': 'Cutting_plane',
               'SolverLP':'linprog',
               'LP_acc_constraints':1e-300,
               'SolverNLP':'differential_evolution',
               'NLP_alpha_cut':-0.009,
               'num_support_points': 10,
               'verbose':False}

model.check_avs(options=optimoptions)
model
```

    Belief Model avoids sure  loss

ADG model

  List of Symbols 	[x]

  Domain 	Ω={0, 1}

  List of desirable gambles 	G=posi(ℒ(Ω)+      ∪    [1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 1.0*Piecewise((1, Eq(x, 1)), (0, True)), 1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 0.1*Piecewise((1, Eq(x, 1)), (0, True)), -0.1*Piecewise((1, Eq(x, 0)), (0, True)) + Piecewise((1, Eq(x, 1)), (0, True)), Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True)), -0.5*Piecewise((1, Eq(x, 0)), (0, True)) + 2*Piecewise((1, Eq(x, 1)), (0, True)), -0.4*Piecewise((1, Eq(x, 0)), (0, True)) + 0.5*Piecewise((1, Eq(x, 1)), (0, True)), 0.5*Piecewise((1, Eq(x, 0)), (0, True)) - 0.4*Piecewise((1, Eq(x, 1)), (0, True)), -Piecewise((1, Eq(x, 0)), (0, True)) + 1.1*Piecewise((1, Eq(x, 1)), (0, True)), 1.1*Piecewise((1, Eq(x, 0)), (0, True)) - Piecewise((1, Eq(x, 1)), (0, True))])

  Avoiding sure loss? 	 Yes

So Alice is **rational** or, equivalently, her set of desirable gambles is coherent.

### Inference

ADG allows Alice to make inferences on her model.

For instance, she may want to assess if a certain gamble $f$ is desirable to her, given
she has accepted the gambles in $G$. In other words, she wants to assess if this gamble belongs to the set `model.G`

```python
f=10*g3
f_range=(None,None)
model.natural_extension(f,f_range,options=optimoptions)
```

    True

This is True, because the desirability of $10*g3$ follows from A.3.
Here, `f_range`is the range of $f$, for the moment we can leave it indetermined. We will use it in another notebook. Similarly, we have

```python
f=10*g3+g4
f_range=(None,None)
model.natural_extension(f,f_range,options=optimoptions)
```

    True

```python
f=-g1
f_range=(None,None)
model.natural_extension(f,f_range,options=optimoptions)
```

    False

```python
f=1*Piecewise((1, Eq(x, 1)), (0, True)) - 1.5*Piecewise((1, Eq(x, 0)), (0, True))
f_range=(None,None)
model.natural_extension(f,f_range,options=optimoptions)
```

    False

Another important inference in ADG is `lower prevision`. This allows Alice to determine
her maximum buying price for a gamble: how much she should pay to buy a certain gamble.

```python
lp=model.lower_prevision(f,f_range,options=optimoptions)
print(lp)
```

    -0.30952380952380953

In other words, `lp` is the largest value such that $f-lp$ is desirable to Alice.

```python
f1=f-lp
model.natural_extension(f1,f_range,options=optimoptions)
```

    True

```python
f1=f-lp-0.00000001
model.natural_extension(f1,f_range,options=optimoptions)
```

    False

Similarly, Alice may want to determine her minimum selling price for a gamble. This is the `upper_prevision` of $f$

```python
up=model.upper_prevision(f,f_range,options=optimoptions)
print(up)
```

    -0.19047619047619044

```python
f1=up-f
model.natural_extension(f1,f_range,options=optimoptions)
```

    True

```python
f1=up-f-0.00000001
model.natural_extension(f1,f_range,options=optimoptions)
```

    False

## Updating: inference

Alice may want to evaluate how her inferences would change if she knew that the result of the coin toss will be Head.

This operation is called `updating` in ADG and is formally equivalent to probabilistic conditioning. It is performed by firstly defining an indicator on the event of interest:

```python
h = Piecewise((1.0,Eq(x,1)),(0.0,Eq(x,0)))
```

and then re-computing `lower_prevision` and `upper_prevision` conditionally on this event.

```python
lp=model.lower_prevision(f,f_range,h=h,options=optimoptions)
print(lp)
```

    1.0

```python
up=model.upper_prevision(f,f_range,h=h,options=optimoptions)
print(up)
```

    1.0

Note that, since she knows that the result is Heads, then she is sure that the payoff for $f$ is one.

## Updating: model

Alice may also want to update her whole belief model conditionally on $h$. This can be done as follows:

```python
NewG = model.updating(h,options=optimoptions)
NewG
```

    [1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 1.0*Piecewise((1, Eq(x, 1)), (0, True)) - 1.0,
     1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 0.1*Piecewise((1, Eq(x, 1)), (0, True)) - 0.1,
     -0.1*Piecewise((1, Eq(x, 0)), (0, True)) + Piecewise((1, Eq(x, 1)), (0, True)) - 1.0,
     Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True)) + 0.1,
     -0.5*Piecewise((1, Eq(x, 0)), (0, True)) + 2*Piecewise((1, Eq(x, 1)), (0, True)) - 2.0,
     -0.4*Piecewise((1, Eq(x, 0)), (0, True)) + 0.5*Piecewise((1, Eq(x, 1)), (0, True)) - 0.5,
     0.5*Piecewise((1, Eq(x, 0)), (0, True)) - 0.4*Piecewise((1, Eq(x, 1)), (0, True)) + 0.4,
     -Piecewise((1, Eq(x, 0)), (0, True)) + 1.1*Piecewise((1, Eq(x, 1)), (0, True)) - 1.1,
     1.1*Piecewise((1, Eq(x, 0)), (0, True)) - Piecewise((1, Eq(x, 1)), (0, True)) + 1.0]

It returns the updated set of desirable gambles. We can use it to build a new belief model.

```python
model1 = ADG(ListSymbols,ListDomains)
model1.add_gambleList(NewG)
model1.buildModel()
model1.check_avs()
model1
```

    Belief Model avoids sure  loss

ADG model

  List of Symbols 	[x]

  Domain 	Ω={0, 1}

  List of desirable gambles 	G=posi(ℒ(Ω)+      ∪    [1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 1.0*Piecewise((1, Eq(x, 1)), (0, True)) - 1.0, 1.0*Piecewise((1, Eq(x, 0)), (0, True)) + 0.1*Piecewise((1, Eq(x, 1)), (0, True)) - 0.1, -0.1*Piecewise((1, Eq(x, 0)), (0, True)) + Piecewise((1, Eq(x, 1)), (0, True)) - 1.0, Piecewise((1, Eq(x, 0)), (0, True)) - 0.1*Piecewise((1, Eq(x, 1)), (0, True)) + 0.1, -0.5*Piecewise((1, Eq(x, 0)), (0, True)) + 2*Piecewise((1, Eq(x, 1)), (0, True)) - 2.0, -0.4*Piecewise((1, Eq(x, 0)), (0, True)) + 0.5*Piecewise((1, Eq(x, 1)), (0, True)) - 0.5, 0.5*Piecewise((1, Eq(x, 0)), (0, True)) - 0.4*Piecewise((1, Eq(x, 1)), (0, True)) + 0.4, -Piecewise((1, Eq(x, 0)), (0, True)) + 1.1*Piecewise((1, Eq(x, 1)), (0, True)) - 1.1, 1.1*Piecewise((1, Eq(x, 0)), (0, True)) - Piecewise((1, Eq(x, 1)), (0, True)) + 1.0])

  Avoiding sure loss? 	 Yes

```python
model1.lower_prevision(f,f_range,options=optimoptions)
```

    1.0

This is inference is equivalent to `model1.lower_prevision(f,f_range,h=h,options=optimoptions)`


# Part 1 - Scalars and Vectors

For the questions below it is not sufficient to simply provide answer to the questions, but you must solve the problems and show your work using python (the NumPy library will help a lot!) Translate the vectors and matrices into their appropriate python  representations and use numpy or functions that you write yourself to demonstrate the result or property.

## 1.1 Create a two-dimensional vector and plot it on a graph

```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
```

```
dv_2 = [1,1]
```

```
plt.arrow(0,0, dv_2[0], dv_2[1],head_width=.05, head_length=0.05, color ='orange')
plt.xlim(0,2)
plt.ylim(0,2)
```

## 1.2 Create a three-dimensional vecor and plot it on a graph

```
from mpl_toolkits.mplot3d import Axes3D
```

```
dv_3 = [1,1,1]
```

```
vectors = np.array([[0,0,0,1,1,1]])
```

```
X, Y, Z, U, V, W = zip(*vectors)
```

```
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.quiver(X, Y, Z, U, V, W, length=0.05)
```

## 1.3 Scale the vectors you created in 1.1 by $5$, $\pi$, and $-e$ and plot all four vectors (original + 3 scaled vectors) on a graph. What do you notice about these vectors?

```
from math import e, pi
print(e)
print(pi)
```

    2.718281828459045
    3.141592653589793

```
np_dv_2 = np.array(dv_2)
```

```
np_dv2_e = 2.718281828459045*(np_dv_2)
np_dv2_pi = 3.141592653589793*(np_dv_2)

```

```
print (np_dv2_e)
print (np_dv2_pi)
```

    [2.71828183 2.71828183]
    [3.14159265 3.14159265]

```
plt.arrow(0,0, dv_2[0], dv_2[1],head_width=.05, head_length=23, color ='orange')
plt.arrow(0,0, np_dv2_e[0], np_dv2_e[1],head_width=.05, head_length=40, color ='blue')
plt.arrow(0,0, np_dv2_pi[0], np_dv2_pi[1],head_width=.05, head_length=10, color ='red')
plt.xlim(-1,40)
plt.ylim(-1,40)
plt.title('Vectors')
plt.show()
```

```
red = [0.5,0.8]
```

```
green = np.multiply(3.141592653589793, red)

blue = np.multiply(2.718281828459045, red)
```

```
plt.arrow(0,0, green[0], green[0], head_width=0.05, head_length=0.05, color='green')
plt.arrow(0,0, blue[0], blue[0], head_width=0.05, head_length=0.05, color='blue')
plt.arrow(0,0, red[0], red[0], head_width=0.05, head_length=0.05, color='red')
plt.xlim(-1,2)
plt.ylim(-1,2)
```

## 1.4 Graph vectors $\vec{a}$ and $\vec{b}$ and plot them on a graph

\begin{align}
\vec{a} = \begin{bmatrix} 5 \\ 7 \end{bmatrix}
\qquad
\vec{b} = \begin{bmatrix} 3 \\4 \end{bmatrix}
\end{align}

```
a = [5,7]
b = [3,4]
```

```
plt.arrow(0,0, a[0], a[1], head_width=0.05, head_length=0.05, color='green')
plt.arrow(0,0, b[0], b[1], head_width=0.05, head_length= 0.05, color='blue')
plt.xlim(-1,8)
plt.ylim(-1,8)
```

## 1.5 find $\vec{a} - \vec{b}$ and plot the result on the same graph as $\vec{a}$ and $\vec{b}$. Is there a relationship between vectors $\vec{a} \thinspace, \vec{b} \thinspace \text{and} \thinspace \vec{a-b}$

```
np_a = np.array(a)
np_b = np.array(b)
```

```
print (np_a - np_b)
```

    [2 3]

```
np_ab =(np_a -np_b)
```

```
plt.arrow(0,0, a[0], a[1], head_width=0.05, head_length=0.05, color='green')
plt.arrow(0,0, b[0], b[1], head_width=0.05, head_length= 0.05, color='blue')
plt.arrow(0,0, np_ab[0], np_ab[1,], head_width=0.05, head_length= 0.05, color='red')
plt.xlim(-1,8)
plt.ylim(-1,8)
```

## 1.6 Find $c \cdot d$

\begin{align}
\vec{c} = \begin{bmatrix}7 & 22 & 4 & 16\end{bmatrix}
\qquad
\vec{d} = \begin{bmatrix}12 & 6 & 2 & 9\end{bmatrix}
\end{align}

```
c = [7,22,4,16]
d = [12,6,2,9]
c_np = np.array(c)
d_np = np.array(d)
```

```
import scipy as sp
```

```
c_dot_d = np.dot(c_np, d_np)
c_dot_d
```

    368

##  1.7 Find $e \times f$

\begin{align}
\vec{e} = \begin{bmatrix} 5 \\ 7 \\ 2 \end{bmatrix}
\qquad
\vec{f} = \begin{bmatrix} 3 \\4 \\ 6 \end{bmatrix}
\end{align}

```
e =[5,7,2]
f= [3,4,6]
e_np = np.array(e)
f_np = np.array(f)
```

```
print (f_np * e_np)
```

    [15 28 12]

## 1.8 Find $||g||$ and then find $||h||$. Which is longer?

\begin{align}
\vec{g} = \begin{bmatrix} 1 \\ 1 \\ 1 \\ 8 \end{bmatrix}
\qquad
\vec{h} = \begin{bmatrix} 3 \\3 \\ 3 \\ 3 \end{bmatrix}
\end{align}

```
g = np.array([
      [1],
      [1],
      [1],
      [8]
])
h = np.array([
            [3],
            [3],
            [3],
            [3]
])
```

```
type(g2)
```

    numpy.ndarray

```
g2 = np.linalg(g)
h2 = np.linalg(h)
```

```

```

# Part 2 - Matrices

## 2.1 What are the dimensions of the following matrices? Which of the following can be multiplied together? See if you can find all of the different legal combinations.
\begin{align}
A = \begin{bmatrix}
1 & 2 \\
3 & 4 \\
5 & 6
\end{bmatrix}
\qquad
B = \begin{bmatrix}
2 & 4 & 6 \\
\end{bmatrix}
\qquad
C = \begin{bmatrix}
9 & 6 & 3 \\
4 & 7 & 11
\end{bmatrix}
\qquad
D = \begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
\qquad
E = \begin{bmatrix}
1 & 3 \\
5 & 7
\end{bmatrix}
\end{align}

```
# a [3,2]
# b[1,3]
#c [2,3]
# d [3,3]
# e [2,2]
# cd, cb, ae, bd
```

## 2.2 Find the following products: CD, AE, and BA. What are the dimensions of the resulting matrices? How does that relate to the dimensions of their factor matrices?

```
a = np.array([
              [1,2],
              [3,4],
              [5,6]
])
b = np.array([2,4,6])
c = np.array([
              [9,6,3],
              [4,7,11]
])
d = np.array([
              [1,0,0],
              [0,1,0],
              [0,0,1]
])
e = np.array([
              [1,3],
              [5,7]
])
```

```
np.matmul(c,d)
```

    array([[ 9,  6,  3],
           [ 4,  7, 11]])

```
np.matmul(a,e)
```

    array([[11, 17],
           [23, 37],
           [35, 57]])

```
np.matmul(b,a)
```

    array([44, 56])

## 2.3  Find $F^{T}$. How are the numbers along the main diagonal (top left to bottom right) of the original matrix and its transpose related? What are the dimensions of $F$? What are the dimensions of $F^{T}$?

\begin{align}
F =
\begin{bmatrix}
20 & 19 & 18 & 17 \\
16 & 15 & 14 & 13 \\
12 & 11 & 10 & 9 \\
8 & 7 & 6 & 5 \\
4 & 3 & 2 & 1
\end{bmatrix}
\end{align}

```
f = np.array([
              [20,19,18,17],
              [16,15,14,13],
              [12,11,10,9],
              [8,7,6,5],
              [4,3,2,1]
])
```

```
# the numbers along the main diagnol stay the same
# f [5,4]
# fT [4,5]
```

```
f.T
```

    array([[20, 16, 12,  8,  4],
           [19, 15, 11,  7,  3],
           [18, 14, 10,  6,  2],
           [17, 13,  9,  5,  1]])

# Part 3 - Square Matrices

## 3.1 Find $IG$ (be sure to show your work) 😃

You don't have to do anything crazy complicated here to show your work, just create the G matrix as specified below, and a corresponding 2x2 Identity matrix and then multiply them together to show the result. You don't need to write LaTeX or anything like that (unless you want to).

\begin{align}
G=
\begin{bmatrix}
13 & 14 \\
21 & 12
\end{bmatrix}
\end{align}

```
g = np.array([
              [13,14],
              [21,12]
])
```

## 3.2 Find $|H|$ and then find $|J|$.

\begin{align}
H=
\begin{bmatrix}
12 & 11 \\
7 & 10
\end{bmatrix}
\qquad
J=
\begin{bmatrix}
0 & 1 & 2 \\
7 & 10 & 4 \\
3 & 2 & 0
\end{bmatrix}
\end{align}

```
h = np.array([
              [12,11],
              [7,10]
])
j = np.array([
              [0,1,2],
              [7,10,4],
              [3,2,0]
])
```

```
np.linalg.det(h)
```

    43.000000000000014

```
np.linalg.det(j)
```

    -19.999999999999996

## 3.3 Find $H^{-1}$ and then find $J^{-1}$

```
np.linalg.inv(h)
```

    array([[ 0.23255814, -0.25581395],
           [-0.1627907 ,  0.27906977]])

```
np.linalg.inv(j)
```

    array([[ 0.4 , -0.2 ,  0.8 ],
           [-0.6 ,  0.3 , -0.7 ],
           [ 0.8 , -0.15,  0.35]])

## 3.4 Find $HH^{-1}$ and then find $J^{-1}J$. Is $HH^{-1} == J^{-1}J$? Why or Why not?

Please ignore Python rounding errors. If necessary, format your output so that it rounds to 5 significant digits (the fifth decimal place).

Yes, they will both equal one because a ,matrix multiplied by its inverse will
always equal one

```
h_inv = np.linalg.inv(h)
j_inv = np.linalg.inv(j)
```

```
np.matmul(h,h_inv)
```

    array([[1.00000000e+00, 5.55111512e-16],
           [2.22044605e-16, 1.00000000e+00]])

```
np.matmul(j,j_inv)
```

    array([[ 1.00000000e+00, -5.55111512e-17,  0.00000000e+00],
           [ 0.00000000e+00,  1.00000000e+00,  0.00000000e+00],
           [ 0.00000000e+00,  0.00000000e+00,  1.00000000e+00]])

# Stretch Goals:

A reminder that these challenges are optional. If you finish your work quickly we welcome you to work on them. If there are other activities that you feel like will help your understanding of the above topics more, feel free to work on that. Topics from the Stretch Goals sections will never end up on Sprint Challenges. You don't have to do these in order, you don't have to do all of them.

- Write a function that can calculate the dot product of any two vectors of equal length that are passed to it.
- Write a function that can calculate the norm of any vector
- Prove to yourself again that the vectors in 1.9 are orthogonal by graphing them.
- Research how to plot a 3d graph with animations so that you can make the graph rotate (this will be easier in a local notebook than in google colab)
- Create and plot a matrix on a 2d graph.
- Create and plot a matrix on a 3d graph.
- Plot two vectors that are not collinear on a 2d graph. Calculate the determinant of the 2x2 matrix that these vectors form. How does this determinant relate to the graphical interpretation of the vectors?


# Подбор параметрической неопределённости методами Монте-Карло

Стандарты по оценке неопределённости:
* ГОСТ 34100.3.1-2017
* SO/IEC GUIDE 98-3:2008

```python
# Загрузка нужных библиотек
from scipy import stats, interpolate
import numpy as np
from numpy.random import random
from matplotlib import pyplot as plt
import seaborn as sns
from statsmodels.distributions.empirical_distribution import ECDF
import pandas as pd
from tqdm import tqdm

plt.rcParams['figure.figsize'] = [8, 6]
```

    /usr/local/lib/python3.7/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm

## Загрузка данных

```python
# Получение доступа к гугл диску
from google.colab import drive
drive.mount('/content/drive')
```

    Mounted at /content/drive

```python
# Загрузка данных из Экселя
df = pd.read_excel('/content/drive/MyDrive/Colab Notebooks/PKS_OC.xlsx', sheet_name='data')
# Удалить пустые записи
df.dropna(inplace=True)
# Нормализация значения вероятности [0...1]
df['F(x)']=df['F(x)']/df['F(x)'].max()
# Показать несколько первых строк
df.head()
```

      calc
      var
      x
      F(x)

      0
      CDF_JNB56_TNFD
      Pc(AS)
      2.001000e-12
      0.00

      1
      CDF_JNB56_TNFD
      Pc(AS)
      1.815000e-11
      0.01

      2
      CDF_JNB56_TNFD
      Pc(AS)
      2.815000e-11
      0.02

      3
      CDF_JNB56_TNFD
      Pc(AS)
      3.711000e-11
      0.03

      4
      CDF_JNB56_TNFD
      Pc(AS)
      4.464000e-11
      0.04

```python
# Стоим обратные функции для CDF и заполняем ими хештаблицу хештаблиц
inv_cdf = {}
calc_group = df.groupby(by=['calc'])
for calc_name, calc_df in calc_group:
  var_dic = {}
  var_group = calc_df.groupby(by=['var'])

  for var_name, var_df in var_group:
    # Строим линейную интерполяцию инвертированной CDF. В результате получаем функцию.
    var_dic[var_name] = interpolate.interp1d(var_df['F(x)'],var_df['x'])

  inv_cdf[calc_name] = var_dic
```

## Определение исследуемых выражений

Формула 1.6.4:
$$t_b = \frac{P_c(AS)}{F_c^{00}-F_c^{01}}$$

```python
def f164(P_cas, F_c00, F_c01):
  a =  F_c00 - F_c01
  #print(a)
  #a = np.where(a > 1e-10, a, 1e-10)
  return  P_cas/(a)
```

Формула 1.6.5:
$$t_b = \frac{P_c(AS)}{F_c^{00}-F_b^{00}}$$

```python
def f165(P_cas, F_c00, F_b00):
  a =  F_c00 - F_b00
  #print(a)
  #a = np.where(a > 1e-10, a, 1e-10)
  return  P_cas/(a)
```

## Независимые переменные

```python
# Генерируем случайные значения на оснорве эмперической CDF (https://en.wikipedia.org/wiki/Inverse_transform_sampling
# Количество семплов
N=int(1e+6)
samples_df = pd.DataFrame()
for calc_item in inv_cdf.items():
  for var_item in calc_item[1].items():
    # Генерируем N случайных значений (непрерывное равномерное распределение [0...1])
    uniform_samples = random(N)
    #print(calc_item[0])
    #print(var_item[1])
    samples_df = samples_df.append(pd.DataFrame({
        'calc':calc_item[0],
        'var':var_item[0],
        'value':var_item[1](uniform_samples)}))
```

```python
# Строим гитограммы полученных семплов
calc_group = samples_df.groupby(by=['calc'])
for calc_name, calc_df in calc_group:
  ax = sns.histplot(calc_df,x='value', hue='var', log_scale=True, bins=100)
  ax.set_title(calc_name)
  plt.show()
```

```python
# Расчёт по формуле 1.6.5
v165 = {}
calc_group = samples_df.groupby(by=['calc'])
for calc_name, calc_df in calc_group:

  if calc_name in ['CDF_JNB56-57_TCLIE', 'CDF_JNB56-57_TNFD']:
    v = f165(calc_df[calc_df['var']=='Pc(AS)']['value'],
                         calc_df[calc_df['var']=='Fc00']['value'],
                         calc_df[calc_df['var']=='Fc01']['value'])
  else:

    v = f165(calc_df[calc_df['var']=='Pc(AS)']['value'],
                         calc_df[calc_df['var']=='Fc00']['value'],
                         calc_df[calc_df['var']=='Fb00']['value'])

  # Доверительный интервал в %
  ci = 95
  # Порог отсечения
  truncation = 100-ci #99.73 # Отсечение в %, -1.3 соотвествует 3*sigma
  # Значения перцентилей, соответствующие доверительному интервалу
  p5, p95 = np.percentile(v,[truncation/2, 100 - truncation/2 ])
  # p5, p95 = v165.mean() - 3 * v165.std(), v165.mean() + 3 * v165.std()
  # Откидываем результаты семплирования вне доверительного интервала
  v = v[(v>=p5) & (v<=p95)]
  print(calc_name, f' {ci}% доверительный интервал: [{p5:.2E}, {p95:.2E}] mean={v.mean():.2E}')
  # Сохраняем результаты семплировния для дальнейшего использования
  v165[calc_name] = v
```

    CDF_JNB56-57_TCLIE  95% доверительный интервал: [-0.16670075909165746, 0.22034520013193173] mean=4.41675E-03
    CDF_JNB56-57_TNFD  95% доверительный интервал: [-0.17540623348613035, 0.22058622884319773] mean=3.95249E-03
    CDF_JNB56_TCLIE  95% доверительный интервал: [-0.19555235134725657, 0.21923144106931522] mean=1.81561E-03
    CDF_JNB56_TNFD  95% доверительный интервал: [-0.2485157112009295, 0.27287508135119193] mean=2.49418E-03
    CDF_JNB57_TCLIE  95% доверительный интервал: [-0.1906302558959365, 0.2187029354253208] mean=2.14922E-03
    CDF_JNB57_TNFD  95% доверительный интервал: [-0.2373965493774472, 0.2634378041878419] mean=2.51510E-03

```python
# Выводим результаты семплирвоания
for item in v165.items():
  v = item[1]
  name = item[0]
  ax = sns.histplot(v,bins=100)
  ax.set_title(name)
  #plt.xlim(-3,3)
  #plt.yscale('log')
  plt.show()
```

```python
#import scipy.stats as st

def fit_scipy_distributions(array, bins, plot_hist = True, plot_best_fit = True, plot_all_fits = False):
  if plot_best_fit or plot_all_fits:
    assert plot_hist, "plot_hist must be True if setting plot_best_fit or plot_all_fits to True"

  # Returns un-normalised (i.e. counts) histogram
  y, x = np.histogram(array, bins=bins)

  # Some details about the histogram
  bin_width = x[1]-x[0]
  N = len(array)
  x_mid = (x + np.roll(x, -1))[:-1] / 2.0 # go from bin edges to bin middles

  # selection of available distributions
  # CHANGE THIS IF REQUIRED
  #DISTRIBUTIONS = [st.alpha,st.cauchy,st.cosine,st.laplace,st.levy,st.levy_l,st.norm]
  DISTRIBUTIONS = [stats.norm, stats.lognorm, stats.laplace, stats.cauchy]

  if plot_hist:
      fig, ax = plt.subplots()
      #plt.yscale("log")
      #h = sns.histplot(np.array(array), bins = bins, ax=ax)
      h = ax.hist(array, bins = bins)

  # loop through the distributions and store the sum of squared errors
  # so we know which one eventually will have the best fit
  sses = []
  for dist in DISTRIBUTIONS:
      name = dist.__class__.__name__[:-4]

      params = dist.fit(array)
      arg = params[:-2]
      loc = params[-2]
      scale = params[-1]

      pdf = dist.pdf(x_mid, loc=loc, scale=scale, *arg)
      pdf_scaled = pdf * bin_width * N # to go from pdf back to counts need to un-normalise the pdf

      sse = np.sum((y - pdf_scaled)**2)
      sses.append([sse, name])

      # Not strictly necessary to plot, but pretty patterns
      if plot_all_fits:
          ax.plot(x_mid, pdf_scaled, label = name)

  if plot_all_fits:
      plt.legend(loc=1)

  # CHANGE THIS IF REQUIRED
  ax.set_xlabel('x label')
  ax.set_ylabel('y label')

  # Things to return - df of SSE and distribution name, the best distribution and its parameters
  results = pd.DataFrame(sses, columns = ['SSE','distribution']).sort_values(by='SSE')
  best_name = results.iloc[0]['distribution']
  best_dist = getattr(stats, best_name)
  best_params = best_dist.fit(array)

  if plot_best_fit:
      new_x = np.linspace(x_mid[0] - (bin_width * 2), x_mid[-1] + (bin_width * 2), 1000)
      best_pdf = best_dist.pdf(new_x, *best_params[:-2], loc=best_params[-2], scale=best_params[-1])
      best_pdf_scaled = best_pdf * bin_width * N
      ax.plot(new_x, best_pdf_scaled, label = best_name)
      plt.legend(loc=1)

  if plot_hist:
      plt.show()

  return results, best_name, best_params
```

```python
# Подгонка закона распределени под выборку
for v in v165.items():
  if len(v[1])!=0:
    sses, best_name, best_params = fit_scipy_distributions(v[1],
                                                       bins = 100,
                                                       plot_best_fit = False,
                                                      plot_all_fits=True)
    print(v[0])
    print(sses)
    print(best_name, best_params)
```

## Зависимые переменные

```python
# Загрузка данных с корреляциями из Экселя
corr_df = pd.read_excel('/content/drive/MyDrive/Colab Notebooks/PKS_OC.xlsx', sheet_name='corr')
# Удалить пустые записи
corr_df.dropna(inplace=True)
# Показать несколько первых строк
corr_df.head()
```

      calc
      var
      corr_group
      corr_coef

      1
      CDF_JNB56_TNFD
      Fc00
      1.0
      0.8

      2
      CDF_JNB56_TNFD
      Fb00
      1.0
      0.8

      4
      CDF_JNB56_TCLIE
      Fc00
      2.0
      0.8

      5
      CDF_JNB56_TCLIE
      Fb00
      2.0
      0.8

      7
      CDF_JNB57_TNFD
      Fc00
      3.0
      0.8

```python
corr_groups = corr_df.groupby(by='corr_group')
for group in corr_groups:
  print(group[1]['calc'])
# Парное семплрование
#inv_cdf[]

#P_cas = inv_cdf['Pc(AS)'](random(x_unif.shape[0]))
#F_c00 = inv_cdf['Fc00'](x_unif[:,0])
#F_b00= inv_cdf['Fb00'](x_unif[:,1])

N=int(1e+6)
samples_df = pd.DataFrame()
for calc_item in inv_cdf.items():
  if 'Fc00' in calc_item[1] and 'Fc01' in calc_item:
    print(calc_item[1])
  #for var_item in calc_item[1].items():
    # Коэф. корреляции
    k=0.80327
    # Генерируем скорелированную пару нормально распределённых случайных чисел
    mvnorm = stats.multivariate_normal([0, 0], [[1, k], [k, 1]])
    x = mvnorm.rvs((N,))
    # Задаём по данным нормальное распределение
    norm = stats.norm([0],[1])
    # Генерируем N случайных скорелированных значений (непрерывное равномерное распределение [0...1])
    x_unif = norm.cdf(x)
    # Дополнительное правило, отсечения нефизичных значений
    #x_unif = x_unif[x_unif[:,0]>x_unif[:,1]]

    samples_df = samples_df.append(pd.DataFrame({
        'calc':calc_item[0],
        'var':'Fc00',
        'value':calc_item[1]['Fc00'] (x_unif)}))
    samples_df = samples_df.append(pd.DataFrame({
        'calc':calc_item[0],
        'var':'Fc01',
        'value':calc_item[1]['Fc01'] (x_unif[1])}))
```

    1    CDF_JNB56_TNFD
    2    CDF_JNB56_TNFD
    Name: calc, dtype: object
    4    CDF_JNB56_TCLIE
    5    CDF_JNB56_TCLIE
    Name: calc, dtype: object
    7    CDF_JNB57_TNFD
    8    CDF_JNB57_TNFD
    Name: calc, dtype: object
    10    CDF_JNB57_TCLIE
    11    CDF_JNB57_TCLIE
    Name: calc, dtype: object
    13    CDF_JNB56-57_TNFD
    14    CDF_JNB56-57_TNFD
    Name: calc, dtype: object
    16    CDF_JNB56-57_TCLIE
    17    CDF_JNB56-57_TCLIE
    Name: calc, dtype: object

```python
ax = sns.scatterplot(F_c00,F_b00)
ax.set_yscale('log')
ax.set_xscale('log')
```

```python
sns.histplot([F_c00, F_b00], log_scale=True, bins=50)
```

```python
v165_2 = f165(P_cas,F_c00,F_b00)
```

```python
print(np.mean(v165), np.mean(v165_2))
sns.histplot((v165, v165_2), bins=50)
plt.yscale('log')
```

```python
shape, loc, scale = stats.norm.fit(v165)
x = np.linspace(-10, 10, 100)
pdf = stats.lognorm.pdf(x, shape, loc, scale)

ax = sns.lineplot(x, pdf)

ax.set_xscale('log')
```

```python
sns.scatterplot(x_unif[:,0],x_unif[:,1])
```

## Черновик

Формла 1.7.7:
$$ {(F}_c^{00}-F_b^{00})\cdot t_{so}+{(P}_c\left(AS\right)-P_b\left(AS\right))+{(F}_c^{01}-F_b^{01})\cdot T_{01}+{(F}_c^{02}-F_b^{02})\cdot T_{02}++{(F}_c^{03}-F_b^{03})\cdot T_{03}+{(P}_c\left(OD\right)-P_b\left(OD\right))+{(F}_c^{05}-F_b^{05})\cdot t_r={P_c\left(AS\right)+(F}_c^{01}-F_b^{00})\cdot T_{01}+{(F}_c^{02}-F_b^{00})\cdot T_{02}+{(F}_c^{03}-F_b^{00})\cdot T_{03}+P_c\left(OD\right)+{(F}_c^{04}-F_b^{00})\cdot t_r+{(F}_c^{09}-F_b^{00})\cdot T_{09}+{(F}_c^{10}-F_b^{00})\cdot T_{10}+{(F}_c^{11}-F_b^{00})\cdot T_{11}+{(F}_c^{12}-F_b^{00})\cdot T_{12}+{(F}_c^{13}-F_b^{00})\cdot T_{13}+{P_c\left(LOCA-JEF\right)}^\ast $$
Выразим из неё $t_{so}$

```python
from sympy import symbols, IndexedBase, Idx, Sum, Indexed
from sympy import simplify, solve
F_c = IndexedBase('F_c')
F_b = IndexedBase('F_b')
T = IndexedBase('T')
t_so, t_r, P_cas, P_bas, P_cod, P_bod, = symbols('t_{so} t_r P_{c}(AS) P_{b}(AS) P_{c}(OD) P_{b}(OD)')
P_cloca = symbols('P_{c}(LOCA-JEF)')
i, j = symbols('i j')
a = (P_cas-P_bas)+(F_c[1]-F_b[1])*t_so+Sum((F_c[i]-F_b[i])*T[i],(i,0,3))-(P_cod + P_bod)+(F_c[5]-F_b[5]) * t_r
b = P_cas+Sum((F_c[i]-F_b[0])*T[i],(i,0,3))+P_cod+(F_c[4]-F_b[0])*t_r+Sum((F_c[i]-F_b[0])*T[i],(i,9,13))+P_cloca
```

```python
a
```

```python
b
```

```python
solve(a-b, t_so)
```

$$ t_{so} = \frac{P_b\left(AS\right)+P_b\left(OD\right)+{P_c(LOCA-JEF)}^\ast}{F_c^{00}-F_b^{00}} - \frac{(F_b^{00}-F_b^{01})\cdot T_{01}}{F_c^{00}-F_b^{00}} - \frac{(F_b^{00}-F_b^{02})\cdot T_{02}}{F_c^{00}-F_b^{00}} - \frac{(F_b^{00}-F_b^{03})\cdot T_{03}}{F_c^{00}-F_b^{00}} + \frac{(F_c^{04}-F_b^{00}-F_c^{05}+F_b^{05})\cdot t_r}{F_c^{00}-F_b^{00}} + \frac{(F_c^{09}-F_b^{00})\cdot T_{09}}{F_c^{00}-F_b^{00}} + \frac{(F_c^{10}-F_b^{00})\cdot T_{10}}{F_c^{00}-F_b^{00}} + \frac{(F_c^{11}-F_b^{00})\cdot T_{11}}{F_c^{00}-F_b^{00}} + \frac{(F_c^{11}-F_b^{00})\cdot T_{11}}{F_c^{00}-F_b^{00}}  + \frac{(F_c^{12}-F_b^{00})\cdot T_{12}}{F_c^{00}-F_b^{00}} + \frac{(F_c^{13}-F_b^{00})\cdot T_{13}}{F_c^{00}-F_b^{00}} $$

```python
def t_so():
    Fb = [uniform.rvs(size=batch_size) for i in range(6)]
    Fc = [uniform.rvs(size=batch_size) for i in range(14)]
    T = [uniform.rvs(size=batch_size) for i in range(14)]
    T = [uniform.rvs(size=batch_size) for i in range(14)]
    Tr = uniform.rvs(size=batch_size)

    t =(
        (Fb[0]-Fb[1])*T[1]-
        (Fb[0]-Fb[2])*T[2]-
        (Fb[0]-Fb[3])*T[3]+
        (Fc[4]-Fb[0]-Fc[5]-Fb[5])*Tr+
        (Fc[9]-Fb[0])*T[9]+
        (Fc[10]-Fb[0])*T[10]+
        (Fc[11]-Fb[0])*T[11]+
        (Fc[11]-Fb[0])*T[11]+ # Повторение, описка?
        (Fc[12]-Fb[0])*T[12]+
        (Fc[13]-Fb[0])*T[13]
    )/(Fc[0]-Fb[0])
    return t
```

$$ F_a^{00}\cdot t_{so}+P_a\left(AS\right)+F_a^{01} \cdot T_{01}+F_a^{02}\cdot T_{02}+F_a^{03}\cdot T_{03}+P_a\left(OD\right)+F_a^{05}\cdot t_r $$

```python
def f172():
  Fa = [uniform.rvs(size=batch_size) for i in range(6)]
  T = [uniform.rvs(size=batch_size) for i in range(4)]
  t_so = uniform.rvs(size=batch_size)
  tr = uniform.rvs(size=batch_size)
  Pas = uniform.rvs(size=batch_size)
  Pod = uniform.rvs(size=batch_size)

```


# Sensitivity indices for Sobol's $G^{*}$ function

**Leif Rune Hellevik**, [leif.r.hellevik@ntnu.no](mailto:leif.r.hellevik@ntnu.no)

# Sobol's $G^{*}$function

In our previous notebook
[g_function](interactive_g_function.ipynb),
we demonstrated how polynomial chaos
expansions (with `chaospy`) and
Monte Carlo methods may be used to approximate
the Sobol sensitivity
indices. The example was Sobol's G function and was taken
from
[Saltelli et al. 2010](https://www.sciencedirect.com/science/article/pii/S0010465509003087)
too.

In this notebook we focus on another function, which has proved to be
useful as a test function with analytical
solutions for the sensitivity
indices, namely Sobol's $G^*$ function which
is defined much in the same manner
as the [g_function](interactive_g_function.ipynb):

$$
\begin{equation}
Y=G(X) =  G(X_1, X_2,\ldots,X_k,a_1, a_2,\ldots,a_k)  =
\prod_{i=1}^{k} g_i \label{eq:1} \tag{1}
\end{equation}
$$

which is identical with

$$
\begin{equation}
g_i = \frac{(1+\alpha_i) |2 \left (X_i+ \delta_i -
I(X_i+\delta_i) \right ) -1 |^{\alpha_i}+a_i}{1+{a}_i} \label{eq:2} \tag{2}
\end{equation}
$$

All the input factors $X_i$ are assumed to be uniformly
distributed in the
interval $[0,1]$, an the coefficients $a_i$ are
assumed to be positive real
numbers $(a_i \leq 0)$, $\delta_i \in
[0,1]$, and $\alpha_i >0$. Finally, $
I(X_i+\delta_i)$ denotes the
integer value for $X_i+\delta_i$. Note that for for
$\alpha_i=1$ and
$\delta_i=0$ $g^*$ reduces to $g$ in the
[g_function](interactive_g_function.ipynb) notebook. The $\alpha_i$
and
$\delta_i$ are curvature and shift parameters, respectively.

The number of
factors *k* can be varied as the reader pleases, but the
minimum number to
produce a meaningful inference is set at three.

Run the first cell to
initialise plotting and printing modules for later use (and system
settings)

```
# ipython magic
%matplotlib notebook
%load_ext autoreload
%autoreload 2
import os, sys, inspect
# Use this if you want to include modules from a subfolder
cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],"python_source")))
if cmd_subfolder not in sys.path:
     sys.path.insert(0, cmd_subfolder)

%run python_source/matplotlib_header

import matplotlib.pyplot as plt
from present_output import print_vectors_relerror, print_3vectors_relerror
```

As you will be able to explore below, the sensitivity $S_i$ of $G$ in
([1](#eq:1)) with respect to a specific input factor $X_i$, will depend
on the
value of the corresponding coefficient $a_i$; small values of
$a_i$ (e.g.
$a_i=0$) will yield a high corresponding $S_i$, meaning
that $X_i$ is an
important/influential variable on the variance or
uncertainty of $G$.

We have
implemented Sobol's  $G^*$ function in ([1](#eq:1)) and  ([2](#eq:2)) in the
code snippet below:

```
# model function
import numpy as np
from numba import jit

@jit
def g(Xj,aj,alphaj,deltaj):
    return ((1+alphaj)*np.abs(2*(Xj+deltaj-(Xj+deltaj).astype(int))-1)**alphaj+aj)/(1+aj)

@jit
def G(X,a,alpha,d):
    G_vector=np.ones(X.shape[0])

    for j, aj in enumerate(a):
        np.multiply(G_vector,g(X[:,j],aj,alpha[j],d[j]),G_vector)
    return G_vector
```

The sensitivity indices $S_i$ and $S_{Ti}$ for $Y=G(X)$ in
eq. ([1](#eq:1)) may
be derived as outlined in [Saltelli 2010](https://www.sciencedirect.com/science/article/pii/S0010465509003087)).
The conditional variance $V_i$ may be found to be:

$$
\begin{equation}
V_i \left ( G^*(X_i,a_i,\alpha_i) \right) =
\frac{\alpha_i^2}{(1+2\alpha_i)(1+a_i)^2}
\label{eq:3} \tag{3}
\end{equation}
$$

while the $V_{T_I}$ and the variance $V$ are given by the same
expressions as
for the [g_function](interactive_g_function.ipynb).

$$
\begin{equation}
V_{T_i} = V_i \; \prod_{j\neq i} (1+V_j) \qquad \text{and}
\qquad V = \prod_{i=1}^k (1+V_i) -1
\label{eq:4} \tag{4}
\end{equation}
$$

Consequently the first order sensitivity indices $S_i$ of $Y=G^*(X)$, are given by

$$
\begin{equation}
S_i=\frac{V_i}{V} \qquad \text{and} \qquad
S_{T_i}=\frac{V_{T_i}}{V}
\label{eq:5} \tag{5}
\end{equation}
$$

```
# Analytical computations
f, ax = plt.subplots(1,1)
f.suptitle('G* function with variable coefficients')

# import modules
import numpy as np

def Vi(ai,alphai):
    return alphai**2/((1+2*alphai)*(1+ai)**2)

def V(a_prms,alpha):
    D=1
    for ai,alphai in zip(a_prms,alpha):
        D*=(1+Vi(ai,alphai))
    return D-1

def S_i(a,alpha):
    S_i=np.zeros_like(a)
    for i, (ai,alphai) in enumerate(zip(a,alpha)):
        S_i[i]=Vi(ai,alphai)/V(a,alpha)
    return S_i

def S_T(a,alpha):
    # to be completed
    S_T=np.zeros_like(a)
    Vtot=V(a,alpha)
    for i, (ai,alphai) in enumerate(zip(a,alpha)):
        S_T[i]=(Vtot+1)/(Vi(ai,alphai)+1)*Vi(ai,alphai)/Vtot
    return S_T

def update_Sobol(**kwargs):
    import re
    r = re.compile("([a-zA-Z]+)([0-9]+)")
    ax.clear()
    prm_cat=int(len(kwargs)/k)
    prms=np.zeros((prm_cat,k))

    for key, value in kwargs.items(): #find indx and value for a_prms
        pre,post=r.match(key).groups()
        cat_idx=strings.index(pre)
        prms[cat_idx,int(post)]=value

    Si[:]=S_i(prms[0,:],prms[1,:])
    ST[:]=S_T(prms[0,:],prms[1,:])
    width=0.4
    x_tick_list=np.arange(len(prms[0,:]))+1
    ax.set_xticks(x_tick_list+width/2)
    x_labels=['x'+str(i) for i in np.arange(len(prms[0,:]))]
    ax.set_xticklabels(x_labels)
    ax.set_ylim(0,1)

    ax.bar(x_tick_list,Si,width,color='blue')
    ax.bar(x_tick_list+width,ST,width,color='red')
    ax.legend(['First order indices','Total indices'])

k=4 #number of prms
strings=['a','alpha','delta']
a_lbls=[strings[0]+str(i) for i in np.arange(k)]
alpha_lbls=[strings[1]+str(i) for i in np.arange(k)]
delta_lbls=[strings[2]+str(i) for i in np.arange(k)]
Si=np.zeros(k)
ST=np.zeros(k)
a_prms=np.zeros(k)
alpha=np.zeros_like(a_prms)
delta=np.zeros_like(a_prms)

import ipywidgets as widgets
my_sliders=[]
for i in range(k):
    my_sliders.append(widgets.FloatSlider(min=0, max=15, value=6.52, description=a_lbls[i]))
    my_sliders.append(widgets.FloatSlider(min=0, max=15, value=1.0, description=alpha_lbls[i]))
    my_sliders.append(widgets.FloatSlider(min=0, max=1.0, value=0.5, description=delta_lbls[i]))

slider_dict = {slider.description:slider for slider in my_sliders}
ui_left = widgets.VBox(my_sliders[0::3])
ui_mid  = widgets.VBox(my_sliders[1::3])
ui_right = widgets.VBox(my_sliders[2::3])
ui=widgets.HBox([ui_left,ui_mid,ui_right])

out=widgets.interactive_output(update_Sobol, slider_dict)

display(ui,out)
```

Use the sliders to see how the sensitivities vary with the values of $a_i$, and
reflect on the effect.

Note from the analytical expressions $V_i$ and
$V_{T_i}$ (derived in
[[saltelli2010]](#saltelli2010)) in the sensitivity
indices do not depend on the
shift paramters $\delta_i$.

For $\alpha_i<1$ the
$G^*$ function is concave, while the function is
convex for $\alpha_i>1$, which
is the reason for calling $\alpha$ a
shape parameter.

## Approximation of the sensitivity indices for Sobol's $G^*$ function with spectral expansions

In this
section we show the spectral expansion module
[chaospy](https://github.com/jonathf/chaospy) may be used to compute the Sobol
indices for Sobol's $G^*$ function.  A more in depth treatment of
`chaospy` and
its usage is provided in the separate notebook [A
practical introduction to
polynomial chaos with the chaospy package](introduction_gpc.ipynb). Furthermore,
you may find our previous "A
Guide to Uncertainty Quantification and Sensitivity
Analysis for
Cardiovascular Applications" [Eck et al. 2015](https://onlinelibrary.wiley.com/doi/full/10.1002/cnm.2755) as a
useful
introduction to how polynomial chaos expansions may be used for
UQ&S. We
are therefore focusing on the application of the spectral
expansions and how
they agree with the analytical solutions for the
indices, rather than presenting
the spectral expansion theory.

```
# chaospy G-function with sliders
import chaospy as cp

if not 'jpdf' in globals():
    jpdf = cp.Iid(cp.Uniform(),k) #the joint pdf
    print('Create the joint pdf')

def update_chaospy_G(**kwargs):
    NS=kwargs['NS']
    del kwargs['NS']
    polynomial_order=kwargs['polynomial_order']
    del kwargs['polynomial_order']

    prm_cat=int(len(kwargs)/k)
    prms=np.zeros((prm_cat,k))

    import re
    r = re.compile("([a-zA-Z]+)([0-9]+)")

    for key, value in kwargs.items(): #find indx and value for a_prms
        pre,post=r.match(key).groups()
        cat_idx=strings.index(pre)
        prms[cat_idx,int(post)]=value

    X=jpdf.sample(NS)
    print('Number of samples: ',NS)

    G_sample=G(X.transpose(),prms[0,:],prms[1,:],prms[2,:])

    poly = cp.orth_ttr(polynomial_order, jpdf)
    approx = cp.fit_regression(poly, X, G_sample)

    exp_pc = cp.E(approx, jpdf)
    std_pc = cp.Std(approx, jpdf)
    print("Statistics polynomial chaos\n")
    print('\n        E(Y)  |  std(Y) \n')
    print('pc  : {:2.5f} | {:2.5f}'.format(float(exp_pc), std_pc))
    S_pc = cp.Sens_m(approx, jpdf) #Si from chaospy
    S_tpc = cp.Sens_t(approx, jpdf) #Total effect sensitivity index from chaospy

    row_labels= ['S_'+str(idx) for idx in range(len(a_prms))]
    col_labels=['Chaospy','Analytical','Error (%)']

    print("\nFirst Order Indices")
    print_vectors_relerror(S_pc,Si,col_labels,row_labels,[3,3,0])

    print("\n\nTotal Effect Indices")
    row_labels= ['St_'+str(idx) for idx in range(k)]
    print_vectors_relerror(S_tpc,ST,col_labels,row_labels,[3,3,0])

if (len(my_sliders)==len(a_prms)*3):   #add sliders if not added before
    my_sliders.append(widgets.IntSlider(min=500,max=5000,step=250,value=500,description='NS')) #add slider for samples
    my_sliders.append(widgets.IntSlider(description='polynomial_order', min=1,max=6,value=4)) # add slider for polynomial order

    slider_dict = {slider.description:slider for slider in my_sliders} #add the sliders in the dictionary

    ui_left = widgets.VBox(my_sliders[0::3])
    ui_mid  = widgets.VBox(my_sliders[1::3])
    ui_right = widgets.VBox(my_sliders[2::3])
    ui=widgets.HBox([ui_left,ui_mid,ui_right])

out=widgets.interactive_output(update_chaospy_G, slider_dict)
display(ui,out)

# end chaospy G-function with sliders
```

You may check whether approximated sensitivity indices are independent of the
shift paramters $\delta_i$, as they should according according to the analytical
expressions for $V_i$ in eq. ([3](#eq:3)).

## Comparison of MC and PC approximation of the sensitivity indices

In this section we demonstrate how
Monte Carlo simulations and
polynomial chaos expansions may be used to
estimate the Sobol
indices and compare their estimates with the analytical
solutions.

Readers unfamiliar with how to use python (notebooks) for Monte
Carlo
simulations and polynomial chaos expansions are referred to our
previous
[A brief introduction to UQ and SA with the Monte Carlo
method](https://github.com/lrhgit/uqsa_tutorials/blob/master/monte_carlo.ipynb).
and  [A practical introduction to polynomial chaos with the
chaospy
package](https://github.com/lrhgit/uqsa_tutorials/blob/master/introduction_gpc.ipynb)

```
# mc and pc comparison for Gstar-function with sliders

import monte_carlo as mc

if not 'jpdf' in globals():
    jpdf = cp.Iid(cp.Uniform(),k) #the joint pdf
    print('Create the joint pdf')

def update_Gstar(**kwargs):
    NsPC=kwargs['NsPC']
    del kwargs['NsPC']
    NsMC=kwargs['NsMC']
    del kwargs['NsMC']

    polynomial_order=kwargs['polynomial_order']
    del kwargs['polynomial_order']

    prm_cat=int(len(kwargs)/k)
    prms=np.zeros((prm_cat,k))

    import re
    r = re.compile("([a-zA-Z]+)([0-9]+)")

    for key, value in kwargs.items(): #find indx and value for a_prms
        pre,post=r.match(key).groups()
        cat_idx=strings.index(pre)
        prms[cat_idx,int(post)]=value

    ## Update the analytical indices
    Si[:]=S_i(prms[0,:],prms[1,:])
    ST[:]=S_T(prms[0,:],prms[1,:])

    ## Monte Carlo update
    print('Number of samples for Monte Carlo: ', NsMC)
    X_mc=jpdf.sample(NsMC)
    A, B, C = mc.generate_sample_matrices_mc(NsMC, k, jpdf, sample_method='R') #A, B, C already transposed
    G_A_sample = G(A,prms[0,:],prms[1,:],prms[2,:])
    G_B_sample = G(B,prms[0,:],prms[1,:],prms[2,:])
    G_C_sample_list = np.array([G(C_i,prms[0,:],prms[1,:],prms[2,:]) for C_i in C]).T

    exp_mc = np.mean(G_A_sample)
    std_mc = np.std(G_A_sample)
    print("Statistics Monte Carlo\n")
    print('\n        E(Y)  |  std(Y) \n')
    print('mc  : {:2.5f} | {:2.5f}'.format(float(exp_mc), std_mc))

    S_mc, S_tmc = mc.calculate_sensitivity_indices_mc(G_A_sample, G_B_sample, G_C_sample_list)

    ## update PC estimates
    Xpc=jpdf.sample(NsPC)
    print('Number of samples: ',NsPC)

    G_sample=G(Xpc.transpose(),prms[0,:],prms[1,:],prms[2,:])

    poly = cp.orth_ttr(polynomial_order, jpdf)
    approx = cp.fit_regression(poly, Xpc, G_sample)

    exp_pc = cp.E(approx, jpdf)
    std_pc = cp.Std(approx, jpdf)
    print("Statistics polynomial chaos\n")
    print('\n        E(Y)  |  std(Y) \n')
    print('pc  : {:2.5f} | {:2.5f}'.format(float(exp_pc), std_pc))
    S_pc = cp.Sens_m(approx, jpdf) #Si from chaospy
    S_tpc = cp.Sens_t(approx, jpdf) #Total effect sensitivity index from chaospy

    row_labels= ['S_'+str(idx) for idx in range(len(a_prms))]
    #col_labels=['Chaospy','Analytical','Error (%)']
    col_labels=['Monte Carlo','Err (%)','PolyChaos','Err (%)']

    print("\nFirst Order Indices")
#    print_vectors_relerror(S_pc,Si,col_labels,row_labels,[3,3,0])
    print_3vectors_relerror(S_mc,S_pc, Si, col_labels, row_labels, [3,0,3,0])

    print("\n\nTotal Effect Indices")
    row_labels= ['St_'+str(idx) for idx in range(k)]
#    print_vectors_relerror(S_tpc,ST,col_labels,row_labels,[3,3,0])
    print_3vectors_relerror(S_tmc,S_tpc, ST, col_labels, row_labels, [3,0,3,0])

## Set up the sliders
cmp_sliders=[]
for i in range(k):
    cmp_sliders.append(widgets.FloatSlider(min=0, max=15, value=6.52, description=a_lbls[i]))
    cmp_sliders.append(widgets.FloatSlider(min=0, max=15, value=1.0, description=alpha_lbls[i]))
    cmp_sliders.append(widgets.FloatSlider(min=0, max=1.0, value=0.5, description=delta_lbls[i]))

cmp_sliders.append(widgets.IntSlider(min=500,max=5000,step=250,value=500,description='NsPC')) #add slider for samples
cmp_sliders.append(widgets.IntSlider(min=500,max=50000,step=250,value=500,description='NsMC')) #add slider for samples
cmp_sliders.append(widgets.IntSlider(description='polynomial_order', min=1,max=6,value=4)) # add slider for polynomial order

slider_dict = {slider.description:slider for slider in cmp_sliders} #add the sliders in the dictionary

ui_left = widgets.VBox(cmp_sliders[0::3])
ui_mid  = widgets.VBox(cmp_sliders[1::3])
ui_right = widgets.VBox(cmp_sliders[2::3])
ui=widgets.HBox([ui_left,ui_mid,ui_right])

out=widgets.interactive_output(update_Gstar, slider_dict)
display(ui,out)
```


# Ejercicio 3 - Simulación de distribuciones condicionadas

### Julian Ferres - Nro.Padrón 101483

## Enunciado:

Sea $X$ \~ $N(0,1)$ truncada al intervalo $[-1,1]$

Imagine $m(x) = E[Y | X=x]$ como:

   \begin{equation}
     m(x) := \left\{
 \begin{array}{ll}
      \frac{(x + 2)^2}{2} & \mathrm{si\ } si -1\leq x<-0.5 \\
    \frac{x}{2}+0.875     & \mathrm{si\ } -0.5 \leq x \leq 0\\
    -5(x-0.2)^2 +1.075 & \mathrm{si\ } 0 < x \leq 0.5 \\
    x + 0.125 & \mathrm{si\ } 0.5 \leq x < 1
           \end{array}
         \right.
    \end{equation}

Dado un $x$, la distribución condicional de $Y - m(x)$ es $N(0, \sigma ^2(x))$,

con $\sigma(x)=0.2-0.1 * \cos(2x)$

- Se pide simular $200$ puntos $(X,Y)$, y graficarlos en un plano. Además, vamos a necesitar
Los $200$ pares ordenados en cuestión, para hacer análisis posteriores

- Reconstruir $m(x)$ con los $200$ puntos, para eso:

Realizar una partición de $[-1,1]$ en intervalos de longitud $h$ y en cada intervalo encontrar el polinomio $f$ de grado $M$ que minimice el error cuadratico medio $$ \frac{1}{n} \sum |f(X_i)-Y_i|^2$$

Usar:

1. $h = 0.5$ , $M=1$
2. $h = 0.1$ , $M=1$
3. $h = 0.25$ , $M=2$
4. $h = 0.5$ , $M=2$

## Solución:

#### Importo todas las librerias e inicializo funciones

```python
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from math import cos, pi
from scipy.stats import truncnorm
```

```python
m1 = lambda x: (x+2)**2/2
m2 = lambda x: x/2 + 0.875
m3 = lambda x: -5*(x-0.2)**2 + 1.075
m4 = lambda x: x + 0.125
```

```python
def m(x):
    if -1 <= x < -0.5:
        return m1(x)
    if -0.5 <= x < 0:
        return m2(x)
    if 0 <= x < 0.5:
        return m3(x)
    if 0.5 <= x < 1:
        return m4(x)

m = np.vectorize(m)
```

```python
x_0 = np.linspace(-1,1,1000) #Me genero 1000 valores entre -1 y 1 para graficar m(x) 'suave'
y_0 = m(x_0)
```

#### Normal truncada

```python
a , b = -1 , 1 #Limites de la normal truncada
```

```python
x1 = np.linspace(truncnorm.ppf(0.01, a, b),
                truncnorm.ppf(0.99, a, b), 200) #Genero 200 cuantiles de la normal truncada

plt.plot(x1, truncnorm.pdf(x1, a, b),
        'r-', lw=3, alpha=0.75, label='Normal truncada')
plt.title("Density Plot de X",fontsize='15')
plt.legend(loc='best', frameon= True)
plt.grid()
```

```python
x1 = truncnorm.rvs(a, b, size=200)
#Me genero la muestra de distribucion X
```

```python
sigma = np.vectorize(lambda x : 0.2 - 0.1 * cos(2*pi*x))
normal = np.vectorize(np.random.normal)

y1 = normal( m(x1),sigma(x1))
```

```python
fig, ax = plt.subplots(figsize=(11,7))
plt.plot(x_0, y_0, 'g-', linewidth = 5, label = 'Función m(x)=E[Y|X=x]')
plt.legend(loc='best', frameon= True)

plt.plot(x1, y1, 'ro' ,markersize= 5, alpha = 0.5 ,label = 'Dispersion (X,Y)')
plt.legend(loc='best', frameon= True)

plt.title("Scatter Plot de (X,Y) y Line plot de m(x)", fontsize='15')
plt.xlabel('X')
plt.ylabel('Y')

plt.show()
```

#### La muestra de los $200$ pares con distribución $(X,Y)$ se encuentra en la variable output

## Reconstruyo la regresión

#### Con h=0.5 y M=1

```python
partition = [[],[],[],[]]
for i in range(200):
    partition[int(2*(x1[i]+1))].append(i)
```

```python
polinomio_a_trozos = []

cuadrado_de_los_errores1 = 0

for i in range(4):
    x_aux , y_aux = [x1[j] for j in partition[i]],[y1[j] for j in partition[i]]
    z = np.polyfit(x_aux,y_aux,1)
    polinomio_a_trozos.append(np.poly1d(z))

    #sumo los errores para cada trozo del polinomio
    for j in range(len(x_aux)):
        cuadrado_de_los_errores1 += (polinomio_a_trozos[i](x_aux[j])-y_aux[j])**2
```

```python
xp=[]
xp.append(np.linspace(-1, -0.5, 200))
xp.append(np.linspace(-0.5,0, 200))
xp.append(np.linspace(0, 0.5, 200))
xp.append(np.linspace(0.5,1, 200))
```

```python
fig, ax = plt.subplots(figsize=(11,7))

plt.plot(x1, y1, 'ro', linewidth = 5, alpha = 0.5 ,label = 'Dispersion X,Y')
plt.legend(loc='best', frameon= True)

for i in range(4):
    plt.plot(xp[i], polinomio_a_trozos[i](xp[i]) ,'b-', linewidth = 5 )

plt.plot(x_0, y_0, 'g-', linewidth = 5, alpha = 0.75 ,label = 'Función m(x)=E[Y|X=x]')

plt.legend(loc='best', frameon= True)
plt.title("Estimación m(x) con h=0.5 y M=1", fontsize='15')
plt.xlabel('X')
plt.ylabel('Y')

plt.show()
```

La estimación parece ajustarse bien a la función de regresion, no obstante, el error cuadrático medio es alto ya que no esta Overfitteando
a la muestra.

#### Estimación del error cuadrático medio

```python
(cuadrado_de_los_errores1 / 200)**0.5
```

    0.21568051286590767

#### Con h=0.1 y M=1

```python
partition = [[] for i in range(20)]
for i in range(200):
    partition[int(10*(x1[i]+1))].append(i)
```

```python
polinomio_a_trozos = []
cuadrado_de_los_errores2 = 0

for i in range(20):
    x_aux , y_aux = [x1[j] for j in partition[i]],[y1[j] for j in partition[i]]
    z = np.polyfit(x_aux,y_aux,1)
    polinomio_a_trozos.append(np.poly1d(z))

    #sumo los errores para cada trozo del polinomio
    for j in range(len(x_aux)):
        cuadrado_de_los_errores2 += (polinomio_a_trozos[i](x_aux[j])-y_aux[j])**2
```

```python
xp=[]

for i in range(20):
    xp.append(np.linspace(-1+i*(1/10), -0.9+i*(1/10), 200))
```

```python
fig, ax = plt.subplots(figsize=(11,7))

plt.plot(x1, y1, 'ro', linewidth = 5, alpha = 0.5 ,label = 'Dispersion X,Y')
plt.legend(loc='best', frameon= True)

for i in range(20):
    plt.plot(xp[i], polinomio_a_trozos[i](xp[i]) ,'b-', linewidth = 5 )

plt.plot(x_0, y_0, 'g-', linewidth = 5, alpha = 0.75,label = 'Función m(x)=E[Y|X=x]')
plt.legend(loc='best', frameon= True)
plt.title("Estimación m(x) con h=0.1 y M=1", fontsize='15')
plt.xlabel('X')
plt.ylabel('Y')
plt.show()
```

Se puede observar un claro caso de Overfitting, donde el error cuadrático medio es medianamente bajo, pero no estima correctamente la regresión.

#### Estimación del error cuadrático medio

```python
(cuadrado_de_los_errores2 / 200)**0.5
```

    0.196030417748454

#### Con h=0.25 y M=2

```python
partition = [[] for i in range(8)]
for i in range(200):
    partition[int(4*(x1[i]+1))].append(i)
```

```python
polinomio_a_trozos = []
cuadrado_de_los_errores3 = 0

for i in range(8):
    x_aux , y_aux = [x1[j] for j in partition[i]],[y1[j] for j in partition[i]]
    z = np.polyfit(x_aux,y_aux,2)
    polinomio_a_trozos.append(np.poly1d(z))

    #sumo los errores para cada trozo del polinomio
    for j in range(len(x_aux)):
        cuadrado_de_los_errores3 += (polinomio_a_trozos[i](x_aux[j])-y_aux[j])**2
```

```python
xp=[]

for i in range(8):
    xp.append(np.linspace(-1+i*(1/4), -1+(i+1)*(1/4), 200))
```

```python
fig, ax = plt.subplots(figsize=(11,7))

plt.plot(x1, y1, 'ro', linewidth = 5,alpha = 0.5, label ='Dispersion X,Y')
plt.legend(loc='best', frameon= True)

for i in range(8):
    plt.plot(xp[i], polinomio_a_trozos[i](xp[i]) ,'b-', linewidth = 5 )

plt.plot(x_0, y_0, 'g-', linewidth = 5,alpha = 0.75 ,label = 'Función m(x)=E[Y|X=x]')
plt.legend(loc='best', frameon= True)
plt.title("Estimación m(x) con h=0.25 y M=2", fontsize='15')
plt.xlabel('X')
plt.ylabel('Y')
plt.show()
```

Se puede observar un claro caso de Overfitting, donde el error cuadrático medio es medianamente bajo, pero no estima correctamente la regresión.

#### Estimación del error cuadrático medio

```python
(cuadrado_de_los_errores3 / 200)**0.5
```

    0.20118730938942803

#### Con h=0.5 y M=2

```python
partition = [[] for i in range(4)]
for i in range(200):
    partition[int(2*(x1[i]+1))].append(i)
```

```python
polinomio_a_trozos = []
cuadrado_de_los_errores4 = 0

for i in range(4):
    x_aux , y_aux = [x1[j] for j in partition[i]],[y1[j] for j in partition[i]]
    z = np.polyfit(x_aux,y_aux,2)
    polinomio_a_trozos.append(np.poly1d(z))

    #sumo los errores para cada trozo del polinomio
    for j in range(len(x_aux)):
        cuadrado_de_los_errores4 += (polinomio_a_trozos[i](x_aux[j])-y_aux[j])**2
```

```python
xp=[]
for i in range(4):
    xp.append(np.linspace(-1+i*(1/2), -1+(i+1)*(1/2), 200))
```

```python
fig, ax = plt.subplots(figsize=(11,7))

plt.plot(x1, y1, 'ro', linewidth = 5,alpha = 0.5, label = 'Dispersion X,Y')
plt.legend(loc='best', frameon= True)

for i in range(4):
    plt.plot(xp[i], polinomio_a_trozos[i](xp[i]) ,'b-', linewidth = 5)

plt.plot(x_0, y_0, 'g-', linewidth = 5,alpha = 0.75 ,label = 'Función m(x)=E[Y|X=x]')
plt.legend(loc='best', frameon= True)
plt.title("Estimación m(x) con h=0.5 y M=2", fontsize='15')
plt.xlabel('X')
plt.ylabel('Y')

plt.show()
```

Se ve que el ECM es ligeramente superior a los casos con Overfitting, se ve que predice la regresión de forma bastante acertada.

#### Estimación del error cuadrático medio

```python
(cuadrado_de_los_errores4 / 200)**0.5
```

    0.21280621089954807

```python
(cuadrado_de_los_errores1 / 200)**0.5 , (cuadrado_de_los_errores2 / 200)**0.5 , (cuadrado_de_los_errores3 / 200)**0.5 , (cuadrado_de_los_errores4 / 200)**0.5
```

    (0.21568051286590767,
     0.196030417748454,
     0.20118730938942803,
     0.21280621089954807)

```python

```

Link al Repo de GitHub: https://github.com/julianferres/Aprendizaje-Estadistico.git

Ejercicios/06-Reconstruir Regresion (Penultima Clase)/.ipynb_checkpoints/EstimacionRegresion-checkpoint.ipynb


```python
from IPython.display import Image
Image('../../Python_probability_statistics_machine_learning_2E.png',width=200)
```

We saw with maximum likelihood estimation how we could use the
principle of
maximum likelihood to derive a formula of the data that
would estimate the
underlying parameters (say, $\theta$). Under that
method, the parameter was
fixed, but unknown. If we change our
perspective slightly and consider the
underlying parameter as a random
variable in its own right, this leads to
additional flexibility in
estimation.  This method is the simplest of the family
of Bayesian
statistical methods and is most closely related to maximum
likelihood
estimation. It is very popular in communications and signal
processing
and is the backbone of many important algorithms in those areas.
Given that the parameter $\theta$ is also a random variable, it has a
joint
distribution with the other random variables, say,
$f(x,\theta)$.  Bayes'
theorem gives the following:

$$
\mathbb{P}(\theta|x) =
\frac{\mathbb{P}(x|\theta)\mathbb{P}(\theta)}{\mathbb{P}(x)}
$$

 The $\mathbb{P}(x|\theta)$ term is the usual likelihood term we have
seen
before. The term in the denominator is *prior* probability of the data $x$
and
it explicitly makes a very powerful claim: even before collecting or
processing
any data,  we know what the probability of that data is.  The
$\mathbb{P}(\theta)$ is the prior probability of the
parameter. In other words,
regardless of the data that is collected, this is
the probability of the
parameter itself.

In a particular application, whether or not you feel
justified making these
claims is something that you have to reconcile for
yourself and the problem at
hand.  There are many persuasive  philosophical
arguments one way or the other,
but the main thing to keep in mind when applying
any method is whether or not
the assumptions are reasonable for the problem at
hand.

However, for now, let's just assume that we somehow have
$\mathbb{P}(\theta)$
and the next step is the maximizing of this expression over
the $\theta$.
Whatever results from that maximization is the maximum
a-posteriori (MAP)
estimator for $\theta$. Because the maximization takes place
with respect to
$\theta$ and not $x$, we can ignore the $\mathbb{P}(x)$ part. To
make things
concrete, let us return to our original coin flipping problem.  From
our
earlier analysis, we know that the likelihood function for this problem is
the following:

$$
\ell(\theta) := \theta^k (1-\theta)^{ (n-k) }
$$

 where the probability of the coin coming up heads is $\theta$. The
next step is
the prior probability, $\mathbb{P}(\theta)$. For this example, we
will choose
the $\beta(6,6)$ distribution (shown in the top left panel of
[Figure](#fig:MAP_001)). The $\beta$ family of distributions is a gold mine
because it allows for a wide variety of distributions using few input
parameters. Now that we have all the ingredients, we turn to maximizing the
posterior function, $\mathbb{P}(\theta|x)$. Because the logarithm is convex, we
can use it to make the maximization process easier by converting the product to
a sum without changing the extrema that we are looking for. Thus, we prefer the
to work with the logarithm of $\mathbb{P}(\theta|x)$ as in the following.

$$
\mathcal{L} := \log \mathbb{P}(\theta|x) =  \log \ell(\theta) +
\log\mathbb{P}(\theta) - \log\mathbb{P}(x)
$$

 This is tedious to do by hand and therefore an excellent job
for Sympy.

```python
import numpy as np
import sympy
from sympy import stats as st
from sympy.abc import p,k,n
# setup objective function using sympy.log
obj=sympy.expand_log(sympy.log(p**k*(1-p)**(n-k)*
                         st.density(st.Beta('p',6,6))(p)))
# use calculus to maximize objective
sol=sympy.solve(sympy.simplify(sympy.diff(obj,p)),p)[0]
sol
```

$\displaystyle \frac{k + 5}{n + 10}$

which means that our MAP estimator of $\theta$ is the following:

$$
\hat{\theta}_{MAP} = \frac{k+5}{n+10}
$$

 where $k$ is the number of heads in the sample. This is obviously a
biased
estimator of $\theta$,

$$
\mathbb{E}(\hat{\theta}_{MAP}) = \frac{5+n \theta}{10 +n} \neq \theta
$$

 But is this bias *bad*? Why would anyone want a biased estimator?
Remember that
we constructed this entire estimator using the idea of the prior
probability of
$\mathbb{P}(\theta)$ which *favors* (biases!) the estimate
according to the
prior.  For example, if $\theta=1/2$, the MAP estimator
evaluates to
$\hat{\theta}_{MAP}=1/2$. No bias there! This is because the peak
of the prior
probability is at $\theta=1/2$.

To compute the corresponding variance for this
estimator, we need this
intermediate result,

$$
\mathbb{E}(\hat{\theta}_{MAP}^2) =\frac{25 +10 n \theta + n \theta((n-1)
p+1)}{(10+n)^2}
$$

 which gives the following variance,

$$
\mathbb{V}(\hat{\theta}_{MAP}) = \frac{n (1-\theta) \theta}{(n+10)^2}
$$

Let's pause and compare this to our previous maximum likelihood (ML) estimator
shown below:

$$
\hat{\theta}_{ML} = \frac{1}{n} \sum_{i=1}^n X_i =  \frac{k}{n}
$$

 As we discussed before, the ML-estimator is unbiased with the
following
variance.

$$
\mathbb{V}(\hat{\theta}_{ML}) = \frac{\theta(1-\theta)}{n}
$$

 How does this variance compare to that of the MAP? The ratio of the
two is the
following:

$$
\frac{\mathbb{V}(\hat{\theta}_{MAP})}{\mathbb{V}(\hat{\theta}_{ML})}=\frac{n^2}{(n+10)^2}
$$

  This ratio shows that the variance for the MAP-estimator is smaller
than that
of the the ML-estimator. This is payoff for having a biased
MAP-estimator --- it
requires fewer samples to estimate if the underlying
parameter is consistent
with the prior probability. If not, then it will take
more samples to pull the
estimator away from the bias. In the limit as $n
\rightarrow \infty$, the ratio
goes to one.  This means that the benefit of the
reduced variance vanishes with
enough samples.

The above discussion admits a level of arbitrariness via the
prior
distribution. We don't have to choose just one prior, however. The
following shows how we can use the previous posterior distribution as the
prior
for the next posterior distribution,

$$
\mathbb{P}(\theta|x_{k+1}) =
\frac{\mathbb{P}(x_{k+1}|\theta)\mathbb{P}(\theta|x_k)}{\mathbb{P}(x_{k+1})}
$$

 This is a very different strategy because we are using every data
sample $x_k$
as a parameter for the posterior distribution instead of lumping
all the samples
together in a summation (this is where we got the $k$
term in the prior case).
This case is much harder to analyze because now
every incremental posterior
distribution is itself a random function because of
the injection of the $x$
random variable. On the other hand, this is more in
line with more general
Bayesian methods because it is clear that the output of
this estimation process
is a posterior distribution function, not just a single
parameter estimate.
[Figure](#fig:MAP_001) illustrates this method. The graph
in the top row, far
left shows the prior probability ($\beta(6,6)$) and the dot
on the top shows the
most recent MAP-estimate for $\theta$. Thus, before we
obtain any data, the peak
of the prior probability  is the estimate. The next
graph to right shows the
effect of $x_0=0$ on the incremental prior
probability.  Note that the estimate
has barely moved to the left. This is
because the influence of the data has not
caused the prior probability to drift
away from the original
$\beta(6,6)$-distribution.  The first two rows of the
figure all have $x_k=0$
just to illustrate how far left the original prior
probability can be moved by
those data. The dots on the tops of the sub-graphs
show how the MAP estimate
changes frame-by-frame as more data is incorporated.
The remaining graphs,
proceeding top-down, left-to-right show the incremental
change in the prior
probability for $x_k=1$.  Again, this shows how far to the right
the estimate
can be pulled from where it started. For this example, there
are an equal number
of $x_k=0$ and $x_k=1$ data, which corresponds
to $\theta=1/2$.

The prior probability is the $\beta(6,6)$
distribution shown in the top left panel. The dots near the peaks of each of the
subgraphs indicate the MAP estimate at that frame

**Programming Tip.**
The following is a quick paraphrase of how [Figure](#fig:MAP_001) was
constructed. The first step is to recursively create the
posteriors from the
data. Note the example data is sorted
to make the progression easy to see as a
sequence.

```python
from sympy.abc import p,x
from sympy.stats import density, Beta, Bernoulli
prior = density(Beta('p',6,6))(p)
likelihood=density(Bernoulli('x',p))(x)
data = (0,0,0,0,0,0,0,1,1,1,1,1,1,1,1)
posteriors = [prior]
for i in data:
    posteriors.append(posteriors[-1]*likelihood.subs(x,i))
```

With the posteriors in hand, the next step
is to compute the peak values at
each frame using the
`fminbound` function from Scipy's  `optimize` module.

```python
from matplotlib.pylab import subplots
from scipy.optimize import fminbound

fig,ax = subplots(4,4,sharex=True,
                  subplot_kw={
                              'yticks':[],
                              'xticks':[0,0.5,1],
                              'xticklabels':[0,0.5,1]
                  }
                 )
fig.set_size_inches((8,8))

pvals = np.linspace(0,1,100)
mxvals = []
for i,j in zip(ax.flat,posteriors):
    i.plot(pvals,sympy.lambdify(p,j)(pvals),color='k')
    mxval = fminbound(sympy.lambdify(p,-j),0,1)
    mxvals.append(mxval)
    h = i.axis()[-1]
    i.axis(ymax=h*1.3)
    i.plot(mxvals[-1],h*1.2,'ok')
    i.plot(mxvals[:-1],[h*1.2]*len(mxvals[:-1]),'o')
```

The [Figure](#fig:MAP_002) is the same as [Figure](#fig:MAP_001) except that
the
initial prior probability is the $\beta(1.3,1.3)$-distribution, which has a
wider lobe that the $\beta(6,6)$-distribution. As shown in the figure, this
prior has the ability to be swayed more violently one way or the other based on
the $x_k$ data that is incorporated. This means that it can more quickly adapt
to data that is not so consistent with the initial prior and thus does not
require a large amount of data in order to *unlearn* the prior probability.
Depending on the application, the ability to unlearn the prior probability or
stick with it is a design problem for the analyst. In this example, because the
data are representative of a $\theta=1/2$ parameter, both priors eventually
settle on an estimated posterior that is about the same. However, if this had
not been the case ($\theta \neq 1/2$), then the second prior
would have
produced a better estimate for the same amount of data.

For this example, the
prior probability is the $\beta(1.3,1.3)$ distribution, which has a wider main
lobe than the $\beta(6,6)$ distribution. The dots near the peaks of each of the
subgraphs indicate the MAP estimate at that frame.

Because we have the
entire posterior density available, we can compute
something that is closely
related to the confidence interval we discussed
earlier, except in this
situation, given the Bayesian interpretation, it is
called a *credible interval*
or *credible set*.  The idea is that we want to
find a symmetric interval around
the peak that accounts for 95% (say) of the
posterior density. This means that
we can then say the probability that the
estimated parameter is within the
credible interval is 95%. The computation
requires significant numerical
processing because even though we have the
posterior density in hand, it is hard
to integrate analytically and requires
numerical quadrature (see Scipy's
`integrate` module).  [Figure](#fig:MAP_003)
shows extent of the interval and
the shaded region under the posterior density
that accounts for 95%.

The credible interval
in Bayesian maximum a-posteriori is the interval corresponding to the shaded
region in the posterior density.


오늘은 Basis와 Rank라는 개념에 대해 살펴보겠습니다.

처음 접했을 때는 무언가 복잡하고 어려워보이던 Rank... 이름만으로도 무언가 되게 본질적이고 어려워 보였습니다.
하지만, 찬찬히 곱씹어보면, 그리 어렵지 않은 개념이더라구요! 오늘 간단하게 개념을 살펴보겠습니다.

basis라는 개념을 살펴보기 전에, Span과 Generating sets라는 개념에 대해 살펴보고 가겠습니다!

### 1. Span

Span이란, 쉽게 생각하면 벡터들을 선형조합(Linear Combination)해주는 과정이라 생각할 수 있습니다. 선형조합은 앞에서 살펴보았듯이, 선형결합 연산을 해주는 것이죠!(Addition, Multiplication by scalar)

아래와 같이, 행렬 A가 3개의 컬럼으로 이뤄져있다고 할 때 이 컬럼들은 A라는 하나의 Vector space 혹은 Vector subspace를 이루고 있다고 볼 수 있습니다.

그리고, 이 컬럼(벡터)들을 하나씩 떼어내 선형조합을 한 것을 바로 span of A라고 볼 수 있습니다.

**Span of A : 벡터(부분)공간"A"의 벡터(columns)들을 선형조합한 set**

위와 같이 정의가 될 수 있습니다.

결국, 아래와 같이 행렬 내에서 Span을 할 경우, columns space(열공간)을 얻어낼 수 있습니다!

$$
\begin{align}
A
=
\begin{bmatrix}
10 & 11 & 12\\
11 & 12 & 13\\
12 & 13 & 14\\
\end{bmatrix}
\end{align}
$$

$$
\begin{align}
span(A)
=
c_{1}
\begin{bmatrix}
10\\
11\\
12\\
\end{bmatrix}
+
c_{2}
\begin{bmatrix}
11\\
12\\
13\\
\end{bmatrix}
+
c_{3}
\begin{bmatrix}
12\\
13\\
14\\
\end{bmatrix}
\end{align}
$$

### 2. Generating sets

Generating sets이란 **"Sets of Vectors that span vector space or vector subspace"라고 정의됩니다.(from MML)**

즉, 벡터공간을 Span해주는 벡터들의 집합(set)이 바로 Generating sets라는 것이죠. 위의 행렬을 참고하자면, 아래처럼 3개의 벡터가 곧 "Generating sets"이 될 것입니다!

$$
\begin{align}
\begin{bmatrix}
10\\
11\\
12\\
\end{bmatrix},
\begin{bmatrix}
11\\
12\\
13\\
\end{bmatrix},
\begin{bmatrix}
12\\
13\\
14\\
\end{bmatrix}
\end{align}
$$

### 3. Basis

Basis란 **"Minimal generating set and a maximal linearly independent set of vectors"라고 합니다.(from MML)**

그대로 직역하자면,
    - 벡터(부분)공간을 구성하는 generating sets중 가장 minimal 한 단위이자
    - 가장 Maximal 한 선형독립인 벡터 집합(set)

이 곧 basis 입니다.

즉 **"minimal generating sets == maximal independent sets of vectors"**라는 것이죠.

한 공간을 구성하는 벡터집합(vector sets)이 곧 선형독립인 것들로만 채워져야 위의 Basis 정의가 들어맞습니다.

곧, Basis는 벡터(부분)공간을 구성하는 선형독립인 벡터 set으로 볼 수 있는 것입니다!

### 4. Rank

Rank란 **"The number of Column vector or row vector which is linearly indepdent"** 입니다.

직역하자면, Rank란 **"선형독립인 열 혹은 행벡터의 갯수"** 인 것이지요!

생각보다 간단한 개념입니다.

오늘은 Basis와 Rank라는 개념에 대해 살펴보았습니다. 간단히 글로만 살펴보았지만, 앞으로의 많은 개념, 연산, 예시에서 이 개념들을 활용할 예정입니다. 앞으로 다양한 예시와 개념들과 함께 지속적으로 다시 살펴보며 이해를 더해나가겠습니다 :)

1.Study/2. with computer/1.Math_code/5. Mathematics for Machine Learning/2. Linear Algebra_6. Basis, Rank.ipynb


#### RL Course by David Silver

## Lecture 3  Planning by Dynamic Programming

- Policy Evaluation
- Policy Iteration
- Value Iteration

# 强化学习

强化学习的过程中，机器在环境中通过尝试得到一个策略$\pi$,使其能够在遇到状态x时,就知道要采取的动作$a=\pi(x)$.策略通常有两种表示方法,一种是$\pi:X\to A$,常用于确定性策略,在状态x下只选择a动作;另一种是$\pi:X \times A \to \mathbb{R}$,常用于随机性策略,在状态x下选择动作a的概率为$\pi(x,a)$.

策略的优劣由该策略的长期累积奖赏决定,强化学习的目的就是寻找长期累计奖赏最大化的策略.长期累积奖赏主要有两种计算方式:一种是t步累积奖赏,表示为$E[\frac{1}{T}\sum_{t=1}^{T}r^t]$,另一种是r折扣累积奖赏,表示为$E[\sum_{t=0}^{+\infty}\gamma^t r^{t+1}]$.其中$r^t$为t步获得的奖赏.

## 探索-利用窘境

在讨论长期累积奖赏最大化之前,以K-摇臂赌博机为例讨论下探索-利用窘境. K-摇臂赌博机的目标是长期累积奖赏最大的简化版本,单步奖赏最大化. 选择某一摇臂这个动作对应的奖赏服从一个概率分布,仅通过一次尝试不能够确定该动作的平均奖赏值. 因此,总尝试次数有限的情况下,如果将所有次数都分配给第一遍尝试中奖励最大的摇臂(利用),可能会导致无法找到最优摇臂; 如果将次数平均分配给每个摇臂(探索),则可能导致最终的平均奖赏较低.这个问题称为探索-利用窘境.
在探索和利用之间如何折中有两种经典算法:e-贪心算法和softmax算法, 两者的区别主要体现在非当前最优动作被采纳的概率是否平均以及当前最优动作被采纳的概率优势大小, 两者均可以通过调参在偏向探索和偏向利用之间调整.

        <top></top>
        <top></top>

## 1. 马尔科夫决策过程

强化学习是建立在马尔科夫决策过程之上的一个体系, 在学习强化学习之前,不可避免要先了解什么是马尔科夫决策过程(MDP).

马尔科夫决策过程是在马尔科夫过程的基础上, 增加了动作,奖励的概念, 包括了五个部分:

- 状态S:有限的状态集
- 动作A:动作的有限集
- 概率P:状态转移概率矩阵

    $p^a_{ss^\prime}=P[S_{t+1}=s^\prime|S_t=s,A_t=a]$
- 奖励R:奖励函数

    $R^a_s=E[R_{t+1}|S_t= s, A_t = a]$
- $\gamma$衰减系数:$\gamma\in[0,1]$

对于一个MDP, 可以产生多个策略作为决策过程的依据. 一个策略包括各个给定状态下的动作概率分布
$$\pi(a|s)=P[A_t=a|S_t=s]$$

下图是一个机器人从任意一个状态出发寻找金币的例子。找到金币则获得奖励 1，碰到海盗则损失 1。找到金币或者碰到海盗则机器人停止。

可以把这个问题建模成马尔科夫决策过程。图中不同位置为状态，因此 $S = \{1,\dots,8\}$。机器人采取动作是向东南西北四个方向走，因此A={‘n’,’e’,’s’,’w’}。转移概率方面，当机器人碰到墙壁，则会停在原来的位置；当机器人找到金币时获得奖励 1，当碰到海盗则损失 1, 其他情况不奖励也不惩罚。因此除了 $R_{1,s}=−1, R_{3,s}=1，R_{5,s}=−1 $之外，其他情况$R_{∗,∗}=0$。采用折扣累积奖赏的计算方式,衰减因子$\gamma$。写成代码下面所示。

```python
class Mdp:
    def __init__(self):
        self.state = [1, 2, 3, 4, 5, 6, 7, 8] # set of states
        self.action = ['n','s', 'e', 'w'] # set of actions
        self.reward = dict() # set of reward among s-a pairs
        self.reward['1-s'] = -1.0
        self.reward['3-s'] = 1.0
        self.reward['5-s'] = -1.0
        self.terminate = dict() # set of terminate states
        self.terminate[6] = True
        self.terminate[7] = True
        self.terminate[8] = True

        self.t = dict() # set of state transforms
        self.t['1-e'] = 2
        self.t['1-s'] = 6
        self.t['2-w'] = 1
        self.t['2-e'] = 3
        self.t['3-w'] = 2
        self.t['3-s'] = 7
        self.t['3-e'] = 4
        self.t['4-w'] = 3
        self.t['4-e'] = 5
        self.t['5-w'] = 4
        self.t['5-s'] = 8
        self.gamma = 0.5

    def transform(self, state, action): # return is_terminate, state, reward
        if state in self.terminate: # start at terminate state
            return True, state, 0.0
        key = "%d-%s"%(state, action)
        if key in self.t : # legal state-action pair
            state = self.t[key]
            if key in self.reward:
                return True, state, self.reward[key] # return the terminate state and reward of s-a pair
            else:
                return False, state, 0.0 # not the state-action pair leading to terminal.
        else:
            return False, state, 0.0 # illegal state-action pair, stay at former state with reward of 0

```

疑惑:
    reward 什么时候分配在状态上 什么时候分配在动作上 对实现有影响

## 有模型学习

有模型学习是指MDP的四要素已知的学习方法, 有模型的学习主要包括策略迭代和值迭代两种方式.

### 1. 策略估值

按照slide中迭代的策略估值的公式

迭代计算上图中的各个点.
具体算法如下:

需要注意的是$R^{\pi}$是从当前状态按照策略产生的奖励, 故对于1状态来说, 其$R^{\pi}=0.25$;对于6,7,8状态来说,由于是直接终止,故$R^{\pi} = 0$
以V(1)在迭代结束的时候的迭代步骤为例, 根据[dp](http://www0.cs.ucl.ac.uk/staff/D.Silver/web/Teaching_files/DP.pdf)中的描述, 当迭代结束时, $|v^{k+1}-v^{k}\leq \epsilon$,此时有
$$
\begin{align}
V(1)=0.25+0.5\times (0.25\times0 + 0.5\times-0.335 + 0.25\times-0.008)\\
     =-0.335
\end{align}
$$
完整的计算结果如下:

```python
# 基于迭代的策略估值
def policy(mdp, state, action):
    return 1.0/len(mdp.action)

def policy_evaluation(mdp, policy):
    values = [0.0]*(len(mdp.state)+1)
    error_s = float('inf')
    epsilon = 0.001
    while(error_s > epsilon):
        aux = list(values)
        values = [0.0]*len(values)
        error = 0
        for s in mdp.state:
            for a in mdp.action:
                is_terminal, state, reward = mdp.transform(s, a)
                values[s] += policy(mdp, s, a) * (reward + mdp.gamma * aux[state])
            error = max(error, abs(values[s] - aux[s]))
        error_s = error
    return values

mdp = Mdp()
values = policy_evaluation(mdp, policy)
print(values)
```

    [0.0, -0.3344268798828125, -0.0081329345703125, 0.28360748291015625, -0.0081329345703125, -0.3344268798828125, 0.0, 0.0, 0.0]

```python
# 基于迭代的策略估值 按照贝克曼方程进行向量加速
# V^{k+1} = R^{\pi}+\gamma P^{\pi}v^k
import numpy as np
# init
R = np.array([0, -0.25, 0, 0.25, 0, -0.25, 0, 0, 0], np.float32)
R = R[:,np.newaxis] # format the shape of R from (9,) to (9, 1)
V = np.zeros((9, 1), np.float32)
p = np.zeros((9,9), np.float32)
p[1][1] = 0.5
p[1][2] = 0.25
p[1][6] = 0.25
p[2][1] = 0.25
p[2][2] = 0.5
p[2][3] = 0.25
p[3][2] = 0.25
p[3][3] = 0.25
p[3][4] = 0.25
p[3][7] = 0.25
p[4][3] = 0.25
p[4][4] = 0.5
p[4][5] = 0.25
p[5][4] = 0.25
p[5][5] = 0.5
p[5][8] = 0.25
gamma = 0.5
error_s = np.inf
epsilon = 0.001
p
assert V.shape == (9, 1)
assert R.shape == (9, 1)
assert p.shape == (9, 9)

# evaluation
while (error_s > epsilon):
    V_k = V.copy()
    V = R + gamma*p.dot(V_k)
    error_s = np.max(np.abs(V - V_k))

np.set_printoptions(precision=3) # pretty the output
print(V)
```

    [[ 0.   ]
     [-0.334]
     [-0.008]
     [ 0.284]
     [-0.008]
     [-0.334]
     [ 0.   ]
     [ 0.   ]
     [ 0.   ]]

### 2. 策略迭代

策略迭代是基于策略估值的优化方法。优化步骤如下：

    - step1：利用估值函数对当前状态进行估值
    - step2：按照贪心策略选取估值最大化的动作更新策略
    - step3：如果策略能够被优化（还未稳定），返回step1，否则结束迭代

策略迭代公式
$$\pi'(s)=argmax_{a\in A}q_{\pi}(s,a)$$
策略迭代终止条件
$$v_{\pi}(s)=max_{a \in A}q_{\pi}(s,a)$$

策略迭代的步骤

```python
# policy iteration.
# +++++++++++++++debug information++++++++++++++++++
# import pdb
# pdb.set_trace()
# +++++++++++++++++++ end ++++++++++++++++++++++++++

class Policy:
    def __init__(self, mdp):
        self.epsilon = 0.0005
        self.mdp = mdp
        self.iteration = 0
        self.values = [0.0]*(len(mdp.state)+1) # v(s)
        self.policy = [mdp.action] * (len(mdp.state)+1) # the argmax q(s,a) of s

    def get_policy(self, state, action): # return the prob of s-a pair
        if action in self.policy[state]:
            return 1. / len(self.policy[state])
        else: return 0.0

    def improve_step(self): # return if the policy has changed
        is_optimium = True
        for index in self.mdp.state:
            self.policy[index] = [] # the argmax q(s,a) of s
            max_value = float('-inf')

            for a in self.mdp.action:
                _, next_s, reward = self.mdp.transform(index, a)
                q_s_a = self.mdp.gamma * self.values[next_s] + reward

                if q_s_a > max_value:
                    self.policy[index] = [a]
                    max_value = q_s_a
                elif q_s_a == max_value:
                    self.policy[index].append(a)

            if abs(self.values[index] - max_value) > self.epsilon:  # v_pi(s) != max q(s,a)
                is_optimium = False
        return is_optimium

    def evaluation(self):
        self.iteration += 1
        error_s = float('inf')
        epsilon = 0.001
        while(error_s > epsilon):
            aux = list(self.values)
            self.values = [0.0]*len(self.values)
            error = 0
            for s in self.mdp.state:
                for a in self.mdp.action:
                    is_terminal, state, reward = self.mdp.transform(s, a)
                    self.values[s] += self.get_policy(s, a) * (reward + self.mdp.gamma * aux[state])
                error = max(error, abs(self.values[s] - aux[s]))
            error_s = error
        print("iter %d: " % self.iteration, self.values)

    def optimium(self):
        n = 5
        while(True):
            self.evaluation()
            is_optimium = self.improve_step()

            # print the current policy
#             for s in mdp.state:
#                 for a in mdp.action:
#                     p = policy.get_policy(s, a)
#                     if p > 0:
#                         print(s, a, policy.get_policy(s, a))

            if is_optimium:
                break

mdp = Mdp()
policy = Policy(mdp)
policy.optimium()

```

    iter 1:  [0.0, -0.3344268798828125, -0.0081329345703125, 0.28360748291015625, -0.0081329345703125, -0.3344268798828125, 0.0, 0.0, 0.0]
    iter 2:  [0.0, 0.25, 0.5, 1.0, 0.5, 0.25, 0.0, 0.0, 0.0]

### 3. 值迭代

由策略迭代的过程可以得到策略迭代过程的原则就是

> 一个策略$\pi(a|s)$在s点处达到最优,$V_{\pi}(s)=V_*(s)$,当且仅当对于每个s状态能够达到的状态s',都有$\pi$在该点达到最优值, $V_{\pi}(s') = V_*(s')$

如果已知$V_*(s')$, 则$V_*(s)$可以直接由
$$V_*(s)=max_{a\in A}(R^a_{s\to s'}+V_*(s'))$$
得到.

因此, 可以直接在迭代的每一步根据上一步的V值对这一步的每个状态的估值进行更新, 相比策略迭代方式减少了策略评价过程中得到稳定策略估值的步骤,减少了计算量.
具体算法如下:

```python
# value iteration.
# +++++++++++++++debug information++++++++++++++++++
# import pdb
# pdb.set_trace()
# +++++++++++++++++++ end ++++++++++++++++++++++++++

class Value:
    def __init__(self, mdp):
        self.epsilon = 0.0005
        self.mdp = mdp
        self.iteration = 0
        self.values = [0.0]*(len(mdp.state)+1) # v(s)
        self.policy = ['s']*(len(mdp.state) + 1) # todo update policy inf func step

    """
    return true if iteration terminate
    return false otherwise.
    """
    def step(self):
        v_temp = [float('-inf')] * len(self.values)
        max_diff = 0.0
        for s in self.mdp.state:
            for a in self.mdp.action:
                _, s_t, reward = self.mdp.transform(s, a)
                v_temp[s] = max(v_temp[s], reward + self.mdp.gamma * self.values[s_t])
            max_diff = max(max_diff, abs(v_temp[s] - self.values[s]))
        if max_diff < self.epsilon:
            return True
        else:
            self.values = v_temp
        return False

    def optimum(self):
        iteration = 0
        while True:
            iteration += 1
            is_optimum = self.step()
            print("iter", iteration,":", self.values)
            if is_optimum: break

    def get_policy(self):
        pass
        # for each state return the maximum of Q(s,a)

mdp = Mdp()
value_iter = Value(mdp)
value_iter.optimum()
```

    iter 1 : [-inf, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0]
    iter 2 : [-inf, 0.0, 0.5, 1.0, 0.5, 0.0, 0.0, 0.0, 0.0]
    iter 3 : [-inf, 0.25, 0.5, 1.0, 0.5, 0.25, 0.0, 0.0, 0.0]
    iter 4 : [-inf, 0.25, 0.5, 1.0, 0.5, 0.25, 0.0, 0.0, 0.0]


# その昔 SageMath というのがあった

freeのソフトを使って mathematica の代用となることを目指していた。

アメリカの数学教育の予算がついていたので成果物があるはずで、それを jupyter または google colab に云々。

要は、インタラクティブな形でネット上で成果物を読むことができないか、と考えている。

日本語のチュートリアル
https://doc.sagemath.org/html/ja/tutorial/introduction.html

https://ja.wikipedia.org/wiki/SageMath

https://www.sagemath.org/

sagemath.github.io

https://icu-hsuzuki.github.io/science/computer/education/sage-j.html

```
# sage: 2 + 2
2 + 2
```

    4

```
# sage: factor(-2007)
# factor (-2007) # エラーになる
# -2007を因数分解したい、ということなので
from sympy import *
display(factor(-2007))
display(factorint(-2007))
display(-1*3**2*223)
```

    -2007

    {-1: 1, 3: 2, 223: 1}

    -2007

```
# sage: A = matrix(4,4, range(16)); A
from sympy import *
Matrix([0,1,2,3],[4,5,6,7],[8,9,10,11],[12,13,14,15])
```

# いまここ

https://doc.sagemath.org/html/ja/tutorial/introduction.html


```python
import arviz as az
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import pymc3 as pm

from scipy.special import expit as logistic
```

```python
from traitlets.config.manager import BaseJSONConfigManager
path = "/home/osvaldo/anaconda3/etc/jupyter/nbconfig"
cm = BaseJSONConfigManager(config_dir=path)
cm.update("livereveal", {
              "theme": "serif",
              "transition": "zoom",
              "start_slideshow_at": "selected",
              "controls": "True",
              "progress": "False",
              "shortcut": "False"});
```

```python
az.style.use('arviz-darkgrid')
```

```python
%%HTML
<style>

.CodeMirror {
    width: 100vw;
}

.container {
    width: 99% !important;
}

.rendered_html {
  font-size:0.8em;
}
.rendered_html table, .rendered_html th, .rendered_html tr, .rendered_html td {
     font-size: 100%;
}

body {
  font-family: Ubuntu;
  background: #F0F0F0;
  background-color: #F0F0F0;
}

.reveal h1,
.reveal h2,
.reveal h3,
.reveal h4,
.reveal h5,
.reveal h6 {
  margin: 0 0 20px 0;
  color: #2a2eec;
  font-family: Ubuntu;
  line-height: 0.9em;
  letter-spacing: 0.02em;
  text-transform: none;
  text-shadow: none;
}

.reveal blockquote {
  display: block;
  position: relative;
  background: #fa7c17;
  border-radius: 15px;
  box-shadow: 0px 0px 2px rgba(0, 0, 0, 0.2);
  font-weight: bold;
}

</style>
```

#####

Modelado Jerárquico

Agrupamiento parcial de datos

## Objetivos

* Aprender a construir modelos jerárquicos
* Discutir el concepto de contracción y regularización.
* Discutir las implicaciones de agrupar parcialmente los datos

## Modelos Jerárquicos

## Corrimientos químicos jerárquicos

* Las proteínas son moléculas formadas por un número variable de 20 unidades en un orden variable.

* Para cada una de estas unidades podemos medir un "corrimiento químico" y también podemos calcularlo usando métodos de Química Cuántica

* Como cada una de las 20 unidades son químicamente diferentes, existe variabilidad entre unidades

* Como el entorno químico (la proteína) modula el valor de un corrimiento químico, existe variabilidad dentro de cada unidad

```python
cs_data = pd.read_csv('../datos/chemical_shifts_theo_exp.csv')
diff = cs_data.theo - cs_data.exp
idx = pd.Categorical(cs_data['aa']).codes
groups = len(np.unique(idx))

cs_data.tail()
```

      ID
      aa
      theo
      exp

      1771
      1KS9
      LYS
      55.79
      57.51

      1772
      1KS9
      ARG
      58.91
      59.02

      1773
      1KS9
      LYS
      59.49
      58.92

      1774
      1KS9
      GLU
      59.48
      58.36

      1775
      1KS9
      SER
      58.07
      60.55

```python
with pm.Model() as cs_nh:
    μ = pm.Normal('μ', mu=0, sd=10, shape=groups)
    σ = pm.HalfNormal('σ', sd=10, shape=groups)

    y = pm.Normal('y', mu=μ[idx], sd=σ[idx], observed=diff)

    trace_cs_nh = pm.sample(1000)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [σ, μ]

  <progress value='3000' class='' max='3000', style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [3000/3000 00:07<00:00 Sampling 2 chains, 0 divergences]

```python
with pm.Model() as cs_h:
    # hiper_priors
    μ_mu = pm.Normal('μ_mu', mu=0, sd=10)
    μ_sd = pm.HalfNormal('μ_sd', 10)

    # priors
    μ = pm.Normal('μ', mu=μ_mu, sigma=μ_sd, shape=groups)
    σ = pm.HalfNormal('σ', sd=10, shape=groups)

    y = pm.Normal('y', mu=μ[idx], sigma=σ[idx], observed=diff)

    trace_cs_h = pm.sample(1000)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [σ, μ, μ_sd, μ_mu]

  <progress value='3000' class='' max='3000', style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [3000/3000 00:07<00:00 Sampling 2 chains, 0 divergences]

```python
axes = az.plot_forest([trace_cs_nh, trace_cs_h], model_names=['non_hierarchical', 'hierarchical'],
              var_names='μ', combined=True, r_hat=False, ess=False, figsize=(10, 12),
              colors='cycle');
y_lims = axes[0].get_ylim()
axes[0].vlines(trace_cs_h['μ_mu'].mean(), *y_lims);
```

## Contracción y ventajas de los modelos jerárquicos

* Cada grupo informa a los demás y es informado por los demás.

* La estimación de cada grupo se contrae hacia la media común

* La fuerza de la contracción no es la misma para todos los grupos
     * Los grupos típicos están menos afectados.
     * Los grupos con muestras de mayor tamaño se ven menos afectados.

* Conservar la variación a nivel de grupo y entre grupos

* El modelo jerárquico es una forma de regularización.

## Renacuajos multinivel

* Tenemos 48 tanques llenos de renacuajos

* Queremos modelar la probabilidad de supervivencia

* Las condiciones como la temperatura, el pH, la luz, etc. varían ligeramente entre los tanques (pero no los estamos teniendo en cuenta explícitamente)

* Podemos pensar en cada tanque como un grupo

[statistical rethinking](https://xcelab.net/rm/statistical-rethinking/)

```python
d = pd.read_csv('../datos/reedfrogs.csv', sep=",")
d.head()
```

      density
      pred
      size
      surv
      propsurv

      0
      10
      no
      big
      9
      0.9

      1
      10
      no
      big
      10
      1.0

      2
      10
      no
      big
      7
      0.7

      3
      10
      no
      big
      10
      1.0

      4
      10
      no
      small
      9
      0.9

```python
with pm.Model() as modelo_renacuajos:
    # Hiperpriors
    μ = pm.Normal('μ', 0., 1.)
    σ = pm.HalfNormal('σ', 10.)
    # Prior
    α_tanque = pm.Normal('α_tanque', μ, σ, shape=d.shape[0])
    p = pm.Deterministic('p', pm.math.sigmoid(α_tanque))  # transformación logística
    #likelihood
    surv = pm.Binomial('surv', n=d.density, p=p, observed=d.surv)

    trace_renacuajos = pm.sample(2000, tune=2000)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [α_tanque, σ, μ]

  <progress value='8000' class='' max='8000', style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [8000/8000 00:12<00:00 Sampling 2 chains, 0 divergences]

```python
_, ax = plt.subplots(1, 1, figsize=(12, 5))

ax.scatter(np.arange(0, 48), d.propsurv, color='C0')
ax.scatter(np.arange(0, 48), np.mean(trace_renacuajos['p'], 0), color='C1')
ax.hlines(logistic(np.median(trace_renacuajos['μ'], axis=0)), -1, 49, linestyles='--')

idx = d.density[d.density.diff() > 0].index
ax.vlines(idx + 0.5, -.05, 1.05, lw=.5)
for i, t in zip(np.linspace(0, 48, 7)[1::2], ('pequeño', 'mediano', 'largo')):
    ax.text(i, 0, t, horizontalalignment='center')
ax.set_xlabel('tanques')
ax.set_ylabel('proporción de survivencia')
ax.set_xlim(-1, 48)
ax.set_xticks([])
ax.set_ylim(-.05, 1.05)
ax.grid(False)
```

## Monedas jerárquicas

\begin{align}
\mu &\sim \operatorname{Beta}(\alpha, \beta) \\
\kappa &\sim \operatorname{Gamma}(s, r) \\
\theta &\sim \operatorname{Beta}(\alpha=\mu  \kappa, \beta=(1 - \mu)  \kappa) \\
y &\sim \operatorname{Bin}(n=1, p=\theta)
\end{align}

[Doing Bayesian Data Analysis](https://sites.google.com/site/doingbayesiandataanalysis/)

```python
N = [30, 30, 30]  # Número de tiradas por moneda
z = [3, 3, 3]  # Número de caras por moneda

# vector índice
monedas = np.repeat(np.arange(len(N)), N)
# Lista llena de ceros (cecas) y unos (caras)
datos = []
for i in range(len(N)):
    datos.extend(np.repeat([1, 0], [z[i], N[i]-z[i]]))
```

```python
with pm.Model() as modelo_j:
    # hiperparámetros
    μ = pm.Beta('μ', 2., 2.)
    κ = pm.HalfNormal('κ', 10.)

    # priors
    θ = pm.Beta('θ', μ * κ, (1 - μ) * κ, shape=len(N))
    # likelihood
    y = pm.Bernoulli('y', p=θ[monedas], observed=datos)

    trace_j = pm.sample(2000)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [θ, κ, μ]

  <progress value='5000' class='' max='5000', style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [5000/5000 00:09<00:00 Sampling 2 chains, 0 divergences]

```python
az.plot_trace(trace_j);
```

```python
az.summary(trace_j, kind='stats')
```

      mean
      sd
      hpd_3%
      hpd_97%

      μ
      0.189
      0.086
      0.053
      0.340

      κ
      10.963
      6.407
      0.663
      22.344

      θ[0]
      0.117
      0.054
      0.029
      0.218

      θ[1]
      0.117
      0.053
      0.027
      0.216

      θ[2]
      0.117
      0.053
      0.025
      0.211

## Ejercicios

1. En el modelo `cs_h` ¿Cuál de los siguientes priors generará mayor *contracción* μ = Normal(0, 0.5) o μ = Normal(0, 1). Justificar

2. Convierte el modelo de las propinas del notebook anterior en un modelo jerárquico (agrupando los días de la semana).

3. Repetí el ejemplo de las monedas jerárquicas, esta vez use los siguientes datos:
    * z = [3,3,3]
    * z = [18,3,18]

Registrá el valor de las medias estimadas para $\theta$, para que puedas comparar los 3 ejemplos (incluyendo `z = [18,18,18]`)