 INF-285 - Computación Científica

 The Beginning

  [S]cientific [C]omputing [T]eam

 Version: 1.02

## Table of Contents
* [Introduction](#intro)
* [NumPy](#numpy)
* [MatPlotLib](#matplotlib)
* [SciPy](#scipy)
* [SymPy](#sympy)

* [Libraries](#libraries)
* [Acknowledgements](#acknowledgements)

```python
import numpy as np
import scipy as sp
import matplotlib.pyplot as plt
import sympy as sym

from scipy.integrate import odeint
import scipy.linalg as spla
from scipy import optimize, interpolate
from scipy.linalg import solve_triangular, toeplitz, lu
from scipy.optimize import root
# pip install bitstring
import bitstring as bs

%matplotlib inline

sym.init_printing()
from ipywidgets import interact, widgets, fixed, IntSlider, interact_manual, Checkbox, RadioButtons
from mpl_toolkits.mplot3d import Axes3D
import matplotlib as mpl
mpl.rcParams['font.size'] = 14
mpl.rcParams['axes.labelsize'] = 20
mpl.rcParams['xtick.labelsize'] = 14
mpl.rcParams['ytick.labelsize'] = 14
from matplotlib import cm

# pip install memory_profiler
%load_ext memory_profiler

from sklearn import datasets
from colorama import Fore, Back, Style
# https://pypi.org/project/colorama/
# Fore: BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE, RESET.
# Back: BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE, RESET.
# Style: DIM, NORMAL, BRIGHT, RESET_ALL
textBold = lambda x: Style.BRIGHT+x+Style.RESET_ALL
textBoldH = lambda x: Style.BRIGHT+Back.YELLOW+x+Style.RESET_ALL
textBoldB = lambda x: Style.BRIGHT+Back.BLUE+Fore.BLACK+x+Style.RESET_ALL
textBoldR = lambda x: Style.BRIGHT+Back.RED+Fore.BLACK+x+Style.RESET_ALL
```

# Introduction
[Back to TOC](#toc)

In this jupyter notebook we will briefly introduce part of ecosystem in Python for Scientific Computing with explanation and examples.
We strongly suggest to follow the explanation step by step, read the suggested documents/links, and play with the jupyter notebook changing it to learn!

# NumPy, the core of numerical computing in Python
[Back to TOC](#toc)

We strongly suggests to take a look to the associated paper explaning Arrays in NumPy [link](https://www.nature.com/articles/s41586-020-2649-2).
Now, we will start with some examples provided in Fig. 1 from the previous reference. Also see https://numpy.org.

## a) Data structure: An array and a bi-dimensional array.

```python
# NumPy array
a1 = np.arange(12)
# NumPy ndarray
a2 = np.array([[1, 2, 3], [4, 5, 6]], float)
a3 = np.reshape(a1,(4,3))

print(textBoldB(' Printing their values '))
print(textBoldH('a1:\n'),a1)
print(textBoldH('a2:\n'),a2)
print(textBoldH('a3:\n'),a3)

print(textBoldB(' Printing their datatypes '))
print(textBoldH('a1:\n'),a1.dtype)
print(textBoldH('a2:\n'),a2.dtype)
print(textBoldH('a3:\n'),a3.dtype)

print(textBoldB(' Printing their type name '))
print(textBoldH('a1:\n'),type(a1).__name__)
print(textBoldH('a2:\n'),type(a2).__name__)
print(textBoldH('a3:\n'),type(a3).__name__)
```

     Printing their values
    a1:
     [ 0  1  2  3  4  5  6  7  8  9 10 11]
    a2:
     [[1. 2. 3.]
     [4. 5. 6.]]
    a3:
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]
     Printing their datatypes
    a1:
     int64
    a2:
     float64
    a3:
     int64
     Printing their type name
    a1:
     ndarray
    a2:
     ndarray
    a3:
     ndarray

## b) Indexing (view)

```python
print(textBoldB(' Showing original variable a3 '))
print(textBoldH('a3\n'),a3)

print(textBoldB(' Showing a subset of elements (SLICES) of variable a3 \n the second and third columns '))
print(textBoldH('a3[:,1:]\n'),a3[:,1:])
print(textBoldB(' Storing the outcome in a4 '))
a4 = a3[:,1:]

print(textBoldB(' What does it means it is a "view"? '))
print(textBoldB(' Let\'s change the original a3 and see what happens to a4. '))
a3[0,1]=100
print(textBoldH('a4\n'), a4)
print(textBoldB(' a4 has changed too! So a4 is a "view" of part of a3. '))
print('\n')
print(textBoldB(' So, how do we copy it?'))
print(textBoldB(' Let\'s use np.copy to copy the "view" obtained before,'+
                '\n then modify the original variable and see what had happened'))
a5 = np.copy(a3[:,1:])
print(textBoldH('a3\n'), a3)
print(textBoldH('a4=a3[:,1:])\n'), a4)
print(textBoldH('a5=np.copy(a3[:,1:])\n'), a5)
a3[0,1]=1
print(textBoldB(' Changing a3 as "a3[0,1]=1"'))
print(textBoldH('a3\n'), a3)
print(textBoldH('a4\n'), a4)
print(textBoldH('a5\n'), a5)
print(textBoldB(' Thus, chaning a3 have changed a4 since it is a "view" and not changed a5 since it is a copy. '))
```

     Showing original variable a3
    a3
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]
     Showing a subset of elements (SLICES) of variable a3
     the second and third columns
    a3[:,1:]
     [[ 1  2]
     [ 4  5]
     [ 7  8]
     [10 11]]
     Storing the outcome in a4
     What does it means it is a "view"?
     Let's change the original a3 and see what happens to a4.
    a4
     [[100   2]
     [  4   5]
     [  7   8]
     [ 10  11]]
     a4 has changed too! So a4 is a "view" of part of a3.

     So, how do we copy it?
     Let's use np.copy to copy the "view" obtained before,
     then modify the original variable and see what had happened
    a3
     [[  0 100   2]
     [  3   4   5]
     [  6   7   8]
     [  9  10  11]]
    a4=a3[:,1:])
     [[100   2]
     [  4   5]
     [  7   8]
     [ 10  11]]
    a5=np.copy(a3[:,1:])
     [[100   2]
     [  4   5]
     [  7   8]
     [ 10  11]]
     Changing a3 as "a3[0,1]=1"
    a3
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]
    a4
     [[ 1  2]
     [ 4  5]
     [ 7  8]
     [10 11]]
    a5
     [[100   2]
     [  4   5]
     [  7   8]
     [ 10  11]]
     Thus, chaning a3 have changed a4 since it is a "view" and not changed a5 since it is a copy.

```python
print(textBoldB(' Another way to extract slices is with steps: '))
a6 = a3[:,::2]
print(textBoldH('a3\n'), a3)
print(textBoldH('a6 = a3[:,::2]\n'), a6)
print(textBoldB(' In this case we obtained the first and third columns, \n '+
               ' the pattern consider ini:end:step, where "ini" is the first index considered,\n'+
               ' "end" defines that the pattern will be considere until the index "end-1",\n'+
               ' and "step" indicates the step for selecting the next index, it is 1 by default.'))
```

     Another way to extract slices is with steps:
    a3
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]
    a6 = a3[:,::2]
     [[ 0  2]
     [ 3  5]
     [ 6  8]
     [ 9 11]]
     In this case we obtained the first and third columns,
      the pattern consider ini:end:step, where "ini" is the first index considered,
     "end" defines that the pattern will be considere until the index "end-1",
     and "step" indicates the step for selecting the next index, it is 1 by default.

## c) Indexing (copy)

```python
print(textBoldB(' Now we will list the alternatives to "copy" the data from ndarrays'))
print(textBoldH('a3\n'), a3)

print('\n')
print(textBoldR('with SCALARS:'))
print(textBoldB('o1 = a3[1,2]'))
o1 = a3[1,2]
print(textBoldH('o1\n'), o1)
print(textBoldB('Changing o1: o1 = 111'))
o1 = 111
print(textBoldH('a3\n'), a3)
print(textBoldH('o1\n'), o1)

print('\n')
print(textBoldR('with MASKS:'))
print(textBoldB('o2 = a3[a3>9]'))
o2 = a3[a3>9]
print(textBoldH('o2\n'), o2)

print('\n')
print(textBoldR('with ARRAYS:'))
print(textBoldB('o3 = a3[[0,1],[1,2]]'))
o3 = a3[[0,1],[1,2]]
print(textBoldH('o3\n'), o3)
print(textBoldH('o3:\n'),o3.dtype)
print(textBoldH('o3:\n'),type(o3).__name__)

print('\n')
print(textBoldR('with ARRAYS and BROADCASTING (this will be review next) :'))
print(textBoldB('o4 = a3[[[1],[2]],[1,0]]'))
o4 = a3[[[1],[2]],[1,0]]
print(textBoldH('o4\n'), o4)
print(textBoldH('o4:\n'),o4.dtype)
print(textBoldH('o4:\n'),type(o4).__name__)
```

     Now we will list the alternatives to "copy" the data from ndarrays
    a3
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]

    with SCALARS:
    o1 = a3[1,2]
    o1
     5
    Changing o1: o1 = 111
    a3
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]
    o1
     111

    with MASKS:
    o2 = a3[a3>9]
    o2
     [10 11]

    with ARRAYS:
    o3 = a3[[0,1],[1,2]]
    o3
     [1 5]
    o3:
     int64
    o3:
     ndarray

    with ARRAYS and BROADCASTING (this will be review next) :
    o4 = a3[[[1],[2]],[1,0]]
    o4
     [[4 3]
     [7 6]]
    o4:
     int64
    o4:
     ndarray

## d) Vectorization (THE HEART OF THE HEART OF NUMERICAL COMPUTING!!)

```python
print(textBoldH('a3\n'), a3)
print(textBoldB(' We will define a new ndarray n1 = np.ones((4,3)) and add it to the array a3.'))
n1 = np.ones((4,3))

print(textBoldB(' The first implementation one may think of is the following:'))
def sum_two_arrays(a3,n1):
    out = np.zeros_like(a3) # What does this function do?
    for i in range(4):
        for j in range(3):
            out[i,j] = a3[i,j]+n1[i,j]
    return out
n2 = sum_two_arrays(a3,n1)
```

    a3
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]
     We will define a new ndarray n1 = np.ones((4,3)) and add it to the array a3.
     The first implementation one may think of is the following:

```python
# To quantify the computation time it takes let's measure it:
%timeit n2 = sum_two_arrays(a3,n1)
```

    39.4 µs ± 4.26 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

```python
# The output it shows in the computer we run the experiments was:
# 33.9 µs ± 1.68 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
# This means it took about 34 microseconds.

# Now, let's try the NumPy way! i.e. the vectorized approach!
%timeit n2 = a3+n1
```

    1.08 µs ± 21.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

```python
# The output it shows in the computer we run the experiments was:
# 1.05 µs ± 12.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
# This means it was about 34 times faster!!
# and also it only took "one" line of code where in the previous version took "7" lines of code!!
print(textBoldR(' Given the previous argument, it is crucial you get familir with VECTORIZATION computing with NumPy!! '))
```

     Given the previous argument, it is crucial you get familir with VECTORIZATION computing with NumPy!!

## e) Broadcasting

```python
# Creating a 1D array, this is the stringly suggested way to create "vectors" is NumPy.
# This can be considered as a row vector.
x1 = np.array([1,2,3])
print(textBoldH('x1:'),x1)
print(textBoldH('shape:'),x1.shape)
print(textBoldH('dtype:'),x1.dtype)
print(textBoldH('type name:'),type(x1).__name__)
```

    x1: [1 2 3]
    shape: (3,)
    dtype: int64
    type name: ndarray

```python
# Creating a 2D array with one row.
x2 = x1[np.newaxis] # What does np.newaxis do?
print(textBoldH('x2:'),x2)
print(textBoldH('shape:'),x2.shape)
print(textBoldH('dtype:'),x2.dtype)
print(textBoldH('type name:'),type(x2).__name__)
# This emulates a column vector.
```

    x2: [[1 2 3]]
    shape: (1, 3)
    dtype: int64
    type name: ndarray

```python
# Transpossing the 2D array. This emulates a column vector.
x2 = x1[np.newaxis].T
print(textBoldH('x2:'),x2)
print(textBoldH('shape:'),x2.shape)
print(textBoldH('dtype:'),x2.dtype)
print(textBoldH('type name:'),type(x2).__name__)
```

    x2: [[1]
     [2]
     [3]]
    shape: (3, 1)
    dtype: int64
    type name: ndarray

```python
# Notice that transpossing a 1D array does not do anything (please try it by yourself!).
# Now that we have a 1D array (that emulates a row vector) and
# a 1 column 2d array (that emulates a columns vectors), we can do a "BROADCAST" and see what happend.
x3=x1+x2
print(textBoldH('x3:'),x3)
print(textBoldH('shape:'),x3.shape)
print(textBoldH('dtype:'),x3.dtype)
print(textBoldH('type name:'),type(x3).__name__)
# In this cas we have operated elementwise and broadcasted the array.
```

    x3: [[2 3 4]
     [3 4 5]
     [4 5 6]]
    shape: (3, 3)
    dtype: int64
    type name: ndarray

```python
# The previous operation is equivalent to the following operation.
x11 = np.tile(x1, (3,1)) # Repeat the 1d array 'vertically' 3 times
print(textBoldH('x11:'),x11)
x21 = np.tile(x2, 3) # Repeat the 2d array 'horizontally' 3 times
print(textBoldH('x21:'),x21)
x31 = x11+x21
print(textBoldH('x31:'),x31)
print(textBoldH('shape:'),x31.shape)
print(textBoldH('dtype:'),x31.dtype)
print(textBoldH('type name:'),type(x31).__name__)
print(textBoldR(' As you may suspect, this way of doing the sum is not recommended.\n'+
               ' It was added just for comparison purposes.'))
```

    x11: [[1 2 3]
     [1 2 3]
     [1 2 3]]
    x21: [[1 1 1]
     [2 2 2]
     [3 3 3]]
    x31: [[2 3 4]
     [3 4 5]
     [4 5 6]]
    shape: (3, 3)
    dtype: int64
    type name: ndarray
     As you may suspect, this way of doing the sum is not recommended.
     It was added just for comparison purposes.

## f) Reduction

```python
print(textBoldH('a3\n'), a3)

print('\n')
print(textBoldB(' First reduction, sum along axis 0:'))
print(textBoldB(' s1 = np.sum(a3,axis=0)'))
s1 = np.sum(a3,axis=0)
print(textBoldH('s1'), s1)
print(textBoldH('shape:'),s1.shape)
print(textBoldH('dtype:'),s1.dtype)
print(textBoldH('type name:'),type(s1).__name__)

print('\n')
print(textBoldB(' First reduction, sum along axis 1:'))
print(textBoldB(' s2 = np.sum(a3,axis=1)'))
s2 = np.sum(a3,axis=1)
print(textBoldH('s2'), s2)
print(textBoldH('shape:'),s2.shape)
print(textBoldH('dtype:'),s2.dtype)
print(textBoldH('type name:'),type(s2).__name__)

print('\n')
print(textBoldB(' First reduction, sum along axis 0 and 1:'))
print(textBoldB(' s3 = np.sum(a3)'))
s3 = np.sum(a3)
print(textBoldH('s3'), s3)
print(textBoldH('shape:'),s3.shape)
print(textBoldH('dtype:'),s3.dtype)
print(textBoldH('type name:'),type(s3).__name__)

print('\n')
print(textBoldR(' As you may suspect, the reduction reduces the dimensionality in respect of the original input.'))
```

    a3
     [[ 0  1  2]
     [ 3  4  5]
     [ 6  7  8]
     [ 9 10 11]]

     First reduction, sum along axis 0:
     s1 = np.sum(a3,axis=0)
    s1 [18 22 26]
    shape: (3,)
    dtype: int64
    type name: ndarray

     First reduction, sum along axis 1:
     s2 = np.sum(a3,axis=1)
    s2 [ 3 12 21 30]
    shape: (4,)
    dtype: int64
    type name: ndarray

     First reduction, sum along axis 0 and 1:
     s3 = np.sum(a3)
    s3 66
    shape: ()
    dtype: int64
    type name: int64

     As you may suspect, the reduction reduces the dimensionality in respect of the original input.

## g) Examples

```python
# Create a random matrix and comput its determinant
np.random.seed(0) # Fixing seed for randomnumber generator.
n = 100
A = np.random.random((n,n))
print(textBoldH('|A|:'), np.linalg.det(A))
```

    |A|: 4.189009433279832e+24

```python
# Compute matrix vector product
x = np.random.random(n)
b = A @ x
print(textBoldH('A*x:'), b)
```

    A*x: [22.50387804 27.06165293 25.44380132 23.90306726 23.5383304  25.97354295
     25.39133786 23.23602505 24.96797807 26.35437211 24.27664139 25.67413645
     23.48460197 27.44447023 25.01080204 26.21436807 23.91089268 23.70399375
     26.22314526 23.86964466 23.34695257 24.9999494  24.89781895 24.68551405
     25.01137941 25.3997684  27.07055286 21.46566256 24.26978535 23.4243107
     24.53506025 22.71666134 26.62366171 25.82769683 24.07777227 24.0025882
     23.31802885 23.65800393 22.56287072 21.65606265 23.49534362 25.8842525
     26.99496029 22.16848876 23.42231272 22.87673568 22.68911393 22.00950553
     24.55681272 23.32897479 23.47424422 22.0042083  27.62785605 24.867569
     24.55058081 26.24970021 25.46630341 23.86956795 21.73690518 23.01731327
     25.4796017  22.83023589 25.22949757 25.50838772 25.52477344 23.8568525
     24.69135328 24.45486666 27.27024007 23.65238719 23.42199642 25.47403783
     25.11460573 27.85454033 23.02376118 23.35345807 25.94299815 23.22683254
     24.44753233 25.2004444  23.14280702 25.84015184 24.84311922 23.01210411
     26.16362515 23.67760286 25.1811232  25.06465188 25.37328402 25.90372182
     24.22927176 24.11067019 25.40049423 24.89267375 23.10754772 23.31576435
     24.28890539 27.46372393 24.8354481  25.08037583]

```python
# Solving a linear system of equations A*x=b, i.e. assume we know A and b and we want x.
x_out = np.linalg.solve(A,b)
print(textBoldH('x_out=A^{-1} b:'), x_out)
```

    x_out=A^{-1} b: [0.74826798 0.18020271 0.38902314 0.03760018 0.01178774 0.99626787
     0.48819666 0.37202476 0.19617209 0.80719225 0.70575272 0.0015562
     0.77122667 0.11148275 0.94863268 0.33273608 0.45110278 0.00498091
     0.82439269 0.30816825 0.55644688 0.9266008  0.15622238 0.86732961
     0.50023893 0.92445617 0.82235505 0.44298003 0.08871177 0.03019745
     0.87423081 0.47428492 0.66369491 0.88164841 0.30465899 0.89576302
     0.02753244 0.27992568 0.8117664  0.85322032 0.9448354  0.30301127
     0.78023559 0.9851303  0.52700602 0.25259629 0.03896232 0.75965306
     0.13884538 0.38694261 0.30927788 0.41685067 0.90599899 0.42594776
     0.91274431 0.80540086 0.28837833 0.31808902 0.36984422 0.60564719
     0.67406252 0.97222984 0.92651606 0.61187172 0.04623878 0.54530762
     0.40364022 0.54477873 0.04017637 0.99576221 0.05695837 0.27267993
     0.07212403 0.54761111 0.3518183  0.37529984 0.78916752 0.90402012
     0.16569698 0.27897694 0.47272862 0.11288167 0.38653808 0.30208805
     0.92197634 0.19853412 0.7212114  0.21589321 0.62565838 0.58199073
     0.45872429 0.8169073  0.22271719 0.47728022 0.9824176  0.18326179
     0.54543245 0.29310578 0.12380606 0.85159298]

```python
# Computing the norm 2, between x and x_out
print(textBoldH('||x-x_out||_2:'), np.linalg.norm(x-x_out))
# It seems it is small, why is this?
```

    ||x-x_out||_2: 2.2940419009392607e-13

```python
# Computing dot product
v1 = np.random.random(n)
v2 = np.random.random(n)
print(textBoldH('dot product between v1 and v2:'), np.dot(v1,v2))
```

    dot product between v1 and v2: 22.562561096899618

```python
# Computing elementwise product
print(textBoldH('v1*v2:'), v1*v2)
```

    v1*v2: [0.5040407  0.35124297 0.20272754 0.06808604 0.03793166 0.03130265
     0.16771434 0.03131432 0.13621973 0.08603658 0.15233759 0.63918918
     0.18470824 0.05499963 0.01482564 0.09488504 0.10792356 0.045415
     0.06367632 0.06743177 0.14068657 0.11574464 0.33993602 0.71001683
     0.02322593 0.17319421 0.04309746 0.6421034  0.2301384  0.41128254
     0.0240249  0.26253555 0.32900122 0.68786613 0.7069239  0.19076432
     0.116608   0.91879584 0.81958956 0.26540587 0.15802502 0.00093995
     0.34807158 0.42177808 0.28353462 0.08118476 0.00483724 0.12667942
     0.15228556 0.13161217 0.48897394 0.23173528 0.14190865 0.08554734
     0.40820262 0.01695295 0.04123169 0.19450118 0.07245406 0.44841639
     0.34529915 0.09043179 0.89483999 0.00454414 0.43664874 0.05455569
     0.49905853 0.06178369 0.1334216  0.16378546 0.44445211 0.25810607
     0.26141593 0.39681414 0.01200907 0.02636425 0.00702124 0.51440657
     0.11066247 0.12086425 0.3494806  0.06069188 0.29896651 0.00942196
     0.24711492 0.00450915 0.52898055 0.3297233  0.01349076 0.09048845
     0.14948964 0.05592686 0.30474635 0.37455074 0.17706751 0.0695418
     0.0419936  0.12171033 0.08576855 0.38059043]

```python
# Squaring each element of v1
print(textBoldH('out:'), np.power(v1,2))
```

    out: [3.33712844e-01 3.35043316e-01 2.97389976e-01 3.40284824e-02
     3.23343677e-01 8.35195196e-03 4.65999923e-01 2.64784014e-03
     3.57781451e-02 8.65641121e-02 7.78026242e-02 6.59510128e-01
     8.28004335e-02 5.46977129e-03 4.62164663e-03 2.23243998e-02
     1.55387254e-01 3.87358426e-02 5.52355095e-02 3.38470418e-02
     8.92428375e-02 2.54882336e-01 9.43515134e-01 6.27248860e-01
     2.97633931e-02 6.44569231e-01 5.04232111e-01 8.20470759e-01
     1.88969944e-01 7.51860125e-01 4.87330001e-01 7.45795551e-02
     1.51925446e-01 7.03046345e-01 6.52399127e-01 6.02731826e-02
     6.99354020e-01 8.94456439e-01 7.70479222e-01 1.45380560e-01
     2.92856089e-01 7.15649408e-01 5.13229573e-01 5.02829816e-01
     8.13683961e-02 3.73078949e-01 2.96601313e-03 8.82965771e-02
     4.11233243e-02 1.49780281e-01 5.44982922e-01 6.78816793e-02
     3.20106099e-02 7.09024079e-01 8.37404284e-01 1.99109251e-01
     5.97791604e-02 6.36885484e-02 1.73205699e-01 6.43914668e-01
     3.15261678e-01 3.33316796e-01 9.56791734e-01 6.03343621e-03
     2.25637724e-01 2.29826561e-01 3.06887166e-01 2.82039233e-02
     2.85346922e-02 1.72597507e-01 3.67499362e-01 9.16893692e-02
     2.47525358e-01 9.73689918e-01 9.26046819e-04 2.17032328e-01
     2.24585662e-01 4.78596079e-01 7.09690214e-02 6.39894512e-01
     1.60795723e-01 9.05588097e-03 3.96976232e-01 6.96598071e-04
     5.13805385e-01 1.47386830e-04 3.56015330e-01 7.58452840e-01
     4.87774331e-01 3.89805124e-01 1.86443576e-01 1.13027426e-01
     1.37118773e-01 1.40644816e-01 2.80776661e-01 2.05435121e-02
     7.77715748e-01 3.91720197e-01 1.40202489e-01 9.64740735e-01]

# MatPlotLib, comprehensive 2-D plotting
[Back to TOC](#toc)

In this case, we will go straight to the examples. See https://matplotlib.org and https://github.com/matplotlib/cheatsheets#cheatsheets.

```python
def plot_trig_func(n=5):
    x = np.linspace(0,10,n) # Generates a linearly spaced set of n points from 0 to 10
    y1 = np.sin(x) # Vectorization in action!
    y2 = np.cos(x) # Vectorization in action!
    plt.figure(figsize=(5,5)) # Creating the figure
    plt.plot(x,y1,label=r'$\sin(x)$') # Plotting sin
    plt.plot(x,y2,'r.-',label=r'$\cos(x)$') # Plotting cos
    plt.grid(True) # Adding nice grid
    plt.legend(loc='best') # Adding legend
    plt.title('Plotting trigonometric functions') # Adding title
    plt.xlabel(r'$x$') # Adding x-label
    plt.ylabel(r'$y$') # Adding y-label
    plt.show() # Showing the plot
# The following is a iPython widget called interact,
# it allows us to interact with the function that it is being called.
interact(plot_trig_func,n=(3,1000,1)) # Making this 'tunable'!
```

    interactive(children=(IntSlider(value=5, description='n', max=1000, min=3), Output()), _dom_classes=('widget-i…

    <function __main__.plot_trig_func(n=5)>

```python
def plot_funct_2_var(theta=0):
    x = np.linspace(-2,2,20)
    y = x # Recall this is just a 'view'

    X, Y = np.meshgrid(x,y) # It generates two 2D arrays with with cartesian product of 'x' and 'y'.

    # 'lambda' definition help us to organize better the code
    f = lambda x,y: np.sin(x*np.sin(theta)+y*np.cos(theta))*np.exp(-(x**2+y**2))
    Z = f(X,Y) # Evaluation a function of two variables

    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10,5))
    fig.suptitle('Comparison of two ways of plotting $f(x,y)$')

    # This is useful for images.
    ax1.imshow(Z, cmap='hot', interpolation='nearest')
    ax1.set_ylabel('imshow')

    # This is useful for functions in two variables, i.e. z=f(x,y)
    ax2.pcolormesh(X,Y,Z,cmap='hot', shading='auto')
    ax2.axis('equal')
    ax2.set_ylabel('pcolormesh')
    plt.show()
interact(plot_funct_2_var,theta=(0,2*np.pi,2*np.pi/50))
```

    interactive(children=(FloatSlider(value=0.0, description='theta', max=6.283185307179586, step=0.12566370614359…

    <function __main__.plot_funct_2_var(theta=0)>

```python
def plot_funct_2_var_3D(theta=0):
    x = np.linspace(-2,2,40)
    y = x # Recall this is just a 'view'

    X, Y = np.meshgrid(x,y) # It generates two 2D arrays with with cartesian product of 'x' and 'y'.

    # 'lambda' definition help us to organize better the code
    f = lambda x,y: np.sin(x*np.sin(theta)+y*np.cos(theta))*np.exp(-(x**2+y**2))
    Z = f(X,Y) # Evaluation a function of two variables

    fig, ax = plt.subplots(subplot_kw={"projection": "3d"},figsize=(10,10))

    # Plot the surface.
    surf = ax.plot_surface(X, Y, Z, cmap='hot', linewidth=2, antialiased=True)

    # Add a color bar which maps values to colors.
    fig.colorbar(surf, shrink=0.5, aspect=10)

    plt.show()
interact(plot_funct_2_var_3D,theta=(0,2*np.pi,2*np.pi/50))
```

    interactive(children=(FloatSlider(value=0.0, description='theta', max=6.283185307179586, step=0.12566370614359…

    <function __main__.plot_funct_2_var_3D(theta=0)>

# SciPy, fundamental library for scientific computing
[Back to TOC](#toc)

In this case, we will go straight to the examples. See https://www.scipy.org.

```python
# Numerical Integration
import scipy.integrate as integrate

# Integrating $\int_0^1 \exp(-x^2)\,dx$
result = integrate.quad(lambda x: np.exp(-x**2), 0, 1)
print(result)

# Integrating $\int_0^1 1\,dx=1$
result = integrate.quad(lambda x: 1, 0, 1)
print(result)
```

    (0.7468241328124271, 8.291413475940725e-15)
    (1.0, 1.1102230246251565e-14)

```python
# Minimization
from scipy.optimize import minimize
f = lambda x: np.sin(x[0])*(1+np.cos(x[1]))

x0 = np.array([0, 0])
res = minimize(f, x0, method='nelder-mead', options={'xatol': 1e-8, 'disp': True})
print(res.x)
```

    Optimization terminated successfully.
             Current function value: -2.000000
             Iterations: 88
             Function evaluations: 175
    [-1.57079632e+00 -1.44645246e-08]

```python
# Plotting the previous function
x = np.linspace(-2*np.pi,2*np.pi)
y = x
X,Y = np.meshgrid(x,y)
f = lambda x,y: np.sin(x)*(1+np.cos(y))
Z = f(X,Y)
plt.figure(figsize=(8,8))
plt.pcolormesh(X,Y,Z,shading='auto')
plt.plot(res.x[0],res.x[1],'r.')
plt.colorbar()
plt.title('The red dot is the minimum found numerically')
plt.show()
```

```python
# root finding
from scipy.optimize import root
f = lambda x: x - 0.5 * np.exp(x)
sol = root(f, 0.3)
print(sol)
```

        fjac: array([[-1.]])
         fun: array([-0.30685295])
     message: 'The iteration is not making good progress, as measured by the \n  improvement from the last ten iterations.'
        nfev: 28
         qtf: array([0.30685295])
           r: array([-5.23532303e-05])
      status: 5
     success: False
           x: array([0.69365795])

# SymPy, symbolic mathematics
[Back to TOC](#toc)

In this case, we will go straight to the examples. See https://www.sympy.org/en/index.html.

```python
# Working with algebraic expression versus 'numerical' expressions
sym.sqrt(3)==np.sqrt(3)
```

    False

```python
sym.sqrt(3)>np.sqrt(3)
```

```python
sym.sqrt(3)<np.sqrt(3)
```

```python
# Defining symbolic variables
x = sym.symbols('x')
x
```

```python
y=x**2+sym.cos(x)+(x**2+1)**2
y
```

```python
# Expanding the terms
sym.expand(y)
```

```python
# Computing derivatives
yp=sym.diff(y)
yp
```

```python
# Factorization of expression
sym.factor(yp)
```

```python
# Computing limits
sym.limit((1-sym.cos(x))/x, x, 0)
```

```python
sym.limit((sym.sin(x))/x, x, 0)
```

```python
# Solving algebraic equations
sym.solve(x**2-x+1,x)
```

# Libraries
[Back to TOC](#toc)

### Please make sure you make all of them your BFF!!

* NumPy: https://www.nature.com/articles/s41586-020-2649-2
* Matplotlib: http://matplotlib.org/examples/pylab_examples/simple_plot.html
* Scipy.org: https://www.scipy.org
* SymPy: https://www.sympy.org/en/index.html
* IEEE 754 Floating Point Special Values: https://docs.scipy.org/doc/numpy-1.10.0/user/misc.html

# Acknowledgements
[Back to TOC](#toc)

* _Material originally created by professor Claudio Torres_ (`ctorres@inf.utfsm.cl`). v.1.0. DI UTFSM. August 2021._
* _Update August 2021 - v1.01 - C.Torres_ : Extending jupyter notebook.
* _Update September 2021 - v1.02 - C.Torres_ : Adding missing titles references and reordering libaries section.

```python

```


# Build IRF's Comparison
* After running Dynare (with loop on some parameter) and write the result to Excel with irfs2xlsx.m
* plot few IRFs on the same figure, for comparing alternative parameters
* then read and plot with mathplotlib of Python

```python
! pip install git+https://github.com/cohenimhuji/CohenimPy
! pip install git+https://github.com/cohenimhuji/CohenimPy --upgrade
```

    Collecting git+https://github.com/cohenimhuji/CohenimPy

      Running command git clone --filter=blob:none -q https://github.com/cohenimhuji/CohenimPy 'C:\Users\u30l\AppData\Local\Temp\pip-req-build-7ozahy2p'

      Cloning https://github.com/cohenimhuji/CohenimPy to c:\users\u30l\appdata\local\temp\pip-req-build-7ozahy2p
      Resolved https://github.com/cohenimhuji/CohenimPy to commit 1c125f6aa36f23284d9a22b8e2d08c5b1b06e20b
      Preparing metadata (setup.py): started
      Preparing metadata (setup.py): finished with status 'done'
    Requirement already satisfied: datetime in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (4.3)
    Requirement already satisfied: IPython in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (7.30.0)
    Requirement already satisfied: matplotlib in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (3.5.0)
    Requirement already satisfied: numba in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.54.1)
    Requirement already satisfied: numpy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.20.3)
    Requirement already satisfied: pandas in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.3.4)
    Requirement already satisfied: pathlib in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.0.1)
    Requirement already satisfied: pathos in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.2.7)
    Requirement already satisfied: scipy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.7.3)
    Requirement already satisfied: statsmodels in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.13.1)
    Requirement already satisfied: sympy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.9)
    Requirement already satisfied: zope.interface in c:\users\u30l\anaconda3\lib\site-packages (from datetime->CohenimPy==0.59) (5.4.0)
    Requirement already satisfied: pytz in c:\users\u30l\anaconda3\lib\site-packages (from datetime->CohenimPy==0.59) (2021.3)
    Requirement already satisfied: decorator in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (5.1.0)
    Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (3.0.22)
    Requirement already satisfied: jedi>=0.16 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.17.2)
    Requirement already satisfied: setuptools>=18.5 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (59.2.0)
    Requirement already satisfied: traitlets>=4.2 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (5.1.1)
    Requirement already satisfied: backcall in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.2.0)
    Requirement already satisfied: colorama in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.4.4)
    Requirement already satisfied: matplotlib-inline in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.1.3)
    Requirement already satisfied: pygments in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (2.10.0)
    Requirement already satisfied: pickleshare in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.7.5)
    Requirement already satisfied: pyparsing>=2.2.1 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (3.0.6)
    Requirement already satisfied: python-dateutil>=2.7 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (2.8.2)
    Requirement already satisfied: pillow>=6.2.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (8.4.0)
    Requirement already satisfied: cycler>=0.10 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (0.11.0)
    Requirement already satisfied: packaging>=20.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (21.3)
    Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (1.3.2)
    Requirement already satisfied: fonttools>=4.22.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (4.28.2)
    Requirement already satisfied: llvmlite<0.38,>=0.37.0rc1 in c:\users\u30l\anaconda3\lib\site-packages (from numba->CohenimPy==0.59) (0.37.0)
    Requirement already satisfied: multiprocess>=0.70.11 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.70.11.1)
    Requirement already satisfied: pox>=0.2.9 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.2.9)
    Requirement already satisfied: dill>=0.3.3 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.3.3)
    Requirement already satisfied: ppft>=1.6.6.3 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (1.6.6.3)
    Requirement already satisfied: patsy>=0.5.2 in c:\users\u30l\anaconda3\lib\site-packages (from statsmodels->CohenimPy==0.59) (0.5.2)
    Requirement already satisfied: mpmath>=0.19 in c:\users\u30l\anaconda3\lib\site-packages (from sympy->CohenimPy==0.59) (1.2.1)
    Requirement already satisfied: parso<0.8.0,>=0.7.0 in c:\users\u30l\anaconda3\lib\site-packages (from jedi>=0.16->IPython->CohenimPy==0.59) (0.7.0)
    Requirement already satisfied: six in c:\users\u30l\anaconda3\lib\site-packages (from patsy>=0.5.2->statsmodels->CohenimPy==0.59) (1.16.0)
    Requirement already satisfied: wcwidth in c:\users\u30l\anaconda3\lib\site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->IPython->CohenimPy==0.59) (0.2.5)
    Collecting git+https://github.com/cohenimhuji/CohenimPy

      Running command git clone --filter=blob:none -q https://github.com/cohenimhuji/CohenimPy 'C:\Users\u30l\AppData\Local\Temp\pip-req-build-x74zudet'

      Cloning https://github.com/cohenimhuji/CohenimPy to c:\users\u30l\appdata\local\temp\pip-req-build-x74zudet
      Resolved https://github.com/cohenimhuji/CohenimPy to commit 1c125f6aa36f23284d9a22b8e2d08c5b1b06e20b
      Preparing metadata (setup.py): started
      Preparing metadata (setup.py): finished with status 'done'
    Requirement already satisfied: datetime in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (4.3)
    Requirement already satisfied: IPython in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (7.30.0)
    Requirement already satisfied: matplotlib in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (3.5.0)
    Requirement already satisfied: numba in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.54.1)
    Requirement already satisfied: numpy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.20.3)
    Requirement already satisfied: pandas in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.3.4)
    Requirement already satisfied: pathlib in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.0.1)
    Requirement already satisfied: pathos in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.2.7)
    Requirement already satisfied: scipy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.7.3)
    Requirement already satisfied: statsmodels in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.13.1)
    Requirement already satisfied: sympy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.9)
    Requirement already satisfied: pytz in c:\users\u30l\anaconda3\lib\site-packages (from datetime->CohenimPy==0.59) (2021.3)
    Requirement already satisfied: zope.interface in c:\users\u30l\anaconda3\lib\site-packages (from datetime->CohenimPy==0.59) (5.4.0)
    Requirement already satisfied: setuptools>=18.5 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (59.2.0)
    Requirement already satisfied: decorator in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (5.1.0)
    Requirement already satisfied: matplotlib-inline in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.1.3)
    Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (3.0.22)
    Requirement already satisfied: backcall in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.2.0)
    Requirement already satisfied: colorama in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.4.4)
    Requirement already satisfied: traitlets>=4.2 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (5.1.1)
    Requirement already satisfied: jedi>=0.16 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.17.2)
    Requirement already satisfied: pygments in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (2.10.0)
    Requirement already satisfied: pickleshare in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.7.5)
    Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (1.3.2)
    Requirement already satisfied: cycler>=0.10 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (0.11.0)
    Requirement already satisfied: pillow>=6.2.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (8.4.0)
    Requirement already satisfied: python-dateutil>=2.7 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (2.8.2)
    Requirement already satisfied: fonttools>=4.22.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (4.28.2)
    Requirement already satisfied: packaging>=20.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (21.3)
    Requirement already satisfied: pyparsing>=2.2.1 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (3.0.6)
    Requirement already satisfied: llvmlite<0.38,>=0.37.0rc1 in c:\users\u30l\anaconda3\lib\site-packages (from numba->CohenimPy==0.59) (0.37.0)
    Requirement already satisfied: multiprocess>=0.70.11 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.70.11.1)
    Requirement already satisfied: dill>=0.3.3 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.3.3)
    Requirement already satisfied: pox>=0.2.9 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.2.9)
    Requirement already satisfied: ppft>=1.6.6.3 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (1.6.6.3)
    Requirement already satisfied: patsy>=0.5.2 in c:\users\u30l\anaconda3\lib\site-packages (from statsmodels->CohenimPy==0.59) (0.5.2)
    Requirement already satisfied: mpmath>=0.19 in c:\users\u30l\anaconda3\lib\site-packages (from sympy->CohenimPy==0.59) (1.2.1)
    Requirement already satisfied: parso<0.8.0,>=0.7.0 in c:\users\u30l\anaconda3\lib\site-packages (from jedi>=0.16->IPython->CohenimPy==0.59) (0.7.0)
    Requirement already satisfied: six in c:\users\u30l\anaconda3\lib\site-packages (from patsy>=0.5.2->statsmodels->CohenimPy==0.59) (1.16.0)
    Requirement already satisfied: wcwidth in c:\users\u30l\anaconda3\lib\site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->IPython->CohenimPy==0.59) (0.2.5)

```python
## Using Jupter and Spyder\Atom Together
## https://medium.com/@rrfd/cookiecutter-data-science-organize-your-projects-atom-and-jupyter-2be7862f487e
%load_ext autoreload
%autoreload 2
%matplotlib inline
```

```python
this_project_path = 'c:\\Users\\u30l\\GitHub\\CohenimTools'
this_NB_path = this_project_path+'\\CohenimPy'
import os
current_NB_path = os.path.abspath(os.getcwd())
if this_NB_path !=  current_NB_path:
    print('WARRNING: NOT CORRECT Notebook PATH !')
os.chdir(this_NB_path)
```

```python
## https://medium.com/@rrfd/cookiecutter-data-science-organize-your-projects-atom-and-jupyter-2be7862f487e :
# Link ./src/ with your notebook
# Run this in the project root
os.chdir(this_project_path)
! pip3 install --editable .
os.chdir(this_NB_path)
```

    Obtaining file:///C:/Users/u30l/GitHub/CohenimTools
      Preparing metadata (setup.py): started
      Preparing metadata (setup.py): finished with status 'done'
    Requirement already satisfied: datetime in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (4.3)
    Requirement already satisfied: IPython in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (7.30.0)
    Requirement already satisfied: matplotlib in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (3.5.0)
    Requirement already satisfied: numba in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.54.1)
    Requirement already satisfied: numpy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.20.3)
    Requirement already satisfied: pandas in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.3.4)
    Requirement already satisfied: pathlib in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.0.1)
    Requirement already satisfied: pathos in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.2.7)
    Requirement already satisfied: scipy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.7.3)
    Requirement already satisfied: statsmodels in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (0.13.1)
    Requirement already satisfied: sympy in c:\users\u30l\anaconda3\lib\site-packages (from CohenimPy==0.59) (1.9)
    Requirement already satisfied: zope.interface in c:\users\u30l\anaconda3\lib\site-packages (from datetime->CohenimPy==0.59) (5.4.0)
    Requirement already satisfied: pytz in c:\users\u30l\anaconda3\lib\site-packages (from datetime->CohenimPy==0.59) (2021.3)
    Requirement already satisfied: jedi>=0.16 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.17.2)
    Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (3.0.22)
    Requirement already satisfied: colorama in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.4.4)
    Requirement already satisfied: setuptools>=18.5 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (59.2.0)
    Requirement already satisfied: pickleshare in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.7.5)
    Requirement already satisfied: traitlets>=4.2 in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (5.1.1)
    Requirement already satisfied: backcall in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.2.0)
    Requirement already satisfied: matplotlib-inline in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (0.1.3)
    Requirement already satisfied: pygments in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (2.10.0)
    Requirement already satisfied: decorator in c:\users\u30l\anaconda3\lib\site-packages (from IPython->CohenimPy==0.59) (5.1.0)
    Requirement already satisfied: fonttools>=4.22.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (4.28.2)
    Requirement already satisfied: pillow>=6.2.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (8.4.0)
    Requirement already satisfied: pyparsing>=2.2.1 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (3.0.6)
    Requirement already satisfied: packaging>=20.0 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (21.3)
    Requirement already satisfied: python-dateutil>=2.7 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (2.8.2)
    Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (1.3.2)
    Requirement already satisfied: cycler>=0.10 in c:\users\u30l\anaconda3\lib\site-packages (from matplotlib->CohenimPy==0.59) (0.11.0)
    Requirement already satisfied: llvmlite<0.38,>=0.37.0rc1 in c:\users\u30l\anaconda3\lib\site-packages (from numba->CohenimPy==0.59) (0.37.0)
    Requirement already satisfied: pox>=0.2.9 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.2.9)
    Requirement already satisfied: multiprocess>=0.70.11 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.70.11.1)
    Requirement already satisfied: ppft>=1.6.6.3 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (1.6.6.3)
    Requirement already satisfied: dill>=0.3.3 in c:\users\u30l\anaconda3\lib\site-packages (from pathos->CohenimPy==0.59) (0.3.3)
    Requirement already satisfied: patsy>=0.5.2 in c:\users\u30l\anaconda3\lib\site-packages (from statsmodels->CohenimPy==0.59) (0.5.2)
    Requirement already satisfied: mpmath>=0.19 in c:\users\u30l\anaconda3\lib\site-packages (from sympy->CohenimPy==0.59) (1.2.1)
    Requirement already satisfied: parso<0.8.0,>=0.7.0 in c:\users\u30l\anaconda3\lib\site-packages (from jedi>=0.16->IPython->CohenimPy==0.59) (0.7.0)
    Requirement already satisfied: six in c:\users\u30l\anaconda3\lib\site-packages (from patsy>=0.5.2->statsmodels->CohenimPy==0.59) (1.16.0)
    Requirement already satisfied: wcwidth in c:\users\u30l\anaconda3\lib\site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->IPython->CohenimPy==0.59) (0.2.5)
    Installing collected packages: CohenimPy
      Attempting uninstall: CohenimPy
        Found existing installation: CohenimPy 0.59
        Uninstalling CohenimPy-0.59:
          Successfully uninstalled CohenimPy-0.59
      Running setup.py develop for CohenimPy
    Successfully installed CohenimPy-0.59

```python
import pkg_resources  # part of setuptools
version = pkg_resources.require("CohenimPy")[0].version
print('CohenimPy version is:',version)
```

    CohenimPy version is: 0.5

```python
import numpy as np
import math
#import seaborn as sns  # Machin Learning module
import statsmodels as st
import statsmodels.api as sm
from statsmodels.tsa.seasonal import seasonal_decompose
```

```python
# my functions and oter iports
import pandas as pd
import matplotlib.pyplot as plt

from CohenimPy import ncDate, ncStat
from CohenimPy.ncDate import period
from CohenimPy.ncStat import ols_fit,regs_plot
import CohenimPy.ncPlot as ncPlot
ncPlot.printVersion()
from CohenimPy.ncPlot import multi_save, add_Rec, plot1ax, plot2axs, colors
import CohenimPy.ncPublish as ncPub
```

```python
from IPython.display import set_matplotlib_formats
set_matplotlib_formats('pdf', 'png')
```

    C:\Users\u30l\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: DeprecationWarning: `set_matplotlib_formats` is deprecated since IPython 7.23, directly use `matplotlib_inline.backend_inline.set_matplotlib_formats()`

```python
# %% Set Project Configuraion
# %% shold be after importing: CohenimPy.ncPlot
import config
file_date = config.file_date

# very importent - get Project Main Path
config_project_path = config.project_path
print('src config  project path is:     '+str(config_project_path))

if  this_project_path!= str(config_project_path):
    print('WARRNING: NOT CORRECT PROJECT PATH !')

print('This project path is:     '+str(this_project_path))
print('Current Notebook path is: '+str(current_NB_path))
```

    src config  project path is:     c:\Users\u30l\GitHub\CohenimTools
    This project path is:     c:\Users\u30l\GitHub\CohenimTools
    Current Notebook path is: c:\Users\u30l\GitHub\CohenimTools\CohenimPy

# Read IRFs from Dynare Simulations

Read all IRFs Excels to a list fo df:

```python
path_base = 'G:\\My Drive\\Study\\My_Questions_Ideas_Plans_WP_and_Research\\FXI_w_Yossi Yakhin\\Dynare\\FXI_SOE_dynare_Estimation\\'
path_xlsx = path_base+'xlsx\\'
path_figs =path_base+'figs\\'

sim_list = ['IRF_THETA_2der_0','IRF_THETA_2der_0.1','IRF_THETA_2der_0.2']
Endos = ['Y','pi','i','C','I','EX','IM','w','FX']
Shocks = ['e_i','e_AN']

plotMultiIRFs(sim_list=sim_list,Endos=Endos,Shocks=Shocks,legend_at=0,toSave=True,path_xlsx=path_xlsx,path_figs=path_figs)

```

```python

# for each shock make figure with subplots
for shock in Shocks:
    fig, axs = plt.subplots(nrows=nrows, ncols=nrows,figsize=(12, 12))
    axs=axs.flat
    fig.suptitle('IRF to '+str(shock)+' Shock', fontsize=16)
    #for each endogenious varible make subplot of IRF's for all simulation (parameters)
    for index,endo in enumerate(Endos):
        col = endo+'_'+shock
        toPlot = []
        for df_IRFs in list_df_IRFs:
            toPlot.append(df_IRFs[col])
        df=pd.DataFrame(toPlot)
        df.index=simulations
        df=df.T
        title='$'+df_Endo.at['Tex Name',endo]+'$'
        plot1ax(ax=axs[index],columns=simulations,df=df,title=title, markers='', xlabel=xlabel,ylabel='') # logy=True
        if index==legend_at:
            axs[index].legend(loc=0)
            # Create the legend
        axs[index].ticklabel_format(style='sci',scilimits=(-2,2),useMathText=True)
        #axs[0, 1].remove()
        #tittle
    fig.subplots_adjust(wspace=0.7, hspace=0.6)
    plt.tight_layout()
 #   	mkdir('pdf');
    if toSave:
        try:
            os.mkdir(path_figs)
        except OSError:
            print ("Creation of the directory %s failed" % path_figs)
        else:
            print ("Successfully created the directory %s " % path_figs)
        print('...now saving figures')
        multi_save(name = path_figs+str(shock),pdf=True,png=True)
```

```python
pd.options.display.latex.repr=True
pd.options.display.latex.repr=False
```

```python
%%capture

# Should unmark the seting: "Jupyter: Theme Matplotlib Plots"
# and rerun notebook...
# must Save Notebook before this

#stop - also commant this

os.chdir(this_NB_path)

###################################################
### IMPORTENT : update Notebook file_name below ###
###################################################
## Print Notebook wo code (inputs)

# %notebook -e 2.0-c-ISR_data_exploreation.ipynb
# ! jupyter nbconvert 2.0-c-ISR_data_exploreation.ipynb --no-input --no-prompt --to pdf
! jupyter nbconvert --output-dir='./tex' --no-input --to=latex Compare_Dynare_Irfs_.ipynb --no-prompt
# --template=revtex_nocode.tplx
#--template nc_article

# to try also https://ipypublish.readthedocs.io/en/latest/ :
# ! nbpublish -f latex_ipypublish_all -pdf file_name.ipynb
# also sphinx and jupinx and more....
os.chdir(project_path)
```

```python
#os.chdir(project_path)
#ncPub.NBtoTex(current_NB_path,'2.0-c-ISR_data_exploreation')
```

```python
current_NB_path
this_NB_path

```

    'c:\\Users\\u30l\\GitHub\\CohenimTools\\CohenimPy'


# Tutorial 1: Learning to Predict
**Week 3, Day 4: Reinforcement Learning**

**By Neuromatch Academy**

__Content creators:__ Marcelo Mattar and Eric DeWitt with help from Matt Krause

__Content reviewers:__ Byron Galbraith and Michael Waskom

---

# Tutorial objectives

In this tutorial, we will learn how to estimate state-value functions in a classical conditioning paradigm using Temporal Difference (TD) learning and examine TD-errors at the presentation of the conditioned and unconditioned stimulus (CS and US) under different CS-US contingencies. These exercises will provide you with an understanding of both how reward prediction errors (RPEs) behave in classical conditioning and what we should expect to see if Dopamine represents a "canonical" model-free RPE.

At the end of this tutorial:
* You will learn to use the standard tapped delay line conditioning model
* You will understand how RPEs move to CS
* You will understand how variability in reward size effects RPEs
* You will understand how differences in US-CS timing effect RPEs

```python
# Imports
import numpy as np
import matplotlib.pyplot as plt
```

```python
#@title Figure settings
import ipywidgets as widgets       # interactive display
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
# @title Helper functions
from matplotlib import ticker

def plot_value_function(V, ax=None, show=True):
  """Plot V(s), the value function"""
  if not ax:
    fig, ax = plt.subplots()

  ax.stem(V, use_line_collection=True)
  ax.set_ylabel('Value')
  ax.set_xlabel('State')
  ax.set_title("Value function: $V(s)$")

  if show:
    plt.show()

def plot_tde_trace(TDE, ax=None, show=True, skip=400):
  """Plot the TD Error across trials"""
  if not ax:
    fig, ax = plt.subplots()

  indx = np.arange(0, TDE.shape[1], skip)
  im = ax.imshow(TDE[:,indx])
  positions = ax.get_xticks()
  # Avoid warning when setting string tick labels
  ax.xaxis.set_major_locator(ticker.FixedLocator(positions))
  ax.set_xticklabels([f"{int(skip * x)}" for x in positions])
  ax.set_title('TD-error over learning')
  ax.set_ylabel('State')
  ax.set_xlabel('Iterations')
  ax.figure.colorbar(im)
  if show:
    plt.show()

def learning_summary_plot(V, TDE):
  """Summary plot for Ex1"""
  fig, (ax1, ax2) = plt.subplots(nrows = 2, gridspec_kw={'height_ratios': [1, 2]})

  plot_value_function(V, ax=ax1, show=False)
  plot_tde_trace(TDE, ax=ax2, show=False)
  plt.tight_layout()

def reward_guesser_title_hint(r1, r2):
  """"Provide a mildly obfuscated hint for a demo."""
  if (r1==14 and r2==6) or (r1==6 and r2==14):
    return "Technically correct...(the best kind of correct)"

  if  ~(~(r1+r2) ^ 11) - 1 == (6 | 24): # Don't spoil the fun :-)
    return "Congratulations! You solved it!"

  return "Keep trying...."

#@title Default title text
class ClassicalConditioning:

    def __init__(self, n_steps, reward_magnitude, reward_time):

        # Task variables
        self.n_steps = n_steps
        self.n_actions = 0
        self.cs_time = int(n_steps/4) - 1

        # Reward variables
        self.reward_state = [0,0]
        self.reward_magnitude = None
        self.reward_probability = None
        self.reward_time = None

        self.set_reward(reward_magnitude, reward_time)

        # Time step at which the conditioned stimulus is presented

        # Create a state dictionary
        self._create_state_dictionary()

    def set_reward(self, reward_magnitude, reward_time):

        """
        Determine reward state and magnitude of reward
        """
        if reward_time >= self.n_steps - self.cs_time:
            self.reward_magnitude = 0

        else:
            self.reward_magnitude = reward_magnitude
            self.reward_state = [1, reward_time]

    def get_outcome(self, current_state):

        """
        Determine next state and reward
        """
        # Update state
        if current_state < self.n_steps - 1:
            next_state = current_state + 1
        else:
            next_state = 0

        # Check for reward
        if self.reward_state == self.state_dict[current_state]:
            reward = self.reward_magnitude
        else:
            reward = 0

        return next_state, reward

    def _create_state_dictionary(self):

        """
        This dictionary maps number of time steps/ state identities
        in each episode to some useful state attributes:

        state      - 0 1 2 3 4 5 (cs) 6 7 8 9 10 11 12 ...
        is_delay   - 0 0 0 0 0 0 (cs) 1 1 1 1  1  1  1 ...
        t_in_delay - 0 0 0 0 0 0 (cs) 1 2 3 4  5  6  7 ...
        """
        d = 0

        self.state_dict = {}
        for s in range(self.n_steps):
            if s <= self.cs_time:
                self.state_dict[s] = [0,0]
            else:
                d += 1 # Time in delay
                self.state_dict[s] = [1,d]

class MultiRewardCC(ClassicalConditioning):
  """Classical conditioning paradigm, except that one randomly selected reward,
    magnitude, from a list, is delivered of a single fixed reward."""
  def __init__(self, n_steps, reward_magnitudes, reward_time=None):
    """"Build a multi-reward classical conditioning environment
      Args:
        - nsteps: Maximum number of steps
        - reward_magnitudes: LIST of possible reward magnitudes.
        - reward_time: Single fixed reward time
      Uses numpy global random state.
      """
    super().__init__(n_steps, 1, reward_time)
    self.reward_magnitudes = reward_magnitudes

  def get_outcome(self, current_state):
    next_state, reward = super().get_outcome(current_state)
    if reward:
      reward=np.random.choice(self.reward_magnitudes)
    return next_state, reward

class ProbabilisticCC(ClassicalConditioning):
  """Classical conditioning paradigm, except that rewards are stochastically omitted."""
  def __init__(self, n_steps, reward_magnitude, reward_time=None, p_reward=0.75):
    """"Build a multi-reward classical conditioning environment
      Args:
        - nsteps: Maximum number of steps
        - reward_magnitudes: Reward magnitudes.
        - reward_time: Single fixed reward time.
        - p_reward: probability that reward is actually delivered in rewarding state
      Uses numpy global random state.
      """
    super().__init__(n_steps, reward_magnitude, reward_time)
    self.p_reward = p_reward

  def get_outcome(self, current_state):
    next_state, reward = super().get_outcome(current_state)
    if reward:
      reward*= int(np.random.uniform(size=1)[0] < self.p_reward)
    return next_state, reward
```

---
# Section 1: TD-learning

```python
#@title Video 1: Introduction
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="YoNbc9M92YY", width=854, height=480, fs=1)
print("Video available at https://youtu.be/" + video.id)
video
```

__Environment:__

- The agent experiences the environment in episodes or trials.
- Episodes terminate by transitioning to the inter-trial-interval (ITI) state and they are initiated from the ITI state as well. We clamp the value of the terminal/ITI states to zero.
- The classical conditioning environment is composed of a sequence of states that the agent deterministically transitions through. Starting at State 0, the agent moves to State 1 in the first step, from State 1 to State 2 in the second, and so on.  These states represent time in the tapped delay line representation
- Within each episode, the agent is presented a CS and US (reward).
- The CS is always presented at 1/4 of the total duration of the trial. The US (reward) is then delivered after the CS. The interval between the CS and US is specified by `reward_time`.
- The agent's goal is to learn to predict expected rewards from each state in the trial.

**General concepts**

* Return $G_{t}$: future cumulative reward, which can be written in arecursive form
\begin{align}
G_{t} &= \sum \limits_{k = 0}^{\infty} \gamma^{k} r_{t+k+1} \\
&= r_{t+1} + \gamma G_{t+1}
\end{align}
where $\gamma$ is discount factor that controls the importance of future rewards, and $\gamma \in [0, 1]$. $\gamma$ may also be interpreted as probability of continuing the trajectory.
* Value funtion $V_{\pi}(s_t=s)$: expecation of the return
\begin{align}
V_{\pi}(s_t=s) &= \mathbb{E} [ G_{t}\; | \; s_t=s, a_{t:\infty}\sim\pi] \\
& = \mathbb{E} [ r_{t+1} + \gamma G_{t+1}\; | \; s_t=s, a_{t:\infty}\sim\pi]
\end{align}
With an assumption of **Markov process**, we thus have:
\begin{align}
V_{\pi}(s_t=s) &= \mathbb{E} [ r_{t+1} + \gamma V_{\pi}(s_{t+1})\; | \; s_t=s, a_{t:\infty}\sim\pi] \\
&= \sum_a \pi(a|s) \sum_{r, s'}p(s', r)(r + V_{\pi}(s_{t+1}=s'))
\end{align}

**Temporal difference (TD) learning**

* With a Markovian assumption, we can use $V(s_{t+1})$ as an imperfect proxy for the true value $G_{t+1}$ (Monte Carlo bootstrapping), and thus obtain the generalised equation to calculate TD-error:
\begin{align}
\delta_{t} = r_{t+1} + \gamma V(s_{t+1}) - V(s_{t})
\end{align}

* Value updated by using the learning rate constant $\alpha$:
\begin{align}
V(s_{t}) \leftarrow V(s_{t}) + \alpha \delta_{t}
\end{align}

  (Reference: https://web.stanford.edu/group/pdplab/pdphandbook/handbookch10.html)

__Definitions:__

* TD-error:
\begin{align}
\delta_{t} = r_{t+1} + \gamma V(s_{t+1}) - V(s_{t})
\end{align}

* Value updates:
\begin{align}
V(s_{t}) \leftarrow V(s_{t}) + \alpha \delta_{t}
\end{align}

## Exercise 1: TD-learning with guaranteed rewards

Implement TD-learning to estimate the state-value function in the classical-conditioning world with guaranteed rewards, with a fixed magnitude, at a fixed delay after the conditioned stimulus, CS. Save TD-errors over learning (i.e., over trials) so we can visualize them afterwards.

In order to simulate the effect of the CS, you should only update $V(s_{t})$ during the delay period after CS. This period is indicated by the boolean variable `is_delay`. This can be implemented by multiplying the expression for updating the value function by `is_delay`.

Use the provided code to estimate the value function.

```python
def td_learner(env, n_trials, gamma=0.98, alpha=0.001):
  """ Temporal Difference learning

  Args:
    env (object): the environment to be learned
    n_trials (int): the number of trials to run
    gamma (float): temporal discount factor
    alpha (float): learning rate

  Returns:
    ndarray, ndarray: the value function and temporal difference error arrays
  """
  V = np.zeros(env.n_steps) # Array to store values over states (time)
  TDE = np.zeros((env.n_steps, n_trials)) # Array to store TD errors

  for n in range(n_trials):
    state = 0 # Initial state
    for t in range(env.n_steps):
      # Get next state and next reward
      next_state, reward = env.get_outcome(state)
      # Is the current state in the delay period (after CS)?
      is_delay = env.state_dict[state][0]

      ########################################################################
      ## TODO for students: implement TD error and value function update
      # Fill out function and remove
      raise NotImplementedError("Student excercise: implement TD error and value function update")
      #################################################################################
      # Write an expression to compute the TD-error
      TDE[state, n] = ...

      # Write an expression to update the value function
      V[state] += ...

      # Update state
      state = next_state

  return V, TDE

# Uncomment once the td_learner function is complete
# env = ClassicalConditioning(n_steps=40, reward_magnitude=10, reward_time=10)
# V, TDE = td_learner(env, n_trials=20000)
# learning_summary_plot(V, TDE)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W3D4_ReinforcementLearning/solutions/W3D4_Tutorial1_Solution_6f2c8b56.py)

*Example output:*

## Interactive Demo 1: US to CS Transfer

During classical conditioning, the subject's behavioral response (e.g., salivating) transfers from the unconditioned stimulus (US; like the smell of tasty food) to the conditioned stimulus (CS; like Pavlov ringing his bell) that predicts it. Reward prediction errors play an important role in this process by adjusting the value of states according to their expected, discounted return.

Use the widget below to examine how reward prediction errors change over time. Before training (orange line), only the reward state has high reward prediction error. As training progresses (blue line, slider), the reward prediction errors shift to the conditioned stimulus, where they end up when the trial is complete (green line).

Dopamine neurons, which are thought to carry reward prediction errors _in vivo_, show exactly the same behavior!

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

n_trials = 20000

@widgets.interact
def plot_tde_by_trial(trial = widgets.IntSlider(value=5000, min=0, max=n_trials-1 , step=1, description="Trial #")):
  if 'TDE' not in globals():
    print("Complete Exercise 1 to enable this interactive demo!")
  else:

    fig, ax = plt.subplots()
    ax.axhline(0, color='k') # Use this + basefmt=' ' to keep the legend clean.
    ax.stem(TDE[:, 0], linefmt='C1-', markerfmt='C1d', basefmt=' ',
            label="Before Learning (Trial 0)",
            use_line_collection=True)
    ax.stem(TDE[:, -1], linefmt='C2-', markerfmt='C2s', basefmt=' ',
            label="After Learning (Trial $\infty$)",
            use_line_collection=True)
    ax.stem(TDE[:, trial], linefmt='C0-', markerfmt='C0o', basefmt=' ',
            label=f"Trial {trial}",
            use_line_collection=True)

    ax.set_xlabel("State in trial")
    ax.set_ylabel("TD Error")
    ax.set_title("Temporal Difference Error by Trial")
    ax.legend()
```

## Interactive Demo 2: Learning Rates and Discount Factors

Our TD-learning agent has two parameters that control how it learns: $\alpha$, the learning rate, and $\gamma$, the discount factor. In Exercise 1, we set these parameters to $\alpha=0.001$ and $\gamma=0.98$ for you. Here, you'll investigate how changing these parameters alters the model that TD-learning learns.

Before enabling the interactive demo below, take a moment to think about the functions of these two parameters. $\alpha$ controls the size of the Value function updates produced by each TD-error. In our simple, deterministic world, will this affect the final model we learn? Is a larger $\alpha$ necessarily better in more complex, realistic environments?

The discount rate $\gamma$ applies an exponentially-decaying weight to returns occuring in the future, rather than the present timestep. How does this affect the model we learn? What happens when $\gamma=0$ or $\gamma \geq 1$?

Use the widget to test your hypotheses.

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

@widgets.interact
def plot_summary_alpha_gamma(alpha = widgets.FloatSlider(value=0.0001, min=0.0001, max=0.1, step=0.0001, readout_format='.4f', description="alpha"),
                             gamma = widgets.FloatSlider(value=0.980, min=0, max=1.1, step=0.010, description="gamma")):
  env = ClassicalConditioning(n_steps=40, reward_magnitude=10, reward_time=10)
  try:
    V_params, TDE_params = td_learner(env, n_trials=20000, gamma=gamma, alpha=alpha)
  except NotImplementedError:
    print("Finish Exercise 1 to enable this interactive demo")

  learning_summary_plot(V_params,TDE_params)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W3D4_ReinforcementLearning/solutions/W3D4_Tutorial1_Solution_f6249bde.py)

---
# Section 2: TD-learning with varying reward magnitudes

In the previous exercise, the environment was as simple as possible. On every trial, the CS predicted the same reward, at the same time, with 100% certainty. In the next few exercises, we will make the environment more progressively more complicated and examine the TD-learner's behavior.

## Interactive Demo 3: Match the Value Functions

First, will replace the environment with one that dispenses one of several rewards, chosen at random. Shown below is the final value function $V$ for a TD learner that was trained in an enviroment where the CS predicted a reward of 6 or 14 units; both rewards were equally likely).

Can you find another pair of rewards that cause the agent to learn the same value function? Assume each reward will be dispensed 50% of the time.

Hints:
* Carefully consider the definition of the value function $V$. This can be solved analytically.
* There is no need to change $\alpha$ or $\gamma$.
* Due to the randomness, there may be a small amount of variation.

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

n_trials = 20000
np.random.seed(2020)
rng_state = np.random.get_state()
env = MultiRewardCC(40, [6, 14], reward_time=10)
V_multi, TDE_multi = td_learner(env, n_trials, gamma=0.98, alpha=0.001)

@widgets.interact
def reward_guesser_interaction(r1 = widgets.IntText(value=0, min=0, max=50, description="Reward 1"),
                               r2 = widgets.IntText(value=0, min=0, max=50, description="Reward 2")):
  try:
    env2 = MultiRewardCC(40, [r1, r2], reward_time=10)
    V_guess, _ = td_learner(env2, n_trials, gamma=0.98, alpha=0.001)
    fig, ax = plt.subplots()
    m, l, _ = ax.stem(V_multi, linefmt='y-', markerfmt='yo', basefmt=' ', label="Target",
            use_line_collection=True)
    m.set_markersize(15)
    m.set_markerfacecolor('none')
    l.set_linewidth(4)
    m, _, _ = ax.stem(V_guess, linefmt='r', markerfmt='rx', basefmt=' ', label="Guess",
                      use_line_collection=True)
    m.set_markersize(15)

    ax.set_xlabel("State")
    ax.set_ylabel("Value")
    ax.set_title("Guess V(s)\n" + reward_guesser_title_hint(r1, r2))
    ax.legend()
  except NotImplementedError:
    print("Please finish Exercise 1 first!")
```

## Section 2.1 Examining the TD Error

Run the cell below to plot the TD errors from our multi-reward environment. A new feature appears in this plot? What is it? Why does it happen?

```python
plot_tde_trace(TDE_multi)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W3D4_ReinforcementLearning/solutions/W3D4_Tutorial1_Solution_dea47c05.py)

---
# Section 3: TD-learning with probabilistic rewards

In this environment, we'll return to delivering a single reward of ten units. However, it will be delivered intermittently: on 20 percent of trials, the CS will be shown but the agent will not receive the usual reward; the remaining 80% will proceed as usual.

 Run the cell below to simulate. How does this compare with the previous experiment?

Earlier in the notebook, we saw that changing $\alpha$ had little effect on learning in a deterministic environment. What happens if you set it to an large value, like 1, in this noisier scenario? Does it seem like it will _ever_ converge?

```python
np.random.set_state(rng_state) # Resynchronize everyone's notebooks
n_trials = 20000
try:
  env = ProbabilisticCC(n_steps=40, reward_magnitude=10, reward_time=10,
                        p_reward=0.8)
  V_stochastic, TDE_stochastic = td_learner(env, n_trials*2, alpha=1)
  learning_summary_plot(V_stochastic, TDE_stochastic)
except NotImplementedError:
  print("Please finish Exercise 1 first")
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W3D4_ReinforcementLearning/solutions/W3D4_Tutorial1_Solution_98f98dd2.py)

---
# Summary

In this notebook, we have developed a simple TD Learner and examined how its state representations and reward prediction errors evolve during training. By manipualting its environment and parameters ($\alpha$, $\gamma$), you developed an intuition for how it behaves.

This simple model closely resembles the behavior of subjects undergoing classical conditioning tasks and the dopamine neurons that may underlie that behavior. You may have implemented TD-reset or used the model to recreate a common experimental error. The update rule used here has been extensively studied for [more than 70 years](https://www.pnas.org/content/108/Supplement_3/15647) as a possible explanation for artificial and biological learning.

However, you may have noticed that something is missing from this notebook. We carefully calculated the value of each state, but did not use it to actually do anything. Using values to plan _**Actions**_ is coming up next!

# Bonus

## Exercise 2: Removing the CS

In Exercise 1, you (should have) included a term that depends on the conditioned stimulus. Remove it and see what happens. Do you understand why?
This phenomena often fools people attempting to train animals--beware!

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W3D4_ReinforcementLearning/solutions/W3D4_Tutorial1_Solution_3b5e09e0.py)


# Hydrology --- introduction
> Water cycle, major processes

- toc: true
- badges: true
- comments: false
- categories: [jupyter]

## How much water is there? Where?

> youtube: https://youtu.be/2ObMyytxLz8

## The water cycle

## Global water distribution

| Water source                         | Volume (km$^3$) | % of freshwater | % of total water |
|:--------------------------------------|-----------------------------------|-----------------------|------------------------|
| Oceans, Seas, & Bays                 | 1,338,000,000                     | --                    | 96.54                  |
| Ice caps, Glaciers,
& Permanent Snow | 24,064,000                        | 68.7                  | 1.74                   |
| Groundwater                          | 23,400,000                        | --                    | 1.69                   |
|$\quad$Fresh                            | 10,530,000                        | 30.1                  |   0.76                 |
|$\quad$Saline                           | 12,870,000                        | --                    |   0.93                 |
| Soil Moisture                        | 16,500                            | 0.05                  | 0.001                  |
| Ground Ice
& Permafrost              | 300,000                           | 0.86                  | 0.022                  |
| Lakes                                | 176,400                           | --                    | 0.013                  |
|$\quad$Fresh                            | 91,000                            | 0.26                  | 0.007                  |
|$\quad$Saline                           | 85,400                            | --                    | 0.006                  |
| Atmosphere                           | 12,900                            | 0.04                  | 0.001                  |
| Swamp Water                          | 11,470                            | 0.03                  | 0.0008                 |
| Rivers                               | 2,120                             | 0.006                 | 0.0002                 |
| Biological Water                     | 1,120                             | 0.003                 | 0.0001                 |

\* (Percents are rounded, so will not add to 100)
[https://www.usgs.gov/special-topic/water-science-school/science/fundamentals-water-cycle](https://www.usgs.gov/special-topic/water-science-school/science/fundamentals-water-cycle)

## Energy drives the hydrologic cycle

> A key aspect of the hydrologic cycle is the fact that it is driven by energy inputs (primarily from the sun). At the global scale, the system is essentially closed with respect to water; negligible water is entering or leaving the system. In other words, there is no external forcing in terms of a water flux. Systems with no external forcing will generally eventually come to an equilibrium state. So what makes the hydrologic cycle so dynamic? The solar radiative energy input, which is external to the system, drives the hydrologic cycle. Averaged over the globe, 342 W m$^{-2}$ of solar radiative energy is being continuously input to the system at the top of the atmosphere. This energy input must be dissipated, and this is done, to a large extent, via the hydrologic cycle. Due to this fact, the study of hydrology is not isolated to the study of water storage and movement, but also must often include study of energy storage and movements.

Margulis, 2017, "Introduction to Hydrology"

## Components of the water cycle

### Water storage in oceans

### Evaporation / Sublimation

Evaporation $\longrightarrow$ cooling

### Evapotranspiration

### Water storage in the atmosphere

Cumulonimbus cloud over Africa

Picture of cumulonimbus taken from the International Space Station, over western Africa near the Senegal-Mali border.

If all of the water in the atmosphere rained down at once, it would only cover the globe to a depth of 2.5 centimeters.
$$
\begin{align}
\text{amount of water in the atmosphere} & \qquad V = 12\, 900\, \text{km}^3 \\
\text{surface of Earth} & \qquad S = 4 \pi R^2;\quad R=6371\,\text{km}\\
& \qquad V = S \times h \\
\text{height} & \qquad h = \frac{V}{S} \simeq 2.5\,\text{cm}
\end{align}
$$

Try to calculate this yourself, and click on the button below to check how to do it.

```python
#collapse-hide
# amount of water in the atmosphere
V = 12900 # km^3
# Earth's radius
R = 6371 # km
# surface of Earth = 4 pi Rˆ2
S = 4 * 3.141592 * R**2
# Volume: V = S * h, therefore
# height
h = V / S # in km
h_cm = h * 1e5 # in cm
print(f"The height would be ~ {h_cm:.1f} cm")
```

    The height would be ~ 2.5 cm

### Condensation

### Precipitation

|                   |   Intensity   (cm/h)  |   Median diameter   (mm)  |   Velocity of fall   (m/s)  |   Drops s$^{-1}$ m$^{-2}$  |
|:-------------------|--------------------------|------------------------------------|---------------------------------------|----------------------------------------|
|   Fog             |   0.013                  |   0.01                             |   0.003                               |   67,425,000                           |
|   Mist            |   0.005                  |   0.1                              |   0.21                                |   27,000                               |
|   Drizzle         |   0.025                  |   0.96                             |   4.1                                 |   151                                  |
|   Light rain      |   0.10                   |   1.24                             |   4.8                                 |   280                                  |
|   Moderate rain   |   0.38                   |   1.60                             |   5.7                                 |   495                                  |
|   Heavy rain      |   1.52                   |   2.05                             |   6.7                                 |   495                                  |
|   Excessive rain  |   4.06                   |   2.40                             |   7.3                                 |   818                                  |
|   Cloudburst      |   10.2                   |   2.85                             |   7.9                                 |   1,220                                |

Source: https://www.usgs.gov/special-topic/water-science-school/science/precipitation-and-water-cycle

### Water storage in ice and snow

### Snowmelt runoff to streams

### Surface runoff

### Streamflow

The Mississippi river basin is very large

The Amazon river basin is **Huge**

### Lakes and rivers

Lake Malawi

### Infiltration

### Groundwater storage

Center Pivot irrigation in Nebraska taps the Ogallala Aquifer.

### Groundwater flow and discharge

### Spring

Ein Gedi

Thousand Springs, Idaho

```python

```


# Distillation Prelab
Lee Runyon
CBE Lab I
### Base Equations
mass balances, energy balances, and special equations
#### Molar Flow Balance
$n_{feed}=n_{distillate}n_{bottoms}$

```python
import numpy as np

def bubble_pt_RaoultsLaw(T):
    (VP_a)=np.exp(A_a-B_a/(T+C_a))
    (VP_b)=np.exp(A_b-B_b/(T+C_b))
    P_bubble=za*VP_a+zb*VP_b

```

ref: https://en.wikibooks.org/wiki/Introduction_to_Chemical_Engineering_Processes/Vapor-Liquid_equilibrium
ref: http://www.eng.auburn.edu/~drmills/mans486/Diffusion%20Tube/Antoine_coefficient_table.PDF

```python
A_ipa=4.57795
B_ipa=1221.423
C_ipa=-87.474

A_water=4.6543
B_water=1435.264
C_water=-64.848

#bubble point equation combined
import sympy as sympy
import scipy as scipy
P,PA,PB,yA,yB=sympy.Symbols('P PA PB yA yB')
def VLE(P,PA,PB)
    return P-ya*
for T in range(255,375):
    P=0.9
    PA=10**(Aipa-Bipa/(Cipa+T))#VP_ipa
    PB=10**(Awater-Bwater/(Cwater+T)#VP_h20
    sol,=fsolve(lamda:(yA / PA*(T) + yB / PB*(T)  - 1/P),[yA,yb])
```

```python
#124 C3H8O isopropyl-alcohol 8.11820 1580.920 219.620
from scipy.optimize import fsolve
import numpy as np

sol, = fsolve(lambda x: func(x, 3.2), 3)
print sol
```