# Simple Fortran kernel for Jupyter

## Switching compilers
`%comipler: compiler-name .file-extention`

### default compiler is intel fortran

```fortran
use, intrinsic :: iso_fortran_env
print *, compiler_version()
print *, compiler_options()
end
```

     GCC version 8.3.0
     -mtune=generic -march=x86-64 -Wall

### gfortran

```fortran
%compiler: gfortran
use, intrinsic :: iso_fortran_env
print *, compiler_version()
print *, compiler_options()
end
```

     GCC version 7.4.0
     -mtune=generic -march=x86-64 -Wall

```fortran
%compiler: gfortran-8
use, intrinsic :: iso_fortran_env
print *, compiler_version()
print *, compiler_options()
end
```

     GCC version 8.3.0
     -mtune=generic -march=x86-64 -Wall

## Change file extention
default file extention is .f90 (free format)

for FIXED FORMAT source use .f/.for

```fortran
%compiler: gfortran-8 .f
      PARAMETER(N = 100)
      OPEN(9, FILE = 'SINCOS.XY')
      PI = 4.0 * ATAN(1.0)
      DO 10 I = 0, N
        X = 2.0 * I * PI / N
        WRITE(9, *) X, SIN(X), COS(X)
   10 CONTINUE
      STOP 'NORMAL TERMINATION'
      END
```

## Write text file
- %writefile *fn* [-a]
- %%writefile *fn* [-a]
- -a append

```fortran
%writefile: Laurence_Sterne.txt
“O Cupid! Cupid! prince of Gods and men.”
```

    write to file:Laurence_Sterne.txt

```fortran
%%writefile: Laurence_Sterne.txt -a
Laurence Sterne. (1713–1768).  A Sentimental Journey through France and Italy.
```

    append to file:Laurence_Sterne.txt

## Compile C program
- %compiler: gcc .c

```fortran
%compiler: gcc .c
#include <stdio.h>
int main(void) {
    printf("Hello world\n");
    return 0;
}
```

    Hello world

## Compile C++ program
- %compiler: g++ .c++

```fortran
%compiler: g++ .c++
#include <iostream>

using namespace std;

int main() {
    std::cout << "Hello World!" << std::endl;
    return 0;
}
```

    Hello World!

## Separate module compilation
`%module: object-file-name`

```fortran
%module: m
%fcflags: -O2
!
! The Sieve of Eratosthenes
!
module m_eratos
    implicit none
contains
    integer function nprimes(n)
        integer, intent(in) :: n
        logical, allocatable :: tab(:)
        integer :: i
        allocate(tab(2:n), source = .true.)
        do i = 2, int(sqrt(real(n)))
            if (tab(i)) tab(i**2::i) = .false.
        end do
        nprimes = count(tab)
    end function nprimes
end module m_eratos
```

    [gfort kernel] module objects created successfully: m.o

## Compiler options
`%fcflags: arg1 arg2 ...`
`%ldflags: arg1 arg2 ...`

gfortran *fcflags* source.f90 *ldflags*

to use module, put object-file-names to *fcflags*

```fortran
%fcflags: m.o
program test
    use m_eratos
    implicit none
    integer :: i
    do i = 1, 8
       write(*, *) i, 10**i, nprimes(10**i), nint(10**i / log(10.0**i))
       write(9, *) i, 10**i, nprimes(10**i), nint(10**i / log(10.0**i))
    end do
end program test
```

               1          10           4           4
               2         100          25          22
               3        1000         168         145
               4       10000        1229        1086
               5      100000        9592        8686
               6     1000000       78498       72382
               7    10000000      664579      620421
               8   100000000     5761455     5428681

# Matplotlib Ready
`%fig: args`

```python
import matplotlib.pyplot as plt
import numpy as np
_fig = plt.figure( args )
```

> ...

> ( user inputs are executed by Python's exec command )

> ...

```python
plt.show()
```

### note
I don't know why, but `plt.figure()` doesn't work properlly with `exec/eval` commands. So it is hard coded in the kernel.

### plot number of primes $\pi(n)$
$$
\large\pi(n)\simeq{n \over {\log(n)}}
$$

```fortran
%fcflags: m.o -O2
program prime
    use m_eratos
    implicit none
    integer :: i
    open(9, file = 'primes.csv')
    write(9, '(5g0)') 'n', ',', 'primes', ',', 'n/log(n)'
    do i = 2, 501
      ! write(*, *) i, ',', nprimes(i)
       write(9, '(5g0)') i, ',', nprimes(i), ',', i / log(real(i))
    end do
    stop 'normal end'
end program prime
```

    STOP normal end

```fortran
%fig:
x, y1, y2 = np.loadtxt('primes.csv', unpack = True, delimiter=',', skiprows=1)
plt.grid(which='both')
plt.plot(x, y1)
plt.plot(x, y2)
plt.title('Prime numbers', fontsize = 25)
plt.xlabel('n', fontsize = 20)
plt.ylabel('number of primes', fontsize = 20)
```

```fortran
%fig:
x, y1, y2, y3 = np.loadtxt('fort.9', unpack = True)
plt.yscale('log')
plt.grid(which='both')
plt.plot(x, y2)
plt.plot(x, y3)
plt.title('Prime number theorem', fontsize = 25)
plt.xlabel('10^x', fontsize = 20)
plt.ylabel('number of primes', fontsize = 20)
```

### subplot
sin cos curves

```fortran
%fig:figsize=(6, 2)
x, y1, y2 = np.loadtxt('SINCOS.XY', unpack = True)
plt.subplot(1, 2, 1)
plt.grid(which='both')
plt.plot(x, y1)
plt.subplot(1, 2, 2)
plt.grid(which='both')
plt.plot(x, y2)
```

### Pandas examples
DataFrame is AxesSubplot object.

Internal variable \_fig is accessible and assignable (e.g. \_fig = rhs).
However multiple assignment is not possble.

```fortran
%fig:
import pandas as pd
ax = _fig.add_subplot(1, 1, 1)

x = np.arange(100)
df = pd.DataFrame({'x': x, 'sin(x)': np.sin(x / 5)})
df.plot(ax=ax)
```

```fortran
%fig:
from pandas import *
fig, axes = plt.subplots(nrows=2, ncols=1, figsize=(9, 6))
_fig = fig
# Multiple assignment is not allowed for _fig.

df = DataFrame(np.random.randn(1000, 4))
df.plot(ax=axes[0], legend=False)
df.plot(ax=axes[1], legend=False, kind='hist')
```

## BMP example

```fortran
%module: BMP
    module m_bmp
      use, intrinsic :: iso_fortran_env
      implicit none
      type :: t_bmp_file_header
        sequence
        integer(int16) :: bfType = transfer('BM', 0_int16) ! BitMap
        integer(int32) :: bfSize          ! file size in bytes
        integer(int16) :: bfReserved1 = 0 ! always 0
        integer(int16) :: bfReserved2 = 0 ! always 0
        integer(int32) :: bfOffBits
      end type t_bmp_file_header
      !
      type :: t_bmp_info_header
        sequence
        integer(int32) :: biSize          = Z'28' ! size of bmp_info_header ; 40bytes
        integer(int32) :: biWidth
        integer(int32) :: biHeight
        integer(int16) :: biPlanes        = 1 ! always 1
        integer(int16) :: biBitCount
        integer(int32) :: biCompression   = 0 ! 0:nocompression, 1:8bitRLE, 2:4bitRLE, 3:bitfield
        integer(int32) :: biSizeImage
        integer(int32) :: biXPelsPerMeter = 3780 ! 96dpi
        integer(int32) :: biYPelsPerMeter = 3780 ! 96dpi
        integer(int32) :: biClrUsed       = 0
        integer(int32) :: biClrImportant  = 0
      end type t_bmp_info_header
      !
      type :: t_rgb
        sequence
        integer(int8) :: ib, ig, ir
      end type t_rgb
    contains
      subroutine wr(bmp, fn)
        type(t_rgb), intent(in) :: bmp(:, :)
        character(len = *), intent(in) :: fn
        type(t_bmp_file_header) :: bmp_file_header
        type(t_bmp_info_header) :: bmp_info_header
        integer :: i, nx, ny
        nx = size(bmp, 1)
        ny = size(bmp, 2)
        bmp_file_header%bfSize      = 14 + 40 + 0 + (3 * nx + mod(ny, 4)) * ny
        bmp_file_header%bfOffBits   = 14 + 40
        bmp_info_header%biWidth     = nx
        bmp_info_header%biHeight    = ny
        bmp_info_header%biBitCount  = 24
        bmp_info_header%biSizeImage = (3 * nx + mod(ny, 4)) * ny
        open(9, file = fn // '.bmp', access = 'stream', status = 'unknown')
        write(9) bmp_file_header
        write(9) bmp_info_header
        write(9) (bmp(:, i), repeat(achar(0), mod(nx, 4)), i = 1, ny)
        close(9)
      end subroutine wr
    end module m_bmp
```

    [gfort kernel] module objects created successfully: BMP.o

```fortran
%fcflags: BMP.o
    module m_mandel
      use, intrinsic :: iso_fortran_env
      implicit none
      integer, parameter :: kd = real64
      integer, parameter :: maxiter = 255
    contains
      pure elemental integer function mandel(c)
        complex(kd), intent(in) :: c
        complex(kd) :: z
        z = c
        do mandel = maxiter, 1, -1
          if (abs(z) > 2.0_kd) exit
          z = z**2 + c
        end do
      end function mandel
    end module m_mandel

    program mandelbrot
      use m_bmp
      use m_mandel
      implicit none
      complex(kd), allocatable :: c(:, :)
      type(t_rgb), allocatable :: bmp(:, :)
      integer  :: ix, iy, nx, ny
      real(kd) :: xmin, ymin, xmax, ymax
      xmin = 0.15_kd
      xmax = 0.25_kd
      ymin = 0.50_kd
      ymax = 0.60_kd
      nx = 320
      ny = 320
      allocate( c(nx, ny) )
      forall (ix = 1:nx, iy = 1:ny) c(ix, iy) = cmplx(pos(ix, nx, xmin, xmax), pos(iy, ny, ymin, ymax))
      bmp = to_rgb( mandel(c) )
      call wr(bmp, 'mandel')
      stop 'normal end'
    contains
      pure real(kd) function pos(i, n, rmin, rmax)
        integer , intent(in) :: i, n
        real(kd), intent(in) :: rmin, rmax
        pos = (rmax - rmin) / (n - 1) * i + rmin
      end function pos
      !
      pure elemental type(t_rgb) function to_rgb(k)
        integer, intent(in) :: k
        to_rgb = t_rgb(10 * k,  k, 5 * k)
      end function to_rgb
    end program mandelbrot
```

    STOP normal end

## PIL

```fortran
%fig:
from PIL import Image
im1 = Image.open('mandel.bmp')
plt.imshow(im1)
```

# Display Images   (MIME: image/xxx)
`%image: fn1 fn2 ...`

```fortran
%image: mandel.bmp
```

## Python ; limited functionality (exec line by line)
- %py:
- **print** *text* -> eval(*text*)    ; not pyhon2 nor python3 **print**

```fortran
%py:
from sympy import *
x = Symbol('x')
bernoulli_gen = x / (exp(x) - 1)
print bernoulli_gen

bernoulli_ser = series(bernoulli_gen , x, 0, 10).removeO()
btmp = Poly(bernoulli_ser, x).coeffs()
bernoulli = [0, 0, 0, 0, 0, 0]

indx = [0, 1, 2, 4, 6, 8]
for i in range(6): bernoulli[i] = factorial(indx[i]) * btmp[5 - i]
print bernoulli
```

    x/(exp(x) - 1)
    [1, -1/2, 1/6, -1/30, 1/42, -1/30]

### Bernoulli numbers
$$
B_0=1, B_1=-{1\over2}, B_2={1\over6}, B_4=-{1\over30}, B_6={1\over42}, B_8=-{1\over30}
$$


# Find worst cases

\begin{equation}
\begin{array}{rl}
    \mathcal{F}_L =& \dfrac{4 K I H r}{Q_{in}(1+f)}\\
    u_{c} =& \dfrac{-KI\mathcal{F}_L}{\theta \left(\mathcal{F}_L+1\right)}\\
    \tau =& -\dfrac{r}{|u_{c}|}\\
    C_{\tau,{\rm decay}}=& C_0 \exp{\left(-\lambda \tau \right)}\\
    C_{\tau,{\rm filtr}}=& C_0 \exp{\left(-k_{\rm att} \tau \right)}\\
    C_{\tau,{\rm dilut}} =& C_{in} \left( \dfrac{Q_{in}}{u_c \Delta y \Delta z} \right)\\
    C_{\tau,{\rm both}} =& \dfrac{C_{\rm in}Q_{\rm in}}{u_c \Delta y H \theta} \exp{\left(-\lambda\dfrac{r}{|u_c|}\right)}
\end{array}
\end{equation}

```python
%reset -f
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
from os import system
import os
from matplotlib.gridspec import GridSpec
from drawStuff import *

import jupypft.attachmentRateCFT as CFT
import jupypft.plotBTC as BTC

''' GLOBAL CONSTANTS '''
PI = 3.141592
THETA = 0.35
```

```python
def flowNumber():
    return (4.0*K*I*H*r) / (Qin*(1+f))

def uChar():
    '''Interstitial water velocity'''
    return -(K*I*flowNumber())/(THETA*(flowNumber() + 1))

def tChar():
    return -r/uChar()

def cDecay():
    return C0 * np.exp(-decayRate * tChar())

def cAttach():
    return C0 * np.exp(-attchRate * tChar())

def cDilut():
    return (C0 * Qin) / (-uChar() * delY * delZ * THETA)

def cBoth():
    return (C0 * Qin) / (-uChar() * delY * delZ * THETA) * np.exp(-decayRate * tChar())

def cTrice():
    return (C0 * Qin) / (-uChar() * delY * delZ * THETA) * np.exp(-(decayRate+attchRate) * tChar())

def findSweet():
    deltaConc = np.abs(cBoth() - np.max(cBoth()))
    return np.argmin(deltaConc)
```

```python
K = 10**-2
Qin = 0.24/86400
f = 10
H = 20
r = 40
I = 0.001
C0 = 1.0
qabs = np.abs(uChar()*THETA)

kattDict = dict(
    dp = 1.0E-7,
    dc = 2.0E-3,
    q = qabs,
    theta = THETA,
    visco = 0.0008891,
    rho_f = 999.79,
    rho_p = 1050.0,
    A = 5.0E-21,
    T = 10. + 273.15,
    alpha = 0.01)

decayRate = 3.5353E-06
attchRate,_ = CFT.attachmentRate(**kattDict)
delY,delZ = 1.35,H
```

```python
print("Nondim Flow  = {:.2E}".format(flowNumber()))
print("Charac. Vel  = {:.2E} m/s".format(uChar()))
print("Charac. time = {:.2E} s".format(tChar()))
```

    Nondim Flow  = 1.05E+03
    Charac. Vel  = -2.85E-05 m/s
    Charac. time = 1.40E+06 s

```python
print("Rel concenc. due decay = {:.2E}".format(cDecay()))
print("Rel conc. due dilution = {:.2E}".format(cDilut()))
print("Rel conc. due attachmt = {:.2E}".format(cAttach()))
print("Rel conc. due both eff = {:.2E}".format(cBoth()))
print("Rel conc. due three ef = {:.2E}".format(cTrice()))
```

    Rel concenc. due decay = 7.05E-03
    Rel conc. due dilution = 1.03E-02
    Rel conc. due attachmt = 4.42E-05
    Rel conc. due both eff = 7.26E-05
    Rel conc. due three ef = 3.21E-09

## Hydraulic conductivity

```python
K = 10**-2
H = 20
f = 10
C0 = 1.0
decayRate = 3.5353E-06

Qin_array = np.array([0.24,1.0,10.0,100.])/86400.
rarray = np.array([5.,10.,40.,100.])
Iarray = 10**np.linspace(-5,0,num=100)

Ci = np.zeros([len(rarray),len(Qin_array)])
Ii = np.zeros([len(rarray),len(Qin_array)])
FLi = np.zeros([len(rarray),len(Qin_array)])

for qi,Qin in enumerate(Qin_array):
    for ri,r in enumerate(rarray):
        i = findSweet()

        worstC = -np.log10(cBoth()[i])
        worstGradient = Iarray[i]
        worstFlowNumber = flowNumber()[i]

        Ci[ri,qi] = worstC
        Ii[ri,qi] = worstGradient
        FLi[ri,qi] = worstFlowNumber

myLabels={"Title": { 0: r"$\bf{-\log (C_{\tau}/C_0)}$",
                     1: r"$\bf{I}$ (%)",
                     2: r"$\log(\mathcal{F}_L)$"},
          "Y": "Injection flow rate\n$\\bf{Q_{in}}$ (m³/d)",
          "X": "Setback distance\n$\\bf{r}$ (m)"}

threeHeatplots(data={"I":Ii.T,"C":Ci.T,"FL":FLi.T},\
               ylabel=rarray,xlabel=np.round(Qin_array*86400,decimals=2),myLabels=myLabels);
```
K = 10**-2
H = 20
f = 10
C0 = 1.0
decayRate = 1.119E-5

Qin_array = np.array([0.24,1.0,10.0,100.])/86400.
rarray = np.array([5.,10.,40.,100.])
Iarray = 10**np.linspace(-5,0,num=100)

Ci = np.zeros([len(rarray),len(Qin_array)])
Ii = np.zeros([len(rarray),len(Qin_array)])
FLi = np.zeros([len(rarray),len(Qin_array)])

for qi,Qin in enumerate(Qin_array):
    for ri,r in enumerate(rarray):
        i = findSweet()

        worstC = -np.log10(cBoth()[i])
        worstGradient = Iarray[i]
        worstFlowNumber = flowNumber()[i]

        Ci[ri,qi] = worstC
        Ii[ri,qi] = worstGradient
        FLi[ri,qi] = worstFlowNumber

myLabels={"Title": { 0: r"$\bf{-\log (C_{\tau}/C_0)}$",
                     1: r"$\bf{I}$ (%)",
                     2: r"$\log(\mathcal{F}_L)$"},
          "Y": "Injection flow rate\n$\\bf{Q_{in}}$ (m³/d)",
          "X": "Setback distance\n$\\bf{r}$ (m)"}

threeHeatplots(data={"I":Ii.T,"C":Ci.T,"FL":FLi.T},\
               ylabel=rarray,xlabel=np.round(Qin_array*86400,decimals=2),myLabels=myLabels);
## PLOTRAN SIMULATION RESULTS

```python
minI = np.array([0.00046667, 0.0013    , 0.0048    , 0.012     , 0.00046667,
       0.0013    , 0.0048    , 0.012     , 0.00053333, 0.0013    ,
       0.0048    , 0.012     , 0.00056667, 0.0021    , 0.0053    ,
       0.012     ])

minC = np.array([2.2514572 , 2.62298917, 3.14213329, 3.51421485, 1.64182175,
       2.00913676, 2.52461269, 2.89537637, 0.74130696, 1.0754177 ,
       1.55071976, 1.90646243, 0.18705258, 0.39222131, 0.73428991,
       1.00387133])
```

```python
Qin_array = np.array([0.24,1.0,10.0,100.])/86400.
rarray = np.array([5.,10.,40.,100.])
Iarray = 10**np.linspace(-5,0,num=100)

Ci = np.zeros([len(rarray),len(Qin_array)])
Ii = np.zeros([len(rarray),len(Qin_array)])
FLi = np.zeros([len(rarray),len(Qin_array)])

i = 0
for qi,Qin in enumerate(Qin_array):
    for ri,r in enumerate(rarray):
        worstC = minC[i]
        worstGradient = minI[i]

        Ci[ri,qi] = worstC
        Ii[ri,qi] = worstGradient

        i += 1

myLabels={"Title": { 0: r"$\bf{-\log (C_{\tau}/C_0)}$",
                     1: r"$\bf{I}$ (%)",
                     2: r"$\log(\mathcal{F}_L)$"},
          "Y": "Injection flow rate\n$\\bf{Q_{in}}$ (m³/d)",
          "X": "Setback distance\n$\\bf{r}$ (m)"}

threeHeatplots(data={"I":Ii.T,"C":Ci.T,"FL":Ii.T},\
               ylabel=rarray,xlabel=np.round(Qin_array*86400,decimals=2),myLabels=myLabels);
```

```python

```


## Reinforcement Learning Tutorial -1: Q Learning

#### MD Muhaimin Rahman
sezan92[at]gmail[dot]com

Q learning , can be said one of the most famous -and kind of intuitive- of all Reinforcement learning algorithms. In fact ,the recent all algorithms using Deep learning , are based on the Q learning algorithms. So, to work on recent algorithms, one must have a good idea on Q learning.

### Intuition

First , start with an Intuition. Lets assume , you are in a maze

Okay okay! I admit, it is not a maze. just a house with 5 rooms. And I got it from, this [link](http://mnemstudio.org/path-finding-q-learning-tutorial.htm) . Your goal is to get out of this place, no matter where you are. But you dont know - atleast pretend to - how to get there! After wondering about the map, you stumbled upon a mysterious letter with a lot of numbers in the room.

The matrix has 6 columns and 6 rows. What you will have to do , is to go to the room with highest value. Suppose, you are in room number 2. Then , you will have to move to room number 3 . Then you get out! Look at the picture again! You can try with every state, you are guaranteed to get out of the house, using this matrix! .

In the world of RL, every room is called a ```state```, movement from one state to another is called ```action```. Our game has a very ***JARGONISH*** name, ```Markov Decision Process``` . Maybe they invented this name to freak everybody out. But in short, this process means, your action from current state never depends on previous state. Practically such processes are impossible, but it helps to simplify problems

 Now the question is , how can we get this ?

- First , initialize the matrix as Zeros

- Then we will apply the Q learning update equation

\begin{equation}
Q(s_t,a) = Q(s_t,a) + \alpha (Q'(s_{t+1},a)-Q(s_t,a))
\end{equation}

Here, $s_t$ is state at time $t$ , $s_{t+1}$ means the next state, $a$ is action. Q(s_t,a_t) means Q matrix value for state $s_t$ and action $a_t$ , $Q'(s_{t+1},a)$ means target Q value with state $s_{t+1}$ and the ***BEST ACTION*** for next state. Here $\alpha $ is learning rate}

Before we proceed, let me ask you, does this equation ring a bell ? I mean, haven't you seen a similar equation ?

Yeah, you got it , it is similar to Gradient descent Equation. If you dont know Gradient descent equation, I am sorry, you wont be able to get the future tutorials. So I suggest you get the basic and working Idea of Neural Networks and Gradient descent algorithms

Now ,How can we get $Q'(s_{t+1},a)$ ?

Using Bellman Equation

\begin{equation}
Q'(s_{t+1},a) = r+ \gamma max(Q(s_{t+1},a_t))
\end{equation}

Here, $r$ is reward we get-if we can get - from one state to another state.

It means the target $Q$ value for every state and action is the sum of reward with that state and action, and the maximum $Q$ value of next state multiplied with discount factor $\gamma$. You can get the idea in the later section

***Where did this equation came from ? ***

Okay chill! let's start from the game again ! So suppose , every room has reward, $R_t,R_{t+1},R_{t+2},R_{t+3},R_{t+4},R_{t+5}$.. So obviously , the value of a state will be the expected cumulative reward

\begin{equation}
Q(s,a) = R_t + R_{t+1} + R_{t+2}+ R_{t+3}+ R_{t+4}+ R_{t+5}
\end{equation}

Suppose, someone comes here, and says, He wants to give more weight to sooner rewards than later rewards. What should we do ? We will introduce, discount factor, $\gamma$ , which is $0<\gamma<1$ ..

\begin{equation}
Q(s,a) = R_t + \gamma R_{t+1} + \gamma^2 R_{t+2}+ \gamma^3 R_{t+3}+ \gamma^4 R_{t+4}+ \gamma^5 R_{t+5}
\end{equation}

\begin{equation}
Q(s,a) = R_t + \gamma [R_{t+1} + \gamma R_{t+2}+ \gamma^2 R_{t+3}+ \gamma^3 R_{t+4}+ \gamma^4 R_{t+5}]
\end{equation}

This equation can be rewritten as

\begin{equation}
Q(s_t,a) = R_t+\gamma Q(s_{t+1},a_{t+1})
\end{equation}

Suppose, we have some finite discrete actions in our hand, and each resulting $Q$ values of its own, what we will do ? We will try to take the action of maximum $Q$ value!

\begin{equation}
Q(s_t,a) = R_t+\gamma max(Q(s_{t+1},a))
\end{equation}

### Coding!

Let's start coding!

I will be using ***Open Ai*** gym environment. The Introduction and Installtion of environments are given [here](https://github.com/openai/gym)

```python
import gym
import numpy as np
```

Initialization of Environments

I will use the Mountaincar environment by Open AI gym. It is a classic problem invented from 90s. I intend to use this environment for all algorithms .

In this game, your task is to get the car reach that green flag. For every step you will get -1 .So , your job is to reach the goal position with minimum steps. Maximum steps limit is 200.

```python
env = gym.make('MountainCar-v0')
s = env.reset() #Reset the car
```

```env.reset()``` gives the initial state. State is the position and velocity of the car in a given time

This game's actions can be 0,1,2 . 0 for left, 1 for doing nothing, 2 for right
```env.step(action)``` returns four arguments

- next state
- reward
- terminal , it means if game is over or not
- info , for now , it is unnecessary

#### Hyper Parameters

- ```legal_actions``` number of actions
- ```actions``` the actions list
- ```gamma``` discount factor $\gamma$
- ```lr``` learning rate $\alpha$
- ```num_episodes``` number of episodes
- ```epsilon``` epsilon , to choose random actions
- ```epsilon_decay``` epsilon decay rate

```python
legal_actions=env.action_space.n
actions = [0,1,2]
gamma =0.99
lr =0.5
num_episodes =30000
epsilon =0.5
epsilon_decay =0.99
```

Codeblock to discretize the state. Because ***Q learning*** doesnt work on continuous state space, we have to convert states into 10 discrete states

```python
N_BINS = [10,10]

MIN_VALUES = [0.6,0.07]
MAX_VALUES = [-1.2,-.07]
BINS = [np.linspace(MIN_VALUES[i], MAX_VALUES[i], N_BINS[i]) for i in range(len(N_BINS))]
rList =[]
def discretize(obs):
       return tuple([int(np.digitize(obs[i], BINS[i])) for i in range(len(N_BINS))])
```

#### Q Learning CLass

```python
class QL:
    def __init__(self,Q,policy,
                legal_actions,
                actions,
                gamma,
                lr):
        self.Q = Q #Q matrix
        self.policy =policy
        self.legal_actions=legal_actions
        self.actions = actions
        self.gamma =gamma
        self.lr =lr
    def q_value(self,s,a):
        """Gets the Q value for a certain state and action"""
        if (s,a) in self.Q:
            self.Q[(s,a)]
        else:
            self.Q[s,a]=0
        return self.Q[s,a]
    def action(self,s):
        """Gets the action for cetain state"""
        if s in self.policy:
            return self.policy[s]
        else:
            self.policy[s] = np.random.randint(0,self.legal_actions)
            return self.policy[s]
    def learn(self,s,a,s1,r,done):
        """Updates the Q matrix"""
        if done== False:
            self.Q[(s,a)] =self.q_value(s,a)+ self.lr*(r+self.gamma*max([self.q_value(s1,a1) for a1 in self.actions]) - self.q_value(s,a))
        else:
            self.Q[(s,a)] =self.q_value(s,a)+ self.lr*(r - self.q_value(s,a))
        self.q_values = [self.q_value(s,a1) for a1 in self.actions]
        self.policy[s] = self.actions[self.q_values.index(max(self.q_values))]

```

#### Q Matrix Parameters

- ```Q``` - Q table. We will use dictionary data structure.
- ```policy``` - policy table , it will give us the action for given state

```python
Q = {}
policy ={}
legal_actions =3
QL = QL(Q,policy,legal_actions,actions,gamma,lr)
```

#### Training

### Psuedocode

- get initial state $s_{raw}$
- discretize initial state , $s \gets discretize(s_{raw})$
- set total reward to zero , $r_{total} \gets 0$
- set terminal $d$ to false , $d \gets False$
- for each step
- - choose action based on epsilon greedy policy
- - get next state $s1_{raw} $, reward , $r$, terminal $d$ doing the action
- - $s1 \gets discretize(s1_{raw}) $
- - $r_{total} \gets r_{total}+r$
- - if $d == True $
- - - if $r_{total}<-199$
- - - - then give $r \gets -100$
- - - - Update $Q$ table
- - - - break
- - else
- - - Update $Q$ table
- - - break
- - $s \gets s1$

```python
for i in range(num_episodes):
    s_raw= env.reset() #initialize
    s = discretize(s_raw) #discretize the state
    rAll =0 #total reward
    d = False
    j = 0
    for j in range(200):

        #epsilon greedy. to choose random actions initially when Q is all zeros
        if np.random.random()< epsilon:
            a = np.random.randint(0,legal_actions)
            epsilon = epsilon*epsilon_decay
        else:
            a =QL.action(s)
        s1_raw,r,d,_ = env.step(a)
        rAll=rAll+r
        s1 = discretize(s1_raw)
        env.render()
        if d:
            if rAll<-199:
                r =-100 #punishment, if the game finishes before reaching the goal , we can give punishment
                QL.learn(s,a,s1,r,d)
                print("Failed! Reward %d"%rAll)
            elif rAll>-199:
                print("Passed! Reward %d"%rAll)
            break
        QL.learn(s,a,s1,r,d)
        if j==199:
            print("Reward %d after full episode"%(rAll))

        s = s1
env.close()
```


# Aplicando Python para análisis de precios: simulación de escenarios futuros de precios

> En la clase anterior vimos como importar datos de activos de la base de datos de Yahoo Finance usando el paquete pandas-datareader. En esta clase, veremos como pronosticar escenarios de evolución de precios, suponiendo que los rendimientos diarios se distribuyen normalmente. Como esta evolución de precios es aleatoria, utilizaremos la simulación montecarlo (hacer muchas simulaciones de escenarios de evolución de precios) para obtener probabilidades de que los precios de cierre estén encima de un valor umbral y tomar decisiones con base en estas probabilidades.

**Referencias:**
- http://pandas.pydata.org/
- http://www.learndatasci.com/python-finance-part-yahoo-finance-api-pandas-matplotlib/

## 1. Recordemos como descargar datos...

Antes que nada, para poder hacer simular escenarios de predicción de precios, vamos a recordar lo que hicimos en la clase pasada de descargar los datos de Yahoo Finance, utilizando el paquete `data` de la librería `pandas_datareader`.

Esta vez, utilizaremos los datos de precios de cierre ajustados de activos de la compañía Apple en el año 2016 para nuestra aplicación.

```python
# Importamos librerías
import pandas as pd
pd.core.common.is_list_like = pd.api.types.is_list_like
import pandas_datareader.data as web
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
# Función para descargar precios de cierre ajustados de varios activos a la vez:
def get_closes(tickers, start_date=None, end_date=None, freq='d'):
    # Fecha inicio por defecto (start_date='2010-01-01') y fecha fin por defecto (end_date=today)
    # Frecuencia de muestreo por defecto (freq='d')
    # Importamos paquetes necesarios
    import pandas as pd
    pd.core.common.is_list_like = pd.api.types.is_list_like
    import pandas_datareader.data as web
    # Creamos DataFrame vacío de precios, con el índice de las fechas
    closes = pd.DataFrame(columns = tickers, index=web.YahooDailyReader(symbols=tickers[0], start=start_date, end=end_date, interval=freq).read().index)
    # Agregamos cada uno de los precios con YahooDailyReader
    for ticker in tickers:
        df = web.YahooDailyReader(symbols=ticker, start=start_date, end=end_date, interval=freq).read()
        closes[ticker]=df['Adj Close']
    closes.index_name = 'Date'
    closes = closes.sort_index()
    return closes
```

```python
# Descargamos datos...
# Instrumento: Apple
name = ['AAPL']
# Fechas de interés (inicio y fin): 2010-2016
start, end = '2010-01-01', '2016-12-31'
# Función DataReader
closes = get_closes(tickers=name, start_date=start, end_date=end, freq='d')
closes
```

      AAPL

      Date

      2010-01-04
      20.473503

      2010-01-05
      20.508902

      2010-01-06
      20.182680

      2010-01-07
      20.145369

      2010-01-08
      20.279305

      2010-01-11
      20.100410

      2010-01-12
      19.871763

      2010-01-13
      20.152065

      2010-01-14
      20.035355

      2010-01-15
      19.700523

      2010-01-19
      20.572037

      2010-01-20
      20.255390

      2010-01-21
      19.905249

      2010-01-22
      18.917973

      2010-01-25
      19.426920

      2010-01-26
      19.701479

      2010-01-27
      19.887072

      2010-01-28
      19.065300

      2010-01-29
      18.373636

      2010-02-01
      18.629066

      2010-02-02
      18.737165

      2010-02-03
      19.059557

      2010-02-04
      18.372683

      2010-02-05
      18.698900

      2010-02-08
      18.570709

      2010-02-09
      18.768732

      2010-02-10
      18.666370

      2010-02-11
      19.005989

      2010-02-12
      19.169575

      2010-02-16
      19.458488

      ...
      ...

      2016-11-17
      106.609329

      2016-11-18
      106.715996

      2016-11-21
      108.335266

      2016-11-22
      108.403137

      2016-11-23
      107.850449

      2016-11-25
      108.393433

      2016-11-28
      108.180122

      2016-11-29
      108.073463

      2016-11-30
      107.162018

      2016-12-01
      106.163315

      2016-12-02
      106.560860

      2016-12-05
      105.794868

      2016-12-06
      106.609329

      2016-12-07
      107.656517

      2016-12-08
      108.713409

      2016-12-09
      110.487801

      2016-12-12
      109.857559

      2016-12-13
      111.690132

      2016-12-14
      111.690132

      2016-12-15
      112.300980

      2016-12-16
      112.446442

      2016-12-19
      113.096069

      2016-12-20
      113.396652

      2016-12-21
      113.503304

      2016-12-22
      112.756714

      2016-12-23
      112.979713

      2016-12-27
      113.697243

      2016-12-28
      113.212425

      2016-12-29
      113.183334

      2016-12-30
      112.300980

1762 rows × 1 columns

```python
# Graficamos
closes.plot(figsize=(10,6));
```

## 2. Simulación de rendimientos diarios

Recordemos que los precios diarios de cierre ajustados no son un proceso estocástico estacionario, pero los rendimientos diarios si lo son. Por tanto calculamos los rendimientos a partir de los precios de cierre, obtenemos sus propiedades estadísticas muestrales y proyectamos los rendimientos. Luego, obtenemos la proyección de los precios.

Para una sucesión de precios $\{S_t\}_{t=0}^{n}$, el rendimiento simple $R_t$ se define como el el cambio porcentual

$$
R_t=\frac{S_t-S_{t-1}}{S_{t-1}}\approx \ln\left(\frac{S_t}{S_{t-1}}\right)=r_t.
$$
para $t=1,\ldots,n$.

Para el ejemplo en curso, ¿cómo calcular esto?

Además, supusimos que los rendimientos diarios eran una variable aleatoria con distribución normal (que se caracteriza con su media y varianza). Por tanto obtenemos la media y desviación estandar muestrales. Hagamos una función que retorne lo anterior.

```python
# Calcular rendimientos diarios y graficarlos
ret = closes.pct_change().dropna()
ret.plot(figsize=(10,6));
```

Entonces, suponemos que la diferencia logaritmica de los precios (rendimientos diarios) tiene una distribución normal.

¿Cómo se caracteriza una [distribución normal](https://es.wikipedia.org/wiki/Distribuci%C3%B3n_normal)?

```python
# Calculamos media y desviación estándar
mu, std = ret.mean().AAPL, ret.std().AAPL
mu, std
```

    (0.0011058676595842502, 0.016665129057537725)

Habiendo caracterizado los rendimientos diarios como una variable aleatoria normal con la media y la varianza muestral obtenida de los datos del 2016, podemos generar números aleatorios con estas características para simular el comportamiento de los precios de cierre de las acciones en el 2017 (hay un supuesto de que las cosas no cambiarán fundamentalmente).

Sin embargo, cada simulación que hagamos nos conducirá a distintos resultados (los precios siguen evolucionando aleatoriamente). Entonces, lo que haremos es simular varios escenarios para así ver alguna tendencia y tomar decisiones.

Hagamos una una función que simule varios escenarios de rendimientos diarios rendimientos diarios y que devuelva un dataframe con esta simulación.

```python
# Ayuda en la función np.random.randn
help(np.random.randn)
```

    Help on built-in function randn:

    randn(...) method of mtrand.RandomState instance
        randn(d0, d1, ..., dn)

        Return a sample (or samples) from the "standard normal" distribution.

        If positive, int_like or int-convertible arguments are provided,
        `randn` generates an array of shape ``(d0, d1, ..., dn)``, filled
        with random floats sampled from a univariate "normal" (Gaussian)
        distribution of mean 0 and variance 1 (if any of the :math:`d_i` are
        floats, they are first converted to integers by truncation). A single
        float randomly sampled from the distribution is returned if no
        argument is provided.

        This is a convenience function.  If you want an interface that takes a
        tuple as the first argument, use `numpy.random.standard_normal` instead.

        Parameters
        ----------
        d0, d1, ..., dn : int, optional
            The dimensions of the returned array, should be all positive.
            If no argument is given a single Python float is returned.

        Returns
        -------
        Z : ndarray or float
            A ``(d0, d1, ..., dn)``-shaped array of floating-point samples from
            the standard normal distribution, or a single such float if
            no parameters were supplied.

        See Also
        --------
        random.standard_normal : Similar, but takes a tuple as its argument.

        Notes
        -----
        For random samples from :math:`N(\mu, \sigma^2)`, use:

        ``sigma * np.random.randn(...) + mu``

        Examples
        --------
        >>> np.random.randn()
        2.1923875335537315 #random

        Two-by-four array of samples from N(3, 6.25):

        >>> 2.5 * np.random.randn(2, 4) + 3
        array([[-4.49401501,  4.00950034, -1.81814867,  7.29718677],  #random
               [ 0.39924804,  4.68456316,  4.99394529,  4.84057254]]) #random

```python
# Función que simula varios escenarios de rendimientos diarios
def ret_sim(mu, sigma, ndays, nscen, start_date):
    dates = pd.date_range(start=start_date,periods=ndays)
    return pd.DataFrame(data = sigma*np.random.randn(ndays, nscen)+mu, index = dates)
```

```python
# Simulamos 100 escenarios para todo el 2017
simret = ret_sim(mu, std, 252, 10000, '2017-01-01')
```

```python
# Mostrar
#simret
```

## 3. Proyección de precios de cierre

Por tanto, para calcular los precios, tenemos:

$$\begin{align}
p_i&=p_{i-1}(R_i+1)\\
p_{i+1}&=p_i(R_{i+1}+1)=p_{i-1}(R_i+1)(R_{i+1}+1)\\
&\vdots\\
p_{i+k}&=p_{i-1}(R_i+1)\cdots(R_{i+k}+1).
\end{align}$$

Si hacemos $i=0$ en la última ecuación, tenemos que $p_{k}=p_{-1}\exp(r_0+\cdots+r_{k})$, donde $p_{-1}$ es el último precio reportado en el 2016.

Con los rendimientos, calculamos los precios de cierre...

```python
# Obtenemos los precios
simcloses = closes.iloc[-1].AAPL*((1+simret).cumprod())
#simcloses
```

Concatenamos y graficamos...

```python
# Concatenar y graficar
#simcloses_ex = pd.concat([closes.iloc[-200:], simcloses])
#simcloses_ex.plot(figsize=(10,6), legend=False);
```

```python
closes.iloc[-1].AAPL*1.1
```

    123.53107800000002

## 4. Probabilidad Precio-Umbral

Ya que tenemos muchos escenarios de precios proyectados, podemos ver varias cosas. Por ejemplo, ¿cuál es la probabilidad de que el precio de cierre sobrepase algún valor umbral en algún momento?

```python
# Umbral de 110% del ultimo precio
K = closes.iloc[-1].AAPL*1.1
# Fechas
dates = simcloses.index
# DataFrame de Strikes
Strike = pd.DataFrame(index = dates, columns=['K'], data = K*np.ones(len(dates)))
# Concatenar y graficar
#simcloses_ex_K = pd.concat([simcloses_ex.T, Strike.T]).T
#simcloses_ex_K.plot(figsize=(10,6), legend=False);
```

```python
# Comparar cada escenario en cada fecha
TF = simcloses>K
# Sumamos para cada fecha y dividimos entre el número de escenarios
prob = pd.DataFrame(TF.sum(axis=1)/100)
# Gráfico de probabilidad
prob.plot(figsize=(10,6), legend=False);
```

___
Entonces, ya aprendimos a bajar datos con pandas-datareader. En específico, a partir de los precios de cierre ajustados obtuvimos los rendimientos diarios.

Suponiendo que los rendimientos diarios son un proceso estocástico estacionario de distribución normal, pudimos caracaterizarlo y proyectar varios escenarios de evolución de los precios (montecarlo).

Con estas proyecciones pudimos calcular probabilidades de sobrepasar cierto precio umbral: toma de decisiones.

Created with Jupyter by Esteban Jiménez Rodríguez.


# CF Part 3 - Item-based method
> Collaborative Filtering on MovieLens Latest-small Part 3 - Finding recommendations using memory based item-item similarity method

- toc: true
- badges: true
- comments: true
- categories: [movie, collaborative]
- image:

## Idea
Let $u$ be the active user and $i$ the referenced item
1. If $u$ liked items similar to $i$, he will probably like item $i$.
2. If he hated or disliked items similar to $i$, he will also hate item $i$.

The idea is therefore to look at how an active user $u$ rated items similar to $i$ to know how he would have rated item $i$

## Advantages over user-based CF

1.  Stability  : Items ratings are more stable than users ratings. New ratings on items are unlikely to significantly change the similarity between two items, particularly when the items have many ratings (Michael D. Ekstrand, et al. 2011).
2.  Scalability  : with stable item's ratings, it is reasonable to pre-compute similarities between items in an item-item similarity matrix (similarity between items can be computed offline). This will reduce the scalability concern of the algorithm. (Sarwar et al. 2001), (Michael D. Ekstrand, et al. 2011).

## Algorithm : item-to-item collaborative filtering

The algorithm that defines item-based CF is described as follow (B. Sarwar et al. 2001)(George Karypis 2001) :

        First identify the $k$ most similar items for each item in the catalogue and record the corresponding similarities. To compute similarity between two items we can user the Adjusted Cosine Similarity that has proven to be more efficient than the basic Cosine similarity measure used for user-based collaborative as described in (B. Sarwar et al. 2001). The Adjusted Cosine distance between two items $i$ and $j$ is computed as follow

\begin{equation}
 w_{i,j}= \frac{\sum_{u\in U}(r_{u,i}-\bar{r}_u)(r_{u,j}-\bar{r}_u)}{\sqrt{\sum_{u\in U} (r_{u,i}-\bar{r}_u)^2}\sqrt{\sum_{u\in U} (r_{u,j}-\bar{r}_u)^2}}
\end{equation}

$w_{i,j}$ is the degree of similarity between items $i$ and $j$. This term is computed for all users $u\in U$, where $U$ is the set of users that rated both items $i$ and $j$. Let's denote by $S^{(i)}$ the set of the $k$ most similar items to item $i$.

 To produce top-N recommendations for a given user $u$ that has already purchased a set $I_u$ of items, do the following :

 Find the set $C$ of candidate items by taking the union of all $S^{(i)}, \forall i\in I_u$ and removing each of the items in the set $I_u$.
\begin{equation}
 C = \bigcup_{i\in I_u}\{S^{(i)}\}\smallsetminus I_u
\end{equation}

        $\forall c\in C$, compute similarity between c and the set $I_u$ as follows:
\begin{equation}
 w_{c,I_u} = \sum_{i\in I_u} w_{c,i}, \forall c \in C
\end{equation}

        Sort items in $C$ in decreasing order of $w_{c,I_u}, \forall c \in C$, and return the first $N$ items as the Top-N recommendation list.

Before returning the first $N$ items as top-N recommendation list, we can make predictions about what user $u$ would have given to each items in the top-N recommendation list, rearrange the list in descending order of predicted ratings and return the rearranged list as the final recommendation list. Rating prediction for item-based CF is given by the following formular (B. Sarwar et al. 2001):

\begin{equation}
 \hat{r}_{u,i}=\frac{\sum_{i\in S^{(i)}}r_{u,j}\cdot w_{i,j}}{\sum_{j\in S^{(i)}}|w_{i,j}|}
\end{equation}

### Import useful requirements

```python
import os

if not (os.path.exists("recsys.zip") or os.path.exists("recsys")):
    !wget https://github.com/nzhinusoftcm/review-on-collaborative-filtering/raw/master/recsys.zip
    !unzip recsys.zip
```

### Import requirements

```
matplotlib==3.2.2
numpy==1.19.2
pandas==1.0.5
python==3.7
scikit-learn==0.24.1
scikit-surprise==1.1.1
scipy==1.6.2
```

```python
from sklearn.neighbors import NearestNeighbors
from scipy.sparse import csr_matrix

from recsys.datasets import ml1m, ml100k
from recsys.preprocessing import ids_encoder

import pandas as pd
import numpy as np
import os
import sys
```

### Load ratings

```python
ratings, movies = ml100k.load()
```

### userids and itemids encoding

```python
# create the encoder
ratings, uencoder, iencoder = ids_encoder(ratings)
```

Let's implements the item-based collaborative filtering algorithm described above

### Step 1. Find similarities for each of the items

To compute similarity between two items $i$ and $j$, we need to :

1. find all users who rated both of them,
2. Normalize their ratings on items $i$ and $j$
3. Apply the cosine metric to the normalized ratings to compute similarity between $i$ and $j$

Function ```normalize()``` process the rating dataframe to normalize ratings of all users

```python
def normalize():
    # compute mean rating for each user
    mean = ratings.groupby(by='userid', as_index=False)['rating'].mean()
    norm_ratings = pd.merge(ratings, mean, suffixes=('','_mean'), on='userid')

    # normalize each rating by substracting the mean rating of the corresponding user
    norm_ratings['norm_rating'] = norm_ratings['rating'] - norm_ratings['rating_mean']
    return mean.to_numpy()[:, 1], norm_ratings
```

```python
mean, norm_ratings = normalize()
np_ratings = norm_ratings.to_numpy()
norm_ratings.head()
```

      userid
      itemid
      rating
      rating_mean
      norm_rating

      0
      0
      0
      5
      3.610294
      1.389706

      1
      0
      1
      3
      3.610294
      -0.610294

      2
      0
      2
      4
      3.610294
      0.389706

      3
      0
      3
      3
      3.610294
      -0.610294

      4
      0
      4
      3
      3.610294
      -0.610294

now that each rating has been normalized, we can represent each item by a vector of its normalized ratings

```python
def item_representation(ratings):
    return csr_matrix(
        pd.crosstab(ratings.itemid, ratings.userid, ratings.norm_rating, aggfunc=sum).fillna(0).values
    )
```

```python
R = item_representation(norm_ratings)
```

Let's build and fit our $k$-NN model using sklearn

```python
def create_model(rating_matrix, k=20, metric="cosine"):
    """
    :param R : numpy array of item representations
    :param k : number of nearest neighbors to return
    :return model : our knn model
    """
    model = NearestNeighbors(metric=metric, n_neighbors=k+1, algorithm='brute')
    model.fit(rating_matrix)
    return model
```

#### Similarities computation

Similarities between items can be measured with the *Cosine* or *Eucliedian* distance. The ***NearestNeighbors*** class from the sklearn library simplifies the computation of neighbors. We just need to specify the metric (e.g. cosine or euclidian) that will be used to compute similarities.

The above method, ```create_model```, creates the kNN model and the following ```nearest_neighbors``` method uses the created model to kNN items. It returns nearest neighbors as well as similarities measures for each items.

```nearest_neighbors``` returns :
- ```similarities``` : numpy array of shape $(n,k)$
- ```neighbors``` : numpy array of shape $(n,k)$

where $n$ is the total number of items and $k$ is the number of neighbors to return, specified when creating the kNN model.

```python
def nearest_neighbors(rating_matrix, model):
    """
    compute the top n similar items for each item.
    :param rating_matrix : items representations
    :param model : nearest neighbors model
    :return similarities, neighbors
    """
    similarities, neighbors = model.kneighbors(rating_matrix)
    return similarities[:,1:], neighbors[:,1:]
```

#### Ajusted Cosine Similarity
In the context of item-based collaborative filtering, the adjusted cosine similarity has shown to be more efficient that the cosine or the euclidian distance. Here is the formular to compute the adjusted cosine weight between two items $i$ and $j$ :

\begin{equation}
 w_{i,j}= \frac{\sum_{u\in U}(r_{u,i}-\bar{r}_u)(r_{u,j}-\bar{r}_u)}{\sqrt{\sum_{u\in U} (r_{u,i}-\bar{r}_u)^2}\sqrt{\sum_{u\in U} (r_{u,j}-\bar{r}_u)^2}}.
\end{equation}

This term is computed for all users $u\in U$, where $U$ is the set of users that rated both items $i$ and $j$. Since the *sklearn* library do not directly implement the adjusted cosine similarity metric, we will implement it with the method ```adjusted_cosine```, with some helper function :

- ```save_similarities``` : since the computation of the adjusted cosine similarity is time consuming, around 5 mins for the ml100k dataset, we use this method to save the computed similarities for lated usage.
- ```load_similarities``` : load the saved similarities
- ```cosine``` : cosine distance between two vectors.

```python
def save_similarities(similarities, neighbors, dataset_name):
    base_dir = 'recsys/weights/item2item'
    save_dir = os.path.join(base_dir, dataset_name)
    os.makedirs(save_dir, exist_ok=True)
    similarities_file_name = os.path.join(save_dir, 'similarities.npy')
    neighbors_file_name = os.path.join(save_dir, 'neighbors.npy')
    try:
        np.save(similarities_file_name, similarities)
        np.save(neighbors_file_name, neighbors)
    except ValueError as error:
        print(f"An error occured when saving similarities, due to : \n ValueError : {error}")

def load_similarities(dataset_name, k=20):
    base_dir = 'recsys/weights/item2item'
    save_dir = os.path.join(base_dir, dataset_name)
    similiraties_file = os.path.join(save_dir, 'similarities.npy')
    neighbors_file = os.path.join(save_dir, 'neighbors.npy')
    similarities = np.load(similiraties_file)
    neighbors = np.load(neighbors_file)
    return similarities[:,:k], neighbors[:,:k]

def cosine(x, y):
    return np.dot(x, y) / (np.linalg.norm(x) * np.linalg.norm(y))

def adjusted_cosine(np_ratings, nb_items, dataset_name):
    similarities = np.zeros(shape=(nb_items, nb_items))
    similarities.fill(-1)

    def _progress(count):
        sys.stdout.write('\rComputing similarities. Progress status : %.1f%%' % (float(count / nb_items)*100.0))
        sys.stdout.flush()

    items = sorted(ratings.itemid.unique())
    for i in items[:-1]:
        for j in items[i+1:]:
            scores = np_ratings[(np_ratings[:, 1] == i) | (np_ratings[:, 1] == j), :]
            vals, count = np.unique(scores[:,0], return_counts = True)
            scores = scores[np.isin(scores[:,0], vals[count > 1]),:]

            if scores.shape[0] > 2:
                x = scores[scores[:, 1].astype('int') == i, 4]
                y = scores[scores[:, 1].astype('int') == j, 4]
                w = cosine(x, y)

                similarities[i, j] = w
                similarities[j, i] = w
        _progress(i)
    _progress(nb_items)

    # get neighbors by their neighbors in decreasing order of similarities
    neighbors = np.flip(np.argsort(similarities), axis=1)

    # sort similarities in decreasing order
    similarities = np.flip(np.sort(similarities), axis=1)

    # save similarities to disk
    save_similarities(similarities, neighbors, dataset_name=dataset_name)

    return similarities, neighbors
```

now, we can call the ```adjusted_cosine``` function to compute and save items similarities and neighbors based on the adjusted cosine metric.

uncomment the two lines of the following cell to compute the adjusted cosine between all items. As we have already run the next cell before, we will just load the precomputed similarities for further use.

```python
# nb_items = ratings.itemid.nunique()
# similarities, neighbors = adjusted_cosine(np_ratings, nb_items=nb_items, dataset_name='ml100k')
```

Among the following similarity metrics, choose the one you wish to use for the item-based collaborative filtering :

- **euclidian** or **cosine** : choose *euclidian* or *cosine* to initialise the similarity model through the sklearn library.
- **adjusted_cosine** : choose the *adjusted_cosine* metric to load similarities computed and saved through the ```adjusted_cosine``` function.

In this case, we will use the *adjusted_cosine* metric.

```python
# metric : choose among [cosine, euclidean, adjusted_cosine]

metric = 'adjusted_cosine'

if metric == 'adjusted_cosine':
    similarities, neighbors = load_similarities('ml100k')
else:
    model = create_model(R, k=21, metric=metric)
    similarities, neighbors = nearest_neighbors(R, model)
```

```python
print('neighbors shape : ', neighbors.shape)
print('similarities shape : ', similarities.shape)
```

    neighbors shape :  (1682, 20)
    similarities shape :  (1682, 20)

```neighbors``` and ```similarities``` are numpy array, were each entries are list of 20 neighbors with their corresponding similarities

### Step 2. Top N recommendation for a given user

Top-N recommendations are made for example for a user $u$ who has already rated a set of items $I_u$

#### 2.a- Finding candidate items

To find candidate items for user $u$, we need to :

1. Find the set $I_u$ of items already rated by user $u$,
2. Take the union of similar items as $C$ for all items in $I_u$
3. exclude from the set $C$ all items in $I_u$, to avoid recommend to a user items he has already purchased.

These are done in function ```candidate_items()```

```python
def candidate_items(userid):
    """
    :param userid : user id for which we wish to find candidate items
    :return : I_u, candidates
    """

    # 1. Finding the set I_u of items already rated by user userid
    I_u = np_ratings[np_ratings[:, 0] == userid]
    I_u = I_u[:, 1].astype('int')

    # 2. Taking the union of similar items for all items in I_u to form the set of candidate items
    c = set()

    for iid in I_u:
        # add the neighbors of item iid in the set of candidate items
        c.update(neighbors[iid])

    c = list(c)
    # 3. exclude from the set C all items in I_u.
    candidates = np.setdiff1d(c, I_u, assume_unique=True)

    return I_u, candidates
```

```python
test_user = uencoder.transform([1])[0]
i_u, u_candidates = candidate_items(test_user)
```

```python
print('number of items purchased by user 1 : ', len(i_u))
print('number of candidate items for user 1 : ', len(u_candidates))
```

    number of items purchased by user 1 :  272
    number of candidate items for user 1 :  893

#### 2.b- Find similarity between each candidate item and the set $I_u$

```python
def similarity_with_Iu(c, I_u):
    """
    compute similarity between an item c and a set of items I_u. For each item i in I_u, get similarity between
    i and c, if c exists in the set of items similar to itemid.
    :param c : itemid of a candidate item
    :param I_u : set of items already purchased by a given user
    :return w : similarity between c and I_u
    """
    w = 0
    for iid in I_u :
        # get similarity between itemid and c, if c is one of the k nearest neighbors of itemid
        if c in neighbors[iid] :
            w = w + similarities[iid, neighbors[iid] == c][0]
    return w
```

#### 2.c- Rank candidate items according to their similarities to $I_u$

```python
def rank_candidates(candidates, I_u):
    """
    rank candidate items according to their similarities with i_u
    :param candidates : list of candidate items
    :param I_u : list of items purchased by the user
    :return ranked_candidates : dataframe of candidate items, ranked in descending order of similarities with I_u
    """

    # list of candidate items mapped to their corresponding similarities to I_u
    sims = [similarity_with_Iu(c, I_u) for c in candidates]
    candidates = iencoder.inverse_transform(candidates)
    mapping = list(zip(candidates, sims))

    ranked_candidates = sorted(mapping, key=lambda couple:couple[1], reverse=True)
    return ranked_candidates
```

## Putting all together

Now that we defined all functions necessary to build our item to item top-N recommendation, let's define function ```item2item_topN()``` that makes top-$N$ recommendations for a given user

```python
def topn_recommendation(userid, N=30):
    """
    Produce top-N recommendation for a given user
    :param userid : user for which we produce top-N recommendation
    :param n : length of the top-N recommendation list
    :return topn
    """
    # find candidate items
    I_u, candidates = candidate_items(userid)

    # rank candidate items according to their similarities with I_u
    ranked_candidates = rank_candidates(candidates, I_u)

    # get the first N row of ranked_candidates to build the top N recommendation list
    topn = pd.DataFrame(ranked_candidates[:N], columns=['itemid','similarity_with_Iu'])
    topn = pd.merge(topn, movies, on='itemid', how='inner')
    return topn
```

```python
topn_recommendation(test_user)
```

      itemid
      similarity_with_Iu
      title

      0
      1356
      52.867173
      Ed's Next Move (1996)

      1
      1189
      50.362199
      Prefontaine (1997)

      2
      1516
      31.133267
      Wedding Gift, The (1994)

      3
      1550
      31.031738
      Destiny Turns on the Radio (1995)

      4
      1554
      27.364494
      Safe Passage (1994)

      5
      1600
      27.287712
      Guantanamera (1994)

      6
      1223
      26.631850
      King of the Hill (1993)

      7
      1388
      26.624397
      Gabbeh (1996)

      8
      766
      26.590175
      Man of the Year (1995)

      9
      691
      26.461802
      Dark City (1998)

      10
      1378
      25.787842
      Rhyme & Reason (1997)

      11
      1664
      25.327445
      8 Heads in a Duffel Bag (1997)

      12
      1261
      24.785660
      Run of the Country, The (1995)

      13
      1123
      24.524028
      Last Time I Saw Paris, The (1954)

      14
      1538
      24.492453
      All Over Me (1997)

      15
      1485
      24.345312
      Colonel Chabert, Le (1994)

      16
      1450
      24.262120
      Golden Earrings (1947)

      17
      909
      23.357301
      Dangerous Beauty (1998)

      18
      359
      22.973658
      Assignment, The (1997)

      19
      1369
      22.710078
      Forbidden Christ, The (Cristo proibito, Il) (1...

      20
      1506
      22.325504
      Nelly & Monsieur Arnaud (1995)

      21
      1537
      22.061914
      Cosi (1996)

      22
      1474
      21.877034
      Nina Takes a Lover (1994)

      23
      1467
      21.861203
      Saint of Fort Washington, The (1993)

      24
      1255
      21.750924
      Broken English (1996)

      25
      1499
      21.529748
      Grosse Fatigue (1994)

      26
      1466
      21.063269
      Margaret's Museum (1995)

      27
      1448
      20.846909
      My Favorite Season (1993)

      28
      927
      20.730153
      Flower of My Secret, The (Flor de mi secreto, ...

      29
      1375
      20.627152
      Cement Garden, The (1993)

This dataframe represents the top N recommendation list a user. These items are sorted in decreasing order of similarities with $I_u$.

**Observation** : The recommended items are the most similar to the set $I_u$ of items already purchased by the user.

## Top N recommendation with predictions

Before recommending the previous list to the user, we can go further and predict the ratings the user would have given to each of these items, sort them in descending order of prediction and return the reordered list as the new top N recommendation list.

### Rating prediction

As stated earlier, the predicted rating $\hat{r}_{u,i}$ for a given user $u$ on an item $i$ is obtained by aggregating ratings given by $u$ on items similar to $i$ as follows:

\begin{equation}
 \hat{r}_{u,i}=\frac{\sum_{j\in S^{(i)}}r_{u,j}\cdot w_{i,j}}{\sum_{j\in S^{(i)}}|w_{i,j}|}
\end{equation}

```python
def predict(userid, itemid):
    """
    Make rating prediction for user userid on item itemid
    :param userid : id of the active user
    :param itemid : id of the item for which we are making prediction
    :return r_hat : predicted rating
    """

    # Get items similar to item itemid with their corresponding similarities
    item_neighbors = neighbors[itemid]
    item_similarities = similarities[itemid]

    # get ratings of user with id userid
    uratings = np_ratings[np_ratings[:, 0].astype('int') == userid]

    # similar items rated by item the user of i
    siru = uratings[np.isin(uratings[:, 1], item_neighbors)]
    scores = siru[:, 2]
    indexes = [np.where(item_neighbors == iid)[0][0] for iid in siru[:,1].astype('int')]
    sims = item_similarities[indexes]

    dot = np.dot(scores, sims)
    som = np.sum(np.abs(sims))

    if dot == 0 or som == 0:
        return mean[userid]

    return dot / som
```

Now let's use our ```predict()``` function to predict what ratings the user would have given to the previous top-$N$ list and return the reorganised list (in decreasing order of predictions) as the new top-$N$ list

```python
def topn_prediction(userid):
    """
    :param userid : id of the active user
    :return topn : initial topN recommendations returned by the function item2item_topN
    :return topn_predict : topN recommendations reordered according to rating predictions
    """
    # make top N recommendation for the active user
    topn = topn_recommendation(userid)

    # get list of items of the top N list
    itemids = topn.itemid.to_list()

    predictions = []

    # make prediction for each item in the top N list
    for itemid in itemids:
        r = predict(userid, itemid)

        predictions.append((itemid,r))

    predictions = pd.DataFrame(predictions, columns=['itemid','prediction'])

    # merge the predictions to topN_list and rearrange the list according to predictions
    topn_predict = pd.merge(topn, predictions, on='itemid', how='inner')
    topn_predict = topn_predict.sort_values(by=['prediction'], ascending=False)

    return topn, topn_predict
```

Now, let's make recommendation for user 1 and compare the two list

```python
topn, topn_predict = topn_prediction(userid=test_user)
```

```python
topn_predict
```

      itemid
      similarity_with_Iu
      title
      prediction

      7
      1388
      26.624397
      Gabbeh (1996)
      4.666667

      18
      359
      22.973658
      Assignment, The (1997)
      4.600000

      4
      1554
      27.364494
      Safe Passage (1994)
      4.500000

      14
      1538
      24.492453
      All Over Me (1997)
      4.500000

      27
      1448
      20.846909
      My Favorite Season (1993)
      4.490052

      29
      1375
      20.627152
      Cement Garden, The (1993)
      4.333333

      26
      1466
      21.063269
      Margaret's Museum (1995)
      4.271915

      2
      1516
      31.133267
      Wedding Gift, The (1994)
      4.000000

      23
      1467
      21.861203
      Saint of Fort Washington, The (1993)
      4.000000

      21
      1537
      22.061914
      Cosi (1996)
      4.000000

      10
      1378
      25.787842
      Rhyme & Reason (1997)
      4.000000

      19
      1369
      22.710078
      Forbidden Christ, The (Cristo proibito, Il) (1...
      4.000000

      3
      1550
      31.031738
      Destiny Turns on the Radio (1995)
      3.777778

      1
      1189
      50.362199
      Prefontaine (1997)
      3.666528

      20
      1506
      22.325504
      Nelly & Monsieur Arnaud (1995)
      3.610294

      15
      1485
      24.345312
      Colonel Chabert, Le (1994)
      3.610294

      11
      1664
      25.327445
      8 Heads in a Duffel Bag (1997)
      3.610294

      9
      691
      26.461802
      Dark City (1998)
      3.610294

      6
      1223
      26.631850
      King of the Hill (1993)
      3.610294

      5
      1600
      27.287712
      Guantanamera (1994)
      3.610294

      17
      909
      23.357301
      Dangerous Beauty (1998)
      3.500000

      12
      1261
      24.785660
      Run of the Country, The (1995)
      3.333333

      24
      1255
      21.750924
      Broken English (1996)
      3.265749

      13
      1123
      24.524028
      Last Time I Saw Paris, The (1954)
      3.200000

      16
      1450
      24.262120
      Golden Earrings (1947)
      3.142978

      22
      1474
      21.877034
      Nina Takes a Lover (1994)
      3.000000

      8
      766
      26.590175
      Man of the Year (1995)
      3.000000

      0
      1356
      52.867173
      Ed's Next Move (1996)
      2.280926

      28
      927
      20.730153
      Flower of My Secret, The (Flor de mi secreto, ...
      1.665010

      25
      1499
      21.529748
      Grosse Fatigue (1994)
      1.122032

As you will have noticed, the two lists are sorted in different ways. The second list is organized according to the predictions made for the user.

Note: When making predictions for user $u$ on item $i$, user $u$ may not have rated any of the $k$ most similar items to i. In this case, we consider the mean rating of $u$ as the predicted value.

## Evaluation with Mean Absolute Error

```python
from recsys.preprocessing import train_test_split, get_examples

# get examples as tuples of userids and itemids and labels from normalize ratings
raw_examples, raw_labels = get_examples(ratings, labels_column='rating')

# train test split
(x_train, x_test), (y_train, y_test) = train_test_split(examples=raw_examples, labels=raw_labels)

def evaluate(x_test, y_test):
    print('Evaluate the model on {} test data ...'.format(x_test.shape[0]))
    preds = list(predict(u,i) for (u,i) in x_test)
    mae = np.sum(np.absolute(y_test - np.array(preds))) / x_test.shape[0]
    print('\nMAE :', mae)
    return mae
```

```python
evaluate(x_test, y_test)
```

    Evaluate the model on 10000 test data ...

    MAE : 0.672389703640273

    0.672389703640273

## Summary

As with the User-based CF, we have also summarised the Item-based CF into the python class [ItemToItem](https://github.com/nzhinusoftcm/review-on-collaborative-filtering/blob/master/recsys/memories/ItemToItem.py).

#### ItemToItem : usage

```python
from recsys.memories.ItemToItem import ItemToItem
from recsys.preprocessing import ids_encoder, train_test_split, get_examples
from recsys.datasets import ml100k

# load data
ratings, movies = ml100k.load()

# prepare data
ratings, uencoder, iencoder = ids_encoder(ratings)

# get examples as tuples of userids and itemids and labels from normalize ratings
raw_examples, raw_labels = get_examples(ratings, labels_column='rating')

# train test split
(x_train, x_test), (y_train, y_test) = train_test_split(examples=raw_examples, labels=raw_labels)
```

#### Instanciate the ItemToItem CF

Parameters :
- ```k``` : number of neighbors to consider for each item
- ```metric``` : metric to use when computing similarities : let's use **cosine**
- ```dataset_name``` : in this example, we use the ml100k dataset

```python
# create the Item-based CF
item2item = ItemToItem(ratings, movies, k=20, metric='cosine', dataset_name='ml100k')
```

    Normalize ratings ...
    Create the similarity model ...
    Compute nearest neighbors ...
    Item to item recommendation model created with success ...

```python
# evaluate the algorithm on test dataset
item2item.evaluate(x_test, y_test)
```

    Evaluate the model on 10000 test data ...

    MAE : 0.507794195659005

    0.507794195659005

#### Evaluate the Item-based CF on the ML-1M dataset

```python
from recsys.memories.ItemToItem import ItemToItem
from recsys.preprocessing import ids_encoder, train_test_split, get_examples
from recsys.datasets import ml1m

# load data
ratings, movies = ml1m.load()

# prepare data
ratings, uencoder, iencoder = ids_encoder(ratings)

# get examples as tuples of userids and itemids and labels from normalize ratings
raw_examples, raw_labels = get_examples(ratings, labels_column='rating')

# train test split
(x_train, x_test), (y_train, y_test) = train_test_split(examples=raw_examples, labels=raw_labels)

# create the Item-based CF
item2item = ItemToItem(ratings, movies, k=20, metric='cosine', dataset_name='ml1m')

# evaluate the algorithm on test dataset
print("=========================")
item2item.evaluate(x_test, y_test)
```

    Normalize ratings ...
    Create the similarity model ...
    Compute nearest neighbors ...
    Item to item recommendation model created with success ...
    =========================
    Evaluate the model on 100021 test data ...

    MAE : 0.42514728655396045

    0.42514728655396045

## Model based CF

**User-based** and **Item-based CF** are memory based algorithms. They directly act on the user-item interactions to compute recommendation. To the contrary, model-based algorithms are mathematical models trained on the user-item interactions and used to predict recommendation.

We will start with the SVD (Singular Value Decomposition) algorithm.

## References

1. George Karypis (2001)Evaluation of Item-Based Top-N Recommendation Algorithms
2. Sarwar et al. (2001)  Item-based collaborative filtering recommendation algorithms
3. Michael D. Ekstrand, et al. (2011).  Collaborative Filtering Recommender Systems
4. J. Bobadilla et al. (2013) Recommender systems survey
5. Greg Linden, Brent Smith, and Jeremy York (2003) Amazon.com Recommendations : Item-to-Item Collaborative Filtering

## Author

[Carmel WENGA](https://www.linkedin.com/in/carmel-wenga-871876178/),

PhD student at Université de la Polynésie Française,

Applied Machine Learning Research Engineer,

[ShoppingList](https://shoppinglist.cm), NzhinuSoft.


# Frequent sub-population testing for COVID-19

_[Boaz Barak](https://boazbarak.org), [Mor Nitzan](https://mornitzan.wixsite.com/nitzan), Neta Ravid Tannenbaum, [Janni Yuval](https://eapsweb.mit.edu/people/janniy)_

[Working paper on arXiv](http://arxiv.org/abs/2007.04827) | [Open this notebook in colab](https://colab.research.google.com/drive/1ZyUylIfo3tSwDHsv942Xl6lgMxAcjGCP?usp=sharing) (see also ["playground" notebook"](https://colab.research.google.com/drive/1pgpVBqchciVYl2qnb6q_L33xLmXogmVo?usp=sharing) ) | [Code on GitHub](https://github.com/boazbk/seirsplus)

Consider an institution of $N$ people (think business, school, ...) that
was shut down due to COVID-19 and that we would like to open it back up.
_Testing_ will clearly play a significant role in any opening strategy, but what's the best way to use a limited testing budget?

For example, suppose that we have a budget of $N$ tests per month.
Is it better to test all $N$ members of the institution once per month? Ot is it better to test $\approx N/4$ of them per week?
Or perhaps test $N/30$ members per day?
Moreover, if we are smart about testing, could we avoid the added risk to society from
opening the business?

It turns out that it is almost always better to increase the test frequency (e.g., test $25\%$ per week as opposed to $100\%$ per mont),
and sometimes dramatically so,. Moreover, under reasonable assumptions, a moderate amount of frequent testing (coupled with mitigation once an outbreak is detected),
can  __completely offset__ the added risk from opening the business.

In this notebook, we describe this in a simple example. See the [paper](http://arxiv.org/abs/2007.04827)
for more details and this [repository](https://github.com/boazbk/seirsplus) for the code.

__NOTE:__ This paper has not been yet peer reviewed.

## Setup

Suppose we have an institution of $N=1000$ members, and the prevalence
of COVID-19 in the community is such that each member has probability
$1/5000=0.05\%$ to be infected each day.

(For example,  our own community of
[Cambridge, MA](https://cityofcambridge.shinyapps.io/COVID19/?tab=new)
has about 100K people and the number of daily reported cases has ranged from
$\approx 50$ at the mid April peak to $\approx 2$ recently.
Assuming there are $5$ to $10$ true cases for each reported one,
the daily probability of infection was something like $0.25\%$ to $0.5\%$ at the peak
and is roughly $0.01\%$ to $0.02\%$ now.)

Suppose that if we open up a business, then in expectation each infected member will infect
$R$  individuals within the business, in addition to the people they
will infect in the external society. For example, if each community member infects in expectation $0.9$ people in
external society, and $0.5$ people within their workplace, then opening all businesses can and will
make the difference between controlled and uncontrolled outbreak.

However _testing_ can mitigate this. If we discover an outbreak in a business through randomized testing, we can quarantine
those individuals and ensure they do not continue to infect others.
Moreover, since quarantined individuals also don't infect people in external society,
discovery through workplace testing can cut short their "effective infectious period" and hence
reduce the number of people they would have infected __even compared to the baseline when the workplace was not
open at all!__

We show this using both math (for a simple exponential spread model) and code (building on an SEIR model of [McGee et al](https://github.com/ryansmcgee/seirsplus)).

### Frequent testing outperforms infrequent testing in simulations

Let us run a simulation for the setting we just described.
We use the standard SEIR (suspected-exposed-infected-recovered) model with COVID-19 parameters
taken from the literature. (We expect that as we learn more on COVID-19 and test technology improves, some parameters will change but the
qualitative  conclusions will remain the same; at the moment we assume the mean incubation period is $5.2$ days,
the infectious period is $14$ days, the probability for a false negative for an individual in the incubation period is $67\%$
and is $20\%$ in the infectious period, with no false positives.)

For this particular run, let's assume that each infected person infects on average two people in their workplace.
Here are two execution of such a simulation - the first where we test $100\%$ of the population every $28$ days, and the second where
we test $25\%$ every week. We keep track of the number of __E__xposed, __I__nfected, and __R__ecovered individuals, and stop the
simulation once we get the first positive result.

```python
%load_ext autoreload
%autoreload 2
from util import * # Python file containing all code, available on https://github.com/boazbk/seirsplus
```

```python
print("Test 100% of people every 28 days")
sim(base,R=2,period=28,fraction_tested=1);
print("Test 25% of people every 7 days")
log = sim(base,R=2, period=7,fraction_tested=1/4)
print(f"Outbreak detected at {log['t']} days")
```

In this particular instance, testing every week resulted in discovery of the outbreak at 14 days (the last time we ran this), leading to better outcome. But this does not have to always be the case - there is significant amount of stochasticity here
and sometimes the fact that we only test frequently will cause us to miss an infection.

To compare the two parameter settings, we need to define a cost function. We define the __societal risk__ as
the average number of infected people per day until the first detection.
The rationale is that this corresponds to the number of _infection opportunities_: chances that
for individuals to infect both co-workers and members of the external society until the outbreak
was detected. (We do not model what happens after an outbreak is detected since that would depend
strongly on the particular institution; such detection may a number of mitigation mechanisms including contact tracing, isolation,
temporary closures, widepread testing, and more.)

Let's use a "violin graph" to compare the distribution of this societal risk under both testing regimes.

```python
infrequent = [sim(base,R=2,period=28,fraction_tested=1, plot=False)["risk"] for i in range(50)];
frequent = [sim(base,R=2,period=7,fraction_tested=1/4, plot=False)["risk"] for i in range(50)];

```

```python
violins([infrequent,frequent], ["100% / 28 days", "25% / week"])
```

We see that frequent testing reduces the mean social social risk,
but more than that it also significantly drops the "tail" of the distribution - the chance that
the social risk will be much higher than (in this case) an average of 6 infected days.

Moreover, we can also compare to the setting when the business is closed but we don't test at all.

```python
closed = [sim({**base , "T":56 },R=0,period=28,fraction_tested=0, plot=False)["risk"] for i in range(50)];
```

```python
violins([closed,infrequent,frequent], ["Business closed", "100% / 28 days", "25% / week"])
```

We see that testing $25\%$ of the population per week is not only better than testing everyone
once a week, but in fact even reduces the infection opportunities compared to the baseline
where the business is closed!

Overall this conclusion holds for many (but not all!) parameters for the external infection probability
and internal reproductive number.

<small>Figure: Comparing the societal risk for the  28/4 testing policy (i.e., 25% each week) against the baseline when business is closed, 28/1 (i.e., 100% every 4 weeks), as well as the case when the business is opened and no testing takes place.
For every value of the external probability and internal reproductive number we track the percentage of the 28/4 risk wrt the comparands (less than 100%  means that 28/4 policy improves on the comparand)
 </small>

### Is more frequent testing always better?

Since we saw that increasing the frequency to once a week gives better outcome,
you might wonder if increasing it even more to daily testing would be even better.
This sometimes is the case, but not always, and regardless we often see rapidly "diminishing returns"
with frequency.

```python
veryfrequent = [sim(base,R=2,period=1,fraction_tested=1/28, plot=False)["risk"] for i in range(50)];
```

```python
violins([closed,infrequent,frequent, veryfrequent], ["Business closed", "100% / 28 days", "25% / week", "3.6% / day"])

```

## Can we prove this?

The above are results from simulations, but can we _prove_ that more frequent testing always helps?
To do so, we consider a simpler setting where an initial infection arrives at time $t_0$,
and at time $t-t_0$ there are $C^{t-t_0}$ infected individuals.
The factor $C$ is the growth per time unit (for example, if units are months then $C=10$
might be reasonable for COVID-19, but it of course depends on how well connected is the workplace
or institution).

In this model we ignore factors such as incubation period, false negatives, and simply ask, assuming the following
question: if we have a budget to test $p$ fraction of the individuals per time unit,
and use that by testing $\epsilon p$ fraction every $\epsilon$ time units, what is
the expected number of infected people at detection?

Let's assume $t_0=0$ for simplicity. If at each $\epsilon$ time units we test each individual
with probability $p$ independently, then the probability we will miss  at time $t$ all infected
individuals is  $(1 - \epsilon p)^{C^t}$. The expected risk is the sum over all $n$ of the probability
we missed detection up to time $(n-1)\epsilon$, times the probability we detected at time $n\epsilon$,
times $C^{n \epsilon}$ or in other words

$$\sum_{n=0}^{\infty}  \left( \prod_{i=0}^{n-1} (1 - \epsilon p)^{C^{\epsilon i}} \right) \cdot (1 - (1-\epsilon p)^{C^{\epsilon n}} \cdot C^{\epsilon n}$$

using the formula for arithmetic progressions this is

$$\sum_{n=0}^{\infty}   (1 - \epsilon p)^{(C^{\epsilon n} - 1)(C^\epsilon - 1)}  \cdot (1 - (1-\epsilon p)^{C^{\epsilon n}} \cdot C^{\epsilon n}$$

now letting $\epsilon \rightarrow 0$,  $(1-\epsilon p)^{1/\epsilon} \rightarrow \exp(-p)$,
while $\tfrac{1}{\epsilon}(C^{\epsilon} -1 ) \rightarrow \ln C$
and  $1-(1-\epsilon)^{C^{\epsilon n}} \approx  pC^tdt$ (where $dt = \epsilon$ and $t=\epsilon n$).
Hence the total expected cost converges to the integral

$$\int_{t=0}^\infty \exp(\tfrac{-p(C^t-1)}{\ln C}) p \cdot C^{2t} dt$$

Lucikly, [SymPy](https://www.sympy.org/) knows how to solve this integral

```python
import sympy as sp
t = sp.Symbol('t')
C = sp.Symbol('C')
p = sp.Symbol('p')
sp.Q.is_true(p>0)
sp.Q.is_true(p<1)
sp.Q.is_true(C>1)
sp.integrate(sp.exp(-p*(C**t - 1)/sp.log(C))*p*C**(2*t),t)
```

$\displaystyle \begin{cases} \frac{\left(- C^{t} p - \log{\left(C \right)}\right) e^{- \frac{p \left(C^{t} - 1\right)}{\log{\left(C \right)}}}}{p} & \text{for}\: p \neq 0 \\\begin{cases} \frac{C^{2 t} p}{2 \log{\left(C \right)}} & \text{for}\: 2 \log{\left(C \right)} \neq 0 \\p t & \text{otherwise} \end{cases} & \text{otherwise} \end{cases}$

Hence the total expected cost equals

$$-\exp\left(\tfrac{p\cdot(1-C^t)}{\ln C}\right)\left( \tfrac{\ln C}{p} + C^t \right)\Bigr|^{t=\infty}_{t=0}  = 1 +\tfrac{\ln C}{p}$$

We see that with frequent enough testing, the cost scales only _logarithmically_ with $C$.
In contrast, consider the case $p=1$. If we test the population every time unit, then
since the initial infection can come at any point, the risk would be $\int_0^1 C^t dt = (C-1)/\ln C$,
which scales _nearly linearly_ with $C$.

However, for small values of $C$, it can be the case that the risk when testing everyone per time period
is lower than the risk in the very frequent setting. However, this difference is limited:
[Wolfram Alpha](https://www.wolframalpha.com/input/?i=max+%281+%2B+ln%28x%29+-+%28x-1%29%2Fln%28x%29%29)
tells us that the maximum of $1 + \ln C - (C-1)/\ln C$ for $C>1$ is $3-e \approx 0.28$.

As usual, see the [paper](http://arxiv.org/abs/2007.04827) for more details. We also created a ["playground" notebook"](https://colab.research.google.com/drive/1pgpVBqchciVYl2qnb6q_L33xLmXogmVo?usp=sharing) where you can test out simulations of different scenarios