```python
import numpy as np
import matplotlib.pyplot as plt
```

# Funciones para que funcionen mis demas codigos xd

```python
# Funcion de la que me tomo mas tiempo hacerlo
def gauss(a,b):
    '''Funcion que retorna un array "x" despues de hacer una eliminacion de Gauss con pivoteo'''

    n = a.shape[0]
    # https://numpy.org/doc/1.18/reference/generated/numpy.ndarray.html
    # estamos creando un matriz columna
    x = np.ndarray( shape=(n,1), dtype = np.float64 )
    a_copy = a.copy() # con el fin de hacer segumiento a la matriz al final .
    s = [] # lista para alojar los valores maximos (es parte del pivoteo)
    l = [] # usada para reordenar como se hara la eliminacion

    # #### LLENAMOS LA LISTA "S" CON LOS VALORES MAXIMOS  #######
    for i in range(0,n):
        l.append(i) # vamos agregando .Establecemos esta matriz
        smax = 0 # el maximo de una fila es importante para el pivote
        for j in range(0,n):
            # se escoje el maximo entre todos los elemnos de una fila
            smax = max(smax,abs(a[i][j]))

        s.append(smax) # alojamos el maximo de la fila

    # ---------- print('l inicial ' ,l) (SEGUIMIENTO)
    # --------- print('s inicial ', s) (SEGUIMIENTO)

    # ##### PIVOTEO(ESCOJEMOS LA FILAS PIVOTES Y ELIMINAMOS) #########
    # k hace referencia a la columna donde se hara ceros seran creados en el array a_ij
    # Pero recuerda que los ceros no se crean realmente . Por que en esos espacios almacenamos
    # otras cosas(los multiplicadorese para descomposicion LU) .
    for k in range(0,n-1):

        rmax = 0
        # ##### SE ESCOJE LA FILA PIVOTE EN ESTE BLOQUE #########
        for i in range (k,n):

            # esto es para escoger la correcta fila pivote
            # se esta dividiendo (elemnto de columna k y fila [k-n][k])/los Smax
            r = abs( a[l[i]][k] / s[l[i]] )
            if (r > rmax):
                # j = i es para escoger el correcto pivote
                # rmax = r es para escoger la maxima proporcion(ratio)
                rmax,j = r,i
                # al final no usaremos rmax , solo es necesario para detener el if cuando sea necesario

        l[j],l[k] = l[k],l[j] # luego se cambia el lugar donde ocurre el maximo proporcion

        # ####### ELIMINACION GAUSSIANA PERO LA FILA PIVOTE LO DETERMINA "l" ##################
        for i in range(k+1,n):
            xmult = a[l[i]][k]/a[l[k]][k]
            a[l[i]][k] = xmult # los guardo para fines de LU y para eliminar "b"

            a_copy[l[i]][k] -= xmult*a[l[k]][k] # para hacer segumiento(se puede borrar)

            # este for hara segumiento de que se haga operaciones en toda la fila que no es pivot
            for j in range(k+1,n):
                a[l[i]][j] -= xmult*a[l[k]][j]
                a_copy[l[i]][j] -= xmult*a[l[k]][j] # para hacer seguimiento(se puede borrar)

    # --------- print('l final ' ,l) (SEGUIMIENTO)
    # --------- print('a final copy \n' , a_copy) (SEGUIMIENTO)
    # ##### SEGIMOS PIVOTEANDO PERO PARA LA MATRIZ b###
    for k in range(0,n-1):

        # recuerda el que ahora maneja el orden sera la lista "l"
        # debemos hacer las operaciones en el mismo orden que hemos hecho para "a"
        for i in range(k+1,n):
            b[l[i]] -= a[l[i]][k]*b[l[k]]

    #------ print('b final \n ' , b) (SEGUIMIENTO)

    # ######### AHORA HACEMOS LA SUSTITUCION BACKWARD ##########

    # espero se entienda por que -1 . Es por la cuenta por cero
    x[n-1] = b[l[n-1]]/a[l[n-1]][n-1]

    for i in range(n-2,-1,-1):
        summ = b[l[i]]
        for j in range(i+1,n):
            summ -= a[l[i]][j]*x[j]

        x[i] = summ/a[l[i]][i]

    return x
```

# Aca implementare el metodo de Euler simple

```python
def euler(f,t0=0,x0=0,limsup=0,n=100):
    '''
    Funcion que recibe como argumentos
    - f = funcion de R2-> R
    - t0,x0 = condicion inicial (valor inicial)
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    Y restorna un array de dos dimensiones , donde en la primera esta el dominio y la segunda es el rango predicho
    '''
    h = (limsup-t0)/n
    t = t0
    x = x0
    cal_valores = [] # rango predicho
    valores_t = [] # dominio donde se predice
    cal_valores.append(x)
    valores_t.append(t)
    for k in range(0,n):
        x = x + h*f(t,x)
        t = t + h
        cal_valores.append(x)
        valores_t.append(t)

    lista_resultados = np.array([valores_t,cal_valores],dtype=np.float64)

    return np.array(lista_resultados)
```

```python
# Probamos nuestra funcion si funciona xd
def funcion1prueba(t,x):
    return np.float64(1+x**2+t**3)

lista = euler(f=funcion1prueba,t0=1,x0=-4,limsup=2,n=5)
lista # x(2) aprox 4.2358...
```

    array([[ 1.        ,  1.2       ,  1.4       ,  1.6       ,  1.8       ,
             2.        ],
           [-4.        , -0.4       ,  0.1776    ,  0.93270835,  2.12589733,
             4.39618521]])

```python
np.linspace(0,1,4)
```

    array([0.        , 0.33333333, 0.66666667, 1.        ])

# Aca implementare el metodo de HEUN simple con iteracion

```python
def suma(a=10,b=20):
    return a/b
```

```python
def heun(f,t0=0,x0=0,limsup=0,n=100,error=10**(-3)):
    '''
    Funcion que recibe como argumentos
    - f = funcion de R2-> R
    - t0,x0 = condicion inicial (valor inicial)
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    - error = es el error para que se detenga las iteraciones en heun del corrector
    Y restorna un array de dos dimensiones , donde en la primera esta el dominio y la segunda es el rango predicho
    '''

    h = (limsup-t0)/n
    t = t0
    x = x0
    cal_valores = [] # rango predicho
    valores_t = [] # dominio donde se predice
    cal_valores.append(x)
    valores_t.append(t)
    for k in range(0,n):
        error_iter = error
        maximo = 0 # para controlar el while
        xe = x + h*f(t,x) # // predictor (fase intermedia)

        # con el while hacemos la correccion hasta tener una convergencia
        while(error_iter >= error and maximo < 20 ):
            x_old = xe
            xe = x + (f(t,x)+f(t+h,xe))/2 * h  # // corrector
            error_iter = abs((xe-x_old)/xe*100)
            maximo += 1 # este es para evitar que el programa se cuelgue

        x = xe # si no pones esto , (vas a tener muchos problemas)
        t = t + h
        cal_valores.append(xe)
        valores_t.append(t)

    lista_resultados = np.array([valores_t,cal_valores],dtype=np.float64)

    return np.array(lista_resultados)
```

```python
# probemos si funciona
def funcion2prueba(t,x):
    return np.float64(4*np.e**(0.8*t)-0.5*x)

lista = heun(f=funcion2prueba,t0=0,x0=2,limsup=1,n=1,error=10**(-3))
lista # x(1) real 6.1946314
```

    array([[0.        , 1.        ],
           [2.        , 6.36087068]])

# Aca aplicare metodo del punto medio

```python
def midpoint(f,t0=0,x0=0,limsup=0,n=100):
    '''
    Funcion que recibe como argumentos
    - f = funcion de R2-> R
    - t0,x0 = condicion inicial (valor inicial)
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    Y restorna un array de dos dimensiones , donde en la primera esta el dominio y la segunda es el rango predicho
    '''

    h = (limsup-t0)/n
    t = t0
    x = x0
    cal_valores = [] # rango predicho
    valores_t = [] # dominio donde se predice
    cal_valores.append(x)
    valores_t.append(t)
    for k in range(0,n):
        xe = x + h/2*f(t,x)
        x = x + h*f(t+h/2,xe)
        t = t + h
        cal_valores.append(x)
        valores_t.append(t)

    lista_resultados = np.array([valores_t,cal_valores],dtype=np.float64)

    return np.array(lista_resultados)
```

```python
# probemos si nuestro codigo funciona
# probemos si funciona
def funcion2prueba(t,x):
    return np.float64(4*np.e**(0.8*t)-0.5*x)

lista = midpoint(f=funcion2prueba,t0=0,x0=2,limsup=1,n=1)
lista # x(1) real 6.1946314
```

    array([[0.        , 1.        ],
           [2.        , 6.21729879]])

# Runge kutta

```python
# implemento runge kuta 4
def RK4(f=None,t0=0,x0=0,limsup=0,n=10):
    '''
    Funcion que recibe como argumentos
    - f = funcion de R2-> R
    - t0,x0 = condicion inicial (valor inicial)
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    Y restorna un array de dos dimensiones , donde en la primera esta el dominio y la segunda es el rango predicho
    '''
    h = (limsup-t0)/n
    t = t0
    x = x0
    cal_valores = [] # rango predicho
    valores_t = [] # dominio donde se predice
    cal_valores.append(x)
    valores_t.append(t)
    for i in range(0,n):
        k1 = h*f(t,x)
        k2 = h*f(t+1/2*h,x+1/2*k1)
        k3 = h*f(t+1/2*h,x+1/2*k2)
        k4 = h*f(t+h,x+k3)
        x = x + 1/6*(k1+2*k2+2*k3+k4)
        t = t0 + (i+1)*h
        cal_valores.append(x)
        valores_t.append(t)

    lista_resultados = np.array([valores_t,cal_valores],dtype=np.float64)
    return lista_resultados

# implemento runge kuta 3
def RK3(f=None,t0=0,x0=0,limsup=0,n=10):
    '''
    Funcion que recibe como argumentos
    - f = funcion de R2-> R
    - t0,x0 = condicion inicial (valor inicial)
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    Y restorna un array de dos dimensiones , donde en la primera esta el dominio y la segunda es el rango predicho
    '''
    h = (limsup-t0)/n
    t = t0
    x = x0
    cal_valores = [] # rango predicho
    valores_t = [] # dominio donde se predice
    cal_valores.append(x)
    valores_t.append(t)
    for i in range(0,n):
        k1 = h*f(t,x)
        k2 = h*f(t+1/2*h,x+1/2*k1)
        k3 = h*f(t+h,x-k1+2*k2)
        x = x + 1/6*(k1+4*k2+k3)
        t = t0 + (i+1)*h
        cal_valores.append(x)
        valores_t.append(t)

    lista_resultados = np.array([valores_t,cal_valores],dtype=np.float64)
    return lista_resultados
```

```python
# probamos si funciona
def funcion2prueba(t,x):
    return np.float64(4*np.e**(0.8*t)-0.5*x)

lista1 = RK4(f=funcion2prueba,t0=0,x0=2,limsup=1,n=2)
lista2 = RK3(f=funcion2prueba,t0=0,x0=2,limsup=1,n=2)
lista1,lista2 # x(1) real 6.1946314
```

    (array([[0.        , 0.5       , 1.        ],
            [2.        , 3.7516995 , 6.19504199]]),
     array([[0.        , 0.5       , 1.        ],
            [2.        , 3.75036978, 6.1918979 ]]))

```python
# implemento runge kuta 3
def RK2_ralston(f=None,t0=0,x0=0,limsup=0,n=10):
    '''
    Funcion que recibe como argumentos
    - f = funcion de R2-> R
    - t0,x0 = condicion inicial (valor inicial)
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    Y restorna un array de dos dimensiones , donde en la primera esta el dominio y la segunda es el rango predicho
    '''
    h = (limsup-t0)/n
    t = t0
    x = x0
    cal_valores = [] # rango predicho
    valores_t = [] # dominio donde se predice
    cal_valores.append(x)
    valores_t.append(t)
    for i in range(0,n):
        k1 = f(t,x)
        k2 = f(t+3/4*h,x+3/4*k1*h)
        x = x + 1/3*(k1+2*k2)*h
        t = t0 + (i+1)*h
        cal_valores.append(x)
        valores_t.append(t)
    lista_resultados = np.array([valores_t,cal_valores],dtype=np.float64)
    return lista_resultados
```

```python
# probamos si funciona
def funcion2prueba(t,x):
    return np.float64(4*np.e**(0.8*t)-0.5*x)

lista = RK2_ralston(f=funcion2prueba,t0=0,x0=2,limsup=1,n=4)
lista # x(1) real 6.1946314
```

    array([[0.        , 0.25      , 0.5       , 0.75      , 1.        ],
           [2.        , 2.8110145 , 3.75844035, 4.87753599, 6.21077359]])

# Resolvemos el problema de la derivada dejado por el profesor

```python
print("//////////////////////// Ejercicio 1 ///////////////////")
# Problema a resolver
# dy/dx = y*t**(3)-1.5*y con y(0)=1 , resolver en el intervalor [0,2]
Pasos = 10
def diff1(t,y):
    return np.float64(y*t**(3)-1.5*y)

# la funcion "y" analitica es:
def funcion_resuelta(t):
    return np.float64(np.e**( (t**4)/4-1.5*t ) )

# creamos el lugar donde graficaremos
# fig es la figura y axes son los ejes (son elementos de cada figura)
fig, axes = plt.subplots(nrows=1,ncols=1 , figsize = (8,6))
# graficamos el analitico
x = np.linspace(0,2,100)
axes.plot(x,funcion_resuelta(x),'b',label="Analitico")

# resolvemos el problema con el metodo de euler
solucion_euler = euler(f=diff1,t0=0,x0=1,limsup=2,n=Pasos)
axes.plot(solucion_euler[0],solucion_euler[1],'r-*',label="Euler")

# resolvemos el problema con el metodo de heund
solucion_heund = heun(f=diff1,t0=0,x0=1,limsup=2,n=Pasos,error=10**(-3))
axes.plot(solucion_heund[0],solucion_heund[1],'g-*',label="Heund")

# resolvemos el problema con el metodo de ralston
solucion_ralston = RK2_ralston(f=diff1,t0=0,x0=1,limsup=2,n=Pasos)
# axes.plot(solucion_ralston[0],solucion_ralston[1],'m-*',label="ralston")

# resolvemos el problema con el metodo de RK4
solucion_RK4 = RK4(f=diff1,t0=0,x0=1,limsup=2,n=Pasos)
# axes.plot(solucion_RK4[0],solucion_RK4[1],'k-*',label="ralston")

# grafiquemos el campo de pendientes
eje_x = np.linspace(0, 2, 20)
eje_y = np.linspace(0, 4.5, 20)
## para entender como funciona meshgrid
# https://numpy.org/doc/stable/reference/generated/numpy.meshgrid.html
X,Y  = np.meshgrid(eje_x, eje_y)
tangentes = diff1(X,Y)
proyeccion_y = np.sin(np.arctan(tangentes))
proyeccion_x = np.cos(np.arctan(tangentes))
axes.quiver(X, Y, proyeccion_x, proyeccion_y,width=0.002,color="k")

# añadidos de imagen
axes.grid(True)
axes.legend( loc='upper left', shadow=True)
fig.suptitle('Resultado de la pregunta 1 de aplicar distintos metodos', fontsize=20)

plt.show()

```

# Sistemas de ecuaciones lineales

```python
def RK4_system(F=None,t0=0,X0=0,limsup=0,nstep=10):
    '''
    Funcion que recibe como argumentos
    - F = funcion de Rn+1-> Rn F(X,t)
    - t0,X0 = condicion inicial (valor inicial) X0 es una matriz columna de orden "n"
    - limsup = hasta donde se quiere hacer la prediccion
    - nstep = numero de pasos
    Y restorna un array de n+1 dimensiones , donde en cada fila el primer elemento es el tiempo y los restantes son los valores predichos en ese tiempo
    Si se desea tener por columnas podrias usar retornar la transpuesta de la matriz (OJO) .
    '''
    h = (limsup-t0)/nstep
    t = t0
    X = np.array(X0,dtype=np.float64)
    # https://numpy.org/doc/stable/reference/generated/numpy.insert.html ( documentacion )
    valor_inicio = np.insert(X,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
    cal_valores = []
    cal_valores.append(valor_inicio) # agregamos la parte inicial
    for i in range(0,nstep):
        K1 = F(t,X)
        K2 = F(t+1/2*h,X+1/2*K1*h)
        K3 = F(t+1/2*h,X+1/2*K2*h)
        K4 = F(t+1/2*h,X+ K3*h)
        t = t0 + (i+1)*h
        X = X + 1/6*(K1+ 2*K2+ 2*K3 + K4)*h
        valor_calculado = np.insert(X,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
        cal_valores.append(valor_calculado) # agregamos la parte inicial

    lista_resultados = np.array(cal_valores,dtype=np.float64)
    return lista_resultados # aca puedes retornar esto o su transpuesta
```

```python
# Probemos nuestra funcion
def Funcionprueba(t,X):
    diffx = X[0] - X[1] + 2*t - t**2- t**3
    diffy = X[0] + X[1] - 4*t**2 + t**3
    return np.array([diffx,diffy],dtype=np.float64)

respuesta = RK4_system(F=Funcionprueba,t0=0,X0=np.array([1,0]),limsup=1,nstep=10)
respuesta
```

    array([[0.        , 1.        , 0.        ],
           [0.1       , 1.10812373, 0.10981831],
           [0.2       , 1.23402758, 0.23611003],
           [0.3       , 1.37506076, 0.3748197 ],
           [0.4       , 1.52809382, 0.52179317],
           [0.5       , 1.68948175, 0.67271496],
           [0.6       , 1.85503049, 0.82303526],
           [0.7       , 2.01996803, 0.96788605],
           [0.8       , 2.17892194, 1.10198559],
           [0.9       , 2.32590516, 1.21953096],
           [1.        , 2.4543123 , 1.31407829]])

# Aca implementare Euler implicito y explicito para sistema de ecuaciones diferenciales

### La forma implicita y explicita
aca resolvere un tipo de ecuaciones lineales con euler implicito
$$ Y' = AY $$
donde $A$ es un matriz constante y $Y$ es $[y1,y2,y3,y4....]$ . Luego el metodo de euler implicito involucra
$$ Y_{i+1} = Y_{i} + Y'_{i+1}\cdot h $$
$$ Y_{i+1} = Y_{i} + AY_{i+1}\cdot h $$
Finalmente el sistema iterativo para **euler implicito** queda como
$$ \left( I - A h \right) Y_{i+1} = Y_i $$
En cammbio para **euler explicito** tenemos
$$ \left( I + A h \right) Y_{i} = Y_{i+1} $$

```python
# Metodo de Euler implicito
def euler_implicito(A=None,Yi0=None,t0=0,limsup=0,n=100):
    '''
    Funcion que recibe como argumentos
    - A = matriz para el problema de Y' = AY a resolver
    - t0,Yi0 = condiciones iniciales, Yi0 es un array fila de orden "n"
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    Y restorna un array de n+1 dimensiones , donde en cada fila el primer elemento es el tiempo y los restantes son los valores predichos en ese tiempo
    Si se desea tener por columnas podrias usar retornar la transpuesta de la matriz (OJO) .
    '''
    # armamos la condicion inicial
    h = (limsup-t0)/n
    t = t0
    Y = np.array(Yi0,dtype=np.float64)
    # https://numpy.org/doc/stable/reference/generated/numpy.insert.html ( documentacion )
    valor_inicio = np.insert(Y,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
    cal_valores = []
    cal_valores.append(valor_inicio) # agregamos la parte inicial

    # armamos la matriz que se usara en la relacion de recurrencia (I-Ah)Y_i+1 = Y_i
    orden = A.shape[1] # nos da el orden de la matriz cuadrada
    # https://numpy.org/doc/stable/reference/generated/numpy.identity.html
    I = np.identity(orden , dtype=np.float64)
    B = I- A*h # la matriz para la recurrencia

    # hacemos la recurrencia
    for i in range(0,n):
        # recuerda que se devuelve una matriz columan y lo que quieres es matriz fila por eso el T ( transpuesta)
        Y = gauss(B.copy(),Y.T).T
        t = t0 + (i+1)*h
        # OJO ( [[1,3]] es lo mimso [1,3] para numpy . Haz la prueba y veras )
        valor_calculado = np.insert(Y,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
        cal_valores.append(valor_calculado) # agregamos la parte inicial

    lista_resultados = np.array(cal_valores,dtype=np.float64)
    return lista_resultados # aca puedes retornar esto o su transpuesta

# # Metodo de Euler explicito
def euler_explicito(A=None,Yi0=None,t0=0,limsup=0,n=100):
    '''
    Funcion que recibe como argumentos
    - A = matriz para el problema de Y' = AY a resolver
    - t0,Yi0 = condiciones iniciales, Yi0 es un array fila de orden "n"
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    Y restorna un array de n+1 dimensiones , donde en cada fila el primer elemento es el tiempo y los restantes son los valores predichos en ese tiempo
    Si se desea tener por columnas podrias usar retornar la transpuesta de la matriz (OJO) .
    '''
    # armamos la condicion inicial
    h = (limsup-t0)/n
    t = t0
    Y = np.array(Yi0,dtype=np.float64)
    # https://numpy.org/doc/stable/reference/generated/numpy.insert.html ( documentacion )
    valor_inicio = np.insert(Y,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
    cal_valores = []
    cal_valores.append(valor_inicio) # agregamos la parte inicial

    # armamos la matriz que se usara en la relacion de recurrencia (I-Ah)Y_i+1 = Y_i
    orden = A.shape[1] # nos da el orden de la matriz cuadrada
    # https://numpy.org/doc/stable/reference/generated/numpy.identity.html
    I = np.identity(orden , dtype=np.float64)
    B = I + A*h # la matriz para la recurrencia

    # hacemos la relacion de recurrencia
    for i in range(0,n):
        # recuerda que se devuelve una matriz columan y lo que quieres es matriz fila por eso el T ( transpuesta)
        Y = (B.dot(Y.T)).T
        t = t0 + (i+1)*h
        # OJO ( [[1,3]] es lo mimso [1,3] para numpy . Haz la prueba y veras )
        valor_calculado = np.insert(Y,0,t) # lo que hace es crear una copia de X y aumentarle al inicio el "t"
        cal_valores.append(valor_calculado) # agregamos la parte inicial

    lista_resultados = np.array(cal_valores,dtype=np.float64)
    return lista_resultados # aca puedes retornar esto o su transpuesta
```

# veamos si funciona nuestro codigo

### La solucion teorica
$$ \begin{equation}
\begin{pmatrix}
X_1' \\
x_2'
\end{pmatrix} =
\begin{pmatrix}
999 & 1999 \\
-1000 & -2000
\end{pmatrix}
\begin{pmatrix}
X_1 \\
X_2
\end{pmatrix}
\end{equation} $$
Es de la forma $x_1 = Ae^{mt}$ , $x_2 = Be^{mt}$ donde m esta dado por
$$ m^2 - (999 -2000)m + ( (999) \cdot (-2000) - (1999)\cdot (-1000) ) = 0 $$

```python
# usare simpy x1 = x y x2 = y
from sympy import *
t = symbols('t')
x, y = symbols('x, y', cls=Function)
eq = (Eq(Derivative(x(t),t), 999*x(t) + 1999*y(t)), Eq(Derivative(y(t),t), -1000*x(t) + -2000*y(t)))
ics = {x(0): 1, y(0): 1}
resuelto = dsolve(eq,ics=ics)
resuelto
```

    [Eq(x(t), 3998*exp(-t)/999 - 2999*exp(-1000*t)/999),
     Eq(y(t), -2000*exp(-t)/999 + 2999*exp(-1000*t)/999)]

```python
# solucion de x(t)
x1 = resuelto[0]
x1
```

$\displaystyle x{\left(t \right)} = \frac{3998 e^{- t}}{999} - \frac{2999 e^{- 1000 t}}{999}$

```python
# solucion de y(t)
x2 = resuelto[1]
x2
```

$\displaystyle y{\left(t \right)} = - \frac{2000 e^{- t}}{999} + \frac{2999 e^{- 1000 t}}{999}$

```python
0.2/100

```

    0.002

```python
print("/////////////// Ejercicio 2 /////////////////////")

MatrizA = np.array([[999,1999],[-1000,-2000]],dtype=np.float64)
# euler implicito
resultado1 = euler_implicito(A=MatrizA,Yi0=np.array([1,1]),t0=0,limsup=0.2,n=100)
# euler explicito
resultado2 = euler_explicito(A=MatrizA,Yi0=np.array([1,1]),t0=0,limsup=0.2,n=100)

# aplicando runge kutta
# Probemos nuestra funcion
def Funcionproblema(t,X):
    x1 = X[0]
    x2 = X[1]
    diffx1 = 999*x1 + 1999*x2
    diffx2 = -1000*x1 - 2000*x2
    return np.array([diffx1,diffx2],dtype=np.float64)

resultado3 = RK4_system(F=Funcionproblema,t0=0,X0=np.array([1,1]),limsup=0.2,nstep=4)

# solucion real calculado magicamente
def solx(t):
    return np.float64( 3998/999*np.e**(-t) - 2999/999*np.e**(-1000*t))
def soly(t):
    return np.float64( -2000/999*np.e**(-t) + 2999/999*np.e**(-1000*t))
resultado1, resultado2,resultado3,solx(0.2),soly(0.2)
```

    (array([[ 0.00000000e+00,  1.00000000e+00,  1.00000000e+00],
            [ 2.00000000e-03,  2.99334664e+00, -9.97338656e-01],
            [ 4.00000000e-03,  3.65248611e+00, -1.66046218e+00],
            [ 6.00000000e-03,  3.86690046e+00, -1.87885262e+00],
            [ 8.00000000e-03,  3.93308367e+00, -1.94900399e+00],
            [ 1.00000000e-02,  3.94986707e+00, -1.96974763e+00],
            [ 1.20000000e-02,  3.95019439e+00, -1.97402728e+00],
            [ 1.40000000e-02,  3.94504686e+00, -1.97282420e+00],
            [ 1.60000000e-02,  3.93808488e+00, -1.96979879e+00],
            [ 1.80000000e-02,  3.93052856e+00, -1.96617118e+00],
            [ 2.00000000e-02,  3.92278457e+00, -1.96234806e+00],
            [ 2.20000000e-02,  3.91498845e+00, -1.95846499e+00],
            [ 2.40000000e-02,  3.90718536e+00, -1.95456714e+00],
            [ 2.60000000e-02,  3.89939034e+00, -1.95066957e+00],
            [ 2.80000000e-02,  3.89160838e+00, -1.94677727e+00],
            [ 3.00000000e-02,  3.88384112e+00, -1.94289190e+00],
            [ 3.20000000e-02,  3.87608908e+00, -1.93901401e+00],
            [ 3.40000000e-02,  3.86835242e+00, -1.93514377e+00],
            [ 3.60000000e-02,  3.86063117e+00, -1.93128122e+00],
            [ 3.80000000e-02,  3.85292533e+00, -1.92742638e+00],
            [ 4.00000000e-02,  3.84523486e+00, -1.92357922e+00],
            [ 4.20000000e-02,  3.83755974e+00, -1.91973974e+00],
            [ 4.40000000e-02,  3.82989994e+00, -1.91590792e+00],
            [ 4.60000000e-02,  3.82225543e+00, -1.91208376e+00],
            [ 4.80000000e-02,  3.81462618e+00, -1.90826722e+00],
            [ 5.00000000e-02,  3.80701215e+00, -1.90445831e+00],
            [ 5.20000000e-02,  3.79941333e+00, -1.90065699e+00],
            [ 5.40000000e-02,  3.79182967e+00, -1.89686327e+00],
            [ 5.60000000e-02,  3.78426115e+00, -1.89307711e+00],
            [ 5.80000000e-02,  3.77670773e+00, -1.88929851e+00],
            [ 6.00000000e-02,  3.76916939e+00, -1.88552746e+00],
            [ 6.20000000e-02,  3.76164610e+00, -1.88176393e+00],
            [ 6.40000000e-02,  3.75413782e+00, -1.87800792e+00],
            [ 6.60000000e-02,  3.74664453e+00, -1.87425940e+00],
            [ 6.80000000e-02,  3.73916620e+00, -1.87051836e+00],
            [ 7.00000000e-02,  3.73170280e+00, -1.86678479e+00],
            [ 7.20000000e-02,  3.72425429e+00, -1.86305867e+00],
            [ 7.40000000e-02,  3.71682065e+00, -1.85933999e+00],
            [ 7.60000000e-02,  3.70940184e+00, -1.85562874e+00],
            [ 7.80000000e-02,  3.70199785e+00, -1.85192489e+00],
            [ 8.00000000e-02,  3.69460863e+00, -1.84822843e+00],
            [ 8.20000000e-02,  3.68723416e+00, -1.84453935e+00],
            [ 8.40000000e-02,  3.67987441e+00, -1.84085764e+00],
            [ 8.60000000e-02,  3.67252935e+00, -1.83718327e+00],
            [ 8.80000000e-02,  3.66519896e+00, -1.83351624e+00],
            [ 9.00000000e-02,  3.65788319e+00, -1.82985652e+00],
            [ 9.20000000e-02,  3.65058203e+00, -1.82620411e+00],
            [ 9.40000000e-02,  3.64329543e+00, -1.82255900e+00],
            [ 9.60000000e-02,  3.63602339e+00, -1.81892115e+00],
            [ 9.80000000e-02,  3.62876586e+00, -1.81529057e+00],
            [ 1.00000000e-01,  3.62152281e+00, -1.81166724e+00],
            [ 1.02000000e-01,  3.61429422e+00, -1.80805114e+00],
            [ 1.04000000e-01,  3.60708006e+00, -1.80444225e+00],
            [ 1.06000000e-01,  3.59988030e+00, -1.80084057e+00],
            [ 1.08000000e-01,  3.59269491e+00, -1.79724608e+00],
            [ 1.10000000e-01,  3.58552386e+00, -1.79365876e+00],
            [ 1.12000000e-01,  3.57836713e+00, -1.79007860e+00],
            [ 1.14000000e-01,  3.57122468e+00, -1.78650559e+00],
            [ 1.16000000e-01,  3.56409649e+00, -1.78293971e+00],
            [ 1.18000000e-01,  3.55698252e+00, -1.77938095e+00],
            [ 1.20000000e-01,  3.54988276e+00, -1.77582929e+00],
            [ 1.22000000e-01,  3.54279716e+00, -1.77228472e+00],
            [ 1.24000000e-01,  3.53572571e+00, -1.76874723e+00],
            [ 1.26000000e-01,  3.52866837e+00, -1.76521680e+00],
            [ 1.28000000e-01,  3.52162512e+00, -1.76169341e+00],
            [ 1.30000000e-01,  3.51459593e+00, -1.75817705e+00],
            [ 1.32000000e-01,  3.50758077e+00, -1.75466772e+00],
            [ 1.34000000e-01,  3.50057961e+00, -1.75116539e+00],
            [ 1.36000000e-01,  3.49359243e+00, -1.74767005e+00],
            [ 1.38000000e-01,  3.48661919e+00, -1.74418168e+00],
            [ 1.40000000e-01,  3.47965987e+00, -1.74070028e+00],
            [ 1.42000000e-01,  3.47271444e+00, -1.73722583e+00],
            [ 1.44000000e-01,  3.46578287e+00, -1.73375832e+00],
            [ 1.46000000e-01,  3.45886514e+00, -1.73029772e+00],
            [ 1.48000000e-01,  3.45196122e+00, -1.72684403e+00],
            [ 1.50000000e-01,  3.44507108e+00, -1.72339724e+00],
            [ 1.52000000e-01,  3.43819469e+00, -1.71995732e+00],
            [ 1.54000000e-01,  3.43133203e+00, -1.71652427e+00],
            [ 1.56000000e-01,  3.42448306e+00, -1.71309808e+00],
            [ 1.58000000e-01,  3.41764776e+00, -1.70967872e+00],
            [ 1.60000000e-01,  3.41082611e+00, -1.70626619e+00],
            [ 1.62000000e-01,  3.40401808e+00, -1.70286047e+00],
            [ 1.64000000e-01,  3.39722363e+00, -1.69946154e+00],
            [ 1.66000000e-01,  3.39044274e+00, -1.69606941e+00],
            [ 1.68000000e-01,  3.38367539e+00, -1.69268404e+00],
            [ 1.70000000e-01,  3.37692155e+00, -1.68930543e+00],
            [ 1.72000000e-01,  3.37018119e+00, -1.68593356e+00],
            [ 1.74000000e-01,  3.36345428e+00, -1.68256842e+00],
            [ 1.76000000e-01,  3.35674080e+00, -1.67921000e+00],
            [ 1.78000000e-01,  3.35004071e+00, -1.67585829e+00],
            [ 1.80000000e-01,  3.34335401e+00, -1.67251326e+00],
            [ 1.82000000e-01,  3.33668065e+00, -1.66917491e+00],
            [ 1.84000000e-01,  3.33002060e+00, -1.66584322e+00],
            [ 1.86000000e-01,  3.32337386e+00, -1.66251819e+00],
            [ 1.88000000e-01,  3.31674038e+00, -1.65919979e+00],
            [ 1.90000000e-01,  3.31012014e+00, -1.65588801e+00],
            [ 1.92000000e-01,  3.30351311e+00, -1.65258285e+00],
            [ 1.94000000e-01,  3.29691927e+00, -1.64928428e+00],
            [ 1.96000000e-01,  3.29033859e+00, -1.64599229e+00],
            [ 1.98000000e-01,  3.28377105e+00, -1.64270688e+00],
            [ 2.00000000e-01,  3.27721662e+00, -1.63942802e+00]]),
     array([[ 0.00000000e+00,  1.00000000e+00,  1.00000000e+00],
            [ 2.00000000e-03,  6.99600000e+00, -5.00000000e+00],
            [ 4.00000000e-03,  9.84008000e-01,  1.00800000e+00],
            [ 6.00000000e-03,  6.98003998e+00, -4.99201600e+00],
            [ 8.00000000e-03,  9.68079904e-01,  1.01596803e+00],
            [ 1.00000000e-02,  6.96414374e+00, -4.98406390e+00],
            [ 1.20000000e-02,  9.52215457e-01,  1.02390422e+00],
            [ 1.40000000e-02,  6.94831103e+00, -4.97614358e+00],
            [ 1.60000000e-02,  9.36414404e-01,  1.03180870e+00],
            [ 1.80000000e-02,  6.93254157e+00, -4.96825491e+00],
            [ 2.00000000e-02,  9.20676492e-01,  1.03968159e+00],
            [ 2.20000000e-02,  6.91683514e+00, -4.96039777e+00],
            [ 2.40000000e-02,  9.05001469e-01,  1.04752303e+00],
            [ 2.60000000e-02,  6.90119147e+00, -4.95257202e+00],
            [ 2.80000000e-02,  8.89389083e-01,  1.05533313e+00],
            [ 3.00000000e-02,  6.88561030e+00, -4.94477754e+00],
            [ 3.20000000e-02,  8.73839084e-01,  1.06311201e+00],
            [ 3.40000000e-02,  6.87009141e+00, -4.93701421e+00],
            [ 3.60000000e-02,  8.58351223e-01,  1.07085982e+00],
            [ 3.80000000e-02,  6.85463452e+00, -4.92928190e+00],
            [ 4.00000000e-02,  8.42925251e-01,  1.07857666e+00],
            [ 4.20000000e-02,  6.83923940e+00, -4.92158049e+00],
            [ 4.40000000e-02,  8.27560922e-01,  1.08626267e+00],
            [ 4.60000000e-02,  6.82390580e+00, -4.91390986e+00],
            [ 4.80000000e-02,  8.12257989e-01,  1.09391796e+00],
            [ 5.00000000e-02,  6.80863347e+00, -4.90626987e+00],
            [ 5.20000000e-02,  7.97016206e-01,  1.10154267e+00],
            [ 5.40000000e-02,  6.79342217e+00, -4.89866042e+00],
            [ 5.60000000e-02,  7.81835329e-01,  1.10913690e+00],
            [ 5.80000000e-02,  6.77827166e+00, -4.89108137e+00],
            [ 6.00000000e-02,  7.66715115e-01,  1.11670079e+00],
            [ 6.20000000e-02,  6.76318168e+00, -4.88353261e+00],
            [ 6.40000000e-02,  7.51655321e-01,  1.12423446e+00],
            [ 6.60000000e-02,  6.74815201e+00, -4.87601401e+00],
            [ 6.80000000e-02,  7.36655707e-01,  1.13173802e+00],
            [ 7.00000000e-02,  6.73318240e+00, -4.86852546e+00],
            [ 7.20000000e-02,  7.21716031e-01,  1.13921159e+00],
            [ 7.40000000e-02,  6.71827260e+00, -4.86106683e+00],
            [ 7.60000000e-02,  7.06836053e-01,  1.14665530e+00],
            [ 7.80000000e-02,  6.70342238e+00, -4.85363801e+00],
            [ 8.00000000e-02,  6.92015536e-01,  1.15406927e+00],
            [ 8.20000000e-02,  6.68863151e+00, -4.84623887e+00],
            [ 8.40000000e-02,  6.77254242e-01,  1.16145361e+00],
            [ 8.60000000e-02,  6.67389973e+00, -4.83886930e+00],
            [ 8.80000000e-02,  6.62551934e-01,  1.16880844e+00],
            [ 9.00000000e-02,  6.65922683e+00, -4.83152918e+00],
            [ 9.20000000e-02,  6.47908377e-01,  1.17613388e+00],
            [ 9.40000000e-02,  6.64461256e+00, -4.82421839e+00],
            [ 9.60000000e-02,  6.33323335e-01,  1.18343005e+00],
            [ 9.80000000e-02,  6.63005669e+00, -4.81693681e+00],
            [ 1.00000000e-01,  6.18796575e-01,  1.19069706e+00],
            [ 1.02000000e-01,  6.61555898e+00, -4.80968433e+00],
            [ 1.04000000e-01,  6.04327864e-01,  1.19793504e+00],
            [ 1.06000000e-01,  6.60111921e+00, -4.80246083e+00],
            [ 1.08000000e-01,  5.89916970e-01,  1.20514409e+00],
            [ 1.10000000e-01,  6.58673714e+00, -4.79526620e+00],
            [ 1.12000000e-01,  5.75563661e-01,  1.21232433e+00],
            [ 1.14000000e-01,  6.57241253e+00, -4.78810032e+00],
            [ 1.16000000e-01,  5.61267709e-01,  1.21947588e+00],
            [ 1.18000000e-01,  6.55814517e+00, -4.78096307e+00],
            [ 1.20000000e-01,  5.47028883e-01,  1.22659886e+00],
            [ 1.22000000e-01,  6.54393483e+00, -4.77385434e+00],
            [ 1.24000000e-01,  5.32846956e-01,  1.23369337e+00],
            [ 1.26000000e-01,  6.52978126e+00, -4.76677402e+00],
            [ 1.28000000e-01,  5.18721699e-01,  1.24075953e+00],
            [ 1.30000000e-01,  6.51568426e+00, -4.75972199e+00],
            [ 1.32000000e-01,  5.04652888e-01,  1.24779745e+00],
            [ 1.34000000e-01,  6.50164358e+00, -4.75269814e+00],
            [ 1.36000000e-01,  4.90640295e-01,  1.25480726e+00],
            [ 1.38000000e-01,  6.48765901e+00, -4.74570236e+00],
            [ 1.40000000e-01,  4.76683696e-01,  1.26178905e+00],
            [ 1.42000000e-01,  6.47373033e+00, -4.73873453e+00],
            [ 1.44000000e-01,  4.62782868e-01,  1.26874294e+00],
            [ 1.46000000e-01,  6.45985730e+00, -4.73179455e+00],
            [ 1.48000000e-01,  4.48937588e-01,  1.27566904e+00],
            [ 1.50000000e-01,  6.44603971e+00, -4.72488230e+00],
            [ 1.52000000e-01,  4.35147633e-01,  1.28256747e+00],
            [ 1.54000000e-01,  6.43227734e+00, -4.71799767e+00],
            [ 1.56000000e-01,  4.21412783e-01,  1.28943833e+00],
            [ 1.58000000e-01,  6.41856996e+00, -4.71114055e+00],
            [ 1.60000000e-01,  4.07732818e-01,  1.29628173e+00],
            [ 1.62000000e-01,  6.40491735e+00, -4.70431083e+00],
            [ 1.64000000e-01,  3.94107517e-01,  1.30309779e+00],
            [ 1.66000000e-01,  6.39131930e+00, -4.69750841e+00],
            [ 1.68000000e-01,  3.80536664e-01,  1.30988661e+00],
            [ 1.70000000e-01,  6.37777559e+00, -4.69073316e+00],
            [ 1.72000000e-01,  3.67020039e-01,  1.31664830e+00],
            [ 1.74000000e-01,  6.36428600e+00, -4.68398499e+00],
            [ 1.76000000e-01,  3.53557427e-01,  1.32338298e+00],
            [ 1.78000000e-01,  6.35085031e+00, -4.67726379e+00],
            [ 1.80000000e-01,  3.40148612e-01,  1.33009074e+00],
            [ 1.82000000e-01,  6.33746831e+00, -4.67056944e+00],
            [ 1.84000000e-01,  3.26793378e-01,  1.33677170e+00],
            [ 1.86000000e-01,  6.32413979e+00, -4.66390185e+00],
            [ 1.88000000e-01,  3.13491511e-01,  1.34342596e+00],
            [ 1.90000000e-01,  6.31086453e+00, -4.65726089e+00],
            [ 1.92000000e-01,  3.00242799e-01,  1.35005363e+00],
            [ 1.94000000e-01,  6.29764231e+00, -4.65064648e+00],
            [ 1.96000000e-01,  2.87047029e-01,  1.35665481e+00],
            [ 1.98000000e-01,  6.28447294e+00, -4.64405850e+00],
            [ 2.00000000e-01,  2.73903989e-01,  1.36322962e+00]]),
     array([[ 0.00000000e+00,  1.00000000e+00,  1.00000000e+00],
            [ 5.00000000e-02, -7.22831244e+05,  7.22833146e+05],
            [ 1.00000000e-01, -1.74047356e+11,  1.74047356e+11],
            [ 1.50000000e-01, -4.19078765e+16,  4.19078765e+16],
            [ 2.00000000e-01, -1.00907601e+22,  1.00907601e+22]]),
     3.27656211291869,
     -1.6391006067627265)

# Graficamos para ver mejor la solucion

```python
# creamos el lugar donde graficaremos
# fig es la figura y axes son los ejes (son elementos de cada figura)
fig, axes = plt.subplots(nrows=1,ncols=2 , figsize = (15,6))
# graficamos el analitico
x = np.linspace(0,0.2,100)

# para X1
axes[0].plot(x,solx(x),'b',label="Analitico para x(t)")
axes[0].plot(resultado1.T[0],resultado1.T[1],'r-*',label="Euler implicito")
axes[0].plot(resultado2.T[0],resultado2.T[1],'g-*',label="Euler explicito")
axes[0].set_ylim(0.8,4.2)
axes[0].set_title('Para X1')
axes[0].grid(True)
axes[0].legend( loc='lower right', shadow=True)

axes[1].plot(x,soly(x),'b',label="Analitico para y(t)")
axes[1].plot(resultado1.T[0],resultado1.T[2],'r-*',label="Euler implicito")
axes[1].plot(resultado2.T[0],resultado2.T[2],'g-*',label="Euler explicito")
axes[1].set_ylim(-2.1,1.1)
axes[1].set_title('Para x2')
axes[1].grid(True)
axes[1].legend( loc='upper right', shadow=True)
# añadidos de imagen

fig.suptitle('Resultado de la pregunta de aplicar Euler Implicito y explicito', fontsize=20)
plt.show()
```

# Aca implementare heun modificado

```python
def heun_modificado(f,t0=0,x0=0,x1=0,limsup=0,n=100,error=10**(-3)):
    '''
    Funcion que recibe como argumentos
    - f = funcion de R2-> R
    - t0,x0,xant0 = condicion inicial (valor inicial) , x0 y x1 son los que se necesita para este metodo multipaso
    - limsup = hasta donde se quiere hacer la prediccion
    - n = numero de pasos
    - error = es el error para que se detenga las iteraciones en heun del corrector
    Y restorna un array de dos dimensiones , donde en la primera esta el dominio y la segunda es el rango predicho
    '''

    h = (limsup-t0)/n
    t = t0
    xant = x0
    x = x1
    cal_valores = [] # rango predicho
    valores_t = [] # dominio donde se predice
    cal_valores.append(x)
    valores_t.append(t)
    for k in range(0,n):
        error_iter = error
        maximo = 0 # para controlar el while
        xe = xant + 2*h*f(t,x) # // predictor (fase intermedia)
        xant = x # como ya usamos el x ahora se convierte en xant , a partir de aca cambia el x ( por eso es apropiado hace aca el cambio)

        # con el while hacemos la correccion hasta tener una convergencia
        while(error_iter >= error and maximo < 15 ):
            x_old = xe
            xe = x + (f(t,x)+f(t+h,xe))/2 * h  # // corrector
            error_iter = abs((xe-x_old)/xe*100)
            maximo += 1 # este es para evitar que el programa se cuelgue

        x = xe # es importante que pongas esto , si no lo pones tendras cosas raras
        t = t + h
        cal_valores.append(xe)
        valores_t.append(t)

    lista_resultados = np.array([valores_t,cal_valores],dtype=np.float64)

    return np.array(lista_resultados)

```

# Aplicamos al problema que dejo el profe

```python
# hallamos la solucion analitica
# usare simpy
from sympy import *
t = symbols('t')
x = symbols('x', cls=Function)
eq = Eq(Derivative(x(t),t),-0.5*x(t)+exp(-t))
ics = {x(2): 4.143883}
resuelto = dsolve(eq,ics=ics)
resuelto # solucion analitica
```

$\displaystyle x{\left(t \right)} = \left(12.0000007405032 - 2.0 e^{- 0.5 t}\right) e^{- 0.5 t}$

```python
# probemos si funciona
def diffx(t,x):
    return np.float64(-0.5*x+np.e**(-t))

resultado4 = heun_modificado(f=diffx,t0=2,x0=5.222138,x1=4.143883,limsup=3,n=2,error=10**(-5))

# solucion real
def solxreal(t):
    return np.float64( ( 12.0000007405032-2.0*np.e**(-0.5*t) )*np.e**(-0.5*t) )

# comprobamos
resultado4 , solxreal(2)
```

    (array([[2.        , 2.5       , 3.        ],
            [4.143883  , 3.27133572, 2.57367712]]),
     4.1438829999999856)

# Visualizamos graficamente

```python
# creamos el lugar donde graficaremos
# fig es la figura y axes son los ejes (son elementos de cada figura)
fig, axes = plt.subplots(nrows=1,ncols=1 , figsize = (15,6))
# graficamos el analitico
x = np.linspace(2,3,100)
# para X1
axes.plot(x,solxreal(x),'b',label="Analitico para x(t)")
axes.plot(resultado4[0],resultado4[1],'r-*',label="Heun modificado")
axes.grid(True)
axes.legend( loc='upper right', shadow=True)

fig.suptitle('Resultado de la pregunta de aplicar Heun modificado', fontsize=20)
plt.show()
```


## Learning Process

Here in this section we are going to partly describy and in the remaining to test and evaluate performance of various machine learning models that we selected and adopted to built up learning models for accomplishing supervised machine learning tasks related to classification problems. More precisely, we focused on binary classification problemsa, since the target variables, that is T-OR-D feature, amongst the 12 features from which the dataset is made of, and from which the more or less hundered of records are described, is a binary categorical feature which can assume the two values represented by labels DECK and THROUGH describing in two distinct manner a property about each bridge within the dataset, that property refers to the system used for constructing the bridge surface, that is commonly called deck, for let veicheles or trains or whatevert to cross the rivers that are three distinct rivers: A, M, O. Where A stands for Allegheny river, while M stands for Monongahela river and lastly O stands for Ohio river.

Before describing the fine tuning process applied to the different models accordingly to their own major properties, characteristics and features, we have decided and established to test each model performance by looking at how well all of them is going just exploiting the default setting and runnng cross-validation protocol, in other word also referred to as policy, to check the accuracy level, as ana instance and some other metrics.

To be more detailed we follow the common machine leaning working flow, that requires to split the data set, after having preprocessed it properly and in a suitable manner to meet mahcine learning models needs, into subsets commonly referred  to as training set and test set. Where, the former is exploited to build up a inference model and the latter is used to check model's performance as weel as behavior up on held out sample of istances never seen before and so those examples that the learning model wasn't provided with to learn weights and select right hyper-params to plug in back into model at the end of training procedure, in order to later test the model with the found weights as well as hyper-parameters and if it meet our requirements in terms of performance values reached at test time, then it will be ready for deployments.

\begin{figure}[h]
\caption{Machine Learning Workflow Example}
\centering
\includegraphics[width=0.5\textwidth]{images/machine_learning_workflow_2.png}
\end{figure}

As we can see from the image just above what we have to do, following the purposed machine learning scheme or work flow are the subsequent steps:

__(1) Split Dataset into Train and Test sets__:
After having done with preprocessing phase we separate or divide the dataset into training set and test set, where usually the training set is bigger than the test set, and in some cases  the test set, for recording some kind of performance measures, can be made of just a single instance against which the model will be tested and checked.

__(2) Split Train set into smaller Train set and a Validation set__:
Once  have made the first split then we hold out a part for later checking the test set and we focus on training set. In fact once we have selected a machine learning model amongst those we want to adopt to fit a model and compare the results obtained we try to indentify the best fit parameters to setting the model with them. To reach such a goal we can adopt different aproaches to split further the training set into a validation set and a smaller train set in order to emulate before doing test phase a possible beahvior of the trainng model once we think it is ready for the following phase that is test step.
There are several procedure for establishing how to divide training set into two new subsets, that are validation and a little bit smaller train set, where all of them are also connected or reffered to a learning algorithm called cross-validation which consists roughly speaking into testing the model against a smaller portion of training set in order to record and measure the performance of a model before saying we are ready to proceed with test phase. Among the existing Cross-Validation Procedure we adopt and so describe briefly the following:

- **K-fold Cross-Validation**: which is a cross validation protocol exploited so that we split the training set into K-folds, in other words K-subsets all of the same size, a part eventually the last one which will be the remainder set of samples. One at a time the K-fold are left out and the model is firstly trained against K-1 subsets(folds) and then test against the left out fold for recording the performanc. At the end of the k-times we have trained the model we have recorded the performance measures and we so can average the results and understand how model in average is well doing. In oder words we can either assume the mean value as the driving value to assume the model as satisfying our constraint on performance measures or adopt the best result amongst the k-trains as the settings for hyper-params to adopt. This procedure is feasible and suitable if we do not carea botu the fact that, in cases of classification tasks, the categories might be balanced or not in terms of number of instances, as well as, it can be adopted also if we want to show a learning curve and some other performance graphics or schemes as Confusion matrtices and ROC or recall-precision curves. Latly usual values for K are: 5, 10, 20.

\begin{figure}[h]
\caption{Cross Validation Example}
\centering
\includegraphics[width=0.5\textwidth]{images/machine_learning_workflow/cross_validation.png}
\end{figure}

- **Leave One Out Cross-Validation**: It is a special case of the K-fold Cross-Validation. In fact, instead of adopting the usual values as 5, 10, 20, for K as the number of possible subsets, we establish to identify each single instance as a possible fold. It's clear that this algorithm requires more time to be completed and does not allow to show up the graphics cited just above, since we do not enough data for confusion matrix and ROC or recall-precision curves.

- **Stratified Cross-Validation**: It is a good compromise when the dataset is still large enough to be exploited for training purposes but we decide to split into K-folds the traning set such that each subset have the same proportion of samples coming from the differn classes. This operation reveals to be necessary or even mandatory when we detect that the dataset does not show the same number of samples, more or less, for each class, in other word when the dataset is unbalanced for a given attribute that is the target attribute. This means that, while trying to mitigate the issue about the unbalanced dataset we thunk as well as hope that this management let the model to fit a model which will not be affected heavily just from the most numerous class, bus still learn how to classify the samples coming from the other less numerous classes, without too mcuh misclassifying errors. As usual also with Stratified Cross-Validation we are able to show same graphics as with plain K-fold Cross-Validation, the difference is that the folds are not randomly sampled from the original training set, but yet are sampled in the same proportion per each class in order to have the same number of samples for each class inside each fold.

We try out all of the three described cross-validation techniques to measure how weel default settings for different models are doing to gaina a base line against wich compare later results coming from fine tuning process carried out exploiting grid search technique for selecting the best combinatin of purposed values for each candidate machine learning technique.

__(3) Grid Search approach__: is the technique adopted when taking into account one at a time all the machine learning algorithm for selecting the best set of hyper-parameters. It consists into defining for each model a grid of possible values for different hyper-parameters that in some sense represent our degree of freedom referred to some of the properties that characterize all the different models. The grid of values might be real number ranging within a more or less interval, or a string value used to trigger a certain feature of a model combined with other related aspects of the machine learning algorithm of the given model. We recall that the standard grid search will proceed until all possible combination of provided values have been test and training with such settings have been carried out. Opposite to classic grid search it is another technique called Random Grid Search, which implies instead to let a model to choose or sample randomly the hyper-parameters within the ranges or intervals related to each hyper-params. The latter technique can be potentially less expensive since we test a reduced number of combination but might be sub-optimal even if the results can be still acceptable and meaningful.

\begin{figure}[h]
\caption{Grid Search Example}
\centering
\includegraphics[width=0.5\textwidth]{images/machine_learning_workflow/grid_search.png}
\end{figure}

__(4) Metrics and model performance's evaluation tools__: before deploying we have to test our model against a test set, that is a subset created or obtained from overall dataset which was, after preprocessing phase that turns feature values somehow into numbers, devided into two distinct sets generally of different size. The test set evaluation implies exploiting some metrics such as Accuracy, but yet there exist several others that partly are derived from confusion matrix evaluation tool, such as Precsio, Recal F1-score, and the like.

So what we understand is that we can make use of a bouch of metrics but rather than using directly those metrics we can explore model's performance by means of more useful tools such as Confusion Matrix, Roc curve in order to better understanding model's behavior when fed with unobserved new samples, as well as, how to set a threshold for determing when a target variable will suggest us that the classified sample belong to one class or the other. So, here we briefly describe which instruments we exploit to measure model's peformance starting from confusion matrix and moving ahead toward ROC curve.

**Confusion Matrix**: in statistics a confusion matrix it's a grid or matrix of numbers and in the simplest scenario, correspoding to a binary classification task, it aims at showing how well a model was going when applied onto unknow or preiviously unseen data points and  samples, in the following manner. Arbitrary we establish that along the rows we have the fraction of samples that the model has calssified or has assigned to agiven class, and so the rows account for *Predicted valuers*. Vice versa, along the columns we have the total number of samples for each class that all together resemble the so called *Actual Values* as we illustrate in the picture just below:

\begin{figure}[h]
\caption{Confusion Matrix Example}
\centering
\includegraphics[width=0.5\textwidth]{images/machine_learning_workflow/confusion_matrix.png}
\end{figure}

So, such a table of numbers allows us to measure the fraction of correctly classified examples belonging to the Positive class, also referred to as *True Positive(TP)* or to the Negative class, also named *True Negative(TN)* and at the same time we can derive also the fractions of wrongly classified Positive samples and Negative ones, respectively knwon as *False Positve(FP)* and *False Negative (FN)*. It is clear that looking at the diagonal matrix we can undestand that the larger the value along the diagonal the better the model was able to correctly identify the samples accordingly to their own actual class.
From the four statistics measure depiceted above, that are TP,TN, FP, and FN, by the time have been dirved other useful metrics that can be exploited when the most used and well knwon measurement performance of accuracy is not enough due to the fact as an instance we wanto to analyze deeper our model behavior toward optimization of some goal or because we have to deal with dataset which are not balanced through classes to be predicted and so we wanto some other metrics to ensure the goodness of our solutions.

**Roc Curve**: for the reason cited some lines earlier by exploiting the four basis metrics have been developed other useful tools for ensuring the goodness of a solution, among those tools we decide to adopt the following one, knwon as Roc Curve. It is a curve, whose acronim stands for Receiver Operating Curve, largely employed in the field of *Decison Theory* and aims at finding, siggesting or showing how moedl's performance varywhen we are going to set different thresholds for a simple scenario in which we are going to solve a binary classification problem. Such a curve require to show on the x-axis the fraction of samples corresponding to *False Positve Rate (FPR)* at a different values of the model's hyper-params corresponding to threshold set for classifying items at inference time, as well as on the x-axis the fraction of samples corresponding to *True Positive Rate(TRP)* in order to plot a curve that originates at coordinates (0,0) and terminates at coordinates (1,1), varying in the middle accordingly to the pairs of values recorded at a given threshold for (FPR,TPR) pair. We are also reporting two driving curves that are respectvely the curve related to the *Random Classifier* which corresponds to a classifier that for each instance is just randomly selecting the predicted class, and the *Perfetct Classifier* that always correctly classifyes all the samples. Our goal by analysing such a graphics is to identify the threshold value such that we are near the points on the curve that are not so far from the upper-left corner so that to maximise the TPR as well as minimizing the FPR. Another useful quantity related to Roc Curve is represented by the so called amount Area Under the Curve (AUC) that suggests us how much area under the rco curve whas accounted while varying the threshold for classiffying the test samples, in particular the higher the value the better the classifier is doing varying the threshlds. Lastly we notice that the Random Sample accounts for an AUC equls to 0.5 %, while the Perfect Classifier for 1.0 % so we aim at obtaing a value for AUC that is in at least in the between but that approaches the unity. Here, below is presented an example of Roc Curve example:

\begin{figure}[h]
\caption{Roc Curve Example}
\centering
\includegraphics[width=0.5\textwidth]{images/machine_learning_workflow/roc_curve.jpg}
\end{figure}

Lastly, before moving ahead with describing Machine Learning Models we provide a brief list of other useful metrics that can be exploited if necessary during our analyses:

\begin{figure}[h]
\caption{Roc Curve Example}
\centering
\includegraphics[width=0.5\textwidth]{images/machine_learning_workflow/summary_statistics.jpg}
\end{figure}

**Learning Curve**:
learning curves constitute a great tool to diagnose bias and variance in any supervised learning algorithm. It comes in handt when we have to face against the so called **Bias-Variance Trade-Off**. In order to explain what that trade-off implies we have to say briefly what follows:
- In supervised learning, we assume there's a real relationship between feature(s) and target and estimate this unknown relationship with a model. Provided the assumption is true, there really is a model, which we'll call $f$, which describes perfectly the relationship between features and target.
- In practice, $f$ is almost always completely unknown, and we try to estimate it with a model $\hat{f}$. We use a certain training set and get a certain $\hat{f}$. If we use a different training set, we are very likely to get a different $\hat{f}$. As we keep changing training sets, we get different outputs for $\hat{f}$. The amount by which $\hat{f}$ varies as we change training sets is called **variance**.
- While, for most real-life scenarios, however, the true relationship between features and target is complicated and far from linear. Simplifying assumptions give **bias** to a model. The more erroneous the assumptions with respect to the true relationship, the higher the bias, and vice-versa.
- In practice, however, we need to accept a trade-off. We can’t have both low bias and low variance, so we want to aim for something in the middle, knowing that:
\begin{equation}
\begin{cases}
Y = f(X) + \quad irreducible \quad error \\
f(X) = \hat{f(X)} + \quad reducible \quad error \\
Y = \hat{f(X)} + \quad reducible \quad error + \quad irreducible \quad error \\
 \end{cases}
\end{equation}

\begin{figure}[H]
	\centering
	\begin{subfigure}{.3\textwidth}
		\includegraphics[width=\textwidth]{images/machine_learning_workflow/biasvariance.png}
		\caption{Atom 1}
	\end{subfigure}
%%%%%%%%%%%%%%
	\begin{subfigure}{.3\textwidth}
		\includegraphics[width=\textwidth]{images/machine_learning_workflow/irr_error.png}
		\caption{Atom 2}
	\end{subfigure}
%%%%%%%%%%%%%%
\end{figure}

\begin{figure}[H]
	\centering
	\begin{subfigure}{.3\textwidth}
		\includegraphics[width=\textwidth]{images/machine_learning_workflow/add_data.png}
		\caption{Atom 1}
	\end{subfigure}
%%%%%%%%%%%%%%
	\begin{subfigure}{.3\textwidth}
		\includegraphics[width=\textwidth]{images/machine_learning_workflow/low_high_var.png}
		\caption{Atom 2}
	\end{subfigure}
%%%%%%%%%%%%%%
\end{figure}

**P-Value Analysis**: Here, in the following we are going to describe shortly what is and how we have exploited the statistical tool for assessing model performance from a particular point of view represented by the so called *P-value Analysis*. The p-value is widely used in *statistical hypothesis testing*, specifically in *null hypothesis significance testing*. In this method, as part of experimental design, before performing the experiment, one first chooses a model (the null hypothesis) and a threshold value for p, called the *significance level* of the test, traditionally *5% or 1%* and denoted as $\alpha$. If the p-value is less than the chosen significance level ($\alpha$), that suggests that the observed data is sufficiently inconsistent with the null hypothesis and that the null hypothesis may be rejected. However, that does not prove that the tested hypothesis is true. When the p-value is calculated correctly, this test guarantees that the type I error rate is at most $\alpha$. For typical analysis, using the standard $\alpha = 0.05$  cutoff, the null hypothesis is rejected when $\rho < .05$ and not rejected when $\rho > .05$. The p-value does not, in itself, support reasoning about the probabilities of hypotheses but is only a tool for deciding whether to reject the null hypothesis.

In our  particular case:

- Null hypothesis ($H_{0}$): we ask ourselve whether a given particular model is able to predict class labels, with performance that matches just a uniform model's performance with $Prob(THROUGH) = 0.5$

- Test statistic: shuffleing target labels and measuring accuracy score to determine significance level

- Alpha level ($\alpha$, designated threshold of significance): 0.05

__(5)Deployment__: The last step for building a machine learning system comprise the deployment of one or more machine learning trained models that fit and satisfy constraints that was intially fixed before doing any kind of analysis. The main goal of deployemnt is to employ such statistics models for predicting, in other words making inference, against new data, unknown obeservatons for which we do not knwo their target class values.

### Pricipal Component Analysis

After having investigate the data points inside the dataset, I move one to another section of my report where I decide to explore examples that made up the entire dataset using a particular technique in the field of statistical analysis that corresponds, precisely, to so called Principal Component Analysis. In fact, the major objective of this section is understand whether it is possible to transform, by means of some kind of linear transformation given by a mathematical calculation, the original data examples into reprojected representation that allows me to retrieve most useful information to be later exploited at training time. So, lets dive a bit whitin what is and which are main concepts, pros and cons about Principal Component Analysis.

Firstly, we know that **Principal Component Analysis**, more shortly PCA, is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called *principal components*. This transformation is defined in such a way that:
- the first principal component has the largest possible variance (that is, accounts for as much of the variability in the data as possible),
- and each succeeding component in turn has the highest variance possible under the constraint that it is orthogonal to the preceding components.

The resulting vectors, each being a linear combination of the variables and containing n observations, are an uncorrelated orthogonal basis set. PCA is sensitive to the relative scaling of the original variables.

PCA is mostly used as a tool in *exploratory data analysis* and for making predictive models, for that reasons I used such a technique here, before going through the different learning technique for producing my models.

#### Several Different Implementation

From the theory and the filed of research in statistics, we know that out there, there are several different implementation and way of computing principal component analysis, and each adopted technique has different performance as well as numerical stability. The three major derivations are:
- PCA by means of an iterative based procedure of extraing pricipal components one after the other selecting each time the one that account for the most of variance along its own axis, within the remainig subspace to be derived.
- The second possible way of performing PCA is done via calculation of *Covariance Matrix* applied to attributes, that are our independent predictive variables, used to represent data points.
- Lastly, it is used the technique known as *Singular Valued Decomposition* applied to the overall data points within our dataset.

Reading scikit-learn documentation, I discovered that PCA's derivation uses the *LAPACK implementation* of the *full SVD* or a *randomized truncated SVD* by the method of *Halko et al. 2009*, depending on the shape of the input data and the number of components to extract. Therefore I will descrive mainly that way of deriving the method with respect to the others that, instead, will be described more briefly and roughly.

#### PCA's Iterative based Method
Going in order, as depicted briefly above, I start describing PCA obtained by means of iterative based procedure to extract one at a time a new principal componet explointing the data points at hand.

We begin, recalling that, PCA is defined as an orthogonal linear transformation that transforms the data to a new coordinate system such that the greatest variance by some scalar projection of the data comes to lie on the first coordinate (called the first principal component), the second greatest variance on the second coordinate, and so on.

We suppose to deal with a data matrix X, with column-wise zero empirical mean, where each of the n rows represents a different repetition of the experiment, and each of the p columns gives a particular kind of feature.

From a math poitn of view, the transformation is defined by a set of p-dimensional vectors of weights or coefficients $\mathbf {w} _{(k)}=(w_{1},\dots ,w_{p})_{(k)}$ that map each row vector $\mathbf{x}_{(i)}$ of X to a new vector of principal component scores ${\displaystyle \mathbf {t} _{(i)}=(t_{1},\dots ,t_{l})_{(i)}}$, given by: ${\displaystyle {t_{k}}_{(i)}=\mathbf {x} _{(i)}\cdot \mathbf {w} _{(k)}\qquad \mathrm {for} \qquad i=1,\dots ,n\qquad k=1,\dots ,l}$.

In this way all the individual variables ${\displaystyle t_{1},\dots ,t_{l}}$ of t considered over the data set successively inherit the maximum possible variance from X, with each coefficient vector w constrained to be a unit vector.

More precisely, the first component In order to maximize variance has to satisfy the following expression:

${\displaystyle \mathbf {w} _{(1)}={\underset {\Vert \mathbf {w} \Vert =1}{\operatorname {\arg \,max} }}\,\left\{\sum _{i}\left(t_{1}\right)_{(i)}^{2}\right\}={\underset {\Vert \mathbf {w} \Vert =1}{\operatorname {\arg \,max} }}\,\left\{\sum _{i}\left(\mathbf {x} _{(i)}\cdot \mathbf {w} \right)^{2}\right\}}$

So, with $w_{1}$ found, the first principal component of a data vector $x_{1}$ can then be given as a score $t_{1(i)} = x_{1} ⋅ w_{1}$ in the transformed co-ordinates, or as the corresponding vector in the original variables, $(x_{1} ⋅ w_{1})w_{1}$.

The others remainig components are computed as folloes. The kth component can be found by subtracting the first k − 1 principal components from X, as in the following expression:

- ${\displaystyle \mathbf {\hat {X}} _{k}=\mathbf {X} -\sum _{s=1}^{k-1}\mathbf {X} \mathbf {w} _{(s)}\mathbf {w} _{(s)}^{\rm {T}}}$

- and then finding the weight vector which extracts the maximum variance from this new data matrix  ${\mathbf {w}}_{{(k)}}={\underset {\Vert {\mathbf {w}}\Vert =1}{\operatorname {arg\,max}}}\left\{\Vert {\mathbf {{\hat {X}}}}_{{k}}{\mathbf {w}}\Vert ^{2}\right\}={\operatorname {\arg \,max}}\,\left\{{\tfrac {{\mathbf {w}}^{T}{\mathbf {{\hat {X}}}}_{{k}}^{T}{\mathbf {{\hat {X}}}}_{{k}}{\mathbf {w}}}{{\mathbf {w}}^{T}{\mathbf {w}}}}\right\}$

It turns out that:
- from the formulas depicted above me get the remaining eigenvectors of $X^{T}X$, with the maximum values for the quantity in brackets given by their corresponding eigenvalues. Thus the weight vectors are eigenvectors of $X^{T}X$.
- The kth principal component of a data vector $x_(i)$ can therefore be given as a score $t_{k(i)} = x_{(i)} ⋅ w_(k)$ in the transformed co-ordinates, or as the corresponding vector in the space of the original variables, $(x_{(i)} ⋅ w_{(k)}) w_{(k)}$, where $w_{(k)}$ is the kth eigenvector of $X^{T}X$.
- The full principal components decomposition of X can therefore be given as: ${\displaystyle \mathbf {T} =\mathbf {X} \mathbf {W}}$, where W is a p-by-p matrix of weights whose columns are the eigenvectors of $X^{T}X$.

#### Covariance Matrix for PCA analysis

PCA made from covarian matrix computation requires the calculation of sample covariance matrix of the dataset as follows: $\mathbf{Q} \propto \mathbf{X}^T \mathbf{X} = \mathbf{W} \mathbf{\Lambda} \mathbf{W}^T$.

The empirical covariance matrix between the principal components becomes ${\displaystyle \mathbf {W} ^{T}\mathbf {Q} \mathbf {W} \propto \mathbf {W} ^{T}\mathbf {W} \,\mathbf {\Lambda } \,\mathbf {W} ^{T}\mathbf {W} =\mathbf {\Lambda } }$.

#### Singular Value Decomposition for PCA analysis

Finally, the principal components transformation can also be associated with another matrix factorization, the singular value decomposition (SVD) of X, ${\displaystyle \mathbf {X} =\mathbf {U} \mathbf {\Sigma } \mathbf {W} ^{T}}$, where more precisely:
- Σ is an n-by-p rectangular diagonal matrix of positive numbers $σ_{(k)}$, called the singular values of X;
- instead U is an n-by-n matrix, the columns of which are orthogonal unit vectors of length n called the left singular vectors of X;
- Then, W is a p-by-p whose columns are orthogonal unit vectors of length p and called the right singular vectors of X.

factorizingn the matrix ${X^{T}X}$, it can be written  as:

${\begin{aligned}\mathbf {X} ^{T}\mathbf {X} &=\mathbf {W} \mathbf {\Sigma } ^{T}\mathbf {U} ^{T}\mathbf {U} \mathbf {\Sigma } \mathbf {W} ^{T}\\&=\mathbf {W} \mathbf {\Sigma } ^{T}\mathbf {\Sigma } \mathbf {W} ^{T}\\&=\mathbf {W} \mathbf {\hat {\Sigma }} ^{2}\mathbf {W} ^{T}\end{aligned}}$

Where we recall that ${\displaystyle \mathbf {\hat {\Sigma }} }$ is the square diagonal matrix with the singular values of X and the excess zeros chopped off that satisfies ${\displaystyle \mathbf {{\hat {\Sigma }}^{2}} =\mathbf {\Sigma } ^{T}\mathbf {\Sigma } } {\displaystyle \mathbf {{\hat {\Sigma }}^{2}} =\mathbf {\Sigma } ^{T}\mathbf {\Sigma } }$. Comparison with the eigenvector factorization of $X^{T}X$ establishes that the right singular vectors W of X are equivalent to the eigenvectors of $X^{T}X$ , while the singular values $σ_{(k)}$ of X are equal to the square-root of the eigenvalues $λ_{(k)}$ of $X^{T}X$ .

At this point we understand that using the singular value decomposition the score matrix T can be written as:

${\begin{aligned} \mathbf{T} & = \mathbf{X} \mathbf{W} \\ & = \mathbf{U}\mathbf{\Sigma}\mathbf{W}^T \mathbf{W} \\ & = \mathbf{U}\mathbf{\Sigma} \end{aligned}}$

so each column of T is given by one of the left singular vectors of X multiplied by the corresponding singular value. This form is also the polar decomposition of T.

Efficient algorithms exist to calculate the SVD, as in scikit-learn package, of X without having to form the matrix $X^{T}X$, so computing the SVD is now the standard way to calculate a principal components analysis from a data matrix

```python
from utils.all_imports import *;
%matplotlib inline
# Set seed for notebook repeatability
np.random.seed(0)
```

    None

```python
# =========================================================================== #
# READ INPUT DATASET
# =========================================================================== #
dataset_path, dataset_name, column_names, TARGET_COL = get_dataset_location()
estimators_list, estimators_names = get_estimators()

dataset, feature_vs_values = load_brdiges_dataset(dataset_path, dataset_name)
columns_2_avoid = ['ERECTED', 'LENGTH', 'LOCATION']
# Array used for storing graphs
plots_names = list(map(lambda xi: f"{xi}_learning_curve.png", estimators_names))
pca_kernels_list = ['linear', 'poly', 'rbf', 'cosine', 'sigmoid']
cv_list = list(range(10, 1, -1))
```

```python
# Make distinction between Target Variable and Predictors
# --------------------------------------------------------------------------- #
rescaledX, y, columns = prepare_data_for_train(dataset, target_col=TARGET_COL)
```

    Summary about Target Variable {target_col}
    --------------------------------------------------
    2    57
    1    13
    Name: T-OR-D, dtype: int64
    shape features matrix X, after normalizing:  (70, 11)

```python
# sns.pairplot(dataset, hue=TARGET_COL, height=2.5);
```

```python
df = show_table_pc_analysis(X=rescaledX); df.to_csv('cumulative_varation_explained.csv') # df
```

    Cumulative varation explained(percentage) up to given number of pcs:

```python
n_components = rescaledX.shape[1]
pca = PCA(n_components=n_components)
# pca = PCA(n_components=2)

#X_pca = pca.fit_transform(X)
pca = pca.fit(rescaledX)
X_pca = pca.transform(rescaledX)

fig = show_cum_variance_vs_components(pca, n_components)

py.sign_in('franec94', 'STMaADdoKsk66UekCPGa')
py.iplot(fig, filename='selecting-principal-components {}'.format('Standardize'))
```

#### Major Pros & Cons of PCA

### Test Error Rate versus Training Sample Size

Here, with the following set of graphics we aim at describing how the *Test Error Rate* changes across different *Training Set Size*, in  particular running the tests simply emploing the different estimators or classification algorithm with their default setting and running the training phase via cross-validation technique and adopint increasing training set size. Main goal with the illustration of such a bounch of graphics is to grasph a basic idea, that might be used to lead the training phase about how much the training set size affects the performance of a classifier at least with default settings of the hyper-parameter of the latter.

```python
try_comparing_various_online_solvers_by_kernel(X=rescaledX, y=y, kernels=None, heldout=[.75, .7, .65, .6, .55, .5, .45, .4, .35, .33, .3, .25, .2], rounds=20, n_classes=-1, n_components=9, estimators=estimators_list, cv=StratifiedKFold(2), verbose=0, show_fig=True, save_fig=False, stratified_flag=True, n_splits=3, random_state=42, gridshape=(3, 2), figsize=(15, 15), title="try comparing various online solvers by kernel", fig_name="try_comparing_various_online_solvers_by_kernel.png")
```

The main results we can get from the analysis we can carry out looking at the graphics just reported above is what follows:

- the *GuassianNB Classifier* in four out of five trials, that are more precisely trial for which we have employed kernel-tricks such as Linear, Polynomial, Rbf, Sigmoid the resulting estimators follow a trend is characterized by increasing and decrising in test error rate with some large picks after we have used at least half of the samples from the dataset. Furthermore, the Polynomial based Estimator results to be the worst among those cosidered earlier, in fact near the half of the training set size the trend even becomes increasing going apart from the decreasing trend characterized from alternating local maxima and minima. Finally, when observing how such a classifier behaves when trained adopting Cosine trick we can refer that it follows a trend somewhat similar to the one followed also from other classifier expect for decision trees classifier.

- the estimators based of *Decision Trees Classifier*, together with also *GuassianNB Classifiers*, are the two techniques amongst the others that were importantly affected by the training set size chosen for performing such analysis. In fact, even if in general is characterized from a decreasing trend, such a trend seems to be less stable than the others observed for the remaning classification techniques, in fact the trend is highly fluctuating with large picks when crossing and going towards larger training set sizes. The worst cases were when the classification technique was performed after having pre-preprocessed data examples by means of kernel-Pca based on Linear and Rbf tricks, because *Test Error Rate* turned to increasing after a given training set size which is larger than half of samples. While if we compare the generalization represented by Random Forest classifiers with respect to Decision Trees Classifier, it seems that the former across the different trials adopint also different kernel-trick for pre-processing the training examples follows a more stable decreasing trend for Test Error Rate versus training set size graphics, where just in two cases out of five which are trials where we have exploited Rbf and Sigmoid kernel-Pca tricks the random forest models seems to follwo in the second half of the Test Erro Rate curve a more fluctuating trend.

- while, speaking about all the other learning algorithms, what we have noticed is that in general the several training phase executed for the models varying the training set size follows a decreasing trend, which means either that allowing the estimatores to be trained by means of larger training set they gain more insights about how can be characterized training examples, in order to better discriminate between the two classes, or that models overfit to much against the training set. However, all the models seem to start at the very beginning with a Test Error Rate somewhat yet not too large and keep reducing it across the different training set size. More precisely, we have reported that Knn Classifier performs better across the different attempts when the training set size was in the range between half and *70%* of the training set size. While SVMs, RandomForests, and Sgd classifiers seem to reach better performance when training examples in the training set reaches let to use a traning set size in the range between *70% and 80%* of the training set size. Lastly the logisti regression classifier seems to follow a decreasing trend for Test Error Rate which results to be the most conservative, with a reduced number of fluctuation which also have been a contained changes or fluctuations.

### References
- Metrics and Diagnositc Tools:
    - (__Confusion Matrix__) https://en.wikipedia.org/wiki/Confusion_matrix
    - (__F-1-Sccore, Accuracy, and Precision-Recall__) https://towardsdatascience.com/beyond-accuracy-precision-and-recall-3da06bea9f6c
    - (__Roc Curve__) https://en.wikipedia.org/wiki/Receiver_operating_characteristic
    - (__P-value__) https://en.wikipedia.org/wiki/P-value
- Statistics:
    - (__Correlation and dependence__) https://en.wikipedia.org/wiki/Correlation_and_dependence

pittsburgh-bridges-data-set-analysis/models-analyses/merge_analyses/latex_merge/Section 1.2 - Learning Process and Pca Analysis.ipynb