# Redes Neuronales

Inicialmente, las redes neuronales fueron inspiradas en el cerebro humano. Sin embargo, después de cierto tiempo se ha dejado de tratar de emular cómo funciona el cerebro y se ha tomado un enfoque en encontrar las configuraciones más apropiadas de las redes neuronales para desarrollar diferentes tareas, incluyendo visión artificial, procesamiento de lenguaje natural y reconocimiento de voz.

Se puede describir una red neuronal como un modelo matemático para procesamiento de información.

- El procesamiento de información se realiza en las neuronas.
- Las neuronas son conectadas e intercambian información y señales entre ellas a través de enlaces de conexión.
- Los enlaces de conexión entre las neuronas pueden ser fuertes o débiles y esto determina cómo la información es procesada.
- Cada neurona tiene un estado interno que es determinado por todas las conexiones con otras neuronas.
- Cada neurona tiene una función de activación que es calculada en su estado y determina la señal de salida de la neurona.

Se pueden identificar dos características principales para una red neuronal.

- Arquitectura de la red: Esto describe el conjunto de conexiones, es decir "feedforward", "recurrent", "multi/single layered" y así.
- El aprendizaje: Describe lo que comúnmente se denomina entrenamiento. La forma más común es descenso de gradiente y backpropagation.

Una neurona es una función matemática que toma uno o más valores de entrada y calcula una salida como un valor numérico.

Una neurona está definida como

\begin{equation}
y=f(\sum_{i} x_i w_i +b)
\end{equation}

1. Primero se calcula el valor $\sum x_i w_i$ de las entradas $x_i$ y los pesos $w_i$. Aquí $x_i$ son los valores de entrada numéricos o las salidas de otras neuronas.
2. Los pesos $w_i$ son valores numéricos que representan la fuerza de las entradas o, alternativamente, la fuerza de las conexiones entre las neuronas.
3. Los pesos $b$ son los valores especiales cuya entrada es 1.

Luego se utiliza la suma pesada de una entrada hacia la función de activación $f$, cuyo valor es conocido como función de transferencia. Hay muchos tipos de funciones de activación, pero en general tienen el requerimiento de ser no lineales.

# Tipos de funciones de activación.

- f(a)=a. Esta función de activación es conocida como la identidad.
- f(a)=0 o 1.
- f(a)=$\frac{1}{1+exp(-a)}$. Esta función es la más usada comunmente, uede ser interpretada como la probabilidad de una neurona de activarse.
- f(a)=$\frac{2}{1+exp(-a)}-1$: Esta función de activación es llamada sigmoide bipolar y es simplemente una sigmoide logística reescalada y traducida al rango {-1,1}
- f(a)=$\frac{exp(a)-exp(-a)}{exp(a)+exp(-a)}$. Esta función de activación es llamada tangente hiperbólica.
- f(a)=$a$ si $a\geq 0$, o $0$ si $a<0$. Esta función de activación es probablemente la más cercana a su contraparte biológica. Es una mezcla de la identidad y una función de umbral.

```python
# Ejercicio: Hacer las funciones de activación previas

import numpy as np
import matplotlib.pyplot as plt
a=np.linspace(-5,5,100)

def ida(a):
    return a
def taf(a):
    return np.sign(a)

def sigmoid(a):
    return 1/(1+np.exp(-a))

def bip(a):
    return 2/(1+np.exp(-a))-1

def tanh(a):
    return np.tanh(a)
def relu(a):
    return a*(np.sign(a)*0.5+0.5)

plt.plot(a,relu(a))

plt.show()
```

```python
import matplotlib.pyplot as plt
import numpy

weight_value=1000

#Se modifica para cambiar donde empieza la función de escalón
bias_value_1=5000

# Dice dónde el escalóln termina
bias_value_2=-5000

plt.axis([-10,10,-1,10])

print("la función de escalon empieza en {0} y termina en {1}".format(-bias_value_1/weight_value,-bias_value_2/weight_value))

inputs=numpy.arange(-10,10,0.01)
outputs=list()

for x in inputs:
    y1=1.0/(1.0+numpy.exp(-weight_value*x-bias_value_1))
    y2=1.0/(1.0+numpy.exp(-weight_value*x-bias_value_2))

    w=7
    y=y1*w-y2*w

    outputs.append(y)
plt.plot(inputs,outputs,lw=2,color="black")
plt.show()

```

```python
from matplotlib.colors import ListedColormap
import numpy as np

def tanh(x): return (1.0-np.exp(-2*x))/(1.0+np.exp(-2*x))
def tanh_derivative(x):
    return (1+tanh(x))*(1-tanh(x))

class NeuralNetwork:
    def __init__(self, net_arch):
        self.activation_func = tanh
        self.activation_derivative = tanh_derivative
        self.layers = len(net_arch)
        self.steps_per_epoch = 1000
        self.net_arch = net_arch
        self.weights=[]

        for layer in range(len(net_arch) - 1):
            w = 2 * np.random.rand(net_arch[layer] + 1,net_arch[layer + 1]) - 1
            self.weights.append(w)

    def fit(self, data, labels, learning_rate=0.1, epochs=10):
        ones = np.ones((1, data.shape[0]))
        Z = np.concatenate((ones.T, data), axis=1)
        training = epochs * self.steps_per_epoch
        for k in range(training):
            if k % self.steps_per_epoch == 0:
                print("epochs: {}".format(k/self.steps_per_epoch))
                for s in data:
                    print(s,nn.predict(s))
        sample=np.random.randint(data.shape[0])
        y=[Z[sample]]

        for i in range(len(self.weights) - 1):
            activation = np.dot(y[i], self.weights[i])
            activation_f = self.activation_func(activation)

            activation_f = np.concatenate((np.ones(1), np.array(activation_f)))
            y.append(activation_f)

        activation = np.dot(y[-1], self.weights[-1])
        activation_f = self.activation_func(activation)
        y.append(activation_f)

        error = labels[sample] - y[-1]
        delta_vec = [error * self.activation_derivative(y[-1])]

        for i in range(self.layers - 2, 0, -1):
            error = delta_vec[-1].dot(self.weights[i][1:].T)
            error = error * self.activation_derivative(y[i][1:])
            delta_vec.append(error)
        delta_vec.reverse()

        for i in range(len(self.weights)):
            layer = y[i].reshape(1, nn.net_arch[i] + 1)
            delta = delta_vec[i].reshape(1,nn.net_arch[i + 1] )
            self.weights[i] += learning_rate * layer.T.dot(delta)

    def predict(self, x):
        val=np.concatenate((np.ones(1).T,np.array(x)))
        for i in range(0,len(self.weights)):
            val = self.activation_func(np.dot(val, self.weights[i]))
            val = np.concatenate((np.ones(1).T, np.array(val)))
        return val[1]

    def plot_decision_regions(self,X,y,points=200):
        markers=('o','^')
        colors=('red','blue')
        cmap=ListedColormap(colors)

        x1_min, x1_max = X[:, 0].min() - 1, X[:, 0].max() + 1
        x2_min, x2_max = X[:, 1].min() - 1, X[:, 1].max() + 1

        resolution=max(x1_max-x1_min,x2_max-x2_min)/float(points)

        xx1, xx2 = np.meshgrid(np.arange(x1_min,x1_max,resolution),np.arange(x2_min, x2_max,resolution))

        input=np.array([xx1.ravel(),xx2.ravel()]).T
        Z=np.empty(0)
        for i in range(input.shape[0]):
            val=nn.predict(np.array(input[i]))
            if val < 0.5:
                val = 0
            if val >= 0.5:
                val = 1
            Z = np.append(Z, val)
        Z = Z.reshape(xx1.shape)
        plt.pcolormesh(xx1, xx2, Z, cmap=cmap)
        plt.xlim(xx1.min(), xx1.max())
        plt.ylim(xx2.min(), xx2.max())

        classes = ["False", "True"]
        for idx, cl in enumerate(np.unique(y)):
            plt.scatter(x=X[y == cl, 0],
            y=X[y == cl, 1],
            alpha=1.0,
            c=colors[idx],
            edgecolors='black',
            marker=markers[idx],
            s=80,
            label=classes[idx])
        plt.xlabel('x-axis')
        plt.ylabel('y-axis')
        plt.legend(loc='upper left')
        plt.show()

if __name__=='__main__':
    np.random.seed(0)
    nn = NeuralNetwork([2, 2, 1])
    X = np.array([[0, 0],
    [0, 1],
    [1, 0],
    [1, 1]])
    y = np.array([0, 0, 1, 1])
    nn.fit(X, y, epochs=10)
    print("Final prediction")
    for s in X:
        print(s, nn.predict(s))
        nn.plot_decision_regions(X, y)
```

# Aprendizaje de características

Para ilustrar cómo funciona el aprendizaje profundo, consideremos la tarea de reconocer una simple figura geométrica, por ejemplo, un cubo, como se ve en el siguiente diagrama. El cubo es compuesto de bordes (o líneas), que se cruzan en vértices. Digamos que cada punto posible en el espacio tridimensional está asociado con una neurona (olvide por un momento que esto requerirá un número infinito de neuronas). Todos los puntos / neuronas están en la primera (entrada) capa de una red de alimentación de múltiples capas. Un punto de entrada / neurona está activo si el punto correspondiente se encuentra en una línea. Los puntos / neuronas que se encuentran en una línea común (borde) tener fuertes conexiones positivas con un solo borde / neurona común en la siguiente capa. Por el contrario, tienen conexiones negativas con todas las demás neuronas en la siguiente capa. Lo único excepción son las neuronas que se encuentran en los vértices. Cada una de esas neuronas se encuentra simultáneamente en tres bordes, y está conectado a sus tres neuronas correspondientes en la capa posterior.

Ahora tenemos dos capas ocultas, con diferentes niveles de abstracción: la primera para los puntos. y el segundo para bordes. Pero esto no es suficiente para codificar un cubo completo en la red. Probemos con otra capa para vértices. Aquí, cada tres bordes activos / neuronas del segundo capa, que forma un vértice, tiene una conexión positiva significativa a un único común vértice / neurona de la tercera capa. Como un borde del cubo forma dos vértices, cada uno borde / neurona tendrá conexiones positivas a dos vértices / neuronas y negativas conexiones a todos los demás. Finalmente, presentaremos la última capa oculta (cubo). El cuatro los vértices / neuronas que forman un cubo tendrán conexiones positivas con un solo cubo / neurona del cubo /capa:

El ejemplo de representación del cubo está demasiado simplificado, pero podemos sacar varias conclusiones
de eso. Una de ellas es que las redes neuronales profundas se prestan bien jerárquicamente datos organizados Por ejemplo, una imagen consta de píxeles, que forman líneas, bordes, regiones, y así. Esto también es cierto para el habla, donde los bloques de construcción se llaman fonemas; como así como texto, donde tenemos caracteres, palabras y oraciones. En el ejemplo anterior, dedicamos capas a entidades de cubo específicas deliberadamente, pero en practica, no haríamos eso. En cambio, una red profunda "descubrirá" características automáticamente durante el entrenamiento. Estas características pueden no ser inmediatamente obvias y, en en general, no sería interpretable por los humanos. Además, no sabríamos el nivel de características codificadas en las diferentes capas de la red. Nuestro ejemplo es más parecido al clásico. algoritmos de aprendizaje automático, donde el usuario tiene que usar su propia experiencia para seleccionar lo que ellos piensan son las mejores características. Este proceso se llama ingeniería de características, y puede ser laborioso y lento. Permitir que una red descubra automáticamente las características no solo son más fáciles, sino que son muy abstractas, lo que las hace menos sensible al ruido. Por ejemplo, la visión humana puede reconocer objetos de diferentes formas, tamaños, en diferentes condiciones de iluminación, e incluso cuando su vista está parcialmente oscurecida. Podemos reconocer a las personas con diferentes cortes de pelo, rasgos faciales e incluso cuando usan un sombrero o un bufanda que les cubre la boca. Del mismo modo, las características abstractas que aprende la red lo ayudarán para reconocer mejor las caras, incluso en condiciones más difíciles.

# Algoritmos de Deep Learning

Podríamos definir el aprendizaje profundo como una clase de técnicas de aprendizaje automático, donde la información se procesa en capas jerárquicas para comprender las representaciones y características de los datos en niveles crecientes de complejidad. En la práctica, todos los algoritmos de aprendizaje profundo son redes neuronales, que comparten algunas propiedades básicas comunes. Todos consisten en neuronas interconectadas que están organizadas en capas. En lo que difieren es en la arquitectura de la red (o la forma en que las neuronas están organizadas en la red) y, a veces, en la forma en que se entrenan. Con eso en mente, echemos un vistazo a las principales clases de redes neuronales. La siguiente lista no es exhaustiva, pero representa la gran mayoría de los algoritmos en uso hoy en día:

- Perceptrones multicapa (MLP): una red neuronal con propagación de alimentación directa, capas completamente conectadas y al menos una capa oculta.

- Redes neuronales convolucionales (CNN): una CNN es una red neuronal con varios tipos de capas especiales. Por ejemplo, las capas convolucionales aplican un filtro a la imagen de entrada (o sonido) deslizando ese filtro por toda la señal entrante, para producir un mapa de activación n-dimensional. Existe alguna evidencia de que las neuronas en las CNN se organizan de manera similar a cómo se organizan las células biológicas en la corteza visual del cerebro. Hemos mencionado CNN varias veces hasta ahora, y eso no es una coincidencia: hoy, superan a todos los demás algoritmos de ML en una gran cantidad de tareas de visión por computadora y PNL.

- Redes recurrentes: este tipo de red tiene un estado interno (o memoria), que se basa en la totalidad o en parte de los datos de entrada ya alimentados a la red. La salida de una red recurrente es una combinación de su estado interno (memoria de entradas) y la última muestra de entrada. Al mismo tiempo, el estado interno cambia para incorporar datos recién ingresados. Debido a estas propiedades, las redes recurrentes son buenas candidatas para tareas que funcionan en datos secuenciales, como texto o datos de series de tiempo.

- Codificadores automáticos: una clase de algoritmos de aprendizaje no supervisados, en los que la forma de salida es la misma que la entrada que permite a la red aprender mejor las representaciones básicas.

```python

```

# Softmax y entropía cruzada
La función softmax es una generalización del concepto de regresión logística para múltiples clases. Veamos la siguiente fórmula:

\begin{equation}
F(x_i)=\frac{e^{x_i}}{\sum_{j=1}^{n}e^{x_j}}
\end{equation}

Aquí, i, j = 0, 1, 2, ... ny xi representan cada uno de n valores reales arbitrarios, correspondientes a n clases mutuamente excluyentes. El softmax "aplasta" los valores de entrada en el intervalo (0, 1), similar a la función logística. Pero tiene la propiedad adicional de que la suma de todas las salidas aplastadas suma 1. Podemos interpretar las salidas softmax como una distribución de probabilidad normalizada de las clases. Entonces, tiene sentido usar una función de pérdida, que compara la diferencia entre las probabilidades de clase estimadas y la distribución de clase real (la diferencia se conoce como crossentropía). Como mencionamos en el paso 5 de esta sección, la distribución real suele ser un vector codificado en caliente, donde la clase real tiene una probabilidad de 1, y todos los demás tienen una probabilidad de 0. La función de pérdida que hace esto se llama Pérdida de entropía cruzada:

\begin{equation}
H(p,q)=\sum_{i=1}^{n}{p_i(x)log(q_i (x))}
\end{equation}

Aquí, qi (x) es la probabilidad estimada de que la salida pertenezca a la clase i (de n clases totales) y pi (x) es la probabilidad real. Cuando utilizamos valores objetivo codificados en caliente para pi (x), solo la clase objetivo tiene un valor distinto de cero (1) y todos los demás son ceros. En este caso, la pérdida de entropía cruzada solo capturará el error en la clase de destino y descartará todos los demás errores. En aras de la simplicidad, asumiremos que aplicamos la fórmula
sobre una sola muestra de entrenamiento.

```python
import warnings
warnings.filterwarnings("ignore")
```

```python
import tensorflow
with tensorflow.device("/gpu:1"):
    pass
# model definition here
#Here's an example:
#"/cpu:0": the main CPU of your machine
#"/gpu:0": the first GPU of your machine, if one exists
#"/gpu:1": the second GPU of your machine, if a second exists
#"/gpu:2": the third GPU of your machine, if a third exists, and so on

```

```python
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.utils import np_utils

(X_train,Y_train),(X_test,Y_test)=mnist.load_data()
plt.imshow(X_test[0])
X_train=X_train.reshape(60000,784)
X_test=X_test.reshape(10000,784)

import matplotlib.pyplot as plt

classes=10
Y_train=np_utils.to_categorical(Y_train,classes)
Y_test=np_utils.to_categorical(Y_test,classes)
print(Y_test[0])
input_size=784
batch_size=100
hidden_neurons=100
epochs=10

model=Sequential([Dense(hidden_neurons,input_dim=input_size), Activation('sigmoid'),Dense(classes),Activation('softmax')])

model.compile(loss='categorical_crossentropy',metrics=['accuracy'],optimizer='sgd')

model.fit(X_train,Y_train,batch_size=batch_size,nb_epoch=epochs,verbose=1)

score=model.evaluate(X_test,Y_test,verbose=1)

print('Test accuracy:',score)

weights=model.layers[0].get_weights()

```

```python

```

```python
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import numpy
fig = plt.figure()
w = weights[0].T
for neuron in range(hidden_neurons):
    ax = fig.add_subplot(10, 10, neuron + 1)
    ax.axis("off")
    ax.imshow(numpy.reshape(w[neuron], (28, 28)), cmap=cm.Greys_r)
plt.show()
```

```python
from keras.datasets import cifar10
from keras.layers.core import Dense, Activation
from keras.models import Sequential
from keras.utils import np_utils

(X_train, Y_train), (X_test, Y_test) = cifar10.load_data()

X_train = X_train.reshape(50000, 3072)
X_test = X_test.reshape(10000, 3072)
classes = 10
Y_train = np_utils.to_categorical(Y_train, classes)
Y_test = np_utils.to_categorical(Y_test, classes)
input_size = 3072
batch_size = 100
epochs = 100

model = Sequential([Dense(1024, input_dim=input_size),Activation('relu'),Dense(512),Activation('relu'),Dense(512),Activation('sigmoid'),Dense(classes),Activation('softmax')])

model.compile(loss='categorical_crossentropy',metrics=['accuracy'],optimizer='sgd')

model.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs,validation_data=(X_test, Y_test), verbose=1)

```

    Train on 50000 samples, validate on 10000 samples
    Epoch 1/100
    50000/50000 [==============================] - 5s 100us/step - loss: 2.1783 - acc: 0.1827 - val_loss: 2.0974 - val_acc: 0.2161
    Epoch 2/100
    50000/50000 [==============================] - 4s 76us/step - loss: 2.0231 - acc: 0.2486 - val_loss: 1.9859 - val_acc: 0.2604
    Epoch 3/100
    50000/50000 [==============================] - 4s 77us/step - loss: 1.9524 - acc: 0.2854 - val_loss: 1.9488 - val_acc: 0.2747
    Epoch 4/100
    50000/50000 [==============================] - 4s 77us/step - loss: 1.9187 - acc: 0.3005 - val_loss: 1.8829 - val_acc: 0.3214
    Epoch 5/100
    50000/50000 [==============================] - 4s 78us/step - loss: 1.8811 - acc: 0.3198 - val_loss: 1.8578 - val_acc: 0.3231
    Epoch 6/100
    50000/50000 [==============================] - 4s 76us/step - loss: 1.8533 - acc: 0.3292 - val_loss: 1.8288 - val_acc: 0.3348
    Epoch 7/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.8185 - acc: 0.3415 - val_loss: 1.8235 - val_acc: 0.3455
    Epoch 8/100
    50000/50000 [==============================] - 4s 77us/step - loss: 1.7925 - acc: 0.3528 - val_loss: 1.7932 - val_acc: 0.3574
    Epoch 9/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.7643 - acc: 0.3618 - val_loss: 1.7683 - val_acc: 0.3660
    Epoch 10/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.7477 - acc: 0.3680 - val_loss: 1.7113 - val_acc: 0.3798
    Epoch 11/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.7303 - acc: 0.3756 - val_loss: 1.7235 - val_acc: 0.3854
    Epoch 12/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.7080 - acc: 0.3866 - val_loss: 1.6981 - val_acc: 0.3877
    Epoch 13/100
    50000/50000 [==============================] - 4s 78us/step - loss: 1.6914 - acc: 0.3921 - val_loss: 1.6913 - val_acc: 0.3967
    Epoch 14/100
    50000/50000 [==============================] - 4s 78us/step - loss: 1.6767 - acc: 0.3951 - val_loss: 1.6676 - val_acc: 0.4020
    Epoch 15/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.6661 - acc: 0.4009 - val_loss: 1.6577 - val_acc: 0.4085
    Epoch 16/100
    50000/50000 [==============================] - 6s 118us/step - loss: 1.6457 - acc: 0.4126 - val_loss: 1.7004 - val_acc: 0.3944
    Epoch 17/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.6359 - acc: 0.4142 - val_loss: 1.6234 - val_acc: 0.4179
    Epoch 18/100
    50000/50000 [==============================] - 5s 90us/step - loss: 1.6218 - acc: 0.4163 - val_loss: 1.6255 - val_acc: 0.4162
    Epoch 19/100
    50000/50000 [==============================] - 4s 80us/step - loss: 1.6055 - acc: 0.4247 - val_loss: 1.6418 - val_acc: 0.4101
    Epoch 20/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.5900 - acc: 0.4304 - val_loss: 1.5921 - val_acc: 0.4355
    Epoch 21/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.5807 - acc: 0.4339 - val_loss: 1.5878 - val_acc: 0.4314
    Epoch 22/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.5672 - acc: 0.4397 - val_loss: 1.5928 - val_acc: 0.4314
    Epoch 23/100
    50000/50000 [==============================] - 4s 79us/step - loss: 1.5570 - acc: 0.4408 - val_loss: 1.5611 - val_acc: 0.4385
    Epoch 24/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.5523 - acc: 0.4444 - val_loss: 1.5583 - val_acc: 0.4416
    Epoch 25/100
    50000/50000 [==============================] - 4s 80us/step - loss: 1.5449 - acc: 0.4457 - val_loss: 1.5478 - val_acc: 0.4471
    Epoch 26/100
    50000/50000 [==============================] - 4s 80us/step - loss: 1.5267 - acc: 0.4555 - val_loss: 1.5635 - val_acc: 0.4446
    Epoch 27/100
    50000/50000 [==============================] - 4s 78us/step - loss: 1.5233 - acc: 0.4553 - val_loss: 1.5280 - val_acc: 0.4497
    Epoch 28/100
    50000/50000 [==============================] - 4s 88us/step - loss: 1.5114 - acc: 0.4575 - val_loss: 1.5298 - val_acc: 0.4555
    Epoch 29/100
    50000/50000 [==============================] - 4s 81us/step - loss: 1.5086 - acc: 0.4592 - val_loss: 1.5356 - val_acc: 0.4525
    Epoch 30/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.4947 - acc: 0.4649 - val_loss: 1.5176 - val_acc: 0.4576
    Epoch 31/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.4896 - acc: 0.4693 - val_loss: 1.4967 - val_acc: 0.4678
    Epoch 32/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.4837 - acc: 0.4698 - val_loss: 1.5119 - val_acc: 0.4597
    Epoch 33/100
    50000/50000 [==============================] - 5s 90us/step - loss: 1.4720 - acc: 0.4724 - val_loss: 1.4917 - val_acc: 0.4630
    Epoch 34/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.4652 - acc: 0.4748 - val_loss: 1.4913 - val_acc: 0.4689
    Epoch 35/100
    50000/50000 [==============================] - 4s 88us/step - loss: 1.4564 - acc: 0.4777 - val_loss: 1.4959 - val_acc: 0.4650
    Epoch 36/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.4468 - acc: 0.4819 - val_loss: 1.5042 - val_acc: 0.4574
    Epoch 37/100
    50000/50000 [==============================] - 4s 81us/step - loss: 1.4442 - acc: 0.4832 - val_loss: 1.4873 - val_acc: 0.4634
    Epoch 38/100
    50000/50000 [==============================] - 4s 85us/step - loss: 1.4353 - acc: 0.4842 - val_loss: 1.5064 - val_acc: 0.4617
    Epoch 39/100
    50000/50000 [==============================] - 4s 81us/step - loss: 1.4346 - acc: 0.4874 - val_loss: 1.4934 - val_acc: 0.4643
    Epoch 40/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.4173 - acc: 0.4946 - val_loss: 1.4545 - val_acc: 0.4791
    Epoch 41/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.4078 - acc: 0.4989 - val_loss: 1.4925 - val_acc: 0.4653
    Epoch 42/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.4132 - acc: 0.4958 - val_loss: 1.4676 - val_acc: 0.4753
    Epoch 43/100
    50000/50000 [==============================] - 5s 95us/step - loss: 1.4011 - acc: 0.4976 - val_loss: 1.4602 - val_acc: 0.4816
    Epoch 44/100
    50000/50000 [==============================] - 5s 100us/step - loss: 1.3979 - acc: 0.4978 - val_loss: 1.4389 - val_acc: 0.4843
    Epoch 45/100
    50000/50000 [==============================] - 5s 108us/step - loss: 1.3876 - acc: 0.5055 - val_loss: 1.4901 - val_acc: 0.4696
    Epoch 46/100
    50000/50000 [==============================] - 5s 96us/step - loss: 1.3902 - acc: 0.5007 - val_loss: 1.5024 - val_acc: 0.4686
    Epoch 47/100
    50000/50000 [==============================] - 5s 96us/step - loss: 1.3798 - acc: 0.5080 - val_loss: 1.4699 - val_acc: 0.4767
    Epoch 48/100
    50000/50000 [==============================] - 5s 97us/step - loss: 1.3802 - acc: 0.5057 - val_loss: 1.4573 - val_acc: 0.4829
    Epoch 49/100
    50000/50000 [==============================] - 4s 88us/step - loss: 1.3720 - acc: 0.5073 - val_loss: 1.4239 - val_acc: 0.4882
    Epoch 50/100
    50000/50000 [==============================] - 4s 86us/step - loss: 1.3632 - acc: 0.5137 - val_loss: 1.4383 - val_acc: 0.4867
    Epoch 51/100
    50000/50000 [==============================] - 4s 88us/step - loss: 1.3547 - acc: 0.5126 - val_loss: 1.4643 - val_acc: 0.4828
    Epoch 52/100
    50000/50000 [==============================] - 4s 90us/step - loss: 1.3466 - acc: 0.5180 - val_loss: 1.4495 - val_acc: 0.4878
    Epoch 53/100
    50000/50000 [==============================] - 5s 99us/step - loss: 1.3442 - acc: 0.5193 - val_loss: 1.4264 - val_acc: 0.4846
    Epoch 54/100
    50000/50000 [==============================] - 5s 96us/step - loss: 1.3385 - acc: 0.5211 - val_loss: 1.4610 - val_acc: 0.4885
    Epoch 55/100
    50000/50000 [==============================] - 5s 94us/step - loss: 1.3282 - acc: 0.5234 - val_loss: 1.4150 - val_acc: 0.4972
    Epoch 56/100
    50000/50000 [==============================] - 5s 97us/step - loss: 1.3255 - acc: 0.5248 - val_loss: 1.4380 - val_acc: 0.4874
    Epoch 57/100
    50000/50000 [==============================] - 5s 105us/step - loss: 1.3170 - acc: 0.5295 - val_loss: 1.3911 - val_acc: 0.5089
    Epoch 58/100
    50000/50000 [==============================] - 5s 96us/step - loss: 1.3143 - acc: 0.5304 - val_loss: 1.4412 - val_acc: 0.4866
    Epoch 59/100
    50000/50000 [==============================] - 4s 85us/step - loss: 1.3120 - acc: 0.5307 - val_loss: 1.4223 - val_acc: 0.4957
    Epoch 60/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.3089 - acc: 0.5336 - val_loss: 1.4069 - val_acc: 0.5013
    Epoch 61/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.2988 - acc: 0.5352 - val_loss: 1.4492 - val_acc: 0.4865
    Epoch 62/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.2934 - acc: 0.5361 - val_loss: 1.4143 - val_acc: 0.4940
    Epoch 63/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.2871 - acc: 0.5398 - val_loss: 1.3954 - val_acc: 0.5039
    Epoch 64/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.2796 - acc: 0.5413 - val_loss: 1.3905 - val_acc: 0.5097
    Epoch 65/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.2686 - acc: 0.5461 - val_loss: 1.4012 - val_acc: 0.5004
    Epoch 66/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.2606 - acc: 0.5502 - val_loss: 1.3996 - val_acc: 0.5117
    Epoch 67/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.2595 - acc: 0.5502 - val_loss: 1.4035 - val_acc: 0.5005
    Epoch 68/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.2558 - acc: 0.5513 - val_loss: 1.3989 - val_acc: 0.5105
    Epoch 69/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.2517 - acc: 0.5542 - val_loss: 1.4048 - val_acc: 0.5075
    Epoch 70/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.2454 - acc: 0.5569 - val_loss: 1.3911 - val_acc: 0.5111
    Epoch 71/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.2354 - acc: 0.5582 - val_loss: 1.3855 - val_acc: 0.5111
    Epoch 72/100
    50000/50000 [==============================] - 5s 94us/step - loss: 1.2333 - acc: 0.5568 - val_loss: 1.4110 - val_acc: 0.5014
    Epoch 73/100
    50000/50000 [==============================] - 5s 95us/step - loss: 1.2267 - acc: 0.5635 - val_loss: 1.4218 - val_acc: 0.5005
    Epoch 74/100
    50000/50000 [==============================] - 4s 87us/step - loss: 1.2239 - acc: 0.5626 - val_loss: 1.3804 - val_acc: 0.5072
    Epoch 75/100
    50000/50000 [==============================] - 5s 96us/step - loss: 1.2105 - acc: 0.5689 - val_loss: 1.3710 - val_acc: 0.5146
    Epoch 76/100
    50000/50000 [==============================] - 5s 91us/step - loss: 1.2092 - acc: 0.5706 - val_loss: 1.3825 - val_acc: 0.5142
    Epoch 77/100
    50000/50000 [==============================] - 6s 113us/step - loss: 1.2020 - acc: 0.5693 - val_loss: 1.4336 - val_acc: 0.4964
    Epoch 78/100
    50000/50000 [==============================] - 4s 88us/step - loss: 1.1982 - acc: 0.5705 - val_loss: 1.3763 - val_acc: 0.5213
    Epoch 79/100
    50000/50000 [==============================] - 4s 85us/step - loss: 1.1940 - acc: 0.5742 - val_loss: 1.3887 - val_acc: 0.5137
    Epoch 80/100
    50000/50000 [==============================] - 6s 122us/step - loss: 1.1866 - acc: 0.5778 - val_loss: 1.3790 - val_acc: 0.5155
    Epoch 81/100
    50000/50000 [==============================] - 4s 84us/step - loss: 1.1849 - acc: 0.5749 - val_loss: 1.4099 - val_acc: 0.5047
    Epoch 82/100
    50000/50000 [==============================] - 4s 86us/step - loss: 1.1762 - acc: 0.5804 - val_loss: 1.3757 - val_acc: 0.5191
    Epoch 83/100
    50000/50000 [==============================] - 4s 88us/step - loss: 1.1732 - acc: 0.5806 - val_loss: 1.3690 - val_acc: 0.5169
    Epoch 84/100
    50000/50000 [==============================] - 5s 93us/step - loss: 1.1674 - acc: 0.5845 - val_loss: 1.4169 - val_acc: 0.5098
    Epoch 85/100
    50000/50000 [==============================] - 5s 95us/step - loss: 1.1631 - acc: 0.5835 - val_loss: 1.3976 - val_acc: 0.5095
    Epoch 86/100
    50000/50000 [==============================] - 5s 98us/step - loss: 1.1570 - acc: 0.5871 - val_loss: 1.3729 - val_acc: 0.5203
    Epoch 87/100
    50000/50000 [==============================] - 5s 94us/step - loss: 1.1518 - acc: 0.5909 - val_loss: 1.3584 - val_acc: 0.5223
    Epoch 88/100
    50000/50000 [==============================] - 5s 93us/step - loss: 1.1447 - acc: 0.5916 - val_loss: 1.3580 - val_acc: 0.5280
    Epoch 89/100
    50000/50000 [==============================] - 5s 93us/step - loss: 1.1411 - acc: 0.5917 - val_loss: 1.4336 - val_acc: 0.5034
    Epoch 90/100
    50000/50000 [==============================] - 6s 123us/step - loss: 1.1349 - acc: 0.5958 - val_loss: 1.3716 - val_acc: 0.5118
    Epoch 91/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.1276 - acc: 0.5975 - val_loss: 1.3844 - val_acc: 0.5163
    Epoch 92/100
    50000/50000 [==============================] - 4s 85us/step - loss: 1.1183 - acc: 0.6021 - val_loss: 1.3758 - val_acc: 0.5182
    Epoch 93/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.1191 - acc: 0.5991 - val_loss: 1.3949 - val_acc: 0.5174
    Epoch 94/100
    50000/50000 [==============================] - 4s 86us/step - loss: 1.1087 - acc: 0.6046 - val_loss: 1.3686 - val_acc: 0.5222
    Epoch 95/100
    50000/50000 [==============================] - 5s 110us/step - loss: 1.0997 - acc: 0.6100 - val_loss: 1.3793 - val_acc: 0.5196
    Epoch 96/100
    50000/50000 [==============================] - 4s 82us/step - loss: 1.0935 - acc: 0.6097 - val_loss: 1.3982 - val_acc: 0.5105
    Epoch 97/100
    50000/50000 [==============================] - 5s 102us/step - loss: 1.0914 - acc: 0.6116 - val_loss: 1.3512 - val_acc: 0.5242
    Epoch 98/100
    50000/50000 [==============================] - 6s 127us/step - loss: 1.0754 - acc: 0.6188 - val_loss: 1.3880 - val_acc: 0.5252
    Epoch 99/100
    50000/50000 [==============================] - 5s 92us/step - loss: 1.0730 - acc: 0.6177 - val_loss: 1.3652 - val_acc: 0.5225
    Epoch 100/100
    50000/50000 [==============================] - 4s 83us/step - loss: 1.0640 - acc: 0.6201 - val_loss: 1.3744 - val_acc: 0.5254

    <keras.callbacks.History at 0x27059a17e88>

```python

print(model.predict_classes(X_test))

print(Y_test.reshape(-1))

cm=confusion_matrix(Y_test.reshape(-1), model.predict_classes(X_test))
cm/np.sum(cm,axis=1)*100

```

    [3 8 0 ... 3 4 7]
    [3 8 8 ... 5 1 7]

    array([[64.1,  5.6,  3.5,  1.4,  3.6,  0.9,  1.8,  3.8, 12.7,  2.6],
           [ 3.3, 71.1,  1.1,  1.2,  1.1,  1.3,  1. ,  2.7,  6.8, 10.4],
           [10.9,  2.4, 33.1,  5.7, 17.4,  8.4,  7.1, 11.5,  2. ,  1.5],
           [ 4.4,  3.3,  6.1, 27.5,  9.3, 21.1,  9.2, 12.1,  3.3,  3.7],
           [ 7. ,  2.1,  9.4,  4.3, 46.6,  4.9,  6.4, 15.2,  2.5,  1.6],
           [ 2.8,  0.9,  7.3, 14.1,  9.5, 42.1,  6. , 13.4,  2.6,  1.3],
           [ 2.2,  2.5,  5.4,  5.9, 14.3,  7.3, 53.4,  5.7,  1.1,  2.2],
           [ 5.1,  1.5,  2.6,  3.1,  5.6,  5.3,  1.7, 70.8,  1.2,  3.1],
           [10.7,  8.2,  1.2,  1.5,  2.8,  1.2,  0.7,  1.9, 67.9,  3.9],
           [ 4.8, 25.9,  1. ,  1.7,  0.6,  2.3,  1.5,  6.9,  6.5, 48.8]])

```python
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import matplotlib.gridspec as gridspec
import numpy
import random

fig = plt.figure()
outer_grid = gridspec.GridSpec(10, 10, wspace=0.0, hspace=0.0)
weights = model.layers[0].get_weights()
w = weights[0].T

for i, neuron in enumerate(random.sample(range(0, 1023), 100)):
    ax = plt.Subplot(fig, outer_grid[i])
    ax.imshow(numpy.mean(numpy.reshape(w[i], (32, 32, 3)), axis=2),
    cmap=cm.Greys_r)
    ax.set_xticks([])
    ax.set_yticks([])
    fig.add_subplot(ax)
plt.show()
```

```python
# Ejercicio Utilicen Redes neuronales para clasificar
```

```python

```

```python

```

```python
import matplotlib.pyplot as plt
import numpy as np
mean = [0, 0]
cov = [[1, 0], [0, 100]]  # diagonal covariance

cov2 = [[2, 0], [0, 1]]  # diagonal covariance

x1, x2 = np.random.multivariate_normal(mean, cov, 100).T
x3, x4 = np.random.multivariate_normal(mean, cov2, 100).T
```

```python
from mpl_toolkits import mplot3d

zdata = 15 * np.random.random(1000)
xdata = np.sin(zdata) + 0.1 * np.random.randn(1000)
ydata = np.cos(zdata) + 0.1 * np.random.randn(1000)

fig = plt.figure()
ax = plt.axes(projection='3d')

ax.scatter3D(xdata, ydata, zdata, cmap='Greens');
```

```python
print(X.shape)
covX=np.cov(X)
print(covX)
```

    (3, 1000)
    [[ 2.79463640e-02 -4.03523147e-04 -6.37819697e-03]
     [-4.03523147e-04  2.63174637e-02  6.63009104e-03]
     [-6.37819697e-03  6.63009104e-03  9.83000217e-01]]

```python
from numpy import linalg as LA

w, v = LA.eig(covX)
print(v)
```

    [[ 0.00668036 -0.97855495 -0.20587761]
     [-0.00693215  0.20583193 -0.9785628 ]
     [-0.99995366 -0.00796433  0.00540845]]

```python

```

```python
print(w)
print(v)
```

    [0.98308879 0.02797933 0.02619592]
    [[ 0.00668036 -0.97855495 -0.20587761]
     [-0.00693215  0.20583193 -0.9785628 ]
     [-0.99995366 -0.00796433  0.00540845]]

```python
fig = plt.figure(figsize=(10,10))
ax = plt.axes(projection='3d')

ax.scatter3D(xdata, ydata, zdata, cmap='Greens');
for i in range(len(v)):
    ax.plot([0,v[i][2]],[0,v[i][1]],[0,v[i][0]])
plt.show()
```

```python
print(v.shape)
dat=np.dot(X.T,v)
plt.scatter(dat[:,0],dat[:,1])
```

```python

```


# Univariate HP with Exponential Delay

In this notebook, we present an example use case of fitting (calibrating) and sampling from a univariate Hawkes (self-exciting) process with exponential delay density. Concretely, the HP is determined by the conditional intensity function

$$
\begin{align}
\lambda^*(t) &= \mu + \alpha \sum_{t_j < t} g_\theta(t - t_j) \\
g_\theta(x) &= -\theta \exp (-\theta x )
\end{align}
$$

```python
%matplotlib inline
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
from hawkeslib.model.uv_exp import UnivariateExpHawkesProcess as UVHP
```

## Example 1: FX Price Jump Data

Fit to real data and get Monte Carlo estimates of the distribution of number of occurrences via Ogata's modified thinning method.

```python
df = pd.read_csv("example_data.csv", header=None)
df.columns = ["mark", "time"]
```

```python
# take all mark 21 and fit a univariate exp HP
td = np.array(df.loc[df.mark == 21, :].get("time")).astype(np.float64)

proc = UVHP()
proc.fit(td, method="em")
```

    -58574.97098100516

```python
# verify that the unconditional mean makes sense
mu, alpha, theta = proc.get_params()

N_approx = mu * td[-1] / (1 - alpha)

print(N_approx)
print(len(td))
```

    8005.669015748907
    8003

```python
%%time
# take unconditioned samples from this HP, and estimate the distribution of
# number of occurrences in a ten second period

counts = []
for j in range(100000):
    n = len(proc.sample(T=10000))  # time in millis
    counts.append(n)
```

    CPU times: user 5.65 s, sys: 100 ms, total: 5.75 s
    Wall time: 1.52 s

```python
plt.title("Number of Occurrences in 10secs")
plt.hist(counts, bins=30)
plt.show()
```

## Example 2: Simulated Data

We simulate HPs and try to fit their parameters.

```python
param_pairs = []
for j in range(100):
    # priors
    pars = np.random.gamma(3.), np.random.rand() * .3, 1/np.random.gamma(1.)

    hp = UVHP()
    hp.set_params(*pars)
    smp = hp.sample(10000, method="branching")

    # fit with gd
    hp2 = UVHP()
    hp2.fit(smp, method="gd")

    # fit with em
    hp3 = UVHP()
    hp3.fit(smp, method="em", reltol=1e-8)

    comb_pars = hp.get_params() + hp2.get_params() + hp3.get_params()
    param_pairs.append(comb_pars)
```

```python
Z = np.array(param_pairs)

f, ax = plt.subplots(2, 3, figsize=(15,10))
f.suptitle("Fitted vs Actual Parameters")
for k in range(3):
    ax[0][k].plot(Z[:, k], Z[:, k+3], "k.")
    ax[0][k].set_title(["mu", "alpha", "theta"][k])

    ax[1][k].plot(Z[:, k], Z[:, k+6], "k.")

ax[0,2].loglog()
ax[1,2].loglog()

ax[0,0].set_ylabel("Grad. Descent")
ax[1,0].set_ylabel("Expectation-Maximization")

```

```python

```


# Capítulo 4

```python
# Ajuste de gráficas

import matplotlib as mpl
import matplotlib.pyplot as plt

plt.rcParams['font.family'] = 'serif'
plt.rcParams['figure.figsize'] = (8.0, 8.0)
plt.rcParams['font.size'] = 18
mpl.rcParams['mathtext.fontset'] = 'cm'
mpl.rcParams['mathtext.rm'] = 'serif'
plt.rcParams['lines.markersize']=10

```

## Arreglos Numpy

```python
import numpy as np
numeros_1al8 = range(1,9)
enteros8bytes = np.array(numeros_1al8)
enteros4bytes = np.array(numeros_1al8,dtype='int32')
enteros2bytes = np.array(numeros_1al8,dtype='int16')
enteros1byte  = np.array(numeros_1al8,dtype='int8')
MatrizEnteros8bytes = np.array([numeros_1al8,numeros_1al8,
                                numeros_1al8])

```

```python
complejos8bytes = np.array(numeros_1al8, dtype='complex64')
complejos16bytes = np.array(numeros_1al8, dtype='complex128')
complejos32bytes = np.array(numeros_1al8, dtype='complex256')
```

```python
enteros8bytes = np.array(enteros8bytes, dtype='float64')
enteros8bytes.astype('float64')

```

```python
enteros8bytes.shape
MatrizEnteros8bytes.shape
np.array(np.pi).shape
np.array([np.pi]).shape
```

```python
enteros8bytes.reshape(8,1)
enteros8bytes.reshape(2,4)
enteros8bytes.reshape(4,2)
enteros8bytes.reshape(2,2,2)
```

```python
enteros8bytes.reshape((4,2),order='F')
```

### Ejemplo 4.1

```python
P = np.ones((7,7)) + np.diag(np.arange(10,71,10))

P = [[ 1 for j in range(7)] for i in range(7)]
for i in range(7):
    P[i][i] = (i+1)*10 +1

P
```

### Ejemplo 4.2

```python
import matplotlib.pyplot as plt

def Vsm(m):
    return 1001.38-4.86*m+34.69*m**2

def Vcs(n):
    return 1003.0+16.4*n+2.1*n**(3/2)+0.003*n**(5/2)

def Vsp(m):
    return 17.963-0.1094*m**(3/2)-0.0002*m**2

x1 = np.linspace(0.0, 0.6)
ysm = Vsm(x1)
x2 = np.linspace(0.0, 0.5)
ycs = Vcs(x2)

f, (ax1, ax2) = plt.subplots(1, 2)
ax1.plot(x1, ysm, 'o-',color='black')
ax1.set_ylabel(r'Volumen molar del agua $cm^3$')
ax1.set_xlabel(r'Sulfato de magnesio (mol kg$^{-1})$')
ax1.set_ylim(1001,1012)
ax2.plot(x2, ycs, '-',color='black')
ax2.set_xlabel(r'Cloruro de sodio (mol kg$^{-1})$')
ax2.set_ylim(1001,1012)

```

### Ejemplo 4.3

```python
np.arange(-2*np.pi,-2*np.pi+26*.5,.5)
```

```python
import matplotlib.pyplot as plt

# Partición del eje X en intervalos de longitud .5
puntosEnX = np.arange(-2*np.pi,2*np.pi,.5)
# Partición del eje Y en 10 intervalos de igual longitud
puntosEnY = np.linspace(0,4*np.pi,11)
# Malla
xs, ys = np.meshgrid(puntosEnX,puntosEnY)
# Cálculo de la función sen(x)+cos(x)
f_eval = np.sin(xs) + np.cos(ys)
fig = plt.figure(figsize=(10,10))
# Contornos de la función
plt.contourf(puntosEnX,puntosEnY,f_eval,cmap='gray')
plt.colorbar()

```

### Ejemplo 4.4

```python
import matplotlib.pyplot as plt
np.random.seed(0)
plt.scatter(np.random.rand(100,1),np.random.rand(100,1), c='k')
plt.xlabel(r'$X$')
plt.ylabel(r'$Y$')
```

### Ejemplo 4.5

```python
DistC  = np.fromfunction( lambda i,j: abs(i-j), (5,5),dtype=int)
```

## Problemas centrales

### Juego de la vida

```python
def JuegoVidaV0(B):
    n, m = np.array(B).shape
    Bt = np.zeros((n,m))
    for i in range(1, n-1):
        for j in range(1,m-1):
            vecinas = (B[i-1,j-1] + B[i,j-1] + B[i+1,j-1]+\
                       B[i-1,j]   +            B[i+1,j]  +\
                       B[i-1,j+1] + B[i,j+1]+ B[i+1,j+1])
            if (B[i,j]==0) & (vecinas == 3):
                Bt[i,j]= 1
            elif (B[i,j]==1) & ((vecinas == 3)|(vecinas == 2)):
                Bt[i,j] = 1
    return(Bt)
```

### Vecino más cercano

```python
import math
def VMC0(X):
    if type(X) !=np.ndarray:
        X = np.array(X)
    n, p = X.shape
    D = np.zeros((n,n))
    for i in range(n):
        for j in range(n):
            if i != j:
                D[i,j] = math.sqrt(sum([(x-y)**2\
                for x,y in zip(X[i,],X[j,])]))
            else:
                D[i,i] = np.inf
    return D
```

## Eliminación de ciclos `for`

### Funciones universales

```python
temperaturasF = np.random.uniform(20,90,(100000,1))
temperaturasC = [(x-32)/1.8 for x in temperaturasF]

temperaturasCN = (temperaturasF-32)/1.8
```

```python
%timeit [(x-32)/1.8 for x in temperaturasF]
```

```python
%timeit (temperaturasF-32)/1.8
```

### Ejemplo 4.7

```python
import math
np.random.seed(0)
n = 100000
# Coordenadas cartesianas
X = np.random.random((n,1))
Y = np.random.random((n,1))
```

```python
r = [math.sqrt(i**2+j**2) for i,j in zip(X,Y)]
theta = [math.atan2(i,j) for i,j in zip(X,Y)]
```

```python
rN = np.sqrt(X**2+Y**2)
thetaN = np.arctan2(Y,X)

```

```python
plt.subplot(1, 2, 1)
plt.hist(rN)
plt.subplot(1, 2, 2)
plt.hist(thetaN)
```

```python
%timeit [math.sqrt(i**2+j**2) for i,j in zip(X,Y)]
```

```python
%timeit np.sqrt(X**2+Y**2)
```

## Agregaciones

```python
%timeit min(temperaturasF)
```

```python
%timeit np.min(temperaturasF)
```

### Ejemplo 4.8

```python
X0 = 2*np.random.rand(100000,2)-1
piEstimado = 4*np.mean( (X0**2).sum(axis=1) < 1)
```

### Ejemplo 4.9

```python
c = (1/np.arange(1,101)).sum()
p_i = 1/(c*np.arange(1,101))
```

```python
def NumeroHill(p,q):
    return ((p**q).sum())**(1/(1-q))
```

```python
def NumeroHill_1(p):
    return np.exp(-((p*np.log(p)).sum()))
```

```python
qs = np.linspace(0, 3, 301)
Dq = np.zeros(len(qs))
Dq[qs != 1] = NumeroHill(p_i, qs[qs != 1])    # Produce un error
```

```python
a = np.random.uniform(0,100,(100,200))
a. mean()
a.mean(axis=0)       # promedio por columnas
a.mean(axis=1)       # promedio por filas
```

```python
### Broadcasting
```

```python
a = np.array([1,2,3,4])
b = np.fromfunction(lambda i,j: i*10+j+1,(3,4))
c = np.array(-1)
d = np.fromfunction(lambda i,j: (-i*10),(3,1))
```

```python
a**b
a - c
a + d
b*d
```

```python
d = np.fromfunction(lambda i,j: (-j*10),(1,3))
```

### Ejemplo 4.10

```python
X = np.random.uniform(0,100,(10,2))
X_Estandarizada =  (X - X.mean(axis=0))/X.std(axis=0)
```

```python
XBroad, XMediasBroad = np.broadcast_arrays(X,X.mean(axis=0))
```

### Ejemplo 4.11

```python
X0 = np.random.rand(10,2)
Y  = np.array([-1,1])
Z = X0 + Y[:,np.newaxis, np.newaxis]
```

```python
X0Broad, YBroad = np.broadcast_arrays(X0,\
                       Y[:,np.newaxis, np.newaxis])
X0Broad.strides
YBroad.strides

```

### Indexación

```python
a = np.array([2, 3.5, -2, -4.5, 4, 5, -3.5, 17] )
mascara = [True, False, False, True, True, False, False, True]
a[mascara]
```

```python
a[a > 0]
a[a**2 < 16]
```

### Ejemplo 4.12

```python
a[ abs(a - np.mean(a)) > 2*np.std(a) ] = np.median(a)
```

### Ejemplo 4.13

```python
P = np.zeros((7,7))
P[1:-1,1:-1] =  1
P[2:-2,2:-2] =  2
P[3:-3,3:-3] =  3

```

```python
def matrizPiramide(n=3):
    P = np.zeros((2*n+1,2*n+1),dtype="int")
    for i in range(1,n+1):
        P[i:-i,i:-i] = i
    return(P)
```

```python
a = np.random.randint(1,10,(5,5))
a[ range(5), range(4,-1,-1)]
```

### Ejemplo 4.14

```python
P[P> 1-epsilon ] = 1-epsilon
P[P<epsilon] = epsilon
```

```python
sumaXfila = P.sum(axis=1)
sumaXfila.shape
```

```python
sumaXfilas[:, np.newaxis]
```

```python
def probTotalAcotada(P, epsilon):
    P[ P > 1-epsilon ] = 1-epsilon
    P[ P < epsilon ] = epsilon
    sumaXfila = P.sum(axis=1)
    P = P / sumaXfila[:, np.newaxis]
    return P
```

### Ejemplo 4.15

```python
def NumeroHillv2(p,qs=np.linspace(0,3,301)):
    sumaPq = (p[:,np.newaxis]**qs[np.newaxis,:]).sum(axis=0)
    numerosHill = np.full(len(qs), NumeroHill_1(p))
    numerosHill[qs != 1] = sumaPq[qs!=1]**(1/(1-qs[qs != 1]))
    return numerosHill
```

### Ejemplo 4.16: Conjunto de Mandelbrot

```python
orbitas = np.zeros((4,31))
ValoresC = np.array([.2,-1.38,.3,-1.9])

def fc(z,c):
    return z**2 + c

for i in range(1,31):
    orbitas[:,i] = fc(orbitas[:,i-1],ValoresC)

for i in range(4):
    plt.subplot(2, 2, i+1)
    plt.plot(range(31), orbitas[i,], 'ko-')
    plt.xlabel('Iteración')
    plt.ylim(-3,3)
    plt.text(18,-2.5,r'$c=$'+str(ValoresC[i])+r'$+0i$')

```

```python
def mandelbrot(npixels = 500, iteracionesMax = 20):
    # Crea una malla de tamaño npixels X npixels
    # en los intervalos [-2,1] y [-1.5,1.5]
    x, y = np.meshgrid(np.linspace(-2, 1, npixels),
                       np.linspace(-1.5, 1.5, npixels))
    c = x + 1j * y
    z = c.copy()
    m = np.zeros((npixels, npixels))
    for k in range(iteracionesMax):
        # Índices de los puntos en la malla tal
        # que el módulo de z_k  es menor que 10
        indices = np.abs(z) <= 10
        # Siguiente puntos en la órbita
        z[indices] = z[indices] ** 2 + c[indices]
        m[indices] = k
    return m

```

```python
M = mandelbrot(iteracionesMax=200)
```

```python
fig, ax = plt.subplots(1, 1, figsize=(10, 10))
ax.imshow(200-M, cmap=plt.cm.gray)
ax.set_axis_off()
```

```python
fig, ax = plt.subplots(1, 1, figsize=(10, 10))
ax.imshow(np.log(M), cmap=plt.cm.gray)
ax.set_axis_off()
```

```python
fig, ax = plt.subplots(1, 1, figsize=(10, 10))
ax.imshow(np.log(M), cmap=plt.cm.hot)
ax.set_axis_off()

```

### Ejemplo 4.17: Regla 30 y 90

```python
identificador = 4*estado[:-2] + 2*estado[1:-1] + estado[2:]
```

```python
regla = np.array([0,1,1,1,1,0,0,0])
estado = regla[identificador]
```

```python
def WolframRegla(numero=30, iteraciones=30):
    nCA = 60                 # Ancho de la matriz de salida
    if nCA < iteraciones:
        nCA = 2*iteraciones  # Una mejor visualización
    # obtención de la regla
    regla = list(map(int,list(format(numero,'010b'))))
    regla.reverse()
    regla = np.array(regla)
    # inicialización matriz de salida
    salida = np.zeros((iteraciones+1,nCA),dtype='int8')
    salida[0,nCA//2] = 1
    # ciclo con el número de iteraciones deseadas
    for i in range(iteraciones):
        salida[i+1,1:-1] = regla[salida[i,:-2]*4+\
                                 salida[i,1:-1]*2+salida[i,2:]]
    return salida

```

```python
plt.imshow(1-WolframRegla(30), cmap=plt.cm.gray)
```

```python
plt.imshow(1-WolframRegla(90), cmap=plt.cm.gray)
```

### Juego de la vida

```python
def NuevaGeneracion(Actual):
    # Cálculo de vecinos
    V = (Actual[0:-2,0:-2]+Actual[0:-2,1:-1]+Actual[0:-2,2:]+
         Actual[1:-1,0:-2]+                  Actual[1:-1,2:]+
         Actual[2:  ,0:-2]+Actual[2:  ,1:-1]+Actual[2:  ,2:])
    # Reglas
    # Tres vecinos y celda vacía
    nacimientos = (Actual[1:-1,1:-1]==0) &  (V==3)
    # Dos o tres vecinos y célula viva
    sobreviven = ((V==2) | (V==3)) & (Actual[1:-1,1:-1]==1)
    # Nuevos estados. Por defecto célula muerta
    Actual.fill(0)
    # Asigna el valor 1 a las celdas donde hubo nacimientos o
    # donde la célula sobrevivió
    Actual[1:-1,1:-1][nacimientos | sobreviven] = 1
    return Actual
```

```python
pulsar = np.zeros((17,17),dtype='int8')
filasVivas = [2,4,5,6,7,9,10,11,12,14]
columnasVivas = [[4,5,6,10,11,12],
                 [2,7,9,14],
                 [2,7,9,14],
                 [2,7,9,14],
                 [4,5,6,10,11,12],
                 [4,5,6,10,11,12],
                 [2,7,9,14],
                 [2,7,9,14],
                 [2,7,9,14],
                 [4,5,6,10,11,12]]
for i in range(len(filasVivas)):
    pulsar[filasVivas[i],columnasVivas[i]] = 1

```

```python
plt.imshow(1-pulsar, cmap=plt.cm.gray)
```

```python
plt.imshow(1-NuevaGeneracion(pulsar), cmap=plt.cm.gray)
```

```python
glider = np.zeros((12,12),dtype='int8')
glider[1:4,3] = 1
glider[2,1] = 1
glider[3,2] = 1
```

```python
plt.imshow(1-glider, cmap=plt.cm.gray)
```

```python
plt.imshow(1-NuevaGeneracion(glider), cmap=plt.cm.gray)
```

### Leopardo

```python
def bordes(M,k):
    tam = M.shape[0]
    vTam = tam + k
    V = np.zeros((tam+2*k,tam+2*k),dtype='int8')
    V[k:vTam,k:vTam] = M
    # Bordes izquierdo y derecho
    V[k:vTam,0:k] = M[:,-k:]
    V[k:vTam,-k:] = M[:,:k]
    # Bordes superior e inferior
    V[0:k,k:vTam] = M[-k:,:]
    V[-k:,k:vTam] = M[:k,:]
    # Esquinas
    V[:k,:k]   = M[-k:,-k:]
    V[-k:,-k:] = M[:k,:k]
    V[-k:,:k]  = M[:k,-k:]
    V[:k,-k:]  = M[-k:,:k]
    return V

def conteoAct(M):
    V = (M[0:-2,0:-2]+M[0:-2,1:-1]+M[0:-2,2:]+
         M[1:-1,0:-2]+M[1:-1,1:-1]+M[1:-1,2:]+
         M[2:  ,0:-2]+M[2:  ,1:-1]+M[2:  ,2:])
    return(V)

def conteoInh(M, k):
    m = M.shape[0]
    n = m - 2*k
    V = np.zeros((n,n))
    for i in range(k,(m-k)):
        for j in range(k,(m-k)):
            V[i-k,j-k] = np.sum(M[(i-k):(i+k+1),(j-k):(j+k+1)])
    return V

def leopardo(P, inh = 6, wa = 1, wi= 0.1):
    k1 = inh-1
    Ve = bordes(P, inh)
    I = conteoInh(Ve,inh)
    A = conteoAct(Ve[k1:-k1,k1:-k1])
    Pnueva = np.zeros(P.shape)
    Pnueva[ (wa*A >  wi*I)]=1
    return Pnueva

```

```python
P0 = np.random.randint(0,2,(100,100))
P1 = leopardo(P0,5)
P2 = leopardo(P1,5)
P99 = P2.copy()
for i in range(97):
    P99 = leopardo(P99,5)
```

```python
fig, ax = plt.subplots(2, 2, figsize=(20, 20))
ax[0,0].imshow(1-P0, plt.cm.copper)
ax[0,0].set_axis_off()
ax[0,1].imshow(1-P1, plt.cm.copper)
ax[0,1].set_axis_off()
ax[1,0].imshow(1-P2, plt.cm.copper)
ax[1,0].set_axis_off()
ax[1,1].imshow(1-P99, plt.cm.copper)
ax[1,1].set_axis_off()

```

### Vecino más cercano

```python
def distEuclidiana(X):
    n, p = X.shape
    diferencias = X.reshape(n,1,p) - X
    distancias =  (diferencias**2).sum(axis=2)
    np.fill_diagonal(distancias, np.Inf)
    return distancias
```

```python
def VMC(X):
    distancias = distEuclidiana(X)
    k = np.argmin(distancias,0)
    return X[k,:]
```

```python
from matplotlib.collections import LineCollection

# Arreglo con 200 puntos aleatorios en R^2
X = np.random.random((200,2))
# Vecinos más cercanos
vecinos = VMC(X)
# Coordenadas de los puntos y su vecino más cercano
coordenadas = np.zeros((200,2,2))
coordenadas[:,0,:] = X
coordenadas[:,1,:] = vecinos
lineas = LineCollection(coordenadas)
fig, ax = plt.subplots(1,1,figsize = (10, 10))
ax.scatter(X[:,0], X[:,1], c = 'black')
ax.add_artist(lineas)
```

### Normalización cuantílica

```python
ys = np.mean(np.sort(X,axis=0),axis=1)
```

```python
def rangos(x):
    n = len(x)
    orden =  x.argsort()
    rango = np.arange(n,dtype='int')
    rango[orden] = np.arange(n)
    return rango

def normCuantilica0(X):
    r = np.apply_along_axis(rangos,0,X)
    ys = np.mean(np.sort(X,axis=0),axis=1)
    z = ys[r]
    return z

```

```python
X0 = np.array([[1, 6, 4, 3], [2, 5, 7, 5],
               [3, 4, 1, 5],[4, 3, 6, 4],
               [5, 2, 2, 5],[6, 1, 8, 8]])
normCuantilica0(X0)
```

```python
def empatesM(x):
    u = np.unique(x)
    return len(u)<len(x)

def empates(x):
    u, pos,conteos = np.unique(x,return_inverse=True,
                               return_counts=True)
    return pos, conteos

def normCuantilica(X):
    r  = np.apply_along_axis(rangos,0,X)
    ys = np.mean(np.sort(X,axis=0),axis=1)
    z = ys[r]
    colEmpates = np.apply_along_axis(empatesM,0,X)
    if sum(colEmpates) > 0:
        for i in np.arange(len(colEmpates))[colEmpates]:
            posEmp, conEmp = empates(X[:,i])
            ysEmp = np.bincount(posEmp, z[:,i])/conEmp
            z[:,i] = ysEmp[posEmp]
    return z
normCuantilica(X0)
```

```python
X = -np.log(np.random.random((200,2)))
Z = normCuantilica(X)
```

## Álgebra lineal

```python
from numpy import linalg as nl
A = np.array([[0.9,0.3,0.1],[0.1,0.5,0.2],[0.0,0.2,0.7]])
L = np.fromfunction(lambda x,y: 2+ 3*x - 2*y,(3,2))
R = np.random.randint(-10,11,(3,3))
b = np.array([30.0,25.0,10.0])
D = np.array([[4, 5],[-2, 5]])
E = np.eye(3, k= -1)
```

```python
5*A
A*R
```

```python
AR = A.dot(R) # o
AR = A@R
Ab = A.dot(b)  # o
Ab = A@b
```

```python
M = 3*A@A - A@R.T - 4*E
M@M@M
```

```python
nl.matrix_power(M,3)
```

### Ejemplo 4.18

```python
R  = np.array([[1,-1,1],[0,10,25],[20,10,0]])
V  = np.array([0,90,80])
nl.solve(R,V)
```

```python
nl.inv(R)@V
```

### Ejemplo 4.19

```python
A = [[2,1,-1],[5, 0, 2], [3,-1, 6]]
Lambda = Symbol('lambda')
Asimb  = Matrix(A)
PolCarac = (Asimb - Lambda*np.eye(3)).det().expand()
```

```python
Anum = np.array(A)
-nl.matrix_power(Anum,3)+8*nl.matrix_power(Anum,2)-\
                         12*Anum-15*np.eye(3)
```

### Ejemplo 4.20

```python
val, vec = nl.eig(MatrixCom([-42,-43,28,18,-10,1]))
```

### Ejemplo 4.21

```python
p0 = np.array([200,100,100,100,200,200,100]).reshape(7,1)
LeslieFocas = np.zeros((6,6))
LeslieFocas = np.diag(Supervivencia[:-1],k=-1)
LeslieFocas[0,] = Fecundidad
p3 = nl.matrix_power(LeslieFocas,3)@p0
sum(p3)
```

```python
val_prop, vec_prop = nl.eig(LeslieFocas)
abs(val_prop)
```

## Polinomios

```python
Pol = np.polynomial.Polynomial
p_gr = Pol([1,1,-1])
p2 = Pol.fromroots([-1 + 1j, -1-1j, 3])
```

```python
p_gr.roots()
p2.roots()
```

### Ejemplo 4.22

```python
from sympy import *
h, z, R = symbols('h z R')
integrate(pi*(R**2-z**2), (z, -R, -R+h)).simplify()
```

```python
R, F  = 200, 200
V0 = 4/3*np.pi*R**3
T  = V0/F
h  = np.zeros(200)
tiempo = np.linspace(0, T, 200)
Coeficientes = [0, 0, 200*np.pi, -np.pi/3]
for i, t in enumerate(tiempo):
    Coeficientes[0] = F*t - V0
    p = Pol(Coeficientes)
    raices = p.roots()
    h[i] = raices[ (0 <= raices) & (raices <= 2*R) ][0]
```

```python
plt.plot(tiempo/60, h, 'k-')
plt.xlabel('Tiempo (h)')
plt.ylabel('Altura del combustible en el tanque (cm)')
```

### Ejemplo 4.23

```python
R = 0.08206         # L atm /(mol K)
a = 4.225/1.01325   # L^3 atm/ (mol^2)
b = 0.03707         # L/mol
T1 = 298            # K
P1 = 1              # atm
coeficientes1 = [-a*b, a, -(P1*b+R*T1), P1]
Pol(coeficientes1).roots()
T2 = 500            # K
P2 = 120            # atm
coeficientes2 = [-a*b, a, -(P2*b+R*T2), P2]
Pol(coeficientes2).roots()
```

```python
def WaalsVM(T, P, a = 4.225/1.01325, b = 0.03713):
    R = 0.08206           # L atm / (mol K)
    Tc = 8*a/(27*b*R)     # K
    Pc = a/(27*b**2)      # atm
    coeficientes = [-a*b, a, -(P*b + R*T), P]
    z = Pol(coeficientes).roots()
    return z, Tc, Pc
```

```python
def isoterma(V,T = 405, a = 4.225/1.01325, b = 0.03707):
    R = 0.08206       # L atm /(mol K)
    P = (R*T)/(V-b) - a/V**2
    return P
```


```python
import sympy

from sympy import symbols, init_session, sqrt
init_session(quiet=True)
```

    IPython console for SymPy 0.7.6.1 (Python 3.5.4-64-bit) (ground types: python)

```python
tau0, p0, b, P, a, Gamma, delta, sigma, sigma_tau_0 = symbols('tau0 p0 b P a Gamma delta sigma, sigma_tau_0',
                                                              positive=True, real=True)
```

```python
T = 2 * tau0 * sqrt(1 - b**2)
tau = 2 * tau0 * p0 / sqrt(1 - b**2)
theta = tau/T
Q = sqrt(Gamma * T) * delta/sigma
sigma_tau = T * sqrt(6 * theta) / Q
sigma_T = T * sqrt(2 * theta) / Q
```

$$ p_0 = \frac{\tau T}{4 \tau_0^2}$$

$$ \sigma_{p_0}^2 = \left(\frac{\partial p_0}{\partial \tau}\right)^2 \sigma_\tau^2 + \left(\frac{\partial p_0}{\partial \tau}\frac{\partial p_0}{\partial T}\right) \mathrm{cov}(\tau,T)+\left(\frac{\partial p_0}{\partial T}\right)^2 \sigma_T^2 +\left(\frac{\partial p_0}{\partial \tau_0}\right)^2 \sigma_{\tau_0}^2 $$

$$ \sigma_{p_0}^2 = \left(\frac{T  \sigma_\tau}{4  \tau_0^2} \right)^2 +
               \left(\frac{\tau  T }{16 \tau_0^4} b  \theta^2 T^2\right) +
               \left(\frac{\tau  \sigma_T}{4 \tau_0^2} \right)^2 +
               \left(\frac{\tau  T }{2 \tau_0^3}  \sigma_{\tau_0}\right)^2 $$

```python
sigma_p0_sq = ((T * sigma_tau / 4 / tau0**2)**2 +
               (tau * T / 16 / tau0**4) * b * theta**2 * T**2 +
               (tau * sigma_T / 4 / tau0**2)**2 +
               (tau * T / 2 / tau0**3 * sigma_tau_0)**2)
```

```python
sqrt(sigma_p0_sq).simplify()
```

```python
print(sqrt(sigma_p0_sq).simplify())
```

    sqrt(p0)*sqrt(-(-Gamma*b*delta**2*p0**2*tau0**2*(-b**2 + 1)**(3/2) - 4*Gamma*delta**2*p0*sigma_tau_0**2*(-b**2 + 1)**(5/2) + p0**2*sigma**2*tau0*(b**2 - 1) - 3*sigma**2*tau0*(-b**2 + 1)**3)/(-b**2 + 1)**(5/2))/(sqrt(Gamma)*delta*tau0)

```python

```


# Computation for the EigenValues of a 3DOF System in Structural Dynamics

In this example, we will look on how to solve a 3DOF system using python.
This uses the library of Sympy (Symbolic Python) to solve the equations in terms of variables (letters)

$$ m \ddot(x) + c \dot(x) + k (x) = 0 $$

## Step 1: Loading libraries

```python
# imports numpy and matplotlib
%pylab inline

# imports sympy
import sympy as sm

# prints output using LaTeX for pretty viewing
sm.init_printing(use_latex=True)
```

    Populating the interactive namespace from numpy and matplotlib

## Step 2: Establish Mass Matrix

```python
# assigning masses
m1 = 1000 #(kg)
m2 = 1500 #(kg)
m3 = 2000 #(kg)

# making the mass matrix
m = sm.Matrix([[m1,0,0],[0,m2,0],[0,0,m3]])

# printing the mass matrix
m
```

## Step 3: Establish Stiffness Matrix

```python
# computing for the stiffnesses
k1 = 600000 #(N/m)
k2 = 1200000 #(N/m)
k3 = 1800000 #(N/m)

# making the stiffness matrix
k = sm.Matrix([[k1,-k1,0],[-k1,k1+k2,-k2],[0,-k2,k2+k3]])

# printing the stiffness matrix
k
```

## Step 4: Solving the EigenValue Problem
Making the Matrix

```python
# assigning the "x" variable as a symbol
x = sm.Symbol('x')

# making the Eigen matrix
A = k-m*x**2

# printing the matrix
A
```

Getting the Determinant of the Matrix

```python
sm.det(A).simplify()
```

Solving the determinants and listing the types of modes

```python
# This step solves the equation in line
B = sm.solve(sm.det(A),x**2)

# This line converts the array into real number
C = real(B)

# This line makes an array into a complex numbers
D = array((C.astype(complex)))

# Gets the squareroot of the real part of the complex number
E = 1/(sqrt(D.real) / 2 / math.pi)

# printing the Modes
print("Period 1 = {0}, Period 2 = {1}, Period 3 = {2}".format(E[0],E[1],E[2]))
```

    Period 1 = 0.2023720283163326, Period 2 = 0.43267656159437806, Period 3 = 0.13629624070103266


## RKR Computation
All Code and Markdown written by Gary Zeri, Chapman University Student and member of the LaRue Cat Lab

All equations and information within this notebook originated from The Computation of RKR Potential Energy Curves of Diatomic Molecules using Matematica, written by Peter Senn.

The RKR method is a procedure used to determine the potential energy curves of diatomic molecules by calculating the classical turning points, $r_-$ and $r_+$ from the molecule's diatomic constants.

The turning points, $r_{\pm}$, can be computed using the following equation:
$$r_{\pm} = \frac{f(v)}{2} \cdot \left[\sqrt{1 + \frac{1}{f(v)g(v)}} \pm 1\right]$$

It should be noted that in the above equation the fraction in the square root has a one in the numerator while in the paper by Peter Senn the numerator has a four. The four is not present in the Matematica code in Senn's paper and in Lecture #21: Construction of Potential Curves by the Rydberg-Klein-Rees Method (RKR) from MIT OpenCourseWare, the four is not included as well, leading to the omission of the four in the equation presented here to provide an exact equation to that used within the below code.

$f(v)$ and $g(v)$ are defined as follows:
$$f(v) = \int^v_{\frac{-1}{2}}{ \left[ \frac{dv^{'}}{\sqrt{E(v)-E(v^{'})}{}} \right] }$$

$$g(v) = \int_{\frac{-1}{2}}^v{\left[ \frac{\frac{\partial E}{\partial J}}{\sqrt{E(v) - E(v^{'})}}\right]dv^{'}}$$

In the original paper by Senn, there are constants in front of the integrals that are omitted here as they are used to scale the values to a desired unit system. Since this RKR code will use wavenumbers for energy and angstroms for bond distance, the constant of $\frac{8.211609}{\sqrt{\mu}}$ will be multiplied by the final answers to perform the conversion to Hartrees as an energy value.

For vibronic states with no rotation and $J=0$, $E(v)$ and $\frac{\partial E}{\partial J}$ can be represented as:
$$E(v) = \omega_e(v+ \frac{1}{2}) - \omega_ex_e(v+\frac{1}{2})^2 + \omega_ey_e(v+\frac{1}{2})^3 + \omega_ez_e(v+\frac{1}{2})^4 + \dots$$

$$\frac{\partial E}{\partial J} = B(v) = B_e -\alpha_e(v+\frac{1}{2}) + y_e(v+\frac{1}{2})^2 + \dots$$

An important computational issue to note with the RKR method is that the integrand term $\frac{dv^{'}}{\sqrt{E(v) - E(v^{'})}}$ will become one over zero since the integration is from $\frac{-1}{2}$ to $v$, thus the integrand will become $\frac{1}{\sqrt{E(v) - E(v)}}$ when $v^{'} = v$ at the end of the integration.

In order to deal with this issue, the integral is changed from $\int^{v}_{\frac{-1}{2}}$ to $\int^{v-\delta}_{\frac{-1}{2}}$, where $\delta$ is some extremely small value. A correction term is then added to both $f(v)$ and $g(v)$ to account for the missing endpoint of the integration resulting in new and more easily computable versions of $f(v)$ and $g(v)$:

$$f(v) = \int^{v-\delta}_{\frac{-1}{2}}{\frac{dv^{'}}{\sqrt{E(v) - E(v^{'})}} + \sqrt{\frac{\delta}{Q_v}}}$$

$$g(v) = \int^{v-\delta}_{\frac{-1}{2}}{\frac{B(v^{'})}{\sqrt{E(v)-E(v^{'})}}dv^{'} } + 2B(v)\sqrt{\frac{\delta}{Q_v}} $$

Where $Q_v$ is the following series:
$$Q_v = w_e-2\omega_ex_e(v+\frac{1}{2}) + 3\omega_ey_e(v+\frac{1}{2})^2 + 4\omega_ez_e(v+\frac{1}{2})^3 + \dots$$

The following code only uses the terms listed above in the computation for all series listed.

## RKR With Excited States

In order to allow for RKR computations with excited states, $E(v)$ and $\frac{\partial E}{\partial J}$ must include the $J$ term. The full $E(v)$ equation is included below:
\begin{align}
    E(v) &= \omega_e(v+\frac{1}{2}) - \omega_ex_e(v+\frac{1}{2})^2 + \omega_ey_e(v+\frac{1}{2})^3 + \omega_ez_e(v+\frac{1}{2}) + \left[B_e-\alpha_e(v+\frac{1}{2}) + y_e(v+\frac{1}{2})^2\right] \cdot J(J+1)-D_eJ^2(J+1)^2 \\
    E(v) &= c(v) + \left[B_e-\alpha_e(v+\frac{1}{2}) + y_e(v+\frac{1}{2})^2\right] \cdot J(J+1)-D_eJ^2(J+1)^2 \\
    E(v) &= c(v) + f(v, J) - g(J)
\end{align}

\begin{align}
    \frac{\partial E}{\partial J} &= \frac{\partial}{\partial J}\left[ c(v) + f(v, J) - g(J)\right] \\
    \frac{\partial E}{\partial J} &= 0 + \frac{\partial}{\partial J}f(v, J) - \frac{\partial}{\partial J}g(J) \\
    \frac{\partial E}{\partial J} &= \frac{\partial}{\partial J}f(v, J) - \frac{\partial}{\partial J}g(J) \\
\end{align}

\begin{align}
    \frac{\partial }{\partial J}f(v, J) &= \frac{\partial }{\partial J}\left[d(v)\cdot h(J)\right] \\
    &= d(v) \frac{\partial}{\partial J}h(v) \\
    &= d(v) \frac{\partial}{\partial J}\left[J(J+1)\right] \\
    &= d(v) \cdot \left(2J + 1\right) \\
    &= \left[B_e-\alpha_e(v+\frac{1}{2}) + y_e(v+\frac{1}{2})^2\right] \cdot \left(2J + 1\right) \\
    &= B_e \cdot \left(2J + 1\right) - \alpha_e(v+\frac{1}{2})\left(2J + 1\right) + y_e(v+\frac{1}{2})^2 \left(2J + 1\right) \\
    &= 2JB_e + B_e -2J\alpha_e(v+\frac{1}{2}) -2\alpha_e(v+\frac{1}{2} ) + 2Jy_e(v+\frac{1}{2})^2 + y_e(v+\frac{1}{2})^2 \\
    &= 2J\left[B_e -\alpha_e(v+\frac{1}{2}) + y_e(v+\frac{1}{2})^2\right] + B_e - \alpha_e(v+\frac{1}{2}) + y_e(v+\frac{1}{2})^2 \\
    &= 2Jd(v) + d(v) \\
    \frac{\partial}{\partial J}f(v, J) &= d(v)\left(2J + 1\right)
\end{align}

\begin{align}
    \frac{\partial }{\partial J}g(J) &= \frac{\partial}{\partial J} \left[D_e J^2(J+1)^2 \right] \\
        &= D_e \frac{\partial}{\partial J}\left[ J^2(J+1)^2\right] \\
        &= D_e \left[ \frac{\partial}{\partial J}[J^2] \cdot (J+1)^2 + J^2 \cdot \frac{\partial}{\partial J}[(J+1)^2]\right] \\
        &= D_e \left[ 2J \cdot (J+1)^2 + J^2 \cdot 2(J+1)\right] \\
        &= 2J(J+1)D_e \left[ (J+1) + J\right] \\
        &= 2J(J+1)D_e\left[ 2J + 1 \right] \\
    \frac{\partial}{\partial J}g(J) &= 2D_eJ(J+1)(2J+1)
\end{align}

\begin{align}
        \frac{\partial E}{\partial J} &= \frac{\partial}{\partial J}f(v, J) - \frac{\partial}{\partial J}g(J) \\
       \frac{\partial E}{\partial J} &= d(v)(2J + 1) - 2D_eJ(J+1)(2J+1) \\
       \frac{\partial E}{\partial J} &= \left[B_e -\alpha_e(v+\frac{1}{2}) + y_e(v+\frac{1}{2})^2\right](2J+1) - 2D_eJ(J+1)(2J+1)
\end{align}

```python
#Import all needed classes here and create all global variables

import numpy as np
from tqdm import tqdm
from scipy.integrate import quad as integrate

from plotly.offline import iplot, init_notebook_mode
init_notebook_mode(connected=True)

#Should be a very small value
#Is the distance from v that the integration stops at
delta = pow(10, -3)

#Reduced Molecular Mass
#In Non Hartree Atomic Units, each proton has an amu of 1
#u = (12 * 16) / (12+16)
#u = (14 * 16) / (14 + 16)
u = 0.5
#Diatomic Constants, must be in Wavenumbers (1 / cm)
#Currently Has Data for COA
'''
COA
we = 1518.28
wxe = 19.4
wye = 0
wze = 0
Be = 1.6115
alphae = 0.02325
ye = 0
De = 0.00000733
'''
'''
#NOX
we = 1904.20
wxe = 14.075
wye = 0
wze = 0
Be = 1.67195
alphae = 0.0171
ye = 0
De = 0.54 * pow(10, -6)
'''
'''
#NOA
we = 2374.31
wxe = 10.106
wye = -0.0465
wze = 0
Be = 1.9965
alphae = 0.01915
ye = 0
De = 5.4 * pow(10, -6)
'''

#H2X
we = 4401.21
wxe = 121.33
wye = 0
wze = 0
Be = 60.853
alphae = 3.062
ye = 0
De = 0.0471
re = .74144
```

```python
#Define All Functions Here

def E(v, J=0):
    term = v + 0.5
    d = (we * term) - (wxe*pow(term, 2)) + (wye*pow(term, 3)) + (wze*pow(term,4))
    return d * ( 2*J + 1) - 2*De*J*(J+1)*(2*J+1)

def B(v):
    term = v + 0.5
    return Be - (alphae * term)  + ye*pow(term, 2)

#Used in the correctionFactor calculation
def Q(v):
    term = v + 0.5
    return we - (2*wxe*term) + (3*wye*pow(term, 2)) + (4*wze*pow(term,3))

#Used to correct integrals that stop delta away from v
def correctionFactor(v):
    return 2 * np.sqrt(delta / Q(v))

def integralRadical(v, vPrime):
    return np.sqrt(E(v) - E(vPrime))

def f(v):
    integrand = lambda vPrime: 1 / integralRadical(v, vPrime)
    return integrate(integrand, -0.5, v-delta)[0] + correctionFactor(v)

def g(v):
    integrand = lambda vPrime : B(vPrime) / integralRadical(v, vPrime)
    return integrate(integrand, -0.5, v-delta)[0] + (B(v)*correctionFactor(v))

#v refers to a float energy level for the potential well
#returns the tuple (r+, r-)
def RKR(v):

    c0 = (8.211609 * f(v) ) / (2 * np.sqrt(u))

    radicand = 1 / (f(v) * g(v))
    c1 = np.sqrt(1 + radicand)

    return c0 * (c1 + 1), c0 * (c1 - 1)
```

```python
#Lists to hold data
x = []
y = []
ddX = []
ddX2 = []
ddx = []
ddx2 = []
leftAsympCutOff = False

x.append(re)
y.append(0)

#Loop to generate all data to graph
print("Building RKR Potential")
for v in tqdm(np.arange(-.499, 17, .01)):

    energy = E(v, 0)
    xPoints = RKR(v)

    if(len(x) != 0):
        #Compute First Derivative
        ddX.append( (min(xPoints) + min(x[-2:])) / 2 )
        slope = (energy-y[-1]) / ( min(xPoints) - min(x[-2:]) )
        ddx.append( slope )

        if(len(ddx) > 1):
            #Compute 2nd Derivative
            ddX2.append( (ddX[-2] + ddX[-1]) / 2 )
            ddx2.append( (ddx[-1] - ddx[-2]) / (ddX[-1] - ddX[-2]) )

            #Determine if Cutoff should be used
            if(ddx2[-1] <= 0):
                leftAsympCutOff = True

    if(not leftAsympCutOff):
        x.extend( xPoints )
        y.extend( [energy] * 2 )
    else:
        x.append(max(xPoints))
        y.append(energy)

print("Graphing Potential")
#Prepare data for graphing
figure = {
    "data":[
        {
            "type":"scatter",
            "x":x,
            "y":y,
            "connectgaps":True,
            "mode":"markers",
            "name":"RKR Potential",
        }
    ],

    "layout":{
        "xaxis":{"title":"Bond Distance non-Hartree Atomic Units"},
        "yaxis":{"title":"Energy in Wavenumbers"},
        "title":{"text":"RKR Generated Potential Energy Curve"}
    }
}

figure["data"].append(
    {
        "type":"scatter",
        "x":ddX,
        "y":ddx,
        "name":"1st Derivative"
    }
)

figure["data"].append(
    {
        "type":"scatter",
        "x":[ddX[ddx.index(min(ddx))]] * 2,
        "y":[0, pow(10, 4) * 2],
        "name":"1st Derivative Turning Point"
    }
)

figure["data"].append(
    {
        "type":"scatter",
        "x":ddX2,
        "y":ddx2,
        "name":"2nd Derivative"
    }
)

iplot(figure)
```

```python
#Test for the RKR Class
from rkr import rkr as rkrClass

rkr = rkrClass()

rkr.setDiatomicConstants(alphae, Be, we, wxe, wye, wze, ye)
rkr.setReducedMass(0.5)
rkr.setDelta(delta)

x, y = rkr.graphData()

figure = {

    "data":[
        {
            "type":"scatter",
            "x":x,
            "y":y,
            "connectgaps":True,
            "mode":"markers",
            "name":"RKR Potential",
        }
    ],

    "layout":{
        "xaxis":{"title":"Bond Distance non-Hartree Atomic Units"},
        "yaxis":{"title":"Energy in Wavenumbers"},
        "title":{"text":"RKR Generated Potential Energy Curve"}
    }
}

iplot(figure)
```

```python
E(900)
```

```python

```


# Variational Autoencoder

As suggested by the title, VAE is extended from autoencoders. In autoencoders, we have an encoder which maps the input $x$ to a latent variable $z$ and a decoder then reconstruct $\hat{x}$ from $z$. The learned feature $z$ could serve as a representation of the input since most important information is already included in $z$ if $\hat{x}$ is close to $x$.

In VAE, we add probability to the autoencoder and define the density function of $x$ based on $z$ in the form of:

$$
\begin{equation}
p_\theta(x) = \int p_\theta(z)p_\theta(x|z)dz
\end{equation}
$$
In practice, we choose a simple distribution like Gaussian for $p(z)$ and use a neural network to represent the complex conditional distribution $p(x|z)$. We learn the parameters $\theta$ by maximizing the likelihood of the training data.

Although the integration over $z$ is intractable, we could optimize a lower bound of $p(x)$ with the help of the encoder and bayes rule. The encoder provides a conditional probability distribution $q_\phi (z|x)$. As a result, the logarithm of the likelihood of training data could be written as:

$$
\begin{align}
log p_\theta(x^{i}) &=  E_{z\sim q_\phi(z|x^{i})} \big[log \frac{p_\theta(z)\ p_\theta(x^{i}|z)}{p_\theta(z|x^i))} \frac{q_\phi(z|x^{i})}{q_\phi(z|x^i}\big]  \\
 &= E_z\big[log p_\theta(x^i|z)\big] - E_z\big[\frac{q_\phi(z|x^i)}{p_\theta(z)}\big] + E_z\big[\frac{q_\phi(z|x^i)}{p_\theta(z|x^i)}\big] \\
 &= E_z\big[logp_\theta(x^i|z)\big] - D_{KL}(q_\phi(z|x^i) \| p_\theta(z)) + D_{KL}(q_\phi(z|x^i) \|p_\theta(z|x^i) )
\end{align}
$$

Although the third term $D_{KL}(q_\phi(z|x^i) \|p_\theta(z|x^i)$ in equation $4$ is still intractable, it's always bigger than 0. We ignore it and optimize the first two terms which is a lower bound of $log p_\theta(x^i)$.

The maximization of $logp_\theta(x^i)$ could be factorized into maximization of $E_z\big[logp_\theta(x^i|z)\big]$ (reconstruct input as best as possible) and minimization of $D_{KL}(q_\phi(z|x^i) \| p_\theta(z))$ (make approximate posteior probability distribution close to prior )

Then the architecture of VAE is like below:

As we can see, the output of the encoder and decoder are nolonger deterministic $z$ or $x^i$ but are probability distributions! Based on probability distributions, we sample $z$ and $x^i$. Besides that, with reparameterization trick, we make sampling also differentiable. Okay! Let's get through a project of VAE on MNIST to better understand how it works.

```python
# import packages
import torch
import torch.utils.data
from torch import nn, optim
from torch.nn import functional as F
from torchvision import datasets, transforms
from torchvision.utils import save_image

from easydict import EasyDict as edict
import os
```

```python
# initialize parameters and dataset
args = edict()
args.batch_size = 128
args.epochs = 10
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

train_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data/MNIST', train=True, download=True,
                   transform=transforms.ToTensor()),
    batch_size=args.batch_size, shuffle=True)
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data/MNIST', train=False, transform=transforms.ToTensor()),
    batch_size=args.batch_size, shuffle=True)
```

```python
# define VAE
class VAE(nn.Module):
    def __init__(self):
        super(VAE, self).__init__()

        self.en1 = nn.Linear(784, 400)
        self.en2_mu = nn.Linear(400, 20)
        self.en2_logvar = nn.Linear(400, 20)
        self.de1 = nn.Linear(20, 400)
        self.de2 = nn.Linear(400, 784)

    def encode(self, x):
        h1 = F.relu(self.en1(x))
        return self.en2_mu(h1), self.en2_logvar(h1)

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5*logvar)
        eps = torch.randn_like(std)
        return mu + eps*std

    def decode(self, z):
        h3 = F.relu(self.de1(z))
        return torch.sigmoid(self.de2(h3))

    def forward(self, x):
        mu, logvar = self.encode(x.view(-1, 784))
        z = self.reparameterize(mu, logvar)
        return self.decode(z), mu, logvar

```

Here we assume the prior of $z$ is a Gaussian distribution and the conditional probability distribution of the decoder is a bernoulli distribution. That's why the encoder has two outputs and the decoder has only one. Reparameterize function simulates the sampling based on $\mu + randn * std$ that makes $z$ also differentiable.

```python
def loss_function(recon_x, x, mu, logvar):
    BCE = F.binary_cross_entropy(recon_x, x.view(-1, 784), reduction='sum')
    KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())

    return BCE + KLD
```

As introduced above, there are two terms for optimization which are BCE and KLD in the loss function defined above. Actually the bernoulli distribution might be a little inappropriate here because the inputs in MNIST daataset are not binary. However, since most values in the input are 0 or 1, we still think of them as approximately binary. Then the binary cross entropy between the inputs and the reconstructed bernoulli probability $y$ is indeed the first term in equation $4$.

$$
\begin{align}
logp(x|z) &= log(x^y(1-x)^{1-y})  \\
&= ylog(x) + (1-y)log(1-x) \\
\end{align}
$$

Then let's look into the second term in equation $4$ which is the KL divergence between $q(z|x)$ and $p(z)$. Since both of them are Gaussian distributions, we assume $q \sim \mathcal{N}(\mu_1, \sigma_1)$ and $p \sim \mathcal{N}(\mu_2,\sigma_2)$. Then it could be written as (The derivation between two gaussian distributions are inducted later, here we just show the conclusion):
$$
\begin{align}D_{KL}(q_\phi(z|x)\|p(z)) & = -\frac{1}{2} + log(\frac{\sigma_2}{\sigma_1}) + \frac{\sigma_1^2 + (\mu_1-\mu_2)^2}{2\sigma_2^2}  \\&= -\frac{1}{2}(1 + log\sigma_1 - \sigma_1^2-\mu_1^2) \qquad where \ \mu_2 = 0, \sigma_2 = 1
\end{align}
$$

Then, we can see that they match exactly with BCE and KLD in the loss function. The derivation for KL divergence between two gaussian distributions is shown below. You may skip it if you just want to know the basic idea of VAE.

$$
p \sim \mathcal{N}(\mu_1, \sigma_1), q \sim \mathcal{N}(\mu_2, \sigma_2) \\\begin{align}D_{KL}(p\|q) & = -\int p(x)log(q(x))dx + \int p(x) log(p(x))dx  \\&= -\int p(x) log\frac{1}{(2\pi\sigma_2^2)^{1/2}}e^{-\frac{(x-\mu_2)^2}{2\sigma_2^2}}dx + \int p(x)log\frac{1}{(2\pi\sigma_1^2)^{1/2}}e^{-\frac{(x-\mu_1)^2}{2\sigma_1^2}}dx  \\&= \frac{1}{2}log(2\pi\sigma_2^2) + \int p(x)\frac{(x-\mu_2)^2}{2\sigma_2^2}dx - \frac{1}{2} log(2\pi\sigma_1^2) -\int p(x)\frac{(x-\mu_1)^2}{2\sigma_1^2}dx \\&= log\frac{\sigma_2}{\sigma_1} + \frac{1}{2\sigma_2^2} \big[\int p(x)x^2dx - \int p(x)2x\mu_2dx + \int p(x)\mu_2^2dx \big] - \frac{1}{2}  \\&= log\frac{\sigma_2}{\sigma_1}  + \frac{\sigma_1^2 + \mu_1^2 -2\mu_1\mu_2 + \mu_2^2}{2\sigma_2^2} - \frac{1}{2}
\end{align}
$$

```python
# Then we create the model and optimizer
model = VAE().to(device)
optimizer = optim.Adam(model.parameters(), lr=1e-3)
```

```python
# Define train and test function
def train(epoch):
    model.train()
    train_loss = 0
    for batch_idx, (data, _) in enumerate(train_loader):
        data = data.to(device)
        optimizer.zero_grad()
        recon_batch, mu, logvar = model(data)
        loss = loss_function(recon_batch, data, mu, logvar)
        loss.backward()
        train_loss += loss.item()
        optimizer.step()

    print('====> Epoch: {} Average loss: {:.4f}'.format(
          epoch, train_loss / len(train_loader.dataset)))

def test(epoch):
    model.eval()
    test_loss = 0
    with torch.no_grad():
        for i, (data, _) in enumerate(test_loader):
            data = data.to(device)
            recon_batch, mu, logvar = model(data)
            test_loss += loss_function(recon_batch, data, mu, logvar).item()
            if i == 0:
                n = min(data.size(0), 8)
                comparison = torch.cat([data[:n],
                                      recon_batch.view(args.batch_size, 1, 28, 28)[:n]])
                save_image(comparison.cpu(),
                         'VAE_result/reconstruction_' + str(epoch) + '.png', nrow=n)

    test_loss /= len(test_loader.dataset)
    print('====> Test set loss: {:.4f}'.format(test_loss))

```

```python
# Do the trainning and see results
for epoch in range(1, args.epochs + 1):
    train(epoch)
    test(epoch)
    with torch.no_grad():
        sample = torch.randn(64, 20).to(device)
        sample = model.decode(sample).cpu()
        save_image(sample.view(64, 1, 28, 28),
                   'VAE_result/sample_' + str(epoch) + '.png')

```

Then we can see the reconstruction and sampling results in VAE_result directory!

```python

```

Hands on Generative Models/Hands on VAE.ipynb


# Quiz 1
# Machine Learning 2018-1

A logistic regression model is a statistical classification method that uses a generalized linear regression model to estimate $P(C=1 | \mathbf{x})$, the probability of the sample $\mathbf{x}\in\mathbb{R}^2$ belonging to class $C_1$.

\begin{equation}
y=P(C=1|\mathbf{x},\mathbf{w})=\sigma(w_0+w_1x_0 + w_2x_1)
\end{equation}
where
\begin{equation}
\sigma(x)=\frac{1}{1+e^{-x}}
\end{equation}

### 1.
Write a function that implements a logistic regression model.

```python
import numpy as np

def ro(x):
    return  1 / (1 + np.exp(-x))

def f_1(w, x):
    '''
     w: weight vector with shape (3,)
     x: input sample with shape (1,2)
     returns: P(C=1|x,w)
    '''
    ### Your code here

    y = ro(w[0] + w[1]*x[0] + w[2]*x[1])

    return  y

print(f_1(np.array([0, 1, 2]), np.array([1, 2])))

lst = []
for x1 in np.linspace(-2, 1.5, 4):
    for x2 in np.linspace(-2, 1.5, 4):
        lst.append([x1, x2])
X = np.array(lst)

print(X[0, :])
```

    0.9933071490757153
    [-2. -2.]

```python

```

### 2.
Assume that the cost of a false positive (predicting class $C_1$ when the real class is $C_0$) is $L_0$ and the cost of a false negative is $L_1$. Write a function that calculates the risk of classifying a sample $\mathbf{x}$ in class $y \in \{0,1\}$.

```python
def f_2(w, L, x, y):
    '''
     w: weight vector with shape (3,)
     L: loss vector with shape (2,)
     x: input sample with shape (2,1)
     y: class value {0, 1}
     returns: R(y|x,w)
    '''
    ### Your code here

    p = f_1(w, x)

    if y == 0:
        R =  L[1]*p
    else:
        R =  L[0]*(1-p)

    return  R
W1 = np.array([0, 1, 2])
L1 = np.array([0.9, 2, 0.3])
X = np.array([-2, -2])
y = 0

f_2(W1, L1, X, y)
```

    0.004945246313269549

### 3.
Write a function that implements a classifier that makes the prediction that minimizes the risk.

```python
def f_3(w, L, x):
    '''
     w: weight vector with shape (3,)
     L: loss vector with shape (2,)
     x: input sample with shape (2,1)
     returns: predicted class {0, 1}
    '''
    ### Your code here

    r0 = f_2(w, L, x, 0)

    r1 = f_2(w, L, x, 1)

    # Choose the minimum risk

    if r0 <= r1 :
        return 0
    else:
        return 1

    return  0
```

### 4.
Write a function that implements a classifier that makes the prediction that minimizes the risk, but that can also reject the sample. The cost of rejection is $L_2$.

```python
def f_4(w, L, x):
    '''
     w: weight vector with shape (3,)
     L: loss vector with shape (3,)
     x: input sample with shape (2,1)
     returns: predicted class {0, 1, 2}. Rejection is 2.
    '''
    ### Your code here

    rejectRisk = L[2]

    risk0 = f_2(w, L, x, 0)

    risk1 = f_2(w, L, x, 1)

    r = [risk0, risk1, rejectRisk]

    # Choose the minimum risk

    if np.argmin(r) == 0 : # risk of 0
        return 0
    elif np.argmin(r) == 1: # risk of 1
        return 1
    elif np.argmin(r) == 2: # risk of reject
        return 2

    return -1

# Test the function
C2= [ 0.,  0.,  2.,  1.,  0.,  0.,  2.,  1.,  0.,  2.,  1.,  1.,  0.,  2.,  1.,  1.]

for i in range(len(lst)):
    print(C2[i], "=", f_4(W1, L1, X[i, :]))
```

    0.0 = 0
    0.0 = 0
    2.0 = 2
    1.0 = 1
    0.0 = 0
    0.0 = 0
    2.0 = 2
    1.0 = 1
    0.0 = 0
    2.0 = 2
    1.0 = 1
    1.0 = 1
    0.0 = 0
    2.0 = 2
    1.0 = 1
    1.0 = 1

### Grader

Run the following cell to grade your quiz.

```python
def compare(val1, val2, error):
    if abs(val1 - val2) > error:
        return False
    return True

lst = []
for x1 in np.linspace(-2, 1.5, 4):
    for x2 in np.linspace(-2, 1.5, 4):
        lst.append([x1, x2])
X = np.array(lst)

W1 = np.array([0, 1, 2])
L1 = np.array([0.9, 2, 0.3])
W2 = np.array([-0.3, 1, -0.5])
Y1= [ 0.00247262,  0.02492443, 0.20860853,  0.73105858,  0.00789708,  0.07585818,
  0.45842952,  0.89721598,  0.02492443,  0.20860853,  0.73105858,  0.9655548,
  0.07585818,  0.45842952,  0.89721598,  0.98901306]
R10= [ 0.00494525,  0.04984885,  0.41721705,  1.46211716,  0.01579417,  0.15171636,
  0.91685903,  1.79443195,  0.04984885,  0.41721705,  1.46211716,  1.93110961,
  0.15171636,  0.91685903,  1.79443195,  1.97802611]
R11= [ 0.89777464,  0.87756802,  0.71225233,  0.24204728,  0.89289262,  0.83172764,
  0.48741343,  0.09250562,  0.87756802,  0.71225233,  0.24204728,  0.03100068,
  0.83172764,  0.48741343,  0.09250562,  0.00988825]
C1= [ 0.,  0.,  0.,  1.,  0.,  0.,  1.,  1.,  0.,  0.,  1.,  1.,  0.,  1.,  1.,  1.]
C2= [ 0.,  0.,  2.,  1.,  0.,  0.,  2.,  1.,  0.,  2.,  1.,  1.,  0.,  2.,  1.,  1.]
Y2= [ 0.21416502,  0.13200647,  0.07822826,  0.04521747,  0.46671596,  0.32812743,
  0.21416502,  0.13200647,  0.73756162,  0.61063923,  0.46671596,  0.32812743,
  0.90024951,  0.83433491,  0.73756162,  0.61063923]

def test1():
    for i in range(len(lst)):
        if not compare(Y1[i], f_1(W1, X[i, :]), 0.0001):
            return False
        if not compare(Y2[i], f_1(W2, X[i, :]), 0.0001):
            return False
    return True

def test2():
    for i in range(len(lst)):
        if not compare(R10[i], f_2(W1, L1, X[i, :], 0), 0.0001):
            return False
        if not compare(R11[i], f_2(W1, L1, X[i, :], 1), 0.0001):
            return False
    return True

def test3():
    for i in range(len(lst)):
        if not compare(C1[i], f_3(W1, L1[:2], X[i, :]), 0.0001):
            return False
    return True

def test4():
    for i in range(len(lst)):
        if not compare(C2[i], f_4(W1, L1, X[i, :]), 0.0001):
            return False
    return True

def evaluate():
    score = 0
    for test in [test1, test2, test3, test4]:
        if test():
            score += 1
    return score

print ("Score: ", evaluate(), "/ 4")
```

    Score:  4 / 4

```python

```


This notebook loads data from a ship roll decay test simulation and tries to predict the parameters in a matematical model describing the dynamics of this system. If you are not familiar with the ship roll decay test, you can check out this
[Video](https://youtu.be/gN123b8gM-o).

```python
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import matplotlib.pyplot as plt
import sympy as sp
import sympy.physics.mechanics as me
from sympy.physics.vector.printing import vlatex
from IPython.display import Math
import statsmodels.api as sm
from sklearn.metrics import r2_score
from scipy.integrate import solve_ivp

# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using "Save & Run All"
# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session
```

# Loading data from the simulation

```python
data = pd.read_csv('/kaggle/input/kvlcc2-roll-decay/fnpf_kvlcc2_rolldecay_0kn.csv', index_col=0)
```

This file contains time series for roll angle: $\phi$ (**phi**) and its first and second time derivatives $\dot{\phi}$ (**phi1d**) and $\ddot{\phi}$ (**phi2d**).

```python
data.head()
```

The ship is forces into an initial roll angle at the start of the test. When the ship is released it starts an oscillation motion that is decaying over time, due to the hydrodynamic roll damping of the ship.

```python
fig,ax=plt.subplots()
fig.set_size_inches(20,5)
data.plot(y='phi', ax=ax);
ax.set_ylabel(r'Roll angle $\phi$ $[rad]$')
ax.grid(True)
```

The ship is now behaving very much like the classical spring-mass-damper system in dynamics:

This system can be described by the following differential equation:

```python
phi = me.dynamicsymbols('phi')  # Roll angle
phi_dot = phi.diff()
phi_dot_dot = phi_dot.diff()

A_44 = sp.symbols('A_44')  # description='Total mass moment of inertia', unit='kg*m**2'
B_44 = sp.Function('B_{44}')(phi_dot)
C_44 = sp.Function('C_{44}')(phi)

lhs = A_44*phi_dot_dot + B_44 + C_44
rhs = 0
roll_equation_himeno = sp.Eq(lhs=lhs, rhs=rhs)
Math(vlatex(roll_equation_himeno))
```

where $A_{44}$ is the total roll intertia (representing the mass $m$), $B_{44}(\dot{\phi})$ is a function for the roll damping ($c$ in the figure above) and $C_{44}(\phi)$ is the stiffness (represented by the spring force $k$ in the figure). The $A_{44}=1$ is assumed to give this equation one unique solution. The equation is now ritten as:

```python
roll_equation_normalized = roll_equation_himeno.subs(A_44,1)
Math(vlatex(roll_equation_normalized))
```

## Stiffness model
The following polynomial will be used for the stiffness model:

```python
C_1, C_3, C_5 = sp.symbols('C_1 C_3 C_5')
restoring_equation_cubic = sp.Eq(C_44, C_1 * phi + C_3 * phi ** 3 + C_5 * phi ** 5)
restoring_equation_cubic
```

## Damping model
The following polynomial will be used for the damping model:

```python
B_1, B_2, B_3 = sp.symbols('B_1 B_2 B_3')
b44_cubic_equation = sp.Eq(B_44, B_1 * phi_dot + B_2 * phi_dot * sp.Abs(phi_dot) + B_3 * phi_dot ** 3)
Math(vlatex(b44_cubic_equation))
```

The model can now be rewritten with these polynomials:

```python
roll_decay_equation_cubic = roll_equation_normalized.subs([
    (C_44, restoring_equation_cubic.rhs),
    (B_44, b44_cubic_equation.rhs),

])
Math(vlatex(roll_decay_equation_cubic))
```

```python
def calculate_features(data):
    phi=data['phi']
    phi1d=data['phi1d']
    data_features = -np.array([phi1d, np.abs(phi1d)*phi1d, phi1d**3, phi, phi**3, phi**5]).T
    return data_features

def generate_features_and_label(data):

    data_features = calculate_features(data=data)

    columns = ['B_1','B_2','B_3','C_1','C_3','C_5']
    X = pd.DataFrame(data=data_features, index=data.index, columns=columns)

    y = data['phi2d']

    return X,y

X,y = generate_features_and_label(data=data)
```

# Regression
Regression to estimate the parameters $B_1$, $B_2$,..., from the time series.

## Linear regression

```python
model = sm.OLS(y,X)
results = model.fit()
```

```python
results.summary()
```

```python
y_pred = results.predict(X)
```

## Simpler model

```python
def generate_features_and_label_simple(data):

    phi=data['phi']
    phi1d=data['phi1d']
    phi2d=data['phi2d']

    X = -np.array([phi1d, phi, phi**3, phi**5]).T
    y = phi2d.values

    return X,y

X2,y2 = generate_features_and_label_simple(data=data)
```

```python
model2 = sm.OLS(y2,X2)
results2 = model2.fit()
```

```python
results2.summary()
```

```python
y_pred2 = results2.predict(X2)
```

```python
fig,ax=plt.subplots()
fig.set_size_inches(20,5)
data.plot(y='phi2d', ax=ax, label=r'$\ddot{\phi}$');

ax.plot(data.index, y_pred, '--', label=r'$\ddot{\phi}_{pred}$')
ax.plot(data.index, y_pred2, ':', label=r'$\ddot{\phi}_{pred2}$')

ax.set_ylabel(r'Roll angle acceleration $\ddot{\phi}$ $[rad/s^2]$')

ax.grid(True)
ax.legend();
```

## Simulate

```python
def updater(t, states):
    phi = states[0]
    phi1d = states[1]

    s = {
        'phi':phi,
        'phi1d':phi1d
    }
    x = calculate_features(data=s)
    phi2d = results.predict(x)[0]

    d_states_dt = np.array([phi1d, phi2d])
    return d_states_dt

y0 = [data.iloc[0]['phi'],
      data.iloc[0]['phi1d'],
     ]
t = data.index
t_span = [t[0], t[-1]]
result = solve_ivp(fun=updater, t_span=t_span, y0=y0, t_eval=t)
df_result = pd.DataFrame(index=result.t, data=result.y.T, columns = ['phi','phi1d'])
```

```python
fig,ax=plt.subplots()
fig.set_size_inches(20,5)
data.plot(y='phi', ax=ax, label=r'$\phi$');

df_result.plot(y='phi', style='--', label=r'$\phi_{sim}$', ax=ax)

ax.set_ylabel(r'Roll angle $\phi$ $[rad]$')

ax.grid(True)
ax.legend();

```

```python
r2_score(y_true=data['phi'], y_pred=df_result['phi'])
```

```python
error = data['phi']-df_result['phi']
error.hist(bins=100);
```