### Checklist for submission

It is extremely important to make sure that:

1. Everything runs as expected (no bugs when running cells);
2. The output from each cell corresponds to its code (don't change any cell's contents without rerunning it afterwards);
3. All outputs are present (don't delete any of the outputs);
4. Fill in all the places that say `# YOUR CODE HERE`, or "**Your answer:** (fill in here)".
5. Never copy/paste any notebook cells. Inserting new cells is allowed, but it should not be necessary.
6. The notebook contains some hidden metadata which is important during our grading process. **Make sure not to corrupt any of this metadata!** The metadata may for example be corrupted if you copy/paste any notebook cells, or if you perform an unsuccessful git merge / git pull. It may also be pruned completely if using Google Colab, so watch out for this. Searching for "nbgrader" when opening the notebook in a text editor should take you to the important metadata entries.
7. Although we will try our very best to avoid this, it may happen that bugs are found after an assignment is released, and that we will push an updated version of the assignment to GitHub. If this happens, it is important that you update to the new version, while making sure the notebook metadata is properly updated as well. The safest way to make sure nothing gets messed up is to start from scratch on a clean updated version of the notebook, copy/pasting your code from the cells of the previous version into the cells of the new version.
8. If you need to have multiple parallel versions of this notebook, make sure not to move them to another directory.
9. Although not forced to work exclusively in the course `conda` environment, you need to make sure that the notebook will run in that environment, i.e. that you have not added any additional dependencies.

**FOR HA1, HA2, HA3 ONLY:** Failing to meet any of these requirements might lead to either a subtraction of POEs (at best) or a request for resubmission (at worst).

We advise you to perform the following steps before submission to ensure that requirements 1, 2, and 3 are always met: **Restart the kernel** (in the menubar, select Kernel$\rightarrow$Restart) and then **run all cells** (in the menubar, select Cell$\rightarrow$Run All). This might require a bit of time, so plan ahead for this (and possibly use Google Cloud's GPU in HA1 and HA2 for this step). Finally press the "Save and Checkout" button before handing in, to make sure that all your changes are saved to this .ipynb file.

### Fill in name of notebook file
This might seem silly, but the version check below needs to know the filename of the current notebook, which is not trivial to find out programmatically.

You might want to have several parallel versions of the notebook, and it is fine to rename the notebook as long as it stays in the same directory. **However**, if you do rename it, you also need to update its own filename below:

```python
nb_fname = "IHA1.ipynb"
```

### Fill in group number and member names (use NAME2 and GROUP only for HA1, HA2 and HA3):

```python
NAME1 = "Aladdin Persson"
NAME2 = ""
GROUP = ""
```

### Check Python version

```python
from platform import python_version_tuple
assert python_version_tuple()[:2] == ('3','7'), "You are not running Python 3.7. Make sure to run Python through the course Conda environment."
```

### Check that notebook server has access to all required resources, and that notebook has not moved

```python
import os
nb_dirname = os.path.abspath('')
assignment_name = os.path.basename(nb_dirname)
assert assignment_name in ['IHA1', 'IHA2', 'HA1', 'HA2', 'HA3'], \
    '[ERROR] The notebook appears to have been moved from its original directory'
```

### Verify correct nb_fname

```python
from IPython.display import display, HTML
try:
    display(HTML(r''.format(nb_fname=nb_fname)))
except NameError:
    assert False, 'Make sure to fill in the nb_fname variable above!'
```

### Verify that your notebook is up-to-date and not corrupted in any way

```python
import sys
sys.path.append('..')
from ha_utils import check_notebook_uptodate_and_not_corrupted
check_notebook_uptodate_and_not_corrupted(nb_dirname, nb_fname)
```

    Matching current notebook against the following URL:
    http://raw.githubusercontent.com/JulianoLagana/deep-machine-learning/master/home-assignments/IHA1/IHA1.ipynb
    [SUCCESS] No major notebook mismatch found when comparing to latest GitHub version. (There might be minor updates, but even that is the case, submitting your work based on this notebook version would be acceptable.)

# IHA1 - Assignment

Welcome to the first individual home assignment!

This assignment consists of two parts:
 * Python and NumPy exercises;
 * Build a deep neural network for forward propagation.

The focus of this assignment is for you to gain practical knowledge with implementing forward propagation of deep neural networks without using any deep learning framework. You will also gain practical knowledge in two of Python's scientific libraries [NumPy](https://docs.scipy.org/doc/numpy-1.13.0/index.html) and [Matplotlib](https://matplotlib.org/devdocs/index.html).

Skeleton code is provided for most tasks and every part you are expected to implement is marked with **TODO**. Throughout the assignment you will also need to submit written answers to some questions. These questions are mainly to make you reflect on some particular topics and your answers will not be graded in detail.

We expect you to search and learn by yourself any commands you think are useful for these tasks. Don't limit yourself to what was taught in CL1. Use the help function, [stackoverflow](https://stackoverflow.com/), google, the [python documentation](https://docs.python.org/3.5/library/index.html) and the [NumPy](https://docs.scipy.org/doc/numpy-1.13.0/index.html) documentation to your advantage.

**IMPORTANT NOTE**: The tests available are not exhaustive, meaning that if you pass a test you have avoided the most common mistakes, but it is still not guaranteed that your solution is 100% correct.

Lets start by importing the necessary libraries below.

```python
import numpy as np
import matplotlib.pyplot as plt
from utils.tests.iha1Tests import *
```

## 1. Lists and arrays introduction
First, we will warm up with a Python exercise and few NumPy exercises.

### 1.1 List comprehensions
Examine the code snippet provided below.

```python
myList = []
for i in range(25):
    if i % 2 == 0:
        myList.append(i**2)

print(myList)
```

    [0, 4, 16, 36, 64, 100, 144, 196, 256, 324, 400, 484, 576]

This is not a very "[pythonic](http://docs.python-guide.org/en/latest/writing/style/)" way of writing. Lets re-write the code above using a [list comprehension](https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions). The result will be less code, more readable and elegant. Your solution should be able to fit into one line of code.

```python
myList = [i**2 for i in range(25) if i % 2 == 0] # TODO
# YOUR CODE HERE
print(myList)
```

    [0, 4, 16, 36, 64, 100, 144, 196, 256, 324, 400, 484, 576]

```python
# sample output from cell above for reference
# [0, 4, 16, 36, 64, 100, 144, 196, 256, 324, 400, 484, 576]
```

### 1.2 Numpy array vs numpy vectors
Run the cell below to create a numpy array.

```python
myArr = np.array([1, 9, 25, 49, 81, 121, 169, 225, 289, 361, 441, 529])
print(myArr)
print(myArr.shape)
```

    [  1   9  25  49  81 121 169 225 289 361 441 529]
    (12,)

One of the core features of numpy is to efficiently perform linear algebra operations.
There are two types of one-dimensional representations in numpy: arrays of shape (x,) and vectors of shape (x,1).

The above result indicates that **myArr** is an array of 12 elements with shape (12,).

Numpy's arrays and vectors both have the type of `numpy.ndarray` but have in some cases different characteristics and it is important to separate the two types because it will save a lot of debugging time later on. Read more about numpy shapes [here](https://stackoverflow.com/a/22074424).

Run the code below to see how the transpose operation behaves differently between an array and vector.

```python
# print the shape of an array and the shape of a transposed array
print('myArr is an array of shape:')
print(myArr.shape)
print('The transpose of myArr has the shape:')
print(myArr.T.shape)

# print the shape of a vector and the transpose of a vector
myVec = myArr.reshape(12,1)
print('myVec is a vector of shape:')
print(myVec.shape)
print('The transpose of myVec has the shape:')
print(myVec.T.shape)
```

    myArr is an array of shape:
    (12,)
    The transpose of myArr has the shape:
    (12,)
    myVec is a vector of shape:
    (12, 1)
    The transpose of myVec has the shape:
    (1, 12)

### 1.3 Numpy exercises
Now run the cell below to create the numpy array `numbers` and then complete the exercises sequentially.

```python
numbers = np.arange(24)
print(numbers)
```

    [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]

```python
# TODO: reshape numbers into a 6x4 matrix
numbers = numbers.reshape(6, 4)
# YOUR CODE HERE
print(numbers)

```

    [[ 0  1  2  3]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22 23]]

```python
# sample output from cell above for reference
#  [[ 0  1  2  3]
#   [ 4  5  6  7]
#   [ 8  9 10 11]
#   [12 13 14 15]
#   [16 17 18 19]
#   [20 21 22 23]]
```

```python
# test case
test_numpy_reshape(numbers)
```

    Test passed

```python
# TODO: set the element of the last row of the last column to zero
# Hint: Try what happends when indices are negative
numbers[5,3] = 0
# YOUR CODE HERE
print(numbers)
```

    [[ 0  1  2  3]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22  0]]

```python
# sample output from cell above for reference
#  [[ 0  1  2  3]
#   [ 4  5  6  7]
#   [ 8  9 10 11]
#   [12 13 14 15]
#   [16 17 18 19]
#   [20 21 22  0]]
```

```python
# test case
test_numpy_neg_ix(numbers)
```

    Test passed

```python
# TODO: set every element of the 0th row to 0
numbers[0, :] = 0
# YOUR CODE HERE
print(numbers)
```

    [[ 0  0  0  0]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22  0]]

```python
# sample output from cell above for reference
#  [[ 0  0  0  0]
#   [ 4  5  6  7]
#   [ 8  9 10 11]
#   [12 13 14 15]
#   [16 17 18 19]
#   [20 21 22  0]]
```

```python
# test case
test_numpy_row_ix(numbers)
```

    Test passed

```python
# TODO: append a 1x4 row vector of zeros to `numbers`,
# resulting in a 7x4 matrix where the new row of zeros is the last row
# Hint: A new matrix must be created in the procedure. Numpy arrays are not dynamic.
print(numbers)
print(numbers.shape)
numbers = np.append(numbers, [[0,0,0,0]], axis=0)
# YOUR CODE HERE
print(numbers)
print(numbers.shape)
```

    [[ 0  0  0  0]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22  0]]
    (6, 4)
    [[ 0  0  0  0]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22  0]
     [ 0  0  0  0]]
    (7, 4)

```python
# sample output from cell above for reference
#   [[ 0  0  0  0]
#    [ 4  5  6  7]
#    [ 8  9 10 11]
#    [12 13 14 15]
#    [16 17 18 19]
#    [20 21 22  0]
#    [ 0  0  0  0]]
#  (7, 4)
```

```python
# test case
test_numpy_append_row(numbers)
```

    Test passed

```python
# TODO: set all elements above 10 to the value 1
numbers[numbers>10] = 1
# YOUR CODE HERE
print(numbers)
```

    [[ 0  0  0  0]
     [ 4  5  6  7]
     [ 8  9 10  1]
     [ 1  1  1  1]
     [ 1  1  1  1]
     [ 1  1  1  0]
     [ 0  0  0  0]]

```python
# sample output from cell above for reference
#  [[ 0  0  0  0]
#   [ 4  5  6  7]
#   [ 8  9 10  1]
#   [ 1  1  1  1]
#   [ 1  1  1  1]
#   [ 1  1  1  0]
#   [ 0  0  0  0]]
```

```python
# test case
test_numpy_bool_matrix(numbers)
```

    Test passed

```python
# TODO: compute the sum of every row and replace `numbers` with the answer
# `numbers` will be a (7,) array as a result
numbers = numbers.sum(axis=1)
# YOUR CODE HERE
print(numbers.shape)
print(numbers)
```

    (7,)
    [ 0 22 28  4  4  3  0]

```python
# sample output from cell above for reference
#   (7,)
#   [ 0 22 28  4  4  3  0]
```

```python
# test case
test_numpy_sum(numbers)
```

    Test passed

## 2 Building your deep neural network
It is time to start implementing your first feed-forward neural network. In this lab you will only focus on implementing the forward propagation procedure.

As you know, a feed-forward neural network consists of a sequence of layers, where the output from one layer is the input to the following layer. Below you can find an example of a feed-forward neural network with 2 hidden layers (blue) and 1 output layer (red).

**What would be your approach to implement a feed-forward neural network? Please provide a brief and high level explanation on how would you implement the several components of network; for instance, would you use any `python` class or only functions?** We will not grade this answer in detail. The objective with this question is to help you reflect upon how to structure the code. This will hopefully enable you to understand the code below more easily. As long as it is clear from your answer that you have tried to briefly describe the overall structure of the code, we will accept your answer.

**Your answer:**  I would create a class for the neural network and create internal functions for each of the parts of the neural network, which would for example be to initialize weights, one for forward propogation, backward propogation, and a main function with the specifics of number of update steps/epochs to run it.

In this exercise, you will implement several components of a feed-forward network, such as a layers, activation functions and weight initialization functions, and from those create your own feed-forward neural network implementation.

NOTE: When using a neural network, due to memory and computational constraints, you can not forward propagate the entire dataset at once. Therefore, you divide the dataset into a number of sets/parts called batches. A batch will make up for the first dimension of every input to a layer and the notation `(BATCH_SIZE, NUM_FEATURES)` simply means the dimension of a batch of samples where every sample has `NUM_FEATURES` features.

### 2.1 Activation functions
You will start by defining a few activation functions that are later needed by the neural network.

#### 2.1.1 ReLU
The neural network will use the ReLU activation function in every layer except for the last. ReLU does element-wise comparison of the input matrix. For example, if the input is `X`, and `X[i,j] == 2` and `X[k,l] == -1`, then after applying ReLU, `X[i,j] == 2` and `X[k,l] == 0` should be true.

The formula for implementing ReLU for a single neuron $i$ is:
\begin{equation}
relu(z_i) =
    \begin{cases}
      0, & \text{if}\ z_i \leq 0 \\
      z_i, & \text{otherwise}
    \end{cases}
\end{equation}

Now implement `relu` in vectorized form

```python
def relu(z):
    """ Implement the ReLU activation function

    Arguments:
    z - the input of the activation function. Has a type of 'numpy.ndarray'

    Returns:
    a - the output of the activation function. Has a type of numpy.ndarray and the same shape as 'z'
    """
    a = np.maximum(0, z) # TODO
    # YOUR CODE HERE

    return a
```

```python
# test case
test_relu(relu)
```

    Test passed

Can you name some of the advantages of the ReLU activation function? (We will not grade this answer in detail.)

**Your answer:** It seems to me that the gradients of it are incredibly simple, either 0 if it didn't allow input to be passed or 1 if it did. This probably has computational benefits which I can imagine is quite beneficial as NNs usually train a long time. Comparing to other like sigmoid or tanh the gradient is stable in the way that it's always 1 which is not the case for sigmoid for example. For large values of z the sigmoid will return a gradient close to 0, which perhaps is not intuitive because it means it'll be difficult to update.

#### 2.1.2 Sigmoid
The sigmoid activation function is common for binary classification. This is because it squashes its input to the range [0,1].
Implement the activation function `sigmoid` using the formula:
\begin{equation}
    \sigma(z) = \frac{1}{1 + e^{-z}}
\end{equation}

```python
def sigmoid(z):
    """ Implement the sigmoid activation function

    Arguments:
    z - the input of the activation function. Has a type of 'numpy.ndarray'

    Returns:
    a - the output of the activation function. Has a type of 'numpy.ndarray' and the same shape as 'z'
    """

    a = 1/(1 + np.exp(-z)) # TODO
    # YOUR CODE HERE

    return a
```

```python
# test case
test_sigmoid(sigmoid)
```

    Test passed

Why is the sigmoid activation function useful for binary classification? Feel free to use some numerical examples to show how the magnitudes of the variable $z$ affect the output of the sigmoid layer. (We will not grade this answer in detail.)

**Your answer:** Because it outputs values between (0,1) it is easy to use a threshold of say 0.5 and set everything above to 1 and otherwise, and hence it makes sense to use this for binary classification.

#### 2.1.3 Visualization
Make a plot using matplotlib to visualize the activation functions between the input interval [-3,3]. The plot should have the following properties
 * one plot should contain a visualization of both `ReLU` and `sigmoid`;
 * x-axis: range of values between [-3,3], **hint**: np.linspace;
 * y-axis: the value of the activation functions at a given input `x`;
 * a legend explaining which line represents which activation function.

```python
# TODO: make a plot of ReLU and sigmoid values in the interval [-3,3]
x = np.arange(-3, 3, 0.01)
y1 = relu(x)
y2 = sigmoid(x)
plt.plot(x,y1, label="ReLU")
plt.xlabel("x")
plt.ylabel("y")

plt.plot(x,y2, label="sigmoid")
plt.xlabel("x")
plt.ylabel("y")
plt.title("PLot of ReLU and Sigmoid between [-3, 3]")

plt.legend()
# YOUR CODE HERE
```

#### 2.1.4 Softmax
You will use the softmax activation function / classifier as the final layer of your neural network later in the assignment. Implement `softmax` according to the formula below. The subtraction of the maximum value is there solely to avoid overflows in a practical implementation.
\begin{equation}
softmax(z_i) = \frac{e^{z_i - max(\mathbf{z})}}{ \sum^j e^{z_j - max(\mathbf{z})}}
\end{equation}

```python
def softmax(z):
    """ Implement the softmax activation function

    Arguments:
    z - the input of the activation function, shape (BATCH_SIZE, FEATURES) and type 'numpy.ndarray'

    Returns:
    a - the output of the activation function, shape (BATCH_SIZE, FEATURES) and type 'numpy.ndarray'
    """
    max_z = np.max(z, axis=1, keepdims=True)
    z = z - max_z
    z = np.exp(z)
    a = z / z.sum(axis=1, keepdims=True)
    # YOUR CODE HERE

    return a
```

```python
# test case
test_softmax(softmax)
```

    Test passed

What are the main differences between using `sigmoid` and `softmax` for multi-class classification problems? And when the number of classes is 2? (We will not grade this answer in detail.)

**Your answer:** Softmax will make the output nodes actually correspond to probabilities which will make it easy to interpret (sum will be 1) which is not the case for sigmoid. When googling I found that sigmoid is simply softmax when the classes=2 but I couldn't rederive that by my own.

### 2.2 Initialize weights
You will implement a helper function that takes the shape of a layer as input, and returns an initialized weight matrix $\mathbf{W}$ and bias vector $\mathbf{b}$ as output. The matrix $\mathbf{W}$ should be sampled from a normal distribution with mean 0 and standard deviation 2, and $\mathbf{b}$ should be initialized to all zeros.

```python
def initialize_weights(layer_shape):
    """ Implement initialization of the weight matrix and biases

    Arguments:
    layer_shape - a tuple of length 2, type (int, int), that determines the dimensions of the weight matrix: (input_dim, output_dim)

    Returns:
    w - a weight matrix with dimensions of 'layer_shape', (input_dim, output_dim), that is normally distributed with
        properties mu = 0, stddev = 2. Has a type of 'numpy.ndarray'
    b - a vector of initialized biases with shape (1,output_dim), all of value zero. Has a type of 'numpy.ndarray'
    """
    w = np.random.normal(loc=0, scale=2, size=layer_shape) # TODO
    b = np.zeros((1, layer_shape[1])) # TODO
    # YOUR CODE HERE

    return w, b
```

```python
# test case
test_initialize_weights(initialize_weights)
```

    Test passed

### 2.3 Feed-forward neural network layer module
To build a feed-forward neural network of arbitrary depth you are going to define a neural network layer as a module that can be used to stack layers upon each other. Your task is to complete the `Layer` class by following the descriptions in the comments. Recall the formula for forward propagation of an arbitrary layer $l$:

\begin{equation}
\mathbf{a}^{[l]} = g(\mathbf{z}^{[l]}) = g(\mathbf{a}^{[l-1]}\mathbf{w}^{[l]} +\mathbf{b}^{[l]})
\end{equation}

where $g$ is the activation function given by `activation_fn`, which can be relu, sigmoid or softmax.

```python
class Layer:
    """
    TODO: Build a class called Layer that satisfies the descriptions of the methods
    Make sure to utilize the helper functions you implemented before
    """

    def __init__(self, input_dim, output_dim, activation_fn=relu):
        """
        Arguments:
        input_dim - the number of inputs of the layer. type int
        output_dim - the number of outputs of the layer. type int
        activation_fn - a reference to the activation function to use. Should be 'relu' as a default
                        possible values are the 'relu', 'sigmoid' and 'softmax' functions you implemented earlier.
                        Has the type 'function'

        Attributes:
        w - the weight matrix of the layer, should be initialized with 'initialize_weights'
            and has the shape (INPUT_FEATURES, OUTPUT_FEATURES) and type 'numpy.ndarray'
        b - the bias vector of the layer, should be initialized with 'initialize_weights'
            and has the shape (1, OUTPUT_FEATURES) and type 'numpy.ndarray'
        activation_fn - a reference to the activation function to use.
                        Has the type 'function'
        """
        self.w, self.b = initialize_weights((input_dim, output_dim)) # TODO
        self.activation_fn = activation_fn # TODO
        # YOUR CODE HERE

    def forward_prop(self, a_prev):
        """ Implement the forward propagation module of the neural network layer
        Should use whatever activation function that 'activation_fn' references to

        Arguments:
        a_prev - the input to the layer, which may be the data 'X', or the output from the previous layer.
            a_prev has the shape of (BATCH_SIZE, INPUT_FEATURES) and the type 'numpy.ndarray'

        Returns:
        a - the output of the layer when performing forward propagation. Has the type 'numpy.ndarray'
        """
        a = self.activation_fn(np.dot(a_prev, self.w) + self.b) # TODO
        # YOUR CODE HERE

        return a
```

```python
# test case, be sure that you pass the previous activation function tests before running this test
test_layer(Layer, relu, sigmoid, softmax)
```

    Test passed

### 2.4 Backward propagation

In the previous exercise you only implemented the forward propagation of the layer. However, when training a network, you would also need to implement the backward propagation of the layer in order to be able to update its weights. (You won't need to do this in this assignment since you will not train the network. Instead you will load the network weights, which are obtained from a previously trained network.)

Provide a high-level explanation of how you would modify your current implementation of the `Layer` class in order to add the backward propagation. (We will not grade this in detail. It is enough that your answer demonstrates that you have thought about how to perform backprop using these classes.)

**Your answer:** You would need to first cache the values of the current weights and the values of the activation values. And the idea is then to start at the end of the graph with the loss and you want to compute the gradients of each weight with respect to the loss, since the gradient will the direction of steepest descent we will then update with:

weight = weight - learning_rate * grad(weight)

And this should minimize the loss for the next time it sees the same example.

### 2.5 Example - Logistic regression
Binary logistic regression is a classifier where classification is performed by applying the sigmoid activation function to a linear combination of input values. You will now try out your neural network layer by utilizing it as a linear combination of input values and apply the sigmoid activation function to classify a simple problem.

The cell below defines a dataset of 5 points of either class `0` or class `1`. Your assignment is to:
1. Create an instance of a `Layer` with the sigmoid activation function;
2. Manually tune the weights `w` and bias `b` of your layer.

You can use `test_logistic` to visually inspect how your classifier is performing. In order to  be successful in this exercise you need select values for the weights `w` and bias `b` such that all points are classified correctly.

```python
# Run this cell to create the dataset
X_s = np.array([[1, 2],
               [5, 3],
               [8, 8],
               [7, 5],
               [3, 6]])
Y_s = np.array([0,0,1,0,1])

test_logistic(X_s, Y_s)
```

```python
# create an instance of layer
l = Layer(2,1,sigmoid)

# TODO: manually tune weights
l.w = np.array([-0.5,1])
l.b = np.array([-2])
# YOUR CODE HERE

# testing your choice of weights with this function
test_logistic(X_s,Y_s,l,sigmoid)
```

### 2.6 Feed-forward neural network
You will now define the actual neural network class. It is an L-layer neural network, meaning that the number of layers and neurons in each layer is specified as input by the user. Once again, you will only focus on implementing the forward propagation part.

Read the descriptions in the comments and complete the **TODO**s.

```python
class NeuralNetwork:
    """
    TODO: Implement an L-layer neural network class by utilizing the Layer module defined above
    Each layer should use 'relu' activation function, except for the output layer, which should use 'softmax'
    """

    def __init__(self, input_n, layer_dims):
        """
        Arguments:
        input_n    - the number of inputs to the network. Should be the same as the length of a data sample
                     Has type int
        layer_dims - a python list or tuple of the number of neurons in each layer. Layer 'l' should have a weight matrix
                     with the shape ('layer_dims[l-1]', 'layer_dims[l]').
                     'layer_dims[-1]' is the dimension of the output layer.
                     Layer 1 should have the dimensions ('input_n', 'layer_dims[0]').
                     len(layer_dims) is the depth of the neural network
        Attributes:
        input_n - the number of inputs to the network. Has type int
        layers  - a python list of each layer in the network. Each layer should use the 'relu' activation function,
                  except for the last layer, which should use 'softmax'.
                  Has type 'list' containing layers of type 'Layer'
        """

        self.input_n = input_n # TODO
        self.layers = [] # TODO

        prev_nodes = self.input_n
        for idx, layer in enumerate(layer_dims):
            if idx != len(layer_dims)-1:
                self.layers.append(Layer(prev_nodes, layer, activation_fn=relu))
            else:
                self.layers.append(Layer(prev_nodes, layer, activation_fn=softmax))

            prev_nodes = layer

        # YOUR CODE HERE

    def forward_prop(self, x):
        """
        Implement the forward propagation procedure through the entire network, from input to output.
        You will now connect each layer's forward propagation function into a chain of layer-wise forward propagations.

        Arguments:
        x - the input data, which has the shape (BATCH_SIZE, NUM_FEATURES) and type 'numpy.ndarray'

        Returns:
        a - the output of the last layer after forward propagating through the every layer in 'layers'.
            Should have the dimension (BATCH_SIZE, layers[-1].w.shape[1]) and type 'numpy.ndarray'
        """
        for l in self.layers:
            a = l.forward_prop(x)
            x = a

        # YOUR CODE HERE
        return a
```

```python
# test case
test_neuralnetwork(NeuralNetwork)
```

    Test passed

## 3 Making predictions with a neural network
In practice, its common to load weights to your neural network that has already been trained.
In this section, you will create an instance of your neural network, load trained weights from disk, and perform predictions.

### 3.1 Load weights from disk
Create an instance of `NeuralNetwork` with input size $28 \times 28 = 784$, two hidden layers of size 100 and an output layer of size 10. Thereafter, load the weights contained in `./utils/ann_weights.npz` to your network.

```python
ann = NeuralNetwork(784, layer_dims=[100, 100, 10]) # TODO: create instance of ann
# YOUR CODE HERE

# load weights
weights = np.load('./utils/ann_weights.npz')
for l in range(len(ann.layers)):
    ann.layers[l].w = weights['w' + str(l)]
    ann.layers[l].b = weights['b' + str(l)]
```

### 3.2 Prediction
Now, implement the function `predict_and_correct` which does the following:
1. Load `./utils/test_data.npz` from disk;
2. Extract test data `X` and `Y` from file;
3. Perform for every pair of data:
    a. plot the image `x`; <br>
    b. make a prediction using your neural network by forward propagating and picking the most probable class;
    c. check whether the prediction is correct (compare with the ground truth number `y`);
    d. print the predicted label and wether it was correct or not.

```python
def predict_and_correct(ann):
    """ Load test data from file and predict using your neural network.
    Make a prediction for ever data sample and print it along with wether it was a correct prediction or not

    Arguments:
    ann - the neural network to use for prediction. Has type `NeuralNetwork`

    Returns: # for test case purposes
    A `numpy.ndarray` of predicted classes (integers [0-9]) with shape (11,)
    """
    data = np.load('./utils/test_data.npz')
    X, cls = data['X'], data['Y']

    cls_preds = ann.forward_prop(X) # TODO: make a predicted number for every image in X
    cls_preds = cls_preds.argmax(1)
    # YOUR CODE HERE

    for i in range(len(X)):
        plt.imshow(X[i].reshape(28,28), cmap='gray')
        plt.show()
        correct = cls_preds[i] == cls[i]
        print('The prediction was {0}, it was {1}!'.format(cls_preds[i], 'correct' if correct else 'incorrect'))

    return cls_preds

cls_pred = predict_and_correct(ann)
```

```python
# final test case
test_predict_and_correct_answer(cls_pred)
```

    Test passed, you have implemented your neural network correctly!

## Congratulations!
You have successfully implemented a neural network from scratch using only NumPy!


Table of Contents

1  Backtesing

```python
%tensorflow_version 2.x
!git clone https://github.com/tensortrade-org/tensortrade.git
!pip install -U ta
```

    TensorFlow 2.x selected.
    Cloning into 'tensortrade'...
    remote: Enumerating objects: 287, done.
    remote: Counting objects: 100% (287/287), done.
    remote: Compressing objects: 100% (201/201), done.
    remote: Total 7670 (delta 188), reused 171 (delta 85), pack-reused 7383
    Receiving objects: 100% (7670/7670), 53.96 MiB | 20.97 MiB/s, done.
    Resolving deltas: 100% (5115/5115), done.
    Collecting ta
      Downloading https://files.pythonhosted.org/packages/a1/69/602e636dd2ed3bc15fcae514c5918049a8e02490d660f601f464c91985e1/ta-0.5.15.tar.gz
    Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from ta) (1.18.2)
    Requirement already satisfied, skipping upgrade: pandas in /usr/local/lib/python3.6/dist-packages (from ta) (0.25.3)
    Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->ta) (2018.9)
    Requirement already satisfied, skipping upgrade: python-dateutil>=2.6.1 in /usr/local/lib/python3.6/dist-packages (from pandas->ta) (2.8.1)
    Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.6.1->pandas->ta) (1.12.0)
    Building wheels for collected packages: ta
      Building wheel for ta (setup.py) ... done
      Created wheel for ta: filename=ta-0.5.15-cp36-none-any.whl size=23439 sha256=1a825dfd41b8282243041c3fd15cbee7bfc5d824dfa6d7df4c8437b835793e88
      Stored in directory: /root/.cache/pip/wheels/53/b3/ec/93bd3d04147adcf06fab4ba9926b1150c987d8ffa053374709
    Successfully built ta
    Installing collected packages: ta
    Successfully installed ta-0.5.15

```python
cd tensortrade
```

    /content/tensortrade

```python
!pip install -r requirements.txt
```

    Requirement already satisfied: numpy>=1.16.4 in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 1)) (1.18.2)
    Requirement already satisfied: pandas>=0.25.0 in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 2)) (0.25.3)
    Requirement already satisfied: gym>=0.14.0 in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 3)) (0.17.1)
    Collecting pyyaml>=5.1.2
      Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz (269kB)
         |████████████████████████████████| 276kB 2.8MB/s
    Collecting sympy>=1.4
      Downloading https://files.pythonhosted.org/packages/ce/5b/acc12e3c0d0be685601fc2b2d20ed18dc0bf461380e763afc9d0a548deb0/sympy-1.5.1-py2.py3-none-any.whl (5.6MB)
         |████████████████████████████████| 5.6MB 9.1MB/s
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.25.0->-r requirements.txt (line 2)) (2018.9)
    Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.25.0->-r requirements.txt (line 2)) (2.8.1)
    Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from gym>=0.14.0->-r requirements.txt (line 3)) (1.5.0)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from gym>=0.14.0->-r requirements.txt (line 3)) (1.12.0)
    Requirement already satisfied: cloudpickle<1.4.0,>=1.2.0 in /usr/local/lib/python3.6/dist-packages (from gym>=0.14.0->-r requirements.txt (line 3)) (1.3.0)
    Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from gym>=0.14.0->-r requirements.txt (line 3)) (1.4.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.4->-r requirements.txt (line 5)) (1.1.0)
    Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from pyglet<=1.5.0,>=1.4.0->gym>=0.14.0->-r requirements.txt (line 3)) (0.16.0)
    Building wheels for collected packages: pyyaml
      Building wheel for pyyaml (setup.py) ... done
      Created wheel for pyyaml: filename=PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl size=44621 sha256=48c62b8ff4a26e3dad8fa0e7fa549206625c3f5c2778c6b1bb0c3421d9bfcc69
      Stored in directory: /root/.cache/pip/wheels/a7/c1/ea/cf5bd31012e735dc1dfea3131a2d5eae7978b251083d6247bd
    Successfully built pyyaml
    Installing collected packages: pyyaml, sympy
      Found existing installation: PyYAML 3.13
        Uninstalling PyYAML-3.13:
          Successfully uninstalled PyYAML-3.13
      Found existing installation: sympy 1.1.1
        Uninstalling sympy-1.1.1:
          Successfully uninstalled sympy-1.1.1
    Successfully installed pyyaml-5.3.1 sympy-1.5.1

```python
!mkdir /content/agents
!mkdir /content/data
```

```python
import pandas as pd
import numpy as np
%matplotlib inline

import ta
from tensortrade.data import DataFeed, Module
from tensortrade.exchanges import Exchange
from tensortrade.exchanges.services.execution.simulated import execute_order
from tensortrade.data import Stream, DataFeed, Module
from tensortrade.instruments import USD, BTC, ETH, LTC
from tensortrade.wallets import Wallet, Portfolio
from tensortrade.environments import TradingEnvironment

from tensortrade.agents import DQNAgent
from tensortrade.agents import A2CAgent
```

```python
import pandas as pd

def load_csv(filename):
    df = pd.read_csv(filename)
    # Convert the date column type from string to datetime for proper sorting.
    df['date'] = pd.to_datetime(df['date'])

    # Make sure historical prices are sorted chronologically, oldest first.
    df.sort_values(by='date', ascending=True, inplace=True)
    df.reset_index(drop=True, inplace=True)

    # Format timestamps as you want them to appear on the chart buy/sell marks.
    df['date'] = df['date'].dt.strftime('%Y-%m-%d')

    # PlotlyTradingChart expects 'datetime' as the timestamps column name.
    df.rename(columns={'date': 'datetime'}, inplace=True)

    return df

df = load_csv('/content/data/hz_data.csv')
df.tail()
```

      datetime
      open
      high
      low
      close
      volume
      y2_hat

      587
      2019-10-19
      7950.48
      8096.76
      7875.00
      7960.49
      3461.125292
      0.003603

      588
      2019-10-20
      7965.39
      8314.77
      7874.68
      8235.74
      6358.924991
      0.003452

      589
      2019-10-21
      8236.49
      8352.00
      8156.00
      8213.65
      4919.734168
      0.000227

      590
      2019-10-22
      8206.53
      8314.77
      7989.15
      8025.90
      5678.934831
      -0.000325

      591
      2019-10-23
      8021.57
      8052.04
      7920.40
      7995.47
      1588.417045
      0.000585

```python
import ta
from tensortrade.data import DataFeed, Module

dataset = ta.add_all_ta_features(df, 'open', 'high', 'low', 'close', 'volume', fillna=True)
dataset.head(3)
```

```python
price_history = dataset[['datetime', 'open', 'high', 'low', 'close', 'volume']]  # chart data
display(price_history.head(3))

dataset.drop(columns=['datetime', 'open', 'high', 'low', 'close', 'volume'], inplace=True)
```

      datetime
      open
      high
      low
      close
      volume

      0
      2018-03-11
      8795.04
      9768.37
      8450.00
      9535.04
      16407.54238

      1
      2018-03-12
      9535.04
      9892.00
      8742.07
      9120.75
      18751.02058

      2
      2018-03-13
      9115.24
      9482.79
      8830.00
      9142.32
      16001.47535

```python
from tensortrade.exchanges import Exchange
from tensortrade.exchanges.services.execution.simulated import execute_order
from tensortrade.data import Stream, DataFeed, Module
from tensortrade.instruments import USD, BTC, ETH, LTC
from tensortrade.wallets import Wallet, Portfolio
from tensortrade.environments import TradingEnvironment

coinbase = Exchange("coinbase", service=execute_order)(
    Stream("USD-BTC", price_history['close'].tolist())
)

portfolio = Portfolio(USD, [
    Wallet(coinbase, 10000 * USD),
    Wallet(coinbase, 10 * BTC),
])

with Module("coinbase") as coinbase_ns:
    nodes = [Stream(name, dataset[name].tolist()) for name in dataset.columns]

feed = DataFeed([coinbase_ns])
feed.next()
```

    {'coinbase:/momentum_ao': 0.0,
     'coinbase:/momentum_kama': 9535.04,
     'coinbase:/momentum_mfi': 50.0,
     'coinbase:/momentum_roc': 0.0,
     'coinbase:/momentum_rsi': 100.0,
     'coinbase:/momentum_stoch': 82.30163004315938,
     'coinbase:/momentum_stoch_signal': 82.30163004315938,
     'coinbase:/momentum_tsi': 0.0,
     'coinbase:/momentum_uo': 50.0,
     'coinbase:/momentum_wr': -17.69836995684063,
     'coinbase:/others_cr': 0.0,
     'coinbase:/others_dlr': 0.0,
     'coinbase:/others_dr': 37.03698680179694,
     'coinbase:/trend_adx': 0.0,
     'coinbase:/trend_adx_neg': 0.0,
     'coinbase:/trend_adx_pos': 0.0,
     'coinbase:/trend_aroon_down': 4.0,
     'coinbase:/trend_aroon_ind': 0.0,
     'coinbase:/trend_aroon_up': 4.0,
     'coinbase:/trend_cci': 0.0,
     'coinbase:/trend_dpo': -2577.035287162161,
     'coinbase:/trend_ema_fast': 9535.04,
     'coinbase:/trend_ema_slow': 9535.04,
     'coinbase:/trend_ichimoku_a': 9109.185000000001,
     'coinbase:/trend_ichimoku_b': 9109.185000000001,
     'coinbase:/trend_kst': 370.3698680179695,
     'coinbase:/trend_kst_diff': 0.0,
     'coinbase:/trend_kst_sig': 370.3698680179695,
     'coinbase:/trend_macd': 0.0,
     'coinbase:/trend_macd_diff': 0.0,
     'coinbase:/trend_macd_signal': 0.0,
     'coinbase:/trend_mass_index': 1.0,
     'coinbase:/trend_psar': 9535.04,
     'coinbase:/trend_psar_down': -1.0,
     'coinbase:/trend_psar_down_indicator': 0.0,
     'coinbase:/trend_psar_up': -1.0,
     'coinbase:/trend_psar_up_indicator': 0.0,
     'coinbase:/trend_trix': 36.24669700257624,
     'coinbase:/trend_visual_ichimoku_a': 6998.578475506762,
     'coinbase:/trend_visual_ichimoku_b': 7113.944231418929,
     'coinbase:/trend_vortex_ind_diff': 0.0,
     'coinbase:/trend_vortex_ind_neg': 1.0,
     'coinbase:/trend_vortex_ind_pos': 1.0,
     'coinbase:/volatility_atr': 0.0,
     'coinbase:/volatility_bbh': 9535.04,
     'coinbase:/volatility_bbhi': 0.0,
     'coinbase:/volatility_bbl': 9535.04,
     'coinbase:/volatility_bbli': 0.0,
     'coinbase:/volatility_bbm': 9535.04,
     'coinbase:/volatility_bbp': 0.0,
     'coinbase:/volatility_bbw': 0.0,
     'coinbase:/volatility_dch': 9535.04,
     'coinbase:/volatility_dchi': 1.0,
     'coinbase:/volatility_dcl': 9535.04,
     'coinbase:/volatility_dcli': 1.0,
     'coinbase:/volatility_kcc': 9251.136666666667,
     'coinbase:/volatility_kch': 10569.506666666668,
     'coinbase:/volatility_kchi': 0.0,
     'coinbase:/volatility_kcl': 7932.766666666666,
     'coinbase:/volatility_kcli': 0.0,
     'coinbase:/volatility_kcp': 0.6076721001438645,
     'coinbase:/volatility_kcw': 28.501794914570873,
     'coinbase:/volume_adi': 10599.807277524375,
     'coinbase:/volume_cmf': 0.6460326008631875,
     'coinbase:/volume_em': 0.0,
     'coinbase:/volume_fi': 0.0,
     'coinbase:/volume_nvi': 1000.0,
     'coinbase:/volume_obv': 16407.542380000003,
     'coinbase:/volume_sma_em': 0.0,
     'coinbase:/volume_vpt': 6080.713723685023,
     'coinbase:/y2_hat': -0.002261962}

```python
env = TradingEnvironment(
    feed=feed,
    portfolio=portfolio,
    action_scheme='managed-risk',
    reward_scheme='risk-adjusted',
    window_size=20
)
```

    /usr/local/lib/python3.6/dist-packages/gym/logger.py:30: UserWarning:

    WARN: Box bound precision lowered by casting to float32

```python
from tensortrade.agents import DQNAgent
agent = DQNAgent(env)
agent.train(n_episodes=10, n_steps=200, save_path="../agents/")
```

    ====      AGENT ID: 0d76af7d-78d9-432a-b3b1-c430b56229e0      ====
    [2020-03-21 10:11:42 PM] Episode: 1/10 Step: 150/200
    [2020-03-21 10:11:49 PM] Episode: 1/10 Step: 200/200
    [2020-03-21 10:11:56 PM] Episode: 2/10 Step: 50/200
    [2020-03-21 10:12:03 PM] Episode: 2/10 Step: 100/200
    [2020-03-21 10:12:10 PM] Episode: 2/10 Step: 150/200
    [2020-03-21 10:12:16 PM] Episode: 2/10 Step: 200/200
    [2020-03-21 10:12:23 PM] Episode: 3/10 Step: 50/200
    [2020-03-21 10:12:30 PM] Episode: 3/10 Step: 100/200
    [2020-03-21 10:12:37 PM] Episode: 3/10 Step: 150/200
    [2020-03-21 10:12:44 PM] Episode: 3/10 Step: 200/200
    [2020-03-21 10:12:51 PM] Episode: 4/10 Step: 50/200
    [2020-03-21 10:12:58 PM] Episode: 4/10 Step: 100/200
    [2020-03-21 10:13:04 PM] Episode: 4/10 Step: 150/200
    [2020-03-21 10:13:11 PM] Episode: 4/10 Step: 200/200
    [2020-03-21 10:13:18 PM] Episode: 5/10 Step: 50/200
    [2020-03-21 10:13:25 PM] Episode: 5/10 Step: 100/200
    [2020-03-21 10:13:32 PM] Episode: 5/10 Step: 150/200
    [2020-03-21 10:13:38 PM] Episode: 5/10 Step: 200/200
    [2020-03-21 10:13:45 PM] Episode: 6/10 Step: 50/200
    [2020-03-21 10:13:52 PM] Episode: 6/10 Step: 100/200
    [2020-03-21 10:13:59 PM] Episode: 6/10 Step: 150/200
    [2020-03-21 10:14:06 PM] Episode: 6/10 Step: 200/200
    [2020-03-21 10:14:13 PM] Episode: 7/10 Step: 50/200
    [2020-03-21 10:14:20 PM] Episode: 7/10 Step: 100/200
    [2020-03-21 10:14:27 PM] Episode: 7/10 Step: 150/200
    [2020-03-21 10:14:34 PM] Episode: 7/10 Step: 200/200
    [2020-03-21 10:14:40 PM] Episode: 8/10 Step: 50/200
    [2020-03-21 10:14:47 PM] Episode: 8/10 Step: 100/200
    [2020-03-21 10:14:54 PM] Episode: 8/10 Step: 150/200
    [2020-03-21 10:15:01 PM] Episode: 8/10 Step: 200/200
    [2020-03-21 10:15:08 PM] Episode: 9/10 Step: 50/200
    [2020-03-21 10:15:15 PM] Episode: 9/10 Step: 100/200
    [2020-03-21 10:15:22 PM] Episode: 9/10 Step: 150/200
    [2020-03-21 10:15:29 PM] Episode: 9/10 Step: 200/200
    [2020-03-21 10:15:36 PM] Episode: 10/10 Step: 50/200
    [2020-03-21 10:15:43 PM] Episode: 10/10 Step: 100/200
    [2020-03-21 10:15:49 PM] Episode: 10/10 Step: 150/200
    [2020-03-21 10:15:56 PM] Episode: 10/10 Step: 200/200

    911000.4083567932

```python
# from tensortrade.agents import A2CAgent
# agent = A2CAgent(env)
# agent.train(n_episodes=4, n_steps=200, save_path="../agents/")
```

```python
%matplotlib inline
print(portfolio.performance.net_worth)
portfolio.performance.net_worth.plot(figsize=(16,10))
```

## Backtesing

```python
df1 = load_csv("/content/data/BTCUSD_1d.csv")
df1 = df1[1500:-600].reset_index(drop = True)
df1.tail()
```

      datetime
      open
      high
      low
      close
      volume

      858
      2018-02-26
      9595.99
      10461.97
      9376.34
      10324.70
      16156.805056

      859
      2018-02-27
      10329.99
      10850.00
      10150.00
      10566.30
      10408.479938

      860
      2018-02-28
      10572.49
      11064.75
      10255.07
      10314.90
      10490.749058

      861
      2018-03-01
      10314.99
      11090.00
      10223.41
      10903.13
      9481.665146

      862
      2018-03-02
      10917.37
      11175.00
      10774.01
      11029.99
      8329.097901

```python
dataset1 = ta.add_all_ta_features(df1, 'open', 'high', 'low', 'close', 'volume', fillna=True)
dataset1.head(3)
```

    /usr/local/lib/python3.6/dist-packages/ta/trend.py:543: RuntimeWarning:

    invalid value encountered in double_scalars

    /usr/local/lib/python3.6/dist-packages/ta/trend.py:547: RuntimeWarning:

    invalid value encountered in double_scalars

      datetime
      open
      high
      low
      close
      volume
      volume_adi
      volume_obv
      volume_cmf
      volume_fi
      momentum_mfi
      volume_em
      volume_sma_em
      volume_vpt
      volume_nvi
      volatility_atr
      volatility_bbm
      volatility_bbh
      volatility_bbl
      volatility_bbw
      volatility_bbp
      volatility_bbhi
      volatility_bbli
      volatility_kcc
      volatility_kch
      volatility_kcl
      volatility_kcw
      volatility_kcp
      volatility_kchi
      volatility_kcli
      volatility_dcl
      volatility_dch
      volatility_dchi
      volatility_dcli
      trend_macd
      trend_macd_signal
      trend_macd_diff
      trend_ema_fast
      trend_ema_slow
      trend_adx
      trend_adx_pos
      trend_adx_neg
      trend_vortex_ind_pos
      trend_vortex_ind_neg
      trend_vortex_ind_diff
      trend_trix
      trend_mass_index
      trend_cci
      trend_dpo
      trend_kst
      trend_kst_sig
      trend_kst_diff
      trend_ichimoku_a
      trend_ichimoku_b
      trend_visual_ichimoku_a
      trend_visual_ichimoku_b
      trend_aroon_up
      trend_aroon_down
      trend_aroon_ind
      trend_psar
      trend_psar_up
      trend_psar_down
      trend_psar_up_indicator
      trend_psar_down_indicator
      momentum_rsi
      momentum_tsi
      momentum_uo
      momentum_stoch
      momentum_stoch_signal
      momentum_wr
      momentum_ao
      momentum_kama
      momentum_roc
      others_dr
      others_dlr
      others_cr

      0
      2015-10-22
      267.90
      280.04
      267.39
      275.21
      30166.425443
      7130.246014
      30166.425443
      0.236364
      0.000000
      50.0
      0.000000
      0.000000
      -27137.171080
      1000.000000
      0.0
      275.210000
      275.210000
      275.210000
      0.000000
      0.000000
      0.0
      0.0
      274.213333
      286.863333
      261.563333
      9.226393
      0.539394
      0.0
      0.0
      275.21
      275.21
      1.0
      1.0
      0.000000
      0.000000
      0.000000
      275.210000
      275.210000
      0.0
      0.0
      0.0
      1.0
      1.0
      0.0
      -89.022715
      1.000000
      0.000000
      2470.627543
      -899.771929
      -899.771929
      0.000000
      273.715
      273.715
      2619.953082
      2524.904009
      4.0
      4.0
      0.0
      275.21
      -1.0
      -1.00
      0.0
      0.0
      100.0
      0.0
      50.0
      61.818182
      61.818182
      -38.181818
      0.0
      275.21
      0.0
      -89.977193
      0.000000
      0.000000

      1
      2015-10-23
      275.21
      280.19
      273.46
      278.60
      21572.268470
      18509.377228
      51738.693913
      0.357747
      73129.990115
      50.0
      97024.103092
      97024.103092
      -26877.178494
      1012.317866
      0.0
      276.905000
      280.295000
      273.515000
      2.448493
      0.750000
      0.0
      0.0
      275.815000
      285.505000
      266.125000
      7.026449
      0.643705
      0.0
      0.0
      275.21
      278.60
      1.0
      0.0
      0.270427
      0.054085
      0.216342
      275.731538
      275.461111
      0.0
      0.0
      0.0
      1.0
      1.0
      0.0
      0.002406
      1.923694
      66.666667
      2468.932543
      -899.154631
      -899.463280
      0.308649
      273.790
      273.790
      2619.953082
      2524.904009
      8.0
      4.0
      4.0
      278.60
      -1.0
      -1.00
      0.0
      0.0
      100.0
      100.0
      50.0
      87.578125
      74.698153
      -12.421875
      0.0
      278.60
      0.0
      1.231787
      1.224262
      1.231787

      2
      2015-10-24
      278.60
      284.00
      278.22
      282.84
      20586.335018
      30832.685041
      72325.028931
      0.426307
      75152.285881
      50.0
      120309.418743
      108666.760917
      579.026763
      1027.724283
      0.0
      278.883333
      285.126075
      272.640592
      4.476956
      0.816901
      0.0
      0.0
      277.772222
      286.158889
      269.385556
      6.038521
      0.802133
      0.0
      0.0
      275.21
      282.84
      1.0
      0.0
      0.817452
      0.206759
      0.610694
      276.825148
      276.007695
      0.0
      0.0
      0.0
      1.0
      1.0
      0.0
      0.009324
      2.784692
      100.000000
      2466.954210
      -898.434147
      -899.120236
      0.686089
      275.695
      275.695
      2619.953082
      2524.904009
      12.0
      4.0
      8.0
      280.04
      -1.0
      280.04
      0.0
      1.0
      100.0
      100.0
      50.0
      93.016255
      80.804187
      -6.983745
      0.0
      282.84
      0.0
      1.521895
      1.510431
      2.772428

```python
price_history1 = dataset1[['datetime', 'open', 'high', 'low', 'close', 'volume']]  # chart data
display(price_history1.head(3))

dataset1.drop(columns=['datetime', 'open', 'high', 'low', 'close', 'volume'], inplace=True)
dataset1['y2_hat']=np.nan
```

      datetime
      open
      high
      low
      close
      volume

      0
      2015-10-22
      267.90
      280.04
      267.39
      275.21
      30166.425443

      1
      2015-10-23
      275.21
      280.19
      273.46
      278.60
      21572.268470

      2
      2015-10-24
      278.60
      284.00
      278.22
      282.84
      20586.335018

```python
coinbase = Exchange("coinbase", service=execute_order)(
    Stream("USD-BTC", price_history1['close'].tolist())
)

portfolio = Portfolio(USD, [
    Wallet(coinbase, 10000 * USD),
    Wallet(coinbase, 10 * BTC),
])

with Module("coinbase") as coinbase_ns:
    nodes = [Stream(name, dataset1[name].tolist()) for name in dataset1.columns]

feed = DataFeed([coinbase_ns])
feed.next()

env = TradingEnvironment(
    feed=feed,
    portfolio=portfolio,
    action_scheme='managed-risk',
    reward_scheme='risk-adjusted',
    window_size=20
)
agent = DQNAgent(env)
agent.restore(path="/content/agents/DQN1_10.hdf5")
```

    /usr/local/lib/python3.6/dist-packages/gym/logger.py:30: UserWarning:

    WARN: Box bound precision lowered by casting to float32

```python
state, reward, done, info = agent.env.step(action)
action = agent.env.action_space.sample()
n_steps = 200
step = 0

while not done and (step < n_steps):
    state, reward, done, info = agent.env.step(action)
    action = agent.get_action(state)
    print('step:', step, '-- action:', action)
    step += 1
```

    step: 0 -- action: 84
    step: 1 -- action: 84
    step: 2 -- action: 84
    step: 3 -- action: 84
    step: 4 -- action: 84
    step: 5 -- action: 84
    step: 6 -- action: 84
    step: 7 -- action: 84
    step: 8 -- action: 84
    step: 9 -- action: 84
    step: 10 -- action: 84
    step: 11 -- action: 84
    step: 12 -- action: 84
    step: 13 -- action: 53
    step: 14 -- action: 84
    step: 15 -- action: 70
    step: 16 -- action: 41
    step: 17 -- action: 45
    step: 18 -- action: 41
    step: 19 -- action: 41
    step: 20 -- action: 41
    step: 21 -- action: 41
    step: 22 -- action: 84
    step: 23 -- action: 20
    step: 24 -- action: 84
    step: 25 -- action: 84
    step: 26 -- action: 84
    step: 27 -- action: 84
    step: 28 -- action: 84
    step: 29 -- action: 84
    step: 30 -- action: 84
    step: 31 -- action: 84
    step: 32 -- action: 53
    step: 33 -- action: 84
    step: 34 -- action: 20
    step: 35 -- action: 53
    step: 36 -- action: 84
    step: 37 -- action: 84
    step: 38 -- action: 84
    step: 39 -- action: 84
    step: 40 -- action: 84
    step: 41 -- action: 84
    step: 42 -- action: 84
    step: 43 -- action: 84
    step: 44 -- action: 84
    step: 45 -- action: 84
    step: 46 -- action: 84
    step: 47 -- action: 84
    step: 48 -- action: 84
    step: 49 -- action: 127
    step: 50 -- action: 84
    step: 51 -- action: 84
    step: 52 -- action: 84
    step: 53 -- action: 45
    step: 54 -- action: 84
    step: 55 -- action: 84
    step: 56 -- action: 53
    step: 57 -- action: 127
    step: 58 -- action: 169
    step: 59 -- action: 41
    step: 60 -- action: 20
    step: 61 -- action: 20
    step: 62 -- action: 20
    step: 63 -- action: 70
    step: 64 -- action: 84
    step: 65 -- action: 84
    step: 66 -- action: 84
    step: 67 -- action: 84
    step: 68 -- action: 84
    step: 69 -- action: 84
    step: 70 -- action: 84
    step: 71 -- action: 84
    step: 72 -- action: 84
    step: 73 -- action: 84
    step: 74 -- action: 20
    step: 75 -- action: 53
    step: 76 -- action: 20
    step: 77 -- action: 84
    step: 78 -- action: 84
    step: 79 -- action: 84
    step: 80 -- action: 84
    step: 81 -- action: 41
    step: 82 -- action: 84
    step: 83 -- action: 84
    step: 84 -- action: 41
    step: 85 -- action: 41
    step: 86 -- action: 84
    step: 87 -- action: 41
    step: 88 -- action: 20
    step: 89 -- action: 84
    step: 90 -- action: 84
    step: 91 -- action: 84
    step: 92 -- action: 84
    step: 93 -- action: 84
    step: 94 -- action: 84
    step: 95 -- action: 20
    step: 96 -- action: 84
    step: 97 -- action: 84
    step: 98 -- action: 84
    step: 99 -- action: 84
    step: 100 -- action: 41
    step: 101 -- action: 127
    step: 102 -- action: 84
    step: 103 -- action: 41
    step: 104 -- action: 84
    step: 105 -- action: 84
    step: 106 -- action: 84
    step: 107 -- action: 84
    step: 108 -- action: 84
    step: 109 -- action: 84
    step: 110 -- action: 84
    step: 111 -- action: 84
    step: 112 -- action: 84
    step: 113 -- action: 84
    step: 114 -- action: 84
    step: 115 -- action: 84
    step: 116 -- action: 84
    step: 117 -- action: 84
    step: 118 -- action: 84
    step: 119 -- action: 84
    step: 120 -- action: 84
    step: 121 -- action: 84
    step: 122 -- action: 84
    step: 123 -- action: 84
    step: 124 -- action: 84
    step: 125 -- action: 84
    step: 126 -- action: 84
    step: 127 -- action: 84
    step: 128 -- action: 20
    step: 129 -- action: 84
    step: 130 -- action: 84
    step: 131 -- action: 84
    step: 132 -- action: 84
    step: 133 -- action: 41
    step: 134 -- action: 41
    step: 135 -- action: 20
    step: 136 -- action: 84
    step: 137 -- action: 84
    step: 138 -- action: 84
    step: 139 -- action: 84
    step: 140 -- action: 127
    step: 141 -- action: 84
    step: 142 -- action: 173
    step: 143 -- action: 169
    step: 144 -- action: 84
    step: 145 -- action: 154
    step: 146 -- action: 84
    step: 147 -- action: 84
    step: 148 -- action: 84
    step: 149 -- action: 53
    step: 150 -- action: 41
    step: 151 -- action: 84
    step: 152 -- action: 41
    step: 153 -- action: 84
    step: 154 -- action: 84
    step: 155 -- action: 84
    step: 156 -- action: 84
    step: 157 -- action: 84
    step: 158 -- action: 127
    step: 159 -- action: 84
    step: 160 -- action: 84
    step: 161 -- action: 84
    step: 162 -- action: 84
    step: 163 -- action: 20
    step: 164 -- action: 84
    step: 165 -- action: 53
    step: 166 -- action: 84
    step: 167 -- action: 84
    step: 168 -- action: 84
    step: 169 -- action: 41
    step: 170 -- action: 84
    step: 171 -- action: 84
    step: 172 -- action: 84
    step: 173 -- action: 84
    step: 174 -- action: 84
    step: 175 -- action: 127
    step: 176 -- action: 84
    step: 177 -- action: 84
    step: 178 -- action: 84
    step: 179 -- action: 41
    step: 180 -- action: 70
    step: 181 -- action: 84
    step: 182 -- action: 84
    step: 183 -- action: 41
    step: 184 -- action: 84
    step: 185 -- action: 41
    step: 186 -- action: 53
    step: 187 -- action: 84
    step: 188 -- action: 84
    step: 189 -- action: 84
    step: 190 -- action: 41
    step: 191 -- action: 41
    step: 192 -- action: 41
    step: 193 -- action: 41
    step: 194 -- action: 41
    step: 195 -- action: 41
    step: 196 -- action: 84
    step: 197 -- action: 84
    step: 198 -- action: 84
    step: 199 -- action: 84

```python
env.portfolio.performance.net_worth.plot(figsize=(16,10))
```

```python

```


# **FLISol BA 2021** - Carlos Bustillo

### 0. Consideraciones

Requiere que los archivos 'sim.py', 'simConst.py', 'remoteapi.dll' estén alojados en la misma carpeta que este cuaderno de Jupyter.
Incluir en el script de Coppelia la línea: simRemoteApi.start(19999)

```python
# Se importa las librerías necesarias
import sim
import sympy as sp
import matplotlib.pyplot as plt # pyplot
import numpy as np
```

### 1. Establecer la conexión

Utilizaremos las funciones del API Remoto de VREP.

Para más detalles refiérase a la documentación de la librería:

http://www.coppeliarobotics.com/helpFiles/en/remoteApiFunctionsPython.htm

```python
def connect(port):
# Establece la conexión a VREP
# port debe coincidir con el puerto de conexión en VREP
# retorna el número de cliente o -1 si no puede establecer conexión
    sim.simxFinish(-1) # Por si acaso, cierre todas las conexiones abiertas.
    clientID=sim.simxStart('127.0.0.1',port,True,True,2000,5) # Conectarse
    if clientID == 0: print("Conectado a", port)
    else: print("No se pudo conectar")
    return clientID
```

```python
# Obtenemos los manejadores para cada una de las articulaciones del Robot
def getHandlers():
    retCode,effector=sim.simxGetObjectHandle(clientID,'effector',sim.simx_opmode_blocking)
    retCode,target=sim.simxGetObjectHandle(clientID,'Cuboid',sim.simx_opmode_blocking)
    retCode,joint1=sim.simxGetObjectHandle(clientID,'MTB_joint1',sim.simx_opmode_blocking)
    retCode,joint2=sim.simxGetObjectHandle(clientID,'MTB_joint2',sim.simx_opmode_blocking)
    retCode,joint3=sim.simxGetObjectHandle(clientID,'MTB_joint3',sim.simx_opmode_blocking)
    retCode,joint4=sim.simxGetObjectHandle(clientID,'MTB_joint4',sim.simx_opmode_blocking)
    joint = [joint1, joint2, joint3, joint4]
    return joint, effector, target
```

### 2. Inicializar

```python
# Obtenemos el manejador del sensor de visión
clientID = connect(19999)
joint, effector, target = getHandlers()
retCode,sensorHandle=sim.simxGetObjectHandle(clientID,'Vision_sensor',sim.simx_opmode_blocking)
```

    Conectado a 19999

### 3. Visualizar Imagen captada

```python
# Obtenemos la imagen
retCode, resolution, image=sim.simxGetVisionSensorImage(clientID,sensorHandle,0,sim.simx_opmode_oneshot_wait)
```

```python
img = np.array(image, dtype=np.uint8)
img.resize([resolution[1],resolution[0],3])
plt.imshow(img)
```

```python

```


```python
!pip install -r ../requirements.txt
```

    Requirement already satisfied: numpy in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from -r ../requirements.txt (line 1)) (1.15.0)
    Requirement already satisfied: scipy in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from -r ../requirements.txt (line 2)) (1.1.0)
    Requirement already satisfied: matplotlib in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from -r ../requirements.txt (line 3)) (2.1.1)
    Requirement already satisfied: Pillow in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from -r ../requirements.txt (line 4)) (4.3.0)
    Requirement already satisfied: ipywidgets>=0.6.0 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from -r ../requirements.txt (line 5)) (7.2.1)
    Requirement already satisfied: requests in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from -r ../requirements.txt (line 6)) (2.14.2)
    Collecting SimPEG (from -r ../requirements.txt (line 7))
      Using cached https://files.pythonhosted.org/packages/4f/b5/042629700a6bec03b3a693c4dcb15b15087668c5ae5f4142ba8d7b8aee8e/SimPEG-0.9.2-py2.py3-none-any.whl
    Collecting em_examples (from -r ../requirements.txt (line 8))
      Using cached https://files.pythonhosted.org/packages/ca/01/3b3c3f7402fc2967811398ab00789bd273afd2cb1ccada0e0774bff54e30/em_examples-0.0.30-py2.py3-none-any.whl
    Collecting empymod (from -r ../requirements.txt (line 9))
      Using cached https://files.pythonhosted.org/packages/0a/24/77231d9af2438d8913b457c58e739a039542623fdb7d2ee02a09cf940442/empymod-1.7.3-py3-none-any.whl
    Collecting deepdish (from -r ../requirements.txt (line 10))
      Using cached https://files.pythonhosted.org/packages/6e/39/2a47c852651982bc5eb39212ac110284dd20126bdc7b49bde401a0139f5d/deepdish-0.3.6-py2.py3-none-any.whl
    Collecting pymatsolver>=0.1.2 (from -r ../requirements.txt (line 11))
      Using cached https://files.pythonhosted.org/packages/0f/5e/30e92dfe738cc3e28f52344484aa2c6e44872787b2edd789df0eaae04e3b/pymatsolver-0.1.2.tar.gz
    Requirement already satisfied: six>=1.10 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from matplotlib->-r ../requirements.txt (line 3)) (1.10.0)
    Requirement already satisfied: python-dateutil>=2.0 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from matplotlib->-r ../requirements.txt (line 3)) (2.5.3)
    Requirement already satisfied: pytz in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from matplotlib->-r ../requirements.txt (line 3)) (2016.6.1)
    Requirement already satisfied: cycler>=0.10 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from matplotlib->-r ../requirements.txt (line 3)) (0.10.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from matplotlib->-r ../requirements.txt (line 3)) (2.1.4)
    Requirement already satisfied: olefile in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from Pillow->-r ../requirements.txt (line 4)) (0.44)
    Requirement already satisfied: ipykernel>=4.5.1 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (4.7.0)
    Requirement already satisfied: traitlets>=4.3.1 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (4.3.1)
    Requirement already satisfied: nbformat>=4.2.0 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (4.2.0)
    Requirement already satisfied: widgetsnbextension~=3.2.0 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (3.2.0)
    Requirement already satisfied: ipython>=4.0.0 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (6.2.1)
    Collecting properties>=0.5.2 (from SimPEG->-r ../requirements.txt (line 7))
      Using cached https://files.pythonhosted.org/packages/61/9c/034bb67472f33d399a789875e98c3fc47c8ad9f6a59efbf23f3d65a6dd1e/properties-0.5.3.tar.gz
    Requirement already satisfied: cython in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from SimPEG->-r ../requirements.txt (line 7)) (0.24.1)
    Collecting vectormath>=0.2.0 (from SimPEG->-r ../requirements.txt (line 7))
      Using cached https://files.pythonhosted.org/packages/c7/44/297304759e6e5076b5fe96cdef1dd7ff6e1c67b1e735a8284ab7f5555712/vectormath-0.2.0.tar.gz
    Collecting discretize>=0.2.0 (from SimPEG->-r ../requirements.txt (line 7))
      Using cached https://files.pythonhosted.org/packages/8b/0c/5419bb36abfd8ddc45c7a63d553126523ce67074073442f5f0dcb05ff694/discretize-0.3.2.tar.gz
    Collecting geoana>=0.0.4 (from SimPEG->-r ../requirements.txt (line 7))
      Using cached https://files.pythonhosted.org/packages/60/34/5715c1b3e35b4b7e0fdef463405c9ae5b073563f48118f2a54165be78cdc/geoana-0.0.5-py2.py3-none-any.whl
    Requirement already satisfied: future in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from em_examples->-r ../requirements.txt (line 8)) (0.16.0)
    Requirement already satisfied: jupyter in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from em_examples->-r ../requirements.txt (line 8)) (1.0.0)
    Requirement already satisfied: tables in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from deepdish->-r ../requirements.txt (line 10)) (3.4.2)
    Collecting pyMKL>=0.0.3 (from pymatsolver>=0.1.2->-r ../requirements.txt (line 11))
      Using cached https://files.pythonhosted.org/packages/43/9d/0657fdb64d31a83c5caab80a70e55dac7658ba3fac7d1f43cbcca726b286/pyMKL-0.0.3.tar.gz
    Requirement already satisfied: jupyter_client in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipykernel>=4.5.1->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (5.1.0)
    Requirement already satisfied: tornado>=4.0 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipykernel>=4.5.1->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (4.4.1)
    Requirement already satisfied: notebook>=4.4.1 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from widgetsnbextension~=3.2.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (5.0.0)
    Requirement already satisfied: setuptools>=18.5 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (40.0.0)
    Requirement already satisfied: jedi>=0.10 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (0.11.0)
    Requirement already satisfied: decorator in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (4.0.10)
    Requirement already satisfied: pickleshare in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (0.7.4)
    Requirement already satisfied: simplegeneric>0.8 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (0.8.1)
    Requirement already satisfied: prompt_toolkit<2.0.0,>=1.0.4 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (1.0.15)
    Requirement already satisfied: pygments in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (2.1.3)
    Requirement already satisfied: pexpect in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (4.0.1)
    Collecting utm (from geoana>=0.0.4->SimPEG->-r ../requirements.txt (line 7))
      Using cached https://files.pythonhosted.org/packages/92/d9/8bdb3d477dcaf75e1a1021772892f729a055787a08aa472582c5d35a0b36/utm-0.4.2.tar.gz
    Requirement already satisfied: numexpr>=2.5.2 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from tables->deepdish->-r ../requirements.txt (line 10)) (2.6.2)
    Requirement already satisfied: jupyter_core in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from jupyter_client->ipykernel>=4.5.1->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (4.4.0)
    Requirement already satisfied: pyzmq>=13 in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from jupyter_client->ipykernel>=4.5.1->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (15.4.0)
    Requirement already satisfied: parso==0.1.* in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from jedi>=0.10->ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (0.1.1)
    Requirement already satisfied: wcwidth in /home/nbuser/anaconda3_420/lib/python3.5/site-packages (from prompt_toolkit<2.0.0,>=1.0.4->ipython>=4.0.0->ipywidgets>=0.6.0->-r ../requirements.txt (line 5)) (0.1.7)
    Building wheels for collected packages: pymatsolver, properties, vectormath, discretize, pyMKL, utm
      Running setup.py bdist_wheel for pymatsolver ... done
      Stored in directory: /home/nbuser/.cache/pip/wheels/09/66/a3/c540b8d29d1edab2bd4fabe368987aedc7fda18e742672be5b
      Running setup.py bdist_wheel for properties ... done
      Stored in directory: /home/nbuser/.cache/pip/wheels/76/46/be/6a1402b7cfa1ca30e7fda543437f5dd615abef8aeb7a3d62c6
      Running setup.py bdist_wheel for vectormath ... done
      Stored in directory: /home/nbuser/.cache/pip/wheels/a2/86/4d/f9890cfc126a83639eb2651f1e66f6d7f7bf0322b237f5a17b
      Running setup.py bdist_wheel for discretize ... done
      Stored in directory: /home/nbuser/.cache/pip/wheels/09/6d/73/3b88a7371a5b8cdcf860f4fbd2c5001ff59054b63b2c37a0cc
      Running setup.py bdist_wheel for pyMKL ... done
      Stored in directory: /home/nbuser/.cache/pip/wheels/d6/40/40/df64f35d394a22fbecae00749cb8e7071798a3554fac9e5abe
      Running setup.py bdist_wheel for utm ... done
      Stored in directory: /home/nbuser/.cache/pip/wheels/1d/2d/5a/4c6320941f607ffefd353e1bb715e33a250a59f835cf0fbdd8
    Successfully built pymatsolver properties vectormath discretize pyMKL utm
    Installing collected packages: properties, vectormath, pyMKL, pymatsolver, discretize, utm, geoana, SimPEG, deepdish, empymod, em-examples
    Successfully installed SimPEG-0.9.2 deepdish-0.3.6 discretize-0.3.2 em-examples-0.0.30 empymod-1.7.3 geoana-0.0.5 properties-0.5.3 pyMKL-0.0.3 pymatsolver-0.1.2 utm-0.4.2 vectormath-0.2.0

```python
from em_examples import DCLayers
from IPython.display import display
%matplotlib inline
```

    /home/nbuser/anaconda3_420/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
      return f(*args, **kwds)
    /home/nbuser/anaconda3_420/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
      return f(*args, **kwds)

```python
from matplotlib import rcParams
rcParams['font.size'] = 14
```

# Purpose

## Investigating DC Resistivity Data

Using the widgets contained in this notebook we will explore the physical principals governing DC resistivity including the behavior of currents, electric field, electric potentials in a two layer earth.

The measured data in a DC experiment are potential differences, we will demonstrate how these  provide information about subsurface physical properties.

### Background: Computing Apparent Resistivity

In practice we cannot measure the potentials everywhere, we are limited to those locations where we place electrodes. For each source (current electrode pair) many potential differences are measured between M and N electrode pairs to characterize the overall distribution of potentials. The widget below allows you to visualize the potentials, electric fields, and current densities from a dipole source in a simple model with 2 layers. For different electrode configurations you can measure the potential differences and see the calculated apparent resistivities.

In a uniform halfspace the potential differences can be computed by summing up the potentials at each measurement point from the different current sources based on the following equations:

\begin{align}
    V_M = \frac{\rho I}{2 \pi} \left[ \frac{1}{AM} - \frac{1}{MB} \right] \\
    V_N = \frac{\rho I}{2 \pi} \left[ \frac{1}{AN} - \frac{1}{NB} \right]
\end{align}
where $AM$, $MB$, $AN$, and $NB$ are the distances between the corresponding electrodes.

The potential difference $\Delta V_{MN}$ in a dipole-dipole survey can therefore be expressed as follows,

\begin{equation}
    \Delta V_{MN} = V_M - V_N = \rho I \underbrace{\frac{1}{2 \pi} \left[ \frac{1}{AM} - \frac{1}{MB} - \frac{1}{AN} + \frac{1}{NB} \right]}_{G}
\end{equation}

and the resistivity of the halfspace $\rho$ is equal to,

$$
    \rho = \frac{\Delta V_{MN}}{IG}
$$

In this equation $G$ is often referred to as the geometric factor.

In the case where we are not in a uniform halfspace the above equation is used to compute the apparent resistivity ($\rho_a$) which is the resistivity of the uniform halfspace which best reproduces the measured potential difference.

In the top plot the location of the A electrode is marked by the red +, the B electrode is marked by the blue -, and the M/N potential electrodes are marked by the black dots. The $V_M$ and $V_N$ potentials are printed just above and to the right of the black dots. The calculted apparent resistivity is shown in the grey box to the right. The bottom plot can show the resistivity model, the electric fields (e), potentials, or current densities (j) depending on which toggle button is selected. Some patience may be required for the plots to update after parameters have been changed.

# LayeredEarth app

## Parameters:

 - **A**: (+) Current electrode  location
 - **B**: (-) Current electrode  location
 - **M**: (+) Potential electrode  location
 - **N**: (-) Potential electrode  location
 - **$\rho_1$**: Resistivity of the first layer
 - **$\rho_2$**: Resistivity of the second layer
 - **h**: Thickness of the first layer
 - **Plot**: Choice of 2D plot (Model, Potential, Electric field, Currents)

```python
out = DCLayers.plot_layer_potentials_app()
display(out)
```

    MyApp(children=(FloatSlider(value=-30.0, continuous_update=False, description='A', max=30.0, min=-30.0, step=1…

```python

```


# Hep Recommender
> A recommender system for scientific articles in the field of High Energy Physics.

- toc: true
- badges: true
- comments: true
- categories: [jupyter]
- image: images/hep_recommender.png

# Introduction

In this note I want to discuss [hep-recommender](https://hep-recommender.herokuapp.com/), a recommender system for scientific papers in the field of High Energy Physics (HEP) aimed to help researchers in the exploration of relevant literature.  This is a personal project I have been working recently in collaboration with [José Eliel Camargo Molina](https://github.com/JoseEliel).    A [submission](https://devpost.com/software/hep-recommender) of this project to the [Facebook Artificial Intelligence Hackathon (2020)](https://fbai1.devpost.com) won the third place.

There are different approaches to determine the degree of similarity between articles in order to identify related work. Some works have used text-mining and natural language processing methods. Another popular approach is based on citation analysis, where the similarity between two articles is estimated based on bibliographic information [1,2,4,5]. Finally, other works have used a combination of the previous approaches, see for instance [3].

In this project, we develop a recommender system based on citation analysis. More specifically, we will develop a recommender system that elaborates on the ideas of co-citation analysis [1,4] and co-citation proximity analysis (CPA) [2]. Co-citation analysis is based on the premise that articles which are frequently cited together (by the same papers) should be related to each other. CPA extends this idea by incorporating the notion that the closer the citations are to each other within the article text, the more likely it is that they are related. While these methods are relatively simple they provide a high quality of related article recommendations.   Our recommender system relies on a distributed representation of articles obtained by training a Skip-Gram model on reference lists.  This model also captures the notion that articles cited close to each other on the text are similar.

In this note I want to go to some detail about the model powering our recommender system, how it compares to other approaches, and how we made it available via a web application.

# Related work

I would like to start discussing some of the standard methods in the literature to quantify the similarity between two articles.  I will define the following concepts:

* bibliometric coupling

* co-citation count

* co-citation proximity analysis (CPA)

Bibliometric couplign reflects the idea that two papers sharing a large portion of their references should be similar [5].  Co-citation count assigns a high degree of similarity to articles which are frequenly cited together (regardless of where the citations occcured within the text) [1].   CPA extends co-citation count by assigning more weigth to cases where the articles are cited close to each other within the text [2].  In general CPA is expected to give better results than co-citation count, though it requires some work in order to parse the references of each article keeping information about the location of the references within the text.

Lets formalize these concepts.  We consider a set of papers $\{  \omega_1, \omega_2, \ldots \omega_n  \}$. We define the quantities $c_{i,j}$, where $c_{i,j}=1$ if $\omega_i$ is cited by $\omega_j$ and $c_{i,j}=0$ otherwise.  Then we are ready to introduce our three measures of similarity:

* bibliographic coupling is defined as $\sum_{k=1}^{n}   c_{k,i}   c_{k,j}$

* co-citation count is defined as $\sum_{k=1}^{n}   c_{i,k}   c_{j,k}$

* CPA can be used to define a measure $\sum_{k=1}^{n} \eta_{i,j}^{k}  c_{i,k}   c_{j,k}$, where $n_{ij}^{k}$ are coefficients that penalize cases where the citations occur far from each other within the text.

In the original CPA article [2], $\eta$ was fixed to $1$ if the citations occur in the same sentence, $1/2$ if they only occur in the same paragraph, $1/4$ if they only occur in the same chapter, and $1/8$ if they only appear on the same article. Parametric representations of $\eta$ have also been proposed [7].

# Distributed representation on co-citations

We propose a model that relies on the same idea behind CPA, namely, that the proximity of references within an article provide valuable information regarding the similarity of two articles.    However, we wanted to simplify the process of extracting the necessary data as much as possible.

We assume that it is possible to extract the list of references of each article in the order of appearance within the text. For instance the following article

> Higgs boson pair production in gluon fusion is the most promising process to find out whether the Higgs boson self-coupling is Standard-Model-like. Early studies of Higgs boson pair production within an EFT framework can be found in Refs. [1–3]. Many phenomenological investigations about the potential of this process to reveal New Physics have been performed since, see e.g. Refs. [4–8]...

would give rise to a reference list [1,2,3,4,5,6,7,8,...].   In this way, articles which are frequently close to each other on these reference lists should be very similar, as the community is citing them close to each other within the text of the articles.

We will use an approach that has proven to be very fruitful in Natural Language Processing (NLP).  We will take the lists of references and train a Skip-Gram model, such that articles which tend to be cited close to each other will have similar embeddings.

Our dataset consists of ordered lists of references.  Suppose we have a total of $n$ unique articles in our dataset and for each of these articles we have its reference list.  The Skip-gram model maximizes the following objective function:

\begin{align}
\frac{1}{n} \sum_{q=1}^{n}    \left[  \frac{1}{n_q}   \sum_{i=1}^{n_q}  \sum_{ -c \leq j  \leq c ,\, j\neq 0  } \log p(w_{i+j}^{q}| w_i^{q}) \right]
\end{align}

Here $\omega_{i}^{q}$ represents the embedding of the $i$-th article appearing on the reference list of article $q$; $n_q$ represents the size of the reference list for article $q$, and $c$ represents the context-window size of the Skip-Gram model.  The probability is modelled using  a softmax function

\begin{align}
p(    \omega_{i}  |    \omega_j  )   = \frac{   \exp(   \omega_{i} \cdot \omega_{j}  )}{     \sum_{l=1}^{n}   \exp(   \omega_{l}  \cdot \omega_j   )   }
\end{align}

Just like in the Skip-Gram model used in NLP, in order to make the model training more efficient we need to implement negative sampling or hierarchical softmax.      A good implementation of the Skip-Gram model is provided by the [Gensim](https://radimrehurek.com/gensim/auto_examples/index.html) library for instance.

In order to explore the structure of the trained embeddings, I took a sample of articles and made a t-SNE visualization of their embeddings in two dimensions:

For this, I picked a set of articles that were published on the [arXiv](https://arxiv.org).  Each arXiv category was displayed with a different color.  On this plot, some of the arXiv categories that appear are: astro-ph (red), hep-ex (blue), hep-th (black) and hep-ph (green), among others.  As expected, the embeddings form cluster around their categories.

# Data

Regarding open acces digital libraries, the research community in the field of High Energy Physics mainly uses the [INSPIRE-HEP](https://inspirehep.net) and the [arXiv](https://arxiv.org). INSPIRE-HEP provides an API from which data for articles in the field of HEP can be retrieved, we use this API to extract the data we need.  Below I show some exploratory example.  Lets first define some classes that talk to the API and handle the response.

```python
import requests
from typing import List, Dict

class InspireAPI:
    """
    Simple wrapper class around the INSPIRE API

    https://inspirehep.net

    methods:
       literature: gives access to the literature endpoint
    """

    LITERATURE = "https://inspirehep.net/api/literature/"

    def __init__(self):
        pass

    def literature(self, record_id: str):
        """
        Returns api response for a given record_id
        """
        url = self.LITERATURE + record_id
        return LiteratureRecord(requests.get(url).json())

class LiteratureRecord:
    """
    Datamodel class for handling literature record data,
    implementing basic methods to access the properties
    """

    def __init__(self, data: Dict):
        self.data = data

    @property
    def record_id(self) -> str:
        """
        Returns the INSPIRE id of the article
        """
        return self.data['id']

    @property
    def metadata(self) -> Dict:
        """
        Returns article metadata
        """
        return self.data['metadata']

    @property
    def references(self) -> List[str]:
        """
        Returns reference list of the article as a List of INSPIRE article ids
        """
        if self.metadata.get("references"):
            return [
                element["record"]["$ref"].split("/")[-1]
                for element in self.metadata["references"]
                if element.get("record")
            ]
```

We can now look at one example. All the bibliographic data is contained on the *metadata* property of our LiteratureRecord class.  I implemented a *references* property in order to retrieve the list of references (only the article identifiers).   Lets extract data for the article with the identifier ['11883'](https://labs.inspirehep.net/literature/11883),

```python
inspireapi = InspireAPI()

record = inspireapi.literature('11883')

record.references
```

    ['40440',
     '12289',
     '12290',
     '14006',
     '12291',
     '12288',
     '9159',
     '43800',
     '43801']

In this way we are able to extract the lists of references we need for our model.  Note that the INSPIRE-API returns references in the order of appearance on the reference section of the article, and by tradition, the HEP community orders references by the order of appearance on the text.

# Recommendations

We can use the Skip-Gram model embeddings and some metric on the vector space to provide recommendations for articles with at least some citations.  Very recent articles or unpopular articles with no citations will not have embeddings and we must find other method to provide recommendations for these articles.   For this reason, we consider two scenarios when we want to provide recommendations for a given article:

*i)* The article has embeddings produced by our Skip-Gram model.

*ii)* The article has no embeddings from the Skip-Gram model.

In case *i)* we retrieve the top similar articles using cosine similarity as the metric.  Cosine similarity is defined as $\cos \theta_{12} = \hat \omega_{1} \cdot \hat \omega_{2}$, where $\hat \omega = \omega/|\omega|$ is a unit norm vector.   In this case we retrieve as recommendations the articles whose embedding have a smaller angle difference with the original article.

In case *ii)*, we dont have an embedding but we can build one out of the references of the article.  We retrieve the references of the article by calling the INSPIRE-HEP api and then take the average vector $\omega_{\rm{avg}}^q =  1/n_q \sum_{i=1}^{n_q} \omega_i^{q}$ of the references. We can then proceed to retrieve the top similar articles using cosine similarity as before.

These two simple methods are able to provide recommendations for most articles.

# Web application

The recommender system has been made available at [hep-recommender](https://hep-recommender.herokuapp.com/).   It is a Flask web application
currently hosted on Heroku with a gunicorn server. Storage of the model artifacts is done in AWS S3.

# Discussion

To see one example of the application, we can look the recommendations of similar articles for:

["Broken Symmetries and the Masses of Gauge Bosons"](https://labs.inspirehep.net/literature/11883) by Higgs, Peter W. (1964).

This article is one of the works behind the 2013 Physics nobel prize for the theoretical development of the so-called Higgs mechanism.  The recommendations from our system are very good, including the other articles which were also responsible for these theoretical developments as aknowledged by the community.

Another interesting example is this one

["Unitary Symmetry and Leptonic Decays"](http://www.hep-recommender.com/?article=4510) by Cabibbo, Nicola (1963)

This article introduced the idea of quark mixing when only two quark generations were known.  The recommendations include the article

["CP Violation in the Renormalizable Theory of Weak Interaction"](https://labs.inspirehep.net/literature/81350) by Kobayashi, Makoto; Maskawa, Toshihide (1973)

which extends the idea of Cabibbo to three quark generations and gave rise to what is now known as the CKM matrix.

Exploring a large set of articles I was surprised by the high quality of the recommendations.  Several experts on the field of HEP have also been using the application and find the recommendations satisfactory.

# References

[[1]](https://www.semanticscholar.org/paper/Co-citation-in-the-scientific-literature%3A-A-new-of-Small/da30b84925764b550b55c7d00596f8f1b9608fe2) H. Small, “Co-citation in the scientific literature: A new measure of the relationship between two documents” Journal of the American Society for Information Science, vol. 24, no. 4, pp. 265–269, 1973.

[[2]](https://www.semanticscholar.org/paper/Citation-Proximity-Analysis-%28CPA%28-%3A-A-New-Approach-Gipp-Beel/22b519fcf4b623bef5cd702f2e1c995640d4c833) B. Gipp and J. Beel, “Citation proximity analysis (cpa) : A new approach for identifying related work based on co-citation analysis” in Proceedings of the 12th International Confer- ence on Scientometrics and Informetrics, vol. 1 (B. Larsen, ed.), (Sao Paulo), pp. 571–575, BIREME/PANO/WHO, 2009.

[[3]](https://www.semanticscholar.org/paper/A-Scalable-Hybrid-Research-Paper-Recommender-System-Kanakia-Shen/bb246e08bc6641672c2bb2b93d4214eccf3f84b6) A. Kanakia, Z. Shen, D. Eide, and K. Wang, “A scalable hybrid research paper recommender system for microsoft academic” CoRR, vol. abs/1905.08880, 2019.

[[4]](https://www.semanticscholar.org/paper/System-of-Document-Connections-Based-on-References-Marshakova-shaikevich/2d871489eb7288dd1bec4be99bc363efd4933d48) I. V. Marshakova-shaikevich, “System of document connections based on references” 2009.

[[5]](https://www.semanticscholar.org/paper/Bibliographic-coupling-between-scientific-papers-Kessler/68300052245f0f3aed1a4d65943e436cf1227242) M. M. Kessler, “Bibliographic coupling between scientific papers” American Documentation, vol. 14, no. 1, pp. 10–25, 1963.

[[6]](https://www.semanticscholar.org/paper/Efficient-Estimation-of-Word-Representations-in-Mikolov-Chen/330da625c15427c6e42ccfa3b747fb29e5835bf0) T. Mikolov, K. Chen, G. S. Corrado, and J. Dean, “Efficient estimation of word representations in vector space” CoRR, vol. abs/1301.3781, 2013.

[[7]](https://www.semanticscholar.org/paper/Evaluating-link-based-recommendations-for-Wikipedia-Schwarzer-Schubotz/7c72a2008f94078ce50ac251c6437038bbcf0185) M. Schwarzer, M. Schubotz, N. Meuschke, C. Breitinger, V. Markl, and B. Gipp, “Evaluating link-based recommendations for wikipedia” 2016 IEEE/ACM Joint Conference on Digital Libraries (JCDL), pp. 191–200, 2016.


# Writing a SAS Global Forum Paper in Jupyter Lab

This paper outlines how you can submit your SAS Global forum paper directly from Jupyter lab

In the cells below you will see examples of common elements in SAS Global Forum papers

This template leverages [Jupyter Book](https://jupyterbook.org/start/overview.html)

The [MyST Cheat sheet](https://jupyterbook.org/reference/cheatsheet.html) is here

For questions post issues to: <>

# Headings

# This is a heading (only for title)

## this is a subheading (section headings)

### this is a heading3

This is normal text

**This code is in bold**

This is *italic* text

You can use either underscore `_` or asterisk `*` to augment your text.

A single symbol for italics _underscore_ or *asterisk*

A double symbol for bold __underscore__ or **asterisk**

# SAS Code

The preferred way to represent code is in a code cell.
You can modify the cell metadata to present combinations of code, output, or both removed or hidden by default.  This behavior is active in the HTML version of your paper.

You can see the cell metadata in the `property inspector`.

## To create inline runable code, use a code cell

To create code that is not runable use the single backtick '\`' for code that is less than one line long
and triple backtick '\`\`\`' for blocks of code

Here is an example of a single backtick `data _null_`

Here is an example of a triple backtick (use sas to get SAS syntax coloring in your code block. It only works with code blocks).
```sas
proc print data=sashelp.class(obs=10)
run;
```

```sas
/* This cell should be removed */
proc print data=sashelp.class;
run;
```

```sas
/* This output should be removed */
proc print data=sashelp.class;
run;
```

```sas
/* This code should be removed */
proc print data=sashelp.class;
run;
```

```sas
/* This code should be hidden */
proc print data=sashelp.class;
run;
```

```sas
/* This output should be hidden */
proc print data=sashelp.class;
run;
```

```sas
/* This cell should be hidden */
proc print data=sashelp.class;
run;
```

# Math Equations

You can specify your math equations between `$$` and then will be rendered by [MathJax](https://www.mathjax.org). [Here](https://davidhamann.de/2017/06/12/latex-cheat-sheet/) is a good reference and tutorial.

$$
a_1 = b_1 + c_1
$$

$$
y = \beta_0 + \beta_1 X_1 +\beta_2 X_2 + \epsilon
$$

\begin{equation}
  \int_0^\infty \frac{x^3}{e^x-1}\,dx = \frac{\pi^4}{15}
  \label{eq:sample}
\end{equation}

# Images

* Include an image via URL

* Include an image via local file

* Format an image

```{image} ../images/sas-campus.jpg
:alt: SAS Cary Campus
:class: bg-primary
:width: 200px
:align: center
```

# Figures

# Tables

[Tables reference](https://www.markdownguide.org/extended-syntax/#tables)

| Syntax      | Description | Test Text     |
| :---        |    :----:   |          ---: |
| Header      | Title       | Here's this   |
| Paragraph   | Text        | And more      |

# Footnotes

This is a very important finding

This is a example of footnote and tooltip[1](#fn1 "footnote and tooltip 1").
Another footnote[2](#fn2).
Another tooltip[3](#_blank "tooltip 3").

footnote and tooltip 1
footnote 2

```{note}
Here is a note
```

# This cell and All cells above should be removed before publication!!!

Paper ####-2020

My Great Paper Title

Jane Doe, Anywhere USA

## ABSTRACT

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum

## INTRODUCTION

Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source. Lorem Ipsum comes from sections 1.10.32 and 1.10.33 of "de Finibus Bonorum et Malorum" (The Extremes of Good and Evil) by Cicero, written in 45 BC. This book is a treatise on the theory of ethics, very popular during the Renaissance. The first line of Lorem Ipsum, "Lorem ipsum dolor sit amet..", comes from a line in section 1.10.32.

```sas

```

## CONCLUSION

```sas

```

```sas

```

## References

## Recommended Reading

## Acknowledgments

## CONTACT INFORMATION

Your comments and questions are valued and encouraged. Contact the author at:

```
Jane Doe
Professor
Department of Statistics
Anywhere, NC
Jane.Doe@sas.com
```

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Other brand and product names are trademarks of their respective companies.