# A short & practical introduction to Tensor Flow!

Part 5

Example of a single-layer one-directional long short-term memory network (LSTM) trained with
[connectionist temporal classification](http://www.cs.toronto.edu/~graves/icml_2006.pdf) to predict character sequences from nFeatures x nFrames
arrays of Mel-Frequency Cepstral Coefficients.  This is test code to run on the
8-item data set in the "sample_data" directory, for those without access to TIMIT.

Author: [Jon Rein](https://github.com/jonrein/tensorflow_CTC_example)

Adapted by: Pablo M. Olmos (olmos@tsc.uc3m.es)

Date: March 2017

```python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
from tensorflow.python.ops import ctc_ops as ctc
import numpy as np
from utils import load_batched_data
```

```python
# Lets check what version of tensorflow we have installed. The provided scripts should run with tf 1.0 and above

print(tf.__version__)
```

    1.0.1

```python
batchSize = 4 # Batch of sequences

## Change according to the folder where you saved the dataset provided
INPUT_PATH = 'XX/MCC_sample_data_phoneme_recog/mfcc' #directory of MFCC nFeatures x nFrames 2-D array .npy files
TARGET_PATH = 'XX/MCC_sample_data_phoneme_recog/char_y/' #directory of nCharacters 1-D array .npy files

####Load data
print('Loading data')
batchedData, maxTimeSteps, totalN = load_batched_data(INPUT_PATH, TARGET_PATH, batchSize)
```

    Loading data

```python
####Learning Parameters
learningRate = 0.001
momentum = 0.9
nEpochs = 200

####Network Parameters
nFeatures = 26 #12 MFCC coefficients + energy, and derivatives
nHidden = 128
nClasses = 28#27 characters, plus the "blank" for CTC
```

## Creating the computation graph

We will create a LSTM layer with 128 memory cells. On top of this, we use a fully connected soft-max layer.

We use [CTC classification](http://www.cs.toronto.edu/~graves/icml_2006.pdf) to compute the loss function that we can optimize by gradient descend. This function is already provided in the TF [contributions library](        https://www.tensorflow.org/versions/r0.10/api_docs/python/nn/conectionist_temporal_classification__ctc_)

### LSTMs

Recall the fundamental model

Also, the un-regularized cost function is

\begin{align}
J(\boldsymbol{\theta})=\frac{1}{N}\sum_{n=1}^N\sum_{t=1}^{T_n}d(\boldsymbol{y}_t^{(n)},\text{softmax}(\boldsymbol{W}_h \boldsymbol{h}_t^{(n)}+\mathbf{b}))
\end{align}
where $d(\cdot,\cdot)$ is the cross-entropy loss function.

```python
####Define graph
print('Defining graph')
graph = tf.Graph()
with graph.as_default():

    #### We start by encoding the forward LSTM with nHidden cells

    #i(t) parameters
    # Input gate: input, previous output, and bias.
    ix = tf.Variable(tf.truncated_normal([nFeatures, nHidden], -0.1, 0.1))   ##W^ix
    im = tf.Variable(tf.truncated_normal([nHidden, nHidden], -0.1, 0.1)) ## W^ih
    ib = tf.Variable(tf.zeros([1, nHidden])) ##b_i

    #f(t) parameters
    # Forget gate: input, previous output, and bias.
    fx = tf.Variable(tf.truncated_normal([nFeatures, nHidden], -0.1, 0.1)) ##W^fx
    fm = tf.Variable(tf.truncated_normal([nHidden, nHidden], -0.1, 0.1)) ##W^fh
    fb = tf.Variable(tf.zeros([1, nHidden])) ##b_f

    #g(t) parameters
    # Memory cell: input, state and bias.
    cx = tf.Variable(tf.truncated_normal([nFeatures, nHidden], -0.1, 0.1)) ##W^gx
    cm = tf.Variable(tf.truncated_normal([nHidden, nHidden], -0.1, 0.1)) ##W^gh
    cb = tf.Variable(tf.zeros([1, nHidden]))  ##b_g

    #o(t) parameters
    # Output gate: input, previous output, and bias.
    ox = tf.Variable(tf.truncated_normal([nFeatures, nHidden], -0.1, 0.1))  ##W^ox
    om = tf.Variable(tf.truncated_normal([nHidden, nHidden], -0.1, 0.1))  ##W^oh
    ob = tf.Variable(tf.zeros([1, nHidden])) ##b_o

    # Variable saving state across unrollings.
    saved_output = tf.Variable(tf.zeros([batchSize, nHidden]), trainable=False) #h(t)
    saved_state = tf.Variable(tf.zeros([batchSize, nHidden]), trainable=False) #s(t)
    ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####

    # Definition of the cell computation.
    def lstm_cell(i, o, state):
        """Create a LSTM cell. See e.g.: http://arxiv.org/pdf/1402.1128v1.pdf
        Note that in this formulation, we omit the various connections between the
        previous state and the gates."""

        input_gate = tf.sigmoid(tf.matmul(i, ix) + tf.matmul(o, im) + ib)
        forget_gate = tf.sigmoid(tf.matmul(i, fx) + tf.matmul(o, fm) + fb)
        update = tf.matmul(i, cx) + tf.matmul(o, cm) + cb
        state = forget_gate * state + input_gate * tf.tanh(update)    #tf.tanh(update) is g(t)
        output_gate = tf.sigmoid(tf.matmul(i, ox) + tf.matmul(o, om) + ob)
        return output_gate * tf.tanh(state), state      #h(t) is output_gate * tf.tanh(state)
    ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####

    # Classifier weights and biases (over h(t) and b_h(t) to labels)
    w = tf.Variable(tf.truncated_normal([nHidden, nClasses], -0.1, 0.1))
    b = tf.Variable(tf.zeros([nClasses]))

    ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####  ####

    # Now we define the placeholders for the input data

    train_data = list()
    for _ in range(maxTimeSteps):
        train_data.append(tf.placeholder(tf.float32, shape=[batchSize,nFeatures]))

    targetIxs = tf.placeholder(tf.int64)
    targetVals = tf.placeholder(tf.int32)
    targetShape = tf.placeholder(tf.int64)
    targetY = tf.SparseTensor(targetIxs, targetVals, targetShape)
    seqLengths = tf.placeholder(tf.int32, shape=(batchSize))

    # Given the input, we indicate how to compute the hidden states

    # Unrolled forward LSTM loop.
    outputs = list()
    output = saved_output
    state = saved_state
    for i in train_data:
        output, state = lstm_cell(i, output, state)
        outputs.append(output)

    # State saving across unrollings.

    # With this, we make sure that the loss is not computed until all states are first evaluate
    with tf.control_dependencies([saved_output.assign(output),saved_state.assign(state)]):
        #Classifier.
        logits = tf.reshape(tf.matmul(tf.concat(axis=0,values=outputs),w)+b,[-1,batchSize,nClasses])
        loss = tf.reduce_mean(ctc.ctc_loss(inputs=logits, labels=targetY,  sequence_length=seqLengths))

    ####Optimizing
    optimizer = tf.train.MomentumOptimizer(learningRate, momentum).minimize(loss)

    ####Evaluating
    # We use greedy decoding to estimate label sequences for training data
    logitsMaxTest = tf.slice(tf.argmax(logits, 2), [0, 0], [seqLengths[0], 1])
    predictions = tf.to_int32(ctc.ctc_beam_search_decoder(logits, seqLengths)[0][0])
    errorRate = tf.reduce_sum(tf.edit_distance(predictions, targetY, normalize=False)) / tf.to_float(tf.size(targetY.values))
```

    Defining graph

```python
####Run session
with tf.Session(graph=graph) as session:
    print('Initializing')
    tf.global_variables_initializer().run()

    for epoch in range(nEpochs):
        print('Epoch', epoch+1, '...')
        batchErrors = np.zeros(len(batchedData))
        batchRandIxs = np.random.permutation(len(batchedData)) #randomize batch order
        for batch, batchOrigI in enumerate(batchRandIxs):
            batchInputs, batchTargetSparse, batchSeqLengths = batchedData[batchOrigI]
            batchTargetIxs, batchTargetVals, batchTargetShape = batchTargetSparse
            feedDict = {targetIxs: batchTargetIxs, targetVals: batchTargetVals,
                        targetShape: batchTargetShape, seqLengths: batchSeqLengths}
            for i in range(maxTimeSteps):
                feedDict[train_data[i]] = batchInputs[i,:,:]

            _, l, er, lmt, logits_out = session.run([optimizer, loss, errorRate, logitsMaxTest,logits], feed_dict=feedDict)
            print(np.unique(lmt)) #print unique argmax values of first sample in batch; should be blank for a while, then spit out target values
            if (batch % 1) == 0:
                print('Minibatch', batch, '/', batchOrigI, 'loss:', l)
                print('Minibatch', batch, '/', batchOrigI, 'error rate:', er)
            batchErrors[batch] = er*len(batchSeqLengths)
        epochErrorRate = batchErrors.sum() / totalN
        print('Epoch', epoch+1, 'error rate:', epochErrorRate)

```

    Initializing
    Epoch 1 ...
    [ 0  1  2  3  4  5  6  7  9 10 11 12 14 15 16 17 18 19 20 21 22 23 24 25 26
     27]
    Minibatch 0 / 0 loss: 869.418
    Minibatch 0 / 0 error rate: 1.85075
    [ 0  1  2  3  4  5  6  7  9 10 14 15 16 18 19 20 21 22 25 26 27]
    Minibatch 1 / 1 loss: 904.927
    Minibatch 1 / 1 error rate: 1.46866
    Epoch 1 error rate: 1.65970146656
    Epoch 2 ...
    [ 0  1  3  4  5  6  7  8 10 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27]
    Minibatch 0 / 0 loss: 806.009
    Minibatch 0 / 0 error rate: 1.84701
    [ 0  1  3  4  6  7 10 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27]
    Minibatch 1 / 1 loss: 771.455
    Minibatch 1 / 1 error rate: 1.32239
    Epoch 2 error rate: 1.58470147848
    Epoch 3 ...
    [ 0  4  5 12 14 15 16 24 27]
    Minibatch 0 / 1 loss: 585.832
    Minibatch 0 / 1 error rate: 0.797015
    [ 0 27]
    Minibatch 1 / 0 loss: 322.648
    Minibatch 1 / 0 error rate: 0.839552
    Epoch 3 error rate: 0.818283587694
    Epoch 4 ...
    [ 0 27]
    Minibatch 0 / 1 loss: 277.227
    Minibatch 0 / 1 error rate: 0.979104
    [ 0 27]
    Minibatch 1 / 0 loss: 234.57
    Minibatch 1 / 0 error rate: 0.985075
    Epoch 4 error rate: 0.982089549303
    Epoch 5 ...
    [ 0 27]
    Minibatch 0 / 0 loss: 264.399
    Minibatch 0 / 0 error rate: 0.985075
    [ 0 27]
    Minibatch 1 / 1 loss: 368.805
    Minibatch 1 / 1 error rate: 0.985075
    Epoch 5 error rate: 0.98507463932
    Epoch 6 ...
    [ 0 27]
    Minibatch 0 / 1 loss: 395.835
    Minibatch 0 / 1 error rate: 0.985075
    [ 0  6 27]
    Minibatch 1 / 0 loss: 321.503
    Minibatch 1 / 0 error rate: 0.985075
    Epoch 6 error rate: 0.98507463932
    Epoch 7 ...
    [ 0 27]
    Minibatch 0 / 1 loss: 415.518
    Minibatch 0 / 1 error rate: 0.985075
    [ 0  6 27]
    Minibatch 1 / 0 loss: 310.699
    Minibatch 1 / 0 error rate: 0.981343
    Epoch 7 error rate: 0.983208954334
    Epoch 8 ...
    [ 0 12 27]
    Minibatch 0 / 1 loss: 390.025
    Minibatch 0 / 1 error rate: 0.967164
    [ 0  6 12 27]
    Minibatch 1 / 0 loss: 266.782
    Minibatch 1 / 0 error rate: 0.873134
    Epoch 8 error rate: 0.920149236917
    Epoch 9 ...
    [ 0 12 27]
    Minibatch 0 / 1 loss: 328.451
    Minibatch 0 / 1 error rate: 0.844776
    [ 0  6 12 27]
    Minibatch 1 / 0 loss: 233.808
    Minibatch 1 / 0 error rate: 0.742537
    Epoch 9 error rate: 0.79365670681
    Epoch 10 ...
    [ 0 12 14 26 27]
    Minibatch 0 / 1 loss: 280.828
    Minibatch 0 / 1 error rate: 0.785075
    [ 0  1 12 14 26 27]
    Minibatch 1 / 0 loss: 220.602
    Minibatch 1 / 0 error rate: 0.731343
    Epoch 10 error rate: 0.758208960295
    Epoch 11 ...
    [ 0  1  4 12 14 15 22 26 27]
    Minibatch 0 / 0 loss: 227.794
    Minibatch 0 / 0 error rate: 0.727612
    [ 0  1  4  5 12 14 26 27]
    Minibatch 1 / 1 loss: 270.691
    Minibatch 1 / 1 error rate: 0.743284
    Epoch 11 error rate: 0.735447764397
    Epoch 12 ...
    [ 0  1  4  5 12 14 15 26 27]
    Minibatch 0 / 1 loss: 273.665
    Minibatch 0 / 1 error rate: 0.752239
    [ 0  1  4 12 14 15 22 26 27]
    Minibatch 1 / 0 loss: 241.86
    Minibatch 1 / 0 error rate: 0.738806
    Epoch 12 error rate: 0.745522379875
    Epoch 13 ...
    [ 0  4 12 14 22 26 27]
    Minibatch 0 / 0 loss: 239.378
    Minibatch 0 / 0 error rate: 0.723881
    [ 0  4  5 12 22 26 27]
    Minibatch 1 / 1 loss: 270.804
    Minibatch 1 / 1 error rate: 0.791045
    Epoch 13 error rate: 0.75746268034
    Epoch 14 ...
    [ 0  4 12 22 26 27]
    Minibatch 0 / 1 loss: 269.324
    Minibatch 0 / 1 error rate: 0.829851
    [ 0 12 22 26 27]
    Minibatch 1 / 0 loss: 226.784
    Minibatch 1 / 0 error rate: 0.772388
    Epoch 14 error rate: 0.80111938715
    Epoch 15 ...
    [ 0 12 22 26 27]
    Minibatch 0 / 0 loss: 223.324
    Minibatch 0 / 0 error rate: 0.764925
    [ 0  4  5 12 26 27]
    Minibatch 1 / 1 loss: 258.681
    Minibatch 1 / 1 error rate: 0.764179
    Epoch 15 error rate: 0.764552235603
    Epoch 16 ...
    [ 0  4 12 14 26 27]
    Minibatch 0 / 1 loss: 252.699
    Minibatch 0 / 1 error rate: 0.743284
    [ 0  4 12 14 22 26 27]
    Minibatch 1 / 0 loss: 212.51
    Minibatch 1 / 0 error rate: 0.671642
    Epoch 16 error rate: 0.707462668419
    Epoch 17 ...
    [ 0  4 12 14 19 26 27]
    Minibatch 0 / 1 loss: 244.301
    Minibatch 0 / 1 error rate: 0.707463
    [ 0  4 12 14 26 27]
    Minibatch 1 / 0 loss: 205.924
    Minibatch 1 / 0 error rate: 0.660448
    Epoch 17 error rate: 0.683955222368
    Epoch 18 ...
    [ 0  4 12 14 26 27]
    Minibatch 0 / 1 loss: 238.389
    Minibatch 0 / 1 error rate: 0.701493
    [ 0  4 12 14 26 27]
    Minibatch 1 / 0 loss: 197.096
    Minibatch 1 / 0 error rate: 0.656716
    Epoch 18 error rate: 0.679104477167
    Epoch 19 ...
    [ 0  4 12 14 17 26 27]
    Minibatch 0 / 0 loss: 194.246
    Minibatch 0 / 0 error rate: 0.649254
    [ 0  4 12 14 26 27]
    Minibatch 1 / 1 loss: 237.298
    Minibatch 1 / 1 error rate: 0.698507
    Epoch 19 error rate: 0.67388060689
    Epoch 20 ...
    [ 0  4 12 14 26 27]
    Minibatch 0 / 1 loss: 236.57
    Minibatch 0 / 1 error rate: 0.689552
    [ 0  1  4 12 14 15 17 26 27]
    Minibatch 1 / 0 loss: 189.302
    Minibatch 1 / 0 error rate: 0.66791
    Epoch 20 error rate: 0.678731352091
    Epoch 21 ...
    [ 0  4 12 14 26 27]
    Minibatch 0 / 1 loss: 231.954
    Minibatch 0 / 1 error rate: 0.680597
    [ 0  1  4  9 12 14 15 17 26 27]
    Minibatch 1 / 0 loss: 185.598
    Minibatch 1 / 0 error rate: 0.649254
    Epoch 21 error rate: 0.66492536664
    Epoch 22 ...
    [ 0  1  4  9 12 14 15 17 26 27]
    Minibatch 0 / 0 loss: 182.795
    Minibatch 0 / 0 error rate: 0.63806
    [ 0  4 12 14 26 27]
    Minibatch 1 / 1 loss: 224.726
    Minibatch 1 / 1 error rate: 0.665672
    Epoch 22 error rate: 0.651865661144
    Epoch 23 ...
    [ 0  1  4  9 10 12 14 15 17 26 27]
    Minibatch 0 / 0 loss: 179.998
    Minibatch 0 / 0 error rate: 0.634328
    [ 0  4 12 14 26 27]
    Minibatch 1 / 1 loss: 221.414
    Minibatch 1 / 1 error rate: 0.668657
    Epoch 23 error rate: 0.651492536068
    Epoch 24 ...
    [ 0  1  4  9 10 12 14 15 17 26 27]
    Minibatch 0 / 0 loss: 177.096
    Minibatch 0 / 0 error rate: 0.630597
    [ 0  4 12 14 15 26 27]
    Minibatch 1 / 1 loss: 218.618
    Minibatch 1 / 1 error rate: 0.665672
    Epoch 24 error rate: 0.648134320974
    Epoch 25 ...
    [ 0  4  5  9 10 12 14 15 17 26 27]
    Minibatch 0 / 0 loss: 174.653
    Minibatch 0 / 0 error rate: 0.649254
    [ 0  4  9 12 14 15 22 26 27]
    Minibatch 1 / 1 loss: 215.852
    Minibatch 1 / 1 error rate: 0.668657
    Epoch 25 error rate: 0.658955216408
    Epoch 26 ...
    [ 0  4  5  9 10 12 14 15 17 22 26 27]
    Minibatch 0 / 0 loss: 172.132
    Minibatch 0 / 0 error rate: 0.63806
    [ 0  4  9 12 14 15 22 26 27]
    Minibatch 1 / 1 loss: 212.63
    Minibatch 1 / 1 error rate: 0.665672
    Epoch 26 error rate: 0.651865661144
    Epoch 27 ...
    [ 0  4  5  9 10 12 14 15 19 22 26 27]
    Minibatch 0 / 0 loss: 169.311
    Minibatch 0 / 0 error rate: 0.641791
    [ 0  4  9 12 14 15 22 26 27]
    Minibatch 1 / 1 loss: 209.269
    Minibatch 1 / 1 error rate: 0.674627
    Epoch 27 error rate: 0.658208966255
    Epoch 28 ...
    [ 0  4  5  9 10 12 14 15 19 22 26 27]
    Minibatch 0 / 0 loss: 166.589
    Minibatch 0 / 0 error rate: 0.641791
    [ 0  1  4  9 12 14 15 22 26 27]
    Minibatch 1 / 1 loss: 206.217
    Minibatch 1 / 1 error rate: 0.671642
    Epoch 28 error rate: 0.656716406345
    Epoch 29 ...
    [ 0  4  5  9 10 12 14 15 19 22 26 27]
    Minibatch 0 / 0 loss: 164.174
    Minibatch 0 / 0 error rate: 0.630597
    [ 0  1  4  9 12 14 22 26 27]
    Minibatch 1 / 1 loss: 203.413
    Minibatch 1 / 1 error rate: 0.665672
    Epoch 29 error rate: 0.648134320974
    Epoch 30 ...
    [ 0  4  5  9 10 12 14 15 19 22 26 27]
    Minibatch 0 / 0 loss: 162.118
    Minibatch 0 / 0 error rate: 0.623134
    [ 0  1  4  9 12 14 22 26 27]
    Minibatch 1 / 1 loss: 200.618
    Minibatch 1 / 1 error rate: 0.659701
    Epoch 30 error rate: 0.641417890787
    Epoch 31 ...
    [ 0  1  4  9 12 14 22 26 27]
    Minibatch 0 / 1 loss: 199.327
    Minibatch 0 / 1 error rate: 0.653731
    [ 0  4  5  9 10 12 14 15 17 19 22 26 27]
    Minibatch 1 / 0 loss: 159.894
    Minibatch 1 / 0 error rate: 0.623134
    Epoch 31 error rate: 0.638432830572
    Epoch 32 ...
    [ 0  1  4  9 12 14 15 21 22 26 27]
    Minibatch 0 / 1 loss: 195.326
    Minibatch 0 / 1 error rate: 0.656716
    [ 0  4  5  9 10 12 14 15 17 19 22 26 27]
    Minibatch 1 / 0 loss: 158.654
    Minibatch 1 / 0 error rate: 0.623134
    Epoch 32 error rate: 0.63992536068
    Epoch 33 ...
    [ 0  1  4  9 12 14 15 21 22 26 27]
    Minibatch 0 / 1 loss: 192.108
    Minibatch 0 / 1 error rate: 0.644776
    [ 0  4  5  9 10 12 14 15 17 19 22 26 27]
    Minibatch 1 / 0 loss: 157.225
    Minibatch 1 / 0 error rate: 0.630597
    Epoch 33 error rate: 0.637686550617
    Epoch 34 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 26 27]
    Minibatch 0 / 0 loss: 156.36
    Minibatch 0 / 0 error rate: 0.623134
    [ 0  1  4  9 12 14 15 21 22 26 27]
    Minibatch 1 / 1 loss: 188.401
    Minibatch 1 / 1 error rate: 0.635821
    Epoch 34 error rate: 0.629477620125
    Epoch 35 ...
    [ 0  1  4  9 12 14 15 21 22 26 27]
    Minibatch 0 / 1 loss: 187.768
    Minibatch 0 / 1 error rate: 0.641791
    [ 0  4  5  9 10 12 14 15 17 19 22 26 27]
    Minibatch 1 / 0 loss: 152.905
    Minibatch 1 / 0 error rate: 0.604478
    Epoch 35 error rate: 0.623134315014
    Epoch 36 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 26 27]
    Minibatch 0 / 0 loss: 152.016
    Minibatch 0 / 0 error rate: 0.604478
    [ 0  1  4  9 12 14 15 21 22 26 27]
    Minibatch 1 / 1 loss: 184.178
    Minibatch 1 / 1 error rate: 0.635821
    Epoch 36 error rate: 0.620149254799
    Epoch 37 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 149.262
    Minibatch 0 / 0 error rate: 0.597015
    [ 0  1  4  9 12 14 15 21 22 25 26 27]
    Minibatch 1 / 1 loss: 182.583
    Minibatch 1 / 1 error rate: 0.626866
    Epoch 37 error rate: 0.611940294504
    Epoch 38 ...
    [ 0  1  4  9 12 14 15 21 22 25 26 27]
    Minibatch 0 / 1 loss: 182.091
    Minibatch 0 / 1 error rate: 0.620896
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 146.369
    Minibatch 1 / 0 error rate: 0.597015
    Epoch 38 error rate: 0.608955204487
    Epoch 39 ...
    [ 0  1  4  9 12 14 15 17 21 22 25 26 27]
    Minibatch 0 / 1 loss: 179.141
    Minibatch 0 / 1 error rate: 0.608955
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 144.821
    Minibatch 1 / 0 error rate: 0.597015
    Epoch 39 error rate: 0.602985054255
    Epoch 40 ...
    [ 0  1  4  9 12 14 15 17 21 22 25 26 27]
    Minibatch 0 / 1 loss: 176.727
    Minibatch 0 / 1 error rate: 0.608955
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 143.294
    Minibatch 1 / 0 error rate: 0.589552
    Epoch 40 error rate: 0.599253714085
    Epoch 41 ...
    [ 0  1  4  9 12 14 15 17 21 22 25 26 27]
    Minibatch 0 / 1 loss: 174.356
    Minibatch 0 / 1 error rate: 0.597015
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 141.787
    Minibatch 1 / 0 error rate: 0.58209
    Epoch 41 error rate: 0.589552223682
    Epoch 42 ...
    [ 0  1  4  5  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 172.076
    Minibatch 0 / 1 error rate: 0.585075
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 140.278
    Minibatch 1 / 0 error rate: 0.574627
    Epoch 42 error rate: 0.57985073328
    Epoch 43 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 140.145
    Minibatch 0 / 0 error rate: 0.578358
    [ 0  1  4  5  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 169.255
    Minibatch 1 / 1 error rate: 0.58209
    Epoch 43 error rate: 0.580223888159
    Epoch 44 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 137.297
    Minibatch 0 / 0 error rate: 0.563433
    [ 0  1  2  4  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 167.681
    Minibatch 1 / 1 error rate: 0.579104
    Epoch 44 error rate: 0.571268647909
    Epoch 45 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 135.386
    Minibatch 0 / 0 error rate: 0.55597
    [ 0  1  2  4  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 166.009
    Minibatch 1 / 1 error rate: 0.573134
    Epoch 45 error rate: 0.564552217722
    Epoch 46 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 133.615
    Minibatch 0 / 0 error rate: 0.552239
    [ 0  1  2  4  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 164.226
    Minibatch 1 / 1 error rate: 0.567164
    Epoch 46 error rate: 0.559701502323
    Epoch 47 ...
    [ 0  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 131.971
    Minibatch 0 / 0 error rate: 0.541045
    [ 0  1  2  4  5  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 162.364
    Minibatch 1 / 1 error rate: 0.570149
    Epoch 47 error rate: 0.555597007275
    Epoch 48 ...
    [ 0  1  2  4  5  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 161.984
    Minibatch 0 / 1 error rate: 0.567164
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 130.101
    Minibatch 1 / 0 error rate: 0.537313
    Epoch 48 error rate: 0.552238821983
    Epoch 49 ...
    [ 0  1  2  4  5  8  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 158.859
    Minibatch 0 / 1 error rate: 0.558209
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 129.074
    Minibatch 1 / 0 error rate: 0.533582
    Epoch 49 error rate: 0.545895516872
    Epoch 50 ...
    [ 0  1  2  4  5  8  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 156.657
    Minibatch 0 / 1 error rate: 0.549254
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 127.891
    Minibatch 1 / 0 error rate: 0.529851
    Epoch 50 error rate: 0.539552211761
    Epoch 51 ...
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 128.157
    Minibatch 0 / 0 error rate: 0.537313
    [ 0  1  2  4  5  8  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 154.057
    Minibatch 1 / 1 error rate: 0.537313
    Epoch 51 error rate: 0.537313461304
    Epoch 52 ...
    [ 0  1  2  4  5  8  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 153.972
    Minibatch 0 / 1 error rate: 0.540299
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 124.71
    Minibatch 1 / 0 error rate: 0.522388
    Epoch 52 error rate: 0.531343281269
    Epoch 53 ...
    [ 0  1  2  4  5  8  9 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 151.306
    Minibatch 0 / 1 error rate: 0.522388
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 123.4
    Minibatch 1 / 0 error rate: 0.522388
    Epoch 53 error rate: 0.522388041019
    Epoch 54 ...
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 149.463
    Minibatch 0 / 1 error rate: 0.513433
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 122.077
    Minibatch 1 / 0 error rate: 0.522388
    Epoch 54 error rate: 0.517910450697
    Epoch 55 ...
    [ 0  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 122.463
    Minibatch 0 / 0 error rate: 0.526119
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 147.196
    Minibatch 1 / 1 error rate: 0.504478
    Epoch 55 error rate: 0.515298515558
    Epoch 56 ...
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 147.171
    Minibatch 0 / 1 error rate: 0.504478
    [ 0  1  4  5  8  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 119.054
    Minibatch 1 / 0 error rate: 0.485075
    Epoch 56 error rate: 0.494776129723
    Epoch 57 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 119.514
    Minibatch 0 / 0 error rate: 0.496269
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 144.033
    Minibatch 1 / 1 error rate: 0.501493
    Epoch 57 error rate: 0.49888060987
    Epoch 58 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 116.725
    Minibatch 0 / 0 error rate: 0.488806
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 142.663
    Minibatch 1 / 1 error rate: 0.489552
    Epoch 58 error rate: 0.489179104567
    Epoch 59 ...
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 142.575
    Minibatch 0 / 1 error rate: 0.483582
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 114.872
    Minibatch 1 / 0 error rate: 0.488806
    Epoch 59 error rate: 0.48619402945
    Epoch 60 ...
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 139.86
    Minibatch 0 / 1 error rate: 0.480597
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 113.874
    Minibatch 1 / 0 error rate: 0.481343
    Epoch 60 error rate: 0.480970144272
    Epoch 61 ...
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 138.035
    Minibatch 0 / 1 error rate: 0.468657
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 112.803
    Minibatch 1 / 0 error rate: 0.466418
    Epoch 61 error rate: 0.4675373137
    Epoch 62 ...
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 136.325
    Minibatch 0 / 1 error rate: 0.465672
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 111.647
    Minibatch 1 / 0 error rate: 0.466418
    Epoch 62 error rate: 0.466044768691
    Epoch 63 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 112.237
    Minibatch 0 / 0 error rate: 0.470149
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 134.35
    Minibatch 1 / 1 error rate: 0.459701
    Epoch 63 error rate: 0.46492536366
    Epoch 64 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 109.272
    Minibatch 0 / 0 error rate: 0.462687
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 133.365
    Minibatch 1 / 1 error rate: 0.447761
    Epoch 64 error rate: 0.455223888159
    Epoch 65 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 0 / 0 loss: 107.762
    Minibatch 0 / 0 error rate: 0.462687
    [ 0  1  2  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 132.266
    Minibatch 1 / 1 error rate: 0.447761
    Epoch 65 error rate: 0.455223888159
    Epoch 66 ...
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 131.898
    Minibatch 0 / 1 error rate: 0.447761
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 106.034
    Minibatch 1 / 0 error rate: 0.421642
    Epoch 66 error rate: 0.434701502323
    Epoch 67 ...
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 129.609
    Minibatch 0 / 1 error rate: 0.438806
    [ 0  1  4  5  9 10 12 14 15 17 19 22 25 26 27]
    Minibatch 1 / 0 loss: 105.124
    Minibatch 1 / 0 error rate: 0.41791
    Epoch 67 error rate: 0.428358212113
    Epoch 68 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 106.273
    Minibatch 0 / 0 error rate: 0.440298
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 1 loss: 127.286
    Minibatch 1 / 1 error rate: 0.435821
    Epoch 68 error rate: 0.438059702516
    Epoch 69 ...
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 127.132
    Minibatch 0 / 1 error rate: 0.432836
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 102.744
    Minibatch 1 / 0 error rate: 0.421642
    Epoch 69 error rate: 0.427238807082
    Epoch 70 ...
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 124.849
    Minibatch 0 / 1 error rate: 0.41791
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 101.73
    Minibatch 1 / 0 error rate: 0.399254
    Epoch 70 error rate: 0.408582091331
    Epoch 71 ...
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 1 loss: 123.275
    Minibatch 0 / 1 error rate: 0.41194
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 100.653
    Minibatch 1 / 0 error rate: 0.399254
    Epoch 71 error rate: 0.405597016215
    Epoch 72 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 101.741
    Minibatch 0 / 0 error rate: 0.432836
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 121.414
    Minibatch 1 / 1 error rate: 0.40597
    Epoch 72 error rate: 0.419402986765
    Epoch 73 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 98.4771
    Minibatch 0 / 0 error rate: 0.406716
    [ 0  1  2  4  5  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 120.382
    Minibatch 1 / 1 error rate: 0.39403
    Epoch 73 error rate: 0.400373131037
    Epoch 74 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 97.0958
    Minibatch 0 / 0 error rate: 0.399254
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 119.239
    Minibatch 1 / 1 error rate: 0.391045
    Epoch 74 error rate: 0.395149245858
    Epoch 75 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 95.8222
    Minibatch 0 / 0 error rate: 0.402985
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 117.98
    Minibatch 1 / 1 error rate: 0.391045
    Epoch 75 error rate: 0.397014915943
    Epoch 76 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 117.92
    Minibatch 0 / 1 error rate: 0.4
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 94.3704
    Minibatch 1 / 0 error rate: 0.38806
    Epoch 76 error rate: 0.394029855728
    Epoch 77 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 95.6203
    Minibatch 0 / 0 error rate: 0.402985
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 114.963
    Minibatch 1 / 1 error rate: 0.376119
    Epoch 77 error rate: 0.389552235603
    Epoch 78 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 92.7221
    Minibatch 0 / 0 error rate: 0.384328
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 113.719
    Minibatch 1 / 1 error rate: 0.376119
    Epoch 78 error rate: 0.380223885179
    Epoch 79 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 91.5891
    Minibatch 0 / 0 error rate: 0.36194
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 112.465
    Minibatch 1 / 1 error rate: 0.373134
    Epoch 79 error rate: 0.367537304759
    Epoch 80 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 90.4798
    Minibatch 0 / 0 error rate: 0.358209
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 111.203
    Minibatch 1 / 1 error rate: 0.370149
    Epoch 80 error rate: 0.364179104567
    Epoch 81 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 89.3991
    Minibatch 0 / 0 error rate: 0.354478
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 109.94
    Minibatch 1 / 1 error rate: 0.367164
    Epoch 81 error rate: 0.360820889473
    Epoch 82 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 88.3502
    Minibatch 0 / 0 error rate: 0.347015
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 108.677
    Minibatch 1 / 1 error rate: 0.361194
    Epoch 82 error rate: 0.354104489088
    Epoch 83 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 87.3284
    Minibatch 0 / 0 error rate: 0.335821
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 107.412
    Minibatch 1 / 1 error rate: 0.355224
    Epoch 83 error rate: 0.345522388816
    Epoch 84 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 86.3305
    Minibatch 0 / 0 error rate: 0.328358
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 106.147
    Minibatch 1 / 1 error rate: 0.352239
    Epoch 84 error rate: 0.340298503637
    Epoch 85 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 85.3557
    Minibatch 0 / 0 error rate: 0.328358
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 104.888
    Minibatch 1 / 1 error rate: 0.355224
    Epoch 85 error rate: 0.341791048646
    Epoch 86 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 84.3986
    Minibatch 0 / 0 error rate: 0.324627
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 103.639
    Minibatch 1 / 1 error rate: 0.343284
    Epoch 86 error rate: 0.333955228329
    Epoch 87 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 83.4513
    Minibatch 0 / 0 error rate: 0.317164
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 102.404
    Minibatch 1 / 1 error rate: 0.334328
    Epoch 87 error rate: 0.325746268034
    Epoch 88 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 82.5094
    Minibatch 0 / 0 error rate: 0.317164
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 101.184
    Minibatch 1 / 1 error rate: 0.334328
    Epoch 88 error rate: 0.325746268034
    Epoch 89 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 101.453
    Minibatch 0 / 1 error rate: 0.337313
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 81.3588
    Minibatch 1 / 0 error rate: 0.302239
    Epoch 89 error rate: 0.319776117802
    Epoch 90 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 83.0071
    Minibatch 0 / 0 error rate: 0.309702
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 98.5342
    Minibatch 1 / 1 error rate: 0.325373
    Epoch 90 error rate: 0.31753732264
    Epoch 91 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 79.9263
    Minibatch 0 / 0 error rate: 0.291045
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 97.457
    Minibatch 1 / 1 error rate: 0.322388
    Epoch 91 error rate: 0.306716412306
    Epoch 92 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 97.8064
    Minibatch 0 / 1 error rate: 0.313433
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 78.6847
    Minibatch 1 / 0 error rate: 0.287313
    Epoch 92 error rate: 0.300373136997
    Epoch 93 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 80.298
    Minibatch 0 / 0 error rate: 0.302239
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 95.0365
    Minibatch 1 / 1 error rate: 0.301493
    Epoch 93 error rate: 0.301865667105
    Epoch 94 ...
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 77.2443
    Minibatch 0 / 0 error rate: 0.283582
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 94.0263
    Minibatch 1 / 1 error rate: 0.295522
    Epoch 94 error rate: 0.289552241564
    Epoch 95 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 94.4072
    Minibatch 0 / 1 error rate: 0.301493
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 76.0317
    Minibatch 1 / 0 error rate: 0.279851
    Epoch 95 error rate: 0.290671646595
    Epoch 96 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 92.0953
    Minibatch 0 / 1 error rate: 0.295522
    [ 0  1  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 75.3567
    Minibatch 1 / 0 error rate: 0.279851
    Epoch 96 error rate: 0.287686571479
    Epoch 97 ...
    [ 0  1  4  5  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 76.9106
    Minibatch 0 / 0 error rate: 0.294776
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 90.4627
    Minibatch 1 / 1 error rate: 0.280597
    Epoch 97 error rate: 0.287686556578
    Epoch 98 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 73.9285
    Minibatch 0 / 0 error rate: 0.272388
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 89.5396
    Minibatch 1 / 1 error rate: 0.280597
    Epoch 98 error rate: 0.276492536068
    Epoch 99 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 89.98
    Minibatch 0 / 1 error rate: 0.277612
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 72.7317
    Minibatch 1 / 0 error rate: 0.261194
    Epoch 99 error rate: 0.269402980804
    Epoch 100 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 74.232
    Minibatch 0 / 0 error rate: 0.276119
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 87.3823
    Minibatch 1 / 1 error rate: 0.262687
    Epoch 100 error rate: 0.269402995706
    Epoch 101 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 71.407
    Minibatch 0 / 0 error rate: 0.253731
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 86.4593
    Minibatch 1 / 1 error rate: 0.262687
    Epoch 101 error rate: 0.258208960295
    Epoch 102 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 86.9061
    Minibatch 0 / 1 error rate: 0.262687
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 70.3033
    Minibatch 1 / 0 error rate: 0.25
    Epoch 102 error rate: 0.25634329021
    Epoch 103 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 84.6797
    Minibatch 0 / 1 error rate: 0.253731
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 69.6908
    Minibatch 1 / 0 error rate: 0.238806
    Epoch 103 error rate: 0.24626865238
    Epoch 104 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 83.5111
    Minibatch 0 / 1 error rate: 0.253731
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 69.0457
    Minibatch 1 / 0 error rate: 0.235075
    Epoch 104 error rate: 0.244402982295
    Epoch 105 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 82.3969
    Minibatch 0 / 1 error rate: 0.253731
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 68.3647
    Minibatch 1 / 0 error rate: 0.220149
    Epoch 105 error rate: 0.236940294504
    Epoch 106 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 81.3314
    Minibatch 0 / 1 error rate: 0.250746
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 67.6539
    Minibatch 1 / 0 error rate: 0.220149
    Epoch 106 error rate: 0.235447764397
    Epoch 107 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 69.1542
    Minibatch 0 / 0 error rate: 0.231343
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 80.0115
    Minibatch 1 / 1 error rate: 0.244776
    Epoch 107 error rate: 0.238059699535
    Epoch 108 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 66.2762
    Minibatch 0 / 0 error rate: 0.216418
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 79.2521
    Minibatch 1 / 1 error rate: 0.241791
    Epoch 108 error rate: 0.229104474187
    Epoch 109 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 65.3968
    Minibatch 0 / 0 error rate: 0.212687
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 78.4421
    Minibatch 1 / 1 error rate: 0.241791
    Epoch 109 error rate: 0.227238804102
    Epoch 110 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 64.576
    Minibatch 0 / 0 error rate: 0.212687
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 77.579
    Minibatch 1 / 1 error rate: 0.238806
    Epoch 110 error rate: 0.225746266544
    Epoch 111 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 63.8086
    Minibatch 0 / 0 error rate: 0.212687
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 76.6767
    Minibatch 1 / 1 error rate: 0.229851
    Epoch 111 error rate: 0.22126865387
    Epoch 112 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 77.1958
    Minibatch 0 / 1 error rate: 0.226866
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 62.9073
    Minibatch 1 / 0 error rate: 0.208955
    Epoch 112 error rate: 0.217910453677
    Epoch 113 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 64.6208
    Minibatch 0 / 0 error rate: 0.220149
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 74.6293
    Minibatch 1 / 1 error rate: 0.226866
    Epoch 113 error rate: 0.223507463932
    Epoch 114 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 61.8849
    Minibatch 0 / 0 error rate: 0.208955
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 73.7717
    Minibatch 1 / 1 error rate: 0.214925
    Epoch 114 error rate: 0.211940303445
    Epoch 115 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 74.3308
    Minibatch 0 / 1 error rate: 0.214925
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 61.0029
    Minibatch 1 / 0 error rate: 0.205224
    Epoch 115 error rate: 0.210074625909
    Epoch 116 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 62.6907
    Minibatch 0 / 0 error rate: 0.220149
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 71.8942
    Minibatch 1 / 1 error rate: 0.2
    Epoch 116 error rate: 0.210074625909
    Epoch 117 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 59.9677
    Minibatch 0 / 0 error rate: 0.205224
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 71.1161
    Minibatch 1 / 1 error rate: 0.191045
    Epoch 117 error rate: 0.198134325445
    Epoch 118 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 71.6925
    Minibatch 0 / 1 error rate: 0.18806
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 59.0717
    Minibatch 1 / 0 error rate: 0.208955
    Epoch 118 error rate: 0.198507465422
    Epoch 119 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 69.6842
    Minibatch 0 / 1 error rate: 0.18806
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 58.5599
    Minibatch 1 / 0 error rate: 0.19403
    Epoch 119 error rate: 0.191044777632
    Epoch 120 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 60.218
    Minibatch 0 / 0 error rate: 0.205224
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 68.4744
    Minibatch 1 / 1 error rate: 0.185075
    Epoch 120 error rate: 0.195149250329
    Epoch 121 ...
    [ 0  1  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 57.5049
    Minibatch 0 / 0 error rate: 0.190299
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 67.7868
    Minibatch 1 / 1 error rate: 0.185075
    Epoch 121 error rate: 0.187686569989
    Epoch 122 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 68.372
    Minibatch 0 / 1 error rate: 0.18806
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 56.6059
    Minibatch 1 / 0 error rate: 0.179104
    Epoch 122 error rate: 0.183582089841
    Epoch 123 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 58.2054
    Minibatch 0 / 0 error rate: 0.190299
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 66.2101
    Minibatch 1 / 1 error rate: 0.18209
    Epoch 123 error rate: 0.186194032431
    Epoch 124 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 55.6127
    Minibatch 0 / 0 error rate: 0.175373
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 65.5385
    Minibatch 1 / 1 error rate: 0.173134
    Epoch 124 error rate: 0.174253731966
    Epoch 125 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 54.9491
    Minibatch 0 / 0 error rate: 0.171642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 64.8513
    Minibatch 1 / 1 error rate: 0.170149
    Epoch 125 error rate: 0.170895524323
    Epoch 126 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 54.3069
    Minibatch 0 / 0 error rate: 0.16791
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 64.1526
    Minibatch 1 / 1 error rate: 0.170149
    Epoch 126 error rate: 0.169029846787
    Epoch 127 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 53.6844
    Minibatch 0 / 0 error rate: 0.16791
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 63.4486
    Minibatch 1 / 1 error rate: 0.170149
    Epoch 127 error rate: 0.169029846787
    Epoch 128 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 64.0354
    Minibatch 0 / 1 error rate: 0.179104
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 52.919
    Minibatch 1 / 0 error rate: 0.16791
    Epoch 128 error rate: 0.173507459462
    Epoch 129 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 54.5046
    Minibatch 0 / 0 error rate: 0.182836
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 61.9022
    Minibatch 1 / 1 error rate: 0.164179
    Epoch 129 error rate: 0.173507459462
    Epoch 130 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 52.0385
    Minibatch 0 / 0 error rate: 0.164179
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 61.2601
    Minibatch 1 / 1 error rate: 0.161194
    Epoch 130 error rate: 0.162686564028
    Epoch 131 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 61.8688
    Minibatch 0 / 1 error rate: 0.164179
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 51.2691
    Minibatch 1 / 0 error rate: 0.160448
    Epoch 131 error rate: 0.162313431501
    Epoch 132 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 60.0986
    Minibatch 0 / 1 error rate: 0.152239
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 50.8191
    Minibatch 1 / 0 error rate: 0.160448
    Epoch 132 error rate: 0.156343281269
    Epoch 133 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 59.3511
    Minibatch 0 / 1 error rate: 0.149254
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 50.3408
    Minibatch 1 / 0 error rate: 0.156716
    Epoch 133 error rate: 0.152985073626
    Epoch 134 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 51.9895
    Minibatch 0 / 0 error rate: 0.171642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 58.4231
    Minibatch 1 / 1 error rate: 0.146269
    Epoch 134 error rate: 0.158955223858
    Epoch 135 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 59.0491
    Minibatch 0 / 1 error rate: 0.149254
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 49.2
    Minibatch 1 / 0 error rate: 0.160448
    Epoch 135 error rate: 0.154850743711
    Epoch 136 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 57.4319
    Minibatch 0 / 1 error rate: 0.143284
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 48.716
    Minibatch 1 / 0 error rate: 0.152985
    Epoch 136 error rate: 0.148134328425
    Epoch 137 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 56.7704
    Minibatch 0 / 1 error rate: 0.140299
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 48.223
    Minibatch 1 / 0 error rate: 0.149254
    Epoch 137 error rate: 0.144776113331
    Epoch 138 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 49.8639
    Minibatch 0 / 0 error rate: 0.16791
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 55.9265
    Minibatch 1 / 1 error rate: 0.140299
    Epoch 138 error rate: 0.154104471207
    Epoch 139 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 47.2999
    Minibatch 0 / 0 error rate: 0.141791
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 55.4332
    Minibatch 1 / 1 error rate: 0.143284
    Epoch 139 error rate: 0.142537310719
    Epoch 140 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 56.0748
    Minibatch 0 / 1 error rate: 0.140299
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 46.5281
    Minibatch 1 / 0 error rate: 0.134328
    Epoch 140 error rate: 0.137313432992
    Epoch 141 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 48.127
    Minibatch 0 / 0 error rate: 0.145522
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 54.2575
    Minibatch 1 / 1 error rate: 0.131343
    Epoch 141 error rate: 0.138432838023
    Epoch 142 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 54.9123
    Minibatch 0 / 1 error rate: 0.131343
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 45.5468
    Minibatch 1 / 0 error rate: 0.130597
    Epoch 142 error rate: 0.130970150232
    Epoch 143 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 47.0821
    Minibatch 0 / 0 error rate: 0.141791
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 53.1109
    Minibatch 1 / 1 error rate: 0.125373
    Epoch 143 error rate: 0.133582092822
    Epoch 144 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 53.7626
    Minibatch 0 / 1 error rate: 0.128358
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 44.6109
    Minibatch 1 / 0 error rate: 0.130597
    Epoch 144 error rate: 0.129477612674
    Epoch 145 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 52.2001
    Minibatch 0 / 1 error rate: 0.125373
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 44.2147
    Minibatch 1 / 0 error rate: 0.130597
    Epoch 145 error rate: 0.127985075116
    Epoch 146 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 51.5938
    Minibatch 0 / 1 error rate: 0.125373
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 43.8079
    Minibatch 1 / 0 error rate: 0.126866
    Epoch 146 error rate: 0.126119405031
    Epoch 147 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 45.2066
    Minibatch 0 / 0 error rate: 0.13806
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 50.8199
    Minibatch 1 / 1 error rate: 0.125373
    Epoch 147 error rate: 0.131716422737
    Epoch 148 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 51.4782
    Minibatch 0 / 1 error rate: 0.122388
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 42.8649
    Minibatch 1 / 0 error rate: 0.123134
    Epoch 148 error rate: 0.122761193663
    Epoch 149 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 44.1791
    Minibatch 0 / 0 error rate: 0.130597
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 49.8341
    Minibatch 1 / 1 error rate: 0.116418
    Epoch 149 error rate: 0.123507458717
    Epoch 150 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 50.4733
    Minibatch 0 / 1 error rate: 0.119403
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 41.9678
    Minibatch 1 / 0 error rate: 0.115672
    Epoch 150 error rate: 0.11753731221
    Epoch 151 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 43.1771
    Minibatch 0 / 0 error rate: 0.123134
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 48.8935
    Minibatch 1 / 1 error rate: 0.119403
    Epoch 151 error rate: 0.121268656105
    Epoch 152 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 49.4761
    Minibatch 0 / 1 error rate: 0.119403
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 41.1175
    Minibatch 1 / 0 error rate: 0.115672
    Epoch 152 error rate: 0.11753731221
    Epoch 153 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 42.2187
    Minibatch 0 / 0 error rate: 0.119403
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 47.9501
    Minibatch 1 / 1 error rate: 0.119403
    Epoch 153 error rate: 0.119402982295
    Epoch 154 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 40.4421
    Minibatch 0 / 0 error rate: 0.115672
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 47.516
    Minibatch 1 / 1 error rate: 0.119403
    Epoch 154 error rate: 0.11753731221
    Epoch 155 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 48.0486
    Minibatch 0 / 1 error rate: 0.116418
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 39.8703
    Minibatch 1 / 0 error rate: 0.108209
    Epoch 155 error rate: 0.112313430756
    Epoch 156 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 40.8535
    Minibatch 0 / 0 error rate: 0.11194
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 46.4636
    Minibatch 1 / 1 error rate: 0.110448
    Epoch 156 error rate: 0.111194033176
    Epoch 157 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 39.2471
    Minibatch 0 / 0 error rate: 0.108209
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 46.0085
    Minibatch 1 / 1 error rate: 0.107463
    Epoch 157 error rate: 0.107835821807
    Epoch 158 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 38.8318
    Minibatch 0 / 0 error rate: 0.108209
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 45.5396
    Minibatch 1 / 1 error rate: 0.107463
    Epoch 158 error rate: 0.107835821807
    Epoch 159 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 38.4313
    Minibatch 0 / 0 error rate: 0.108209
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 45.0683
    Minibatch 1 / 1 error rate: 0.104478
    Epoch 159 error rate: 0.106343284249
    Epoch 160 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 38.0439
    Minibatch 0 / 0 error rate: 0.104478
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 44.601
    Minibatch 1 / 1 error rate: 0.104478
    Epoch 160 error rate: 0.104477614164
    Epoch 161 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 45.3143
    Minibatch 0 / 1 error rate: 0.107463
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 37.5644
    Minibatch 1 / 0 error rate: 0.100746
    Epoch 161 error rate: 0.104104477912
    Epoch 162 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 38.4683
    Minibatch 0 / 0 error rate: 0.104478
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 43.6048
    Minibatch 1 / 1 error rate: 0.101493
    Epoch 162 error rate: 0.102985076606
    Epoch 163 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 44.35
    Minibatch 0 / 1 error rate: 0.107463
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 36.9077
    Minibatch 1 / 0 error rate: 0.0895522
    Epoch 163 error rate: 0.098507463932
    Epoch 164 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 37.7765
    Minibatch 0 / 0 error rate: 0.0932836
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 42.6985
    Minibatch 1 / 1 error rate: 0.0985075
    Epoch 164 error rate: 0.0958955213428
    Epoch 165 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 43.4398
    Minibatch 0 / 1 error rate: 0.104478
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 36.2402
    Minibatch 1 / 0 error rate: 0.0858209
    Epoch 165 error rate: 0.095149256289
    Epoch 166 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 42.006
    Minibatch 0 / 1 error rate: 0.0955224
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 35.9527
    Minibatch 1 / 0 error rate: 0.0858209
    Epoch 166 error rate: 0.0906716436148
    Epoch 167 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 36.7521
    Minibatch 0 / 0 error rate: 0.0895522
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 41.4075
    Minibatch 1 / 1 error rate: 0.0955224
    Epoch 167 error rate: 0.0925373136997
    Epoch 168 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 42.1208
    Minibatch 0 / 1 error rate: 0.101493
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 35.2686
    Minibatch 1 / 0 error rate: 0.0858209
    Epoch 168 error rate: 0.0936567187309
    Epoch 169 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 36.0119
    Minibatch 0 / 0 error rate: 0.0858209
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 40.6214
    Minibatch 1 / 1 error rate: 0.0955224
    Epoch 169 error rate: 0.0906716436148
    Epoch 170 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 41.3203
    Minibatch 0 / 1 error rate: 0.101493
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 34.6042
    Minibatch 1 / 0 error rate: 0.0783582
    Epoch 170 error rate: 0.0899253748357
    Epoch 171 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 39.9853
    Minibatch 0 / 1 error rate: 0.0925373
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 34.3231
    Minibatch 1 / 0 error rate: 0.0783582
    Epoch 171 error rate: 0.0854477621615
    Epoch 172 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 39.5545
    Minibatch 0 / 1 error rate: 0.0895522
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 34.0364
    Minibatch 1 / 0 error rate: 0.0783582
    Epoch 172 error rate: 0.0839552246034
    Epoch 173 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 34.6713
    Minibatch 0 / 0 error rate: 0.0820896
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 38.989
    Minibatch 1 / 1 error rate: 0.0865672
    Epoch 173 error rate: 0.0843283571303
    Epoch 174 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 39.7128
    Minibatch 0 / 1 error rate: 0.0955224
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 33.376
    Minibatch 1 / 0 error rate: 0.0708955
    Epoch 174 error rate: 0.0832089558244
    Epoch 175 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 38.381
    Minibatch 0 / 1 error rate: 0.0895522
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 33.0976
    Minibatch 1 / 0 error rate: 0.0708955
    Epoch 175 error rate: 0.0802238807082
    Epoch 176 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 33.6622
    Minibatch 0 / 0 error rate: 0.0746269
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 37.8526
    Minibatch 1 / 1 error rate: 0.0835821
    Epoch 176 error rate: 0.079104475677
    Epoch 177 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 38.6074
    Minibatch 0 / 1 error rate: 0.0895522
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 32.4604
    Minibatch 1 / 0 error rate: 0.0708955
    Epoch 177 error rate: 0.0802238807082
    Epoch 178 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 37.2777
    Minibatch 0 / 1 error rate: 0.0835821
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 32.193
    Minibatch 1 / 0 error rate: 0.0708955
    Epoch 178 error rate: 0.0772388055921
    Epoch 179 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 36.8947
    Minibatch 0 / 1 error rate: 0.0835821
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 31.9226
    Minibatch 1 / 0 error rate: 0.0708955
    Epoch 179 error rate: 0.0772388055921
    Epoch 180 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 36.5194
    Minibatch 0 / 1 error rate: 0.0776119
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 31.6497
    Minibatch 1 / 0 error rate: 0.0671642
    Epoch 180 error rate: 0.0723880603909
    Epoch 181 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 36.1528
    Minibatch 0 / 1 error rate: 0.0746269
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 31.3749
    Minibatch 1 / 0 error rate: 0.0671642
    Epoch 181 error rate: 0.0708955228329
    Epoch 182 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 35.7959
    Minibatch 0 / 1 error rate: 0.0686567
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 31.0991
    Minibatch 1 / 0 error rate: 0.0671642
    Epoch 182 error rate: 0.0679104477167
    Epoch 183 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 35.4484
    Minibatch 0 / 1 error rate: 0.0686567
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 30.8232
    Minibatch 1 / 0 error rate: 0.0671642
    Epoch 183 error rate: 0.0679104477167
    Epoch 184 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 35.1097
    Minibatch 0 / 1 error rate: 0.0686567
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 30.548
    Minibatch 1 / 0 error rate: 0.0671642
    Epoch 184 error rate: 0.0679104477167
    Epoch 185 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 31.0552
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 34.6792
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 185 error rate: 0.0679104477167
    Epoch 186 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 30.0452
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 34.4228
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 186 error rate: 0.0679104477167
    Epoch 187 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 29.723
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 34.1534
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 187 error rate: 0.0679104477167
    Epoch 188 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 35.0012
    Minibatch 0 / 1 error rate: 0.0746269
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 29.3311
    Minibatch 1 / 0 error rate: 0.0671642
    Epoch 188 error rate: 0.0708955228329
    Epoch 189 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 29.835
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 33.5194
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 189 error rate: 0.0679104477167
    Epoch 190 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 28.9151
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 33.2403
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 190 error rate: 0.0679104477167
    Epoch 191 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 28.6415
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 32.9534
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 191 error rate: 0.0679104477167
    Epoch 192 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 28.3787
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 32.6618
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 192 error rate: 0.0679104477167
    Epoch 193 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 28.125
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 32.3682
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 193 error rate: 0.0679104477167
    Epoch 194 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 27.8788
    Minibatch 0 / 0 error rate: 0.0671642
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 32.0748
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 194 error rate: 0.0679104477167
    Epoch 195 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 32.9447
    Minibatch 0 / 1 error rate: 0.0746269
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 27.5741
    Minibatch 1 / 0 error rate: 0.0634328
    Epoch 195 error rate: 0.0690298490226
    Epoch 196 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 28.0762
    Minibatch 0 / 0 error rate: 0.0634328
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 31.4427
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 196 error rate: 0.0660447739065
    Epoch 197 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 32.3096
    Minibatch 0 / 1 error rate: 0.0746269
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 27.1546
    Minibatch 1 / 0 error rate: 0.0634328
    Epoch 197 error rate: 0.0690298490226
    Epoch 198 ...
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 0 / 1 loss: 30.9647
    Minibatch 0 / 1 error rate: 0.0686567
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 1 / 0 loss: 26.9722
    Minibatch 1 / 0 error rate: 0.0634328
    Epoch 198 error rate: 0.0660447739065
    Epoch 199 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 27.4377
    Minibatch 0 / 0 error rate: 0.0634328
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 30.5643
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 199 error rate: 0.0660447739065
    Epoch 200 ...
    [ 0  1  2  3  4  5  8  9 10 12 14 15 17 19 21 22 25 26 27]
    Minibatch 0 / 0 loss: 26.5984
    Minibatch 0 / 0 error rate: 0.0634328
    [ 0  1  2  4  5  7  8  9 10 11 12 14 15 17 18 19 21 22 23 25 26 27]
    Minibatch 1 / 1 loss: 30.3287
    Minibatch 1 / 1 error rate: 0.0686567
    Epoch 200 error rate: 0.0660447739065

```python
logits_out.shape
```

    (423, 4, 28)

```python
import matplotlib.pyplot as plt
```

```python
# Lets visualize the prediction for the first training sequence
np.argmax(logits_out[:,0,:],axis=1)
```

    array([27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 10, 27, 26, 26, 26, 27, 27, 27,
           27,  9, 12, 19, 27, 27, 27, 27, 27,  0, 27, 26, 22, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 25, 27, 27, 27, 27, 27, 27, 27, 27, 15, 12, 27,  0,  8, 26, 26,
           26, 26, 26, 17, 26, 26, 14, 12,  2, 27, 27,  0, 27, 27, 27, 12, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 21,  4,  0, 15, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27,  5, 22, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 15, 19,
           25, 25,  4, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27,  0, 27, 27, 27, 27, 27, 27, 15, 26, 14, 12, 27,
           27, 27, 27, 27, 27, 27, 27, 26,  1,  0,  1,  4, 25, 26,  1,  0, 27,
           27, 15,  9, 22, 11,  7,  0, 19, 27, 14, 10,  0,  0, 11, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 14, 27, 14, 27, 27, 12, 19, 22, 14,  0,  0, 18,  9, 22, 25,  5,
            5, 12,  0,  0,  9, 26, 27, 27, 17, 27, 27, 27, 27, 27, 27, 26, 26,
            9,  0,  0, 12, 21, 21,  4,  0,  0, 14, 27, 27, 27, 27, 27, 14, 26,
            1,  0,  0, 23, 26, 26,  1,  4, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
           27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27])

```python
%matplotlib notebook
plt.plot(logits_out[:,0,27],'k-',label='blank')
plt.plot(logits_out[:,0,10],'b-',label='Index 10')
plt.plot(logits_out[:,0,9],'r-',label='Index 9')
plt.plot(logits_out[:,0,12],'g-',label='Index 12')
plt.legend(loc=4)
plt.xlabel('Time Steps')
plt.ylabel('Output Logits')
```

    <IPython.core.display.Javascript object>

    <matplotlib.text.Text at 0x12aa610f0>

More about the spykes that tend to apear at the RNN output when using CTC

[Supervised Sequence Labelling](https://www.cs.toronto.edu/~graves/preprint.pdf)

```python

```


Una vez resuelta la ecuación de Laplace en coordenadas esféricas, por el método de separación de variables:
$$\nabla ^2 V(r, \theta, \phi) = 0$$
se obtienen los siguientes modos normales:
$$ V_{lm}(r, \theta, \phi) = \left( D_{lm} r^l + E_{lm} \frac{1}{r ^{-(l+1)}} \right) Y_l ^m(\theta, \phi) $$
Escritos también de la forma:
$$ V_{lm}(r, \theta, \phi) = \left( D_{lm} r^l + E_{lm} \frac{1}{r ^{-(l+1)}} \right) e^{im\phi} P_l ^m (cos \theta). $$

Una vez evaluada las condiciones de frontera, es posible corroborar qué:
$$ V(r, \theta, \phi) = \sum_ {l= 0} ^ \infty \sum_{-l} ^{l} \left( D_{lm} r^l \right) e^{im\phi} P_l ^m (cos \theta), ~~ l + m = 2k + 1 $$
Donde la solución divergente en el origen ha sido removida para garantizar que el potencial sea finito en el punto (0,0,0) contenido dentro del plano z = 0. La condición en la frontera de $z = 0 \rightarrow V(r, \theta, \phi) = 0$, restringe los posibles valores de $l$ y $m$ aquellos tales que $l + m = 2k + 1$.

Imponiendo la siguiente condición a la frontera,
$$ \sum_ {l= 0} ^ \infty \sum_{-l} ^{l} \left( D_{lm} a^l \right)  Y_l ^m(\theta, \phi) = f(\theta, \phi) $$
Donde:
$$ f(\theta, \phi) = \left\{ \begin{array}{cc}
V_0 sin{\frac{\phi}{2}} ~ cos^2 \theta ~ sin{\frac{\phi}{2}} & 0 \leq \theta \leq \frac{\pi}{2} \\
0 & \frac{\pi}{2} \leq \theta \leq \pi
\end{array}  \right. $$
Multiplicando por $\hat{Y}_{l'} ^{m'}$:
$$ \sum_ {l= 0} ^ \infty \sum_{-l} ^{l} \left( D_{lm} a^l \right)  \hat{Y}_{l'} ^{m'} Y_l ^m = f(\theta, \phi) ~\hat{Y}_{l'} ^{m'}$$
Integrando:
$$ \sum_ {l= 0} ^ \infty \sum_{-l} ^{l} \left( D_{lm} a^l \right) \int_0 ^\pi \int_0 ^{2\pi}  \hat{Y}_{l'} ^{m'} Y_l ^m ~sin\theta d\theta d\phi= V_0 \int_0 ^{\frac{\pi}{2}} \int_0 ^{2\pi} sin{\frac{\phi}{2}} ~ cos^2 \theta ~\hat{Y}_{l'} ^{m'} ~ sin\theta d\theta d\phi + \int_\frac{\pi}{2} ^\pi \int_0 ^{2\pi} 0 ~sin\theta d\theta d\phi$$
Como:
$$ \int_0 ^\pi \int_0 ^{2\pi}\hat{Y}_{l'} ^{m'} Y_l ^m ~sin\theta d\theta d\phi = \delta_{l, l'} \delta_{m, m'} $$

$$ D_{lm} a^l = V_0 \int_0 ^{\frac{\pi}{2}} \int_0 ^{2\pi} sin{\frac{\phi}{2}} ~ cos^2 \theta ~\hat{Y}_{l'} ^{m'} ~ sin\theta d\theta d\phi $$

Y siendo $\hat{Y}_l ^m (\theta, \phi) = N ~e^{-im\phi} P_l ^m (cos \theta)$, con $N = (-1)^m \sqrt{\frac{(2l + 1)(l-m)!}{4\pi (l+m)!}} $;

$$ D_{lm} a^l = V_0 N \int_0 ^{\frac{\pi}{2}} \int_0 ^{2\pi} sin{\frac{\phi}{2}} ~ cos^2 \theta ~e^{-im\phi} P_l ^m (cos \theta) ~ sin\theta d\theta d\phi $$
Separando las componentes en $\theta$ y $\phi$:
$$ D_{lm} = \frac{V_0 N}{a^l} \left[ \int_0 ^{\frac{\pi}{2}} cos^2 \theta ~sin\theta ~P_l ^m (cos \theta) d\theta \right] \left[\int_0 ^{2\pi}sin{\frac{\phi}{2}} ~ e^{-im\phi}   d\phi \right]     $$

Con el cambio de variable: $x = cos\theta$, $- sin \theta d\theta = dx$.
\begin{equation}
 D_{lm} = \frac{V_0 N}{a^l} \left[ -\int_1 ^0 x^2 ~P_l ^m (x) dx \right] \left[\int_0 ^{2\pi}sin{\frac{\phi}{2}} ~ e^{-im\phi}   d\phi \right]
\end{equation}
Concentrándonos en la segunda integral, recordando que $sin \alpha = \frac{e^{i \alpha} - e ^{-i \alpha}}{2i} $;
$$
\int sin{\frac{\phi}{2}} ~ e^{-im\phi}   d\phi   = \int \left( \frac{e^{i \frac{\phi}{2}} - e^{-i \frac{\phi}{2}}      }{2i} \right) e ^{-i m \phi} d\phi $$

$$ = \frac{1}{2i} \int \left(    e^{i\frac{\phi}{2}} e ^{-im\phi} - e^{-i \frac{\phi}{2}} e ^{-im\phi} \right)d\phi
 $$

$$ = \frac{1}{2i} \int \left(  e ^{i \phi \left( \frac{1}{2} - m  \right) } - e ^{- i \phi \left( \frac{1}{2} + m  \right)} \right)  d\phi $$
Separando la integral y efectuando los cambios de variable pertinentes:

$$ = \left. \frac{1}{2} \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) } e ^{- i \phi \left( \frac{1}{2} + m  \right)}  - \frac{1}{ \left( \frac{1}{2} - m  \right) } e ^{i \phi \left( \frac{1}{2} - m  \right)}  \right\} \right|_0 ^{2\pi}$$

$$ = \frac{1}{2} \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) } e ^{- i \pi \left( 1 + 2m  \right)}  - \frac{1}{ \left( \frac{1}{2} - m  \right) } e ^{i \pi \left( 1 - 2m  \right)}  \right\}  - \frac{1}{2} \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) }  - \frac{1}{ \left( \frac{1}{2} - m  \right) }   \right\}$$

Usando la fórmula de Euler:

$$e ^{i  \pi \left( 1 +2 m  \right) }  = cos ((2m +1) \pi) + i ~sin ((2m+1) \pi)$$

Sabemos que $m$ es un entero, por lo tanto $(2m+1)\pi$ es un múltiplo entero \emph{impar} de $\pi$ y $sin ((2m+1) \pi) = 0, \forall m$. Con el mismo argumento, $cos((2m+1)\pi) = -1$.

Análogamente:
$$e ^{i  \pi \left( 1 -2 m  \right) }  = cos ((1 - 2m) \pi) - i ~sin ((1 - 2m) \pi)$$
$$ = cos((2m-1) \pi) + i ~sin ((2m-1)\pi)) $$
Y por razones idénticas a las expuestas con anterioridad: $sin((2m-1)\pi) = 0$ y $cos((2m-1)\pi) = -1$, $\forall m$.

Hecho esto:
$$ = \frac{1}{2} \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) } (-1)  - \frac{1}{ \left( \frac{1}{2} - m  \right) } (-1)  \right\}  - \frac{1}{2} \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) }  - \frac{1}{ \left( \frac{1}{2} - m  \right) }   \right\}$$
$$ =  - \frac{1}{2} \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) }   - \frac{1}{ \left( \frac{1}{2} - m  \right) }  \right\}  - \frac{1}{2} \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) }  - \frac{1}{ \left( \frac{1}{2} - m  \right) }   \right\}$$
$$ = -  \left\{  \frac{1}{ \left( \frac{1}{2} + m  \right) }   - \frac{1}{ \left( \frac{1}{2} - m  \right) }  \right\} $$

Y la expresión para los $D_{lm}$ se convierte en:
$$  D_{lm} = \left\{ (-1)^m \sqrt{\frac{(2l + 1)(l-m)!}{4\pi (l+m)!}} \frac{V_0}{a^l} \left[\frac{1}{ \left( \frac{1}{2} - m  \right) } -  \frac{1}{ \left( \frac{1}{2} + m  \right) }    \right] \right\}  \int_0 ^1 x^2 ~P_l ^m (x) dx    $$

Definiendo $M_{lm}$:
$$ M_{lm} = (-1)^m \sqrt{\frac{(2l + 1)(l-m)!}{4\pi (l+m)!}} \frac{V_0}{a^l} \left[\frac{1}{ \left( \frac{1}{2} - m  \right) } -  \frac{1}{ \left( \frac{1}{2} + m  \right) }    \right] $$

Es necesario notar que, para valores de $m = 0 $, los coeficientes $M_{lm} = 0$.

```
import numpy as np
from matplotlib import pyplot as plt
from matplotlib import cm, colors
import scipy as spy
import scipy.integrate as integrate
from types import *
from mpl_toolkits.mplot3d import axes3d, Axes3D
import matplotlib.cm as cm

%matplotlib inline
```

Introducimos los valores iniciales del problema. Por simplicidad, eligiremos una esfera unitaria y un potencial eléctrico de 5 volts.

```
a = 1.   #Radio de la Esfera
V0 = 5.  #Potencial eléctrico V0
```

Definimos una clase que permitirá efectuar operaciones con el grado y el orden de los polinomios con mayor eficiacia.

```
class Grado_Pol:

    def __init__ (self, l):
        if float(l).is_integer():                     # Comprueba que el grado sea un entero
            self.Grado = l
        else:
            print("El grado es invalido")

    def Orden (self):
        lista = []
        Emes = np.arange(-self.Grado, self.Grado +1)
        for m in Emes:
            if (self.Grado + m)%2 != 0:               # Elige unicamente los valores de m que satisfacen l+m = impar
                lista.append(m)

        return lista                                 # Devuelve una lista de valores posibles

    def No_Orden (self):
        return len(self.Orden())                      # Indica la cardinalidad de la lista anterior

```

A continuación, las funciones especiales que serán necesarias: Los polinomios asociados de legendre y los armonicos esferico, de los que únicamente será posible retomar la parte real.

```
def Legendre (x, grado, orden):
        return spy.special.lpmv(orden, grado, x)
def Ylm (r, th, phi, grado, orden):
    return spy.special.sph_harm(orden, grado, phi, th)
```

Definimos los coeficientes $D_{lm}$.
La integral $\int_0 ^1 x^2 ~P_l ^m (x) dx $ será realizada con la libreria scypy.integrate.quad

```
def Clm (l, m):                                  # El coeficiente Clm es un auxiliar, permite conocer informacion util

    def Mlm (l, m):
        return ((-1.)**m)*np.sqrt(((2.*l + 1)*np.math.factorial(l-m))/(4*np.pi*np.math.factorial(l+m)))*(V0 / (a**l))*((1. / (0.5 - m)) - (1. / (0.5 + m)))
    integral = integrate.quad(Legendre, 1., 0., args=(l,m))

    return Mlm(l,m)*integral[0], integral[1], Mlm(l,m)
    # Clm[0] = el coeficiente buscado
    # Clm[1] = error del metodo quad de scipy
    # Clm[2] = Valor de Mlm, para ver si efectivamente es cero.

def Dlm (l,m):
    return Clm(l,m)[0]

```

Para la funcion potencial, es necesario realizar la "suma infinita" sobre los grados $l$ del polinomio. Por simplicidad, la variable de grado a sido renombrada como *Rango*, y puede manipularse para tomar una *mejor aproximacion*. Como primera propuesta, para ahorrar recursos de computo, ha sido designada con un valor predeterminado de 4.

```
def V(r, th, phi, Rango=4):
    V = 0.                           #Fuera del ciclo. Queremos sumar las todas las m's, sobre todas las i's.
    for i in range (1,Rango+1):
        l = Grado_Pol(i)
        if not l.Orden():
            print("El grado ", l.Grado, " no es valido")
        else:
            #print("Para el grado ", i, " hay ", l.No_Orden(), " ordenes validos.")
            for m in l.Orden():
                D = Dlm(i,m)
                if D < 1e-6 :
                    D = 0
                V = V + D*Ylm(r, th,phi, i, m)
                #print (D, "  l = ", i, " m = ", m, "  Error = ", Clm(i,m)[1], "Mlm = ", Clm(i,m)[2] )

    return V

```

La funcion a graficar es, entonces, una funcion de 4 variables: tres coordenadas espaciales (en coordenadas esfericas), y una cuarta coordenada que representa el valor del potencial $V(r, \theta, \phi)$. Para visualizar una imagen, la variable $r$ será fijada como parámetro. La informacion del potencial sera incluida como un mapa de colores por medio de las librerias matplotlib.cm y matplotlib.colors, e incorporada a una plot_surface de la función $f(r, \theta, \phi)$ como una máscara de color, para cada punto.

Considerando únicamente la componente real de V y mapeando los puntos para Th y Ph obtenidos (un arreglo de N por N. De incluir la variable r sería de N x N x N):

```
"""
color_map= np.real(V(a,Th,Ph,5))
norm = colors.Normalize()
color = norm(color_map)

colorbar = cm.ScalarMappable(cmap = cm.jet)
colorbar.set_array(color_map)
"""
```

    '\ncolor_map= np.real(V(a,Th,Ph,5))\nnorm = colors.Normalize()\ncolor = norm(color_map)\n\ncolorbar = cm.ScalarMappable(cmap = cm.jet)\ncolorbar.set_array(color_map)\n'

Se hace necesario crear una malla de puntos para:
$$ 0 \leq \theta \leq \pi/2 ~; ~ 0 \leq \phi \leq \pi$$

```
#Estos espacios son necesarios para la creacion de una malla de C x C puntos
C = 100
```

```
rr = np.linspace(0, a, C)
tt = np.linspace(0, (np.pi)/2. , C)
pp = np.linspace(0, 2*(np.pi), C)
RR,TT, PP = np.meshgrid(rr, tt, pp)
Th,Ph = np.meshgrid( tt, pp)
```

Incluyendo el mapa de colores:

```
"""
** Esto tambien, necesito que corra para diferentes colores del radio

color_map= np.real(V(R,Th,Ph,5))
norm = colors.Normalize()
color = norm(color_map)

colorbar = cm.ScalarMappable(cmap = cm.jet)
colorbar.set_array(color_map)
"""
```

    '\n** Esto tambien, necesito que corra para diferentes colores del radio\n\ncolor_map= np.real(V(R,Th,Ph,5))\nnorm = colors.Normalize()\ncolor = norm(color_map)\n\ncolorbar = cm.ScalarMappable(cmap = cm.jet)\ncolorbar.set_array(color_map)\n'

```
def graficas (fig  , N=1, r = a, th=Th, phi=Ph, Rango=4, func=V, tag="$V$"):

    contador = 0.
    for i in range(1,N+1):

        R = i*(r/N)
        progreso = (25./N)

        contador += progreso
        print(contador, "%")
        X = R*np.sin(Th)*np.cos(Ph)
        Y = R*np.sin(Th)*np.sin(Ph)
        Z = R*np.cos(Th)

        if type(func) == type(th):
            color_map= np.real(func)
        else:
            color_map= np.real(func(R,th,phi,Rango))

        #color_map= np.real(func(R,th,phi,Rango))
        norm = colors.Normalize()
        color = norm(color_map)

        contador += progreso
        print(contador, "%")

        colorbar = cm.ScalarMappable(cmap = cm.jet)
        colorbar.set_array(color_map)

        if N != 2:
            ax = fig.add_subplot(np.ceil(N/2),np.ceil(N/2), i, projection="3d")
        else:
            ax = fig.add_subplot(N,1, i, projection="3d")

        contador+= progreso
        print(contador, "%")

        ax.set_xlim(-1,1)
        ax.set_ylim(-1,1)
        ax.set_zlim(-1,1)
        ax.set_title(r"Grafica de %s $(r, \theta, \phi)$ para $r = %2.2f$ "%(tag,R))
        ax.plot_surface(X, Y, Z,  rstride=1, cstride=1, facecolors=cm.jet(color))

        contador += progreso
        print(contador, "%")

        plt.colorbar(colorbar, shrink=0.7)

    print ("Puntos graficados: ",Th[0].size)
    print("Grado maximo del polinimio: ", Rango)
    print ("**************************************")
```

El campo eléctrico $\vec{E}$ se define como:
$$ - \vec{\nabla} V = \vec{E}$$
Donde el operador gradiente debe incluirse en coordenadas esfericas, siendo:
$$\vec{\nabla} = \frac{\partial}{\partial r} \hat{r} + \frac{1}{r} \frac{\partial}{\partial \theta} \hat{\theta} + \frac{1}{r~sin\theta} \frac{\partial}{\partial \phi} \hat{\phi}$$
Implementando lo anterior en una funcion, haciendo cortes para diferentes radios:

```
def E (R, TT, PP ,Rango=4):
    et,ep = np.gradient(np.real(V(R,TT,PP, Rango=Rango)))

    Et = (-1./(a*np.ones_like(TT)))*et                 #Factores de escala
    Ep = (-1./a*np.sin(TT))*ep
    return Et, Ep
```

Para hacer cortes en la coordenada radial, será necesario tomar $r =$ constante. Así, serán consideradas solo las componentes $E_{\theta}$ y $E_{\phi}$.

```
E_theta = E(a,Th,Ph)[0]
E_phi = E(a,Th,Ph)[1]
```

Graficando para  $E_{\theta}$, para $r = a$:

```
figT= plt.figure(121)
graficas(figT, func=E_theta, Rango= 3, tag="$Et$")
```

Analogamente para   $E_{\phi}$, para $r = a$:

```
figP= plt.figure(122)
graficas(figP, func=E_phi, Rango= 5, tag="$Ep$")
```

A continuación, una inocua funcion para guardar imagenes .png y hacer un gif despues.

```
def guardar (cuadros = 10, funcion=E_phi, archivo="Ep"):
    for i in range(1,cuadros+1):
        t = 1 - (i/float(cuadros))
        r = a - t
        fig=plt.figure()
        graficas(fig, r = (a-t), func=funcion, tag="$E$")
        fig.savefig(archivo+"r=%1.2f_%d"%(r,(i-1))+".png")
        print(" --------->  Guardada %s"%(archivo+"_r=%d_%i"%(r,(i-1)) ))
        fig.clf
        plt.close
```

```
guardar()
```

```

```