Consider a feedforward neural network with three inputs, two units in the first hidden layer, two units in the second hidden layer, and three units in the output layer. The activation function for hidden layer 1 is ReLU, for hidden layer 2 is sigmoid, and for the output layer is softmax.

The initial weights are given by the matrices
$W^{(1)} =\begin{pmatrix}0.1&0.3&0.7\\0.9& 0.4& 0.4\end{pmatrix}$ , $W^{(2)} =\begin{pmatrix}0.4&0.3\\0.7&0.2\end{pmatrix}$, $W^{(3)} =\begin{pmatrix}0.5&0.6\\0.6& 0.7\\0.3& 0.2\end{pmatrix}$,
and all the biases are unit vectors.

Assuming that the input $\begin{pmatrix}0.1\\0.7\\0.3\end{pmatrix}$
corresponds to the output $\begin{pmatrix}1\\0\\0\end{pmatrix}$ , manually compute the updated weights and biases after a single epoch (forward + backward pass), clearly stating all derivatives that you have used. You should use a learning rate of 1.

As a practical exercise, you should modify the implementation of a stochastic gradient descent routine in the back-propagation Python notebook. ML_in_Finance-Backpropagation.ipynb. Note that the notebook example corresponds to the example in Chapter 4 of the textbook (and shown in class 1), which uses sigmoid activated hidden layers only. Compare the weights and biases obtained by TensorFlow (or your ANN library of choice) with those obtained by your procedure after 200 epoch

Solution:

```python
import numpy as np
from numpy.linalg import norm

import copy
import os

def relu(x):
  return x*(np.sign(x)+1.)/2.

def sigmoid(x):
  return 1./(1.+np.exp(-x))

def softmax(x):
  return np.exp(x)/sum(np.exp(x))
```

```python
W0_1 = np.array([[0.1,0.3,0.7], [0.9,0.4,0.4]])
b_1 = np.array([1.,1.])

W0_2 = np.array([[0.4,0.3], [0.7,0.2]])
b_2 = np.array([1.,1.])

W0_3 = np.array([[0.5,0.6], [0.6,0.7], [0.3,0.2]])
b_3 = np.array([1.,1.,1.])
```

```python
xtrain = np.array([0.1,0.7,0.3])
ytrain = np.array([1,0,0])
```

```python
##Forward Pass
#HL1 ReLU
Z1 = relu(W0_1 @ xtrain+b_1)
#print(Z1)
#HL2 Sigmoid
Z2 = sigmoid(W0_2 @ Z1+b_2)
#print(Z2)
#OL Softmax
ypred = softmax(W0_3 @ Z2+b_3)
print(ypred)
print("Loss: ",-ytrain@np.log(ypred))
```

    [0.35985276 0.43045902 0.20968822]
    Loss:  1.0220603290496275

$\begin{align}
  \frac{\partial\mathcal{L}}{\partial b^{(3)}_{k}}&=\sum_{i}\frac{\partial\mathcal{L}}{\partial Z^{(3)}_{i}}\frac{\partial Z^{(3)}_{i}}{\partial b^{(3)}_{k}}\\
  &=\sum_{i}\sum_{j}\frac{\partial\mathcal{L}}{\partial Z^{(3)}_{i}}\frac{\partial Z^{(3)}_{i}}{\partial I^{(3)}_{j}}\frac{\partial I^{(3)}_{j}}{\partial b^{(3)}_{k}}\\
  &=\sum_{i}\sum_{j}-\frac{Y_{i}}{Z^{(3)}_{i}}\left(Z_{i}^{(3)}(\delta_{ij}-Z^{(3)}_{j})\right)(\delta_{jk})\\
  &=Z^{(3)}_{k}-Y_{k}\\
  \\
  \frac{\partial\mathcal{L}}{\partial w^{(3)}_{ij}}&=\sum_{k}\frac{\partial\mathcal{L}}{\partial Z^{(3)}_{k}}\frac{\partial Z^{(3)}_{k}}{\partial w^{(3)}_{ij}}\\
  &=\sum_{k}\sum_{l}\frac{\partial\mathcal{L}}{\partial Z^{(3)}_{k}}\frac{\partial Z^{(3)}_{k}}{\partial I^{(3)}_{l}}\frac{\partial I^{(3)}_{l}}{\partial w^{(3)}_{ij}}\\
  &=\sum_{k}\sum_{l}-\frac{Y_{k}}{Z^{(3)}_{k}}\left(Z_{k}^{(3)}(\delta_{kl}-Z^{(3)}_{l})\right)(\delta_{il}Z_{j}^{(2)})\\
  &=\sum_{k}\sum_{l}-Y_{k}\left(\delta_{kl}-Z^{(3)}_{l}\right)(\delta_{il}Z_{j}^{(2)})\\
  &=Z_{j}^{(2)}\sum_{k}-Y_{k}\left(\delta_{ik}-Z^{(3)}_{i}\right)\\
  &=Z_{j}^{(2)}(Z_{i}^{(3)}-Y_{i})\\
  \\
\frac{\partial\mathcal{L}}{\partial Z^{(2)}_{i}}&=\sum_{j}\frac{\partial\mathcal{L}}{\partial Z^{(3)}_{j}}\frac{\partial Z^{(3)}_{j}}{\partial Z^{(2)}_{i}}\\
  &=\sum_{j}\sum_{k}\frac{\partial\mathcal{L}}{\partial Z^{(3)}_{j}}\frac{\partial Z^{(3)}_{j}}{\partial I^{(3)}_{k}}\frac{\partial I^{(3)}_{k}}{\partial Z^{(2)}_{i}}\\
  &=\sum_{j}\sum_{k}-\frac{Y_{j}}{Z^{(3)}_{j}}\left(Z_{j}^{(3)}(\delta_{jk}-Z^{(3)}_{k})\right)(w^{(3)}_{ki})\\
  &=\sum_{j}\sum_{k}-\frac{Y_{j}}{Z^{(3)}_{j}}\left(Z_{j}^{(3)}(\delta_{jk}-Z^{(3)}_{k})\right)(w^{(3)}_{ki})\\
\end{align}$

so we have

$\begin{align}
\nabla_{b^{(3)}}\mathcal{L}&=\delta^{(3)}=Z^{(3)}-Y\\
\nabla_{W^{(3)}}\mathcal{L}&=\delta^{(3)}\otimes Z^{(2)}\\
\nabla_{b^{(2)}}\mathcal{L}&=\delta^{(2)}=(\nabla_{I^{(2)}}\sigma^{(2)})(W^{(3)})^{\intercal}\delta^{(3)}\\
&=Z^{(2)}\circ(1-Z^{(2)})\circ(W^{(3)})^{\intercal}\delta^{(3)}\\
\nabla_{W^{(2)}}\mathcal{L}&=\delta^{(2)}\otimes Z^{(1)}\\
\nabla_{b^{(1)}}\mathcal{L}&=\delta^{(1)}=(\nabla_{I^{(1)}}\sigma^{(1)})(W^{(2)})^{\intercal}\delta^{(2)}\\
&=H(I^{(1)})\circ(W^{(2)})^{\intercal}\delta^{(2)}\\
\nabla_{W^{(2)}}\mathcal{L}&=\delta^{(1)}\otimes X
\end{align}
$

```python
##Backward pass
delta3 = ypred-ytrain
dLdW3 = np.outer(delta3,Z2)
print(delta3,dLdW3)
delta2 = Z2*(1-Z2)*(W0_3.T @ delta3)
dLdW2 = np.outer(delta2,Z1)
print(delta2,dLdW2)
delta1 = np.heaviside(W0_1@xtrain+b_1,1)*(W0_2.T @ delta2)
dLdW1 = np.outer(delta1,xtrain)
print(delta1,dLdW1)

b_3-=delta3
W0_3-=dLdW3
b_2-=delta2
W0_2-=dLdW2
b_1-=delta1
W0_1-=dLdW1

print("Updated Weights and Biases")

print(W0_1,b_1)
print(W0_2,b_2)
print(W0_3,b_3)
```

    [-0.64014724  0.43045902  0.20968822] [[-0.56510762 -0.58176209]
     [ 0.37999957  0.39119865]
     [ 0.18510806  0.19056344]]
    [ 0.00011468 -0.00338424] [[ 0.000164    0.00017088]
     [-0.00483946 -0.00504251]]
    [-0.00232309 -0.00064244] [[-2.32309311e-04 -1.62616517e-03 -6.96927932e-04]
     [-6.42442430e-05 -4.49709701e-04 -1.92732729e-04]]
    Updated Weights and Biases
    [[0.10023231 0.30162617 0.70069693]
     [0.90006424 0.40044971 0.40019273]] [1.00232309 1.00064244]
    [[0.399836   0.29982912]
     [0.70483946 0.20504251]] [0.99988532 1.00338424]
    [[1.06510762 1.18176209]
     [0.22000043 0.30880135]
     [0.11489194 0.00943656]] [1.64014724 0.56954098 0.79031178]

```python
k=1
while(k<201):
  k+=1
  print("Iteration: ", k)
  ##Forward Pass
  #HL1 ReLU
  Z1 = relu(W0_1 @ xtrain+b_1)
  #print(Z1)
  #HL2 Sigmoid
  Z2 = sigmoid(W0_2 @ Z1+b_2)
  #print(Z2)
  #OL Softmax
  ypred = softmax(W0_3 @ Z2+b_3)
  print(ypred)
  print("Loss: ",-ytrain@np.log(ypred))

  ##Backward pass
  delta3 = ypred-ytrain
  dLdW3 = np.outer(delta3,Z2)
  #print(delta3,dLdW3)
  delta2 = Z2*(1-Z2)*(W0_3.T @ delta3)
  dLdW2 = np.outer(delta2,Z1)
  #print(delta2,dLdW2)
  delta1 = np.heaviside(W0_1@xtrain+b_1,1)*(W0_2.T @ delta2)
  dLdW1 = np.outer(delta1,xtrain)
  #print(delta1,dLdW1)

  b_3-=delta3
  W0_3-=dLdW3
  b_2-=delta2
  W0_2-=dLdW2
  b_1-=delta1
  W0_1-=dLdW1

  print("Updated Weights and Biases")

  print(W0_1,b_1)
  print(W0_2,b_2)
  print(W0_3,b_3)
```

    Iteration:  2
    [0.87954341 0.06457517 0.05588142]
    Loss:  0.12835236020870527
    Updated Weights and Biases
    [[0.10137754 0.30964278 0.70413262]
     [0.90060169 0.40421181 0.40180506]] [1.0137754  1.00601688]
    [[0.41579591 0.3164272 ]
     [0.71908065 0.21985315]] [1.01101734 1.01331745]
    [[ 1.17145869  1.29143659]
     [ 0.16298704  0.25000631]
     [ 0.06555426 -0.0414429 ]] [1.76060383 0.50496581 0.73443036]
    Iteration:  3
    [0.92228519 0.04126648 0.03644833]
    Loss:  0.08090078793434305
    Updated Weights and Biases
    [[0.10221891 0.31553238 0.70665673]
     [0.90101094 0.40707657 0.40303281]] [1.02218911 1.01010938]
    [[0.42745461 0.32846864]
     [0.72932741 0.2304363 ]] [1.01904728 1.02037493]
    [[ 1.24061327  1.36261042]
     [ 0.12626604  0.21221308]
     [ 0.03312069 -0.0748235 ]] [1.83831864 0.46369933 0.69798203]
    Iteration:  4
    [0.94220177 0.03053017 0.02726806]
    Loss:  0.05953583479959617
    Updated Weights and Biases
    [[0.10288945 0.32022618 0.70866836]
     [0.90134475 0.40941325 0.40403425]] [1.02889454 1.0134475 ]
    [[0.43669346 0.3379647 ]
     [0.73738435 0.23871755]] [1.02535246 1.02587349]
    [[ 1.29232758  1.41575814]
     [ 0.09894951  0.1841394 ]
     [ 0.0087229  -0.09989754]] [1.89611687 0.43316916 0.67071397]
    Iteration:  5
    [0.95388593 0.0242723  0.02184177]
    Loss:  0.04721118634517901
    Updated Weights and Biases
    [[0.10344909 0.32414365 0.71034728]
     [0.90162812 0.41139685 0.40488436]] [1.03449092 1.01628121]
    [[0.44436604 0.3458215 ]
     [0.74404218 0.24553524]] [1.03055088 1.03038439]
    [[ 1.33376113  1.45829241]
     [ 0.07714083  0.16175134]
     [-0.01090196 -0.12004376]] [1.94223094 0.40889686 0.6488722 ]
    Iteration:  6
    [0.9616053  0.02015607 0.01823863]
    Loss:  0.03915120608280936
    Updated Weights and Biases
    [[0.10393039 0.32751273 0.71179117]
     [0.90187508 0.41312558 0.40562525]] [1.0393039  1.01875083]
    [[0.45093613 0.35252897]
     [0.74972329 0.25133513]] [1.03497566 1.03421046]
    [[ 1.36837595  1.49379434]
     [ 0.05896908  0.14311389]
     [-0.02734503 -0.13690824]] [1.98062565 0.38874079 0.63063357]
    Iteration:  7
    [0.96709741 0.01723711 0.01566548]
    Loss:  0.03345605316228914
    Updated Weights and Biases
    [[0.10435317 0.33047219 0.71305951]
     [0.90209438 0.41466068 0.40628315]] [1.0435317  1.02094383]
    [[0.45668555 0.35838364]
     [0.75468157 0.25638418]] [1.03882789 1.03753261]
    [[ 1.39812358  1.52428088]
     [ 0.0433848   0.12714251]
     [-0.04150838 -0.15142339]] [2.01352824 0.37150368 0.61496808]
    Iteration:  8
    [0.97120946 0.01505729 0.01373324]
    Loss:  0.029213117563831773
    Updated Weights and Biases
    [[0.10473046 0.33311322 0.71419138]
     [0.90229188 0.41604314 0.40687563]] [1.0473046  1.02291877]
    [[0.46179905 0.36357929]
     [0.75908226 0.26085556]] [1.04223867 1.04046793]
    [[ 1.42421682  1.55100455]
     [ 0.02973818  0.11316618]
     [-0.053955   -0.16417073]] [2.04231877 0.35644639 0.60123484]
    Iteration:  9
    [0.97440568 0.01336657 0.01222775]
    Loss:  0.025927552828981213
    Updated Weights and Biases
    [[0.10507131 0.33549914 0.71521392]
     [0.9024717  0.41730188 0.40741509]] [1.05071306 1.02471697]
    [[0.46640466 0.36824985]
     [0.76303914 0.26486825]] [1.04529844 1.04309672]
    [[ 1.44746259  1.57479815]
     [ 0.01759813  0.10074003]
     [-0.06506072 -0.17553818]] [2.06791309 0.34307981 0.58900709]
    Iteration:  10
    [0.97696239 0.01201656 0.01102104]
    Loss:  0.02330712020071898
    Updated Weights and Biases
    [[0.10538227 0.33767586 0.7161468 ]
     [0.90263688 0.41845813 0.40791063]] [1.05382265 1.02636876]
    [[0.4705949  0.37249186]
     [0.76663413 0.26850765]] [1.04807227 1.04547651]
    [[ 1.46842567  1.5962442 ]
     [ 0.00666365  0.08955363]
     [-0.07508932 -0.18579783]] [2.0909507  0.33106325 0.57798605]
    Iteration:  11
    [0.9790546  0.01091353 0.01003187]
    Loss:  0.021167867391973044
    Updated Weights and Biases
    [[0.10566825 0.33967773 0.71700474]
     [0.90278971 0.41952798 0.40836913]] [1.05668247 1.02789711]
    [[0.47443889 0.37637732]
     [0.76992815 0.27183721]] [1.05060859 1.04764995]
    [[ 1.48751713  1.61576648]
     [-0.00328389  0.07938161]
     [-0.08423324 -0.19514809]] [2.1118961  0.32014971 0.56795418]
    Iteration:  12
    [0.98079863 0.00999531 0.00920607]
    Loss:  0.01938811367291565
    Updated Weights and Biases
    [[0.10593303 0.34153119 0.71779908]
     [0.90293199 0.4205239  0.40879596]] [1.05933028 1.02931985]
    [[0.47798969 0.37996136]
     [0.77296786 0.27490538]] [1.05294445 1.0496496 ]
    [[ 1.50504567  1.63368307]
     [-0.0124084   0.0700551 ]
     [-0.09263727 -0.20373817]] [2.13109748 0.31015441 0.55874812]
    Iteration:  13
    [0.98227487 0.009219   0.00850613]
    Loss:  0.017884103003570954
    Updated Weights and Biases
    [[0.10617958 0.34325704 0.71853873]
     [0.90306511 0.4214558  0.40919534]] [1.06179578 1.03065115]
    [[0.48128895 0.38328721]
     [0.77578976 0.27775002]] [1.05510884 1.05150083]
    [[ 1.52124913  1.6502389 ]
     [-0.02083597  0.06144426]
     [-0.10041316 -0.21168316]] [2.14882261 0.30093541 0.55024199]
    Iteration:  14
    [0.98354069 0.00855406 0.00790525]
    Loss:  0.016596270340631637
    Updated Weights and Biases
    [[0.10641028 0.34487194 0.71923083]
     [0.90319024 0.42233171 0.40957073]] [1.06410277 1.03190244]
    [[0.48436996 0.38638935]
     [0.77842297 0.28040128]] [1.05712488 1.05322385]
    [[ 1.5363147   1.66562669]
     [-0.02866569  0.05344708]
     [-0.10764901 -0.21907377]] [2.16528192 0.29238135 0.54233673]
    Iteration:  15
    [0.98463813 0.00797812 0.00738374]
    Loss:  0.015481081906068636
    Updated Weights and Biases
    [[0.10662706 0.34638945 0.71988119]
     [0.90330831 0.42315816 0.40992493]] [1.06627064 1.03308309]
    [[0.48725979 0.38929578]
     [0.78089111 0.28288361]] [1.05901128 1.05483499]
    [[ 1.55039238  1.68000078]
     [-0.03597687  0.04598195]
     [-0.1144155  -0.22598274]] [2.18064378 0.28440322 0.53495299]
    Iteration:  16
    [0.98559873 0.00747445 0.00692682]
    Loss:  0.014505974042139025
    Updated Weights and Biases
    [[0.10683154 0.34782076 0.72049461]
     [0.90342009 0.42394062 0.41026027]] [1.06831537 1.03420089]
    [[0.48998072 0.39202952]
     [0.78321361 0.28521704]] [1.06078345 1.05634765]
    [[ 1.56360424  1.69348676]
     [-0.04283401  0.03898255]
     [-0.12077023 -0.23246931]] [2.19504505 0.27692877 0.52802618]
    Iteration:  17
    [0.98644659 0.00703025 0.00652316]
    Loss:  0.013646097998312075
    Updated Weights and Biases
    [[0.10702503 0.34917522 0.7210751 ]
     [0.90352624 0.42468368 0.41057872]] [1.07025032 1.03526239]
    [[0.49255135 0.39460977]
     [0.78540663 0.28741827]] [1.06245418 1.05777296]
    [[ 1.576051    1.70618819]
     [-0.04929023  0.03239423]
     [-0.12676077 -0.23858242]] [2.20859847 0.26989852 0.52150301]
    Iteration:  18
    [0.98720046 0.00663558 0.00616396]
    Loss:  0.012882160242001632
    Updated Weights and Biases
    [[0.10720868 0.35046074 0.72162603]
     [0.90362731 0.4253912  0.41088194]] [1.07208678 1.03627315]
    [[0.49498733 0.39705266]
     [0.78748379 0.28950132]] [1.06403425 1.05912028]
    [[ 1.58781671  1.71819146]
     [-0.05538985  0.02617145]
     [-0.13242686 -0.24436291]] [2.22139801 0.26326294 0.51533906]
    Iteration:  19
    [0.98787515 0.00628261 0.00584224]
    Loss:  0.012198950566520442
    Updated Weights and Biases
    [[0.10738343 0.35168404 0.7221503 ]
     [0.90372379 0.42606654 0.41117137]] [1.07383435 1.03723791]
    [[0.49730201 0.39937192]
     [0.78945665 0.29147808]] [1.06553279 1.06039753]
    [[ 1.59897224  1.72956943]
     [-0.0611702   0.02027584]
     [-0.13780204 -0.24984527]] [2.23352285 0.25698033 0.50949682]
    Iteration:  20
    [0.98848252 0.00596506 0.00555241]
    Loss:  0.011584315088619651
    Updated Weights and Biases
    [[0.10755013 0.3528509  0.72265039]
     [0.90381608 0.42671256 0.41144824]] [1.07550129 1.0381608 ]
    [[0.49950682 0.40157931]
     [0.79133511 0.29335874]] [1.06695764 1.06161148]
    [[ 1.60957795  1.74038409]
     [-0.06666305  0.01467478]
     [-0.14291491 -0.25505887]] [2.24504033 0.25101526 0.50394441]
    Iteration:  21
    [0.98903217 0.00567787 0.00528997]
    Loss:  0.011028424641181308
    Updated Weights and Biases
    [[0.10770948 0.35396633 0.72312843]
     [0.90390454 0.42733176 0.41171361]] [1.07709475 1.03904538]
    [[0.50161164 0.40368498]
     [0.79312774 0.2951521 ]] [1.06831555 1.06276798]
    [[ 1.61968563  1.75068863]
     [-0.07189563  0.00934029]
     [-0.14779    -0.26002892]] [2.25600816 0.2453374  0.49865444]
    Iteration:  22
    [0.98953193 0.00541688 0.00505119]
    Loss:  0.010523243891044792
    Updated Weights and Biases
    [[0.1078621  0.35503468 0.72358629]
     [0.90398948 0.42792634 0.41196843]] [1.07862097 1.03989477]
    [[0.50362508 0.40569777]
     [0.79484198 0.29686579]] [1.06961238 1.0638721 ]
    [[ 1.62934006  1.76052905]
     [-0.07689148  0.0042482 ]
     [-0.15244859 -0.26477724]] [2.26647623 0.23992052 0.49360325]
    Iteration:  23
    [0.98998831 0.00517867 0.00483301]
    Loss:  0.010062139693487153
    Updated Weights and Biases
    [[0.10800854 0.35605979 0.72402562]
     [0.90407117 0.42849822 0.41221352]] [1.08008541 1.04071174]
    [[0.50555468 0.40762542]
     [0.79648434 0.29850648]] [1.07085328 1.06492828]
    [[ 1.63858025e+00  1.76994536e+00]
     [-8.16710815e-02 -6.22515689e-04]
     [-1.56909166e-01 -2.69322847e-01]] [2.27648791 0.23474184 0.48877024]
    Iteration:  24
    [0.99040672 0.0049604  0.00463288]
    Loss:  0.009639587833506902
    Updated Weights and Biases
    [[0.10814929 0.35704502 0.72444787]
     [0.90414987 0.42904911 0.41244962]] [1.08149289 1.04149872]
    [[0.50740708 0.40947472]
     [0.79806056 0.30008006]] [1.07204274 1.0659404 ]
    [[ 1.64744034  1.77897264]
     [-0.08625237 -0.00529025]
     [-0.16118797 -0.27368239]] [2.28608119 0.22978145 0.48413736]
    Iteration:  25
    [0.99079171 0.00475965 0.00444864]
    Loss:  0.009250950458533573
    Updated Weights and Biases
    [[0.10828477 0.35799338 0.72485431]
     [0.90422579 0.42958052 0.41267737]] [1.08284768 1.04225789]
    [[0.50918816 0.4112517 ]
     [0.79957569 0.30159171]] [1.07318477 1.06691191]
    [[ 1.65595044  1.78764177]
     [-0.09065113 -0.00977121]
     [-0.16529931 -0.27787055]] [2.29528948 0.2250218  0.47968872]
    Iteration:  26
    [0.99114711 0.00457441 0.00427848]
    Loss:  0.008892305077855288
    Updated Weights and Biases
    [[0.10841536 0.35890752 0.72524608]
     [0.90429912 0.43009382 0.41289735]] [1.0841536  1.04299117]
    [[0.51090315 0.41296171]
     [0.80103426 0.30304604]] [1.07428291 1.06784585]
    [[ 1.6641372   1.79598007]
     [-0.09488135 -0.01407973]
     [-0.16925586 -0.28190034]] [2.30414237 0.22044739 0.47541024]
    Iteration:  27
    [0.99147622 0.00440295 0.00412083]
    Loss:  0.008560311700414696
    Updated Weights and Biases
    [[0.10854141 0.35978984 0.72562422]
     [0.90437003 0.43059022 0.41311009]] [1.08541406 1.04370031]
    [[0.51255672 0.41460952]
     [0.8024403  0.30444718]] [1.07534032 1.06874496]
    [[ 1.67202434  1.80401189]
     [-0.09895544 -0.01822855]
     [-0.1730689  -0.28578333]] [2.31266614 0.21604444 0.47128941]
    Iteration:  28
    [0.99178185 0.00424378 0.00397437]
    Loss:  0.008252108537762037
    Updated Weights and Biases
    [[0.10866321 0.36064247 0.72598963]
     [0.90443869 0.43107081 0.41331606]] [1.0866321  1.04438688]
    [[0.51415309 0.41619944]
     [0.8037974  0.3057988 ]] [1.07635983 1.06961168]
    [[ 1.67963303  1.81175894]
     [-0.1028845  -0.02222907]
     [-0.17674853 -0.28952987]] [2.3208843  0.21180066 0.46731504]
    Iteration:  29
    [0.99206641 0.00409565 0.00383794]
    Loss:  0.007965229351517857
    Updated Weights and Biases
    [[0.10878105 0.36146735 0.72634315]
     [0.90450523 0.4315366  0.41351569]] [1.0878105  1.04505229]
    [[0.51569601 0.41773531]
     [0.80510883 0.30710424]] [1.077344   1.07044818]
    [[ 1.68698232  1.81924073]
     [-0.10667851 -0.02609147]
     [-0.18030381 -0.29314925]] [2.32881789 0.20770501 0.4634771 ]
    Iteration:  30
    [0.99233201 0.00395743 0.00371056]
    Loss:  0.007697537377711139
    Updated Weights and Biases
    [[0.10889517 0.36226622 0.72668552]
     [0.90456978 0.43198849 0.41370935]] [1.08895174 1.04569784]
    [[0.51718892 0.41922062]
     [0.80637753 0.30836647]] [1.07829513 1.07125647]
    [[ 1.69408932  1.82647481]
     [-0.11034641 -0.02982497]
     [-0.18374291 -0.29664984]] [2.33648588 0.20374758 0.45976654]
    Iteration:  31
    [0.99258049 0.00382817 0.00359134]
    Loss:  0.007447172073045418
    Updated Weights and Biases
    [[0.10900581 0.36304068 0.72701743]
     [0.90463247 0.43242728 0.41389741]] [1.09005811 1.04632469]
    [[0.51863491 0.42065854]
     [0.80760615 0.30958823]] [1.0792153  1.07203831]
    [[ 1.70096952  1.83347707]
     [-0.11389633 -0.03343786]
     [-0.1870732  -0.30003922]] [2.34390539 0.19991941 0.4561752 ]
    Iteration:  32
    [0.99281344 0.00370703 0.00347953]
    Loss:  0.007212505870142702
    Updated Weights and Biases
    [[0.10911317 0.36379216 0.7273395 ]
     [0.90469339 0.43285374 0.41408017]] [1.09113166 1.04693391]
    [[0.52003681 0.42205194]
     [0.80879711 0.31077198]] [1.08010642 1.07279535]
    [[ 1.70763698  1.84026189]
     [-0.11733559 -0.03693766]
     [-0.19030139 -0.30332424]] [2.35109194 0.19621238 0.45269567]
    Iteration:  33
    [0.99303228 0.00359326 0.00337446]
    Loss:  0.006992108812929068
    Updated Weights and Biases
    [[0.10921743 0.364522   0.72765229]
     [0.90475265 0.43326854 0.41425795]] [1.09217428 1.04752649]
    [[0.52139718 0.42340343]
     [0.80995264 0.31191997]] [1.0809702  1.07352906]
    [[ 1.71410447  1.84684237]
     [-0.12067088 -0.04033122]
     [-0.19343359 -0.30651115]] [2.35805967 0.19261912 0.44932122]
    Iteration:  34
    [0.99323824 0.00348622 0.00327553]
    Loss:  0.0067847194457300395
    Updated Weights and Biases
    [[0.10931877 0.3652314  0.72795631]
     [0.90481033 0.43367233 0.414431  ]] [1.09318771 1.04810332]
    [[0.52271838 0.42471542]
     [0.81107474 0.31303425]] [1.08180823 1.07424081]
    [[ 1.72038364  1.85323045]
     [-0.12390829 -0.04362478]
     [-0.19647535 -0.30960567]] [2.36482142 0.1891329  0.44604568]
    Iteration:  35
    [0.99343244 0.00338533 0.00318223]
    Loss:  0.006589220702423551
    Updated Weights and Biases
    [[0.10941735 0.36592146 0.72825206]
     [0.90486652 0.43406566 0.41459957]] [1.09417352 1.04866523]
    [[0.52400257 0.4259901 ]
     [0.81216527 0.31411671]] [1.08262195 1.07493182]
    [[ 1.72648518  1.85943705]
     [-0.1270534  -0.04682405]
     [-0.19943178 -0.31261301]] [2.37138898 0.18574757 0.44286345]
    Iteration:  36
    [0.99361585 0.00329007 0.00309409]
    Loss:  0.006404619821317148
    Updated Weights and Biases
    [[0.10951332 0.36659322 0.72853995]
     [0.9049213  0.43444909 0.4147639 ]] [1.09513317 1.04921299]
    [[0.52525173 0.4272295 ]
     [0.81322594 0.31516907]] [1.08341269 1.07560324]
    [[ 1.73241886  1.86547221]
     [-0.13011131 -0.04993426]
     [-0.20230754 -0.31553795]] [2.37777313 0.1824575  0.43976936]
    Iteration:  37
    [0.99378933 0.00319998 0.00301068]
    Loss:  0.006230031522577576
    Updated Weights and Biases
    [[0.1096068  0.36724761 0.72882041]
     [0.90497473 0.4348231  0.41492419]] [1.09606802 1.04974729]
    [[0.52646771 0.42843547]
     [0.8142583  0.31619294]] [1.08418168 1.07625611]
    [[ 1.73819368  1.87134512]
     [-0.13308673 -0.05296021]
     [-0.20510695 -0.31838491]] [2.3839838  0.17925752 0.43675868]
    Iteration:  38
    [0.99395369 0.00311466 0.00293165]
    Loss:  0.006064663846087807
    Updated Weights and Biases
    [[0.10969793 0.36788552 0.72909379]
     [0.90502688 0.43518815 0.41508064]] [1.09697931 1.05026879]
    [[0.52765218 0.42960972]
     [0.81526382 0.31718979]] [1.08493005 1.07689141]
    [[ 1.74381793  1.87706428]
     [-0.13598397 -0.05590635]
     [-0.20783396 -0.32115793]] [2.39003011 0.17614286 0.43382703]
    Iteration:  39
    [0.99410961 0.00303374 0.00285665]
    Loss:  0.005907806171361446
    Updated Weights and Biases
    [[0.10978682 0.36850773 0.72936046]
     [0.90507781 0.43554467 0.41523343]] [1.09786819 1.0507781 ]
    [[0.52880672 0.43075386]
     [0.81624383 0.31816097]] [1.08565883 1.07751002]
    [[ 1.74929928  1.88263753]
     [-0.13880705 -0.05877675]
     [-0.21049223 -0.32386078]] [2.3959205  0.17310912 0.43097038]
    Iteration:  40
    [0.99425773 0.00295689 0.00278538]
    Loss:  0.005758819037026635
    Updated Weights and Biases
    [[0.10987357 0.36911501 0.72962072]
     [0.90512758 0.43589305 0.41538273]] [1.09873573 1.05127578]
    [[0.52993277 0.43186935]
     [0.81719957 0.31910774]] [1.086369   1.07811278]
    [[ 1.75464481  1.88807213]
     [-0.14155964 -0.06157521]
     [-0.21308517 -0.32649692]] [2.40166277 0.17015224 0.428185  ]
    Iteration:  41
    [0.99439862 0.0028838  0.00271758]
    Loss:  0.005617125452144348
    Updated Weights and Biases
    [[0.10995829 0.36970805 0.72987488]
     [0.90517624 0.43623365 0.41552871]] [1.09958292 1.05176236]
    [[0.53103169 0.43295756]
     [0.81813219 0.32003129]] [1.08706145 1.07870045]
    [[ 1.7598611   1.89337479]
     [-0.14424518 -0.06430522]
     [-0.21561592 -0.32906957]] [2.40726415 0.16726843 0.42546742]
    Iteration:  42
    [0.9945328  0.00281422 0.00265299]
    Loss:  0.005482203450360682
    Updated Weights and Biases
    [[0.11004107 0.3702875  0.73012321]
     [0.90522383 0.43656683 0.4156715 ]] [1.10041071 1.05223833]
    [[0.53210473 0.43401977]
     [0.81904278 0.32093268]] [1.08773703 1.07927375]
    [[ 1.76495425  1.89855178]
     [-0.14686686 -0.06697005]
     [-0.21808739 -0.33158173]] [2.41273135 0.16445422 0.42281443]
    Iteration:  43
    [0.99466073 0.00274789 0.00259139]
    Loss:  0.005353579684281697
    Updated Weights and Biases
    [[0.11012199 0.37085396 0.73036598]
     [0.90527041 0.4368929  0.41581124]] [1.10121994 1.05270414]
    [[0.53315306 0.43505715]
     [0.81993233 0.32181294]] [1.0883965  1.07983334]
    [[ 1.76992995  1.90360891]
     [-0.14942763 -0.06957273]
     [-0.22050232 -0.33403618]] [2.41807063 0.16170633 0.42022305]
    Iteration:  44
    [0.99478283 0.00268459 0.00253258]
    Loss:  0.005230823894365037
    Updated Weights and Biases
    [[0.11020114 0.371408   0.73060343]
     [0.90531602 0.43721216 0.41594807]] [1.10201143 1.05316023]
    [[0.53417777 0.43607082]
     [0.82080178 0.32267302]] [1.08904059 1.08037984]
    [[ 1.7747935   1.90855159]
     [-0.15193026 -0.07211608]
     [-0.22286324 -0.33643551]] [2.42328779 0.15902174 0.41769047]
    Iteration:  45
    [0.99489951 0.00262412 0.00247637]
    Loss:  0.005113544116118423
    Updated Weights and Biases
    [[0.11027859 0.37195016 0.73083578]
     [0.9053607  0.43752489 0.4160821 ]] [1.10278594 1.05360699]
    [[0.53517988 0.43706181]
     [0.82165199 0.3235138 ]] [1.08966998 1.08091383]
    [[ 1.77954984  1.91338489]
     [-0.15437732 -0.07460273]
     [-0.22517252 -0.33878216]] [2.42838828 0.15639762 0.4152141 ]
    Iteration:  46
    [0.9950111 0.0025663 0.0024226]
    Loss:  0.005001382513119942
    Updated Weights and Biases
    [[0.11035442 0.37248092 0.73106325]
     [0.90540448 0.43783137 0.41621344]] [1.10354417 1.05404481]
    [[0.53616036 0.4380311 ]
     [0.8224838  0.32433611]] [1.0902853  1.08143585]
    [[ 1.78420361  1.91811354]
     [-0.15677122 -0.07703516]
     [-0.22743239 -0.34107838]] [2.43337718 0.15383132 0.4127915 ]
    Iteration:  47
    [0.99511794 0.00251095 0.00237111]
    Loss:  0.0048940117425445
    Updated Weights and Biases
    [[0.11042868 0.37300075 0.73128604]
     [0.9054474  0.43813182 0.41634221]] [1.10428679 1.05447403]
    [[0.5371201  0.43897958]
     [0.82329796 0.32514072]] [1.09088716 1.08194641]
    [[ 1.78875913  1.92274197]
     [-0.15911423 -0.07941566]
     [-0.22964491 -0.34332631]] [2.43825924 0.15132038 0.41042039]
    Iteration:  48
    [0.99522033 0.00245792 0.00232176]
    Loss:  0.004791131775457785
    Updated Weights and Biases
    [[0.11050144 0.37351009 0.73150433]
     [0.9054895  0.4384265  0.4164685 ]] [1.10501442 1.054895  ]
    [[0.53805994 0.43990811]
     [0.82409519 0.32592836]] [1.0914761  1.08244598]
    [[ 1.79322048  1.92727434]
     [-0.16140845 -0.0817464 ]
     [-0.23181204 -0.34552794]] [2.44303891 0.14886246 0.40809863]
    Iteration:  49
    [0.99531853 0.00240706 0.00227441]
    Loss:  0.004692467106838367
    Updated Weights and Biases
    [[0.11057277 0.37400936 0.7317183 ]
     [0.9055308  0.43871561 0.41659241]] [1.10572765 1.05530802]
    [[0.53898068 0.44081749]
     [0.82487618 0.32669972]] [1.09205265 1.08293503]
    [[ 1.79759147  1.93171456]
     [-0.16365587 -0.08402942]
     [-0.2339356  -0.34768514]] [2.44772038 0.1464554  0.40582422]
    Iteration:  50
    [0.99541279 0.00235826 0.00222895]
    Loss:  0.00459776430071389
    Updated Weights and Biases
    [[0.1106427  0.37449893 0.73192811]
     [0.90557134 0.43899938 0.41671402]] [1.10642705 1.0557134 ]
    [[0.53988306 0.44170849]
     [0.82564155 0.32745543]] [1.09261731 1.08341395]
    [[ 1.80187569  1.93606629]
     [-0.16585836 -0.08626661]
     [-0.23601733 -0.34979968]] [2.45230759 0.14409714 0.40359526]
    Iteration:  51
    [0.99550335 0.00231138 0.00218527]
    Loss:  0.004506789824370946
    Updated Weights and Biases
    [[0.11071131 0.37497919 0.73213394]
     [0.90561114 0.43927798 0.41683342]] [1.10711312 1.05611141]
    [[0.54076778 0.4425818 ]
     [0.82639192 0.32819612]] [1.09317053 1.08388316]
    [[ 1.80607652  1.94033298]
     [-0.16801767 -0.08845979]
     [-0.23805884 -0.3518732 ]] [2.45680424 0.14178577 0.40140999]
    Iteration:  52
    [0.99559042 0.00226631 0.00214327]
    Loss:  0.004419328132686974
    Updated Weights and Biases
    [[0.11077864 0.37545046 0.73233591]
     [0.90565023 0.43955162 0.41695069]] [1.10778637 1.05650231]
    [[0.54163552 0.4434381 ]
     [0.82712784 0.32892234]] [1.09371277 1.08434302]
    [[ 1.81019714  1.9445179 ]
     [-0.17013548 -0.09061063]
     [-0.24006167 -0.35390727]] [2.46121382 0.13951946 0.39926672]
    Iteration:  53
    [0.9956742  0.00222295 0.00210284]
    Loss:  0.0043351799695212955
    Updated Weights and Biases
    [[0.11084473 0.37591308 0.73253418]
     [0.90568864 0.43982046 0.41706591]] [1.10844726 1.05688637]
    [[0.5424869  0.44427803]
     [0.82784986 0.32963464]] [1.09424442 1.08479389]
    [[ 1.81424057  1.94862412]
     [-0.17221332 -0.09272074]
     [-0.24202725 -0.35590337]] [2.46553962 0.13729651 0.39716388]
    Iteration:  54
    [0.99575488 0.00218121 0.00206391]
    Loss:  0.004254160858001673
    Updated Weights and Biases
    [[0.11090962 0.37636736 0.73272887]
     [0.90572638 0.44008467 0.41717914]] [1.10909624 1.05726381]
    [[0.54332252 0.44510218]
     [0.82855847 0.33033354]] [1.09476588 1.08523611]
    [[ 1.81820964  1.95265453]
     [-0.1742527  -0.09479164]
     [-0.24395695 -0.35786289]] [2.46978474 0.13511529 0.39509996]
    Iteration:  55
    [0.99583261 0.002141   0.00202639]
    Loss:  0.004176099755647466
    Updated Weights and Biases
    [[0.11097337 0.3768136  0.73292011]
     [0.90576349 0.44034441 0.41729046]] [1.10973371 1.05763487]
    [[0.54414292 0.44591112]
     [0.82925417 0.33101951]] [1.09527753 1.08566998]
    [[ 1.82210705  1.95661189]
     [-0.17625499 -0.09682473]
     [-0.24585206 -0.35978716]] [2.47395214 0.13297429 0.39307357]
    Iteration:  56
    [0.99590756 0.00210223 0.00199021]
    Loss:  0.00410083785370396
    Updated Weights and Biases
    [[0.11103601 0.37725206 0.73310803]
     [0.90579998 0.44059983 0.41739993]] [1.11036008 1.05799975]
    [[0.54494865 0.44670538]
     [0.82993739 0.33169301]] [1.09577971 1.0860958 ]
    [[ 1.82593533  1.96049879]
     [-0.17822153 -0.09882138]
     [-0.2477138  -0.36167741]] [2.47804458 0.13087206 0.39108336]
    Iteration:  57
    [0.99597987 0.00206483 0.00195529]
    Loss:  0.004028227502969514
    Updated Weights and Biases
    [[0.11109757 0.37768301 0.73329272]
     [0.90583587 0.44085107 0.4175076 ]] [1.11097573 1.05835867]
    [[0.54574022 0.44748548]
     [0.83060858 0.33235447]] [1.09627275 1.08651386]
    [[ 1.82969689  1.9643177 ]
     [-0.18015356 -0.10078287]
     [-0.24954333 -0.36353484]] [2.4820647  0.12880723 0.38912807]
    Iteration:  58
    [0.99604969 0.00202873 0.00192157]
    Loss:  0.00395813125082976
    Updated Weights and Biases
    [[0.1111581  0.37810671 0.7334743 ]
     [0.90587118 0.44109826 0.41761354]] [1.11158101 1.0587118 ]
    [[0.5465181  0.4482519 ]
     [0.83126814 0.33300432]] [1.09675697 1.08692443]
    [[ 1.83339403  1.96807096]
     [-0.18205228 -0.1027104 ]
     [-0.25134176 -0.36536056]] [2.48601501 0.12677849 0.3872065 ]
    Iteration:  59
    [0.99611714 0.00199386 0.001889  ]
    Loss:  0.0038904209763019866
    Updated Weights and Biases
    [[0.11121763 0.37852339 0.73365288]
     [0.90590593 0.44134154 0.4177178 ]] [1.11217627 1.05905935]
    [[0.54728274 0.4490051 ]
     [0.83191646 0.33364293]] [1.09723267 1.08732776]
    [[ 1.83702891  1.97176077]
     [-0.1839188  -0.10460513]
     [-0.25311011 -0.36715564]] [2.48989787 0.12478463 0.3853175 ]
    Iteration:  60
    [0.99618233 0.00196017 0.00185751]
    Loss:  0.003824977111642391
    Updated Weights and Biases
    [[0.11127618 0.37893327 0.73382855]
     [0.90594015 0.44158103 0.41782044]] [1.11276182 1.05940147]
    [[0.54803459 0.44974552]
     [0.83255391 0.33427069]] [1.09770013 1.0877241 ]
    [[ 1.8406036   1.97538925]
     [-0.18575421 -0.10646816]
     [-0.2548494  -0.36892109]] [2.49371554 0.12282446 0.38345999]
    Iteration:  61
    [0.99624538 0.00192758 0.00182704]
    Loss:  0.0037616879405767347
    Updated Weights and Biases
    [[0.1113338  0.37933658 0.73400139]
     [0.90597384 0.44181685 0.41792151]] [1.11333797 1.05973835]
    [[0.54877405 0.45047356]
     [0.83318084 0.33488794]] [1.09815963 1.08811367]
    [[ 1.84412006  1.9789584 ]
     [-0.18755951 -0.10830052]
     [-0.25656055 -0.37065788]] [2.49747017 0.12089688 0.38163295]
    Iteration:  62
    [0.99630639 0.00189605 0.00179756]
    Loss:  0.0037004489644965615
    Updated Weights and Biases
    [[0.1113905  0.37973352 0.73417151]
     [0.90600701 0.4420491  0.41802104]] [1.11390503 1.06007014]
    [[0.54950151 0.45118963]
     [0.83379758 0.33549502]] [1.09861141 1.08849669]
    [[ 1.84758015  1.98247013]
     [-0.18933569 -0.1101032 ]
     [-0.25824446 -0.37236693]] [2.50116378 0.11900084 0.37983539]
    Iteration:  63
    [0.99636546 0.00186553 0.00176901]
    Loss:  0.0036411623290665027
    Updated Weights and Biases
    [[0.11144633 0.38012428 0.73433898]
     [0.9060397  0.44227789 0.4181191 ]] [1.11446326 1.06039698]
    [[0.55021736 0.45189411]
     [0.83440446 0.33609226]] [1.09905573 1.08887337]
    [[ 1.85098566  1.98592625]
     [-0.19108366 -0.11187715]
     [-0.259902   -0.3740491 ]] [2.50479832 0.11713531 0.37806637]
    Iteration:  64
    [0.99642268 0.00183597 0.00174136]
    Loss:  0.0035837363046264618
    Updated Weights and Biases
    [[0.11150129 0.38050906 0.73450388]
     [0.9060719  0.44250333 0.41821571]] [1.11501294 1.06071904]
    [[0.55092195 0.45258734]
     [0.83500178 0.33667996]] [1.09949282 1.08924392]
    [[ 1.85433828  1.9893285 ]
     [-0.1928043  -0.11362326]
     [-0.26153398 -0.37570523]] [2.50837564 0.11529934 0.37632502]
    Iteration:  65
    [0.99647813 0.00180732 0.00171455]
    Loss:  0.003528084814595028
    Updated Weights and Biases
    [[0.11155543 0.38088802 0.7346663 ]
     [0.90610364 0.44272551 0.41831093]] [1.11555432 1.06103644]
    [[0.55161562 0.45326969]
     [0.83558983 0.33725841]] [1.0999229  1.08960852]
    [[ 1.85763963  1.99267852]
     [-0.19449846 -0.1153424 ]
     [-0.26314117 -0.37733613]] [2.51189751 0.11349202 0.37461047]
    Iteration:  66
    [0.9965319  0.00177955 0.00168855]
    Loss:  0.0034741270067776653
    Updated Weights and Biases
    [[0.11160876 0.38126135 0.73482629]
     [0.90613493 0.44294453 0.4184048 ]] [1.11608765 1.06134932]
    [[0.55229869 0.45394147]
     [0.83616888 0.33782789]] [1.10034619 1.08996735]
    [[ 1.86089126  1.99597791]
     [-0.19616693 -0.11703537]
     [-0.26472433 -0.37894253]] [2.51536561 0.11171248 0.37292192]
    Iteration:  67
    [0.99658406 0.00175261 0.00166333]
    Loss:  0.003421786863095778
    Updated Weights and Biases
    [[0.11166132 0.38162921 0.73498395]
     [0.90616578 0.44316047 0.41849734]] [1.11661315 1.06165781]
    [[0.55297148 0.45460299]
     [0.83673921 0.33838867]] [1.10076289 1.09032058]
    [[ 1.86409464  1.99922815]
     [-0.19781048 -0.11870297]
     [-0.26628415 -0.38052518]] [2.51878155 0.10995987 0.37125859]
    Iteration:  68
    [0.99663468 0.00172647 0.00163885]
    Loss:  0.0033709928437758905
    Updated Weights and Biases
    [[0.11171311 0.38199175 0.73513932]
     [0.9061962  0.44337342 0.41858861]] [1.11713106 1.06196203]
    [[0.55363429 0.45525457]
     [0.83730107 0.338941  ]] [1.10117319 1.09066839]
    [[ 1.86725118  2.0024307 ]
     [-0.19942985 -0.12034594]
     [-0.26782133 -0.38208476]] [2.52214686 0.1082334  0.36961974]
    Iteration:  69
    [0.99668383 0.00170109 0.00161507]
    Loss:  0.0033216775625053803
    Updated Weights and Biases
    [[0.11176416 0.38234911 0.73529248]
     [0.90622621 0.44358347 0.41867863]] [1.11764159 1.0622621 ]
    [[0.5542874  0.45589647]
     [0.83785469 0.33948513]] [1.10157728 1.09101093]
    [[ 1.87036223  2.00558693]
     [-0.20102572 -0.12196499]
     [-0.26933651 -0.38362194]] [2.52546303 0.1065323  0.36800467]
    Iteration:  70
    [0.99673158 0.00167645 0.00159198]
    Loss:  0.0032737774894514814
    Updated Weights and Biases
    [[0.11181449 0.38270146 0.73544348]
     [0.90625581 0.44379069 0.41876744]] [1.11814494 1.06255812]
    [[0.55493108 0.45652899]
     [0.83840032 0.34002129]] [1.10197534 1.09134835]
    [[ 1.87342909  2.00869817]
     [-0.20259878 -0.12356081]
     [-0.27083031 -0.38513736]] [2.52873146 0.10485586 0.36641269]
    Iteration:  71
    [0.99677797 0.0016525  0.00156953]
    Loss:  0.003227232679396885
    Updated Weights and Biases
    [[0.11186413 0.38304892 0.73559239]
     [0.90628502 0.44399515 0.41885506]] [1.11864131 1.06285021]
    [[0.5555656  0.45715238]
     [0.83893817 0.3405497 ]] [1.10236754 1.0916808 ]
    [[ 1.876453    2.01176566]
     [-0.20414967 -0.12513405]
     [-0.27230333 -0.38663161]] [2.53195349 0.10320336 0.36484316]
    Iteration:  72
    [0.99682307 0.00162922 0.00154771]
    Loss:  0.003181986522549217
    Updated Weights and Biases
    [[0.11191309 0.38339162 0.73573927]
     [0.90631385 0.44419693 0.41894154]] [1.11913089 1.06313847]
    [[0.55619121 0.4577669 ]
     [0.83946846 0.34107059]] [1.10275405 1.09200841]
    [[ 1.87943514  2.01479064]
     [-0.20567899 -0.12668534]
     [-0.27375614 -0.38810529]] [2.53513042 0.10157414 0.36329545]
    Iteration:  73
    [0.99686693 0.00160658 0.00152648]
    Loss:  0.0031379855158476097
    Updated Weights and Biases
    [[0.11196139 0.3837297  0.73588416]
     [0.9063423  0.4443961  0.4190269 ]] [1.11961385 1.063423  ]
    [[0.55680815 0.45837278]
     [0.83999139 0.34158415]] [1.10313501 1.09233133]
    [[ 1.88237665  2.01777425]
     [-0.20718735 -0.12821529]
     [-0.2751893  -0.38955896]] [2.53826348 0.09996756 0.36176896]
    Iteration:  74
    [0.99690961 0.00158456 0.00150583]
    Loss:  0.0030951790528252086
    Updated Weights and Biases
    [[0.11200904 0.38406327 0.73602711]
     [0.90637039 0.44459273 0.41911117]] [1.12009038 1.0637039 ]
    [[0.55741666 0.45897026]
     [0.84050717 0.34209058]] [1.10351059 1.09264967]
    [[ 1.88527862  2.02071761]
     [-0.2086753  -0.12972446]
     [-0.27660332 -0.39099315]] [2.54135388 0.09838299 0.36026313]
    Iteration:  75
    [0.99695114 0.00156313 0.00148573]
    Loss:  0.0030535192302974445
    Updated Weights and Biases
    [[0.11205606 0.38439245 0.73616819]
     [0.90639812 0.44478687 0.41919437]] [1.12056064 1.06398125]
    [[0.55801694 0.45955956]
     [0.84101597 0.34259008]] [1.10388092 1.09296356]
    [[ 1.8881421   2.0236218 ]
     [-0.21014339 -0.13121343]
     [-0.27799871 -0.39240838]] [2.54440274 0.09681986 0.3587774 ]
    Iteration:  76
    [0.99699157 0.00154227 0.00146615]
    Loss:  0.003012960670323256
    Updated Weights and Biases
    [[0.11210248 0.38471735 0.73630744]
     [0.90642551 0.4449786  0.41927654]] [1.12102479 1.06425515]
    [[0.55860923 0.46014091]
     [0.84151799 0.34308282]] [1.10424614 1.09327313]
    [[ 1.89096811  2.02648784]
     [-0.21159215 -0.13268271]
     [-0.27937596 -0.39380514]] [2.54741116 0.09527758 0.35731125]
    Iteration:  77
    [0.99703096 0.00152196 0.00144709]
    Loss:  0.0029734603560503995
    Updated Weights and Biases
    [[0.1121483  0.38503809 0.7364449 ]
     [0.90645257 0.44516797 0.4193577 ]] [1.12148299 1.06452567]
    [[0.55919372 0.46071449]
     [0.84201339 0.34356899]] [1.1046064  1.09357848]
    [[ 1.89375761  2.02931672]
     [-0.21302207 -0.13413281]
     [-0.28073554 -0.39518391]] [2.55038021 0.09375563 0.35586417]
    Iteration:  78
    [0.99706933 0.00150217 0.00142851]
    Loss:  0.0029349774801968598
    Updated Weights and Biases
    [[0.11219354 0.38535477 0.73658061]
     [0.90647929 0.44535504 0.41943787]] [1.12193538 1.06479291]
    [[0.55977061 0.46128052]
     [0.84250236 0.34404874]] [1.10496181 1.09387972]
    [[ 1.89651152  2.03210939]
     [-0.21443364 -0.13556425]
     [-0.28207788 -0.39654515]] [2.55331088 0.09225346 0.35443566]
    Iteration:  79
    [0.99710672 0.00148288 0.0014104 ]
    Loss:  0.0028974733050524813
    Updated Weights and Biases
    [[0.11223821 0.38566748 0.73671463]
     [0.90650569 0.44553986 0.41951708]] [1.12238211 1.06505695]
    [[0.56034009 0.46183918]
     [0.84298503 0.34452225]] [1.1053125  1.09417695]
    [[ 1.89923076  2.03486677]
     [-0.21582732 -0.13697747]
     [-0.28340344 -0.3978893 ]] [2.55620416 0.09077058 0.35302526]
    Iteration:  80
    [0.99714318 0.00146408 0.00139274]
    Loss:  0.0028609110329853526
    Updated Weights and Biases
    [[0.11228233 0.38597633 0.736847  ]
     [0.90653178 0.44572249 0.41959535]] [1.12282332 1.06531785]
    [[0.56090235 0.46239066]
     [0.84346158 0.34498966]] [1.1056586  1.09447029]
    [[ 1.90191617  2.03758973]
     [-0.21720355 -0.13837295]
     [-0.28471262 -0.39921678]] [2.55906099 0.0893065  0.35163252]
    Iteration:  81
    [0.99717873 0.00144575 0.00137552]
    Loss:  0.002825255686549883
    Updated Weights and Biases
    [[0.11232591 0.3862814  0.73697774]
     [0.90655757 0.44590298 0.41967271]] [1.12325914 1.06557569]
    [[0.56145756 0.46293513]
     [0.84393216 0.34545114]] [1.1060002  1.09475982]
    [[ 1.9045686   2.04027912]
     [-0.21856278 -0.13975112]
     [-0.28600582 -0.400528  ]] [2.56188225 0.08786075 0.350257  ]
    Iteration:  82
    [0.99721342 0.00142786 0.00135872]
    Loss:  0.0027904739973753727
    Updated Weights and Biases
    [[0.11236897 0.38658279 0.73710691]
     [0.90658305 0.44608138 0.41974916]] [1.1236897  1.06583054]
    [[0.56200589 0.46347277]
     [0.8443969  0.34590681]] [1.10633744 1.09504565]
    [[ 1.90718884  2.04293576]
     [-0.21990541 -0.1411124 ]
     [-0.28728343 -0.40182336]] [2.56466884 0.08643288 0.34889828]
    Iteration:  83
    [0.99724726 0.00141042 0.00134232]
    Loss:  0.002756534303090415
    Updated Weights and Biases
    [[0.11241151 0.38688058 0.73723453]
     [0.90660825 0.44625773 0.41982474]] [1.12411511 1.06608248]
    [[0.56254751 0.46400374]
     [0.84485595 0.34635683]] [1.1066704  1.09532785]
    [[ 1.90977766  2.04556044]
     [-0.22123184 -0.1424572 ]
     [-0.28854583 -0.40310324]] [2.56742158 0.08502247 0.34755595]
    Iteration:  84
    [0.9972803  0.00139339 0.00132632]
    Loss:  0.0027234064516200034
    Updated Weights and Biases
    [[0.11245355 0.38717486 0.73736065]
     [0.90663316 0.44643209 0.41989947]] [1.12453552 1.06633156]
    [[0.56308258 0.46452819]
     [0.84530944 0.34680133]] [1.1069992  1.09560652]
    [[ 1.91233582  2.04815391]
     [-0.22254246 -0.14378591]
     [-0.28979336 -0.404368  ]] [2.57014128 0.08362908 0.34622964]
    Iteration:  85
    [0.99731256 0.00137676 0.00131069]
    Loss:  0.0026910617122401816
    Updated Weights and Biases
    [[0.1124951  0.38746571 0.7374853 ]
     [0.90665779 0.4466045  0.41997336]] [1.12495101 1.06657785]
    [[0.56361124 0.46504629]
     [0.84575751 0.34724044]] [1.10732393 1.09588174]
    [[ 1.91486402  2.05071691]
     [-0.22383764 -0.14509892]
     [-0.29102638 -0.40561799]] [2.57282872 0.08225232 0.34491895]
    Iteration:  86
    [0.99734406 0.00136052 0.00129542]
    Loss:  0.0026594726928453115
    Updated Weights and Biases
    [[0.11253617 0.3877532  0.73760852]
     [0.90668214 0.44677499 0.42004643]] [1.12536172 1.06682142]
    [[0.56413365 0.46555817]
     [0.84620027 0.34767429]] [1.10764469 1.0961536 ]
    [[ 1.91736297  2.05325015]
     [-0.22511774 -0.14639659]
     [-0.29224523 -0.40685356]] [2.57548466 0.0808918  0.34362353]
    Iteration:  87
    [0.99737484 0.00134466 0.0012805 ]
    Loss:  0.0026286132629259515
    Updated Weights and Biases
    [[0.11257677 0.38803742 0.73773032]
     [0.90670623 0.44694362 0.4201187 ]] [1.12576775 1.06706232]
    [[0.56464995 0.46606399]
     [0.84663786 0.34810299]] [1.10796156 1.09642217]
    [[ 1.91983332  2.0557543 ]
     [-0.2263831  -0.14767926]
     [-0.29345022 -0.40807504]] [2.57810982 0.07954714 0.34234303]
    Iteration:  88
    [0.99740491 0.00132916 0.00126592]
    Loss:  0.0025984584818010896
    Updated Weights and Biases
    [[0.11261692 0.38831843 0.73785076]
     [0.90673006 0.44711043 0.42019018]] [1.12616919 1.06730061]
    [[0.56516028 0.46656388]
     [0.84707038 0.34852666]] [1.10827465 1.09668752]
    [[ 1.92227574  2.05823004]
     [-0.22763407 -0.14894729]
     [-0.29464167 -0.40928274]] [2.58070491 0.07821798 0.34107711]
    Iteration:  89
    [0.99743431 0.00131401 0.00125167]
    Loss:  0.0025689845316947224
    Updated Weights and Biases
    [[0.11265662 0.38859631 0.73796985]
     [0.90675363 0.44727544 0.4202609 ]] [1.12656616 1.06753635]
    [[0.56566477 0.46705797]
     [0.84749795 0.34894542]] [1.10858404 1.09694974]
    [[ 1.92469083  2.06067798]
     [-0.22887096 -0.15020101]
     [-0.29581988 -0.41047698]] [2.5832706  0.07690397 0.33982543]
    Iteration:  90
    [0.99746305 0.0012992  0.00123774]
    Loss:  0.0025401686552767293
    Updated Weights and Biases
    [[0.11269587 0.38887112 0.73808762]
     [0.90677696 0.44743871 0.42033088]] [1.12695874 1.06776959]
    [[0.56616354 0.46754639]
     [0.84792067 0.34935937]] [1.10888981 1.09720888]
    [[ 1.92707922  2.06309876]
     [-0.23009408 -0.15144072]
     [-0.29698514 -0.41165804]] [2.58580754 0.07560477 0.33858769]
    Iteration:  91
    [0.99749116 0.00128472 0.00122411]
    Loss:  0.0025119890973261485
    Updated Weights and Biases
    [[0.1127347  0.38914293 0.73820411]
     [0.90680004 0.44760026 0.42040011]] [1.12734704 1.06800038]
    [[0.56665673 0.46802927]
     [0.84833867 0.34976863]] [1.10919203 1.09746503]
    [[ 1.92944148  2.06549297]
     [-0.23130375 -0.15266675]
     [-0.29813773 -0.41282622]] [2.58831638 0.07432004 0.33736358]
    Iteration:  92
    [0.99751866 0.00127056 0.00121078]
    Loss:  0.0024844250502017327
    Updated Weights and Biases
    [[0.11277311 0.3894118  0.73831934]
     [0.90682288 0.44776014 0.42046863]] [1.12773115 1.06822877]
    [[0.56714446 0.46850673]
     [0.84875203 0.35017329]] [1.10949079 1.09771824]
    [[ 1.93177818  2.06786118]
     [-0.23250024 -0.15387938]
     [-0.29927793 -0.4139818 ]] [2.59079772 0.07304948 0.3361528 ]
    Iteration:  93
    [0.99754556 0.0012567  0.00119774]
    Loss:  0.002457456602833834
    Updated Weights and Biases
    [[0.11281111 0.3896778  0.73843334]
     [0.90684548 0.44791837 0.42053645]] [1.12811114 1.06845482]
    [[0.56762683 0.46897889]
     [0.84916086 0.35057345]] [1.10978617 1.09796857]
    [[ 1.93408986  2.07020395]
     [-0.23368385 -0.15507891]
     [-0.300406   -0.41512504]] [2.59325216 0.07179278 0.33495506]
    Iteration:  94
    [0.99757189 0.00124314 0.00118497]
    Loss:  0.0024310646929758366
    Updated Weights and Biases
    [[0.11284871 0.38994098 0.73854614]
     [0.90686786 0.448075   0.42060357]] [1.12848712 1.06867857]
    [[0.56810397 0.46944585]
     [0.84956525 0.35096921]] [1.11007823 1.09821611]
    [[ 1.93637706  2.07252182]
     [-0.23485485 -0.15626561]
     [-0.3015222  -0.41625621]] [2.59568027 0.07054963 0.3337701 ]
    Iteration:  95
    [0.99759766 0.00122987 0.00117247]
    Loss:  0.0024052310624723444
    Updated Weights and Biases
    [[0.11288592 0.39020141 0.73865775]
     [0.90689001 0.44823004 0.42067002]] [1.12885916 1.06890006]
    [[0.56857599 0.46990773]
     [0.84996529 0.35136067]] [1.11036705 1.09846089]
    [[ 1.93864029  2.07481532]
     [-0.23601351 -0.15743976]
     [-0.30262678 -0.41737556]] [2.59808261 0.06931976 0.33259763]
    Iteration:  96
    [0.99762289 0.00121688 0.00116023]
    Loss:  0.0023799382153276796
    Updated Weights and Biases
    [[0.11292273 0.39045914 0.7387682 ]
     [0.90691194 0.44838355 0.42073581]] [1.12922734 1.06911935]
    [[0.56904299 0.47036463]
     [0.85036109 0.35174791]] [1.11065269 1.09870298]
    [[ 1.94088004  2.07708495]
     [-0.23716007 -0.15860162]
     [-0.30371997 -0.41848333]] [2.60045972 0.06810288 0.3314374 ]
    Iteration:  97
    [0.9976476  0.00120415 0.00114824]
    Loss:  0.002355169378368623
    Updated Weights and Biases
    [[0.11295917 0.39071422 0.73887752]
     [0.90693365 0.44853553 0.42080094]] [1.12959174 1.06933648]
    [[0.56950507 0.47081666]
     [0.85075271 0.35213102]] [1.11093523 1.09894244]
    [[ 1.94309681  2.0793312 ]
     [-0.2382948  -0.15975144]
     [-0.30480201 -0.41957976]] [2.60281212 0.06689873 0.33028915]
    Iteration:  98
    [0.99767181 0.00119169 0.0011365 ]
    Loss:  0.0023309084643187584
    Updated Weights and Biases
    [[0.11299524 0.3909667  0.73898573]
     [0.90695515 0.44868603 0.42086544]] [1.12995244 1.06955148]
    [[0.56996233 0.47126392]
     [0.85114026 0.35251008]] [1.11121472 1.09917932]
    [[ 1.94529105  2.08155455]
     [-0.23941792 -0.16088946]
     [-0.30587312 -0.42066509]] [2.60514031 0.06570704 0.32915265]
    Iteration:  99
    [0.99769552 0.00117948 0.001125  ]
    Loss:  0.002307140037112342
    Updated Weights and Biases
    [[0.11303095 0.39121665 0.73909285]
     [0.90697644 0.44883508 0.42092932]] [1.1303095 1.0697644]
    [[0.57041487 0.4717065 ]
     [0.8515238  0.35288518]] [1.11149123 1.09941367]
    [[ 1.94746322  2.08375546]
     [-0.24052969 -0.16201594]
     [-0.30693353 -0.42173953]] [2.60744479 0.06452756 0.32802765]
    Iteration:  100
    [0.99771876 0.00116752 0.00111372]
    Loss:  0.002283849279292448
    Updated Weights and Biases
    [[0.1130663  0.39146411 0.7391989 ]
     [0.90699753 0.44898269 0.42099258]] [1.13066301 1.06997528]
    [[0.57086279 0.47214449]
     [0.85190343 0.3532564 ]] [1.11176482 1.09964555]
    [[ 1.94961375  2.08593439]
     [-0.24163031 -0.16313109]
     [-0.30798344 -0.4228033 ]] [2.60972604 0.06336004 0.32691393]
    Iteration:  101
    [0.99774153 0.0011558  0.00110267]
    Loss:  0.002261021961345581
    Updated Weights and Biases
    [[0.1131013  0.39170912 0.73930391]
     [0.90701842 0.44912891 0.42105525]] [1.13101303 1.07018415]
    [[0.57130617 0.472578  ]
     [0.85227921 0.35362381]] [1.11203555 1.099875  ]
    [[ 1.95174309  2.08809175]
     [-0.24272002 -0.16423515]
     [-0.30902307 -0.42385661]] [2.6119845  0.06220424 0.32581126]
    Iteration:  102
    [0.99776386 0.0011443  0.00109184]
    Loss:  0.002238644412843735
    Updated Weights and Biases
    [[0.11313596 0.39195174 0.73940789]
     [0.90703911 0.44927375 0.42111732]] [1.13135962 1.07039107]
    [[0.57174511 0.4730071 ]
     [0.85265123 0.35398749]] [1.11230347 1.10010207]
    [[ 1.95385163  2.09022798]
     [-0.24379903 -0.16532832]
     [-0.3100526  -0.42489966]] [2.61422064 0.06105994 0.32471942]
    Iteration:  103
    [0.99778575 0.00113304 0.00108121]
    Loss:  0.0022167034952678523
    Updated Weights and Biases
    [[0.11317029 0.392192   0.73951086]
     [0.90705961 0.44941724 0.42117882]] [1.13170286 1.07059605]
    [[0.57217969 0.47343188]
     [0.85301956 0.35434751]] [1.11256864 1.10032681]
    [[ 1.95593979  2.09234349]
     [-0.24486755 -0.16641083]
     [-0.31107224 -0.42593266]] [2.61643489 0.0599269  0.32363821]
    Iteration:  104
    [0.99780722 0.00112199 0.00107079]
    Loss:  0.002195186576402195
    Updated Weights and Biases
    [[0.11320428 0.39242996 0.73961284]
     [0.90707991 0.4495594  0.42123974]] [1.1320428  1.07079914]
    [[0.57260999 0.47385243]
     [0.85338426 0.35470394]] [1.11283112 1.10054927]
    [[ 1.95800795  2.09443867]
     [-0.24592577 -0.16748288]
     [-0.31208218 -0.42695579]] [2.61862767 0.05880491 0.32256741]
    Iteration:  105
    [0.99782828 0.00111115 0.00106057]
    Loss:  0.002174081506191195
    Updated Weights and Biases
    [[0.11323795 0.39266566 0.73971385]
     [0.90710004 0.44970026 0.42130011]] [1.13237952 1.07100038]
    [[0.5730361  0.47426882]
     [0.8537454  0.35505686]] [1.11309094 1.10076949]
    [[ 1.9600565   2.09651391]
     [-0.2469739  -0.16854467]
     [-0.3130826  -0.42796924]] [2.62079939 0.05769377 0.32150684]
    Iteration:  106
    [0.99784894 0.00110052 0.00105054]
    Loss:  0.0021533765939656634
    Updated Weights and Biases
    [[0.11327131 0.39289914 0.73981392]
     [0.90711998 0.44983985 0.42135994]] [1.13271306 1.07119978]
    [[0.57345809 0.47468114]
     [0.85410306 0.35540632]] [1.11334818 1.10098751]
    [[ 1.96208579  2.09856958]
     [-0.24801212 -0.16959638]
     [-0.31407367 -0.4289732 ]] [2.62295045 0.05659325 0.3204563 ]
    Iteration:  107
    [0.99786921 0.00109009 0.0010407 ]
    Loss:  0.0021330605869477495
    Updated Weights and Biases
    [[0.11330435 0.39313044 0.73991304]
     [0.90713974 0.44997818 0.42141922]] [1.13304348 1.0713974 ]
    [[0.57387604 0.47508947]
     [0.8544573  0.3557524 ]] [1.11360287 1.10120338]
    [[ 1.96409619  2.10060606]
     [-0.24904062 -0.17063822]
     [-0.31505558 -0.42996784]] [2.62508124 0.05550316 0.3194156 ]
    Iteration:  108
    [0.99788911 0.00107985 0.00103104]
    Loss:  0.0021131226499489808
    Updated Weights and Biases
    [[0.11333708 0.39335959 0.74001125]
     [0.90715933 0.45011528 0.42147798]] [1.13337085 1.07159326]
    [[0.57429003 0.47549387]
     [0.85480818 0.35609515]] [1.11385507 1.10141712]
    [[ 1.96608806  2.10262369]
     [-0.25005958 -0.17167036]
     [-0.31602848 -0.43095333]] [2.62719213 0.05442331 0.31838456]
    Iteration:  109
    [0.99790864 0.0010698  0.00102156]
    Loss:  0.0020935523461901124
    Updated Weights and Biases
    [[0.11336952 0.39358665 0.74010856]
     [0.90717874 0.45025117 0.42153622]] [1.13369522 1.07178739]
    [[0.57470013 0.47589442]
     [0.85515576 0.35643464]] [1.11410481 1.1016288 ]
    [[ 1.96806172  2.10462282]
     [-0.25106917 -0.17269299]
     [-0.31699255 -0.43192984]] [2.62928349 0.05335351 0.317363  ]
    Iteration:  110
    [0.99792781 0.00105994 0.00101225]
    Loss:  0.002074339619165226
    Updated Weights and Biases
    [[0.11340166 0.39381164 0.74020499]
     [0.90719798 0.45038587 0.42159394]] [1.13401663 1.07197982]
    [[0.5751064  0.47629118]
     [0.8555001  0.35677093]] [1.11435215 1.10183844]
    [[ 1.9700175   2.10660379]
     [-0.25206957 -0.17370626]
     [-0.31794794 -0.43289753]] [2.63135568 0.05229357 0.31635075]
    Iteration:  111
    [0.99794664 0.00105026 0.00100311]
    Loss:  0.0020554747754891254
    Updated Weights and Biases
    [[0.11343352 0.39403461 0.74030055]
     [0.90721706 0.4505194  0.42165117]] [1.13433516 1.07217058]
    [[0.57550892 0.47668424]
     [0.85584127 0.35710407]] [1.11459713 1.10204607]
    [[ 1.97195574  2.10856692]
     [-0.25306094 -0.17471037]
     [-0.3188948  -0.43385656]] [2.63340905 0.05124331 0.31534764]
    Iteration:  112
    [9.97965125e-01 1.04074498e-03 9.94130315e-04]
    Loss:  0.0020369484686639963
    Updated Weights and Biases
    [[0.11346508 0.39425558 0.74039525]
     [0.90723597 0.45065179 0.42170791]] [1.13465083 1.0723597 ]
    [[0.57590776 0.47707365]
     [0.85617931 0.35743412]] [1.11483979 1.10225175]
    [[ 1.97387674  2.11051254]
     [-0.25404344 -0.17570546]
     [-0.3198333  -0.43480708]] [2.63544392 0.05020257 0.31435351]
    Iteration:  113
    [9.97983285e-01 1.03140455e-03 9.85310829e-04]
    Loss:  0.0020187516837080126
    Updated Weights and Biases
    [[0.11349637 0.3944746  0.74048911]
     [0.90725472 0.45078304 0.42176416]] [1.13496372 1.0725472 ]
    [[0.57630297 0.47745948]
     [0.85651429 0.35776115]] [1.11508018 1.10245549]
    [[ 1.9757808   2.11244094]
     [-0.25501723 -0.1766917 ]
     [-0.32076357 -0.43574924]] [2.63746064 0.04917117 0.3133682 ]
    Iteration:  114
    [9.98001125e-01 1.02222921e-03 9.76646099e-04]
    Loss:  0.002000875722595269
    Updated Weights and Biases
    [[0.11352739 0.3946917  0.74058216]
     [0.90727331 0.45091319 0.42181994]] [1.13527385 1.07273312]
    [[0.57669463 0.47784179]
     [0.85684625 0.35808519]] [1.11531832 1.10265734]
    [[ 1.97766822  2.11435244]
     [-0.25598246 -0.17766924]
     [-0.32168576 -0.4366832 ]] [2.63945951 0.04814894 0.31239155]
    Iteration:  115
    [9.98018653e-01 1.01321464e-03 9.68132090e-04]
    Loss:  0.0019833121904558916
    Updated Weights and Biases
    [[0.11355813 0.3949069  0.74067439]
     [0.90729175 0.45104224 0.42187525]] [1.13558129 1.07291749]
    [[0.57708279 0.47822065]
     [0.85717526 0.3584063 ]] [1.11555428 1.10285733]
    [[ 1.97953929  2.11624733]
     [-0.25693928 -0.17863824]
     [-0.32260001 -0.43760908]] [2.64144086 0.04713572 0.31142342]
    Iteration:  116
    [9.98035878e-01 1.00435666e-03 9.59764906e-04]
    Loss:  0.001966052982488544
    Updated Weights and Biases
    [[0.11358861 0.39512025 0.74076582]
     [0.90731003 0.45117022 0.42193009]] [1.13588607 1.07310031]
    [[0.57746752 0.47859611]
     [0.85750135 0.35872454]] [1.11578807 1.1030555 ]
    [[ 1.98139429  2.11812588]
     [-0.25788784 -0.17959885]
     [-0.32350645 -0.43852703]] [2.64340498 0.04613136 0.31046365]
    Iteration:  117
    [9.98052808e-01 9.95651241e-04 9.51540787e-04]
    Loss:  0.0019490902715484674
    Updated Weights and Biases
    [[0.11361882 0.39533177 0.74085647]
     [0.90732816 0.45129714 0.42198449]] [1.13618824 1.07328164]
    [[0.57784887 0.47896823]
     [0.85782458 0.35903996]] [1.11601975 1.10325187]
    [[ 1.98323349  2.11998839]
     [-0.25882827 -0.1805512 ]
     [-0.32440522 -0.43943719]] [2.64535217 0.04513571 0.30951211]
    Iteration:  118
    [9.98069449e-01 9.87094484e-04 9.43456098e-04]
    Loss:  0.0019324164963639779
    Updated Weights and Biases
    [[0.11364878 0.39554149 0.74094635]
     [0.90734615 0.45142303 0.42203844]] [1.13648784 1.07346147]
    [[0.57822691 0.47933708]
     [0.85814501 0.35935259]] [1.11624934 1.10344647]
    [[ 1.98505717  2.12183512]
     [-0.25976072 -0.18149543]
     [-0.32529645 -0.44033969]] [2.64728272 0.04414862 0.30856866]
    Iteration:  119
    [9.98085810e-01 9.78682622e-04 9.35507325e-04]
    Loss:  0.001916024350349943
    Updated Weights and Biases
    [[0.11367849 0.39574945 0.74103548]
     [0.90736399 0.4515479  0.42209196]] [1.13678492 1.07363985]
    [[0.57860167 0.4797027 ]
     [0.85846266 0.35966249]] [1.11647688 1.10363934]
    [[ 1.98686557  2.12366634]
     [-0.26068532 -0.18243169]
     [-0.32618025 -0.44123464]] [2.64919691 0.04316994 0.30763315]
    Iteration:  120
    [9.98101897e-01 9.70412016e-04 9.27691074e-04]
    Loss:  0.0018999067709793184
    Updated Weights and Biases
    [[0.11370795 0.39595567 0.74112386]
     [0.90738168 0.45167176 0.42214504]] [1.13707952 1.07381679]
    [[0.57897323 0.48006515]
     [0.8587776  0.35996971]] [1.11670242 1.1038305 ]
    [[ 1.98865896  2.1254823 ]
     [-0.2616022  -0.18336011]
     [-0.32705677 -0.44212219]] [2.65109502 0.04219952 0.30670546]
    Iteration:  121
    [9.98117717e-01 9.62279147e-04 9.20004061e-04]
    Loss:  0.0018840569296841865
    Updated Weights and Biases
    [[0.11373717 0.39616018 0.74121151]
     [0.90739923 0.45179463 0.4221977 ]] [1.13737168 1.07399232]
    [[0.57934164 0.48042448]
     [0.85908987 0.36027429]] [1.11692597 1.10401999]
    [[ 1.99043758  2.12728326]
     [-0.26251148 -0.18428082]
     [-0.3279261  -0.44300245]] [2.6529773  0.04123725 0.30578546]
    Iteration:  122
    [9.98133276e-01 9.54280612e-04 9.12443111e-04]
    Loss:  0.001868468222253086
    Updated Weights and Biases
    [[0.11376614 0.39636301 0.74129843]
     [0.90741665 0.45191652 0.42224994]] [1.13766144 1.07416646]
    [[0.57970694 0.48078074]
     [0.85939951 0.36057627]] [1.11714757 1.10420783]
    [[ 1.99220168  2.12906947]
     [-0.2634133  -0.18519394]
     [-0.32878838 -0.44387553]] [2.65484402 0.04028296 0.30487301]
    Iteration:  123
    [9.98148582e-01 9.46413117e-04 9.05005149e-04]
    Loss:  0.0018531342596963256
    Updated Weights and Biases
    [[0.11379488 0.39656418 0.74138465]
     [0.90743392 0.45203746 0.42230177]] [1.13794883 1.07433923]
    [[0.58006918 0.48113399]
     [0.85970656 0.36087569]] [1.11736727 1.10439405]
    [[ 1.99395148  2.13084116]
     [-0.26430777 -0.18609959]
     [-0.32964372 -0.44474156]] [2.65669544 0.03933655 0.30396801]
    Iteration:  124
    [9.98163639e-01 9.38673479e-04 8.97687203e-04]
    Loss:  0.0018380488595557678
    Updated Weights and Biases
    [[0.11382339 0.39676373 0.74147017]
     [0.90745107 0.45215746 0.4223532 ]] [1.1382339  1.07451065]
    [[0.58042843 0.48148427]
     [0.86001107 0.36117261]] [1.11758508 1.10457868]
    [[ 1.99568723  2.13259856]
     [-0.26519501 -0.18699791]
     [-0.33049222 -0.44560065]] [2.6585318  0.03839788 0.30307032]
    Iteration:  125
    [9.98178455e-01 9.31058613e-04 8.90486394e-04]
    Loss:  0.0018232060376306939
    Updated Weights and Biases
    [[0.11385167 0.39696168 0.741555  ]
     [0.90746807 0.45227652 0.42240422]] [1.13851668 1.07468075]
    [[0.58078471 0.48183163]
     [0.86031308 0.36146705]] [1.11780103 1.10476173]
    [[ 1.99740913  2.13434191]
     [-0.26607514 -0.187889  ]
     [-0.33133399 -0.44645291]] [2.66035335 0.03746682 0.30217983]
    Iteration:  126
    [9.98193035e-01 9.23565534e-04 8.83399934e-04]
    Loss:  0.001808600000098595
    Updated Weights and Biases
    [[0.11387972 0.39715804 0.74163916]
     [0.90748495 0.45239468 0.42245486]] [1.13879721 1.07484954]
    [[0.58113809 0.48217612]
     [0.86061262 0.36175906]] [1.11801517 1.10494325]
    [[ 1.99911742  2.13607143]
     [-0.26694827 -0.18877298]
     [-0.33216915 -0.44729845]] [2.66216031 0.03654325 0.30129643]
    Iteration:  127
    [9.98207384e-01 9.16191352e-04 8.76425124e-04]
    Loss:  0.001794225136008853
    Updated Weights and Biases
    [[0.11390755 0.39735286 0.74172265]
     [0.9075017  0.45251193 0.42250511]] [1.13907551 1.07501704]
    [[0.5814886  0.48251778]
     [0.86090974 0.36204867]] [1.11822751 1.10512324]
    [[ 2.0008123   2.13778733]
     [-0.26781451 -0.18964996]
     [-0.33299779 -0.44813737]] [2.66395293 0.03562706 0.30042001]
    Iteration:  128
    [9.98221507e-01 9.08933266e-04 8.69559349e-04]
    Loss:  0.0017800760101280703
    Updated Weights and Biases
    [[0.11393516 0.39754615 0.74180549]
     [0.90751833 0.45262829 0.42255498]] [1.13935164 1.07518328]
    [[0.58183629 0.48285667]
     [0.86120447 0.36233593]] [1.11843809 1.10530174]
    [[ 2.00249399  2.13948983]
     [-0.26867397 -0.19052006]
     [-0.33382002 -0.44896977]] [2.66573142 0.03471813 0.29955045]
    Iteration:  129
    [9.98235411e-01 9.01788563e-04 8.62800073e-04]
    Loss:  0.0017661473561193636
    Updated Weights and Biases
    [[0.11396256 0.39773793 0.74188768]
     [0.90753483 0.45274378 0.42260448]] [1.13962561 1.07534826]
    [[0.58218121 0.48319281]
     [0.86149684 0.36262087]] [1.11864694 1.10547877]
    [[ 2.00416268  2.14117914]
     [-0.26952675 -0.19138337]
     [-0.33463594 -0.44979576]] [2.66749601 0.03381634 0.29868765]
    Iteration:  130
    [9.98249101e-01 8.94754614e-04 8.56144840e-04]
    Loss:  0.0017524340700357335
    Updated Weights and Biases
    [[0.11398975 0.39792823 0.74196924]
     [0.9075512  0.45285841 0.42265361]] [1.13989746 1.07551202]
    [[0.5825234  0.48352625]
     [0.86178691 0.36290353]] [1.11885407 1.10565436]
    [[ 2.00581859  2.14285545]
     [-0.27037296 -0.19224001]
     [-0.33544563 -0.45061544]] [2.66924691 0.03292159 0.2978315 ]
    Iteration:  131
    [9.98262580e-01 8.87828870e-04 8.49591269e-04]
    Loss:  0.0017389312041129511
    Updated Weights and Biases
    [[0.11401672 0.39811706 0.74205017]
     [0.90756746 0.45297219 0.42270237]] [1.14016723 1.07567456]
    [[0.5828629  0.48385704]
     [0.8620747  0.36318394]] [1.11905952 1.10582852]
    [[ 2.00746189  2.14451898]
     [-0.27121269 -0.19309008]
     [-0.3362492  -0.45142889]] [2.67098433 0.03203376 0.29698191]
    Iteration:  132
    [9.98275854e-01 8.81008860e-04 8.43137051e-04]
    Loss:  0.0017256339608448688
    Updated Weights and Biases
    [[0.11404349 0.39830446 0.74213048]
     [0.90758359 0.45308514 0.42275077]] [1.14043495 1.07583591]
    [[0.58319975 0.48418522]
     [0.86236024 0.36346213]] [1.11926332 1.10600128]
    [[ 2.00909279  2.1461699 ]
     [-0.27204605 -0.19393367]
     [-0.33704674 -0.45223622]] [2.67270848 0.03115275 0.29613878]
    Iteration:  133
    [9.98288928e-01 8.74292187e-04 8.36779944e-04]
    Loss:  0.001712537687324052
    Updated Weights and Biases
    [[0.11407006 0.39849045 0.74221019]
     [0.90759961 0.45319726 0.42279883]] [1.14070064 1.07599609]
    [[0.58353398 0.48451082]
     [0.86264358 0.36373815]] [1.11946548 1.10617265]
    [[ 2.01071146  2.14780841]
     [-0.27287313 -0.19477089]
     [-0.33783833 -0.45303752]] [2.67441955 0.03027846 0.295302  ]
    Iteration:  134
    [9.98301806e-01 8.67676527e-04 8.30517777e-04]
    Loss:  0.0016996378698401003
    Updated Weights and Biases
    [[0.11409643 0.39867504 0.7422893 ]
     [0.90761551 0.45330857 0.42284653]] [1.14096434 1.0761551 ]
    [[0.58386565 0.48483389]
     [0.86292474 0.36401201]] [1.11966604 1.10634267]
    [[ 2.0123181   2.14943469]
     [-0.27369403 -0.19560183]
     [-0.33862407 -0.45383287]] [2.67611774 0.02941078 0.29447148]
    Iteration:  135
    [9.98314492e-01 8.61159623e-04 8.24348439e-04]
    Loss:  0.0016869301287143444
    Updated Weights and Biases
    [[0.11412261 0.39885825 0.74236782]
     [0.9076313  0.45341908 0.42289389]] [1.14122607 1.07631297]
    [[0.58419479 0.48515446]
     [0.86320375 0.36428377]] [1.11986502 1.10651135]
    [[ 2.01391288  2.15104894]
     [-0.27450883 -0.19642657]
     [-0.33940404 -0.45462236]] [2.67780325 0.02854962 0.29364713]
    Iteration:  136
    [9.98326991e-01 8.54739287e-04 8.18269884e-04]
    Loss:  0.001674410213365405
    Updated Weights and Biases
    [[0.11414859 0.3990401  0.74244576]
     [0.90764697 0.4535288  0.42294092]] [1.14148586 1.07646972]
    [[0.58452143 0.48547257]
     [0.86348065 0.36455344]] [1.12006244 1.1066787 ]
    [[ 2.01549596  2.15265131]
     [-0.27531763 -0.19724523]
     [-0.34017833 -0.45540608]] [2.67947626 0.02769488 0.29282886]
    Iteration:  137
    [9.98339306e-01 8.48413393e-04 8.12280125e-04]
    Loss:  0.0016620739975900956
    Updated Weights and Biases
    [[0.11417437 0.39922062 0.74252312]
     [0.90766254 0.45363775 0.42298761]] [1.14174375 1.07662535]
    [[0.58484561 0.48578826]
     [0.86375547 0.36482105]] [1.12025832 1.10684476]
    [[ 2.01706753  2.15424199]
     [-0.27612051 -0.19805787]
     [-0.34094702 -0.45618412]] [2.68113695 0.02684647 0.29201658]
    Iteration:  138
    [9.98351443e-01 8.42179878e-04 8.06377232e-04]
    Loss:  0.001649917475049159
    Updated Weights and Biases
    [[0.11419998 0.39939983 0.74259993]
     [0.90767799 0.45374592 0.42303397]] [1.14199976 1.07677989]
    [[0.58516737 0.48610155]
     [0.86402824 0.36508665]] [1.1204527  1.10700954]
    [[ 2.01862775  2.15582114]
     [-0.27691756 -0.19886459]
     [-0.34171019 -0.45695655]] [2.68278551 0.02600429 0.2912102 ]
    Iteration:  139
    [9.98363404e-01 8.36036738e-04 8.00559330e-04]
    Loss:  0.0016379367549498873
    Updated Weights and Biases
    [[0.11422539 0.39957773 0.74267617]
     [0.90769333 0.45385334 0.42308   ]] [1.14225391 1.07693335]
    [[0.58548674 0.4864125 ]
     [0.86429899 0.36535025]] [1.12064558 1.10717306]
    [[ 2.02017678  2.15738894]
     [-0.27770887 -0.19966548]
     [-0.34246791 -0.45772345]] [2.68442211 0.02516825 0.29040964]
    Iteration:  140
    [9.98375193e-01 8.29982028e-04 7.94824600e-04]
    Loss:  0.001626128057911233
    Updated Weights and Biases
    [[0.11425062 0.39975436 0.74275187]
     [0.90770857 0.45396002 0.42312572]] [1.14250623 1.07708574]
    [[0.58580376 0.48672112]
     [0.86456774 0.36561189]] [1.12083699 1.10733533]
    [[ 2.02171478  2.15894553]
     [-0.27849451 -0.20046062]
     [-0.34322027 -0.45848491]] [2.68604691 0.02433827 0.28961482]
    Iteration:  141
    [9.98386815e-01 8.24013857e-04 7.89171271e-04]
    Loss:  0.001614487712008471
    Updated Weights and Biases
    [[0.11427568 0.39992973 0.74282703]
     [0.90772371 0.45406596 0.42317113]] [1.14275675 1.07723708]
    [[0.58611845 0.48702745]
     [0.86483453 0.36587159]] [1.12102696 1.10749638]
    [[ 2.02324191  2.16049109]
     [-0.27927457 -0.20125009]
     [-0.34396735 -0.459241  ]] [2.6876601  0.02351425 0.28882565]
    Iteration:  142
    [9.98398272e-01 8.18130386e-04 7.83597625e-04]
    Loss:  0.0016030121489822463
    Updated Weights and Biases
    [[0.11430055 0.40010385 0.74290165]
     [0.90773874 0.45417117 0.42321622]] [1.14300549 1.07738739]
    [[0.58643086 0.48733153]
     [0.86509939 0.36612938]] [1.1212155  1.10765622]
    [[ 2.02475832  2.16202577]
     [-0.28004912 -0.20203398]
     [-0.3447092  -0.45999179]] [2.68926183 0.02269612 0.28804205]
    Iteration:  143
    [9.98409568e-01 8.12329831e-04 7.78101990e-04]
    Loss:  0.001591697900608184
    Updated Weights and Biases
    [[0.11432525 0.40027674 0.74297575]
     [0.90775367 0.45427567 0.423261  ]] [1.14325249 1.07753667]
    [[0.58674102 0.48763339]
     [0.86536233 0.36638529]] [1.12140264 1.10781487]
    [[ 2.02626416  2.16354971]
     [-0.28081824 -0.20281235]
     [-0.34544592 -0.46073737]] [2.69085226 0.02188379 0.28726395]
    Iteration:  144
    [9.98420707e-01 8.06610455e-04 7.72682742e-04]
    Loss:  0.0015805415952184626
    Updated Weights and Biases
    [[0.11434977 0.40044842 0.74304932]
     [0.90776849 0.45437946 0.42330548]] [1.14349775 1.07768495]
    [[0.58704895 0.48793306]
     [0.86562339 0.36663935]] [1.12158839 1.10797235]
    [[ 2.02775957  2.16506307]
     [-0.28158201 -0.20358528]
     [-0.34617756 -0.46147779]] [2.69243155 0.02107718 0.28649127]
    Iteration:  145
    [9.98431691e-01 8.00970571e-04 7.67338300e-04]
    Loss:  0.001569539954366085
    Updated Weights and Biases
    [[0.11437413 0.40061891 0.74312239]
     [0.90778322 0.45448256 0.42334967]] [1.14374131 1.07783223]
    [[0.58735469 0.48823057]
     [0.8658826  0.36689157]] [1.12177277 1.10812867]
    [[ 2.02924471  2.166566  ]
     [-0.2823405  -0.20435286]
     [-0.34690421 -0.46221314]] [2.69399986 0.02027621 0.28572393]
    Iteration:  146
    [9.98442524e-01 7.95408536e-04 7.62067128e-04]
    Loss:  0.0015586897896275864
    Updated Weights and Biases
    [[0.11439832 0.40078823 0.74319495]
     [0.90779785 0.45458497 0.42339356]] [1.14398318 1.07797853]
    [[0.58765827 0.48852595]
     [0.86613997 0.36714199]] [1.12195581 1.10828385]
    [[ 2.0307197   2.16805863]
     [-0.28309378 -0.20511515]
     [-0.34762591 -0.46294348]] [2.69555734 0.0194808  0.28496186]
    Iteration:  147
    [9.98453210e-01 7.89922754e-04 7.56867730e-04]
    Loss:  0.0015479879995356999
    Updated Weights and Biases
    [[0.11442234 0.40095638 0.74326702]
     [0.90781239 0.4546867  0.42343716]] [1.1442234  1.07812386]
    [[0.58795971 0.48881922]
     [0.86639554 0.36739064]] [1.12213752 1.1084379 ]
    [[ 2.03218469  2.16954111]
     [-0.28384193 -0.20587223]
     [-0.34834276 -0.46366888]] [2.69710413 0.01869088 0.28420499]
    Iteration:  148
    [9.98463750e-01 7.84511673e-04 7.51738652e-04]
    Loss:  0.0015374315666371642
    Updated Weights and Biases
    [[0.1144462  0.40112338 0.74333859]
     [0.90782682 0.45478776 0.42348047]] [1.14446198 1.07826823]
    [[0.58825905 0.48911042]
     [0.86664932 0.36763753]] [1.12231792 1.10859085]
    [[ 2.03363981  2.17101357]
     [-0.28458501 -0.20662417]
     [-0.3490548  -0.4643894 ]] [2.69864038 0.01790637 0.28345325]
    Iteration:  149
    [9.98474148e-01 7.79173779e-04 7.46678478e-04]
    Loss:  0.0015270175546686453
    Updated Weights and Biases
    [[0.11446989 0.40128926 0.74340968]
     [0.90784117 0.45488816 0.4235235 ]] [1.14469894 1.07841166]
    [[0.58855631 0.48939958]
     [0.86690135 0.36788268]] [1.12249703 1.1087427 ]
    [[ 2.0350852   2.17247615]
     [-0.2853231  -0.20737103]
     [-0.3497621  -0.46510512]] [2.70016623 0.0171272  0.28270657]
    Iteration:  150
    [9.98484407e-01 7.73907602e-04 7.41685830e-04]
    Loss:  0.0015167431058457383
    Updated Weights and Biases
    [[0.11449343 0.40145402 0.74348029]
     [0.90785542 0.45498791 0.42356625]] [1.14493431 1.07855416]
    [[0.58885153 0.48968672]
     [0.86715165 0.36812613]] [1.12267486 1.10889347]
    [[ 2.03652097  2.17392898]
     [-0.28605625 -0.20811289]
     [-0.35046472 -0.46581609]] [2.70168182 0.01635329 0.28196489]
    Iteration:  151
    [9.98494529e-01 7.68711711e-04 7.36759367e-04]
    Loss:  0.0015066054382599117
    Updated Weights and Biases
    [[0.11451681 0.40161768 0.74355043]
     [0.90786957 0.45508702 0.42360872]] [1.14516811 1.07869574]
    [[0.58914472 0.48997188]
     [0.86740023 0.3683679 ]] [1.12285143 1.10904318]
    [[ 2.03794727  2.17537218]
     [-0.28678454 -0.20884981]
     [-0.35116274 -0.46652238]] [2.70318729 0.01558458 0.28122813]
    Iteration:  152
    [9.98504518e-01 7.63584710e-04 7.31897783e-04]
    Loss:  0.0014966018433795916
    Updated Weights and Biases
    [[0.11454004 0.40178025 0.74362011]
     [0.90788364 0.45518549 0.42365092]] [1.14540035 1.07883641]
    [[0.58943592 0.49025506]
     [0.86764713 0.368608  ]] [1.12302677 1.10919184]
    [[ 2.03936422  2.1768059 ]
     [-0.28750802 -0.20958185]
     [-0.3518562  -0.46722404]] [2.70468278 0.01482099 0.28049623]
    Iteration:  153
    [9.98514375e-01 7.58525243e-04 7.27099806e-04]
    Loss:  0.00148672968364602
    Updated Weights and Biases
    [[0.11456311 0.40194175 0.74368932]
     [0.90789762 0.45528333 0.42369285]] [1.14563107 1.07897618]
    [[0.58972515 0.49053632]
     [0.86789236 0.36884647]] [1.12320088 1.10933947]
    [[ 2.04077193  2.17823024]
     [-0.28822676 -0.21030909]
     [-0.35254517 -0.46792115]] [2.7061684  0.01406247 0.27976913]
    Iteration:  154
    [9.98524104e-01 7.53531986e-04 7.22364196e-04]
    Loss:  0.0014769863901669855
    Updated Weights and Biases
    [[0.11458603 0.40210219 0.74375808]
     [0.90791151 0.45538055 0.42373452]] [1.14586027 1.07911507]
    [[0.59001244 0.49081565]
     [0.86813594 0.36908331]] [1.12337379 1.10948607]
    [[ 2.04217053  2.17964533]
     [-0.28894083 -0.21103158]
     [-0.3532297  -0.46861375]] [2.7076443  0.01330894 0.27904677]
    Iteration:  155
    [9.98533707e-01 7.48603653e-04 7.17689747e-04]
    Loss:  0.0014673694604977285
    Updated Weights and Biases
    [[0.1146088  0.40226158 0.74382639]
     [0.90792531 0.45547716 0.42377593]] [1.14608797 1.07925309]
    [[0.59029781 0.49109311]
     [0.86837791 0.36931856]] [1.1235455  1.10963166]
    [[ 2.04356013  2.18105129]
     [-0.28965028 -0.21174938]
     [-0.35390985 -0.46930191]] [2.70911059 0.01256033 0.27832908]
    Iteration:  156
    [9.98543186e-01 7.43738988e-04 7.13075283e-04]
    Loss:  0.001457876456506448
    Updated Weights and Biases
    [[0.11463142 0.40241994 0.74389426]
     [0.90793902 0.45557317 0.42381707]] [1.1463142  1.07939024]
    [[0.59058129 0.4913687 ]
     [0.86861827 0.36955223]] [1.12371603 1.10977626]
    [[ 2.04494085  2.18244824]
     [-0.29035517 -0.21246255]
     [-0.35458568 -0.46998568]] [2.7105674  0.01181659 0.277616  ]
    Iteration:  157
    [9.98552544e-01 7.38936769e-04 7.08519657e-04]
    Loss:  0.0014485050023240573
    Updated Weights and Biases
    [[0.1146539  0.40257729 0.74396169]
     [0.90795265 0.45566858 0.42385796]] [1.14653898 1.07952654]
    [[0.5908629  0.49164245]
     [0.86885705 0.36978434]] [1.12388541 1.10991987]
    [[ 2.0463128   2.18383629]
     [-0.29105556 -0.21317116]
     [-0.35525724 -0.47066513]] [2.71201486 0.01107766 0.27690748]
    Iteration:  158
    [9.98561782e-01 7.34195803e-04 7.04021752e-04]
    Loss:  0.0014392527823696138
    Updated Weights and Biases
    [[0.11467623 0.40273362 0.7440287 ]
     [0.9079662  0.4557634  0.4238986 ]] [1.14676232 1.079662  ]
    [[0.59114267 0.49191439]
     [0.86909426 0.37001492]] [1.12405364 1.11006252]
    [[ 2.0476761   2.18521555]
     [-0.29175151 -0.21387526]
     [-0.35592459 -0.47134029]] [2.71345308 0.01034346 0.27620346]
    Iteration:  159
    [9.98570905e-01 7.29514930e-04 6.99580479e-04]
    Loss:  0.0014301175394514048
    Updated Weights and Biases
    [[0.11469842 0.40288897 0.74409527]
     [0.90797966 0.45585763 0.42393899]] [1.14698424 1.07979662]
    [[0.59142062 0.49218453]
     [0.86932994 0.37024398]] [1.12422074 1.1102042 ]
    [[ 2.04903084  2.18658614]
     [-0.29244307 -0.21457491]
     [-0.35658777 -0.47201123]] [2.71488217 0.00961395 0.27550388]
    Iteration:  160
    [9.98579912e-01 7.24893016e-04 6.95194776e-04]
    Loss:  0.0014210970729397854
    Updated Weights and Biases
    [[0.11472048 0.40304333 0.74416143]
     [0.90799304 0.4559513  0.42397913]] [1.14720476 1.07993043]
    [[0.59169677 0.49245291]
     [0.86956409 0.37047155]] [1.12438673 1.11034495]
    [[ 2.05037715  2.18794817]
     [-0.2931303  -0.21527017]
     [-0.35724685 -0.472678  ]] [2.71630226 0.00888905 0.27480869]
    Iteration:  161
    [9.98588807e-01 7.20328959e-04 6.90863608e-04]
    Loss:  0.0014121892370065287
    Updated Weights and Biases
    [[0.11474239 0.40319672 0.74422717]
     [0.90800634 0.45604439 0.42401903]] [1.14742389 1.08006342]
    [[0.59197114 0.49271954]
     [0.86979675 0.37069763]] [1.12455161 1.11048476]
    [[ 2.05171511  2.18930173]
     [-0.29381325 -0.21596108]
     [-0.35790186 -0.47334065]] [2.71771345 0.00816872 0.27411782]
    Iteration:  162
    [9.98597592e-01 7.15821680e-04 6.86585965e-04]
    Loss:  0.0014033919389312309
    Updated Weights and Biases
    [[0.11476417 0.40334916 0.7442925 ]
     [0.90801956 0.45613693 0.42405868]] [1.14764166 1.08019561]
    [[0.59224376 0.49298444]
     [0.87002791 0.37092226]] [1.1247154  1.11062364]
    [[ 2.05304484  2.19064693]
     [-0.29449198 -0.2166477 ]
     [-0.35855287 -0.47399923]] [2.71911586 0.0074529  0.27343124]
    Iteration:  163
    [9.98606269e-01 7.11370129e-04 6.82360862e-04]
    Loss:  0.0013947031374693126
    Updated Weights and Biases
    [[0.11478581 0.40350065 0.74435742]
     [0.9080327  0.4562289  0.4240981 ]] [1.14785807 1.08032701]
    [[0.59251465 0.49324764]
     [0.87025761 0.37114544]] [1.12487812 1.11076162]
    [[ 2.05436644  2.19198388]
     [-0.29516653 -0.21733009]
     [-0.35919991 -0.47465379]] [2.72050959 0.00674153 0.27274888]
    Iteration:  164
    [9.98614839e-01 7.06973282e-04 6.78187337e-04]
    Loss:  0.0013861208412796022
    Updated Weights and Biases
    [[0.11480732 0.40365121 0.74442195]
     [0.90804576 0.45632033 0.42413729]] [1.14807315 1.08045762]
    [[0.59278383 0.49350916]
     [0.87048587 0.3713672 ]] [1.12503978 1.1108987 ]
    [[ 2.05568     2.19331268]
     [-0.29583696 -0.2180083 ]
     [-0.35984304 -0.47530438]] [2.72189475 0.00603456 0.27207069]
    Iteration:  165
    [9.98623305e-01 7.02630140e-04 6.74064453e-04]
    Loss:  0.0013776431074100461
    Updated Weights and Biases
    [[0.11482869 0.40380084 0.74448608]
     [0.90805875 0.45641122 0.42417624]] [1.14828692 1.08058746]
    [[0.59305131 0.49376902]
     [0.87071269 0.37158755]] [1.12520039 1.1110349 ]
    [[ 2.05698562  2.19463343]
     [-0.29650332 -0.21868237]
     [-0.36048231 -0.47595105]] [2.72327145 0.00533193 0.27139662]
    Iteration:  166
    [9.98631669e-01 6.98339726e-04 6.69991294e-04]
    Loss:  0.0013692680398371996
    Updated Weights and Biases
    [[0.11484994 0.40394956 0.74454981]
     [0.90807165 0.45650158 0.42421496]] [1.14849938 1.08071654]
    [[0.59331713 0.49402723]
     [0.8709381  0.37180651]] [1.12535997 1.11117022]
    [[ 2.0582834   2.19594621]
     [-0.29716565 -0.21935237]
     [-0.36111775 -0.47659385]] [2.72463978 0.00463359 0.27072663]
    Iteration:  167
    [9.98639932e-01 6.94101089e-04 6.65966967e-04]
    Loss:  0.0013609937880588205
    Updated Weights and Biases
    [[0.11487105 0.40409738 0.74461316]
     [0.90808449 0.4565914  0.42425346]] [1.14871055 1.08084486]
    [[0.5935813  0.49428383]
     [0.87116211 0.3720241 ]] [1.12551852 1.11130467]
    [[ 2.05957344  2.19725114]
     [-0.29782401 -0.22001832]
     [-0.36174943 -0.47723281]] [2.72599985 0.00393949 0.27006067]
    Iteration:  168
    [9.98648096e-01 6.89913299e-04 6.61990600e-04]
    Loss:  0.0013528185457365566
    Updated Weights and Biases
    [[0.11489204 0.40424431 0.74467613]
     [0.90809724 0.4566807  0.42429173]] [1.14892045 1.08097243]
    [[0.59384384 0.49453882]
     [0.87138474 0.37224033]] [1.12567607 1.11143826]
    [[ 2.06085581  2.19854829]
     [-0.29847844 -0.2206803 ]
     [-0.36237737 -0.47786799]] [2.72735175 0.00324957 0.26939868]
    Iteration:  169
    [9.98656163e-01 6.85775449e-04 6.58061342e-04]
    Loss:  0.001344740549387827
    Updated Weights and Biases
    [[0.11491291 0.40439036 0.74473873]
     [0.90810993 0.45676949 0.42432978]] [1.14912909 1.08109926]
    [[0.59410477 0.49479223]
     [0.87160601 0.37245523]] [1.12583261 1.11157102]
    [[ 2.06213062  2.19983777]
     [-0.29912899 -0.22133833]
     [-0.36300163 -0.47849944]] [2.72869559e+00 2.56379858e-03 2.68740614e-01]
    Iteration:  170
    [9.98664135e-01 6.81686654e-04 6.54178360e-04]
    Loss:  0.0013367580771219986
    Updated Weights and Biases
    [[0.11493365 0.40453555 0.74480095]
     [0.90812254 0.45685776 0.42436761]] [1.14933649 1.08122537]
    [[0.59436411 0.49504407]
     [0.87182594 0.3726688 ]] [1.12598817 1.11170294]
    [[ 2.06339795  2.20111967]
     [-0.2997757  -0.22199248]
     [-0.36362225 -0.47912719]] [2.73003145e+00 1.88211193e-03 2.68086435e-01]
    Iteration:  171
    [9.98672013e-01 6.77646048e-04 6.50340843e-04]
    Loss:  0.0013288694474230729
    Updated Weights and Biases
    [[0.11495427 0.40467987 0.7448628 ]
     [0.90813508 0.45694553 0.42440523]] [1.14954267 1.08135075]
    [[0.59462187 0.49529437]
     [0.87204453 0.37288106]] [1.12614276 1.11183403]
    [[ 2.06465789  2.20239406]
     [-0.30041862 -0.22264278]
     [-0.36423926 -0.47975128]] [2.73135944e+00 1.20446588e-03 2.67436095e-01]
    Iteration:  172
    [9.98679799e-01 6.73652786e-04 6.46547999e-04]
    Loss:  0.0013210730179728718
    Updated Weights and Biases
    [[0.11497476 0.40482334 0.74492429]
     [0.90814754 0.45703279 0.42444263]] [1.14974762 1.08147542]
    [[0.59487808 0.49554314]
     [0.87226181 0.37309203]] [1.12629639 1.11196431]
    [[ 2.06591052  2.20366105]
     [-0.3010578  -0.22328928]
     [-0.36485272 -0.48037177]] [2.73267964e+00 5.30813095e-04 2.66789547e-01]
    Iteration:  173
    [9.98687495e-01 6.69706044e-04 6.42799051e-04]
    Loss:  0.001313367184517497
    Updated Weights and Biases
    [[0.11499514 0.40496597 0.74498542]
     [0.90815994 0.45711957 0.42447981]] [1.14995138 1.08159938]
    [[0.59513276 0.49579041]
     [0.87247778 0.37330172]] [1.12644906 1.11209379]
    [[ 2.06715593  2.2049207 ]
     [-0.30169327 -0.22393202]
     [-0.36546266 -0.48098869]] [ 2.73399215e+00 -1.38892949e-04  2.66146748e-01]
    Iteration:  174
    [9.98695102e-01 6.65805015e-04 6.39093244e-04]
    Loss:  0.0013057503797703827
    Updated Weights and Biases
    [[0.1150154  0.40510777 0.74504619]
     [0.90817226 0.45720585 0.42451679]] [1.15015396 1.08172265]
    [[0.59538592 0.49603618]
     [0.87269248 0.37351015]] [1.1266008  1.11222247]
    [[ 2.06839421  2.20617312]
     [-0.30232508 -0.22457105]
     [-0.36606912 -0.48160208]] [ 2.73529704e+00 -8.04697964e-04  2.65507654e-01]
    Iteration:  175
    [9.98702621e-01 6.61948911e-04 6.35429836e-04]
    Loss:  0.0012982210723550514
    Updated Weights and Biases
    [[0.11503554 0.40524875 0.74510661]
     [0.90818452 0.45729166 0.42455357]] [1.15035536 1.08184522]
    [[0.59563758 0.49628048]
     [0.8729059  0.37371733]] [1.1267516  1.11235036]
    [[ 2.06962543  2.20741838]
     [-0.30295328 -0.2252064 ]
     [-0.36667215 -0.48221198]] [ 2.73659442e+00 -1.46664688e-03  2.64872224e-01]
    Iteration:  176
    [9.98710055e-01 6.58136964e-04 6.31808107e-04]
    Loss:  0.0012907777657823398
    Updated Weights and Biases
    [[0.11505556 0.40538892 0.74516668]
     [0.90819671 0.45737698 0.42459014]] [1.1505556  1.08196712]
    [[0.59588775 0.49652332]
     [0.87311807 0.37392327]] [1.12690149 1.11247748]
    [[ 2.07084967  2.20865657]
     [-0.30357789 -0.22583813]
     [-0.36727178 -0.48281844]] [ 2.73788437e+00 -2.12478384e-03  2.64240416e-01]
    Iteration:  177
    [9.98717404e-01 6.54368419e-04 6.28227348e-04]
    Loss:  0.0012834189974637565
    Updated Weights and Biases
    [[0.11507547 0.40552829 0.74522641]
     [0.90820883 0.45746183 0.4246265 ]] [1.15075469 1.08208834]
    [[0.59613646 0.49676472]
     [0.87332899 0.374128  ]] [1.12705048 1.11260383]
    [[ 2.07206702  2.20988775]
     [-0.30419897 -0.22646627]
     [-0.36786805 -0.48342148]] [ 2.73916696 -0.00277915  0.26361219]
    Iteration:  178
    [9.98724671e-01 6.50642543e-04 6.24686870e-04]
    Loss:  0.0012761433377576328
    Updated Weights and Biases
    [[0.11509527 0.40566686 0.7452858 ]
     [0.90822089 0.45754622 0.42466267]] [1.15095266 1.08220889]
    [[0.59638372 0.4970047 ]
     [0.87353869 0.37433152]] [1.12719856 1.11272942]
    [[ 2.07327754  2.21111202]
     [-0.30481655 -0.22709086]
     [-0.36846099 -0.48402116]] [ 2.74044229 -0.00342979  0.2629875 ]
    Iteration:  179
    [9.98731855e-01 6.46958616e-04 6.21185997e-04]
    Loss:  0.0012689493890462823
    Updated Weights and Biases
    [[0.11511495 0.40580465 0.74534485]
     [0.90823288 0.45763014 0.42469863]] [1.1511495  1.08232878]
    [[0.59662955 0.49724326]
     [0.87374718 0.37453385]] [1.12734576 1.11285426]
    [[ 2.07448132  2.21232945]
     [-0.30543068 -0.22771195]
     [-0.36905065 -0.4846175 ]] [ 2.74171044 -0.00407675  0.26236632]
    Iteration:  180
    [9.98738960e-01 6.43315936e-04 6.17724069e-04]
    Loss:  0.0012618357848454957
    Updated Weights and Biases
    [[0.11513452 0.40594166 0.74540357]
     [0.9082448  0.45771361 0.4247344 ]] [1.15134523 1.08244801]
    [[0.59687395 0.49748044]
     [0.87395446 0.37473501]] [1.12749208 1.11297836]
    [[ 2.07567843  2.21354011]
     [-0.30604138 -0.22832957]
     [-0.36963706 -0.48521055]] [ 2.74297148 -0.00472007  0.26174859]
    Iteration:  181
    [9.98745986e-01 6.39713816e-04 6.14300439e-04]
    Loss:  0.0012548011889429252
    Updated Weights and Biases
    [[0.11515399 0.4060779  0.74546196]
     [0.90825666 0.45779662 0.42476998]] [1.15153986 1.08256661]
    [[0.59711696 0.49771624]
     [0.87416056 0.37493499]] [1.12763754 1.11310172]
    [[ 2.07686895  2.21474409]
     [-0.3066487  -0.22894375]
     [-0.37022025 -0.48580033]] [ 2.74422549 -0.00535978  0.26113429]
    Iteration:  182
    [9.98752934e-01 6.36151583e-04 6.10914478e-04]
    Loss:  0.0012478442945652382
    Updated Weights and Biases
    [[0.11517334 0.40621339 0.74552002]
     [0.90826846 0.45787919 0.42480537]] [1.15173341 1.08268456]
    [[0.59735858 0.49795068]
     [0.87436549 0.37513383]] [1.12778214 1.11322436]
    [[ 2.07805294  2.21594144]
     [-0.30725268 -0.22955454]
     [-0.37080027 -0.4863869 ]] [ 2.74547256 -0.00599593  0.26052338]
    Iteration:  183
    [9.98759806e-01 6.32628580e-04 6.07565566e-04]
    Loss:  0.001240963823571148
    Updated Weights and Biases
    [[0.11519259 0.40634812 0.74557777]
     [0.90828019 0.45796131 0.42484056]] [1.15192589 1.08280188]
    [[0.59759883 0.49818377]
     [0.87456925 0.37533152]] [1.1279259  1.11334629]
    [[ 2.07923048  2.21713225]
     [-0.30785334 -0.23016198]
     [-0.37137714 -0.48697027]] [ 2.74671275 -0.00662856  0.25991581]
    Iteration:  184
    [9.98766603e-01 6.29144165e-04 6.04253100e-04]
    Loss:  0.0012341585256733197
    Updated Weights and Biases
    [[0.11521173 0.40648212 0.74563519]
     [0.90829186 0.458043   0.42487557]] [1.15211731 1.08291857]
    [[0.59783772 0.49841553]
     [0.87477187 0.37552809]] [1.12806881 1.1134675 ]
    [[ 2.08040164  2.21831659]
     [-0.30845074 -0.2307661 ]
     [-0.3719509  -0.48755049]] [ 2.74794615 -0.00725771  0.25931156]
    Iteration:  185
    [9.98773326e-01 6.25697708e-04 6.00976489e-04]
    Loss:  0.0012274271776832534
    Updated Weights and Biases
    [[0.11523077 0.40661537 0.7456923 ]
     [0.90830346 0.45812425 0.42491039]] [1.15230768 1.08303465]
    [[0.59807528 0.49864597]
     [0.87497335 0.37572354]] [1.1282109  1.11358802]
    [[ 2.08156648  2.21949452]
     [-0.3090449  -0.23136694]
     [-0.37252158 -0.48812759]] [ 2.74917282 -0.00788341  0.25871058]
    Iteration:  186
    [9.98779976e-01 6.22288594e-04 5.97735154e-04]
    Loss:  0.001220768582782257
    Updated Weights and Biases
    [[0.1152497  0.4067479  0.7457491 ]
     [0.90831501 0.45820508 0.42494503]] [1.15249701 1.08315011]
    [[0.5983115  0.49887512]
     [0.87517371 0.3759179 ]] [1.12835217 1.11370784]
    [[ 2.08272508  2.22066612]
     [-0.30963585 -0.23196453]
     [-0.37308922 -0.4887016 ]] [ 2.75039285 -0.00850569  0.25811285]
    Iteration:  187
    [9.98786555e-01 6.18916221e-04 5.94528529e-04]
    Loss:  0.0012141815698143883
    Updated Weights and Biases
    [[0.11526853 0.40687972 0.74580559]
     [0.9083265  0.45828548 0.42497949]] [1.15268531 1.08326497]
    [[0.59854642 0.49910297]
     [0.87537296 0.37611116]] [1.12849263 1.11382697]
    [[ 2.08387749  2.22183146]
     [-0.31022364 -0.2325589 ]
     [-0.37365385 -0.48927255]] [ 2.75160629 -0.00912461  0.25751832]
    Iteration:  188
    [9.98793064e-01 6.15579999e-04 5.91356060e-04]
    Loss:  0.0012076649926029235
    Updated Weights and Biases
    [[0.11528726 0.40701082 0.74586178]
     [0.90833792 0.45836546 0.42501377]] [1.15287259 1.08337922]
    [[0.59878003 0.49932955]
     [0.87557111 0.37630334]] [1.12863229 1.11394543]
    [[ 2.0850238   2.22299059]
     [-0.3108083  -0.2331501 ]
     [-0.3742155  -0.48984049]] [ 2.75281323 -0.00974019  0.25692696]
    Iteration:  189
    [9.98799503e-01 6.12279352e-04 5.88217204e-04]
    Loss:  0.0012012177292883449
    Updated Weights and Biases
    [[0.11530589 0.40714121 0.74591766]
     [0.90834929 0.45844502 0.42504787]] [1.15305887 1.08349289]
    [[0.59901236 0.49955487]
     [0.87576818 0.37649446]] [1.12877115 1.11406321]
    [[ 2.08616405  2.22414359]
     [-0.31138985 -0.23373816]
     [-0.3747742  -0.49040543]] [ 2.75401372 -0.01035247  0.25633875]
    Iteration:  190
    [9.98805875e-01 6.09013715e-04 5.85111431e-04]
    Loss:  0.001194838681687179
    Updated Weights and Biases
    [[0.11532442 0.40727091 0.74597325]
     [0.9083606  0.45852417 0.42508179]] [1.15324416 1.08360596]
    [[0.59924342 0.49977894]
     [0.87596416 0.37668452]] [1.12890923 1.11418033]
    [[ 2.08729832  2.22529051]
     [-0.31196834 -0.2343231 ]
     [-0.37532998 -0.49096741]] [ 2.75520785 -0.01096148  0.25575363]
    Iteration:  191
    [9.98812179e-01 6.05782536e-04 5.82038221e-04]
    Loss:  0.0011885267746719063
    Updated Weights and Biases
    [[0.11534285 0.40739992 0.74602854]
     [0.90837185 0.45860292 0.42511554]] [1.15342846 1.08371845]
    [[0.59947323 0.50000178]
     [0.87615909 0.37687353]] [1.12904654 1.1142968 ]
    [[ 2.08842666  2.22643143]
     [-0.31254379 -0.23490496]
     [-0.37588287 -0.49152647]] [ 2.75639567 -0.01156727  0.2551716 ]
    Iteration:  192
    [9.98818418e-01 6.02585272e-04 5.78997065e-04]
    Loss:  0.0011822809555674892
    Updated Weights and Biases
    [[0.11536118 0.40752825 0.74608353]
     [0.90838304 0.45868126 0.42514911]] [1.15361178 1.08383037]
    [[0.59970179 0.5002234 ]
     [0.87635296 0.37706152]] [1.12918308 1.11441262]
    [[ 2.08954915  2.22756641]
     [-0.31311624 -0.23548378]
     [-0.37643291 -0.49208263]] [ 2.75757725 -0.01216985  0.2545926 ]
    Iteration:  193
    [9.98824591e-01 5.99421395e-04 5.75987464e-04]
    Loss:  0.0011761001935705244
    Updated Weights and Biases
    [[0.11537941 0.4076559  0.74613824]
     [0.90839417 0.4587592  0.42518252]] [1.15379414 1.08394172]
    [[0.59992912 0.50044381]
     [0.87654579 0.37724848]] [1.12931886 1.11452779]
    [[ 2.09066583  2.22869551]
     [-0.31368571 -0.23605958]
     [-0.37698012 -0.49263592]] [ 2.75875266 -0.01276927  0.25401661]
    Iteration:  194
    [9.98830701e-01 5.96290385e-04 5.73008930e-04]
    Loss:  0.001169983479183563
    Updated Weights and Biases
    [[0.11539755 0.40778288 0.74619266]
     [0.90840525 0.45883675 0.42521575]] [1.15397555 1.0840525 ]
    [[0.60015523 0.50066303]
     [0.87673759 0.37743442]] [1.12945389 1.11464233]
    [[ 2.09177677  2.22981878]
     [-0.31425224 -0.2366324 ]
     [-0.37752453 -0.49318638]] [ 2.75992196 -0.01336556  0.2534436 ]
    Iteration:  195
    [9.98836747e-01 5.93191734e-04 5.70060986e-04]
    Loss:  0.0011639298236681586
    Updated Weights and Biases
    [[0.1154156  0.40790921 0.7462468 ]
     [0.90841627 0.45891391 0.42524882]] [1.15415601 1.08416273]
    [[0.60038013 0.50088106]
     [0.87692836 0.37761937]] [1.12958818 1.11475624]
    [[ 2.09288203  2.23093629]
     [-0.31481586 -0.23720227]
     [-0.37806617 -0.49373402]] [ 2.76108521 -0.01395875  0.25287354]
    Iteration:  196
    [9.98842732e-01 5.90124944e-04 5.67143162e-04]
    Loss:  0.001157938258513971
    Updated Weights and Biases
    [[0.11543355 0.40803487 0.74630066]
     [0.90842724 0.45899068 0.42528172]] [1.15433553 1.0842724 ]
    [[0.60060384 0.50109792]
     [0.87711813 0.37780333]] [1.12972173 1.11486952]
    [[ 2.09398167  2.2320481 ]
     [-0.31537659 -0.23776921]
     [-0.37860507 -0.49427889]] [ 2.76224248 -0.01454888  0.2523064 ]
    Iteration:  197
    [9.98848655e-01 5.87089528e-04 5.64255001e-04]
    Loss:  0.0011520078349248114
    Updated Weights and Biases
    [[0.11545141 0.40815989 0.74635424]
     [0.90843815 0.45906707 0.42531446]] [1.15451413 1.08438153]
    [[0.60082638 0.50131363]
     [0.8773069  0.37798631]] [1.12985455 1.11498219]
    [[ 2.09507573  2.23315426]
     [-0.31593448 -0.23833326]
     [-0.37914126 -0.494821  ]] [ 2.76339383 -0.01513597  0.25174214]
    Iteration:  198
    [9.98854519e-01 5.84085007e-04 5.61396051e-04]
    Loss:  0.0011461376233192957
    Updated Weights and Biases
    [[0.11546918 0.40828427 0.74640755]
     [0.90844901 0.45914308 0.42534703]] [1.15469182 1.08449011]
    [[0.60104774 0.50152818]
     [0.87749468 0.37816831]] [1.12998665 1.11509426]
    [[ 2.09616429  2.23425483]
     [-0.31648954 -0.23889445]
     [-0.37967475 -0.49536038]] [ 2.76453931 -0.01572005  0.25118075]
    Iteration:  199
    [9.98860323e-01 5.81110914e-04 5.58565873e-04]
    Loss:  0.0011403267128478804
    Updated Weights and Biases
    [[0.11548686 0.40840802 0.74646058]
     [0.90845982 0.45921871 0.42537945]] [1.15486859 1.08459816]
    [[0.60126794 0.50174161]
     [0.87768148 0.37834936]] [1.13011804 1.11520571]
    [[ 2.09724739  2.23534987]
     [-0.3170418  -0.2394528 ]
     [-0.38020559 -0.49589707]] [ 2.76567898 -0.01630116  0.25062218]
    Iteration:  200
    [9.98866069e-01 5.78166789e-04 5.55764036e-04]
    Loss:  0.0011345742109229449
    Updated Weights and Biases
    [[0.11550445 0.40853113 0.74651334]
     [0.90847057 0.45929397 0.4254117 ]] [1.15504447 1.08470567]
    [[0.601487   0.50195391]
     [0.87786731 0.37852945]] [1.13024873 1.11531658]
    [[ 2.09832509  2.23643944]
     [-0.31759129 -0.24000834]
     [-0.38073379 -0.49643109]] [ 2.76681291 -0.01687933  0.25006642]
    Iteration:  201
    [9.98871758e-01 5.75252184e-04 5.52990114e-04]
    Loss:  0.0011288792427649214
    Updated Weights and Biases
    [[0.11552195 0.40865362 0.74656584]
     [0.90848127 0.45936886 0.4254438 ]] [1.15521946 1.08481266]
    [[0.60170493 0.50216509]
     [0.87805218 0.37870861]] [1.13037872 1.11542685]
    [[ 2.09939744  2.23752358]
     [-0.31813805 -0.24056111]
     [-0.38125939 -0.49696247]] [ 2.76794116 -0.01745458  0.24951343]

Comparing to Tensorflow

```python
from keras.models import Sequential
from keras.layers import Dense
from tensorflow.keras.optimizers import SGD
```

```python
W01 = np.array([[0.1,0.3,0.7], [0.9,0.4,0.4]])
b1 = np.array([1.,1.])

W02 = np.array([[0.4,0.3], [0.7,0.2]])
b2 = np.array([1.,1.])

W03 = np.array([[0.5,0.6], [0.6,0.7], [0.3,0.2]])
b3 = np.array([1.,1.,1.])
```

```python
# Create the model
model = Sequential()
# Add the first hidden layer
model.add(Dense(2, input_dim=3, activation='relu', weights = [W01.T, b1]))
# Second hidden layer
model.add(Dense(2, activation='sigmoid', weights = [W02.T, b2]))
# Output layer
model.add(Dense(3, activation='softmax', weights = [W03.T, b3]))

sgd = SGD(lr=1)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['categorical_crossentropy'])
model.get_weights()
```

    /usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
      "The `lr` argument is deprecated, use `learning_rate` instead.")

    [array([[0.1, 0.9],
            [0.3, 0.4],
            [0.7, 0.4]], dtype=float32),
     array([1., 1.], dtype=float32),
     array([[0.4, 0.7],
            [0.3, 0.2]], dtype=float32),
     array([1., 1.], dtype=float32),
     array([[0.5, 0.6, 0.3],
            [0.6, 0.7, 0.2]], dtype=float32),
     array([1., 1., 1.], dtype=float32)]

```python
model.fit(xtrain.reshape((1,3)), ytrain.reshape((1, 3)), epochs=200, batch_size=1)
```

    Epoch 1/200
    1/1 [==============================] - 1s 540ms/step - loss: 1.0221 - categorical_crossentropy: 1.0221
    Epoch 2/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.1284 - categorical_crossentropy: 0.1284
    Epoch 3/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0809 - categorical_crossentropy: 0.0809
    Epoch 4/200
    1/1 [==============================] - 0s 11ms/step - loss: 0.0595 - categorical_crossentropy: 0.0595
    Epoch 5/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0472 - categorical_crossentropy: 0.0472
    Epoch 6/200
    1/1 [==============================] - 0s 11ms/step - loss: 0.0392 - categorical_crossentropy: 0.0392
    Epoch 7/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0335 - categorical_crossentropy: 0.0335
    Epoch 8/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0292 - categorical_crossentropy: 0.0292
    Epoch 9/200
    1/1 [==============================] - 0s 11ms/step - loss: 0.0259 - categorical_crossentropy: 0.0259
    Epoch 10/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0233 - categorical_crossentropy: 0.0233
    Epoch 11/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0212 - categorical_crossentropy: 0.0212
    Epoch 12/200
    1/1 [==============================] - 0s 3ms/step - loss: 0.0194 - categorical_crossentropy: 0.0194
    Epoch 13/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0179 - categorical_crossentropy: 0.0179
    Epoch 14/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0166 - categorical_crossentropy: 0.0166
    Epoch 15/200
    1/1 [==============================] - 0s 3ms/step - loss: 0.0155 - categorical_crossentropy: 0.0155
    Epoch 16/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0145 - categorical_crossentropy: 0.0145
    Epoch 17/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0136 - categorical_crossentropy: 0.0136
    Epoch 18/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0129 - categorical_crossentropy: 0.0129
    Epoch 19/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0122 - categorical_crossentropy: 0.0122
    Epoch 20/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0116 - categorical_crossentropy: 0.0116
    Epoch 21/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0110 - categorical_crossentropy: 0.0110
    Epoch 22/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0105 - categorical_crossentropy: 0.0105
    Epoch 23/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0101 - categorical_crossentropy: 0.0101
    Epoch 24/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0096 - categorical_crossentropy: 0.0096
    Epoch 25/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0093 - categorical_crossentropy: 0.0093
    Epoch 26/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0089 - categorical_crossentropy: 0.0089
    Epoch 27/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0086 - categorical_crossentropy: 0.0086
    Epoch 28/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0083 - categorical_crossentropy: 0.0083
    Epoch 29/200
    1/1 [==============================] - 0s 9ms/step - loss: 0.0080 - categorical_crossentropy: 0.0080
    Epoch 30/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0077 - categorical_crossentropy: 0.0077
    Epoch 31/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0074 - categorical_crossentropy: 0.0074
    Epoch 32/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0072 - categorical_crossentropy: 0.0072
    Epoch 33/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0070 - categorical_crossentropy: 0.0070
    Epoch 34/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0068 - categorical_crossentropy: 0.0068
    Epoch 35/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0066 - categorical_crossentropy: 0.0066
    Epoch 36/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0064 - categorical_crossentropy: 0.0064
    Epoch 37/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0062 - categorical_crossentropy: 0.0062
    Epoch 38/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0061 - categorical_crossentropy: 0.0061
    Epoch 39/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0059 - categorical_crossentropy: 0.0059
    Epoch 40/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0058 - categorical_crossentropy: 0.0058
    Epoch 41/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0056 - categorical_crossentropy: 0.0056
    Epoch 42/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0055 - categorical_crossentropy: 0.0055
    Epoch 43/200
    1/1 [==============================] - 0s 9ms/step - loss: 0.0054 - categorical_crossentropy: 0.0054
    Epoch 44/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0052 - categorical_crossentropy: 0.0052
    Epoch 45/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0051 - categorical_crossentropy: 0.0051
    Epoch 46/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0050 - categorical_crossentropy: 0.0050
    Epoch 47/200
    1/1 [==============================] - 0s 13ms/step - loss: 0.0049 - categorical_crossentropy: 0.0049
    Epoch 48/200
    1/1 [==============================] - 0s 14ms/step - loss: 0.0048 - categorical_crossentropy: 0.0048
    Epoch 49/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0047 - categorical_crossentropy: 0.0047
    Epoch 50/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0046 - categorical_crossentropy: 0.0046
    Epoch 51/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0045 - categorical_crossentropy: 0.0045
    Epoch 52/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0044 - categorical_crossentropy: 0.0044
    Epoch 53/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0043 - categorical_crossentropy: 0.0043
    Epoch 54/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0043 - categorical_crossentropy: 0.0043
    Epoch 55/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0042 - categorical_crossentropy: 0.0042
    Epoch 56/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0041 - categorical_crossentropy: 0.0041
    Epoch 57/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0040 - categorical_crossentropy: 0.0040
    Epoch 58/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0040 - categorical_crossentropy: 0.0040
    Epoch 59/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0039 - categorical_crossentropy: 0.0039
    Epoch 60/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0038 - categorical_crossentropy: 0.0038
    Epoch 61/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0038 - categorical_crossentropy: 0.0038
    Epoch 62/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0037 - categorical_crossentropy: 0.0037
    Epoch 63/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0036 - categorical_crossentropy: 0.0036
    Epoch 64/200
    1/1 [==============================] - 0s 14ms/step - loss: 0.0036 - categorical_crossentropy: 0.0036
    Epoch 65/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0035 - categorical_crossentropy: 0.0035
    Epoch 66/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0035 - categorical_crossentropy: 0.0035
    Epoch 67/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0034 - categorical_crossentropy: 0.0034
    Epoch 68/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0034 - categorical_crossentropy: 0.0034
    Epoch 69/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0033 - categorical_crossentropy: 0.0033
    Epoch 70/200
    1/1 [==============================] - 0s 3ms/step - loss: 0.0033 - categorical_crossentropy: 0.0033
    Epoch 71/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0032 - categorical_crossentropy: 0.0032
    Epoch 72/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0032 - categorical_crossentropy: 0.0032
    Epoch 73/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0031 - categorical_crossentropy: 0.0031
    Epoch 74/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0031 - categorical_crossentropy: 0.0031
    Epoch 75/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0031 - categorical_crossentropy: 0.0031
    Epoch 76/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0030 - categorical_crossentropy: 0.0030
    Epoch 77/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0030 - categorical_crossentropy: 0.0030
    Epoch 78/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0029 - categorical_crossentropy: 0.0029
    Epoch 79/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0029 - categorical_crossentropy: 0.0029
    Epoch 80/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0029 - categorical_crossentropy: 0.0029
    Epoch 81/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0028 - categorical_crossentropy: 0.0028
    Epoch 82/200
    1/1 [==============================] - 0s 11ms/step - loss: 0.0028 - categorical_crossentropy: 0.0028
    Epoch 83/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0028 - categorical_crossentropy: 0.0028
    Epoch 84/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0027 - categorical_crossentropy: 0.0027
    Epoch 85/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0027 - categorical_crossentropy: 0.0027
    Epoch 86/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0027 - categorical_crossentropy: 0.0027
    Epoch 87/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0026 - categorical_crossentropy: 0.0026
    Epoch 88/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0026 - categorical_crossentropy: 0.0026
    Epoch 89/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0026 - categorical_crossentropy: 0.0026
    Epoch 90/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0025 - categorical_crossentropy: 0.0025
    Epoch 91/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0025 - categorical_crossentropy: 0.0025
    Epoch 92/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0025 - categorical_crossentropy: 0.0025
    Epoch 93/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0025 - categorical_crossentropy: 0.0025
    Epoch 94/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0024 - categorical_crossentropy: 0.0024
    Epoch 95/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0024 - categorical_crossentropy: 0.0024
    Epoch 96/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0024 - categorical_crossentropy: 0.0024
    Epoch 97/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0024 - categorical_crossentropy: 0.0024
    Epoch 98/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0023 - categorical_crossentropy: 0.0023
    Epoch 99/200
    1/1 [==============================] - 0s 11ms/step - loss: 0.0023 - categorical_crossentropy: 0.0023
    Epoch 100/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0023 - categorical_crossentropy: 0.0023
    Epoch 101/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0023 - categorical_crossentropy: 0.0023
    Epoch 102/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0022 - categorical_crossentropy: 0.0022
    Epoch 103/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0022 - categorical_crossentropy: 0.0022
    Epoch 104/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0022 - categorical_crossentropy: 0.0022
    Epoch 105/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0022 - categorical_crossentropy: 0.0022
    Epoch 106/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0022 - categorical_crossentropy: 0.0022
    Epoch 107/200
    1/1 [==============================] - 0s 9ms/step - loss: 0.0021 - categorical_crossentropy: 0.0021
    Epoch 108/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0021 - categorical_crossentropy: 0.0021
    Epoch 109/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0021 - categorical_crossentropy: 0.0021
    Epoch 110/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0021 - categorical_crossentropy: 0.0021
    Epoch 111/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0021 - categorical_crossentropy: 0.0021
    Epoch 112/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0020 - categorical_crossentropy: 0.0020
    Epoch 113/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0020 - categorical_crossentropy: 0.0020
    Epoch 114/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0020 - categorical_crossentropy: 0.0020
    Epoch 115/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0020 - categorical_crossentropy: 0.0020
    Epoch 116/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0020 - categorical_crossentropy: 0.0020
    Epoch 117/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0019 - categorical_crossentropy: 0.0019
    Epoch 118/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0019 - categorical_crossentropy: 0.0019
    Epoch 119/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0019 - categorical_crossentropy: 0.0019
    Epoch 120/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0019 - categorical_crossentropy: 0.0019
    Epoch 121/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0019 - categorical_crossentropy: 0.0019
    Epoch 122/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0019 - categorical_crossentropy: 0.0019
    Epoch 123/200
    1/1 [==============================] - 0s 15ms/step - loss: 0.0019 - categorical_crossentropy: 0.0019
    Epoch 124/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0018 - categorical_crossentropy: 0.0018
    Epoch 125/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0018 - categorical_crossentropy: 0.0018
    Epoch 126/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0018 - categorical_crossentropy: 0.0018
    Epoch 127/200
    1/1 [==============================] - 0s 20ms/step - loss: 0.0018 - categorical_crossentropy: 0.0018
    Epoch 128/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0018 - categorical_crossentropy: 0.0018
    Epoch 129/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0018 - categorical_crossentropy: 0.0018
    Epoch 130/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0018 - categorical_crossentropy: 0.0018
    Epoch 131/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0017 - categorical_crossentropy: 0.0017
    Epoch 132/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0017 - categorical_crossentropy: 0.0017
    Epoch 133/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0017 - categorical_crossentropy: 0.0017
    Epoch 134/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0017 - categorical_crossentropy: 0.0017
    Epoch 135/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0017 - categorical_crossentropy: 0.0017
    Epoch 136/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0017 - categorical_crossentropy: 0.0017
    Epoch 137/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0017 - categorical_crossentropy: 0.0017
    Epoch 138/200
    1/1 [==============================] - 0s 14ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 139/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 140/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 141/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 142/200
    1/1 [==============================] - 0s 13ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 143/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 144/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 145/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 146/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0016 - categorical_crossentropy: 0.0016
    Epoch 147/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 148/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 149/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 150/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 151/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 152/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 153/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 154/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 155/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 156/200
    1/1 [==============================] - 0s 23ms/step - loss: 0.0015 - categorical_crossentropy: 0.0015
    Epoch 157/200
    1/1 [==============================] - 0s 23ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 158/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 159/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 160/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 161/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 162/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 163/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 164/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 165/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 166/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 167/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 168/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0014 - categorical_crossentropy: 0.0014
    Epoch 169/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 170/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 171/200
    1/1 [==============================] - 0s 4ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 172/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 173/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 174/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 175/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 176/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 177/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 178/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 179/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 180/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 181/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0013 - categorical_crossentropy: 0.0013
    Epoch 182/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 183/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 184/200
    1/1 [==============================] - 0s 15ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 185/200
    1/1 [==============================] - 0s 15ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 186/200
    1/1 [==============================] - 0s 19ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 187/200
    1/1 [==============================] - 0s 10ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 188/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 189/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 190/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 191/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 192/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 193/200
    1/1 [==============================] - 0s 7ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 194/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 195/200
    1/1 [==============================] - 0s 11ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 196/200
    1/1 [==============================] - 0s 6ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 197/200
    1/1 [==============================] - 0s 8ms/step - loss: 0.0012 - categorical_crossentropy: 0.0012
    Epoch 198/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0011 - categorical_crossentropy: 0.0011
    Epoch 199/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0011 - categorical_crossentropy: 0.0011
    Epoch 200/200
    1/1 [==============================] - 0s 5ms/step - loss: 0.0011 - categorical_crossentropy: 0.0011

    <keras.callbacks.History at 0x7fb084a36650>

```python
model.predict(xtrain.reshape((1, 3)))
```

    array([[9.9887174e-01, 5.7525293e-04, 5.5299088e-04]], dtype=float32)

```python
model.get_weights()
```

    [array([[0.11550441, 0.9084708 ],
            [0.40853125, 0.4592939 ],
            [0.74651366, 0.4254116 ]], dtype=float32),
     array([1.1550443, 1.084706 ], dtype=float32),
     array([[0.6014873 , 0.87786746],
            [0.50195366, 0.3785293 ]], dtype=float32),
     array([1.1302494, 1.1153164], dtype=float32),
     array([[ 2.0983257 , -0.3175912 , -0.3807338 ],
            [ 2.2364383 , -0.24000825, -0.49643087]], dtype=float32),
     array([ 2.7668118 , -0.01687927,  0.25006643], dtype=float32)]

Hey! Lookie there! They're the same!