# Attempt to Replicate the Black Scholes Model Using a Neural Network

```python
import torch
import torch.nn as nn
import numpy as np
import torch.nn.functional as F
import matplotlib.pyplot as plt
import scipy.stats as ss
```

## Define the Function
Here we define our function, the Black Scholes Model (BSM). First, we must initialize the option class, then the Geometric Brownian Motion Class, and finally the BSM class.
Then we test the equation with a test value of 2.

```python
'''=========
option class init
=========='''
class VanillaOption:
    def __init__(
        self,
        otype = 1, # 1: 'call'
                  # -1: 'put'
        strike = 110.,
        maturity = 1.,
        market_price = 10.):
      self.otype = otype               # Put or Call
      self.strike = strike             # Strike K
      self.maturity = maturity         # Maturity T
      self.market_price = market_price #this will be used for calibration

    def payoff(self, s): #s: excercise price
      otype = self.otype
      k = self.strike
      maturity = self.maturity
      return np.max([0, (s - k)*otype])

```

```python
'''============
Gbm class
============='''

class Gbm:
    def __init__(self,
                 init_state = 100.,
                 drift_ratio = .0475,
                 vol_ratio = .2
                ):
        self.init_state = init_state
        self.drift_ratio = drift_ratio
        self.vol_ratio = vol_ratio
```

```python
'''========
Black-Scholes-Merton formula.
=========='''

def bsm_price(self, vanilla_option):
    s0 = self.init_state
    sigma = self.vol_ratio
    r = self.drift_ratio

    otype = vanilla_option.otype
    k = vanilla_option.strike
    maturity = vanilla_option.maturity

    d1 = 1/(sigma*np.sqrt(maturity))*(np.log(s0/k) + (r + np.power(sigma,2)/2)*(maturity))
    d2 = 1/(sigma*np.sqrt(maturity))*(np.log(s0/k) + (r - np.power(sigma,2)/2)*(maturity))
    return (otype * s0 * ss.norm.cdf(otype * d1) #line break needs parenthesis
            - otype * np.exp(-r * maturity) * k * ss.norm.cdf(otype * d2))

Gbm.bsm_price = bsm_price
```

```python
'''=======
Get BSM prices given an option and a Tensor
======='''

def prices_bsm(self, vanilla_option, data):

  # Create the list
  a = []

  # Get tensor size
  sizeT = list(data.size())[0]

  for i in range(sizeT):
    self.init_state = data[i].item()
    callPrice = gbm1.bsm_price(vanilla_option)
    a.append(callPrice)

  # Convert to array then tensor
  arrayOut = np.array(a)
  outputData = torch.from_numpy(arrayOut)
  return outputData

Gbm.prices_bsm = prices_bsm
```

```python
def f(s, k = 10):
  gbm = Gbm(init_state=s)
  option = VanillaOption(strike=k)
  return gbm.bsm_price(option)

batch_size = 21
x_list = np.linspace(8, 11, batch_size)
y_list = np.array([f(x) for x in x_list])
#plt.plot(x_list, y_list)
```

## Create Model
Next, we create the neural network model. This is done first by setting the inner and outer dimensions with variables. Next we code the model and vary the internal dimensions to attempt to improve the model. At this level, this is essentially a simple linear algebra exercise:

If we have input $x$, internal parameters $a,b$, and solution $f(x)$ then in the one-dimensional case we have:

\begin{equation}
  \left(
    a_{1}x+b_{2}
  \right)
  a_{2} + b_{2}
  = f(x)
\end{equation}

However, we want to get a better estimate for the true equation. So we increase the interior dimension which corresponds to the number of neurons inside the network. For example, we raised the inner dimension to 3. In matrix form we have:

\begin{equation}
\left(
  \begin{bmatrix} x \end{bmatrix}
  \begin{bmatrix} a_{1} & a_{2} & a_{3} \end{bmatrix}
  +
  \begin{bmatrix} b_{1} & b_{2} & b_{3} \end{bmatrix}
\right)
 \begin{bmatrix} a_{4} \\ a_{5} \\ a_{6} \end{bmatrix}
 +
 \begin{bmatrix} b_{4} \\ \end{bmatrix}
 =
 \begin{bmatrix} f(x) \end{bmatrix}
\end{equation}

Graphically, we can render this second neural network as:

**ADD DESCRIPTION**

```python
#model
#nn.Linear
H1 = 50; H2 = 11 #number of hidden layer
model = nn.Sequential(
    nn.Linear(1, H1),
    nn.Sigmoid(),
    nn.Linear(H1, H2),
    nn.Sigmoid(),
    nn.Linear(H2,2),
    nn.Sigmoid(),
    nn.Linear(2,1)
)
```

Here we define the Loss function as the Mean Squared Error(MSE).

Note that by doing so, we are essentially 'cheating' the system. In most applications, we would not know the function $f$ so we would be unable to find the MSE.

```python
#loss function
criterion = nn.MSELoss()
```

Next we choose a learning rate and a method for learning. The learning rate is the percent of the data that is accepted in each iteration. The Methods we tried were SGD and Adam.

```python
#optimizer
learning_rate = 0.1
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, momentum=0.7)
```

```python
batch_size = np.size(x_list)
x_train0 = torch.from_numpy(x_list).reshape(batch_size,1).float()
y_train0 = torch.from_numpy(y_list).reshape(batch_size,1).float()
```

## Train the Model

First we create the training data. This is a batch of random points that we pass through the BSM.

```python
# Create Training Data

#batch_size = 1000

# Create a list
#a = []

# Generate randomized data
#x_train = torch.randn(batch_size, 1)
# Modify the random data to be a stock price
#x_train0 = np.abs(x_train*100)

# Create static training data for conistency
#x_train = torch.tensor([[100], [105], [110], [115], [120]])

# Initialize GBM
#gbm1 = Gbm()
#option = VanillaOption()

# Call the prices_bsm to get y_train
#y_train0 = gbm1.prices_bsm(option, x_train)

#print(x_train[1:10])
#print(y_train[1:10])

```

Once we have the training data, we pass this collection of inputs and solutions into the model. With each iteration we calculate the loss and attempt to optimize the model to further reduce the loss.

```python
# Normalization

def linear_transform(xx, l = 0, u= 1):
  M = torch.max(xx)
  m = torch.min(xx)
  return (u-l)/(M-m)*(xx-m)+l, m, M, l, u
x_train, x_m, x_M, x_l, x_u = linear_transform(x_train0, -1, 1)
y_train, y_m, y_M, y_l, y_u = linear_transform(y_train0, 0, 1)
```

```python
# Train the model

num_epochs = 1000

for epoch in range(num_epochs):

    # Forward pass
    outputs = model(x_train.float())
    loss = criterion(outputs, y_train.float())

    # Backward and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if epoch == 0 or (epoch+1) % 50 == 0:
        print ('Epoch [{}/{}], Loss: {:.4f}'.format(epoch+1,
                                                    num_epochs, loss.item()))
        #print(outputs[1:10])

```

    Epoch [1/1000], Loss: 0.6988
    Epoch [50/1000], Loss: 0.0928
    Epoch [100/1000], Loss: 0.0900
    Epoch [150/1000], Loss: 0.0841
    Epoch [200/1000], Loss: 0.0695
    Epoch [250/1000], Loss: 0.0363
    Epoch [300/1000], Loss: 0.0059
    Epoch [350/1000], Loss: 0.0012
    Epoch [400/1000], Loss: 0.0007
    Epoch [450/1000], Loss: 0.0005
    Epoch [500/1000], Loss: 0.0003
    Epoch [550/1000], Loss: 0.0002
    Epoch [600/1000], Loss: 0.0002
    Epoch [650/1000], Loss: 0.0001
    Epoch [700/1000], Loss: 0.0001
    Epoch [750/1000], Loss: 0.0001
    Epoch [800/1000], Loss: 0.0001
    Epoch [850/1000], Loss: 0.0001
    Epoch [900/1000], Loss: 0.0001
    Epoch [950/1000], Loss: 0.0001
    Epoch [1000/1000], Loss: 0.0001

## Testing the Model

```python
def learnedfun(x):
  out = (1-(-1))/(x_M-x_m)*(x-x_m)+(-1.)
  out = model(out)
  out = (y_M- y_m)*out+y_m
  return out

y_pred = learnedfun(x_train0)
```

```python
# Test with training data
plt.scatter(x_train0.detach().numpy(), y_train0.detach().numpy(), label='true')
plt.plot(x_train0.detach().numpy(), y_pred.detach().numpy(), label='pred')

plt.legend()
plt.show()
```

```python
# Test the Model with Random data

# Generate random data
x_0 = np.random.randint(90, 120, batch_size)

# Get BSM Prices as determined by the formula
y_0 = np.array([f(x, 105) for x in x_0])

# Transform to tensor
x_0 = torch.from_numpy(x_0).reshape(batch_size,1).float()
y_0 = torch.from_numpy(y_0).reshape(batch_size,1).float()

# Normalize
x_, x_m, x_M, x_l, x_u = linear_transform(x_0)
y_, y_m, y_M, y_l, y_u = linear_transform(y_0)

# Plot x_ versus formula prices
plt.scatter(x_0.detach().numpy(), y_0.detach().numpy(), label='true')

# Get BSM Prices as determined by the model
y_pred = learnedfun(x_0)

# Plot x_ versus the model prices
plt.scatter(x_0.detach().numpy(), y_pred.detach().numpy(), label='pred')

plt.legend()

```

```python

```


# Chapter 3 - Developing Templates

Generating SoftMax distributions from normals could get quite tedious – for any sufficiently complicated shape, the number of normals to be used could be excessive. Let's add a layer of abstraction onto all our work.

##Polygon Construction

We can put everything together from all we've talked about (shifting the distribution and generating weights from normals) to a more tangible process: generating a softmax distribution from a [polytope](http://en.wikipedia.org/wiki/Polytope). Let's motivate this with an example first.

Imagine you worked at the Pentagon as an HRI researcher. One day, while pondering the nature of language, you happened to look out your window and spot an intruder. If you called a human security officer, you might say something like, "I see an intruder in front of the Heliport facade." We can use our SoftMax classifier to translate this same sentence for a security bot to understand.

First, we'd need to divide the space in a similar way we did for the Pac-Man problem:

As opposed to our Pac-Man problem, we can't assign weights by inspection. Instead, we'll use our weights-from-normals tactic to generate our weights for each class, and our shifted bias tactic to place those weights appropriately.

### Step 1: Define Polytope
We can use a geometry library like [Shapely](http://toblerity.org/shapely/shapely.html) to define custom polytopes (in this case, a pentagon). For a quick way to get ideal pentagon vertex coordinates, you can either calculate them by hand or use [some online tools](http://www.mathsisfun.com/geometry/pentagon.html).

Let's try a pentagon with the following coordinates (starting at the corner between the South Parking Entrance and the Heliport Facade):

$$
\begin{align}
P_1 &= (P_{1x}, P_{1y}) = (-1.90,-0.93) \\
P_2 &= (-1.40,1.45) \\
P_3 &= (1.03,1.71) \\
P_4 &= (2.02,-0.51) \\
P_5 &= (0.21,-2.15) \\
\end{align}
$$

### Step 2: Get Normals and Offsets
We want to get six classes, so we'd like to specify $\frac{6(6-1)}{2} = 15$ normal vectors in order to use our transformation matrix $A$. But, we only have six unknowns, so we can reduce the size of our $A$ matrix. That is, we can use:

$$
\mathbf{N} = \begin{bmatrix}
\mathbf{n}_{0,1}^T \\
\mathbf{n}_{0,2}^T \\
\mathbf{n}_{0,3}^T \\
\mathbf{n}_{0,4}^T \\
\mathbf{n}_{0,5}^T \\
\mathbf{n}_{1,2}^T \\
\end{bmatrix}
= \begin{bmatrix}
-1 & 1 & 0 & 0 & 0 & 0 \\
-1 & 0 & 1 & 0 & 0 & 0 \\
-1 & 0 & 0 & 1 & 0 & 0 \\
-1 & 0 & 0 & 0 & 1 & 0 \\
-1 & 0 & 0 & 0 & 0 & 1 \\
0 & -1 & 1 & 0 & 0 & 0 \\
\end{bmatrix}
\begin{bmatrix}
\mathbf{w}_{0}^T \\
\mathbf{w}_{1}^T \\
\mathbf{w}_{2}^T \\
\mathbf{w}_{3}^T \\
\mathbf{w}_{4}^T \\
\mathbf{w}_{5}^T \\
\end{bmatrix}
= \mathbf{A}\mathbf{W}
$$

Where $\mathbf{n}_{0,1}$ is the boundary between the interior and the South Parking Entrance, and so on.

Except, we can be smarter about this. We only care about the *relative* weights, so why not define one class and solve for the weights of all other classes? Since we have one interior class with weights $w_0$, simply define $w_0 = \begin{bmatrix}0 & 0 \end{bmatrix}^T$ and $b_0 = 0$, leaving us with the following five equations and five unkowns:

$$
\mathbf{N} = \begin{bmatrix}
\mathbf{n}_{0,1}^T \\
\mathbf{n}_{0,2}^T \\
\mathbf{n}_{0,3}^T \\
\mathbf{n}_{0,4}^T \\
\mathbf{n}_{0,5}^T \\
\end{bmatrix}
= \begin{bmatrix}
1 & 0 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 & 0 \\
0 & 0 & 0 & 1 & 0 \\
0 & 0 & 0 & 0 & 1 \\
\end{bmatrix}
\begin{bmatrix}
\mathbf{w}_{1}^T \\
\mathbf{w}_{2}^T \\
\mathbf{w}_{3}^T \\
\mathbf{w}_{4}^T \\
\mathbf{w}_{5}^T \\
\end{bmatrix}
= \mathbf{A}\mathbf{W}
$$

Does it make sense that the weights we'd use correspond directly to the class boundaries of each class with some zero-weighted interior class? Yes: think of a class boundary as defined by its normal vector. Those normal vectors point exactly in the direction of greatest probability of a given class.

Thus, we have:

$$
\mathbf{n}_{0,i} = \mathbf{w}_{i} \; \forall i \in N
$$

We have the normals, but solving for the class biases will require digging deeper. We need the equation for a normal fixed to the surface of the polytope (not simply its magnitude and direction!).

In $\mathbb{R}^2$, we know that a line is uniquely defined by two points passing through it – a face's bounding vertices, for instance. This can help us find the normal vectors and offsets, giving us the weights and biases.

Recall the specification of our hyperplanes in $\mathbb{R}^2$:

\begin{align}
0 &= (\mathbf{w}_i - \mathbf{w}_j)^T\mathbf{x} + (b_i - b_j) \\
  &= (w_{i,x} - w_{j,x})x + (w_{i,y} - w_{j,y})y + (b_i - b_j) \\
  &= w_{i,x}x + w_{i,y}y + b_i
\end{align}

Where the last line assumes $j$ is the interior class with weights and a bias of 0.

Since we have two points on this line segment (and any third point from a linear combination of the first two), we can use their $x$ and $y$ values to calculate our weights:

\begin{equation}\label{eq:nullspace}
\begin{bmatrix}
x_1 & y_1 & 1 \\
x_2 & y_2 & 1 \\
x_3 & y_3 & 1 \\
\end{bmatrix}
\begin{bmatrix}
w_{i,x}\\
w_{i,y}\\
b_i
\end{bmatrix}
=\begin{bmatrix}
0\\
0\\
0
\end{bmatrix}
\end{equation}

The non-trivial solution to $\ref{eq:nullspace}$ can be found through various decomposition techniques. We use [Singular Value Decomposition](http://en.wikipedia.org/wiki/Singular_value_decomposition).

In short, given any polygon, we can use its vertices to find the equations of the normals representing the class boundaries between the interior class and an exterior class for each face. Let's try this out and see if it works well.

Note that this part, as well as several future ones, require long swaths of code
to be fully explained. Rather than include the code in this document, you can always find it
on [our Github](https://github.com/COHRINT/cops_and_robots/blob/master/src/cops_and_robots/robo_tools/fusion/softmax.py).

```python
import numpy as np
%matplotlib inline

from cops_and_robots.robo_tools.fusion.softmax import SoftMax, make_regular_2D_poly
poly = make_regular_2D_poly(5, max_r=2, theta=-np.pi/4)
labels = ['Interior','Heliport Facade','Mall Terrace Entrance',
              'River Terrace Entrance', 'Concourse Entrance',
              'South Parking Entrance']
sm = SoftMax(poly=poly, class_labels=labels, resolution=0.1)
sm.plot(plot_poly=True, plot_normals=False)

```

**NOTE: 3D Plotting currently borked**

Well, that looks like the class boundaries are lining up just fine, but what about the probability distributions themselves? They seem a bit diffuse. If you remember from Chapter 1, we can simply multiply the weights and biases by the same value to raise the steepness of each class. Let's try that:

```python
steepness = 5
sm = SoftMax(poly=poly, class_labels=labels, resolution=0.1, steepness=5)
sm.plot(plot_poly=True, plot_normals=False)
```

As expected, our boundaries stayed the same but our probabilities are less spread out. Looking good!

However, we need to address a few assumptions. Most importantly, our interior class will not always be centered at the origin. Let's look at a shifted coordinate frame again, with the center of our polygon at $(-2,3)$:

\begin{align}
\mathbf{x}' &= \begin{bmatrix}x & y\end{bmatrix}^T + \begin{bmatrix}2 & -3\end{bmatrix}^T = \begin{bmatrix}x + 2 & y -3\end{bmatrix}^T \\
0 &= (\mathbf{w}_i - \mathbf{w}_j)^T \mathbf{x}' + (b_i - b_j) \\
&= (\mathbf{w}_i - \mathbf{w}_j)^T \mathbf{x} + (\mathbf{w}_i - \mathbf{w}_j)^T \mathbf{b} + (b_i - b_j)\\
&= \mathbf{w}_i^T \mathbf{x} + \mathbf{w}_i^T \mathbf{b} + b_i\\
&= w_{i,x}x + w_{i,y}y + \begin{bmatrix}w_{i,x} & w_{i,y}\end{bmatrix}\begin{bmatrix}-2 \\ 3\end{bmatrix}  + b_i\\
&= w_{i,x}x + w_{i,y}y -2 w_{i,x}  + 3w_{i,y} + b_i\\
&= w_{i,x}(x - 2) + w_{i,y}(y + 3) + b_i\\
\end{align}

$$
\begin{bmatrix}
x_1 & y_1 & 1 \\
x_2 & y_2 & 1 \\
x_3 & y_3 & 1 \\
\end{bmatrix}
\begin{bmatrix}
w_{i,x}\\
w_{i,y}\\
b_i
\end{bmatrix}
=\begin{bmatrix}
0\\
0\\
0
\end{bmatrix}
$$

```python
poly = make_regular_2D_poly(5, max_r=2, theta=-np.pi/4, origin=(-2,3))
sm = SoftMax(poly=poly, class_labels=labels, resolution=0.1, steepness=5)
sm.plot(plot_poly=True, plot_normals=False)
```

Great! We've successfully decomposed the space around the Pentagon, so we can tell the automatic security bots where the suspect is without having to pull out a map of the Pentagon and show them directly where on the map our intruder may be. That is, we've replaced communication of specific coordinates with the communication of 'zones' formed by spatial relationships to landmarks.

However, the methodology build up to this point doesn't work for all cases. For instance: what happens if we want to use a non-symmetric shape to develop a SoftMax model? [Chapter 3](03_mms.ipynb) will dive into some of these pitfalls and how to get around them

```python
from shapely.geometry import Polygon
import numpy as np

polygon = Polygon(((-1.8996,-0.92915),
                   (-1.395,1.4523),
                   (1.0256,1.7093),
                   (2.018,-0.51393),
                   (0.21001,-2.145),))
pts = polygon.exterior.coords[:]
normals = np.zeros((len(pts) - 1, 2))
biases = np.zeros(len(pts) - 1)
for i in range(len(pts) - 1):
    slope = (pts[i + 1][1] - pts[i][1]) / (pts[i + 1][0] - pts[i][0])
    normals[i] = np.array((-slope, 1))
    biases[i] = pts[i][1] - slope * pts[i][0]
print(normals)
print(biases)
```

    [[-4.71948078  1.        ]
     [-0.10617202  1.        ]
     [ 2.24025595  1.        ]
     [-0.90214548  1.        ]
     [ 0.57633875  1.        ]]
    [ 8.03597568  1.60040997  4.0069065  -2.33445957 -2.0239631 ]

##From Polygons to Polytopes

```python
from collada import Collada
import numpy as np
from scipy.spatial import ConvexHull
from mpl_toolkits.mplot3d import Axes3D
from mpl_toolkits.mplot3d.art3d import Poly3DCollection
import matplotlib.pyplot as plt

lim_ = 1 # Fleming
mesh = Collada('/Users/nick/Downloads/Desk.dae')
# lim_ = 250 # Pentagon
# mesh = Collada('/Users/nick/Downloads/Pentagon.dae')

mpu = mesh.assetInfo.unitmeter # how many real-world meters are in one distance unit

# Grab all model vertices
geometries = mesh.geometries
primitives = [ _.primitives for _ in geometries]
trisets = [ _[0] for _ in primitives]
vertices = [ a.vertex for a in trisets]
vertices = np.asarray(vertices) * mpu

# Find convex hull
verts = np.zeros([1,3])
for vert in vertices:
    verts = np.vstack((verts,vert))
verts = np.subtract(verts[1:,:],np.array([0,3.8,0]))
hull = ConvexHull(verts)

# Get all facets (faces) of the convex hull
facets = []
for simplex in hull.simplices:
    facet = [(verts[simplex[0],0],verts[simplex[0],1],verts[simplex[0],2]),
             (verts[simplex[1],0],verts[simplex[1],1],verts[simplex[1],2]),
             (verts[simplex[2],0],verts[simplex[2],1],verts[simplex[2],2]),
             ]
    facets.append(facet)

# Plot vertices and convex hull
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(verts[:,0], verts[:,1],verts[:,2])
ax.add_collection3d(Poly3DCollection(facets, alpha=0.5))

ax.auto_scale_xyz([-lim_, lim_], [-lim_, lim_], [-lim_, lim_])
plt.show()

```

```python
from IPython.core.display import HTML

# Borrowed style from Probabilistic Programming and Bayesian Methods for Hackers
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Roboto:100,100italic,500,300,300italic,400' rel='stylesheet' type='text/css'>

<style>
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1, h2, h3, h4 {
        font-family: "Roboto", "wingdings", sans-serif;
    }
    h1{
        font-weight: 500;
    }
    h2{
        font-weight: 400;
    }
    h3{
        font-weight: 300 !important;
/*         font-style: italic; */
    }
    h4{
        font-weight: 300 !important;
        font-style: italic;
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: "HelveticaNeue-light", "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 120%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 22pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

```python

```


# Leaky Integrate-and-Fire Neuron

The integrate-and-fire neuron was first envisioned by the French neuroscientist [Louis Lapicque](https://en.wikipedia.org/wiki/Louis_Lapicque) in 1907. Although the neuron theory was already stablished some years before, the nature of the action potentials was still unknown [[1]](#References). Therefore, the integrate-and-fire model tries the reproduce the electrical activity of a neuron, disregarding the mechanisms underlying it. In this notebook, we will deal with a modification of the this model: the leaky integrate-and-fire neuron.

A neuron can be modeled as an RC circuit. The figure below shows the analogy between a neuron and a RC circuit. The main idea behind the model is the following: the cell membrane separates both the external and internal environment of a cell. The internal environment –named [cytosol](https://en.wikipedia.org/wiki/Cytosol)– is rich in negative charged molecules; the external environment is however, majoritarily compounded by positive charged molecules. Hence, the cell membrane that divides both environments functions as a insulator. If an external current $I$ is applied to the neuron, this will charge the membrane. This current can be understood as the entrance of positive ions towards the cytosol. Whenever the current is droped, the cell membrane will slowly return to its original charge. Therefore, the cell membrane acts similarly to a condensor. These properties allow the neuron to be represented as a RC circuit.

The cell membrane can be represented by a first order differential equation (ODE) [[2]](#References):

$$
\begin{equation}
I(t) = \frac{V(t) - V_R}{R} + C \frac{d V}{dt},
\end{equation}
$$

where $V(t)$ is the internal potential over time, $V_R$ the resting membrane potential, $I(t)$ the injected current over time, $R$ the resistance of the membrane and $C$ its capacitance. We will assume that the current are is kept constant, $I$. Bearing this in mind, we can rearrange the equation:

$$
\begin{equation}
\tau\frac{dV}{dt} = - (V(t) - V_R) + RI,
\end{equation}
$$

where $\tau = RC$. This ODE can be solved by separation of variables for $V(t_0) = V_R$:

$$
\begin{equation}
\int^{V(t)}_{V_R} \frac{1}{-(V-V_R)+RI} = \int^{t}_{0} \frac{1}{\tau}dt \\
\end{equation}
$$

$$
\begin{equation}
-\Big [\log |-(V-V_R)+RI| \Big ]^{V(t)}_{V_R} = \frac{t}{\tau} \\
\end{equation}
$$

$$
\begin{equation}
\log{\frac{-(V-V_R)+RI}{RI}} = \frac{t}{\tau} \\
\end{equation}
$$

$$
\begin{equation}
\boxed{V(t) = V_R + RI(1 - e^{-\frac{t}{\tau}})}
\end{equation}
$$

Finally we have to introduce the firing. Whenever the potential of the cell reaches a given threshold $\theta$, a spike will be produced. Consequently, the potential of the cell will decrease back its resting potential. Mathematically, we can describe this behavior as a piecewise function:

$$
V(t) =
\begin{cases}
0, &\quad\text{if } V(t) > \theta \\
V(t) = V_R + RI(1 - e^{-\frac{t}{\tau}}), &\quad\text{otherwise}
\end{cases}
$$

Note that $t$ is also reseted after a spike is produced.

```python
# Add temporary path to code
import sys
sys.path.append("..")

# Import the module with the LeakyIntegrateAndFire model
from neural_models import LeakyIntegrateAndFire

# Create the model
model = LeakyIntegrateAndFire(VR=-70, R=100, C=0.3, theta=-55)

# Print the model parameters
print(model)

# Run the model
model.run(current=0.3)
```

    LeakyIntegrateAndFire(VR=-70, R=100, C=0.3, theta=-55")

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

# Plot the results
plt.figure(figsize=(8, 5))
plt.plot(model.tvec, model.V, color='royalblue')
plt.plot(model.tvec, np.repeat(model.theta, len(model.V)),
         color='lightcoral', linestyle='--')
plt.xlabel('time [t]', fontsize=12)
plt.ylabel('Voltage [V]', fontsize=12)
plt.grid(alpha=0.3)
plt.show()
```

The plot shows how the voltage of the cell changes as a function of time. The red dashed line corresponds to $\theta$. For $t = 0$, the neuron is at its resting potential. At $t>0$ the current is applied. The membrane potential increases until reaching the threshold. Afterward, the neuron spikes and its potential decreases back to $V_R$ (according to the equation above).

In addition, it will be interesting to compute the period and spiking frequency given a constant intensity. Analitically, it will be the time when $V(t) = \theta$:

$$
\theta = V_R + RI(1 - e^{-\frac{T}{\tau}}),
$$

where $T$ is the spiking period. By isolating $T$:

$$
T = - \tau \log{\Big |1 - \frac{\theta - V_R}{RI} \Big|}
$$

The frequency is by definition:

$$
f = \frac{1}{T}
$$

```python
period = - model.tau * np.log(1 - (model.theta - model.VR) / (model.R * model.current))

print(period)
```

    20.79441541679836

For the example above, the period $T = 20.784$ implies the neuron fires approximately each 21ms. If we want to observe a determined number of $N$ spikes, we can calculate the time of simulation needed as follows:

$$
t = NT = \frac{N}{f}
$$

For example, if we want to observe $N = 10$ spikes, we have to run the simulation for approximately 208 ms:

```python
# Run and plot the model
model.run(t=208, current=0.3)
```

```python
# Plot the results
plt.figure(figsize=(8, 5))
plt.plot(model.tvec, model.V, color='royalblue')
plt.plot(model.tvec, np.repeat(model.theta, len(model.V)),
         color='lightcoral', linestyle='--')
plt.xlabel('time [t]', fontsize=12)
plt.ylabel('Voltage [V]', fontsize=12)
plt.grid(alpha=0.3)
plt.show()
```

# References
1. [Lapicque's introduction of the integrate-and-fire model neuron](https://web.archive.org/web/20070613230629/http://neurotheory.columbia.edu/~larry/AbbottBrResBul99.pdf)
2. [Neuronal Dynamics: Chapter 1](https://neuronaldynamics.epfl.ch/book.html)


**NOTE:** *The slope-deflection sign convention may seem strange to those used to matirx stiffness analysis, but it makes sense.  None of the slope deflection equations explicitly state a
member 'direction' and it doesn't matter.  For example, whether you consider the column AB as
going from A to B or as going from B to A, a +ive shear at end A is still directed toward the
left. In matrix analysis, that direction matters.*

## Kulak & Grondin - Example 8.2
This solves a close approximation to the first-order example of Kulak and Grondin.  The major
differences are:
* the lateral resistance of the outside column stacks are ignored; only the central rigid frame is included.
* this method does not account for axial changes of length in any of the members.
The results obtained here are
compared with those obtained via a [first-order matrix analysis](../matrix-methods/frame2d/50-test-KG82sd.ipynb),
and found to agree to about 6 significant figures.

```python
from IPython import display
display.SVG('KG-8.2sd.svg')
```

### Solve for joint rotations and storey translations using slope-deflection

```python
import sympy as sy                  # use symbolic algebra
sy.init_printing()                  # print pretty math
from sdutil2 import SD, FEF         # slope-deflection library
```

```python
sy.var('theta_a theta_b theta_c theta_d theta_e theta_f Delta_b Delta_c')
```

```python
E = 200000
Ic = 222E6   # W310x97
Ib = 488E6   # W460x106
Hf = 21900   # Horizontal load at F
He = 43400   # Horizontal load at E

Lab = 6500   # Length of column
Lbc = 5500   # Length of column
Lbe = 10500  # Length of beam
```

```python
Mab,Mba,Vab,Vba = SD(Lab,E*Ic,theta_a,theta_b,Delta_b)             # column AB, BC
Mbc,Mcb,Vbc,Vcb = SD(Lbc,E*Ic,theta_b,theta_c,Delta_c-Delta_b)

Mde,Med,Vde,Ved = SD(Lab,E*Ic,theta_d,theta_e,Delta_b)             # column DE, EF
Mef,Mfe,Vef,Vfe = SD(Lbc,E*Ic,theta_e,theta_f,Delta_c-Delta_b)

Mbe,Meb,Vbe,Veb = SD(Lbe,E*Ib,theta_b,theta_e) + FEF.udl(Lbe,55)   # beams BE, CF
Mcf,Mfc,Vcf,Vfc = SD(Lbe,E*Ib,theta_c,theta_f) + FEF.udl(Lbe,45)
```

```python
eqns = [ Mba+Mbe+Mbc,      # sum of moments at B = 0
         Mcb+Mcf,          # sum of moments at C = 0
         Med+Meb+Mef,      # sum of moments at E = 0
         Mfe+Mfc,          # sum of moments at F = 0
         -Vab-Vde+Hf+He,   # sum of Fx @ base of storey 1 = 0
         -Vbc-Vef+Hf,      # sum of Fx @ base of storey 2 = 0
         theta_a,          # fixed support at A, rotation = 0
         theta_d]          # fixed support at F, rotation = 0
```

```python
soln = sy.solve(eqns)
soln
```

### Determine end moments and shears

#### Demonstrate how to access and convert one end moment:

```python
Mab
```

```python
Mab.subs(soln)
```

```python
Mab.subs(soln).n(4) * 1E-6
```

```python
V = globals()   # another way to access global variables
V['Mab']
```

```python
Mab is V['Mab']
```

    True

```python
V['Mab'].subs(soln).n()
```

#### Determine end moments by back substitution:

```python
# collect the end moments in all 6 members
allm = []
V = globals()
for m in 'ab,bc,de,ef,be,cf'.split(','):
    mj = V['M'+m].subs(soln).n()*1E-6          # Mxy
    mk = V['M'+m[::-1]].subs(soln).n()*1E-6    # Myx
    allm.append((m.upper(),mj,mk))
allm
```

    [('AB', -49.9080183705027, 47.4941396356060),
     ('BC', 250.526060428376, 252.096857589821),
     ('DE', -189.782099213905, -232.254022051198),
     ('EF', -290.011616822927, -333.061301195269),
     ('BE', -298.020200063982, 522.265638874126),
     ('CF', -252.096857589821, 333.061301195269)]

```python
[(m,round(a,1),round(b,1)) for m,a,b in allm]  # display to one decimal place
```

    [('AB', -49.9, 47.5),
     ('BC', 250.5, 252.1),
     ('DE', -189.8, -232.3),
     ('EF', -290.0, -333.1),
     ('BE', -298.0, 522.3),
     ('CF', -252.1, 333.1)]

#### Determine end shears by back substitution:

```python
# collect the end shears in all 6 members
allv = []
V = globals()
for m in 'ab,bc,de,ef,be,cf'.split(','):
    mj = V['V'+m].subs(soln).n()*1E-3          # Mxy
    mk = V['V'+m[::-1]].subs(soln).n()*1E-3    # Myx
    allv.append((m.upper(),mj,mk))
allv
```

    [('AB', 0.371365959214883, 0.371365959214883),
     ('BC', -91.3859850942176, -91.3859850942176),
     ('DE', 64.9286340407851, 64.9286340407851),
     ('EF', 113.285985094218, 113.285985094218),
     ('BE', 267.393291541891, -310.106708458109),
     ('CF', 228.539100609005, -243.960899390995)]

```python
[(m,round(a,1),round(b,1)) for m,a,b in allv]   # display to one decimal place
```

    [('AB', 0.4, 0.4),
     ('BC', -91.4, -91.4),
     ('DE', 64.9, 64.9),
     ('EF', 113.3, 113.3),
     ('BE', 267.4, -310.1),
     ('CF', 228.5, -244.0)]

### Now compare to matrix method solution (Frame2D):

```python
import pandas as pd

dd = '../matrix-methods/frame2d/data/KG82sd.d/all'   # location of mm solution
```

### Compare Moments:
#### Convert current member end moments solution to tabular form:

```python
mems = pd.DataFrame(allm,columns=['ID','MZJ','MZK']).set_index('ID')
mems
```

      MZJ
      MZK

      ID

      AB
      -49.9080183705027
      47.4941396356060

      BC
      250.526060428376
      252.096857589821

      DE
      -189.782099213905
      -232.254022051198

      EF
      -290.011616822927
      -333.061301195269

      BE
      -298.020200063982
      522.265638874126

      CF
      -252.096857589821
      333.061301195269

#### Fetch solution from Frame2D:

```python
mefs = pd.read_csv(dd+'/member_end_forces.csv').set_index('MEMBERID').loc[mems.index]
mems2 = mefs[['MZJ','MZK']] * -1E-6    # convert sign and to kN-m
mems2
```

      MZJ
      MZK

      ID

      AB
      -49.908014
      47.494144

      BC
      250.526061
      252.096859

      DE
      -189.782098
      -232.254022

      EF
      -290.011612
      -333.061297

      BE
      -298.020204
      522.265634

      CF
      -252.096859
      333.061297

#### Compare member end moments in the two solutions:

```python
mdiff = (100*(1-mems/mems2))   # calculate % diff
mdiff
```

      MZJ
      MZK

      ID

      AB
      -9.17049660653646e-6
      8.35047321290361e-6

      BC
      9.14187947564926e-8
      6.57745258259013e-7

      DE
      -5.40915578994827e-7
      4.17979983957650e-8

      EF
      -1.72059335667996e-6
      -1.36026414576662e-6

      BE
      1.40762708156217e-6
      -9.36849420263286e-7

      CF
      6.57745302667934e-7
      -1.36026414576662e-6

```python
mdiff.abs().max()
```

    MZJ    0.000009
    MZK    0.000008
    dtype: float64

The maximum difference in member end moments is 0.000009% (about 7 or 8 sig figs).

### Compare Shears:

#### Convert our end shears to tabular form:

```python
mevs = pd.DataFrame(allv,columns=['ID','FYJ','FYK']).set_index('ID')
mevs
```

      FYJ
      FYK

      ID

      AB
      0.371365959214883
      0.371365959214883

      BC
      -91.3859850942176
      -91.3859850942176

      DE
      64.9286340407851
      64.9286340407851

      EF
      113.285985094218
      113.285985094218

      BE
      267.393291541891
      -310.106708458109

      CF
      228.539100609005
      -243.960899390995

#### Extract the end shears from Frame2D results:

```python
mevs2 = mefs[['FYJ','FYK']] * 1E-3
mevs2[['FYK']] *= -1    # change sign on end k
mevs2
```

      FYJ
      FYK

      ID

      AB
      0.371365
      0.371365

      BC
      -91.385985
      -91.385985

      DE
      64.928634
      64.928634

      EF
      113.285983
      113.285983

      BE
      267.393292
      -310.106708

      CF
      228.539101
      -243.960899

#### Compare end shears in the two results:

```python
vdiff = 100*(1-mevs/mevs2)
vdiff
```

      FYJ
      FYK

      ID

      AB
      -0.000353904629113444
      -0.000353904629113444

      BC
      3.75466946422875e-7
      3.75466946422875e-7

      DE
      -2.20237894588138e-7
      -2.20237894588138e-7

      EF
      -1.52798038666191e-6
      -1.52798038666191e-6

      BE
      3.23684223868526e-7
      -2.79100675903976e-7

      CF
      2.57897703193066e-7
      -2.41594921845945e-7

```python
vdiff.abs().max()
```

    FYJ    0.000354
    FYK    0.000354
    dtype: float64

The maximum difference is about 0.00004% which is high, but end shears on column AB are very small.

### Compare Displacements

```python
deltw = pd.DataFrame([('B', soln[Delta_b], soln[theta_b]),
                      ('C', soln[Delta_c], soln[theta_c])],columns=['ID','DX','RZ']).set_index('ID')
deltw
```

      DX
      RZ

      ID

      B
      23.3628188885579
      0.00712966246666336

      C
      34.3068306465552
      0.00722695283139248

```python
disp = pd.read_csv(dd+'/node_displacements.csv').set_index('NODEID').loc[deltw.index][['DX','RZ']]
disp['RZ'] *= -1
disp
```

      DX
      RZ

      ID

      B
      23.362818
      0.007130

      C
      34.306830
      0.007227

```python
diffd = (100*(1-deltw/disp))
diffd
```

      DX
      RZ

      ID

      B
      -3.52157611960280e-6
      -6.27118601492782e-7

      C
      -2.71776707805316e-6
      6.06124439528344e-7

```python
diffd.abs().max()
```

    DX    3.521576e-06
    RZ    6.271186e-07
    dtype: float64

Max difference in displacement is 0.000006%.

Note that the matrix method solution was accomplished by setting A very high in order to minimize
effects of axial deformation.  But there is a limit to how high this can be set, due to numerical
instability in equation solving (probably).  Setting $A=10^{10}$ seems to be about as high as is possible - larger values lead to worse results.

```python

```