# Performance

**ENGSCI233: Computational Techniques and Computer Systems**

*Department of Engineering Science, University of Auckland*

# 0 What's in this Notebook?

Having mostly bug-free code is a **necessary but not sufficient** condition for a good computer program. This notebook is all about understanding code performance, how algorithms scale with larger and larger problems, how to identify bottlenecks, and strategies to speed up execution.

Things you need to know:
- That Big O notation measures how algorithm runtime grows with problem size. You can read this from a graph by running the algorithm several times at different problem sizes.
- Profiling tells you how long your code spends running different functions, so you can figure out which parts are too slow.
- When calculations are independent of each other, they can be run in parallel. This gives you speed-up and efficiency.

```python
# imports and environment: this cell must be executed before any other in the notebook
%matplotlib inline
from performance233 import*
```

## 1 Measuring algorithms

***How should we decide if one algorithm is better than another?***

One way to assess an algorithm is to count how many operations it takes to solve a problem of size $N$, and to compare this against another implementation. For instance, suppose we wish to sort an array of $N$ random numbers in ascending order; we could use:

1. Heapsort, which takes $k_1N\log_2 N+k_2$ operations to complete,
2. or insertion sort, which takes $k_3 N^2 + k_4$ operations to complete.

Depending on the values of $[k_1, k_2, k_3, k_4]$, we may assess one algorithm as superior to the other for a given problem size.

A second approach is to compare the asymptotic scaling of each algorithm: how do they perform as $N$ gets **really really large**. The latter we denote using $\mathcal{O}()$, called ["Big O notation"](https://www.interviewcake.com/article/java/big-o-notation-time-and-space-complexity) or order-of-complexity. This notation ignores **constant multiplicative factors** and focuses on the functional form.

From a scaling perspective, heapsort with order-of-complexity $\mathcal{O}(N\log_2N)$ is superior to insertion sort with $\mathcal{O}(N^2)$.

### 1.1 Graphing $\mathcal{O}()$

How do we decide if an algorithm is $\mathcal{O}(N^2)$ or $\mathcal{O}(N\log_2N)$? Both have graphs that are concave up...

***Execute the cell below.***

```python
compare_scaling()
```

A useful diagnostic is to plot execution time on log-log axes, for a few doublings of the problem size, i.e., $N$, $2N$, $4N$, $8N$, etc. If the problem has scaling $\mathcal{O}(N^\alpha)$, then the plot will be linear on log-log axes, and $\alpha$ can be read off as the slope.

***Execute the cell below.***

```python
scaling_loglog()
```

***Read the slope off the middle plot and verify that it is 2, i.e., $\mathcal{O}(N^2)$. ***

***There are two-ways to construct a log-log plot: (middle) by explicitly log converting the $x$ and $y$ quantities, and (right) by calling the Python commands `ax.set_xscale('log')` and `ax.set_yscale('log')`.***

### 1.2 Other metrics

**Scaling of the execution time** is just one consideration when deciding on an algorithm implementation. Depending on the application or hardware available, consideration may also have to be given to memory use, stability, or preconditions of the algorithm input (some search algorithms are *very* fast if the input is already sorted).

## 2 Profiling for optimisation

***How should we choose which parts of a computer program should be improved?***

Large computer programs might comprise hundreds of different functions or methods calling each other in a complex sequence. Just one poorly written implementation can **slow the entire program**. How can we locate these bottlenecks?

Profilers are used to automatically analyse the performance of our code. This could include how efficiently is uses memory or how quickly it performs tasks. Here, we will focus on the latter, introducing the idea of code execution time and looking at how a profiler can be used to generate **execution time** statistics for a computer program.

### 2.1 Measuring time

Ultimately, profiling relies on measuring **how long it takes** to run different parts of the code. So we need a measure of [`time`](https://www.programiz.com/python-programming/time#time).

***Run the cell below and answer the questions.***

```python
# import the time module
from time import time

# get the current time IN SECONDS from the system clock
t0 = time()
print(t0)
```

***Run the cell above over and over (Ctrl+Enter ***\*wait\* *** Ctrl+Enter ***\*wait\* *** Ctrl+Enter). How does the output change?***

> *~ your answer here ~*

***Divide `t0` by 3600, to print the number of hours. Convert this to a number of days and a number of years.***

> *~ your answer here ~*

***Subtract the current time IN YEARS from today's date. When did the clock start?***

> *~ your answer here ~*

Once we can measure time, we can measure **time differences**.

***Run the cell below and answer the questions.***

```python
# start the clock
tstart = time()

# do something you want to time
    # e.g., find the location of the largest element in an array of random numbers
i = np.argmax(np.random.rand(1000))

# stop the clock
tend = time()
print(tend-tstart, 'seconds')
```

***How long does it take to find the max value? Do you believe the answer?***

> *~ your answer here ~*

***Run the cell above over and over (Ctrl+Enter ***\*wait\* *** Ctrl+Enter ***\*wait\* *** Ctrl+Enter). How does the output change?***

> *~ your answer here ~*

Sometimes an operation executes **too quickly** to be timed accurately using `time()`. In which case, we can repeat the operation `N` times, and divide the total execution time by `N`.

***Run the cell below and answer the questions.***

```python
# start the clock
tstart = time()

# do something you want to time
    # e.g., find the location of the largest element in an array of random numbers
N = 100
for j in range(N):
    i = np.argmax(np.random.rand(1000))

# stop the clock
tend = time()
print((tend-tstart)/N, 'seconds')
```

***How long does it take to find the max value? Does this agree with the previous estimate?***

> *~ your answer here ~*

***Increase the value of `N`. Does the estimate of the execution time change (or at least stabilise?)***

> *~ your answer here ~*

### 2.2 Profilers

So, one way to get a sense of where our code is slow, is by writing in a whole bunch of `time()` and `print()` calls. This takes forever when you have a complicated code... and then you just have to go back in and pull them out when you're finished optimising.

It's much better to use a ***PROFILER***. In the lab, you will use the [`cProfile`](https://docs.python.org/3.2/library/profile.html) module to study the efficiency of an LU factorisation implementation. There is not much more to say here except to study the typical output of such a profiler.

There's a lot of useful information to unpack here. After some general header information (e.g., total runtime) the profiler goes on to rank different function and method calls by the total time the code has spend in them. The columns give:

1. The **total number** of times the function or method was called.
2. The **total time** spent in that function or method (**excludes other function or method calls**).
3. Total time **per function or method call** (excludes time spent calling other functions or methods).
4. Cumulative time spent in that function or method - `tend-tstart` in the example above (**includes other functions of methods**).
5. Cumulative time per function or method call - `(tend-tstart)/N` in the example above (includes time spent calling other functions of methods).
6. The **name** of the function or method.

From the print out above, we can identify that the large majority of time is spent in the `row_reduction` function, and this is a consequence of ***both*** the large number of function calls (199) ***and*** the relatively slow function execution (0.043 seconds, compared to the next slowest `lu_factor` at 0.005 seconds per call).

Perhaps we should take a look at improving `row_reduction`? You'll do this in the lab.

## 3 Concurrency and Parallelisation

***How can we make optimised code even faster?***

Throughout the 80's and 90's, clever engineers devised new methods to squeeze more and more transistors onto microchips. The result was that computing speeds roughly **doubled every 2 years**: the so-called [Moore's Law](https://en.wikipedia.org/wiki/Moore%27s_law). While research into transistor miniaturisation [continues to this day](https://www.technologyreview.com/s/602592/worlds-smallest-transistor-is-cool-but-wont-save-moores-law/), in practice, gains in computing power are achieved through **multi-core processing**. Many desktops now come with 8-core chips (8 independent processors) as standard, although if you're rereading these notes in 5 years time that will probably [sound primitive](https://i.pinimg.com/originals/93/44/66/9344663cd0094039d4bacd47f67d48fe.jpg).

**Concurrency** is the idea that you can do two or more things at the same time2. It is ubiquitous in computing: multiple apps ***concurrently*** running on your phone, 30 or so ENGSCI students ***concurrently*** working through some contrived lab problem on a networked desktop machine each Wednesday morning. In each case, we can think of a **shared resource** (your phone's memory, a pool of Desktops) being accessed by **independent processes** (phone apps, ENGSCI nerds).

The same concepts can be applied to **parallelise** your code.

2[Scary concept if you're a male.](https://vignette.wikia.nocookie.net/tehmeme/images/5/5d/Y0UJC.png/revision/latest?cb=20120505151500)

### 3.1 An example

***- - - - CLASS CODING EXERCISE - - - -***

How long does it take Python to factorise a 3000$\times$3000 matrix? How about 10 of them?

```python
# import some pacakges
from scipy.linalg import lu
import numpy as np
from time import time

N = 3000
n = 10

# create some matrices
As = []
for i in range(n):
    As.append(np.random.rand(N,N))

# factorise one matrix using lu()
t0 = time()
P,L,U = lu(As[0])
t1 = time()
print('factorising 1 matrix: ',t1-t0, 'seconds')

# factorise ten matrices using lu()
# *** your code here ***

print('factorising {:d} matrices: '.format(n),t1-t0, 'seconds')

# free up the memory
del(As)
```

***Does the time for factorising 1 matrix vs. 10 matrices scale as you expect?***

> *~ your answer here ~*

***Explain whether we need to have FINISHED factorising the FIRST matrix before starting on the SECOND.***

> *~ your answer here ~*

### 3.2 Thinking about parallelisation

A multi-core microprocessor contains several independent processing units. We can think of these as forming a **pool** of workers. At any given time, some of the workers may be **idle** while others will be busy completing an **assigned task**. When a new request comes along, it will be either assigned to an available worker or, in the event everyone is busy, stored in a **queue**. When a worker finishes a task, they are **returned to the pool** ready to receive the next queued assignment.

***If the input to one assignment does not depend on the outcome of another, then two workers can complete their tasks simultaneously***.

The case is relatively easy to treat in Python using the [`multiprocessing`](https://docs.python.org/3.4/library/multiprocessing.html?highlight=process) library, the [`Pool`](https://docs.python.org/3.4/library/multiprocessing.html?highlight=process#multiprocessing.pool.Pool) class, and the [`map()`](https://docs.python.org/3.4/library/multiprocessing.html?highlight=process#multiprocessing.pool.Pool.map) method. Although, we cannot do multiprocessing inside this Jupyter Notebook, I have included a supplementary script `parallel_example.py` that demonstrates how to parallelise a loop and what kind of speed-ups can be achieved.

***Visual Studio Code printout for `parallel_example.py`, a parallelised version of Example 3.1.***

Finally, if the two assignments are **related to one another**, two workers may still be able to complete them simultaneously, although they may need to **communicate with each other** from time to time. To treat this case, we require a [message passing protocol](https://en.wikipedia.org/wiki/Message_passing). You won't need to do that in this course.

### 3.3 Speed-up

The purpose of parallelisation is to reduce the execution time of a program. By measuring execution time for different sized pools, we can develop a sense of relative gains and diminishing returns. We define the parallel speedup, $S$, and parallel efficiency, $E$,

\begin{equation}
S=\frac{T_s}{T_p},\quad\quad E = \frac{S}{n_p}
\end{equation}

where $T_s$ is the **serial execution time** and $T_p$ is the parallel execution time for a pool of size $n_p$.

***Run the cell below to plot parallel speed-up and efficiency for `parallel_example.py`.***

```python
parallel_analysis()
```

***What does the phrase "linear speedup" imply?***

> *~ your answer here ~*

***Which sections of the lefthand plot would you consider "sublinear" and "supralinear"?***

> *~ your answer here ~*

***Explain how the parallel speedup plot shows diminishing returns.***

> *~ your answer here ~*

***Why might the speedup get WORSE for a very large pool?***

> *~ your answer here ~*

```python
# solution code for 3.1
#t0 = time()
#for A in As:
#    P,L,U = lu(A)
#t1 = time()
```


# "Spin Glass Models 4: Ising Model - Simulation"
> "In this blog post we will introduce ways of simulating the Ising models introduced in the previous post. We start by looking at a fairly basic MCMC implementation. From this we will introduce a computational technique that has implications way beyond spin glasses. We will end with a more modern higher performance approach to simulating the Ising model."
- toc: true
- author: Lewis Cole (2020)
- branch: master
- badges: false
- comments: false
- categories: [Spin-Glass, Magnet, Ising, Edwards-Anderson, MCMC, Simulated-Annealing, Wolff]
- hide: false
- search_exclude: false
- image: https://github.com/lewiscoleblog/blog/raw/master/images/spin-glass/hc_graph.png

___
This is the fourth blog post in a series - you can find the previous blog post [here](https://lewiscoleblog.com/spin-glass-models-4)
___

# Setup
In this blog we will limit ourselves to trying to find minimum energy states of the Ising model - we could use these schemes in order to estimate other thermodynamic properties also. The models will be in 2-dimensions on a square lattice since the results of these will be easier to display, it should be fairly obvious how we could modify this code for other dimensions. We will also use Gaussian distributed interaction strengths since we do not require mathematical tractability when we are simulating.

## Interactions
Unlike the Sherrington-Kirkpatrick model we will have to be a bit "clever" in defining the interaction strengths. I have decided to do this using 4 (NxN) arrays representing the up, down, left and right interaction strengths. This is a bit wasteful in terms of memory use but I think it is worth the trade-off to improve readability of the code. We then have the following symmetries:
```python
up[i,j] == down[i-1,j]
left[i,j] == right[i,j-1]
```
Where these indices exist, for the boundary coniditons we will take the following:
```python
up[0,j] == 0
down[N-1,j] == 0
left[i,0] == 0
right[i,N-1] == 0
```
For all possible $i, j$. This is a fixed boundary condition, the edge cases will have only 3 neighbours and corner cases only 2. This can introduce some instability for small spin-glasses but this is fine for our purposes. Modifying this to allow for periodic boundary conditions (i.e. a toroidal geometry) would not be particularly difficult.

## Mixing, Convergence, etc.
The methods presented below are Markov-Chain Monte-Carlo (MCMC) methods. These methods are notoriously finicky and require parameter tuning, running multiple chains, etc. in order to ensure good performance. As to not get distracted in this blog post I will not mention these concerns but please keep them in mind when reading on. For now we will run a single chain and look at the results it produces, in practice one would not do this.

# Local Update Methods
We begin by looking at local update methods. In the Sherrington-Kirkpatrick blog post we looked at one such (very bad) method: the greedy gradient descent. Local update methods mean at each simulation step we update 1 site only.

## Metropolis-Hastings
First we look at the "classic" approach to simulating the Ising model. In many texts and online references this will be the only method that is presented, it leads some to believe (incorrectly) that the simulation method is somehow part of the Ising model itself. This is the method I presented (without elaboration) in my Cython/Numba blog post.

We present Metropolis-Hastings in it's most general form first (our application to the Ising model will appear below):
1. Initialize the system in state $x_t$
2. Sample a new proposed state from some distribution $Q(x'_{t+1} | x_t)$
3. Accept this new state with probability $min\left[\alpha, 1 \right]$, else $x_{t+1} = x_t$, where:
$$ \alpha = \frac{P(x'_{t+1})Q(x_t |x'_{t+1})} {P(x_t)Q(x'_{t+1} |x_{t})} $$
4. Repeat steps 2. and 3.

In this context $P(.)$ represents the probability distribution we wish to estimate. We notice that this does not need to be standardized (i.e. we can ignore any difficult partition functions) so this is a (relatively) easy algorithm to implement. We find that the sequence $x_t$ forms a Markov-Chain - hence the term Markov-Chain Monte Carlo (MCMC).

We can see that that Metropolis-Hastings consists of 2 steps: a proposal step then an acceptance/rejection step. We see that there are essentially no constraints on $Q(.)$ so we can use something easy to sample from. We only need to be careful that it has a support that contains the support of $P(.)$ - otherwise we will not be able to sample all possible values as required. Selecting a suitable $Q(.)$ is at the heart of the algorithm however, the best performance will be where $Q$ and $P$ are very similar. Since $P$ is generally unknown in advance this often requires a bit of trial and error. There are many ways to test whether the algorithm is doing a "good job" but we will not cover them here (in a later set of blog posts I should probably go deeper into MCMC methods - here we're just concerned with the Ising model in particular).

Moving back to the Ising model picture, we can apply the Metropolis-Hastings methodology via:
1. First pick a random site
2. Calculate the change in energy associated with "flipping" this sites spin (going from +1 to -1 or vice versa)
3. If the energy decreases accept the flipped state and start again with a new site
4. If energy increases accept the flipped state with some acceptance probability or keep the same state
5. Begin again

We can see that in contrast to the "greedy hill climber" approach in the Sherrington-Kirkpatrick blog post we are not simply declining into a local minima, there is some probability that we can jump to a state of higher energy and escape a local minima to find a "better" one. In theory if we wait long enough we should find ourselves in a global minimum state. We now derive the acceptance probability as given by Metropolis-Hastings:

Recall that the Gibbs distribution for the Ising model with a given configuration is:
$$ P(\sigma) = \frac{exp(-\beta H_{\sigma})}{Z_{\beta}} $$

Since we are uniformly selecting new states the $Q(.)$ terms in $\alpha$ cancel. If we denote the propsed state as: $\hat{\sigma}$ then the relative likelihood is:
$$ \alpha = \frac{P(\hat{\sigma})}{P(\sigma)} = \frac{exp(-\beta H_{\hat{\sigma}})}{exp(-\beta H_{\sigma})} = exp\left(-\beta \left(H_{\hat{\sigma}} - H_{\sigma}\right)\right) = exp(-\beta \Delta H)  $$

To summarise by Metropolis-Hastings we then accept the new configuration with probability:
$$ p_{flip} = min\left[1, exp(-\beta \Delta H) \right] $$

In this particular case we can calculate $\Delta H$ quite efficiently since we are changing the spin of only 1 site, the contributions of all other sites "cancels out". We can write this down as:
$$ \Delta H = 2\sigma_{i,j} \sum_{(x,y) \in \langle i, j \rangle} J_{x,y} \sigma_{x,y} $$

This introduces some efficiency in a code implementation since we do not have to calculate the energy of the whole configuration each time we want to update a site's spin.

An example implementation of this can be seen below:

```python
#hide
import warnings
warnings.filterwarnings('ignore')
```

```python
# An implementation of an Ising spin-glass of size NxN
# With fixed boundary conditions using Metropolis-Hastings
# Connectivity is initialized as a Gaussian distribution N(0, s^2/N)
# Updates occur at randomly selected sites

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Fix random seed
np.random.seed(123)

# Set size of model N and initial spins
N = 32
spins = np.random.choice([-1, 1], (N,N))

# Fix number of timesteps and some containers
timesteps = 10000
mag = np.zeros(timesteps+1)
energy = np.zeros(timesteps+1)

# Initialize interaction arrays
# Have 4 arrays: up, down, left right
# These represent the interaction strengths to the
# up/down/left/right neighbours of a site
# There is a symmetry between these matrices
# This is not the most memory efficient solution
s_h = 1
s_v = 1
up = np.zeros((N,N))
down = np.zeros((N,N))
left = np.zeros((N,N))
right = np.zeros((N,N))

up[1:N,:] = np.random.rand(N-1,N) * s_v
down[0:N-1,:] = up[1:N,:]
left[:,1:N] = np.random.rand(N,N-1) * s_h
right[:,0:N-1] = left[:,1:N]

mag[0] = spins.sum()

for i in range(N):
    for j in range(N):
        if i == 0:
            up_neighbour = 0
            down_neighbour = spins[i+1,j]
        elif i == N-1:
            up_neighbour = spins[i-1,j]
            down_neighbour = 0
        else:
            up_neighbour = spins[i-1,j]
            down_neighbour = spins[i+1,j]
        if j == 0:
            left_neighbour = 0
            right_neighbour = spins[i,j+1]
        elif j == N-1:
            left_neighbour = spins[i,j-1]
            right_neighbour = 0
        else:
            left_neighbour = spins[i,j-1]
            right_neighbour = spins[i,j+1]

        energy[0] += spins[i,j]*(up[i,j]*up_neighbour + down[i,j]*down_neighbour + left[i,j]*left_neighbour + right[i,j]*right_neighbour)

# Avoid double count - each neighbour pair
# counted twice in above since loop over each site
energy[0] /= 2

# Fix beta (inverse temerature) - from analysis we know that
# system in glassy-phase for T<s so beta>1/s. Performance
# of random updates isn't good so don't select temperature
# too low
beta = 1

# Define proposal step
def proposal(s_array):
    _N = s_array.shape[0]
    return np.random.choice(_N, 2)

def energy_change(spin_site, bt, s_array, up_array, down_array, left_array, right_array):
    i = spin_site[0]
    j = spin_site[1]

    if i == 0:
        up_neighbour = 0
        down_neighbour = s_array[i+1,j]
    elif i == N-1:
        up_neighbour = s_array[i-1,j]
        down_neighbour = 0
    else:
        up_neighbour = s_array[i-1,j]
        down_neighbour = s_array[i+1,j]
    if j == 0:
        left_neighbour = 0
        right_neighbour = s_array[i,j+1]
    elif j == N-1:
        left_neighbour = s_array[i,j-1]
        right_neighbour = 0
    else:
        left_neighbour = s_array[i,j-1]
        right_neighbour = s_array[i,j+1]

    dE_tmp = 2*s_array[i,j]*(up_array[i,j]*up_neighbour + down_array[i,j]*down_neighbour + left_array[i,j]*left_neighbour + right_array[i,j]*right_neighbour)
    return dE_tmp

def acceptance(bt, energy):
    if energy <= 0:
        return -1
    else:
        prob = np.exp(-bt*energy)
        if prob > np.random.random():
            return -1
        else:
            return 1

# Define update step
dE = 0
dM = 0

def update(bt, s_array, up_array, down_array, left_array, right_array):
    global dE
    global dM

    # Proposal Step
    site = proposal(s_array)

    # Calculate energy change
    dE = energy_change(site, bt, s_array, up_array, down_array, left_array, right_array)
    dM = -2*s_array[site[0],site[1]]

    # Acceptance step
    accept = acceptance(bt, dE)

    if accept == -1:
        s_array[site[0], site[1]] *= -1
    else:
        dE = 0
        dM = 0

    return s_array

def _main_loop(ts, s_array, up_array, down_array, left_array, right_array):
    s_temp = s_array.copy()
    for i in range(ts):
        update_step = update(beta, s_temp, up_array, down_array, left_array, right_array)
        s_temp = update_step
        energy[i+1] = energy[i] + dE
        mag[i+1] = mag[i] + dM

#### Run Main Loop
_main_loop(timesteps, spins, up, down, left, right)
mag = mag / (N**2)
energy = energy / (N**2)

# plot magnetism and energy evolving in time
fig, ax1 = plt.subplots()
ax1.set_xlabel("Time step")
ax1.set_ylabel("Magnetism", color='blue')
ax1.plot(mag, color='blue')

ax2 = ax1.twinx()
ax2.set_ylabel("Energy", color='red')
ax2.plot(energy, color='red')

plt.show()
```

## Gibbs Sampling
Many presentations of the Ising model only show the Metropolis-Hastings scheme, as such there is a misconception that the Metropolis-Hastings sampling is somehow part of the Ising model itself. This is not true, an alternative to Metropolis-Hastings is Gibbs-Sampling. We can describe Gibbs-Sampling in general terms as:
1. Pick an initial state $\pmb{x_t} = \left(x^1_t, x^2_t, ... , x^N_t\right)$
2. For each $N$ dimension sample: $x^i_{t+1} \sim P\left(x^i_{t+1} | x^i_{t+1}, x^2_{t+1}, ... , x^{i-1}_{t+1}, x^{i+1}_t, ... x^N_i\right)$ and define next state as: $\pmb{x_{i+1}} = \left(x_1^{i+1}, x_2^{i+1}, ... , x_N^{i+1}\right)$
3. Repeat sampling for each successive state

We can see this is just a special case of Metropolis-Hastings, if we denote $\pmb{x_t^{-j}} = \left(x_t^1, x_t^2, ... , x_t^{j-1}, x_t^{j+1}, ..., x_t^N\right)$ (all components apart from the jth). Then we can set:
$$Q(x_{t+1}^j \pmb{x_t^{-j}} | \pmb{x_t} ) = P(x_{t+1}^j | \pmb{x_t^{-j}})$$
In the Metropolis-Hastings scheme, the acceptance probablilty then becomes:
\begin{align}
min\left[1 , \frac{Q(x_{t+1}^j \pmb{x_t^{-j}} | \pmb{x_t} ) P(\pmb{x_t})}{Q(\pmb{x_t} | x_{t+1}^j \pmb{x_t^{-j}} ) P(x_{t+1}^j \pmb{x_t^{-j}})} \right] & = min \left[1 , \frac{P(\pmb{x_t})P(x_{t+1}^j | \pmb{x_t^{-j}})}{P(x_{t+1}^j \pmb{x_t^{-j}})P(x_{t}^j | \pmb{x_t^{-j}})} \right] \\
&= min\left[1 , \frac{P(x_{t}^j | \pmb{x_t^{-j}})P(\pmb{x_t^{-j}})P(x_{t+1}^j | \pmb{x_t^{-j}})}{P(x_{t+1}^j | \pmb{x_t^{-j}})P(\pmb{x_t^{-j}})P(x_{t}^j | \pmb{x_t^{-j}})} \right] \\
&= min \left[1, 1 \right] = 1
\end{align}
Which results in the Gibbs procedure as described above.

We can implement Gibbs sampling in the context of an Ising model as:
1. Pick a random site $(i,j)$
2. Set spin to +1 with probability $p_{ij}$, or -1 with probability $(1-p_{ij})$
3. Begin again

The probability is defined as:
$$ p_{ij} = \frac{1}{1+ exp(- \beta \Delta H / \sigma_{i,j})} $$

At lower temperatures this should behave similarly to the Metropolis-Hastings. However the Metropolis-Hastings method has approximately twice the probability of accepting energetically unfavourable states, as such the Gibbs might be less efficient.

Note: this is true for the Ising model with methods as descibed - it is note a general point on Gibbs/Metropolis-Hastings. When sampling from higher dimensional distributions Gibbs samplers sample each dimension independently whereas Metropolis-Hastings samples points from the high-dimensional space. In some situations the Gibbs sampler can perform significantly better.

Since this is only a minor update to the Metropolis-Hastings code we will not present it here.

## Glauber Dynamics (and Heat-Bath)
Another confusion I have seen is that Metropolis-Hastings is the only acceptance-rejection scheme. This is not true, we can also define alternative acceptance probabilites. One such example is Glauber dynamics where we can express the acceptance probability as:
$$ p_{flip} = \frac{1}{2}\left( 1 - tanh\left( \beta \Delta H / 2\right) \right)\frac{exp(-\beta \Delta H/2)}{exp(\beta \Delta H/2) + exp(-\beta \Delta H/2)} $$
We will not derive this here nor simulate using Glauber dynamics (but we could modify 1 or 2 lines of the code above to ahcieve this), it is just to illustrate another option. For the Ising model Glauber dynamics has a lower acceptance probability for all possible states, as such its performance is likely to be slightly worse than Metropolis-Hastings.

For the Ising model Glauber-Dynamics is identical to the Heat-Bath method.

## Simulated Annealing and Simulated Tempering
We now move onto our first "improvement" to the Metropolis-Hastings scheme: simulated annealing.

The concept is very simple and takes inspiration from physical systems. Essentially we just start the system in a high temperature and gradually cool it down. This makes intuitive sense since at higher temperatures we have an increased chance of jumping out of local-minima and get closer to a better overall minima. However we will struggle to actually locate the minima at high temperature for the same reason. In contrast with a low temperature we will be able to locate a nearby local-minima very accurately but will not be able to jump out of it to find a better one. By starting off "hot" the samples will jump between many local minima, as we slowly cool down there should be fewer jumps between local minima and it should eventually get stuck in the domain of a "good" local minima (not far from the global minima ideally) as we cool the temperature further we should get closer and closer to that minima. This is similar to the process of annealing metals by heating them then cooling them to reorganize the crystalline structure.

Simulated annealing has proved very useful in the field of combinatorial optimization in situations where we want to quickly generate "good" solutions (not necessarily "best"). Variations on the idea can be seen in many areas (e.g. variable learning rate algorithms in deep learning can be thought of as a form of simulated annealing). We can also note there is nothing in the method that is particular to Metropolis-Hastings (or the other variations presented) - it can be used with pretty much any simulation method.

We have not descibed "how" we would want to decrease the temperature over time. This is part of the "issue" with simulated annealing (and many MCMC algorithms in general) - there is just as much art as their is science to implementing them. There are no real hard fast rules for getting good results, one in essense has to just try various options until something works (or on well studied problems borrow schemes from others).

We will take a simple approach where we decrease temperature by 10% every 500 steps starting from a temperature of 4 (this was chosen arbitrarily - it is not a suggestion of what might work well in this situation!) We can make use of the code example above for Metropolis-Hastings to give a compact implementation of:

```python
# An implementation of a Metropolis-Hastings algorithm
# with simulated annealing applied to a 2d Ising spin glass

# Fix random seed
np.random.seed(123)

# Set size of model N and initial spins
N = 32
spins = np.random.choice([-1, 1], (N,N))

# Set up initial beta
beta = 1/4

def _main_loop_SA(ts, bt_initial, s_array, up_array, down_array, left_array, right_array):
    s_temp = s_array.copy()
    bt_live = bt_initial
    for i in range(ts):
        if ts % 500 == 0:
            bt_live *= 1/0.9
        update_step = update(bt_live, s_temp, up_array, down_array, left_array, right_array)
        s_temp = update_step
        energy[i+1] = energy[i] + dE
        mag[i+1] = mag[i] + dM

#### Run Main Loop
_main_loop_SA(timesteps, beta, spins, up, down, left, right)
mag = mag / (N**2)
energy = energy / (N**2)

# plot magnetism and energy evolving in time
fig, ax1 = plt.subplots()
ax1.set_xlabel("Time step")
ax1.set_ylabel("Magnetism", color='blue')
ax1.plot(mag, color='blue')

ax2 = ax1.twinx()
ax2.set_ylabel("Energy", color='red')
ax2.plot(energy, color='red')

plt.show()
```

We can see that the system settled down to a lower energy state more quickly and smoothly than with the vanilla Metropolis-Hastings scheme.

Although simulated annealing can improve on vanilla Metropolis-Hastings it can still struggle to find a global minima of the system. There are various "hacks" that can further improve this however - one such example being the concept of restarting. Again this is a very "obvious" thing to try - we store the "best" state we've visited so far in a simulation, if we "get stuck" somewhere above this energy level we "jump back" to this best state and try again.

We can extend the simulated annealing idea further to the concept of "simulated tempering". Here we treat the temperature of the system as a variable in itself, the teperature can go up as well as down during the simulation. This can further improve convergence properties since it allows the system to escape energy boundaries more easily by increasing temperature. This can remove the need to use restarting since a higher temperature is always available as an option at all times.

One such simulated tempering scheme is "parallel tempering" - as the name suggests this involves running many Markov-Chains in parallel and "jumping" between chains as the algorithm progresses. In some instances the cost of running multiple chains is less than the improvement in performance. Again however there is an art to selecting the correct number of chains and temperatures to run in parallel, most times there is no substitute for just trying things and running tests on the results. For the interests of brevity we will not present a full code here - but we note that for 2 chains at temperatures $T_1$ and $T_2$ our proposed update is to switch the states between the two chains (or swap temperatures of the 2 chains) the acceptance probability is then:
$$p_{flip} = min\left[1, exp((H_1-H_2)(\beta_1 - \beta_2)) \right] $$
Where $H_1$ is the energy as defined by the Hamiltonian for chain with temperature $T_1 = 1/\beta_1$ and similar for $H_2$. This can be easily adapted to more chains and temperatures.

# Cluster Update Methods
We now have a few options for simulating the Ising model, however they are by no means perfect. The issue still remains of falling into local optima instead of a global optima. From our previous mathematical study we know that energy minima for the Ising model are "far away" from each other, that is they have very little overlapping spins. By flipping individual spins 1 by 1 it is very hard to make the chains explore the energy landscape fully. The natural way to solve this is to flip multiple spins simultaneously at each step. From the general definition of the Metropolis-Hastings method there is nothing stopping us in following this line of reasoning.

Unfortunately this makes things much harder, the complications arise in finding a valid scheme for flipping multiple spins at once. We have glossed over the mathematical foundations of MCMC here but the proposal/acceptance probabilities need to be selected in "smart way"  in order for the resulting Markov chain to have certain properties. When looking at more than 1 spin at a time in the Ising model this proved fairly difficult. This is evidenced by the original Metropolis-Hastings scheme being proposed in 1953 yet the first multi-spin method not being proposed and justified until the late 1980s.

## Wolff Algorithm
The main idea of the algorithm is to look for "clusters" of spin sites with the same spin. We then decide to flip the spin of all the sites within this cluster at once. We then pick a new cluster and repeat this process as necessary. The pseudocode for this algorithm as it applies to the Ising model is:
1. A site $x$ with spin $\sigma_x$ is selected at random and added to an empty cluster
2. For each neighbour $y$ of $x$ such that $\sigma_y = \sigma_x$ we add $y$ to the cluster stack with probability $p_{xy} = 1 - exp(-2\beta J_{xy})$ else move onto next neighbour
3. After all neighbours are exhausted select next site in the cluster stack and repeat the previous step until the cluster stack is exhausted
4. Once the cluster is fully specified flip the spins of all sites in the cluster and begin again

We can see that like the Gibbs sampling algorithm, here the Wolff algorithm is "rejection free" - that is all proposed sites are flipped. We also note that there is nothing in this method that is incompatible with simulated annealing/tempering - these techniques are often used together.

Some of the more computer-science focussed readers may be thinking: "creating clusters is computationally expensive, will a brute force local update method not be better?" Which is a valid concern; there a 2 things to consider here - firstly there are many efficient cluster generating algorithms from percolation theory which can help speed up this process (we presented a very simple method above for clarity.) And secondly the local update methods will take a very long time to make "big jumps" away from the current configuration - even though there is some probability to make unfavourable movements most of the time these will not be accepted, to jump to a different local energy minima we would need many such unfavourable moves which means we could be waiting for a very long time! This is why spin glass models form a good test bed for optimization algorithms as they are one of the simplest to define models with "difficult" energy landscapes.

We should find this algorithm performs better in general, especially near the 2nd order phase transition whereby successive samples become increasingly correlated (whereas we would want more independent samples). We will try and produce plots of the 2nd order themodynamic variables: heat capacity and susceptibility. We should expect to see an approximate discontinuity at the critical temperature. To do this we will re-run the Wolff algorithm multiple times for each target temperature. For each target temperature we will run 1500 "burn" steps and then evaluate our variables over the next 2500 steps. This should be long enough to get some reasonable results.

Note that the Wolff algorithm does not "converge" to a low energy state like the preceeding algorithms, instead it samples from the entire space in a "smart way" - even if it finds itself in the global energy minima there is still a relatively high probability of escaping. As such the graphs we produced before will look more "wiggly" - I've heard the term "fat caterpillar" used to describe the graph of a well mixed MCMC algorithm. If we were interested in finding a ground state we could keep track of the configuration corresponding to the lowest observed energy state so far (we will not do this in our code however).

In the example below we'll run a constant $J=1$ to try and reproduce the heat capacity we found analytically in the previous blog post as a proof of concept. I will leave the functionality for general interaction strengths should you wish to experiment.

An implementation of this method can be seen below (note: this is a very slow code since we're using Python lists! It would be a prime candidate for being sped up using Cython/Numba/etc.):

```python
# An implementation of the Wolff algorithm
# With simulated annealing applied to a
# 2d Ising spin glass

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Fix random seed
np.random.seed(123)

# Set size of model NxN and initial spins
N = 32
spins_initial = np.random.choice([-1, 1], (N,N))

# Fix time-steps
burn = 1500
evaluation = 2500
time_steps = burn + evaluation

# Initialize interaction arrays
# Have 4 arrays: up, down, left right
# These represent the interaction strengths to the
# up/down/left/right neighbours of a site
# There is a symmetry between these matrices
# This is not the most memory efficient solution
s_h = 1
s_v = 1
up = np.zeros((N,N))
down = np.zeros((N,N))
left = np.zeros((N,N))
right = np.zeros((N,N))

# Using J=1 constant so graphs are easier to generate
# Replace with comments to give an EA spin glass
up[1:N,:] = 1 #np.random.rand(N-1,N) * s_v
down[0:N-1,:] = up[1:N,:]
left[:,1:N] = 1 #np.random.rand(N,N-1) * s_h
right[:,0:N-1] = left[:,1:N]

# Create function to find neigbour sites
def nbr_udlr(s_site, s_array):
    _N = s_array.shape[0]
    i = s_site[0]
    j = s_site[1]

    if i == 0:
        up_site = 0
        down_site = [i+1, j]
    elif i == _N-1:
        up_site = [i-1,j]
        down_site = 0
    else:
        up_site = [i-1,j]
        down_site = [i+1, j]
    if j == 0:
        left_site = 0
        right_site = [i,j+1]
    elif j == N-1:
        left_site = [i,j-1]
        right_site = 0
    else:
        left_site = [i,j-1]
        right_site = [i,j+1]

    return [up_site, down_site, left_site, right_site]

# Create function to return interactions strength
def int_strength(s_site, udlr, up_array, down_array, left_array, right_array):
    if udlr == 0:
        return up_array[s_site[0], s_site[1]]
    if udlr == 1:
        return down_array[s_site[0], s_site[1]]
    if udlr == 2:
        return left_array[s_site[0], s_site[1]]
    if udlr == 3:
        return right_array[s_site[0], s_site[1]]

def energy_calc(s_array, up_array, down_array, left_array, right_array):
    _N = s_array.shape[0]
    energy = 0
    for i in range(_N):
        for j in range(_N):
            if i == 0:
                up_neighbour = 0
                down_neighbour = s_array[i+1,j]
            elif i == N-1:
                up_neighbour = s_array[i-1,j]
                down_neighbour = 0
            else:
                up_neighbour = s_array[i-1,j]
                down_neighbour = s_array[i+1,j]
            if j == 0:
                left_neighbour = 0
                right_neighbour = s_array[i,j+1]
            elif j == N-1:
                left_neighbour = s_array[i,j-1]
                right_neighbour = 0
            else:
                left_neighbour = s_array[i,j-1]
                right_neighbour = s_array[i,j+1]

            energy += s_array[i,j]*(up_array[i,j]*up_neighbour + down_array[i,j]*down_neighbour + left_array[i,j]*left_neighbour + right_array[i,j]*right_neighbour)
    return -energy/2

def wolff_step(bt, s_array, up_array, down_array, left_array, right_array):
    _N = s_array.shape[0]
    initial_site = np.random.choice(_N, 2)
    initial_site = [initial_site[0], initial_site[1]]
    old_spin = s_array[initial_site[0], initial_site[1]]
    cluster = [initial_site]
    stack = [initial_site]

    while stack != []:
        site = stack[np.random.choice(len(stack))]

        # Cycle neigbours
        nbr = nbr_udlr(site, s_array)
        for i in range(4):
            nbr_live = nbr[i]
            if nbr_live == 0:
                continue
            nbr_spin = s_array[nbr_live[0], nbr_live[1]]
            if nbr_spin == old_spin:
                if nbr_live not in cluster:
                    p = 1 - np.exp(-2*bt*int_strength(site, i, up_array, down_array, left_array, right_array))
                    if np.random.random() < p:
                        cluster.append(nbr_live)
                        stack.append(nbr_live)
        stack.remove(site)

    for site in cluster:
        s_array[site[0], site[1]] *= -1
    return s_array

###### Main Code
# Create useful constants
N1 = evaluation*N*N
N2 = evaluation*evaluation*N*N

# Define temp ranges
temp_steps = 20
temp_min = 1.75
temp_max = 2.75

temp_array = np.linspace(temp_min, temp_max, num=temp_steps)
M = np.zeros(temp_steps)
E = np.zeros(temp_steps)
C = np.zeros(temp_steps)
X = np.zeros(temp_steps)

for t in range(temp_steps):
    spins = spins_initial.copy()
    M1 = 0
    M2 = 0
    E1 = 0
    E2 = 0
    beta = 1/temp_array[t]
    for i in range(time_steps):
        spins = wolff_step(beta, spins, up, down, left, right)
        if i > burn:
            mag_tmp = abs(spins.sum())
            M1 += mag_tmp
            M2 += mag_tmp**2
            energy_tmp = energy_calc(spins, up, down, left, right)
            E1 += energy_tmp
            E2 += energy_tmp**2

        M[t] = M1 / N1
        E[t] = E1 / N1
        C[t] = (E2/N1 - E1**2/N2)*beta**2
        X[t] = (M2/N1 - M1**2/N2)*beta

# Create plots
fig, axs = plt.subplots(2, 2, figsize=(10,10), gridspec_kw={'hspace': 0.25, 'wspace': 0.25})
axs[0, 0].scatter(temp_array, M, color='Red')
axs[0, 0].set_title("Magnetism")
axs[0, 0].set(xlabel="T", ylabel="Magnetism")

axs[0, 1].scatter(temp_array, E, color='Blue')
axs[0, 1].set_title("Energy")
axs[0, 1].set(xlabel="T", ylabel="Energy")

axs[1, 0].scatter(temp_array, X, color='Red')
axs[1, 0].set_title("Susceptibility")
axs[1, 0].set(xlabel="T", ylabel="Susceptibility")

axs[1, 1].scatter(temp_array, C, color='Blue')
axs[1, 1].set_title("Heat Capacity")
axs[1, 1].set(xlabel="T", ylabel="Heat Capacity")

plt.show()
```

The plots here are a bit noisy but they loosely match our previous theoretical findings.

## Swendsen-Wang Algorithm
Another cluster algorithm is the Swendsen-Wang. Unlike the Wolff algorithm Swendsen-Wang looks at multiple clusters concurrently and applies a spin-flip to all clusters. It was propsed 2 years prior to the Wolff method.

In pseudo code it can be presented as:
1. From an initialized spin configuration for each neighbour pair of sites $\langle x, y \rangle$ we specify a bond: $b_{x,y} \in \{0, 1 \}$ Where we sample according to:
\begin{align}
& \mathbb{P}(b_{x,y} = 0 | \sigma_x \neq \sigma_y) = 1 \\
& \mathbb{P}(b_{x,y} = 1 | \sigma_x \neq \sigma_y) = 0 \\
& \mathbb{P}(b_{x,y} = 0 | \sigma_x = \sigma_y) = exp(-2 \beta J_{xy})  \\
& \mathbb{P}(b_{x,y} = 1 | \sigma_x = \sigma_y) = 1 - exp(-2 \beta J_{xy})
\end{align}
2. Generate clusters using bonds. If there exists a bond between sites $b_{x,y} = 1$ then the sites belong to same cluster
3. For each cluster with probability 1/2 flip all spins within the cluster to get a new configuration
4. Repeat process of generating bonds and clusters

We can see this is slightly different to the Wolff algorithm since it looks at multiple clusters within a given step. The performance of the Swendsen-Wang is slightly worse than that of the Wolff since it has a lower probability of flipping large clusters (in the case of Ising models). Both algorithms have been adpated and used to alternative spin glass models (as well as models outside of spin glasses).

We won't present an implementation of this method here since we already looked at the Wolff algorithm for an example of a cluster algorithm.

# Conclusion
In this blog post we have looked at a variety of MCMC methods for simulating Ising models. We started by looking at various local update methods, which we now know do not behave optimally. We extended these ideas to cluster update methods which can show better performance for Ising models. We also looked at the very intuitive simulated annealing and simulated tempering methods, which have been used in optimization problems far outside the realms of spin glass models or even statsitical physics in general.

___
This is the fourth blog post in a series - you can find the next blog post [here](https://lewiscoleblog.com/spin-glass-models-5)


# Free Body Diagram for particles

Renato Naville Watanabe

```python
import numpy as np
import matplotlib.pyplot as plt

%matplotlib notebook
```

### Steps to draw a free-body diagram

1 - Draw separately each object considered in the problem. How you separate depends on what questions you want to answer.

2 - Identify the forces acting on each object. If you are analyzing more than one object, remember the third Newton law (action and reaction), and identify where the reaction of a force is being applied.

3 - Draw all the identified forces, representing them as vectors. The vectors should be represented with the origin in the object. In the case of particles, the origin should be in the center of the particle.

4 - If necessary, you should represent the reference frame in the free-body diagram.

5 - After this, you can solve the problem using the Second Newton Law (see, e.g,  [Newton Laws](https://nbviewer.jupyter.org/github/BMClab/bmc/blob/master/Notebooks/newtonLawForParticles.ipynb)) to find the motion of the particle.

## Basic element and forces

### Spring

Is an element used to represent a force proportional to some length or displacement. It produces a force in the same direction of the vector linking the spring extremities and opposite to its length or displacement from an equilibrium length. Frequently it has a linear relation, but it could be nonlinear as well. The force exerted by the spring in one of the extremities is:

\begin{equation}
\vec{\bf{F}} = - k(||\vec{\bf{r}}||-l_0)\frac{\vec{\bf{r}}}{||\vec{\bf{r}}||} = -k\vec{\bf{r}} +kl_0\frac{\vec{\bf{r}}}{||\vec{\bf{r}}||} = -k\left(1-\frac{l_0}{||\vec{\bf{r}}||}\right)\vec{\bf{r}}
\end{equation}

where $\vec{\bf{r}}$ is the vector linking the extremity applying the force to the other extremity and $l_0$ is the equilibrium length of the spring.

Since the spring element is a massless element, the force in  both extremities have the same absolute value and opposite directions.

### Damping

Is an element used to represent a force proportional to the velocity of displacement.  It produces a force in the same direction of the vector linking the  element extremities and opposite to its velocity.

Frequently it has a linear relation, but it could be nonlinear as well. The force exerted by the damping element in one of its extremities is:

\begin{equation}
\vec{\bf{F}} = - b||\vec{\bf{v}}||\frac{\vec{\bf{v}}}{||\vec{\bf{v}}||} = -b\vec{\bf{v}} = -b\frac{d\vec{\bf{r}}}{dt}
\end{equation}

where $\vec{\bf{r}}$ is the vector linking the extremity applying the force to the other extremity.

For the same reason of the spring, the force in  both extremities have the same absolute value and opposite directions.

### Gravity

The gravity force acts on two masses, each one atracting each other:

\begin{equation}
\vec{{\bf{F}}} = - G\frac{m_1m_2}{||\vec{\bf{r}}||^2}\frac{\vec{\bf{r}}}{||\vec{\bf{r}}||}
\end{equation}

where  $G = 6.67.10^{−11} Nm^2/kg^2$ and $\vec{\bf{r}}$ is a vector with length equal to the distance between the masses and directing towards the other mass. Note the forces acting on each mass have the same absolute value.

Since the mass of the Earth is $m1=5.9736×10^24kg$ and its radius is 6.371×10^6m (see [this notebook](https://nbviewer.jupyter.org/github/BMClab/bmc/blob/master/Notebooks/KineticsFundamentalConcepts.ipynb), for details), the gravity force on Earth is:

\begin{equation}
\vec{{\bf{F}}} = m\vec{\bf{g}}
\end{equation}

with the absolute value of $\vec{\bf{g}}$ approximately equal to 9.81 $m/s^2$,  and point towards the center of Earth.

Below, will be shown some examples of how to draw the free-body diagram and obtain the motion equations to solve the problems.

## 1) No force acting on the particle

The most trivial situation is a particle with no force acting on it.

The free-body diagram is below, with no force vectors acting on the particle.

<figure>

In this situation, the resultant force is:

\begin{equation}
\vec{\bf{F}} = \vec{\bf{0}}
\end{equation}

And the second Newton law for this particle is:

\begin{equation}
m\frac{d^2\vec{\bf{r}}}{dt^2} = \vec{\bf{0}} \rightarrow \frac{d^2\vec{\bf{r}}}{dt^2} = \vec{\bf{0}}
\end{equation}

The motion of of the particle can be found by integrating twice both times, getting the following:

\begin{equation}
\vec{\bf{r}} = \vec{\bf{v_0}}t + \vec{\bf{r0}}
\end{equation}

The particle continues to change its position with the same velocity it was at the beginning of the analysis. This could be predicted by the first Newton law.

## 2) Gravity force acting on the particle

Now, we will consider a ball with the gravity force acting on it. The free-body diagram is depicted below.

<figure>

The only force acting on the ball is the gravitational force:

\begin{equation}
\hat{\bf{F_g}} = - mg \hat{\bf{j}}
\end{equation}

So, we write the Second Newton Law:

\begin{equation}
\hat{\bf{F_g}} = m \frac{d^2\vec{\bf{r}}}{dt^2} \rightarrow - mg \hat{\bf{j}} = m \frac{d^2\vec{\bf{r}}}{dt^2} \rightarrow - g \hat{\bf{j}} =\frac{d^2\vec{\bf{r}}}{dt^2}
\end{equation}

Now, we can separate the equation in two components (x and y):

\begin{equation}
0 = \frac{d^2x}{dt^2}
\end{equation}

and

\begin{equation}
- g = \frac{d^2y}{dt^2}
\end{equation}

These equations were solved in detail in [this Notebook about Newton laws](https://nbviewer.jupyter.org/github/BMClab/bmc/blob/master/Notebooks/newtonLawForParticles.ipynb).

## 3) Ground reaction force

Now, we will analyze the situation of a particle in contact with the ground. To simplify the analysis, only the vertical movement will be considered.

<figure>

The forces acting on the particle are the ground reaction force (often called as normal force) and the gravity force. The free-body diagram of the particle is below:

<figure>

So, the resultant force in the particle is:

\begin{equation}
    \vec{\bf{F}} = \overrightarrow{\bf{GRF}} + m\vec{\bf{g}} = \overrightarrow{\bf{GRF}} - mg\hat{\bf{j}}
\end{equation}

Considering only the y direction:

\begin{equation}
    F = GRF - mg
\end{equation}

Applying the second Newton law to the particle:

\begin{equation}
    m \frac{d^2y}{dt^2} = GRF - mg
\end{equation}

Note that since we have no information about how the force GRF varies along time, we cannot solve this equation. To find the position of the particle along time, one would have to measure the ground reaction force. See [the notebook on Vertical jump](http://nbviewer.jupyter.org/github/BMClab/bmc/blob/master/notebooks/VerticalJump.ipynb) for an application of this model.

## 4) Linear spring in horizontal movement

The example below is mass attached to a spring. The other extremity of the spring is fixed.

<figure>

The only force force acting on the mass is from the spring. Below is the free-body diagram from the mass.

<figure>

Since the movement is horizontal, we can neglect the gravity force.

\begin{equation}
    \vec{\bf{F}} = -k\left(1-\frac{l_0}{||\vec{\bf{r}}||}\right)\vec{\bf{r}}
\end{equation}

Applying the second Newton law to the mass:

\begin{equation}
    m\frac{d^2\vec{\bf{r}}}{dt^2} = -k\left(1-\frac{l_0}{||\vec{\bf{r}}||}\right)\vec{\bf{r}} \rightarrow \frac{d^2\vec{\bf{r}}}{dt^2} = -\frac{k}{m}\left(1-\frac{l_0}{||\vec{\bf{r}}||}\right)\vec{\bf{r}}
\end{equation}

Since the movement is unidimensional, we can deal with it scalarly:

\begin{equation}
    \frac{d^2x}{dt^2} = -\frac{k}{m}\left(1-\frac{l_0}{x}\right)x = -\frac{k}{m}(x-l_0)
\end{equation}

To solve this equation numerically, we must break the equations into two first-order differential equation:

\begin{equation}
    \frac{dv_x}{dt} =  -\frac{k}{m}(x-l_0)
\end{equation}

\begin{equation}
    \frac{dx}{dt} =  v_x
\end{equation}

In the numerical solution below, we will use $k = 40 N/m$, $m = 2 kg$, $l_0 = 0.5 m$ and the mass starts from the position $x = 0.8m$ and at rest.

```python
k = 40
m = 2
l0 = 0.5
x0 = 0.8
v0 = 0

x = x0
v = v0

dt = 0.001
t = np.arange(0, 3, dt)

r = np.array([x])

for i in t[1:]:
    dxdt = v
    dvxdt = -k/m*(x-l0)
    x = x + dt*dxdt
    v = v + dt*dvxdt
    r = np.vstack((r,np.array([x])))

plt.figure()
plt.plot(t,r)
plt.show()
```

## 5) Linear spring in bidimensional movement in horizontal plane

This example has two masses attached to the spring. To solve the motion of both masses, we must draw a free-body diagram for each one of the masses.

<figure>

The only force acting on each mass is the force due to the spring. Since the movement is happening in the horizontal plane, the gravity force can be neglected.

<figure>

So, the forces acting on mass 1 is:

\begin{equation}
    \vec{\bf{F_1}} = k\left(||\vec{\bf{r_2}}-\vec{\bf{r_1}}||-l_0\right)\frac{(\vec{\bf{r_2}}-\vec{\bf{r_1}})}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}
\end{equation}

and the forces acting on mass 2 is:

\begin{equation}
    \vec{\bf{F_2}} =k\left(||\vec{\bf{r_2}}-\vec{\bf{r_1}}||-l_0\right)\frac{(\vec{\bf{r_1}}-\vec{\bf{r_2}})}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}
\end{equation}

Applying the second Newton law for the masses:

\begin{equation}
     m_1\frac{d^2\vec{\bf{r_1}}}{dt^2} = k\left(||\vec{\bf{r_2}}-\vec{\bf{r_1}}||-l_0\right)\frac{(\vec{\bf{r_2}}-\vec{\bf{r_1}})}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||} \rightarrow \frac{d^2\vec{\bf{r_1}}}{dt^2} = -\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_1}}+\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_2}} \rightarrow \frac{d^2x_1\hat{\bf{i}}}{dt^2}+\frac{d^2y_1\hat{\bf{j}}}{dt^2} = -\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_1\hat{\bf{i}}+y_1\hat{\bf{j}})+\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_2\hat{\bf{i}}+y_2\hat{\bf{j}})
\end{equation}

\begin{equation}
     m_2\frac{d^2\vec{\bf{r_2}}}{dt^2} = k\left(||\vec{\bf{r_2}}-\vec{\bf{r_1}}||-l_0\right)\frac{(\vec{\bf{r_1}}-\vec{\bf{r_2}})}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||} \rightarrow \frac{d^2\vec{\bf{r_2}}}{dt^2} = -\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_2}}+\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_1}} \rightarrow \frac{d^2x_2\hat{\bf{i}}}{dt^2}+\frac{d^2y_2\hat{\bf{j}}}{dt^2} = -\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_2\hat{\bf{i}}+y_2\hat{\bf{j}})+\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_1\hat{\bf{i}}+y_1\hat{\bf{j}})
\end{equation}

Now, we can separate the equations for each of the coordinates:

\begin{equation}
    \frac{d^2x_1}{dt^2} = -\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)x_1+\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)x_2=-\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_1-x_2)
\end{equation}

\begin{equation}
    \frac{d^2y_1}{dt^2} = -\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)y_1+\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)y_2=-\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_1-y_2)
\end{equation}

\begin{equation}
    \frac{d^2x_2}{dt^2} = -\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)x_2+\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)x_1=-\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_2-x_1)
\end{equation}

\begin{equation}
    \frac{d^2y_2}{dt^2} = -\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)y_2+\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)y_1=-\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_2-y_1)
\end{equation}

To solve these equations numerically, you must break these equations into first-order  equations:

\begin{equation}
    \frac{dv_{x_1}}{dt} = -\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_1-x_2)
\end{equation}

\begin{equation}
    \frac{dv_{y_1}}{dt} = -\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_1-y_2)
\end{equation}

\begin{equation}
    \frac{dv_{x_2}}{dt} = -\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_2-x_1)
\end{equation}

\begin{equation}
    \frac{dv_{y_2}}{dt} = -\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_2-y_1)
\end{equation}

\begin{equation}
    \frac{dx_1}{dt} = v_{x_1}
\end{equation}

\begin{equation}
    \frac{dy_1}{dt} = v_{y_1}
\end{equation}

\begin{equation}
    \frac{dx_2}{dt} = v_{x_2}
\end{equation}

\begin{equation}
    \frac{dy_2}{dt} = v_{y_2}
\end{equation}

Note that if you did not wanted to know the details about the motion of each mass, but only the motion of the center of mass of the masses-spring system, you could have modeled the whole system as a single particle.

To solve the equations numerically, we will use the $m_1=1 kg$, $m_2 = 2 kg$, $l_0 = 0.5 m$, $k = 90 N/m$ and $x_{1_0} = 0 m$, $x_{2_0} = 0 m$, $y_{1_0} = 1 m$, $y_{2_0} = -1 m$, $v_{x1_0} = -2 m/s$, $v_{x2_0} = 0 m$, $v_{y1_0} = 0 m$, $v_{y2_0} = 0 m$.

```python
x01 = 0
y01= 0.5
x02 = 0
y02 = -0.5
vx01 = 0.1
vy01 = 0
vx02 = -0.1
vy02 = 0

x1= x01
y1 = y01
x2= x02
y2 = y02
vx1= vx01
vy1 = vy01
vx2= vx02
vy2 = vy02
r1 = np.array([x1,y1])
r2 = np.array([x2,y2])

k = 30
m1 = 1
m2 = 1
l0 = 0.5

dt = 0.0001
t = np.arange(0,5,dt)

for i in t[1:]:

    dvx1dt = -k/m1*(x1-x2)*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))
    dvx2dt = -k/m2*(x2-x1)*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))
    dvy1dt = -k/m1*(y1-y2)*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))
    dvy2dt = -k/m2*(y2-y1)*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))

    dx1dt = vx1
    dx2dt = vx2
    dy1dt = vy1
    dy2dt = vy2

    x1 = x1 + dt*dx1dt
    x2 = x2 + dt*dx2dt
    y1 = y1 + dt*dy1dt
    y2 = y2 + dt*dy2dt

    vx1 = vx1 + dt*dvx1dt
    vx2 = vx2 + dt*dvx2dt
    vy1 = vy1 + dt*dvy1dt
    vy2 = vy2 + dt*dvy2dt

    r1 = np.vstack((r1,np.array([x1,y1])))
    r2 = np.vstack((r2,np.array([x2,y2])))

springLength = np.sqrt((r1[:,0]-r2[:,0])**2+(r1[:,1]-r2[:,1])**2)
plt.figure()
plt.plot(t, springLength)
plt.show()

```

## 6)Particle with gravity and linear air resistance

Below is the free-body diagram of a particle with the gravity force and a linear drag force due to the air resistance.

<figure>

the forces being applied in the ball are:

\begin{equation}
\vec{\bf{F}} = -mg \hat{\bf{j}} - b\vec{\bf{v}} = -mg \hat{\bf{j}} - b\frac{d\vec{\bf{r}}}{dt} = -mg \hat{\bf{j}} - b\left(\frac{dx}{dt}\hat{\bf{i}}+\frac{dy}{dt}\hat{\bf{j}}\right) = - b\frac{dx}{dt}\hat{\bf{i}} - \left(mg + b\frac{dy}{dt}\right)\hat{\bf{j}}
\end{equation}

Writing down the Second Newton Law:

\begin{equation}
\vec{\bf{F}} = m \frac{d^2\vec{\bf{r}}}{dt^2} \rightarrow - b\frac{dx}{dt}\hat{\bf{i}} - \left(mg + b\frac{dy}{dt}\right)\hat{\bf{j}} = m\left(\frac{d^2x}{dt^2}\hat{\bf{i}}+\frac{d^2y}{dt^2}\hat{\bf{j}}\right)
\end{equation}

Now, we can separate into one equation for each coordinate:

\begin{equation}
- b\frac{dx}{dt} = m\frac{d^2x}{dt^2} -\rightarrow \frac{d^2x}{dt^2} = -\frac{b}{m} \frac{dx}{dt}
\end{equation}

\begin{equation}
-mg - b\frac{dy}{dt} = m\frac{d^2y}{dt^2} \rightarrow \frac{d^2y}{dt^2} = -\frac{b}{m}\frac{dy}{dt} - g
\end{equation}

These equations were solved  in [this notebook](https://nbviewer.jupyter.org/github/BMClab/bmc/blob/master/Notebooks/newtonLawForParticles.ipynb).

## 7) Particle with gravity and nonlinear air resistance

Below, is the free-body diagram of a particle with the gravity force and a drag force due to the air resistance proportional to the square of the particle velocity.

<figure>

The forces being applied in the ball are:

\begin{equation}
\vec{\bf{F}} = -mg \hat{\bf{j}} - bv^2\hat{\bf{e_t}} = -mg \hat{\bf{j}} - b (v_x^2+v_y^2) \frac{v_x\hat{\bf{i}}+v_y\hat{\bf{j}}}{\sqrt{v_x^2+v_y^2}} =   -mg \hat{\bf{j}} - b \sqrt{v_x^2+v_y^2} \,(v_x\hat{\bf{i}}+v_y\hat{\bf{j}}) =  -mg \hat{\bf{j}} - b \sqrt{\left(\frac{dx}{dt} \right)^2+\left(\frac{dy}{dt} \right)^2} \,\left(\frac{dx}{dt} \hat{\bf{i}}+\frac{dy}{dt}\hat{\bf{j}}\right)
\end{equation}

Writing down the Second Newton Law:

\begin{equation}
\vec{\bf{F}} = m \frac{d^2\vec{\bf{r}}}{dt^2} \rightarrow -mg \hat{\bf{j}} - b \sqrt{\left(\frac{dx}{dt} \right)^2+\left(\frac{dy}{dt} \right)^2} \,\left(\frac{dx}{dt} \hat{\bf{i}}+\frac{dy}{dt}\hat{\bf{j}}\right) = m\left(\frac{d^2x}{dt^2}\hat{\bf{i}}+\frac{d^2y}{dt^2}\hat{\bf{j}}\right)
\end{equation}

Now, we can separate into one equation for each coordinate:

\begin{equation}
- b \sqrt{\left(\frac{dx}{dt} \right)^2+\left(\frac{dy}{dt} \right)^2} \,\frac{dx}{dt} = m\frac{d^2x}{dt^2} \rightarrow \frac{d^2x}{dt^2} = - \frac{b}{m} \sqrt{\left(\frac{dx}{dt} \right)^2+\left(\frac{dy}{dt} \right)^2} \,\frac{dx}{dt}
\end{equation}

\begin{equation}
-mg - b \sqrt{\left(\frac{dx}{dt} \right)^2+\left(\frac{dy}{dt} \right)^2} \,\frac{dy}{dt} = m\frac{d^2y}{dt^2} \rightarrow \frac{d^2y}{dt^2} = - \frac{b}{m} \sqrt{\left(\frac{dx}{dt} \right)^2+\left(\frac{dy}{dt} \right)^2} \,\frac{dy}{dt} -g
\end{equation}

These equations were solved numerically in [this notebook](https://nbviewer.jupyter.org/github/BMClab/bmc/blob/master/Notebooks/newtonLawForParticles.ipynb).

## 8) Linear spring and damping on bidimensional horizontal  movement

This situation is very similar to the example of horizontal movement with one spring and two masses, with a damper added in parallel to the spring.

<figure>

Now, the forces acting on each mass are the force due to the spring and the force due to the damper.

<figure>

So, the forces acting on mass 1 is:

\begin{equation}
    \vec{\bf{F_1}} = b\frac{d(\vec{\bf{r_2}}-\vec{\bf{r_1}})}{dt} +  k\left(||\vec{\bf{r_2}}-\vec{\bf{r_1}}||-l_0\right)\frac{(\vec{\bf{r_2}}-\vec{\bf{r_1}})}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||} =  b\frac{d(\vec{\bf{r_2}}-\vec{\bf{r_1}})}{dt} +  k\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(\vec{\bf{r_2}}-\vec{\bf{r_1}})
\end{equation}

and the forces acting on mass 2 is:

\begin{equation}
    \vec{\bf{F_2}} = b\frac{d(\vec{\bf{r_1}}-\vec{\bf{r_2}})}{dt} +  k\left(||\vec{\bf{r_2}}-\vec{\bf{r_1}}||-l_0\right)\frac{(\vec{\bf{r_1}}-\vec{\bf{r_2}})}{||\vec{\bf{r_1}}-\vec{\bf{r_2}}||}= b\frac{d(\vec{\bf{r_1}}-\vec{\bf{r_2}})}{dt} +  k\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(\vec{\bf{r_1}}-\vec{\bf{r_2}})
\end{equation}

Applying the second Newton law for the masses:

\begin{equation}
     m_1\frac{d^2\vec{\bf{r_1}}}{dt^2} = b\frac{d(\vec{\bf{r_2}}-\vec{\bf{r_1}})}{dt}+k\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(\vec{\bf{r_2}}-\vec{\bf{r_1}}) \rightarrow \frac{d^2\vec{\bf{r_1}}}{dt^2} = -\frac{b}{m_1}\frac{d\vec{\bf{r_1}}}{dt} -\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_1}} + \frac{b}{m_1}\frac{d\vec{\bf{r_2}}}{dt}+\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_2}}  \rightarrow \frac{d^2x_1\hat{\bf{i}}}{dt^2}+\frac{d^2y_1\hat{\bf{j}}}{dt^2} = -\frac{b}{m_1}\left(\frac{dx_1\hat{\bf{i}}}{dt}+\frac{dy_1\hat{\bf{j}}}{dt}\right)-\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_1\hat{\bf{i}}+y_1\hat{\bf{j}})+\frac{b}{m_1}\left(\frac{dx_2\hat{\bf{i}}}{dt}+\frac{dy_2\hat{\bf{j}}}{dt}\right)+\frac{k}{m_1}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_2\hat{\bf{i}}+y_2\hat{\bf{j}}) = -\frac{b}{m_1}\left(\frac{dx_1\hat{\bf{i}}}{dt}+\frac{dy_1\hat{\bf{j}}}{dt}-\frac{dx_2\hat{\bf{i}}}{dt}-\frac{dy_2\hat{\bf{j}}}{dt}\right)-\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_1\hat{\bf{i}}+y_1\hat{\bf{j}}-x_2\hat{\bf{i}}-y_2\hat{\bf{j}})
\end{equation}

\begin{equation}
     m_2\frac{d^2\vec{\bf{r_2}}}{dt^2} = b\frac{d(\vec{\bf{r_1}}-\vec{\bf{r_2}})}{dt}+k\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(\vec{\bf{r_1}}-\vec{\bf{r_2}}) \rightarrow \frac{d^2\vec{\bf{r_2}}}{dt^2} = -\frac{b}{m_2}\frac{d\vec{\bf{r_2}}}{dt} -\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_2}} + \frac{b}{m_2}\frac{d\vec{\bf{r_1}}}{dt}+\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)\vec{\bf{r_1}}  \rightarrow \frac{d^2x_2\hat{\bf{i}}}{dt^2}+\frac{d^2y_2\hat{\bf{j}}}{dt^2} = -\frac{b}{m_2}\left(\frac{dx_2\hat{\bf{i}}}{dt}+\frac{dy_2\hat{\bf{j}}}{dt}\right)-\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_2\hat{\bf{i}}+y_2\hat{\bf{j}})+\frac{b}{m_2}\left(\frac{dx_1\hat{\bf{i}}}{dt}+\frac{dy_1\hat{\bf{j}}}{dt}\right)+\frac{k}{m_2}\left(1-\frac{l_0}{||\vec{\bf{r_2}}-\vec{\bf{r_1}}||}\right)(x_1\hat{\bf{i}}+y_1\hat{\bf{j}})=-\frac{b}{m_2}\left(\frac{dx_2\hat{\bf{i}}}{dt}+\frac{dy_2\hat{\bf{j}}}{dt}-\frac{dx_1\hat{\bf{i}}}{dt}-\frac{dy_1\hat{\bf{j}}}{dt}\right)-\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_2\hat{\bf{i}}+y_2\hat{\bf{j}}-x_1\hat{\bf{i}}-y_1\hat{\bf{j}})
\end{equation}

Now, we can separate the equations for each of the coordinates:

\begin{equation}
    \frac{d^2x_1}{dt^2} = -\frac{b}{m_1}\left(\frac{dx_1}{dt}-\frac{dx_2}{dt}\right)-\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_1-x_2)
\end{equation}

\begin{equation}
    \frac{d^2y_1}{dt^2} = -\frac{b}{m_1}\left(\frac{dy_1}{dt}-\frac{dy_2}{dt}\right)-\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_1-y_2)
\end{equation}

\begin{equation}
    \frac{d^2x_2}{dt^2} = -\frac{b}{m_2}\left(\frac{dx_2}{dt}-\frac{dx_1}{dt}\right)-\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_2-x_1)
\end{equation}

\begin{equation}
    \frac{d^2y_2}{dt^2} = -\frac{b}{m_2}\left(\frac{dy_2}{dt}-\frac{dy_1}{dt}\right)-\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_2-y_1)
\end{equation}

If you want to solve these equations numerically, you must break these equations into first-order  equations:

\begin{equation}
    \frac{dv_{x_1}}{dt} = -\frac{b}{m_1}\left(v_{x_1}-v_{x_2}\right)-\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_1-x_2)
\end{equation}

\begin{equation}
    \frac{dv_{y_1}}{dt} =  -\frac{b}{m_1}\left(v_{y_1}-v_{y_2}\right)-\frac{k}{m_1}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_1-y_2)
\end{equation}

\begin{equation}
    \frac{dv_{x_2}}{dt} = -\frac{b}{m_2}\left(v_{x_2}-v_{x_1}\right)-\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(x_2-x_1)
\end{equation}

\begin{equation}
    \frac{dv_{y_2}}{dt} = -\frac{b}{m_2}\left(v_{y_2}-v_{y_1}\right)-\frac{k}{m_2}\left(1-\frac{l_0}{\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}}\right)(y_2-y_1)
\end{equation}

\begin{equation}
    \frac{dx_1}{dt} = v_{x_1}
\end{equation}

\begin{equation}
    \frac{dy_1}{dt} = v_{y_1}
\end{equation}

\begin{equation}
    \frac{dx_2}{dt} = v_{x_2}
\end{equation}

\begin{equation}
    \frac{dy_2}{dt} = v_{y_2}
\end{equation}

To solve the equations numerically, we will use the $m_1=1 kg$, $m_2 = 2 kg$, $l_0 = 0.5 m$, $k = 10 N/m$, $b = 0.6 Ns/m$ and $x_{1_0} = 0 m$, $x_{2_0} = 0 m$, $y_{1_0} = 1 m$, $y_{2_0} = -1 m$, $v_{x1_0} = -2 m/s$, $v_{x2_0} = 1 m$, $v_{y1_0} = 0 m$, $v_{y2_0} = 0 m$.

```python
x01 = 0
y01= 1
x02 = 0
y02 = -1

vx01 = -2
vy01 = 0
vx02 = 1
vy02 = 0

x1= x01
y1 = y01
x2= x02
y2 = y02
vx1= vx01
vy1 = vy01
vx2= vx02
vy2 = vy02
r1 = np.array([x1,y1])
r2 = np.array([x2,y2])

k = 10
m1 = 1
m2 = 2
b = 0.6
l0 = 0.5

dt = 0.001
t = np.arange(0,5,dt)

for i in t[1:]:
    dvx1dt = -b/m1*(vx1-vx2) -k/m1*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))*(x1-x2)
    dvx2dt = -b/m2*(vx2-vx1) -k/m2*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))*(x2-x1)
    dvy1dt = -b/m1*(vy1-vy2) -k/m1*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))*(y1-y2)
    dvy2dt = -b/m2*(vy2-vy1) -k/m2*(1-l0/np.sqrt((x2-x1)**2+(y2-y1)**2))*(y2-y1)
    dx1dt = vx1
    dx2dt = vx2
    dy1dt = vy1
    dy2dt = vy2
    x1 = x1 + dt*dx1dt
    x2 = x2 + dt*dx2dt
    y1 = y1 + dt*dy1dt
    y2 = y2 + dt*dy2dt
    vx1 = vx1 + dt*dvx1dt
    vx2 = vx2 + dt*dvx2dt
    vy1 = vy1 + dt*dvy1dt
    vy2 = vy2 + dt*dvy2dt
    r1 = np.vstack((r1,np.array([x1,y1])))
    r2 = np.vstack((r2,np.array([x2,y2])))

springDampLength = np.sqrt((r1[:,0]-r2[:,0])**2+(r1[:,1]-r2[:,1])**2)
plt.figure()
plt.plot(t, springDampLength)
plt.show()

plt.figure()
plt.plot(r1[:,0], r1[:,1],'r.')
plt.plot(r2[:,0], r2[:,1],'b.')
plt.plot((m1*r1[:,0]+m2*r2[:,0])/(m1+m2), (m1*r1[:,1]+m2*r2[:,1])/(m1+m2),'g.')
plt.xlim(-2,2)
plt.ylim(-2,2)
plt.show()
```

## 9) Simple muscle model

The diagram below shows a simple muscle model. The spring in the left represents the tendinous tissues and the spring in the right represents the elastic properties of the muscle fibers. The damping is present to model the viscous properties of the muscle fibers, the element CE is the contractile element (force production) and the mass $m$ is the muscle mass.

The length $L_{MT}$ is the length of the muscle plus the tendon. In our model $L_{MT}$ is constant, but it could be a function of the joint angle.

<figure>

The length of the tendon will be denoted by $l_T(t)$ and the muscle length, by $l_{m}(t)$. Both lengths are related by each other  by the following expression:

\begin{equation}
l_t(t) + l_m(t) = L_{MT}
\end{equation}

The free-body diagram of the muscle mass is depicted below.

<figure>

The resultant force being applied in the muscle mass is:

$\vec{\bf{F}} = -k_T(||\vec{\bf{r_m}}||-l_{t_0})\frac{\vec{\bf{r_m}}}{||\vec{\bf{r_m}}||} + b\frac{d(L_{MT}\hat{\bf{i}} - \vec{\bf{r_{m}}})}{dt} + k_m (||L_{MT}\hat{\bf{i}} - \vec{\bf{r_{m}}}||-l_{m_0})\frac{L_{MT}\hat{\bf{i}} - \vec{\bf{r_{m}}}}{||L_{MT}\hat{\bf{i}} - \vec{\bf{r_{m}}}||} +\vec{\bf{F}}{\bf{_{CE}}}(t)$

Since the model is unidimensional, we can assume that the force $\vec{\bf{F}}\bf{_{CE}}(t)$ is in the x direction, so the analysis will be done only in this direction.

$F = -k_T(l_t-l_{t_0}) + b\frac{d(L_{MT} - l_t)}{dt} + k_m (l_m-l_{m_0}) + F_{CE}(t) = -k_T(l_t-l_{t_0}) -b\frac{dl_t}{dt} + k_m (L_{MT}-l_t-l_{m_0}) + F_{CE}(t) = -b\frac{dl_t}{dt}-(k_T+k_m)l_t+F_{CE}(t)+k_Tl_{t_0}+k_m(L_{MT}-l_{m_0})$

Applying the second Newton law:

$m\frac{d^2l_t}{dt^2} = -b\frac{dl_t}{dt}-(k_T+k_m)l_t+F_{CE}(t)+k_Tl_{t_0}+k_m(L_{MT}-l_{m_0})$

To solve this equation, we must break the equation into two first-order differential equations:

\begin{equation}
\frac{dvt}{dt} = - \frac{b}{m}v_t - \frac{k_T+k_m}{m}l_t +\frac{F_{CE}(t)}{m} + \frac{k_T}{m}l_{t_0}+\frac{k_m}{m}(L_{MT}-l_{m_0})
\end{equation}

\begin{equation}
\frac{d l_t}{dt} = v_t
\end{equation}

Now, we can solve these equations by using some numerical method. To obtain the solution, we will use the damping factor of the muscle as $b = 10\,Ns/m$, the muscle mass is $m = 2 kg$, the stiffness of the tendon as $k_t=1000\,N/m$ and the elastic element of the muscle as $k_m=1500\,N/m$. The tendon-length is $L_{MT} = 0.35\,m$, and the tendon equilibrium length is $l_{t0} = 0.28\,m$ and the muscle fiber equilibrium length is $l_{m0} = 0.07\,m$. Both the tendon and the muscle fiber are at their equilibrium lengths and at rest.

Also, we will model the force of the contractile element as a Heaviside step of $90\,N$ (90 N beginning at $t=0$), but normally it is modeled as a function of $l_m$ and $v_m$ having a neural activation signal as input.

```python
m = 2
b = 10
km = 1500
kt = 1000
lt0 = 0.28
lm0 = 0.07

Lmt = 0.35
vt0 = 0

dt = 0.0001
t = np.arange(0, 10, dt)

Fce = 90

lt = lt0
vt = vt0
ltp = np.array([lt0])
lmp = np.array([lm0])
Ft = np.array([0])
for i in range(1,len(t)):
    dvtdt = -b/m*vt-(kt+km)/m*lt  + Fce/m + kt/m*lt0 +km/m*(Lmt-lm0)
    dltdt = vt
    vt = vt + dt*dvtdt
    lt = lt + dt*dltdt
    Ft = np.vstack((Ft,np.array(kt*(lt-lt0))))
    ltp = np.vstack((ltp,np.array(lt)))
    lmp = np.vstack((lmp,np.array(Lmt - lt)))

plt.figure()
plt.plot(t,Ft)
plt.xlabel('t')
plt.ylabel('Tendon force (N)')
plt.show()
plt.figure()
plt.plot(t,ltp)
plt.plot(t,lmp)
plt.xlabel('t')
plt.ylabel('Length (m)')
plt.show()
```

## Problems

Solve the problems 3.3.9, 3.3.20, 10.1.6, 12.1.6(a,b,c,d,f), 12.1.7, 12.1.10 (a,b) from Ruina and Pratap book

## References

- Ruina A, Rudra P (2015) [Introduction to Statics and Dynamics](http://ruina.tam.cornell.edu/Book/index.html). Oxford University Press.
- Nigg & Herzog (2006) [Biomechanics of the Musculo-skeletal System](https://books.google.com.br/books?id=hOIeAQAAIAAJ&dq=editions:ISBN0470017678). 3rd Edition. Wiley.

```python

```


# Part D: Comparison of toroidal meniscus models with different profile shapes

## Introduction

So far all the capillary entry pressures for the percoaltion examples were calculated using the ``Standard`` physics model which is the ``Washburn`` model for straight walled capillary tubes. This has been shown to be a bad model for fibrous media where the walls of throats are converging and diverging. In the study [Capillary Hysteresis in Neutrally Wettable Fibrous Media: A Pore Network Study of a Fuel Cell Electrode](http://link.springer.com/10.1007/s11242-017-0973-2) percolation in fibrous media was simulated using a meniscus model that assumed the contrictions between fibers are similar to a toroid:

This model was first proposed by Purcell and treats the inner solid profile as a circle. As the fluid invades through the center of the torus the meniscus is pinned to the surface and the "effective" contact angle becomes influenced by the converging diverging geometry and is a function of the filling angle $\alpha$. The shape of the meniscus as the invading phase moves upwards through the torus with key model parameters is shown below.

Different intrinsic contact angles through invading phase are shown above: (a) 60$^\circ$, (b) 90$^\circ$ and (c) 120$^\circ$. All scenarios clearly show an inflection of the meniscus curvature signifying a switch in the sign of the capillary pressure from negative to positive. This inflection is predicted to occur for all contact angles by the model with varying filling angle. The capillary pressure can be shown to be:

$P_C = -2\sigma cos(\theta-\alpha))/(r+R(1-cos(\alpha))$

A consequence of the circular solid profile is that all fluid behaves as non-wetting fluid because $\alpha$ can range from -90$^\circ$ to 90$^\circ$ degrees and so even if $\theta$ is 0 then the meniscus is still pinned at zero capillary pressure at the very furthest part of the throat where the $\alpha$ is 90$^\circ$

Considering other shapes of solid profile this situation can be avoided. It will be shown by reformulating the Purcell model in a more general way that allows for a flexible defintion of the solid profile that filling angle can be limited to values below 90 and allow for spontaneous imbibition (percolation threshold below zero) of highly wetting fluids.

## Set up
We will set up a trivially small network with one throat to demonstrate the use of the meniscus model. Here we do the imports and define a few functions for plotting.

```python
import matplotlib
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import sympy as syp
from sympy import lambdify, symbols
from sympy import atan as sym_atan
from sympy import cos as sym_cos
from sympy import sin as sym_sin
from sympy import sqrt as sym_sqrt
from sympy import pi as sym_pi
from ipywidgets import interact, fixed
from IPython.display import display
import warnings
np.random.seed(10)
warnings.simplefilter(action='ignore')
matplotlib.rcParams['figure.figsize'] = (5, 5)
```

```python
theta = 60
fiberRad = 5e-6
throatRad = 1e-5
max_bulge = 1e-5
```

Now we define our two pore network and add the meniscus model in several modes: 'max' returns the maximum pressure experienced by the meniscus as it transitions through the throat, i.e. the burst entry pressure. 'touch' is the pressure at which the meniscus has protruded past the throat center a distance defined by the 'touch_length' dictionary key. In network simulations this could be set to the pore_diameter. Finally the 'men' mode accepts a target_Pc parameter and returns all the mensicus information required for assessing cooperative filling or plotting.

```python
import openpnm as op
import openpnm.models.physics as pm
net = op.network.Cubic(shape=[2, 1, 1], spacing=5e-5)
geo = op.geometry.StickAndBall(network=net,
                               pores=net.pores(),
                               throats=net.throats())
phase = op.phases.Water(network=net)
phase['pore.contact_angle'] = theta
phys = op.physics.Standard(network=net,
                           phase=phase,
                           geometry=geo)
geo['throat.diameter'] = throatRad*2
geo['throat.touch_length'] = max_bulge
```

We define a plotting function that uses the meniscus data:
$\alpha$ is filling angle as defined above,
$radius$ is the radius of curvature of the mensicus,
$center$ is the position of the centre of curvature relative to the throat center along the axis of the throat,
$\gamma$ is the angle between the throat axis and the line joining the meniscus center and meniscus contact point.

```python
def plot_meniscus(target_Pc, meniscus_model=None, ax=None):
    throatRad = geo['throat.diameter'][0]/2
    theta = np.deg2rad(phys['pore.contact_angle'][0])
    throat_a = phys['throat.scale_a']
    throat_b = phys['throat.scale_b']
    x_points = np.arange(-0.99, 0.99, 0.01)*throat_a
    if ax is None:
        fig, ax = plt.subplots()
    if meniscus_model.__name__ == 'purcell':
        # Parameters for plotting fibers
        x, R, rt, s, t = syp.symbols('x, R, rt, s, t')
        y = R*syp.sqrt(1- (x/R)**2)
        r = rt + (R-y)
        rx = syp.lambdify((x, R, rt), r, 'numpy')
        ax.plot(x_points, rx(x_points, fiberRad, throatRad), 'k-');
        ax.plot(x_points, -rx(x_points, fiberRad, throatRad), 'k-');
        phys.add_model(propname='throat.meniscus',
                       model=meniscus_model,
                       mode='men',
                       r_toroid=fiberRad,
                       target_Pc=target_Pc)
    elif meniscus_model.__name__ == 'sinusoidal':
        x, a, b, rt, sigma, theta = syp.symbols('x, a, b, rt, sigma, theta')
        y = (sym_cos(sym_pi*x/(2*a)))*b
        r = rt + (b-y)
        rx = lambdify((x, a, b, rt), r, 'numpy')
        ax.plot(x_points, rx(x_points, throat_a, throat_b, throatRad), 'k-');
        ax.plot(x_points, -rx(x_points, throat_a, throat_b, throatRad), 'k-');
        phys.add_model(propname='throat.meniscus',
                       model=meniscus_model,
                       mode='men',
                       r_toroid=fiberRad,
                       target_Pc=target_Pc)
    else:
        # General Ellipse
        x, a, b, rt, sigma, theta = syp.symbols('x, a, b, rt, sigma, theta')
        profile_equation = phys.models['throat.entry_pressure']['profile_equation']
        if profile_equation == 'elliptical':
            y = sym_sqrt(1 - (x/a)**2)*b
        elif profile_equation == 'sinusoidal':
            y = (sym_cos(sym_pi*x/(2*a)))*b
        r = rt + (b-y)
        rx = lambdify((x, a, b, rt), r, 'numpy')
        ax.plot(x_points, rx(x_points, throat_a, throat_b, throatRad), 'k-');
        ax.plot(x_points, -rx(x_points, throat_a, throat_b, throatRad), 'k-');
        phys.add_model(propname='throat.meniscus',
                       model=meniscus_model,
                       profile_equation=profile_equation,
                       mode='men',
                       target_Pc=target_Pc)
    men_data = {}
    men_data['alpha'] = phys['throat.meniscus.alpha']
    men_data['gamma'] = phys['throat.meniscus.gamma']
    men_data['radius'] = phys['throat.meniscus.radius']
    men_data['center'] = phys['throat.meniscus.center']
    arc_cen = men_data['center']
    arc_rad = men_data['radius']
    arc_angle = men_data['gamma']
    angles = np.linspace(-arc_angle, arc_angle, 100)
    arcx = arc_cen + arc_rad*np.cos(angles)
    arcy = arc_rad*np.sin(angles)
    ax.plot(arcx, arcy, 'b-')
    ax.scatter(phys['throat.meniscus.pos'], phys['throat.meniscus.rx']);
    ax.axis('equal')
    ax.ticklabel_format(style='sci', axis='both', scilimits=(-6,-6))
    return ax
```

# Circular (Purcell)

```python
circular_model = pm.meniscus.purcell

phys.add_model(propname='throat.max',
               model=circular_model,
               mode='max',
               r_toroid=fiberRad)
phys.add_model(propname='throat.touch',
               model=circular_model,
               mode='touch',
               r_toroid=fiberRad)
phys.add_model(propname='throat.meniscus',
               model=circular_model,
               mode='men',
               r_toroid=fiberRad,
               target_Pc=1000)
touch_Pc = phys['throat.touch'][0]
print('Pressure at maximum bulge', np.around(touch_Pc, 0))
max_Pc_circle = phys['throat.max'][0]
print('Circular profile critical entry pressure', np.around(max_Pc_circle, 0))
```

    Pressure at maximum bulge 7213.0
    Circular profile critical entry pressure 8165.0

We can see that the touch_Pc calculated earlier, corresponds with the tip of the meniscus exceeding the max_bulge parameter. Try changing this and re-running to see what happens.

```python
#NBVAL_IGNORE_OUTPUT
ax = plot_meniscus(target_Pc=touch_Pc, meniscus_model=circular_model)
ax.plot([max_bulge, max_bulge], [-throatRad, throatRad], 'r--');
```

```python
#NBVAL_IGNORE_OUTPUT
ax = plot_meniscus(target_Pc=max_Pc_circle, meniscus_model=circular_model)
```

We can interact with the mensicus model by changing the target_Pc parameter.

```python
#NBVAL_IGNORE_OUTPUT
interact(plot_meniscus,
         target_Pc=(-2000, max_Pc_circle, 1),
         meniscus_model=fixed(circular_model),
         ax=fixed(None));
```

    interactive(children=(FloatSlider(value=3082.0, description='target_Pc', max=8165.324889242946, min=-2000.0, s…

Here we can see that the critical entry pressure for the circular profile is positive, even though the intrinsic contact angle is highly non-wetting

# Sinusoidal

Now we can start to compare the different meniscus models:

```python
sinusoidal_model = pm.meniscus.sinusoidal
```

```python
display(sinusoidal_model)
```

    <function openpnm.models.physics.meniscus.sinusoidal(target, mode='max', target_Pc=None, num_points=1000.0, r_toroid=5e-06, throat_diameter='throat.diameter', pore_diameter='pore.diameter', touch_length='throat.touch_length', surface_tension='pore.surface_tension', contact_angle='pore.contact_angle')>

```python
phys.add_model(propname='throat.meniscus',
               model=sinusoidal_model,
               mode='men',
               r_toroid=fiberRad,
               target_Pc=1000)
```

The equation for the solid sinusoidal profile is:

```python
x, a, b, rt, sigma, theta = syp.symbols('x, a, b, rt, sigma, theta')
y = (sym_cos(sym_pi*x/(2*a)))*b
r = rt + b-y
r
```

$\displaystyle - b \cos{\left(\frac{\pi x}{2 a} \right)} + b + rt$

```python
# Derivative of profile
rprime = r.diff(x)
rprime
```

$\displaystyle \frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a}$

```python
# Filling angle
alpha = sym_atan(rprime)
alpha
```

$\displaystyle \operatorname{atan}{\left(\frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a} \right)}$

```python
# angle between y axis, meniscus center and meniscus contact point
eta = sym_pi - (theta + alpha)
eta
```

$\displaystyle - \theta - \operatorname{atan}{\left(\frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a} \right)} + \pi$

```python
# angle between x axis, meniscus center and meniscus contact point
gamma = sym_pi/2 - eta
gamma
```

$\displaystyle \theta + \operatorname{atan}{\left(\frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a} \right)} - \frac{\pi}{2}$

```python
# Radius of curvature of meniscus
rm = r/sym_cos(eta)
rm
```

$\displaystyle - \frac{- b \cos{\left(\frac{\pi x}{2 a} \right)} + b + rt}{\cos{\left(\theta + \operatorname{atan}{\left(\frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a} \right)} \right)}}$

```python
# distance along x-axis from center of curvature to meniscus contact point
d = rm*sym_sin(eta)
d
```

$\displaystyle - \frac{\left(- b \cos{\left(\frac{\pi x}{2 a} \right)} + b + rt\right) \sin{\left(\theta + \operatorname{atan}{\left(\frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a} \right)} \right)}}{\cos{\left(\theta + \operatorname{atan}{\left(\frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a} \right)} \right)}}$

```python
# Capillary Pressure
p = 2*sigma/rm
p
```

$\displaystyle - \frac{2 \sigma \cos{\left(\theta + \operatorname{atan}{\left(\frac{\pi b \sin{\left(\frac{\pi x}{2 a} \right)}}{2 a} \right)} \right)}}{- b \cos{\left(\frac{\pi x}{2 a} \right)} + b + rt}$

```python
phys.add_model(propname='throat.max',
               model=sinusoidal_model,
               mode='max',
               r_toroid=fiberRad)
phys.add_model(propname='throat.touch',
               model=sinusoidal_model,
               mode='touch',
               r_toroid=fiberRad)
max_Pc_sin = phys['throat.max'][0]
print(max_Pc_sin)
```

    4729.770413396985

```python
#NBVAL_IGNORE_OUTPUT
plot_meniscus(target_Pc=max_Pc_sin, meniscus_model=sinusoidal_model);
```

```python
#NBVAL_IGNORE_OUTPUT
interact(plot_meniscus,
         target_Pc=(-2000, max_Pc_sin, 1),
         meniscus_model=fixed(sinusoidal_model),
         ax=fixed(None));
```

    interactive(children=(FloatSlider(value=1364.0, description='target_Pc', max=4729.770413396985, min=-2000.0, s…

Now the crtical entry pressure is negative signifying that spontaneous imbibition will occur

# General Elliptical

Similarly we can define an elliptical profile and use the same method to determine the capillary pressure:

```python
y = sym_sqrt(1 - (x/a)**2)*b
y
```

$\displaystyle b \sqrt{1 - \frac{x^{2}}{a^{2}}}$

In-fact this is the model that OpenPNM uses for Purcell as well with a = b = fiber radius

```python
# Scale ellipse in x direction
phys['throat.scale_a'] = fiberRad
# Scale ellipse in y direction
phys['throat.scale_b'] = fiberRad
general_model = pm.meniscus.general_toroidal
phys.add_model(propname='throat.entry_pressure',
               model=general_model,
               profile_equation='elliptical',
               mode='max')
max_Pc_ellipse = phys['throat.entry_pressure'][0]
print(max_Pc_ellipse)
```

    8165.324889242946

```python
#NBVAL_IGNORE_OUTPUT
plot_meniscus(target_Pc=max_Pc_ellipse, meniscus_model=general_model);
```

```python
max_Pc_ellipse
```

    8165.324889242946

```python
#NBVAL_IGNORE_OUTPUT
interact(plot_meniscus,
         target_Pc=(-2000, max_Pc_ellipse, 1),
         meniscus_model=fixed(general_model),
         ax=fixed(None));
```

    interactive(children=(FloatSlider(value=3082.0, description='target_Pc', max=8165.324889242946, min=-2000.0, s…

The two scale factors can now be used to determine a wide range of capillary behaviours with one general model. Below we run the model for a range of scaling factors showing the effect on the sign and magnitude of the entry pressure.

```python
#NBVAL_IGNORE_OUTPUT
bs = np.linspace(0.2, 1.0, 4)*throatRad
phys['throat.scale_a'] = throatRad
elliptical_pressures = []
sinusoidal_pressures = []
fig, (ax1, ax2) = plt.subplots(2, len(bs), figsize=(10, 10))
for i in range(len(bs)):
    phys['throat.scale_b'] = bs[i]
    phys.add_model(propname='throat.entry_pressure',
                   model=general_model,
                   profile_equation='elliptical',
                   mode='max',
                   num_points=1000)
    Pc = phys['throat.entry_pressure']
    elliptical_pressures.append(Pc)
    plot_meniscus(target_Pc=Pc, meniscus_model=general_model, ax=ax1[i])
for i in range(len(bs)):
    phys['throat.scale_b'] = bs[i]
    phys.add_model(propname='throat.entry_pressure',
                   model=general_model,
                   profile_equation='sinusoidal',
                   mode='max',
                   num_points=1000)
    Pc = phys['throat.entry_pressure']
    sinusoidal_pressures.append(Pc)
    plot_meniscus(target_Pc=Pc, meniscus_model=general_model, ax=ax2[i])
```

```python
#NBVAL_IGNORE_OUTPUT
plt.figure()
plt.plot(bs/throatRad, elliptical_pressures, 'g-');
plt.plot(bs/throatRad, sinusoidal_pressures, 'r-');
```

Here we can see that the two different shaped profiles lead to quite different capiallary behaviour. The elliptical profile always resuls in positive pressure and the meniscus is basically pinned to the end of the throat where highest pressure occurs as alpha always reaches 90. Whereas the sinusiodal model allows for spontaneous imbibition where a breakthrough may occur at negative capillary pressure for wetting fluids if the wall angle is shallow.

examples/percolation/Part D - Meniscus Model Comparison.ipynb