```python
from tectosaur2.nb_config import setup

setup()
```

# Slip on an infinite strike slip fault in an elastic half space

Last time we solved for the displacement and stress given slip on an infinitely long strike slip fault. This time, we'll make the problem more interesting (and a bit harder!) by adding in a free surface. We'll replicate classical solutions for the surface displacement on a half-space given slip on a fault. In the next section, we'll add in a topographic free surface and have some fun!

The goal here is to reproduce the classical results for an antiplane fault as a screw dislocation in an elastic half space. This is a great test of a numerical boundary integral solver because it has several particularly difficult features:
- **Infinite surfaces**: The free surface in the analytical solution is infinite. Numerical methods are normally very bad at infinity! The traditional solution to this is to use half-space Green's functions for a boundary integral method - effectively, this solves the problem analytically.
- **The fault tip**: Constant slip on a finite fault will result in a stress singularity at the bottom tip of the fault. This results from the infinitesimal jump from non-zero slip on the fault side of the fault tip to zero slip on the non-fault side of the fault tip. Or, from a calculus perspective, $\frac{\partial u_z}{\partial y} = \lim_{\Delta y \to 0} \frac{\Delta u_z}{\Delta y}$. At the fault tip, $\Delta u_z$ remains constant and non-zero as $\Delta y$ converges to zero. The result is that the limit is infinite.
- **The fault-surface intersection**: Considering either surface alone, there will be another singularity at the fault-surface intersection for the same logic as the fault tip: for the free surface, there is a jump in the displacement across the fault and for the fault, there is effectively another fault tip. However, these singularities exactly cancel leaving an entirely regular displacement and stress solution.

I'm going to solve all these issues via a QBX-based direct numerical solution. The methodology used here:
1. Can model very long but not truly infinite surfaces [^infinite].
2. Can calculate displacement and stress arbitrarily close a singularity [^caveat].

[^infinite]: It would probably be approximate truly infinite surfaces with something like [Gauss-Laguerre quadrature](https://en.wikipedia.org/wiki/Gauss–Laguerre_quadrature). I've never needed to do this, but it'd be cool. While writing this, I'm trying to avoid getting [nerd-sniped](https://xkcd.com/356/) into trying this out.

[^caveat]: Floating point error will scale inversely with the distance from the singularity. So, if you want to calculate values very very close to the singularity (something like 1e-6 or less) then the accuracy will be correspondingly lower.

```python
import warnings
import numpy as np
import sympy
import matplotlib.pyplot as plt
from tectosaur2 import gauss_rule, refine_surfaces, integrate_term, pts_grid
from tectosaur2.laplace2d import double_layer, hypersingular
```

## Constructing the mesh

Step one is to set up the geometry that we will be modeling. To do this, we use the `refine_surfaces` function. `refine_surfaces` adaptively refines the specified surfaces into "panels" that follow a careful set of rules designed to guarantee that our numerical integration operations are accurate based on the "stage-1" rules layed out in section 3 of {cite:t}`walaFastAlgorithmQuadrature2019`. It takes as arguments:
* A list of surfaces. Here, we will be specifying the surfaces symbolically using `sympy`. The variables $t$ is implicitly assumed to vary in `[-1, 1]`.
  - So, for a vertical fault extending one unit deep, we specify that $y = \frac{1}{2}(t + 1)$.
  - And for a horizontal free surface extending 1000 fault lengths away, we specify $x = 1000t$.
* A quadrature rule that will be used to integrate over each panel.
* An optional list of control points which has shape `Nx4`. These points allow for specifying regions of the surface that should be refined more than the default.
  - The first two components specify the location of the control point. In this case, the control point is located at the fault-surface intersection.
  - The third component specifies the control radius of the control point. Every panel within this radius will be refined
  - The fourth component specifies the maximum length of a panel within the control radius of the control point. So, the end result of the control point below is that every panel with 10 fault lengths of the origin (the fault-surface intersection) will have a maximum length of 0.2.

I've plotted the resulting surface panels in the figure. The nodes on the surface lines are the edges of panels.
- In the zoomed in figure, you can see the effect of the control point setting the fault-surface intersection length scale.
- In the zoomed out figure, you can see the effect of the adaptive refinement. One of the criteria is that no panel can be more than twice the length of any panel with a panel length's radius of that panel. This results in the step wise increasing length of panels.

```python
surf_half_L = 1000
t = sympy.var("t")
fault, free = refine_surfaces(
    [
        (t, t * 0, (t + 1) * -0.5), # the fault
        (t, -t * surf_half_L, 0 * t) # the free surface
    ],
    gauss_rule(8),
    control_points=np.array([(0, 0, 10, 0.2)]),
)
print(
    f"The free surface mesh has {free.n_panels} panels with a total of {free.n_pts} points."
)
print(
    f"The fault mesh has {fault.n_panels} panels with a total of {fault.n_pts} points."
)
plt.figure(figsize=(8, 4))
plt.subplot(1, 2, 1)
plt.title("Zoomed in")
plt.plot(fault.panel_edges[:, 0], fault.panel_edges[:, 1], "r-o")
plt.plot(free.panel_edges[:, 0], free.panel_edges[:, 1], "k-o")
plt.axis("scaled")
plt.xlim([-1, 1])
plt.ylim([-1.5, 0.5])
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.subplot(1, 2, 2)
plt.title("Zoomed out")
plt.plot(free.panel_edges[:, 0], free.panel_edges[:, 1], "k-o", markersize=4)
plt.axis("equal")
plt.xlabel("$x$")
plt.ylabel("$y$")
plt.tight_layout()
plt.show()
```

## Solving for surface displacement

The first step in solving the problem is to calculate the surface displacement given unit slip on the fault. To do this, we are going to invert the relevant integral equation.

The governing integral equation here is:

\begin{equation}
u_z(x) = -\int_{S} \frac{\partial G(x, y)}{\partial y} u_z(y) dy  ~~~~~~~~~~~~~ \forall x \in \Omega / \partial\Omega
\end{equation}

where $G(x, y)$ is the fundamental solution of the Laplace equation, $u_z(y)$ is the slip at point $y$, $\Omega$ is the domain of interest (the Earth!) and $\partial \Omega$ is the boundary of the domain (the free surface **and** the fault) and $u(x)$ is the displacement that we would like to calculate. Finally, $x$ is restricted to lie in the interior of the domain.

To get to something that we can solve numerically to recover, we need a few steps:
1. Separate the surface into two portions: the free surface and the fault.
2. Model the fault as two co-located surface with a jump in displacement.
3. Take the limit as $x$ approaches the free surface.

The result is:

\begin{equation}
u_z(x) = -\lim_{\hat{x} \to x}\int_{H} \frac{\partial G(\hat{x}, y)}{\partial y} u_z(y) dy -\int_{F} \frac{\partial G(x, y)}{\partial y} \Delta u_z(y) dy  ~~~~~~~~~~~~~ \forall x \in H
\end{equation}

where $H$ is the free surface, $F$ is the fault, $\Delta u_z(y)$ is fault slip.

In partially discretized form where we've decomposed the displacement and slip fields into basis functions, the resulting linear system will look like:

\begin{align}
u_i = -A_{ij}u_j - B_{ij}(\Delta u)_j \\

A_{ik} = \lim_{\hat{x} \to x_i}\int_{H} \frac{\partial G(\hat{x}, y)}{\partial y} \psi_k(y) dy \\

B_{ij} = \int_{F} \frac{\partial G(x_i, y)}{\partial y} \phi_j(y) dy\\

u(x) = \sum_{i} u_i \psi_i(x)\\
\Delta u(x) = \sum_{i} (\Delta u)_i \phi_i(x)\\
\end{align}

Since we want to calculate the unknown surface displacement $u_i$ from the known slip $(\Delta u)_j$, we can rearrange the linear system to:
\begin{equation}
(I + A_{ij})u_j = -B_{ij}(\Delta u)_j
\end{equation}

The remaining task is to compute the matrices $A$ and $B$ and then invert the left hand side of this linear system.

### Implementation

To construct these matrices, we will use the `integrate_term` function. `integrate_term` takes as parameters:
1. The kernel function. In this case, the integral of $\frac{\partial G(x,y)}{\partial y}$ is also known as the "double layer" potential in classical potential theory because it represents the effect of two co-located sheets of oppositely charged particles. Beyond the mathematical correspondence, this also maps nicely in a conceptual sense to our setting: the double layer potential represents the effect of a jump in displacement across a surface (the fault!).
2. The observation points. Since we want to calculate the displacement at the free surface, we pass the nodes of the free surface mesh as the observation points: `free.pts`. Note that because these points are on the boundary of the domain, the integrals are computed as limits to the boundary $\hat{x} \to x$.
3. A flexible number of source surfaces. The numerical integration is much easier if we can handle all the relevant surfaces at once in the form of the original governing equation before we had separated the free surface and fault. `integrate_term` will automatically separate the subcomponents of the resulting matrix
4. The direction for any limits to the boundary. In this case, we want to approach the boundary from the interior, so we pass `limit_direction=1.0`. If we had wanted to approach the boundary from the exterior, we could pass `limit_direction=-1.0`.
5. Any singularities. See below for more details.
6. Whether to return a report on the internal details of the integration process. See the later section of debugging integration for details on the values that are returned as part of the report.

`integrate_term` will return a list of matrices. Each matrix will have the shape:
```
(number of observation points, observation kernel dimension, number of source coefficients, source kernel dimension)
```

```{margin}
In comparison, the hypersingular integral has an observation kernel dimension of 2 and source kernel dimension of 1. And the double-layer-equivalent T* kernel of plane strain elasticity has observation and source kernel dimensions of 2 because it comes from a vector PDE.
```

In the case of the double layer potential, both kernel dimensions are 1, so you might notice a few places where we index the matrices as `mat[:,0,:,0]`. This is simply to get rid of those extra array dimensions.

#### Details on singularities

The model has stress singularities. This is generally a sign that you should reconsider your modeling choices, but in this case it's okay because we're trying to match an existing analytical solution. Computing integrals near singularities in the solution is quite difficult. However, `integrate_term` will do its best to handle those singularities. For optimal accuracy, you can specify the location of the singularities. Preferably, we will also specify the location of any "possible singularities" as well.

**What is a possible-singularities?** Any given point will only be a singularity for some choices of slip field.  For example, the fault tip will produce a stress singularity for any slip distribution that is non-zero at the tip. But, if the slip field happens to be exactly zero at the fault tip, then there is no singularity. This is uncertainty is irrelevant to the integration algorithm because the algorithm is producing a matrix that can be used for arbitrary slip distributions. So, to handle the general case, it's important to assume that any "possible singularity" will be an actual singularity.

In this model, there is a possible-singularity that turns out not to be a singularity: the fault-surface intersection. Here, either the step function in the surface displacement or the nonzero fault tip slip would cause a stress singularity on its own. But, when combined the singularities produced by these two features exactly cancel, leaving behind an entirely non-singular displacement field.

```python
%%time
# Specify singularities
singularities = np.array([
    [-surf_half_L,0],
    [surf_half_L,0],
    [0,0],
    [0,-1],
])

# Integrate!
(surf_disp_to_surf_disp, fault_slip_to_surf_disp), reports = integrate_term(
    double_layer,
    free.pts,
    free,
    fault,
    limit_direction=1.0,
    singularities=singularities,
    return_report=True
)

# Specify a constant unit slip along the fault.
slip = np.ones(fault.n_pts)

# rhs = B \Delta u
rhs = -fault_slip_to_surf_disp[:,0,:,0].dot(slip)

# lhs = I + A
lhs = np.eye(free.n_pts) + surf_disp_to_surf_disp[:,0,:,0]

surf_disp = np.linalg.solve(lhs, rhs)
```

    CPU times: user 461 ms, sys: 85.7 ms, total: 547 ms
    Wall time: 924 ms

Not too much code! Now that we've solved for surface displacement, let's check against the classic arctan screw dislocation solution! That solution {cite:p}`segallEarthquakeVolcanoDeformation2010` is:

\begin{equation}
u_z = \frac{-s}{2\pi}\bigg[\tan^{-1}(\frac{x}{y + d_1}) - \tan^{-1}(\frac{x}{y - d_1}) - \tan^{-1}(\frac{x}{y + d_2}) + \tan^{-1}(\frac{x}{y - d_2}) \bigg]
\end{equation}

I'll make two plots. One will show the nearfield displacement and accuracy. The other will show the (very) farfield displacement and accuracy. We're getting 14 digits of accuracy in the nearfield, almost machine precision and 10-11 digits in the farfield. This is from solving the integral equation with ~200 eighth order panels and in less than half a second of runtime!

```python
analytical = -np.arctan(-1 / free.pts[:,0]) / np.pi
%matplotlib inline
for XV in [4, 1000]:
    plt.figure(figsize=(12, 6))
    plt.suptitle(f"{XV} fault lengths")
    plt.subplot(1, 2, 1)
    plt.plot(free.pts[:, 0], surf_disp, "ro")
    plt.plot(free.pts[:, 0], analytical, "bo")
    plt.xlabel("$x$")
    plt.ylabel("$u_z$")
    plt.title("Displacement")
    plt.xlim([-XV, XV])
    plt.ylim([-0.6, 0.6])

    plt.subplot(1, 2, 2)
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        plt.plot(free.pts[:, 0], np.log10(np.abs(surf_disp - analytical)))
    plt.xlabel("$x$")
    plt.ylabel("$\log_{10}|u_{\textrm{BIE}} - u_{\textrm{analytic}}|$")
    plt.title("Error (number of digits of accuracy)")
    plt.tight_layout()
    plt.xlim([-XV, XV])
    plt.show()
```

## Interior evaluation of displacement and stress.

Now that we've solved for surface displacement, we have everything we need to evaluate displacement and stress in the interior of the domain. We'll start by producing a grid of observation points near the fault.

```python
nobs = 200
zoomx = [-1.5, 1.5]
zoomy = [-3, 0]
xs = np.linspace(*zoomx, nobs)
ys = np.linspace(*zoomy, nobs)
obs_pts = pts_grid(xs, ys)
obsx = obs_pts[:, 0]
obsy = obs_pts[:, 1]
```

Next, we'll build the relevant integral equation matrices. To calculate displacement, we'll return to the original integral equation and basically just directly integrate it using `integrate_term`.

\begin{equation}
u_z(x) = -\int_{H} \frac{\partial G(x, y)}{\partial y} u_z(y) dy -\int_{F} \frac{\partial G(x, y)}{\partial y} \Delta u_z(y) dy  ~~~~~~~~~~~~~ \forall x \in \Omega / \partial\Omega
\end{equation}

The `integrate_term` call will hopefully look very familiar. Except for the list of observation points, it's exactly the same as the call from solving for surface displacement. After constructing the matrices, we will multiply them by surface displacement and slip respectively to compute interior displacement.

```python
K = double_layer
(free_to_disp, fault_to_disp), report = integrate_term(
    double_layer,
    obs_pts,
    free,
    fault,
    singularities=singularities,
    return_report=True,
)
interior_disp = -free_to_disp[:, 0, :, 0].dot(surf_disp) - fault_to_disp[:, 0, :, 0].dot(slip)
```

Calculating stress requires computing the derivatives of the displacement. Numerical differentiation is notoriously error prone, so instead, we'll analytically differentiate the integral equation:

\begin{align}
\sigma_{1z}(x) = \mu\bigg[-\int_{H} \frac{\partial^2 G(x, y)}{\partial x_1 \partial \textbf{y}} u_z(y) dy -\int_{F} \frac{\partial^2 G(x, y)}{\partial x_1 \partial \textbf{y}} \Delta u_z(y) dy\bigg] \\
\sigma_{2z}(x) = \mu\bigg[-\int_{H} \frac{\partial^2 G(x, y)}{\partial x_2 \partial \textbf{y}} u_z(y) dy -\int_{F} \frac{\partial^2 G(x, y)}{\partial x_2 \partial \textbf{y}} \Delta u_z(y)\bigg] dy
\end{align}

The result is that we can perform a very similar integration but with the double layer kernel replaced with the $\frac{\partial^2 G(x, y)}{\partial x_k \partial \textbf{y}}$. The new kernel is called the "hypersingular" kernel because of the $\frac{1}{r^2}$ dominant behavior of the kernel. The `integrate_term` is the same as the displacement calculation but with the `hypersingular` kernel. The other difference is that the kernel dimension is 2 instead of 1 because there are two stres because there are two stress components.

```python
(free_to_stress, fault_to_stress), report = integrate_term(
    hypersingular,
    obs_pts,
    free,
    fault,
    singularities=singularities,
    return_report=True,
)
interior_stress = -free_to_stress[:,:,:,0].dot(surf_disp) - fault_to_stress[:,:,:,0].dot(slip)
interior_sxz = interior_stress[:, 0]
interior_syz = interior_stress[:, 1]
```

    /Users/tbent/Dropbox/active/eq/tectosaur2/tectosaur2/integrate.py:201: UserWarning: Some integrals failed to converge during adaptive integration. This an indication of a problem in either the integration or the problem formulation.
      warnings.warn(
    /Users/tbent/Dropbox/active/eq/tectosaur2/tectosaur2/integrate.py:208: UserWarning: Some expanded integrals reached maximum expansion order. These integrals may be inaccurate.
      warnings.warn(

Finally, we'll compare with the analytical solutions for both displacement and stress. We're achieving 9 digits of accuracy at a minimum with even better accuracy closer to the free surface and fault!

```python
analytical_disp = (
    1.0 / (2 * np.pi) * (np.arctan((obsy + 1) / obsx) - np.arctan((obsy - 1) / obsx))
)
with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    disp_err = np.log10(np.abs(interior_disp - analytical_disp))

rp = obsx ** 2 + (obsy + 1) ** 2
ri = obsx ** 2 + (obsy - 1) ** 2
analytical_sxz = -(1.0 / (2 * np.pi)) * (((obsy + 1) / rp) - ((obsy - 1) / ri))
analytical_syz = (1.0 / (2 * np.pi)) * ((obsx / rp) - (obsx / ri))
sxz_err = np.log10(np.abs(interior_sxz - analytical_sxz))
syz_err = np.log10(np.abs(interior_syz - analytical_syz))
plt.figure(figsize=(12, 8))

plots = [
    ("analytical_disp", "$u_z$"),
    ("analytical_sxz", "$\sigma_{xz}$"),
    ("analytical_syz", "$\sigma_{yz}$"),
    ("disp_err", r"$\log_{10}|u_{\textrm{BIE}} - u_{\textrm{analytic}}|$"),
    ("sxz_err", r"$\log_{10}|\sigma_{xz,\textrm{BIE}} - \sigma_{xz,\textrm{analytic}}|$"),
    ("syz_err", r"$\log_{10}|\sigma_{yz,\textrm{BIE}} - \sigma_{yz,\textrm{analytic}}|$")
]
for i, (k, title) in enumerate(plots):
    plt.subplot(2, 3, 1 + i)
    plt.title(title)
    v = locals()[k].reshape((nobs, nobs))
    v2d = v.reshape((nobs, nobs))
    if i < 3:
        levels = np.linspace(-0.5, 0.5, 11)
    else:
        levels = np.linspace(-14, -9, 11)
    cntf = plt.contourf(xs, ys, v2d, levels=levels, extend="both")
    plt.contour(
        xs,
        ys,
        v2d,
        colors="k",
        linestyles="-",
        linewidths=0.5,
        levels=levels,
        extend="both",
    )
    plt.colorbar(cntf)
    # plt.xlim([-0.01, 0.01])
    # plt.ylim([-0.02, 0.0])

plt.tight_layout()
plt.show()
```

## Tools for debugging the integration algorithm

In this final section, I'll show some of the details that come from the reports produced by the integration algorithm. The figures below are useful for debugging and understanding the integration algorithm. A quick summary of each figure:
- `p_all`: This shows the QBX expansion order used for each interior point. The dark blue indicates zero expansion order which means that QBX was not used for those points.
- `n_subsets`: The shows the number of adaptive integration steps that were taken in accurately integrating the QBX expansions. For example, if the integration interval `[-1, 1]` is split into three subsets: `[-1, 0], [0, 0.5], [0.5, 1]`, then `n_subsets = 3`.
- `direction`: The shows the direction that the expansion center was offset from the nearby source surface with respect to the nearest source surface normal vector. So, `direction = 1` indicates that the expansion center for that observation is offset in the direction of the normal and `direction = -1` indicates that the expansion center was offset in the direction opposite the normal vector.
- `on_surface`: This field is equal to 1 if the point is lying exactly on the nearest source surface.
- `closest_src_pts`: This field shows the `(x, y)` coordinates of the nearest source surface point.

The `p_all` and `n_subsets` fields are particularly useful for understanding what is happening when a integral is failing to be computed properly.

```python
def quick_plot(k):
    plt.title(k)
    v = globals()[k].reshape((nobs, nobs))
    levels = np.linspace(np.min(v), np.max(v), 11)
    cntf = plt.contourf(xs, ys, v, levels=levels, extend="both")
    plt.contour(
        xs,
        ys,
        v,
        colors="k",
        linestyles="-",
        linewidths=0.5,
        levels=levels,
        extend="both",
    )
    plt.colorbar(cntf)

n_subsets = np.zeros(obs_pts.shape[0])
n_subsets[report["use_qbx"]] = report["n_subsets"]

p_all = np.zeros(obs_pts.shape[0])
p_all[report["use_qbx"]] = report["p"]

direction = np.zeros(obs_pts.shape[0])
direction[report["use_qbx"]] = report["direction"]

on_surface = np.zeros(obs_pts.shape[0])
on_surface[report["use_qbx"]] = report["on_surface"]

closest_src_pts = np.zeros((obs_pts.shape[0], 2))
closest_src_pts[report["use_qbx"]] = report["closest_src_pts"]
closest_src_ptsx = closest_src_pts[:, 0]
closest_src_ptsy = closest_src_pts[:, 1]

plt.rcParams['text.usetex'] = False
plt.figure(figsize=(12, 8))
for i, k in enumerate(
    [
        "p_all",
        "n_subsets",
        "direction",
        "on_surface",
        "closest_src_ptsx",
        "closest_src_ptsy",
    ]
):
    plt.subplot(2, 3, 1 + i)
    quick_plot(k)
    plt.xlim([-0.25, 0.25])
    plt.ylim([-0.5, 0])
plt.tight_layout()
plt.show()
plt.rcParams['text.usetex'] = True
```

These figures are showing the expansion centers and radii at the fault-surface intersection and at the fault tip.
- The black dots showing the expansion center and the black circles show the radius of accuracy for those expansion centers.
- The blue dots show the observation points where we are evaluating stress. These always lie on one of the expansion circles. This is intentional because it allows the expansion center to be maximally far from any source surface.
- The red lines and dots show the source surfaces: the free surface and the fault.

These figures can be useful for identifying expansion centers/radii that are incorrect. The two criterion for expansion circles are that:
1. They do not intersect the source surfaces at more than the one point where they touch but do not cross. This criterion is not strictly required and a small deviation is okay but seeing an expansion center intersecting a source surface at more than just a touching point is an indication of potential problems.
2. The circle must be 3 times it's radius away from any singular point specified in the `singularities` list above. In these figures, you can see that the expansion radii get quite small near the fault tip and the fault surface intersection.

```python
cs = report["exp_centers"]
rs = report["exp_rs"]
plt.figure(figsize=(8, 4))
for i, V in enumerate([(0, -0.025), (0, -1)]):
    plt.subplot(1, 2, 1 + i)
    plt.title("Fault-surface intersection" if i == 0 else "Fault tip")
    W = 0.03
    xrange = [V[0] - W, V[0] + W]
    yrange = [V[1] - W, V[1] + W]
    plt.plot(fault.pts[:, 0], fault.pts[:, 1], "r-o")
    plt.plot(free.pts[:, 0], free.pts[:, 1], "r-o")
    plt.plot(obs_pts[:, 0], obs_pts[:, 1], "b.", markersize=10)
    plt.plot(cs[:, 0], cs[:, 1], "k.", markersize=10)
    for i in range(cs.shape[0]):
        in_x_range = (
            cs[i, 0] - rs[i] < xrange[1] and cs[i, 0] - rs[i] > xrange[0]
        ) or (cs[i, 0] + rs[i] > xrange[0] and cs[i, 0] + rs[i] < xrange[1])
        in_y_range = (
            cs[i, 1] - rs[i] < yrange[1] and cs[i, 1] - rs[i] > yrange[0]
        ) or (cs[i, 1] + rs[i] > yrange[0] and cs[i, 1] + rs[i] < yrange[1])
        if in_x_range and in_y_range:
            plt.gca().add_patch(plt.Circle(cs[i], rs[i], color="k", fill=False))
    plt.axis("scaled")
    plt.xlim(xrange)
    plt.ylim(yrange)
plt.tight_layout()
plt.show()
```

It's also useful to examine cases where the expansion order is very high or the number of adaptive integration subsets is very high. These could indicate either integration failures or mistaken convergence conditions.

```python
print("p really high in", np.sum(report["p"] > 40), "cases")
print(
    "maximum number of subsets required by adaptive integration: ",
    report["n_subsets"].max(),
)
```

    p really high in 2 cases
    maximum number of subsets required by adaptive integration:  104


# Gaussian process regression with varying output noise

This notebook shows how to construct a Gaussian process model where different noise is assumed for different data points. The model is:

\begin{align}
f(\cdot) &\sim \mathcal{GP}\big(0, k(\cdot, \cdot)\big) \\
y_i | f, x_i &\sim \mathcal N\big(y_i; f(x_i), \sigma^2_i\big)
\end{align}

We'll demonstrate two methods. In the first demonstration, we'll assume that the noise variance is known for every data point. We'll incorporate the known noise variances $\sigma^2_i$ into the data matrix $\mathbf Y$, make a likelihood that can deal with this structure, and implement inference using variational GPs with natural gradients.

In the second demonstration, we'll assume that the noise variance is not known, but we'd like to estimate it for different groups of data. We'll show how to construct an appropriate likelihood for this task and set up inference similarly to the first demonstration, with optimization over the noise variances.

```python
import numpy as np
import tensorflow as tf
import gpflow
from gpflow.ci_utils import ci_niter
from gpflow.optimizers import NaturalGradient
from gpflow import set_trainable
import matplotlib.pyplot as plt

%matplotlib inline
```

## Demo 1: known noise variances
### Generate synthetic data
We create a utility function to generate synthetic data, including noise that varies amongst the data:

```python
np.random.seed(1)  # for reproducibility

def generate_data(N=80):
    X = np.random.rand(N)[:, None] * 10 - 5  # Inputs, shape N x 1
    F = 2.5 * np.sin(6 * X) + np.cos(3 * X)  # Mean function values
    NoiseVar = 2 * np.exp(-((X - 2) ** 2) / 4) + 0.3  # Noise variances
    Y = F + np.random.randn(N, 1) * np.sqrt(NoiseVar)  # Noisy data
    return X, Y, NoiseVar

X, Y, NoiseVar = generate_data()
```

Here's a plot of the data, with error bars representing two standard deviations:

```python
fig, ax = plt.subplots(1, 1, figsize=(12, 6))
_ = ax.errorbar(
    X.squeeze(),
    Y.squeeze(),
    yerr=2 * (np.sqrt(NoiseVar)).squeeze(),
    marker="x",
    lw=0,
    elinewidth=1.0,
    color="C1",
)
```

### Make a Y matrix that includes the variances
We need to tell the GP model what the variance is for each data point. To do this, we'll concatenate the observations with the variances into a single data matrix:

```python
Y_data = np.hstack([Y, NoiseVar])
```

### Make a new likelihood

To cope with this data structure, we'll build a new likelihood. Note how the code extracts the observations `Y` and the variances `NoiseVar` from the data. For more information on creating new likelihoods, see [Likelihood design](../tailor/likelihood_design.ipynb). Here, we're implementing the `log_prob` function (which computes the log-probability of the data given the latent function) and `variational_expectations`, which computes the expected log-probability under a Gaussian distribution on the function, and is needed in the evaluation of the evidence lower bound (ELBO). Check out the docstring for the `Likelihood` object for more information on what these functions do.

```python
class HeteroskedasticGaussian(gpflow.likelihoods.Likelihood):
    def __init__(self, **kwargs):
        # this likelihood expects a single latent function F, and two columns in the data matrix Y:
        super().__init__(latent_dim=1, observation_dim=2, **kwargs)

    def _log_prob(self, F, Y):
        # log_prob is used by the quadrature fallback of variational_expectations and predict_log_density.
        # Because variational_expectations is implemented analytically below, this is not actually needed,
        # but is included for pedagogical purposes.
        # Note that currently relying on the quadrature would fail due to https://github.com/GPflow/GPflow/issues/966
        Y, NoiseVar = Y[:, 0], Y[:, 1]
        return gpflow.logdensities.gaussian(Y, F, NoiseVar)

    def _variational_expectations(self, Fmu, Fvar, Y):
        Y, NoiseVar = Y[:, 0], Y[:, 1]
        return (
            -0.5 * np.log(2 * np.pi)
            - 0.5 * tf.math.log(NoiseVar)
            - 0.5 * (tf.math.square(Y - Fmu) + Fvar) / NoiseVar
        )

    # The following two methods are abstract in the base class.
    # They need to be implemented even if not used.

    def _predict_log_density(self, Fmu, Fvar, Y):
        raise NotImplementedError

    def _predict_mean_and_var(self, Fmu, Fvar):
        raise NotImplementedError
```

### Put it together with Variational Gaussian Process (VGP)
Here we'll build a variational GP model with the previous likelihood on the dataset that we generated. We'll use the natural gradient optimizer (see [Natural gradients](natural_gradients.ipynb) for more information).

The variational GP object is capable of variational inference with any GPflow-derived likelihood. Usually, the inference is an inexact (but pretty good) approximation, but in the special case considered here, where the noise is Gaussian, it will achieve exact inference. Optimizing over the variational parameters is easy using the natural gradients method, which provably converges in a single step.

Note: We mark the variational parameters as not trainable so that they are not included in the `model.trainable_variables` when we optimize using the Adam optimizer. We train the variational parameters separately using the natural gradient method.

```python
# model construction
likelihood = HeteroskedasticGaussian()
kernel = gpflow.kernels.Matern52(lengthscales=0.5)
model = gpflow.models.VGP((X, Y_data), kernel=kernel, likelihood=likelihood, num_latent_gps=1)
```

Notice that we specify num_latent_gps=1, as the VGP model would normally infer this from the shape of Y_data, but we handle the second column manually in the HeteroskedasticGaussian likelihood.

```python
natgrad = NaturalGradient(gamma=1.0)
adam = tf.optimizers.Adam()

set_trainable(model.q_mu, False)
set_trainable(model.q_sqrt, False)

for _ in range(ci_niter(1000)):
    natgrad.minimize(model.training_loss, [(model.q_mu, model.q_sqrt)])
    adam.minimize(model.training_loss, model.trainable_variables)
```

```python
# let's do some plotting!
xx = np.linspace(-5, 5, 200)[:, None]

mu, var = model.predict_f(xx)

plt.figure(figsize=(12, 6))
plt.plot(xx, mu, "C0")
plt.plot(xx, mu + 2 * np.sqrt(var), "C0", lw=0.5)
plt.plot(xx, mu - 2 * np.sqrt(var), "C0", lw=0.5)

plt.errorbar(
    X.squeeze(),
    Y.squeeze(),
    yerr=2 * (np.sqrt(NoiseVar)).squeeze(),
    marker="x",
    lw=0,
    elinewidth=1.0,
    color="C1",
)
_ = plt.xlim(-5, 5)
```

### Questions for the reader
1) What is the difference in meaning between the orange vertical bars and the blue regions in the prediction?

2) Why did we not implement `conditional_mean` and `conditional_var` in the HeteroskedasticGaussian likelihood? What could be done here?

2) What are some better kernel settings for this dataset? How could they be estimated?

## Demo 2: grouped noise variances

In this demo, we won't assume that the noise variances are known, but we will assume that they're known in two groups. This example represents a case where we might know that an instrument has varying fidelity for different regions, but we do not know what those fidelities are.

Of course it would be straightforward to add more groups, or even one group per data point. We'll stick with two for simplicity.

```python
np.random.seed(1)  # for reproducibility and to make it independent from demo 1
```

### Generate data

```python
def generate_data(N=100):
    X = np.random.rand(N)[:, None] * 10 - 5  # Inputs, shape N x 1
    F = 2.5 * np.sin(6 * X) + np.cos(3 * X)  # Mean function values
    groups = np.where(X > 0, 0, 1)
    NoiseVar = np.array([0.02, 0.5])[groups]  # Different variances for the two groups
    Y = F + np.random.randn(N, 1) * np.sqrt(NoiseVar)  # Noisy data
    return X, Y, groups

X, Y, groups = generate_data()
```

```python
# here's a plot of the raw data.
fig, ax = plt.subplots(1, 1, figsize=(12, 6))
_ = ax.plot(X, Y, "kx")
```

### Data structure

In this case, we need to let the model know which group each data point belongs to. We'll use a similar trick to the above, stacking the group identifier with the data:

```python
Y_data = np.hstack([Y, groups])
```

### Build a likelihood

This time, we'll use a builtin likelihood, `SwitchedLikelihood`, which is a container for other likelihoods, and applies them to the first `Y_data` column depending on the index in the second. We're able to access and optimize the parameters of those likelihoods. Here, we'll (incorrectly) initialize the variances of our likelihoods to 1, to demonstrate how we can recover reasonable values for these through maximum-likelihood estimation.

```python
likelihood = gpflow.likelihoods.SwitchedLikelihood(
    [gpflow.likelihoods.Gaussian(variance=1.0), gpflow.likelihoods.Gaussian(variance=1.0)]
)
```

```python
# model construction (notice that num_latent_gps is 1)
kernel = gpflow.kernels.Matern52(lengthscales=0.5)
model = gpflow.models.VGP((X, Y_data), kernel=kernel, likelihood=likelihood, num_latent_gps=1)
```

```python
for _ in range(ci_niter(1000)):
    natgrad.minimize(model.training_loss, [(model.q_mu, model.q_sqrt)])
```

We've now fitted the VGP model to the data, but without optimizing over the hyperparameters. Plotting the data, we see that the fit is not terrible, but hasn't made use of our knowledge of the varying noise.

```python
# let's do some plotting!
xx = np.linspace(-5, 5, 200)[:, None]

mu, var = model.predict_f(xx)

fig, ax = plt.subplots(1, 1, figsize=(12, 6))
ax.plot(xx, mu, "C0")
ax.plot(xx, mu + 2 * np.sqrt(var), "C0", lw=0.5)
ax.plot(xx, mu - 2 * np.sqrt(var), "C0", lw=0.5)

ax.plot(X, Y, "C1x", mew=2)
_ = ax.set_xlim(-5, 5)
```

### Optimizing the noise variances
Here we'll optimize over both the noise variance and the variational parameters, applying natural gradients interleaved with the Adam optimizer.

As before, we mark the variational parameters as not trainable, in order to train them separately with the natural gradient method.

```python
likelihood = gpflow.likelihoods.SwitchedLikelihood(
    [gpflow.likelihoods.Gaussian(variance=1.0), gpflow.likelihoods.Gaussian(variance=1.0)]
)
kernel = gpflow.kernels.Matern52(lengthscales=0.5)
model = gpflow.models.VGP((X, Y_data), kernel=kernel, likelihood=likelihood, num_latent_gps=1)

set_trainable(model.q_mu, False)
set_trainable(model.q_sqrt, False)

for _ in range(ci_niter(1000)):
    natgrad.minimize(model.training_loss, [(model.q_mu, model.q_sqrt)])
    adam.minimize(model.training_loss, model.trainable_variables)
```

### Plotting the fitted model

Now that the noise variances have been estimated, we can see the final model fit.
The predictive variance is higher on the left side of the plot, where we know that the data have different variance.
We'll plot the known underlying function in green to see how effectively we've recovered the ground truth.
We can also print the model to examine the estimated noise variances:

```python
# let's do some plotting!
xx = np.linspace(-5, 5, 200)[:, None]

mu, var = model.predict_f(xx)

fig, ax = plt.subplots(1, 1, figsize=(12, 6))
ax.plot(xx, mu, "C0")
ax.plot(xx, mu + 2 * np.sqrt(var), "C0", lw=0.5)
ax.plot(xx, mu - 2 * np.sqrt(var), "C0", lw=0.5)

ax.plot(X, Y, "C1x", mew=2)
ax.set_xlim(-5, 5)
_ = ax.plot(xx, 2.5 * np.sin(6 * xx) + np.cos(3 * xx), "C2--")
```

```python
from gpflow.utilities import print_summary

print_summary(model, fmt="notebook")
```

name                                  	class    	transform       	prior  	trainable  	shape        	dtype  	value

VGP.kernel.variance                   	Parameter	Softplus        	       	True       	()           	float64	2.0487746335100274

VGP.kernel.lengthscales               	Parameter	Softplus        	       	True       	()           	float64	0.21743510667548904

VGP.likelihood.likelihoods[0].variance	Parameter	Softplus + Shift	       	True       	()           	float64	0.2914816857759117

VGP.likelihood.likelihoods[1].variance	Parameter	Softplus + Shift	       	True       	()           	float64	0.6055446990450579

VGP.q_mu                              	Parameter	                	       	False      	(100, 1)     	float64	[[1.15102310e+00...

VGP.q_sqrt                            	Parameter	FillTriangular  	       	False      	(1, 100, 100)	float64	[[[2.06898404e-01, 0.00000000e+00, 0.00000000e+00...


Markdown macros

$\newcommand{\I}{\mathrm{i}}
\newcommand{\E}{\mathrm{e}}
\newcommand{\D}{\mathop{}\!\mathrm{d}}
\newcommand{\bra}[1]{\langle{#1}|}
\newcommand{\ket}[1]{|{#1}\rangle}
\newcommand{\braket}[1]{\langle{#1}\rangle}
\newcommand{\bm}[1]{\boldsymbol{#1}}$

# Imports

```python
# numpy
import numpy as np
```

```python
# matplotlib
import matplotlib
import matplotlib.pylab as plt
```

```python
%matplotlib inline
from IPython.display import Image
```

```python
# sympy
import sympy as sy
sy.init_printing(use_unicode = True)
```

```python
# figure style
# ATTENTION: commentez l'option 'usetex' si elle ne marche pas;
#            vous avez besoin de LaTeX pour l'utiliser
matplotlib.rcParams['text.usetex'] = True
#
matplotlib.rcParams['font.family'] = 'serif'
matplotlib.rcParams['font.weight'] = 'normal'
matplotlib.rcParams['font.size'] = 14
matplotlib.rcParams['axes.labelsize'] = 18
matplotlib.rcParams['figure.figsize'] = (6.0,4.5)
matplotlib.rcParams['lines.linewidth'] = 0.5
matplotlib.rcParams['xtick.top'] = True
matplotlib.rcParams['ytick.right'] = True
```

# Amorce de glissement: décrochage d'une échelle

On étudie la chute d'une échelle; on calcule la condition de glissement au point de rotation.

L'échelle est de masse $m$ et longueur $l$; le moment d'inertie (par rapport à sa base) est
$$I = \frac{ml^2}{3}$$.
Le centre de masse est au milieu $l/2$

```python
Image(filename='decrochage.png', width=250)
```

On a un repère $(X,Y,Z)$ au point $O$. L'échelle tourne autour de $O$ d'un angle $\theta(t)$, dépendent du temps.

Le repère mobile $(x,y,z)$ attaché à l'échelle, au centre de masse $G$ (sur lequel est appliqué le poids $mg$), est relié au repère fixe par un rotation:
$$\begin{pmatrix}\hat{\bm x} \\ \hat{\bm y} \\ \hat{\bm z} \end{pmatrix} = C_z(\theta)\begin{pmatrix}\hat{\bm X} \\ \hat{\bm Y} \\ \hat{\bm Z} \end{pmatrix}$$

Le coordonées du centre de masse sont
$$\bm G = \frac{l}{2}\big(\cos\theta \hat{\bm X} + \sin\theta \hat{\bm Y}\big) = \frac{l}{2} \hat{\bm x}$$
ce qui donne la vitesse
$$\dot{\bm G} = \frac{l \dot\theta}{2}\big(-\sin\theta \hat{\bm X} + \cos\theta \hat{\bm Y}\big) = \frac{l \dot\theta}{2} \hat{\bm y}$$
est l'accelération
\begin{align*}
\ddot{\bm G} &= -\frac{l}{2}\big(\ddot\theta \sin\theta + \dot\theta^2 \cos\theta \big) \hat{\bm X} + \frac{l}{2}\big(\ddot\theta \cos\theta - \dot\theta^2 \sin\theta \big)\hat{\bm Y} \\
&= \frac{l \ddot\theta}{2} \hat{\bm y}-\frac{l \dot\theta^2}{2} \hat{\bm x}
\end{align*}

### Moment cinétique

La vitesse angulaire du centre de masse, dans le repère mobile est
$$\bm \omega = \dot{\theta} \hat{\bm z} = \dot{\theta} \hat{\bm Z}$$
(on note que la rotation ne change pas l'axe $Z$).

Le moment cinétique et le moment sont:
$$\bm L_G = I_G \omega I\dot{\theta} \hat{\bm Z}, \quad \frac{\D \bm L_G}{\D t} = I_G\ddot{\theta} \hat{\bm Z}$$
toujours au centre de masse.

Le moment $\dot{\bm L}_G$, satisfait:
$$\dot{\bm L}_G = I_G\dot{\bm \omega} + \bm \omega \times I_G \bm \omega$$
où, pour calculer $\dot{I}_G$ on a utilisé le passage par le repère fixe $I = R I_0 R^T$.

### Équations de mouvement

Le forces sont:

* le poids $-mg \hat{\bm X}$
* la force normale en $O$: $\bm N = N \hat{\bm X}$
* et la force de frottement statique: $\bm f = f \hat{\bm Y}$

L'équation de Newton est:
$$ m \ddot G = (N-mg) \hat{\bm X} + f \hat{\bm Y}$$
ou, en remplaçant l'accelération du centre de masse par son expression, on obtient
$$N = mg - \frac{ml}{2}\big(\ddot\theta \sin\theta + \dot\theta^2 \cos\theta \big)$$
et
$$f =  \frac{ml}{2}\big(\ddot\theta \cos\theta - \dot\theta^2 \sin\theta \big)$$

Le terme de rotation de l'équation d'Euler s'annule $\bm \omega\times I_G\bm\omega=0$, d'où:
$$I_G \dot{\bm \omega} = \bm G \times \big(-mg \hat{\bm X}\big)$$
pusique seule le poids exerce un moment par rapport à $O$:
$$\ddot{\theta} = \frac{3g}{2l} \sin\theta$$

## Frottement

Au point de contact de deux solides on a une force de réaction:
$$\bm R = \bm T + \bm N$$
qu'on décompose en partie tangentielle et normale à la surface de contact.

Frottement dynamiue: la force s'oppose au mouvement tangentiel,
$$\bm T = - \mu_d N \hat{\bm v}_t$$
où $\hat{\bm v}_t$ est la direction du mouvement au point de contact, et $N$ est la grandeur de la force normale en ce point.

Frottement statique ($\bm V_t = 0$):
$$T = \mu_s N$$

Le frottement dynamique est plus petit que le frottement statique: $\mu_d < \mu_s$.

En utilisant les formules:

* $\ddot\theta = (3g/2l)\sin\theta$
* et $\dot\theta = (3g/l)(1 - \cos\theta$ (n suppose qu'on part du sommet à vitesse nulle)

pour exprimer les forces en fonction de l'angle, on obtient:
$$N = \frac{mg}{4}(3\cos\theta - 1)^2$$
et
$$f = \frac{3mg}{4} \sin\theta(3\cos\theta - 2)$$

### Condition de glissement

Le décrochage se produit au moment ou les forces $N$ et $f$ atteignent la limite du frottement statique $f = \mu N$:
$$\frac{3mg}{4} \sin\theta(3\cos\theta - 2) = \mu \frac{mg}{4}(3\cos\theta - 1)^2$$
ou
$$\sin\theta(3\cos\theta - 2)=\frac{\mu}{3}(3\cos\theta - 1)^2$$

On cherche d'abord une solution graphique. On choisit une valeur du coefficient de frottement $\mu \sim 0.1$.

```python
# Graphe
mu = 0.1
t = np.linspace(0, np.pi/2, 129)
f = np.sin(t)*(3*np.cos(t) - 2)
N = (mu/3)*(3*np.cos(t) - 1)**2
```

```python
fig, ax = plt.subplots()
ax.plot(180*t/np.pi, f, color = '#800', label = r"$f$")
ax.plot(180*t/np.pi, N, color = '#008', label = r"$N$")
ax.set_xlabel('angle en degrés')
ax.set_ylim(-0.1,0.4)
ax.legend();
```

On calcule maintenant la valeur de $\theta$ numériquement

```python
import scipy
from scipy.optimize import root_scalar
```

```python
def fN(t):
    mu = 0.1
    f = np.sin(t)*(3*np.cos(t) - 2)
    N = (mu/3)*(3*np.cos(t) - 1)**2
    return f-N
```

```python
sol = root_scalar(fN, x0 = 0.5, x1 = 1.3)
180*sol.root/np.pi
```

On trouve finalement que pour $\mu = 0.1$ l'angle de décrochage est de $\theta \approx 47^\circ$.

# Brouillon

```python
# font
# matplotlib.rcParams['font.serif'] = "STIX Two Text"
# matplotlib.rcParams['mathtext.fontset'] = 'stix'
```

```python
fig, ax = plt.subplots()
ax.plot(t, f, color = '#800', label = r"$f$")
ax.plot(t, N, color = '#008', label = r"$N$")
ax.set_ylim(-0.1,0.4)
ax.legend();
```

```python

```


# Black Scholes 模型

Black-Scholes模型是Fischer Black和Myron Scholes在1973的论文"The Pricing of Options and Corporate Liabilities"中提出的。自论文出版以来，这个模型成为了投资者广泛使用的工具。时至今日，它依然是确定期权公允价格的最好的方式之一。

## 假设

* 欧式期权之内在到期日行权
* 在期权的生命周期内，没有股息分派
* 市场走势不能被预测
* 无风险利率和波动率为常数
* 标的物的走势服从对数正态分布

## 无股息 Black-Scholes 公式

在Black-Schole公式中, 定义如下参数.

* $S$, 在时间$t$时的资产价格
* $T$, 期权到期时间. 期权的剩余时间定义为$T−t$
* $K$, 期权行权价
* $r$, 无风险收益率，假设在其在时间段$t$和$T$中为常量
* $\sigma$, 标的资产的波动率, 即标的资产回报的标准差

### $N(d)$是服从正态分布的随机变量$Z$的积累分布函数

$$N(d) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^d e^{-\frac{1}{2}x^2} dx$$

$C(S,t)$是在时间$t$的看涨期权的价值；$P(S,t)$是在时间$t$的看跌期权的价值。

Black-Scholes看涨期权公式是:

$$C(S,t) = SN(d_1) - Ke^{-r(T - t)} N(d_2)$$

看跌期权公式是：

$$P(S,t) = Ke^{-r(T - t)}N(-d_2) - SN(-d_1)$$

其中

$$d_1 = \frac{\ln \left(\frac{S}{K} \right) + \left(r + \frac{\sigma^2}{2} \right)(T - t)}{\sigma \sqrt{T - t}}$$

$$d_2 = d_1 - \sigma \sqrt{T - t} = \frac{\ln \left(\frac{S}{K} \right) + \left(r - \frac{\sigma^2}{2}\right)(T - t)}{\sigma \sqrt{T - t}}$$

## 无股息Black-Scholes公式的Python实现

```python
import numpy as np
import scipy.stats as si
import sympy as sy
import sympy.stats as systats
sy.init_printing()
```

```python
def euro_call(S, K, T, r, sigma):

    #S: spot price
    #K: strike price
    #T: time to maturity
    #r: interest rate
    #sigma: volatility of underlying asset

    N = systats.Normal("x", 0.0, 1.0)

    d1 = (sy.ln(S / K) + (r + 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))
    d2 = (sy.ln(S / K) + (r - 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))

    call = (S * systats.cdf(N)(d1) - K * sy.exp(-r * T) * systats.cdf(N)(d2))

    return call
```

```python
def euro_put(S, K, T, r, sigma):

    #S: spot price
    #K: strike price
    #T: time to maturity
    #r: interest rate
    #sigma: volatility of underlying asset

    N = systats.Normal("x", 0.0, 1.0)

    d1 = (sy.ln(S / K) + (r + 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))
    d2 = (sy.ln(S / K) + (r - 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))

    put = (K * sy.exp(-r * T) * systats.cdf(N)(-d2) - S * systats.cdf(N)(-d1))

    return put
```

测试一下

```python
euro_call(50, 100, 1, 0.05, 0.25)
```

```python
euro_call(50, 100, 1, 0.05, 0.25).evalf()
```

### 符号计算扩展

```python
S, K, T, t, r, sigma = sy.symbols("S K T t r sigma")
call_price = euro_call(S, K, T-t, r, sigma)
```

#### Delta
$$\Delta = \frac{\partial V}{\partial S}$$

```python
delta = sy.diff(call_price, S)
```

#### Gamma
$$\Gamma = \frac{\partial^2 V}{\partial S^2}$$

```python
gamma = sy.diff(call_price, S, 2)
```

#### Theta
$$\Theta = \frac{\partial V}{\partial t}$$

```python
theta = sy.diff(call_price, t)
```

## 图像

```python
import matplotlib
import matplotlib.pyplot as plt
```

```python
# Data for plotting
s = np.arange(40.0, 200.0, 1.0)
c = []
for s_i in s:
    c.append(call_price.evalf(subs={
        S: s_i,
        K: 100,
        T: 1,
        t: 0.5,
        r: 0.05,
        sigma: 0.25},n=2))

fig, ax = plt.subplots()
ax.plot(s, c)

ax.set(xlabel='S', ylabel='C',
       title='Call Price Value of S')
ax.grid()

plt.show()
```

```python
## Delta Plot
s = np.arange(40.0, 200.0, 1.0)
c = []
for s_i in s:
    c.append(delta.evalf(subs={
        S: s_i,
        K: 100,
        T: 1,
        t: 0.5,
        r: 0.05,
        sigma: 0.25},n=2))

fig, ax = plt.subplots()
ax.plot(s, c)

ax.set(xlabel='S', ylabel='Delta',
       title='Call Delta Value of S')
ax.grid()

plt.show()
```

```python
## Gamma Plot
s = np.arange(40.0, 200.0, 1.0)
c = []
for s_i in s:
    c.append(gamma.evalf(subs={
        S: s_i,
        K: 100,
        T: 1,
        t: 0.5,
        r: 0.05,
        sigma: 0.25},n=2))

fig, ax = plt.subplots()
ax.plot(s, c)

ax.set(xlabel='S', ylabel='Gamma',
       title='Call Gamma Value of S')
ax.grid()

plt.show()
```

## 内容主要引用自
[Black-Scholes Formula and Python Implementation](https://aaronschlegel.me/black-scholes-formula-python.html)


# Plot a Function, Save the plot to a file
This notebook shows how to plot an elabortae function, then send to output to a file.  The example is adapted from a server-side implementation of the same problem:

    In the server-side implementation the following steps are used
    1) Input interface is an html web page, with input cells (built into html)
    2) Action button, uses POST method to send request to the python script housed in cgi-bin directory linked to the webroot.
    3) Parse the form , structure the inputs -- in the code below, the interactive inputs `` variable = float(input{"message")``replace the parser script.
    4) Run the calculations - prepare the plot
    5) Send the plot to a file
    6) Create output html, use the plot as an image source.
    7) Done!

Here we will only do step 3 (modified) and step 4; there is substantial added work to do the web interface -- that's shown at the end of this notebook.

For our notebook, we will need ``math`` and ``matplotlib``. There are someother imports from the cgi-bin script, that are commented out.

```python

```

```python
# This is script is a test for SERVER-SIDE computation
# Values to enter via a web interface and leave in a graphic
# T Cleveland 2016-0216
#
# Import mathlibrary to get special functions
#
from math import sqrt,erf,erfc,exp  # get special math functions
#
# Import graphics routines for picture making
#
from matplotlib import pyplot as plt
#
# Import system calls to nicely exit the program -- may need to change for server-side processing
#
#import sys ## used in the html/cgi-bin, not needed here
```

```python

```

```python

```

## Forward Define the Functions
This step is important, the functions must be defined before they are called -- in an interpreter, this is usually done at the top of the script.  Other scripting languages store the scripts at the end (JavaScript usually keeps scripts at end of the file -- it internally promotes then to the top before it runs its JIT bytecode compiler.

In a compiled language, this is not as necessary (predefinition is, location not so much).

These prototype functions are usually written so that they are organic with respect to their variables, so there is no leakage -- in these two functions, the input list is just names, and the output is just a value that must be assigned in the calling script.

Our first function evaluates:

\begin{equation}
C(x,t) = \frac{a}{b}
\end{equation}

```python
# contaminant function
def concentration(c_initial,c_source,space,time,t_pulse,dispersion,velocity,retardation,mu,lamda):
# PARAMETERS (ARGUMENTS)
# c_initial == concentration everywhere at time = 0, excpet at source
# c_source == concentration at source (x=0), for any time
# space == distance from source at x=0
# time  == elapsed time from t=0
# t_pulse == duration of finite source pulse
# dispersion == dispersion (diffusion) coefficient (length^2/time)
# velocity == species velocity (mean section velocity/porosity) (length/time)  Calculate before call function
# retardation == retardation coefficient (adjusted time) Calculate before call function
# mu == 1st order decay rate coefficient (time^-1)
# lamda == 0th order rate constant (concentration/time)
#################################################
## COMPUTE A                                   ##
#################################################
    termA1 = (1.0-0.5*erfc(((retardation*space)-(velocity*time))/(2.0*sqrt(dispersion*retardation*time))))
    termA2 = (-0.5*exp(velocity*space/dispersion)*erfc(((retardation*space)+(velocity*time))/(2.0*sqrt(dispersion*retardation*time))))
    termA3 = (exp(-mu*time/retardation))
    termA = termA1*(termA2+termA3)
#################################################
# error handler for negative (undefined) time ###
#################################################
    if(time < 0):
        print("negative time -- no solution")
        concentration = (-999.9)
        return (concentration)
################################################
# During the finite pulse                    ###
################################################
    if(time <= t_pulse):
        concentration = (lamda/mu)+(c_initial - lamda/mu)*termA + (c_source - lamda/mu)*termB(space,time,velocity,dispersion,retardation,mu)
        return(concentration)
    else:
        concentration = (lamda/mu)+(c_initial - lamda/mu)*termA + (c_source - lamda/mu)*termB(space,time,velocity,dispersion,retardation,mu) -c_source*termB(space,time-t_pulse,velocity,disersion,retardation,mu)
        return(concentration)
#################################################
# error handler for impossible
    print("no path to this message")
    concentration = -999.0
    return(concentration)
#################################################
```

Here is the other function, this uses a concept of adjusted time to account for a linear equilibrium isotherm for the underlying mass transport process.  Code wise nothing special, just another funcion

```python

```

```python
# term B function to handle time shift
def termB(space,time,velocity,dispersion,retardation,mu):
## COMPUTE B ##
## Compute adjusted velocity
    u = velocity* sqrt(1.0+(4.0*mu*dispersion)/(velocity**2)) # cm/day
#
    termB1 =(0.5)*exp(0.5*((velocity-u)*space)/(dispersion))
    termB2 = erfc(((retardation*space)-(u*time))/(2.0*sqrt(dispersion*retardation*time)))
    termB3 = termB1*termB2
#
    termB4 =(0.5)*exp(0.5*((velocity+u)*space)/(dispersion))
    termB5 = erfc(((retardation*space)+(u*time))/(2.0*sqrt(dispersion*retardation*time)))
    termB6 = termB4*termB5
#
    termB = termB3+termB6
    return(termB)

#################################################
```

```python
# lets test
c_initial = float(input("Initial Concentration 0"))
c_source = float(input("Source Concentration 1000"))
space = float(input("Space 1"))
time  = float(input("Time 10"))
t_pulse = float(input("Pulse Duration 100"))
dispersion = float(input("Dispersion Coefficient 1"))
velocity = float(input("Velocity .5"))
retardation = float(input("Retardation Factor 1.0"))
mu = 0.0000001
lamda = 0.0

#c_initial = 0.0
#c_source = 1000.0
#space = 1.
#time  = 10.
#t_pulse = 100.0
#dispersion = 1.0
#velocity = 0.5
#retardation = 1.0
#mu = 0.0000001
#lamda = 0.0

#
# forward define and initialize vectors for a profile plot
#
how_many_points = 50
x = range(0,how_many_points,1) # null list
c = [0.0 for i in range(how_many_points)] # null list
#
# initialize the vectors so can use ordinary arithmetic
#dist = [0.1,1.0,2.0,3.0,4.0,5.0]
#dist = range(0,50,1)
for i in range(0,how_many_points,1):
    c[i] = concentration(c_initial,c_source,x[i],time,t_pulse,dispersion,velocity,retardation,mu,lamda)
#    print (x[i],c[i])  # cool, now can build plot
#
# Building the Plot
#
plt.plot(x,c, color='red', marker='o', linestyle = 'solid')  # make the plot object
plt.title(" Concentration Profile ") # caption the plot object
plt.xlabel(" Distance from Source ") # label x-axis
plt.ylabel(" Concentration        ") # label y-axis
plt.savefig("1D-ADR.png")
plt.show() # plot to stdio -- has to be last call as it kills prior objects
plt.close('all') # needed when plt.show call not invoked
print("end of script")
#sys.exit() # used to elegant exit for CGI-BIN use
```

```python

```

```python

```


This notebook is part of the `nbsphinx` documentation: https://nbsphinx.readthedocs.io/.

# Markdown Cells

We can use *emphasis*, **boldface**, `preformatted text`.

> It looks like strike-out text is not supported: ~~strikethrough~~.

* Red
* Green
* Blue

***

1. One
1. Two
1. Three

Arbitrary Unicode characters should be supported, e.g. łßō.
Note, however, that this only works if your HTML browser and your LaTeX processor provide the appropriate fonts.

## Equations

Inline equations like $\text{e}^{i\pi} = -1$
can be created by putting a LaTeX expression between two dollar signs, like this:
`$\text{e}^{i\pi} = -1$`.

**Note:**

Avoid leading and trailing spaces around math expressions, otherwise errors like the following will occur when Sphinx is running:

    ERROR: Unknown interpreted text role "raw-latex".

See also the [pandoc docs](http://pandoc.org/MANUAL.html#math):

> Anything between two `$` characters will be treated as TeX math. The opening `$` must have a non-space character immediately to its right, while the closing `$` must have a non-space character immediately to its left, and must not be followed immediately by a digit.

Equations can also be displayed on their own line like this:
\begin{equation}
\int\limits_{-\infty}^\infty f(x) \delta(x - x_0) dx = f(x_0).
\end{equation}

This can be done by simply using one of the LaTeX math environments, like so:

```
\begin{equation}
\int\limits_{-\infty}^\infty f(x) \delta(x - x_0) dx = f(x_0)
\end{equation}
```

### Automatic Equation Numbering

This is not automatically enabled in Jupyter notebooks,
but you can install a notebook extension in order to enable equation numbering:
https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/equation-numbering/readme.html.

Automatic Equation Numbering is enabled on http://nbviewer.jupyter.org/,
see e.g. the latest version of this very notebook at the link http://nbviewer.jupyter.org/github/spatialaudio/nbsphinx/blob/master/doc/markdown-cells.ipynb.

When using `nbsphinx`, you can use the following `mathjax_config` setting in your `conf.py` file
to enable automatic equation numbering in HTML output.
In LaTeX output, the equations are numbered by default.

```python
mathjax_config = {
    'TeX': {'equationNumbers': {'autoNumber': 'AMS', 'useLabelIds': True}},
}
```

You can use `\label{...}` to give a unique label to an equation:

\begin{equation}
\phi = \frac{1 + \sqrt{5}}{2}
\label{golden-mean}
\end{equation}

```
\begin{equation}
\phi = \frac{1 + \sqrt{5}}{2}
\label{golden-mean}
\end{equation}
```

If automatic equation numbering is enabled,
you can later reference that equation using its label.
You can use `\eqref{golden-mean}` for a reference with parentheses: \eqref{golden-mean},
or `\ref{golden-mean}` for a reference without them: \ref{golden-mean}.

In HTML output, these equation references only work for equations within a single HTML page.
In LaTeX output, equations from other notebooks can be referenced, e.g. \eqref{fibonacci-recurrence}.

### Manual Equation Numbering

If you prefer to assign equation numbers (or some kind of names) manually,
you can do so with `\tag{...}`:

\begin{equation}
a^2 + b^2 = c^2
\tag{99.4}
\label{pythagoras}
\end{equation}

```
\begin{equation}
a^2 + b^2 = c^2
\tag{99.4}
\label{pythagoras}
\end{equation}
```

The above equation has the number \ref{pythagoras}.

## Citations

According to https://nbconvert.readthedocs.io/en/latest/latex_citations.html,
`nbconvert` supports citations using a special HTML-based syntax.
`nbsphinx` supports the same syntax.

Example: Kluyver et al. (2016).

```html
<cite data-cite="kluyver2016jupyter">Kluyver et al. (2016)</cite>
```

You don't actually have to use `<cite>`,
any inline HTML tag can be used, e.g. `<strong>`:
Python: An Ecosystem for Scientific Computing.

```html
<strong data-cite="perez2011python">Python: An Ecosystem for Scientific Computing</strong>
```

You'll also have to define a list of references,
see [the section about references](a-normal-rst-file.rst#references).

There is also a Notebook extension which may or may not be useful: https://github.com/takluyver/cite2c.

## Code

We can also write code with nice syntax highlighting:

```python3
print("Hello, world!")
```

## Tables

A     | B     | A and B
------|-------|--------
False | False | False
True  | False | False
False | True  | False
True  | True  | True

## Images

Local image:

Remote image:

### SVG support for LaTeX

LaTeX doesn't support SVG images, but there are Sphinx extensions that can be used for automatically converting SVG images for inclusion in LaTeX output.

Just include one of the following options in the list of `extensions`
in your `conf.py` file.

* `'sphinxcontrib.inkscapeconverter'` or `'sphinxcontrib.rsvgconverter'`:
  See https://github.com/missinglinkelectronics/sphinxcontrib-svg2pdfconverter
  for installation instructions.

  The external programs `inkscape` or `rsvg-convert` (Debian/Ubuntu package `librsvg2-bin`)
  are needed, respectively.

* `'sphinx.ext.imgconverter'`:
  This is a built-in Sphinx extension, see
  http://www.sphinx-doc.org/en/master/usage/extensions/imgconverter.html.

  This needs the external program `convert` from *ImageMagick*.

  The disadvantage of this extension is that SVGs are converted to bitmap images.

## Cell Attachments

Images can also be embedded in the notebook itself. Just drag an image file into the Markdown cell you are just editing or copy and paste some image data from an image editor/viewer.

The generated Markdown code will look just like a "normal" image link, except that it will have an `attachment:` prefix:

This is a cell attachment:

In the Jupyter Notebook, there is a speciall "Attachments" cell toolbar which you can use to see all attachments of a cell and delete them, if needed.

## HTML Elements (HTML only)

It is allowed to use plain HTML elements within Markdown cells.
Those elements are passed through to the HTML output and are ignored for the LaTeX output.
Below are a few examples.

HTML5 [audio](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio) elements can be created like this:

```html
<audio src="https://example.org/audio.ogg" controls>alternative text</audio>
```

Example:

<audio src="https://upload.wikimedia.org/wikipedia/commons/6/61/DescenteInfinie.ogg" controls>The HTML audio element is not supported!</audio>

HTML5 [video](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video) elements can be created like this:

```html

```

Example:

The alternative text is shown in browsers that don't support those elements. The same text is also shown in Sphinx's LaTeX output.

**Note:** You can also use local files for the `<audio>` and `<video>` elements, but you have to create a link to the source file somewhere, because only then are the local files copied to the HTML output directory!
You should do that anyway to make the audio/video file accessible to browsers that don't support the `<audio>` and `<video>` elements.

## Info/Warning Boxes

**Warning:**

This is an *experimental feature*!
Its usage will probably change in the future or it might be removed completely!

Until there is an info/warning extension for Markdown/CommonMark (see [this issue](https://github.com/jupyter/notebook/issues/1292)), such boxes can be created by using HTML `<div>` elements like this:

```html
<div class="alert alert-info">

**Note:** This is a note!

</div>
```

For this to work reliably, you should obey the following guidelines:

* The `class` attribute has to be either `"alert alert-info"` or `"alert alert-warning"`, other values will not be converted correctly.
* No further attributes are allowed.
* For compatibility with CommonMark, you should add an empty line between the `<div>` start tag and the beginning of the content.

**Note:**

The text can contain further Markdown formatting.
It is even possible to have nested boxes:

... but please don't *overuse* this!

## Links to Other Notebooks

Relative links to local notebooks can be used:
[a link to a notebook in a subdirectory](subdir/a-notebook-in-a-subdir.ipynb),
[a link to an orphan notebook](orphan.ipynb)
(latter won't work in LaTeX output, because orphan pages are not included there).

This is how a link is created in Markdown:

```
[a link to a notebook in a subdirectory](subdir/a-notebook-in-a-subdir.ipynb)
```

Markdown also supports *reference-style* links:
[a reference-style link][mylink],
[another version of the same link][mylink].

[mylink]: subdir/a-notebook-in-a-subdir.ipynb

These can be created with this syntax:

```
[a reference-style link][mylink]

[mylink]: subdir/a-notebook-in-a-subdir.ipynb
```

Links to sub-sections are also possible, e.g.
[this subsection](subdir/a-notebook-in-a-subdir.ipynb#A-Sub-Section).

This link was created with:

```
[this subsection](subdir/a-notebook-in-a-subdir.ipynb#A-Sub-Section)
```

You just have to remember to replace spaces with hyphens!

BTW, links to sections of the current notebook work, too, e.g.
[beginning of this section](#Links-to-Other-Notebooks).

This can be done, as expected, like this:

```
[beginning of this section](#Links-to-Other-Notebooks)
```

## Links to `*.rst` Files (and Other Sphinx Source Files)

Links to files whose extension is in the configuration value [source_suffix](http://www.sphinx-doc.org/en/master/config.html#confval-source_suffix), will be converted to links to the generated HTML/LaTeX pages. Example: [A reStructuredText file](a-normal-rst-file.rst).

This was created with:

```
[A reStructuredText file](a-normal-rst-file.rst)
```

Links to sub-sections are also possible. Example: [Sphinx Directives](a-normal-rst-file.rst#sphinx-directives-for-info-warning-boxes).

This was created with:

```
[Sphinx Directives](a-normal-rst-file.rst#sphinx-directives-for-info-warning-boxes)
```

**Note:**

Sphinx section anchors are different from Jupyter section anchors!
To create a link to a subsection in an `.rst` file (or another non-notebook source file), you not only have to replace spaces with hyphens, but also slashes and some other characters.
In case of doubt, just check the target HTML page generated by Sphinx.

## Links to Local Files

Links to local files (other than Jupyter notebooks and other Sphinx source files) are also possible, e.g. [requirements.txt](requirements.txt).

This was simply created with:

```
[requirements.txt](requirements.txt)
```

The linked files are automatically copied to the HTML output directory.
For LaTeX output, links are created,
but the files are not copied to the target directory.

## Links to Domain Objects

Links to [Sphinx domain objects](http://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html) (such as a Python class or JavaScript function) are also possible. For example:
[example_python_function()](a-normal-rst-file.rst#example_python_function).

This was created with:

```
[example_python_function()](a-normal-rst-file.rst#example_python_function)
```

This is especially useful for use with the Sphinx [autodoc](http://www.sphinx-doc.org/en/master/ext/autodoc.html) extension!


# Hopfield model project
Artificial Neural Networks (ANN) are computational techniques that aim to realize a very simplified model of the human brain. In this way, ANN try to learn tasks mimicking the behavior of brain. The brain is composed of a large set of elements, specialized cells called neurons. In analogy with the human brain, ANN are computational methods that use a large set of elementary computational: the neurons. Each single neuron is a very simple entity but the power of the brain is given by the fact that neurons are numerous and strongly interconnected between them. The human brain is one of the most computationally efficient device that is why in the late years a lot of effort has been done in order to develop an artificial version of it, as a matter of fact companies like Telsa motors are developing self driving cars which are based on ANN to implement the behavior of the human brain in computer systems.

The easiest way to represent a neuron is either (on or off) or spin up/down in physics) with the synapsys either eccitative or inibitive. We can easily map the neural network into a spin system by mapping neurons in spins $\{s_i\}_{i=1,\cdots,N}$ and synapsys into the magnetic coupling $J_{ij}$.
By a specific representation of these coupling costant we can define the Hopfield model.

## The Hopfield Model
The Hopfield model is a fully connected neural network able to recall stored memories starting from a noisy or distorted input. The Hopfield network consists of N neurons connected through symmetric bidirectional links.The neuron interactions are encoded in the connection matrix, a N × N real symmetric matrix without self-interaction terms whose $J_{ij}$ entries define the weight of the connection between neuron i and j.

The model respect the following mapping:

$$
n = \{0,1\} \longrightarrow s = \{-1,1\}      \mbox {     where n  is the binary operator and s the spin}
$$

The correct transformation of this mapping is:
$$
\begin{equation}
s = f(n) = \frac{2n-1}{2}
\end{equation}
$$
We define the synapses as:
$$
J_{ij}=\begin{cases}
+1 & \mbox{ excitatory synapses} \\
-1 & \mbox{ inhibitory synapses}
\end{cases}
$$

A neuron is activated if it receives a sufficient number of active impulsive, and we can compute these impulses as follows:
$$
\begin{equation}
h_i(t) = \sum^N_{j=1,j\neq i} J_{ij}(s_j(t)+1)
\label{eq:h}
\end{equation}
$$
It is important to notice that $j\neq i$ since the neuron DOES NOT interacts with itself.
To decide if these impulses are sufficient to activate the neuron we apply a non-linear function to the impulses:
$$
\begin{equation}
s_i(t+1)=sign{\left(h_i(t)-\theta_i\right)}
\end{equation}
$$
where $\theta_i$ is the threshold. We choose this threshold in a way that is useful for the calculations:
$$
\theta_i = \sum^N_{j=1,j\neq i} J_{ij}.
$$
By applying this threshold we obtain in Equation $\eqref{eq:upd}$ $\textbf{update rule}$ that we will use below in the code:
$$
\begin{equation}
s_i(t+1)=sign{\left(\sum^N_{j=1,j\neq i} J_{ij}s_j(t)\right)}
\label{eq:upd}
\end{equation}
$$

The $\textbf{Hopfield model}$ that we will use in the following consists in a specific choice of the synapses (ferromagnetic couplings):
$$
\
\begin{equation}
\begin{cases}
J_{ii}=0 & \mbox{ known as Hebb rule} \\
J_{ij}=\frac{1}{N}\sum_{\mu=1}^{p} \xi^\mu_i\xi^\mu_j
\end{cases}
\label{eq:hop}
\end{equation}
$$
where the $\vec{\xi^\mu}$ are $p<<N$ excitatory pattern with $\xi^\mu_i=\{+1,-1\}$.
This choice encode these patterns in the couplings and gives to the system some interesting properties:
- if the system start from a configuration equal to a pattern $\vec{\xi^\mu}$ and apply the update rule of Eq $\eqref{eq:upd}$ it stays in that pattern $\forall t$.
- by solving the system from a statistical mechanics point of view turns out that the system has many minima and this minima are all and only the patterns.

The actual resolution of this system is quite long and it is not the aim of this work, but it is instructive to see the proof of the stability of the patterns.
$$
s(1)=sign{\left(\sum^N_{j=1} J_{ij}s_j(0)\right)}\overset{s_j(0)=\xi^\mu_j}{=}
sign{\left(\sum^N_{j=1} \frac{1}{N}\sum^p_{\nu=1}\xi_i^\nu\xi_j^\nu\xi_j^\mu\right)}
\\=sign{\left(\sum^N_{\nu=1}\xi_i^\nu\frac{1}{N}\sum^p_{j=1}\xi_j^\nu\xi_j^\mu \right)}
=sign{\left(\sum^N_{\nu=1}\xi_i^\nu(\delta_{\mu\nu}+O(N^{-\frac{1}{2}}) \right)}
\simeq sign\left(\xi^\mu_i \right)=\xi^\mu_i
$$
And so the property enuciated before holds.

From now on we will talk about spins and not neurons anymore.

This model take into account fully connected system, where each spin is connected to all the others. But what happen if we consider an interaction lenght $R$? It is an interesting question to look at how the results vary with this assumption.

It is also really interesting to notice that this type of system does not need training, and so it can be reallly interesting if the results are good.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import scipy as scp

```

```python
# Parameters
L = 10   # Lenght of the edge of the image (change to 11?)
N = L**2 # Number of spins
p = 10   # Number of patterns
MF = 0   # Use or not the Mean Field strategy: if MF=1 uses MF, if MF = 0 only spins at
         # a distance R interacts
R = 3    # The 3-nearest neighbor interacts
np.random.seed(1234) # Seed to make the random process reproducible
```

```python
# Patterns
def random_patterns(N,p):
    xi = np.random.randint(0,2,N*p).reshape(p,N) # Each line is a pattern
    xi[xi==0]=-1
    return xi

xi = random_patterns(N,p)
idx = np.random.randint(0,p)
plt.imshow(xi[idx].reshape(L,L),cmap='Greys') # This is an example of pattern
# -1 = white, +1 = black
```

```python
# Coupling constant
# Row i is the connections of the i-th neuron with the others.
# Note that the Hopfield model requires Jii=0
# Note that J is a symmetrical matrix (J=J.T)
def MF_coupling(xi,N):
    J = 1/N * np.array( [ (xi[:,i]*xi[:,j]).sum() for i in range(N) for j in range(N) ] )
    J = J.reshape(N,N)
    for i in range(len(J)):
        J[i,i] = 0
    return J
def R_coupling(xi,N,R):
    J = MF_coupling(xi,N)
    for i in range( J.shape[0] ):
        J_temp = J[i].reshape(L,L)
        for j in range(L):
            y = (i%L -j)**2 # Look at indexes should be ok -1
            for k in range(L):
                if np.sqrt( (i//L - k)**2 + y ) > R: J_temp[j,k] = 0
        J[i] = J_temp.reshape(1,N)
    return J
```

```python
if MF: J = MF_coupling(xi,N)
else: J = R_coupling(xi,N,R)
```

```python
# Update rule
# def update(sys):
#     N = len(sys)
#     for i in range(N):
#         temp = 0
#         for j in range(N): #Maybe it is possible to set the cycle on J[i][j]!=0 only
#             temp +=  J[i][j]*sys[j]
#         if temp == 0: temp = 2*np.random.randint(0,2)-1
#         sys[i] = np.sign(temp)
#     return np.sign(sys)

def update(sys1):
    sys = deepcopy(sys1)
    N = len(sys)
    temp = np.zeros(N)
    for i in range(N):
        for j in range(N): #Maybe it is possible to set the cycle on J[i][j]!=0 only
            temp[i] +=  J[i][j]*sys[j]
        if temp[i] == 0: temp[i] = 2*np.random.randint(0,2)-1
    sys = np.sign(temp)
    return np.sign(sys)

def deterministic_hopfield(sys, t):
    for i in range(t):
        sys = update(sys)
    return sys

def error_im(xi_idx, sys):
    wrong_pixels = (np.abs( sys-xi_idx )/2).sum()
    return wrong_pixels/len(sys)

def assign_pattern(xi,sys):
    errors = [ error_im(i,sys) for i in xi ]
    correct = np.argmin(errors)
    return correct

def total_error(xi,t):
    errors = []
    prediction = []
    for mu in range(len(xi)):
        sys = deterministic_hopfield(xi[mu],t)
        errors.append( error_im(xi[mu],sys) )
        if assign_pattern(xi,sys)==mu:
            prediction.append( 1 )
        else: prediction.append( 0 )
    errors = (np.array(errors)).mean()
    prediction = np.array(prediction).sum()/len(xi)
    return errors, prediction

```

```python
# actual algorithm
from copy import deepcopy
idx = np.random.randint(0,p)
sys = deepcopy(xi[idx])

sys = deterministic_hopfield(sys,100)
wrong_pixels = error_im(xi[idx],sys)
assigned_pattern = assign_pattern(xi,sys)

fig, ax = plt.subplots(1,2, figsize = (10,6))
ax[0].set_title('Pattern')
ax[0].imshow(xi[idx].reshape(L,L), cmap='Greys')
ax[1].set_title('Finishing configuration')
ax[1].imshow(sys.reshape(L,L), cmap='Greys')
print('The error of the algorithm is %f' %(wrong_pixels))
print('The algorithm recognised the pattern %i and the correct pattern is %i' %(assigned_pattern,idx))
```

## Analisys

```python
# We now look at the scaling with regards to the parameters
L = 16
N = L**2

P = np.arange(5,N/2,20)
scores_16 = [[],[]]
errors_16 = [[],[]]

for MF in range(2):
    for p in P:
        p = int(p)
        xi = random_patterns(N,p)
        if MF: J = MF_coupling(xi,N)
        else: J = R_coupling(xi,N,R)
        err, pred = total_error(xi,30)
        scores_16[MF].append( pred )
        errors_16[MF].append(err)

```

```python
figp, axp = plt.subplots(1,2, figsize=(14,6))
axp[0].set_title('Dependance of the error by number of pattern p')
axp[0].set_xlabel('Number of pattern p')
axp[0].set_ylabel('Average error along the p patterns')
axp[0].plot(P,errors_16[0],  '--bo', label='Error, no MF')
axp[0].plot(P,errors_16[1], '--go', label='Error, MF')
axp[0].legend()

axp[1].set_title('Dependance of the prediction of the correct pattern by number of pattern p')
axp[1].set_xlabel('Number of pattern p')
axp[1].set_ylabel('Fraction of pattern correctly predicted')
axp[1].plot(P,scores_16[0], '--bo', label='Score, no MF')
axp[1].plot(P,scores_16[1], '--go', label='Score, MF')
axp[1].legend()
```

The dependance of error with patterns generally follows an increasing tendency at the beginning of the curve, which then flatters, reaching a plateau. The growth is particularly steep in the case of the Non Mean Field model, which reaches it maximum within the first $40$ $p$-patterns. It is instead is less pronounced for the Mean Field theory: here the highest value ($\approx 0.25$) is reached only after about $70$ patterns. The plateau of the Non Mean Field graph is, on the contrary, very stable, showing almost no oscillation around the value of $0.15$, which is also lower.

Concerning the dependence of the prediction of the correct pattern with number of patterns, it can be observed that, in case of Non Mean Field model, the score is always constant whichever the pattern, thus the independence of the error with  number of patterns and the perfect effectivness of the predictions. With the Mean Field model, however, this stability is not preserved: in fact, after passing the first few patterns, the score drops to about $70\%$, and oscillates around that value.

By observing both graphs, it can be stated that there exist an optimal value of $p$ patterns which optimize the performances of the algorithm, which in this case is around $20$ (about $8\%$ of the system size). In order to check if this tendency is preserved with other sizes of the system, different sizes are studied.

```python
L = 10
N = L**2
P_10 = np.arange(5,N/2,5)
scores_10 = []
errors_10 = []
sigma_10 = []

MF = 0
for pi in P_10:
    err = []
    pred = []
    for n in range(20):
        p = int(pi)
        xi = random_patterns(N,p)
        if MF: J = MF_coupling(xi,N)
        else: J = R_coupling(xi,N,R)
        e, p = total_error(xi,30)
        err.append(e)
        pred.append(p)
    pred = np.array(pred)
    err = np.array(err)
    sigma_10.append( pred.std() )
    scores_10.append( pred.mean() )
    errors_10.append( err.mean() )
```

```python
figp, axp = plt.subplots(1,2, figsize=(14,6))
axp[0].set_title('Dependance of the error by number of pattern p')
axp[0].set_xlabel('Number of pattern p')
axp[0].set_ylabel('Average error along the p patterns')
axp[0].plot(P_10,errors_10,  '--co', label='Error, no MF, L =10')
axp[0].plot(P,errors_16[0],  '--bo', label='Error, no MF, L =16')
axp[0].legend()

axp[1].set_title('Dependance of the prediction of the correct pattern by number of pattern p')
axp[1].set_xlabel('Number of pattern p')
axp[1].set_ylabel('Fraction of pattern correctly predicted')
axp[1].plot(P_10,scores_10, '--co', label='Score, no MF, L =10')
axp[1].plot(P,scores_16[0],  '--bo', label='Score, no MF, L =16')
axp[1].legend()
```

```python
L = 6
N = L**2
P_6 = np.arange(2,N/2,1)
scores_6 = []
errors_6 = []
sigma_6 = []

MF = 0

for pi in P_6:
    err = []
    pred = []
    for n in range(50):
        p = int(pi)
        xi = random_patterns(N,p)
        if MF: J = MF_coupling(xi,N)
        else: J = R_coupling(xi,N,R)
        e, p = total_error(xi,30)
        err.append(e)
        pred.append(p)
    pred = np.array(pred)
    err = np.array(err)
    sigma_6.append( pred.std() )
    scores_6.append( pred.mean() )
    errors_6.append( err.mean() )
```

```python
L = 8
N = L**2
P_8 = np.arange(2,N/2,2)
scores_8 = []
errors_8 = []
sigma_8 = []

MF = 0

for pi in P_8:
    err = []
    pred = []
    for n in range(40):
        p = int(pi)
        xi = random_patterns(N,p)
        if MF: J = MF_coupling(xi,N)
        else: J = R_coupling(xi,N,R)
        e, p = total_error(xi,30)
        err.append(e)
        pred.append(p)
    pred = np.array(pred)
    err = np.array(err)
    sigma_8.append( pred.std() )
    scores_8.append( pred.mean() )
    errors_8.append( err.mean() )

```

```python
figp, axp = plt.subplots(1,2, figsize=(14,6))
axp[0].set_title('Dependance of the error by number of pattern p')
axp[0].set_xlabel('Number of pattern p')
axp[0].set_ylabel('Average error along the p patterns')
axp[0].plot(P_6,errors_6, '--co', label='Error, no MF, L = 6')
axp[0].plot(P_8,errors_8, '--ro', label='Error, no MF, L = 8')
axp[0].plot(P_10,errors_10, '--ko', label='Error, no MF, L = 10')
axp[0].legend()

axp[1].set_title('Dependance of the prediction of the correct pattern by number of pattern p')
axp[1].set_xlabel('Number of pattern p')
axp[1].set_ylabel('Fraction of pattern correctly predicted')
axp[1].plot(P_6, scores_6, '--co', label='Score, no MF, L = 6')
axp[1].plot(P_8, scores_8, '--ro', label='Score, no MF, L = 8')
axp[1].plot(P_10, scores_10, '--ko', label='Score, no MF, L = 10')
axp[1].legend()
```

Comment these results
- optimal number of patterns <20, so % of system

```python
# How does the time scales with the number of spins N?
import time as time
L = np.arange(10,28,4)
N = L**2
p = 10
times = [[],[]]

for MF in range(2):
    for n in N:
        xi = np.random.randint(0,2,n*p).reshape(p,n)
        xi[xi==0]=-1
        J = 1/n * np.array( [ (xi[:,i]*xi[:,j]).sum() for i in range(n) for j in range(n) ] )
        J = J.reshape(n,n)
        for k in range(len(J)):
            J[k,k] = 0
        idx = np.random.randint(0,p)
        start = time.time()
        deterministic_hopfield(xi[idx],30)
        end = time.time()
        times[MF].append(end-start)

```

```python
fig, ax = plt.subplots(figsize = (8, 8))
ax.plot(N,times[0], '--bo', label = 'Data with 10 patterns, no MF')
ax.plot(N,times[1], '--go', label = 'Data with 10 patterns, MF')
ax.set(xlabel = ' Number of spins N', ylabel =' Implementation time [s]',
              title = '  Dependency of the time implementation on the size of the system')
ax.legend()
```

The two curves above show both an exponential growth of the implementation time as a function of the size of the system. The tendency does not change for both Mean Field and Non Mean Field models: this is consistent with the fact that the two algorithms are implemented with the same logic, just by changing the couplong rule.

The variation of implementation time with the number of spins follows a linear form with different slopes. Implementation time increases as the number of spins increases, for N in (500, 700), the implementation time increases slowly meaning that more time is required when a number of spins are used. In general the implementation time grows exponentially with number of spins.

```python
# How does the time scales with the number of pattern p with a fixed N?

L = 16
N = L**2
P = np.arange(10,N,16)
times = [[],[]]

for MF in range(2):
    for p in P:
        p = int(p)
        xi = np.random.randint(0,2,N*p).reshape(p,N)
        xi[xi==0]=-1
        J = 1/N * np.array( [ (xi[:,i]*xi[:,j]).sum() for i in range(N) for j in range(N) ] )
        J = J.reshape(N,N)
        for k in range(len(J)):
            J[k,k] = 0
        idx = np.random.randint(0,p)
        start = time.time()
        deterministic_hopfield(xi[idx],30)
        end = time.time()
        times[MF].append(end-start)

```

```python
fig, ax = plt.subplots(figsize = (8, 8))
ax.plot(P,times[0], '--bo', label = 'Data with 16x16 spins, no MF')
ax.plot(P,times[1], '--go', label = 'Data with 16x16 spins, MF')
ax.set(xlabel = 'Number of patterns p', ylabel =' Implementation time [s]',
              title = ' Dependency of the time implementation on the number of patterns')
ax.legend()
```

In this graph the implementation time ad a function of the number of patterns is displayed. The shape of the curve does not outline any particular tendency, and oscillates randomly showing instability.
The Mean Field model has less pronounced fluctuations with respect to the Non Mean Field one.

It can be concluded that the Non Mean Field model is more stable and leads to smaller errors, but is less computationally efficient with respect to the Mean Field one, which is unstable given the considerations stated in the cells above.

```python
MF = 0
L = 10
N = L**2
P = 10
error_R = [[],[]]
sigmas_R = [[],[]]
R = np.arange(3,L)

for MF in range(2):
    for r in R:
        supp = []
        for n in range(50):
            xi = random_patterns(N,P)
            if MF: J = MF_coupling(xi,N)
            else: J = R_coupling(xi,N,r)
            err, pred = total_error(xi,30)
            supp.append(err)
        supp = np.array(supp)
        error_R[MF].append(supp.mean())
        sigmas_R[MF].append(supp.std())

```

```python
fig, ax = plt.subplots(figsize = (8,8))
ax.plot(R,error_R[0], '--bo', label = 'no MF')
ax.errorbar(R,error_R[0], yerr=sigmas_R[0], fmt='none', color='b')
ax.plot(R,error_R[1], '--go', label = 'MF')
ax.errorbar(R,error_R[1], yerr=sigmas_R[1], fmt='none', color='g')

ax.set(xlabel = 'Number of neighbors R', ylabel =' Error ',
              title = ' Dependency of the error on the number of neighbors')
ax.legend()

```

## Distinguish criteria of similar patterns

The aim of this section is to define in a more rigorous way when two patterns are considered $\textit{similar}$ and when they are easily distinguishable by the MF and NMF models.

In order to do so, we define a new error function which weights the different pixels between two patterns with respect to their distance: so we will consider bigger the error if wrong pixels are close to each other.

```python
L=10
N=L**2
```

```python
def w_error(sys11, sys22, alpha):
    sys1 = deepcopy(sys11)
    sys2 = deepcopy(sys22)
    N = sys1.shape[0]
    L = int(np.sqrt(N))
    sys1 = sys1.reshape(L,L)
    sys2 = sys2.reshape(L,L)
    wrong1 = [] #wrong black pixels in sys1
    wrong2 = [] # in sys2
    min_dist = []
    for i in range(L):
        for j in range(L):
            if sys1[i][j] == sys2[i][j]: continue
            elif sys1[i][j]== 1 : # if black in sys1
                wrong1.append( np.array([i,j]) )
            elif sys2[i][j]== 1: # if black in sys2
                wrong2.append( np.array([i,j]) )
    # compute distances between wrong pixels of the two systems
    for i in wrong1:
        distances = []
        for j in wrong2:
            if np.linalg.norm(i-j) !=0 :
                distances.append( np.linalg.norm(i-j) )
        min_dist.append( np.array(distances).min() )
    min_dist = np.array(min_dist)
    err =  (1/(min_dist)**alpha).sum()
    err = err/((N+1)//2) #normalization due to worst case scenario (a chessboard with more black cells)
    return err

```

```python
def plot(x, y):
    L = int(np.sqrt(x.shape[0]))
    fig, ax = plt.subplots(1,2, figsize=(10,6))
    ax[0].set_title('Original pattern')
    ax[0].imshow(x.reshape(L,L),cmap='Greys')
    ax[1].set_title('Corrupted pattern')
    ax[1].imshow(y.reshape(L,L),cmap='Greys')
    plt.show()
```

```python
# straight verical lines patterns

patt = [np.zeros(N)-1 for j in range(L)]
for i in range(L):
    for j in range(L):
        patt[i][i+L*j]=1

fig, ax = plt.subplots(2, int(len(patt)/2), figsize=(22,10))
err_patt = []

for i in range(int(len(patt)/2)):
    ax[0][i].set_title('Pattern %i' %(i+1))
    ax[0][i].imshow(patt[i].reshape(L,L),cmap='Greys')
    ax[1][i].set_title('Pattern %i' %(i+int(len(patt)/2)+1))
    ax[1][i].imshow(patt[i+int(len(patt)/2)].reshape(L,L),cmap='Greys')
plt.show()

for i in patt:
    err_patt.append(w_error(patt[0],i,2))

figure = plt.figure(figsize=(6,4))
plt.grid('-.', alpha=0.5)
plt.scatter(np.arange(1, len(err_patt)+1), err_patt, marker='X', color='darkred')
plt.title("Error with respect to pattern 1")
plt.xlabel("Pattern number")
plt.ylabel("Error")

plt.show()
```

```python
# we try our algorithm to see if the patterns are distinguishable.

patt = np.array(patt)
ind = np.arange(len(patt))

for idx in ind:
    for MF in range(2):
        if MF: J = MF_coupling(patt,N)
        else: J = R_coupling(patt,N,R)

        sys = deepcopy(patt[idx])

        sys = deterministic_hopfield(sys,100)
        wrong_pixels = error_im(patt[idx],sys)
        assigned_pattern = assign_pattern(patt,sys)

        fig, ax = plt.subplots(1,2, figsize = (8,5))
        ax[0].set_title('Pattern')
        ax[0].imshow(patt[idx].reshape(L,L), cmap='Greys')
        ax[1].set_title('Finishing configuration')
        ax[1].imshow(sys.reshape(L,L), cmap='Greys')
        plt.show()
        print('The error of the algorithm is %f' %(wrong_pixels))
        print('The algorithm recognised the pattern %i and the correct pattern is %i' %(assigned_pattern,idx))
```

```python
print(J.mean(axis=0).reshape(L,L))
```

    [[0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]
     [0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063 0.063]]

```python
# chessboard
chess = [np.zeros(N)-1 for j in range(2)]
c = np.array([np.zeros((L,L))-1 for j in range(2)])

for i in range(2):
    if i == 0:
        c[i][::2, ::2] = 1
        c[i][1::2, 1::2] = 1
    else:
        c[i][1::2, ::2] = 1
        c[i][::2, 1::2] = 1
    chess[i] = c[i].reshape(1,N)[0]

err_chess = []

fig, ax = plt.subplots(1, 2, figsize=(16,6))

ax[0].set_title('Pattern 1')
ax[0].imshow(chess[0].reshape(L,L),cmap='Greys')
ax[1].set_title('Pattern 2')
ax[1].imshow(chess[1].reshape(L,L),cmap='Greys')
plt.show()

for i in chess:
    err_chess.append(w_error(chess[0],i,2))

print("Error between the two patterns: ", err_chess[1])
```

```python
# we try our algorithm to see if the two chessboards are distinguishable.

chess = np.array(chess)
ind = [0,1]

for idx in ind:
    for MF in range(2):
        if MF: J = MF_coupling(chess,N)
        else: J = R_coupling(chess,N,R)

        sys = deepcopy(chess[idx])

        sys = deterministic_hopfield(sys,100)
        wrong_pixels = error_im(chess[idx],sys)
        assigned_pattern = assign_pattern(chess,sys)

        fig, ax = plt.subplots(1,2, figsize = (8,5))
        ax[0].set_title('Pattern')
        ax[0].imshow(chess[idx].reshape(L,L), cmap='Greys')
        ax[1].set_title('Finishing configuration')
        ax[1].imshow(sys.reshape(L,L), cmap='Greys')
        plt.show()
        print('The error of the algorithm is %f' %(wrong_pixels))
        print('The algorithm recognised the pattern %i and the correct pattern is %i' %(assigned_pattern,idx))
```

```python
print(J.mean(axis=0))
```

    [-0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002 -0.0002
     -0.0002]

```python
# frames
frames = [np.zeros(N)-1 for j in range((L+1)//2)]

for i in range((L+1)//2):
    for j in range(N):
        if j%L == i:
            frames[i][j] = 1
        if j%L == (L-i-1) :
            frames[i][j] = 1
        if j//L == i:
            frames[i][j] = 1
        if j//L == (L-i-1) :
            frames[i][j] = 1
    for k in range(i):
        for j in range(N):
            if j%L == k:
                frames[i][j] = -1
            if j%L == (L-k-1) :
                frames[i][j] = -1
            if j//L == k:
                frames[i][j] = -1
            if j//L == (L-k-1) :
                frames[i][j] = -1

err_frame = []

fig, ax = plt.subplots(1, (L+1)//2 , figsize=(16,6))

for i in range((L+1)//2):
    ax[i].set_title('Pattern %i' %(i+1))
    ax[i].imshow(frames[i].reshape(L,L),cmap='Greys')
plt.show()

for i in frames:
    err_frame.append(w_error(frames[0],i,2))

figure = plt.figure(figsize=(6,4))
plt.grid('-.', alpha=0.5)
plt.scatter(np.arange(1, 1+len(err_frame)), err_frame, marker='X', color='darkred')
plt.title("Error with respect to pattern 1")
plt.xlabel("Pattern number")
plt.xticks(np.arange(1, 1+len(err_frame)))
plt.ylabel("Error")

plt.show()

```

```python
frames = np.array(frames)
ind = np.arange(len(frames))

for idx in ind:
    for MF in range(2):
        if MF: J = MF_coupling(frames,N)
        else: J = R_coupling(frames,N,R)

        sys = deepcopy(frames[idx])

        sys = deterministic_hopfield(sys,100)
        wrong_pixels = error_im(frames[idx],sys)
        assigned_pattern = assign_pattern(frames,sys)

        fig, ax = plt.subplots(1,2, figsize = (8,5))
        ax[0].set_title('Pattern')
        ax[0].imshow(frames[idx].reshape(L,L), cmap='Greys')
        ax[1].set_title('Finishing configuration')
        ax[1].imshow(sys.reshape(L,L), cmap='Greys')
        plt.show()
        print('The error of the algorithm is %f' %(wrong_pixels))
        print('The algorithm recognised the pattern %i and the correct pattern is %i' %(assigned_pattern,idx))
```

```python
print(J.mean(axis=0).std())
```

    0.003731809212700987

```python
f = np.array(frames[4:])
print(f.reshape(L,L))
J = MF_coupling(f,N)
print(J[23].reshape(L,L))
```

    [[-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
     [-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
     [-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
     [-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
     [-1. -1. -1. -1.  1.  1. -1. -1. -1. -1.]
     [-1. -1. -1. -1.  1.  1. -1. -1. -1. -1.]
     [-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
     [-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
     [-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
     [-1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]]
    [[ 0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.    0.01  0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01 -0.01 -0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01 -0.01 -0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01]
     [ 0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01  0.01]]

```python
# random
diff = [np.zeros(N)-1 for j in range(5)]

for i in range(5):
    for j in range(L):
        k = np.random.randint(0,N)
        diff[i][k] = 1

fig, ax = plt.subplots(1, (L+1)//2 , figsize=(16,6))

for i in range((L+1)//2):
    ax[i].set_title('Pattern %i' %(i+1))
    ax[i].imshow(diff[i].reshape(L,L),cmap='Greys')
plt.show()
```

```python
# random
diff = [np.zeros(N)-1 for j in range(2)]

for i in range(2):
    for j in range(L):
        k = np.random.randint(0,N)
        diff[i][k] = 1

fig, ax = plt.subplots(1, 2 , figsize=(10,6))

for i in range(2):
    ax[i].set_title('Pattern %i' %(i+1))
    ax[i].imshow(diff[i].reshape(L,L),cmap='Greys')
plt.show()
```

```python
avg_rnd = np.array(diff).mean(axis=0)
avg_fr = np.array(frames).mean(axis=0)
avg_patt = np.array(patt).mean(axis=0)
avg_ch = np.array(chess).mean(axis=0)

print(avg_rnd.reshape(L,L))
print(avg_fr.reshape(L,L))
print(avg_patt.reshape(L,L))
print(avg_ch.reshape(L,L))

plt.imshow(avg_rnd.reshape(L,L),cmap='Greys')
plt.show()
plt.imshow(avg_fr.reshape(L,L),cmap='Greys')
plt.show()
plt.imshow(avg_patt.reshape(L,L),cmap='Greys')
plt.show()
plt.imshow(avg_ch.reshape(L,L),cmap='Greys')
plt.show()

```

```python
diff = np.array(diff)
ind = np.arange(len(diff))

for idx in ind:
    for MF in range(2):
        if MF: J = MF_coupling(diff,N)
        else: J = R_coupling(diff,N,R)

        sys = deepcopy(diff[idx])

        sys = deterministic_hopfield(sys,100)
        wrong_pixels = error_im(diff[idx],sys)
        assigned_pattern = assign_pattern(diff,sys)

        fig, ax = plt.subplots(1,2, figsize = (8,5))
        ax[0].set_title('Pattern')
        ax[0].imshow(diff[idx].reshape(L,L), cmap='Greys')
        ax[1].set_title('Finishing configuration')
        ax[1].imshow(sys.reshape(L,L), cmap='Greys')
        plt.show()
        print('The error of the algorithm is %f' %(wrong_pixels))
        print('The algorithm recognised the pattern %i and the correct pattern is %i' %(assigned_pattern,idx))
```

```python
print(J.mean(axis=0).std())
```

    0.01084544143868747

```python

```