### Instructions

When running the notebook the first time, make sure to run all cells before making changes in the notebook. Hit Shift + Enter to run the selected cell or, in the top menu, click on: `Kernel` > `Restart Kernel and Run All Cells...` to rerun the whole notebook. If you make any changes in a cell, rerun that cell.

If you make any changes in a coding cell, rerun the notebook by `Run` > `Run Selected Cell and All Below`

```python
# Import dependencies
import sys
sys.path.append('python/')
import numpy as np
import matplotlib.pyplot as plt
import rc_animation as rc_ani
import time
startTime = time.time()
from IPython.display import HTML
```

# Introduction to Rotation Curves

A rotation curve plots the rotational velocity of stars in a galaxy in terms of their distance from the center, also known as the radius. Using photometric data of the luminous matter, a rotation curve of a galaxy can be created. It is used to estimate the mass inside a radius by equating the centripetal force to the gravitational force:

\begin{equation}
\frac{mv^2}{r}=\frac{G M_{enc}(r)}{r^2}
\end{equation}

>where:

        $v$ = rotational velocity

        $G$ = gravitational constant

        $M_{enc}(r)$ = enclosed mass as a function of radius

        $r$ = radius or distance from the center of the galaxy

Rotation curves of some spiral galaxies reveal a discrepancy in the measured and luminous mass. The relationship between the rotational velocity and mass is important to the study of Dark Matter. Plotting the rotation curves of edge-on spiral galaxies and their components can be used as a tool to find the Dark Matter rotation curve and its corresponding mass.

This notebook aims to demonstrate three different rotation curve types - rigid body (frisbee) rotation, planet-like rotation, and flat rotation.

### Rigid body (frisbee) rotation

The most straight-forward rotation curve is that of a rigid body, i.e. a solid disk. The rotational velocity in this case is proportional to the radius of the rotating object, which can be denoted by:

\begin{equation}
v \propto r
\end{equation}

Because of this, the rigid body has a rotation curve that increases linearly with radius. To demonstrate this, let's create two arrays for the radius and the rotational velocity called `radiusRB` and `velocityRB`, respectively:

```python
radiusRB = np.array([1,2,3,4,5])
velocityRB = np.array([0.1,0.2,0.3,0.4,0.5])
```

```python
try: # Display animation
    display(HTML(''))
except FileNotFoundError: #Make it first
    rc_ani.MakeAnimation(radiusRB,velocityRB,
                         100,1,                                         # time and dt
                         'images/rigidbody.gif','Rigid Body Rotation')  # file name and title of plot
    display(HTML(''))
```

```python
rc_ani.PlotRotationCurve(radiusRB,velocityRB,'Rigid Body Rotation Curve')
```

In rigid body rotation, all pieces of the body must maintain the same relative position to each other throughout the rotation. Another way to think about this is that all parts of a rigid body have the same angular velocity -- that is, the rate at which their angle from the horizontal changes is the same.

### Planet-like rotation

Non-rigid rotating bodies have more flexibility in their rotation. The most well-known example is the Keplerian model, which models orbital rotation in solar systems and planetary motion. This model has most of its mass concentrated in the center of the rotating object with satellites free to orbit around the center. With planet-like rotation, rotational velocity is inversely proportional to the square root of the radius:

\begin{equation}
v \propto \frac{1}{\sqrt{r}}
\end{equation}

One example is our Solar System with eight planets orbiting the Sun at the center.

```python
# Parameters
G = 6.67408e-11         # Gravitational constant (in m^3 kg^-1 s^-2)
AU = 1.496e11           # AU Astronomical Unit (in meters)

# Masses of the planets (M) in kg
M_Earth = 5.97e24
M_Sun = 3.3e5*M_Earth
M_Mercury = 0.0553*M_Earth
M_Venus = 0.815*M_Earth
M_Mars = 0.107*M_Earth
M_Jupiter = 317.8*M_Earth
M_Saturn = 95.2*M_Earth
M_Uranus = 14.5*M_Earth
M_Neptune = 17.1*M_Earth

# Make array of masses
massesSS = np.array([M_Mercury,M_Venus,M_Earth,M_Mars,M_Jupiter,M_Saturn,M_Uranus,M_Neptune])

# Distances of planets from the Sun (r) in meters
r_Earth = AU
r_Mercury = 0.387*AU
r_Venus = 0.723*AU
r_Mars = 1.52*AU
r_Jupiter = 5.20*AU
r_Saturn = 9.57*AU
r_Uranus = 19.17*AU
r_Neptune = 30.18*AU

# Make array of radii
radiusSS = np.array([r_Mercury,r_Venus,r_Earth,r_Mars,r_Jupiter,r_Saturn,r_Uranus,r_Neptune])

# Orbital velocity of planets (v) in m/s
v_Mercury = np.sqrt((G*M_Sun)/r_Mercury)
v_Venus = np.sqrt((G*(M_Sun+M_Mercury))/r_Venus)
v_Earth = np.sqrt((G*(M_Sun+M_Mercury+M_Venus))/r_Earth)
v_Mars = np.sqrt((G*(M_Sun+M_Mercury+M_Venus+M_Earth))/r_Mars)
v_Jupiter = np.sqrt((G*(M_Sun+M_Mercury+M_Venus+M_Earth+M_Mars))/r_Jupiter)
v_Saturn = np.sqrt((G*(M_Sun+M_Mercury+M_Venus+M_Earth+M_Mars+M_Jupiter))/r_Saturn)
v_Uranus = np.sqrt((G*(M_Sun+M_Mercury+M_Venus+M_Earth+M_Mars+M_Jupiter+M_Saturn))/r_Uranus)
v_Neptune = np.sqrt((G*(M_Sun+M_Mercury+M_Venus+M_Earth+M_Mars+M_Jupiter+M_Saturn+M_Uranus))/r_Neptune)

# Make array of velocities
velocitySS = np.array([v_Mercury,v_Venus,v_Earth,v_Mars,v_Jupiter,v_Saturn,v_Uranus,v_Neptune])
```

```python
try: #Load the animation if it already exists
    HTML('')
except FileNotFoundError: #And create it if it doesn't
    rc_ani.MakeAnimation(1e-3*radiusSS,1e-3*velocitySS,   # change distance unit to km, velocity unit to km/s
                         5e10,1e7,                        # time and dt
                         'images/solarsystem.gif','Planet-like Rotation',
                         1e8,1e8,                         # xlim and ylim
                         size=True,masses=massesSS)
    # Display animation
    HTML('')
```

```python
rc_ani.PlotRotationCurve(1e-3*radiusSS,1e-3*velocitySS,
                  'Planet-like Rotation Curve',
                  xlabel='Radius (km)',ylabel='Velocity (km/s)',
                  xlim=1e8,ylim=5)
```

In a planet-like rotation, none of the bodies are attached to one another in any way, so they can move in relation to one another. Often, there is a single, dominant source of mass at the center -- in the case of our Solar system, the Sun is massive enough that we can ignore the masses of planets in our calculations. However, if there is a significant amount of mass between the center and the bodies we are observing, we may get different results...

### Flat rotation

The mass distribution of a galaxy based on luminous matter is similar to the mass distribution of the Keplerian model. Surprisingly, some of the rotation curves of galaxies are entirely different than the rotation curve of planetary motion. The curve for spiral galaxies quickly increases from the center to a relatively constant velocity:

\begin{equation}
v \sim constant
\end{equation}

This type of rotation curve suggests there is more mass spread throughout the galaxy than we can see. This 'missing mass' is what we call Dark Matter.

```python
# Arbitrary distance and velocity for animation and plot
radiusFR = np.array([1,2,3,4,5,6,7])
velocityFR = np.array([0.1,0.24,0.29,0.31,0.31,0.31,0.31])
```

```python
try:
    display(HTML(''))
except FileNotFoundError:
    rc_ani.MakeAnimation(radiusFR,velocityFR,
                         200,1,                                       # time and dt
                         'images/flatrotation.gif','Flat Rotation')   # file name and title of plot
    # Display animation
    display(HTML(''))
```

```python
rc_ani.PlotRotationCurve(radiusFR,velocityFR,'Flat Rotation Curve')
```

We see that velocity increases as we move outward. This is because, for a galaxy, mass is spread out throughout the galaxy, rather than being largely concentrated at the center. However, what we would expect to see is for the velocity to start dropping off at some point because the concentration of mass is still expected to be lower as we approach the edges, even if not negligibly so. This is why the flatness seems to indicate that there is 'missing' mass in our data.

```python
#NBVAL_IGNORE_OUTPUT
#Because the calculation time naturally varies.
# Calculation time
executionTime3 = (time.time() - startTime)
ttt3=executionTime3/60
print(f'Execution time: {ttt3:.2f} minutes')
```

    Execution time: 0.01 minutes

### References

>Carroll, B. W., & Ostlie, D. A. (2018). **An Introduction to Modern Astrophysics**. Cambridge University Press.

>Williams, D. (2021, December 27). **Planetary Fact Sheet**. NASA. Retrieved February 27, 2022, from https://nssdc.gsfc.nasa.gov/planetary/factsheet/


This notebook is part of https://github.com/AudioSceneDescriptionFormat/splines, see also https://splines.readthedocs.io/.

[back to overview](hermite.ipynb)

# Uniform Cubic Hermite Splines

We derive the basis matrix as well as the basis polynomials
for cubic (= degree 3) Hermite splines.
The derivation for other degrees is left as an exercise for the reader.

In this notebook,
we consider *uniform* spline segments,
i.e. the parameter in each segment varies from $0$ to $1$.
The derivation for *non-uniform* cubic Hermite splines
can be found in [a separate notebook](hermite-non-uniform.ipynb).

```python
import sympy as sp
sp.init_printing(order='grevlex')
```

We load a few tools from [utility.py](utility.py):

```python
from utility import NamedExpression, NamedMatrix
```

```python
t = sp.symbols('t')
```

We are considering a single cubic polynomial segment of a Hermite spline
(which is sometimes called a *Ferguson cubic*).

To simplify the indices in the following derivation,
we are looking at the fifth polynomial segment $\boldsymbol{p}_4(t)$
from $\boldsymbol{x}_4$ to $\boldsymbol{x}_5$, where $0 \le t \le 1$.
The results will be easily generalizable to an arbitrary
polynomial segment $\boldsymbol{p}_i(t)$
from $\boldsymbol{x}_i$ to $\boldsymbol{x}_{i+1}$.

The polynomial has 4 coefficients, $\boldsymbol{a_4}$ to $\boldsymbol{d_4}$.

```python
coefficients = sp.Matrix(sp.symbols('a:dbm4')[::-1])
coefficients
```

Combined with the *monomial basis* ...

```python
b_monomial = sp.Matrix([t**3, t**2, t, 1]).T
b_monomial
```

... the coefficients form an expression
for our polynomial segment $\boldsymbol{p}_4(t)$:

```python
p4 = NamedExpression('pbm4', b_monomial.dot(coefficients))
p4
```

For more information about polynomials,
see [Polynomial Parametric Curves](polynomials.ipynb).

Let's also calculate the first derivative
(a.k.a. velocity, a.k.a. tangent vector),
while we are at it:

```python
pd4 = p4.diff(t)
pd4
```

To generate a Hermite spline segment,
we have to provide the value of the polynomial
at the start and end point of the segment
(at times $t = 0$ and $t = 1$, respectively).
We also have to provide the first derivative at those same points.

\begin{align*}
\boldsymbol{x}_4 &= \left.\boldsymbol{p}_4\right\rvert_{t=0}\\
\boldsymbol{x}_5 &= \left.\boldsymbol{p}_4\right\rvert_{t=1}\\
\boldsymbol{\dot{x}}_4 &= \left.\frac{d}{dt}\boldsymbol{p}_4\right\rvert_{t=0}\\
\boldsymbol{\dot{x}}_5 &= \left.\frac{d}{dt}\boldsymbol{p}_4\right\rvert_{t=1}
\end{align*}

We call those 4 values the *control values* of the segment.

Evaluating the polynomial and its derivative
at times $0$ and $1$ leads to 4 expressions for our 4 control values:

```python
x4 = p4.evaluated_at(t, 0).with_name('xbm4')
x5 = p4.evaluated_at(t, 1).with_name('xbm5')
xd4 = pd4.evaluated_at(t, 0).with_name('xdotbm4')
xd5 = pd4.evaluated_at(t, 1).with_name('xdotbm5')
```

```python
display(x4, x5, xd4, xd5)
```

## Basis Matrix

Given an input vector of control values ...

```python
control_values_H = NamedMatrix(sp.Matrix([x4.name,
                                          x5.name,
                                          xd4.name,
                                          xd5.name]))
control_values_H.name
```

... we want to find a way to transform those into the coefficients of our cubic polynomial.

```python
M_H = NamedMatrix(r'{M_\text{H}}', 4, 4)
```

```python
coefficients_H = NamedMatrix(coefficients, M_H.name * control_values_H.name)
coefficients_H
```

This way, we can express our previously unknown coefficients
in terms of the given control values.

However, in order to make it easy to determine
the coefficients of the *basis matrix* $M_H$,
we need the equation the other way around
(by left-multiplying by the inverse):

```python
control_values_H.expr = M_H.name.I * coefficients
control_values_H
```

We can now insert the expressions for the control values
that we obtained above ...

```python
substitutions = x4, x5, xd4, xd5
```

```python
control_values_H.subs_symbols(*substitutions)
```

... and from this equation we can directly read off
the matrix coefficients of ${M_H}^{-1}$:

```python
M_H.I = sp.Matrix(
    [[expr.coeff(cv) for cv in coefficients]
     for expr in control_values_H.subs_symbols(*substitutions).name])
M_H.I
```

The same thing for copy & paste purposes:

```python
print(_.expr)
```

This transforms the coefficients of the polynomial into our control values,
but we need it the other way round,
which we can simply get by inverting the matrix:

```python
M_H
```

Again, for copy & paste:

```python
print(_.expr)
```

## Basis Polynomials

Multiplying the monomial basis with this matrix
yields the *Hermite basis polynomials*:

```python
b_H = NamedMatrix(r'{b_\text{H}}', b_monomial * M_H.expr)
b_H.factor().simplify().T
```

Let's plot the basis polynomials
with some help from [helper.py](helper.py):

```python
from helper import plot_basis
```

```python
plot_basis(*b_H.expr, labels=sp.symbols('xbm_i xbm_i+1 xdotbm_i xdotbm_i+1'))
```

Note that the basis function associated with $\boldsymbol{x}_i$
has the value $1$ at the beginning,
while all others are $0$ at that point.
For this reason,
the linear combination of all basis functions at $t=0$
simply adds up to the value $\boldsymbol{x}_i$
(which is exactly what we wanted to happen!).

Similarly,
the basis function associated with $\boldsymbol{\dot{x}}_i$
has a first derivative of $+1$ at the beginning,
while all others have a first derivative of $0$.
Therefore,
the linear combination of all basis functions at $t=0$
turns out to have a first derivative of $\boldsymbol{\dot{x}}_i$
(what a coincidence!).

While $t$ progresses towards $1$,
both functions must relinquish their influence
to the other two basis functions.

At the end (when $t=1$),
the basis function associated with $\boldsymbol{x}_{i+1}$
is the only one that has a non-zero value.
More concretely, it has the value $1$.
Finally,
the basis function associated with $\boldsymbol{\dot{x}}_{i+1}$
is the only one with a non-zero first derivative.
In fact, it has a first derivative of exactly $+1$
(the function values leading up to that have to be negative
because the final function value has to be $0$).

This can be summarized by:

```python
sp.Matrix([[
    b.subs(t, 0),
    b.subs(t, 1),
    b.diff(t).subs(t, 0),
    b.diff(t).subs(t, 1),
] for b in b_H.expr])
```

## Example Plot

To quickly check whether the matrix $M_H$ does what we expect,
let's plot an example segment:

```python
import numpy as np
```

If we use the same API as for the other splines,
we can reuse the helper functions for plotting
from [helper.py](helper.py).

```python
from helper import plot_spline_2d, plot_tangents_2d
```

```python
class UniformHermiteSegment:

    grid = 0, 1

    def __init__(self, control_values):
        self.coeffs = sp.lambdify([], M_H.expr)() @ control_values

    def evaluate(self, t):
        t = np.expand_dims(t, -1)
        return t**[3, 2, 1, 0] @ self.coeffs
```

<div class="alert alert-info">

Note

The `@` operator is used here to do
[NumPy's matrix multiplication](https://numpy.org/doc/stable/reference/generated/numpy.matmul.html).

</div>

```python
vertices = [0, 0], [5, 1]
tangents = [2, 3], [0, -2]
```

```python
s = UniformHermiteSegment([*vertices, *tangents])
```

```python
plot_spline_2d(s, chords=False)
plot_tangents_2d(tangents, vertices)
```

## Relation to Bézier Splines

Above, we were using two positions (start and end)
and two tangent vectors (at those same two positions) as control values:

```python
control_values_H.name
```

What about using four positions (and no tangent vectors) instead?

Let's use the point $\boldsymbol{\tilde{x}}_4$ as a "handle"
(connected to $\boldsymbol{x}_4$) that controls the tangent vector.
Same for $\boldsymbol{\tilde{x}}_5$ (connected to $\boldsymbol{x}_5$).

And since the tangents looked unwieldily long in the plot above
(compared to the effect they have on the shape of the curve),
let's put the handles only at a third of the length of the tangents,
shall we?

\begin{align*}
\tilde{\boldsymbol{x}}_4
&=
\boldsymbol{x}_4 + \frac{\dot{\boldsymbol{x}}_4}{3}
\\
\tilde{\boldsymbol{x}}_5
&=
\boldsymbol{x}_5 - \frac{\dot{\boldsymbol{x}}_5}{3}
\end{align*}

```python
control_values_B = NamedMatrix(sp.Matrix([
    x4.name,
    sp.Symbol('xtildebm4'),
    sp.Symbol('xtildebm5'),
    x5.name,
]), sp.Matrix([
    x4.name,
    x4.name + xd4.name / 3,
    x5.name - xd5.name / 3,
    x5.name,
]))
control_values_B
```

Now let's try to come up with a matrix
that transforms our good old Hermite control values
into our new control points.

```python
M_HtoB = NamedMatrix(r'{M_\text{H$\to$B}}', 4, 4)
```

```python
NamedMatrix(control_values_B.name, M_HtoB.name * control_values_H.name)
```

We can immediately read the matrix coefficients
off the previous expression.

```python
M_HtoB.expr = sp.Matrix([[expr.coeff(cv) for cv in control_values_H.name]
                         for expr in control_values_B.expr])
M_HtoB.pull_out(sp.S.One / 3)
```

```python
print(_.expr)
```

The inverse of this matrix transforms our new control points
into Hermite control values:

```python
M_BtoH = NamedMatrix(r'{M_\text{B$\to$H}}', M_HtoB.I.expr)
M_BtoH
```

```python
print(_.expr)
```

When we combine $M_H$ with this new matrix,
we get a matrix which leads us to a new set of basis polynomials
associated with the 4 control points.

```python
M_B = NamedMatrix(r'{M_\text{B}}', M_H.name * M_BtoH.name)
M_B
```

```python
M_B = M_B.subs_symbols(M_H, M_BtoH).doit()
M_B
```

```python
b_B = NamedMatrix(r'{b_\text{B}}', b_monomial * M_B.expr)
b_B.T
```

```python
plot_basis(
    *b_B.expr,
    labels=sp.symbols('xbm_i xtildebm_i xtildebm_i+1 xbm_i+1'))
```

Those happen to be the cubic *Bernstein* polynomials and
it turns out that we just invented *Bézier* curves!
See [the section about Bézier splines](bezier.ipynb)
for more information about them.

We chose the additional control points to be located
at $\frac{1}{3}$ of the tangent vector.
Let's quickly visualize this
using the example from above and $M_\text{H$\to$B}$:

```python
points = sp.lambdify([], M_HtoB.expr)() @ [*vertices, *tangents]
```

```python
import matplotlib.pyplot as plt
```

```python
plot_spline_2d(s, chords=False)
plot_tangents_2d(tangents, vertices)
plt.scatter(*points.T, marker='X', color='black')
plt.annotate(r'$\quad\tilde{\bf{x}}_0$', points[1])
plt.annotate(r'$\tilde{\bf{x}}_1\quad$', points[2], ha='right');
```


```python
%matplotlib inline
```

# Wasserstein unmixing with PyTorch

In this example we estimate mixing parameters from distributions that minimize
the Wasserstein distance. In other words we suppose that a target
distribution $\mu^t$ can be expressed as a weighted sum of source
distributions $\mu^s_k$ with the following model:

\begin{align}\mu^t = \sum_{k=1}^K w_k\mu^s_k\end{align}

where $\mathbf{w}$ is a vector of size $K$ and belongs in the
distribution simplex $\Delta_K$.

In order to estimate this weight vector we propose to optimize the Wasserstein
distance between the model and the observed $\mu^t$ with respect to
the vector. This leads to the following optimization problem:

\begin{align}\min_{\mathbf{w}\in\Delta_K} \quad W \left(\mu^t,\sum_{k=1}^K w_k\mu^s_k\right)\end{align}

This minimization is done in this example with a simple projected gradient
descent in PyTorch. We use the automatic backend of POT that allows us to
compute the Wasserstein distance with :any:`ot.emd2` with
differentiable losses.

```python
# Author: Remi Flamary <remi.flamary@polytechnique.edu>
#
# License: MIT License

# sphinx_gallery_thumbnail_number = 2

import numpy as np
import matplotlib.pylab as pl
import ot
import torch
```

## Generate data

```python
nt = 100
nt1 = 10  #

ns1 = 50
ns = 2 * ns1

rng = np.random.RandomState(2)

xt = rng.randn(nt, 2) * 0.2
xt[:nt1, 0] += 1
xt[nt1:, 1] += 1

xs1 = rng.randn(ns1, 2) * 0.2
xs1[:, 0] += 1
xs2 = rng.randn(ns1, 2) * 0.2
xs2[:, 1] += 1

xs = np.concatenate((xs1, xs2))

# Sample reweighting matrix H
H = np.zeros((ns, 2))
H[:ns1, 0] = 1 / ns1
H[ns1:, 1] = 1 / ns1
# each columns sums to 1 and has weights only for samples form the
# corresponding source distribution

M = ot.dist(xs, xt)
```

## Plot data

```python
pl.figure(1)
pl.scatter(xt[:, 0], xt[:, 1], label='Target $\mu^t$', alpha=0.5)
pl.scatter(xs1[:, 0], xs1[:, 1], label='Source $\mu^s_1$', alpha=0.5)
pl.scatter(xs2[:, 0], xs2[:, 1], label='Source $\mu^s_2$', alpha=0.5)
pl.title('Sources and Target distributions')
pl.legend()
```

## Optimization of the model wrt the Wasserstein distance

```python
# convert numpy arrays to torch tensors
H2 = torch.tensor(H)
M2 = torch.tensor(M)

# weights for the source distributions
w = torch.tensor(ot.unif(2), requires_grad=True)

# uniform weights for target
b = torch.tensor(ot.unif(nt))

lr = 2e-3  # learning rate
niter = 500  # number of iterations
losses = []  # loss along the iterations

# loss for the minimal Wasserstein estimator

def get_loss(w):
    a = torch.mv(H2, w)  # distribution reweighting
    return ot.emd2(a, b, M2)  # squared Wasserstein 2

for i in range(niter):

    loss = get_loss(w)
    losses.append(float(loss))

    loss.backward()

    with torch.no_grad():
        w -= lr * w.grad  # gradient step
        w[:] = ot.utils.proj_simplex(w)  # projection on the simplex

    w.grad.zero_()
```

## Estimated weights and convergence of the objective

```python
we = w.detach().numpy()
print('Estimated mixture:', we)

pl.figure(2)
pl.semilogy(losses)
pl.grid()
pl.title('Wasserstein distance')
pl.xlabel("Iterations")
```

## Ploting the reweighted source distribution

```python
pl.figure(3)

# compute source weights
ws = H.dot(we)

pl.scatter(xt[:, 0], xt[:, 1], label='Target $\mu^t$', alpha=0.5)
pl.scatter(xs[:, 0], xs[:, 1], color='C3', s=ws * 20 * ns, label='Weighted sources $\sum_{k} w_k\mu^s_k$', alpha=0.5)
pl.title('Target and reweighted source distributions')
pl.legend()
```


```python
import numpy as np
import pykat
import matplotlib.pyplot as plt
```

# Analytics

## Input - output (2x2 matrices)

**Quadrature picture, aka 2-photon formalism** (The one we use for the analytics)

In the frequency domain, the vacuum noise incident on the cavity can in the quadrature-picture be expressed as:

\begin{align}
\mathbf{a}(\Omega) = \begin{bmatrix}
    \hat{a}_1(\Omega) \\
    \hat{a}_2(\Omega)
\end{bmatrix}
\end{align}

The field reflected off an overcoupled cavity ($r_2 = 1$)  is given by:

\begin{align}
\mathbf{b}(\Omega) = \begin{bmatrix}
    \hat{b}_1(\Omega) \\
    \hat{b}_2(\Omega)
\end{bmatrix} = \mathbf{M}\, \mathbf{a}(\Omega)
\end{align}

where the matric describing the reflection off the cavity is:

\begin{align}
\mathbf{M} &= -\mathbf{R} + \mathbf{T} ( \mathbf{I} - \mathbf{N}\, \mathbf{S}\, \mathbf{N}\, \mathbf{R} )^{-1} \mathbf{N}\, \mathbf{S}\, \mathbf{N}\, \mathbf{T} \\ \\
\mathbf{R} &- \text{Reflection off input mirror} \\
\mathbf{T} &- \text{Transmission through input mirror} \\
\mathbf{S} &- \text{Roundtrip propagation} \\
\mathbf{I} &- \text{Identity matrix} \\\\
\end{align}

and the matrix for the non-linear crystal is:

\begin{align}
\mathbf{N} &\equiv \mathbf{N}(r, \phi) = \begin{bmatrix} \cosh(r) + \sinh(r) \cos(2\phi) & \sinh(r) \sin(2\phi) \\
                              \sinh(r) \sin(2\phi) & \cosh(r) - \sinh(r) \cos(2\phi)
                     \end{bmatrix}
\end{align}

The homodyne readout power-fluctuations operator is given by:

\begin{align}
\hat{P}(\Omega) =  A_1 \big(\hat{b}_1(\Omega) + \hat{b}_1^\dagger(\Omega) \big) + A_2 \big(\hat{b}_2(\Omega) + \hat{b}_2^\dagger(\Omega) \big)
\end{align}

where $A_1 = \sqrt{P_0}\cos(\theta)$ and $A_2 = \sqrt{P_0}\sin(\theta)$, $P_0$ is the local oscillator power, and $\theta$ is the local oscillator phase. The power spectral density (PSD) and amplitude spectral density (ASD) of the quantum noise becomes:

\begin{align}
\text{PSD} &= ⟨0|\, \hat{P}(\Omega) \hat{P}^\dagger(\Omega) \,| 0 ⟩ =  2 P_0 \hbar \omega_0 \begin{bmatrix} \cos \theta & \sin\theta \end{bmatrix} \mathbf{M} \,\mathbf{M}^\dagger \begin{bmatrix} \cos \theta \\ \sin\theta \end{bmatrix} \\
\text{ASD} &= \sqrt{\text{PSD}}
\end{align}

**Alternatively** Express the power fluctuation operator as:

\begin{align}
\hat{P}(\Omega) =  \mathbf{A}^\text{T} \left(\mathbf{b} + \left(\mathbf{b}^\dagger \right)^\text{T} \right)
\end{align}

where $\mathbf{A}^\text{T} = \sqrt{P_0}\begin{bmatrix} \cos \theta & \sin \theta \end{bmatrix}$, and then

\begin{align}
\text{PSD} &= ⟨0|\, \hat{P}(\Omega) \hat{P}^\dagger(\Omega) \,| 0 ⟩ =  2 \hbar \omega_0 \mathbf{A}^\text{T} \mathbf{M} \,\mathbf{M}^\dagger \mathbf{A} \\
\text{ASD} &= \sqrt{\text{PSD}}
\end{align}

**Sideband picture, aka 1-photon formalism** (Used internally by Finesse)

\begin{align}
\mathbf{N} &\equiv \mathbf{N}(r, \phi) = \begin{bmatrix} \cosh(r) & \text{e}^{2i\phi} \sinh(r) \\
                              \text{e}^{-2i\phi} \sinh(r) & \cosh(r)
                     \end{bmatrix}
\end{align}

\begin{align}
\begin{bmatrix}
\hat{b}_{\omega_0 + \Omega} \\
\hat{b}^\dagger_{\omega_0 - \Omega}
\end{bmatrix} = \mathbf{N} \begin{bmatrix}
\hat{a}_{\omega_0 + \Omega} \\
\hat{a}^\dagger_{\omega_0 - \Omega}
\end{bmatrix}
\end{align}

# Functions

Functions for doing the analytics:

```python
# Speed of light
c = 299792458.0
# Planck's reduced constant
hbar = 1.0545718e-34

# Propagation through vacuum
def space(phi, L, W=0):
    return np.exp(-1j*W*L/c)*np.array([[np.cos(phi), -np.sin(phi)],
                                       [np.sin(phi), np.cos(phi)]] )

# Transmission
def trans(t):
    return np.eye(2)*t

# Reflection
def refl(r, neg=False):
    if neg:
        r *= -1
    return np.eye(2)*r

# Nonlinear element
def nle(r, phi):
    '''
    Nonlinear element.

    r    - Squeeze exponent.
    phi  - Squeeze angle.
    '''
    return np.array([[np.cosh(r) + np.sinh(r)*np.cos(2*phi), np.sinh(r)*np.sin(2*phi)],
                     [np.sinh(r)*np.sin(2*phi), np.cosh(r) - np.sinh(r)*np.cos(2*phi)]])

# Reflection off cavity with internal NLE.
def r_cav(r1, t1, r2, t2, phi, L, r_sq=0, phi_sq=0, W=0):
    # Must to allow second source through end mirror to be correct for quantum noise.
    # Thus, currenlty incorrect for quantum noise calculations if r2 != 1.

    N = nle(r_sq, phi_sq)
    R1p = refl(r1)
    R1n = refl(r1, neg=True)
    T1 = trans(t1)
    R2 = refl(r2)
    S = space(phi, L, W)

    # Roundtrip.
    M = np.eye(2) - N@S@R2@S@N@R1p
    M = np.linalg.inv(M)

    # M_rt = eye(2) - N*S*R2*S*N*R1p
    # M_rt = M_rt.inv()

    return R1n + T1@M@N@S@R2@S@N@T1

def nle_in_cav(r1, t1, r2, t2, phi, L, r_sq, phi_sq, P_LO, F):
    '''
    Computes the ASD of quantum noise reflected off a cavity with an internal NLE.

    r1      - Input mirror amplitude reflection
    t1      - Input mirror amplitude transmission
    r2      - End mirror amplitude reflection
    t2      - End mirror amplitude transmission
    phi     - Cavity tuning [deg]
    L       - Cavity length [m]
    r_sq    - Squeeze exponent.
    phi_sq  - Squeeze angle [deg]
    P_LO    - Power of the local oscillator [W].
    F       - Frequency offset at which to compute quantum noise [Hz].
    '''

    hbar = 1.0545718e-34
    c0 = 299792458.0

    lam0 = 1.064e-6
    f0 = c0/lam0
    w0 = 2.0*np.pi*f0
    W = 2.0*np.pi*F

    phi *= np.pi/180.0
    phi_sq *= np.pi/180.0

    # Fields
    N = 5
    a = 0
    b = 2
    c = 4
    d = 6
    e = 8

    SRC = np.zeros([2*N])
    SRC[a:a+2] = 1
    SRC[e:e+2] = 1

    SRC *= np.sqrt(hbar*w0/2.0)

    # The noise matrix
    N_Mat = np.eye(2*N, dtype=complex)*-1

    # a to a (source)
    N_Mat[a:a+2, a:a+2] = np.eye(2)

    # a to b
    N_Mat[b:b+2, a:a+2] = r_cav(r1, t1, r2, t2, phi, L, r_sq, phi_sq, W)

    # b to c
    N_Mat[c:c+2, b:b+2] = refl(np.sqrt(2))

    # b to d
    N_Mat[d:d+2, b:b+2] = trans(np.sqrt(2))

    # e to c
    N_Mat[c:c+2, e:e+2] = trans(np.sqrt(2))

    # e to d
    N_Mat[d:d+2, e:e+2] = refl(np.sqrt(2), neg=True)

    # e to e (source)
    N_Mat[e:e+2, e:e+2] = np.eye(2)

    N = np.linalg.inv(N_Mat)

    # Coherent field
    C = np.sqrt(P_LO)/np.sqrt(2.0)
    D = -np.sqrt(P_LO)/np.sqrt(2.0)

    # ASD readout for both quadratures
    Q1 = np.sqrt(np.sum(np.abs(N[c,:]*SRC*C - N[d,:]*SRC*D)**2))
    Q2 = np.sqrt(np.sum(np.abs(N[c+1,:]*SRC*C - N[d+1,:]*SRC*D)**2))

    return Q1, Q2

# Photodetctor
def pd(A):
    # A is the complex amplitue
    return np.abs(A)**2

# Amplitude detector
def ad(A):
    return A[0] - 1j*A[1]

# Homodyne readout (not finished. Do not use.)
def HD_readout(M, idx1, idx2, A, a):
    '''
    A    - Coherent part of the local oscillator.
    a    - Noise part of the local oscillator
    b    - Noise part to be measured

    Returns
    --------
    P_HD   - Coherent power
    p_hd   - Noise power operator
    '''

    # From cav
    # T1 = trans(1.0/np.sqrt(2.0))
    # R1 = refl(1.0/np.sqrt(2.0))

    # From LO
    T = trans(1.0/sqrt(2.0))
    R = refl(1.0/sqrt(2.0), neg=True)

    # Coherent
    C = T*A
    D = R*A

    # Readout, coherent

    P_C = C[0]*C[0].conjugate() + C[1]*C[1].conjugate()
    P_D = D[0]*D[0].conjugate() + D[1]*D[1].conjugate()

    P_HD = P_C - P_D

    # Readout, quantum noise
    p_c = C[0]*Dagger(c[0]) + C[0].conjugate()*c[0] + C[1]*Dagger(c[1]) + C[1].conjugate()*c[1]
    p_d = D[0]*Dagger(d[0]) + D[0].conjugate()*d[0] + D[1]*Dagger(d[1]) + D[1].conjugate()*d[1]

    p_hd = p_c - p_d

    return P_HD, p_hd

# Power Spectral Density. Not Finished. Do not use.
def PSD(p):
    return (p*Dagger(p) + Dagger(p)*p)/2
```

# Cavity response test

No NLE in the cavity. Just testing the "analytic" equations vs. Finesse.

```python
# Controller (setting the parameters here)
# -----------------------------------------

# Input laser power
P0 = 10
# Input laser phase
phi0 = 0

# Power reflection and transmission coeffiecients
R1 = 0.81
T1 = 1.0-R1

R2 = 0.9
T2 = 1.0-R2

# NLE-parameters
r_sq = 0
phi_sq = 0

# Tuning of end mirror
phi = 0
# Cavity length
L = 1.0
# Frequency offset
F = 10e6
```

## Finesse model

```python
base = pykat.finesse.kat()
base.parse('''

l laser 1 0 0 n0
s s0 0 n0 nIM1

m IM 0.81 0.19 0 nIM1 nIM2
s sCav 1 nIM2 nEM1
m EM 1 0 0 nEM1 nEM2

pd P nIM1
ad A 0 nIM1

yaxis abs:deg
''')
```

```python
# Finesse run
kat = base.deepcopy()
kat.laser.P = P0
kat.laser.phase = phi0
kat.laser.f = F
kat.A.f = F

kat.IM.R = R1
kat.IM.T = T1
kat.EM.R = R2
kat.EM.T = T2

kat.EM.phi = phi
kat.sCav.L = L

# Sweeping the cavity tuning. Note that positive phi at the end mirror means shorter cavity.
kat.parse('xaxis EM phi lin -90 90 201')

out = kat.run()
P_f = out['P'].real
A_f = out['A']
```

## Analytics

```python
# Analytic
# ---------------

# Transforming parameters
phi0 *= np.pi/180

r1 = np.sqrt(R1)
t1 = np.sqrt(T1)
r2 = np.sqrt(R2)
t2 = np.sqrt(T2)

W = 2.0*np.pi*F

# Input field
E1 = np.sqrt(P0)*np.cos(phi0)
E2 = np.sqrt(P0)*np.sin(phi0)
# ... as vector
E = np.array([[E1],[E2]])

# Arrays for storing output power and complex amplitude
P_a = np.zeros(len(out.x))
A_a = np.zeros(len(out.x), dtype=complex)

for k,phi in enumerate(out.x):
    phi *= np.pi/180
    # Matrix describing reflection off cavity
    M = r_cav(r1, t1, r2, t2, -phi, L, r_sq=0, phi_sq=0, W=W)

    # Output field
    E2 = M @ E

    # Computing complex amplitude and power. Rotating 180 deg to make it match Finesse.
    A_a[k] = ad(E2)*np.exp(-1j*np.pi)
    P_a[k] = pd(A_a[k])
```

## Plotting

```python
fig = plt.figure(dpi=120, figsize=(9,6))
ax = fig.add_subplot(221)
ax.plot(out.x, P_f, label='Finesse')
ax.plot(out.x, P_a,'--', label='Analytic')

ax.legend()
ax.grid()
ax.set_yscale('linear')
ax.set_xlabel(r'Cavity tuning [deg]')
ax.set_ylabel(r'Reflected power [W]')
ax.set_title('Frequency offset: {:.1f} MHz'.format(F/1e6))

# Magnitude
ax = fig.add_subplot(222)
ax.plot(out.x, np.abs(A_f), label='Finesse')
ax.plot(out.x, np.abs(A_a),'--', label='Analytic')

ax.legend()
ax.grid()
ax.set_yscale('linear')
# ax.set_xlabel(r'Cavity tuning [deg]')
ax.set_ylabel(r'Reflected magnitude [$\sqrt{\mathrm{W}}$]')

# Phase
ax = fig.add_subplot(224)
ax.plot(out.x, np.angle(A_f), label='Finesse')
ax.plot(out.x, np.angle(A_a),'--', label='Analytic')

ax.legend()
ax.grid()
ax.set_yscale('linear')
ax.set_xlabel(r'Cavity tuning [deg]')
ax.set_ylabel(r'Phase [deg]')

plt.show(fig)
```

# Crystal in cavity

All parameters for the analytics are extracted from Finesse. Thus, to change parameters, alter the Finesse model below.

## Base Finesse model

```python
base = pykat.finesse.kat()
base.parse('''

l laser 1 0 10 n0
s s0 0 n0 nBS4

dbs FI nFIa nFIb nFIc nFId

s s1 0 nFIc nIM1

m IM 0.9 0.1 0 nIM1 nIM2
s sCav1 0 nIM2 nNLE1
# nle NLE {0} {1} nNLE1 nNLE2
s sCav 1 nNLE2 nEM1
m EM 1 0 0.0 nEM1 nEM2

s s2 0 nFId nBS1

bs BS 0.5 0.5 0 0 nBS1 nBS2 nBS3 nBS4

hd HD 180 nBS2 nBS3
qhd QHD 180 nBS2 nBS3

pd P1 nBS2
pd P2 nBS3

fsig sig 1

yaxis abs
''')
```

DBS

1 -- > 3

2 ---> 1

3 ---> 4

4 ---> 2

```python
# Extracting parameters that will stay the same from the Finesse-model
r1 = np.sqrt(base.IM.R.value)
t1 = np.sqrt(base.IM.T.value)
r2 = np.sqrt(base.EM.R.value)
t2 = np.sqrt(base.EM.T.value)
L = base.sCav.L.value

# r_sq = np.log(10**(x_sq/10.0))
```

## Sweeping cavity tuning. No squeeze.

Just for checking if numbers agree without squeezing.

```python
# Single-trip squeeze dB
x_sq = 0
# Squeeze angle
phi_sq = 0
# Frequency
F = 1e6

kat = base.deepcopy()

# Adding non-linear element
kat.parse('nle NLE {0} {1} nNLE1 nNLE2'.format(x_sq, phi_sq))

# Setting frequency
kat.signals.f = F

# Adding xaxis and running Finesse.
kat.noxaxis = False
kat.parse('xaxis EM phi lin -90 90 200')
out = kat.run()

# Extracting parameters from kat-instance (to be used in analytic runs)
F = kat.signals.f.value
P_LO = kat.laser.P.value
phi_LO = kat.laser.phase.value*np.pi/180.0
# phi = kat.IM.phi.value - kat.EM.phi.value
phis = -out.x

f0 = c/kat.lambda0
w0 = 2.0*np.pi*f0

r_sq = np.log(10**(x_sq/10.0))

# Running analytics (Big matrix)
# ------------------
# Converting from squeeze magnitude in dB (Finesse) to squeeze exponent r (analytics).
# Note that we use 10 instead of 20. Thus, using formula for power dB even though we want this in amplitude dB.

Q1 = np.zeros(len(out.x))
Q2 = np.zeros(len(out.x))

for k, phi in enumerate(phis):
    Q1[k], Q2[k] = nle_in_cav(r1, t1, r2, t2, phi, L, r_sq, phi_sq, P_LO, F)

Q = np.real(np.exp(-1j*phi_LO)*(Q1 + 1j*Q2))

# Running analytics 2 (2x2 Matrix)
A0 = np.sqrt(P_LO)
alpha = 2*np.sqrt(hbar*w0/2.0)

Q_theta = np.zeros(len(out.x), dtype=float)

for k, phi in enumerate(phis):
    M = r_cav(r1, t1, r2, t2, phi*np.pi/180.0, L, r_sq, phi_sq*np.pi/180.0, W=2.0*np.pi*F)

    Q_theta[k] = A0*alpha*np.sqrt((np.abs(M[0,:])**2).sum()*np.cos(phi_LO)**2 +
                                  (np.abs(M[1,:])**2).sum()*np.sin(phi_LO)**2 +
                                  2.0*np.cos(phi_LO)*np.sin(phi_LO)*(np.real(M[0,0].conj()*M[1,0]) +
                                                                      np.real(M[0,1].conj()*M[1,1])) )

```

```python
# Plotting

fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out.x, out['QHD'], label='Finesse')
# Note that the extra factor of sqrt(2) is needed to make a match.
# ax.plot(out.x, Q,'--', label='Analytic')
ax.plot(out.x, Q_theta,':', label='Analytic 2')

ax.legend()
ax.grid()
ax.set_yscale('log')
ax.set_xlabel(r'Cavity tuning [deg]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
ax.set_title('Frequency: {:.1e} Hz. Squeeze: {} dB'.format(F, x_sq))
plt.show(fig)

# Relative difference
print('Relative difference between Finesse and Analytic:   {:.1e}'
      .format( max((Q - out['QHD'])/out['QHD'])))
print('Relative difference between Finesse and Analytic 2: {:.1e}'
      .format( max((Q_theta - out['QHD'])/out['QHD'])))
print('Relative difference between Analytic and Analytic 2: {:.1e}'
      .format( max((Q - Q_theta)/Q_theta)))
```

Good, numbers agree. Then we go ahaed and switch on the NLE!

## Sweeping squeeze magnitude with cavity on resonance

```python
# Single-trip squeeze dB
x_sq = 0
# Squeeze angle
phi_sq = 33
# Frequency
F = 1e0
# Local oscillator phase
phi_LO = 12
# ------------------------------

kat = base.deepcopy()

# Adding non-linear element
kat.parse('nle NLE {0} {1} nNLE1 nNLE2'.format(x_sq, phi_sq))

# Setting frequency
kat.signals.f = F
kat.EM.phi = 12
kat.IM.phi = 0

# Setting local oscillator phase
kat.laser.phase = phi_LO

# Running once without squeezing
kat.noxaxis = True
out0 = kat.run()

# Adding xaxis and running Finesse.
kat.noxaxis = False
kat.parse('xaxis NLE r lin -1 1 400')

out = kat.run()

# Extracting parameters from kat-instance (to be used in analytic runs)
F = kat.signals.f.value
P_LO = kat.laser.P.value
phi_LO = -kat.laser.phase.value*np.pi/180.0
phi = kat.IM.phi.value - kat.EM.phi.value

f0 = c/kat.lambda0
w0 = 2.0*np.pi*f0

# Squeeze exponent (note 10 in stead of 20...)
r_sqs = np.log(10**(out.x/10.0))

# Running analytics (Big matrix)
# ------------------
# Converting from squeeze magnitude in dB (Finesse) to squeeze exponent r (analytics).
# Note that we use 10 instead of 20. Thus, using formula for power dB even though we want this in amplitude dB.

Q1 = np.zeros(len(out.x))
Q2 = np.zeros(len(out.x))

for k, r_sq in enumerate(r_sqs):
    Q1[k], Q2[k] = nle_in_cav(r1, t1, r2, t2, phi, L, r_sq, phi_sq, P_LO, F)

Q = np.real(np.exp(-1j*phi_LO)*(Q1 + 1j*Q2))

# Running analytics 2 (2x2 Matrix)
A0 = np.sqrt(P_LO)
alpha = 2*np.sqrt(hbar*w0/2.0)

A = np.array([[np.cos(phi_LO)],[np.sin(phi_LO)]])

Q_theta = np.zeros(len(out.x), dtype=float)

for k, r_sq in enumerate(r_sqs):
    M = r_cav(r1, t1, r2, t2, phi*np.pi/180.0, L, r_sq, phi_sq*np.pi/180.0, W=2.0*np.pi*F)

    Q_theta[k] = A0*alpha*np.sqrt((np.abs(M[0,:])**2).sum()*np.cos(phi_LO)**2 +
                                  (np.abs(M[1,:])**2).sum()*np.sin(phi_LO)**2 +
                                  2.0*np.cos(phi_LO)*np.sin(phi_LO)*(np.real(M[0,0].conj()*M[1,0]) +
                                                                      np.real(M[0,1].conj()*M[1,1])) )

    Q_theta[k] = A0*alpha*np.sqrt((A.T@M@M.T.conj()@A)[0,0])

r_sq = 0
M = r_cav(r1, t1, r2, t2, phi*np.pi/180.0, L, r_sq, -phi_sq*np.pi/180.0, W=2.0*np.pi*F)

Q0 = A0*alpha*np.sqrt((np.abs(M[0,:])**2).sum()*np.cos(phi_LO)**2 +
                              (np.abs(M[1,:])**2).sum()*np.sin(phi_LO)**2 +
                              2.0*np.cos(phi_LO)*np.sin(phi_LO)*(np.real(M[0,0].conj()*M[1,0]) +
                                                                  np.real(M[0,1].conj()*M[1,1])) )
```

```python
kat.laser.phase
```

```python
# Plotting

fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out.x, out['QHD'], label='Finesse')
# Note that the extra factor of sqrt(2) is needed to make a match.
ax.plot(out.x, Q,'--', label='Analytic')
ax.plot(out.x, Q_theta,':', label='Analytic 2')

ax.legend()
ax.grid()
ax.set_yscale('log')
ax.set_xlabel(r'Single-pass squeeze magnitude [dB]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
ax.set_title('Frequency: {:.1e} Hz. Cavity detuning: {} deg. Squeeze angle: {} deg'.format(F, phi, phi_sq))
plt.show(fig)

# Relative difference
print('Relative difference between Finesse and Analytic:   {:.1e}'
      .format( max((Q - out['QHD'])/out['QHD'])))
print('Relative difference between Finesse and Analytic 2: {:.1e}'
      .format( max((Q_theta - out['QHD'])/out['QHD'])))
print('Relative difference between Analytic and Analytic 2: {:.1e}'
      .format( max((Q - Q_theta)/Q_theta)))
```

```python
Q /out['QHD']
```

```python
# Plotting

fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out.x, 20*np.log10(out['QHD']/out0['QHD']), label='Finesse')
# Note that the extra factor of sqrt(2) is needed to make a match.
# ax.plot(out.x, Q1*np.sqrt(2),'--', label='Analytic')
ax.plot(out.x, 20*np.log10(Q_theta/Q0),':', label='Analytic 2')

ax.legend()
ax.grid()
ax.set_yscale('linear')
ax.set_xlabel(r'Single-pass squeeze magnitude [dB]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
ax.set_title('Frequency: {:.1e} Hz. Cavity detuning: {} deg. Squeeze angle: {} deg'.format(F, phi, phi_sq))
plt.show(fig)

```

```python
kat.IM.R
```

```python

```

```python

```

```python
# Adding xaxis and running Finesse.
kat = base.deepcopy()
kat.noxaxis = False
kat.parse('xaxis NLE r lin -1 1 400')
out = kat.run()

# Extracting some more parameters
F = kat.signals.f.value
P_LO = kat.laser.P.value
phi_LO = kat.laser.phase.value*np.pi/180.0
phi = kat.IM.phi.value - kat.EM.phi.value

f0 = c/kat.lambda0
w0 = 2.0*np.pi*f0
```

```python
# Running analytics (Big matrix)
# ------------------
# Converting from squeeze magnitude in dB (Finesse) to squeeze exponent r (analytics).
# Note that we use 10 instead of 20. Thus, using formula for power dB even though we want this in amplitude dB.
r_sqs = np.log(10**(out.x/10.0))

Q1 = np.zeros(len(out.x))
Q2 = np.zeros(len(out.x))

for k, r_sq in enumerate(r_sqs):
    Q1[k], Q2[k] = nle_in_cav(r1, t1, r2, t2, phi, L, r_sq, phi_sq, P_LO, F)
```

```python
# Running analytics 2 (2x2 Matrix)
A0 = np.sqrt(P_LO)
alpha = 2*np.sqrt(hbar*w0/2.0)

Q_theta = np.zeros(len(out.x), dtype=float)

for k, r_sq in enumerate(r_sqs):
    M = r_cav(r1, t1, r2, t2, phi*np.pi/180.0, L, r_sq, phi_sq*np.pi/180.0, W=2.0*np.pi*F)

    Q_theta[k] = A0*alpha*np.sqrt((np.abs(M[0,:])**2).sum()*np.cos(phi_LO)**2 +
                                  (np.abs(M[1,:])**2).sum()*np.sin(phi_LO)**2 +
                                  2.0*np.cos(phi_LO)*np.sin(phi_LO)*(np.real(M[0,0].conj()*M[1,0]) +
                                                                      np.real(M[0,1].conj()*M[1,1])) )

```

```python
# Plotting

fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out.x, out['QHD'], label='Finesse')
# Note that the extra factor of sqrt(2) is needed to make a match.
ax.plot(out.x, Q1*np.sqrt(2),'--', label='Analytic')
ax.plot(out.x, Q_theta,':', label='Analytic 2')

ax.legend()
ax.grid()
ax.set_yscale('log')
ax.set_xlabel(r'Single-pass squeeze magnitude [dB]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
ax.set_title('Frequency: {:.1e} Hz. Cavity detuning: {} deg'.format(F, phi))
plt.show(fig)
```

# Sweeping cavity tuning

With frequency and squeeze magnitude set to arbitrary values.

```python
# Single-trip squeeze dB
x_sq = 0.01
# Squeeze angle
phi_sq = 39
# Frequency
F = 1e9
# -------------------------

kat = base.deepcopy()

# Adding non-linear element
kat.parse('nle NLE {0} {1} nNLE1 nNLE2'.format(x_sq, phi_sq))

# Setting frequency
kat.signals.f = F

# Adding xaxis and running Finesse.
kat.noxaxis = False
kat.parse('xaxis EM phi lin -90 90 200')
out = kat.run()

# Extracting parameters from kat-instance (to be used in analytic runs)
F = kat.signals.f.value
P_LO = kat.laser.P.value
phi_LO = -kat.laser.phase.value*np.pi/180.0
# phi = kat.IM.phi.value - kat.EM.phi.value
phis = -out.x

f0 = c/kat.lambda0
w0 = 2.0*np.pi*f0

r_sq = np.log(10**(x_sq/10.0))

# Running analytics (Big matrix)
# ------------------
# Converting from squeeze magnitude in dB (Finesse) to squeeze exponent r (analytics).
# Note that we use 10 instead of 20. Thus, using formula for power dB even though we want this in amplitude dB.

Q1 = np.zeros(len(out.x))
Q2 = np.zeros(len(out.x))

for k, phi in enumerate(phis):
    Q1[k], Q2[k] = nle_in_cav(r1, t1, r2, t2, phi, L, r_sq, phi_sq, P_LO, F)

# Running analytics 2 (2x2 Matrix)
A0 = np.sqrt(P_LO)
alpha = 2*np.sqrt(hbar*w0/2.0)

A = np.array([[np.cos(phi_LO)],[np.sin(phi_LO)]])

Q_theta = np.zeros(len(out.x), dtype=float)

for k, phi in enumerate(phis):
    M = r_cav(r1, t1, r2, t2, phi*np.pi/180.0, L, r_sq, phi_sq*np.pi/180.0, W=2.0*np.pi*F)

    Q_theta[k] = A0*alpha*np.sqrt((np.abs(M[0,:])**2).sum()*np.cos(phi_LO)**2 +
                                  (np.abs(M[1,:])**2).sum()*np.sin(phi_LO)**2 +
                                  2.0*np.cos(phi_LO)*np.sin(phi_LO)*(np.real(M[0,0].conj()*M[1,0]) +
                                                                      np.real(M[0,1].conj()*M[1,1])) )

    #Q_theta[k] = A0*alpha*np.sqrt((A.T@M@M.T.conj()@A)[0,0])
```

```python
# Plotting

fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out.x, out['QHD'], label='Finesse')
# Note that the extra factor of sqrt(2) is needed to make a match.
#ax.plot(out.x, Q1*np.sqrt(2),'--', label='Analytic')
ax.plot(out.x, Q_theta,':', label='Analytic 2')

ax.legend()
ax.grid()
ax.set_yscale('log')
ax.set_xlabel(r'Cavity tuning [deg]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
ax.set_title('Frequency: {:.1e} Hz. Squeeze: {} dB. Angle: {} deg'.format(F, x_sq, phi_sq))
plt.show(fig)

# Relative difference
print('Relative difference between Finesse and Analytic:   {:.1e}'
      .format( max((Q1*np.sqrt(2) - out['QHD'])/out['QHD'])))
print('Relative difference between Finesse and Analytic 2: {:.1e}'
      .format( max((Q_theta - out['QHD'])/out['QHD'])))
print('Relative difference between Analytic and Analytic 2: {:.1e}'
      .format( max((Q1*np.sqrt(2) - Q_theta)/Q_theta)))
```

```python

```

```python

```

```python
kat = base.deepcopy()
kat.noxaxis = False
kat.parse('xaxis EM phi lin -10 10 400')
out = kat.run()

# Extracting parameters
F = kat.signals.f.value
P_LO = kat.laser.P.value
phi_LO = kat.laser.phase.value
r_sq = np.log(10**(x_sq/10.0))
```

```python
# Running analytics
phis = -out.x
Q1 = np.zeros(len(out.x))
Q2 = np.zeros(len(out.x))

# ASDs for the two quadratures. Lopoping over the different tunings used in the Finesse run
for k, phi in enumerate(out.x):
    # Note that we set -phi, as positive phi gives shorter cavity in Finesse, while it
    # gives longer cavity in the analytics.
    Q1[k], Q2[k] = nle_in_cav(r1, t1, r2, t2, -phi, L, r_sq, phi_sq, P_LO, F)
```

```python
fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out.x, out['QHD'], label='Finesse')
ax.plot(out.x, Q1*np.sqrt(2),'--', label='Analytic')
# ax.plot(out.x, Q_theta*np.sqrt(2),'--', label='Analytic 2')

ax.legend()
ax.grid()
ax.set_yscale('log')
ax.set_xlabel(r'Cavity tuning [deg]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
ax.set_title('Frequency: {:.1e} Hz. SQZ magnitude: {} dB'.format(F, x_sq))
plt.show(fig)
```

```python

```

# Testing SQZ vs. NLE

```python
base_NLE = pykat.finesse.kat()
base_NLE.parse('''

l laser 1 0 0 n0
s s0 0 n0 nBS4

s s1 0 nQN nNLE1

s s2 0 nNLE2 nBS1

bs BS 0.5 0.5 0 0 nBS1 nBS2 nBS3 nBS4

qhd QHD 180 nBS2 nBS3

fsig sig 1

yaxis abs

printnoises
''')

base_SQ = pykat.finesse.kat()
base_SQ.parse('''

l laser 1 0 0 n0
s s0 0 n0 nBS4

sq SQZ 0 0 0 nSQZ

s s2 0 nSQZ nBS1

bs BS 0.5 0.5 0 0 nBS1 nBS2 nBS3 nBS4

qhd QHD 180 nBS2 nBS3

fsig sig 1

yaxis abs
''')
```

```python
x_sq = 1
phi_sq = 0

kat_sq = base_SQ.deepcopy()
kat_nle = base_NLE.deepcopy()

# Setting squeeze magnitude and angle
kat_sq.SQZ.db = x_sq
kat_sq.SQZ.angle = phi_sq
kat_nle.parse('nle NLE {0} {1} nNLE1 nNLE2'.format(x_sq, phi_sq))

# Adding x-axis
kat_sq.parse('xaxis SQZ r lin 0 10 100')
kat_nle.parse('xaxis NLE r lin 0 10 100')

out_sq = kat_sq.run()
out_nle = kat_nle.run()
```

```python
fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out_sq.x, out_sq['QHD'], label='SQ')
ax.plot(out_nle.x, out_nle['QHD'],'--', label='NLE')

ax.legend()
ax.grid()
ax.set_yscale('log')
ax.set_xlabel(r'Squeeze magnitude [dB]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
# ax.set_title('Frequency: {:.1e} Hz. SQZ magnitude: {} dB'.format(F, x_sq))
plt.show(fig)
```

```python
fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out_sq.x, out_sq['QHD']/out_sq['QHD'][0], label='SQ')
ax.plot(out_nle.x, out_nle['QHD']/out_sq['QHD'][0],'--', label='NLE')

ax.legend()
ax.grid()
ax.set_yscale('linear')
ax.set_ylim(0,1)
ax.set_xlabel(r'Input squeeze parameter magnitude [dB]')
ax.set_ylabel(r'Relative quantum noise [N(0 dB)]')
# ax.set_title('Frequency: {:.1e} Hz. SQZ magnitude: {} dB'.format(F, x_sq))
plt.show(fig)
```

```python
fig = plt.figure(dpi=120)
ax = fig.add_subplot(111)
ax.plot(out_sq.x, 20*np.log10(out_sq['QHD']/out_sq['QHD'][0]), label='SQ')
ax.plot(out_nle.x, 20*np.log10(out_nle['QHD']/out_sq['QHD'][0]),'--', label='NLE')

ax.legend()
ax.grid()
ax.set_yscale('linear')
ax.set_xlabel(r'Squeeze magnitude [dB]')
ax.set_ylabel(r'Quantum noise [W/$\sqrt{\mathrm{Hz}}$]')
# ax.set_title('Frequency: {:.1e} Hz. SQZ magnitude: {} dB'.format(F, x_sq))
plt.show(fig)
```

# Testing with signal

## vs. frequency

```python
base = pykat.finesse.kat()
base.parse('''

l laser 1 0 0 n0
s s0 0 n0 nFIa

dbs FI nFIa nFIb nFIc nFId

s s1 1 nFIc nNLE1

s s2 0 nNLE2 nM1

m M 1 0 0 nM1 nM2

pd1 S1 $fs nFId

pd P0 nFId

ad A1 $mfs nFId
ad A2 $fs nFId

fsig sig M phase 1 0

yaxis abs:deg
''')

x_sq = 10
phi_sq = 15

kat = base.deepcopy()
kat.parse('nle NLE {0} {1} nNLE1 nNLE2'.format(x_sq, phi_sq))
kat.parse('xaxis sig f log 1 1G 200')

out = kat.run()
out.plot()
```

```python
fig = plt.figure(dpi=120)

ax = fig.add_subplot(211)
ax.plot(out.x, np.abs(out['A1']), label='Lower sideband')
ax.plot(out.x, np.abs(out['A2']), '--', label='Upper sideband')

ax.grid()
ax.set_yscale('linear')
ax.set_xscale('log')

ax.set_ylabel(r'Sideband amplitude')
ax.legend()

ax = fig.add_subplot(212)
ax.plot(out.x, np.angle(out['A1'])*180/np.pi, label='Lower sideband')
ax.plot(out.x, np.angle(out['A2'])*180/np.pi, '--', label='Upper sideband')

ax.grid()
ax.set_ylabel(r'Sideband phase')
ax.legend()
ax.set_xscale('log')

ax.set_xlabel(r'Frequency [Hz]')
# ax.set_title('Frequency: {:.1e} Hz. SQZ magnitude: {} dB'.format(F, x_sq))
plt.show(fig)
```

## vs squeeze magnitude

```python
base = pykat.finesse.kat()
base.parse('''

l laser 1 0 0 n0
s s0 0 n0 nFIa

dbs FI nFIa nFIb nFIc nFId

s s1 100 nFIc nNLE1

s s2 0 nNLE2 nM1

m M 1 0 0 nM1 nM2

pd1 S1 $fs nFId

pd P0 nFId

ad A1 $mfs nFId
ad A2 $fs nFId

fsig sig M phase 100 0

yaxis abs:deg
''')

x_sq = 10
phi_sq = 0

kat = base.deepcopy()
kat.parse('nle NLE {0} {1} nNLE1 nNLE2'.format(x_sq, phi_sq))
kat.parse('xaxis NLE r lin 0 10 200')

out = kat.run()
out.plot()
```

```python
fig = plt.figure(dpi=120)

ax = fig.add_subplot(211)
ax.plot(out.x, np.abs(out['A1']), label='Lower sideband')
ax.plot(out.x, np.abs(out['A2']), '--', label='Upper sideband')

ax.grid()
ax.set_yscale('linear')
ax.set_ylabel(r'Sideband amplitude')
ax.legend()

ax = fig.add_subplot(212)
ax.plot(out.x, np.angle(out['A1'])*180/np.pi, label='Lower sideband')
ax.plot(out.x, np.angle(out['A2'])*180/np.pi, '--', label='Upper sideband')

ax.grid()
ax.set_ylabel(r'Sideband phase')
ax.legend()

ax.set_xlabel(r'Squeeze magnitude [dB]')
# ax.set_title('Frequency: {:.1e} Hz. SQZ magnitude: {} dB'.format(F, x_sq))
plt.show(fig)
```

```python
fig = plt.figure(dpi=120)

ax = fig.add_subplot(211)
ax.plot(out.x, np.abs(out['A1'])/np.abs(out['A1'][0]), label='Lower sideband')
ax.plot(out.x, np.abs(out['A2'])/np.abs(out['A2'][0]), '--', label='Upper sideband')

ax.grid()
ax.set_yscale('linear')
ax.set_ylabel(r'Relative sideband amplitude')
ax.legend()

ax = fig.add_subplot(212)
ax.plot(out.x, 20*np.log10(np.abs(out['A1'])/np.abs(out['A1'][0])), label='Lower sideband')
ax.plot(out.x, 20*np.log10(np.abs(out['A2'])/np.abs(out['A2'][0])), '--', label='Upper sideband')

ax.grid()
ax.set_ylabel(r'Relative sideband amplitude [dB]')
ax.legend()

ax.set_xlabel(r'Squeeze magnitude [dB]')
# ax.set_title('Frequency: {:.1e} Hz. SQZ magnitude: {} dB'.format(F, x_sq))
plt.show(fig)
```

```python

```

```python

```

```python
# Without NLE

kat = pykat.finesse.kat()
kat.parse('''

l laser 1 0 20 n0
s s0 0 n0 nFIa

dbs FI nFIa nFIb nFIc nFId

s s1 0 nFIc nM1

m M 1 0 0 nM1 nM2

pd1 S1 $fs nFId

pd P0 nFId

ad A1 $fs nFId

fsig sig M phase 1 0

yaxis abs:deg
xaxis sig f log 1 1G 200
''')
out = kat.run()
out.plot()
```

```python

```

```python
out['S1']
```


```python
from sympy import *
```

```python
init_session()
```

    IPython console for SymPy 1.0 (Python 2.7.12-64-bit) (ground types: gmpy)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at http://docs.sympy.org/1.0/

```python
t, P, e_r, e_d, delta_e, rho_e, g_e, i_r, i_d, delta_i, rho_i, g_i, b = symbols('t P \\tau_{er} \\tau_{ed} \\delta_e \\rho_e \\bar{g}_e \\tau_{ir} \\tau_{id} \\delta_i \\rho_i \\bar{g}_i \\beta', positive=True, real=True)
```

```python
C = g_e*(P*(tau_i/(tau_i-1))*exp((t-delta_i)/i_r)*(1 - exp(-(t-delta_i)/(i_r*(1+(1/tau_i))))) - (tau_e/(tau_e-1))*exp(t/e_r)*(1 - exp(-t/(e_r*(1+(1/tau_e))))))
```

```python
C
```

```python
C_prime = diff(C,t)
```

```python
C_prime
```

```python
C_check = C_prime.subs({P:2, tau_i:10, tau_e:4, delta_i:1/g_e, i_r : 2.8, e_r: 5, t: 4})
```

```python
simplify(C_check)
```

```python
p = plot(C_check, (g_e,0,20))
#p = plot_implicit(C_prime, (t,0,10), (g_e,0,0.5))
```

```python
p.show()
```

```python
p.show()
```

```python

```


# A simple demonstration of coregionalization

This notebook shows how to construct a multi-output GP model using GPflow. We will consider a regression problem for functions $f: \mathbb{R}^D \rightarrow \mathbb{R}^P$. We assume that the dataset is of the form $(X_1, f_1), \dots, (X_P, f_P)$, that is, we do not necessarily observe all the outputs for a particular input location (for cases where there are fully observed outputs for each input, see [Multi-output Gaussian processes in GPflow](./multioutput.ipynb) for a more efficient implementation).

For this problem, we model $f$ as a *coregionalized* Gaussian process, which assumes a kernel of the form:

$$\textrm{cov}(f_i(X), f_j(X^\prime)) = k(X, X^\prime) \cdot B[i, j].$$

The covariance of the $i$th function at $X$ and the $j$th function at $X^\prime$ is a kernel applied at $X$ and $X^\prime$, times the $(i, j)$th entry of a positive definite $P \times P$ matrix $B$. This is known as the **intrinsic model of coregionalization (ICM)** (Bonilla and Williams, 2008).

To make sure that B is positive-definite, we parameterize it as:

$$B = W W^\top + \textrm{diag}(\kappa).$$

To build such a model in GPflow, we need to perform the following two steps:

 * Create the kernel function defined previously, using the `Coregion` kernel class.
 * Augment the training data X with an extra column that contains an integer index to indicate which output an observation is associated with. This is essential to make the data work with the `Coregion` kernel.

```python
import gpflow
import tensorflow as tf
import numpy as np
import matplotlib
%matplotlib inline

from gpflow.ci_utils import ci_niter

matplotlib.rcParams['figure.figsize'] = (12, 6)
plt = matplotlib.pyplot
np.random.seed(123)
```

## Data preparation
We start by generating some training data to fit the model with. For this example, we choose the following two correlated functions for our outputs:
$$
\begin{align}
y_1 &= \sin(6x) + \epsilon_1, \qquad \epsilon_1 \sim \mathcal{N}(0, 0.009) \\
y_2 &= \sin(6x + 0.7) + \epsilon_2, \qquad \epsilon_2 \sim \mathcal{N}(0, 0.01) \\
\end{align}
$$

```python
# make a dataset with two outputs, correlated, heavy-tail noise. One has more noise than the other.
X1 = np.random.rand(100, 1) # Observed locations for first output
X2 = np.random.rand(50, 1) * 0.5 # Observed locations for second output

Y1 = np.sin(6*X1) + np.random.randn(*X1.shape) * 0.03
Y2 = np.sin(6*X2+ 0.7) + np.random.randn(*X2.shape) * 0.1

plt.figure(figsize=(8, 4))
plt.plot(X1, Y1, 'x', mew=2)
plt.plot(X2, Y2, 'x', mew=2);
```

## Data formatting for the coregionalized model
We add an extra column to our training dataset that contains an index that specifies which output is observed.

```python
# Augment the input with ones or zeros to indicate the required output dimension
X_augmented = np.vstack((np.hstack((X1, np.zeros_like(X1))), np.hstack((X2, np.ones_like(X2)))))

# Augment the Y data
Y_augmented = np.vstack((np.hstack((Y1, np.zeros_like(Y1))), np.hstack((Y2, np.ones_like(Y2)))))
```

## Building the coregionalization kernel
We build a coregionalization kernel with the Matern 3/2 kernel as the base kernel. This acts on the leading ([0]) data dimension of the augmented X values. The `Coregion` kernel indexes the outputs, and acts on the last ([1]) data dimension (indices) of the augmented X values. To specify these dimensions, we use the built-in `active_dims` argument in the kernel constructor. To construct the full multi-output kernel, we take the product of the two kernels (for a more in-depth tutorial on kernel combination, see [Manipulating kernels](./kernels.ipynb)).

```python
output_dim = 2 # Number of outputs
rank = 1 # Rank of W

# Base kernel
k = gpflow.kernels.Matern32(active_dims=[0])

# Coregion kernel
coreg = gpflow.kernels.Coregion(output_dim=output_dim, rank=rank, active_dims=[1])
coreg.W.assign(np.random.rand(output_dim, rank))  # Initialize the W matrix to break symmetry

kern = k * coreg
```

**Note:** By default, the `W` matrix is initialized with zeros; however, this is a saddle point in the objective, so the value of `W` is not optimized to fit the data. Hence, re-initializing the matrix to random entries should give a more accurate result.

## Constructing the model
The final element in building the model is to specify the likelihood for each output dimension. To do this, use the `SwitchedLikelihood` object in GPflow.

```python
# This likelihood switches between Gaussian noise with different variances for each f_i:
lik = gpflow.likelihoods.SwitchedLikelihood([gpflow.likelihoods.Gaussian(), gpflow.likelihoods.Gaussian()])

# now build the GP model as normal
m = gpflow.models.VGP((X_augmented, Y_augmented), kernel=kern, likelihood=lik, num_latent=1)
# Here we specify num_latent=1 to avoid getting two outputs when predicting as Y_augmented is 2-dimensional

# closure
@tf.function
def objective_closure():
    return - m.log_marginal_likelihood()

# fit the covariance function parameters
maxiter = ci_niter(10000)
gpflow.optimizers.Scipy().minimize(objective_closure, m.trainable_variables, options=dict(maxiter=maxiter),
                                   method='L-BFGS-B')
```

          fun: -225.6617724526606
     hess_inv: <11483x11483 LbfgsInvHessProduct with dtype=float64>
          jac: array([ 2.49150975e-02, -8.71126486e-03,  8.32274805e-04, ...,
           -9.46678965e-11,  2.04116098e-11,  3.06369938e-03])
      message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
         nfev: 1289
          nit: 1179
       status: 0
      success: True
            x: array([ 7.15966290e-02, -7.52441296e-01, -1.34821048e+00, ...,
           -3.54882923e-12,  9.69372022e-13,  4.49216825e-01])

That's it: the model is trained. Let's plot the model fit to see what's happened.

```python
def plot_gp(x, mu, var, color, label):
    plt.plot(x, mu, color=color, lw=2, label=label)
    plt.fill_between(x[:, 0],
                     (mu - 2*np.sqrt(var))[:, 0],
                     (mu + 2*np.sqrt(var))[:, 0],
                     color=color, alpha=0.4)

def plot(m):
    plt.figure(figsize=(8, 4))
    xtest = np.linspace(0, 1, 100)[:,None]
    line, = plt.plot(X1, Y1, 'x', mew=2)
    mu, var = m.predict_f(np.hstack((xtest, np.zeros_like(xtest))))
    plot_gp(xtest, mu, var, line.get_color(), 'Y1')

    line, = plt.plot(X2, Y2, 'x', mew=2)
    mu, var = m.predict_f(np.hstack((xtest, np.ones_like(xtest))))
    plot_gp(xtest, mu, var, line.get_color(), 'Y2')

    plt.legend()

plot(m)
```

From the plots, we can see:

 - The first function (blue) has low posterior variance everywhere because there are so many observations, and the noise variance is small.
 - The second function (orange) has higher posterior variance near the data, because the data are more noisy, and very high posterior variance where there are no observations (x > 0.5).
 - The model has done a reasonable job of estimating the noise variance and lengthscales.
 - The model recognises the correlation between the two functions and is able to suggest (with uncertainty) that because x > 0.5 the orange curve should follow the blue curve (which we know to be the truth from the data-generating procedure).

The covariance matrix between outputs is as follows:

```python
B = coreg.W.numpy() @ coreg.W.numpy().T + np.diag(coreg.kappa.numpy())
print('B =', B)
plt.imshow(B);
```

## References

Bonilla, Edwin V., Kian M. Chai, and Christopher Williams. "Multi-task Gaussian process prediction." _Advances in neural information processing systems_. 2008.


# Обучение без учителя
- В задачах машинного обучения без учителя нет целевой переменной (правильных ответов), есть только признаковое описание объектов. Необходимо выявить некоторые скрытые закономерности данных, понять, как объекты устроены, есть ли в данных какая-то структура.
- **Кластеризация**: k-means, DBSCAN, ...
- **Визуализация**: PCA, t-SNE, ...
- Задача кластеризации состоит в том, чтобы разбить объекты, которые даны в выборке, на несколько кластеров, так чтобы объекты в одном кластере были похожи друг на друга, а объекты в разных кластерах были не похожи друг на друга.

# 1. k-means
Кластеры представляют собой центроиды, то есть каждый кластер задаётся координатами центра, а точки относятся к тому кластеру, чей центр к ним ближе.

**Минимизируем полную сумму квадратов расстояний от каждой точки до среднего значения назначенного ей кластера**

Итерационный алгоритм:
1. берём k случайных центров (*k средних*)
2. присоединяем все объекты к ближайшему к ним кластеру
3. пересчитываем координаты кластеров, чтобы они оказались в центре масс точек, которые относятся к ним. То есть для каждой центроиды берём все точки, которые к ней относятся, считаем среднее.
4. снова ищем для каждой точки ближайший центроид
- Если ни у одной точки её назначение не изменилось, то останавливаемся и сохраняем кластеры.
### Достоинства:
+ Очень простой алгоритм
+ Работает даже на больших данных
### Недостатки:
- Надо задавать число k руками
- Не всегда находит кластеры правильно (ищет только кластеры выпуклой формы)
- сильно зависит от начальной инициализации центров кластеров (разный результат от запуска к запуску)

```python
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn import datasets
from sklearn.metrics import accuracy_score
```

```python
np.random.seed(42)
```

```python
X1 = np.random.normal(loc=[0,-10], size=(100,2))
X2 = np.random.normal(loc=[-10, 0], size=(100,2))
X3 = np.random.normal(loc=[0, 0], size=(100,2))
X = np.vstack((X1,X2,X3))
y = np.array([1]*100 + [2]*100 + [3]*100)
```

```python
k_means = KMeans(n_clusters=3)
```

```python
clusters = k_means.fit_predict(X)
```

```python
plt.scatter(X[:, 0], X[:, 1], c=clusters)
plt.title('K-Means')
plt.show()
```

Попробуем разное количество кластеров:

```python
plt.figure(figsize=(12, 12))
for n_c in range(2, 8):
    k_means = KMeans(n_clusters=n_c)
    clusters = k_means.fit_predict(X)
    plt.subplot(3, 2, n_c-1)
    plt.scatter(X[:, 0], X[:, 1], c=clusters)
    plt.title('n_clusters = {}'.format(n_c))
plt.show()
```

### Оценка качества разбиения, выбор числа кластеров, SSE, elbow method
Дана матрица данных $X$ и дано число $k$ предполагаемых кластеров. Цель кластеризации представить данные в виде групп кластеров $C=\{C_1, C_2, \ldots, C_k\}$. Каждый кластер имеет свой центр:

\begin{equation}
\mu_i = \frac{1}{n_i} \sum \limits_{x_j \in C_i} x_j
\end{equation}

где $n_i = |C_i|$ - это количество точек в кластере $C_i$.

Таким образом, мы имеем некоторые кластеры $C=\{C_1, C_2, \ldots, C_k\}$ и необходимо оценить качество разбиения. Для этого будем вычислять сумму квадратических ошибок (**SSE, sum of squared error**):

\begin{equation}
SSE(C) = \sum \limits_{i=1}^{k} \sum \limits_{x_j \in C_i} ||x_j - \mu_i||^2
\end{equation}

Сумма квадратов расстояний между центроидом и каждым объектом кластера.
Цель найти
\begin{equation}
C^* = arg\min\limits_C \{SSE(C)\}
\end{equation}

Выбирается число кластеров в месте "преломления" графика

# 2. DBSCAN
Идея: хотим найти скопления точек, не зная заранее количество кластеров

Нужно задать два параметра:
- размер окрестности точки $\epsilon$
- minPts - минимальное количество точек в $\epsilon$ - окрестности

Тогда все точки можно разделить на три типа:
- Core (ядровые) точки - у которых в $\epsilon$ - окрестности от minPts соседей
- Border (граничные) точки - не Core точки, но достижимы из Core точек
- Noise (шумовые) точки - все остальные, меньше minPts соседей в $\epsilon$ - окрестности

- minPts = 4
- вокруг каждой точки есть своя $\epsilon$ - окрестность
- точка А и все красные точки - core (ядровые), так как $\epsilon$ окрестность вокруг них содержит по 4 и более точек (включая себя).
- точки B и С -border points, они не core points (ядровые), но они достижимы из A через другие ядровые точки, поэтому они тоже принадлежат кластеру
- точка N - шум

### Алгоритм:
1. берём следующую точку, ищем соседей в $\epsilon$ окрестности
2. если их как минимум minPts, то начинаем поиск связной компоненты из этой core-точки (обход в ширину)
3. иначе помечаем точку как noise, она может быть позднее к какой-то Core-точке как Border-точка

Начинаем с точки P2. minPts=2. Нашли ближайших соседей P2 - P1 и P3, они внутри окрестности. Всего в окрестности находится 3 точки, это больше minPts, поэтому P2 - это core-точка. Из неё нужно запустить поиск связной компоненты. Для этого проверим соседей P1 и P3.

В окрестности P1 находится две точки, она тоже core. Но никаких новых точек из неё не было найдено.

Проверим теперь P3 точку. У неё в окрестности три точки (core), причем P4 - новая точка. То есть из начальной точки P2 можно достичь точки P4 по цепочке из core точек.

Core. Достигаем новой точки P5. Проверяем P5, больше не находим непосещенных точек. Это значит, что кластер найден.

Берём следующую непомеченную точку P6. Ищем соседей в окрестности. Их нет. Noise.

### Достоинства:
+ Не нужно задавать количество кластеров
+ Кластеры могут быть любой формы
+ Может работать с шумными данными
### Недостатки:
- Долго работает на больших данных
- Чувствителен к выбору гиперпараметров (лучше использовать иерархический HDBSCAN)

```python
from sklearn.cluster import DBSCAN
```

```python
from sklearn.cluster import DBSCAN
plt.figure(figsize=(12, 36))
i = 1
for sample in [2, 5, 10]:
    for e in [0.2, 1, 3, 5, 10]:
        dbscan = DBSCAN(eps=e, min_samples=sample)
        clusters = dbscan.fit_predict(X)
        plt.subplot(9, 2, i)
        plt.scatter(X[:, 0], X[:, 1], c=clusters)
        plt.title('eps = {}, min_samples = {}'.format(e, sample))
        i += 1
    i += 1
plt.show()
```

- Если маленький радиус е и мало объектов, то DBSCAN находит много выбросов
- Если увеличить радиус и не менять количество объектов, то часть из них остается выбросами
- Если еще увеличить радиус, то выбросы пропадают
- Если еще сильнее увеличить радиус, то выделяется только один кластер и т.д.

### Kmeans и DBSCAN на более сложных данных

```python
n_samples = 1500
X = datasets.make_circles(n_samples=n_samples,
                          factor = 0.5,
                          noise = 0.05)
X = X[0]
X.shape
```

    (1500, 2)

```python
dbscan = DBSCAN(eps=0.1, min_samples=5)
clusters = dbscan.fit_predict(X)
```

```python
plt.scatter(X[:, 0], X[:, 1], c=clusters)
plt.title('DBSCAN');

```

```python
k_means = KMeans(n_clusters=2)
clusters = k_means.fit_predict(X)
plt.scatter(X[:, 0], X[:, 1], c = clusters);
```

# Пример

```python
from sklearn.datasets import load_digits
```

```python
digits = load_digits()
X, y = digits['data'], digits['target']
```

```python
plt.imshow(X[1].reshape(8, 8), cmap='gray')
```

```python
km = KMeans(n_clusters=10)
```

```python
clusters = km.fit_predict(X)
```

```python
pred = np.zeros(X.shape[0])
```

```python
for i in range(10):
    bc = np.bincount(y[clusters == i])
    pred[clusters == i] = bc.argmax() # преобладающая метка в кластере
```

```python
from sklearn.metrics import accuracy_score
```

```python
accuracy_score(y, pred)
```

    0.7924318308291597

Ошибки кластеризации для нуля:

```python
incorrect_indices = np.where(np.logical_and(pred == 0, y!=0))[0]
```

```python
for i in range(2):
    plt.imshow(X[incorrect_indices[i]].reshape(8, 8), cmap='gray')
    plt.title('Real digit is {}'.format(y[incorrect_indices[i]]))
    plt.show()
```

```python
correct_indices = np.where(np.logical_and(pred == 0, y == 0))[0]
```

```python
for i in range(2):
    plt.imshow(X[correct_indices[i]].reshape(8, 8), cmap='gray')
```

## Другие алгоритмы кластеризации

# Домашнее задание 2
## Задание 1
1. Реализовать kmeans
2. Визуализировать сходимость центров кластеров
3. Оценить $SSE$ для значений $k = 1, \ldots, 10$ и построить график зависимости $SSE$ от количества кластеров.

### Алгоритм к-средних
На вход алгоритм получает матрицу данных $D$, количество кластеров $k$, и критерий остановки $\epsilon$:
1. t = 0
2. случайным образом инициализируем $k$ центров кластеров: $\mu_1^t, \mu_2^t, \ldots, \mu_k^t \in R^d$;
3. повторять
    4. $t = t + 1$;
    5. $C_j = 0$ для всех $j = 1, \ldots, k$
    6. для каждого $x_j \in D$
        7. $j^* = arg\min\limits_C \{||X_j - \mu_i^{t-1}||^2\}$ \\\ присваиваем $x_j$ к ближайшему центру
        8. $C_{j^*} = C_{j^*} \cup {x_j}$
    9. для каждого i=1 до k
        10. $\mu_i = \frac{1}{|C_i|} \sum_{x_j \in C_i} x_j$
11. пока $\sum_{i=1}^k ||\mu_i^{t} - \mu_i^{t-1}||^2 \leq \epsilon$

```python
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
X, Y = make_blobs(n_samples = 1000, n_features=2, centers=5, cluster_std = 1.2, random_state=17)
plt.scatter(X[:,0], X[:,1])
```

## Задание 2
1. Исследуйте данные с помощью pandas. Посмотрите на признаки и их распределения. (Например, постройте график распределения автомобилей по годам, типу топлива и т.д.)
2. Кластеризуйте данные с помощью KMeans из sklearn.clustering. Найдите оптимальное число кластеров
3. Произведите анализ получившихся кластеров:

Пример: первый кластер содержит полноприводные автомобили немецкого производства с АКПП, небольшим пробегом и высокой стоимостью, второй - японцы с правым рулём ...

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

df = pd.read_csv('data/2.csv', encoding='cp1251')
df = df.drop(columns=['Модель', 'Цвет'])
df.head()
```

      Марка
      Год
      Состояние
      Пробег
      Объем
      Топливо
      Мощность
      Кузов
      Привод
      КПП
      Руль
      Хозяев в ПТС
      Цена

      0
      Volkswagen
      2013.0
      БУ
      42000.0
      1200.0
      бензин
      105.0
      хэтчбек
      передний
      автомат
      левый
      1 владелец
      689196.0

      1
      Skoda
      2012.0
      БУ
      62000.0
      1800.0
      бензин
      152.0
      кроссовер
      полный
      механика
      левый
      1 владелец
      639196.0

      2
      Renault
      2015.0
      БУ
      4700.0
      1600.0
      бензин
      106.0
      хэтчбек
      передний
      механика
      левый
      1 владелец
      629196.0

      3
      Nissan
      2012.0
      БУ
      70000.0
      1600.0
      бензин
      110.0
      хэтчбек
      передний
      автомат
      левый
      1 владелец
      479196.0

      4
      УАЗ
      2014.0
      БУ
      50000.0
      2700.0
      бензин
      128.0
      внедорожник
      полный
      механика
      левый
      1 владелец
      599196.0


Calculus

From

This section covers how to do basic calculus tasks such as derivatives, integrals, limits, and series expansions in SymPy.  If you are not familiar with the math of any part of this section, you may safely skip it.

```julia
    >>> from sympy import *
    >>> x, y, z = symbols('x y z')
    >>> init_printing(use_unicode=True)
```

In Julia

we have

all functions from sympy are imported by default

unicode printing is enabled by default

double quotes are for strings

so the above can be:

```julia
using SymPy
x, y, z = symbols("x y z")
```

    (x, y, z)

We can also use the convenient @vars macro, as with

```julia
@vars x y z
```

    (x, y, z)

Derivatives

To take derivatives, use the diff function.

```julia
    >>> diff(cos(x), x)
    -sin(x)
    >>> diff(exp(x**2), x)
         ⎛ 2⎞
         ⎝x ⎠
    2⋅x⋅ℯ
```

In  Julia

save for ** becoming ^ this is the same

```julia
diff(cos(x), x)
```

\begin{equation*}- \sin{\left (x \right )}\end{equation*}

```julia
diff(exp(x^2), x)
```

\begin{equation*}2 x e^{x^{2}}\end{equation*}

diff can take multiple derivatives at once.  To take multiple derivatives, pass the variable as many times as you wish to differentiate, or pass a number after the variable.  For example, both of the following find the third derivative of x^4.

```julia
    >>> diff(x**4, x, x, x)
    24⋅x
    >>> diff(x**4, x, 3)
    24⋅x
```

In Julia

```julia
diff(x^4, x, x, x)
```

\begin{equation*}24 x\end{equation*}

```julia
diff(x^4, x, 3)
```

\begin{equation*}24 x\end{equation*}

You can also take derivatives with respect to many variables at once.  Just pass each derivative in order, using the same syntax as for single variable derivatives.  For example, each of the following will compute $\frac{\partial^7}{\partial x\partial y^2\partial z^4} e^{x y z}$.

```julia
    >>> expr = exp(x*y*z)
    >>> diff(expr, x, y, y, z, z, z, z)
     3  2 ⎛ 3  3  3       2  2  2                ⎞  x⋅y⋅z
    x ⋅y ⋅⎝x ⋅y ⋅z  + 14⋅x ⋅y ⋅z  + 52⋅x⋅y⋅z + 48⎠⋅ℯ
    >>> diff(expr, x, y, 2, z, 4)
     3  2 ⎛ 3  3  3       2  2  2                ⎞  x⋅y⋅z
    x ⋅y ⋅⎝x ⋅y ⋅z  + 14⋅x ⋅y ⋅z  + 52⋅x⋅y⋅z + 48⎠⋅ℯ
    >>> diff(expr, x, y, y, z, 4)
     3  2 ⎛ 3  3  3       2  2  2                ⎞  x⋅y⋅z
    x ⋅y ⋅⎝x ⋅y ⋅z  + 14⋅x ⋅y ⋅z  + 52⋅x⋅y⋅z + 48⎠⋅ℯ
```

In Julia:

```julia
expr = exp(x*y*z)
```

\begin{equation*}e^{x y z}\end{equation*}

```julia
diff(expr, x, y, y, z, z, z, z)
```

\begin{equation*}x^{3} y^{2} \left(x^{3} y^{3} z^{3} + 14 x^{2} y^{2} z^{2} + 52 x y z + 48\right) e^{x y z}\end{equation*}

```julia
diff(expr, x, y, 2, z, 4)
```

\begin{equation*}x^{3} y^{2} \left(x^{3} y^{3} z^{3} + 14 x^{2} y^{2} z^{2} + 52 x y z + 48\right) e^{x y z}\end{equation*}

```julia
diff(expr, x, y, y, z, 4)
```

\begin{equation*}x^{3} y^{2} \left(x^{3} y^{3} z^{3} + 14 x^{2} y^{2} z^{2} + 52 x y z + 48\right) e^{x y z}\end{equation*}

diff can also be called as a method.  The two ways of calling diff are exactly the same, and are provided only for convenience.

```julia
    >>> expr.diff(x, y, y, z, 4)
     3  2 ⎛ 3  3  3       2  2  2                ⎞  x⋅y⋅z
    x ⋅y ⋅⎝x ⋅y ⋅z  + 14⋅x ⋅y ⋅z  + 52⋅x⋅y⋅z + 48⎠⋅ℯ
```

In Julia:

```julia
expr.diff(x, y, y, z, 4)
```

\begin{equation*}x^{3} y^{2} \left(x^{3} y^{3} z^{3} + 14 x^{2} y^{2} z^{2} + 52 x y z + 48\right) e^{x y z}\end{equation*}

To create an unevaluated derivative, use the Derivative class.  It has the same syntax as diff.

```julia
    >>> deriv = Derivative(expr, x, y, y, z, 4)
    >>> deriv
         7
        ∂     ⎛ x⋅y⋅z⎞
    ──────────⎝ℯ     ⎠
      4   2
      ∂z  ∂y  ∂x
```

In Julia,

classes are not exported, so we use sympy.Derivative:

```julia
deriv = sympy.Derivative(expr, x, y, y, z, 4)
deriv
```

\begin{equation*}\frac{\partial^{7}}{\partial z^{4}\partial y^{2}\partial x} e^{x y z}\end{equation*}

To evaluate an unevaluated derivative, use the doit method.

```julia
    >>> deriv.doit()
     3  2 ⎛ 3  3  3       2  2  2                ⎞  x⋅y⋅z
    x ⋅y ⋅⎝x ⋅y ⋅z  + 14⋅x ⋅y ⋅z  + 52⋅x⋅y⋅z + 48⎠⋅ℯ
```

In Julia:

```julia
deriv.doit()
```

\begin{equation*}x^{3} y^{2} \left(x^{3} y^{3} z^{3} + 14 x^{2} y^{2} z^{2} + 52 x y z + 48\right) e^{x y z}\end{equation*}

These unevaluated objects are useful for delaying the evaluation of the derivative, or for printing purposes.  They are also used when SymPy does not know how to compute the derivative of an expression (for example, if it contains an undefined function, which are described in the :ref:Solving Differential Equations <tutorial-dsolve> section).

Derivatives of unspecified order can be created using tuple (x, n) where n is the order of the derivative with respect to x.

```julia
    >>> m, n, a, b = symbols('m n a b')
    >>> expr = (a*x + b)**m
    >>> expr.diff((x, n))
      n
     ∂ ⎛         m⎞
    ───⎝(a⋅x + b) ⎠
      n
    ∂x
```

In Julia:

```julia
@vars m n a b
expr = (a*x + b)^m
expr.diff((x, n))
```

\begin{equation*}\frac{\partial^{n}}{\partial x^{n}} \left(a x + b\right)^{m}\end{equation*}

Integrals

To compute an integral, use the integrate function.  There are two kinds of integrals, definite and indefinite.  To compute an indefinite integral, that is, an antiderivative, or primitive, just pass the variable after the expression.

```julia
    >>> integrate(cos(x), x)
    sin(x)
```

In Julia:

```julia
integrate(cos(x), x)
```

\begin{equation*}\sin{\left (x \right )}\end{equation*}

Note that SymPy does not include the constant of integration.  If you want it, you can add one yourself, or rephrase your problem as a differential equation and use dsolve to solve it, which does add the constant (see :ref:tutorial-dsolve).

Quick Tip:

$\infty$ in SymPy is oo (that's the lowercase letter "oh" twice).  This    is because oo looks like $\infty$, and is easy to type.

To compute a definite integral, pass the argument (integration_variable, lower_limit, upper_limit).  For example, to compute

$$

   \int_0^\infty e^{-x}\,dx,
$$

we would do

```julia
    >>> integrate(exp(-x), (x, 0, oo))
    1
```

In Julia:

```julia
integrate(exp(-x), (x, 0, oo))
```

\begin{equation*}1\end{equation*}

As with indefinite integrals, you can pass multiple limit tuples to perform a multiple integral.  For example, to compute

$$
   \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} e^{- x^{2} - y^{2}}\, dx\, dy,
$$

do

```julia
    >>> integrate(exp(-x**2 - y**2), (x, -oo, oo), (y, -oo, oo))
    π
```

In Julia:

```julia
integrate(exp(-x^2 - y^2), (x, -oo, oo), (y, -oo, oo))
```

\begin{equation*}\pi\end{equation*}

If integrate is unable to compute an integral, it returns an unevaluated Integral object.

```julia
    >>> expr = integrate(x**x, x)
    >>> print(expr)
    Integral(x**x, x)
    >>> expr
    ⌠
    ⎮  x
    ⎮ x  dx
    ⌡
```

In Julia:

```julia
expr = integrate(x^x, x)
expr
```

\begin{equation*}\int x^{x}\, dx\end{equation*}

As with Derivative, you can create an unevaluated integral using Integral.  To later evaluate this integral, call doit.

```julia
    >>> expr = Integral(log(x)**2, x)
    >>> expr
    ⌠
    ⎮    2
    ⎮ log (x) dx
    ⌡
    >>> expr.doit()
             2
    x⋅log (x) - 2⋅x⋅log(x) + 2⋅x
```

In Julia:

the Integral class is not exported, so it must be qualified:

```julia
expr = sympy.Integral(log(x)^2, x)
expr
```

\begin{equation*}\int \log{\left (x \right )}^{2}\, dx\end{equation*}

```julia
expr.doit()
```

\begin{equation*}x \log{\left (x \right )}^{2} - 2 x \log{\left (x \right )} + 2 x\end{equation*}

integrate uses powerful algorithms that are always improving to compute both definite and indefinite integrals, including heuristic pattern matching type algorithms, a partial implementation of the Risch algorithm <http://en.wikipedia.org/wiki/Risch_algorithm>, and an algorithm using Meijer G-functions <http://en.wikipedia.org/wiki/Meijer_g-function> that is useful for computing integrals in terms of special functions, especially definite integrals.  Here is a sampling of some of the power of integrate.

```julia
    >>> integ = Integral((x**4 + x**2*exp(x) - x**2 - 2*x*exp(x) - 2*x -
    ...     exp(x))*exp(x)/((x - 1)**2*(x + 1)**2*(exp(x) + 1)), x)
    >>> integ
    ⌠
    ⎮ ⎛ 4    2  x    2        x          x⎞  x
    ⎮ ⎝x  + x ⋅ℯ  - x  - 2⋅x⋅ℯ  - 2⋅x - ℯ ⎠⋅ℯ
    ⎮ ──────────────────────────────────────── dx
    ⎮               2        2 ⎛ x    ⎞
    ⎮        (x - 1) ⋅(x + 1) ⋅⎝ℯ  + 1⎠
    ⌡
    >>> integ.doit()
                     x
       ⎛ x    ⎞     ℯ
    log⎝ℯ  + 1⎠ + ──────
                   2
                  x  - 1

    >>> integ = Integral(sin(x**2), x)
    >>> integ
    ⌠
    ⎮    ⎛ 2⎞
    ⎮ sin⎝x ⎠ dx
    ⌡
    >>> integ.doit()
                    ⎛√2⋅x⎞
    3⋅√2⋅√π⋅fresnels⎜────⎟⋅Γ(3/4)
                    ⎝ √π ⎠
    ─────────────────────────────
               8⋅Γ(7/4)

    >>> integ = Integral(x**y*exp(-x), (x, 0, oo))
    >>> integ
    ∞
    ⌠
    ⎮  y  -x
    ⎮ x ⋅ℯ   dx
    ⌡
    0
    >>> integ.doit()
    ⎧ Γ(y + 1)    for -re(y) < 1
    ⎪
    ⎪∞
    ⎪⌠
    ⎨⎮  y  -x
    ⎪⎮ x ⋅ℯ   dx    otherwise
    ⎪⌡
    ⎪0
    ⎩
```

In Julia:

```julia
integ = sympy.Integral((x^4 + x^2*exp(x) - x^2 - 2*x*exp(x) - 2*x - exp(x))*exp(x)/((x - 1)^2*(x + 1)^2*(exp(x) + 1)), x)
integ
```

\begin{equation*}\int \frac{\left(x^{4} + x^{2} e^{x} - x^{2} - 2 x e^{x} - 2 x - e^{x}\right) e^{x}}{\left(x - 1\right)^{2} \left(x + 1\right)^{2} \left(e^{x} + 1\right)}\, dx\end{equation*}

```julia
integ.doit()
```

\begin{equation*}\log{\left (e^{x} + 1 \right )} + \frac{e^{x}}{x^{2} - 1}\end{equation*}

```julia
integ = sympy.Integral(sin(x^2), x)
integ.doit()
```

\begin{equation*}\frac{3 \sqrt{2} \sqrt{\pi} S\left(\frac{\sqrt{2} x}{\sqrt{\pi}}\right) \Gamma\left(\frac{3}{4}\right)}{8 \Gamma\left(\frac{7}{4}\right)}\end{equation*}

```julia
integ = sympy.Integral(x^y*exp(-x), (x, 0, oo))
integ.doit()
```

\begin{equation*}\begin{cases} \Gamma\left(y + 1\right) & \text{for}\: - \Re{\left(y\right)} < 1 \\\int\limits_{0}^{\infty} x^{y} e^{- x}\, dx & \text{otherwise} \end{cases}\end{equation*}

This last example returned a Piecewise expression because the integral does not converge unless $\Re(y) > 1.$

Limits

SymPy can compute symbolic limits with the limit function.  The syntax to compute

$$
   \lim_{x\to x_0} f(x)
$$

is limit(f(x), x, x0).

```julia
    >>> limit(sin(x)/x, x, 0)
    1
```

In Julia:

```julia
limit(sin(x)/x, x, 0)
```

\begin{equation*}1\end{equation*}

In Julia, a pair can be used to indicate the limit:

```julia
limit(sin(x)/x, x=>0)
```

\begin{equation*}1\end{equation*}

limit should be used instead of subs whenever the point of evaluation is a singularity.  Even though SymPy has objects to represent $\infty$, using them for evaluation is not reliable because they do not keep track of things like rate of growth.  Also, things like $\infty - \infty$ and $\frac{\infty}{\infty}$ return $\mathrm{nan}$ (not-a-number).  For example

```julia
    >>> expr = x**2/exp(x)
    >>> expr.subs(x, oo)
    nan
    >>> limit(expr, x, oo)
    0
```

In Julia:

```julia
expr = x^2/exp(x)
expr.subs(x, oo)
```

\begin{equation*}\mathrm{NaN}\end{equation*}

```julia
limit(expr, x, oo)
```

\begin{equation*}0\end{equation*}

Like Derivative and Integral, limit has an unevaluated counterpart, Limit.  To evaluate it, use doit.

```julia
    >>> expr = Limit((cos(x) - 1)/x, x, 0)
    >>> expr
         ⎛cos(x) - 1⎞
     lim ⎜──────────⎟
    x─→0⁺⎝    x     ⎠
    >>> expr.doit()
    0
```

In Julia:

```julia
expr = sympy.Limit((cos(x) - 1)/x, x, 0)
expr
```

\begin{equation*}\lim_{x \to 0^+}\left(\frac{\cos{\left (x \right )} - 1}{x}\right)\end{equation*}

```julia
expr.doit()
```

\begin{equation*}0\end{equation*}

To evaluate a limit at one side only, pass '+' or '-' as a third argument to limit.  For example, to compute

$$
   \lim_{x\to 0^+}\frac{1}{x},
$$

do

```julia
    >>> limit(1/x, x, 0, '+')
    ∞
```

In Julia:

```julia
limit(1/x, x => 0, "+")
```

\begin{equation*}\infty\end{equation*}

As opposed to

```julia
    >>> limit(1/x, x, 0, '-')
    -∞
```

In Julia:

```julia
limit(1/x, x, 0, "-")
```

\begin{equation*}-\infty\end{equation*}

Series Expansion

SymPy can compute asymptotic series expansions of functions around a point. To compute the expansion of f(x) around the point x = x_0 terms of order x^n, use f(x).series(x, x0, n).  x0 and n can be omitted, in which case the defaults x0=0 and n=6 will be used.

```julia
    >>> expr = exp(sin(x))
    >>> expr.series(x, 0, 4)
             2
            x     ⎛ 4⎞
    1 + x + ── + O⎝x ⎠
            2
```

In Julia:

```julia
expr = exp(sin(x))
expr.series(x, 0, 4)
```

\begin{equation*}1 + x + \frac{x^{2}}{2} + O\left(x^{4}\right)\end{equation*}

The O\left (x^4\right ) term at the end represents the Landau order term at x=0 (not to be confused with big O notation used in computer science, which generally represents the Landau order term at $x=\infty$).  It means that all x terms with power greater than or equal to x^4 are omitted.  Order terms can be created and manipulated outside of series.  They automatically absorb higher order terms.

```julia
    >>> x + x**3 + x**6 + O(x**4)
         3    ⎛ 4⎞
    x + x  + O⎝x ⎠
    >>> x*O(1)
    O(x)
```

In Julia:

O is not exported, so we must qualify it:

```julia
x + x^3 + x^6 + sympy.O(x^4)

```

\begin{equation*}x + x^{3} + O\left(x^{4}\right)\end{equation*}

```julia
x*sympy.O(1)
```

\begin{equation*}O\left(x\right)\end{equation*}

If you do not want the order term, use the removeO method.

```julia
    >>> expr.series(x, 0, 4).removeO()
     2
    x
    ── + x + 1
    2
```

In Julia:

```julia
expr.series(x, 0, 4).removeO()
```

\begin{equation*}\frac{x^{2}}{2} + x + 1\end{equation*}

The O notation supports arbitrary limit points (other than 0):

```julia
    >>> exp(x - 6).series(x, x0=6)
                2          3          4          5
         (x - 6)    (x - 6)    (x - 6)    (x - 6)         ⎛       6       ⎞
    -5 + ──────── + ──────── + ──────── + ──────── + x + O⎝(x - 6) ; x → 6⎠
            2          6          24        120
```

In Julia:

```julia
exp(x - 6).series(x, x0=6)
```

\begin{equation*}-5 + \frac{\left(x - 6\right)^{2}}{2} + \frac{\left(x - 6\right)^{3}}{6} + \frac{\left(x - 6\right)^{4}}{24} + \frac{\left(x - 6\right)^{5}}{120} + x + O\left(\left(x - 6\right)^{6}; x\rightarrow 6\right)\end{equation*}

Finite differences

So far we have looked at expressions with analytic derivatives and primitive functions respectively. But what if we want to have an expression to estimate a derivative of a curve for which we lack a closed form representation, or for which we don't know the functional values for yet. One approach would be to use a finite difference approach.

The simplest way the differentiate using finite differences is to use the differentiate_finite function:

```julia
    >>> f, g = symbols('f g', cls=Function)
    >>> differentiate_finite(f(x)*g(x))
    -f(x - 1/2)⋅g(x - 1/2) + f(x + 1/2)⋅g(x + 1/2)
```

In Julia:

differentiate_finite is not exported

```julia
f, g = symbols("f g", cls=sympy.Function)
sympy.differentiate_finite(f(x)*g(x))
```

\begin{equation*}- f{\left (x - \frac{1}{2} \right )} g{\left (x - \frac{1}{2} \right )} + f{\left (x + \frac{1}{2} \right )} g{\left (x + \frac{1}{2} \right )}\end{equation*}

(The functions f and g can also be created with the command @symfuns f g, using the @symfuns macro.)

If we want to expand the intermediate derivative we may pass the flag evaluate=True:

```julia
    >>> differentiate_finite(f(x)*g(x), evaluate=True)
    (-f(x - 1/2) + f(x + 1/2))⋅g(x) + (-g(x - 1/2) + g(x + 1/2))⋅f(x)
```

In Julia:

```julia
sympy.differentiate_finite(f(x)*g(x), evaluate=true)
```

\begin{equation*}\left(- f{\left (x - \frac{1}{2} \right )} + f{\left (x + \frac{1}{2} \right )}\right) g{\left (x \right )} + \left(- g{\left (x - \frac{1}{2} \right )} + g{\left (x + \frac{1}{2} \right )}\right) f{\left (x \right )}\end{equation*}

This form however does not respect the product rule.

If you already have a Derivative instance, you can use the as_finite_difference method to generate approximations of the derivative to arbitrary order:

```julia
    >>> f = Function('f')
    >>> dfdx = f(x).diff(x)
    >>> dfdx.as_finite_difference()
    -f(x - 1/2) + f(x + 1/2)
```

In Julia:

```julia
f = sympy.Function("f")
dfdx = f(x).diff(x)
dfdx.as_finite_difference()
```

\begin{equation*}- f{\left (x - \frac{1}{2} \right )} + f{\left (x + \frac{1}{2} \right )}\end{equation*}

here the first order derivative was approximated around x using a minimum number of points (2 for 1st order derivative) evaluated equidistantly using a step-size of 1. We can use arbitrary steps (possibly containing symbolic expressions):

```julia
    >>> f = Function('f')
    >>> d2fdx2 = f(x).diff(x, 2)
    >>> h = Symbol('h')
    >>> d2fdx2.as_finite_difference([-3*h,-h,2*h])
    f(-3⋅h)   f(-h)   2⋅f(2⋅h)
    ─────── - ───── + ────────
         2        2        2
      5⋅h      3⋅h     15⋅h
```

In Julia:

```julia
f = sympy.Function("f")
d2fdx2 = f(x).diff(x, 2)
h = sympy.Symbol("h")
d2fdx2.as_finite_difference([-3*h,-h,2*h])
```

\begin{equation*}\frac{f{\left (- 3 h \right )}}{5 h^{2}} - \frac{f{\left (- h \right )}}{3 h^{2}} + \frac{2 f{\left (2 h \right )}}{15 h^{2}}\end{equation*}

If you are just interested in evaluating the weights, you can do so manually:

```julia
    >>> finite_diff_weights(2, [-3, -1, 2], 0)[-1][-1]
    [1/5, -1/3, 2/15]
```

In Julia:

the finite_diff_weights function that is not exported:

```julia
sympy.finite_diff_weights(2, [-3, -1, 2], 0)[end][end]
```

\begin{equation*}\frac{2}{15}\end{equation*}

note that we only need the last element in the last sublist returned from finite_diff_weights. The reason for this is that the function also generates weights for lower derivatives and using fewer points (see the documentation of finite_diff_weights for more details).

If using finite_diff_weights directly looks complicated, and the as_finite_difference method of Derivative instances is not flexible enough, you can use apply_finite_diff which takes order, x_list, y_list and x0 as parameters:

```julia
    >>> x_list = [-3, 1, 2]
    >>> y_list = symbols('a b c')
    >>> apply_finite_diff(1, x_list, y_list, 0)
      3⋅a   b   2⋅c
    - ─── - ─ + ───
       20   4    5
```

In Julia,

apply_finite_diff is not exported:

```julia
x_list = [-3, 1, 2]
y_list = symbols("a b c")
sympy.apply_finite_diff(1, x_list, y_list, 0)
```

\begin{equation*}- \frac{3 a}{20} - \frac{b}{4} + \frac{2 c}{5}\end{equation*}

return to index