# Wheldon, Kirk, and Finlay Work-Precision Diagrams
### David Widmann, Chris Rackauckas

# Wheldon, Kirk, and Finlay

We study algorithms for solving constant delay differential equations with a test problem from W.H. Enright and H. Hayashi, "The evaluation of numerical software for delay differential equations", 1997. It is a model of chronic granulocytic leukemia that was published by T. Wheldon, J. Kirk and H. Finlay in "Cyclical granulopoiesis in chronic granulocytic leukemia: A simulation study", 1974, and is given by
```math
\begin{align}
y_1'(t) &= \frac{1.1}{1 + \sqrt{10}y_1(t-20)^{5/4}} - \frac{10y_1(t)}{1 + 40y_2(t)} \\
y_2'(t) &= \frac{100y_1(t)}{1 + 40y_2(t)} - 2.43y_2(t)
\end{align}
```

```julia
using DelayDiffEq, DiffEqDevTools, DiffEqProblemLibrary, Plots
using DiffEqProblemLibrary.DDEProblemLibrary: importddeproblems; importddeproblems()
import DiffEqProblemLibrary.DDEProblemLibrary: prob_dde_wheldon
gr()

sol = solve(prob_dde_wheldon, MethodOfSteps(Vern9(), max_fixedpoint_iters=1000); reltol=1e-14, abstol=1e-14)
test_sol = TestSolution(sol)
plot(sol)
```

## Low order RK methods

### High tolerances

First we compare final errors of solutions with low order RK methods at high tolerances.

```julia
abstols = 1.0 ./ 10.0 .^ (4:7)
reltols = 1.0 ./ 10.0 .^ (1:4)

setups = [Dict(:alg=>MethodOfSteps(BS3())),
          Dict(:alg=>MethodOfSteps(Tsit5())),
          Dict(:alg=>MethodOfSteps(RK4())),
          Dict(:alg=>MethodOfSteps(DP5())),
          Dict(:alg=>MethodOfSteps(OwrenZen3())),
          Dict(:alg=>MethodOfSteps(OwrenZen4())),
          Dict(:alg=>MethodOfSteps(OwrenZen5()))]
names = ["BS3", "Tsit5", "RK4", "DP5", "OwrenZen3", "OwrenZen4", "OwrenZen5"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:final)
plot(wp)
```

Next we test interpolation errors:

```julia
abstols = 1.0 ./ 10.0 .^ (4:7)
reltols = 1.0 ./ 10.0 .^ (1:4)

setups = [Dict(:alg=>MethodOfSteps(BS3())),
          Dict(:alg=>MethodOfSteps(Tsit5())),
          Dict(:alg=>MethodOfSteps(RK4())),
          Dict(:alg=>MethodOfSteps(DP5())),
          Dict(:alg=>MethodOfSteps(OwrenZen3())),
          Dict(:alg=>MethodOfSteps(OwrenZen4())),
          Dict(:alg=>MethodOfSteps(OwrenZen5()))]
names = ["BS3", "Tsit5", "RK4", "DP5", "OwrenZen3", "OwrenZen4", "OwrenZen5"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:L2)
plot(wp)
```

Both interpolation tests and tests of final error show similar results. `BS3` does quite well but only `OwrenZen4`, `OwrenZen5`, and `RK4` achieve interpolation errors of about 1e-5.

### Low tolerances

We repeat our tests at low tolerances.

```julia
abstols = 1.0 ./ 10.0 .^ (8:11)
reltols = 1.0 ./ 10.0 .^ (5:8)

setups = [Dict(:alg=>MethodOfSteps(BS3())),
          Dict(:alg=>MethodOfSteps(Tsit5())),
          Dict(:alg=>MethodOfSteps(RK4())),
          Dict(:alg=>MethodOfSteps(DP5())),
          Dict(:alg=>MethodOfSteps(OwrenZen3())),
          Dict(:alg=>MethodOfSteps(OwrenZen4())),
          Dict(:alg=>MethodOfSteps(OwrenZen5()))]
names = ["BS3", "Tsit5", "RK4", "DP5", "OwrenZen3", "OwrenZen4", "OwrenZen5"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:final)
plot(wp)
```

```julia
abstols = 1.0 ./ 10.0 .^ (8:11)
reltols = 1.0 ./ 10.0 .^ (5:8)

setups = [Dict(:alg=>MethodOfSteps(BS3())),
          Dict(:alg=>MethodOfSteps(Tsit5())),
          Dict(:alg=>MethodOfSteps(RK4())),
          Dict(:alg=>MethodOfSteps(DP5())),
          Dict(:alg=>MethodOfSteps(OwrenZen3())),
          Dict(:alg=>MethodOfSteps(OwrenZen4())),
          Dict(:alg=>MethodOfSteps(OwrenZen5()))]
names = ["BS3", "Tsit5", "RK4", "DP5", "OwrenZen3", "OwrenZen4", "OwrenZen5"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:L2)
plot(wp)
```

Out of the compared methods, `Tsit5`, `DP5`, and `OwrenZen5` seem to be the best methods for this problem at low tolerances, but also `OwrenZen4` performs similarly well. `OwrenZen5` and `OwrenZen4` can even achieve interpolation errors below 1e-9.

## Lazy interpolants

### High tolerances

We compare the Verner methods, which use lazy interpolants, at high tolerances. As reference we include `OwrenZen4`.

```julia
abstols = 1.0 ./ 10.0 .^ (4:7)
reltols = 1.0 ./ 10.0 .^ (1:4)

setups = [Dict(:alg=>MethodOfSteps(Vern6())),
          Dict(:alg=>MethodOfSteps(Vern7())),
          Dict(:alg=>MethodOfSteps(Vern8())),
          Dict(:alg=>MethodOfSteps(Vern9())),
          Dict(:alg=>MethodOfSteps(OwrenZen4()))]
names = ["Vern6", "Vern7", "Vern8", "Vern9", "OwrenZen4"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:final)
plot(wp)
```

```julia
abstols = 1.0 ./ 10.0 .^ (4:7)
reltols = 1.0 ./ 10.0 .^ (1:4)

setups = [Dict(:alg=>MethodOfSteps(Vern6())),
          Dict(:alg=>MethodOfSteps(Vern7())),
          Dict(:alg=>MethodOfSteps(Vern8())),
          Dict(:alg=>MethodOfSteps(Vern9())),
          Dict(:alg=>MethodOfSteps(OwrenZen4()))]
names = ["Vern6", "Vern7", "Vern8", "Vern9", "OwrenZen4"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:L2)
plot(wp)
```

### Low tolerances

We repeat these tests and compare the Verner methods also at low tolerances.

```julia
abstols = 1.0 ./ 10.0 .^ (8:11)
reltols = 1.0 ./ 10.0 .^ (5:8)

setups = [Dict(:alg=>MethodOfSteps(Vern6())),
          Dict(:alg=>MethodOfSteps(Vern7())),
          Dict(:alg=>MethodOfSteps(Vern8())),
          Dict(:alg=>MethodOfSteps(Vern9())),
          Dict(:alg=>MethodOfSteps(OwrenZen4()))]
names = ["Vern6", "Vern7", "Vern8", "Vern9", "OwrenZen4"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:final)
plot(wp)
```

```julia
abstols = 1.0 ./ 10.0 .^ (8:11)
reltols = 1.0 ./ 10.0 .^ (5:8)

setups = [Dict(:alg=>MethodOfSteps(Vern6())),
          Dict(:alg=>MethodOfSteps(Vern7())),
          Dict(:alg=>MethodOfSteps(Vern8())),
          Dict(:alg=>MethodOfSteps(Vern9())),
          Dict(:alg=>MethodOfSteps(OwrenZen4()))]
names = ["Vern6", "Vern7", "Vern8", "Vern9", "OwrenZen4"]
wp = WorkPrecisionSet(prob_dde_wheldon,abstols,reltols,setups;names=names,
                      appxsol=test_sol,maxiters=Int(1e5),error_estimate=:L2)
plot(wp)
```

It seems `Vern6` and `Vern7` are both well suited for the problem at low tolerances and outperform `OwrenZen4`, whereas at high tolerances `OwrenZen4` is more efficient.

```julia
using DiffEqBenchmarks
DiffEqBenchmarks.bench_footer(WEAVE_ARGS[:folder],WEAVE_ARGS[:file])
```


```python
%reload_ext autoreload
%autoreload 2
import pandas as pd
import warnings
warnings.filterwarnings("ignore", category=RuntimeWarning);
warnings.filterwarnings("ignore", category=DeprecationWarning);
#Need to fix deprecation warning
warnings.filterwarnings("ignore");
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
%config InlineBackend.figure_format = 'retina'

```

```python
from hmmlearn.hmm import GaussianHMM
from hmm import GaussianFSHMM
```

## HMM and FSHMM testing

#### Generate synthetic data

We generate two features from an HMM following the parameters from the paper, and three features that are random noise.

```python
startprob = np.array([0.4, 0.6])

transmat = np.array([[0.75, 0.25],
                     [0.4, 0.6]])

means = np.array([[10.0,  20.0],
                  [30.0, 60.0]])

covars = 25.0 * np.tile(np.identity(2), (2, 1, 1))
model = GaussianHMM(n_components=2, covariance_type="diag")

model.startprob_ = startprob
model.transmat_ = transmat
model.means_ = means
model._covars_ = covars
```

```python
X, Y = model.sample(500)
```

```python
x1 = np.random.normal(0, 1, 500)
x2 = np.random.normal(0, 1, 500)
x3 = np.random.normal(0, 1, 500)
```

```python
df = pd.DataFrame(X)
df['3'] = x1
df['4'] = x2
df['5'] = x3
```

```python
sns.set(style="whitegrid", palette="tab10")
df.plot(figsize=(16,9));
```

```python
# To make the plot interactive using cufflinks uncomment the following
# import plotly
# import cufflinks as cf
# from plotly.offline import init_notebook_mode, plot, iplot
# import colorlover as cl

# plotly.offline.init_notebook_mode(connected=False)
# cf.set_config_file(offline=True, world_readable=False, theme='white')
# df.iplot()
```

### FSHMM

#### Test from paper

```python
%reload_ext autoreload
%autoreload 2
from hmm import GaussianFSHMM
```

```python
model_new_new = GaussianFSHMM(k=50, n_components=2, algorithm='viterbi', covariance_type="diag", init_params='m', n_iter=1000)
```

```python
model_new_new.transmat_ = np.array([[0.5, 0.5], [0.5, 0.5]])
model_new_new.startprob_ = np.array([0.5, 0.5])
model_new_new._covars_ = 4.0 * np.tile(np.identity(5), (2, 1, 1))
```

```python
df_norm = (df-df.mean())/df.std()
df_scaled = (df-df.min())/(df.max()-df.min())
```

```python
# df_norm.iplot()
# df_scaled.iplot()
```

```python
model_new_new.fit(df_norm)
```

    GaussianFSHMM(k=None)

#### Feature saliencies of all features. First two saliencies are almost one and the last three features (added noise) are close to zero.

```python
rho_df = pd.DataFrame(model_new_new.rho_, index=['rho1', 'rho2', 'rho3', 'rho4', 'rho5'], columns=['Saliency']).T.round(3)
rho_df
```

      rho1
      rho2
      rho3
      rho4
      rho5

      Saliency
      0.989
      0.994
      0.109
      0.105
      0.096

```python
rho_df.T.plot.bar(figsize=(11,6));
```

```python
model_new_new.monitor_.converged
```

    True

```python
model_new_new.monitor_
```

    ConvergenceMonitor(history=[-3079.812411067629, -3110.98786033821], iter=1000,
              n_iter=1000, tol=1e-09, verbose=False)

#### Transition matrix

The transition matrix from the original HMM was:

$$\begin{pmatrix}
           0.75 & 0.25\\
           0.4, & 0.6
    \end{pmatrix}$$

```python
pd.DataFrame(model_new_new.transmat_).round(3)
```

      0
      1

      0
      0.748
      0.252

      1
      0.367
      0.633

#### Parameter estimates for $\mu$
The means for the two-state HMM were:

$$\begin{align}
    S0_{\mu} =
        \begin{pmatrix}
            10\\
           20
    \end{pmatrix}
    \qquad
    S1_{\mu} =
        \begin{pmatrix}
            30\\
           60
    \end{pmatrix}
    \nonumber
  \end{align}$$

and the other three time series had a mean of 0.

```python
pd.DataFrame((model_new_new.means_)*np.array(df.std())+np.array(df.mean()), columns=['mu1', 'mu2', 'mu3', 'mu4', 'mu5'], index=['S0', 'S1']).round(3)
```

      mu1
      mu2
      mu3
      mu4
      mu5

      S0
      10.297
      20.390
      -0.176
      0.229
      -0.092

      S1
      29.767
      59.342
      -0.296
      -0.020
      -0.241

#### Parameter estimates for $\sigma$

The variances for the two-state HMM were:

$$\begin{align}
    S0_{\sigma^2} =
        \begin{pmatrix}
            25\\
           25
    \end{pmatrix}
    \qquad
    S1_{\sigma^2} =
        \begin{pmatrix}
            25\\
           25
    \end{pmatrix}
    \nonumber
  \end{align}$$

therefore the std for each component and each state was 5, and the other three time series had a std of 1.

```python
pd.DataFrame(np.sqrt((model_new_new._covars_)*np.array(df.std())+np.array(df.mean())), columns=['sigma1', 'sigma2', 'sigma3', 'sigma4', 'sigma5'], index=['S0', 'S1']).round(3)
```

      sigma1
      sigma2
      sigma3
      sigma4
      sigma5

      S0
      4.692
      6.219
      0.998
      1.016
      1.084

      S1
      4.766
      6.277
      0.962
      1.003
      1.076

#### Parameter estimates for $\epsilon$, the last three values (the irrelevant features) are close to zero, which is the true value.

```python
pd.DataFrame(model_new_new.epsilon_, index=['eps1', 'eps2', 'eps3', 'eps4', 'eps5'], columns=['MAP']).T.round(3)
```

      eps1
      eps2
      eps3
      eps4
      eps5

      MAP
      -0.125
      -0.064
      0.03
      -0.027
      0.029

#### Parameter estimates for $\tau$, the last three features are close to one, which is the true value.

```python
pd.DataFrame(model_new_new.tau_, index=['tau1', 'tau2', 'tau3', 'tau4', 'tau5'], columns=['MAP']).T.round(3)
```

      tau1
      tau2
      tau3
      tau4
      tau5

      MAP
      0.959
      0.796
      0.987
      0.982
      0.982

```python
# HTML('''''')
```

```python

```

```python

```


```julia
using LinearAlgebra
using ForwardDiff
using Symbolics
using StaticArrays
using BenchmarkTools
```

```julia
function f(x)
    @show x
    @show zero(x)
    @assert x > zero(x)
    sin(x)/x
end
```

    f (generic function with 1 method)

```julia
ForwardDiff.derivative(f, 1.0)
```

    x = Dual{ForwardDiff.Tag{typeof(f), Float64}}(1.0,1.0)
    zero(x) = Dual{ForwardDiff.Tag{typeof(f), Float64}}(0.0,0.0)

    -0.30116867893975674

```julia
ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}}(1.0,1.0) > 0
```

    true

```julia
module My

struct Foo{T}
    a::T
    b::T
end

end
```

    Main.My

```julia
methods(My.Foo)
```

# 1 method for type constructor:

 Main.My.Foo(a::T, b::T) where T in Main.My at In[5]:4

```julia
methods(My.Foo{Int})
```

# 1 method for type constructor:

 Main.My.Foo{T}(a, b) where T in Main.My at In[5]:4

```julia
My.Foo(a::T, b::T) where T<:Real = My.Foo{T}(a, b)
```

```julia
methods(My.Foo)
```

# 2 methods for type constructor:

 Main.My.Foo(a::T, b::T) where T<:Real in Main at In[8]:1

 Main.My.Foo(a::T, b::T) where T in Main.My at In[5]:4

```julia
@which My.Foo(1im, 2im)
```

Main.My.Foo(a::T, b::T) where T in Main.My at In[5]:4

```julia
@which My.Foo(1, 2)
```

Main.My.Foo(a::T, b::T) where T<:Real in Main at In[8]:1

```julia
G(x, A) = exp(-dot(x, A, x)/2)
```

    G (generic function with 1 method)

```julia
struct Bar{F, G}
    f::F
    ∇f::G
end
```

```julia
function Bar(f)
    ∇f(x, param) = ForwardDiff.gradient(x -> f(x, param), x)
    Bar(f, ∇f)
end
```

    Bar

```julia
bar = Bar(G)
```

    Bar{typeof(G), var"#∇f#2"{typeof(G)}}(G, var"#∇f#2"{typeof(G)}(G))

```julia
A = SA[
    2 -1
    -1 2
]
```

    2×2 SMatrix{2, 2, Int64, 4} with indices SOneTo(2)×SOneTo(2):
      2  -1
     -1   2

```julia
bar.f(SA[1, 2], A)
```

    0.049787068367863944

```julia
bar.∇f(SA[1, 2], A)
```

    2-element SVector{2, Float64} with indices SOneTo(2):
      0.0
     -0.14936120510359183

```julia
@variables a[axes(A)...], x[axes(SA[1, 2])...]
```

    2-element Vector{Symbolics.Arr{Num, N} where N}:
     a[SOneTo(2),SOneTo(2)]
     x[SOneTo(2)]

```julia
aa = collect(a)
```

\begin{equation}
\left[
\begin{array}{cc}
a{{_1}}ˏ{_1} & a{{_1}}ˏ{_2} \\
a{{_2}}ˏ{_1} & a{{_2}}ˏ{_2} \\
\end{array}
\right]
\end{equation}

```julia
collect(aa) - aa
```

\begin{equation}
\left[
\begin{array}{cc}
0 & 0 \\
0 & 0 \\
\end{array}
\right]
\end{equation}

```julia
xx = collect(x)
```

\begin{equation}
\left[
\begin{array}{c}
x{_1} \\
x{_2} \\
\end{array}
\right]
\end{equation}

```julia
_G = G(xx, aa) # |> expand |> simplify
```

\begin{equation}
e^{ - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_1} + x{_2} a{{_2}}ˏ{_1} \right) x{_1} - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_2} + x{_2} a{{_2}}ˏ{_2} \right) x{_2}}
\end{equation}

```julia
_∇G = Symbolics.gradient(_G, xx)
```

\begin{equation}
\left[
\begin{array}{c}
\left(  - x{_1} a{{_1}}ˏ{_1} - \frac{1}{2} x{_2} a{{_1}}ˏ{_2} - \frac{1}{2} x{_2} a{{_2}}ˏ{_1} \right) e^{ - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_1} + x{_2} a{{_2}}ˏ{_1} \right) x{_1} - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_2} + x{_2} a{{_2}}ˏ{_2} \right) x{_2}} \\
\left(  - \frac{1}{2} x{_1} a{{_1}}ˏ{_2} - \frac{1}{2} x{_1} a{{_2}}ˏ{_1} - x{_2} a{{_2}}ˏ{_2} \right) e^{ - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_1} + x{_2} a{{_2}}ˏ{_1} \right) x{_1} - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_2} + x{_2} a{{_2}}ˏ{_2} \right) x{_2}} \\
\end{array}
\right]
\end{equation}

```julia
_∇G / _G
```

\begin{equation}
\left[
\begin{array}{c}
\left(  - \frac{1}{2} a{{_1}}ˏ{_2} - \frac{1}{2} a{{_2}}ˏ{_1} \right) x{_2} - x{_1} a{{_1}}ˏ{_1} \\
\left(  - \frac{1}{2} a{{_1}}ˏ{_2} - \frac{1}{2} a{{_2}}ˏ{_1} \right) x{_1} - x{_2} a{{_2}}ˏ{_2} \\
\end{array}
\right]
\end{equation}

```julia
∇G = build_function(_∇G, xx, vec(aa); expression=Val(false))[1]
```

    RuntimeGeneratedFunction(#=in Symbolics=#, #=using Symbolics=#, :((ˍ₋arg1, ˍ₋arg2)->begin
              #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:282 =#
              #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:283 =#
              let var"x[1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg1[1]), var"x[2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg1[2]), var"a[1, 1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[1]), var"a[2, 1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[2]), var"a[1, 2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[3]), var"a[2, 2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[4])
                  #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:375 =#
                  (SymbolicUtils.Code.create_array)(typeof(ˍ₋arg1), nothing, Val{1}(), Val{(2,)}(), (*)((+)((*)(-1//1, var"x[1]", var"a[1, 1]"), (*)(-1//2, var"x[2]", var"a[1, 2]"), (*)(-1//2, var"x[2]", var"a[2, 1]")), (exp)((+)((*)(-1//2, (+)((*)(var"x[1]", var"a[1, 1]"), (*)(var"x[2]", var"a[2, 1]")), var"x[1]"), (*)(-1//2, (+)((*)(var"x[1]", var"a[1, 2]"), (*)(var"x[2]", var"a[2, 2]")), var"x[2]")))), (*)((+)((*)(-1//2, var"x[1]", var"a[1, 2]"), (*)(-1//2, var"x[1]", var"a[2, 1]"), (*)(-1//1, var"x[2]", var"a[2, 2]")), (exp)((+)((*)(-1//2, (+)((*)(var"x[1]", var"a[1, 1]"), (*)(var"x[2]", var"a[2, 1]")), var"x[1]"), (*)(-1//2, (+)((*)(var"x[1]", var"a[1, 2]"), (*)(var"x[2]", var"a[2, 2]")), var"x[2]")))))
              end
          end))

```julia
∇G(SA[1, 2], A)
```

    2-element SVector{2, Float64} with indices SOneTo(2):
      0.0
     -0.14936120510359183

```julia
function Bar(f, x, param)
    x_sym = collect(x)
    param_sym = collect(param)
    _f = simplify(expand(f(x_sym, param_sym)))
    _∇f = Symbolics.gradient(_f, x_sym)
    ff = build_function(_f, vec(x_sym), vec(param_sym); expression=Val(false))
    ∇f = build_function(_∇f, vec(x_sym), vec(param_sym); expression=Val(false))[1]
    Bar(ff, ∇f)
end
```

    Bar

```julia
@variables x[1:2] a[1:2, 1:2]
car = Bar(G, x, a)
```

    Bar{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2), Symbolics.var"#_RGF_ModTag", Symbolics.var"#_RGF_ModTag", (0xe8e4a897, 0x0a819225, 0x068af023, 0xc2872ebe, 0xd95a665b)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2), Symbolics.var"#_RGF_ModTag", Symbolics.var"#_RGF_ModTag", (0x65d79911, 0x5e34d083, 0xb302542c, 0xda9f6066, 0xf5f1eee2)}}(RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2), Symbolics.var"#_RGF_ModTag", Symbolics.var"#_RGF_ModTag", (0xe8e4a897, 0x0a819225, 0x068af023, 0xc2872ebe, 0xd95a665b)}(quote
        #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:282 =#
        #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:283 =#
        let var"x[1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg1[1]), var"x[2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg1[2]), var"a[1, 1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[1]), var"a[2, 1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[2]), var"a[1, 2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[3]), var"a[2, 2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[4])
            (exp)((+)((*)((+)((*)(-1//2, var"a[1, 2]"), (*)(-1//2, var"a[2, 1]")), var"x[1]", var"x[2]"), (*)(-1//2, (^)(var"x[1]", 2), var"a[1, 1]"), (*)(-1//2, (^)(var"x[2]", 2), var"a[2, 2]")))
        end
    end), RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2), Symbolics.var"#_RGF_ModTag", Symbolics.var"#_RGF_ModTag", (0x65d79911, 0x5e34d083, 0xb302542c, 0xda9f6066, 0xf5f1eee2)}(quote
        #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:282 =#
        #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:283 =#
        let var"x[1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg1[1]), var"x[2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg1[2]), var"a[1, 1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[1]), var"a[2, 1]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[2]), var"a[1, 2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[3]), var"a[2, 2]" = #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:169 =# @inbounds(ˍ₋arg2[4])
            #= D:\.julia\packages\SymbolicUtils\Hwe4r\src\code.jl:375 =#
            (SymbolicUtils.Code.create_array)(typeof(ˍ₋arg1), nothing, Val{1}(), Val{(2,)}(), (*)((+)((*)((+)((*)(-1//2, var"a[1, 2]"), (*)(-1//2, var"a[2, 1]")), var"x[2]"), (*)(-1//1, var"x[1]", var"a[1, 1]")), (exp)((+)((*)((+)((*)(-1//2, var"a[1, 2]"), (*)(-1//2, var"a[2, 1]")), var"x[1]", var"x[2]"), (*)(-1//2, (^)(var"x[1]", 2), var"a[1, 1]"), (*)(-1//2, (^)(var"x[2]", 2), var"a[2, 2]")))), (*)((+)((*)((+)((*)(-1//2, var"a[1, 2]"), (*)(-1//2, var"a[2, 1]")), var"x[1]"), (*)(-1//1, var"x[2]", var"a[2, 2]")), (exp)((+)((*)((+)((*)(-1//2, var"a[1, 2]"), (*)(-1//2, var"a[2, 1]")), var"x[1]", var"x[2]"), (*)(-1//2, (^)(var"x[1]", 2), var"a[1, 1]"), (*)(-1//2, (^)(var"x[2]", 2), var"a[2, 2]")))))
        end
    end))

```julia
x = SA[1, 2]
A = SA[2 -1; -1 2]
car.f(x, A)
```

    0.049787068367863944

```julia
car.∇f(x, A)
```

    2-element SVector{2, Float64} with indices SOneTo(2):
      0.0
     -0.14936120510359183

```julia
@btime G($x, $A)
@btime $bar.f($x, $A)
@btime $car.f($x, $A)
```

      10.822 ns (0 allocations: 0 bytes)
      11.100 ns (0 allocations: 0 bytes)
      171.429 ns (0 allocations: 0 bytes)

    0.049787068367863944

```julia
@btime $∇G($x, $A)
@btime $bar.∇f($x, $A)
@btime $car.∇f($x, $A)
```

      412.000 ns (0 allocations: 0 bytes)
      14.830 ns (0 allocations: 0 bytes)
      552.406 ns (0 allocations: 0 bytes)

    2-element SVector{2, Float64} with indices SOneTo(2):
      0.0
     -0.14936120510359183

```julia
dG(x, A) = (A + A')*x/2*G(x, A)
#dG2(x, A) = A*x*G(x, A)
@btime $dG2($x, $A)
```

      12.500 ns (0 allocations: 0 bytes)

    2-element SVector{2, Float64} with indices SOneTo(2):
     0.0
     0.14936120510359183

```julia
dG(xx, aa)
```

\begin{equation}
\left[
\begin{array}{c}
\left( x{_1} a{{_1}}ˏ{_1} + \frac{1}{2} \left( a{{_1}}ˏ{_2} + a{{_2}}ˏ{_1} \right) x{_2} \right) e^{ - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_1} + x{_2} a{{_2}}ˏ{_1} \right) x{_1} - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_2} + x{_2} a{{_2}}ˏ{_2} \right) x{_2}} \\
\left( x{_2} a{{_2}}ˏ{_2} + \frac{1}{2} \left( a{{_1}}ˏ{_2} + a{{_2}}ˏ{_1} \right) x{_1} \right) e^{ - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_1} + x{_2} a{{_2}}ˏ{_1} \right) x{_1} - \frac{1}{2} \left( x{_1} a{{_1}}ˏ{_2} + x{_2} a{{_2}}ˏ{_2} \right) x{_2}} \\
\end{array}
\right]
\end{equation}

```julia

```


# CF Part 2 - User-based method
> Collaborative Filtering on MovieLens Latest-small Part 2 - Finding recommendations using memory based user-user similarity method

- toc: true
- badges: true
- comments: true
- categories: [movie, collaborative]
- image:

### Download tools

```python
import os

if not (os.path.exists("recsys.zip") or os.path.exists("recsys")):
    !wget https://github.com/nzhinusoftcm/review-on-collaborative-filtering/raw/master/recsys.zip
    !unzip recsys.zip
```

### Import requirements
```
matplotlib==3.2.2
numpy==1.19.2
pandas==1.0.5
python==3.7
scikit-learn==0.24.1
scikit-surprise==1.1.1
scipy==1.6.2
```

```python
from sklearn.neighbors import NearestNeighbors
from scipy.sparse import csr_matrix

from recsys.datasets import ml100k
from recsys.preprocessing import ids_encoder

import pandas as pd
import numpy as np
import zipfile
```

### Load MovieLen ratings

```python
ratings, movies = ml100k.load()
```

### userids and itemids encoding

```python
# create the encoder
ratings, uencoder, iencoder = ids_encoder(ratings)
```

### Transform rating dataframe to matrix

```python
def ratings_matrix(ratings):
    return csr_matrix(pd.crosstab(ratings.userid, ratings.itemid, ratings.rating, aggfunc=sum).fillna(0).values)

R = ratings_matrix(ratings)
```

Memory based collaborative filtering (CF) also known as nearest neighbors based CF makes recommendation based on similar behavious of users and items. There are two types of memory based CF : user-based and item-based CF. Both of these algorithm usually proceed in three stages :

1. Similarity computation (between users or items)
2. Rating prediction (using ratings of similar users or items)
3. Top-N recommendation

## Idea

Let $u$ be the user for which we plan to make recommendations.

1. Find other users whose past rating behavior is similar to that of $u$
2. Use their ratings on other items to predict what the current user will like

## Algorithm : user-to-user collaborative filtering

The entire process of user-to-user CF algorithm is described as follow (J. Bobadilla et al. 2013): For an active user $u$,

 First identify the set $G_u$ of $k$ most similar users. $G_u$ is the group users similar to the active user $u$. The similarity between two users $u$ and $v$ can be measured by the cosine similarity measure as follows :

\begin{equation}
 w_{u,v}=\frac{\vec{r}_u \cdot \vec{r}_v}{\|\vec{r}_u\|_2 \ast \|\vec{r}_v\|_2} = \frac{\sum_{i\in I}r_{u,i}r_{v,i}}{\sqrt{\sum_{i\in I} (r_{u,i})^2}\sqrt{\sum_{i\in I} (r_{v,i})^2}}
\end{equation}

$w_{u,v}$ is the degree of similarity between users $u$ and $v$. This term is computed for all $v\in U$, where $U$ is the set of all users. There remains the question of how many neighbors to select. As experimented by (Herlocker et al. 1999), $k\in [20,50]$ is a reasonable starting point in many domains.

 Find the set $C$ of candidate items, purchased by the group and not purchased by the active user $u$. Candidate items have to be the most frequent items purchased by the group.

Aggregate ratings of users in $G_u$ to make predictions for user $u$ on items he has not already purchased. Several aggregation approaches are often used such as average, weighted sum, ajusted weighted sum. By using weighted sum, the predicted rating of user $u$ on item $i$ denoted by $\hat{r}_{u,i}$ is computed as follow :

\begin{equation}
 \hat{r}_{u,i}=\bar{r}_u + \frac{\sum_{v\in G_u}(r_{v,i}-\bar{r}_v)\cdot w_{u,v}}{\sum_{v\in G_u}|w_{u,v}|}.
\end{equation}

Ratings of similar users are weighted by the corresponding similarity with the active user. Summation are made over all the users who have rated item $i$. Subtracting the user’s mean rating $\bar{r}_v$ compensates for differences in users’ use of the rating scale as some users will tend to give higher ratings than others (Michael D. Ekstrand, et al. 2011). This prediction is made for all items $i \in C$ not purchased by user $u$.

The Top-$N$ recommendations are obtained by choosing the $N$ items which provide most satisfaction to the user according to prediction.

### Step 1. Identify $G_u$, the set of $k$ users similar to an active user $u$

To find the $k$ most similar users to $u$, we use the cosine similarity and compute $w_{u,v}$ for all $v\in U$. Fortunately, libraries such as scikit-learn (sklearn) are very useful for such tasks :

1. First of all, we create a nearest neighbors model with sklearn through the function ```create_model()```. This function creates and fit a nearest neighbors model with user's ratings. We can choose ```cosine``` or ```euclidian``` based similarity metric. ```n_neighbors=21``` define the number of neighbors to return. With $k=20$ neighbors, $|G_u|=21$ as $G_u$ contains $20$ similar users added to the active user $u$. That is why ```n_neighbors=21```. Each row $r_u$ of the rating matrix $R$ represents ratings of user $u$ on all items of the database. Missing ratings are replaced with $0.0$.
```python
R[u,:] # uth row of the rating matrix R. Ratings of user u on all items in the database
```
2. Function ```nearest_neighbors()``` returns the knn users for each user.

```python
def create_model(rating_matrix, metric):
    """
    - create the nearest neighbors model with the corresponding similarity metric
    - fit the model
    """
    model = NearestNeighbors(metric=metric, n_neighbors=21, algorithm='brute')
    model.fit(rating_matrix)
    return model
```

```python
def nearest_neighbors(rating_matrix, model):
    """
    :param rating_matrix : rating matrix of shape (nb_users, nb_items)
    :param model : nearest neighbors model
    :return
        - similarities : distances of the neighbors from the referenced user
        - neighbors : neighbors of the referenced user in decreasing order of similarities
    """
    similarities, neighbors = model.kneighbors(rating_matrix)
    return similarities[:, 1:], neighbors[:, 1:]
```

Let's call functions ```create_model()``` and ```nearest_neighbors()``` to respectively create the $k$-NN model and compute the nearest neighbors for a given user

```python
model = create_model(rating_matrix=R, metric='cosine') # we can also use the 'euclidian' distance
similarities, neighbors = nearest_neighbors(R, model)
```

### Step 2. Find candidate items

The set $C$ of candidate items are the most frequent ones purchased by users in $G_u$ for an active user $u$ and not purchased by $u$.

Function ```find_candidate_items()``` : find items purchased by these similar users as well as their frequency. Note that the frequency of the items in the set $C$ can be computed by just counting the actual occurrence frequency of that items.

1. ```Gu_items``` : frequent items of $G_u$ in decreasing order of frequency.
2. ```active_items``` : items already purchased by the active user
3. ```candidates``` : frequent items of $G_u$ not purchased by the active user $u$

```python
def find_candidate_items(userid):
    """
    Find candidate items for an active user

    :param userid : active user
    :param neighbors : users similar to the active user
    :return candidates : top 30 of candidate items
    """
    user_neighbors = neighbors[userid]
    activities = ratings.loc[ratings.userid.isin(user_neighbors)]

    # sort items in decreasing order of frequency
    frequency = activities.groupby('itemid')['rating'].count().reset_index(name='count').sort_values(['count'],ascending=False)
    Gu_items = frequency.itemid
    active_items = ratings.loc[ratings.userid == userid].itemid.to_list()
    candidates = np.setdiff1d(Gu_items, active_items, assume_unique=True)[:30]

    return candidates
```

### Step 3. Rating prediction

Now it's time to predict what score the active user $u$ would have given to each of the top-30 candidate items.

To predict the score of $u$ on a candidate item $i$ ,we need :
1. Similarities between $u$ and all his neighbors $v \in G_u$ who rated item $i$ : function ```nearest_neighbors()``` returns similar users of a user as well as their corresponding similarities.
2. Normalized ratings of all $v \in G_u$ on item $i$. The normalized rating of user $v$ on item $i$ is defined by $r_{v,i}-\bar{r}_v$.

Next, let's compute the mean rating of each user and the normalized ratings for each item. The DataFrame ```mean``` contains mean rating for each user. With the mean rating of each user, we can add an extra column ```norm_rating``` to the ```ratings```'s DataFrame which can be accessed to make predictions.

```python
# mean ratings for each user
mean = ratings.groupby(by='userid', as_index=False)['rating'].mean()
mean_ratings = pd.merge(ratings, mean, suffixes=('','_mean'), on='userid')

# normalized ratings for each items
mean_ratings['norm_rating'] = mean_ratings['rating'] - mean_ratings['rating_mean']

mean = mean.to_numpy()[:, 1]
```

```python
np_ratings = mean_ratings.to_numpy()
```

Let us define function ```predict``` that predict rating between user $u$ and item $i$. Recall that the prediction formula is defined as follow :

\begin{equation}
 \hat{r}_{u,i}=\bar{r}_u + \frac{\sum_{v\in G_u}(r_{v,i}-\bar{r}_v)\cdot w_{u,v}}{\sum_{v\in G_u}|w_{u,v}|}.
\end{equation}

```python
def predict(userid, itemid):
    """
    predict what score userid would have given to itemid.

    :param
        - userid : user id for which we want to make prediction
        - itemid : item id on which we want to make prediction

    :return
        - r_hat : predicted rating of user userid on item itemid
    """
    user_similarities = similarities[userid]
    user_neighbors = neighbors[userid]
    # get mean rating of user userid
    user_mean = mean[userid]

    # find users who rated item 'itemid'
    iratings = np_ratings[np_ratings[:, 1].astype('int') == itemid]

    # find similar users to 'userid' who rated item 'itemid'
    suri = iratings[np.isin(iratings[:, 0], user_neighbors)]

    # similar users who rated current item (surci)
    normalized_ratings = suri[:,4]
    indexes = [np.where(user_neighbors == uid)[0][0] for uid in suri[:, 0].astype('int')]
    sims = user_similarities[indexes]

    num = np.dot(normalized_ratings, sims)
    den = np.sum(np.abs(sims))

    if num == 0 or den == 0:
        return user_mean

    r_hat = user_mean + np.dot(normalized_ratings, sims) / np.sum(np.abs(sims))

    return r_hat
```

Now, we can make rating prediction for a given user on each item in his set of candidate items.

```python
def user2userPredictions(userid, pred_path):
    """
    Make rating prediction for the active user on each candidate item and save in file prediction.csv

    :param
        - userid : id of the active user
        - pred_path : where to save predictions
    """
    # find candidate items for the active user
    candidates = find_candidate_items(userid)

    # loop over candidates items to make predictions
    for itemid in candidates:

        # prediction for userid on itemid
        r_hat = predict(userid, itemid)

        # save predictions
        with open(pred_path, 'a+') as file:
            line = '{},{},{}\n'.format(userid, itemid, r_hat)
            file.write(line)
```

```python
import sys

def user2userCF():
    """
    Make predictions for each user in the database.
    """
    # get list of users in the database
    users = ratings.userid.unique()

    def _progress(count):
        sys.stdout.write('\rRating predictions. Progress status : %.1f%%' % (float(count/len(users))*100.0))
        sys.stdout.flush()

    saved_predictions = 'predictions.csv'
    if os.path.exists(saved_predictions):
        os.remove(saved_predictions)

    for count, userid in enumerate(users):
        # make rating predictions for the current user
        user2userPredictions(userid, saved_predictions)
        _progress(count)
```

```python
user2userCF()
```

    Rating predictions. Progress status : 99.9%

### Step 4. Top-N recommendation

Function ```user2userRecommendation()``` reads predictions for a given user and return the list of items in decreasing order of predicted rating.

```python
def user2userRecommendation(userid):
    """
    """
    # encode the userid
    uid = uencoder.transform([userid])[0]
    saved_predictions = 'predictions.csv'

    predictions = pd.read_csv(saved_predictions, sep=',', names=['userid', 'itemid', 'predicted_rating'])
    predictions = predictions[predictions.userid==uid]
    List = predictions.sort_values(by=['predicted_rating'], ascending=False)

    List.userid = uencoder.inverse_transform(List.userid.tolist())
    List.itemid = iencoder.inverse_transform(List.itemid.tolist())

    List = pd.merge(List, movies, on='itemid', how='inner')

    return List
```

```python
user2userRecommendation(212)
```

      userid
      itemid
      predicted_rating
      title

      0
      212
      483
      4.871495
      Casablanca (1942)

      1
      212
      357
      4.764547
      One Flew Over the Cuckoo's Nest (1975)

      2
      212
      50
      4.660002
      Star Wars (1977)

      3
      212
      98
      4.613636
      Silence of the Lambs, The (1991)

      4
      212
      64
      4.550733
      Shawshank Redemption, The (1994)

      5
      212
      194
      4.522336
      Sting, The (1973)

      6
      212
      174
      4.521300
      Raiders of the Lost Ark (1981)

      7
      212
      134
      4.414819
      Citizen Kane (1941)

      8
      212
      187
      4.344531
      Godfather: Part II, The (1974)

      9
      212
      196
      4.303696
      Dead Poets Society (1989)

      10
      212
      523
      4.281802
      Cool Hand Luke (1967)

      11
      212
      216
      4.278246
      When Harry Met Sally... (1989)

      12
      212
      100
      4.260087
      Fargo (1996)

      13
      212
      168
      4.206139
      Monty Python and the Holy Grail (1974)

      14
      212
      435
      4.122984
      Butch Cassidy and the Sundance Kid (1969)

      15
      212
      135
      4.115228
      2001: A Space Odyssey (1968)

      16
      212
      83
      4.106995
      Much Ado About Nothing (1993)

      17
      212
      69
      4.086366
      Forrest Gump (1994)

      18
      212
      70
      4.086328
      Four Weddings and a Funeral (1994)

      19
      212
      275
      3.985037
      Sense and Sensibility (1995)

      20
      212
      153
      3.981619
      Fish Called Wanda, A (1988)

      21
      212
      514
      3.956640
      Annie Hall (1977)

      22
      212
      521
      3.937792
      Deer Hunter, The (1978)

      23
      212
      97
      3.906106
      Dances with Wolves (1990)

      24
      212
      173
      3.879325
      Princess Bride, The (1987)

      25
      212
      660
      3.847897
      Fried Green Tomatoes (1991)

      26
      212
      215
      3.709920
      Field of Dreams (1989)

      27
      212
      258
      3.583718
      Contact (1997)

      28
      212
      202
      3.508617
      Groundhog Day (1993)

      29
      212
      237
      3.039041
      Jerry Maguire (1996)

Let us make top n recommendation for a given user.

## Evaluation with Mean Absolute Error (MAE)

```python
from recsys.preprocessing import train_test_split, get_examples

# get examples as tuples of userids and itemids and labels from normalize ratings
raw_examples, raw_labels = get_examples(ratings, labels_column='rating')

# train test split
(x_train, x_test), (y_train, y_test) = train_test_split(examples=raw_examples, labels=raw_labels)

def evaluate(x_test, y_test):
    print('Evaluate the model on {} test data ...'.format(x_test.shape[0]))
    preds = list(predict(u,i) for (u,i) in x_test)
    mae = np.sum(np.absolute(y_test - np.array(preds))) / x_test.shape[0]
    print('\nMAE :', mae)
    return mae
```

```python
evaluate(x_test, y_test)
```

    Evaluate the model on 10000 test data ...

    MAE : 0.7505910931068639

    0.7505910931068639

## Summary

We have summarised all the steps of building the user-based collaborative filtering into a python class for further user. Click [UserToUser.py](https://github.com/nzhinusoftcm/review-on-collaborative-filtering/blob/master/recsys/memories/UserToUser.py) for more details on the **UserToUser** class definition.

```python
from recsys.memories.UserToUser import UserToUser

# load ml100k ratings
ratings, movies = ml100k.load()

# prepare data
ratings, uencoder, iencoder = ids_encoder(ratings)

# get examples as tuples of userids and itemids and labels from normalize ratings
raw_examples, raw_labels = get_examples(ratings, labels_column='rating')

# train test split
(x_train, x_test), (y_train, y_test) = train_test_split(examples=raw_examples, labels=raw_labels)
```

```python
# create the user-based CF
usertouser = UserToUser(ratings, movies, metric='cosine')
```

    Normalize users ratings ...
    Initialize the similarity model ...
    Compute nearest neighbors ...
    User to user recommendation model created with success ...

```python
# evaluate the user-based CF on the ml100k test data
usertouser.evaluate(x_test, y_test)
```

    Evaluate the model on 10000 test data ...

    MAE : 0.7505910931068639

    0.7505910931068639

## Evaluation on the ML-1M dataset (this may take some time)

```python
from recsys.datasets import ml1m
from recsys.preprocessing import ids_encoder, get_examples, train_test_split
from recsys.memories.UserToUser import UserToUser

# load ml100k ratings
ratings, movies = ml1m.load()

# prepare data
ratings, uencoder, iencoder = ids_encoder(ratings)

# get examples as tuples of userids and itemids and labels from normalize ratings
raw_examples, raw_labels = get_examples(ratings, labels_column='rating')

# train test split
(x_train, x_test), (y_train, y_test) = train_test_split(examples=raw_examples, labels=raw_labels)

# create the user-based CF
usertouser = UserToUser(ratings, movies, k=20, metric='cosine')

# evaluate the user-based CF on the ml1m test data
print("==========================")
usertouser.evaluate(x_test, y_test)
```

    Normalize users ratings ...
    Initialize the similarity model ...
    Compute nearest neighbors ...
    User to user recommendation model created with success ...
    ==========================
    Evaluate the model on 100021 test data ...

    MAE : 0.732267005840993

    0.732267005840993

## Limitations of user-based CF

1.  Sparsity  : In general, users interact with less than 20% of items. This leads the rating matrix to be highly sparse. For example, the movielen-100k contains 100k ratings from 943 users on 1682 items. The pourcentage of sparsity in this case is around $94\%$. A recommender system based on nearest neighbor algorithms may be unable to make any item recommendations for a particular user. As a result the accuracy of recommendations may be poor (Sarwar et al. 2001).

2.  Stability of user's ratings  : As a user rates and re-rates items, their rating vector will change along with their similarity to other users. A user’s neighborhood is determined not only by their ratings but also by the ratings of other users, so their neighborhood can change as a result of new ratings supplied by any user in the system (Michael D. Ekstrand, et al. 2011).

3.  Scalability  : Due to the non-stability of users ratings, finding similar users in advance is complicated. For this reason, most user-based CF systems find neighborhoods each time predictions or recommendations are needed. However, these are huge computations that grows with both the number of users and the number of items. With millions of users and items, a typical web-based recommender system running existing algorithms will suffer serious scalability concerns (Sarwar et al. 2001), (Michael D. Ekstrand, et al. 2011).

## References

1. Herlocker et al. (1999) An Algorithmic Framework for Performing Collaborative Filtering
2. Sarwar et al. (2001)  Item-based collaborative filtering recommendation algorithms
3. Michael D. Ekstrand, et al. (2011).  Collaborative Filtering Recommender Systems
4. J. Bobadilla et al. (2013) Recommender systems survey

## Author

[Carmel WENGA](https://www.linkedin.com/in/carmel-wenga-871876178/),

PhD student at Université de la Polynésie Française,

Applied Machine Learning Research Engineer,

[ShoppingList](https://shoppinglist.cm), NzhinuSoft.


# Holonomic Robot

**need to fix formatting**

Robots come in a variety of types and configurations: wheeled, tracked, legs, flying, etc. Common wheeled robots typically have two wheels (directly driven) with a caster wheel to make the robot stable. There are some without the caster wheel and employ a control system to keep them upright (inverted pendulum
problem) and resemble a Segway scooter. All of these two wheeled robot are
non-holonomic systems.

A **nonholonomic system** in physics and mathematics is a system whose state depends on the path taken in order to achieve it. A car is a typical example of a nonholonomic system. It can occupy any location (x,y) in any orientation ($\phi$), but the path from one location/orientation to another position/orientation is not always linear. Thus you have to parallel park.

A **holonomic system** is not constrained by this. Thus it can move anywhere at will.

```python
%matplotlib inline
```

```python
from __future__ import division
from __future__ import print_function
import numpy as np
from numpy.linalg import norm
from math import cos, sin, pi
import matplotlib.pyplot as plt
```

## Soccer

These types of robots are great for soccer, especially the goalie. Typical configuration are 3 or 4 motors. Omni wheels allow slip perpendicular to the axis.

Due to these constraints, a holonomic robot which could travel in any direction and immediately change its position and orientation is much more desirable. There are a variety of different wheels which make this type of robot possible such as mecanum or omni wheels.

Omni wheels operate like standard wheels in that the force is produced normal to the motor's axis of rotation and as a result of friction. However, there are a series of smaller wheels which ring the main wheel and allow the wheel to slip in the direction of the motor rotational axis. Note that no force is produced parallel to the motor axis, just slippage.

## Dynamics

The dynamics for a holonomic robot with 4 omni directional wheels (can be derived using Euler-Largrange ($\mathcal{L}$) which defines a system's kinectic ($T$) and potential ($V$) energies in relation to a set of generalized coordinates ($q$) and generalized forces ($Q$):

\begin{equation}
    \newcommand{\dpar}[2]{\frac{\partial #1}{\partial #2}}
\end{equation}

\begin{equation}
	\mathcal{L}=T-V \\
	\frac{d}{dt} \left\{ \dpar{ \mathcal{L} }{\dot q} \right\} - \dpar{ \mathcal{L} }{q} = Q \\
	T = \frac{1}{2}M v_w^2+ \frac{1}{2}J \dot \psi^2 + \frac{1}{2} J_w (\dot \theta_1^2 + \dot \theta_2^2 + \dot \theta_3^2 + \dot \theta_4^2) \\
	V = 0
\end{equation}

However, the dynamics must be calculated from an inertial reference frame (${W}$) and take into account the rotating body frame dynamics (${B'}$). Now, assume the body frame is offset from the center of mass (CM) by $x_m$ and $y_m$ which compose a vector $r_m$. Thus the velocity of the robot in the rotating frame would be:

\begin{equation*}
	v_w = v_{B'} + \dot \psi \times r_m \\
	v_w = v_{B'} +
	\begin{bmatrix}
		0 & 0 & \dot \psi
	\end{bmatrix}^T
	\times
	\begin{bmatrix}
		x_m & y_m & 0
	\end{bmatrix}^T
    =
	\begin{bmatrix}
		\dot x & \dot y & 0
	\end{bmatrix}^T +
	\begin{bmatrix}
		-y_m \dot \psi & x_m \dot \psi & 0
	\end{bmatrix}^T \\
	v_{B'} = \begin{bmatrix}
		\dot x & \dot y & 0
	\end{bmatrix}^T
\end{equation*}

where $v_{B'}$ is the speed of the body frame. Now substituting that into
the above kinetic energy equation $T$, we get:

\begin{equation}
	T = \frac{1}{2}M( ( \dot x - \dot \psi y )^2 + (\dot y + \dot \psi x)^2)+ \dots \\
	T = \frac{1}{2}M( \dot x^2 - 2 \dot \psi y_m \dot x +\dot \psi^2 y_m^2 + \dot y^2 + 2 \dot \psi x_m \dot y + \dot \psi^2 x_m^2)+ \frac{1}{2}J \dot \psi^2 + \frac{1}{2} J_w (\dot \theta_1^2 + \dot \theta_2^2 + \dot \theta_3^2 + \dot \theta_4^2)  \\
	\frac{d}{dt} \left\{ \dpar{ \mathcal{L} }{\dot x} \right\} = M ( \ddot x - \ddot \psi y - \dot \psi \dot y ) \hspace{1cm} \dpar{ \mathcal{L} }{x} = M(\dot \psi \dot y + \dot \psi^2 x) \\
	\frac{d}{dt} \left\{ \dpar{ \mathcal{L} }{\dot y} \right\} = M (\ddot y + \ddot \psi x + \dot \psi \dot x) \hspace{1cm} \dpar{ \mathcal{L} }{y} = M( -\dot \psi \dot x + \dot \psi^2 y) \\
	\frac{d}{dt} \left\{ \dpar{ \mathcal{L} }{\dot \psi} \right\} = J \ddot \psi \hspace{1cm} \dpar{ \mathcal{L} }{\phi} = 0 \\
	\frac{d}{dt} \left\{ \dpar{ \mathcal{L} }{\dot \theta} \right\} = J_w \sum \limits_{i=1}^4 \ddot \theta_i \hspace{1cm} \dpar{ \mathcal{L} }{\theta} = 0
\end{equation}

Now we make the following assumptions: ${B'}$ is coincident with
${B}$, $x_m = 0$, $y_m = 0$, $\dot x = v_x$,
$\dot y = v_y$

\begin{equation*}
	F_x = M (\ddot x - 2 \dot \psi \dot y ) \\
	F_y = M (\ddot y + 2 \dot \psi \dot x) \\
	T = J \ddot \psi \\
	\tau_w = J_w \ddot \theta_1 \hspace{1cm}
	\tau_w = J_w \ddot \theta_2 \hspace{1cm}
	\tau_w = J_w \ddot \theta_3 \hspace{1cm}
	\tau_w = J_w \ddot \theta_4
\end{equation*}

\begin{equation*}
	\begin{bmatrix}
		F_x \\
		F_y \\
		T
	\end{bmatrix} =
	\begin{bmatrix}
		M & 0 & 0 \\
		0 & M & 0 \\
		0 & 0 & J
	\end{bmatrix}
	\begin{bmatrix}
		\ddot x \\
		\ddot y \\
		\ddot \psi
	\end{bmatrix} +
	\begin{bmatrix}
		0 & -2M \dot \psi & 0 \\
		2M \dot \psi & 0 & 0 \\
		0 & 0 & 0
	\end{bmatrix}
	\begin{bmatrix}
		\dot x \\
		\dot y \\
		\dot \psi
	\end{bmatrix}
	= \mathcal{M} \ddot X + \mathcal{O} \dot X = Q
\end{equation*}

## World Coordinates

Now the dynamics derived so far are all in the body frame and we could stop
here and develop a controller which performs velocity control. However, position
control is more useful and a transform needs to be performed to move the
velocities and accelerations into the world frame.

\begin{equation}
	\dot X^W = R_B^W \dot X^B \\
	R_B^W =
	\begin{bmatrix}
		\cos \psi & \sin \psi & 0 \\
		-\sin \psi & \cos \psi & 0 \\
		0 & 0 & 1
	\end{bmatrix} \\
	\ddot X^W = \dot R_B^W \dot X^B + R_B^W \ddot X^B \\
	\dot R_B^W =
	\begin{bmatrix}
		\sin \psi & -\cos \psi & 0 \\
		\cos \psi & \sin \psi & 0 \\
		0 & 0 & 1
	\end{bmatrix}
\end{equation}

Now, substituting this into the dynamics, gives dynamics in the world
coordinate system of:

\begin{equation}
	F = \mathcal{M} (\dot R \dot X + R \ddot X )  + \mathcal{O} R \dot X \\
	F = \mathcal{M} R \ddot X + (\mathcal{M}  \dot R + \mathcal{O} R) \dot X
\end{equation}

## External Forces and Torques

Now summing the forces into their body referenced $x$ and $y$ directions and the torque about the $z$ axis, gives us:

\begin{equation}
    \sum F_x=f_1 \sin(\phi) - f_2 \sin(\phi) - f_3 \sin(\phi) + f_4 \sin(\phi) \\
	\sum F_y=f_1 \cos(\phi) + f_2 \cos(\phi) - f_3 \cos(\phi) - f_4 \cos(\phi) \\
	\sum T=L(f_1+f_2+f_3+f_4)
\end{equation}

Additionally, we can simplify this by assuming all of the angles are the same
(e.g., $\phi_1 = \phi_2 = \phi_3 = \phi_4$) and can now put this into a
matrix form:

\begin{equation}
	\begin{bmatrix}
		F_x \\
		F_y \\
		T
	\end{bmatrix} =
	\begin{bmatrix}
		\sin(\phi) & 0 & 0 \\
		0 & \cos(\phi) & 0 \\
		0 & 0 & L
	\end{bmatrix}
	\begin{bmatrix}
		1 & -1 & -1 & 1\\
		1 & 1 & -1 & -1\\
		1 & 1 & 1& 1
	\end{bmatrix}
	\begin{bmatrix}
		f_1 \\
		f_2 \\
		f_3 \\
		f_4
	\end{bmatrix}
\end{equation}

where $\phi$ is again the angle of the motors, $f_i$ is the magnitude of the force produced by the motors, and $L$ is the radius of the robot.

where $pinv()$ is defined as the pseudoinverse since $A(\phi)$ is not a square matrix. Finally, substituting these into the original equation, we can calculate the torques given the desired accelerations.

\begin{equation}
	\begin{bmatrix} \tau_1 \\  \tau_2 \\  \tau_3 \\  \tau_4 \end{bmatrix} = \frac {M r_w} {4}
	\begin{bmatrix}
		-1 & 1 & 1 \\
		-1 & -1 & 1 \\
		1 & -1 & 1 \\
		1 & 1 & 1
	\end{bmatrix}
	\begin{bmatrix}
		\frac{1}{\sin(\phi)} & 0 & 0 \\
		0 & \frac{1}{\cos(\phi)} & 0 \\
		0 & 0 & \frac{1}{2}
	\end{bmatrix}
	\begin{bmatrix}
		a_x \\
		a_y \\
		R \dot \omega
	\end{bmatrix}
\end{equation}

Now looking at this equation, we notice that $\phi$ can not be equal to 0, 90, 180, 270, or 360 otherwise we get a singularity in the $A(\phi)$ matrix. This however is not an issue in the real world, since the motors would occupy the same physical space and the robot would essentially only have 2 and not 4 motors.

## Holonomic Robot Kinematics

Number of equivalent motors for any direction under linear movement only, no rotational movement allowed.

Now performing a similar exercise for what was done with the dynamics, looking
at $coordinate$, the velocity of motor 1is given by
$v_1 = -\sin(\phi) v_x + \cos(\phi) v_y + R \omega$. Performing this for
each wheel gives:

\begin{equation}
	\begin{bmatrix}
		v_1 \\
		v_2 \\
		v_3 \\
		v_4
	\end{bmatrix} =
	\begin{bmatrix}
		-\sin(\phi)  & \cos(\phi) & L \\
		-\sin(\phi) & -\cos(\phi) & L \\
		 \sin(\phi) & -\cos(\phi) & L \\
		 \sin(\phi)  & \cos(\phi) & L
	\end{bmatrix}
	\begin{bmatrix}
		v_x \\
		v_y \\
		\omega
	\end{bmatrix} =
	\begin{bmatrix}
		-1 & 1 & 1 \\
		-1 & -1 & 1 \\
		1 & -1 & 1 \\
		1 & 1 & 1
	\end{bmatrix}
	\begin{bmatrix}
		\sin(\phi) & 0 & 0 \\
		0 & \cos(\phi) & 0 \\
		0 & 0 & L
	\end{bmatrix}
	\begin{bmatrix}
		v_x \\
		v_y \\
		\omega
	\end{bmatrix}
\end{equation}

Now setting $\omega$ to zero and calculating only linear movement, we can
determine the number of equivalent motors as shown in
\figref{fig:equivalent_motors}. For example, setting
$\phi$ to 30 $^\circ$ (the red line in \figref{fig:equivalent_motors})
and traveling in the x direction only ($\begin{bmatrix} v_x & v_y & \omega \end{bmatrix}^T = \begin{bmatrix}1& 0 & 0 \end{bmatrix}^T$),
the above equation simplifies to $4 \sin(30)$ or 2 equivalent motors.
Repeating for the y direction results in $4 \cos(30)$ or 3.46 equivalent
motors.

Now it is interesting to note that when $\phi$ is set to 30 $^\circ$,
the robot has more equivalent motors when going forward or backwards, while a
$\phi$ of 60 $^\circ$ provides more equivalent motors moving left or right.
When the motors are are angled at 45 $^\circ$, movement is clearly equally
optimized for both forward/backwards and left/right ( $2 \sin(45)$ is 2.83 motors)
movement.

\figref{fig:equivalent_motors} tells us that no mater how the 4 motors are
oriented in a realistic configuration, the robot will never have the equivalent
use of all 4 motors. Movement in one direction or another can be optimized, but
then a sacrifice is made in another direction. This fact is intuitively obvious.

Another issue is these results are also ideal. This logic assumes that the wheels
will not slip and have good traction in any orientation. Unfortunately real world
results do not mimic this situation and the robot's performance will be reduced.

## Equvilent Motors

Since this robot has 4 motors with omni wheels, in certain configurations, you can get more power/speed in certain directions than others. Let's look at a configuration with the wheels oriented at the $\phi$ angles of: 30, 45, and 60 degrees.

```python
def motors(phi, angles):
    """
    in:
        phi - orientation of motors
        angles - array of angles from 0 ... 2*pi
    out:
        array of results
    """
    phi = phi*pi/180
    ans = []
    for angle in angles:
        a = np.array([
                [-sin(phi),  cos(phi), 1.0],
                [-sin(phi), -cos(phi), 1.0],
                [ sin(phi), -cos(phi), 1.0],
                [ sin(phi),  cos(phi), 1.0]
            ])
        b = np.array([
                cos(angle),
                sin(angle),
                0.0
            ])
        v = sum(abs(a.dot(b)))
        ans.append(v)
    return ans
```

```python
theta = np.arange(0,2*pi,0.1)
em30 = motors(30, theta)
em45 = motors(45, theta)
em60 = motors(60, theta)
```

```python
plt.polar(theta, em30, label='30');
plt.polar(theta, em45, label='45');
plt.polar(theta, em60, label='60');
plt.grid(True)
plt.legend(loc='upper left');
```

As seen above, 30 and 60 degrees orientations favor certain directions. However, 45 degrees is better in all directions and all around movement.

## Control

Using the equations above to transform desired movement into motor actions, the robot can move in any direction. A basic summery of robot movement to motor direction is shown above.

# References

---

* Alexander Gloye, Raul Rojas, Holonomic Control of a Robot with an Omnidirectional Drive, accepted for publication by Künstliche Intelligenz, Springer-Verlag, 2006.

* http://en.wikipedia.org/wiki/Non-holonomic_system

* http://en.wikipedia.org/wiki/Lagrangian_mechanics

* http://www.kornylak.com

* R. Balakrishna, Ashitava Ghosal, "Modeling of Slip for Wheeled Mobile Robots," lEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. I I , NO. I , FEBRUARY 1995, pp. 126-132

* J. Agullo, S. Cardona, and J. Vivancos, “Kinematics of vehicles with directional sliding wheels,” Mechanisms and Muchine Theory, vol. 22, no. 4, pp. 295-301, 1987.

* Pseudoinverse: for m > n: $A_{left}^{-1}=(A^TA)^{-1}A^T$ or m < n: $A_{right}^{-1} = A^T(AA^T)^{-1}$ such that $AA^{-1}=I$ or $A^{-1}A=I$

* Masayoshi Wada (2010). Motion Control of a Four-wheel-drive Omnidirectional Wheelchair with High Step Climbing Capability, Climbing and Walking Robots, Behnam Miripour (Ed.), ISBN: 978-953-307-030-8, [InTech](http://www.intechopen.com/books/climbing-and-walking-robots/motion-control-of-a-four-wheeldrive-omnidirectional-wheelchair-with-high-step-climbing-capability)

-----------

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.


## Introduction and Theory

One of the biggest challenges of dynamical systems theory or nonlinear dynamics is the development of mathematical techniques that provide us with the capability of exploring the geometrical template of structures that governs transport in phase space. Since the early 1900, the idea of pursuing a qualitative description of the solutions of differential equations, which emerged from the pioneering work carried out by Henri Poincaré on the three body problem of celestial mechanics {% cite hp1890 --file LDs %}, has had a profound impact on our understanding of the nonlinear character of natural phenomena. Indeed, this powerful approach has now been widely embraced by the scientific community and its essence was nicely captured by Vladimir Arnold's statement that a complete description of classical mechanics boils down to the geometrical analysis of phase space.

In this section we present the details of a mathematical tool whose potential brings us  one step closer to fulfilling the long sought after dream envisioned by Poincaré. The method is known in the literature as Lagrangian descriptors (LDs) and has the capability of revealing the geometrical template of phase space structures that characterizes trajectories with qualitatively distinct dynamical behavior. As we will see, this method provides us with a systematic way of exploring phase space by means of looking at its dynamical skeleton using low-dimensional slices. This procedure allows for a complete reconstruction, that is, a *phase space tomography*, of the intricate geometry of underlying invariant manifolds that characterize transport mechanisms.

Consider a general dynamical system:

\begin{equation}
\dfrac{d\mathbf{x}}{dt} = \mathbf{v}(\mathbf{x},t) \;,\quad \mathbf{x} \in \mathbb{R}^{n} \;,\; t \in \mathbb{R} \;,
\label{eq:gtp_dynSys}
\end{equation}

where the vector field satifies $\mathbf{v}(\mathbf{x},t) \in C^{r} (r \geq 1)$ in $\mathbf{x}$ and continuous in time. If the vector field does not depend on time, the system is called *autonomous*, and it is *non-autonomous* otherwise. For any initial condition $\mathbf{x}(t_0) = \mathbf{x}_0$ this system of differential equations has a unique solution with the same regularity as that of the vector field, which also depends continuously on the initial data {% cite coddington1984 --file LDs %}. The vector field can be specified as an analytical model, or it could also have been retrieved from numerical simulations as a discrete spatio-temporal dataset.

The method of Lagrangian Descriptors was first introduced to analyze Lagrangian transport and mixing in geophysical flows {% cite madrid2009 mendoza2010 --file LDs %}. Initially, LDs relied on the arclength of the trajectories as they evolve from their initial conditions forward and backward in time {% cite mendoza2010 mancho2013lagrangian --file LDs %}. Since its proposal as a nonlinear dynamics tool, it was used to plan a transoceanic autonomous underwater vehicle missions {% cite ramos2018 --file LDs %}, manage marine oil spills {% cite gg2016 --file LDs %}, analyze the Stratospheric Polar Vortex {% cite alvaro1 alvaro2 curbelo2019a curbelo2019b --file LDs %}, and recently in chemical reaction dynamics {% cite craven2015lagrangian craven2016deconstructing craven2017lagrangian revuelta2019unveiling --file LDs %}.

Our goal in this section is to give an introduction to the method of Lagrangian descriptors, which can reveal the geometry of phase space structures that determine transport in dynamical systems, such as in Eq. \eqref{eq:gtp_dynSys}. The geometry of phase space structures is encoded in the trajectories of the system, which can be extracted using LDs from their initial conditions. The simple idea behind LDs is to seed a given phase space region with initial conditions and integrate a bounded and positive quantity (an intrinsic geometrical and/or physical property of the dynamical system under study) along trajectories for a finite time. This approach reveals the invariant phase space structures that make up the dynamical skeleton governing reaction dynamics. This is similar to the visualization techniques developed in fluid mechanics experiments in the laboratory to uncover patterns of flow structures by studying the evolution of particles in a moving fluid {% cite chien1986 --file LDs %}. A powerful analogy can be found in iron filings that align with the magnetic field lines of a magnet.

One of the biggest challenges that one is faced with when exploring the high-dimensional phase space of a dynamical system, such as those that occur in Hamiltonian systems, is that of the qualitative description of the behavior of ensembles of initial conditions, and recovering from their trajectory evolution the underlying template of geometrical phase space structures that governs the dynamical transport mechanisms of the flow. The problem that naturally arises in this context is that the trajectories of ensembles of initial conditions that start nearby might get ''lost'' with respect to each other very quickly, making the use of classical nonlinear dynamics techniques (Small Alignment Index, Lyapunov Exponents, etc...) that rely on tracking the location of neighboring trajectories computationally expensive and difficult to interpret. Furthermore, when Poincaré maps are applied for the dynamical analysis of high-dimensional systems, one might encounter the issue of trajectories not coming back to the selected surface of section, which would yield no relevant information whatsoever.

The method of Lagrangian descriptors provides tremendous advantages in comparison to other methodologies to overcome these issues. For instance, it is a computationally inexpensive and straightforward to implement tool to explore nonlinear dynamics. But probably, the key and revolutionary idea behind the success of this technique is that it focuses on integrating a positive scalar function along trajectories of initial conditions of the system instead of tracking their phase space location. In this way, by emphasizing initial conditions, it directly targets the building blocks where the dynamical structure of phase space is encoded. The methodology offered by LDs has thus the capability of producing a complete and detailed geometrical *phase space tomography* in high dimensions by means of using low-dimensional phase space probes to extract the intersections of the phase space invariant manifolds with these slices {% cite demian2017 naik2019a naik2019b GG2019 --file LDs %}. Any phase space slice can be selected and sampled with a high-resolution grid of initial conditions, and no information regarding the dynamical skeleton af invariant manifolds at the given slice is lost as the trajectories evolve in time. Moreover, this analysis does not rely on trajectories coming back to the chosen slice, as is required for Ponicaré maps to work. In this respect, there is also another key point that needs to be highlighted which demonstrates the real potential of LDs with respect to other classical nonlinear dynamics techniques. Using LDs we can obtain *all* the invariant manifolds of the dynamical system *simultaneously*, the hyperbolic stable and unstable manifolds coming from *any* NHIM in phase space, and also the KAM tori. This provides an edge over the classical approach of computing stable and unstable manifolds that relies on locating first the NHIMs in phase space individually, and for every NHIM globalize the manifolds separately, for which a knowledge of the eigendirections is crucial. Consequently, the application of LDs offers the capability of recovering *all* the relevant phase space structures in one *shot* without having to study the local dynamics about equilibrium points of the dynamical system.

### Formulations for Lagrangian Descriptors

The Lagrangian descriptor is a scalar valued non-negative function $M$, that is integrated forward and backward for a fixed intergation time $\tau$. Originally $M$ was defined using the arclength:

\begin{equation}
M(\mathbf{x_0},t_0,\tau) = \int_{t_0-\tau}^{t_0+\tau} ||\mathbf{v}(\mathbf{x}(t;\mathbf{x_0},t)|| \; dt \;,
\label{eq:M_function}
\end{equation}

where $||\cdot||$ is the Euclidean distance. $M$ can be naturally broken down into a forward ($M^f$) and backward ($M^b$) integral:

\begin{equation}
M(\mathbf{x}_{0},t_0,\tau) = M^b(\mathbf{x}_{0},t_0,\tau) + M^f(\mathbf{x}_{0},t_0,\tau) \;,
\end{equation}

where we have that:

\begin{equation}
M^f(\mathbf{x}_{0},t_0,\tau) = \int^{t_0+\tau}_{t_0} ||\mathbf{v}(\mathbf{x}(t;\mathbf{x}_0),t)|| \; dt \;\;,\;\;
M^b(\mathbf{x}_{0},t_0,\tau) = \int^{t_0}_{t_0-\tau} ||\mathbf{v}(\mathbf{x}(t;\mathbf{x}_0),t)|| \; dt \;.
\end{equation}

The advantage of this splitting is that $M^f$ highlights the stable manifolds of the dynamical system, and $M^b$ recovers the unstable manfifolds, while $M$ shows all the invariant manifolds simultaneously.
The intuitive idea why this tool works is that the influence of phase space structures on trajectories will result in differences (abrupt change) in arclength of nearby trajectories in the neighborhood of a phase space structure.
The method captures this distinct dynamical behavior separated by invariant phase space structures that results in abrupt changes of values of $M$. This detection of invariant manifolds has been mathematically quantified in terms of ''singular structures'' {% cite mancho2013lagrangian lopesino2017 --file LDs %}, where $M$ is non-differentiable. Once the manifolds are known, one can compute the NHIM at their intersection by means of a root search algorithm. An alternative method to recover the manifolds and their associated NHIM is by minimizing the function $M$ using a search optimization algorithm. This second procedure and some interesting variations are described in {% cite feldmaier2019 --file LDs %}.

We remark that there is no general ''golden rule'' for selecting the value of $\tau$ for exploring phase space. The *appropriate* (usually chosen by trial and error) value of $\tau$ will unveil the relevant geometrical template of phase space structures. A very low value of $\tau$ will not reveal any structures and very high value may lead to obscurity due to differences in magnitude of LD values. This means that $\tau$ is intimately related to the time scales of the dynamical phenomena that occur in the system under consideration. One needs to bear in mind the compromise that exists between the complexity of the structures revealed by the method to explain a certain dynamical phenomenon.

An alternative definition of LDs relies on the $p$-norm of the vector field of the dynamical system, where $p \in (0,1]$ and was first introduced in {% cite lopesino2017 --file LDs %} as:

\begin{equation}
M_p(\mathbf{x}_{0},t_0,\tau) = \sum_{k=1}^{n} \int^{t_0+\tau}_{t_0-\tau}  |v_{k}(\mathbf{x}(t;\mathbf{x}_0),t)|^p \; dt  \;,
\label{eq:Mp_function}
\end{equation}

Although this alternative definition of LDs does not have such an intuitive physical interpretation as that of arclength, it has been shown allow for a rigorous analysis of the notion of ''singular structures'' and to establish the mathematical connection of this notion to invariant stable and unstable manifolds in phase space. Furthermore, forward integration will reveal stable manifolds and backward evolution unveils unstable manifolds, as in the arclegth version of LDs. Another important aspect of the $p$-norm of LDs is that, since in the definition all the vector field components contribute separately, one can naturally decompose the LD in a way that allows to isolate distinct dynamical effects such as hyperbolic and elliptic behavior. This was used in \cite {demian2017,naik2019a} to show that the method can be used to successfully detect NHIMs and their stable and unstable manifolds in Hamiltonian systems. Furthermore, it is important to remark that with this definition of LDs one can mathematically prove that these phase space structures are detected as singularities of the $M_p$ scalar field, that is, at points where the function is non-differentiable and therefore its gradient takes very large values {% cite lopesino2017 demian2017 naik2019a --file LDs %}. Moreover, in this context it has also been shown that:

\begin{equation}
\mathcal{W}^u(\mathbf{x}_{0},t_0) = \textrm{argmin } M_p^{(b)}(\mathbf{x}_{0},t_0,\tau) \quad,\quad \mathcal{W}^s(\mathbf{x}_{0},t_0) = \textrm{argmin } M_p^{(f)}(\mathbf{x}_{0},t_0,\tau) \;,
\label{eq:min_LD_manifolds}
\end{equation}

where $\mathcal{W}^u$ and $\mathcal{W}^s$ are, respectively, the unstable and stable manifolds calculated at time $t_0$ and $\textrm{argmin}(\cdot)$ denotes the phase space coordinates $\mathbf{x}_0$ that minimize the function $M_p$. In addition, NHIMs at time $t_0$ can be calculated as the intersection of the stable and unstable manifolds:

\begin{equation}
\mathcal{N}(\mathbf{x}_{0},t_0) = \mathcal{W}^u(\mathbf{x}_{0},t_0) \cap \mathcal{W}^s(\mathbf{x}_{0},t_0) = \textrm{argmin} M_p(\mathbf{x}_{0},t_0,\tau)
\label{eq:min_NHIM_LD}
\end{equation}

At this point, we would like to discuss the issues that arise from the definitions of LDs provided in Eqs. \eqref{eq:M_function} and \eqref{eq:Mp_function} when they are applied to analyze dynamics in open Hamiltonian systems. Notice that in both definitions all the initial conditions are integrated for the same time $\tau$. Recent studies have revealed  {% cite junginger2017chemical naik2019b GG2019 --file LDs %} that computing fixed-time LDs, that is, integrating all initial conditions chosen on a phase space surface for the same integration time $\tau$, could give rise to issues related to the fact that some of the trajectories that escape from the potential energy surface can go to infinity in finite time or at an increasing rate. The trajectories that show this behavior will give NaN values in the LD scalar field, hiding some regions of the phase space, and therefore obscuring the detection of invariant manifolds. In order to circumvent this problem we explain here the approach that has been recently adopted in the literature {% cite junginger2017chemical naik2019b GG2019 --file LDs %} known as variable integration time Lagrangian descriptors. In this methodology, LDs are calculated, at any initial condition, for a fixed initial integration time $\tau_0$ or until the trajectory of that initial condition leaves a certain phase space region $\mathcal{R}$ that we call the {\em interaction region}. Therefore, the total integration time in this strategy depends on the initial conditions themselves, that is $\tau(\mathbf{x}_0)$. In this variable-time formulation, given a fixed integration time $\tau_0 > 0$, the $p$-norm definition of LDs with $p \in (0,1]$ has the form:

\begin{equation}
M_p(\mathbf{x}_{0},t_0,\tau) = \sum_{k=1}^{n} \int^{t_0 + \tau^{+}_{\mathbf{x}_0}}_{t_0 - \tau^{-}_{\mathbf{x}_0}}  |v_{k}(\mathbf{x}(t;\mathbf{x}_0),t)|^p \; dt  \;,
\label{eq:Mp_vt}
\end{equation}

and the total integration time is defined as:

\begin{equation}
\tau^{\pm}_{\mathbf{x}_{0}} = \min \left\lbrace \tau_0 \, , \, |t^{\pm}|_{\big| \mathbf{x}\left(t^{\pm}; \, \mathbf{x}_{0}\right) \notin \mathcal{R}} \right\rbrace \; ,
\end{equation}

where $t^{+}$ and $t^{-}$ are the times for which the trajectory leaves the interaction region $\mathcal{R}$ in forward and backward time, respectively. It is important to highlighting that we select a large enough interaction region, the variable integration time LD definition given above in Eq. \eqref{eq:Mp_vt} will approach the fixed-time LD definition in Eq. \eqref{eq:Mp_function}. Thus, NHIMs and their stable and unstable manifolds will be captured by the phase space points for which the LD is non-differentiable and local minimum behavior given in Eqs. \eqref{eq:min_LD_manifolds} and \eqref{eq:min_NHIM_LD} is recovered. Moreover, KAM tori are also detected by contour values of the time-averaged LD. Therefore, the variable integration time LDs provides us with a suitable methodology to study the phase space structures that characterize escaping dynamics in open Hamiltonians, since it avoids the issue of trajectories escaping to infinity very fast.

## Revealing the Phase Space Structure

### Linear System

We describe this result for the two degrees-of-freedom system given by the linear quadratic Hamiltonian
associated to a index-1 saddle at the origin. This Hamiltonian and the equations of motion are
given by the expressions:

\begin{equation}
H(x,y,p_x,p_y) = \dfrac{\lambda}{2}\left(p_x^2 - x^2\right) + \dfrac{\omega}{2} \left(p_y^2 + y^2 \right) \quad,\quad \begin{cases}
\dot{x} = \lambda \, p_x \\
\dot{p}_{x} = \lambda \, x \\
\dot{y} = \omega \, p_y \\
\dot{p}_{y} = -\omega \, y
\end{cases}
\label{eq:index1_Ham}
\end{equation}

Given the initial condition $\mathbf{x}_0 = \mathbf{x}(t_0) = \left(x_0,y_0,p_x^0,p_y^0\right) \in \mathbb{R}^4$, the general solution is:

\begin{equation}
\begin{split}
x(t) = \frac{1}{2} \left[A e^{\lambda t} + B e^{-\lambda t}\right] \;\;&,\;\;
p_x(t) = \frac{1}{2} \left[A e^{\lambda t} - B e^{-\lambda t}\right] \\[.1cm]
y(t) = y_0 \cos(\omega t) + p^0_y \sin(\omega t) \;\;&,\;\;
p_y(t) = p^0_y \cos(\omega t) - y_0 \sin(\omega t)
\end{split}
\label{eq:gen_sol_linham}
\end{equation}

where $A = x_0 + p_x^0$ and $B = x_0 - p_x^0$. Since the system is autonomous, we can choose without loss of generality the initial time as $t_0 = 0$. Decomposing the $p$-norm LDs into the hyperbolic and elliptic components of the system we get:

\begin{equation}
M_{p}(\mathbf{x}_{0},\tau) = M^{h}_{p}(\mathbf{x}_{0},\tau) + M^{e}_{p}(\mathbf{x}_{0},\tau) \;,
\label{eq:hyp_comp_M}
\end{equation}

where the hyperbolic ($M^{h}$) and elliptic ($M^{e}$) parts are:

\begin{equation}
M^{h}_{p}(\mathbf{x}_{0},\tau) = \int^{\tau}_{-\tau} |\dot{x}(t;\mathbf{x}_0)|^p + |\dot{p}_{x}(t;\mathbf{x}_0)|^p \; dt \;\;,\;\; M^{e}_{p}(\mathbf{x}_{0},\tau) = \int^{\tau}_{-\tau} |\dot{y}(t;\mathbf{x}_0)|^p + |\dot{p}_{y}(t;\mathbf{x}_0)|^p \; dt
\label{eq:ell_comp_M}
\end{equation}

By means of an asympotic analysis when the integration time is sufficiently large, i.e. $\tau \gg 1$, it is straightforward to show {% cite lopesino2017 demian2017 naik2019a --file LDs %} that the asymptotic behavior of the hyperbolic component is:

\begin{equation}
M_{p}^{h}\left(\mathbf{x}_0,\tau\right) \sim \left(\dfrac{\lambda}{2}\right)^{p-1}  \dfrac{|A|^{p} + |B|^{p}}{p} \, e^{p \lambda \tau}
\label{eq:M_hyp_asymp}
\end{equation}

Therefore, this result shows that $M_{\gamma}^{h}$ grows exponentially with $\tau$ and also that the leading order singularities in $M_{p}^{h}$ occur when $|A| = 0$, that is, when $p^0_x = - x_0$, which corresponds to initial conditions on the stable manifold of the NHIM, or in the case where $|B| = 0$, that is, $p^0_x = x_0$, corresponding to initial conditions on the unstable manifold of the NHIM. Moreover, $M_{p}^{h}$ is non-differentiable at the NHIM, since it is given by the intersection of the stable and unstable manifolds.

\begin{equation}
\lim_{\tau \to \infty} \dfrac{1}{2\tau} M^{e}_{p}(\mathbf{x}_{0},\tau) = \dfrac{2}{\pi} \left(\omega R\right)^{p} B\left(\dfrac{p+1}{2},\dfrac{1}{2}\right)
\label{eq:lim_m_gamma}
\end{equation}

where $R$ is the radius of the circular periodic orbit described by the $y$ DoF in the center space $y-p_y$ . So the time average of the elliptic part of LDs converges to a value that depends on the energy of the $y$ DoF, and therefore in the limit its value is constant for all points of the periodic orbit. This result is of interest because it allows us to use LDs as a tool to recover phase space KAM tori, given that the dynamical system under study satisfies the conditions of the Ergodic Partition Theorem {% cite mezic1999 --file LDs %}. Therefore, if one analyzes the time average of LDs on a specific initial condition in phase space $\mathbf{x}_0$, and its value converges, then this initial copndition would lie in an invariant phase space set cosisting of points that share the same time average value. Therefore, the contours of the time average of LDs, when it converges, identify invariant phase space sets. It is also important to highlight here that the limit value to which $M^{e}$ converges is directly related to the limit value to which the classical arclength definition of LDs tends when just applied to the elliptic component of the system. Indeed:

\begin{equation}
\displaystyle{\lim_{\tau \to \infty} \dfrac{1}{2\tau} M^{e}_{p}} = \dfrac{2}{\pi} \left(\lim_{\tau \to \infty} \dfrac{1}{2\tau} M^{e}\right)^{p} B\left(\dfrac{p+1}{2},\dfrac{1}{2}\right)
\end{equation}

where $M^e$ represents the arclength LD in Eq. \eqref{eq:M_function} applied only to the $y$ DoF.

To conclude this theoretical discussion we will show how LDs attains a local minimum at the phase space points corresponding to the stable and unstable manifolds of the NHIM and a global minimum at the NHIM. Given an energy of the system, $H = H_0$, above that of the origin, we know from the Lyapunov Subcenter Manifold Theorem {% cite weinstein1973 moser1976 rabinowitz1982 --file LDs %} that a family of NHIM parametrized by the energy bifurcates from the index-1 saddle. The phase space points that lie on the stable (or unstable) manifold, that is $x_0 = -p_x^0$ ($x_0 = p_x^0$), contribute to the hyperbolic component of LDs described in Eq. \eqref{eq:M_hyp_asymp} with $|A| = 0$ ($|B| = 0$) so that the Lagrangian descriptor has a local minimum at the manifold. Moreover, if the initial condition is on the NHIM, that is $x_0 = p_x^0 = 0$, then $|A| = |B| = 0$ and consequently $M^{h}_{p}(\mathbf{x}_{0},\tau) = 0$. Furthermore, all the energy of the system for these points concentrates on the $y$ DoF which evolves periodically, implying that $M^{e}_{p}(\mathbf{x}_{0},\tau) > 0$. As a result, LDs attain a global minimum at those points.

In order to support the theretical argument presented above, we provide also a numerical computation of the $p$-norm LDs in the saddle space $x-p_x$ of the linear Hamiltonian given in Eq. \eqref{eq:index1_Ham} using $p = 1/2$ and for an integration time $\tau = 10$. In Fig. [fig:1](#fig:index1_lds) we illustrate how the method detects the stable (red) and unstable (blue) manifolds of the unstable periodic orbit at the origin, and these manifolds can be directly extracted from the ridges of the scalar field $||\nabla M_p||$. Moreover, we show in Fig. [fig:1](#fig:index1_lds) C) that the LD scalar field is non-differentiable at the manifolds and also attains a local minimum on them, and we do so by looking at the values taken by $M_p$ along the line $p_x = 1/2$.

<a id="fig:index1_lds"></a>

<figcaption style="text-align:center;font-size:14px"><b>fig:1 </b><em> Phase portrait in the saddle space of the linear Hamiltonian given in Eq. \eqref{eq:index1_Ham}. A) Applicaton of the $p$-norm definition of LDs in Eq. \eqref{eq:Mp_function} using $p = 1/2$ with $\tau = 10$; B) Invariant stable (blue) and unstable (red) manifolds of the unstable periodic orbit at the origin extracted from the gradient of the $M_p$ function; C) Value of LDs along the line $p_x = 0.5$ depicted in panel A) to illustrate how the method detects the stable and unstable manifolds at points where the scalar field is singular or non-differentiable and attains a local minimum.</em></figcaption><hr>

### Time-dependent double well potential

In the following example, we illustrate how the arclength definition of LDs captures the stable and unstable manifolds that determine the phase portrait of the time-dependent double well potential, commonly known in dynamical systems as the forced Duffing oscillator. The Duffing equation arises when studying the motion of a particle on a line, i.e. a one degree of freedom system, subjected to the influence of a symmetric double well potential and an external forcing. The second order ODE that describes this oscillator is given by:

\begin{equation}
\ddot{x} + x^3 - x = \varepsilon f(t) \quad \Leftrightarrow \quad
\begin{cases}
\dot{x} = y \\
\dot{y} = x - x^3 + \varepsilon f(t) \\
\end{cases}
\end{equation}

where $\varepsilon$ measures the strength of the forcing term $f(t)$, and we choose for this example a sinusoidal force $f(t) = \sin(t)$. In the autonomous case, i.e. $\varepsilon = 0$, the system has three equilibrium points: a saddle located at the origin and two diametrally opoosed centers at the points $(\pm 1,0)$. The stable and unstable manifolds that emerge from the saddle point form two homoclininc orbits in the form of a figure eight around the two center equilibria:

\begin{equation}
\mathcal{W}^{s} = \mathcal{W}^{u} = \left\{(x,y) \in \mathbb{R}^2 \; \Big| \; 2y^2 + x^4 - 2x^2 = 0 \right\}
\label{eq:duff_homocMani}
\end{equation}

We begin by computing LDs for the unforced Duffing system using $\tau = 2$. For this small integration time, the method highlights the saddle and center fixed points, since the arclength at those points is always zero. Moreover, in this case the phase portrait still looks blurry as shown in Fig. [fig:2](#fig:duffing1_lds) A), and this is a consequence of trajectories not having sufficient time to evolve in order to make distinct dynamical behaviors ditinguishable. If we now increase the integration time to $\tau = 10$, we can see in Fig. [fig:2](#fig:duffing1_lds) B) that the homoclinic connection formed by the stable and unstable manifolds of the saddle point at the origin becomes clearly visible. Moreover, observe that the manifolds are located at points where the scalar values taken by LDs change abruptly. This property is demonstrated in Fig. [fig:2](#fig:duffing1_lds) C), where we have depicted the value of function $M$ along the line $y = 0.5$. Notice that sharp changes in the scalar field of LDs at the manifolds are also related to local minima.

<a id="fig:duffing1_lds"></a>
<figcaption style="text-align:center;font-size:14px"><b>fig:2 </b><em> Phase portrait of the autonomous and undamped Duffing oscillator obtained by applying the arclength definition of LDs in Eq. \eqref{eq:M_function}. A) LDs with $\tau = 2$; B) LDs with $\tau = 10$; C) Value of LDs along the line $y = 0.5$ depicted in panel B) illustrating how the method detects the stable and unstable manifolds at points where the scalar field changes abruptly.</em></figcaption><hr>

We move on to compute LDs for the forced Duffing oscillator. In this situation, the vector field is time-dependent and thus the dynamical system is nonautonomous. The consequence is that the homoclinic connection breaks up and the stable and unstable manifolds intersect, forming an intricate tangle that gives rise to chaos. We illustrate this phenomenon by computing LDs with $\tau = 10$ to reconstruct the phase portrait at the initial time $t_0 = 0$. This result is shown in Fig. [fig:3](#fig:duffing2_lds) C), and we also depict the forward $(M^f)$ and backward $(M^b)$ contributions of LDs in Fig. [fig:3](#fig:duffing2_lds) A) and B) respectively, demonstrating that the method can be used to recover the stable and unstable manifolds separately. Furthermore, by taking the value of LDs along the line $y = 0.5$, the location of the invariant manifolds are highlighted at points corresponding to sharp changes (and local minima) in the scalar field values of LDs.

<a id="fig:duffing2_lds"></a>
<figcaption style="text-align:center;font-size:14px"><b>fig:3 </b><em> Phase portrait of the nonautonomous and undamped Duffing oscillator obtained at time $t = 0$ by applying the arclength definition of LDs in Eq. \eqref{eq:M_function} with an integration time $\tau = 10$. A) Forward LDs detect stable manifolds; B) Backward LDs highlight unstable manifolds of the system; C) Total LDs (forward $+$ backward) showing that all invariant manifolds are recovered simultaneously. D) Value taken by LDs along the line $y = 0.5$ in panel C) to illustrate how the method detects the stable and unstable manifolds at points where the scalar field changes abruptly.</em></figcaption><hr>

### Cubic potential

In order to illustrate the issues encountered by the fixed integration time LDs and how the variable integration approach resolves them, we apply the method to a basic one degree-of-freedom Hamiltonian known as the ''fish potential'', which is given by the formula:

\begin{equation}
H = \dfrac{1}{2} p_x^2 + \dfrac{1}{2} x^2 + \dfrac{1}{3} x^3 \quad \Leftrightarrow \quad
\begin{cases}
\dot{x} = p_x \\
\dot{p}_{x} = - x - x^2
\end{cases} \;.
\label{eq:fish_Ham}
\end{equation}

This dynamical system has a saddle point at the point $(-1,0)$ from which a homoclinic orbit emerges, which surrounds the elliptic point located at the origin. By applying the $p$-norm LD with $p = 1/2$ and integrating all initial conditions for the same time $\tau = 3$ we clearly observe in Fig. [fig:4](#fig:fish_lds) A) the problems that appear in the detection of phase space structures due to trajectories escaping to infinity in finite time. If we increase $\tau$ further, very large and NaN values of LDs completely obscure the phase portrait of the system. On the other hand, if now we use the variable integration time LDs with $\tau = 8$ and we select for the interaction region a circle of radius $r = 15$ centered at the origin, the homoclininc orbit and the equilibrium points are nicely captured. Moreover, we can extract the stable and unstable manifolds of the system from the sharp ridges in the gradient of the scalar field, due to the fact that the method is non-differentiable at the location of the manifolds.

<a id="fig:fish_lds"></a>

<figcaption style="text-align:center;font-size:14px"><b>fig:4 </b><em> Phase portrait of the ''fish potential'' Hamiltonian in Eq. \eqref{eq:fish_Ham} revealed by the $p$-norm LDs with $p = 1/2$. A) Fixed-time integration LDs in Eq. \eqref{eq:Mp_function} with $\tau = 3$; B) Variable-time integration definition of LDs in Eq. \eqref{eq:Mp_vt} with $\tau = 8$; C) Invariant stable (blue) and unstable (red) manifolds of the saddle fixed point  extracted from the gradient of the variable time $M_p$ function.</em></figcaption><hr>

### Hénon-Heiles Hamiltonian

We conclude this tutorial on how to apply the method of Lagrangian descriptors to unveil the dynamical skeleton of a four-dimensional phase space by applying this tool to a hallmark Hamiltonian system of nonlinear dynamics, the H\'enon-Heiles Hamiltonian introduced in 1964 to study the motion of stars in galaxies {% cite henon1964 --file LDs %}. This system is described by:

\begin{equation}
H = \dfrac{1}{2} \left(p_x^2 + p_y^2\right) + \dfrac{1}{2}\left(x^2 + y^2\right) + x^2y - \dfrac{1}{3} y^3 \quad \Leftrightarrow \quad
\begin{cases}
\dot{x} = p_x \\
\dot{p}_{x} = - x - 2xy \\
\dot{y} = p_y \\
\dot{p}_{y} = - y - x^2 + y^2
\end{cases} \;.
\label{eq:henon_system}
\end{equation}

which has four equilibrium points: one minimum located at the origin and three saddle-center points at $(0,1)$ and $(\pm \sqrt{3}/2,-1/2)$. The potential energy surface is $V(x,y) = x^2/2 + y^2/2 + x^2y - y^3/3$ which has a $\pi/3$ rotational symmetry and is characterized by a central scattering region about the origin and three escape channels, see Fig. [fig:5](#fig:henonHeiles_pes) below for details.

<a id="fig:henonHeiles_pes"></a>

<figcaption style="text-align:center;font-size:14px"><b>fig:5 </b><em> Potential energy surface for the H\'enon-Heiles system.</em></figcaption><hr>

In order to analyze the phase space of the H\'enon-Heiles Hamiltonian by means of the variable integration time LDs, we fix an energy $H = H_0$ of the system and choose an interaction region $\mathcal{R}$ defined in configuration space by a circle of radius $15$ centered at the origin. For our analysis we consider the following phase space slices:

\begin{eqnarray}
\mathcal{U}^{+}_{y,p_y} & = \left\{(x,y,p_x,p_y) \in \mathbb{R}^4 \;|\; H = H_0 \;,\; x = 0 \;,\; p_x > 0\right\} \\[.1cm]
\mathcal{V}^{+}_{x,p_x} &= \left\{(x,y,p_x,p_y) \in \mathbb{R}^4 \;|\; H = H_0 \;,\; y = 0 \;,\; p_y > 0\right\}
\label{eq:psos}
\end{eqnarray}

Once we have fixed the surfaces of section (SOS) where we want to compute LDs, we select a grid of initial conditions and, after discarding those that are energetically unfeasible, we integrate the remaining conditions both forward and backward in time, and compute LDs using the definition in Eq. \eqref{eq:Mp_vt} with $p = 1/2$ along the trajectory for the whole fixed integration time or until the initial condition leaves the interaction region $\mathcal{R}$, what happens first. The result obtained when the LDs values are plotted will reveal the stable and unstable manifolds and also de KAM tori in the surface of section under consideration. Since the stable and unstable manifolds are detected at points where the LD scalar function is non-differentiable, we can directly extract them from the gradient, that is, using $||\nabla \mathcal{M}_p||$. We begin by looking at the phase space structures on the SOS $\mathcal{U}^{+}_{y,p_y}$. To do so, we fix an energy for the system $H = 1/12$, which is below that of the saddle-center equilibrium points. For that energy level, the exit channels of the PES are closed, and therefore, all trajectories are trapped in the scattering region of the central minimum at the origin. We can clearly see in Fig. [fig:6](#fig:henonHeiles_lds) A)-B) that the computation of the $p$-norm variable integration time LDs with $p = 1/2$ using $\tau = 50$ reveals that the motion of the system is completely regular. The method nicely captures the UPO present in the central region of the PES and also its stable and unstable manifolds which form a homoclininc cnnection. In order to demonstrate how the intricate details of chaotic motion are captured by LDs, we increase the energy of the system to $H = 1/3$. This energy level is now above that of the index-1 saddles of the PES, and consequently, phase space bottlenecks open in the energy manifold allowing trajectories of the system to escape to infinity from the scattering region. When we apply LDs using $\tau = 10$ on the SOSs defined in Eq. \eqref{eq:psos}, we observe in Figs. [fig:6](#fig:henonHeiles_lds) C)-F) that we can detect with high-fidelity the intricate homoclinic tangle formed by the stable and unstable manifolds of the UPO associated to the upper index-1 saddle of the PES. Moreover, observe that despite the issue of trajectories escaping to infinity in finite time, LDs succeed in revealing the template of geometrical phase space structures that governs transport and escape dynamics from the PES of the H\'enon-Heiles Hamiltonian system.

<a id="fig:henonHeiles_lds"></a>

<figcaption style="text-align:center;font-size:14px"><b>fig:6 </b><em> Phase space structures of the H\'enon-Heiles Hamiltonian as revealed by the $p$-norm variable integration time LDs with $p = 1/2$. A) LDs computed for $\tau = 50$ in the SOS $\mathcal{U}^{+}_{y,p_y}$ with energy $H = 1/12$; C) LDs for $\tau = 10$ in the SOS $\mathcal{U}^{+}_{y,p_y}$ with energy $H = 1/3$; E) LDs for $\tau = 10$ in the SOS $\mathcal{V}^{+}_{x,p_x}$ with energy $H = 1/3$;. In the right panels we have extracted the invariant stable (blue) and unstable (red) manifolds from the gradient of LDs.</em></figcaption><hr>

To finish this chapter we would like to mention that the method of Lagrangian descriptors has also been adapted to explore the template of geometrical structures present in the phase space of stochastic dynamical systems {% cite balibrea2016lagrangian --file LDs %}. This achievenment clearly evidences the versatility that this mathematical technique brings to the nonlinear dynamics community. We are confident that the analysis of stochastic processes that play a crucial role in chemical reaction dynamics by means of LDs will surely help to shed light and provide new and interesting insights in the development of Chemistry. Definitely, Lagrangian descriptors has become the ultimate tool to reveal the strong bond that exists between Chemistry and Mathematics in phase space.

## Implications for Reaction Dynamics

## References

{% bibliography --file LDs --cited %}


```python
# Aquí van las librerías y funciones que va a usar

import matplotlib.pyplot as plt
import sympy as sym
%matplotlib inline
sym.init_printing()

#!pip install control
import control

## Puede que requiera incorporar más librerías o definir nuevas funciones.  Hágalo a continuación

```

# Tarea

Nombres: **Ponga aquí sus nombres completos**

1. Analice el efecto de la realimentación para un proceso modelado por

$$ G_p(s) = \frac{0.75}{s^2 + 4s + 2}$$

y una estrategia de contro definida por:

$$G_c(s) = k_c$$

- ¿El sistema $G_p$ es estable?
- ¿Qué efecto tiene realimentar el sistema con el controlador definido?
- ¿Puede identificar qué efecto tiene aumentar $k_c$ sobre el valor final de la señal controlada?
- ¿Puede identificar qué efecto tiene aumentar $k_c$ sobre el valor final de la señal de error?

```python
## Escriba su código
```

Escriba sus análisis
- si requiere, inserte más celdas
- use viñetas

```python
## si requiere más celdas, insértelas
```

2. Analice el efecto de la realimentación para un proceso modelado por

$$ G_p(s) = \frac{2}{s^3 + 2s^2 + 4s + 2}$$

y una estrategia de contro definida por:

$$G_c(s) = k_c$$

- ¿El sistema $G_p$ es estable?
- ¿Qué efecto tiene realimentar el sistema con el controlador definido?
- ¿Puede identificar qué efecto tiene aumentar $k_c$ sobre el valor final de la señal controlada?
- ¿Puede identificar qué efecto tiene aumentar $k_c$ sobre el valor final de la señal de error?

```python
## Escriba su código
```

Escriba sus análisis
- si requiere, inserte más celdas
- use viñetas

```python
## si requiere más celdas, insértelas
```

```python

```