            prepared by Özlem Salehi (QTurkey)

This cell contains some macros. If there is a problem with displaying mathematical formulas, please run this cell to load these macros.

$ \newcommand{\bra}[1]{\langle #1|} $
$ \newcommand{\ket}[1]{|#1\rangle} $
$ \newcommand{\braket}[2]{\langle #1|#2\rangle} $
$ \newcommand{\dot}[2]{ #1 \cdot #2} $
$ \newcommand{\biginner}[2]{\left\langle #1,#2\right\rangle} $
$ \newcommand{\mymatrix}[2]{\left( \begin{array}{#1} #2\end{array} \right)} $
$ \newcommand{\myvector}[1]{\mymatrix{c}{#1}} $
$ \newcommand{\myrvector}[1]{\mymatrix{r}{#1}} $
$ \newcommand{\mypar}[1]{\left( #1 \right)} $
$ \newcommand{\mybigpar}[1]{ \Big( #1 \Big)} $
$ \newcommand{\sqrttwo}{\frac{1}{\sqrt{2}}} $
$ \newcommand{\dsqrttwo}{\dfrac{1}{\sqrt{2}}} $
$ \newcommand{\onehalf}{\frac{1}{2}} $
$ \newcommand{\donehalf}{\dfrac{1}{2}} $
$ \newcommand{\hadamard}{ \mymatrix{rr}{ \sqrttwo & \sqrttwo \\ \sqrttwo & -\sqrttwo }} $
$ \newcommand{\vzero}{\myvector{1\\0}} $
$ \newcommand{\vone}{\myvector{0\\1}} $
$ \newcommand{\stateplus}{\myvector{ \sqrttwo \\  \sqrttwo } } $
$ \newcommand{\stateminus}{ \myrvector{ \sqrttwo \\ -\sqrttwo } } $
$ \newcommand{\myarray}[2]{ \begin{array}{#1}#2\end{array}} $
$ \newcommand{\X}{ \mymatrix{cc}{0 & 1 \\ 1 & 0}  } $
$ \newcommand{\Z}{ \mymatrix{rr}{1 & 0 \\ 0 & -1}  } $
$ \newcommand{\Htwo}{ \mymatrix{rrrr}{ \frac{1}{2} & \frac{1}{2} & \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} & \frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & \frac{1}{2} & -\frac{1}{2} & -\frac{1}{2} \\ \frac{1}{2} & -\frac{1}{2} & -\frac{1}{2} & \frac{1}{2} } } $
$ \newcommand{\CNOT}{ \mymatrix{cccc}{1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0} } $
$ \newcommand{\norm}[1]{ \left\lVert #1 \right\rVert } $
$ \newcommand{\pstate}[1]{ \lceil \mspace{-1mu} #1 \mspace{-1.5mu} \rfloor } $

 Quantum Fourier Transform

What makes Fourier Transform special is that it can be computed faster on a quantum computer than on a classical computer. In this notebook, we will learn about the quantum analogue of Discrete Fourier Transform.

Let's recall the definition for $DFT$. $DFT$ of the vector $x=\myvector{x_0~x_1\dots~x_{N-1}}^T$ is the complex vector $y=\myvector{y_0~y_1\dots y_{N-1}}^T$ where

$$
y_k=\frac{1}{\sqrt{N}} \sum_{j=0}^{N-1}e^{\frac{2\pi i j k }{N}}x_j.
$$

Now suppose that we have an $N=2^n$-dimensional quantum state vector $x=\myvector{x_0~x_1\dots~x_{N-1}}^T$ representing the state
$$
\ket{\psi} = \sum_{j=0}^{N-1} x_j \ket{j}.
$$

Quantum Fourier Transform $QFT$ of state $\ket{\psi}$ is given by
$$
\ket{\phi}=\sum_{k=0}^{N-1}y_k \ket{k} = \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1} \sum_{j=0}^{N-1}e^{\frac{2\pi i j k }{N}}x_j \ket{k},
$$
where $y_k$ is defined as above for $k=0,\dots,N-1$.

Hence for a basis state $\ket{j}$ represented by the vector $\myvector{x_0~x_1\dots~x_{N-1}}^T$ where $x_j=1$ and the rest of the entries are 0, its $QFT$ is given by

$$
 \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1}e^{\frac{2\pi i j k }{N}} \ket{k}.
$$

$e^{\frac{2\pi i}{N}}$ is often denoted by $\omega$. The above expression can be equivalently expressed as
$$
 \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1}{\omega^{ j k} } \ket{k}.
$$

Task 1 (on paper)

Apply $QFT$ to the basis state $\ket{10}$ and find the new quantum state.

click for our solution

Task 2 (on paper)

Apply $QFT$ to the state $ \ket \psi=\frac{1}{\sqrt{2}} \ket {01}+\frac{1}{\sqrt{2}} \ket {10} $.

click for our solution

Task 3 (on paper)

Apply $QFT$ to the quantum state $\ket{\psi}=\alpha \ket{0} + \beta \ket{1}$ and find the new quantum state. Conclude that applying 1 qubit $QFT$ is equivalent to applying Hadamard gate.

click for our solution

Remark: Note that two vectors that differ by a relative phase can not be distinguished when measured. For instance the vectors $\frac{1}{2} \myvector{1 \\ -i \\ -1 \\ i}$ and $\frac{1}{2} \myvector{1 \\ -1 \\ 1 \\ -1}$ are not distinguishable. But when $QFT$ is applied, then we obtain the vectors $ \myvector{0 \\ 1 \\ 0 \\ 0}$ and $ \myvector{ 0\\ 0 \\ 1 \\ 0}$ respectively and now the two vectors are distinguishable.

QFT Operator

Let's try to find out the matrix corresponding to $QFT$. Given $x$, the entries of the vector $y$ corresponding to $QFT$ of the vector $x$ is given by

$$
y_k=\frac{1}{\sqrt{N}} \sum_{j=0}^{N-1}e^{\frac{2\pi i j k }{N}}x_j = \frac{1}{\sqrt{N}} \sum_{j=0}^{N-1}\omega^{j k}x_j .
$$

Hence, we can equivalently write:

\begin{align*}
y_0 &= \frac{1}{\sqrt{N}} \left (x_0 + x_1 + x_2 + \cdots + x_{N-1}  \right ) \\
y_1 &= \frac{1}{\sqrt{N}} \left (x_0 + x_1\omega  + x_2 \omega^2+ \cdots + x_{N-1} \omega^{N-1}  \right )\\
y_2 &= \frac{1}{\sqrt{N}} \left (x_0 + x_1\omega^2  + x_2 \omega^4+ \cdots + x_{N-1} \omega^{2N-2}  \right )\\
\vdots \\
y_{N-1} &= \frac{1}{\sqrt{N}} \left (x_0 + x_1\omega^{N-1}  + x_2 \omega^{2N-2}+ \cdots + x_{N-1} \omega^{(N-1)(N-1)}  \right ).
\end{align*}

From above, the matrix representation can be given by the following matrix.

$$ \frac{1}{\sqrt{N}} \mymatrix{rrrrrr}{1 & 1 & 1 &1  & \cdots & 1\\
                   1 & \omega & \omega^2 & \omega^3 & \cdots& \omega^{N-1} \\
                   1 & \omega^2 &\omega^4 & \omega^6  & \cdots& \omega^{2N-2} \\
                  1 & \omega^3 & \omega^6 & \omega^9 & \cdots& \omega^{3N-3} \\
                  \vdots &  \vdots &  \vdots &  \vdots &  \vdots &  \vdots \\
                    1 & \omega^{N-1} & \omega^{2N-2} & \omega^{3N-3} & \cdots& \omega^{(N-1)(N-1)} \\
                    }  $$

 Observation

Let $\ket{j}$ be a basis state where $j=0,\dots N-1$. Then after applying $QFT$, the vector that is obtained is exactly the $j+1$'st column of the $QFT$ opreator.

Task 4 (on paper)

Apply $QFT$ to the basis state $\ket{10}$ using the matrix representation and find the new quantum state.

click for our solution

Properties of QFT (Optional)

 $QFT$ is unitary: You can check this from the matrix corresponding to $QFT$.

 Linear shift: Linear shift of a state vector causes relative phase shift of its $QFT$. The converse is also true.

  If  $QFT$ $\myvector{\alpha_0 \\ \alpha_1 \\ \vdots \\ \alpha_{N-1} } = \myvector{\beta_0 \\ \beta_1 \\ \vdots \\ \beta_{N-1}}$, then $QFT$ $\myvector{\alpha_{N-1} \\ \alpha_0 \\ \vdots \\ \alpha_{N-2} } = \myvector{\beta_0 \\ \omega \beta_1 \\ \vdots \\ \omega^{N-1}\beta_{N-1}}$ and
      $QFT$  $\myvector{\alpha_0 \\ \omega  \alpha_1 \\ \vdots \\ \omega^{N-1}\alpha_{N-1} } = \myvector{\beta_1 \\ \beta_2 \\ \vdots \\ \beta_{0}}$.

Task 5 (on paper)

What is the quantum state obtained after applying $QFT$ to the state $\ket{11}$? Find using Task 4 and the linear shift property.

click for our solution

 Circuit implementation for $QFT$

Recall that after applying $QFT$ to the state $\ket{\psi}=\alpha\ket{0} + \beta \ket{1}$, we get  $\frac{\alpha+\beta}{\sqrt{2}} \ket{0} + \frac{\alpha-\beta}{\sqrt{2}} \ket{1}$, which is the state obtained after applying Hadamard to the state $\ket{\psi}$. Hence, the effect of applying $QFT$ to a single qubit is equivalent to applying Hadamard.

Now let us look at the effect of applying $QFT$ to a $n$-qubit system. Let $N=2^n$.

Let $\ket{j}$ be a basis state where $j=0,\dots,N-1$. We will use the binary representation for $j$, that is $j=j_1j_2\cdots j_n$ for $j_i \in \{0,1\}$. More formally

\begin{equation} \label{eq: binary}
j=j_12^{n-1} + j_2 2^{n-2} + \cdots + j_n2^0 = 2^n \sum_{l=1}^n 2^{-l} j_l~~~~~~~~(1)
\end{equation}

We will stick to the expression above but equivalently you may encounter the following expression for $j$ in various places.

\begin{align*}
j&=2^n(j_1 2^{-1} + j_2 2^{-2} + \cdots j_n 2^{-n}) \\
&= 2^n (0.j_1j_2\cdots j_n)
\end{align*}

Now let's use this representation to express the state obtained after applying $QFT$ to $\ket{j}$.

\begin{align*}
\ket{j}=\ket{j_1j_2\cdots j_n} &\rightarrow  \frac{1}{\sqrt{N}} \sum_{k=0}^{N-1}e^{\frac{2\pi i j k }{N}} \ket{k}\\
&=  \frac{1}{2^{n/2}} \sum_{k_1=0}^{1} \cdots \sum_{k_n=0}^{1}e^{2\pi i j (\sum_{o=1}^n k_o2^{-o})} \ket{k_1k_2\cdots k_n} \mbox{   by replacing $k$ using Equation (1)} \\
&=\frac{1}{2^{n / 2}} \sum_{k_{1}=0}^{1}  \sum_{k_2=0}^{1}\cdots \sum_{k_{n}=0}^{1} \prod_{o=1}^{n} e^{2 \pi i j k_o 2^{-o}}|k_{1} \ldots k_{n}\rangle\\
&=  \frac{1}{2^{n/2}}\otimes_{o=1}^n  \sum_{k_o=0}^{1} e^{2\pi i jk_o 2^{-o} } \ket{k_o} \mbox{   by writing the quantum state as the tensor product of $n$ qubits} \\
&=  \frac{1}{2^{n/2}} \otimes_{o=1}^n \biggl (\ket{0} + e^{2\pi i j 2^{-o} } \ket{1} \biggr) \\
&= \frac{1}{2^{n/2}} \biggl ( \bigl(\ket{0} + e^{2\pi i j 2^{-1}} \ket{1} \bigr) \otimes  \bigl(\ket{0} + e^{2\pi i j 2^{-2}}\ket{1} \bigr) \otimes \cdots \otimes  \bigl(\ket{0} + e^{2\pi i j 2^{-n}}\ket{1} \bigr ) \biggr )~~~~~~ (2)
\end{align*}

Let's find the correct gates for applying $QFT$. We are going to use two gates: Hadamard and controlled rotation gate. Check the following unitary operator:

$$ CR_k = \mymatrix{cccc}{1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & e^{\frac{2\pi i}{2^{k}}} }  $$

$CR_k$ operator puts a relative phase of $e^{\frac{2\pi i}{2^k}}$ in front of the quantum state if both the controlled and the target qubits are in state $\ket{1}$. It corresponds to a rotation around $z$-axis.

\begin{align*}
CR_k \ket{00} &\rightarrow \ket{00} \\
CR_k \ket{01} &\rightarrow \ket{01} \\
CR_k \ket{10} &\rightarrow \ket{10} \\
CR_k \ket{11} &\rightarrow  e^{\frac{2\pi i}{{2^k}}}\ket{11} \\
\end{align*}

Let $\ket{j_1j_2\cdots j_n}$ be the input state. Applying Hadamard to the first qubit, we get

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i j_1 2^{-1}} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

since $e^{\pi i j_1}=1$ if $j_1=0$ and $e^{\pi i j_1} = -1$ if $j_1=1$.

Now let's apply the opreator $CR_2$ where second qubit is the control and the first qubit is the target. The resulting state is

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i  (j_1 2^{-1}+{j_2}{2^{-2}}  )} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

After applying $CR_i$ where qubit $i$ is the controller and the first qubit is the target consecutively for $i=3\dots n$, we obtain the state

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i  (j_1 2^{-1}+{j_2}{2^{-2}} +{j_3}{2^{-3}} \dots {j_n}{2^{-n}}   )} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

Noting that $j=2^n(j_1 2^{-1} + j_2 2^{-2} + \cdots j_n 2^{-n})$, we get

$$
\frac{1}{2^{1/2}} \bigl (\ket{0} + e^{2\pi i j 2^{-n}} \ket{1} \bigr ) \ket{j_2\cdots j_n}
$$

Task 6 (on paper)

Continue with the same procedure this time using the second qubit as the target and obtain the following state:

$$
\frac{1}{2^{1/{4}}} \bigl (\ket{0} + e^{2\pi i j 2^{-n}} \ket{1} \bigr ) \bigl (\ket{0} + e^{2\pi i j 2^{-n+1}  } \ket{1} \bigr ) \ket{j_3\cdots j_n}
$$

- First apply Hadamard to second qubit

- Then apply $CR_i$ where qubit $i+1$ is the control and the second qubit is the target for $i= 2 \dots n-1$.

click for our solution

The above procedure is repeated for each qubit and finally we get the state

$$
\frac{1}{2^{n/2}} \bigl (\ket{0} + e^{2\pi i j 2^{-n}} \ket{1} \bigr ) \otimes  (\ket{0} + e^{2\pi i j 2^{-n+1}} \ket{1} \bigr ) \otimes  \cdots \otimes  (\ket{0} + e^{2\pi i j 2^{-1}} \ket{1} \bigr  )
$$

Note that this is exactly Equation (2) except that the qubits are in the reverse order. This also proves that $QFT$ is unitary, as the gates we used are unitary gates. Overall, the $QFT$ circuit is given below:

Now we are ready to implement our circuit in Cirq. To implement $CR_k$, we will use the $CZPowGate$. Let's start by importing Cirq.

```python
import cirq
```

CZPowGate

In Cirq, $CZPowGate$ is given by the following unitary matrix:

$$ CZPowGate(t) = \mymatrix{cccc}{1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & e^{\pi i t} } $$

To use $CZPowGate$, you should use the following command to create a gate with parameter $t$:

cirq.CZPowGate(exponent=t)

In our case, by letting $t=2/2^k$ we can create the gate $CR_k$.

CRk = cirq.CZPowGate(exponent=2/2^k)

Then we can use $CR_k$ in our circuit by providing the control and target gates.

circuit.append(CRk(q0,q1))

Task 7

Implement the circuit for applying $QFT$ to a 2 qubit system. Try it using $\ket{01}$ as your input state. Simulate your circuit and compare it with the solution of Task 1.

```python
import cirq
from cirq import CZPowGate, X, H, SWAP

#Create two qubits
q1, q2 = cirq.LineQubit.range(2)

#Create and initialize circuit
circuit = cirq.Circuit(X(q1))

#Define CR_2 gate with t=2/2^2

#Apply Hadamard to first qubit

#Apply CR_2 where first qubit is target and second qubit is control

#Apply Hadamard to second qubit

#Swap the qubits

#Simulate the circuit and print the results
print('Simulate the circuit:')
s=cirq.Simulator()
results=s.simulate(circuit)
print(results)
```

click for our solution

In the following tasks, to impose the order of application of the gates you may need to use the parameter
circuit.append(...,strategy=InsertStrategy.NEW)

 when appending gates to your circuit.

Task 8

Implement the circuit which applies $QFT$ to any $n$-qubit system. Define a function named qft which takes the number of qubits, the qubits and the circuit as its parameter. Simulate the circuit for $n=3$ on all possible input states.

```python
from cirq.circuits import InsertStrategy
from cirq import H, SWAP
import cirq

def qft(n,qubits,circuit):

    #Your code here

```

```python
n=3

inputs = ['000','001','010','011','100','101','110','111']

for input in inputs:

    #Create n qubits
    #

    #Create circuit
    #

    #initialization
    #

    #Call qft function
    #

    #Printing circuit
    #

    #Simulating circuit
    #
```

click for our solution

Inverse Quantum Fourier Transform

Inverse Quantum Fourier Transform ($QFT^{\dagger}$) is the transformation which satisfies $QFT\cdot QFT^{\dagger}=I$. It is defined exactly the same with the exponents having negative sign.

$$
 QFT^{\dagger} \ket{k}=\frac{1}{\sqrt{N}} \sum_{l=0}^{N-1}e^{-\frac{2\pi i j k }{N}}\ket{l}.
$$

To implement $QFT^{\dagger}$, one should apply all the operations in reverse order to undo the circuit.

Task 9

Implement the circuit which applies $QFT^{\dagger}$ to any $n$-qubit system. Define a function named iqft which takes the number of qubits, the qubits and the circuit as its parameter. Simulate the circuit for different $n$ values. Apply both $QFT$ and $QFT^{\dagger}$ and check whether the corresponding unitary matrix is the identity matrix. Write your function to a file so that you can use it later.

```python
#%%writefile iqft.py
import cirq
from cirq.circuits import InsertStrategy
from cirq import H, SWAP, CZPowGate

def iqft(n,qubits,circuit):

    #Your code here

```

```python
n=4 #change n

#Create circuit
circuit = cirq.Circuit()

#Create n qubits
#

#Call qft function

#Call iqft function
#

#Printing circuit
#

#Simulating circuit
#

#Unitary matrix representation
print(cirq.unitary(circuit))
```

click for our solution

 Discussion on the number of gates

For the first qubit, we apply a single Hadamard gate followed by $n-1$ controlled rotation gates, which makes $n$ gates in total.

For the second qubit we apply a single Hadamard gate followed by $n-2$ controlled rotation gates, which makes $n-1$ gates in total.

Overall, $n + (n-1) + \cdots + 1 = \frac{(n)(n+1)}{2}$ gates are required.

Furthermore, $\frac{n}{2}$ $SWAP$ gates are required, each of which can be implemented using three $CNOT$ gates.

To sum up, $\theta(n^2)$ gates are required for applying $QFT$ to an $n$-qubit system represented by a vector of size $N=2^n$.

The best known classical algorithm for computing the Discrete Fourier Transform of $2^n$ entries, such as Fast Fourier Transform (FFT) requires $\Theta(n2^n)$, equivalently $\Theta(N \log N)$ gates, which means that the classical algorithm requires exponentially many more operations to compute $DFT$ (Note that the obvious algorithm for computing $DFT$ requires ${2}^{2n}$ gates). An improved algorithm presented [here](https://ieeexplore.ieee.org/document/892139) provides a circuit for computing $QFT$ with $O(n \log n)$ gates.

Nevertheless, this does not mean that we can use $QFT$ directly to accelerate the classical computation process. There are two reasons for this: The first reason is that the amplitudes can not be accessed directly after applying $QFT$. The second reason is that we may not know how to efficiently prepare the input state to $QFT$.

Next, we will see some applications of $QFT$.


# Dependencies

```python
import numpy as np
import scipy as sp
import seaborn as sn
import pandas as pd
import matplotlib
matplotlib.use('agg')
from matplotlib import pyplot as plt
import matplotlib.animation
import time
from IPython.display import HTML, Image, Video
from tqdm import tqdm
import os
import xarray as xr
import gc
import multiprocessing

from mystatsfunctions import OLSE,LMoments
from moarpalettes import get_palette

matplotlib.rcParams['axes.prop_cycle']=matplotlib.cycler('color',list(get_palette.Petroff10().to_sn_palette()))
matplotlib.rcParams['font.family']='Helvetica'
matplotlib.rcParams["animation.html"] = "jshtml"
matplotlib.rcParams['animation.embed_limit'] = 2**30

# %matplotlib inline
```

# Introduction
This notebook uses the Lorenz '63 system to demonstrate the conceptual differences between conventional "probabilistic" climate-model based attribution and our forecast-based approach.

The specific system used is the Palmer '99 variant of the Lorenz dynamical model, which includes a forcing (at an angle $\theta$ in the xy place) to the system, representing some external forcing - eg. anthropogenic greenhouse gas emissions. The equations of this system are as follows:

$
\begin{align}
\dot{x} & = \sigma(y-x) + f_0 \cos\theta \\
\dot{y} & = x (\rho - z) - y + f_0 \sin\theta \\
\dot{z} & = xy - \beta z
\end{align}
$

Lorenz, E. N. (1963). Deterministic Nonperiodic Flow. Journal of the Atmospheric Sciences. [DOI](https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2)

Palmer, T. N. (1999). A Nonlinear Dynamical Perspective on Climate Prediction. Journal of Climate, 12(2), 575–591. [DOI](https://doi.org/10.1175/1520-0442(1999)012<0575:ANDPOC>2.0.CO;2)

## Define the system

```python
def lorenz(xyz, t, rho=28, sigma=10, beta=8/3, F=0, theta=0):

    "Defines the lorez63 dynamical system. Standard default values."

    x, y, z = xyz
    x_dot = sigma * (y - x) + F * np.cos(theta)
    y_dot = x * rho - x * z - y + F * np.sin(theta)
    z_dot = x * y - beta * z
    return [x_dot, y_dot, z_dot]
```

## Integrate the system for long times
Analogous to running a climate model for many years.

We will use a timestep of 1/100 units & run the system for 100 000 units x 100 runs. Each run will start using the final point of the previous run.

```python
# set solver parameters
# start_time = pd.to_datetime('1750-01-01')
# end_time = pd.to_datetime('2250-01-01')
# timeindex = pd.date_range(start_time,end_time,freq='1h')[:-1]
# runlength = (end_time.to_pydatetime()-start_time.to_pydatetime()).days
# timestep = runlength / timeindex.size
## run over this many ICs to generate branches
branches = 101

# set Lorenz system parameters
s = 10
r = 28
b = 8/3
F = (8,np.deg2rad(-40))
```

```python
# ## original ICs
# IC0 = [0.01,0,0]
# IC1 = IC0[:]

# ## ICs if restarting mid-integration
# # start_from = 42
# # X = xr.open_dataset('Lorenz63-realisations/00'+str(start_from)+'.nc')
# # IC0 = X.isel(time=0).sel(type='unforced').to_array().values.flatten()
# # IC1 = X.isel(time=0).sel(type='forced').to_array().values.flatten()

# for branch in tqdm(np.arange(branches)[:]):

#     runlength = 100000
#     timestep = 0.01
#     start = branch * runlength
#     end = (branch+1) * runlength
#     timeindex = np.arange(start,end,timestep)

#     # unforced system
#     X0 = sp.integrate.odeint(lorenz, IC0, np.arange(0,runlength+timestep,timestep), (r,s,b,0,0)).reshape(-1,1,3)
#     IC0 = X0[-1,0,:]
#     X0 = xr.DataArray(data=X0[:-1],dims=['time','branch','dim'],coords=dict(time=timeindex,branch=[branch],dim=['x','y','z'])).to_dataset(dim="dim")

#     # system with external forcing
#     ## define forcing in terms of magnitude + direction
#     X1 = sp.integrate.odeint(lorenz, IC1, np.arange(0,runlength+timestep,timestep), (r,s,b,*F)).reshape(-1,1,3)
#     IC1 = X1[-1,0,:]
#     X1 = xr.DataArray(data=X1[:-1],dims=['time','branch','dim'],coords=dict(time=timeindex,branch=[branch],dim=['x','y','z'])).to_dataset(dim="dim")

#     X = xr.concat([X0.expand_dims({'type':['unforced']}),X1.expand_dims({'type':['forced']})],dim='type')

#     X.to_netcdf('./Lorenz63-realisations/'+f"{branch:04d}"+'.nc')
```

Compute the diagnostics required from the system.

We'll use $|x+y|$ as our "impact measure".
## define diagnostic
X['impact'] = xr.ufuncs.fabs(X.x+X.y)

## convert to pandas for easy resampling
X_df = X.to_dataframe().reset_index()

## determine daily max (preserving coord values) & mean values
X_day = X_df.groupby(['type',X_df.time.dt.date]).apply(lambda x: x.iloc[x.impact.argmax()]).drop('type',axis=1)
X_day['impact_mean'] = X_df.groupby(['type',X_df.time.dt.date]).mean().impact
X_day = X_day.rename(dict(time='maxtime',impact='impact_max'),axis=1).to_xarray()
X_day = X_day.assign_coords(time=pd.to_datetime(X_day.time.values))

## determine yearly max of max & max of mean
X_df = X_day.to_dataframe().reset_index()
X_yearmax = X_df.groupby(['type',X_df.time.dt.year]).apply(lambda x: x.iloc[x.impact_max.argmax()]).drop(['type','time'],axis=1)
X_yearmax['impact_mean'] = X_df.groupby(['type',X_df.time.dt.year]).max().impact_mean
X_yearmax = X_yearmax.to_xarray()Show what these DataArrays look like.XX_dayX_yearmax## x-y variable, reshape into "days"
X0_xy = (X0[:,0]-1.2*X0[:,1]).reshape(24,-1)
X1_xy = (X1[:,0]-1.2*X1[:,1]).reshape(24,-1)

## daymax of x-y
### X0
X0_xy_argmax = np.argmax(X0_xy,axis=0)
X0_xy_max = np.take_along_axis(X0_xy, X0_xy_argmax[None], axis=0).flatten()
### X1
X1_xy_argmax = np.argmax(X1_xy,axis=0)
X1_xy_max = np.take_along_axis(X1_xy, X1_xy_argmax[None], axis=0).flatten()

## split into "nodes"
### we'll define the nodes using the line x=0
### node1 = x > 0
### node2 = x < 0
#### X0
X0_node1 = (X0[:,0] > 0).reshape(24,-1)
X0_xy_max_node1_select = np.take_along_axis(X0_node1, X0_xy_argmax[None], axis=0).flatten()
X0_xy_max_node1 = X0_xy_max[X0_xy_max_node1_select]
X0_xy_max_node2 = X0_xy_max[~X0_xy_max_node1_select]
#### X1
X1_node1 = (X1[:,0] > 0).reshape(24,-1)
X1_xy_max_node1_select = np.take_along_axis(X1_node1, X1_xy_argmax[None], axis=0).flatten()
X1_xy_max_node1 = X1_xy_max[X1_xy_max_node1_select]
X1_xy_max_node2 = X1_xy_max[~X1_xy_max_node1_select]### Plot these simulations# define a color palette
plot_colors = dict(forced=get_palette.Petroff6().to_sn_palette()[0],unforced=get_palette.Petroff6().to_sn_palette()[1])fig = plt.figure()

# create gridspec
gs = matplotlib.gridspec.GridSpec(3,2,figure=fig)

# generate axes
ax_dist = [fig.add_subplot(gs[i,1]) for i in np.arange(3)]
ax_xy = fig.add_subplot(gs[:,0])
ax_x = ax_xy.twinx()
ax_y = ax_xy.twiny()

# plot daymax(x-y) distribution
dist_bins = np.linspace(40,46,25)
[ax_dist[0].hist(X_yearmax.sel(type=x).impact_max,histtype='step',bins=dist_bins,color=plot_colors[x],label=x) for x in ['unforced','forced']]
## by nodes (right first, left second)
[ax_dist[1].hist(X_yearmax.where(X_yearmax.x>0).sel(type=x).dropna('time').impact_max,histtype='step',bins=dist_bins,color=plot_colors[x]) for x in ['unforced','forced']]
[ax_dist[2].hist(X_yearmax.where(X_yearmax.x<0).sel(type=x).dropna('time').impact_max,histtype='step',bins=dist_bins,color=plot_colors[x]) for x in ['unforced','forced']]

# plot main "butterfly"
[ax_xy.plot(X.sel(type=x).isel(time=slice(1000,6000)).x,X.sel(type=x).isel(time=slice(1000,6000)).y,lw=0.1,color=plot_colors[x]) for x in ['unforced','forced']]

# plot distributions on each axis
## x histogram
[ax_x.hist(X.sel(type=x).x,density=True,histtype='step',bins=100,color=plot_colors[x],label=x) for x in ['unforced','forced']]
## y histogram
[ax_y.hist(X.sel(type=x).y,density=True,histtype='step',bins=100,color=plot_colors[x],orientation='horizontal') for x in ['unforced','forced']]

# axes layouts
## dist
[sn.despine(ax=a) for a in ax_dist]
ax_dist[0].legend(loc='upper right',frameon=False)
ax_dist[2].set_xlabel('annual maximum of $|x+y|$')
[a.set_yticks([]) for a in ax_dist]
[a.set_ylim(0,70) for a in ax_dist]
[a.set_xlim(dist_bins[0],dist_bins[-1]) for a in ax_dist]
ax_dist[0].set_title('all',loc='left')
ax_dist[1].set_title('right node',loc='left')
ax_dist[2].set_title('left node',loc='left')
## xy
ax_xy.set_xlim(-30,30)
ax_xy.set_ylim(-30,30)
ax_xy.set_xlabel('x')
ax_xy.set_ylabel('y')
ax_xy.axvline(0,ls=':',color='xkcd:grey',lw=0.5)
ax_xy.text(0.03,0.97,'left node',transform=ax_xy.transAxes,va='top',ha='left')
ax_xy.text(0.97,0.97,'right node',transform=ax_xy.transAxes,va='top',ha='right')
### add arrow showing direction of forcing
ax_xy.arrow(x=20,y=-20,dx=10*np.cos(F[1]),dy=10*np.sin(F[1]),length_includes_head=True,width=0.1,head_length=2,head_width=1,overhang=1,fc='k',ec='k')
ax_xy.arrow(x=19,y=-20,dx=1+10*np.cos(F[1]),dy=0,length_includes_head=True,width=0.1,head_length=0,head_width=0,fc='k',ec='k')
ax_xy.text(20,-15,'$f_0$',ha='right',fontsize=15)
## x
ax_x.set_ylim(0,0.5)
ax_x.axis('off')
## y
ax_y.set_xlim(0,0.5)
ax_y.axis('off')

# fig layout
fig.suptitle(r'Lorenz system: $\rho=28$, $\sigma=10$, $\beta=8/3$, $f_0=3$, $\theta=75 \degree$',fontweight='bold')
fig.patch.set_facecolor('xkcd:white')
fig.set_size_inches(10,5)
fig.dpi=100

plt.tight_layout()## Probabilistic attribution framing
For illustration, let's use a "severe" impacts threshold of |x+y| > 42.# extract impact events
impact_event_count = (X_day.impact_max>42).sum('time')

# ratio of event counts is RR
FAR = 1 - (impact_event_count.sel(type='unforced')/impact_event_count.sel(type='forced')).values[()]And thus the Fraction of Attributable Risk for all severe impact events is:print(np.round(FAR,3))# Extreme event attribution of a specific event
Here, we are going to look at particular events that occurred in the left hand node of the forced system.## Forecast-based attribution
We're gonna use xarray for this to keep track of the dimensions.Define function to produce forecasts based on ICs from the long system integrationdef generate_forecasts(event,
                       leads = [1/8,1/4,1/2,3/4,1,2,3,5,10,20],
                       ens_size = 51,
                       ICP_noise_scale = 0.005,
                       fc_timestep_size='15min'):

    # choose leads (current freq "days")
    inidates = event.maxtime.values - pd.to_timedelta(leads,unit='d').round('1h')

    # extract initial conditions before the event
    ICs = X.sel(type='forced',time=inidates).rename(time='inidate').drop('impact').to_array()

    # generate perturbed initial conditions
    ICPs = xr.DataArray(data=np.random.normal(0,ICP_noise_scale,ens_size*3).reshape(ens_size,3),dims=['number','variable'],coords=dict(number=np.arange(ens_size),variable=['x','y','z']))
    ## make the first intial condition the "control"
    ICPs[0] = 0

    # add perturbations to control ICs to get array of ICs
    ICs = (ICs + ICPs).transpose('inidate','number','variable')

    ## integrate each ensemble member until a couple of days after the event
    ## convert timestep size to units of days
    fc_timestep = pd.to_timedelta(fc_timestep_size).total_seconds()/(24*3600)
    fc_end = event.maxtime.values + pd.Timedelta('2d')
    max_fc_length = (fc_end - inidates).days.max() / fc_timestep

    ### create results array
    fcs0 = np.empty((ens_size,int(max_fc_length),3,inidates.size))+np.nan
    fcs1 = np.empty((ens_size,int(max_fc_length),3,inidates.size))+np.nan

    for i,inidate in enumerate(ICs.inidate.values):

        fc_length = (fc_end - inidate)
        fc_timesteps = int(fc_length.total_seconds()/(fc_timestep*(24*60*60)))
        ics = ICs.sel(inidate=inidate).values

        fc0 = np.array([sp.integrate.odeint(lorenz, ic, np.arange(0,fc_timesteps*fc_timestep-1e-11,fc_timestep), (r,s,b,0,0)) for ic in ics])
        fc1 = np.array([sp.integrate.odeint(lorenz, ic, np.arange(0,fc_timesteps*fc_timestep-1e-11,fc_timestep), (r,s,b,*F)) for ic in ics])

        fcs0[:,-fc_timesteps:,:,i] = fc0
        fcs1[:,-fc_timesteps:,:,i] = fc1

    # wrangle into nice forecast DataArray
    fc_timeindex = pd.date_range(inidates.min(),fc_end,freq=fc_timestep_size)[:-1]
    fcs0 = xr.DataArray(data=fcs0,dims=['number','time','variable','inidate'],coords=dict(number=np.arange(ens_size),time=fc_timeindex,variable=['x','y','z'],inidate=inidates)).to_dataset(dim="variable")
    fcs1 = xr.DataArray(data=fcs1,dims=['number','time','variable','inidate'],coords=dict(number=np.arange(ens_size),time=fc_timeindex,variable=['x','y','z'],inidate=inidates)).to_dataset(dim="variable")
    ## join datasets together
    fcs = xr.concat([fcs0.expand_dims({'type':['unforced']}),fcs1.expand_dims({'type':['forced']})],dim='type')

    # create impact variable
    fcs['impact'] = xr.ufuncs.fabs(fcs.x+fcs.y)

    return fcs### Event 1
The 1945 annual maximum event (largest right node event bewteen 1900 and 2100).X_event = X_yearmax.sel(time=1945,type='forced')
fcs = generate_forecasts(X_event, leads=[1/24,1/12,1/8,1/6,1/4,1/2,3/4,1,2,5,10,20], ens_size=51, ICP_noise_scale=1, fc_timestep_size='12min')g=sn.displot(data=fcs.impact.max('time').to_dataframe().reset_index(),x='impact',hue='type',col='inidate',col_wrap=6,kind='ecdf',palette=plot_colors)

[a.axvline(X_event.impact_max,ls=':',color='grey',lw=1) for a in g.axes]

''sn.relplot(data=fcs.sel(number=0).to_dataframe().reset_index(),x='x',y='y',hue='type',col='inidate',col_wrap=6,kind='line',palette=plot_colors,sort=False)### Event 2
The 2065 annual maximum event (largest left node event bewteen 1900 and 2100).X_event = X_yearmax.sel(time=2065,type='forced')
fcs = generate_forecasts(X_event, leads=[1/24,1/12,1/8,1/6,1/4,1/2,3/4,1,2,5,10,20], ens_size=51, ICP_noise_scale=1, fc_timestep_size='12min')g=sn.displot(data=fcs.impact.max('time').to_dataframe().reset_index(),x='impact',hue='type',col='inidate',col_wrap=6,kind='ecdf',palette=plot_colors)

[a.axvline(X_event.impact_max,ls=':',color='grey',lw=1) for a in g.axes]

''sn.relplot(data=fcs.sel(number=9).to_dataframe().reset_index(),x='x',y='y',hue='type',col='inidate',col_wrap=6,kind='line',palette=plot_colors,sort=False)sn.relplot(data=fcs.to_dataframe().reset_index(),
           x='time',
           y='impact',
           hue='type',
           col='inidate',
           size='number',
           sizes=(0.5,0.5),
           col_wrap=6,
           kind='line',
           palette=plot_colors,
           sort=False,
           facet_kws=dict(sharey=False,sharex=False))Things you can demonstrate here:
- Situations where the forcing affects the predictability
- Situations where the forcing impacts the event## Alternative event definition
Rather than using $|x+y|$, here we'll try to define a line in phase space above which we'll define impact events.

We'll use $0.83 \cdot x - y$ and look for events < -10print('relative occupancy of left node vs right')
((X.x<0).sum('time') / (X.x>0).sum('time')).to_pandas()X['impact_II'] = np.fabs(X.x)

impact_lim = 12

print('left node')
print('count')
print((X.impact_II>impact_lim).where(X.x<0).sum('time').to_pandas())
print('magnitude')
print((X.impact_II).where((X.impact_II>impact_lim)&(X.x<0)).mean('time').to_pandas())

print('right node')
print('count')
print((X.impact_II>impact_lim).where(X.x>0).sum('time').to_pandas())
print('magnitude')
print((X.impact_II).where((X.impact_II>impact_lim)&(X.x>0)).mean('time').to_pandas())## define diagnostic
X['impact_II'] = 0.9*X.x-X.y

## convert to pandas for easy resampling
XII_df = X.to_dataframe().reset_index()

## determine daily max (preserving coord values) & mean values
XII_day = XII_df.groupby(['type',XII_df.time.dt.date]).apply(lambda x: x.iloc[x.impact_II.argmax()]).drop('type',axis=1)
XII_day['impact_II_mean'] = XII_df.groupby(['type',XII_df.time.dt.date]).mean().impact_II
XII_day = XII_day.rename(dict(time='maxtime',impact_II='impact_II_max'),axis=1).to_xarray()
XII_day = XII_day.assign_coords(time=pd.to_datetime(XII_day.time.values))

## determine yearly max of max & max of mean
XII_df = XII_day.to_dataframe().reset_index()
XII_yearmax = XII_df.groupby(['type',XII_df.time.dt.year]).apply(lambda x: x.iloc[x.impact_II_max.argmax()]).drop(['type','time'],axis=1)
XII_yearmax['impact_II_mean'] = XII_df.groupby(['type',XII_df.time.dt.year]).min().impact_II_mean
XII_yearmax = XII_yearmax.to_xarray()fig = plt.figure()

# create gridspec
gs = matplotlib.gridspec.GridSpec(3,2,figure=fig)

# generate axes
ax_dist = [fig.add_subplot(gs[i,1]) for i in np.arange(3)]
ax_xy = fig.add_subplot(gs[:,0])
ax_x = ax_xy.twinx()
ax_y = ax_xy.twiny()

# plot daymax(x-y) distribution
dist_bins = np.linspace(11,15,25)
[ax_dist[0].hist(XII_yearmax.sel(type=x).impact_II_max,histtype='step',bins=dist_bins,color=plot_colors[x],label=x) for x in ['unforced','forced']]
## by nodes (right first, left second)
[ax_dist[1].hist(XII_yearmax.where(XII_yearmax.x>0).sel(type=x).dropna('time').impact_II_max,histtype='step',bins=dist_bins,color=plot_colors[x]) for x in ['unforced','forced']]
[ax_dist[2].hist(XII_yearmax.where(XII_yearmax.x<0).sel(type=x).dropna('time').impact_II_max,histtype='step',bins=dist_bins,color=plot_colors[x]) for x in ['unforced','forced']]

# plot main "butterfly"
[ax_xy.plot(X.sel(type=x).isel(time=slice(1000,6000)).x,X.sel(type=x).isel(time=slice(1000,6000)).y,lw=0.1,color=plot_colors[x]) for x in ['unforced','forced']]

# plot distributions on each axis
## x histogram
[ax_x.hist(X.sel(type=x).x,density=True,histtype='step',bins=100,color=plot_colors[x],label=x) for x in ['unforced','forced']]
## y histogram
[ax_y.hist(X.sel(type=x).y,density=True,histtype='step',bins=100,color=plot_colors[x],orientation='horizontal') for x in ['unforced','forced']]

# axes layouts
## dist
[sn.despine(ax=a) for a in ax_dist]
ax_dist[0].legend(loc='upper right',frameon=False)
ax_dist[2].set_xlabel('annual maximum of $0.85 \cdot x-y$')
[a.set_yticks([]) for a in ax_dist]
[a.set_ylim(0,100) for a in ax_dist]
[a.set_xlim(dist_bins[0],dist_bins[-1]) for a in ax_dist]
ax_dist[0].set_title('all',loc='left')
ax_dist[1].set_title('right node',loc='left')
ax_dist[2].set_title('left node',loc='left')
## xy
ax_xy.set_xlim(-30,30)
ax_xy.set_ylim(-30,30)
ax_xy.set_xlabel('x')
ax_xy.set_ylabel('y')
ax_xy.axvline(0,ls=':',color='xkcd:grey',lw=0.5)
ax_xy.text(0.03,0.95,'left node',transform=ax_xy.transAxes,va='top',ha='left')
ax_xy.text(0.97,0.95,'right node',transform=ax_xy.transAxes,va='top',ha='right')
### add arrow showing direction of forcing
ax_xy.arrow(x=20,y=-20,dx=10*np.cos(F[1]),dy=10*np.sin(F[1]),length_includes_head=True,width=0.1,head_length=2,head_width=1,overhang=1,fc='k',ec='k')
ax_xy.arrow(x=21,y=-20,dx=-1+10*np.cos(F[1]),dy=0,length_includes_head=True,width=0.1,head_length=0,head_width=0,fc='k',ec='k')
ax_xy.text(20,-15,'$f_0$',ha='right',fontsize=15)
## x
ax_x.set_ylim(0,0.5)
ax_x.axis('off')
## y
ax_y.set_xlim(0,0.5)
ax_y.axis('off')
## add in threshold line & shading
xrange = np.arange(-30,30+1e-6,0.01)
yrange = 0.87*xrange-10
ax_xy.plot(xrange,yrange,ls='--',color='xkcd:red',lw=0.5)
ax_xy.fill_between(xrange,yrange,-30,alpha=0.04,color='xkcd:red',lw=0)

# fig layout
fig.suptitle(r'Lorenz system: $\rho=28$, $\sigma=10$, $\beta=8/3$, $f_0=3$, $\theta=75 \degree$',fontweight='bold')
fig.patch.set_facecolor('xkcd:white')
fig.set_size_inches(10,5)
fig.dpi=100

plt.tight_layout()## event in the left node
event_l = XII_day.sel(time='1989-05-30',type='forced')
fcs_l = generate_forecasts(event_l, leads=[1/24,1/12,1/8,1/6,1/4,1/2,3/4,1,2,5,10,20], ens_size=501, ICP_noise_scale=1, fc_timestep_size='12min')
fcs_l['impact_II'] = 0.87*fcs_l.x-fcs_l.y## event in the right node
event_r = XII_yearmax.sel(time=2235,type='forced')
fcs_r = generate_forecasts(event_r, leads=[1/24,1/12,1/8,1/6,1/4,1/2,3/4,1,2,5,10,20], ens_size=501, ICP_noise_scale=1, fc_timestep_size='12min')
fcs_r['impact_II'] = 0.87*fcs_r.x-fcs_r.yg=sn.displot(data=fcs_l.impact_II.sel(time='1989-05-30').max('time').to_dataframe().reset_index(),x='impact_II',hue='type',col='inidate',col_wrap=6,kind='ecdf',palette=plot_colors)

[a.axvline(event_l.impact_II_max,ls=':',color='grey',lw=1) for a in g.axes]

''sn.relplot(data=fcs_l.sel(number=12).to_dataframe().reset_index(),x='x',y='y',hue='type',col='inidate',col_wrap=6,kind='line',palette=plot_colors,sort=False)sn.relplot(data=fcs_l.to_dataframe().reset_index(),
           x='time',
           y='impact_II',
           hue='type',
           col='inidate',
           size='number',
           sizes=(0.5,0.5),
           col_wrap=6,
           kind='line',
           palette=plot_colors,
           sort=False,
           facet_kws=dict(sharey=False,sharex=False))## Animation
We'll create an animation with the following elements:
- Trajectory plot
    - 10 ensemble members up to event (solid)
    - 10 ensemble members post event (solid, transparent)
    - Dots that pulse when event occurs
- Histograms of event
    - Histograms that build up as events occur### left node event## plot options
inidate = '1989-05-29 060000'
event_start_time = '1989-05-30 000000'
event_end_time = '1989-05-30 120000'
choose_mems = np.random.choice(500,10)fig,axes = plt.subplots(1,2,figsize=(10,5))
ax,ax1=axes
sn.despine()
ax.set_xlabel('x')
ax.set_ylabel('y')
ax1.set_xlabel('x')
ax1.set_xlim(11,16)
ax1.set_ylim(0,75)

fr_no = 150

hist0_points = []
hist1_points = []

for lnum,mem in enumerate(choose_mems):

    X0 = fcs_l.sel(number=mem,inidate=inidate,type='unforced',time=slice(inidate,event_end_time))
    X1 = fcs_l.sel(number=mem,inidate=inidate,type='forced',time=slice(inidate,event_end_time))

    maxtime0 = X0.sel(time=slice(event_start_time,event_end_time)).impact_II.idxmax('time')
    maxtime1 = X1.sel(time=slice(event_start_time,event_end_time)).impact_II.idxmax('time')

    argtime0 = ((maxtime0 - X0.time.isel(time=0)).dt.seconds//(12*60)).values[0]
    argtime1 = ((maxtime1 - X1.time.isel(time=0)).dt.seconds//(12*60)).values[0]

    X00 = X0.where(X0.time<=maxtime0).isel(time=slice(0,fr_no)).squeeze()
    ax.plot(X00.x,X00.y,lw=1,color=plot_colors['unforced'],zorder=5)

    X10 = X1.where(X1.time<=maxtime1).isel(time=slice(0,fr_no)).squeeze()
    ax.plot(X10.x,X10.y,lw=1,color=plot_colors['forced'],zorder=5)

    if fr_no >= argtime0:
        X01 = X0.where(X0.time>=maxtime0).isel(time=slice(0,fr_no)).squeeze()
        ax.plot(X01.x,X01.y,lw=1,color=plot_colors['unforced'],alpha=0.1,zorder=3)

        X0.sel(time=maxtime0).squeeze().plot.scatter('x','y',color=plot_colors['unforced'],ax=ax,zorder=10,alpha=np.exp(0.1*(argtime0-fr_no)))

    if fr_no >= argtime1:
        X11 = X1.where(X1.time>=maxtime1).isel(time=slice(0,fr_no)).squeeze()
        ax.plot(X11.x,X11.y,lw=1,color=plot_colors['forced'],alpha=0.1,zorder=3)

        X1.sel(time=maxtime1).squeeze().plot.scatter('x','y',color=plot_colors['forced'],ax=ax,zorder=10,alpha=np.exp(0.1*(argtime1-fr_no)))

## get histogram points
event_max0 = fcs_l.where(fcs_l.x<0).sel(inidate=inidate,type='unforced',time=slice(event_start_time,event_end_time)).impact_II.max('time').squeeze().values
actual_max0 = fcs_l.where(fcs_l.x<0).sel(inidate=inidate,type='unforced',time=slice(inidate,end_time)).isel(time=slice(0,fr_no)).impact_II.max('time').squeeze().values
hist0_points = event_max0[event_max0<=actual_max0]

event_max1 = fcs_l.where(fcs_l.x<0).sel(inidate=inidate,type='forced',time=slice(event_start_time,event_end_time)).impact_II.max('time').squeeze().values
actual_max1 = fcs_l.where(fcs_l.x<0).sel(inidate=inidate,type='forced',time=slice(inidate,end_time)).isel(time=slice(0,fr_no)).impact_II.max('time').squeeze().values
hist1_points = event_max1[event_max1<=actual_max1]

## plot max points
ax.plot(xmax0_points,ymax0_points)

## plot ICs
fcs_l.sel(number=choose_mems,type='forced',inidate=inidate,time=inidate).plot.scatter('x','y',color='k',ax=ax,marker='.')
fcs_l.sel(type='forced',inidate=inidate,time=inidate).plot.scatter('x','y',color='grey',ax=ax,marker='.',s=1,zorder=-1)

## plot histograms
bins = np.linspace(11,16,41)
ax1.hist(hist0_points, bins=bins, lw=1.5, color=plot_colors['unforced'], histtype='step')
ax1.hist(hist1_points, bins=bins, lw=1.5, color=plot_colors['forced'], histtype='step')

# figure layout

```python

```

```python

```

Trying to demo that there's a difference between storyline & probabilistic -> can get different answers

Question is the answers from my approach give more reliable guide to full ensemble vs. eg. data assimilation

Limit in which has to be so -> do we converge to that result or not?

One argument we could use: if we back off far enough sampling bigger range of possible forcing impacts as we sample range of states

Moving from state with little forcing impact into region with large forcing impact

Lorenz -> very sensitive to forcing about x,y=0

```python

```

## Reattempt using |x| as the metric of choice
Note: to make loading data in considerable quicker, don't decode_times in the xarray open call.

```python
def preprocess_to_extremes(ds):

    ds = ds.load()

    rh_events = ds.groupby('time.year').apply(lambda x: x.sel(time=x.x.idxmax('time')))
    lh_events = ds.groupby('time.year').apply(lambda x: x.sel(time=x.x.idxmin('time')))

    return xr.concat([rh_events.expand_dims({'event':['max']}),lh_events.expand_dims({'event':['min']})],'event')
```

```python
X = xr.open_mfdataset('./Lorenz63-realisations/00*.nc',preprocess=preprocess_to_extremes)
```

```python
test=xr.open_dataset('./Lorenz63-realisations/0000.nc',decode_times=False)
```

```python
test.sel(time=test.isel(time=slice(10000,None)).x.idxmax('time'))
```

<div><svg style="position: absolute; width: 0; height: 0; overflow: hidden">
<defs>
<symbol id="icon-database" viewBox="0 0 32 32">
<path d="M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z"></path>
<path d="M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
<path d="M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
</symbol>
<symbol id="icon-file-text2" viewBox="0 0 32 32">
<path d="M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z"></path>
<path d="M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
</symbol>
</defs>
</svg>
<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.
 *
 */

:root {
  --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));
  --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));
  --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));
  --xr-border-color: var(--jp-border-color2, #e0e0e0);
  --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);
  --xr-background-color: var(--jp-layout-color0, white);
  --xr-background-color-row-even: var(--jp-layout-color1, white);
  --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);
}

html[theme=dark],
body.vscode-dark {
  --xr-font-color0: rgba(255, 255, 255, 1);
  --xr-font-color2: rgba(255, 255, 255, 0.54);
  --xr-font-color3: rgba(255, 255, 255, 0.38);
  --xr-border-color: #1F1F1F;
  --xr-disabled-color: #515151;
  --xr-background-color: #111111;
  --xr-background-color-row-even: #111111;
  --xr-background-color-row-odd: #313131;
}

.xr-wrap {
  display: block;
  min-width: 300px;
  max-width: 700px;
}

.xr-text-repr-fallback {
  /* fallback to plain text repr when CSS is not injected (untrusted notebook) */
  display: none;
}

.xr-header {
  padding-top: 6px;
  padding-bottom: 6px;
  margin-bottom: 4px;
  border-bottom: solid 1px var(--xr-border-color);
}

.xr-header > div,
.xr-header > ul {
  display: inline;
  margin-top: 0;
  margin-bottom: 0;
}

.xr-obj-type,
.xr-array-name {
  margin-left: 2px;
  margin-right: 10px;
}

.xr-obj-type {
  color: var(--xr-font-color2);
}

.xr-sections {
  padding-left: 0 !important;
  display: grid;
  grid-template-columns: 150px auto auto 1fr 20px 20px;
}

.xr-section-item {
  display: contents;
}

.xr-section-item input {
  display: none;
}

.xr-section-item input + label {
  color: var(--xr-disabled-color);
}

.xr-section-item input:enabled + label {
  cursor: pointer;
  color: var(--xr-font-color2);
}

.xr-section-item input:enabled + label:hover {
  color: var(--xr-font-color0);
}

.xr-section-summary {
  grid-column: 1;
  color: var(--xr-font-color2);
  font-weight: 500;
}

.xr-section-summary > span {
  display: inline-block;
  padding-left: 0.5em;
}

.xr-section-summary-in:disabled + label {
  color: var(--xr-font-color2);
}

.xr-section-summary-in + label:before {
  display: inline-block;
  content: '►';
  font-size: 11px;
  width: 15px;
  text-align: center;
}

.xr-section-summary-in:disabled + label:before {
  color: var(--xr-disabled-color);
}

.xr-section-summary-in:checked + label:before {
  content: '▼';
}

.xr-section-summary-in:checked + label > span {
  display: none;
}

.xr-section-summary,
.xr-section-inline-details {
  padding-top: 4px;
  padding-bottom: 4px;
}

.xr-section-inline-details {
  grid-column: 2 / -1;
}

.xr-section-details {
  display: none;
  grid-column: 1 / -1;
  margin-bottom: 5px;
}

.xr-section-summary-in:checked ~ .xr-section-details {
  display: contents;
}

.xr-array-wrap {
  grid-column: 1 / -1;
  display: grid;
  grid-template-columns: 20px auto;
}

.xr-array-wrap > label {
  grid-column: 1;
  vertical-align: top;
}

.xr-preview {
  color: var(--xr-font-color3);
}

.xr-array-preview,
.xr-array-data {
  padding: 0 5px !important;
  grid-column: 2;
}

.xr-array-data,
.xr-array-in:checked ~ .xr-array-preview {
  display: none;
}

.xr-array-in:checked ~ .xr-array-data,
.xr-array-preview {
  display: inline-block;
}

.xr-dim-list {
  display: inline-block !important;
  list-style: none;
  padding: 0 !important;
  margin: 0;
}

.xr-dim-list li {
  display: inline-block;
  padding: 0;
  margin: 0;
}

.xr-dim-list:before {
  content: '(';
}

.xr-dim-list:after {
  content: ')';
}

.xr-dim-list li:not(:last-child):after {
  content: ',';
  padding-right: 5px;
}

.xr-has-index {
  font-weight: bold;
}

.xr-var-list,
.xr-var-item {
  display: contents;
}

.xr-var-item > div,
.xr-var-item label,
.xr-var-item > .xr-var-name span {
  background-color: var(--xr-background-color-row-even);
  margin-bottom: 0;
}

.xr-var-item > .xr-var-name:hover span {
  padding-right: 5px;
}

.xr-var-list > li:nth-child(odd) > div,
.xr-var-list > li:nth-child(odd) > label,
.xr-var-list > li:nth-child(odd) > .xr-var-name span {
  background-color: var(--xr-background-color-row-odd);
}

.xr-var-name {
  grid-column: 1;
}

.xr-var-dims {
  grid-column: 2;
}

.xr-var-dtype {
  grid-column: 3;
  text-align: right;
  color: var(--xr-font-color2);
}

.xr-var-preview {
  grid-column: 4;
}

.xr-var-name,
.xr-var-dims,
.xr-var-dtype,
.xr-preview,
.xr-attrs dt {
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
  padding-right: 10px;
}

.xr-var-name:hover,
.xr-var-dims:hover,
.xr-var-dtype:hover,
.xr-attrs dt:hover {
  overflow: visible;
  width: auto;
  z-index: 1;
}

.xr-var-attrs,
.xr-var-data {
  display: none;
  background-color: var(--xr-background-color) !important;
  padding-bottom: 5px !important;
}

.xr-var-attrs-in:checked ~ .xr-var-attrs,
.xr-var-data-in:checked ~ .xr-var-data {
  display: block;
}

.xr-var-data > table {
  float: right;
}

.xr-var-name span,
.xr-var-data,
.xr-attrs {
  padding-left: 25px !important;
}

.xr-attrs,
.xr-var-attrs,
.xr-var-data {
  grid-column: 1 / -1;
}

dl.xr-attrs {
  padding: 0;
  margin: 0;
  display: grid;
  grid-template-columns: 125px auto;
}

.xr-attrs dt,
.xr-attrs dd {
  padding: 0;
  margin: 0;
  float: left;
  padding-right: 10px;
  width: auto;
}

.xr-attrs dt {
  font-weight: normal;
  grid-column: 1;
}

.xr-attrs dt:hover span {
  display: inline-block;
  background: var(--xr-background-color);
  padding-right: 10px;
}

.xr-attrs dd {
  grid-column: 2;
  white-space: pre-wrap;
  word-break: break-all;
}

.xr-icon-database,
.xr-icon-file-text2 {
  display: inline-block;
  vertical-align: middle;
  width: 1em;
  height: 1.5em !important;
  stroke-width: 0;
  stroke: currentColor;
  fill: currentColor;
}
</style><pre class='xr-text-repr-fallback'><xarray.Dataset>
Dimensions:  (branch: 1, type: 2)
Coordinates:
  * type     (type) object 'unforced' 'forced'
    time     (type, branch) int64 2825812 2651661
  * branch   (branch) int64 0
Data variables:
    x        (type, branch) float64 19.58 19.47
    y        (type, branch) float64 18.77 18.4
    z        (type, branch) float64 43.92 42.76</pre><div class='xr-wrap' hidden><div class='xr-header'><div class='xr-obj-type'>xarray.Dataset</div></div><ul class='xr-sections'><li class='xr-section-item'><input id='section-7d77bd0e-c7b2-423c-8460-d1d6fbd5429d' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-7d77bd0e-c7b2-423c-8460-d1d6fbd5429d' class='xr-section-summary'  title='Expand/collapse section'>Dimensions:</label><div class='xr-section-inline-details'><ul class='xr-dim-list'><li><span class='xr-has-index'>branch</span>: 1</li><li><span class='xr-has-index'>type</span>: 2</li></ul></div><div class='xr-section-details'></div></li><li class='xr-section-item'><input id='section-b4f73550-97a8-4e06-b00d-050d566f4a30' class='xr-section-summary-in' type='checkbox'  checked><label for='section-b4f73550-97a8-4e06-b00d-050d566f4a30' class='xr-section-summary' >Coordinates: <span>(3)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>type</span></div><div class='xr-var-dims'>(type)</div><div class='xr-var-dtype'>object</div><div class='xr-var-preview xr-preview'>'unforced' 'forced'</div><input id='attrs-601d0a94-8e37-4ab8-82c2-ede0c69e7ac8' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-601d0a94-8e37-4ab8-82c2-ede0c69e7ac8' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-f1b4816c-cc36-435f-825c-23d259516452' class='xr-var-data-in' type='checkbox'><label for='data-f1b4816c-cc36-435f-825c-23d259516452' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array(['unforced', 'forced'], dtype=object)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>time</span></div><div class='xr-var-dims'>(type, branch)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>2825812 2651661</div><input id='attrs-4ba05572-8ecb-48ac-ba1c-957055163835' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-4ba05572-8ecb-48ac-ba1c-957055163835' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-c5a59395-5775-4754-9dde-1cf7fac3ba58' class='xr-var-data-in' type='checkbox'><label for='data-c5a59395-5775-4754-9dde-1cf7fac3ba58' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>hours since 1750-01-01 00:00:00</dd><dt><span>calendar :</span></dt><dd>proleptic_gregorian</dd></dl></div><div class='xr-var-data'><pre>array([[2825812],
       [2651661]])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>branch</span></div><div class='xr-var-dims'>(branch)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>0</div><input id='attrs-cf4649b4-c969-4db4-83f4-9f6b80a69b06' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-cf4649b4-c969-4db4-83f4-9f6b80a69b06' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-45e1df8f-b9bd-4dba-80c1-2df4f54e9cb4' class='xr-var-data-in' type='checkbox'><label for='data-45e1df8f-b9bd-4dba-80c1-2df4f54e9cb4' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([0])</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-19d49d97-9ad1-47f3-87f1-35c232f6efaf' class='xr-section-summary-in' type='checkbox'  checked><label for='section-19d49d97-9ad1-47f3-87f1-35c232f6efaf' class='xr-section-summary' >Data variables: <span>(3)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span>x</span></div><div class='xr-var-dims'>(type, branch)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-8a685f91-cf4b-438a-b6d3-1d77e3820acb' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-8a685f91-cf4b-438a-b6d3-1d77e3820acb' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-015698d7-0544-42cb-98d7-0a014802e7e9' class='xr-var-data-in' type='checkbox'><label for='data-015698d7-0544-42cb-98d7-0a014802e7e9' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([[19.580437],
       [19.474154]])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>y</span></div><div class='xr-var-dims'>(type, branch)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-a1513a18-922c-4512-a52a-cb62a6b15e2d' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-a1513a18-922c-4512-a52a-cb62a6b15e2d' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-23fce7b6-cac6-411b-8cce-f94522063142' class='xr-var-data-in' type='checkbox'><label for='data-23fce7b6-cac6-411b-8cce-f94522063142' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([[18.771317],
       [18.397154]])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>z</span></div><div class='xr-var-dims'>(type, branch)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-64db895a-e2ad-4171-a7dd-23fd9cc1384b' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-64db895a-e2ad-4171-a7dd-23fd9cc1384b' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-040a7fbf-81ab-46a1-94a1-5460a3231422' class='xr-var-data-in' type='checkbox'><label for='data-040a7fbf-81ab-46a1-94a1-5460a3231422' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([[43.921189],
       [42.7558  ]])</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-c65af658-9c81-4025-863a-35902ff9e196' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-c65af658-9c81-4025-863a-35902ff9e196' class='xr-section-summary'  title='Expand/collapse section'>Attributes: <span>(0)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'></dl></div></li></ul></div></div>

```python
(np.sqrt((test.sel(type='forced')-test_event).to_array()**2).sum('variable')).min()
```

<div><svg style="position: absolute; width: 0; height: 0; overflow: hidden">
<defs>
<symbol id="icon-database" viewBox="0 0 32 32">
<path d="M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z"></path>
<path d="M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
<path d="M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
</symbol>
<symbol id="icon-file-text2" viewBox="0 0 32 32">
<path d="M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z"></path>
<path d="M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
</symbol>
</defs>
</svg>
<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.
 *
 */

:root {
  --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));
  --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));
  --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));
  --xr-border-color: var(--jp-border-color2, #e0e0e0);
  --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);
  --xr-background-color: var(--jp-layout-color0, white);
  --xr-background-color-row-even: var(--jp-layout-color1, white);
  --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);
}

html[theme=dark],
body.vscode-dark {
  --xr-font-color0: rgba(255, 255, 255, 1);
  --xr-font-color2: rgba(255, 255, 255, 0.54);
  --xr-font-color3: rgba(255, 255, 255, 0.38);
  --xr-border-color: #1F1F1F;
  --xr-disabled-color: #515151;
  --xr-background-color: #111111;
  --xr-background-color-row-even: #111111;
  --xr-background-color-row-odd: #313131;
}

.xr-wrap {
  display: block;
  min-width: 300px;
  max-width: 700px;
}

.xr-text-repr-fallback {
  /* fallback to plain text repr when CSS is not injected (untrusted notebook) */
  display: none;
}

.xr-header {
  padding-top: 6px;
  padding-bottom: 6px;
  margin-bottom: 4px;
  border-bottom: solid 1px var(--xr-border-color);
}

.xr-header > div,
.xr-header > ul {
  display: inline;
  margin-top: 0;
  margin-bottom: 0;
}

.xr-obj-type,
.xr-array-name {
  margin-left: 2px;
  margin-right: 10px;
}

.xr-obj-type {
  color: var(--xr-font-color2);
}

.xr-sections {
  padding-left: 0 !important;
  display: grid;
  grid-template-columns: 150px auto auto 1fr 20px 20px;
}

.xr-section-item {
  display: contents;
}

.xr-section-item input {
  display: none;
}

.xr-section-item input + label {
  color: var(--xr-disabled-color);
}

.xr-section-item input:enabled + label {
  cursor: pointer;
  color: var(--xr-font-color2);
}

.xr-section-item input:enabled + label:hover {
  color: var(--xr-font-color0);
}

.xr-section-summary {
  grid-column: 1;
  color: var(--xr-font-color2);
  font-weight: 500;
}

.xr-section-summary > span {
  display: inline-block;
  padding-left: 0.5em;
}

.xr-section-summary-in:disabled + label {
  color: var(--xr-font-color2);
}

.xr-section-summary-in + label:before {
  display: inline-block;
  content: '►';
  font-size: 11px;
  width: 15px;
  text-align: center;
}

.xr-section-summary-in:disabled + label:before {
  color: var(--xr-disabled-color);
}

.xr-section-summary-in:checked + label:before {
  content: '▼';
}

.xr-section-summary-in:checked + label > span {
  display: none;
}

.xr-section-summary,
.xr-section-inline-details {
  padding-top: 4px;
  padding-bottom: 4px;
}

.xr-section-inline-details {
  grid-column: 2 / -1;
}

.xr-section-details {
  display: none;
  grid-column: 1 / -1;
  margin-bottom: 5px;
}

.xr-section-summary-in:checked ~ .xr-section-details {
  display: contents;
}

.xr-array-wrap {
  grid-column: 1 / -1;
  display: grid;
  grid-template-columns: 20px auto;
}

.xr-array-wrap > label {
  grid-column: 1;
  vertical-align: top;
}

.xr-preview {
  color: var(--xr-font-color3);
}

.xr-array-preview,
.xr-array-data {
  padding: 0 5px !important;
  grid-column: 2;
}

.xr-array-data,
.xr-array-in:checked ~ .xr-array-preview {
  display: none;
}

.xr-array-in:checked ~ .xr-array-data,
.xr-array-preview {
  display: inline-block;
}

.xr-dim-list {
  display: inline-block !important;
  list-style: none;
  padding: 0 !important;
  margin: 0;
}

.xr-dim-list li {
  display: inline-block;
  padding: 0;
  margin: 0;
}

.xr-dim-list:before {
  content: '(';
}

.xr-dim-list:after {
  content: ')';
}

.xr-dim-list li:not(:last-child):after {
  content: ',';
  padding-right: 5px;
}

.xr-has-index {
  font-weight: bold;
}

.xr-var-list,
.xr-var-item {
  display: contents;
}

.xr-var-item > div,
.xr-var-item label,
.xr-var-item > .xr-var-name span {
  background-color: var(--xr-background-color-row-even);
  margin-bottom: 0;
}

.xr-var-item > .xr-var-name:hover span {
  padding-right: 5px;
}

.xr-var-list > li:nth-child(odd) > div,
.xr-var-list > li:nth-child(odd) > label,
.xr-var-list > li:nth-child(odd) > .xr-var-name span {
  background-color: var(--xr-background-color-row-odd);
}

.xr-var-name {
  grid-column: 1;
}

.xr-var-dims {
  grid-column: 2;
}

.xr-var-dtype {
  grid-column: 3;
  text-align: right;
  color: var(--xr-font-color2);
}

.xr-var-preview {
  grid-column: 4;
}

.xr-var-name,
.xr-var-dims,
.xr-var-dtype,
.xr-preview,
.xr-attrs dt {
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
  padding-right: 10px;
}

.xr-var-name:hover,
.xr-var-dims:hover,
.xr-var-dtype:hover,
.xr-attrs dt:hover {
  overflow: visible;
  width: auto;
  z-index: 1;
}

.xr-var-attrs,
.xr-var-data {
  display: none;
  background-color: var(--xr-background-color) !important;
  padding-bottom: 5px !important;
}

.xr-var-attrs-in:checked ~ .xr-var-attrs,
.xr-var-data-in:checked ~ .xr-var-data {
  display: block;
}

.xr-var-data > table {
  float: right;
}

.xr-var-name span,
.xr-var-data,
.xr-attrs {
  padding-left: 25px !important;
}

.xr-attrs,
.xr-var-attrs,
.xr-var-data {
  grid-column: 1 / -1;
}

dl.xr-attrs {
  padding: 0;
  margin: 0;
  display: grid;
  grid-template-columns: 125px auto;
}

.xr-attrs dt,
.xr-attrs dd {
  padding: 0;
  margin: 0;
  float: left;
  padding-right: 10px;
  width: auto;
}

.xr-attrs dt {
  font-weight: normal;
  grid-column: 1;
}

.xr-attrs dt:hover span {
  display: inline-block;
  background: var(--xr-background-color);
  padding-right: 10px;
}

.xr-attrs dd {
  grid-column: 2;
  white-space: pre-wrap;
  word-break: break-all;
}

.xr-icon-database,
.xr-icon-file-text2 {
  display: inline-block;
  vertical-align: middle;
  width: 1em;
  height: 1.5em !important;
  stroke-width: 0;
  stroke: currentColor;
  fill: currentColor;
}
</style><pre class='xr-text-repr-fallback'><xarray.DataArray ()>
array(1.65350325)
Coordinates:
    type     <U6 'forced'
    event    <U3 'max'
    year     int64 1813</pre><div class='xr-wrap' hidden><div class='xr-header'><div class='xr-obj-type'>xarray.DataArray</div><div class='xr-array-name'></div></div><ul class='xr-sections'><li class='xr-section-item'><div class='xr-array-wrap'><input id='section-8ce1c1e3-7407-4da3-b160-afcf1b6a8c15' class='xr-array-in' type='checkbox' checked><label for='section-8ce1c1e3-7407-4da3-b160-afcf1b6a8c15' title='Show/hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-array-preview xr-preview'><span>1.654</span></div><div class='xr-array-data'><pre>array(1.65350325)</pre></div></div></li><li class='xr-section-item'><input id='section-e2a46dd8-e185-44f7-b8c5-a307ad4574db' class='xr-section-summary-in' type='checkbox'  checked><label for='section-e2a46dd8-e185-44f7-b8c5-a307ad4574db' class='xr-section-summary' >Coordinates: <span>(3)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span>type</span></div><div class='xr-var-dims'>()</div><div class='xr-var-dtype'><U6</div><div class='xr-var-preview xr-preview'>'forced'</div><input id='attrs-02def4f6-842f-403b-bbb1-008f506bf247' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-02def4f6-842f-403b-bbb1-008f506bf247' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-1d7a7f82-675c-4bef-8c27-0f98004b19ca' class='xr-var-data-in' type='checkbox'><label for='data-1d7a7f82-675c-4bef-8c27-0f98004b19ca' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array('forced', dtype='<U6')</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>event</span></div><div class='xr-var-dims'>()</div><div class='xr-var-dtype'><U3</div><div class='xr-var-preview xr-preview'>'max'</div><input id='attrs-18041c6e-52c8-49f1-a3ae-217553ca8aa3' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-18041c6e-52c8-49f1-a3ae-217553ca8aa3' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-9fc6fcd9-4281-4d5a-9f4f-61b051d70b69' class='xr-var-data-in' type='checkbox'><label for='data-9fc6fcd9-4281-4d5a-9f4f-61b051d70b69' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array('max', dtype='<U3')</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>year</span></div><div class='xr-var-dims'>()</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>1813</div><input id='attrs-cca4378b-994f-4260-8380-e5dac6c9cf06' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-cca4378b-994f-4260-8380-e5dac6c9cf06' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-a87cac5b-aed8-41b3-8960-baa0699aa1a0' class='xr-var-data-in' type='checkbox'><label for='data-a87cac5b-aed8-41b3-8960-baa0699aa1a0' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array(1813)</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-df8993cc-88ab-4a96-b6c6-80a5f188aed4' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-df8993cc-88ab-4a96-b6c6-80a5f188aed4' class='xr-section-summary'  title='Expand/collapse section'>Attributes: <span>(0)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'></dl></div></li></ul></div></div>

## convert to pandas for easy resampling
X_df = X.to_dataframe().reset_index()

## determine daily max (preserving coord values) & mean values
X_day = X_df.groupby(['type',X_df.time.dt.date]).apply(lambda x: x.iloc[x.metric.argmax()]).drop('type',axis=1)
X_day['metric_mean'] = X_df.groupby(['type',X_df.time.dt.date]).mean().metric
X_day = X_day.rename(dict(time='maxtime',metric='metric_max'),axis=1).to_xarray()
X_day = X_day.assign_coords(time=pd.to_datetime(X_day.time.values))

## determine yearly max of max & max of mean
X_df = X_day.to_dataframe().reset_index()
X_yearmax = X_df.groupby(['type',X_df.time.dt.year]).apply(lambda x: x.iloc[x.metric_max.argmax()]).drop(['type','time'],axis=1)
X_yearmax['metric_mean'] = X_df.groupby(['type',X_df.time.dt.year]).max().metric_mean
X_yearmax = X_yearmax.to_xarray()

```python
def locate_analogs(X0, n=1000, max_dist=2):

    """Finds analogs for X0 along the attractor defined by X."""

    X0 = X0.squeeze()

    E = xr.ufuncs.sqrt(((X.sel(type=X0.type.values) - X0).to_array()**2).sum('variable'))

    analog_times = E.rename('E').to_dataframe().sort_values('E').iloc[:n].index.values

    analogs = X.sel(time=analog_times,type=X0.type.values).drop('metric')

    return analogs.rename(time='number').assign_coords(number=np.arange(n))
```

```python
E = locate_analogs(rh_event)
```

```python
E_ranked = E.rename('E').to_dataframe().sort_values('E')
```

```python
E_ranked
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>type</th>
      <th>E</th>
    </tr>
    <tr>
      <th>time</th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>1926-07-30 19:00:00</th>
      <td>forced</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>2210-06-13 07:00:00</th>
      <td>forced</td>
      <td>0.553556</td>
    </tr>
    <tr>
      <th>1993-12-09 16:00:00</th>
      <td>forced</td>
      <td>0.754235</td>
    </tr>
    <tr>
      <th>2141-05-03 00:00:00</th>
      <td>forced</td>
      <td>0.804809</td>
    </tr>
    <tr>
      <th>2044-05-12 09:00:00</th>
      <td>forced</td>
      <td>1.019194</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>1817-08-05 12:00:00</th>
      <td>forced</td>
      <td>61.589837</td>
    </tr>
    <tr>
      <th>2047-07-29 16:00:00</th>
      <td>forced</td>
      <td>61.698691</td>
    </tr>
    <tr>
      <th>2221-06-22 23:00:00</th>
      <td>forced</td>
      <td>61.705505</td>
    </tr>
    <tr>
      <th>1758-08-06 21:00:00</th>
      <td>forced</td>
      <td>61.786215</td>
    </tr>
    <tr>
      <th>1759-09-24 20:00:00</th>
      <td>forced</td>
      <td>61.924374</td>
    </tr>
  </tbody>
</table>
<p>4382904 rows × 2 columns</p>
</div>

```python

```

```python

```

```python

```

```python

```

```python
def generate_forecasts(event,
                       leads = [1/8,1/4,1/2,3/4,1,2,3,5,10,20],
                       ens_size = 51,
                       ICP_type = 'random',
                       ran_gen = sp.stats.norm(0,0.005),
                       fc_timestep_size='15min'):

    """
    Creates a set of initalised Lorenz63 runs.

    Can choose how IC pertubrations are generated.
    """

    # choose leads (current freq "days")
    inidates = event.maxtime.values - pd.to_timedelta(leads,unit='d').round('1h')

    # extract initial conditions before the event
    ICs = X.sel(type='forced',time=inidates).rename(time='inidate').drop('metric')

    # generate perturbed initial conditions
    if ICP_type == 'random':

        ICPs = xr.DataArray(data=ran_gen.rvs(ens_size*3).reshape(ens_size,3),dims=['number','variable'],coords=dict(number=np.arange(ens_size),variable=['x','y','z']))
        ## make the first intial condition the "control"
        ICPs[0] = 0
        # add perturbations to control ICs to get array of ICs
        ICs = (ICs.to_array() + ICPs).transpose('inidate','number','variable')

    elif ICP_type == 'analog':

        ICs = ICs.groupby('inidate').apply(locate_analogs,n=ens_size)
        ICs = ICs.to_array().transpose('inidate','number','variable')

    ## integrate each ensemble member until a couple of days after the event
    ## convert timestep size to units of days
    fc_timestep = pd.to_timedelta(fc_timestep_size).total_seconds()/(24*3600)
    fc_end = event.maxtime.values + pd.Timedelta('2d')
    max_fc_length = ((fc_end - inidates).days+(fc_end - inidates).seconds/(3600*24)).max() / fc_timestep

    ### create results array
    fcs0 = np.empty((ens_size,int(max_fc_length),3,inidates.size))+np.nan
    fcs1 = np.empty((ens_size,int(max_fc_length),3,inidates.size))+np.nan

    for i,inidate in enumerate(ICs.inidate.values):

        fc_length = (fc_end - inidate)
        fc_timesteps = int(fc_length.total_seconds()/(fc_timestep*(24*60*60)))
        ics = ICs.sel(inidate=inidate).values

        fc0 = np.array([sp.integrate.odeint(lorenz, ic, np.arange(0,fc_timesteps*fc_timestep-1e-11,fc_timestep), (r,s,b,0,0)) for ic in ics])
        fc1 = np.array([sp.integrate.odeint(lorenz, ic, np.arange(0,fc_timesteps*fc_timestep-1e-11,fc_timestep), (r,s,b,*F)) for ic in ics])

        fcs0[:,-fc_timesteps:,:,i] = fc0
        fcs1[:,-fc_timesteps:,:,i] = fc1

    # wrangle into nice forecast DataArray
    fc_timeindex = pd.date_range(inidates.min(),fc_end,freq=fc_timestep_size)[:-1]
    fcs0 = xr.DataArray(data=fcs0,dims=['number','time','variable','inidate'],coords=dict(number=np.arange(ens_size),time=fc_timeindex,variable=['x','y','z'],inidate=inidates)).to_dataset(dim="variable")
    fcs1 = xr.DataArray(data=fcs1,dims=['number','time','variable','inidate'],coords=dict(number=np.arange(ens_size),time=fc_timeindex,variable=['x','y','z'],inidate=inidates)).to_dataset(dim="variable")
    ## join datasets together
    fcs = xr.concat([fcs0.expand_dims({'type':['unforced']}),fcs1.expand_dims({'type':['forced']})],dim='type')

    # create impact variable
#     fcs['impact'] = xr.ufuncs.fabs(fcs.x+fcs.y)

    return fcs
```

```python
## event I (right hand lobe): '1926-07-30T19:00:00.000000000'
## event II (left hand lobe): '1897-01-17T04:00:00.000000000'
rh_event = X_day.sel(time='1926-07-30',type='forced')
fcs_r = generate_forecasts(rh_event, leads=[1/4,1,2], ens_size=1001, ICP_type='analog', ran_gen=sp.stats.norm(0,1), fc_timestep_size='1min')
fcs_r['metric'] = np.fabs(fcs_r.x)
```

```python
lh_event = X_day.sel(time='1897-01-17',type='forced')
fcs_l = generate_forecasts(lh_event, leads=[1/4,1,2], ens_size=1001, ICP_type='analog', ran_gen=sp.stats.norm(0,1), fc_timestep_size='1min')
fcs_l['metric'] = np.fabs(fcs_l.x)
```

```python
"""Choose whether to plot interactively (keep off for generating anomations)"""
%matplotlib inline
```

```python
g=sn.displot(data=fcs_l.x.min('time').to_dataframe().reset_index(),x='x',hue='type',col='inidate',kind='hist',element='step',bins=np.linspace(-21,-16,51))
g.fig.patch.set_facecolor('xkcd:white')
# g.fig.savefig('./fcs_l.png',dpi=200)
```

```python
g=sn.displot(data=fcs_r.x.max('time').to_dataframe().reset_index(),x='x',hue='type',col='inidate',kind='hist',element='step',bins=np.linspace(16,21,51))
g.fig.patch.set_facecolor('xkcd:white')
# g.fig.savefig('./fcs_r.png',dpi=200)
```

```python

```

```python

```

Animate the left node event first.

```python
## left node
event_start_time = '1897-01-17T00:00:00'
event_end_time = '1897-01-17T08:00:00'
choose_mems = np.random.choice(500,15)
plot_colors = dict(forced=get_palette.Petroff6().to_sn_palette()[0],unforced=get_palette.Petroff6().to_sn_palette()[1])

def generate_plot(fr_no,inidate):

    fig,axes = plt.subplots(1,2,figsize=(10,5))
    ax,ax1=axes
    sn.despine()
    ax.set_xlabel('x')
    ax.set_ylabel('y')
    ax1.set_xlabel('x')
    ax1.set_xlim(-21,-16)
    ax1.set_ylim(0,150)

    hist0_points = []
    hist1_points = []

    for lnum,mem in enumerate(choose_mems):

        X0 = fcs_l.sel(number=mem,inidate=inidate,type='unforced',time=slice(inidate,event_end_time))
        X1 = fcs_l.sel(number=mem,inidate=inidate,type='forced',time=slice(inidate,event_end_time))

        maxtime0 = X0.x.idxmin('time')
        maxtime1 = X1.x.idxmin('time')

        argtime0 = ((maxtime0 - X0.time.isel(time=0)).astype(float)//(1e9*60)).values[()].astype(int)
        argtime1 = ((maxtime1 - X1.time.isel(time=0)).astype(float)//(1e9*60)).values[()].astype(int)

        X00 = X0.where(X0.time<=maxtime0).isel(time=slice(0,fr_no)).squeeze()
        ax.plot(X00.x,X00.y,lw=1,color=plot_colors['unforced'],zorder=5)

        X10 = X1.where(X1.time<=maxtime1).isel(time=slice(0,fr_no)).squeeze()
        ax.plot(X10.x,X10.y,lw=1,color=plot_colors['forced'],zorder=5)

        if fr_no >= argtime0:
            X01 = X0.where(X0.time>=maxtime0).isel(time=slice(0,fr_no)).squeeze()
            ax.plot(X01.x,X01.y,lw=1,color=plot_colors['unforced'],alpha=0.1,zorder=3)

            X0.sel(time=maxtime0).squeeze().plot.scatter('x','y',color=plot_colors['unforced'],ax=ax,zorder=10,alpha=np.exp(0.02*(argtime0-fr_no)))

        if fr_no >= argtime1:
            X11 = X1.where(X1.time>=maxtime1).isel(time=slice(0,fr_no)).squeeze()
            ax.plot(X11.x,X11.y,lw=1,color=plot_colors['forced'],alpha=0.1,zorder=3)

            X1.sel(time=maxtime1).squeeze().plot.scatter('x','y',color=plot_colors['forced'],ax=ax,zorder=10,alpha=np.exp(0.02*(argtime1-fr_no)))

    ## get histogram points
    if fr_no>0:
        event_max0 = fcs_l.sel(inidate=inidate,type='unforced',time=slice(inidate,event_end_time)).x.min('time').squeeze().values
        actual_max0 = fcs_l.sel(inidate=inidate,type='unforced',time=slice(inidate,event_end_time)).isel(time=slice(0,fr_no)).x.min('time').squeeze().values
        hist0_points = event_max0[event_max0==actual_max0]

        event_max1 = fcs_l.sel(inidate=inidate,type='forced',time=slice(inidate,event_end_time)).x.min('time').squeeze().values
        actual_max1 = fcs_l.sel(inidate=inidate,type='forced',time=slice(inidate,event_end_time)).isel(time=slice(0,fr_no)).x.min('time').squeeze().values
        hist1_points = event_max1[event_max1==actual_max1]

    ## plot ICs
    fcs_l.sel(number=choose_mems,type='forced',inidate=inidate,time=inidate).plot.scatter('x','y',color='k',ax=ax,marker='.')
    fcs_l.sel(type='forced',inidate=inidate,time=inidate).plot.scatter('x','y',color='grey',ax=ax,marker='.',s=1,zorder=-1)

    ## plot histograms
    bins = np.linspace(-21,-16,51)
    ax1.hist(hist0_points, bins=bins, lw=1.5, color=plot_colors['unforced'], histtype='step',label='unforced')
    ax1.hist(hist1_points, bins=bins, lw=1.5, color=plot_colors['forced'], histtype='step',label='forced')

    # figure layout
    ## x,y lim
    xlim0 = fcs_l.sel(number=0,type='forced',inidate=inidate,time=inidate).x.values[()] + np.array([-4,4])
    ylim0 = fcs_l.sel(number=0,type='forced',inidate=inidate,time=inidate).y.values[()] + np.array([-4,4])

    xlim1 = np.array([-25,25])
    ylim1 = np.array([-25,25])

    frames_to_zoom = 120

    curr_xlim = ax.set_xlim()
    curr_ylim = ax.set_ylim()

    xlim_min = np.min([curr_xlim[0],np.max([xlim1[0],xlim0[0]+(fr_no-180)*(xlim1[0]-xlim0[0])/frames_to_zoom])])
    xlim_max = np.max([curr_xlim[1],np.min([xlim1[1],xlim0[1]+(fr_no-180)*(xlim1[1]-xlim0[1])/frames_to_zoom])])

    ylim_min = np.min([curr_ylim[0],np.max([ylim1[0],ylim0[0]+(fr_no-180)*(ylim1[0]-ylim0[0])/frames_to_zoom])])
    ylim_max = np.max([curr_ylim[1],np.min([ylim1[1],ylim0[1]+(fr_no-180)*(ylim1[1]-ylim0[1])/frames_to_zoom])])

    ax.set_xlim(xlim_min,xlim_max)
    ax.set_ylim(ylim_min,ylim_max)

    ax1.legend(frameon=False,loc='upper right')

    fig.savefig('../Figures/animation-figs/'+f"{fr_no:04d}"+'_'+str(inidate)+'.png',dpi=200)

    fig.clear()
    plt.close(fig)
    gc.collect()

for inidate in tqdm(fcs_l.inidate.values):

    total_frames = int((pd.to_datetime(event_end_time) - pd.to_datetime(inidate)).total_seconds()/60)

    P1 = multiprocessing.Pool(processes=4)

    P1.starmap(generate_plot,[(fr_no,inidate) for fr_no in np.arange(total_frames)])

    P1.close()
```

Then the right.

```python
## right node
event_start_time = '1926-07-30T15:00'
event_end_time = '1926-07-30T23:00'
choose_mems = np.random.choice(500,15)
plot_colors = dict(forced=get_palette.Petroff6().to_sn_palette()[0],unforced=get_palette.Petroff6().to_sn_palette()[1])

def generate_plot(fr_no,inidate):

    fig,axes = plt.subplots(1,2,figsize=(10,5))
    ax,ax1=axes
    sn.despine()
    ax.set_xlabel('x')
    ax.set_ylabel('y')
    ax1.set_xlabel('x')
    ax1.set_xlim(16,21)
    ax1.set_ylim(0,150)

    hist0_points = []
    hist1_points = []

    for lnum,mem in enumerate(choose_mems):

        X0 = fcs_r.sel(number=mem,inidate=inidate,type='unforced',time=slice(inidate,event_end_time))
        X1 = fcs_r.sel(number=mem,inidate=inidate,type='forced',time=slice(inidate,event_end_time))

        maxtime0 = X0.x.idxmax('time')
        maxtime1 = X1.x.idxmax('time')

        argtime0 = ((maxtime0 - X0.time.isel(time=0)).astype(float)//(1e9*60)).values[()].astype(int)
        argtime1 = ((maxtime1 - X1.time.isel(time=0)).astype(float)//(1e9*60)).values[()].astype(int)

        X00 = X0.where(X0.time<=maxtime0).isel(time=slice(0,fr_no)).squeeze()
        ax.plot(X00.x,X00.y,lw=1,color=plot_colors['unforced'],zorder=5)

        X10 = X1.where(X1.time<=maxtime1).isel(time=slice(0,fr_no)).squeeze()
        ax.plot(X10.x,X10.y,lw=1,color=plot_colors['forced'],zorder=5)

        if fr_no >= argtime0:
            X01 = X0.where(X0.time>=maxtime0).isel(time=slice(0,fr_no)).squeeze()
            ax.plot(X01.x,X01.y,lw=1,color=plot_colors['unforced'],alpha=0.1,zorder=3)

            X0.sel(time=maxtime0).squeeze().plot.scatter('x','y',color=plot_colors['unforced'],ax=ax,zorder=10,alpha=np.exp(0.02*(argtime0-fr_no)))

        if fr_no >= argtime1:
            X11 = X1.where(X1.time>=maxtime1).isel(time=slice(0,fr_no)).squeeze()
            ax.plot(X11.x,X11.y,lw=1,color=plot_colors['forced'],alpha=0.1,zorder=3)

            X1.sel(time=maxtime1).squeeze().plot.scatter('x','y',color=plot_colors['forced'],ax=ax,zorder=10,alpha=np.exp(0.02*(argtime1-fr_no)))

    ## get histogram points
    if fr_no>0:
        event_max0 = fcs_r.sel(inidate=inidate,type='unforced',time=slice(inidate,event_end_time)).x.max('time').squeeze().values
        actual_max0 = fcs_r.sel(inidate=inidate,type='unforced',time=slice(inidate,event_end_time)).isel(time=slice(0,fr_no)).x.max('time').squeeze().values
        hist0_points = event_max0[event_max0==actual_max0]

        event_max1 = fcs_r.sel(inidate=inidate,type='forced',time=slice(inidate,event_end_time)).x.max('time').squeeze().values
        actual_max1 = fcs_r.sel(inidate=inidate,type='forced',time=slice(inidate,event_end_time)).isel(time=slice(0,fr_no)).x.max('time').squeeze().values
        hist1_points = event_max1[event_max1==actual_max1]

    ## plot ICs
    fcs_r.sel(number=choose_mems,type='forced',inidate=inidate,time=inidate).plot.scatter('x','y',color='k',ax=ax,marker='.')
    fcs_r.sel(type='forced',inidate=inidate,time=inidate).plot.scatter('x','y',color='grey',ax=ax,marker='.',s=1,zorder=-1)

    ## plot histograms
    bins = np.linspace(16,21,51)
    ax1.hist(hist0_points, bins=bins, lw=1.5, color=plot_colors['unforced'], histtype='step',label='unforced')
    ax1.hist(hist1_points, bins=bins, lw=1.5, color=plot_colors['forced'], histtype='step',label='forced')

    # figure layout
    ## x,y lim
    xlim0 = fcs_r.sel(number=0,type='forced',inidate=inidate,time=inidate).x.values[()] + np.array([-4,4])
    ylim0 = fcs_r.sel(number=0,type='forced',inidate=inidate,time=inidate).y.values[()] + np.array([-4,4])

    xlim1 = np.array([-25,25])
    ylim1 = np.array([-25,25])

    frames_to_zoom = 120

    curr_xlim = ax.set_xlim()
    curr_ylim = ax.set_ylim()

    xlim_min = np.min([curr_xlim[0],np.max([xlim1[0],xlim0[0]+(fr_no-180)*(xlim1[0]-xlim0[0])/frames_to_zoom])])
    xlim_max = np.max([curr_xlim[1],np.min([xlim1[1],xlim0[1]+(fr_no-180)*(xlim1[1]-xlim0[1])/frames_to_zoom])])

    ylim_min = np.min([curr_ylim[0],np.max([ylim1[0],ylim0[0]+(fr_no-180)*(ylim1[0]-ylim0[0])/frames_to_zoom])])
    ylim_max = np.max([curr_ylim[1],np.min([ylim1[1],ylim0[1]+(fr_no-180)*(ylim1[1]-ylim0[1])/frames_to_zoom])])

    ax.set_xlim(xlim_min,xlim_max)
    ax.set_ylim(ylim_min,ylim_max)

    ax1.legend(frameon=False,loc='upper right')

    fig.savefig('../Figures/animation-figs/'+f"{fr_no:04d}"+'_'+str(inidate)+'.png',dpi=200)

    fig.clear()
    plt.close(fig)
    gc.collect()

for inidate in tqdm(fcs_r.inidate.values):

    total_frames = int((pd.to_datetime(event_end_time) - pd.to_datetime(inidate)).total_seconds()/60)

    P1 = multiprocessing.Pool(processes=4)

    P1.starmap(generate_plot,[(fr_no,inidate) for fr_no in np.arange(total_frames)])

    P1.close()
```

```python
for inidate in tqdm(fcs_r.inidate.values):

    fig=plt.figure()

    hist0_points = fcs_r.sel(inidate=inidate,type='unforced',time=slice(inidate,event_end_time)).x.max('time').squeeze().values
    hist1_points = fcs_r.sel(inidate=inidate,type='forced',time=slice(inidate,event_end_time)).x.max('time').squeeze().values

    bins = np.linspace(16,21,51)
    plt.hist(hist0_points, bins=bins, lw=1.5, color=plot_colors['unforced'], histtype='step')
    plt.hist(hist1_points, bins=bins, lw=1.5, color=plot_colors['forced'], histtype='step')

    plt.savefig('./test'+str(inidate)+'.png',dpi=200)

    fig.clear()

    plt.close()
```

```python

```


# Árvore de decisão: o aprendizado de máquina presente no computador de classificação de rochas

Sumário da apresentação

Introdução

Metodologia

Importando as dependências necessárias

Criando o seu dado (Tabela de Frequências):

Transformando o seu dado em um objeto do pandas

## Introdução

* Árvores de decisão são metaheurísticas dentro do campo da inteligência artificial que têm sido amplamente utilizadas para construir modelos de classificação, pois esses modelos se assemelham muito ao raciocínio humano sendo fáceis de entender.

 * O algoritmo é estruturado como uma série modelos sequenciais, que combinam logicamenteum conjunto de testes simples, onde cada teste compara um atributo numérico com um valor limite ou um atributo nominal com um conjunto de valores possíveis \cite{Kotsiantis2013}.

## Metodologia

* O algoritmo de árvore de decisão \cite{sklearn_api} é proposto para a solução de um problema de classificação de rochas simples do tipo binária\cite{scikit-learn}.

* Classificação do tipo binária baseia-se em dois aspectos principais a entropia e o ganho de informação \cite{Sayad2020}. O ganho de informação ou divergência \textit{Kullback–Leibler} é a diferença das entropia nas iterações i e i+1. Onde

 $$ i = 0,1,2,3 ..., n \ \forall \ n \in \mathbb{I} $$

* A entropia pode ser definida com base na tabela de frequências que carrega as informações do seu dado. Quando a tabela de frequencia possui dois atributos a entropia pode ser escrita como:

\begin{equation}
E(t,x) = \sum_{c \in x}  P(c)E(c)
\end{equation}

Existe uma variação da árvore de decião que utiliza  a rega de bayes sendo conhecia como árvore de decisão bayesiana.
O teorema de bayes e uma igualdade simples que afirma $prob(A \ e \ B) = prob(B \ e\  A)$ :

\begin{equation}
P(B|A) = \frac{prob(A|B) prob(B)}{prob(A)} \nonumber
\end{equation}

Onde

* $prob(A)$ probabilidade total
* $prob(B)$ é chamado de probabilidade a priori a qual será modificada pela experiência
* $prob(A|B)$ é a verossimilança que determina a experiência
* $prob(B|A)$ é a probabilidade posterior, ou o nível de crença após a realização do experimento

Esse teorema é útil quando interpretado como uma regra para indução: os dados e o evento A são considerados como sucessores de B, o grau de crença anterior a realização do experimento.

## Importando as dependências necessárias

```python
# -*- coding: utf-8 -*-
%matplotlib inline
import pandas as pd # biblioteca que trabalha com tabela de dados
from sklearn import tree # modelo de árvore de decisão do Sci-kit
from IPython.display import Image # pacote que carrega imagens
from PIL import Image, ImageFilter # pacote que processa imagens
```

## Criando o seu dado (Tabela de Frequências):

```python
features = [[140, 1], [130, 1],
           [150, 0], [170, 0]]# diâmetro e gomos
labels = [0, 0, 1, 1] # 0 é maçã e 1 é laranja
```

 ## Transformando o seu dado em um objeto do pandas

```python
Tabela = {'diâmetro': [140, 130, 150, 170],
          'gomo': [1, 1, 0 ,0 ],
          'frutas': [0, 0, 1, 1]
          }
TF = pd.DataFrame(data=Tabela)
print(TF)

```

       diâmetro  gomo  frutas
    0       140     1       0
    1       130     1       0
    2       150     0       1
    3       170     0       1

```python

# o classificador encontra padrões nos dados de treinamento
clf = tree.DecisionTreeClassifier() # instância do classificador
clf = clf.fit(features, labels) # fit encontra padrões nos dados

# iremos utilizar para classificar uma nova fruta
print(clf.predict([[9, 0]]))
```

    [0]

# No caso do computador classificador de rochas ...

[Chaves classificadoras](imagens/perguntas.pdf)

```python
im = Image.open( 'imagens/perguntas.png' )
im.show()
```

```python
# no caso classificação de rochas (Banco de rochas):

nr = 15 # número total de rochas do treinamento

BD = {'Possui cristais?': [1,1,1,1,1,0,0,0,0,0,0,0,0,0,0],
      'Possui camadas?' : [1,1,0,0,0],
      'As camadas possuem materiais diferentes:': [1,0]
      }

rochas = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] # Ids de rochas
```

# References

(Kotsiantis, 2013) Kotsiantis S. B., ``_Decision trees: a recent overview_'', Artificial Intelligence Review, vol. 39, number 4, pp. 261--283, Apr 2013.  [online](https://doi.org/10.1007/s10462-011-9272-4)

(Buitinck, Louppe et al., 2013) L. Buitinck, G. Louppe, M. Blondel et al., ``_API design for machine learning software: experiences from the scikit-learn
               project_'', ECML PKDD Workshop: Languages for Data Mining and Machine Learning,  2013.

(Pedregosa, Varoquaux et al., 2011) Pedregosa F., Varoquaux G., Gramfort A. et al., ``_Scikit-learn: Machine Learning in Python_'', Journal of Machine Learning Research, vol. 12, number , pp. 2825--2830,  2011.

(Saed, 2020) Dr. Saed, ``_An Introduction to Data Science: Decision Tree_'',  2020.  [online](https://www.saedsayad.com/data_mining_map.htm)

```python

```