##### Copyright 2020 The TensorFlow Authors.

```python
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Hello, many worlds

    View on TensorFlow.org

    Run in Google Colab

    View source on GitHub

    Download notebook

This tutorial shows how a classical neural network can learn to correct qubit calibration errors. It introduces Cirq, a Python framework to create, edit, and invoke Noisy Intermediate Scale Quantum (NISQ) circuits, and demonstrates how Cirq interfaces with TensorFlow Quantum.

## Setup

```python
try:
    # %tensorflow_version only works in Colab.
    %tensorflow_version 2.x
except Exception:
    pass
```

Install TensorFlow Quantum:

```python
!pip install tensorflow-quantum
```

Now import TensorFlow and the module dependencies:

```python
import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy
import numpy as np

# visualization tools
%matplotlib inline
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

## 1. The Basics

### 1.1 Cirq and parameterized quantum circuits

Before exploring TensorFlow Quantum (TFQ), let's look at some Cirq basics. Cirq is a Python library for quantum computing from Google. You use it to define circuits, including static and parameterized gates.

Cirq uses SymPy symbols to represent free parameters.

```python
a, b = sympy.symbols('a b')
```

The following code creates a two-qubit circuit using your parameters:

```python
# Create two qubits
q0, q1 = cirq.GridQubit.rect(1, 2)

# Create a circuit on these qubits using the parameters you created above.
circuit = cirq.Circuit(
    cirq.rx(a).on(q0),
    cirq.ry(b).on(q1), cirq.CNOT(control=q0, target=q1))

SVGCircuit(circuit)
```

To evaluate circuits, you can use the `cirq.Simulator` interface. You replace free parameters in a circuit with specific numbers by passing in a `cirq.ParamResolver` object. The following code calculates the raw state vector output of your parameterized circuit:

```python
# Calculate a state vector with a=0.5 and b=-0.5.
resolver = cirq.ParamResolver({a: 0.5, b: -0.5})
output_state_vector = cirq.Simulator().simulate(circuit, resolver).final_state
output_state_vector
```

State vectors are not directly accessible outside of simulation (notice the complex numbers in the output above). To be physically realistic, you must specify a measurement, which converts a state vector into a real number that classical computers can understand. Cirq specifies measurements using combinations of the Pauli operators $\hat{X}$, $\hat{Y}$, and $\hat{Z}$. As illustration, the following code measures $\hat{Z}_0$ and $\frac{1}{2}\hat{Z}_0 + \hat{X}_1$ on the state vector you just simulated:

```python
z0 = cirq.Z(q0)

qubit_map={q0: 0, q1: 1}

z0.expectation_from_wavefunction(output_state_vector, qubit_map).real
```

```python
z0x1 = 0.5 * z0 + cirq.X(q1)

z0x1.expectation_from_wavefunction(output_state_vector, qubit_map).real
```

### 1.2 Quantum circuits as tensors

TensorFlow Quantum (TFQ) provides `tfq.convert_to_tensor`, a function that converts Cirq objects into tensors. This allows you to send Cirq objects to our quantum layers and quantum ops. The function can be called on lists or arrays of Cirq Circuits and Cirq Paulis:

```python
# Rank 1 tensor containing 1 circuit.
circuit_tensor = tfq.convert_to_tensor([circuit])

print(circuit_tensor.shape)
print(circuit_tensor.dtype)
```

This encodes the Cirq objects as `tf.string` tensors that `tfq` operations decode as needed.

```python
# Rank 1 tensor containing 2 Pauli operators.
pauli_tensor = tfq.convert_to_tensor([z0, z0x1])
pauli_tensor.shape
```

### 1.3 Batching circuit simulation

TFQ provides methods for computing expectation values, samples, and state vectors. For now, let's focus on *expectation values*.

The highest-level interface for calculating expectation values is the `tfq.layers.Expectation` layer, which is a `tf.keras.Layer`. In its simplest form, this layer is equivalent to simulating a parameterized circuit over many `cirq.ParamResolvers`; however, TFQ allows batching following TensorFlow semantics, and circuits are simulated using efficient C++ code.

Create a batch of values to substitute for our `a` and `b` parameters:

```python
batch_vals = np.array(np.random.uniform(0, 2 * np.pi, (5, 2)), dtype=np.float32)
```

Batching circuit execution over parameter values in Cirq requires a loop:

```python
cirq_results = []
cirq_simulator = cirq.Simulator()

for vals in batch_vals:
    resolver = cirq.ParamResolver({a: vals[0], b: vals[1]})
    final_state = cirq_simulator.simulate(circuit, resolver).final_state
    cirq_results.append(
        [z0.expectation_from_wavefunction(final_state, {
            q0: 0,
            q1: 1
        }).real])

print('cirq batch results: \n {}'.format(np.array(cirq_results)))
```

The same operation is simplified in TFQ:

```python
tfq.layers.Expectation()(circuit,
                         symbol_names=[a, b],
                         symbol_values=batch_vals,
                         operators=z0)
```

## 2. Hybrid quantum-classical optimization

Now that you've seen the basics, let's use TensorFlow Quantum to construct a *hybrid quantum-classical neural net*. You will train a classical neural net to control a single qubit. The control will be optimized to correctly prepare the qubit in the `0` or `1` state, overcoming a simulated systematic calibration error. This figure shows the architecture:

Even without a neural network this is a straightforward problem to solve, but the theme is similar to the real quantum control problems you might solve using TFQ. It demonstrates an end-to-end example of a quantum-classical computation using the `tfq.layers.ControlledPQC` (Parametrized Quantum Circuit) layer inside of a `tf.keras.Model`.

For the implementation of this tutorial, this is architecture is split into 3 parts:

- The *input circuit* or *datapoint circuit*: The first three $R$ gates.
- The *controlled circuit*: The other three $R$ gates.
- The *controller*: The classical neural-network setting the parameters of the controlled circuit.

### 2.1 The controlled circuit definition

Define a learnable single bit rotation, as indicated in the figure above. This will correspond to our controlled circuit.

```python
# Parameters that the classical NN will feed values into.
control_params = sympy.symbols('theta_1 theta_2 theta_3')

# Create the parameterized circuit.
qubit = cirq.GridQubit(0, 0)
model_circuit = cirq.Circuit(
    cirq.rz(control_params[0])(qubit),
    cirq.ry(control_params[1])(qubit),
    cirq.rx(control_params[2])(qubit))

SVGCircuit(model_circuit)
```

### 2.2 The controller

Now define controller network:

```python
# The classical neural network layers.
controller = tf.keras.Sequential([
    tf.keras.layers.Dense(10, activation='elu'),
    tf.keras.layers.Dense(3)
])
```

Given a batch of commands, the controller outputs a batch of control signals for the controlled circuit.

The controller is randomly initialized so these outputs are not useful, yet.

```python
controller(tf.constant([[0.0],[1.0]])).numpy()
```

### 2.3 Connect the controller to the circuit

Use `tfq` to connect the controller to the controlled circuit, as a single `keras.Model`.

See the [Keras Functional API guide](https://www.tensorflow.org/guide/keras/functional) for more about this style of model definition.

First define the inputs to the model:

```python
# This input is the simulated miscalibration that the model will learn to correct.
circuits_input = tf.keras.Input(shape=(),
                                # The circuit-tensor has dtype `tf.string`
                                dtype=tf.string,
                                name='circuits_input')

# Commands will be either `0` or `1`, specifying the state to set the qubit to.
commands_input = tf.keras.Input(shape=(1,),
                                dtype=tf.dtypes.float32,
                                name='commands_input')

```

Next apply operations to those inputs, to define the computation.

```python
dense_2 = controller(commands_input)

# TFQ layer for classically controlled circuits.
expectation_layer = tfq.layers.ControlledPQC(model_circuit,
                                             # Observe Z
                                             operators = cirq.Z(qubit))
expectation = expectation_layer([circuits_input, dense_2])
```

Now package this computation as a `tf.keras.Model`:

```python
# The full Keras model is built from our layers.
model = tf.keras.Model(inputs=[circuits_input, commands_input],
                       outputs=expectation)
```

The network architecture is indicated by the plot of the model below.
Compare this model plot to the architecture diagram to verify correctness.

Note: May require a system install of the `graphviz` package.

```python
tf.keras.utils.plot_model(model, show_shapes=True, dpi=70)
```

This model takes two inputs: The commands for the controller, and the input-circuit whos output the controller is attempting to correct.

### 2.4 The dataset

The model attempts to output the correct correct measurement value of $\hat{Z}$ for each command. The commands and correct values are defined below.

```python
# The command input values to the classical NN.
commands = np.array([[0], [1]], dtype=np.float32)

# The desired Z expectation value at output of quantum circuit.
expected_outputs = np.array([[1], [-1]], dtype=np.float32)
```

This is not the entire training dataset for this task.
Each datapoint in the dataset also needs an input circuit.

### 2.4 Input crcuit definition

The input-circuit below defines the random miscalibration the model will learn to correct.

```python
random_rotations = np.random.uniform(0, 2 * np.pi, 3)
noisy_preparation = cirq.Circuit(
  cirq.rx(random_rotations[0])(qubit),
  cirq.ry(random_rotations[1])(qubit),
  cirq.rz(random_rotations[2])(qubit)
)
datapoint_circuits = tfq.convert_to_tensor([
  noisy_preparation
] * 2)  # Make two copied of this circuit
```

There are two copies of the circuit, one for each datapoint.

```python
datapoint_circuits.shape
```

### 2.5 Training

With the inputs defined you can test-run the `tfq` model.

```python
model([datapoint_circuits, commands]).numpy()
```

Now run a standard training process to adjust these values towards the `expected_outputs`.

```python
optimizer = tf.keras.optimizers.Adam(learning_rate=0.05)
loss = tf.keras.losses.MeanSquaredError()
model.compile(optimizer=optimizer, loss=loss)
history = model.fit(x=[datapoint_circuits, commands],
                    y=expected_outputs,
                    epochs=30,
                    verbose=0)
```

```python
plt.plot(history.history['loss'])
plt.title("Learning to Control a Qubit")
plt.xlabel("Iterations")
plt.ylabel("Error in Control")
plt.show()
```

From this plot you can see that the neural network has learned to overcome the systematic miscalibration.

### 2.6 Verify outputs
Now use the trained model, to correct the qubit calibration errors. With Cirq:

```python
def check_error(command_values, desired_values):
  """Based on the value in `command_value` see how well you could prepare
  the full circuit to have `desired_value` when taking expectation w.r.t. Z."""
  params_to_prepare_output = controller(command_values).numpy()
  full_circuit = noisy_preparation + model_circuit

  # Test how well you can prepare a state to get expectation the expectation
  # value in `desired_values`
  for index in [0, 1]:
    state = cirq_simulator.simulate(
        full_circuit,
        {s:v for (s,v) in zip(control_params, params_to_prepare_output[index])}
    ).final_state
    expectation = z0.expectation_from_wavefunction(state, {qubit: 0}).real
    print(f'For a desired output (expectation) of {desired_values[index]} with'
          f' noisy preparation, the controller\nnetwork found the following '
          f'values for theta: {params_to_prepare_output[index]}\nWhich gives an'
          f' actual expectation of: {expectation}\n')

check_error(commands, expected_outputs)
```

The value of the loss function during training provides a rough idea of how well the model is learning. The lower the loss, the closer the expectation values in the above cell is to `desired_values`. If you aren't as concerned with the parameter values, you can always check the outputs from above using `tfq`:

```python
model([datapoint_circuits, commands])
```

## 3 Learning to prepare eigenstates of different operators

The choice of the $\pm \hat{Z}$ eigenstates corresponding to 1 and 0 was arbitrary. You could have just as easily wanted 1 to correspond to the $+ \hat{Z}$ eigenstate and 0 to correspond to the $-\hat{X}$ eigenstate. One way to accomplish this is by specifying a different measurement operator for each command, as indicated in the figure below:

This requires use of tfq.layers.Expectation. Now your input has grown to include three objects: circuit, command, and operator. The output is still the expectation value.

### 3.1 New model definition

Lets take a look at the model to accomplish this task:

```python
# Define inputs.
commands_input = tf.keras.layers.Input(shape=(1),
                                       dtype=tf.dtypes.float32,
                                       name='commands_input')
circuits_input = tf.keras.Input(shape=(),
                                # The circuit-tensor has dtype `tf.string`
                                dtype=tf.dtypes.string,
                                name='circuits_input')
operators_input = tf.keras.Input(shape=(1,),
                                 dtype=tf.dtypes.string,
                                 name='operators_input')
```

Here is the controller network:

```python
# Define classical NN.
controller = tf.keras.Sequential([
    tf.keras.layers.Dense(10, activation='elu'),
    tf.keras.layers.Dense(3)
])
```

Combine the circuit and the controller into a single `keras.Model` using `tfq`:

```python
dense_2 = controller(commands_input)

# Since you aren't using a PQC or ControlledPQC you must append
# your model circuit onto the datapoint circuit tensor manually.
full_circuit = tfq.layers.AddCircuit()(circuits_input, append=model_circuit)
expectation_output = tfq.layers.Expectation()(full_circuit,
                                              symbol_names=control_params,
                                              symbol_values=dense_2,
                                              operators=operators_input)

# Contruct your Keras model.
two_axis_control_model = tf.keras.Model(
    inputs=[circuits_input, commands_input, operators_input],
    outputs=[expectation_output])
```

### 3.2 The dataset

Now you will also include the operators you wish to measure for each datapoint you supply for `model_circuit`:

```python
# The operators to measure, for each command.
operator_data = tfq.convert_to_tensor([[cirq.X(qubit)], [cirq.Z(qubit)]])

# The command input values to the classical NN.
commands = np.array([[0], [1]], dtype=np.float32)

# The desired expectation value at output of quantum circuit.
expected_outputs = np.array([[1], [-1]], dtype=np.float32)
```

### 3.3 Training

Now that you have your new inputs and outputs you can train once again using keras.

```python
optimizer = tf.keras.optimizers.Adam(learning_rate=0.05)
loss = tf.keras.losses.MeanSquaredError()

two_axis_control_model.compile(optimizer=optimizer, loss=loss)

history = two_axis_control_model.fit(
    x=[datapoint_circuits, commands, operator_data],
    y=expected_outputs,
    epochs=30,
    verbose=1)
```

```python
plt.plot(history.history['loss'])
plt.title("Learning to Control a Qubit")
plt.xlabel("Iterations")
plt.ylabel("Error in Control")
plt.show()
```

The loss function has dropped to zero.

The `controller` is available as a stand-alone model. Call the controller, and check its response to each command signal. It would take some work to correctly compare these outputs to the contents of `random_rotations`.

```python
controller.predict(np.array([0,1]))
```

Success: See if you can adapt the `check_error` function from your first model to work with this new model architecture.


```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import pytest

import torch
import torch.distributions as dist
import torch.nn as nn
import torch.nn.functional as F

from chmp.ds import Loop, define, mpl_set
from ipytest import magics
from tqdm import tqdm
```

```python
__file__ = 'TimeSeries.ipynb'
floatx = torch.float32
```

# Pytorch API Tests

```python
%%run_pytest[clean]

def test_distributions_sample_api():
    p = dist.Normal(torch.zeros(3, dtype=floatx), torch.ones(3, dtype=floatx))

    assert p.sample().shape == (3,)
    assert p.sample((2,)).shape == (2, 3)
    assert p.sample((1, 2)).shape == (1, 2, 3)

    with pytest.raises(TypeError):
        p.sample(2)

    # can only use tuple arguments
    with pytest.raises(TypeError):
        assert p.sample([2]).shape == (2, 3)

def test_module_multiple_inputs():
    class CustomModule(nn.Module):
        def forward(self, a, b):
            return (a * b).sum()

    mod = CustomModule()
    a = torch.tensor([1.0, 2.0], requires_grad=True)
    b = torch.tensor([3.0, 4.0], requires_grad=True)

    c = mod(a, b)
    c.backward()

    assert a.grad.detach().numpy() == pytest.approx([3.0, 4.0])
    assert b.grad.detach().numpy() == pytest.approx([1.0, 2.0])
```

    ================================================================================ test session starts ================================================================================
    platform darwin -- Python 3.6.4, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
    rootdir: /Volumes/Home/Code/misc-exp, inifile: pytest.ini
    collected 2 items

    TimeSeries.py ..                                                                                [100%]

    ============================================================================= 2 passed in 0.07 seconds ==============================================================================

```python
import bisect
```

```python
class PiecewiseSchedule:
    def __init__(self, schedule):
        if schedule  and schedule[0][0] is None:
            _, self.left = schedule[0]
            schedule = schedule[1:]

        else:
            _, self.left = schedule[0]

        if schedule and schedule[-1][0] is None:
            _, self.right = schedule[-1]
            schedule = schedule[:-1]

        else:
            _, self.right = schedule[-1]

        self.x = [x for x, _ in schedule]
        self.y = [y for _, y in schedule]

    def __call__(self, x):
        if x < self.x[0]:
            return self.left

        elif x >= self.x[-1]:
            return self.right

        else:
            interval = bisect.bisect_left(self.x[1:-1], x)
            u = (x - self.x[interval]) / (self.x[interval + 1] - self.x[interval])
            return self.interpolate(u, self.y[interval], self.y[interval + 1])

        raise RuntimeError()

    def interpolate(self, u, x0, x1):
        raise NotImplementedError()

class PiecewiseLinearSchedule(PiecewiseSchedule):
    @staticmethod
    def interpolate(u, x0, x1):
        return (1 - u) * x0 + u * x1

class PiecewiseLogSchedule(PiecewiseSchedule):
    @staticmethod
    def interpolate(u, x0, x1):
        return (x0 ** (1 - u)) * (x1 ** u)
```

```python
%%run_pytest[clean] -l

def test_piecewise_linear_schedule():
    schedule = PiecewiseLinearSchedule([
        (None, 20),
        (10, 30),
        (20, 40),
        (None, 50),
    ])

    assert schedule(5.0) == pytest.approx(20.0)
    assert schedule(10.0) == pytest.approx(30.0)
    assert schedule(12.5) == pytest.approx(32.5)
    assert schedule(15.0) == pytest.approx(35.0)
    assert schedule(17.5) == pytest.approx(37.5)
    assert schedule(20.0) == pytest.approx(50.0)
    assert schedule(25.0) == pytest.approx(50.0)

def test_piecewise_linear_schedule_no_boundaries():
    schedule = PiecewiseLinearSchedule([
        (10, 30),
        (20, 40),
    ])

    assert schedule(5) == pytest.approx(30)
    assert schedule(10) == pytest.approx(30)
    assert schedule(12.5) == pytest.approx(32.5)
    assert schedule(15.0) == pytest.approx(35.0)
    assert schedule(17.5) == pytest.approx(37.5)
    assert schedule(20) == pytest.approx(40)
    assert schedule(25) == pytest.approx(40)

```

    ================================================================================ test session starts ================================================================================
    platform darwin -- Python 3.6.4, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
    rootdir: /Volumes/Home/Code/misc-exp, inifile: pytest.ini
    collected 2 items

    TimeSeries.py ..                                                                                [100%]

    ============================================================================= 2 passed in 0.07 seconds ==============================================================================

# Support Code

```python
def safe_scale(x):
    return 1e-2 + F.softplus(x)

class SafeScale(nn.Module):
    def forward(x):
        return safe_scale(x)

class Cat(nn.Module):
    def __init__(self, dim, nargs=None):
        super().__init__()
        self.dim = dim
        self.nargs = nargs

    def forward(self, *args):
        if self.nargs is None:
            self.nargs = len(args)

        assert self.nargs == len(nargs)
        return torch.cat(list(args), dim)

class Parallel(nn.Module):
    def __init__(self, *modules):
        super().__init__()
        self.modules = modules

    def forward(self, x):
        return tuple(mod(x) for x in self.modules)

class MultiLinear(nn.Module):
    def __init__(self, in_features, out_features):
        super().__init__()

        self.out_features = out_features
        if not isinstance(in_features, (tuple, list)):
            self.in_features = (in_features,)

        else:
            self.in_features = tuple(in_features)

        self.layer = nn.Linear(sum(self.in_features), self.out_features)

    @property
    def bias(self):
        return self.layer.bias

    @property
    def weight(self):
        return self.layer.weight

    def forward(self, *x):
        # allow MultiLinear to be used in a Sequential Container
        if len(x) == 1 and len(self.in_features) != 1:
            assert isinstance(x[0], tuple)
            x = x[0]

        assert len(x) == len(self.in_features)
        x = torch.cat(list(x), dim=-1)
        return self.layer(x)

class LocationScaleNetwork(nn.Module):
    def __init__(self, n_input, n_hidden, n_output):
        super().__init__()

        self.to_hidden = MultiLinear(n_input, n_hidden)
        self.to_loc = nn.Linear(n_hidden, n_output)
        self.to_scale = nn.Linear(n_hidden, n_output)

        nn.init.kaiming_normal_(self.to_hidden.layer.weight)
        nn.init.constant_(self.to_hidden.layer.bias, 0.0)

        nn.init.xavier_uniform_(self.to_loc.weight)
        nn.init.constant_(self.to_loc.bias, 0.0)

        nn.init.kaiming_normal_(self.to_scale.weight)
        nn.init.constant_(self.to_scale.bias, 0.0)

    def forward(self, *x):
        x = F.elu(self.to_hidden(*x))
        return self.to_loc(x), safe_scale(self.to_scale(x))
```

# "Bouncing Ball" Example

```python
# Create default graph
x = np.zeros((25, 10, 5))

idx = np.arange(10)
pos = np.random.randint(0, 5, size=10)
vel = np.ones(10, dtype=np.int64)
for t in range(0, x.shape[0]):
    x[t, idx, pos] = 1
    pos += vel

    sel = np.logical_or((pos <= 0), (pos >= x.shape[-1] - 1))
    vel[sel] = -vel[sel]
    pos = np.clip(pos, 0, x.shape[-1] - 1)

plt.imshow(x[:, 0, :].T, interpolation='none', origin='lower', aspect='auto')
```

```python
x = torch.from_numpy(x.astype('float32'))
```

## Variational Autoencoder

Generative model:

$$
\begin{align}
    z &\sim \mathcal{N}(0, 1) \\
    x &\sim \mathcal{N}(\mu_p(x), \sigma_p(x))
\end{align}
$$

Variational model:

$$
    z \sim \mathcal{N}(\mu_q(x), \sigma_q(x))
$$

```python
def multi_append(lists, *items):
    for l, i in zip(lists, items):
        l.append(i)
```

```python
class IndependentEncoder(nn.Module):
    def __init__(self, dim_x, n_hidden, dim_z):
        super().__init__()
        self.qz_network = LocationScaleNetwork(dim_x, n_hidden, dim_z)

    def __call__(self, x):
        qz_loc, qz_scale = self.qz_network(x)
        qz = dist.Normal(qz_loc, qz_scale)
        return qz, qz.rsample()

        #res = [], [], []
        #
        #for t in range(len(x)):
        #    qz_loc, qz_scale = self.qz_network(x[t])
        #    eps = torch.normal(torch.zeros_like(qz_loc), torch.ones_like(qz_scale))
        #
        #    multi_append(res, qz_loc, qz_scale, qz_loc + eps * qz_scale)
        #
        #return (
        #    dist.Normal(torch.stack(res[0]), torch.stack(res[1])),
        #    torch.stack(res[2]),
        #)

class IndependentDecoder(nn.Module):
    def __init__(self, dim_x, n_hidden, dim_z):
        super().__init__()
        self.px_network = LocationScaleNetwork(dim_z, n_hidden, dim_x)

    def __call__(self, z):
        px_loc, px_scale = self.px_network(z)
        return dist.Normal(px_loc, px_scale)

class IndependentDecoderLoop(nn.Module):
    def __init__(self, dim_x, n_hidden, dim_z):
        super().__init__()
        self.px_network = LocationScaleNetwork(dim_z, n_hidden, dim_x)

    def __call__(self, z):
        res = [], []

        for t in range(len(z)):
            px_loc, px_scale = self.px_network(z[t])
            multi_append(res, px_loc, px_scale)

        return dist.Normal(torch.stack(res[0]), torch.stack(res[1]))

class StandardNormalPrior(nn.Module):
    def __call__(self, x, z):
        return dist.Normal(0, 1)

class DeepKalmanFilter(nn.Module):
    def __init__(self, *, encoder, decoder, prior):
        super().__init__()
        self.encoder = encoder
        self.decoder = decoder
        self.prior = prior

    def compute_loss(self, x, regularization=1.0):
        qz, z = self.encoder(x)
        px = self.decoder(z)
        pz = self.prior(x, z)

        elbo = (
            px.log_prob(x).sum()
            - regularization * dist.kl.kl_divergence(qz, pz).sum()
        )

        return -elbo / (x.shape[0] * x.shape[1])

    def compute_reconstruction_error(self, x):
        _, z = self.encoder(x)
        x_mean = self.decoder(z).mean
        return float(((x - x_mean) ** 2.0).mean())
```

```python
regularization_schedule = PiecewiseLinearSchedule([
    (0, 1e-2),
    (15_000, 1e-2),
    (20_000, 1e-0),
])

n_hidden = 30
dim_x = 5
dim_z = 2

results = []
for loop, _ in Loop.over(range(10)):
    model = DeepKalmanFilter(
        encoder=IndependentEncoder(dim_x, n_hidden, dim_z),
        decoder=IndependentDecoder(dim_x, n_hidden, dim_z),
        prior=StandardNormalPrior(),
    )
    model.train(True)
    optimizer = torch.optim.Adam(model.parameters(), lr=1e-2, amsgrad=True)
    losses = []

    for idx in loop.nest(range(40_000)):
        optimizer.zero_grad()
        loss = model.compute_loss(
            x, regularization=regularization_schedule(idx),
        )
        loss.backward()
        optimizer.step()

        if (idx % 50) == 0:
            reconst_error = model.compute_reconstruction_error(x)
            losses += [(float(loss), reconst_error)]
            loop.print(f"{loop} {idx}: {loss:.3f} {reconst_error:.3f}")

    results += [np.asarray(losses)]

results = np.asarray(results)
losses = np.asarray(losses)
```

    [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿ 21m 54s / 21m 55s] 39950: -10.698 0.000

```python
_, (ax1, ax2) = plt.subplots(1, 2, figsize=(8, 4))

plt.sca(ax1)
plt.plot(results[..., 0].T)
plt.ylim(-20, +15)

plt.sca(ax2)
plt.plot(results[..., 1].T)
plt.yscale('log')
pass
```

```python
_, z = model.encoder(x)
res = model.decoder(z).mean.detach().numpy()

_, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(16, 4))

plt.sca(ax1)
plt.plot(losses[..., 0])
mpl_set(title='Loss', xlabel='Iteration', ylabel='Loss')

plt.sca(ax2)
plt.pcolormesh(np.asarray(x[:, 0, :]).T)
mpl_set(colorbar=True, title='Input', xlabel='Time t', ylabel='Position x')

plt.sca(ax3)
plt.pcolormesh(res[:, 0, :].T)
mpl_set(colorbar=True, title='Reconstruction', xlabel='Time t', ylabel='Position x')
```

```python
x_ = x.detach().numpy()
z_ = np.asarray([i.detach().numpy() for i in z])
```

```python
for i, c in enumerate('rgbyk'):
    plt.plot(
        z_[..., 0][x_[..., i] == 1].flatten(),
        z_[..., 1][x_[..., i] == 1].flatten(),
        c + '.'
    )
```

## Deep Kalman Filter

Modeled after [Deep Kalman Filters (2015)](https://arxiv.org/abs/1511.05121).

$$
\begin{align}
    z_1 &\sim \mathcal{N}(\mu_{pz,0}, \sigma_{pz,0}) \\
    z_t &\sim \mathcal{N}(\mu_{pz}(z_{t- 1}), \sigma_{pz}(z_{t - 1})) \\
    x_t &\sim \mathcal{N}(\mu_{px}(z_t)), \sigma_{px}(z_t))
\end{align}
$$

Posterior factorization:
$$
    p(z_t|\{x\}) = p(z_t | z_{t - 1}, x_{t:T})
$$

Variational approximation:
$$
\begin{align}
    z_1 &\sim \mathcal{N}(\mu_{qz,0}(x_{t:T}), \sigma_{qz,0}( x_{1:T})) \\
    z_t &\sim \mathcal{N}(\mu_{qz}(z_{t - 1}, x_{t:T}), \sigma_{qz}(z_{t - 1}, x_{t:T}))
\end{align}
$$

Use an RNN to capture the dependence on $x_{t:T}$.

```python
class RecurrentPrior(nn.Module):
    def __init__(self, dim_z, n_hidden):
        super().__init__()
        self.initial_loc = nn.Parameter(
            torch.zeros(dim_z, dtype=torch.float32, requires_grad=True)
        )
        self.initial_scale_raw = nn.Parameter(
            torch.zeros(dim_z, dtype=torch.float32, requires_grad=True)
        )

        self.pz_network = LocationScaleNetwork(dim_z, n_hidden, dim_z)

    def __call__(self, x, z):
        batch_size = x.shape[1]
        pz0_loc = self.initial_loc
        pz0_scale = safe_scale(self.initial_scale_raw)

        pz_loc, pz_scale = self.pz_network(z[:-1, ...])

        pz_loc = torch.cat([
            pz0_loc.expand(1, 1, -1).repeat(1, batch_size, 1),
            pz_loc,
        ], dim=0)
        pz_scale = torch.cat([
            pz0_scale.expand(1, 1, -1).repeat(1, batch_size, 1),
            pz_scale,
        ], dim=0)

        return dist.Normal(pz_loc, pz_scale)

    def simulate(self, time_steps, batch_size=1):
        z = [
            self.get_initial_dist().sample((batch_size,))
        ]

        for _ in range(time_steps - 1):
            pz_loc, pz_scale = self.pz_network(z[-1])
            pz = dist.Normal(pz_loc, pz_scale)
            z += [pz.sample()]

        return torch.stack(z)

    def get_initial_dist(self):
        pz0_loc = self.initial_loc
        pz0_scale = safe_scale(self.initial_scale_raw)
        return dist.Normal(pz0_loc, pz0_scale)
```

```python
class RecurrentEncoder(nn.Module):
    def __init__(self, dim_x, n_hidden, dim_z):
        super().__init__()
        self.summarize_x = nn.GRU(
            input_size=dim_x,
            hidden_size=n_hidden,
        )

        self.qz0_network = LocationScaleNetwork(n_hidden, n_hidden, dim_z)
        self.qz_network = LocationScaleNetwork((dim_z, n_hidden), n_hidden, dim_z)

    def __call__(self, x):
        backward_x = x[torch.arange(len(x) - 1, -1, -1).long(), ...]
        x_summary_backward, _ = self.summarize_x(backward_x)
        x_summary_backward = F.relu(x_summary_backward)

        res = [], [], []

        qz0_loc, qz0_scale = self.qz0_network(x_summary_backward[-1])
        z0 = dist.Normal(qz0_loc, qz0_scale).rsample()

        multi_append(res, qz0_loc, qz0_scale, z0)

        for t in range(1, len(x)):
            z_arg = res[2][t - 1]
            x_arg = x_summary_backward[len(x) - 1 - t]

            qz_loc, qz_scale = self.qz_network(z_arg, x_arg)
            z = dist.Normal(qz_loc, qz_scale).rsample()

            multi_append(res, qz_loc, qz_scale, z)

        qz_loc = torch.stack(res[0])
        qz_scale = torch.stack(res[1])
        z = torch.stack(res[2])

        return dist.Normal(qz_loc, qz_scale), z
```

```python
regularization_schedule = PiecewiseLinearSchedule([
    (0, 1e-2),
    (15_000, 1e-2),
    (20_000, 1e-0),
])

n_hidden = 30
dim_x = 5
dim_z = 2

results = []
for loop, _ in Loop.over(range(1)):
    model = DeepKalmanFilter(
        #encoder=IndependentEncoder(dim_x, n_hidden, dim_z),
        encoder=RecurrentEncoder(dim_x, n_hidden, dim_z),
        decoder=IndependentDecoder(dim_x, n_hidden, dim_z),
        prior=RecurrentPrior(dim_z, n_hidden),
    )
    model.train(True)
    optimizer = torch.optim.Adam(model.parameters(), lr=1e-2, amsgrad=True)
    losses = []

    for idx in loop.nest(range(500_000)):
        optimizer.zero_grad()
        loss = model.compute_loss(
            x, regularization=regularization_schedule(idx),
        )
        loss.backward()
        optimizer.step()

        if (idx % 100) == 0:
            reconst_error = model.compute_reconstruction_error(x)
            losses += [(float(loss), reconst_error)]
            loop.print(f"{loop} {idx}: {loss:.3f} {reconst_error:.3f}")

    results += [np.asarray(losses)]

results = np.asarray(results)
losses = np.asarray(losses)
```

```python
plt.plot(losses[:, 0])
plt.ylim(-20, 10)
```

```python
sim_z = model.prior.simulate(time_steps=20, batch_size=10)
sim_x = model.decoder(sim_z).sample()
sim_x = sim_x.detach().numpy()
```

```python
plt.pcolormesh(sim_x[:, 9, :].T)
```

```python

```


## Data Analytics

### Bootstrap in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

### Bootstrap in Python

Here's a simple workflow, demonstration of bootstrap for modeling workflows. This should help you get started with this important data analytics method to evaluate and integrate uncertainty in any sample statistics or model.

#### Bootstrap

Uncertainty in the sample statistics
* one source of uncertainty is the paucity of data.
* do 200 or even less wells provide a precise (and accurate estimate) of the mean? standard deviation? skew? P13?

Would it be useful to know the uncertainty in these statistics due to limited sampling?
* what is the impact of uncertainty in the mean porosity e.g. 20%+/-2%?

**Bootstrap** is a method to assess the uncertainty in a sample statistic by repeated random sampling with replacement.

Assumptions
* sufficient, representative sampling, identical, idependent samples

Limitations
1. assumes the samples are representative
2. assumes stationarity
3. only accounts for uncertainty due to too few samples, e.g. no uncertainty due to changes away from data
4. does not account for boundary of area of interest
5. assumes the samples are independent
6. does not account for other local information sources

The Bootstrap Approach (Efron, 1982)

Statistical resampling procedure to calculate uncertainty in a calculated statistic from the data itself.
* Does this work?  Prove it to yourself, for uncertainty in the mean solution is standard error:

\begin{equation}
\sigma^2_\overline{x} = \frac{\sigma^2_s}{n}
\end{equation}

Extremely powerful - could calculate uncertainty in any statistic!  e.g. P13, skew etc.
* Would not be possible access general uncertainty in any statistic without bootstrap.
* Advanced forms account for spatial information and sampling strategy (game theory and Journel’s spatial bootstrap (1993).

Steps:

1. assemble a sample set, must be representative, reasonable to assume independence between samples

2. optional: build a cumulative distribution function (CDF)
    * may account for declustering weights, tail extrapolation
    * could use analogous data to support

3. For $\ell = 1, \ldots, L$ realizations, do the following:

    * For $i = \alpha, \ldots, n$ data, do the following:

        * Draw a random sample with replacement from the sample set or Monte Carlo simulate from the CDF (if available).

6. Calculate a realization of the sammary statistic of interest from the $n$ samples, e.g. $m^\ell$, $\sigma^2_{\ell}$. Return to 3 for another realization.

7. Compile and summarize the $L$ realizations of the statistic of interest.

This is a very powerful method.  Let's try it out.

#### Objective

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - sample_data_biased.csv at https://git.io/fh0CW

We need some standard packages. These should have been installed with Anaconda 3.

```python
import numpy as np                        # ndarrys for gridded data
import pandas as pd                       # DataFrames for tabular data
import os                                 # set working directory, run executables
import matplotlib.pyplot as plt           # for plotting
import matplotlib
from scipy import stats                   # summary statistics
import math                               # trig etc.
import scipy.signal as signal             # kernel for moving window calculation
import random                             # random sampling
from scipy.stats import gaussian_kde      # for PDF calculation
from scipy.stats import t                 # Student's t distribution for analytical solution
```

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
os.chdir("c:/PGE383")                     # set the working directory
```

#### Loading Tabular Data

Here's the command to load our comma delimited data file in to a Pandas' DataFrame object.

```python
df = pd.read_csv('sample_data_biased.csv') # load our data table
```

Let's drop some samples so that we increase the variations in bootstrap samples for our demonstration below.

```python
df = df.sample(frac = 0.2)                 # extract 50 random samples to reduce the size of the dataset
print('Using ' + str(len(df)) + ' number of samples')
```

    Using 58 number of samples

Visualizing the DataFrame would be useful and we already learned about these methods in this demo (https://git.io/fNgRW).

We can preview the DataFrame by printing a slice or by utilizing the 'head' DataFrame member function (with a nice and clean format, see below). With the slice we could look at any subset of the data table and with the head command, add parameter 'n=13' to see the first 13 rows of the dataset.

```python
df.head()                                  # display first 4 samples in the table as a preview
```

      X
      Y
      Facies
      Porosity
      Perm

      245
      890
      489
      0
      0.103176
      2.846036

      153
      360
      279
      1
      0.134607
      9.776276

      193
      950
      469
      0
      0.081717
      0.681267

      164
      110
      689
      1
      0.102856
      4.723767

      278
      140
      279
      0
      0.086697
      1.240637

#### Summary Statistics for Tabular Data

The table includes X and Y coordinates (meters), Facies 1 and 0 (1 is sandstone and 0 interbedded sand and mudstone), Porosity (fraction), and permeability as Perm (mDarcy).

There are a lot of efficient methods to calculate summary statistics from tabular data in DataFrames. The describe command provides count, mean, minimum, maximum, and quartiles all in a nice data table. We use transpose just to flip the table so that features are on the rows and the statistics are on the columns.

```python
df.describe().transpose()
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      X
      58.0
      455.172414
      268.819651
      0.000000
      255.000000
      415.000000
      635.000000
      990.000000

      Y
      58.0
      539.689655
      277.417679
      59.000000
      284.000000
      554.000000
      769.000000
      989.000000

      Facies
      58.0
      0.706897
      0.459161
      0.000000
      0.000000
      1.000000
      1.000000
      1.000000

      Porosity
      58.0
      0.130408
      0.038279
      0.058548
      0.104409
      0.123246
      0.150355
      0.220049

      Perm
      58.0
      146.779192
      369.297770
      0.075819
      2.907922
      14.472930
      50.165149
      1906.662897

#### Visualizing Tabular Data with Location Maps

It is natural to set the x and y coordinate and feature ranges manually. e.g. do you want your color bar to go from 0.05887 to 0.24230 exactly? Also, let's pick a color map for display. I heard that plasma is known to be friendly to the color blind as the color and intensity vary together (hope I got that right, it was an interesting Twitter conversation started by Matt Hall from Agile if I recall correctly). We will assume a study area of 0 to 1,000m in x and y and omit any data outside this area.

```python
xmin = 0.0; xmax = 1000.0               # range of x values
ymin = 0.0; ymax = 1000.0               # range of y values
pormin = 0.05; pormax = 0.25;           # range of porosity values
permmin = 0.01; permmax = 10000
nx = 100; ny = 100; csize = 10.0
cmap = plt.cm.plasma                    # color map
cumul_prob = np.linspace(0.0,1.0,100)   # list of cumulative probabilities
```

Let's try out location maps, histograms and scatter plots.

```python
plt.subplot(231)
plt.scatter(df['X'],df['Y'],s = 20,c = df['Porosity'],cmap = plt.cm.inferno,linewidths = 0.3,edgecolor = 'black',alpha = 0.8,vmin = pormin,vmax = pormax)
plt.colorbar(); plt.xlabel('X (m)'); plt.ylabel('Y (m)'); plt.title('Porosity Location Map')

plt.subplot(234)
plt.hist(df['Porosity'],color = 'red',alpha = 0.3,edgecolor='black',bins = np.linspace(pormin,pormax,int(len(df)/3)))
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Porosity Histogram')

plt.subplot(232)
plt.scatter(df['X'],df['Y'],s = 20,c = df['Perm'],cmap = plt.cm.inferno,linewidths = 0.3,edgecolor = 'black',alpha = 0.8,vmin = permmin,vmax = permmax,norm=matplotlib.colors.LogNorm())
plt.colorbar(); plt.xlabel('X (m)'); plt.ylabel('Y (m)'); plt.title('Permeability Location Map')

plt.subplot(235)
plt.hist(df['Perm'],color = 'red',alpha = 0.3,edgecolor='black',bins=np.logspace(np.log10(permmin),np.log10(permmax),int(len(df)/3)))
#sns.kdeplot(x=df['Perm'],color = 'black',alpha = 0.2,levels = 1,log_scale = True,bw_adjust = 1)
plt.xlabel('Permeability (mD)'); plt.ylabel('Frequency'); plt.title('Permeability Histogram'); plt.xscale('log')

plt.subplot(233)
plt.scatter(df['Porosity'],df['Perm'],s = 20,color = 'red',alpha = 0.3,edgecolor='black')
#plt.contour(df['Porosity'],df['Perm'] Z, colors='black');
plt.ylabel('Permeability (mD)'); plt.xlabel('Porosity (fraction)'); plt.title('Permeability-Porosity Scatter Plot')
plt.yscale('log')
sns.kdeplot(x=df['Porosity'],y=df['Perm'],color = 'black',alpha = 0.2,levels = 4)

plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=2.2, wspace=0.2, hspace=0.2)
plt.show()
```

### Bootstrap Method in Python

If you are new to bootstrap and Python, here's the most simple code possible for bootstrap.

* specify the number of bootstrap realizations, $L$
* declare a list to store the bootstrap realizations of the statistic of interest
* loop over L bootstrap realizations
    * n MCS, random samples with replacement for a new realization of the data
    * calculate the realization of the statistic from the realization of the data
* summarize the resulting uncertainty model, histogram, summary statistics etc.

#### Bootstrap of the Sample Mean, Arithmetic Average

Let's demonstrate bootstrap for uncertainty in the arithmetic average with a simple workflow.

```python
por_avg_real = []                      # declare an empty list to store the bootstrap realizations of the statistic
for k in range(0,L):                   # loop over the L bootstrap realizations
    samples = random.choices(df['Porosity'].values, k=len(df)) # n Monte Carlo simulations
    por_avg_real.append(np.average(samples)) # calculate the statistic of interest from the new bootstrap dataset
```

We will compare the bootstrap results with the analytical expression:

\begin{equation}
\overline{x} \pm t_{\frac{\alpha}{2},n-1}\sqrt{\frac{s^2}{n}}
\end{equation}

The remaining code is just a super cool set of plots with the results.

```python
L = 1000                               # set the number of bootstrap realizations
por_avg_real = []                      # declare an empty list to store the bootstrap realizations of the statistic
for k in range(0,L):                   # loop over the L bootstrap realizations
    samples = random.choices(df['Porosity'].values, k=len(df)) # n Monte Carlo simulations
    por_avg_real.append(np.average(samples)) # calculate the statistic of interest from the new bootstrap dataset

fig = plt.subplot(131)
plt.hist(df['Porosity'],color = 'red',alpha = 0.3,edgecolor='black',bins = np.linspace(pormin,pormax,int(len(df)/3)))
#plt.plot([np.average(df['Porosity']),np.average(df['Porosity'])],[0,100])
plt.axvline(x=np.average(df['Porosity']),linestyle="--",c='black')
plt.text(np.average(df['Porosity'])+0.005, 8.8, r'Average = ' + str(round(np.average(df['Porosity']),3)), fontsize=12)
plt.text(np.average(df['Porosity'])+0.005, 8.3, r'St.Dev. = ' + str(round(np.std(df['Porosity']),3)), fontsize=12)
plt.text(np.average(df['Porosity'])+0.005, 7.8, r'P10 = ' + str(round(np.percentile(df['Porosity'],10),3)), fontsize=12)
plt.text(np.average(df['Porosity'])+0.005, 7.3, r'P90 = ' + str(round(np.percentile(df['Porosity'],90),3)), fontsize=12)
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Original Porosity Histogram')

plt.subplot(132)
plt.hist(por_avg_real,color = 'red',alpha = 0.2,edgecolor = 'black',bins=np.linspace(pormin,pormax,100),label = 'bootstrap',density = True) # plot the distribution, could also calculate any summary statistics
analytical = t.pdf(np.linspace(pormin,pormax,100), len(df)-1,loc=np.average(df['Porosity']),scale=math.sqrt(np.var(df['Porosity'].values)/len(df)))
plt.plot(np.linspace(pormin,pormax,100),analytical,color = 'black',label = 'analytical',alpha=0.4)
plt.axvline(x=np.average(por_avg_real),linestyle="--",c='black')
plt.fill_between(np.linspace(pormin,pormax,100), 0, analytical, where = np.linspace(pormin,pormax,100) <= np.percentile(por_avg_real,10), facecolor='red', interpolate=True, alpha = 0.5)
plt.fill_between(np.linspace(pormin,pormax,100), 0, analytical, where = np.linspace(pormin,pormax,100) >= np.percentile(por_avg_real,90), facecolor='red', interpolate=True, alpha = 0.5)
plt.text(np.average(por_avg_real)+0.009, L*0.07, r'Average = ' + str(round(np.average(por_avg_real),3)), fontsize=12)
plt.text(np.average(por_avg_real)+0.009, L*0.066, r'St.Dev. = ' + str(round(np.std(por_avg_real),3)), fontsize=12)
plt.text(np.average(por_avg_real)+0.009, L*0.062, r'P90 = ' + str(round(np.percentile(por_avg_real,90),3)), fontsize=12)
plt.text(np.average(por_avg_real)+0.009, L*0.058, r'P10 = ' + str(round(np.percentile(por_avg_real,10),3)), fontsize=12)
plt.xlabel('Boostrap Realizations and Analytical Sampling Distributions'); plt.ylabel('Frequency'); plt.title('Distribution of Average')
plt.legend()

fig = plt.subplot(133)
plt.hist(df['Porosity'],color = 'red',alpha = 0.2,edgecolor='grey',bins = np.linspace(pormin,pormax,int(len(df)/3)))
#plt.plot([np.average(df['Porosity']),np.average(df['Porosity'])],[0,100])
plt.axvline(x=np.average(df['Porosity']),c='black')
plt.axvline(x=np.percentile(por_avg_real,90),linestyle="--",c='black')
plt.axvline(x=np.percentile(por_avg_real,10),linestyle="--",c='black')
plt.text(np.percentile(por_avg_real,90)+0.009,8.8, r'Average = ' + str(round(np.average(por_avg_real),3)), fontsize=12)
plt.text(np.percentile(por_avg_real,90)+0.009,8.3, r'Average P90 = ' + str(round(np.percentile(por_avg_real,90),3)), fontsize=12)
plt.text(np.percentile(por_avg_real,90)+0.009,7.8, r'Average P10 = ' + str(round(np.percentile(por_avg_real,10),3)), fontsize=12)
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Porosity Histogram with Uncertainty in Average')

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

#### Bootstrap of the Proportion

Let's demonstrate another case for which we have the analytical solution.

We will compare the bootstrap results with the analytical expression:

\begin{equation}
\hat{p} \pm t_{\frac{\alpha}{2},n-1}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}
\end{equation}

```python
L = 100000                               # set the number of bootstrap realizations
sand_prop_real = []                      # declare an empty list to store the bootstrap realizations of the statistic
shale_prop_real = []
for k in range(0,L):                   # loop over the L bootstrap realizations
    samples = random.choices(df['Facies'].values, k=len(df)) # n Monte Carlo simulations
    sand_prop_real.append(samples.count(1)/len(df)) # calculate the statistic of interest from the new bootstrap dataset
    shale_prop_real.append(samples.count(0)/len(df)) # calculate the statistic of interest from the new bootstrap dataset
sand_prop = np.sum(df['Facies'] == 1)/len(df); shale_prop = np.sum(df['Facies'] == 0)/len(df)

fig = plt.subplot(121)
barlist = plt.bar(x=['shale','sand'],height = [1-sand_prop,sand_prop],color = 'red',alpha = 0.3,edgecolor='black')
barlist[0].set_color('blue')
plt.text(-.3, shale_prop+0.02, r'Prop. Shale = ' + str(round(shale_prop,2)), fontsize=12)
plt.text(0.7, sand_prop+0.02, r'Prop. Sand = ' + str(round(sand_prop,2)), fontsize=12)
plt.xlabel('Rock Type / Facies'); plt.ylabel('Frequency'); plt.title('Facies Histogram')
plt.ylim([0,1]); plt.yticks(np.arange(0, 1.1, 0.1))

plt.subplot(122)
plt.hist(sand_prop_real,color = 'red',alpha = 0.2,edgecolor = 'black',bins=np.linspace(0.0,1.0,40),label = 'sand',density = True) # plot the distribution, could also calculate any summary statistics
analytical = t.pdf(np.linspace(0.0,1.0,100), len(df)-1,loc=sand_prop,scale=math.sqrt(sand_prop*(1.0-sand_prop)/len(df)))
plt.plot(np.linspace(0.0,1.0,100),analytical,color = 'black',label = 'analytical',alpha=0.4)
plt.axvline(x=sand_prop,linestyle="--",c='black')
plt.fill_between(np.linspace(0.0,1.0,100), 0, analytical, where = np.linspace(0.0,1.0,100) <= np.percentile(sand_prop_real,10), facecolor='red', interpolate=True, alpha = 0.5)
plt.fill_between(np.linspace(0.0,1.0,100), 0, analytical, where = np.linspace(0.0,1.0,100) >= np.percentile(sand_prop_real,90), facecolor='red', interpolate=True, alpha = 0.5)
plt.text(np.average(sand_prop_real)+0.009, 7.0, r'Prop. = ' + str(round(sand_prop,2)), fontsize=12)
plt.text(np.average(sand_prop_real)+0.009, 6.6, r'P90 = ' + str(round(np.percentile(sand_prop,90),2)), fontsize=12)
plt.text(np.average(sand_prop_real)+0.009, 6.2, r'P10 = ' + str(round(np.percentile(sand_prop,10),2)), fontsize=12)

plt.hist(shale_prop_real,color = 'blue',alpha = 0.2,edgecolor = 'black',bins=np.linspace(0.0,1.0,40),label = 'shale',density = True) # plot the distribution, could also calculate any summary statistics
analytical_shale = t.pdf(np.linspace(0.0,1.0,100), len(df)-1,loc=shale_prop,scale=math.sqrt(shale_prop*(1.0-shale_prop)/len(df)))
plt.plot(np.linspace(0.0,1.0,100),analytical_shale,color = 'black',alpha=0.4)
plt.axvline(x=shale_prop,linestyle="--",c='black')
plt.fill_between(np.linspace(0.0,1.0,100), 0, analytical_shale, where = np.linspace(0.0,1.0,100) <= np.percentile(shale_prop_real,10), facecolor='blue', interpolate=True, alpha = 0.5)
plt.fill_between(np.linspace(0.0,1.0,100), 0, analytical_shale, where = np.linspace(0.0,1.0,100) >= np.percentile(shale_prop_real,90), facecolor='blue', interpolate=True, alpha = 0.5)
plt.text(np.average(shale_prop_real)+0.009, 7.0, r'Prop. = ' + str(round(shale_prop,2)), fontsize=12)
plt.text(np.average(shale_prop_real)+0.009, 6.6, r'P90 = ' + str(round(np.percentile(shale_prop_real,90),2)), fontsize=12)
plt.text(np.average(shale_prop_real)+0.009, 6.2, r'P10 = ' + str(round(np.percentile(shale_prop_real,10),2)), fontsize=12)

plt.xlabel('Boostrap Realizations and Analytical Sampling Distributions'); plt.ylabel('Frequency'); plt.title('Distribution of Bootstrap Proportions')
plt.legend(loc = 'upper left')

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

#### Bootstrap of the Interquartile Range

To prove that we can bootstrap any statistic let's select a more complicated measure of dispersion, the interquartile range.

```python
L = 1000                               # set the number of bootstrap realizations
iqr_real = []                      # declare an empty list to store the bootstrap realizations of the statistic
for k in range(0,L):                   # loop over the L bootstrap realizations
    samples = random.choices(df['Porosity'].values, k=len(df)) # n Monte Carlo simulations
    iqr_real.append(np.percentile(samples,75) - np.percentile(samples,25)) # calculate the statistic of interest from the new bootstrap dataset

iqr = np.percentile(df['Porosity'],75) - np.percentile(df['Porosity'],25)

plt.subplot(111)
plt.hist(iqr_real,color = 'red',alpha = 0.2,edgecolor = 'black',bins=np.linspace(0.0,0.125,100),label = 'bootstrap',density = True) # plot the distribution, could also calculate any summary statistics
plt.axvline(x=iqr,c='black')
plt.axvline(x=np.percentile(iqr_real,10),linestyle="--",c='black')
plt.axvline(x=np.percentile(iqr_real,90),linestyle="--",c='black')
sns.kdeplot(x=iqr_real,color = 'grey',alpha = 0.1,levels = 1,bw_adjust = 1)
plt.text(np.percentile(iqr_real,90)+0.009, L*0.07, r'Average = ' + str(round(np.average(iqr_real),3)), fontsize=12)
plt.text(np.percentile(iqr_real,90)+0.009, L*0.066, r'St.Dev. = ' + str(round(np.std(iqr_real),3)), fontsize=12)
plt.text(np.percentile(iqr_real,90)+0.009, L*0.062, r'P90 = ' + str(round(np.percentile(iqr_real,90),3)), fontsize=12)
plt.text(np.percentile(iqr_real,90)+0.009, L*0.058, r'P10 = ' + str(round(np.percentile(iqr_real,10),3)), fontsize=12)
plt.xlabel('Boostrap Realizations, Sampling Distributions'); plt.ylabel('Frequency'); plt.title('Distribution of Inter Quartile Range')
# plt.legend()

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

#### Bootstrap of the Coefficient of Variation

Here's another statistic that requires multiple measures from the each bootstrap realization of the data.

* this reinforces that we bootstrap dataset realizations and then calculate the statistic on this dataset realization

For the coefficient of variation we will:

* calculate a bootstrap realization of the dataset with $n$ samples with replacement
* calculate the mean and standard deviation from this bootstrapped realization of the dataset
* calculate a boostrap realization of the coefficient of variation as the standard deviation divided by the mean

Repeat this $L$ times and then evaluate the resulting distribution.

```python
L = 1000                               # set the number of bootstrap realizations
cv_real = []                      # declare an empty list to store the bootstrap realizations of the statistic

for k in range(0,L):                   # loop over the L bootstrap realizations
    samples = random.choices(df['Porosity'].values, k=len(df)) # n Monte Carlo simulations
    cv_real.append(np.std(samples)/np.average(samples)) # calculate the statistic of interest from the new bootstrap dataset

cv = np.std(df['Porosity'])/np.average(df['Porosity'])

plt.subplot(111)
plt.hist(cv_real,color = 'red',alpha = 0.2,edgecolor = 'black',bins=np.linspace(0.1,0.4,100),label = 'bootstrap',density = True) # plot the distribution, could also calculate any summary statistics
plt.axvline(x=cv,c='black')
plt.axvline(x=np.percentile(cv_real,10),linestyle="--",c='black')
plt.axvline(x=np.percentile(cv_real,90),linestyle="--",c='black')
sns.kdeplot(x=cv_real,color = 'grey',alpha = 0.1,levels = 1,bw_adjust = 1)
plt.text(np.percentile(cv_real,90)+0.009, 16, r'Average = ' + str(round(np.average(cv_real),3)), fontsize=12)
plt.text(np.percentile(cv_real,90)+0.009, 15, r'St.Dev. = ' + str(round(np.std(cv_real),3)), fontsize=12)
plt.text(np.percentile(cv_real,90)+0.009, 14, r'P90 = ' + str(round(np.percentile(cv_real,90),3)), fontsize=12)
plt.text(np.percentile(cv_real,90)+0.009, 13, r'P10 = ' + str(round(np.percentile(cv_real,10),3)), fontsize=12)
plt.xlabel('Boostrap Realizations, Sampling Distributions'); plt.ylabel('Frequency'); plt.title('Distribution of Coefficient of Variation')
# plt.legend()

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

#### Comments

This was a basic demonstration of bootstrap.

* you can use bootstrap to calculate uncertainty in any statistic!

* you are calculating a set of realizations of the statistic, representing uncertainty due to small sample size

* note the assumptions of bootstrap, including stationarity and representativity

* remember, get a dataset realization by bootstrap and then calculate the realization of the statistic from the dataset realization

* if your statistic has multiple inputs (e.g. P25 and P75), calculate each from the same bootstrap realization of the dataset.

I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```