```python
# -*- coding: utf-8 -*-
#   This work is part of the Core Imaging Library (CIL) developed by CCPi
#   (Collaborative Computational Project in Tomographic Imaging), with
#   substantial contributions by UKRI-STFC and University of Manchester.

#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at

#   http://www.apache.org/licenses/LICENSE-2.0

#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

#   Copyright 2021 UKRI-STFC
#   Authored by: Evangelos Papoutsellis (UKRI-STFC)
```

 Dynamic Sparse CT

In this demo, we focus on different reconstruction methods for sparse dynamic CT using an open-access dynamic dataset available from [Heikkilä_et_al_Zenodo](https://zenodo.org/record/3696817#.YKTRP5MzZp9). The aim is to demonstrate how to increase
the temporal resolution, or to reduce the radiation dose of a CT scan, without sacrificing
the quality of the reconstructions.

The gel phantom simulates diffusion of liquids inside plant stems, namely the flow of iodine-based contrast agents used in high resolution tomographic X-ray imaging of plants. In order to test different reconstruction methods, this radiation resistant phantom with similar diffusion properties was constructed. For more information, please see [Heikkilä_et_al](https://arxiv.org/abs/2003.02841).

 Learning objectives

- Create a **2D + channels** acquisition geometry for the dynamic tomographic data. Channels correspond to different time steps (frames).

- Create **sparse data** using the `Slicer` processor.

- Run FBP reconstruction for every time-channel.

- Setup PDHG for 2 different regularisers:

    * Spatiotemporal Total Variation: regularisation applied equally along the spatial and temporal dimensions (channels).

    * **Directional Total Variation (dTV)**: regularisation applied for each channel using a reference image reference.

We first import all the necessary libraries for this notebook.

```python
# Import libraries

from cil.framework import AcquisitionGeometry

from cil.optimisation.algorithms import PDHG
from cil.optimisation.operators import GradientOperator, BlockOperator
from cil.optimisation.functions import IndicatorBox, BlockFunction, L2NormSquared, MixedL21Norm

from cil.io import NEXUSDataWriter, NEXUSDataReader

from cil.processors import Slicer

from cil.plugins.astra.operators import ProjectionOperator
from cil.plugins.astra.processors import FBP
from cil.plugins.ccpi_regularisation.functions import FGP_dTV, FGP_TV

from cil.utilities.display import show2D, show_geometry
from cil.utilities.jupyter import islicer

from utilities_dynamic_ct import read_frames, read_extra_frames

from IPython.display import clear_output

import numpy as np

import os
```

# Data information: Gel phantom

The sample is an agarose-gel phantom, perfused with a liquid contrast agent in a 50ml Falcon test tube (ø 29 × 115mm). The aim of this experiment was to simulate diffusion of liquids inside plant stems, which cannot withstand high radiation doses from a denser set of measurement angles. After the agarose solidified, five intact plastic straws were made into the gel and filled with 20% sucrose solution to guarantee the diffusion by directing osmosis to the gel body.

Every measurement consists of 360 projections with 282 detector bins obtained from a flat-panel circular-scan cone-beam microCT-scanner. Only the central slice is provided, resulting in a 2D __fanbeam geometry__. The primary measurements consisted of 17 consecutive time frames, with initial stage of no contrast agent followed by steady increase and diffusion into the gel body over time. Data is given in two different resolutions corresponding to reconstructions of size:

* 256 x 256: **GelPhantomData_b4.mat**
* 512 x 512: **GelPhantomData_b2.mat**

For this notebook, a 256x256 resolution is selected. In addition to the primary measurements, a more densely sampled measurements from the first time step and an additional 18th time step are provided in **GelPhantom_extra_frames.mat**

* Pre-scan: **720 projections**
* Post-scan: **1600 projections**

## Load and read dynamic data (mat files)

```python
path = os.path.abspath("/mnt/materials/SIRF/Fully3D/CIL/GelPhantom")
data_mat = "GelPhantomData_b4"
file_info = read_frames(path, data_mat)
```

From the `file_info` variable, we have all the information in order to define our acquisition geometry and create our CIL acquisition data.

**Note that the pixel size of the detector is wrong. The correct pixel size should be doubled.**

```python
# Get sinograms + metadata
sinograms = file_info['sinograms']
frames = sinograms.shape[0]
angles = file_info['angles']
distanceOriginDetector = file_info['distanceOriginDetector']
distanceSourceOrigin = file_info['distanceSourceOrigin']
# Correct the pixel size
pixelSize = 2*file_info['pixelSize']
numDetectors = file_info['numDetectors']
```

## Exercise 1: Create acquisition and image geometries

For this dataset, we have a 2D cone geometry with 17 time channels. Using the metadata above, we can define the acquisition geometry `ag` with

```python

    ag = AcquisitionGeometry.create_Cone2D(source_position = [0, distanceSourceOrigin],
                                           detector_position = [0, -distanceOriginDetector])\
                                        .set_panel(numDetectors, pixelSize)\
                                        .set_channels(frames)\
                                        .set_angles(angles, angle_unit="radian")\
                                        .set_labels(['channel','angle', 'horizontal'])

```

For the image geometry `ig` we use the following code and crop our image domain to `[256,256]`:

```python

    ig = ag.get_ImageGeometry()

```

```python
# Create acquisition + image geometries
ag = AcquisitionGeometry.create_Cone2D(source_position = ...,
                                       detector_position = ...)\
                                    .set_panel(numDetectors, ...)\
                                    .set_channels(...)\
                                    .set_angles(angles, angle_unit="radian")\
                                    .set_labels(['channel','angle', 'horizontal'])

ig = ag.get_ImageGeometry()
ig.voxel_num_x = 256
ig.voxel_num_y = 256
```

## Exercise 1: Solution

```python
# Create acquisition + image geometries
ag = AcquisitionGeometry.create_Cone2D(source_position = [0, distanceSourceOrigin],
                                       detector_position = [0, -distanceOriginDetector])\
                                    .set_panel(numDetectors, pixelSize)\
                                    .set_channels(frames)\
                                    .set_angles(angles, angle_unit="radian")\
                                    .set_labels(['channel','angle', 'horizontal'])

ig = ag.get_ImageGeometry()
ig.voxel_num_x = 256
ig.voxel_num_y = 256
```

Then, we create an `AcquisitionData` by allocating space from the acquisition geometry `ag`. This is filled with every **sinogram per time channel**.

```python
data = ag.allocate()
for i in range(frames):
   data.fill(sinograms[i], channel = i)
```

## Show acquisition data

```python
islicer(data, direction=0, cmap="inferno", title="Time frame")
```

## Show acquisition geometry

```python
show_geometry(ag)
```

## Create Sparse CT data

In order to simulate a sparse acquisition, from the total of 360 projections we select a number of projections depending on the size of the `step`:

- step = 1 --> 360/1 projections
- step = 5 --> 360/5 = 72 projections
- step = 10 --> 360/10 = 36 projections
- step = 20 --> 360/20 = 18 projections

We create the sparse data using the `Slicer` processor. For every case, we show the dynamic data for 4 different time frames and save them using the `NEXUSDataWriter` processor in the `SparseData` directory.

```python
# Create and save Sparse Data with different angular sampling: 18, 36, 72, 360 projections
for step in [1, 5, 10, 20]:

    name_proj = "data_{}".format(int(360/step))
    new_data = Slicer(roi={'angle':(0,360,step)})(data)
    ag = new_data.geometry

    show2D(new_data, slice_list = [0,5,10,16], num_cols=4, origin="upper",
                       cmap="inferno", title="Projections {}".format(int(360/step)), size=(25, 20))

    writer = NEXUSDataWriter(file_name = "SparseData/"+name_proj+".nxs",
                         data = new_data)
    writer.write()

```

For the rest of the notebook, we use the sparse acquisition data `data_36`, i.e., only **36 projections** and perform the following:

- FBP reconstruction per time frame.
- Spatiotemporal TV reconstruction.
- Directional Total variation.

For the other cases, you can change the value of the `num_proj` below. Available data are: `data_18`, `data_36`, `data_72` and `data_360`.

```python
num_proj = 36
reader = NEXUSDataReader(file_name = "SparseData/data_{}.nxs".format(num_proj) )
data = reader.load_data()
ag = data.geometry
```

## Channelwise FBP

For the **channelwise** FBP reconstruction, we perform the following steps

- Allocate a space using the full image geometry (2D+channels) `ig` geometry.
- Extract the 2D acquisition and image geometries, using `ag.subset(channel=0)` and `ig.subset(channel=0)`.
- Run FBP reconstruction using the 2D sinogram data for every time frame.
- Fill the 2D FBP reconstruction with respect to the `channel=i` using the `fill` method.

```python
fbp_recon = ig.allocate()

ag2D = ag.subset(channel=0)
ig2D = ig.subset(channel=0)

for i in range(ig.channels):

    tmp = FBP(ig2D, ag2D)(data.subset(channel=i))
    fbp_recon.fill(tmp, channel=i)
    print("Finish FBP reconstruction for time frame {}".format(i), end='\r')

show2D(fbp_recon, slice_list = [0,5,10,16], num_cols=4, origin="upper",fix_range=(0,0.065),
                   cmap="inferno", title="Projections {}".format(i), size=(25, 20))

name_fbp = "FBP_projections_{}".format(num_proj)
writer = NEXUSDataWriter(file_name = "FBP_reconstructions/"+name_fbp+".nxs",
                     data = fbp_recon)
writer.write()
```

## Exercise 2: Total Variation reconstruction

For the TV reconstruction, we use the **Explicit formulation** of the PDHG algorithm. See the [PDHG notebook](../Week2/03_PDHG.ipynb) for more information.

- Define the `ProjectionOperator` and the `GradientOperator` using `correlation=SpaceChannels`.

```python

        A = ProjectionOperator(ig, ag, 'gpu')
        Grad = GradientOperator(ig, correlation = "SpaceChannels")

```

- Use the `BlockOperator` to define the operator $K$.

```python

        K = BlockOperator(A, Grad)

```

- Use the `BlockFunction` to define the function $\mathcal{F}$ that contains the fidelity term `L2NormSquared(b=data)` and the regularisation term `alpha_tv * MixedL21Norm()`, with `alpha_tv = 0.00063`. Finally, use the `IndicatorBox(lower=0.0)` to enforce a non-negativity constraint for the function $\mathcal{G}$.

```python

        F = BlockFunction(0.5*L2NormSquared(b=data), alpha_tv * MixedL21Norm())
        G = IndicatorBox(lower=0)

```

```python
A = ...
Grad = ...

K = BlockOperator(A, Grad)

alpha_tv = 0.00054
F = BlockFunction(..., alpha_tv * ...)
G = IndicatorBox(lower=0)

normK = K.norm()
sigma = 1./normK
tau = 1./normK

pdhg_tv = PDHG(f = ..., g = ..., operator=..., max_iteration = 500,
            update_objective_interval = 100)
pdhg_tv.run(verbose=1)
```

## Exercise 2: Solution

```python
A = ProjectionOperator(ig, ag, 'gpu')
Grad = GradientOperator(ig, correlation = "SpaceChannels")

K = BlockOperator(A, Grad)

alpha_tv = 0.00054
F = BlockFunction(0.5*L2NormSquared(b=data), alpha_tv * MixedL21Norm())
G = IndicatorBox(lower=0)

normK = K.norm()
sigma = 1./normK
tau = 1./normK

pdhg_tv = PDHG(f = F, g = G, operator=K, max_iteration = 500,
            update_objective_interval = 100)
pdhg_tv.run(verbose=1)
```

## Show TV reconstruction for 4 different time frames

```python
titles_sinos = ["Time-frame {}".format(i) for i in [0,5,10,16]]

show2D(pdhg_tv.solution, slice_list = [0,5,10,16], num_cols=4, origin="upper",fix_range=(0,0.065),
                   cmap="inferno", title=titles_sinos, size=(25, 20))

name_tv = "TV_reconstruction_projections_{}".format(num_proj)
writer = NEXUSDataWriter(file_name = "TV_reconstructions/"+name_tv+".nxs",
                     data = pdhg_tv.solution)
writer.write()
```

## Directional Total Variation

For our final reconstruction, we use a **structure-based prior**, namely the directional Total Variation (dTV) introduced in [Ehrhardt MJ, Arridge SR](https://doi.org/10.1109/tip.2013.2277775).

In comparison with the Total variation regulariser,
$$
\mathrm{TV}(u) = \|\nabla u\|_{2,1} = \sum |\nabla u\|_{2},
$$

in the Direction Total variation, a weight in front of the gradient is used based on a **reference image**. This acts as prior information from which edge structures are propagated into the reconstruction process. For example, an image from another modality, e.g., MRI, , can be used in the PET reconstruction, see [Ehrhardt2016](https://ieeexplore.ieee.org/document/7452643/), [Ehrhardt2016MRI](https://epubs.siam.org/doi/10.1137/15M1047325). Another popular setup, is to use either both modalities or even channels in a joint reconstruction problem simultaneously, improving significantly the quality of the image, see for instance [Knoll et al](https://ieeexplore.ieee.org/document/7466848), [Kazantsev_2018](https://doi.org/10.1088/1361-6420/aaba86).

**Definition:** The dTV regulariser of the image $u$ given the reference image $v$ is defined as

$$\begin{equation}
d\mathrm{TV}(u,v)  := \|D_{v}\nabla u\|_{2,1} = \sum_{i,j=1}^{M,N} \big(|D_{v}\nabla u|_{2}\big)_{i,j},
\label{dTV_definition}
\end{equation}
$$

where the weight $D_{v}$ depends on the normalised gradient $\xi_{v}$ of the reference image $v$,

$$\begin{equation}
D_{v} = \mathbb{I}_{2\times2} - \xi_{v}\xi_{v}^T, \quad \xi_{v} = \frac{\nabla v}{\sqrt{\eta^{2} + |\nabla v|_{2}^{2}}}, \quad \eta>0.
\label{weight_D}
\end{equation}
$$

In this dynamic sparse CT framework, we apply the dTV regulariser for each time frame $t$ which results to the following minimisation problem:

$$
\begin{equation}
u^{*}_{t} = \underset{u}{\operatorname{argmin}}  \, \frac{1}{2}\| A_\text{sc} u_{t}  - b_{t} \|^{2} + \alpha \, d\mathrm{TV}(u_{t}, v_{t})\quad \mbox{(Dynamic dTV)}, \label{dynamic_dtv_problem}
\end{equation}
$$

where $A_\text{sc}$, $b_{t}$, $u^{*}_{t}$, denote the single channel `ProjectionOperator`, the sinogram data and the reconstructed image for the time frame $t$ respectively.

## Reference images

In terms of the reference images $(v_{t})_{t=0}^{16}$, we are going to use the FBP reconstructions of the additional tomographic data. There are two datasets in `GelPhantom_extra_frames.mat` with dense sampled measurements from the first and last (18th) time steps:

- Pre-scan data with 720 projections
- Post-scan data with 1600 projections

We first read the matlab files and create the following acquisition data:

- data_pre_scan
- data_post_scan

## Read matlab files for the extra frames

```python
path = os.path.abspath("/mnt/materials/SIRF/Fully3D/CIL/GelPhantom")
data_mat_extra = "GelPhantom_extra_frames"

pre_scan_info = read_extra_frames(path, data_mat_extra, "GelPhantomFrame1_b4")
post_scan_info = read_extra_frames(path, data_mat_extra, "GelPhantomFrame18_b4")
```

## Acquisition geometry for the 1st frame: 720 projections

```python
ag2D_pre_scan = AcquisitionGeometry.create_Cone2D(source_position = [0,   pre_scan_info['distanceSourceOrigin']],
                                       detector_position = [0, -pre_scan_info['distanceOriginDetector']])\
                                    .set_panel(num_pixels = pre_scan_info['numDetectors'], pixel_size = 2*pre_scan_info['pixelSize'])\
                                    .set_angles(pre_scan_info['angles'])\
                                    .set_labels(['angle', 'horizontal'])
```

## Acquisition geometry for the 18th frame: 1600 projections

```python
ag2D_post_scan = AcquisitionGeometry.create_Cone2D(source_position = [0,   post_scan_info['distanceSourceOrigin']],
                                       detector_position = [0, -post_scan_info['distanceOriginDetector']])\
                                    .set_panel(num_pixels = post_scan_info['numDetectors'], pixel_size = 2*post_scan_info['pixelSize'])\
                                    .set_angles(post_scan_info['angles'])\
                                    .set_labels(['angle', 'horizontal'])
```

## Create Acquisition data: Pre-scan, Post-scan

```python
data_pre_scan = ag2D_pre_scan.allocate()
data_pre_scan.fill(pre_scan_info['sinograms'])

data_post_scan = ag2D_post_scan.allocate()
data_post_scan.fill(post_scan_info['sinograms'])

show2D([data_pre_scan,data_post_scan], title=["Pre-scan 720 projections", "Post-scan 1600 projections"], cmap="inferno", size=(10,10))
```

## FBP reconstruction (Reference images)

For the FBP reconstruction of the pre/post scan we use the 2D image geometry, `ig2D` and the corresponding acquisition geometries `ag2D_pre_scan` and `ag2D_post_scan`.

## Exercise 3: Perform FBP to obtain the reference images

```python
fbp_recon_pre_scan = FBP(..., ag2D_pre_scan)(...)
fbp_recon_post_scan = FBP(ig2D, ...)(...)
```

## Exercise 3: Solution

```python
fbp_recon_pre_scan = FBP(ig2D, ag2D_pre_scan)(data_pre_scan)
fbp_recon_post_scan = FBP(ig2D, ag2D_post_scan)(data_post_scan)
```

## Show and save the reference images

```python
show2D([fbp_recon_pre_scan,fbp_recon_post_scan],
       title=["FBP: Pre-scan", "FBP: Post-scan"], cmap="inferno", origin="upper", fix_range=(0,0.065))

name_fbp_pre_scan = "FBP_pre_scan"
writer = NEXUSDataWriter(file_name = "FBP_reconstructions/"+name_fbp_pre_scan+".nxs",
                     data = fbp_recon_pre_scan)
writer.write()

name_fbp_post_scan = "FBP_post_scan"
writer = NEXUSDataWriter(file_name = "FBP_reconstructions/"+name_fbp_post_scan+".nxs",
                     data = fbp_recon_post_scan)
writer.write()
```

## Edge information from the normalised gradient $\,\xi_{v}$

In the following we compute the normalised gradient $\,\xi_{v}$ for the two reference images using different $\eta$ values:

$$\xi_{v} = \frac{\nabla v}{\sqrt{\eta^{2} + |\nabla v|_{2}^{2}}}, \quad \eta>0$$

```python
def xi_vector_field(image, eta):

    ig = image.geometry
    ig.voxel_size_x = 1.
    ig.voxel_size_y = 1.
    G = GradientOperator(ig)
    numerator = G.direct(image)
    denominator = np.sqrt(eta**2 + numerator.get_item(0)**2 + numerator.get_item(1)**2)
    xi = numerator/denominator

    return (xi.get_item(0)**2 + xi.get_item(1)**2).sqrt()

etas = [0.001, 0.005]

xi_post_scan = []
xi_pre_scan = []

for i in etas:

    xi_post_scan.append(xi_vector_field(fbp_recon_post_scan, i))
    xi_pre_scan.append(xi_vector_field(fbp_recon_pre_scan, i))
```

## Edge information from the pre-scan and post-scan reference images

```python
title_etas = ["$\eta$ = {}".format(eta) for eta in etas]
show2D(xi_pre_scan, cmap="inferno", title=title_etas, origin="upper", num_cols=2, size=(10,10))
show2D(xi_post_scan, cmap="inferno", title=title_etas, origin="upper", num_cols=2,size=(10,10))
```

## Directional Total variation reconstruction

In total we have 17 time frames, and we need 17 reference images.  Due to a slight movement of the sample at the beginning of the experiment, we apply the pre-scan reference image for the first time frame and use the post-scan reference image for the remaining time frames.

One could apply other configurations for the reference image in the intermediate time frames. For example, in order to reconstruct the $(t+1)$th time frame, one could use the $t$th time frame reconstruction as reference. A more sophisticated reference selection approach is applied in hyperspectral computed tomography in [Kazantsev_2018](https://iopscience.iop.org/article/10.1088/1361-6420/aaba86).

## Setup and run the PDHG algorithm

In order to solve the [Dynamic dTV](#dynamic_dTV) problem, we use the implicit formulation of the PDHG algorithm, where the **Fast Gradient Projection** algorithm, under the **dTV** regulariser, is used for the inner proximal problem.

- We first define the single slice `ProjectionOperator` using the 2D image and acquisition geometries and compute the `sigma` and `tau` stepsizes.

```python

    K = ProjectionOperator(ig2D, ag2D, 'gpu')

    normK = K.norm()
    sigma = 1./normK
    tau = 1./normK

```

- We allocate space for the dTV reconstruction, i.e., `dtv_recon` using the full image geometry `ig`.

- Use the following values: `max_iteration=100`, `alpha_dtv = 0.0072`, `eta=0.005`.

- Loop over all the time frames (`tf`) and update :

    - the fidelity term `0.5 * L2NormSquared(b=data.subset(channel=tf)` for the function $\mathcal{F}$,

    - the regularisation term ( $\mathcal{G}$ ) using the `FGP_dTV` function class from the CIL plugin of the CCPi-Regularisation Toolkit. For `tf=0` the pre-scan reference is used and for `tf>0` the post-scan reference is used.

## Define single slice projection operator

```python
K = ProjectionOperator(ig2D, ag2D, 'gpu')
```

## Parameters for the dTV regulariser and the PDHG algorithm

```python
normK = K.norm()
sigma = 1./normK
tau = 1./normK

dtv_recon = ig.allocate()

max_iterations = 100
alpha_dtv = 0.0072
eta = 0.005
```

## Loop over all channels and update:

* the acquisition data in the `L2NormSquared` fidelity term,
* the reference image in the `FGP_dTV` regulariser.

```python
for tf in range(ig.channels):

    F = 0.5 * L2NormSquared(b=data.subset(channel=tf))

    if tf==0:
        G = alpha_dtv * FGP_dTV(reference = fbp_recon_pre_scan, eta=eta, device='gpu')
    else:
        G = alpha_dtv * FGP_dTV(reference = fbp_recon_post_scan, eta=eta, device='gpu')

    pdhg_dtv = PDHG(f=F, g=G, operator = K, tau = tau, sigma = sigma,
            max_iteration = max_iterations, update_objective_interval=100)
    pdhg_dtv.run(verbose = 0)

    clear_output(wait=True)

    print("Finish dTV regularisation for the frame {} with {} projections\n".format(tf,num_proj), end='\r')

    dtv_recon.fill(pdhg_dtv.solution, channel=tf)
```

```python
show2D(dtv_recon, slice_list = [0,5,10,16], num_cols=4, origin="upper",fix_range=(0,0.065),
                   cmap="inferno", title=titles_sinos, size=(25, 20))
```

## FBP vs TV vs dTV reconstructions vs FBP (360 projections).

For our final comparison, we reconstruct the full dataset, i.e., 360 projections and compare it with the FBP, TV and dTV reconstruction with 36 projection.

```python
# Load data with 360 projections
reader = NEXUSDataReader(file_name = "SparseData/data_360.nxs")
data_360 = reader.load_data()
ag2D = data_360.geometry.subset(channel=0)

# Perform channelwise FBP reconstruction
fbp_recon_360 = ig.allocate()
for i in range(ig.channels):
    tmp = FBP(ig2D, ag2D)(data_360.subset(channel=i))
    fbp_recon_360.fill(tmp, channel=i)
```

## Show FBP, TV, dTV with 36 projections and FBP with 360 projections.

```python
show2D(fbp_recon,slice_list = [0,5,10,16], num_cols=4, origin="upper",fix_range=(0,0.065),
                   cmap="inferno", title=titles_sinos, size=(25, 20))

show2D(pdhg_tv.solution,slice_list = [0,5,10,16], num_cols=4, origin="upper",fix_range=(0,0.065),
                   cmap="inferno", title=titles_sinos, size=(25, 20))

show2D(dtv_recon,slice_list = [0,5,10,16], num_cols=4, origin="upper",fix_range=(0,0.065),
                   cmap="inferno", title=titles_sinos, size=(25, 20))

show2D(fbp_recon_360,slice_list = [0,5,10,16], num_cols=4, origin="upper",fix_range=(0,0.065),
                   cmap="inferno", title=titles_sinos, size=(25, 20))
```

## Exercise 4

As an additional **Exercise**, you can try other configurations, namely the datasets `data_18` and `data_72` by changing the [number of projections](#num_proj). The optimal regularisation parameters for spatiotemporal TV and dTV are reported in Table 1 in [Papoutsellis et al](https://arxiv.org/pdf/2102.06126.pdf#page=7).

Conclusions

In this notebook, we presented three different reconstruction methods for undersampled dynamic tomographic data. The **channelwise FBP** and the **Spatiotemporal TV** and **Directional TV** regularisation. We focused on the reconstruction of the dataset with 36 projections out of 360 projections.

So far, we presented how to reconstruct two different multichannel datasets:

* **[Colour](01_Colour_Processing.ipynb)**: 3 channels that encodes information for the red, green and blue colours.
* **Dynamic**: 17 channels that encode temporal information from our acquisition tomographic data.

In the next notebook, we focus on a 4D hyperspectral dataset where the channel dimension contains spectral energy information acquired from an energy-sensitive X-ray detector.

```python

```


# Reimangininig AMM Carbon Finance

## Questions

*Can the prices converge?
*Can a cSDR reflect a global carbon Price?
*Can we levarage the SDR to balance carbon?
*Marble policy update style.
*Equalize smaller pools by

are IMF Weights comparable to ETS allocations per country?

policy update balance

## Assumptions

Although the goal is to demostrate that Balancer could derive a global Carbon price, tackling article 6 of the UN Paris agreement; we choose, for symplification purposes, to demostrate this in the North American fragmented carbon market. Specifically using data from the RGGI(North East US coalition), California's and Quebec's carbon credit markets and offsets.

# System Design

Mark Libertatti's Model

# Differential Specification

\begin{align}
\large Emissions_t &\large = CarbonCredits_{t-1} + {\Delta carbonCredits} \\
\large CarbonOffsets_t &\large = CarbonOffsets_{t-1} + {\Delta CarbonOffsets} \\
\end{align}

\begin{align}
\large {\Delta emissions} = \alpha * globalWarming_{t-1} \\
\large {\Delta CarbonOffsets_t} = -\beta * emissions_{t-1} \\
\large GlobalCarbonPrice = wG7fiat : \Delta globalCT - CarbonOffsets \\
\end{align}

globalCarbonPrice = wG7fiat : ΔglobalCT - CarbonOffsets

## Table of Contents

Dependencies

Modelling

State Variables

System Parameters

Policy Functions

State Update Functions

Partial State Update Blocks

Simulation

Configuration

Execution

Output Preparation

Analysis

## 0. Dependencies

```python
pip install balancerv2cad==0.1.91

```

    Requirement already satisfied: balancerv2cad==0.1.91 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (0.1.91)
    Note: you may need to restart the kernel to use updated packages.

```python
#optional -f show the full list of installed files
%pip show balancerv2cad
```

    Name: balancerv2cad
    Version: 0.1.91
    Summary: A preconfigured Python package, using python poetry
    Home-page: https://github.com/metavisionlabs/balancerv2cad.git
    Author: Nico Rodriguez, Thomas Liu, Marcin Jaczynski
    Author-email: None
    License: MIT
    Location: c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages
    Requires:
    Required-by:
    Note: you may need to restart the kernel to use updated packages.

```python
pip install cadcad
```

    Requirement already satisfied: cadcad in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (0.4.23)
    Requirement already satisfied: fn in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from cadcad) (0.4.3)
    Requirement already satisfied: pathos in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from cadcad) (0.2.8)
    Requirement already satisfied: funcy in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from cadcad) (1.16)
    Requirement already satisfied: pandas in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from cadcad) (1.3.0)
    Requirement already satisfied: pytz>=2017.3 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from pandas->cadcad) (2021.1)
    Requirement already satisfied: python-dateutil>=2.7.3 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from pandas->cadcad) (2.8.1)
    Requirement already satisfied: numpy>=1.17.3 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from pandas->cadcad) (1.21.0)
    Requirement already satisfied: six>=1.5 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from python-dateutil>=2.7.3->pandas->cadcad) (1.16.0)
    Requirement already satisfied: ppft>=1.6.6.4 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from pathos->cadcad) (1.6.6.4)
    Requirement already satisfied: pox>=0.3.0 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from pathos->cadcad) (0.3.0)
    Requirement already satisfied: multiprocess>=0.70.12 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from pathos->cadcad) (0.70.12.2)
    Requirement already satisfied: dill>=0.3.4 in c:\users\sebas\anaconda3\envs\cadcadbalancer\lib\site-packages (from pathos->cadcad) (0.3.4)
    Note: you may need to restart the kernel to use updated packages.

```python
import pandas
#import plotly

# cadCAD configuration
from cadCAD.configuration.utils import config_sim
from cadCAD.configuration import Experiment

# cadCAD Engine
from cadCAD.engine import ExecutionMode, ExecutionContext
from cadCAD.engine import Executor

# cadCAD global simulation configuration list
from cadCAD import configs
```

# MODELING

## 1. State Variables

The key "stocks" of the system are:
1. Price or allowences (??) of regional Carbon Credits of Quebec and California to create a weighted pool.
2. Regional Carbon Offsets of Quebec and California
3. USD and CAD fluctuations.

**Timestep**: daily as per availabe data

```python
#allowences or prices!?
initial_state = {
    'caCC' : 60, # California get actual data
    'qcCC' : 10, # Quebec get actual data
    'rgCC' : 20, # RGGI get actual data
    'caOF' : 60, # California get actual data
    'qcOF' : 10, # Quebec get actual data
    'rgOF' : 20, # RGGI get actual data
    'usdcad' : 1.14, #forex USD to CAD, get actual data
}

initial_state
```

    {'caCC': 60,
     'qcCC': 10,
     'rgCC': 20,
     'caOF': 60,
     'qcOF': 10,
     'rgOF': 20,
     'usdcad': 1.14}

## 2. System Parameters

(These are to be CONTROL static or variable)
The key variables of the system are the fluctuating prices of all the data "stocks".

```python
system_params = {
    'caCC_p' : [15], # California get actual data
    'qcCC_p' : [15], # Quebec get actual data
    'rgCC_p' : [15], # RGGI get actual data
    'caOF_p' : [5], # California get actual data
    'qcOF_p' : [5], # Quebec get actual data
    'rgOF_p' : [5], # RGGI get actual data
    'usdcad_p' : [1.14], #forex USD to CAD, get actual data
}

system_params
```

    {'caCC_p': [15],
     'qcCC_p': [15],
     'rgCC_p': [15],
     'caOF_p': [5],
     'qcOF_p': [5],
     'rgOF_p': [5],
     'usdcad_p': [1.14]}

## 3. Policy Functions

```python
def p_creditVSoffsets(params, substep,state_history, previous_state):

    #Parameters

    #State Variables

    #Calculate rate

    #Calculate change

    return{'how much fresh air is left' : -1.0 * XXX }  ## to reduce XXX instead of increment
```

## 4. State Update Functions
State changes over time.

**For Reference Only**
```python
def state_update_function(params, substep, state_history, previous_state, policy_input):
    variable_value = 0
    return 'variable_name', variable_value
```
* `params` is a Python dictionary containing the **system parameters**
* `substep` is an integer value representing a step within a single `timestep`
* `state_history` is a Python list of all previous states
* `previous_state` is a Python dictionary that defines what the state of the system was at the **previous timestep** or **substep**
* `policy_input` is a Python dictionary of signals or actions from **policy functions**

```python
#possibly iterate on dictionary to create all the many SUFs
#for key in initial_state:
#    print(key)

def new_caCC_p(current_caCC_P, caCC_alpha, caOF):
    'above and below logic to be defined'
    return current_caCC_P + caCC_alpha - caOF
```

```python
#casting variables from initial and system dictionaries

current_caCC_P = initial_state['caCC']
#print ('current_caCC_P: ' + str(current_caCC_P))
caOF = initial_state['caOF']
#print ('caOF: ' + str(caOF))

#updating variables
caCC_alpha = system_params['caOF_p'][0]

print('New caCC budget: ' + str(new_caCC_p(current_caCC_P, caCC_alpha, caOF)))
```

    New caCC budget: 5

```python
def s_caC(params, substep, state_history, previous_state, policy_input):
    'I am lost here   caCC = previous_state['']'

    return 'variable_name', variable_value
```

## 5. Partial State Update Blocks

```python
# reading data from csv
import pandas as pd

# Creating a dataframe from a csv file
df = pd.read_csv('data/test.csv')
df
```

      Insert FX Rates\t\t\t\t

      0
      Date\tEuro\tPound Sterling\tJapanese Yen\tRenm...

      1
      4/1/2016\t1.1411\t1.4286\t112.17\t6.4740

      2
      4/4/2016\t1.1364\t1.4230\t111.68\t6.4809

      3
      4/5/2016\t1.1359\t1.4205\t110.32\t6.4814

      4
      4/6/2016\t1.1349\t1.4071\t110.31\t6.4917

      5
      4/7/2016\t1.1373\t1.4077\t108.34\t6.4831

      6
      4/8/2016\t1.1367\t1.4069\t108.67\t6.4866

      7
      4/11/2016\t1.1391\t1.4235\t108.21\t6.4814

      8
      4/12/2016\t1.1410\t1.4317\t108.31\t6.4706

      9
      4/13/2016\t1.1303\t1.4248\t109.24\t6.4849

      10
      4/14/2016\t1.1260\t1.4140\t109.24\t6.4936

      11
      4/15/2016\t1.1278\t1.4188\t108.97\t6.4864

      12
      4/18/2016\t1.1305\t1.4194\t108.33\t6.4891

      13
      4/19/2016\t1.1335\t1.4370\t109.38\t6.4739

      14
      4/20/2016\t1.1368\t1.4376\t109.22\t6.4730

      15
      4/21/2016\t1.1312\t1.4376\t109.75\t6.4826

      16
      4/22/2016\t1.1269\t1.4377\t110.64\t6.4966

      17
      4/25/2016\t1.1255\t1.4454\t111.08\t6.5066

      18
      4/26/2016\t1.1284\t1.4565\t110.90\t6.5049

      19
      4/27/2016\t1.1309\t1.4608\t111.24\t6.5062

      20
      4/28/2016\t1.1347\t1.4552\t108.08\t6.4857

      21
      4/29/2016\t1.1396\t1.4614\t107.02\t6.4886

      22
      5/2/2016\t1.1513\t1.4662\t106.50\t6.4835

      23
      5/3/2016\t1.1580\t1.4672\t105.84\t6.4908

      24
      5/4/2016\t1.1485\t1.4480\t106.72\t6.5155

      25
      5/5/2016\t1.1426\t1.4461\t107.28\t6.5189

      26
      5/6/2016\t1.1425\t1.4505\t106.98\t6.5134

      27
      5/9/2016\t1.1390\t1.4444\t108.01\t6.5283

      28
      5/10/2016\t1.1363\t1.4423\t109.17\t6.5434

      29
      5/11/2016\t1.1395\t1.4424\t108.76\t6.5340

      30
      5/12/2016\t1.1391\t1.4444\t109.21\t6.5469

      31
      5/13/2016\t1.1345\t1.4412\t108.91\t6.5456

      32
      5/16/2016\t1.1324\t1.4351\t108.82\t6.5467

      33
      5/17/2016\t1.1321\t1.4475\t109.52\t6.5456

      34
      5/18/2016\t1.1273\t1.4482\t109.44\t6.5579

      35
      5/19/2016\t1.1203\t1.4636\t109.93\t6.5668

      36
      5/20/2016\t1.1214\t1.4579\t110.35\t6.5644

      37
      5/23/2016\t1.1207\t1.4469\t109.46\t6.5628

      38
      5/24/2016\t1.1182\t1.4602\t109.61\t6.5647

      39
      5/25/2016\t1.1144\t1.4662\t110.21\t6.5655

      40
      5/26/2016\t1.1179\t1.4695\t109.99\t6.5609

      41
      5/27/2016\t1.1177\t1.4654\t109.62\t6.5682

      42
      5/30/2016\t1.1140\t1.4602\t111.11\t6.5903

      43
      5/31/2016\t1.1151\t1.4618\t111.08\t6.5871

      44
      6/1/2016\t1.1158\t1.4446\t109.54\t6.5891

      45
      6/2/2016\t1.1199\t1.4431\t108.96\t6.5882

      46
      6/3/2016\t1.1140\t1.4408\t108.93\t6.5940

      47
      6/6/2016\t1.1334\t1.4392\t107.15\t6.5705

      48
      6/7/2016\t1.1360\t1.4588\t107.76\t6.5712

      49
      6/8/2016\t1.1374\t1.4569\t107.05\t6.5766

      50
      6/9/2016\t1.1342\t1.4466\t106.49\t6.5779

      51
      6/10/2016\t1.1300\t1.4420\t106.76\t6.5914

      52
      6/13/2016\t1.1270\t1.4158\t106.01\t6.5990

      53
      6/14/2016\t1.1221\t1.4164\t105.90\t6.6037

      54
      6/15/2016\t1.1225\t1.4195\t106.30\t6.5971

      55
      6/16/2016\t1.1214\t1.4129\t104.30\t6.6019

      56
      6/17/2016\t1.1262\t1.4256\t104.26\t6.6008

# SIMULATION

## 6. Configuration

```python
sim_config = config_sim{
    'T' : range(10), #number of discrete iterations
    'N' : 1, #number of MCarlo runs
}
```

## 7. Execution

## 8. Simulation Output Preparation

## 9. Simulation Analysis


# Bringing contextual word representations into your models

```python
__author__ = "Christopher Potts"
__version__ = "CS224u, Stanford, Spring 2021"
```

## Contents

1. [Overview](#Overview)
1. [General set-up](#General-set-up)
1. [Hugging Face BERT models and tokenizers](#Hugging-Face-BERT-models-and-tokenizers)
1. [BERT featurization with Hugging Face](#BERT-featurization-with-Hugging-Face)
  1. [Simple feed-forward experiment](#Simple-feed-forward-experiment)
  1. [A feed-forward experiment with the sst module](#A-feed-forward-experiment-with-the-sst-module)
  1. [An RNN experiment with the sst module](#An-RNN-experiment-with-the-sst-module)
1. [BERT fine-tuning with Hugging Face](#BERT-fine-tuning-with-Hugging-Face)
  1. [HfBertClassifier](#HfBertClassifier)
  1. [HfBertClassifier experiment](#HfBertClassifier-experiment)

## Overview

This notebook provides a basic introduction to using pre-trained [BERT](https://github.com/google-research/bert) representations with the Hugging Face library. It is meant as a practical companion to our lecture on contextual word representations. The goal of this notebook is just to help you use these representations in your own work.

If you haven't already, I encourage you to review the notebook [vsm_04_contextualreps.ipynb](vsm_04_contextualreps.ipynb) before working with this one. That notebook covers the fundamentals of these models; this one dives into the details more quickly.

A number of the experiments in this notebook are resource-intensive. I've included timing information for the expensive steps, to give you a sense for how long things are likely to take. I ran this notebook on a laptop with a single NVIDIA RTX 2080 GPU.

## General set-up

The following are requirements that you'll already have met if you've been working in this repository. As you can see, we'll use the [Stanford Sentiment Treebank](sst_01_overview.ipynb) for illustrations, and we'll try out a few different deep learning models.

```python
import os
from sklearn.metrics import classification_report
import torch
import torch.nn as nn
from transformers import BertModel, BertTokenizer

from torch_shallow_neural_classifier import TorchShallowNeuralClassifier
from torch_rnn_classifier import TorchRNNModel
from torch_rnn_classifier import TorchRNNClassifier
from torch_rnn_classifier import TorchRNNClassifierModel
import sst
import utils

device = "cpu" if not torch.cuda.is_available() else 'cuda:0'
device
```

    'cuda:0'

```python
utils.fix_random_seeds()
```

```python
SST_HOME = os.path.join("data", "sentiment")
```

The `transformers` library does a lot of logging. To avoid ending up with a cluttered notebook, I am changing the logging level. You might want to skip this as you scale up to building production systems, since the logging is very good – it gives you a lot of insights into what the models and code are doing.

```python
import logging
logger = logging.getLogger()
logger.level = logging.ERROR
```

## Hugging Face BERT models and tokenizers

We'll illustrate with the BERT-base cased model:

```python
weights_name = 'bert-base-cased'
```

There are lots other options for pretrained weights. See [this Hugging Face directory](https://huggingface.co/models).

Next, we specify a tokenizer and a model that match both each other and our choice of pretrained weights:

```python
bert_tokenizer = BertTokenizer.from_pretrained(weights_name)
```

```python
bert_model = BertModel.from_pretrained(weights_name)
bert_model.to(device)
```

    Some weights of the model checkpoint at bert-base-cased were not used when initializing BertModel: ['cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight']
    - This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
    - This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

    BertModel(
      (embeddings): BertEmbeddings(
        (word_embeddings): Embedding(28996, 768, padding_idx=0)
        (position_embeddings): Embedding(512, 768)
        (token_type_embeddings): Embedding(2, 768)
        (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
        (dropout): Dropout(p=0.1, inplace=False)
      )
      (encoder): BertEncoder(
        (layer): ModuleList(
          (0): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (1): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (2): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (3): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (4): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (5): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (6): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (7): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (8): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (9): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (10): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (11): BertLayer(
            (attention): BertAttention(
              (self): BertSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): BertSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): BertIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
            )
            (output): BertOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
        )
      )
      (pooler): BertPooler(
        (dense): Linear(in_features=768, out_features=768, bias=True)
        (activation): Tanh()
      )
    )

For modeling (as opposed to creating static representations), we will mostly process examples in batches – generally very small ones, as these models consume _a lot_ of memory. Here's a small batch of texts to use as the starting point for illustrations:

```python
example_texts = [
    "Encode sentence 1. [SEP] And sentence 2!",
    "Bert knows Snuffleupagus"]
```

We will often need to pad (and perhaps truncate) token lists so that we can work with fixed-dimensional tensors: The `batch_encode_plus` has a lot of options for doing this:

```python
bert_tokenizer.tokenize('transubstantiation')
bert_tokenizer.convert_tokens_to_ids(toks)
```

    ['trans', '##ub', '##stant', '##iation']

    [2159, 2427]

```python
bert_tokenizer.convert_ids_to_tokens(list(range(17738, 17749)))
```

    ['Sanchez',
     'Oman',
     '##mith',
     'Flint',
     'Platform',
     '##ioned',
     'emission',
     'packet',
     'Persia',
     '##formed',
     'takeover']

```python
example_ids = bert_tokenizer.batch_encode_plus(
    example_texts,
    add_special_tokens=True,
    return_attention_mask=True,
    padding='longest')
```

The `token_type_ids` is used for multi-text inputs like NLI. The `'input_ids'` field gives the indices for each of the two examples:

```python
example_ids['input_ids']
```

    [[101, 13832, 13775, 5650, 122, 119, 102, 1262, 5650, 123, 106, 102],
     [101, 15035, 3520, 156, 14787, 13327, 4455, 28026, 1116, 102, 0, 0]]

Notice that the final two tokens of the second example are pad tokens.

For fine-tuning, we want to avoid attending to padded tokens. The `'attention_mask'` captures the needed mask, which we'll be able to feed directly to the pretrained BERT model:

```python
example_ids['attention_mask']
```

    [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]

Finally, we can run these indices and masks through the pretrained model:

```python
X_example = torch.tensor(example_ids['input_ids'], device=device)
X_example_mask = torch.tensor(example_ids['attention_mask'], device=device)
X_example
with torch.no_grad():
    reps = bert_model(X_example, attention_mask=X_example_mask)
```

    tensor([[  101, 13832, 13775,  5650,   122,   119,   102,  1262,  5650,   123,
               106,   102],
            [  101, 15035,  3520,   156, 14787, 13327,  4455, 28026,  1116,   102,
                 0,     0]], device='cuda:0')

```python
reps.last_hidden_state.shape
```

    torch.Size([2, 12, 768])

Hugging Face BERT models create a special `pooler_output` representation that is the final representation above the [CLS] extended with a single layer of parameters:

```python
reps.pooler_output.shape
```

    torch.Size([2, 768])

We have two examples, each representented by a single vector of dimension 768, which is $d_{model}$ for BERT base using the notation from [the original Transformers paper](https://arxiv.org/abs/1706.03762). This is an easy basis for fine-tuning, as we will see.

We can also access the final output for each state:

```python
reps.last_hidden_state.shape
```

    torch.Size([2, 12, 768])

Here, we have 2 examples, each padded to the length of the longer one (12), and each of those representations has dimension 768. These representations can be used for sequence modeling, or pooled somehow for simple classifiers.

Those are all the essential ingredients for working with these parameters in Hugging Face. Of course, the library has a lot of other functionality, but the above suffices to featurize and to fine-tune.

## BERT featurization with Hugging Face

To start, we'll use the Hugging Face interfaces just to featurize examples to create inputs to a separate model. In this setting, the BERT parameters are frozen.

```python
def bert_phi(text):
    input_ids = bert_tokenizer.encode(text, add_special_tokens=True)
    X = torch.tensor([input_ids], device=device)
    with torch.no_grad():
        reps = bert_model(X)
        return reps.last_hidden_state.squeeze(0).cpu().numpy()
```

### Simple feed-forward experiment

For a simple feed-forward experiment, we can get the representation of the `[CLS]` tokens and use them as the inputs to a shallow neural network:

```python
def bert_classifier_phi(text):
    reps = bert_phi(text)
    #return reps.mean(axis=0)  # Another good, easy option.
    return reps[0]
```

Next we read in the SST train and dev splits:

```python
train = sst.train_reader(SST_HOME)

dev = sst.dev_reader(SST_HOME)
```

Split the input/output pairs out into separate lists:

```python
X_str_train = train.sentence.values
y_train = train.label.values

X_str_dev = dev.sentence.values
y_dev = dev.label.values
```

In the next step, we featurize all of the examples. These steps are likely to be the slowest in these experiments:

```python
%time X_train = [bert_classifier_phi(text) for text in X_str_train]
```

    CPU times: user 1min 28s, sys: 146 ms, total: 1min 28s
    Wall time: 1min 28s

```python
%time X_dev = [bert_classifier_phi(text) for text in X_str_dev]
```

    CPU times: user 11.7 s, sys: 34.9 ms, total: 11.8 s
    Wall time: 11.8 s

Now that all the examples are featurized, we can fit a model and evaluate it:

```python
model = TorchShallowNeuralClassifier(
    early_stopping=True,
    hidden_dim=300)
```

```python
%time _ = model.fit(X_train, y_train)
```

    Stopping after epoch 20. Validation score did not improve by tol=1e-05 for more than 10 epochs. Final error is 5.419588506221771

    CPU times: user 3 s, sys: 54.1 ms, total: 3.06 s
    Wall time: 1.75 s

```python
preds = model.predict(X_dev)
```

```python
print(classification_report(y_dev, preds, digits=3))
```

                  precision    recall  f1-score   support

        negative      0.694     0.785     0.737       428
         neutral      0.443     0.118     0.186       229
        positive      0.678     0.849     0.754       444

        accuracy                          0.672      1101
       macro avg      0.605     0.584     0.559      1101
    weighted avg      0.635     0.672     0.629      1101

### A feed-forward experiment with the sst module

It is straightforward to conduct experiments like the above using `sst.experiment`, which will enable you to do a wider range of experiments without writing or copy-pasting a lot of code.

```python
def fit_shallow_network(X, y):
    mod = TorchShallowNeuralClassifier(
        hidden_dim=300,
        early_stopping=True)
    mod.fit(X, y)
    return mod
```

```python
%%time
_ = sst.experiment(
    sst.train_reader(SST_HOME),
    bert_classifier_phi,
    fit_shallow_network,
    assess_dataframes=sst.dev_reader(SST_HOME),
    vectorize=False)  # Pass in the BERT reps directly!
```

    Stopping after epoch 44. Validation score did not improve by tol=1e-05 for more than 10 epochs. Final error is 5.209178328514099

                  precision    recall  f1-score   support

        negative      0.699     0.797     0.745       428
         neutral      0.413     0.218     0.286       229
        positive      0.728     0.806     0.765       444

        accuracy                          0.680      1101
       macro avg      0.613     0.607     0.598      1101
    weighted avg      0.651     0.680     0.657      1101

    CPU times: user 1min 49s, sys: 536 ms, total: 1min 49s
    Wall time: 1min 46s

### An RNN experiment with the sst module

We can also use BERT representations as the input to an RNN. There is just one key change from how we used these models before:

* Previously, we would feed in lists of tokens, and they would be converted to indices into a fixed embedding space. This presumes that all words have the same representation no matter what their context is.

* With BERT, we skip the embedding entirely and just feed in lists of BERT vectors, which means that the same word can be represented in different ways.

`TorchRNNClassifier` supports this via `use_embedding=False`. In turn, you needn't supply a vocabulary:

```python
def fit_rnn(X, y):
    mod = TorchRNNClassifier(
        vocab=[],
        early_stopping=True,
        use_embedding=False)  # Pass in the BERT hidden states directly!
    mod.fit(X, y)
    return mod
```

```python
%%time
_ = sst.experiment(
    sst.train_reader(SST_HOME),
    bert_phi,
    fit_rnn,
    assess_dataframes=sst.dev_reader(SST_HOME),
    vectorize=False)  # Pass in the BERT hidden states directly!
```

    Stopping after epoch 35. Validation score did not improve by tol=1e-05 for more than 10 epochs. Final error is 0.5038421787321568

                  precision    recall  f1-score   support

        negative      0.708     0.687     0.698       428
         neutral      0.355     0.328     0.341       229
        positive      0.726     0.777     0.751       444

        accuracy                          0.649      1101
       macro avg      0.597     0.597     0.596      1101
    weighted avg      0.642     0.649     0.645      1101

    CPU times: user 3h 26min 32s, sys: 1h 15min 19s, total: 4h 41min 51s
    Wall time: 39min 59s

## BERT fine-tuning with Hugging Face

The above experiments are quite successful – BERT gives us a reliable boost compared to other methods we've explored for the SST task. However, we might expect to do even better if we fine-tune the BERT parameters as part of fitting our SST classifier. To do that, we need to incorporate the Hugging Face BERT model into our classifier. This too is quite straightforward.

### HfBertClassifier

The most important step is to create an `nn.Module` subclass that has, for its parameters, both the BERT model and parameters for our own classifier. Here we define a very simple fine-tuning set-up in which some layers built on top of the output corresponding to `[CLS]` are used as the basis for the SST classifier:

```python
class HfBertClassifierModel(nn.Module):
    def __init__(self, n_classes, weights_name='bert-base-cased'):
        super().__init__()
        self.n_classes = n_classes
        self.weights_name = weights_name
        self.bert = BertModel.from_pretrained(self.weights_name)
        self.bert.train()
        self.hidden_dim = self.bert.embeddings.word_embeddings.embedding_dim
        # The only new parameters -- the classifier:
        self.classifier_layer = nn.Linear(
            self.hidden_dim, self.n_classes)

    def forward(self, indices, mask):
        reps = self.bert(
            indices, attention_mask=mask)
        return self.classifier_layer(reps.pooler_output)
```

As you can see, `self.bert` does the heavy-lifting: it reads in all the pretrained BERT parameters, and I've specified `self.bert.train()` just to make sure that these parameters can be updated during our training process.

In `forward`, `self.bert` is used to process inputs, and then `pooler_output` is fed into `self.classifier_layer`. Hugging Face has already added a layer on top of the actual output for `[CLS]`, so we can specify the model as

$$
\begin{align}
[h_{1}, \ldots, h_{n}] &= \text{BERT}([x_{1}, \ldots, x_{n}]) \\
h &= \tanh(h_{1}W_{hh} + b_{h}) \\
y &= \textbf{softmax}(hW_{hy} + b_{y})
\end{align}$$

for a tokenized input sequence $[x_{1}, \ldots, x_{n}]$.

The Hugging Face documentation somewhat amusingly says, of `pooler_output`,

> This output is usually _not_ a good summary of the semantic content of the input, you're often better with averaging or pooling the sequence of hidden-states for the whole input sequence.

which is entirely reasonable, but it will require more resources, so we'll do the simpler thing here.

For the training and prediction interface, we can subclass `TorchShallowNeuralClassifier` so that we don't have to write any of our own data-handling, training, or prediction code. The central changes are using `HfBertClassifierModel` in `build_graph` and processing the data with `batch_encode_plus`.

```python
del model
```

### HfBertClassifier experiment

That's it! Let's see how we do on the SST binary, root-only problem. Because fine-tuning is expensive, we'll conduct a modest hyperparameter search and run the model for just one epoch per setting evaluation, as we did when [assessing NLI models](nli_02_models.ipynb).

```python
def bert_fine_tune_phi(text):
    return text
```

```python
def fit_hf_bert_classifier_with_hyperparameter_search(X, y):
    basemod = HfBertClassifier(
        weights_name='bert-base-cased',
        batch_size=8,  # Small batches to avoid memory overload.
        max_iter=1,  # We'll search based on 1 iteration for efficiency.
        n_iter_no_change=5,   # Early-stopping params are for the final evaluation.
        early_stopping=True,
        device=device,
        verbose=True,
        display_progress=True)

    param_grid = {
        'gradient_accumulation_steps': [1, 4, 8],
        'eta': [0.00005, 0.0001, 0.001],
        'hidden_dim': [100, 200, 300]}

    bestmod = utils.fit_classifier_with_hyperparameter_search(
        X, y, basemod, cv=3, param_grid=param_grid)

    return bestmod
```

```python
%%time
bert_classifier_xval = sst.experiment(
    sst.train_reader(SST_HOME),
    bert_fine_tune_phi,
    fit_hf_bert_classifier_with_hyperparameter_search,
    assess_dataframes=sst.dev_reader(SST_HOME),
    vectorize=False)  # Pass in the BERT hidden state directly!
```

And now on to the final test-set evaluation, using the best model from above:

```python
optimized_bert_classifier = bert_classifier_xval['model']
```

```python
# Remove the rest of the experiment results to clear out some memory:
del bert_classifier_xval
```

```python
def fit_optimized_hf_bert_classifier(X, y):
    optimized_bert_classifier.max_iter = 1000
    optimized_bert_classifier.fit(X, y)
    return optimized_bert_classifier
```

```python
test_df = sst.sentiment_reader(
    os.path.join(SST_HOME, "sst3-test-labeled.csv"))
```

```python
%%time
_ = sst.experiment(
    sst.train_reader(SST_HOME),
    bert_fine_tune_phi,
    fit_optimized_hf_bert_classifier,
    assess_dataframes=test_df,
    vectorize=False)  # Pass in the BERT hidden state directly!
```

    Stopping after epoch 9. Validation score did not improve by tol=1e-05 for more than 5 epochs. Final error is 7.519404984079301

                  precision    recall  f1-score   support

        negative      0.756     0.825     0.789       912
         neutral      0.338     0.314     0.325       389
        positive      0.821     0.771     0.795       909

        accuracy                          0.713      2210
       macro avg      0.638     0.636     0.636      2210
    weighted avg      0.709     0.713     0.710      2210

    CPU times: user 13min 7s, sys: 19 s, total: 13min 26s
    Wall time: 13min 27s