### Recurrent Neural Network

Let us now move on to Recurrent Neural Network (RNN). Recurrent neural network is good in handling sequential data because they have a memory component which enables this network to remember past (few) information making it better for a model requiring varying length inputs and outputs.

>For example, consider the two sentences “I went to Nepal in 2009” and “In 2009,I went to Nepal.” If we ask a machine learning model to read each sentence and extract the year in which the narrator went to Nepal, we would like it to recognize the year 2009 as the relevant piece of information, whether it appears in the sixth word or the second word of the sentence. Suppose that we trained a feedforward network that processes sentences of ﬁxed length. A traditional fully connected feedforward network would have separate parameters for each input feature, so itwould need to learn all of the rules of the language separately at each position in the sentence.

- Ian Goodfellow, Yoshua Bengio, and Aaron Courville

In this post, we'll implement a simple RNN, applying it to the problem of [part-of-speech (POS) tagging](https://en.wikipedia.org/wiki/Part-of-speech_tagging) problem.

```julia
using PyPlot
```

### Understanding the Data

The data used in solving the POS tagging problem was taken from Graham Neubig's [NLP Programming Tutorial](http://www.phontron.com/teaching.php). Training dataset is a collection of sentences where each word already has a POS tag attached to it :

$$word1\_tag1\;word2\_tag2\;\;...\;\; .\_.$$

The function readWordTagData, reads such a file and outputs an array of :

- unique words
- unique tags
- whole dataset split into tuples containing word and its associated tag

Arrays of unique words and tags will help in [one-hot vector representation](http://lakshgupta.github.io/2015/07/12/VectorRepresentationofWords1/) which we'll feed to our neural network. In order to handle unknown words and unknown tags, I have also added $UNK\_W$ and $UNK\_T$ to these arrays.

```julia
# https://github.com/JuliaLang/julia/issues/14099
const spaces = filter(isspace, Char(0):Char(0x10FFFF));
```

```julia
function readWordTagData(filePath)
    file = open(filePath);
    vocabSet = Set{AbstractString}();
    tagSet = Set{AbstractString}();
    # read line
    for ln in eachline(file)
        word_tag = split(ln, spaces);
        # remove ""
        word_tag = word_tag[word_tag .!= ""]
        # separate word from tag
        for token in word_tag
            tokenSplit = split(token, "_");
            push!(vocabSet, tokenSplit[1]);
            push!(tagSet, tokenSplit[2]);
        end
    end
    close(file);
    # to handle unknown words
    push!(vocabSet, "UNK_W");
    # to handle unknown tags
    push!(tagSet, "UNK_T");
    #println(vocabSet)
    #println(tagSet)
    vocab = collect(vocabSet);
    tags = collect(tagSet);
    # prepare data array
    data = Tuple{AbstractString , AbstractString }[];
    file = open(filePath);
    # read line
    for ln in eachline(file)
        word_tag = split(ln, spaces);
        # remove ""
        word_tag = word_tag[word_tag .!= ""]
        # separate word from tag
        for token in word_tag
            tokenSplit = split(token, "_");
            push!(data, (tokenSplit[1], tokenSplit[2]));
        end
    end
    close(file);
    #println(length(data))
    return vocab, tags, data;
end
```

    readWordTagData (generic function with 1 method)

### Setting up RNN

Looking at a time step $t$, an RNN receives an input $x_t$ along with the hidden state($h_{t-1}$) computed in the previous time step ($t-1$). If you unroll the RNN over few time steps then it becomes easier to understand and train the network, similar to a feedforward neural network. Any network with connections making a cycle can be unrolled into a series of feedforward network representing each time step. All these time steps share the respective weights and biases over the network and our task would be to learn these parameters using the backpropagation algorithm. It'll become clear once we get to the code.

```julia
# read data
vocabListTrain, tagListTrain, dataTrain = readWordTagData("data/pos/wiki-en-train.norm_pos");
# define the network
inputLayerSize = length(vocabListTrain);
hiddenLayerSize = 100;
outputLayerSize = length(tagListTrain);
learningRate = 1e-3;
decayRate = 0.9;
epsVal = 1e-5;
# initialize weights and biases
Wxh = randn(inputLayerSize, hiddenLayerSize)*0.01; # input to hidden
Whh = randn(hiddenLayerSize, hiddenLayerSize)*0.01; # hidden to hidden
Bh = zeros(1, hiddenLayerSize); # hidden bias
Why = randn(hiddenLayerSize, outputLayerSize)*0.01; # hidden to output
By = zeros(1, outputLayerSize); # output bias
```

    1x43 Array{Float64,2}:
     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0

### Backpropagation Through Time (BPTT)

In order to learn the weights for our recurrent network, we have to generalize our backpropagation algorithm and apply it to our recurrent neural networks. This variant we are going to use is known as [backpropagation through time (BPTT)](http://mail.werbos.com/Neural/BTT.pdf) since it works on sequences in time.

In the forward process, we'll take one-hot vector representation of a word and will try to predict the normalized tag probability. For that we are using the tanh activation function at the hidden layer and the softmax at the output layer. Since our network is an RNN, at time step $1$ for the first sequence, the hidden state can be a zero matrix and in the next time step we can use whatever we have calculated in the previous time step i.e., the value of the last hidden state of the first sequence can be used at time step $1$ for the next sequence and so on.

$$a_t = W_{xh}*x_{t} + W_{hh}*h_{t-1} + B_h$$
$$h_t = tanh(a_t)$$
$$z_t = W_{hy}*h_t + B_y$$
$$\hat y_t = softmax(z_t)$$

We'll save the hidden state values and the normalized probabilities at the each time step to use during the backward process.

For each time step we define out cross entropy cost function as:

$$J_t(W_xh, W_hh, W_hy) = J_t(W) = - y_t* \log (\hat y_t)$$

Since our input is a sequence we define the cost for the whole sequence as:

$$J(W) = \sum_{t}J_t(W) = - \sum_{t} y_t* \log (\hat y_t)$$

```julia
function forwardRNN(x::Array{Vector{Float64},1}, y::Array{Vector{Float64},1},
    h::Array{Array{Float64,2},1}, p::Array{Vector{Float64},1}, hPrev::Array{Float64,2})

    global Wxh, Whh, Why, Bh, By;
    cost = 0;
    # for each time t in x
    # unrolling RNN -> Feedforward NN step
    for time in 1:length(x)
        if time == 1
            h[time] = tanh(x[time]'*Wxh + hPrev*Whh .+ Bh);
        else
            h[time] = tanh(x[time]'*Wxh + h[time-1]*Whh .+ Bh);
        end
        # output layer
        score = h[time]*Why .+ By;
        p_softmax = exp(score) / sum(exp(score));
        p[time] = vec(p_softmax); # output probability distribution (at time t)
        cost += -sum(log(y[time]'*p[time])) # assuming y is a one-hot vector
    end
    return cost;
end
```

    forwardRNN (generic function with 1 method)

In the backward process, we apply the chain rule as usual to backpropagate the errors and calculate the gradients. We already defined our cost function as :

$$J(W) = - \sum_{t} y_t* \log (\hat y_t)$$

Since all output units contribute to the error of each hidden unit we sum up all the gradients calculated at each time step in the sequence and use it to update the parameters. So our parameter gradients becomes :

$$\frac{\partial J_t}{\partial W_{hy}} = \sum_t \frac{\partial J}{\partial z_t}*\underbrace{\frac{\partial z_t}{\partial W_{hy}}}_{h_t}$$

$$\frac{\partial J_t}{\partial W_{hh}} = \sum_t \frac{\partial J}{\partial h_t}*\underbrace{\frac{\partial h_t}{\partial W_{hh}}}_{(1-h_t^2)*h_{t-1}}$$

$$\frac{\partial J_t}{\partial W_{xh}} = \sum_t \frac{\partial J}{\partial h_t}*\underbrace{\frac{\partial h_t}{\partial W_{xh}}}_{(1-h_t^2)*x_{t}}$$

where,

$$\frac{\partial J}{\partial z_t} = \underbrace{\frac{\partial J}{\partial J_t}}_{= 1} * \frac{\partial J_t}{\partial z_t}$$

From the [previous post](http://lakshgupta.github.io/2016/01/16/NeuralNetwork2/) we know that:

$$\frac{\partial J_t}{\partial z_t} = \hat y_t - y_t $$

Point to note here is that change in $z_t$ does not affect the cost at other time steps. But that is not the case with $h_t$. A change in $h$ at time step $t$ is going to change the cost at $t+1$, $t+2$ ...$T$. In other words, $h_t$ is going to affect $\hat y_t$ and $h_{t+1}$, which in turn will propagate forward and affect $h$ at other time steps.

So let us look at the end of the sequence, i.e., at time step $T$:
$$\begin{aligned}
\frac{\partial J}{\partial h_T} &= \frac{\partial J}{\partial z_T}*\frac{\partial z_T}{\partial h_T} \\
&= (\hat y_T - y_T)*W_{hy}
\end{aligned}
$$

and at other time step $t$ in the sequence,

$$\begin{aligned}
\underbrace{\frac{\partial J}{\partial h_t}}_{dh} &= \underbrace{\left (\frac{\partial J}{\partial z_t}*\frac{\partial z_t}{\partial h_t} \right )}_{a} + \underbrace{\left (\frac{\partial J}{\partial h_{t+1}}*\frac{\partial h_{t+1}}{\partial h_t}\right )}_{dhnext}\end{aligned}
$$

We can work out $a$ from the other equations above. In $dhnext$, the value of $\frac{\partial J}{\partial h_{t+1}}$ will backpropagate since we would have already calculated this at time step $(t+1)$ before coming on to the time step $t$. For the rest :
$$\frac{\partial h_{t+1}}{\partial h_t} = W_{hh}*(1-h_t^2)$$

A similar but a different way of working out the equations can be seen in Richard Sochers's [Recurrent Neural Network](http://cs224d.stanford.edu/lectures/CS224d-Lecture7.pdf) lecture slide.
 >

In the code we are also using gradient clipping to handle the gradient explosion problem. I first observed this being used in Andrej Karpathy's [min-char-rnn.py](https://gist.github.com/karpathy/d4dee566867f8291f086), and then in [this](http://arxiv.org/abs/1211.5063) paper.
>We propose a gradient norm
clipping strategy to deal with exploding gradients
and a soft constraint for the vanishing
gradients problem.

- Razvan Pascanu, Tomas Mikolov, Yoshua Bengio

```julia
function backwardRNN(x::Array{Vector{Float64},1}, y::Array{Vector{Float64},1},
    h::Array{Array{Float64,2},1}, p::Array{Vector{Float64},1}, hPrev::Array{Float64,2})

    global Wxh, Whh, Why, Bh, By;

    dWxh = zeros(size(Wxh));
    dWhh = zeros(size(Whh));
    dBh = zeros(size(Bh));
    dWhy = zeros(size(Why));
    dBy = zeros(size(By));

    dh = zeros(size(Bh)); # error from the following time step
    dhnext = zeros(size(h[1]));
    for time in length(x):-1:1
        # output layer error
        dy = p[time] - y[time]; #assuming y is a one hot vector
        # output gradient
        dWhy = dWhy + (dy * h[time])';
        dBy = dBy + dy';
        # backpropagate
        dh = (Why*dy)' + dhnext;
        dhRaw = (1 - (h[time].*h[time])) .* dh;
        # hidden layer gradient
        dWxh = dWxh + (x[time] * dhRaw);
        dBh = dBh + dhRaw;
        if time == 1
            dWhh = dWhh + (hPrev' * dhRaw);
        else
            dWhh = dWhh + (h[time-1]' * dhRaw);
        end
        dhnext = dhRaw*Whh;
    end
    # clip to mitigate exploding gradients
    dWxh = clamp(dWxh, -5, 5);
    dWhh = clamp(dWhh, -5, 5);
    dBh = clamp(dBh, -5, 5);
    dWhy = clamp(dWhy, -5, 5);
    dBy = clamp(dBy, -5, 5);

    return dWxh, dWhh, dBh, dWhy, dBy;
end
```

    backwardRNN (generic function with 1 method)

### Gradient Check

The implementation of backpropagation algorithm hasn't changed much over these series of posts but it's always good to keep a tool in hand to check whether our code is correct or not. We first go through the backpropagation step to calculate the analytical gradients. Then we perturb an element in the weight matrix and go through the forward step to calculate the cost calculating the numerical gradient as :

$$\begin{align}
\frac{d}{d\theta}J(\theta) = \lim_{\epsilon \rightarrow 0}
\frac{J(\theta+ \epsilon) - J(\theta-\epsilon)}{2 \epsilon}.
\end{align}$$

By comparing the relative error between the analytical gradient and the numerical gradient we can be sure that there is no bug in calculating the gradients. It should be very very small, less than $1e^{-5}$.

>Use relative error for the comparison. What are the details of comparing the numerical gradient $f′n$ and analytic gradient $f′a$? That is, how do we know if the two are not compatible? You might be tempted to keep track of the difference $∣fa′−fn′∣$ or its square and define the gradient check as failed if that difference is above a threshold. However, this is problematic. For example, consider the case where their difference is 1e-4. This seems like a very appropriate difference if the two gradients are about 1.0, so we'd consider the two gradients to match. But if the gradients were both on order of 1e-5 or lower, then we'd consider 1e-4 to be a huge difference and likely a failure. Hence, it is always more appropriate to consider the relative error :
>
$$\frac{\mid f'_a - f'_n \mid}{\max(\mid f'_a \mid, \mid f'_n \mid)}$$
>
which considers their ratio of the differences to the ratio of the absolute values of both gradients. Notice that normally the relative error formula only includes one of the two terms (either one), but I prefer to max (or add) both to make it symmetric and to prevent dividing by zero in the case where one of the two is zero (which can often happen, especially with ReLUs). However, one must explicitly keep track of the case where both are zero and pass the gradient check in that edge case. In practice :
>
- relative error > 1e-2 usually means the gradient is probably wrong
- 1e-2 > relative error > 1e-4 should make you feel uncomfortable
- 1e-4 > relative error is usually okay for objectives with kinks. But if there are no kinks (e.g. use of tanh nonlinearities and softmax), then 1e-4 is too high.
- 1e-7 and less you should be happy.
>
Also keep in mind that the deeper the network, the higher the relative errors will be. So if you are gradient checking the input data for a 10-layer network, a relative error of 1e-2 might be okay because the errors build up on the way. Conversely, an error of 1e-2 for a single differentiable function likely indicates incorrect gradient.

- Andrej Karpathy

Its been also suggested to:
- not perform the gradient check at the first iteration
- turn off regularization
- check some of the dimensions of the gradient and assume that the others are correct

```julia
# gradient checking
function gradCheck(inputs::Array{Vector{Float64},1}, targets::Array{Vector{Float64},1},
    h::Array{Array{Float64,2},1}, p::Array{Vector{Float64},1}, hPrev::Array{Float64,2})

    paramNameList = ["Wxh", "Whh", "Bh", "Why", "By"];
    # collect paramters
    global Wxh, Whh, Why, Bh, By;
    paramList = [x for x=(Wxh, Whh, Bh, Why, By)];
    num_checks = 2;
    delta = 1e-5;
    # collect parameter gradients
    cost = forwardRNN(inputs, targets, h, p, hPrev);
    dWxh, dWhh, dBh, dWhy, dBy = backwardRNN(inputs, targets, h, p, hPrev);
    dParamList = [x for x=(dWxh, dWhh, dBh, dWhy, dBy)];
    for (param,dparam,name) in zip(paramList, dParamList, paramNameList)
        # validate the size of the parameter and its gradient
        s0 = size(dparam);
        s1 = size(param);
        if s0 != s1
            println("Error dims dont match: ", s0," and ",s1);
        end
        println(name)
        for i in 1:num_checks
            ri = rand(1:length(param));
            # evaluate cost at [x + delta] and [x - delta]
            old_val = param[ri];
            param[ri] = old_val + delta;
            cg0 = forwardRNN(inputs, targets, h, p, hPrev);
            param[ri] = old_val - delta;
            cg1 = forwardRNN(inputs, targets, h, p, hPrev);
            param[ri] = old_val # reset old value for this parameter
            # fetch both numerical and analytic gradient
            grad_analytic = dparam[ri];
            grad_numerical = (cg0 - cg1) / ( 2 * delta );

            rel_error = abs(grad_analytic - grad_numerical) / abs(grad_numerical + grad_analytic);
            println(grad_numerical,", ", grad_analytic, " => ",rel_error);
            # rel_error should be on order of 1e-7 or less
            if rel_error > 1e-5
                error("Gradient check failed.");
            end
        end
        println("Gradient check passed.")
    end
end
```

    gradCheck (generic function with 1 method)

I ran the gradient check for a sample dataset and used the sequence length of 1. This operation is commented out in the code below because it is very computation expensive. The train method loops over each sequence, and updates the weights/biases matrices using the backpropagation algorithm to calculate the gradients for that sequence.

```julia

function train(data::Array{Tuple{AbstractString,AbstractString},1}, vocabList::Array{AbstractString,1}
    , tagList::Array{AbstractString,1}, numItr::Int64, seqLength::Int64, sampleCostAtItr::Int64)

    global Wxh, Whh, Why, Bh, By;
    numIterations =  numItr * length(data);
    costList = []; # store cost per sampled iteration
    ptr = 1;
    p = [zeros(length(tagList)) for i in 1:seqLength];
    h = Array{Float64,2}[zeros(1,hiddenLayerSize) for i in 1:seqLength]; # hidden layers (at time t)
    hPrev = zeros(1, hiddenLayerSize);
   for itr in 1:numIterations
        # take care of the sequence length
        if ptr+seqLength-1 > length(data) # whenever we are looking at the data from the start
            # reset RNN memory
            hPrev = zeros(1, hiddenLayerSize);
            # go from start of data
            ptr = 1
        end
        # generate sequence
        seqData = data[ptr:ptr+seqLength-1];
        x = Vector{Float64}[];
        y = Vector{Float64}[];
        for word_tag in seqData
            word = word_tag[1];
            tag = word_tag[2];
            # convert to one-hot vectors
            # words
            wordVec = zeros(length(vocabList));
            findWord = collect(vocabList.==word)
            if length(findWord[findWord.==true]) == 0
                # unknown word: UNK_W
                findWord[length(findWord)] = true;
            end
            wordVec[findWord] = 1;
            # tags
            tagVec = zeros(length(tagList));
            findTag = collect(tagList.==tag)
            if length(findTag[findTag.==true]) == 0
                # unknown tag: UNK_T
                findTag[length(findTag)] = true;
            end
            tagVec[findTag] = 1;
            # push to the sequence
            push!(x , wordVec);
            push!(y , tagVec);
        end
        # gradient check
        #gradCheck(x, y, h, p, hPrev);

        # feedforward
        cost = forwardRNN(x, y, h, p, hPrev);
        # sample cost
        if itr%sampleCostAtItr == 1
            push!(costList, cost);
        end
        # backpropagate
        dWxh, dWhh, dBh, dWhy, dBy = backwardRNN(x, y, h, p, hPrev);

        # update weights
        Wxh += -learningRate * dWxh;
        Whh += -learningRate * dWhh;
        Bh += -learningRate * dBh;
        Why += -learningRate * dWhy;
        By += -learningRate * dBy;

        # previous output as hidden input for the next sequence
        hPrev = h[length(h)];

        # move data pointer
        ptr += seqLength;
    end
    return costList;
end
```

    train (generic function with 1 method)

We'll run the training step for the whole data 20 times, using an input sequence of size 2. Since there will be lot of iteration we'll sample the cost at every 1000 interation.

```julia
# MAIN
# number of steps to unroll the RNN for
seqLen = 2
# run through the data n times
numIterOverData = 20;
# sample cost after each n iteration
sampleCostAtEveryItr = 1000;
J = train(dataTrain, vocabListTrain, tagListTrain, numIterOverData, seqLen, sampleCostAtEveryItr);
```

```julia
# plot the cost per iteration
sampleIdxJ = [1+sampleCostAtEveryItr*i for i in 0:length(J)-1]
plot(sampleIdxJ, J)
xlabel("Sampled Iterations")
ylabel("Cost")
grid("on")
```

The cost in the graph seems to be decreasing with respect to the iteration. The cost is not smoothly decreasing because we are kind of running a mini-batch gradient descent hence it is just an estimate of the cost over the whole dataset. It's in my TODO list to investigate it more thoroughly some day since the graph looks a bit ugly. For now, let's check how it is performing. To find the accuracy, we'll use the learned model and go through the forward step to make the prediction.

```julia
function findAccuracy(data::Array{Tuple{AbstractString,AbstractString},1}, vocabList::Array{AbstractString,1}
    , tagList::Array{AbstractString,1}, seqLength::Int64)

    correct = 0;
    ptr=1;
    p = [zeros(length(tagList)) for i in 1:seqLength];
    h = Array{Float64,2}[zeros(1,hiddenLayerSize) for i in 1: seqLength]; # hidden layers (at time t)
    hPrev = zeros(1,hiddenLayerSize);
    for i in 1:length(data)/seqLength

        # prepare inputs (we're sweeping from left to right in steps seq_length long)
        if ptr+seqLength-1 > length(data) # whenever we are looking at the data from the start
            break # return if data is alread read
        end
        # generate sequence
        seqData = data[ptr:ptr+seqLength-1];
        x = Vector{Float64}[];
        y = Vector{Float64}[];
        for word_tag in seqData
            word = word_tag[1];
            tag = word_tag[2];
            # convert to one-hot vectors
            # words
            wordVec = zeros(length(vocabList));
            findWord = collect(vocabList.==word)
            if length(findWord[findWord.==true]) == 0
                # unknown word: UNK_W
                findWord[length(findWord)] = true;
            end
            wordVec[findWord] = 1;
            # tags
            tagVec = zeros(length(tagList));
            findTag = collect(tagList.==tag)
            if length(findTag[findTag.==true]) == 0
                # unknown tag: UNK_T
                findTag[length(findTag)] = true;
            end
            tagVec[findTag] = 1;
            # push to the sequence
            push!(x , wordVec);
            push!(y , tagVec);
        end
        # feedforward
        cost = forwardRNN(x, y, h, p, hPrev);
        hPrev = h[size(h,1)];

        prediction = [indmax(p[j]) for j in 1:length(seqData)];
        truth = [indmax(y[j]) for j in 1:length(seqData)];
        # accuracy
        for j in 1:length(seqData)
            if truth[j] == prediction[j]
                correct = correct + 1;
            end
        end
        ptr += seqLength; # move data pointer
    end
    accuracy = correct/length(data)*100;
    return accuracy;
end
```

    findAccuracy (generic function with 1 method)

```julia
accuracy = findAccuracy(dataTrain, vocabListTrain, tagListTrain, seqLen);
println("accuracy: ", accuracy);
```

    accuracy: 90.99910251585072

### Test data

The test data consists of two files, one with regular sentences in English and other with their corresponding tags. I am just mapping the words and tags to produce an array of tuples similar to what we used in the training step.

```julia
function readData(filePath)
    file = open(filePath);
    # read line
    wordList = [];
    for ln in eachline(file)
        words = split(ln, spaces);
        # remove ""
        words = words[words .!= ""]
        # append to the list
        wordList = [words; wordList];
    end
    close(file);
    return wordList;
end
```

    readData (generic function with 1 method)

```julia
# read test data
wordListTest = readData("data/pos/wiki-en-test.norm");
tagListTest = readData("data/pos/wiki-en-test.pos");
dataTest = Tuple{AbstractString , AbstractString }[];

for i in 1:length(wordListTest)
  push!(dataTest, (wordListTest[i], tagListTest[i]));
end
```

```julia
# will use vocab list and tag list created at the training time
accuracy = findAccuracy(dataTest, vocabListTrain, tagListTrain, seqLen);
println("accuracy: ", accuracy);
```

    accuracy: 85.1194389655928

As you can see, RNN performed quite well on both the train and the test data set. But RNN has it own problems. It suffers from vanishing/exploding gradient problem.

>As introduced in Bengio et al. (1994), the exploding
gradients problem refers to the large increase in the
norm of the gradient during training. Such events are
caused by the explosion of the long term components,
which can grow exponentially more then short term
ones. The vanishing gradients problem refers to the
opposite behaviour, when long term components go
exponentially fast to norm 0, making it impossible for
the model to learn correlation between temporally distant
events.

- Razvan Pascanu, Tomas Mikolov, Yoshua Bengio

In order to tackle these problems, people have come up with few other variants of RNN such as [LSTM (Long-Short Term Memory)](https://en.wikipedia.org/wiki/Long_short-term_memory) and GRU (Gated Recurrent Unit). Maybe I'll cover some of them in the coming posts. Till then, keep it constructive and enjoy!

References:

- [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
- [Recurrent	Neural	Networks - cs224d](http://cs224d.stanford.edu/lectures/CS224d-Lecture7.pdf)
- [Sequence Modeling: Recurrent and Recursive Nets](http://www.deeplearningbook.org/contents/rnn.html)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](http://arxiv.org/abs/1506.00019)
- [Backpropagation Through Time: What it does and how to do it](http://mail.werbos.com/Neural/BTT.pdf)
- [A guide to recurrent neural networks and backpropagation](http://faculty.ksu.edu.sa/ghulam/Documents/Downloads/RecurrentNeuralNetwork.pdf)
- [Lec [5.1]: Deep Learning, Recurrent neural network](https://youtu.be/AvyhbrQptHk)
- [Training RNNs](http://www.cs.toronto.edu/~rgrosse/csc321/lec10.pdf)
- [On the difficulty of training Recurrent Neural Networks
](http://arxiv.org/abs/1211.5063)
- [Deep Learning Lecture 12: Recurrent Neural Nets and LSTMs](https://youtu.be/56TYLaQN4N8)
- [NLP Programming Tutorial](http://www.phontron.com/slides/nlp-programming-en-08-rnn.pdf)
- [Statistical Language Models based on Neural Networks](http://www.fit.vutbr.cz/~imikolov/rnnlm/thesis.pdf)
- [A Simple Way to Initialize Recurrent Networks of Rectified Linear Units](http://arxiv.org/abs/1504.00941)


#========================================================================
	# Copyright 2019 Science Technology Facilities Council
	# Copyright 2019 University of Manchester
	#
	# This work is part of the Core Imaging Library developed by Science Technology
	# Facilities Council and University of Manchester
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	#         http://www.apache.org/licenses/LICENSE-2.0.txt

	#
	# Unless required by applicable law or agreed to in writing, software
	# distributed under the License is distributed on an "AS IS" BASIS,
	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	# See the License for the specific language governing permissions and
	# limitations under the License.
	#
	#=========================================================================

Multi-Channel Reconstruction

Learning Objectives

By the end of this notebook, you will be able to:

- Identify the key differences in building Image/Acquisition Geometries and Operators for multi-channel datasets
- Build your own reconstructions using FDK, CGLS and PDHG
- Determine optimum regularisation parameters based on reconstruction method
- Evaluate the effectiveness of each reconstruction routine using energy profiles and elemental maps.

### Prerequisites:

- Acquisition/Image Geometry, Acquisition Data
- AstraProjectorMC/3DMC
- FDK, CGLS, PDHG, TV
- BlockFramework

### Background:

Conventional X-ray detectors only measure the variable density of objects they pass through, giving no insight into what materials are actually inside. This is because standard detectors measure only the number of photons that arrive at each point on the detector.

For multi-channel imaging, one can use an energy-sensitive X-ray detector, which measures the energy of every X-ray photon that arrives at each individual pixel. This provides an additional layer of information which can provide important insight on a sample's composition or structure. However, adapted reconstruction routines are required to account for the extra energy-based dimension.

The additional energy dimension is stored as a histogram of energy 'channels', indicating the number of X-ray photons detected by a pixel within a fine energy range. Typically 200+ channels are acquired, however in order to speed up computation time, we will restrict our dataset to just 40 channels, where the dominant energy signals are known to appear.

```python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from utilities import islicer, link_islicer
from utilities.show_utilities import channel_to_energy, show

from ccpi.framework import ImageGeometry, AcquisitionGeometry, AcquisitionData, ImageData, BlockDataContainer

import numpy as np
import matplotlib.pyplot as plt
import h5py
import os
import sys

from ccpi.io import NEXUSDataReader
from ccpi.optimisation.algorithms import PDHG, CGLS

from ccpi.optimisation.operators import BlockOperator, Gradient
from ccpi.optimisation.functions import L2NormSquared, KullbackLeibler,\
                                MixedL21Norm, BlockFunction, IndicatorBox

from ccpi.astra.utils import *
from ccpi.astra.processors import FBP
from ccpi.astra.operators import AstraProjectorMC, AstraProjector3DMC
```

```python
from ccpi.framework.TestData import data_dir
pathname = data_dir
filename = 'sinogram_centered_channels100_140.h5'

path = os.path.join(pathname , filename)
arrays = {}

with h5py.File(path, 'r+') as f:
    for k, v in f.items():
        arrays[k] = np.array(v)
    X = arrays['SC']
```

The sample we will look at in this notebook is an iodine-stained lizard head. The use of elemental staining is common in biology and medicine, by acting as a contrast agent to provide improved visibility of internal structures for X-ray imaging. Iodine is a popular choice in the clinical and research fields, due to its energy-based properties falling in the typical range for diagnostic X-ray imaging.

The sample was scanned in a lab-based X-ray CT cone-beam system at 50keV, 0.8W, using an energy-sensitive detector to acquire a full 4D dataset. The detector consisted of an 80 x 80 pixel array, with pixel sizes of 250 $\mu$m x 250 $\mu$m. A source-sample distance of 233.0 mm and a sample-detector distance of 245.0 mm gave a geometric magnification of 2.05x. The sample was scanned for 60 projections over a full 360$^{\circ}$ rotation, with 60s exposure time per projection.

A diagram of the style of setup used for spectral imaging is shown below from a paper by [C.K.Egan *et al*, 2015](https://www.nature.com/articles/srep15979#):

As shown by the diagram, each pixel stores its own energy channel histogram, with characteristic signals such as 'Absorption edges' (caused by photoelectric absorption of X-ray photons by the iodine in our sample) producing sharp rises in signal. These are the key features we look for when analysing spectral data.

Setting up Acquisition/Image Geometry

First we need to setup the geometries based on the acquisition system.
These are currently ordered based on the 4D raw dataset.

Run the code to see the current dimensions.

```python
print(X.shape)
```

We can allocate these separately

```python
num_channels = X.shape[0]
num_pixels_h = X.shape[3]
num_pixels_v = X.shape[1]
num_angles = X.shape[2]
```

We set the angles used based on the parameters chosen for data acquisition. An offset is also applied (this just gives us the freedom to adjust the rotation of our reconstructed images).

```python
angles = np.linspace(-180-55,180-55,num_angles,endpoint=False)*np.pi/180
```

We encode all this information into `AcquisitionGeometry` and `ImageGeometry`.

Recall that the Geometry for scanning with a cone-beam X-ray source is defined by the following parameters:
(See **Notebook00_Building_Blocks** for more information)

```python
# Define acquisition geometry from the detector system acquisition settings
# with reordering of the dimension labels to match the raw data

distance_source_center = 233.0  # [mm]
distance_center_detector = 245.0  # [mm]
detector_pixel_size = 0.25  # [mm]

ag = AcquisitionGeometry('cone',
                         '3D',
                         angles,
                         pixel_num_h=num_pixels_h,
                         pixel_size_h=detector_pixel_size,
                         pixel_num_v=num_pixels_v,
                         pixel_size_v=detector_pixel_size,
                         dist_source_center=distance_source_center,
                         dist_center_detector=distance_center_detector,
                         channels=num_channels,
                         dimension_labels = ['channel', 'vertical', 'angle', 'horizontal'])

# Create the 4D acquisition data
data = ag.allocate()
data.fill(X)

# Calculate the geometric magnification to scale the voxel size relative
# to the detector pixel size.
mag = (ag.dist_source_center + ag.dist_center_detector)/ag.dist_source_center

# Define Image Geoemtry
ig = ImageGeometry(voxel_num_x=ag.pixel_num_h,
                     voxel_num_y=ag.pixel_num_h,
                     voxel_num_z=ag.pixel_num_h,
                     voxel_size_x=ag.pixel_size_h/mag,
                     voxel_size_y=ag.pixel_size_h/mag,
                     voxel_size_z=ag.pixel_size_h/mag,
                     channels=num_channels)
```

```python
print(data)
```

We can use an interactive image slicer `islicer` to provide a visualisation of the `AcquisitionData` in any dimension below. Simply by taking a data subset in a particular dimension, we can then visualise the data in any other given dimension.

Run the code below to see three such examples:

1) Projection radiographs for each of the 60 rotation angles acquired in a single channel

2) The sinogram for each energy channel for the central slice subset

3) The spectral signals acquired in each energy channel for a single projection angle

**Note: You can adjust the look of your reconstructions by varying certain parameters**
 - by removing `cmap`, you return to the default colour map of 'gray'
 - the default scaling runs from 0.0 through the full data range, vary this using e.g. `minmax = (0.0 , 2.0)`

```python
islicer(data.subset(channel=20), direction='angle', title = 'Projection Angle', cmap='inferno')
islicer(data.subset(vertical=40), direction='channel', title = 'Sinogram Channel', cmap='inferno')
islicer(data.subset(angle=40), direction='channel', title = 'Channel', cmap='inferno')
```

We setup the tomography operator for 3D multi-channel data using the `AcquisitionGeometry` and `ImageGeometry`

```python
A3DMC = AstraProjector3DMC(ig, ag)
```

# FDK Reconstruction

One of the simplest, and most common, means of image reconstruction for X-ray CT is the use of Filtered BackProjection (FBP). In the case of many lab-based X-ray sources, which utilise a cone-beam rather than parallel- or fan-beams, we use a specific case of FBP: The [Feldkamp-Davis-Kress (FDK)](https://www.osapublishing.org/josaa/abstract.cfm?uri=josaa-1-6-612) algorithm.

The function `FBP` is capable of handling reconstructions for both parallel-beam and cone-beam geometries in 2D and 3D. When supplied with both Acquisition and Image Geometry (`ag`, `ig`), the function recognises and performs the appropriate form of FBP, in this case FDK.
Run the code below to see a reconstruction using the FDK algorithm.  Here we use a `ram-lak` filter as part of the reconstruction.

```python
# Setup geometry for FDK
fdk = FBP(ig, ag, 'ram-lak')
# Provide dataset
fdk.set_input(data)
# Store results
recon = fdk.get_output()
# Check dimensions
print('Shape: {}'.format(recon.shape))
```

```python
recon.dimension_labels
```

We can see we now have three spatial dimensions to our reconstructed data (each of size 80), along with our 40 energy channels. By selecting a single slice of a spatial dimension, we can view the resulting 80x80 reconstructed image for each energy channel using `islicer`.

```python
# Show results
islicer(recon.subset(vertical=46),direction='channel',
        title='FDK: Channel', cmap='inferno', minmax=(0.0,0.4))
```

While some features of the lizard head can be seen in the reconstructed images, much of the signal is shrouded by noise.
In the next section, we will explore the first iterative algorithm - CGLS.

## Running the CGLS algorithm on a 4D dataset

As the next step, we will begin with a standard CGLS algorithm, applied to our 4D dataset

We initialise the operator based on the dimensions of the 4D dataset

Let's test the CGLS algorithm for just 10 iterations

```python
# We initialise
x_init = A3DMC.volume_geometry.allocate()

# Run the CGLS for 10 iterations
cgls = CGLS(x_init = x_init, operator = A3DMC, data = data, max_iteration = 100)
cgls.run(10)
```

We can use custom-made functions to visualise the quality of our reconstructions. Here we are looking at three commonly used views of the reconstructed dataset (axial, coronal and sagittal), and how these vary for different energy channels. Here we use the `show` function for multi-channel datasets, providing parameters to vary:
 - Title, Figure/font size
 - Channel number you wish to view (Currently takes one value for 4D data)
 - Colour map
 - Min/max colour map scaling

Run the code below to see reconstructions for the 10th, 20th and 30th channel in our truncated dataset, with the X-ray energies these correspond to.

```python
# Plot axial, coronal and sagittal views
# Plotter automatically converts channel number to energy
show(cgls.get_output(), title='CGLS 4D Reconstruction', show_channels=10,
     cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.3))
show(cgls.get_output(), title='CGLS 4D Reconstruction', show_channels=20,
     cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.3))
show(cgls.get_output(), title='CGLS 4D Reconstruction', show_channels=30,
     cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.3))
```

As a result of performing our reconstruction, we change the shape and dimensions of our 4D dataset, as we now have 3 spatial dimensions along with the energy channels.
Run the following code to see the new shape and corresponding dimension labels.

```python
print('Shape = ', cgls.get_output().shape)
print('Labels = ', cgls.get_output().dimension_labels)
```

You can use these labels to once more explore visualisations with the islicer function. Try varying the subset dimension and the direction using the label names above.
e.g. try:
 `islicer(cgls.get_output().subset(vertical=46),direction='channel',title='Axial: Channel', cmap='inferno', minmax = (0.0,0.4))`

```python
# Examples slicer
islicer(cgls.get_output().subset(vertical=46),direction='channel',
        title='Axial View: Channel', cmap='inferno', minmax = (0.0,0.4))
```

Now we can see how the visualisation changes if we increase the number of iterations. Using the `show` function, we can visualise the reconstructed image slices at chosen iteration intervals. We'll run it for 30 iterations.

```python
# Initialise
x_init = A3DMC.volume_geometry.allocate()

# Set up max iterations and step size
max_iter = 30
step = 10

# Set up algorithm
cgls2 = CGLS(x_init = x_init, operator = A3DMC, data = data,
             max_iteration = max_iter, update_objective_interval = 10)

for i in range(0, max_iter // step):
    cgls2.run(step)

    # get and visusualise the results
    cgls_out = cgls2.get_output()
    show(cgls_out, title='Iteration {},'.format((i+1) * step) + ' Objective Function: {}'.format(cgls2.loss[-1]) + '\n',\
         show_channels=20,cmap='inferno', figure_size=[15,6], font_size=[25,20], minmax=(0.0,0.4))
```

These images highlight the instability of a basic CGLS algorithm, where the **number of iterations** effectively acts as the algorithm's **regularisation parameter**. As a result, too many iterations leads to a divergence from an optimum solution, and while the objective function continues to reduce, only additional noise is being contributed.

In the next section, we will look at a different algorithm: Primal-Dual Hybrid Gradient (PDHG), combined with a Total Variation (TV) regularising factor, to see if this improves our reconstructed data quality.

## Total Variation Regularisation in 4D Volume using PDHG

The PDHG algorithm and Total Variation regularisation were covered extensively in **Notebook_04_PDHG_Tomography**, but below gives a brief recap of the basics behind each.

### Recap on PDHG

PDHG aims to solve problems of the form:

$$ \begin{equation} \min_{u} \mathcal{F}(K u) + \mathcal{G}(u) \label{min_problem} \end{equation} $$

In order to setup and run PDHG, we need to define the following:

 - The operator $K$.
 - The function $\mathcal{F}$ and $\mathcal{G}$.
 - Step-sizes $\sigma$ and $\tau$ such that $\sigma\tau\|K\|^{2}<1$.

Then we can setup PDHG:

`pdhg = PDHG(f = F, g = G, operator = K, tau = tau, sigma = sigma, max_iteration = maxiter)`

### Applying Total Variation Regularisation

For this notebook, we will be applying Total Variation (TV) regularisation to our PDHG algorithm. TV is a non-smooth regulariser,

$$ \underset{u}{\operatorname{argmin}} \alpha\,\mathrm{TV}(u) + \frac{1}{2} \| \mathcal{A} u - g\|^{2}_{2} + \mathbb{I}_{\{u>0\}}(u) $$

where,

- The Total Variation is taken as a `MixedL21Norm()`, such that $$\mathrm{TV}(u) = \|\nabla u \|_{2,1} = \sum \sqrt{ (\partial_{y}u)^{2} + (\partial_{x}u)^{2} }$$
- $g$ is the Acquisition data obtained from the detector
- $\mathcal{A}$ is the projection operator ( Radon transform ) that maps from an image-space to an acquisition space, i.e.,
$\mathcal{A} : X \rightarrow Y$.
- $\alpha$ is the regularising parameter that measures a trade-off between the fidelity and the regulariser terms
- $\mathbb{I}_{\{u>0\}}(u) : =
\begin{cases}
0, \mbox u>0\\
\infty, \mbox otherwise
\quad
\end{cases}
$ is a positivity constraint for the minimiser, $u$.

## Setting up PDHG for TV

We define K as a `BlockOperator`, containing the Gradient and Projection operator:

$$ K =
\begin{bmatrix}
\nabla\\
A
\end{bmatrix}
$$
K = `BlockOperator(Gradient, A)`

The function $\mathcal{F}$, is a `BlockFunction` with

a function $\alpha\|\cdot\|_{2,1}\quad\Longleftrightarrow\quad$ `MixedL21Norm()` term that represents the Total variation regularisation ,

a function $\|\cdot -g \|_{2}^{2}\quad\Longleftrightarrow\quad$ `L2NormSquared(data)` term that represents the data fitting.

Hence, $\mathcal{F} = [f_{1}, f_{2}] \quad\Longleftrightarrow\quad $ `F = BlockFunction(MixedL21Norm(), L2NormSquared(data))`

Finally, we have the function $\mathcal{G} = \mathbb{I}_{\{u>0\}}(u) \quad\Longleftrightarrow\quad$ `G = IndicatorBox(lower=0)`

Again, we can verify that with the above setting we can express our problem into this form, for $x=u$

$$ \begin{align} \underset{u}{\operatorname{argmin}}\alpha\|\nabla u\|_{2,1} + \frac{1}{2}\|\mathcal{A} u - g\|^{2}_{2} + \mathbb{I}_{\{u>0\}}(u) \\= \underset{u}{\operatorname{argmin}} f_{1}(\nabla u) + f_{2}(\mathcal{A}u) + \mathbb{I}_{\{u>0\}}(u) \\ = \underset{u}{\operatorname{argmin}} \mathcal{F}( \begin{bmatrix} \nabla \\ \mathcal{A} \end{bmatrix}u) + \mathbb{I}_{\{u>0\}}(u)\\ = \underset{u}{\operatorname{argmin}} \mathcal{F}(Ku) + \mathcal{G}(u)\\ = \underset{x}{\operatorname{argmin}} \mathcal{F}(Kx) + \mathcal{G}(x) \end{align} $$

```python
# The operator K is a Block Operator that contains the gradient and the tomography operator
op1 = Gradient(ig)
op2 = A3DMC

# Set up a BlockOperator K
K = BlockOperator(op1, op2, shape=(2,1))

```

$\mathcal{F}$ is a `BlockFunction` of a fidelity term and our TV regularising term:

- $f_{1}$ = $\alpha\|\cdot\|_{2,1}\quad\Longleftrightarrow\quad$ `MixedL21Norm()` term that represents the Total variation regularisation ,

- $f_{2}$ = $\|\cdot -g \|_{2}^{2}\quad\Longleftrightarrow\quad$ `L2NormSquared(data)` term that represents the data fitting.

Therefore as $f_{1}$ and $f_{2}$ act on each element of $K$, we end up with
$$ \mathcal{F}(Ku) = \mathcal{F}(
\begin{bmatrix}
\nabla u \\
A u
\end{bmatrix}) = ( f_{1}(\nabla u), f_{2}(Au) ) $$

```python
alpha = 0.05
f1 = alpha * MixedL21Norm()
f2 = 0.5 * L2NormSquared(b = data)

F = BlockFunction(f1, f2)
```

Next, we have our positivity constraint function $\mathcal{G} = \mathbb{I}_{\{u>0\}}(u) \quad\Longleftrightarrow\quad$ `G = IndicatorBox(lower=0)`.

```python
G = IndicatorBox(lower = 0)
```

Finally, we compute the operator norm of $K$ (`normK = operator.norm()`), and set our step sizes, $\sigma$ and $\tau$.

```python
# Compute the operator norm for K
normK = K.norm()

# Define the step sizes sigma and tau
sigma = 1
tau = 1/(sigma*normK**2)
```

Now we can setup and run the PDHG algorithm. Here it will run for 100 iterations.

Due to the increased complexity of the algorithm, combined with the reconstruction of a 4D dataset, the runtime for 100 iterations alone will be around **4 minutes**. However, we may require many more than 100 iterations to reach an optimum solution. Further, we may wish to reconstruct hundreds of energy channels, rather than just the 40 we use here.

Therefore, consideration must be taken on the number of iterations you perform based on your choice of algorithm and size of your dataset.

**Scroll past the cell below** if you wish to save yourself 4 minutes!
We have provided a final reconstruction after 1000 iterations of PDHG, which required around 40 minutes of runtime.

```python
pdhg_S_100 = PDHG(f = F, g = G, operator = K, tau = tau, sigma = sigma,
            max_iteration = 100, update_objective_interval = 25)
pdhg_S_100.run()

show(pdhg_S_100.get_output(), title='PDHG TV 100 Iterations',
     show_channels=20, cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.4))
```

### 1000 Iterations

We load in the 1000 iteration result using the `NEXUSDataReader()` we saw in **Notebook 03_3D_Diamond_dataset**.

```python
# Load in 1000 iteration dataset
reader = NEXUSDataReader()
reader.set_up(nexus_file = 'pdhg_S_1000.nxs')
pdhg_S_1000 = reader.load_data()

# Show the result
show(pdhg_S_1000, title='PDHG TV 1000 Iterations',
     show_channels=20, cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.4))
```

## Exercise 1: Change correlation to Space-Channel

We can make some adjustments to our framework to see how these affect the reconstruction.
Our gradient component is capable of smoothing our reconstruction both in the spatial domain, but also in the energy domain, by comparing reconstructions to its neighbouring channel.

Try changing the gradient component of our operator, $\nabla$, so that it is correlated in both the spatial and energy channel domain. We can do this by simply setting:

  `op1 = Gradient(ig, correlation='SpaceChannels')`

Keep the tomography operator, $A$ the same.

There are three numerical parameters that have an impact on the resulting reconstruction: alpha ($\alpha$), sigma ($\sigma$) and tau ($\tau$). For our original PDHG reconstruction, these optimum parameters were:
$\alpha$ = 0.05,
$\sigma$ = 1,
$\tau$ = `1/(sigma*normK**2)`.

For this test, we'll adjust our parameters slightly to:
$\alpha$ = 0.07,
$\sigma$ = 1,
$\tau$ = `1/(sigma*normK**2)`.

See if you notice any difference when you visualise the new reconstruction.

```python
# Setup BlockOperator, K
op1 = Gradient(ig, correlation = "SpaceChannels")
op2 = A3DMC

K = BlockOperator(op1, op2, shape=(2,1) )

# Define regularising parameter
alpha = 0.07

# Setup BlockFunction, F
f1 = alpha * MixedL21Norm()
f2 = 0.5 * L2NormSquared(b = data)
F = BlockFunction(f1, f2)

# Positivity Constraint Function
G = IndicatorBox(lower = 0)

# Compute the operator norm for K
normK = K.norm()

# Define the step sizes sigma and tau
sigma = 1
tau = 1/(sigma*normK**2)
```

Once you've made these changes, have a go at running the reconstruction and see what influence your changes
had.

```python
pdhgSC = PDHG(f = F, g = G, operator = K, tau = tau, sigma = sigma,
             max_iteration = 100, update_objective_interval = 25)
pdhgSC.run()
```

```python
show(pdhgSC.get_output(), title='PDHG TV Reconstruction', show_channels=20,
     cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.4))
```

We can directly compare the two types of PDHG reconstruction using `islicer`. As we move across energy channels, see if there are any major differences between the constructions:
 1) PDHG TV using a Spatial correlation for 1000 iterations
 2) PDHG TV using a Space-Channel correlation for 100 iterations

**Note:** If you also ran the earlier code for 100 iterations of PDHG TV using a Spatial correlation, feel free to add in an extra line to compare this too:
`i3 = islicer(pdhg_S_100.get_output().subset(vertical=40),direction='channel',title='100 Iteration Space Corr.: Channel', cmap='inferno', minmax = (0.0,0.4))`

```python
i1 = islicer(pdhg_S_1000.subset(vertical=46),direction='channel',
             title='1000 Iteration Space Corr.: Channel', cmap='inferno', minmax = (0.0,0.4))
i2 = islicer(pdhgSC.get_output().subset(vertical=46),direction='channel',
             title='100 Iteration Space-Channel Corr.: Channel',cmap='inferno', minmax = (0.0,0.4))
#i3 = islicer(pdhg_S_100.get_output().subset(vertical=46),direction='channel',
#             title='100 Iteration Space Corr.: Channel', cmap='inferno', minmax = (0.0,0.4))
link_islicer(i1,i2)
```

# Spatial/Energy Profiles

One way we can directly observe the effect of applying Spatial/Energy Correlation to our algorithms is through the use of pixel profiles across our reconstructed images in each domain respectively.

By plotting signal values across spatial pixels, or across energy channels, the smoothing effect of this correlation feature is seen. We will demonstrate each of these below.

## Extracting Spatial Profiles

By identifying a region of our reconstructed slices were the Iodine signals are strong, we can plot along the pixels in this region and compare the line profiles for each algorithm covered here.

Given the strong signals given by the eyes of the lizard head sample, we will look across this pixel row for a single channel, given by subset values of:
 - `vertical = 46`
 - `horizontal_y = 33`
 - `channel = 20`

**Feel free to adjust these values in the cell below if you wish to explore different regions of the reconstructed images.**

```python
plt.figure(figsize=[12,8])

# Extract line profiles for each algorithm
plt.plot(recon.subset(vertical = 46, horizontal_y = 33, channel = 20).as_array())
plt.plot(cgls.get_output().subset(vertical = 46, horizontal_y = 33, channel = 20).as_array())
plt.plot(pdhg_S_1000.subset(vertical = 46, horizontal_y = 33, channel = 20).as_array())
plt.plot(pdhgSC.get_output().subset(vertical = 46, horizontal_y = 33, channel = 20).as_array())

# Label and add key
plt.xlabel('Pixels X', fontsize=20)
plt.ylabel('Optical Density (no units)', fontsize=20)
plt.legend(labels=['FDK', 'Simple CGLS', 'Space Corr. PDHG TV', 'Space-Channel Corr. PDHG TV'])
```

We can see the reduced noise fluctuations for the Space-correlated PDHG algorithms, compared to the significant noise seen in both FDK and CGLS reconstructions.

## Extracting Energy Profiles

Once we have chosen and optimised our reconstruction routine, we can exploit this additional, energy dimension further by identifying the spectral signals that lie within.

First we can plot out energy profiles for any region-of-interest in our reconstructed slices. This allows us to find, for instance, an absorption edge.
Iodine has a known absorption 'K-edge' of 33.169 keV, so we can test the accuracy of reconstruction algorithms by seeing where this edge falls in each case. A plot of an idealised, theoretical Iodine K-edge is shown below, falling precisely at 33.169 keV.

We start by extracting a 3x3 set of pixels from each of the 3D multi-channel datasets. We can choose the pixels with the highest signal by looking at our previous reconstructed images.

```python
# Collect 3x3 regions of interest for each dataset

# FDK Recon
# Sum along z-direction (vertical)
fdk_ROI1 = np.mean(fdk.get_output().as_array()[:,44:47,:,:],1)
# Sum along y-direction (horizontal_y)
fdk_ROI2 = np.mean(fdk_ROI1[:,31:34,:],1)
# Sum along x-direction (horizontal_x)
fdk_ROI3 = np.mean(fdk_ROI2[:,14:17],1)

# 3D multi-channel 10 iteration CGLS
cgls_ROI1 = np.mean(cgls.get_output().as_array()[:,44:47,:,:],1)
cgls_ROI2 = np.mean(cgls_ROI1[:,31:34,:],1)
cgls_ROI3 = np.mean(cgls_ROI2[:,14:17],1)

# 3D multi-channel space correlated PDHG
pdhg_S_1000_ROI1 = np.mean(pdhg_S_1000.as_array()[:,44:47,:],1)
pdhg_S_1000_ROI2 = np.mean(pdhg_S_1000_ROI1[:,31:34],1)
pdhg_S_1000_ROI3 = np.mean(pdhg_S_1000_ROI2[:,14:17],1)

# 3D multi-channel space-channel correlated PDHG TV
pdhgSC_ROI1 = np.mean(pdhgSC.get_output().as_array()[:,44:47,:],1)
pdhgSC_ROI2 = np.mean(pdhgSC_ROI1[:,31:34],1)
pdhgSC_ROI3 = np.mean(pdhgSC_ROI2[:,14:17],1)
```

```python
# Set channel numbers for reduced dataset (here 100 to 140)
channel_no = np.linspace(100,140,num_channels)
# Apply linear calibration equation to convert channels to energy
e_keV = 0.2786*channel_no + 0.8575

plt.figure(figsize=(10,8))

plt.plot(e_keV,fdk_ROI3)
plt.plot(e_keV,cgls_ROI3)
plt.plot(e_keV,pdhg_S_1000_ROI3)
plt.plot(e_keV,pdhgSC_ROI3)

plt.plot((33.169,33.169),plt.ylim(0.0,0.4))

plt.legend(labels=['FDK', 'Simple CGLS', 'Space Corr. 1000 Iter. PDHG',
                   'Space-Channel Corr. 100 Iter. PDHG', 'I K-edge 33.169 keV'])
plt.ylim(0.1,0.3)
plt.xlim([29,39])

plt.ylabel('Optical Density (no units)',fontsize=20)
plt.xlabel('Energy (keV)',fontsize=20)
```

From our plots, we can see all algorithms experience a sharp rise in signal value due to the iodine absorption K-edge. Compared to the theoretical value line, each of the cases match well with the expected position of the edge.
An important factor to note is the increased smoothness of the signal for PDHG TV, when a 'Space-Channel' correlation is applied to our gradient operator for these methods. This correlation enforces our operator to use the previous energy channel as a reference point for reconstructing data in the next, resulting in a smoother transition across channels.

## Exercise 2: Move to 2D Spatial Geometry with Channels
### Defining 2D `Acquisition/Image Geometry` and `Projector`

From our 4D dataset, we can reduce the dimensionality by taking a single slice along a particular spatial dimension.
We can go from a shape of

                (Channel, Vertical, Angle, Horizontal) = (40, 80, 60, 80)
to a shape of

                        (Channel, Angle, Horizontal) = (40, 60, 80)
by taking a vertical slice subset.

Let's start by reducing our original `AcquisitionData`. For simplicity, choose a central slice subset by setting:

`data2DMC = data.subset(vertical=40)`
`ag2d = data2DMC.geometry`

This will set our new `AcquisitionGeometry`.

We must then alter the dimensionality of `ag2d`. Currently this is stored as `'3D'`, but as we now use only 2 spatial dimensions, we must change it to `'2D'`. So we can do:

`ag2d.dimension = '2D'`

```python
data2DMC = data.subset(vertical=40)
ag2d = data2DMC.geometry
ag2d.dimension = '2D'
```

We still need to manually adjust the `ImageGeometry` parameters we used for the [4D dataset](#4D_Acquisition_Image_Geometry).
Below we have the parameters used for the original dataset.
Which parts of this geometry do we no longer need?
**Delete** what's not needed and then run the code.

```python
ig2d = ImageGeometry(voxel_num_x=ig.voxel_num_x,
                     voxel_num_y=ig.voxel_num_y,
                     voxel_size_x=ag.pixel_size_h/mag,
                     voxel_size_y=ag.pixel_size_h/mag,
                     channels = ig.channels)
```

Now we can setup the tomography operator for 2D multi-channel data using the redefined `AcquisitionGeometry` and `ImageGeometry` for 2D. Let's call it `A2DMC`.

Remember: Last time we used [AstraProjector3DMC](#A3DMC) as we had a 3D multi-channel dataset. Here we instead use `AstraProjectorMC()`.

Note: While `AstraProjector3DMC` required the use of `gpu`, we can setup our 2D multi-channel tomography operator using either `gpu` or `cpu`. The default is the use of `gpu`.

```python
A2DMC = AstraProjectorMC(ig2d, ag2d ,'cpu')
```

```python
data2DMC.dimension_labels

```

## Exercise 3: Running 2D plus channel CGLS

In this exercise, use what you've learned about setting up reconstruction routines, and perform a simple CGLS using our 2D multi-channel data. Then run it for 10 iterations.

Remember, the key parameters you need are:
 - initialisation x_init (`A2DMC.volume_geometry.allocate()`)
 - operator (`A2DMC`)
 - data (`data2DMC`)

```python
x_init = A2DMC.volume_geometry.allocate()

cgls2d = CGLS(x_init = x_init, operator = A2DMC,
              data = data2DMC, max_iteration = 100)
cgls2d.run(10)

```

You can check the results of your reconstruction by running the function `show` below, giving you axial views for two energy channels. Or run an interactive slicer across energy channels.

```python
# Show reconstruction
show(cgls2d.get_output(), title='2D Multi-Channel CGLS', show_channels=[10,20,30],
     cmap='inferno', figure_size = [15,6], font_size=[25,20],minmax=(0.0,0.4))
```

```python
islicer(cgls2d.get_output(),direction='channel',title='Channel',cmap='inferno',minmax=(0.0,0.4))
```

## Exercise 4: Running 2D plus channel regularised CGLS

Here we will expand upon what you learned about BlockFrameworks. In particular, we will cover both `BlockOperators` and `BlockDataContainers`, which were covered in **Notebook 02_Tikhonov_Block_Framework**.

Below gives a brief definition of each type:

`BlockDataContainer` holds datacontainers as a column vector.

`BlockOperator` is a matrix of operators.

### Setting up Regularised CGLS

For our regularisation, we wish to solve:

$$\underset{u}{\mathrm{argmin}}\begin{Vmatrix}\binom{\alpha \nabla}{A} u - \binom{0}{g}\end{Vmatrix}^2_2$$
With the definitions:

$\tilde{A} = \binom{\alpha \nabla}{A} \quad\Longleftrightarrow\quad$ `BlockOperator(alpha*Gradient(ig2d),A2DMC)`
(where $\alpha$ is the regularisation parameter)

$\tilde{g} = \binom{0}{g} \quad\Longleftrightarrow\quad$ `BlockDataContainer(op1.range_geometry().allocate(),data2DMC)`

this can now be recognised as a least squares problem:

$$\underset{u}{\mathrm{argmin}}\begin{Vmatrix}\tilde{A} u - \tilde{g}\end{Vmatrix}^2_2$$
and being a least squares problem, it can be solved using CGLS with $\tilde{A}$ as operator and $\tilde{g}$ as data.

### Build `BlockOperator`, $\tilde{A} = \binom{\alpha \nabla}{A}$

Using the 2D multi-channel data and geometry constructed above, build a `BlockOperator`, $\tilde{A}$, applying a Space-Channel correlation.
Choose a regularisation value around $\alpha$ = 1.5 as a starting point.

```python
# Setup the BlockOperator
op1 = Gradient(ig2d, correlation='SpaceChannels')
op2 = A2DMC
alpha = 1.5
A_block = BlockOperator(alpha*op1,op2)
```

### Build `BlockDataContainer`

Next build a `BlockDataContainer`, $\tilde{g}$, containing an array with the range of the regularising operator, $\alpha \nabla$, and our reduced `AcquisitionData` (`data2DMC`).

```python
# Setup the BlockDataContainer
g_block = BlockDataContainer(op1.range_geometry().allocate(),data2DMC)
```

### Initialise and Run

Now we can initialise the `BlockOperator` and run the algorithm for 50 iterations.

```python
# Initialise the BlockOperator

x_init = A_block.domain_geometry().allocate()

# Setup and run the regularised CGLS algorithm

cgls_reg = CGLS(x_init = x_init, operator = A_block, data = g_block,
                max_iteration = 100,update_objective_interval = 10)
cgls_reg.run(50)
```

Check the reconstruction below. Does the value of $\alpha$ need changing? Click [here](#Choosing_alpha) if you want to go back and vary your initial value of $\alpha$ to see if you can improve the reconstruction.

```python
# Show reconstruction
show(cgls_reg.get_output(), show_channels=[10,20,30], title='2D Multi-Channel Regularised CGLS',
     cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.4))
```

While still noisy, we can see a significant improvement in contrast to the CGLS algorithm when no regularisation is applied.
Next we will try the PDHG algorithm using TV regularisation to see if this fares any better.

## Exercise 5: 2D Multi-channel Total Variation with PDHG

Now let's test our reduced dataset with the PDHG algorithm using Total Variation regularisation.

Based on what our construction looked like for the 4D version, try to recreate this for our 2D multi-channel data. You'll need to adjust the `BlockOperator`, $K$ and `BlockFunction`, $\mathcal{F}$, accordingly.

Play around with the numeric parameters ($\alpha$, $\sigma$ and $\tau$) as well, and see how they affect the reconstruction here. Is there a big change when moving from 3D multi-channel to 2D multi-channel?

```python
# Setup the BlockOperator, K
op1 = Gradient(ig2d, correlation='SpaceChannels')
op2 = A2DMC
K = BlockOperator(op1,op2)

# Define the regularising parameter, alpha
alpha = 0.03

# Setup the BlockFunction, F
f1 = alpha * MixedL21Norm()
f2 = 0.5 * L2NormSquared(b=data2DMC)
F = BlockFunction(f1,f2)

# Define indicator function that provides a positivity constraint
G = IndicatorBox(lower=0)

# Compute the operator norm for K
normK = K.norm()

# Define the step sizes sigma and tau
sigma = 1
tau = 1/(sigma*normK**2)
```

```python
pdhg_2d = PDHG(f=F, g=G, operator=K, tau=tau, sigma=sigma,
               max_iteration = 1000, update_objective_interval = 25)
pdhg_2d.run(500)
```

```python
# Show reconstruction
show(pdhg_2d.get_output(), show_channels=[10,20,30], title='2D Multi-Channel TV PDHG',
     cmap='inferno', figure_size = [15,6], font_size=[25,20], minmax=(0.0,0.4))
```

### Comparing 2D Multi-channel Methods

We can bring together the 2D multi-channel reconstructions we have performed for:
 - Simple CGLS
 - Regularised CGLS
 - PDHG with TV

Using the `islicer` tool, we can directly compare these reconstructions side-by-side and evaluate the level of noise removal and spatial/energy smoothing offered by each method.

```python
i1 = islicer(cgls2d.get_output(),direction='channel',title='Simple CGLS: Channel',
             cmap='inferno',minmax=(0.0,0.4))
i2 = islicer(cgls_reg.get_output(),direction='channel',title='Regularised CGLS: Channel',
             cmap='inferno',minmax=(0.0,0.4))
i3 = islicer(pdhg_2d.get_output(),direction='channel',title='PDHG TV: Channel',
             cmap='inferno',minmax = (0.0,0.4))
link_islicer(i1,i2,i3)
```

## Energy profiles

Once more we can extract energy profiles, to determine the K-edge position in our 2D multi-channel datasets.
In this case, we extract a 3x3 set of pixels by only summing along two spatial domains, y (`horizontal_y`) and x (`horizontal_x`).

```python
# Collect 3x3 regions of interest for each dataset

# 2d multi-channel simple CGLS
# Sum along y-direction
cgls2d_ROI1 = np.mean(cgls2d.get_output().as_array()[:,33:36,:],1)
# Sum along x-direction
cgls2d_ROI2 = np.mean(cgls2d_ROI1[:,60:63],1)

# 2d multi-channel regularised CGLS
cgls_reg_ROI1 = np.mean(cgls_reg.get_output().as_array()[:,33:36,:],1)
cgls_reg_ROI2 = np.mean(cgls_reg_ROI1[:,60:63],1)

# 2d multi-channel PDHG TV
pdhg_2d_ROI1 = np.mean(pdhg_2d.get_output().as_array()[:,33:36,:],1)
pdhg_2d_ROI2 = np.mean(pdhg_2d_ROI1[:,60:63],1)

```

```python
# Set channel numbers for reduced dataset
channel_no = np.linspace(100,140,num_channels)
# Apply linear calibration equation to convert channels to energy
e_keV = 0.2786*channel_no + 0.8575

plt.figure(figsize=(10,8))

plt.plot(e_keV,cgls2d_ROI2)
plt.plot(e_keV,cgls_reg_ROI2)
plt.plot(e_keV,pdhg_2d_ROI2)

plt.plot((33.169,33.169),plt.ylim(0.0,0.4))

plt.legend(labels=['Simple CGLS', 'Regularised (Space-Channel) CGLS',
                   '(Space-Channel) PDHG TV', 'I K-edge 33.169 keV'])
plt.ylim(0.1,0.4)
plt.xlim([29,39])

plt.ylabel('Optical Density (no units)',fontsize=20)
plt.xlabel('Energy (keV)',fontsize=20)
```

## Constructing Elemental Maps

Once we know have identified the position of this edge in our energy profile, we can narrow our dataset and produce an 'Iodine map'. That is, we can select only the energy channels occupied by the absorption edge, so all reconstructed signal is now due only to the iodine contrast agent. This method is known as **'K-edge subtraction'**, which you can read about in more detail in papers such as that by [C.K.Egan *et al*, 2015](https://www.nature.com/articles/srep15979#).
A basic concept is shown below for an energy profile plot. The hashed area highlights the energy range we are interested in, corresponding to the absorption edge.

Based on our plots, we will estimate the start and end of the edge to occur at approximately 32.5 keV and 34 keV respectively.

```python
# Calculate energy channels corresponding to start and end of the K-edge
e_keV = np.array([32.5,34])
channel_no = ((e_keV-0.8575)/0.2786)-100

# Display the channels corresponding to the start and end of the K-edge
print("Start of edge = channel",int(channel_no[0]))
print("End of edge = channel",int(channel_no[1]))

```

```python
# 2d multi-channel simple CGLS
# Sum over all pixels for channels of interest
cgls2d_COI = np.mean(cgls2d.get_output().as_array()[int(channel_no[0]):int(channel_no[1]),:,:],0)

# 2d multi-channel regularised CGLS
cgls_reg_COI = np.mean(cgls_reg.get_output().as_array()[int(channel_no[0]):int(channel_no[1]),:,:],0)

# 2d multi-channel PDHG TV
pdhg_2d_COI = np.mean(pdhg_2d.get_output().as_array()[int(channel_no[0]):int(channel_no[1]),:,:],0)

```

By stacking our iodine maps as an array, we can use `islicer` to move between each, directly comparing the quality of feature definition and degree of background noise for each algorithm.

```python
# Collect together iodine maps
stack_maps = (np.array([cgls2d_COI, cgls_reg_COI, pdhg_2d_COI]))
titles = ['Simple CGLS', 'Regularised CGLS', 'Space-Channel PDHG TV']
islicer(stack_maps, title=titles, direction=0, cmap='inferno',minmax = (0.0,0.4))
```

Conclusions

This notebook focused on bringing together the core elements of the CIL framework you have learned in the previous courses, and seeing how these may be applied to a multi-channel dataset.

We looked at the key differences in building Image/Acquisition Geometries and Operators for multi-channel datasets, and how these vary for both 2D and 3D multi-channel.

We have covered three key algorithms (FDK, CGLS and PDHG) and shown the differences between these for both 2D and 3D multi-channel. This also allowed us to explore some of the ways in which we can vary these reconstruction routines, and how making these adjustments can have significant effects on the quality of your reconstructions.

Finally we have shown the ability to extract spectral properties from the reconstruction, providing an insight to the elemental composition of the sample of interest using energy profiles and elemental maps.

**Click the link below to see the potential for full volume, white-beam reconstructions using both CGLS (Left) and PDHG TV (Right)**
[4D Lizard Full Volume Reconstruction](images/Lizard_gif.gif "segment")