# Neural Contextual Bandits with UCB-based Exploration
Notation:

| Notation                 | Description                                                  |
| :----------------------- | :----------------------------------------------------------- |
| $K$                      | number of arms                                               |
| $T$                      | number of total rounds                                       |
| $t$                      | index of round                                               |
| $x_{t,a}$                | $x_{t,a}\in\mathbb{R}^d$, $a\in [K]$, it is the context, the context consists of $K$ feature vectors $\{x_{t,a}\in\mathbb{R}^d|a\in[K]\}$ |
| $a_t$                    | after observes the context, the agent select an action $a_t$ in round t |
| $r_{t,a_t}$              | the reward after the agent select action $a_t$               |
| $h$                      | we assume that $r_{t,a_t}=h(x_{t,a_t})+\xi_t$, h is an unknown function satisfying $0\le h(x)\le 1$ for any x |
| $\xi_t$                  | $\xi_t$ is v-sub-Gaussian noise conditioned on $x_{1,a_1},\cdots,x_{t-1,a_{t-1}}$, satisfying $\mathbb{E}\xi_t=0$ |
| $L$                      | the depth of neural network                                  |
| $m$                      | number of neural in each layer of network                    |
| $\sigma(x)$              | we define $\sigma(x)=\max\{x,0\}$                            |
| $W_1,\cdots,W_{L-1},W_L$ | the weight in neural network. $W_1\in\mathbb{R}^{m\times d}$, $W_i\in\mathbb{R}^{m\times m}$, $2\le i\le L-1$, $W_L\in\mathbb{R}^{m\times 1}$ |
| $\theta$                 | $\theta=[vec(W_1)^T,\cdots,vec(W_l)^T]\in\mathbb{R}^p$, $p=m+md+m^2(L-1)$ |
| $f(x;\theta)$            | we define $f(x;\theta)=\sqrt{m}W_L\sigma(W_{l-1}\sigma(\cdots\sigma(W_1x)))$ |

Initialization of parameters:

UCB algorithm:

we set $\mathcal{L}(\theta) = \sum_{i=1}^t\frac{(f(x_{i,a_i};\theta)-r_{i,a_i})^2}{2}+\frac{m\lambda||\theta-\theta^{(0)}||^2_2}{2}$

Then the gradient would be
$$
\nabla\mathcal{L}(\theta) = \sum_{i=1}^t(f(x_{i,a_i};\theta)-r_{i,a_i})\nabla f(x_{i,a_i};\theta) + m\lambda(\theta-\theta^{(0)})
$$

Forawar Algorithm of Neural Network
$$
\begin{align}
X_0 &= X\\
X_1 &=\sigma(W_1X_0)\\
X_2 &=\sigma(W_2X_1)\\
\cdots\\
X_{L-1}&=\sigma(W_{L-1}X_{L-2})\\
X_{L} &=W_L X_{L-1}
\end{align}
$$
$f(X) = X_L$

Backward Propagation
$$
\begin{align}
\nabla_{X_L}f &= 1\\
\nabla_{W_L}f &= X_{L-1}\\
\nabla_{X_{L-1}}f &= W_{L}\\
\\
\nabla_{W_{L-1}}f &= \nabla_{W_{L-1}}f(X_{L-1}(W_{L-1}, X_{L-2}), W_L)=\nabla_{X_{L-1}}f \cdot \nabla_{W_{L-1}}X_{L-1}(W_{L-1}, X_{L-2})\\
\nabla_{X_{L-2}}f &= \nabla_{X_{L-2}}f(X_{L-1}(W_{L-1}, X_{L-2}), W_L)=\nabla_{X_{L-1}}f \cdot \nabla_{X_{L-2}}X_{L-1}(W_{L-1}, X_{L-2})\\
\\
\nabla_{W_{L-2}}f &= \nabla_{W_{L-2}}f(X_{L-2}(W_{L-2}, X_{L-3}), W_L,W_{L-1})=\nabla_{X_{L-2}}f \cdot \nabla_{W_{L-2}}X_{L-2}(W_{L-2}, X_{L-3})\\
\nabla_{X_{L-3}}f &= \nabla_{X_{L-3}}f(X_{L-2}(W_{L-2}, X_{L-3}), W_L,W_{L-1})=\nabla_{X_{L-2}}f \cdot \nabla_{X_{L-3}}X_{L-2}(W_{L-2}, X_{L-3})\\
\cdots\\
\nabla_{W_{l}}f &= \nabla_{W_{l}}f(X_{l}(W_{l}, X_{l-1}), W_L,W_{L-1},\cdots,W_{l+1})=\nabla_{X_{l}}f \cdot \nabla_{W_{l}}X_{l}(W_{l}, X_{l-1})\\
\nabla_{X_{l-1}}f &= \nabla_{X_{l-1}}f(X_{l}(W_{l}, X_{l-1}), W_L,W_{L-1},\cdots,W_{l+1})=\nabla_{X_{l}}f \cdot \nabla_{X_{l-1}}X_{l}(W_{l}, X_{l-1})\\
\cdots\\
\nabla_{W_{1}}f &= \nabla_{W_{1}}f(X_{1}(W_{1}, X_{0}), W_L,W_{L-1},\cdots,W_{2})=\nabla_{X_{1}}f \cdot \nabla_{W_{1}}X_{1}(W_{1}, X_{0})\\
\nabla_{X_{0}}f &= \nabla_{X_{0}}f(X_{1}(W_{1}, X_{0}), W_L,W_{L-1},\cdots,W_{2})=\nabla_{X_{1}}f \cdot \nabla_{X_{0}}X_{1}(W_{1}, X_{0})\\
\end{align}
$$

$\nabla_{W_{l}}f$ is a matrix, to be specific, $\nabla_{W_{l}}f=\left[\begin{matrix}\frac{\partial f}{\partial w^{(l)}_{11}}&\cdots &\frac{\partial f}{\partial w^{(l)}_{1m}}\\ \vdots&& \vdots\\ \frac{\partial f}{\partial w^{(l)}_{m1}}&\cdots &\frac{\partial f}{\partial w^{(l)}_{mm}}\end{matrix}\right]$

$$
\begin{align}
\left[\begin{matrix}\frac{\partial f}{\partial w^{(l)}_{11}}&\cdots &\frac{\partial f}{\partial w^{(l)}_{1m}}\\ \vdots&& \vdots\\ \frac{\partial f}{\partial w^{(l)}_{m1}}&\cdots &\frac{\partial f}{\partial w^{(l)}_{mm}}\end{matrix}\right]=&
\left[\begin{matrix}\frac{\partial f}{\partial x^{(l)}_{1}} \frac{\partial x^{(l)}_{1}}{\partial w^{(l)}_{11}}&\cdots &\frac{\partial f}{\partial x^{(l)}_{1}}\frac{\partial x^{(l)}_{1}}{\partial w^{(l)}_{1m}}\\ \vdots&& \vdots\\ \frac{\partial f}{\partial x^{(l)}_{m}}\frac{\partial x^{(l)}_{m}}{\partial w^{(l)}_{m1}}&\cdots &\frac{\partial f}{\partial x^{(l)}_{m}}\frac{\partial x^{(l)}_{m}}{\partial w^{(l)}_{mm}}\end{matrix}\right]\\
=&
\left[\begin{matrix}\frac{\partial f}{\partial x^{(l)}_{1}} \mathbb{1}_{\sigma(w^{(l)}_{11}x^{(l-1)}_1+w^{(l)}_{12}x^{(l-1)}_2+\cdots+w^{(l)}_{1m}x^{(l-1)}_m)>0} x^{(l-1)}_{1}&\cdots &\frac{\partial f}{\partial x^{(l)}_{1}}\mathbb{1}_{\sigma(w^{(l)}_{11}x^{(l-1)}_1+w^{(l)}_{12}x^{(l-1)}_2+\cdots+w^{(l)}_{1m}x^{(l-1)}_m)>0} x^{(l-1)}_{m}\\ \vdots&& \vdots\\ \frac{\partial f}{\partial x^{(l)}_{m}}\mathbb{1}_{\sigma(w^{(l)}_{m1}x^{(l-1)}_1+w^{(l)}_{m2}x^{(l-1)}_2+\cdots+w^{(l)}_{mm}x^{(l-1)}_m)>0} x^{(l-1)}_{1}&\cdots &\frac{\partial f}{\partial x^{(l)}_{m}}\mathbb{1}_{\sigma(w^{(l)}_{m1}x^{(l-1)}_1+w^{(l)}_{m2}x^{(l-1)}_2+\cdots+w^{(l)}_{mm}x^{(l-1)}_m)>0} x^{(l-1)}_{m}\end{matrix}\right]\\
\end{align}
$$

$\nabla_{X_{l-1}}f$ is a vector, to be specific, $\nabla_{X_{l-1}}f=\left[\begin{matrix}\frac{\partial f}{\partial x^{(l-1)}_{1}}\\ \vdots\\ \frac{\partial f}{\partial x^{(l-1)}_{m}}\end{matrix}\right]$
$$
\begin{align}
\left[\begin{matrix}\frac{\partial f}{\partial x^{(l-1)}_{1}}\\ \vdots\\ \frac{\partial f}{\partial x^{(l-1)}_{m}}\end{matrix}\right]=&
\left[\begin{matrix}\frac{\partial f}{\partial x^{(l)}_{1}}\frac{\partial x^{(l)}_1}{\partial x^{(l-1)}_1}+\frac{\partial f}{\partial x^{(l)}_{2}}\frac{\partial x^{(l)}_{2}}{\partial x^{(l-1)}_{1}}+\cdots+\frac{\partial f}{\partial x^{(l)}_{m}}\frac{\partial x^{(l)}_{m}}{\partial x^{(l-1)}_{1}}\\ \vdots\\ \frac{\partial f}{\partial x^{(l)}_{1}}\frac{\partial x^{(l)}_{1}}{\partial x^{(l-1)}_{m}}+\frac{\partial f}{\partial x^{(l)}_{2}}\frac{\partial x^{(l)}_{2}}{\partial x^{(l-1)}_{m}}+\cdots+\frac{\partial f}{\partial x^{(l)}_{m}}\frac{\partial x^{(l)}_{m}}{\partial x^{(l-1)}_{m}}\end{matrix}\right]\\
=&
\left[\begin{matrix}\frac{\partial f}{\partial x^{(l)}_{1}}\mathbb{1}_{\sigma(w^{(l)}_{11}x^{(l-1)}_1+w^{(l)}_{12}x^{(l-1)}_2+\cdots+w^{(l)}_{1m}x^{(l-1)}_m)>0}w^{(l)}_{11}+\frac{\partial f}{\partial x^{(l)}_{2}}\mathbb{1}_{\sigma(w^{(l)}_{21}x^{(l-1)}_1+w^{(l)}_{22}x^{(l-1)}_2+\cdots+w^{(l)}_{2m}x^{(l-1)}_m)>0}w^{(l)}_{21}+\cdots+\frac{\partial f}{\partial x^{(l)}_{m}}\mathbb{1}_{\sigma(w^{(l)}_{m1}x^{(l-1)}_1+w^{(l)}_{m2}x^{(l-1)}_2+\cdots+w^{(l)}_{mm}x^{(l-1)}_m)>0}w^{(l)}_{m1}\\
\vdots\\
\frac{\partial f}{\partial x^{(l)}_{1}}\mathbb{1}_{\sigma(w^{(l)}_{11}x^{(l-1)}_1+w^{(l)}_{12}x^{(l-1)}_2+\cdots+w^{(l)}_{1m}x^{(l-1)}_m)>0}w^{(l)}_{1m}+\frac{\partial f}{\partial x^{(l)}_{2}}\mathbb{1}_{\sigma(w^{(l)}_{21}x^{(l-1)}_1+w^{(l)}_{22}x^{(l-1)}_2+\cdots+w^{(l)}_{2m}x^{(l-1)}_m)>0}w^{(l)}_{2m}+\cdots+\frac{\partial f}{\partial x^{(l)}_{m}}\mathbb{1}_{\sigma(w^{(l)}_{m1}x^{(l-1)}_1+w^{(l)}_{m2}x^{(l-1)}_2+\cdots+w^{(l)}_{mm}x^{(l-1)}_m)>0}w^{(l)}_{mm}\end{matrix}\right ]\\
=&\left[\begin{matrix}w^{(l)}_{11}&\cdots& w^{(l)}_{1m}\\ \vdots&&\vdots\\ w^{(l)}_{m1}&\cdots& w^{(l)}_{mm}\end{matrix}\right]^T
\left[\begin{matrix}\frac{\partial f}{\partial x^{(l)}_{1}}\mathbb{1}_{\sigma(w^{(l)}_{11}x^{(l-1)}_1+w^{(l)}_{12}x^{(l-1)}_2+\cdots+w^{(l)}_{1m}x^{(l-1)}_m)>0}\\ \vdots\\ \frac{\partial f}{\partial x^{(l)}_{m}}\mathbb{1}_{\sigma(w^{(l)}_{m1}x^{(l-1)}_1+w^{(l)}_{m2}x^{(l-1)}_2+\cdots+w^{(l)}_{mm}x^{(l-1)}_m)>0}\end{matrix}\right]\\
\end{align}
$$

 We assume the reward follows reward = context^T * A^T * A * context + \xi

 $\xi$ is a random variable following standard normal distribution N(0, 1)

 A is d\*d matrix, randomly generated from N(0, 1)

 We assume the context is independent from the action and round index.

 Given action a and round index t, the context is randomly sample from a unit ball in dimension d

```python
%reset -f
import numpy as np
import random
from copy import deepcopy
from GameSetting import *
from NeuralNetworkRelatedFunction import *
```

```python
class BestAgent:
    def __init__(self, K, T, d):
        # K is Total number of actions,
        # T is Total number of periods
        # d is the dimension of context
        self.K = K
        self.T = T
        self.d = d
        self.t = 0 # marks the index of period
        self.history_reward = np.zeros(T)
        self.history_action = np.zeros(T)
        self.history_context = np.zeros((d, T))

    def Action(self, context_list):
        # context_list is a d*K matrix, each column represent a context
        # the return value is the action we choose, represent the index of action, is a scalar

        expected_reward = np.zeros(K)
        for kk in range(0, K):
            context = context_list[:, kk]
            expected_reward[kk] = context.transpose().dot(A.transpose().dot(A)).dot(context)
        ind = np.argmax(expected_reward, axis=None)
        self.history_context[:, self.t] = context_list[:, ind]
        self.history_action[self.t] = ind
        return ind

    def Update(self, reward):
        # reward is the realized reward after we adopt policy, a scalar
        self.history_reward[self.t] = reward
        self.t = self.t + 1

    def GetHistoryReward(self):
        return self.history_reward

    def GetHistoryAction(self):
        return self.history_action

    def GetHistoryContext(self):
        return self.history_context
```

```python
class UniformAgent:
    def __init__(self, K, T, d):
        # K is Total number of actions,
        # T is Total number of periods
        # d is the dimension of context
        self.K = K
        self.T = T
        self.d = d
        self.t = 0 # marks the index of period
        self.history_reward = np.zeros(T)
        self.history_action = np.zeros(T)
        self.history_context = np.zeros((d, T))

    def Action(self, context_list):
        # context_list is a d*K matrix, each column represent a context
        # the return value is the action we choose, represent the index of action, is a scalar

        ind = np.random.randint(0, high = K) # we just uniformly choose an action
        self.history_context[:, self.t] = context_list[:, ind]
        return ind

    def Update(self, reward):
        # reward is the realized reward after we adopt policy, a scalar
        self.history_reward[self.t] = reward
        self.t = self.t + 1

    def GetHistoryReward(self):
        return self.history_reward

    def GetHistoryAction(self):
        return self.history_action

    def GetHistoryContext(self):
        return self.history_context
```

```python
class NeuralAgent:
    def __init__(self, K, T, d, L = 2, m = 20, gamma_t = 0.01, v = 0.1, lambda_ = 0.01, delta = 0.01, S = 0.01, eta = 0.001, frequency = 50, batchsize = None):
        # K is Total number of actions,
        # T is Total number of periods
        # d is the dimension of context
        self.K = K
        self.T = T
        self.d = d

        self.L = L
        self.m = m
        self.gamma_t = gamma_t
        self.v = v
        self.lambda_ = lambda_
        self.delta = delta
        self.S = S
        self.eta = eta
        self.frequency = frequency # we train the network after frequency, e.g. per 50 round
        self.batchsize = batchsize
        self.t = 0 # marks the index of period
        self.history_reward = np.zeros(T)
        self.history_action = np.zeros(T)
        self.predicted_reward = np.zeros(T)
        self.predicted_reward_upperbound = np.zeros(T)
        self.history_context = np.zeros((d, T))

        # initialize the value of parameter
        np.random.seed(12345)
        self.theta_0 = {}
        W = np.random.normal(loc = 0, scale = 4 / m, size=(int(m/2), int(m/2)))
        w = np.random.normal(loc = 0, scale = 2 / m, size=(1, int(m/2)))
        for key in range(1, L + 1):
            if key == 1:
                # this paper doesn't present the initialization of w1
                # in its setting, d = m, then he let theta_0["w1"]=[W,0;0,W]
                # but in fact d might not equal to m
                tempW = np.random.normal(loc = 0, scale = 4 / m, size=(int(m/2), int(d/2)))
                self.theta_0["w1"] = np.zeros((m, d))
                self.theta_0["w1"][0:int(m/2), 0:int(d/2)] = tempW
                self.theta_0["w1"][int(m/2):, int(d/2):] = tempW
            elif 2 <= key and key <= L - 1:
                self.theta_0["w" + str(key)] = np.zeros((m, m))
                self.theta_0["w" + str(key)][0:int(m/2), 0:int(m/2)] = W
                self.theta_0["w" + str(key)][int(m/2):, int(m/2):] = W
            else:
                self.theta_0["w" + str(key)] = np.concatenate([w, -w], axis = 1)

        self.p = m + m * d + m * m * (L - 2)
        self.params = deepcopy(self.theta_0)
        self.Z_t_minus1 = lambda_ * np.eye(self.p)
        self.params_history = {}
        self.grad_history = {}

    def Action(self, context_list):
        # context_list is a d*K matrix, each column represent a context
        # the return value is the action we choose, represent the index of action, is a scalar
        U_t_a = np.zeros(K) # the upper bound of K actions
        predict_reward = np.zeros(K)
        for a in range(1, K + 1):
            predict_reward[a - 1] = NeuralNetwork(context_list[:, a - 1], self.params, self.L, self.m)['x' + str(self.L)][0]
            grad_parameter = GradientNeuralNetwork(context_list[:, a - 1], self.params, self.L, self.m)
            grad_parameter = FlattenDict(grad_parameter, self.L)
            Z_t_minus1_inverse = np.linalg.inv(self.Z_t_minus1)
            U_t_a[a - 1] = predict_reward[a - 1] +\
                           self.gamma_t * np.sqrt(grad_parameter.dot(Z_t_minus1_inverse).dot(grad_parameter)/self.m)
        ind = np.argmax(U_t_a, axis=None)
        self.predicted_reward[self.t] = predict_reward[ind]
        self.predicted_reward_upperbound[self.t] = U_t_a[ind]
        self.history_action[self.t] = ind
        self.history_context[:, self.t] = context_list[:, ind]
        return ind

    def Update(self, reward):
        # reward is the realized reward after we adopt policy, a scalar
#         print("round {:d}".format(self.t))
        self.history_reward[self.t] = reward
        ind = self.history_action[self.t]
        context = self.history_context[:, self.t]

        # compute Z_t_minus1
        grad_parameter = GradientNeuralNetwork(context, self.params, self.L, self.m)
        grad_parameter = FlattenDict(grad_parameter, self.L)
        grad_parameter = np.expand_dims(grad_parameter, axis = 1)
        self.Z_t_minus1 = self.Z_t_minus1 + grad_parameter.dot(grad_parameter.transpose()) / self.m

        # train neural network
        if self.t % self.frequency == 0 and self.t > 0:
            J = self.t
        else:
            J = 0

        if self.batchsize == None:
            trainindex = range(0, self.t + 1)
        else:
            if self.batchsize > self.t + 1:
                trainindex = range(0, self.t + 1)
            else:
                trainindex = random.sample(range(0, self.t + 1), self.batchsize)

        grad_loss = {}
        for j in range(J):

            grad_loss = GradientLossFunction(self.history_context[:, trainindex],# we had not update self.t yet, so here we must +1
                                             self.params,
                                             self.L,
                                             self.m,
                                             self.history_reward[trainindex],
                                             self.theta_0,
                                             self.lambda_)
#             if j < 10:
#                 eta = 1e-4
#             else:
#                 eta = self.eta
            eta = self.eta
            for key in self.params.keys():
                self.params[key] = self.params[key] - eta * grad_loss[key]
            loss = LossFunction(self.history_context[:, trainindex],
                            self.params,
                            self.L,
                            self.m,
                            self.history_reward[trainindex],
                            self.theta_0,
                            self.lambda_)
#             print("j {:d}, loss {:4f}".format(j, loss))

        print("round {:d}, predicted reward {:4f},predicted upper bound {:4f},actual reward {:4f}".format(self.t,
                                                                                self.predicted_reward[self.t],
                                                                                self.predicted_reward_upperbound[self.t],
                                                                                reward))

        self.params_history[self.t] = deepcopy(self.params)
        self.grad_history[self.t] = deepcopy(grad_loss)

        self.t = self.t + 1

    def GetHistoryReward(self):
        return self.history_reward

    def GetHistoryAction(self):
        return self.history_action

    def GetHistoryContext(self):
        return self.history_context
```

```python
# Set the parameter of the game
np.random.seed(12345)
K = 4# Total number of actions,
T = 5000 # Total number of periods
d = 20 # the dimension of context
A = np.random.normal(loc=0, scale=1, size=(d, d))
```

```python
# Neural UCB
np.random.seed(12345)

# Set the parameter of the network
# the setting is based on the description of section 7.1 of the papaer
L = 2
m = 20
# we fix gamma in each round, according to the description of section 3.1
gamma_t = 0.01 #{0.01, 0.1, 1, 10}
v = 0.1 #{0.01, 0.1, 1}
lambda_ = 1 #{0.1, 1, 10}
delta = 0.01 #{0.01, 0.1, 1}
S = 0.01 #{0.01, 0.1, 1, 10}
eta = 1e-4 #{0.001, 0.01, 0.1}
frequency = 50
batchsize = 500
# we set J equal to round index t

neuralagent = NeuralAgent(K, T, d, L, m, gamma_t, v, lambda_, delta, S, eta, frequency, batchsize)
for tt in range(1, T + 1):
    # observe \{x_{t,a}\}_{a=1}^{k=1}
    context_list = SampleContext(d, K)

    # compute the upper bound of reward
    ind = neuralagent.Action(context_list)

    # play ind and observe reward
    reward = GetRealReward(context_list[:, ind], A)
    neuralagent.Update(reward)
```

    round 0, predicted reward 0.156496,predicted upper bound 0.163969,actual reward 26.866324
    round 1, predicted reward 0.152098,predicted upper bound 0.157251,actual reward 21.779967
    round 2, predicted reward 0.223601,predicted upper bound 0.229045,actual reward 22.502430
    round 3, predicted reward 0.350837,predicted upper bound 0.356987,actual reward 23.820496
    round 4, predicted reward 0.292728,predicted upper bound 0.296934,actual reward 23.628268
    round 5, predicted reward 0.163629,predicted upper bound 0.168596,actual reward 24.395122
    round 6, predicted reward 0.150042,predicted upper bound 0.153091,actual reward 26.479825
    round 7, predicted reward 0.282388,predicted upper bound 0.287587,actual reward 16.411790
    round 8, predicted reward 0.009460,predicted upper bound 0.014848,actual reward 26.086504
    round 9, predicted reward 0.106620,predicted upper bound 0.111166,actual reward 22.741131
    round 10, predicted reward 0.343633,predicted upper bound 0.348210,actual reward 18.129200
    round 11, predicted reward 0.024985,predicted upper bound 0.028719,actual reward 12.385876
    round 12, predicted reward 0.527528,predicted upper bound 0.533315,actual reward 23.321083
    round 13, predicted reward 0.236167,predicted upper bound 0.241390,actual reward 38.542890
    round 14, predicted reward 0.039366,predicted upper bound 0.044243,actual reward 16.437376
    round 15, predicted reward 0.314568,predicted upper bound 0.319064,actual reward 28.158622
    round 16, predicted reward 0.320190,predicted upper bound 0.325268,actual reward 29.140470
    round 17, predicted reward 0.281421,predicted upper bound 0.286372,actual reward 17.353608
    round 18, predicted reward -0.010189,predicted upper bound -0.005281,actual reward 18.826907
    round 19, predicted reward 0.421241,predicted upper bound 0.425192,actual reward 11.449585
    round 20, predicted reward 0.119419,predicted upper bound 0.122951,actual reward 38.120675
    round 21, predicted reward 0.228744,predicted upper bound 0.233303,actual reward 28.433292
    round 22, predicted reward 0.315052,predicted upper bound 0.320800,actual reward 19.248693
    round 23, predicted reward 0.059136,predicted upper bound 0.063596,actual reward 18.198123
    round 24, predicted reward 0.300010,predicted upper bound 0.305025,actual reward 23.300051
    round 25, predicted reward 0.198885,predicted upper bound 0.203364,actual reward 14.475313
    round 26, predicted reward 0.247887,predicted upper bound 0.252130,actual reward 17.704919
    round 27, predicted reward 0.144905,predicted upper bound 0.148240,actual reward 16.759412
    round 28, predicted reward 0.208747,predicted upper bound 0.212602,actual reward 20.488660
    round 29, predicted reward 0.165630,predicted upper bound 0.169607,actual reward 25.917058
    round 30, predicted reward 0.056182,predicted upper bound 0.059950,actual reward 20.260931
    round 31, predicted reward 0.128695,predicted upper bound 0.132632,actual reward 21.570369
    round 32, predicted reward 0.129648,predicted upper bound 0.135204,actual reward 24.936236
    round 33, predicted reward 0.159791,predicted upper bound 0.164016,actual reward 16.134349
    round 34, predicted reward 0.013973,predicted upper bound 0.018509,actual reward 31.580753
    round 35, predicted reward 0.560455,predicted upper bound 0.564927,actual reward 25.174120
    round 36, predicted reward 0.295000,predicted upper bound 0.299220,actual reward 19.354114
    round 37, predicted reward 0.205437,predicted upper bound 0.210574,actual reward 26.834291
    round 38, predicted reward 0.040120,predicted upper bound 0.044390,actual reward 20.280294
    round 39, predicted reward 0.240947,predicted upper bound 0.244452,actual reward 20.487989
    round 40, predicted reward 0.328114,predicted upper bound 0.332106,actual reward 14.207510
    round 41, predicted reward 0.291775,predicted upper bound 0.296436,actual reward 13.174174
    round 42, predicted reward 0.178332,predicted upper bound 0.182355,actual reward 16.017797
    round 43, predicted reward 0.195788,predicted upper bound 0.199415,actual reward 24.979637
    round 44, predicted reward 0.457667,predicted upper bound 0.462285,actual reward 14.990857
    round 45, predicted reward 0.186996,predicted upper bound 0.190440,actual reward 37.255346
    round 46, predicted reward 0.078680,predicted upper bound 0.082426,actual reward 20.687110
    round 47, predicted reward 0.254361,predicted upper bound 0.259520,actual reward 26.349630
    round 48, predicted reward 0.402168,predicted upper bound 0.406433,actual reward 23.909314
    round 49, predicted reward 0.311260,predicted upper bound 0.315214,actual reward 22.767743
    round 50, predicted reward 0.062184,predicted upper bound 0.066672,actual reward 23.466417
    round 51, predicted reward 8.606091,predicted upper bound 8.640735,actual reward 14.642113
    round 52, predicted reward 17.627259,predicted upper bound 17.662157,actual reward 18.885952
    round 53, predicted reward 7.995933,predicted upper bound 8.030579,actual reward 18.359690
    round 54, predicted reward 5.429062,predicted upper bound 5.460406,actual reward 26.046702
    round 55, predicted reward 18.930439,predicted upper bound 18.971483,actual reward 11.706470
    round 56, predicted reward 14.594812,predicted upper bound 14.628113,actual reward 17.134158
    round 57, predicted reward 16.372851,predicted upper bound 16.406117,actual reward 17.379211
    round 58, predicted reward 16.534704,predicted upper bound 16.571372,actual reward 15.656447
    round 59, predicted reward 14.027420,predicted upper bound 14.063159,actual reward 19.518779
    round 60, predicted reward 24.912907,predicted upper bound 24.942811,actual reward 26.741491
    round 61, predicted reward 16.426400,predicted upper bound 16.462498,actual reward 20.959550
    round 62, predicted reward 14.281727,predicted upper bound 14.319157,actual reward 17.221722
    round 63, predicted reward 15.054633,predicted upper bound 15.085599,actual reward 14.741157
    round 64, predicted reward 32.300895,predicted upper bound 32.336853,actual reward 20.169107
    round 65, predicted reward 7.407692,predicted upper bound 7.441547,actual reward 21.379650
    round 66, predicted reward 17.846936,predicted upper bound 17.878941,actual reward 22.110123
    round 67, predicted reward 16.504289,predicted upper bound 16.537194,actual reward 21.403313
    round 68, predicted reward 18.028844,predicted upper bound 18.057299,actual reward 19.551126
    round 69, predicted reward 7.267333,predicted upper bound 7.292153,actual reward 17.314159
    round 70, predicted reward 14.856813,predicted upper bound 14.885812,actual reward 17.622531
    round 71, predicted reward 16.589743,predicted upper bound 16.621304,actual reward 37.133653
    round 72, predicted reward 31.980646,predicted upper bound 32.008496,actual reward 27.050000
    round 73, predicted reward 18.537559,predicted upper bound 18.565678,actual reward 23.105176
    round 74, predicted reward 14.009138,predicted upper bound 14.035760,actual reward 17.557551
    round 75, predicted reward 11.138431,predicted upper bound 11.164345,actual reward 31.715111
    round 76, predicted reward 22.696676,predicted upper bound 22.725281,actual reward 18.833883
    round 77, predicted reward 11.517352,predicted upper bound 11.544708,actual reward 29.761181
    round 78, predicted reward 19.359240,predicted upper bound 19.389224,actual reward 23.260744
    round 79, predicted reward 23.841858,predicted upper bound 23.874079,actual reward 13.178028
    round 80, predicted reward 17.279209,predicted upper bound 17.308259,actual reward 19.171228
    round 81, predicted reward 13.657639,predicted upper bound 13.679428,actual reward 19.811594
    round 82, predicted reward 18.063326,predicted upper bound 18.088501,actual reward 28.378565
    round 83, predicted reward 28.552671,predicted upper bound 28.576200,actual reward 22.619617
    round 84, predicted reward 23.537586,predicted upper bound 23.562342,actual reward 13.321176
    round 85, predicted reward 17.181897,predicted upper bound 17.206923,actual reward 25.013070
    round 86, predicted reward 30.894249,predicted upper bound 30.916550,actual reward 26.919864
    round 87, predicted reward 32.041375,predicted upper bound 32.065035,actual reward 27.878379
    round 88, predicted reward 16.352468,predicted upper bound 16.375227,actual reward 13.295649
    round 89, predicted reward 11.583217,predicted upper bound 11.610271,actual reward 12.657600
    round 90, predicted reward 17.705822,predicted upper bound 17.726968,actual reward 12.612084
    round 91, predicted reward 16.046766,predicted upper bound 16.071448,actual reward 20.233196
    round 92, predicted reward 14.144616,predicted upper bound 14.164875,actual reward 27.889139
    round 93, predicted reward 16.121896,predicted upper bound 16.145989,actual reward 30.610312
    round 94, predicted reward 15.225252,predicted upper bound 15.248123,actual reward 22.012892
    round 95, predicted reward 16.355666,predicted upper bound 16.380269,actual reward 23.974887
    round 96, predicted reward 8.217883,predicted upper bound 8.240996,actual reward 17.748972
    round 97, predicted reward 26.009223,predicted upper bound 26.028347,actual reward 20.992405
    round 98, predicted reward 14.951986,predicted upper bound 14.975622,actual reward 19.180408
    round 99, predicted reward 26.249606,predicted upper bound 26.269704,actual reward 25.827047
    round 100, predicted reward 16.151437,predicted upper bound 16.174269,actual reward 30.430881
    round 101, predicted reward 27.898543,predicted upper bound 27.929937,actual reward 26.278745
    round 102, predicted reward 18.710856,predicted upper bound 18.740857,actual reward 25.795415
    round 103, predicted reward 25.865475,predicted upper bound 25.894136,actual reward 21.518010
    round 104, predicted reward 20.259688,predicted upper bound 20.287348,actual reward 15.963689
    round 105, predicted reward 20.152761,predicted upper bound 20.185419,actual reward 16.177697
    round 106, predicted reward 21.634282,predicted upper bound 21.661057,actual reward 24.037160
    round 107, predicted reward 20.475148,predicted upper bound 20.507660,actual reward 12.749993
    round 108, predicted reward 17.872910,predicted upper bound 17.897472,actual reward 25.255918
    round 109, predicted reward 18.114403,predicted upper bound 18.139309,actual reward 33.301586
    round 110, predicted reward 21.184504,predicted upper bound 21.212341,actual reward 14.625824
    round 111, predicted reward 19.244821,predicted upper bound 19.274795,actual reward 19.955009
    round 112, predicted reward 29.802609,predicted upper bound 29.828185,actual reward 34.618448
    round 113, predicted reward 14.922558,predicted upper bound 14.945205,actual reward 38.375085
    round 114, predicted reward 25.467172,predicted upper bound 25.494545,actual reward 16.759919
    round 115, predicted reward 24.033530,predicted upper bound 24.061557,actual reward 14.806201
    round 116, predicted reward 16.800245,predicted upper bound 16.823533,actual reward 35.138419
    round 117, predicted reward 20.402109,predicted upper bound 20.435326,actual reward 29.043583
    round 118, predicted reward 20.644041,predicted upper bound 20.669804,actual reward 24.419414
    round 119, predicted reward 18.951313,predicted upper bound 18.976432,actual reward 28.243550
    round 120, predicted reward 25.089538,predicted upper bound 25.114679,actual reward 21.694475
    round 121, predicted reward 22.584291,predicted upper bound 22.608065,actual reward 19.451703
    round 122, predicted reward 25.139846,predicted upper bound 25.161923,actual reward 30.241232
    round 123, predicted reward 26.259558,predicted upper bound 26.285970,actual reward 30.250533
    round 124, predicted reward 23.916421,predicted upper bound 23.942913,actual reward 19.407280
    round 125, predicted reward 25.559071,predicted upper bound 25.586802,actual reward 32.771126
    round 126, predicted reward 18.227899,predicted upper bound 18.252393,actual reward 18.111151
    round 127, predicted reward 22.213214,predicted upper bound 22.236682,actual reward 26.144206
    round 128, predicted reward 14.417640,predicted upper bound 14.441146,actual reward 24.276563
    round 129, predicted reward 22.246411,predicted upper bound 22.273271,actual reward 22.786668
    round 130, predicted reward 16.896966,predicted upper bound 16.917932,actual reward 22.843833
    round 131, predicted reward 26.852486,predicted upper bound 26.876064,actual reward 27.724264
    round 132, predicted reward 20.853834,predicted upper bound 20.879880,actual reward 31.556063
    round 133, predicted reward 15.220276,predicted upper bound 15.239049,actual reward 19.034335
    round 134, predicted reward 28.086191,predicted upper bound 28.107554,actual reward 14.144158
    round 135, predicted reward 18.752047,predicted upper bound 18.779067,actual reward 28.623331
    round 136, predicted reward 28.930499,predicted upper bound 28.955853,actual reward 17.611621
    round 137, predicted reward 19.618000,predicted upper bound 19.643267,actual reward 22.990426
    round 138, predicted reward 24.962505,predicted upper bound 24.983094,actual reward 21.877166
    round 139, predicted reward 19.143336,predicted upper bound 19.168428,actual reward 27.291976
    round 140, predicted reward 19.837250,predicted upper bound 19.863107,actual reward 33.742850
    round 141, predicted reward 17.044638,predicted upper bound 17.073835,actual reward 17.630771
    round 142, predicted reward 15.408549,predicted upper bound 15.437337,actual reward 23.226348
    round 143, predicted reward 22.495624,predicted upper bound 22.520906,actual reward 24.279765
    round 144, predicted reward 24.449657,predicted upper bound 24.467361,actual reward 32.037554
    round 145, predicted reward 28.070409,predicted upper bound 28.093692,actual reward 18.088100
    round 146, predicted reward 21.769857,predicted upper bound 21.795832,actual reward 14.068824
    round 147, predicted reward 21.362847,predicted upper bound 21.382868,actual reward 20.625575
    round 148, predicted reward 24.108577,predicted upper bound 24.128892,actual reward 24.792132
    round 149, predicted reward 31.305155,predicted upper bound 31.328888,actual reward 23.412657
    round 150, predicted reward 26.480364,predicted upper bound 26.503703,actual reward 27.039334
    round 151, predicted reward 19.383585,predicted upper bound 19.415251,actual reward 21.379650
    round 152, predicted reward 24.025492,predicted upper bound 24.050102,actual reward 26.735973
    round 153, predicted reward 23.079243,predicted upper bound 23.104665,actual reward 18.316809
    round 154, predicted reward 25.222249,predicted upper bound 25.245979,actual reward 29.855392
    round 155, predicted reward 21.376171,predicted upper bound 21.395969,actual reward 28.950713
    round 156, predicted reward 14.369346,predicted upper bound 14.400701,actual reward 17.400596
    round 157, predicted reward 17.331990,predicted upper bound 17.357879,actual reward 10.466660
    round 158, predicted reward 22.872946,predicted upper bound 22.895533,actual reward 17.371566
    round 159, predicted reward 18.887204,predicted upper bound 18.919371,actual reward 33.074061
    round 160, predicted reward 26.428769,predicted upper bound 26.453866,actual reward 24.313228
    round 161, predicted reward 18.658765,predicted upper bound 18.685345,actual reward 24.241578
    round 162, predicted reward 21.067331,predicted upper bound 21.092944,actual reward 20.424007
    round 163, predicted reward 24.962838,predicted upper bound 24.988219,actual reward 31.410511
    round 164, predicted reward 25.958915,predicted upper bound 25.980552,actual reward 18.363835
    round 165, predicted reward 22.668678,predicted upper bound 22.693081,actual reward 17.510460
    round 166, predicted reward 26.362126,predicted upper bound 26.390475,actual reward 30.685054
    round 167, predicted reward 20.567619,predicted upper bound 20.592681,actual reward 19.401971
    round 168, predicted reward 29.522772,predicted upper bound 29.544542,actual reward 35.913454
    round 169, predicted reward 23.517825,predicted upper bound 23.542266,actual reward 17.662427
    round 170, predicted reward 21.043265,predicted upper bound 21.066685,actual reward 26.210918
    round 171, predicted reward 31.668816,predicted upper bound 31.692322,actual reward 35.813192
    round 172, predicted reward 26.428517,predicted upper bound 26.448090,actual reward 28.432307
    round 173, predicted reward 29.028598,predicted upper bound 29.050641,actual reward 27.179995
    round 174, predicted reward 23.990750,predicted upper bound 24.012266,actual reward 22.385836
    round 175, predicted reward 22.640801,predicted upper bound 22.664807,actual reward 23.476106
    round 176, predicted reward 23.082837,predicted upper bound 23.103748,actual reward 19.640802
    round 177, predicted reward 23.749021,predicted upper bound 23.774446,actual reward 28.787638
    round 178, predicted reward 19.261646,predicted upper bound 19.288201,actual reward 27.497351
    round 179, predicted reward 25.493315,predicted upper bound 25.518154,actual reward 17.462747
    round 180, predicted reward 19.401808,predicted upper bound 19.427003,actual reward 19.733407
    round 181, predicted reward 22.755637,predicted upper bound 22.781273,actual reward 24.842331
    round 182, predicted reward 32.526842,predicted upper bound 32.546296,actual reward 36.637592
    round 183, predicted reward 25.517931,predicted upper bound 25.540066,actual reward 28.720944
    round 184, predicted reward 24.978386,predicted upper bound 25.003894,actual reward 34.455809
    round 185, predicted reward 23.745775,predicted upper bound 23.769681,actual reward 23.671121
    round 186, predicted reward 25.427070,predicted upper bound 25.447935,actual reward 22.593549
    round 187, predicted reward 21.837370,predicted upper bound 21.858599,actual reward 15.965466
    round 188, predicted reward 26.481024,predicted upper bound 26.504886,actual reward 25.377860
    round 189, predicted reward 22.596720,predicted upper bound 22.623500,actual reward 24.258048
    round 190, predicted reward 25.311981,predicted upper bound 25.332812,actual reward 20.864208
    round 191, predicted reward 20.782321,predicted upper bound 20.800912,actual reward 18.467663
    round 192, predicted reward 22.275420,predicted upper bound 22.296662,actual reward 19.073194
    round 193, predicted reward 20.074677,predicted upper bound 20.099228,actual reward 23.297718
    round 194, predicted reward 27.444798,predicted upper bound 27.465936,actual reward 23.394712
    round 195, predicted reward 18.174278,predicted upper bound 18.196238,actual reward 13.148697
    round 196, predicted reward 22.791176,predicted upper bound 22.811481,actual reward 17.623899
    round 197, predicted reward 22.172199,predicted upper bound 22.195742,actual reward 25.140992
    round 198, predicted reward 19.017947,predicted upper bound 19.042642,actual reward 32.007888
    round 199, predicted reward 21.947309,predicted upper bound 21.970396,actual reward 30.576206
    round 200, predicted reward 28.569220,predicted upper bound 28.589263,actual reward 27.229753
    round 201, predicted reward 21.804745,predicted upper bound 21.831981,actual reward 22.926045
    round 202, predicted reward 26.421969,predicted upper bound 26.445980,actual reward 28.557619
    round 203, predicted reward 24.269991,predicted upper bound 24.291389,actual reward 17.551181
    round 204, predicted reward 22.711272,predicted upper bound 22.734678,actual reward 19.331618
    round 205, predicted reward 23.165566,predicted upper bound 23.192620,actual reward 27.984101
    round 206, predicted reward 22.799348,predicted upper bound 22.821693,actual reward 23.129912
    round 207, predicted reward 29.946937,predicted upper bound 29.970393,actual reward 31.169346
    round 208, predicted reward 32.127192,predicted upper bound 32.149625,actual reward 31.951191
    round 209, predicted reward 23.961492,predicted upper bound 23.985529,actual reward 24.061564
    round 210, predicted reward 20.623061,predicted upper bound 20.645991,actual reward 30.079992
    round 211, predicted reward 28.006112,predicted upper bound 28.028942,actual reward 30.346848
    round 212, predicted reward 28.236721,predicted upper bound 28.261904,actual reward 19.947383
    round 213, predicted reward 17.812111,predicted upper bound 17.834953,actual reward 19.659480
    round 214, predicted reward 22.668847,predicted upper bound 22.686810,actual reward 21.911955
    round 215, predicted reward 25.688613,predicted upper bound 25.713161,actual reward 27.210401
    round 216, predicted reward 25.744912,predicted upper bound 25.768897,actual reward 19.481306
    round 217, predicted reward 24.845274,predicted upper bound 24.866659,actual reward 25.048318
    round 218, predicted reward 21.865141,predicted upper bound 21.884999,actual reward 24.100033
    round 219, predicted reward 26.529381,predicted upper bound 26.550160,actual reward 22.668683
    round 220, predicted reward 22.201611,predicted upper bound 22.223913,actual reward 16.063566
    round 221, predicted reward 21.938489,predicted upper bound 21.962990,actual reward 24.266150
    round 222, predicted reward 21.995987,predicted upper bound 22.017235,actual reward 23.979146
    round 223, predicted reward 27.843543,predicted upper bound 27.862738,actual reward 35.147981
    round 224, predicted reward 22.686701,predicted upper bound 22.707826,actual reward 21.294915
    round 225, predicted reward 29.003230,predicted upper bound 29.022150,actual reward 26.133858
    round 226, predicted reward 25.384285,predicted upper bound 25.410130,actual reward 26.776742
    round 227, predicted reward 23.723504,predicted upper bound 23.741405,actual reward 22.723087
    round 228, predicted reward 18.850782,predicted upper bound 18.871120,actual reward 17.663279
    round 229, predicted reward 25.322292,predicted upper bound 25.345190,actual reward 31.734585
    round 230, predicted reward 24.947435,predicted upper bound 24.965572,actual reward 28.162845
    round 231, predicted reward 24.232749,predicted upper bound 24.253754,actual reward 16.691607
    round 232, predicted reward 29.278461,predicted upper bound 29.295093,actual reward 29.173788
    round 233, predicted reward 22.391432,predicted upper bound 22.411690,actual reward 24.881548
    round 234, predicted reward 25.654450,predicted upper bound 25.676358,actual reward 25.483518
    round 235, predicted reward 25.311130,predicted upper bound 25.334066,actual reward 20.452540
    round 236, predicted reward 23.098871,predicted upper bound 23.120709,actual reward 17.909351
    round 237, predicted reward 34.854073,predicted upper bound 34.873388,actual reward 36.070589
    round 238, predicted reward 26.370952,predicted upper bound 26.393217,actual reward 26.457008
    round 239, predicted reward 20.517089,predicted upper bound 20.540390,actual reward 18.910845
    round 240, predicted reward 20.719461,predicted upper bound 20.736537,actual reward 24.879516
    round 241, predicted reward 30.510980,predicted upper bound 30.534706,actual reward 35.110334
    round 242, predicted reward 22.848970,predicted upper bound 22.869440,actual reward 23.114881
    round 243, predicted reward 28.843733,predicted upper bound 28.866597,actual reward 36.708900
    round 244, predicted reward 26.597703,predicted upper bound 26.618377,actual reward 20.661982
    round 245, predicted reward 28.668092,predicted upper bound 28.691439,actual reward 30.030348
    round 246, predicted reward 24.119923,predicted upper bound 24.142349,actual reward 20.881649
    round 247, predicted reward 25.343047,predicted upper bound 25.364024,actual reward 28.995775
    round 248, predicted reward 23.144703,predicted upper bound 23.162487,actual reward 21.266792
    round 249, predicted reward 21.427712,predicted upper bound 21.447417,actual reward 19.554265
    round 250, predicted reward 21.636136,predicted upper bound 21.659186,actual reward 26.772540
    round 251, predicted reward 31.176602,predicted upper bound 31.195665,actual reward 28.331387
    round 252, predicted reward 36.630393,predicted upper bound 36.650718,actual reward 46.832335
    round 253, predicted reward 20.950706,predicted upper bound 20.969237,actual reward 22.948096
    round 254, predicted reward 22.155958,predicted upper bound 22.178221,actual reward 19.693619
    round 255, predicted reward 22.338459,predicted upper bound 22.362027,actual reward 22.297879
    round 256, predicted reward 21.667692,predicted upper bound 21.686601,actual reward 19.622319
    round 257, predicted reward 24.916570,predicted upper bound 24.939415,actual reward 19.820018
    round 258, predicted reward 25.917908,predicted upper bound 25.939345,actual reward 23.881907
    round 259, predicted reward 20.474679,predicted upper bound 20.494392,actual reward 17.318144
    round 260, predicted reward 23.844313,predicted upper bound 23.866704,actual reward 26.436840
    round 261, predicted reward 27.781794,predicted upper bound 27.803426,actual reward 30.587766
    round 262, predicted reward 32.721881,predicted upper bound 32.744889,actual reward 29.084829
    round 263, predicted reward 25.475526,predicted upper bound 25.496651,actual reward 20.192529
    round 264, predicted reward 26.262947,predicted upper bound 26.286297,actual reward 22.712370
    round 265, predicted reward 26.607803,predicted upper bound 26.628503,actual reward 22.873040
    round 266, predicted reward 28.254518,predicted upper bound 28.272194,actual reward 34.175406
    round 267, predicted reward 22.901525,predicted upper bound 22.923126,actual reward 24.177456
    round 268, predicted reward 28.249704,predicted upper bound 28.268581,actual reward 29.121999
    round 269, predicted reward 20.692437,predicted upper bound 20.713129,actual reward 20.514044
    round 270, predicted reward 21.819916,predicted upper bound 21.837290,actual reward 18.121877
    round 271, predicted reward 27.451308,predicted upper bound 27.472579,actual reward 26.632305
    round 272, predicted reward 19.079472,predicted upper bound 19.100488,actual reward 22.698227
    round 273, predicted reward 35.684243,predicted upper bound 35.701899,actual reward 37.236786
    round 274, predicted reward 23.962647,predicted upper bound 23.983628,actual reward 27.147218
    round 275, predicted reward 26.473766,predicted upper bound 26.495858,actual reward 29.143695
    round 276, predicted reward 30.241188,predicted upper bound 30.258147,actual reward 26.717140
    round 277, predicted reward 23.250596,predicted upper bound 23.271696,actual reward 30.272687
    round 278, predicted reward 32.923956,predicted upper bound 32.944329,actual reward 33.749474
    round 279, predicted reward 30.712262,predicted upper bound 30.729232,actual reward 32.950217
    round 280, predicted reward 26.457784,predicted upper bound 26.478405,actual reward 22.176620
    round 281, predicted reward 26.611222,predicted upper bound 26.630940,actual reward 36.661694
    round 282, predicted reward 27.440935,predicted upper bound 27.461005,actual reward 27.095449
    round 283, predicted reward 25.948729,predicted upper bound 25.966879,actual reward 25.535366
    round 284, predicted reward 22.242942,predicted upper bound 22.261223,actual reward 23.175035
    round 285, predicted reward 24.737832,predicted upper bound 24.756111,actual reward 30.339552
    round 286, predicted reward 22.579617,predicted upper bound 22.595930,actual reward 19.810297
    round 287, predicted reward 24.738486,predicted upper bound 24.757494,actual reward 31.463143
    round 288, predicted reward 27.479055,predicted upper bound 27.498112,actual reward 28.569927
    round 289, predicted reward 24.717680,predicted upper bound 24.734967,actual reward 24.159995
    round 290, predicted reward 25.796865,predicted upper bound 25.813974,actual reward 30.024971
    round 291, predicted reward 27.134781,predicted upper bound 27.154967,actual reward 28.335806
    round 292, predicted reward 15.223371,predicted upper bound 15.243954,actual reward 16.750180
    round 293, predicted reward 28.813027,predicted upper bound 28.831978,actual reward 25.300563
    round 294, predicted reward 21.027566,predicted upper bound 21.048434,actual reward 20.862800
    round 295, predicted reward 22.834194,predicted upper bound 22.849518,actual reward 25.343369
    round 296, predicted reward 27.278277,predicted upper bound 27.297304,actual reward 25.306339
    round 297, predicted reward 27.281453,predicted upper bound 27.300997,actual reward 33.525973
    round 298, predicted reward 21.839927,predicted upper bound 21.857256,actual reward 25.263409
    round 299, predicted reward 19.107514,predicted upper bound 19.127771,actual reward 20.158310
    round 300, predicted reward 23.011957,predicted upper bound 23.033224,actual reward 28.929577
    round 301, predicted reward 18.981318,predicted upper bound 19.002418,actual reward 21.694897
    round 302, predicted reward 19.618039,predicted upper bound 19.638648,actual reward 15.073932
    round 303, predicted reward 26.984925,predicted upper bound 27.004683,actual reward 21.818682
    round 304, predicted reward 28.930407,predicted upper bound 28.950379,actual reward 27.862276
    round 305, predicted reward 22.254693,predicted upper bound 22.275282,actual reward 25.712432
    round 306, predicted reward 26.568585,predicted upper bound 26.588521,actual reward 24.579077
    round 307, predicted reward 27.272811,predicted upper bound 27.294045,actual reward 22.406017
    round 308, predicted reward 28.679666,predicted upper bound 28.698445,actual reward 26.572130
    round 309, predicted reward 24.043329,predicted upper bound 24.061231,actual reward 26.227653
    round 310, predicted reward 21.986542,predicted upper bound 22.006272,actual reward 21.895643
    round 311, predicted reward 23.526961,predicted upper bound 23.546590,actual reward 26.350202
    round 312, predicted reward 18.792024,predicted upper bound 18.808728,actual reward 18.288657
    round 313, predicted reward 23.408697,predicted upper bound 23.428572,actual reward 23.873949
    round 314, predicted reward 35.892369,predicted upper bound 35.907572,actual reward 35.002580
    round 315, predicted reward 27.093949,predicted upper bound 27.113734,actual reward 25.389419
    round 316, predicted reward 27.660863,predicted upper bound 27.679490,actual reward 27.684901
    round 317, predicted reward 23.626363,predicted upper bound 23.645869,actual reward 22.113029
    round 318, predicted reward 27.360997,predicted upper bound 27.382564,actual reward 32.460627
    round 319, predicted reward 25.476729,predicted upper bound 25.496393,actual reward 20.711017
    round 320, predicted reward 29.121892,predicted upper bound 29.138899,actual reward 32.015616
    round 321, predicted reward 28.115336,predicted upper bound 28.132507,actual reward 26.518370
    round 322, predicted reward 26.479674,predicted upper bound 26.499280,actual reward 28.522367
    round 323, predicted reward 35.052191,predicted upper bound 35.071422,actual reward 37.102618
    round 324, predicted reward 34.394233,predicted upper bound 34.410351,actual reward 39.777010
    round 325, predicted reward 25.519277,predicted upper bound 25.540961,actual reward 23.186744
    round 326, predicted reward 25.593085,predicted upper bound 25.612633,actual reward 24.624234
    round 327, predicted reward 34.547747,predicted upper bound 34.565951,actual reward 37.901271
    round 328, predicted reward 26.478763,predicted upper bound 26.496553,actual reward 27.292871
    round 329, predicted reward 20.856017,predicted upper bound 20.873737,actual reward 23.897103
    round 330, predicted reward 32.895425,predicted upper bound 32.910775,actual reward 36.577826
    round 331, predicted reward 27.486014,predicted upper bound 27.507909,actual reward 23.667333
    round 332, predicted reward 18.533416,predicted upper bound 18.552301,actual reward 18.753208
    round 333, predicted reward 24.511399,predicted upper bound 24.528739,actual reward 26.157284
    round 334, predicted reward 25.894504,predicted upper bound 25.913909,actual reward 36.517846
    round 335, predicted reward 25.863861,predicted upper bound 25.883343,actual reward 20.100779
    round 336, predicted reward 18.252186,predicted upper bound 18.273491,actual reward 19.283114
    round 337, predicted reward 24.148518,predicted upper bound 24.165453,actual reward 19.552147
    round 338, predicted reward 22.465870,predicted upper bound 22.487930,actual reward 18.442350
    round 339, predicted reward 24.500859,predicted upper bound 24.519219,actual reward 22.581162
    round 340, predicted reward 26.984431,predicted upper bound 27.002671,actual reward 31.554546
    round 341, predicted reward 31.103481,predicted upper bound 31.116772,actual reward 29.557085
    round 342, predicted reward 24.372299,predicted upper bound 24.389145,actual reward 25.890633
    round 343, predicted reward 24.603341,predicted upper bound 24.619047,actual reward 23.374907
    round 344, predicted reward 23.549593,predicted upper bound 23.567195,actual reward 27.221957
    round 345, predicted reward 19.538368,predicted upper bound 19.560089,actual reward 16.373085
    round 346, predicted reward 24.424098,predicted upper bound 24.438718,actual reward 24.687748
    round 347, predicted reward 25.923067,predicted upper bound 25.942428,actual reward 23.183493
    round 348, predicted reward 24.019259,predicted upper bound 24.040216,actual reward 25.481358
    round 349, predicted reward 24.575239,predicted upper bound 24.596584,actual reward 19.176036
    round 350, predicted reward 26.100265,predicted upper bound 26.118707,actual reward 22.849963
    round 351, predicted reward 27.588916,predicted upper bound 27.609385,actual reward 29.524987
    round 352, predicted reward 22.789218,predicted upper bound 22.809635,actual reward 20.253380
    round 353, predicted reward 20.835463,predicted upper bound 20.854509,actual reward 15.682392
    round 354, predicted reward 25.056255,predicted upper bound 25.076520,actual reward 24.932531
    round 355, predicted reward 30.649971,predicted upper bound 30.664866,actual reward 34.599706
    round 356, predicted reward 29.660338,predicted upper bound 29.676242,actual reward 27.675229
    round 357, predicted reward 32.141569,predicted upper bound 32.158894,actual reward 30.918927
    round 358, predicted reward 27.603080,predicted upper bound 27.620777,actual reward 26.866438
    round 359, predicted reward 24.916010,predicted upper bound 24.935426,actual reward 22.970294
    round 360, predicted reward 30.136226,predicted upper bound 30.154940,actual reward 30.993208
    round 361, predicted reward 22.610151,predicted upper bound 22.630500,actual reward 17.958827
    round 362, predicted reward 26.003524,predicted upper bound 26.020202,actual reward 32.321188
    round 363, predicted reward 22.741048,predicted upper bound 22.758170,actual reward 21.497563
    round 364, predicted reward 27.967536,predicted upper bound 27.985663,actual reward 29.934508
    round 365, predicted reward 21.382053,predicted upper bound 21.402890,actual reward 15.067486
    round 366, predicted reward 22.969964,predicted upper bound 22.985946,actual reward 24.209817
    round 367, predicted reward 27.706727,predicted upper bound 27.726993,actual reward 26.244708
    round 368, predicted reward 28.811462,predicted upper bound 28.827135,actual reward 33.797131
    round 369, predicted reward 34.283689,predicted upper bound 34.297439,actual reward 36.985730
    round 370, predicted reward 33.013575,predicted upper bound 33.028599,actual reward 42.608397
    round 371, predicted reward 27.696474,predicted upper bound 27.713003,actual reward 28.193921
    round 372, predicted reward 21.556794,predicted upper bound 21.574337,actual reward 25.651449
    round 373, predicted reward 21.715519,predicted upper bound 21.732515,actual reward 24.252629
    round 374, predicted reward 25.304167,predicted upper bound 25.320929,actual reward 22.635578
    round 375, predicted reward 20.716057,predicted upper bound 20.733777,actual reward 16.601546
    round 376, predicted reward 26.395439,predicted upper bound 26.412791,actual reward 27.284897
    round 377, predicted reward 25.547962,predicted upper bound 25.565773,actual reward 28.751471
    round 378, predicted reward 28.941368,predicted upper bound 28.960950,actual reward 29.706052
    round 379, predicted reward 24.231701,predicted upper bound 24.249281,actual reward 25.431377
    round 380, predicted reward 28.259054,predicted upper bound 28.273904,actual reward 25.111188
    round 381, predicted reward 24.040467,predicted upper bound 24.059104,actual reward 25.249773
    round 382, predicted reward 31.944053,predicted upper bound 31.960280,actual reward 32.192478
    round 383, predicted reward 27.029066,predicted upper bound 27.046422,actual reward 31.494182
    round 384, predicted reward 21.787705,predicted upper bound 21.803026,actual reward 24.562237
    round 385, predicted reward 22.328852,predicted upper bound 22.347852,actual reward 15.964167
    round 386, predicted reward 26.983816,predicted upper bound 26.998211,actual reward 30.045357
    round 387, predicted reward 27.951507,predicted upper bound 27.969550,actual reward 24.336679
    round 388, predicted reward 25.987478,predicted upper bound 26.003452,actual reward 26.429914
    round 389, predicted reward 21.693097,predicted upper bound 21.710411,actual reward 21.191616
    round 390, predicted reward 23.032123,predicted upper bound 23.050879,actual reward 15.807913
    round 391, predicted reward 28.969273,predicted upper bound 28.982553,actual reward 34.917404
    round 392, predicted reward 30.987251,predicted upper bound 31.000984,actual reward 31.328443
    round 393, predicted reward 28.799859,predicted upper bound 28.813874,actual reward 26.352679
    round 394, predicted reward 29.058180,predicted upper bound 29.074753,actual reward 32.232722
    round 395, predicted reward 26.692433,predicted upper bound 26.708426,actual reward 26.511006
    round 396, predicted reward 26.653988,predicted upper bound 26.672630,actual reward 27.513520
    round 397, predicted reward 21.270435,predicted upper bound 21.290159,actual reward 16.973643
    round 398, predicted reward 30.493747,predicted upper bound 30.508673,actual reward 30.279113
    round 399, predicted reward 25.472415,predicted upper bound 25.487767,actual reward 25.327879
    round 400, predicted reward 34.165869,predicted upper bound 34.180056,actual reward 29.459556
    round 401, predicted reward 23.809432,predicted upper bound 23.824664,actual reward 23.043277
    round 402, predicted reward 22.794905,predicted upper bound 22.816651,actual reward 21.797900
    round 403, predicted reward 22.428384,predicted upper bound 22.443576,actual reward 14.628479
    round 404, predicted reward 24.984283,predicted upper bound 25.003574,actual reward 25.922631
    round 405, predicted reward 27.882618,predicted upper bound 27.899270,actual reward 29.442347
    round 406, predicted reward 21.853912,predicted upper bound 21.870298,actual reward 24.691268
    round 407, predicted reward 23.476167,predicted upper bound 23.491613,actual reward 23.268864
    round 408, predicted reward 26.286944,predicted upper bound 26.303432,actual reward 28.577532
    round 409, predicted reward 24.609468,predicted upper bound 24.624984,actual reward 25.807924
    round 410, predicted reward 21.579328,predicted upper bound 21.596000,actual reward 18.658583
    round 411, predicted reward 33.879143,predicted upper bound 33.894579,actual reward 38.664597
    round 412, predicted reward 26.199254,predicted upper bound 26.214427,actual reward 25.312893
    round 413, predicted reward 21.286987,predicted upper bound 21.305400,actual reward 22.667175
    round 414, predicted reward 21.066425,predicted upper bound 21.085235,actual reward 22.018669
    round 415, predicted reward 24.891024,predicted upper bound 24.908359,actual reward 27.506958
    round 416, predicted reward 21.710575,predicted upper bound 21.728240,actual reward 17.400607
    round 417, predicted reward 20.493534,predicted upper bound 20.510665,actual reward 21.428317
    round 418, predicted reward 23.814785,predicted upper bound 23.831772,actual reward 24.343636
    round 419, predicted reward 26.728624,predicted upper bound 26.744180,actual reward 29.031543
    round 420, predicted reward 25.618887,predicted upper bound 25.637683,actual reward 26.801461
    round 421, predicted reward 31.647766,predicted upper bound 31.662346,actual reward 34.153312
    round 422, predicted reward 35.550705,predicted upper bound 35.564205,actual reward 36.352535
    round 423, predicted reward 25.179455,predicted upper bound 25.197426,actual reward 24.950913
    round 424, predicted reward 28.022263,predicted upper bound 28.037757,actual reward 31.216991
    round 425, predicted reward 25.770606,predicted upper bound 25.786393,actual reward 26.009563
    round 426, predicted reward 27.999365,predicted upper bound 28.014322,actual reward 23.009981
    round 427, predicted reward 30.387061,predicted upper bound 30.402277,actual reward 33.157724
    round 428, predicted reward 31.363332,predicted upper bound 31.375335,actual reward 35.218025
    round 429, predicted reward 28.116772,predicted upper bound 28.134481,actual reward 26.562262
    round 430, predicted reward 21.707069,predicted upper bound 21.724309,actual reward 19.483794
    round 431, predicted reward 26.431525,predicted upper bound 26.445658,actual reward 24.085927
    round 432, predicted reward 22.253566,predicted upper bound 22.270537,actual reward 28.638007
    round 433, predicted reward 21.828722,predicted upper bound 21.844129,actual reward 18.118544
    round 434, predicted reward 29.057391,predicted upper bound 29.070756,actual reward 24.973080
    round 435, predicted reward 30.342250,predicted upper bound 30.354673,actual reward 32.569962
    round 436, predicted reward 22.900178,predicted upper bound 22.917357,actual reward 21.763968
    round 437, predicted reward 30.242726,predicted upper bound 30.258126,actual reward 37.796753
    round 438, predicted reward 25.884359,predicted upper bound 25.900769,actual reward 30.058727
    round 439, predicted reward 30.421777,predicted upper bound 30.435939,actual reward 30.963014
    round 440, predicted reward 25.672133,predicted upper bound 25.686974,actual reward 26.847850
    round 441, predicted reward 23.569586,predicted upper bound 23.583027,actual reward 22.491834
    round 442, predicted reward 29.483692,predicted upper bound 29.499081,actual reward 28.524658
    round 443, predicted reward 36.352439,predicted upper bound 36.364120,actual reward 36.922999
    round 444, predicted reward 25.540743,predicted upper bound 25.558617,actual reward 28.695512
    round 445, predicted reward 25.136522,predicted upper bound 25.153018,actual reward 25.643233
    round 446, predicted reward 25.363969,predicted upper bound 25.378697,actual reward 29.576295
    round 447, predicted reward 27.200960,predicted upper bound 27.219135,actual reward 23.007766
    round 448, predicted reward 27.093009,predicted upper bound 27.106942,actual reward 25.825975
    round 449, predicted reward 25.276567,predicted upper bound 25.289643,actual reward 21.424413
    round 450, predicted reward 38.644627,predicted upper bound 38.655068,actual reward 39.843273
    round 451, predicted reward 24.071545,predicted upper bound 24.087954,actual reward 20.728969
    round 452, predicted reward 30.262285,predicted upper bound 30.277424,actual reward 26.825522
    round 453, predicted reward 23.104371,predicted upper bound 23.122905,actual reward 26.949321
    round 454, predicted reward 29.638855,predicted upper bound 29.653458,actual reward 35.075624
    round 455, predicted reward 29.650331,predicted upper bound 29.667744,actual reward 27.514945
    round 456, predicted reward 28.237349,predicted upper bound 28.255932,actual reward 31.325534
    round 457, predicted reward 26.476769,predicted upper bound 26.490851,actual reward 31.121621
    round 458, predicted reward 25.534189,predicted upper bound 25.547933,actual reward 23.169431
    round 459, predicted reward 28.621635,predicted upper bound 28.638469,actual reward 31.099677
    round 460, predicted reward 27.139624,predicted upper bound 27.154866,actual reward 22.503277
    round 461, predicted reward 21.399211,predicted upper bound 21.414527,actual reward 20.381082
    round 462, predicted reward 26.570639,predicted upper bound 26.585969,actual reward 28.307946
    round 463, predicted reward 32.368089,predicted upper bound 32.382522,actual reward 35.578016
    round 464, predicted reward 19.394188,predicted upper bound 19.414356,actual reward 19.398505
    round 465, predicted reward 26.379518,predicted upper bound 26.396672,actual reward 24.817552
    round 466, predicted reward 26.075640,predicted upper bound 26.093163,actual reward 29.883927
    round 467, predicted reward 25.472238,predicted upper bound 25.490848,actual reward 22.201107
    round 468, predicted reward 29.145126,predicted upper bound 29.161465,actual reward 32.141100
    round 469, predicted reward 22.874865,predicted upper bound 22.890779,actual reward 22.751550
    round 470, predicted reward 24.835682,predicted upper bound 24.852573,actual reward 19.149374
    round 471, predicted reward 23.908657,predicted upper bound 23.925300,actual reward 22.568943
    round 472, predicted reward 24.750952,predicted upper bound 24.764831,actual reward 22.616045
    round 473, predicted reward 26.697952,predicted upper bound 26.710885,actual reward 27.100263
    round 474, predicted reward 24.219802,predicted upper bound 24.236601,actual reward 19.999634
    round 475, predicted reward 22.793237,predicted upper bound 22.809101,actual reward 21.467004
    round 476, predicted reward 23.935144,predicted upper bound 23.951136,actual reward 26.200421
    round 477, predicted reward 29.116324,predicted upper bound 29.130758,actual reward 26.871476
    round 478, predicted reward 20.227742,predicted upper bound 20.246112,actual reward 22.290762
    round 479, predicted reward 27.600397,predicted upper bound 27.617862,actual reward 28.702833
    round 480, predicted reward 25.574860,predicted upper bound 25.590149,actual reward 20.272970
    round 481, predicted reward 32.186276,predicted upper bound 32.200971,actual reward 30.999253
    round 482, predicted reward 33.758654,predicted upper bound 33.773866,actual reward 35.149415
    round 483, predicted reward 25.535650,predicted upper bound 25.552432,actual reward 26.986683
    round 484, predicted reward 24.173798,predicted upper bound 24.191209,actual reward 25.560679
    round 485, predicted reward 30.256448,predicted upper bound 30.272604,actual reward 27.796142
    round 486, predicted reward 37.744111,predicted upper bound 37.756677,actual reward 40.331263
    round 487, predicted reward 29.293546,predicted upper bound 29.311445,actual reward 30.606174
    round 488, predicted reward 22.548467,predicted upper bound 22.566067,actual reward 13.197247
    round 489, predicted reward 23.395587,predicted upper bound 23.411309,actual reward 21.987194
    round 490, predicted reward 26.758036,predicted upper bound 26.772508,actual reward 22.543846
    round 491, predicted reward 29.729294,predicted upper bound 29.745568,actual reward 33.792337
    round 492, predicted reward 26.732969,predicted upper bound 26.748972,actual reward 25.420548
    round 493, predicted reward 25.044596,predicted upper bound 25.062720,actual reward 26.518000
    round 494, predicted reward 22.817496,predicted upper bound 22.833472,actual reward 25.089354
    round 495, predicted reward 31.110455,predicted upper bound 31.123634,actual reward 31.028541
    round 496, predicted reward 24.704976,predicted upper bound 24.719812,actual reward 23.231921
    round 497, predicted reward 23.409165,predicted upper bound 23.424707,actual reward 28.124001
    round 498, predicted reward 27.995826,predicted upper bound 28.010317,actual reward 25.278982
    round 499, predicted reward 22.585831,predicted upper bound 22.602168,actual reward 21.982998
    round 500, predicted reward 26.931161,predicted upper bound 26.947908,actual reward 31.246715
    round 501, predicted reward 24.310699,predicted upper bound 24.325178,actual reward 22.486606
    round 502, predicted reward 24.866481,predicted upper bound 24.882253,actual reward 29.015511
    round 503, predicted reward 29.831992,predicted upper bound 29.849843,actual reward 35.418033
    round 504, predicted reward 31.246927,predicted upper bound 31.263285,actual reward 30.697924
    round 505, predicted reward 25.119634,predicted upper bound 25.131890,actual reward 25.708865
    round 506, predicted reward 27.981628,predicted upper bound 27.998031,actual reward 30.143438
    round 507, predicted reward 28.712107,predicted upper bound 28.728409,actual reward 24.942963
    round 508, predicted reward 20.882081,predicted upper bound 20.899384,actual reward 19.902480
    round 509, predicted reward 23.935727,predicted upper bound 23.953166,actual reward 18.993417
    round 510, predicted reward 27.175284,predicted upper bound 27.188176,actual reward 28.954062
    round 511, predicted reward 27.373657,predicted upper bound 27.388531,actual reward 25.700509
    round 512, predicted reward 23.041380,predicted upper bound 23.054818,actual reward 25.404289
    round 513, predicted reward 26.849626,predicted upper bound 26.863553,actual reward 26.043686
    round 514, predicted reward 30.267237,predicted upper bound 30.281412,actual reward 28.184198
    round 515, predicted reward 30.344908,predicted upper bound 30.358385,actual reward 29.728060
    round 516, predicted reward 21.288392,predicted upper bound 21.301384,actual reward 13.408060
    round 517, predicted reward 28.133802,predicted upper bound 28.153060,actual reward 34.372860
    round 518, predicted reward 23.686337,predicted upper bound 23.703776,actual reward 23.433568
    round 519, predicted reward 23.182246,predicted upper bound 23.197618,actual reward 22.413465
    round 520, predicted reward 22.398971,predicted upper bound 22.414713,actual reward 22.541023
    round 521, predicted reward 22.671854,predicted upper bound 22.690008,actual reward 17.732196
    round 522, predicted reward 29.233865,predicted upper bound 29.246549,actual reward 26.904329
    round 523, predicted reward 27.729657,predicted upper bound 27.743671,actual reward 26.215404
    round 524, predicted reward 31.743227,predicted upper bound 31.757807,actual reward 29.132666
    round 525, predicted reward 31.284375,predicted upper bound 31.301504,actual reward 30.675108
    round 526, predicted reward 31.979924,predicted upper bound 31.993030,actual reward 37.544652
    round 527, predicted reward 31.471695,predicted upper bound 31.488914,actual reward 31.384870
    round 528, predicted reward 32.938010,predicted upper bound 32.952225,actual reward 39.620412
    round 529, predicted reward 24.932454,predicted upper bound 24.949097,actual reward 22.256554
    round 530, predicted reward 25.602341,predicted upper bound 25.616906,actual reward 20.942927
    round 531, predicted reward 24.938046,predicted upper bound 24.953052,actual reward 24.120332
    round 532, predicted reward 23.823877,predicted upper bound 23.841309,actual reward 23.732569
    round 533, predicted reward 25.335646,predicted upper bound 25.349301,actual reward 27.575525
    round 534, predicted reward 34.059130,predicted upper bound 34.070557,actual reward 33.674405
    round 535, predicted reward 30.825005,predicted upper bound 30.838504,actual reward 30.498603
    round 536, predicted reward 25.559143,predicted upper bound 25.572932,actual reward 23.679405
    round 537, predicted reward 29.456657,predicted upper bound 29.471729,actual reward 32.692675
    round 538, predicted reward 27.162760,predicted upper bound 27.178843,actual reward 28.989715
    round 539, predicted reward 21.521821,predicted upper bound 21.536109,actual reward 26.213913
    round 540, predicted reward 34.206068,predicted upper bound 34.221215,actual reward 35.296488
    round 541, predicted reward 26.909462,predicted upper bound 26.923067,actual reward 26.959677
    round 542, predicted reward 23.395673,predicted upper bound 23.410372,actual reward 21.100494
    round 543, predicted reward 29.143683,predicted upper bound 29.159822,actual reward 30.079592
    round 544, predicted reward 26.387360,predicted upper bound 26.403096,actual reward 23.334370
    round 545, predicted reward 22.011177,predicted upper bound 22.023784,actual reward 22.166315
    round 546, predicted reward 22.498167,predicted upper bound 22.515329,actual reward 22.248599
    round 547, predicted reward 33.036011,predicted upper bound 33.049075,actual reward 33.031173
    round 548, predicted reward 23.656828,predicted upper bound 23.670354,actual reward 22.013536
    round 549, predicted reward 28.482609,predicted upper bound 28.499154,actual reward 30.634164
    round 550, predicted reward 26.217812,predicted upper bound 26.231553,actual reward 30.177168
    round 551, predicted reward 28.120700,predicted upper bound 28.137117,actual reward 28.219999
    round 552, predicted reward 29.341951,predicted upper bound 29.354802,actual reward 32.466618
    round 553, predicted reward 33.775906,predicted upper bound 33.787621,actual reward 37.511635
    round 554, predicted reward 24.671900,predicted upper bound 24.683964,actual reward 23.757741
    round 555, predicted reward 28.011424,predicted upper bound 28.023168,actual reward 31.142057
    round 556, predicted reward 31.431850,predicted upper bound 31.446744,actual reward 32.013341
    round 557, predicted reward 34.202273,predicted upper bound 34.214930,actual reward 31.861755
    round 558, predicted reward 26.031261,predicted upper bound 26.043032,actual reward 25.419425
    round 559, predicted reward 22.135163,predicted upper bound 22.149361,actual reward 17.188549
    round 560, predicted reward 20.800725,predicted upper bound 20.818385,actual reward 19.559349
    round 561, predicted reward 28.614820,predicted upper bound 28.629547,actual reward 21.170586
    round 562, predicted reward 35.519158,predicted upper bound 35.529062,actual reward 37.555099
    round 563, predicted reward 22.833849,predicted upper bound 22.846722,actual reward 28.406143
    round 564, predicted reward 28.336841,predicted upper bound 28.348632,actual reward 23.826132
    round 565, predicted reward 22.118057,predicted upper bound 22.132471,actual reward 21.511176
    round 566, predicted reward 28.581254,predicted upper bound 28.595763,actual reward 23.542903
    round 567, predicted reward 28.844317,predicted upper bound 28.857464,actual reward 29.689281
    round 568, predicted reward 27.853755,predicted upper bound 27.869200,actual reward 28.481667
    round 569, predicted reward 25.237962,predicted upper bound 25.252839,actual reward 22.495750
    round 570, predicted reward 28.642019,predicted upper bound 28.652539,actual reward 24.409900
    round 571, predicted reward 22.148535,predicted upper bound 22.164807,actual reward 23.771918
    round 572, predicted reward 24.504439,predicted upper bound 24.518327,actual reward 22.314191
    round 573, predicted reward 24.241904,predicted upper bound 24.255969,actual reward 18.886456
    round 574, predicted reward 31.280969,predicted upper bound 31.296957,actual reward 30.542736
    round 575, predicted reward 28.464712,predicted upper bound 28.476020,actual reward 26.518754
    round 576, predicted reward 24.191523,predicted upper bound 24.209152,actual reward 18.616807
    round 577, predicted reward 24.629994,predicted upper bound 24.643683,actual reward 29.035239
    round 578, predicted reward 29.298599,predicted upper bound 29.310255,actual reward 34.295028
    round 579, predicted reward 29.950593,predicted upper bound 29.965291,actual reward 34.156378
    round 580, predicted reward 34.974815,predicted upper bound 34.985616,actual reward 35.266102
    round 581, predicted reward 26.993075,predicted upper bound 27.008885,actual reward 31.942121
    round 582, predicted reward 33.288810,predicted upper bound 33.298989,actual reward 34.222106
    round 583, predicted reward 28.790474,predicted upper bound 28.803777,actual reward 30.286428
    round 584, predicted reward 23.785150,predicted upper bound 23.801271,actual reward 24.338064
    round 585, predicted reward 22.033484,predicted upper bound 22.051354,actual reward 17.628701
    round 586, predicted reward 29.844004,predicted upper bound 29.855761,actual reward 27.003718
    round 587, predicted reward 24.235717,predicted upper bound 24.251406,actual reward 21.423584
    round 588, predicted reward 33.774264,predicted upper bound 33.787505,actual reward 38.654748
    round 589, predicted reward 32.688382,predicted upper bound 32.702062,actual reward 30.702832
    round 590, predicted reward 32.295844,predicted upper bound 32.308416,actual reward 31.683060
    round 591, predicted reward 20.349055,predicted upper bound 20.361548,actual reward 15.099992
    round 592, predicted reward 28.676019,predicted upper bound 28.688924,actual reward 31.455172
    round 593, predicted reward 22.940663,predicted upper bound 22.951761,actual reward 28.149118
    round 594, predicted reward 23.292972,predicted upper bound 23.307189,actual reward 20.619158
    round 595, predicted reward 29.264048,predicted upper bound 29.277485,actual reward 24.644965
    round 596, predicted reward 27.117096,predicted upper bound 27.130434,actual reward 25.339570
    round 597, predicted reward 22.688290,predicted upper bound 22.701748,actual reward 23.348106
    round 598, predicted reward 32.308630,predicted upper bound 32.321053,actual reward 33.008111
    round 599, predicted reward 26.553441,predicted upper bound 26.568146,actual reward 25.427124
    round 600, predicted reward 25.049846,predicted upper bound 25.064631,actual reward 24.177203
    round 601, predicted reward 21.919285,predicted upper bound 21.933291,actual reward 22.822126
    round 602, predicted reward 29.782101,predicted upper bound 29.794099,actual reward 30.686638
    round 603, predicted reward 25.802484,predicted upper bound 25.815288,actual reward 28.760809
    round 604, predicted reward 23.504970,predicted upper bound 23.519604,actual reward 27.137133
    round 605, predicted reward 30.751474,predicted upper bound 30.765474,actual reward 34.666523
    round 606, predicted reward 25.728789,predicted upper bound 25.743128,actual reward 21.846951
    round 607, predicted reward 20.285344,predicted upper bound 20.299037,actual reward 14.619994
    round 608, predicted reward 32.853706,predicted upper bound 32.864657,actual reward 33.445563
    round 609, predicted reward 26.307635,predicted upper bound 26.320208,actual reward 25.851883
    round 610, predicted reward 23.895676,predicted upper bound 23.908688,actual reward 29.970922
    round 611, predicted reward 25.618939,predicted upper bound 25.632228,actual reward 22.823494
    round 612, predicted reward 34.313677,predicted upper bound 34.326144,actual reward 33.442853
    round 613, predicted reward 26.531160,predicted upper bound 26.543008,actual reward 23.414105
    round 614, predicted reward 24.298306,predicted upper bound 24.313845,actual reward 21.727190
    round 615, predicted reward 30.084970,predicted upper bound 30.097229,actual reward 35.754270
    round 616, predicted reward 35.340616,predicted upper bound 35.353465,actual reward 35.674963
    round 617, predicted reward 25.806906,predicted upper bound 25.821536,actual reward 26.065292
    round 618, predicted reward 22.988087,predicted upper bound 23.002049,actual reward 23.164118
    round 619, predicted reward 29.014622,predicted upper bound 29.026977,actual reward 25.470699
    round 620, predicted reward 29.912038,predicted upper bound 29.927311,actual reward 24.383353
    round 621, predicted reward 26.797193,predicted upper bound 26.811688,actual reward 20.565877
    round 622, predicted reward 27.843564,predicted upper bound 27.859101,actual reward 32.454031
    round 623, predicted reward 32.444582,predicted upper bound 32.457112,actual reward 33.350110
    round 624, predicted reward 25.480746,predicted upper bound 25.492731,actual reward 22.177569
    round 625, predicted reward 24.962072,predicted upper bound 24.977718,actual reward 22.186437
    round 626, predicted reward 27.426681,predicted upper bound 27.438734,actual reward 29.813857
    round 627, predicted reward 25.986337,predicted upper bound 26.000267,actual reward 20.793318
    round 628, predicted reward 28.502355,predicted upper bound 28.516062,actual reward 26.730474
    round 629, predicted reward 23.534040,predicted upper bound 23.549334,actual reward 22.356764
    round 630, predicted reward 21.289496,predicted upper bound 21.304718,actual reward 17.232672
    round 631, predicted reward 24.974678,predicted upper bound 24.989750,actual reward 31.618457
    round 632, predicted reward 24.990303,predicted upper bound 25.003893,actual reward 25.661305
    round 633, predicted reward 24.980352,predicted upper bound 24.991816,actual reward 21.399722
    round 634, predicted reward 23.656661,predicted upper bound 23.668887,actual reward 30.578423
    round 635, predicted reward 24.440622,predicted upper bound 24.453227,actual reward 23.241694
    round 636, predicted reward 28.266996,predicted upper bound 28.277735,actual reward 24.083239
    round 637, predicted reward 22.309860,predicted upper bound 22.323300,actual reward 22.512340
    round 638, predicted reward 25.510221,predicted upper bound 25.523745,actual reward 21.209247
    round 639, predicted reward 25.949709,predicted upper bound 25.959813,actual reward 24.105653
    round 640, predicted reward 25.410241,predicted upper bound 25.424898,actual reward 21.220096
    round 641, predicted reward 29.080304,predicted upper bound 29.089631,actual reward 34.723026
    round 642, predicted reward 35.906825,predicted upper bound 35.917428,actual reward 39.678056
    round 643, predicted reward 29.912052,predicted upper bound 29.924714,actual reward 29.512086
    round 644, predicted reward 27.121181,predicted upper bound 27.132757,actual reward 22.493152
    round 645, predicted reward 28.900565,predicted upper bound 28.912025,actual reward 33.352456
    round 646, predicted reward 25.003507,predicted upper bound 25.015143,actual reward 22.845018
    round 647, predicted reward 26.591438,predicted upper bound 26.603393,actual reward 28.759942
    round 648, predicted reward 23.789573,predicted upper bound 23.802127,actual reward 17.267670
    round 649, predicted reward 27.370695,predicted upper bound 27.382695,actual reward 26.325639
    round 650, predicted reward 29.704972,predicted upper bound 29.715279,actual reward 35.209748
    round 651, predicted reward 27.823362,predicted upper bound 27.835804,actual reward 25.143489
    round 652, predicted reward 28.660976,predicted upper bound 28.673269,actual reward 33.796349
    round 653, predicted reward 27.040076,predicted upper bound 27.055326,actual reward 22.095849
    round 654, predicted reward 24.280061,predicted upper bound 24.296810,actual reward 25.152162
    round 655, predicted reward 20.665063,predicted upper bound 20.677271,actual reward 19.287291
    round 656, predicted reward 23.299256,predicted upper bound 23.314511,actual reward 19.779469
    round 657, predicted reward 25.183425,predicted upper bound 25.196896,actual reward 25.679367
    round 658, predicted reward 25.295513,predicted upper bound 25.307243,actual reward 23.875771
    round 659, predicted reward 31.953659,predicted upper bound 31.965637,actual reward 32.695160
    round 660, predicted reward 23.394320,predicted upper bound 23.408184,actual reward 19.330943
    round 661, predicted reward 30.759183,predicted upper bound 30.771723,actual reward 28.092920
    round 662, predicted reward 27.529858,predicted upper bound 27.541409,actual reward 24.917760
    round 663, predicted reward 24.527274,predicted upper bound 24.542002,actual reward 21.917614
    round 664, predicted reward 28.426453,predicted upper bound 28.440152,actual reward 30.354244
    round 665, predicted reward 30.465620,predicted upper bound 30.477416,actual reward 33.277705
    round 666, predicted reward 25.882692,predicted upper bound 25.895080,actual reward 26.798331
    round 667, predicted reward 20.000207,predicted upper bound 20.012103,actual reward 20.939162
    round 668, predicted reward 26.420549,predicted upper bound 26.435347,actual reward 26.224022
    round 669, predicted reward 25.058354,predicted upper bound 25.071141,actual reward 22.059009
    round 670, predicted reward 26.122280,predicted upper bound 26.136916,actual reward 29.723780
    round 671, predicted reward 22.765318,predicted upper bound 22.781120,actual reward 25.113181
    round 672, predicted reward 28.811271,predicted upper bound 28.822167,actual reward 24.585381
    round 673, predicted reward 20.913715,predicted upper bound 20.926728,actual reward 13.545550
    round 674, predicted reward 28.763675,predicted upper bound 28.775146,actual reward 28.433889
    round 675, predicted reward 27.122443,predicted upper bound 27.134161,actual reward 28.173626
    round 676, predicted reward 29.261312,predicted upper bound 29.273254,actual reward 27.844974
    round 677, predicted reward 31.336438,predicted upper bound 31.348944,actual reward 36.037314
    round 678, predicted reward 25.474902,predicted upper bound 25.485338,actual reward 23.620702
    round 679, predicted reward 29.255538,predicted upper bound 29.268535,actual reward 27.852726
    round 680, predicted reward 21.904424,predicted upper bound 21.918711,actual reward 22.839176
    round 681, predicted reward 22.628945,predicted upper bound 22.640298,actual reward 20.046133
    round 682, predicted reward 24.787028,predicted upper bound 24.800144,actual reward 24.206649
    round 683, predicted reward 36.986348,predicted upper bound 36.997571,actual reward 37.524737
    round 684, predicted reward 25.104974,predicted upper bound 25.118157,actual reward 24.045214
    round 685, predicted reward 26.364570,predicted upper bound 26.375034,actual reward 23.998678
    round 686, predicted reward 27.229590,predicted upper bound 27.240848,actual reward 24.711304
    round 687, predicted reward 29.217145,predicted upper bound 29.228451,actual reward 28.789713
    round 688, predicted reward 23.398924,predicted upper bound 23.411155,actual reward 18.642473
    round 689, predicted reward 20.622328,predicted upper bound 20.637457,actual reward 19.316089
    round 690, predicted reward 23.858333,predicted upper bound 23.868898,actual reward 24.842206
    round 691, predicted reward 27.811200,predicted upper bound 27.821109,actual reward 26.352195
    round 692, predicted reward 28.711179,predicted upper bound 28.721102,actual reward 30.746562
    round 693, predicted reward 35.256331,predicted upper bound 35.266644,actual reward 37.314877
    round 694, predicted reward 24.866797,predicted upper bound 24.877218,actual reward 21.127933
    round 695, predicted reward 27.931200,predicted upper bound 27.944072,actual reward 35.266085
    round 696, predicted reward 23.429944,predicted upper bound 23.442498,actual reward 28.913205
    round 697, predicted reward 36.033358,predicted upper bound 36.044033,actual reward 38.758597
    round 698, predicted reward 28.671868,predicted upper bound 28.683962,actual reward 28.634347
    round 699, predicted reward 24.041999,predicted upper bound 24.056296,actual reward 20.270469
    round 700, predicted reward 29.735634,predicted upper bound 29.744727,actual reward 32.247901
    round 701, predicted reward 23.595525,predicted upper bound 23.606773,actual reward 19.070377
    round 702, predicted reward 36.700805,predicted upper bound 36.711092,actual reward 41.972075
    round 703, predicted reward 32.518814,predicted upper bound 32.532295,actual reward 37.409698
    round 704, predicted reward 22.897298,predicted upper bound 22.914149,actual reward 18.597794
    round 705, predicted reward 27.711103,predicted upper bound 27.723500,actual reward 31.939030
    round 706, predicted reward 22.632287,predicted upper bound 22.644638,actual reward 24.291439
    round 707, predicted reward 26.059641,predicted upper bound 26.070782,actual reward 28.825064
    round 708, predicted reward 26.961535,predicted upper bound 26.975330,actual reward 26.941056
    round 709, predicted reward 26.634880,predicted upper bound 26.647799,actual reward 28.871324
    round 710, predicted reward 28.963541,predicted upper bound 28.973466,actual reward 29.118502
    round 711, predicted reward 23.327633,predicted upper bound 23.340837,actual reward 23.841794
    round 712, predicted reward 27.405371,predicted upper bound 27.417346,actual reward 28.695943
    round 713, predicted reward 23.811798,predicted upper bound 23.827238,actual reward 19.296273
    round 714, predicted reward 26.325488,predicted upper bound 26.335583,actual reward 27.371117
    round 715, predicted reward 27.373028,predicted upper bound 27.385105,actual reward 26.096060
    round 716, predicted reward 22.579025,predicted upper bound 22.591003,actual reward 15.575143
    round 717, predicted reward 24.051914,predicted upper bound 24.063732,actual reward 24.117353
    round 718, predicted reward 30.939546,predicted upper bound 30.950172,actual reward 36.429658
    round 719, predicted reward 27.403342,predicted upper bound 27.414196,actual reward 28.897613
    round 720, predicted reward 33.713592,predicted upper bound 33.723594,actual reward 31.827460
    round 721, predicted reward 26.424671,predicted upper bound 26.436747,actual reward 28.513481
    round 722, predicted reward 23.913046,predicted upper bound 23.926852,actual reward 19.212153
    round 723, predicted reward 29.658253,predicted upper bound 29.670231,actual reward 32.397014
    round 724, predicted reward 25.539232,predicted upper bound 25.551315,actual reward 25.193380
    round 725, predicted reward 27.950927,predicted upper bound 27.962350,actual reward 26.962260
    round 726, predicted reward 25.563421,predicted upper bound 25.576025,actual reward 26.341793
    round 727, predicted reward 20.718541,predicted upper bound 20.731304,actual reward 19.301114
    round 728, predicted reward 29.327862,predicted upper bound 29.340013,actual reward 31.945575
    round 729, predicted reward 27.511380,predicted upper bound 27.521273,actual reward 26.512392
    round 730, predicted reward 30.390499,predicted upper bound 30.400647,actual reward 24.148067
    round 731, predicted reward 33.060098,predicted upper bound 33.071799,actual reward 30.740461
    round 732, predicted reward 28.387689,predicted upper bound 28.398800,actual reward 32.000720
    round 733, predicted reward 22.237775,predicted upper bound 22.251164,actual reward 22.062546
    round 734, predicted reward 29.531601,predicted upper bound 29.542861,actual reward 34.018870
    round 735, predicted reward 25.074806,predicted upper bound 25.087352,actual reward 18.746153
    round 736, predicted reward 29.521365,predicted upper bound 29.532525,actual reward 32.581890
    round 737, predicted reward 25.302241,predicted upper bound 25.314031,actual reward 25.959032
    round 738, predicted reward 25.799731,predicted upper bound 25.810320,actual reward 21.836669
    round 739, predicted reward 24.689648,predicted upper bound 24.700955,actual reward 24.321185
    round 740, predicted reward 27.569243,predicted upper bound 27.582492,actual reward 36.227284
    round 741, predicted reward 23.337877,predicted upper bound 23.349635,actual reward 23.013423
    round 742, predicted reward 27.209366,predicted upper bound 27.221252,actual reward 27.545870
    round 743, predicted reward 24.541307,predicted upper bound 24.551503,actual reward 28.637498
    round 744, predicted reward 27.107219,predicted upper bound 27.119657,actual reward 24.275175
    round 745, predicted reward 22.102178,predicted upper bound 22.114272,actual reward 21.004012
    round 746, predicted reward 31.600246,predicted upper bound 31.609442,actual reward 29.055848
    round 747, predicted reward 21.980852,predicted upper bound 21.994515,actual reward 20.345785
    round 748, predicted reward 28.023262,predicted upper bound 28.034560,actual reward 28.167501
    round 749, predicted reward 23.996476,predicted upper bound 24.009304,actual reward 25.166135
    round 750, predicted reward 27.505889,predicted upper bound 27.515728,actual reward 26.185887
    round 751, predicted reward 26.856880,predicted upper bound 26.870606,actual reward 28.102724
    round 752, predicted reward 23.580675,predicted upper bound 23.592829,actual reward 21.213042
    round 753, predicted reward 24.077495,predicted upper bound 24.089747,actual reward 24.673494
    round 754, predicted reward 29.401312,predicted upper bound 29.411329,actual reward 28.514186
    round 755, predicted reward 27.098119,predicted upper bound 27.109344,actual reward 22.436178
    round 756, predicted reward 25.311490,predicted upper bound 25.322833,actual reward 24.261228
    round 757, predicted reward 26.859981,predicted upper bound 26.872726,actual reward 22.421760
    round 758, predicted reward 29.178360,predicted upper bound 29.190541,actual reward 33.263547
    round 759, predicted reward 22.593373,predicted upper bound 22.605338,actual reward 21.277976
    round 760, predicted reward 25.241146,predicted upper bound 25.252059,actual reward 17.653098
    round 761, predicted reward 33.375835,predicted upper bound 33.387263,actual reward 35.408531
    round 762, predicted reward 31.305173,predicted upper bound 31.315138,actual reward 35.802580
    round 763, predicted reward 23.235094,predicted upper bound 23.247799,actual reward 20.763785
    round 764, predicted reward 25.649072,predicted upper bound 25.659917,actual reward 24.563699
    round 765, predicted reward 28.078454,predicted upper bound 28.091238,actual reward 28.169920
    round 766, predicted reward 27.391475,predicted upper bound 27.402839,actual reward 26.972942
    round 767, predicted reward 22.373162,predicted upper bound 22.386301,actual reward 18.487825
    round 768, predicted reward 23.260673,predicted upper bound 23.271313,actual reward 20.188439
    round 769, predicted reward 21.776244,predicted upper bound 21.786961,actual reward 21.778014
    round 770, predicted reward 24.828267,predicted upper bound 24.841136,actual reward 21.501955
    round 771, predicted reward 30.512396,predicted upper bound 30.522843,actual reward 32.701877
    round 772, predicted reward 31.754116,predicted upper bound 31.764676,actual reward 38.299640
    round 773, predicted reward 28.751194,predicted upper bound 28.761058,actual reward 31.739013
    round 774, predicted reward 28.085391,predicted upper bound 28.095661,actual reward 26.138364
    round 775, predicted reward 32.363952,predicted upper bound 32.372595,actual reward 32.746799
    round 776, predicted reward 25.428522,predicted upper bound 25.440309,actual reward 23.272516
    round 777, predicted reward 20.863313,predicted upper bound 20.874534,actual reward 22.254521
    round 778, predicted reward 25.768808,predicted upper bound 25.778987,actual reward 26.284050
    round 779, predicted reward 27.032101,predicted upper bound 27.041691,actual reward 26.879204
    round 780, predicted reward 19.074132,predicted upper bound 19.088274,actual reward 18.084229
    round 781, predicted reward 26.501985,predicted upper bound 26.513074,actual reward 20.239976
    round 782, predicted reward 32.696557,predicted upper bound 32.705858,actual reward 32.285075
    round 783, predicted reward 25.281444,predicted upper bound 25.292714,actual reward 25.090832
    round 784, predicted reward 25.648193,predicted upper bound 25.658440,actual reward 25.231746
    round 785, predicted reward 26.350143,predicted upper bound 26.360697,actual reward 21.602137
    round 786, predicted reward 23.938717,predicted upper bound 23.951323,actual reward 23.117642
    round 787, predicted reward 28.097901,predicted upper bound 28.109685,actual reward 29.756016
    round 788, predicted reward 26.542701,predicted upper bound 26.552967,actual reward 23.738926
    round 789, predicted reward 20.627285,predicted upper bound 20.639080,actual reward 18.519128
    round 790, predicted reward 28.265549,predicted upper bound 28.275944,actual reward 29.255553
    round 791, predicted reward 28.812325,predicted upper bound 28.823426,actual reward 31.117293
    round 792, predicted reward 30.210461,predicted upper bound 30.219479,actual reward 28.088166
    round 793, predicted reward 29.941829,predicted upper bound 29.950875,actual reward 27.770951
    round 794, predicted reward 27.722995,predicted upper bound 27.732922,actual reward 27.252859
    round 795, predicted reward 27.054522,predicted upper bound 27.065019,actual reward 30.257823
    round 796, predicted reward 24.470999,predicted upper bound 24.482122,actual reward 27.399765
    round 797, predicted reward 28.692899,predicted upper bound 28.702418,actual reward 31.966456
    round 798, predicted reward 25.838020,predicted upper bound 25.849614,actual reward 33.282280
    round 799, predicted reward 25.147515,predicted upper bound 25.157935,actual reward 17.567362
    round 800, predicted reward 24.387634,predicted upper bound 24.397835,actual reward 20.205890
    round 801, predicted reward 23.532342,predicted upper bound 23.543964,actual reward 22.846605
    round 802, predicted reward 27.884054,predicted upper bound 27.895802,actual reward 27.986271
    round 803, predicted reward 25.692897,predicted upper bound 25.702892,actual reward 25.490356
    round 804, predicted reward 21.860758,predicted upper bound 21.874054,actual reward 22.531630
    round 805, predicted reward 20.900756,predicted upper bound 20.912118,actual reward 21.139373
    round 806, predicted reward 27.112579,predicted upper bound 27.122379,actual reward 22.910670
    round 807, predicted reward 26.842669,predicted upper bound 26.854728,actual reward 29.011071
    round 808, predicted reward 29.869483,predicted upper bound 29.880023,actual reward 33.565085
    round 809, predicted reward 30.165421,predicted upper bound 30.176192,actual reward 31.814369
    round 810, predicted reward 29.360002,predicted upper bound 29.369338,actual reward 27.037316
    round 811, predicted reward 31.384445,predicted upper bound 31.394076,actual reward 32.637457
    round 812, predicted reward 28.721762,predicted upper bound 28.731600,actual reward 28.506880
    round 813, predicted reward 29.258244,predicted upper bound 29.269890,actual reward 30.975151
    round 814, predicted reward 32.522399,predicted upper bound 32.530968,actual reward 34.698581
    round 815, predicted reward 22.272241,predicted upper bound 22.284060,actual reward 21.239080
    round 816, predicted reward 24.577344,predicted upper bound 24.587114,actual reward 23.934075
    round 817, predicted reward 26.886820,predicted upper bound 26.898005,actual reward 22.160080
    round 818, predicted reward 25.206361,predicted upper bound 25.215156,actual reward 23.149973
    round 819, predicted reward 23.342509,predicted upper bound 23.354503,actual reward 21.891322
    round 820, predicted reward 29.011169,predicted upper bound 29.022476,actual reward 28.404898
    round 821, predicted reward 19.392035,predicted upper bound 19.404908,actual reward 19.816797
    round 822, predicted reward 22.695155,predicted upper bound 22.707070,actual reward 19.232277
    round 823, predicted reward 25.106694,predicted upper bound 25.118999,actual reward 18.723017
    round 824, predicted reward 20.918101,predicted upper bound 20.930401,actual reward 21.900377
    round 825, predicted reward 25.065597,predicted upper bound 25.076606,actual reward 27.135035
    round 826, predicted reward 22.679684,predicted upper bound 22.690027,actual reward 23.404386
    round 827, predicted reward 32.762609,predicted upper bound 32.771426,actual reward 35.237724
    round 828, predicted reward 29.604668,predicted upper bound 29.615095,actual reward 28.465131
    round 829, predicted reward 23.557833,predicted upper bound 23.570601,actual reward 27.042488
    round 830, predicted reward 19.822079,predicted upper bound 19.833226,actual reward 17.842662
    round 831, predicted reward 23.996575,predicted upper bound 24.007061,actual reward 26.453348
    round 832, predicted reward 31.445175,predicted upper bound 31.454403,actual reward 31.283674
    round 833, predicted reward 25.093622,predicted upper bound 25.102587,actual reward 22.694612
    round 834, predicted reward 29.056867,predicted upper bound 29.066921,actual reward 33.531984
    round 835, predicted reward 25.875434,predicted upper bound 25.884993,actual reward 22.080369
    round 836, predicted reward 22.398874,predicted upper bound 22.409217,actual reward 19.435537
    round 837, predicted reward 25.916951,predicted upper bound 25.927759,actual reward 24.002491
    round 838, predicted reward 26.667904,predicted upper bound 26.678472,actual reward 23.815089
    round 839, predicted reward 29.723578,predicted upper bound 29.733095,actual reward 34.027456
    round 840, predicted reward 23.563872,predicted upper bound 23.574275,actual reward 23.191727
    round 841, predicted reward 21.606126,predicted upper bound 21.618139,actual reward 15.200241
    round 842, predicted reward 27.614567,predicted upper bound 27.624807,actual reward 28.100483
    round 843, predicted reward 31.783663,predicted upper bound 31.793439,actual reward 29.930818
    round 844, predicted reward 28.679346,predicted upper bound 28.689879,actual reward 28.198387
    round 845, predicted reward 26.983627,predicted upper bound 26.994259,actual reward 26.401742
    round 846, predicted reward 26.236613,predicted upper bound 26.248143,actual reward 28.183134
    round 847, predicted reward 30.324847,predicted upper bound 30.333034,actual reward 34.686238
    round 848, predicted reward 27.657172,predicted upper bound 27.669043,actual reward 29.080026
    round 849, predicted reward 33.567923,predicted upper bound 33.577540,actual reward 34.945725
    round 850, predicted reward 29.281172,predicted upper bound 29.291501,actual reward 33.113838
    round 851, predicted reward 27.828208,predicted upper bound 27.839620,actual reward 28.327010
    round 852, predicted reward 26.780135,predicted upper bound 26.793004,actual reward 23.360550
    round 853, predicted reward 31.010718,predicted upper bound 31.022966,actual reward 32.595251
    round 854, predicted reward 25.437261,predicted upper bound 25.447054,actual reward 25.783166
    round 855, predicted reward 32.365177,predicted upper bound 32.376939,actual reward 33.918042
    round 856, predicted reward 19.589394,predicted upper bound 19.600946,actual reward 20.288236
    round 857, predicted reward 33.037090,predicted upper bound 33.046668,actual reward 33.814424
    round 858, predicted reward 22.704821,predicted upper bound 22.716724,actual reward 22.037271
    round 859, predicted reward 22.154065,predicted upper bound 22.164647,actual reward 26.755302
    round 860, predicted reward 22.582001,predicted upper bound 22.590690,actual reward 22.974822
    round 861, predicted reward 23.696238,predicted upper bound 23.707091,actual reward 17.538221
    round 862, predicted reward 32.539201,predicted upper bound 32.548563,actual reward 31.386725
    round 863, predicted reward 28.585130,predicted upper bound 28.596737,actual reward 33.143655
    round 864, predicted reward 26.869282,predicted upper bound 26.880210,actual reward 23.004557
    round 865, predicted reward 23.385054,predicted upper bound 23.396534,actual reward 22.148753
    round 866, predicted reward 28.260269,predicted upper bound 28.271244,actual reward 25.940031
    round 867, predicted reward 32.670621,predicted upper bound 32.680177,actual reward 32.274319
    round 868, predicted reward 28.595282,predicted upper bound 28.604064,actual reward 29.266206
    round 869, predicted reward 32.957538,predicted upper bound 32.968569,actual reward 32.569107
    round 870, predicted reward 24.565103,predicted upper bound 24.577536,actual reward 21.856485
    round 871, predicted reward 26.010493,predicted upper bound 26.019891,actual reward 29.762548
    round 872, predicted reward 25.595294,predicted upper bound 25.606118,actual reward 22.047279
    round 873, predicted reward 27.897421,predicted upper bound 27.905966,actual reward 26.740357
    round 874, predicted reward 28.430572,predicted upper bound 28.440952,actual reward 28.342594
    round 875, predicted reward 22.001200,predicted upper bound 22.010821,actual reward 18.662396
    round 876, predicted reward 29.758096,predicted upper bound 29.768214,actual reward 32.353265
    round 877, predicted reward 34.069052,predicted upper bound 34.078145,actual reward 34.276820
    round 878, predicted reward 26.271325,predicted upper bound 26.280693,actual reward 24.155397
    round 879, predicted reward 30.551692,predicted upper bound 30.561727,actual reward 25.684772
    round 880, predicted reward 29.055314,predicted upper bound 29.065731,actual reward 30.022540
    round 881, predicted reward 26.393363,predicted upper bound 26.404094,actual reward 22.109261
    round 882, predicted reward 27.406245,predicted upper bound 27.417573,actual reward 24.616388
    round 883, predicted reward 29.127524,predicted upper bound 29.135947,actual reward 28.534049
    round 884, predicted reward 26.616957,predicted upper bound 26.630824,actual reward 28.878648
    round 885, predicted reward 29.259624,predicted upper bound 29.271902,actual reward 28.184674
    round 886, predicted reward 29.533000,predicted upper bound 29.543373,actual reward 29.632946
    round 887, predicted reward 23.918541,predicted upper bound 23.928909,actual reward 23.339982
    round 888, predicted reward 24.104271,predicted upper bound 24.115482,actual reward 20.239919
    round 889, predicted reward 30.267438,predicted upper bound 30.274758,actual reward 31.510028
    round 890, predicted reward 34.345512,predicted upper bound 34.355006,actual reward 36.470812
    round 891, predicted reward 25.752483,predicted upper bound 25.763442,actual reward 22.526545
    round 892, predicted reward 25.686934,predicted upper bound 25.698748,actual reward 22.261467
    round 893, predicted reward 35.724418,predicted upper bound 35.733945,actual reward 40.390333
    round 894, predicted reward 30.702329,predicted upper bound 30.712109,actual reward 33.925813
    round 895, predicted reward 21.747230,predicted upper bound 21.758739,actual reward 17.554719
    round 896, predicted reward 38.036557,predicted upper bound 38.045196,actual reward 39.889951
    round 897, predicted reward 23.420665,predicted upper bound 23.431270,actual reward 23.266150
    round 898, predicted reward 26.308440,predicted upper bound 26.317611,actual reward 28.337892
    round 899, predicted reward 24.821557,predicted upper bound 24.831702,actual reward 23.446093
    round 900, predicted reward 23.793568,predicted upper bound 23.802622,actual reward 22.730515
    round 901, predicted reward 30.518416,predicted upper bound 30.528813,actual reward 27.042381
    round 902, predicted reward 25.637554,predicted upper bound 25.647578,actual reward 19.334374
    round 903, predicted reward 34.849846,predicted upper bound 34.857771,actual reward 37.165600
    round 904, predicted reward 20.210724,predicted upper bound 20.220406,actual reward 16.289269
    round 905, predicted reward 33.521785,predicted upper bound 33.529254,actual reward 35.179977
    round 906, predicted reward 30.362469,predicted upper bound 30.372555,actual reward 35.949069
    round 907, predicted reward 31.181892,predicted upper bound 31.191930,actual reward 32.217635
    round 908, predicted reward 30.486405,predicted upper bound 30.496496,actual reward 33.009562
    round 909, predicted reward 31.786348,predicted upper bound 31.794394,actual reward 28.969642
    round 910, predicted reward 25.485059,predicted upper bound 25.496560,actual reward 24.369782
    round 911, predicted reward 27.305720,predicted upper bound 27.315551,actual reward 32.746273
    round 912, predicted reward 23.226867,predicted upper bound 23.236724,actual reward 22.483447
    round 913, predicted reward 30.729196,predicted upper bound 30.738398,actual reward 32.570946
    round 914, predicted reward 25.672494,predicted upper bound 25.682060,actual reward 22.195243
    round 915, predicted reward 29.234529,predicted upper bound 29.243690,actual reward 30.690380
    round 916, predicted reward 34.862647,predicted upper bound 34.871990,actual reward 36.457416
    round 917, predicted reward 26.017965,predicted upper bound 26.025930,actual reward 22.819436
    round 918, predicted reward 37.214953,predicted upper bound 37.225974,actual reward 40.933068
    round 919, predicted reward 31.579631,predicted upper bound 31.588028,actual reward 36.288782
    round 920, predicted reward 29.136594,predicted upper bound 29.147431,actual reward 28.327701
    round 921, predicted reward 27.560340,predicted upper bound 27.569404,actual reward 30.439399
    round 922, predicted reward 27.320390,predicted upper bound 27.330062,actual reward 25.002235
    round 923, predicted reward 29.174340,predicted upper bound 29.184224,actual reward 31.879791
    round 924, predicted reward 30.760740,predicted upper bound 30.770680,actual reward 29.492374
    round 925, predicted reward 28.343745,predicted upper bound 28.353492,actual reward 26.069400
    round 926, predicted reward 30.576085,predicted upper bound 30.586303,actual reward 31.330111
    round 927, predicted reward 22.707349,predicted upper bound 22.716251,actual reward 18.538871
    round 928, predicted reward 29.033502,predicted upper bound 29.043181,actual reward 25.111112
    round 929, predicted reward 22.312785,predicted upper bound 22.322182,actual reward 23.782644
    round 930, predicted reward 27.292130,predicted upper bound 27.299867,actual reward 26.523561
    round 931, predicted reward 28.588491,predicted upper bound 28.598158,actual reward 27.262748
    round 932, predicted reward 28.324270,predicted upper bound 28.333766,actual reward 27.014807
    round 933, predicted reward 28.329341,predicted upper bound 28.339125,actual reward 24.961561
    round 934, predicted reward 23.988441,predicted upper bound 24.000726,actual reward 23.435836
    round 935, predicted reward 28.550687,predicted upper bound 28.559213,actual reward 28.910386
    round 936, predicted reward 24.364734,predicted upper bound 24.376053,actual reward 24.146929
    round 937, predicted reward 26.986933,predicted upper bound 26.996438,actual reward 29.576005
    round 938, predicted reward 22.475645,predicted upper bound 22.485803,actual reward 21.329339
    round 939, predicted reward 22.734291,predicted upper bound 22.744866,actual reward 25.613813
    round 940, predicted reward 25.507594,predicted upper bound 25.516432,actual reward 28.824679
    round 941, predicted reward 21.911848,predicted upper bound 21.919910,actual reward 16.793056
    round 942, predicted reward 31.106604,predicted upper bound 31.115455,actual reward 29.000452
    round 943, predicted reward 29.880215,predicted upper bound 29.889643,actual reward 30.327269
    round 944, predicted reward 26.966195,predicted upper bound 26.974818,actual reward 23.811684
    round 945, predicted reward 24.812993,predicted upper bound 24.823742,actual reward 25.070391
    round 946, predicted reward 22.890476,predicted upper bound 22.900737,actual reward 23.345826
    round 947, predicted reward 23.630063,predicted upper bound 23.639808,actual reward 21.786329
    round 948, predicted reward 18.000927,predicted upper bound 18.011796,actual reward 17.925718
    round 949, predicted reward 25.075051,predicted upper bound 25.084406,actual reward 22.967729
    round 950, predicted reward 32.366342,predicted upper bound 32.376037,actual reward 27.429246
    round 951, predicted reward 23.796774,predicted upper bound 23.808164,actual reward 19.911047
    round 952, predicted reward 22.727352,predicted upper bound 22.737300,actual reward 22.541564
    round 953, predicted reward 33.553001,predicted upper bound 33.560517,actual reward 36.487774
    round 954, predicted reward 34.282469,predicted upper bound 34.291965,actual reward 36.194414
    round 955, predicted reward 25.013200,predicted upper bound 25.023541,actual reward 24.151194
    round 956, predicted reward 31.544004,predicted upper bound 31.552059,actual reward 34.255734
    round 957, predicted reward 31.320985,predicted upper bound 31.329115,actual reward 33.421457
    round 958, predicted reward 24.831890,predicted upper bound 24.841397,actual reward 14.945888
    round 959, predicted reward 22.210638,predicted upper bound 22.220862,actual reward 19.663173
    round 960, predicted reward 29.019493,predicted upper bound 29.027903,actual reward 32.294037
    round 961, predicted reward 32.913037,predicted upper bound 32.921090,actual reward 30.687455
    round 962, predicted reward 26.707050,predicted upper bound 26.716864,actual reward 26.835038
    round 963, predicted reward 25.041408,predicted upper bound 25.050545,actual reward 26.152920
    round 964, predicted reward 28.327536,predicted upper bound 28.336947,actual reward 25.590876
    round 965, predicted reward 26.992367,predicted upper bound 27.002755,actual reward 29.639282
    round 966, predicted reward 29.808254,predicted upper bound 29.815009,actual reward 28.256365
    round 967, predicted reward 30.416997,predicted upper bound 30.425885,actual reward 32.629103
    round 968, predicted reward 29.263141,predicted upper bound 29.271928,actual reward 31.020613
    round 969, predicted reward 31.092536,predicted upper bound 31.100004,actual reward 32.133109
    round 970, predicted reward 25.844967,predicted upper bound 25.853782,actual reward 26.704575
    round 971, predicted reward 25.819152,predicted upper bound 25.828332,actual reward 29.250637
    round 972, predicted reward 30.180160,predicted upper bound 30.189468,actual reward 29.959832
    round 973, predicted reward 29.371642,predicted upper bound 29.380020,actual reward 32.123321
    round 974, predicted reward 26.983843,predicted upper bound 26.991163,actual reward 21.678565
    round 975, predicted reward 31.820875,predicted upper bound 31.829606,actual reward 30.149814
    round 976, predicted reward 25.761042,predicted upper bound 25.771219,actual reward 20.071706
    round 977, predicted reward 33.595235,predicted upper bound 33.604131,actual reward 35.057234
    round 978, predicted reward 29.037935,predicted upper bound 29.048308,actual reward 28.058427
    round 979, predicted reward 22.589109,predicted upper bound 22.598511,actual reward 27.399768
    round 980, predicted reward 23.109104,predicted upper bound 23.118356,actual reward 19.542999
    round 981, predicted reward 26.322172,predicted upper bound 26.331235,actual reward 24.950769
    round 982, predicted reward 28.003614,predicted upper bound 28.013982,actual reward 24.235728
    round 983, predicted reward 32.985694,predicted upper bound 32.994764,actual reward 33.208312
    round 984, predicted reward 30.816937,predicted upper bound 30.825618,actual reward 31.119247
    round 985, predicted reward 21.285567,predicted upper bound 21.294119,actual reward 21.160134
    round 986, predicted reward 25.000497,predicted upper bound 25.009535,actual reward 24.312095
    round 987, predicted reward 28.399174,predicted upper bound 28.406678,actual reward 30.507079
    round 988, predicted reward 29.537816,predicted upper bound 29.545872,actual reward 28.870477
    round 989, predicted reward 25.570045,predicted upper bound 25.578768,actual reward 20.529336
    round 990, predicted reward 22.357220,predicted upper bound 22.367543,actual reward 19.504455
    round 991, predicted reward 26.158675,predicted upper bound 26.169250,actual reward 29.071205
    round 992, predicted reward 26.116971,predicted upper bound 26.125326,actual reward 26.812725
    round 993, predicted reward 34.227371,predicted upper bound 34.235737,actual reward 35.461081
    round 994, predicted reward 21.664158,predicted upper bound 21.672223,actual reward 19.203079
    round 995, predicted reward 30.438566,predicted upper bound 30.445355,actual reward 29.465026
    round 996, predicted reward 31.842443,predicted upper bound 31.849593,actual reward 34.200154
    round 997, predicted reward 30.470036,predicted upper bound 30.479669,actual reward 30.030257
    round 998, predicted reward 27.034569,predicted upper bound 27.043614,actual reward 31.643816
    round 999, predicted reward 34.216969,predicted upper bound 34.225404,actual reward 32.652940
    round 1000, predicted reward 25.186229,predicted upper bound 25.195913,actual reward 26.853681
    round 1001, predicted reward 28.881766,predicted upper bound 28.889166,actual reward 26.444332
    round 1002, predicted reward 34.848284,predicted upper bound 34.855849,actual reward 34.402960
    round 1003, predicted reward 27.570353,predicted upper bound 27.578239,actual reward 25.563840
    round 1004, predicted reward 32.661833,predicted upper bound 32.669316,actual reward 35.035583
    round 1005, predicted reward 33.015759,predicted upper bound 33.022979,actual reward 37.780368
    round 1006, predicted reward 21.121564,predicted upper bound 21.131688,actual reward 20.259578
    round 1007, predicted reward 33.263188,predicted upper bound 33.272230,actual reward 33.261427
    round 1008, predicted reward 36.944546,predicted upper bound 36.952287,actual reward 43.414665
    round 1009, predicted reward 27.218082,predicted upper bound 27.225599,actual reward 28.151473
    round 1010, predicted reward 26.451816,predicted upper bound 26.459810,actual reward 24.414960
    round 1011, predicted reward 27.765261,predicted upper bound 27.773748,actual reward 24.098132
    round 1012, predicted reward 29.327632,predicted upper bound 29.336662,actual reward 30.493431
    round 1013, predicted reward 33.036211,predicted upper bound 33.044299,actual reward 34.227126
    round 1014, predicted reward 25.109896,predicted upper bound 25.118040,actual reward 26.991640
    round 1015, predicted reward 23.638898,predicted upper bound 23.647682,actual reward 25.066309
    round 1016, predicted reward 37.372962,predicted upper bound 37.379374,actual reward 45.710074
    round 1017, predicted reward 35.489903,predicted upper bound 35.498322,actual reward 37.671333
    round 1018, predicted reward 25.289694,predicted upper bound 25.298615,actual reward 29.361853
    round 1019, predicted reward 29.640150,predicted upper bound 29.647967,actual reward 26.282889
    round 1020, predicted reward 24.050774,predicted upper bound 24.060508,actual reward 20.467822
    round 1021, predicted reward 28.784004,predicted upper bound 28.793215,actual reward 29.026090
    round 1022, predicted reward 24.023124,predicted upper bound 24.032350,actual reward 24.818711
    round 1023, predicted reward 24.398369,predicted upper bound 24.406402,actual reward 22.186418
    round 1024, predicted reward 31.433150,predicted upper bound 31.442120,actual reward 32.038714
    round 1025, predicted reward 34.156684,predicted upper bound 34.164791,actual reward 33.814990
    round 1026, predicted reward 25.338096,predicted upper bound 25.346567,actual reward 24.606032
    round 1027, predicted reward 31.979305,predicted upper bound 31.986967,actual reward 34.887648
    round 1028, predicted reward 27.411403,predicted upper bound 27.421946,actual reward 28.521387
    round 1029, predicted reward 30.088854,predicted upper bound 30.097732,actual reward 32.587005
    round 1030, predicted reward 31.446524,predicted upper bound 31.454555,actual reward 30.613176
    round 1031, predicted reward 20.153199,predicted upper bound 20.161566,actual reward 17.041650
    round 1032, predicted reward 19.813663,predicted upper bound 19.823259,actual reward 21.150712
    round 1033, predicted reward 26.727936,predicted upper bound 26.736554,actual reward 26.641290
    round 1034, predicted reward 30.974037,predicted upper bound 30.984465,actual reward 31.550720
    round 1035, predicted reward 29.015579,predicted upper bound 29.023983,actual reward 29.990441
    round 1036, predicted reward 19.014894,predicted upper bound 19.023997,actual reward 13.491184
    round 1037, predicted reward 28.066313,predicted upper bound 28.074348,actual reward 27.651664
    round 1038, predicted reward 27.758094,predicted upper bound 27.765936,actual reward 29.720250
    round 1039, predicted reward 30.247736,predicted upper bound 30.256918,actual reward 32.921113
    round 1040, predicted reward 25.062161,predicted upper bound 25.073029,actual reward 22.651237
    round 1041, predicted reward 27.872072,predicted upper bound 27.881864,actual reward 30.807451
    round 1042, predicted reward 22.825378,predicted upper bound 22.836788,actual reward 19.031730
    round 1043, predicted reward 21.556287,predicted upper bound 21.566019,actual reward 25.480216
    round 1044, predicted reward 30.007119,predicted upper bound 30.014733,actual reward 28.583069
    round 1045, predicted reward 27.054270,predicted upper bound 27.061819,actual reward 23.236354
    round 1046, predicted reward 26.660051,predicted upper bound 26.670215,actual reward 22.564712
    round 1047, predicted reward 25.509671,predicted upper bound 25.519014,actual reward 23.062456
    round 1048, predicted reward 26.835830,predicted upper bound 26.845125,actual reward 26.274926
    round 1049, predicted reward 36.760731,predicted upper bound 36.768580,actual reward 45.913106
    round 1050, predicted reward 28.425241,predicted upper bound 28.432058,actual reward 24.400243
    round 1051, predicted reward 28.795476,predicted upper bound 28.803655,actual reward 32.820535
    round 1052, predicted reward 25.752272,predicted upper bound 25.761362,actual reward 22.957242
    round 1053, predicted reward 26.437895,predicted upper bound 26.446884,actual reward 22.168147
    round 1054, predicted reward 32.021900,predicted upper bound 32.029688,actual reward 29.919992
    round 1055, predicted reward 24.698927,predicted upper bound 24.707781,actual reward 27.582775
    round 1056, predicted reward 26.605999,predicted upper bound 26.615636,actual reward 27.832938
    round 1057, predicted reward 30.680354,predicted upper bound 30.687279,actual reward 34.058149
    round 1058, predicted reward 25.516779,predicted upper bound 25.525390,actual reward 26.418034
    round 1059, predicted reward 28.635557,predicted upper bound 28.642421,actual reward 28.585124
    round 1060, predicted reward 30.732309,predicted upper bound 30.741370,actual reward 28.715284
    round 1061, predicted reward 30.355739,predicted upper bound 30.367039,actual reward 27.396850
    round 1062, predicted reward 27.121886,predicted upper bound 27.131447,actual reward 27.930193
    round 1063, predicted reward 27.182231,predicted upper bound 27.192349,actual reward 30.203713
    round 1064, predicted reward 27.502727,predicted upper bound 27.510878,actual reward 28.185932
    round 1065, predicted reward 35.730208,predicted upper bound 35.738120,actual reward 41.155279
    round 1066, predicted reward 28.512432,predicted upper bound 28.521870,actual reward 26.164187
    round 1067, predicted reward 28.582149,predicted upper bound 28.591336,actual reward 29.420291
    round 1068, predicted reward 27.431265,predicted upper bound 27.440329,actual reward 24.347857
    round 1069, predicted reward 29.681923,predicted upper bound 29.688848,actual reward 29.739797
    round 1070, predicted reward 32.618614,predicted upper bound 32.626477,actual reward 33.213375
    round 1071, predicted reward 25.912331,predicted upper bound 25.919914,actual reward 27.063951
    round 1072, predicted reward 28.168359,predicted upper bound 28.177790,actual reward 23.538810
    round 1073, predicted reward 31.850020,predicted upper bound 31.859293,actual reward 30.874225
    round 1074, predicted reward 30.057116,predicted upper bound 30.064337,actual reward 32.120229
    round 1075, predicted reward 25.244053,predicted upper bound 25.252047,actual reward 25.882728
    round 1076, predicted reward 29.755790,predicted upper bound 29.762463,actual reward 30.144145
    round 1077, predicted reward 30.268613,predicted upper bound 30.276192,actual reward 32.267536
    round 1078, predicted reward 32.016363,predicted upper bound 32.024398,actual reward 36.025785
    round 1079, predicted reward 23.452521,predicted upper bound 23.460323,actual reward 24.402846
    round 1080, predicted reward 29.710607,predicted upper bound 29.719921,actual reward 31.519442
    round 1081, predicted reward 24.823456,predicted upper bound 24.834187,actual reward 21.724728
    round 1082, predicted reward 23.451595,predicted upper bound 23.460302,actual reward 20.742730
    round 1083, predicted reward 28.152674,predicted upper bound 28.159824,actual reward 29.862166
    round 1084, predicted reward 23.095151,predicted upper bound 23.103287,actual reward 27.271216
    round 1085, predicted reward 25.468771,predicted upper bound 25.477641,actual reward 29.822852
    round 1086, predicted reward 22.099428,predicted upper bound 22.106379,actual reward 21.579637
    round 1087, predicted reward 32.397764,predicted upper bound 32.404611,actual reward 30.100223
    round 1088, predicted reward 31.107890,predicted upper bound 31.115026,actual reward 31.628368
    round 1089, predicted reward 24.309523,predicted upper bound 24.318120,actual reward 26.644095
    round 1090, predicted reward 24.285981,predicted upper bound 24.293801,actual reward 19.345540
    round 1091, predicted reward 21.230449,predicted upper bound 21.240921,actual reward 26.182862
    round 1092, predicted reward 26.052487,predicted upper bound 26.061802,actual reward 29.879240
    round 1093, predicted reward 24.142154,predicted upper bound 24.149807,actual reward 21.679050
    round 1094, predicted reward 22.946717,predicted upper bound 22.954447,actual reward 19.756812
    round 1095, predicted reward 24.827567,predicted upper bound 24.837332,actual reward 19.144593
    round 1096, predicted reward 34.089354,predicted upper bound 34.098861,actual reward 40.050292
    round 1097, predicted reward 31.495341,predicted upper bound 31.503173,actual reward 33.988818
    round 1098, predicted reward 28.772020,predicted upper bound 28.780304,actual reward 26.707141
    round 1099, predicted reward 26.848500,predicted upper bound 26.855579,actual reward 25.328072
    round 1100, predicted reward 22.406329,predicted upper bound 22.413099,actual reward 16.024518
    round 1101, predicted reward 31.018768,predicted upper bound 31.027268,actual reward 31.147118
    round 1102, predicted reward 25.510335,predicted upper bound 25.518241,actual reward 27.976161
    round 1103, predicted reward 24.603203,predicted upper bound 24.612382,actual reward 28.095345
    round 1104, predicted reward 29.241981,predicted upper bound 29.251079,actual reward 29.122675
    round 1105, predicted reward 23.929836,predicted upper bound 23.937515,actual reward 21.743737
    round 1106, predicted reward 27.144025,predicted upper bound 27.153035,actual reward 30.599739
    round 1107, predicted reward 27.787954,predicted upper bound 27.796082,actual reward 31.827576
    round 1108, predicted reward 24.672967,predicted upper bound 24.682652,actual reward 19.637642
    round 1109, predicted reward 24.126275,predicted upper bound 24.135794,actual reward 27.940083
    round 1110, predicted reward 31.259944,predicted upper bound 31.268998,actual reward 30.114413
    round 1111, predicted reward 28.169509,predicted upper bound 28.176938,actual reward 26.255659
    round 1112, predicted reward 21.745553,predicted upper bound 21.754681,actual reward 23.276540
    round 1113, predicted reward 27.885784,predicted upper bound 27.891735,actual reward 25.772814
    round 1114, predicted reward 31.561842,predicted upper bound 31.569160,actual reward 33.766068
    round 1115, predicted reward 27.863537,predicted upper bound 27.871964,actual reward 27.511984
    round 1116, predicted reward 24.782354,predicted upper bound 24.790921,actual reward 24.023004
    round 1117, predicted reward 36.511927,predicted upper bound 36.518991,actual reward 38.327290
    round 1118, predicted reward 27.954687,predicted upper bound 27.962143,actual reward 26.462799
    round 1119, predicted reward 31.481743,predicted upper bound 31.489964,actual reward 31.722354
    round 1120, predicted reward 32.098384,predicted upper bound 32.106553,actual reward 33.849197
    round 1121, predicted reward 27.434819,predicted upper bound 27.443496,actual reward 26.371471
    round 1122, predicted reward 20.821211,predicted upper bound 20.830696,actual reward 23.593847
    round 1123, predicted reward 23.081285,predicted upper bound 23.089599,actual reward 23.152537
    round 1124, predicted reward 30.414259,predicted upper bound 30.422567,actual reward 30.062300
    round 1125, predicted reward 24.619841,predicted upper bound 24.627212,actual reward 21.983500
    round 1126, predicted reward 23.062305,predicted upper bound 23.073287,actual reward 26.202295
    round 1127, predicted reward 29.823497,predicted upper bound 29.830861,actual reward 30.942819
    round 1128, predicted reward 28.202292,predicted upper bound 28.210868,actual reward 26.463790
    round 1129, predicted reward 29.571339,predicted upper bound 29.579586,actual reward 29.772938
    round 1130, predicted reward 30.804547,predicted upper bound 30.811475,actual reward 31.961597
    round 1131, predicted reward 26.666465,predicted upper bound 26.674522,actual reward 29.262052
    round 1132, predicted reward 24.728301,predicted upper bound 24.735448,actual reward 22.178366
    round 1133, predicted reward 33.392127,predicted upper bound 33.399224,actual reward 41.805699
    round 1134, predicted reward 26.542149,predicted upper bound 26.551173,actual reward 23.356284
    round 1135, predicted reward 20.527301,predicted upper bound 20.534210,actual reward 21.429998
    round 1136, predicted reward 23.682610,predicted upper bound 23.690009,actual reward 25.285774
    round 1137, predicted reward 32.078320,predicted upper bound 32.084586,actual reward 34.387813
    round 1138, predicted reward 20.181573,predicted upper bound 20.189400,actual reward 16.643054
    round 1139, predicted reward 26.889049,predicted upper bound 26.895294,actual reward 26.285184
    round 1140, predicted reward 24.548215,predicted upper bound 24.556313,actual reward 30.076645
    round 1141, predicted reward 21.929239,predicted upper bound 21.939352,actual reward 23.859871
    round 1142, predicted reward 32.946341,predicted upper bound 32.954476,actual reward 32.255820
    round 1143, predicted reward 27.372042,predicted upper bound 27.380359,actual reward 27.676570
    round 1144, predicted reward 33.036677,predicted upper bound 33.043926,actual reward 34.748440
    round 1145, predicted reward 32.005603,predicted upper bound 32.012192,actual reward 28.161786
    round 1146, predicted reward 26.153203,predicted upper bound 26.163517,actual reward 23.022716
    round 1147, predicted reward 24.789285,predicted upper bound 24.797240,actual reward 21.388414
    round 1148, predicted reward 29.727929,predicted upper bound 29.737268,actual reward 31.579280
    round 1149, predicted reward 27.340529,predicted upper bound 27.349144,actual reward 31.439356
    round 1150, predicted reward 27.857517,predicted upper bound 27.865889,actual reward 27.118446
    round 1151, predicted reward 33.385500,predicted upper bound 33.393459,actual reward 33.785772
    round 1152, predicted reward 24.450656,predicted upper bound 24.458708,actual reward 22.624466
    round 1153, predicted reward 26.673369,predicted upper bound 26.680526,actual reward 26.022005
    round 1154, predicted reward 33.967437,predicted upper bound 33.973852,actual reward 33.876288
    round 1155, predicted reward 23.175177,predicted upper bound 23.182867,actual reward 21.481343
    round 1156, predicted reward 27.313456,predicted upper bound 27.323527,actual reward 27.195304
    round 1157, predicted reward 37.536673,predicted upper bound 37.543419,actual reward 40.155230
    round 1158, predicted reward 23.688126,predicted upper bound 23.697649,actual reward 24.895287
    round 1159, predicted reward 23.930159,predicted upper bound 23.938258,actual reward 20.170827
    round 1160, predicted reward 27.057650,predicted upper bound 27.065585,actual reward 33.763336
    round 1161, predicted reward 31.407692,predicted upper bound 31.414837,actual reward 38.144810
    round 1162, predicted reward 34.509164,predicted upper bound 34.516691,actual reward 35.695704
    round 1163, predicted reward 28.673354,predicted upper bound 28.682364,actual reward 30.256250
    round 1164, predicted reward 24.092708,predicted upper bound 24.099554,actual reward 24.172709
    round 1165, predicted reward 29.340954,predicted upper bound 29.349670,actual reward 29.759134
    round 1166, predicted reward 26.203372,predicted upper bound 26.211821,actual reward 21.566165
    round 1167, predicted reward 24.406649,predicted upper bound 24.414913,actual reward 19.178325
    round 1168, predicted reward 30.393643,predicted upper bound 30.401423,actual reward 33.063067
    round 1169, predicted reward 30.765651,predicted upper bound 30.771755,actual reward 31.061219
    round 1170, predicted reward 30.224640,predicted upper bound 30.231656,actual reward 31.633271
    round 1171, predicted reward 33.907792,predicted upper bound 33.915710,actual reward 35.190282
    round 1172, predicted reward 27.192654,predicted upper bound 27.200301,actual reward 27.572614
    round 1173, predicted reward 33.659236,predicted upper bound 33.665806,actual reward 35.721599
    round 1174, predicted reward 24.800425,predicted upper bound 24.809108,actual reward 23.224682
    round 1175, predicted reward 31.842729,predicted upper bound 31.850805,actual reward 32.013406
    round 1176, predicted reward 29.883781,predicted upper bound 29.891679,actual reward 32.119318
    round 1177, predicted reward 26.158025,predicted upper bound 26.166064,actual reward 27.732645
    round 1178, predicted reward 31.636223,predicted upper bound 31.644679,actual reward 28.831843
    round 1179, predicted reward 22.541979,predicted upper bound 22.550033,actual reward 22.074365
    round 1180, predicted reward 23.706705,predicted upper bound 23.715132,actual reward 24.155402
    round 1181, predicted reward 25.331853,predicted upper bound 25.340456,actual reward 21.678513
    round 1182, predicted reward 31.642284,predicted upper bound 31.649125,actual reward 31.262154
    round 1183, predicted reward 24.959860,predicted upper bound 24.967941,actual reward 28.263622
    round 1184, predicted reward 27.446341,predicted upper bound 27.453666,actual reward 28.496557
    round 1185, predicted reward 35.736098,predicted upper bound 35.742220,actual reward 34.522702
    round 1186, predicted reward 28.726539,predicted upper bound 28.733344,actual reward 25.942714
    round 1187, predicted reward 29.952025,predicted upper bound 29.959035,actual reward 29.621343
    round 1188, predicted reward 28.252147,predicted upper bound 28.259237,actual reward 28.577983
    round 1189, predicted reward 31.120803,predicted upper bound 31.127282,actual reward 33.179832
    round 1190, predicted reward 22.352468,predicted upper bound 22.363073,actual reward 20.358892
    round 1191, predicted reward 28.871611,predicted upper bound 28.878094,actual reward 31.989997
    round 1192, predicted reward 21.038951,predicted upper bound 21.048510,actual reward 17.796118
    round 1193, predicted reward 27.029224,predicted upper bound 27.037131,actual reward 28.384044
    round 1194, predicted reward 28.021052,predicted upper bound 28.028137,actual reward 27.484326
    round 1195, predicted reward 24.022075,predicted upper bound 24.029596,actual reward 20.070403
    round 1196, predicted reward 23.013000,predicted upper bound 23.020228,actual reward 20.556808
    round 1197, predicted reward 32.162603,predicted upper bound 32.169636,actual reward 33.025583
    round 1198, predicted reward 19.775850,predicted upper bound 19.783924,actual reward 20.944849
    round 1199, predicted reward 28.730396,predicted upper bound 28.737290,actual reward 30.136452
    round 1200, predicted reward 29.592203,predicted upper bound 29.601469,actual reward 28.484690
    round 1201, predicted reward 26.486789,predicted upper bound 26.495609,actual reward 27.638935
    round 1202, predicted reward 29.420925,predicted upper bound 29.428064,actual reward 28.351917
    round 1203, predicted reward 29.775554,predicted upper bound 29.782512,actual reward 31.394891
    round 1204, predicted reward 27.501545,predicted upper bound 27.508829,actual reward 30.403654
    round 1205, predicted reward 26.457650,predicted upper bound 26.465160,actual reward 25.310321
    round 1206, predicted reward 33.502228,predicted upper bound 33.510232,actual reward 29.100389
    round 1207, predicted reward 21.398002,predicted upper bound 21.407888,actual reward 17.891261
    round 1208, predicted reward 22.697100,predicted upper bound 22.704641,actual reward 19.234768
    round 1209, predicted reward 23.282534,predicted upper bound 23.290094,actual reward 23.604132
    round 1210, predicted reward 27.696750,predicted upper bound 27.704160,actual reward 25.880377
    round 1211, predicted reward 28.931327,predicted upper bound 28.940794,actual reward 32.126923
    round 1212, predicted reward 25.177306,predicted upper bound 25.185454,actual reward 25.663387
    round 1213, predicted reward 26.751819,predicted upper bound 26.758601,actual reward 27.123719
    round 1214, predicted reward 24.433469,predicted upper bound 24.441116,actual reward 24.695533
    round 1215, predicted reward 23.389448,predicted upper bound 23.396893,actual reward 22.338489
    round 1216, predicted reward 18.519137,predicted upper bound 18.528498,actual reward 18.264187
    round 1217, predicted reward 32.828216,predicted upper bound 32.834525,actual reward 32.617252
    round 1218, predicted reward 34.504769,predicted upper bound 34.511513,actual reward 36.736130
    round 1219, predicted reward 23.537021,predicted upper bound 23.546899,actual reward 21.357628
    round 1220, predicted reward 23.850752,predicted upper bound 23.857640,actual reward 22.837129
    round 1221, predicted reward 27.107003,predicted upper bound 27.114101,actual reward 25.201779
    round 1222, predicted reward 25.986748,predicted upper bound 25.993565,actual reward 27.091506
    round 1223, predicted reward 26.448395,predicted upper bound 26.456672,actual reward 25.821459
    round 1224, predicted reward 24.742709,predicted upper bound 24.750280,actual reward 23.921610
    round 1225, predicted reward 29.708664,predicted upper bound 29.715353,actual reward 24.738428
    round 1226, predicted reward 27.597886,predicted upper bound 27.606745,actual reward 30.568082
    round 1227, predicted reward 26.079097,predicted upper bound 26.086222,actual reward 29.664447
    round 1228, predicted reward 35.767126,predicted upper bound 35.775266,actual reward 36.030788
    round 1229, predicted reward 21.027930,predicted upper bound 21.035726,actual reward 20.293431
    round 1230, predicted reward 21.800435,predicted upper bound 21.809734,actual reward 20.600951
    round 1231, predicted reward 23.921271,predicted upper bound 23.930228,actual reward 20.972149
    round 1232, predicted reward 29.461517,predicted upper bound 29.469006,actual reward 26.000761
    round 1233, predicted reward 30.222786,predicted upper bound 30.230520,actual reward 29.620668
    round 1234, predicted reward 22.788768,predicted upper bound 22.796892,actual reward 24.139155
    round 1235, predicted reward 25.987791,predicted upper bound 25.994742,actual reward 28.723938
    round 1236, predicted reward 28.042098,predicted upper bound 28.049652,actual reward 29.098579
    round 1237, predicted reward 27.188335,predicted upper bound 27.196553,actual reward 22.876282
    round 1238, predicted reward 26.198313,predicted upper bound 26.206896,actual reward 25.222052
    round 1239, predicted reward 23.143033,predicted upper bound 23.152529,actual reward 23.521390
    round 1240, predicted reward 16.671903,predicted upper bound 16.681536,actual reward 15.418046
    round 1241, predicted reward 29.641975,predicted upper bound 29.648342,actual reward 30.485478
    round 1242, predicted reward 24.863830,predicted upper bound 24.870936,actual reward 26.692363
    round 1243, predicted reward 24.851737,predicted upper bound 24.860756,actual reward 23.354130
    round 1244, predicted reward 26.237502,predicted upper bound 26.244275,actual reward 25.490612
    round 1245, predicted reward 28.803463,predicted upper bound 28.810920,actual reward 28.834829
    round 1246, predicted reward 28.868950,predicted upper bound 28.875742,actual reward 25.601671
    round 1247, predicted reward 27.830528,predicted upper bound 27.838509,actual reward 30.470890
    round 1248, predicted reward 30.265790,predicted upper bound 30.273050,actual reward 27.291907
    round 1249, predicted reward 23.633251,predicted upper bound 23.642096,actual reward 18.247801
    round 1250, predicted reward 24.365052,predicted upper bound 24.372325,actual reward 25.100133
    round 1251, predicted reward 27.850152,predicted upper bound 27.857530,actual reward 26.709086
    round 1252, predicted reward 25.814378,predicted upper bound 25.822770,actual reward 23.342311
    round 1253, predicted reward 32.092820,predicted upper bound 32.099812,actual reward 35.455443
    round 1254, predicted reward 28.677990,predicted upper bound 28.685910,actual reward 27.306569
    round 1255, predicted reward 26.598613,predicted upper bound 26.607535,actual reward 24.452870
    round 1256, predicted reward 27.925856,predicted upper bound 27.933161,actual reward 26.748338
    round 1257, predicted reward 33.135087,predicted upper bound 33.142214,actual reward 31.033471
    round 1258, predicted reward 28.774575,predicted upper bound 28.782671,actual reward 28.339583
    round 1259, predicted reward 29.632039,predicted upper bound 29.638882,actual reward 28.352901
    round 1260, predicted reward 27.445836,predicted upper bound 27.452968,actual reward 24.580822
    round 1261, predicted reward 32.327144,predicted upper bound 32.332956,actual reward 32.802689
    round 1262, predicted reward 26.877312,predicted upper bound 26.884394,actual reward 29.723381
    round 1263, predicted reward 23.890884,predicted upper bound 23.898213,actual reward 21.020130
    round 1264, predicted reward 26.745440,predicted upper bound 26.753086,actual reward 28.104068
    round 1265, predicted reward 20.762528,predicted upper bound 20.770013,actual reward 20.205563
    round 1266, predicted reward 23.738793,predicted upper bound 23.745484,actual reward 21.438065
    round 1267, predicted reward 28.498978,predicted upper bound 28.506921,actual reward 28.903007
    round 1268, predicted reward 40.303218,predicted upper bound 40.309627,actual reward 42.787994
    round 1269, predicted reward 27.489454,predicted upper bound 27.495046,actual reward 30.279555
    round 1270, predicted reward 26.199993,predicted upper bound 26.206369,actual reward 21.925785
    round 1271, predicted reward 25.184956,predicted upper bound 25.192518,actual reward 26.315502
    round 1272, predicted reward 24.712095,predicted upper bound 24.720458,actual reward 22.368576
    round 1273, predicted reward 21.505526,predicted upper bound 21.513240,actual reward 19.920674
    round 1274, predicted reward 29.984687,predicted upper bound 29.991537,actual reward 24.356939
    round 1275, predicted reward 27.934444,predicted upper bound 27.941737,actual reward 24.560054
    round 1276, predicted reward 19.946564,predicted upper bound 19.955453,actual reward 15.716410
    round 1277, predicted reward 30.411004,predicted upper bound 30.417245,actual reward 37.308817
    round 1278, predicted reward 30.026037,predicted upper bound 30.032748,actual reward 30.760264
    round 1279, predicted reward 16.023523,predicted upper bound 16.032391,actual reward 14.806897
    round 1280, predicted reward 25.882825,predicted upper bound 25.890392,actual reward 27.555191
    round 1281, predicted reward 21.450600,predicted upper bound 21.459356,actual reward 20.892333
    round 1282, predicted reward 27.429443,predicted upper bound 27.436662,actual reward 28.486520
    round 1283, predicted reward 26.296607,predicted upper bound 26.305014,actual reward 25.925723
    round 1284, predicted reward 30.695787,predicted upper bound 30.704254,actual reward 28.677976
    round 1285, predicted reward 24.640123,predicted upper bound 24.648603,actual reward 19.836621
    round 1286, predicted reward 29.042320,predicted upper bound 29.049536,actual reward 23.852198
    round 1287, predicted reward 26.943581,predicted upper bound 26.950254,actual reward 24.075066
    round 1288, predicted reward 33.281984,predicted upper bound 33.289148,actual reward 31.937274
    round 1289, predicted reward 33.026345,predicted upper bound 33.031737,actual reward 32.781006
    round 1290, predicted reward 23.787496,predicted upper bound 23.796108,actual reward 25.370506
    round 1291, predicted reward 25.337728,predicted upper bound 25.345812,actual reward 23.298693
    round 1292, predicted reward 29.368686,predicted upper bound 29.374430,actual reward 25.013952
    round 1293, predicted reward 32.495003,predicted upper bound 32.500984,actual reward 29.137873
    round 1294, predicted reward 24.477693,predicted upper bound 24.484663,actual reward 28.291366
    round 1295, predicted reward 19.156426,predicted upper bound 19.163588,actual reward 15.488147
    round 1296, predicted reward 34.673000,predicted upper bound 34.680456,actual reward 34.779239
    round 1297, predicted reward 32.255224,predicted upper bound 32.261651,actual reward 35.833010
    round 1298, predicted reward 32.935522,predicted upper bound 32.941789,actual reward 34.457378
    round 1299, predicted reward 28.382984,predicted upper bound 28.389415,actual reward 29.082119
    round 1300, predicted reward 23.728937,predicted upper bound 23.737001,actual reward 21.639623
    round 1301, predicted reward 32.463898,predicted upper bound 32.472791,actual reward 33.430474
    round 1302, predicted reward 37.236515,predicted upper bound 37.242205,actual reward 42.716928
    round 1303, predicted reward 25.418871,predicted upper bound 25.425939,actual reward 28.252370
    round 1304, predicted reward 25.660756,predicted upper bound 25.667858,actual reward 26.856803
    round 1305, predicted reward 23.328785,predicted upper bound 23.337105,actual reward 17.865086
    round 1306, predicted reward 30.750704,predicted upper bound 30.757599,actual reward 29.453537
    round 1307, predicted reward 26.358486,predicted upper bound 26.364549,actual reward 25.121798
    round 1308, predicted reward 29.882974,predicted upper bound 29.889298,actual reward 32.275778
    round 1309, predicted reward 24.854798,predicted upper bound 24.863133,actual reward 33.010122
    round 1310, predicted reward 19.881230,predicted upper bound 19.888589,actual reward 12.425805
    round 1311, predicted reward 25.397745,predicted upper bound 25.404863,actual reward 26.855046
    round 1312, predicted reward 26.459421,predicted upper bound 26.467555,actual reward 23.973820
    round 1313, predicted reward 24.590822,predicted upper bound 24.598106,actual reward 25.114247
    round 1314, predicted reward 27.007916,predicted upper bound 27.014179,actual reward 22.827081
    round 1315, predicted reward 25.652504,predicted upper bound 25.660455,actual reward 24.855073
    round 1316, predicted reward 23.300888,predicted upper bound 23.308508,actual reward 23.610984
    round 1317, predicted reward 25.726284,predicted upper bound 25.732957,actual reward 25.209972
    round 1318, predicted reward 28.754250,predicted upper bound 28.761325,actual reward 24.112306
    round 1319, predicted reward 29.085660,predicted upper bound 29.092117,actual reward 32.725891
    round 1320, predicted reward 31.431103,predicted upper bound 31.437414,actual reward 31.660596
    round 1321, predicted reward 26.446478,predicted upper bound 26.454481,actual reward 32.322167
    round 1322, predicted reward 30.314343,predicted upper bound 30.321434,actual reward 29.357770
    round 1323, predicted reward 34.307727,predicted upper bound 34.313534,actual reward 38.493073
    round 1324, predicted reward 32.007354,predicted upper bound 32.014523,actual reward 38.875420
    round 1325, predicted reward 24.198215,predicted upper bound 24.204728,actual reward 20.074771
    round 1326, predicted reward 26.474164,predicted upper bound 26.481956,actual reward 23.941860
    round 1327, predicted reward 33.959754,predicted upper bound 33.966676,actual reward 36.574315
    round 1328, predicted reward 27.840313,predicted upper bound 27.847833,actual reward 25.579003
    round 1329, predicted reward 31.571502,predicted upper bound 31.577705,actual reward 27.021275
    round 1330, predicted reward 28.756350,predicted upper bound 28.763267,actual reward 29.088038
    round 1331, predicted reward 35.450595,predicted upper bound 35.457331,actual reward 39.564238
    round 1332, predicted reward 19.118400,predicted upper bound 19.126512,actual reward 14.092667
    round 1333, predicted reward 24.113914,predicted upper bound 24.121930,actual reward 23.897260
    round 1334, predicted reward 27.273068,predicted upper bound 27.281172,actual reward 25.002191
    round 1335, predicted reward 26.009298,predicted upper bound 26.017169,actual reward 24.589509
    round 1336, predicted reward 27.539874,predicted upper bound 27.545937,actual reward 27.223491
    round 1337, predicted reward 21.940905,predicted upper bound 21.951095,actual reward 23.285171
    round 1338, predicted reward 23.286766,predicted upper bound 23.295730,actual reward 19.937279
    round 1339, predicted reward 24.383773,predicted upper bound 24.392119,actual reward 25.754892
    round 1340, predicted reward 26.018991,predicted upper bound 26.025447,actual reward 25.121481
    round 1341, predicted reward 25.775074,predicted upper bound 25.781931,actual reward 29.868081
    round 1342, predicted reward 30.907258,predicted upper bound 30.914156,actual reward 27.298560
    round 1343, predicted reward 30.883271,predicted upper bound 30.889367,actual reward 28.647636
    round 1344, predicted reward 18.356102,predicted upper bound 18.363634,actual reward 18.469595
    round 1345, predicted reward 27.609572,predicted upper bound 27.617327,actual reward 25.245069
    round 1346, predicted reward 21.797640,predicted upper bound 21.806121,actual reward 17.287661
    round 1347, predicted reward 26.676081,predicted upper bound 26.682179,actual reward 28.112758
    round 1348, predicted reward 30.423183,predicted upper bound 30.430971,actual reward 23.228766
    round 1349, predicted reward 24.640371,predicted upper bound 24.646043,actual reward 26.272038
    round 1350, predicted reward 34.408143,predicted upper bound 34.415126,actual reward 40.710829
    round 1351, predicted reward 24.882511,predicted upper bound 24.889703,actual reward 21.137777
    round 1352, predicted reward 24.484943,predicted upper bound 24.492338,actual reward 24.264653
    round 1353, predicted reward 25.017080,predicted upper bound 25.023990,actual reward 21.842579
    round 1354, predicted reward 28.081471,predicted upper bound 28.087480,actual reward 26.050884
    round 1355, predicted reward 28.259755,predicted upper bound 28.266923,actual reward 30.861572
    round 1356, predicted reward 36.441090,predicted upper bound 36.446486,actual reward 41.052421
    round 1357, predicted reward 27.743797,predicted upper bound 27.751705,actual reward 26.497405
    round 1358, predicted reward 26.450989,predicted upper bound 26.458571,actual reward 26.606180
    round 1359, predicted reward 24.167632,predicted upper bound 24.175069,actual reward 27.991534
    round 1360, predicted reward 30.472201,predicted upper bound 30.479467,actual reward 29.210796
    round 1361, predicted reward 27.089356,predicted upper bound 27.096945,actual reward 23.449616
    round 1362, predicted reward 26.169362,predicted upper bound 26.175965,actual reward 22.519356
    round 1363, predicted reward 28.863719,predicted upper bound 28.871204,actual reward 25.707545
    round 1364, predicted reward 37.851626,predicted upper bound 37.857498,actual reward 40.165808
    round 1365, predicted reward 25.974774,predicted upper bound 25.981758,actual reward 24.023900
    round 1366, predicted reward 26.091421,predicted upper bound 26.098979,actual reward 23.728129
    round 1367, predicted reward 24.098833,predicted upper bound 24.105337,actual reward 24.780915
    round 1368, predicted reward 31.998051,predicted upper bound 32.004668,actual reward 35.255924
    round 1369, predicted reward 26.907044,predicted upper bound 26.912982,actual reward 27.580112
    round 1370, predicted reward 24.416668,predicted upper bound 24.424024,actual reward 30.367587
    round 1371, predicted reward 27.730275,predicted upper bound 27.737962,actual reward 27.944618
    round 1372, predicted reward 26.589738,predicted upper bound 26.597222,actual reward 23.698709
    round 1373, predicted reward 25.427362,predicted upper bound 25.434961,actual reward 21.337315
    round 1374, predicted reward 28.705949,predicted upper bound 28.714086,actual reward 31.339984
    round 1375, predicted reward 23.555009,predicted upper bound 23.562397,actual reward 26.075031
    round 1376, predicted reward 27.495625,predicted upper bound 27.501105,actual reward 33.085598
    round 1377, predicted reward 26.744357,predicted upper bound 26.752018,actual reward 25.572250
    round 1378, predicted reward 27.009938,predicted upper bound 27.018365,actual reward 26.466370
    round 1379, predicted reward 24.466206,predicted upper bound 24.473095,actual reward 20.435204
    round 1380, predicted reward 23.335284,predicted upper bound 23.342263,actual reward 19.343833
    round 1381, predicted reward 26.179617,predicted upper bound 26.186092,actual reward 27.954429
    round 1382, predicted reward 33.946366,predicted upper bound 33.951703,actual reward 26.510275
    round 1383, predicted reward 23.656888,predicted upper bound 23.666309,actual reward 25.212599
    round 1384, predicted reward 28.502800,predicted upper bound 28.509073,actual reward 29.025683
    round 1385, predicted reward 25.975186,predicted upper bound 25.982037,actual reward 24.719422
    round 1386, predicted reward 25.828692,predicted upper bound 25.835144,actual reward 28.177883
    round 1387, predicted reward 29.410104,predicted upper bound 29.418224,actual reward 27.579475
    round 1388, predicted reward 25.265594,predicted upper bound 25.271801,actual reward 25.563582
    round 1389, predicted reward 23.401160,predicted upper bound 23.408731,actual reward 19.950389
    round 1390, predicted reward 23.861576,predicted upper bound 23.868089,actual reward 22.783415
    round 1391, predicted reward 32.899589,predicted upper bound 32.906437,actual reward 36.604446
    round 1392, predicted reward 23.609508,predicted upper bound 23.615857,actual reward 17.495672
    round 1393, predicted reward 24.474955,predicted upper bound 24.481605,actual reward 19.944561
    round 1394, predicted reward 25.186695,predicted upper bound 25.193893,actual reward 24.253549
    round 1395, predicted reward 24.588996,predicted upper bound 24.594773,actual reward 23.380612
    round 1396, predicted reward 32.979296,predicted upper bound 32.985877,actual reward 37.398036
    round 1397, predicted reward 28.295972,predicted upper bound 28.301596,actual reward 25.779696
    round 1398, predicted reward 30.347622,predicted upper bound 30.354098,actual reward 34.131068
    round 1399, predicted reward 24.343624,predicted upper bound 24.350399,actual reward 26.381054
    round 1400, predicted reward 24.486041,predicted upper bound 24.493635,actual reward 25.586776
    round 1401, predicted reward 24.423513,predicted upper bound 24.430541,actual reward 18.705526
    round 1402, predicted reward 28.368061,predicted upper bound 28.374905,actual reward 27.029106
    round 1403, predicted reward 24.049720,predicted upper bound 24.056538,actual reward 17.050653
    round 1404, predicted reward 30.437486,predicted upper bound 30.443259,actual reward 33.039484
    round 1405, predicted reward 24.189405,predicted upper bound 24.196860,actual reward 25.229495
    round 1406, predicted reward 26.552880,predicted upper bound 26.559838,actual reward 23.392280
    round 1407, predicted reward 17.092521,predicted upper bound 17.100647,actual reward 16.527244
    round 1408, predicted reward 29.735010,predicted upper bound 29.741336,actual reward 26.853729
    round 1409, predicted reward 24.877000,predicted upper bound 24.884464,actual reward 20.666689
    round 1410, predicted reward 25.581625,predicted upper bound 25.589476,actual reward 22.388037
    round 1411, predicted reward 19.841006,predicted upper bound 19.848670,actual reward 18.540553
    round 1412, predicted reward 28.110004,predicted upper bound 28.115698,actual reward 30.870561
    round 1413, predicted reward 33.379664,predicted upper bound 33.386155,actual reward 38.939603
    round 1414, predicted reward 29.869373,predicted upper bound 29.876366,actual reward 25.339540
    round 1415, predicted reward 26.755404,predicted upper bound 26.763136,actual reward 19.276693
    round 1416, predicted reward 22.973555,predicted upper bound 22.981097,actual reward 17.762495
    round 1417, predicted reward 29.780348,predicted upper bound 29.787010,actual reward 31.493964
    round 1418, predicted reward 30.376749,predicted upper bound 30.383654,actual reward 29.384826
    round 1419, predicted reward 24.668380,predicted upper bound 24.674214,actual reward 23.323027
    round 1420, predicted reward 34.493278,predicted upper bound 34.499641,actual reward 37.242465
    round 1421, predicted reward 32.515217,predicted upper bound 32.522649,actual reward 30.518711
    round 1422, predicted reward 28.091697,predicted upper bound 28.097574,actual reward 31.263542
    round 1423, predicted reward 30.008986,predicted upper bound 30.014162,actual reward 28.345657
    round 1424, predicted reward 29.070080,predicted upper bound 29.076229,actual reward 30.426404
    round 1425, predicted reward 28.231120,predicted upper bound 28.238304,actual reward 28.038162
    round 1426, predicted reward 28.788038,predicted upper bound 28.796299,actual reward 28.089514
    round 1427, predicted reward 26.153686,predicted upper bound 26.159042,actual reward 24.198646
    round 1428, predicted reward 30.951259,predicted upper bound 30.958804,actual reward 39.091180
    round 1429, predicted reward 24.911120,predicted upper bound 24.919288,actual reward 23.608351
    round 1430, predicted reward 26.469200,predicted upper bound 26.476920,actual reward 28.580541
    round 1431, predicted reward 28.805757,predicted upper bound 28.811234,actual reward 31.267564
    round 1432, predicted reward 27.915825,predicted upper bound 27.922534,actual reward 27.668926
    round 1433, predicted reward 25.172449,predicted upper bound 25.179082,actual reward 24.705828
    round 1434, predicted reward 27.651968,predicted upper bound 27.658950,actual reward 25.743353
    round 1435, predicted reward 30.249986,predicted upper bound 30.256120,actual reward 32.689312
    round 1436, predicted reward 30.680155,predicted upper bound 30.687526,actual reward 31.161927
    round 1437, predicted reward 24.674447,predicted upper bound 24.680481,actual reward 24.635820
    round 1438, predicted reward 22.466616,predicted upper bound 22.474378,actual reward 21.877732
    round 1439, predicted reward 23.193074,predicted upper bound 23.201432,actual reward 23.205046
    round 1440, predicted reward 18.216712,predicted upper bound 18.225265,actual reward 16.123777
    round 1441, predicted reward 30.414329,predicted upper bound 30.420073,actual reward 32.026188
    round 1442, predicted reward 32.588367,predicted upper bound 32.594816,actual reward 35.579599
    round 1443, predicted reward 32.076214,predicted upper bound 32.081568,actual reward 32.528175
    round 1444, predicted reward 28.812203,predicted upper bound 28.818421,actual reward 22.706790
    round 1445, predicted reward 27.980116,predicted upper bound 27.987629,actual reward 26.088851
    round 1446, predicted reward 27.796689,predicted upper bound 27.803786,actual reward 28.005577
    round 1447, predicted reward 24.129153,predicted upper bound 24.136041,actual reward 24.849823
    round 1448, predicted reward 22.760277,predicted upper bound 22.767887,actual reward 23.127398
    round 1449, predicted reward 33.814807,predicted upper bound 33.819646,actual reward 36.859971
    round 1450, predicted reward 29.356525,predicted upper bound 29.362572,actual reward 27.324439
    round 1451, predicted reward 24.725155,predicted upper bound 24.732145,actual reward 22.825959
    round 1452, predicted reward 21.608100,predicted upper bound 21.617677,actual reward 17.147486
    round 1453, predicted reward 28.839338,predicted upper bound 28.845778,actual reward 33.496776
    round 1454, predicted reward 32.634155,predicted upper bound 32.641400,actual reward 37.836412
    round 1455, predicted reward 26.893353,predicted upper bound 26.900542,actual reward 29.102715
    round 1456, predicted reward 28.634406,predicted upper bound 28.641566,actual reward 31.921372
    round 1457, predicted reward 21.934580,predicted upper bound 21.941245,actual reward 19.993267
    round 1458, predicted reward 25.700483,predicted upper bound 25.707936,actual reward 21.392429
    round 1459, predicted reward 31.020361,predicted upper bound 31.028001,actual reward 30.205887
    round 1460, predicted reward 29.052145,predicted upper bound 29.059072,actual reward 27.710780
    round 1461, predicted reward 22.301559,predicted upper bound 22.307556,actual reward 18.070089
    round 1462, predicted reward 26.854411,predicted upper bound 26.861377,actual reward 28.730703
    round 1463, predicted reward 30.627669,predicted upper bound 30.633722,actual reward 32.414784
    round 1464, predicted reward 17.431423,predicted upper bound 17.440316,actual reward 12.284715
    round 1465, predicted reward 28.070542,predicted upper bound 28.077278,actual reward 30.545715
    round 1466, predicted reward 22.955325,predicted upper bound 22.962655,actual reward 22.773654
    round 1467, predicted reward 28.886319,predicted upper bound 28.893040,actual reward 28.327149
    round 1468, predicted reward 28.316463,predicted upper bound 28.322561,actual reward 29.095552
    round 1469, predicted reward 27.343058,predicted upper bound 27.349383,actual reward 27.610248
    round 1470, predicted reward 22.753688,predicted upper bound 22.760952,actual reward 19.972661
    round 1471, predicted reward 33.354670,predicted upper bound 33.361092,actual reward 34.280660
    round 1472, predicted reward 30.004274,predicted upper bound 30.010428,actual reward 30.065387
    round 1473, predicted reward 23.842806,predicted upper bound 23.850125,actual reward 22.590841
    round 1474, predicted reward 30.196803,predicted upper bound 30.202630,actual reward 34.605467
    round 1475, predicted reward 26.155591,predicted upper bound 26.161939,actual reward 29.554596
    round 1476, predicted reward 30.510168,predicted upper bound 30.516127,actual reward 30.416917
    round 1477, predicted reward 24.784105,predicted upper bound 24.791229,actual reward 24.715052
    round 1478, predicted reward 26.887599,predicted upper bound 26.893602,actual reward 29.244549
    round 1479, predicted reward 24.068587,predicted upper bound 24.075607,actual reward 22.633157
    round 1480, predicted reward 24.074803,predicted upper bound 24.081561,actual reward 20.500122
    round 1481, predicted reward 23.482879,predicted upper bound 23.489040,actual reward 20.748956
    round 1482, predicted reward 27.048213,predicted upper bound 27.053879,actual reward 20.705535
    round 1483, predicted reward 23.961675,predicted upper bound 23.968409,actual reward 21.186820
    round 1484, predicted reward 31.602611,predicted upper bound 31.609689,actual reward 31.851634
    round 1485, predicted reward 30.078528,predicted upper bound 30.084029,actual reward 30.433977
    round 1486, predicted reward 23.210959,predicted upper bound 23.216849,actual reward 20.935627
    round 1487, predicted reward 29.013743,predicted upper bound 29.019198,actual reward 28.340024
    round 1488, predicted reward 22.959711,predicted upper bound 22.967289,actual reward 19.650676
    round 1489, predicted reward 22.279920,predicted upper bound 22.287352,actual reward 21.430090
    round 1490, predicted reward 29.046873,predicted upper bound 29.053521,actual reward 27.864425
    round 1491, predicted reward 23.181576,predicted upper bound 23.186974,actual reward 19.627825
    round 1492, predicted reward 28.660128,predicted upper bound 28.666942,actual reward 27.095329
    round 1493, predicted reward 28.338123,predicted upper bound 28.342975,actual reward 23.968061
    round 1494, predicted reward 24.147575,predicted upper bound 24.155220,actual reward 24.074304
    round 1495, predicted reward 32.279774,predicted upper bound 32.285184,actual reward 36.549700
    round 1496, predicted reward 26.741243,predicted upper bound 26.746761,actual reward 22.345183
    round 1497, predicted reward 25.754481,predicted upper bound 25.760527,actual reward 22.818264
    round 1498, predicted reward 25.844604,predicted upper bound 25.851547,actual reward 27.117820
    round 1499, predicted reward 29.943022,predicted upper bound 29.947746,actual reward 31.505276
    round 1500, predicted reward 29.044884,predicted upper bound 29.051840,actual reward 25.855371
    round 1501, predicted reward 21.996260,predicted upper bound 22.003469,actual reward 19.953404
    round 1502, predicted reward 23.657341,predicted upper bound 23.664629,actual reward 23.055252
    round 1503, predicted reward 23.155983,predicted upper bound 23.162341,actual reward 22.974570
    round 1504, predicted reward 35.507403,predicted upper bound 35.513652,actual reward 35.879713
    round 1505, predicted reward 37.794447,predicted upper bound 37.799855,actual reward 47.375307
    round 1506, predicted reward 26.415423,predicted upper bound 26.421598,actual reward 25.911229
    round 1507, predicted reward 31.203189,predicted upper bound 31.211054,actual reward 36.855749
    round 1508, predicted reward 36.467909,predicted upper bound 36.473071,actual reward 36.701436
    round 1509, predicted reward 35.826946,predicted upper bound 35.832589,actual reward 37.495430
    round 1510, predicted reward 21.449325,predicted upper bound 21.457766,actual reward 19.271116
    round 1511, predicted reward 32.156815,predicted upper bound 32.163558,actual reward 34.957089
    round 1512, predicted reward 26.607845,predicted upper bound 26.615410,actual reward 26.129651
    round 1513, predicted reward 25.353609,predicted upper bound 25.361146,actual reward 25.269248
    round 1514, predicted reward 31.324966,predicted upper bound 31.332601,actual reward 29.659286
    round 1515, predicted reward 22.572735,predicted upper bound 22.580656,actual reward 19.492208
    round 1516, predicted reward 28.529506,predicted upper bound 28.537769,actual reward 29.954764
    round 1517, predicted reward 28.418914,predicted upper bound 28.425049,actual reward 26.685014
    round 1518, predicted reward 21.148340,predicted upper bound 21.154627,actual reward 24.154795
    round 1519, predicted reward 20.236508,predicted upper bound 20.243520,actual reward 20.047127
    round 1520, predicted reward 37.358144,predicted upper bound 37.363256,actual reward 40.095894
    round 1521, predicted reward 28.202080,predicted upper bound 28.208311,actual reward 26.292369
    round 1522, predicted reward 26.836277,predicted upper bound 26.843355,actual reward 27.978175
    round 1523, predicted reward 29.531910,predicted upper bound 29.539014,actual reward 28.520552
    round 1524, predicted reward 26.236420,predicted upper bound 26.244317,actual reward 29.431056
    round 1525, predicted reward 22.540066,predicted upper bound 22.547151,actual reward 18.822201
    round 1526, predicted reward 24.636955,predicted upper bound 24.644541,actual reward 21.509115
    round 1527, predicted reward 24.747515,predicted upper bound 24.754978,actual reward 22.947505
    round 1528, predicted reward 31.757515,predicted upper bound 31.763931,actual reward 31.964514
    round 1529, predicted reward 31.432397,predicted upper bound 31.437625,actual reward 28.589266
    round 1530, predicted reward 27.144614,predicted upper bound 27.151649,actual reward 26.175025
    round 1531, predicted reward 36.568532,predicted upper bound 36.574967,actual reward 44.643200
    round 1532, predicted reward 24.493732,predicted upper bound 24.500511,actual reward 25.371734
    round 1533, predicted reward 29.315485,predicted upper bound 29.322155,actual reward 29.850153
    round 1534, predicted reward 28.832186,predicted upper bound 28.838676,actual reward 23.550509
    round 1535, predicted reward 30.827824,predicted upper bound 30.833979,actual reward 31.222860
    round 1536, predicted reward 22.986020,predicted upper bound 22.993246,actual reward 22.236342
    round 1537, predicted reward 35.206087,predicted upper bound 35.212570,actual reward 42.548779
    round 1538, predicted reward 24.591818,predicted upper bound 24.599620,actual reward 23.470691
    round 1539, predicted reward 28.113764,predicted upper bound 28.120285,actual reward 24.358748
    round 1540, predicted reward 35.267752,predicted upper bound 35.273126,actual reward 39.563120
    round 1541, predicted reward 25.599911,predicted upper bound 25.608212,actual reward 20.343591
    round 1542, predicted reward 28.005058,predicted upper bound 28.012033,actual reward 27.075617
    round 1543, predicted reward 28.659487,predicted upper bound 28.666527,actual reward 26.038195
    round 1544, predicted reward 30.253311,predicted upper bound 30.259667,actual reward 34.575398
    round 1545, predicted reward 25.480253,predicted upper bound 25.487156,actual reward 25.419217
    round 1546, predicted reward 28.421434,predicted upper bound 28.427383,actual reward 29.408950
    round 1547, predicted reward 32.276899,predicted upper bound 32.283314,actual reward 29.240187
    round 1548, predicted reward 19.663317,predicted upper bound 19.668823,actual reward 13.506849
    round 1549, predicted reward 27.780665,predicted upper bound 27.786680,actual reward 32.213828
    round 1550, predicted reward 27.560934,predicted upper bound 27.567802,actual reward 22.370443
    round 1551, predicted reward 27.795992,predicted upper bound 27.802916,actual reward 32.784048
    round 1552, predicted reward 22.648596,predicted upper bound 22.656798,actual reward 18.432335
    round 1553, predicted reward 20.955095,predicted upper bound 20.961044,actual reward 16.551795
    round 1554, predicted reward 29.537429,predicted upper bound 29.543010,actual reward 33.071744
    round 1555, predicted reward 26.520545,predicted upper bound 26.527001,actual reward 25.066876
    round 1556, predicted reward 24.399153,predicted upper bound 24.405658,actual reward 26.460793
    round 1557, predicted reward 30.497275,predicted upper bound 30.503675,actual reward 28.366773
    round 1558, predicted reward 24.301713,predicted upper bound 24.308789,actual reward 23.457073
    round 1559, predicted reward 24.474561,predicted upper bound 24.480224,actual reward 22.863214
    round 1560, predicted reward 25.228873,predicted upper bound 25.234965,actual reward 23.608360
    round 1561, predicted reward 23.846569,predicted upper bound 23.854347,actual reward 17.047728
    round 1562, predicted reward 28.989759,predicted upper bound 28.994978,actual reward 33.762617
    round 1563, predicted reward 35.363705,predicted upper bound 35.369419,actual reward 32.515461
    round 1564, predicted reward 21.748913,predicted upper bound 21.756877,actual reward 21.708969
    round 1565, predicted reward 27.773287,predicted upper bound 27.779271,actual reward 25.488040
    round 1566, predicted reward 34.135939,predicted upper bound 34.142706,actual reward 39.229962
    round 1567, predicted reward 24.729311,predicted upper bound 24.735573,actual reward 22.750445
    round 1568, predicted reward 27.629665,predicted upper bound 27.635711,actual reward 22.736341
    round 1569, predicted reward 28.942889,predicted upper bound 28.948547,actual reward 32.139841
    round 1570, predicted reward 22.135403,predicted upper bound 22.144027,actual reward 21.598473
    round 1571, predicted reward 29.987273,predicted upper bound 29.992606,actual reward 27.745497
    round 1572, predicted reward 31.050585,predicted upper bound 31.056574,actual reward 33.935761
    round 1573, predicted reward 28.035729,predicted upper bound 28.041395,actual reward 28.948308
    round 1574, predicted reward 27.196028,predicted upper bound 27.202220,actual reward 22.513079
    round 1575, predicted reward 30.529918,predicted upper bound 30.536751,actual reward 29.767754
    round 1576, predicted reward 29.786243,predicted upper bound 29.793647,actual reward 28.552597
    round 1577, predicted reward 25.946119,predicted upper bound 25.952352,actual reward 26.088552
    round 1578, predicted reward 26.257568,predicted upper bound 26.264120,actual reward 21.474290
    round 1579, predicted reward 30.658609,predicted upper bound 30.663661,actual reward 29.112503
    round 1580, predicted reward 22.795455,predicted upper bound 22.801874,actual reward 22.377053
    round 1581, predicted reward 23.013390,predicted upper bound 23.020668,actual reward 16.436692
    round 1582, predicted reward 30.218277,predicted upper bound 30.224980,actual reward 30.515080
    round 1583, predicted reward 24.781497,predicted upper bound 24.788134,actual reward 25.005261
    round 1584, predicted reward 33.910396,predicted upper bound 33.917231,actual reward 34.395947
    round 1585, predicted reward 27.145263,predicted upper bound 27.151968,actual reward 28.974411
    round 1586, predicted reward 24.405443,predicted upper bound 24.412534,actual reward 23.164468
    round 1587, predicted reward 23.348794,predicted upper bound 23.356527,actual reward 21.580811
    round 1588, predicted reward 31.333787,predicted upper bound 31.339516,actual reward 28.676350
    round 1589, predicted reward 24.558528,predicted upper bound 24.565343,actual reward 22.837678
    round 1590, predicted reward 25.947871,predicted upper bound 25.954307,actual reward 23.834143
    round 1591, predicted reward 25.780172,predicted upper bound 25.787194,actual reward 22.132077
    round 1592, predicted reward 28.168913,predicted upper bound 28.175321,actual reward 24.323680
    round 1593, predicted reward 27.353371,predicted upper bound 27.359627,actual reward 28.196486
    round 1594, predicted reward 30.588506,predicted upper bound 30.594793,actual reward 35.270784
    round 1595, predicted reward 34.804783,predicted upper bound 34.810661,actual reward 33.020025
    round 1596, predicted reward 23.107544,predicted upper bound 23.114990,actual reward 20.277862
    round 1597, predicted reward 23.336538,predicted upper bound 23.343189,actual reward 26.889266
    round 1598, predicted reward 29.110846,predicted upper bound 29.116361,actual reward 28.067417
    round 1599, predicted reward 17.167349,predicted upper bound 17.175602,actual reward 16.086935
    round 1600, predicted reward 33.312012,predicted upper bound 33.317459,actual reward 31.833784
    round 1601, predicted reward 32.558454,predicted upper bound 32.565621,actual reward 29.499012
    round 1602, predicted reward 28.837829,predicted upper bound 28.843727,actual reward 31.203812
    round 1603, predicted reward 27.496335,predicted upper bound 27.503199,actual reward 26.788869
    round 1604, predicted reward 30.450759,predicted upper bound 30.458419,actual reward 34.925962
    round 1605, predicted reward 23.275850,predicted upper bound 23.283062,actual reward 20.449564
    round 1606, predicted reward 31.817892,predicted upper bound 31.823707,actual reward 34.956623
    round 1607, predicted reward 22.454349,predicted upper bound 22.461418,actual reward 21.721957
    round 1608, predicted reward 24.688636,predicted upper bound 24.694602,actual reward 27.849367
    round 1609, predicted reward 25.635062,predicted upper bound 25.642175,actual reward 26.121616
    round 1610, predicted reward 28.835234,predicted upper bound 28.841289,actual reward 28.319314
    round 1611, predicted reward 29.218568,predicted upper bound 29.224121,actual reward 27.313885
    round 1612, predicted reward 30.267385,predicted upper bound 30.274486,actual reward 27.015882
    round 1613, predicted reward 29.673568,predicted upper bound 29.679142,actual reward 35.357550
    round 1614, predicted reward 30.531573,predicted upper bound 30.537104,actual reward 30.881705
    round 1615, predicted reward 24.462536,predicted upper bound 24.469246,actual reward 23.560290
    round 1616, predicted reward 25.126677,predicted upper bound 25.132426,actual reward 22.758818
    round 1617, predicted reward 28.202420,predicted upper bound 28.207840,actual reward 32.474977
    round 1618, predicted reward 26.124160,predicted upper bound 26.129455,actual reward 21.160881
    round 1619, predicted reward 22.200545,predicted upper bound 22.208755,actual reward 18.975220
    round 1620, predicted reward 27.299343,predicted upper bound 27.307053,actual reward 23.565488
    round 1621, predicted reward 29.330768,predicted upper bound 29.338762,actual reward 26.930291
    round 1622, predicted reward 20.804965,predicted upper bound 20.812333,actual reward 20.001350
    round 1623, predicted reward 33.274205,predicted upper bound 33.280821,actual reward 37.730266
    round 1624, predicted reward 28.522977,predicted upper bound 28.530435,actual reward 25.675728
    round 1625, predicted reward 24.243651,predicted upper bound 24.250635,actual reward 19.705192
    round 1626, predicted reward 19.268511,predicted upper bound 19.276445,actual reward 17.009337
    round 1627, predicted reward 21.268342,predicted upper bound 21.275480,actual reward 22.333582
    round 1628, predicted reward 30.019580,predicted upper bound 30.025766,actual reward 31.020486
    round 1629, predicted reward 24.768397,predicted upper bound 24.776499,actual reward 26.537827
    round 1630, predicted reward 25.975148,predicted upper bound 25.982675,actual reward 25.857780
    round 1631, predicted reward 25.395997,predicted upper bound 25.404597,actual reward 23.782274
    round 1632, predicted reward 21.490461,predicted upper bound 21.498161,actual reward 21.594375
    round 1633, predicted reward 23.405962,predicted upper bound 23.412400,actual reward 24.818518
    round 1634, predicted reward 21.846968,predicted upper bound 21.853358,actual reward 18.337528
    round 1635, predicted reward 24.764429,predicted upper bound 24.771186,actual reward 23.823825
    round 1636, predicted reward 22.897617,predicted upper bound 22.902885,actual reward 23.149018
    round 1637, predicted reward 28.847835,predicted upper bound 28.855257,actual reward 31.522367
    round 1638, predicted reward 30.544413,predicted upper bound 30.550631,actual reward 31.953249
    round 1639, predicted reward 25.094654,predicted upper bound 25.101444,actual reward 23.208591
    round 1640, predicted reward 20.056987,predicted upper bound 20.063234,actual reward 16.783669
    round 1641, predicted reward 28.382138,predicted upper bound 28.388895,actual reward 26.099833
    round 1642, predicted reward 22.673832,predicted upper bound 22.681646,actual reward 23.910519
    round 1643, predicted reward 28.322698,predicted upper bound 28.328995,actual reward 29.395805
    round 1644, predicted reward 25.625800,predicted upper bound 25.632452,actual reward 28.251698
    round 1645, predicted reward 28.246634,predicted upper bound 28.253298,actual reward 28.519346
    round 1646, predicted reward 23.287352,predicted upper bound 23.293901,actual reward 18.502728
    round 1647, predicted reward 28.692300,predicted upper bound 28.698730,actual reward 27.863075
    round 1648, predicted reward 24.109068,predicted upper bound 24.114722,actual reward 20.461912
    round 1649, predicted reward 29.067040,predicted upper bound 29.074121,actual reward 29.318704
    round 1650, predicted reward 25.634593,predicted upper bound 25.641858,actual reward 25.079598
    round 1651, predicted reward 32.152091,predicted upper bound 32.158471,actual reward 31.118403
    round 1652, predicted reward 24.370437,predicted upper bound 24.376981,actual reward 25.067602
    round 1653, predicted reward 30.690991,predicted upper bound 30.696910,actual reward 29.227906
    round 1654, predicted reward 26.698443,predicted upper bound 26.705622,actual reward 27.120538
    round 1655, predicted reward 22.833620,predicted upper bound 22.841341,actual reward 24.459103
    round 1656, predicted reward 24.639406,predicted upper bound 24.647266,actual reward 23.824216
    round 1657, predicted reward 26.019663,predicted upper bound 26.026331,actual reward 24.861720
    round 1658, predicted reward 23.366523,predicted upper bound 23.374710,actual reward 17.289297
    round 1659, predicted reward 31.954595,predicted upper bound 31.962078,actual reward 33.175393
    round 1660, predicted reward 34.047319,predicted upper bound 34.053347,actual reward 36.652558
    round 1661, predicted reward 26.157387,predicted upper bound 26.163808,actual reward 32.674773
    round 1662, predicted reward 33.249582,predicted upper bound 33.256345,actual reward 36.160137
    round 1663, predicted reward 26.705633,predicted upper bound 26.711934,actual reward 28.500450
    round 1664, predicted reward 39.768086,predicted upper bound 39.774749,actual reward 41.854409
    round 1665, predicted reward 26.251930,predicted upper bound 26.257907,actual reward 26.007447
    round 1666, predicted reward 23.253364,predicted upper bound 23.260529,actual reward 27.184070
    round 1667, predicted reward 21.192483,predicted upper bound 21.198601,actual reward 21.038350
    round 1668, predicted reward 33.814195,predicted upper bound 33.820753,actual reward 31.274187
    round 1669, predicted reward 34.954134,predicted upper bound 34.958639,actual reward 36.108553
    round 1670, predicted reward 28.589514,predicted upper bound 28.596037,actual reward 28.921584
    round 1671, predicted reward 31.924715,predicted upper bound 31.930056,actual reward 35.351475
    round 1672, predicted reward 28.523474,predicted upper bound 28.530091,actual reward 28.356801
    round 1673, predicted reward 22.799223,predicted upper bound 22.804315,actual reward 14.353874
    round 1674, predicted reward 22.692279,predicted upper bound 22.698578,actual reward 21.021609
    round 1675, predicted reward 31.668677,predicted upper bound 31.674953,actual reward 34.558286
    round 1676, predicted reward 24.931861,predicted upper bound 24.938505,actual reward 22.621798
    round 1677, predicted reward 27.710963,predicted upper bound 27.717597,actual reward 25.449821
    round 1678, predicted reward 31.321084,predicted upper bound 31.327201,actual reward 33.167178
    round 1679, predicted reward 30.527136,predicted upper bound 30.533631,actual reward 32.147161
    round 1680, predicted reward 21.806834,predicted upper bound 21.813304,actual reward 19.640140
    round 1681, predicted reward 29.981175,predicted upper bound 29.986360,actual reward 32.371832
    round 1682, predicted reward 25.763911,predicted upper bound 25.770598,actual reward 26.392054
    round 1683, predicted reward 25.248193,predicted upper bound 25.253639,actual reward 21.347786
    round 1684, predicted reward 30.411005,predicted upper bound 30.416155,actual reward 29.167566
    round 1685, predicted reward 20.541960,predicted upper bound 20.549321,actual reward 18.188682
    round 1686, predicted reward 28.812984,predicted upper bound 28.818379,actual reward 29.215460
    round 1687, predicted reward 31.553890,predicted upper bound 31.560346,actual reward 32.559151
    round 1688, predicted reward 24.169843,predicted upper bound 24.176810,actual reward 20.566403
    round 1689, predicted reward 20.577370,predicted upper bound 20.584554,actual reward 15.060943
    round 1690, predicted reward 30.813378,predicted upper bound 30.819558,actual reward 33.293819
    round 1691, predicted reward 27.481228,predicted upper bound 27.487713,actual reward 26.296436
    round 1692, predicted reward 17.732101,predicted upper bound 17.737926,actual reward 14.857797
    round 1693, predicted reward 21.195273,predicted upper bound 21.202832,actual reward 22.052190
    round 1694, predicted reward 26.709979,predicted upper bound 26.716118,actual reward 24.410480
    round 1695, predicted reward 29.752902,predicted upper bound 29.759525,actual reward 28.115260
    round 1696, predicted reward 34.299936,predicted upper bound 34.304896,actual reward 39.168357
    round 1697, predicted reward 26.491029,predicted upper bound 26.495814,actual reward 26.846165
    round 1698, predicted reward 21.506032,predicted upper bound 21.512925,actual reward 14.895809
    round 1699, predicted reward 24.794321,predicted upper bound 24.801176,actual reward 18.363251
    round 1700, predicted reward 25.519636,predicted upper bound 25.527258,actual reward 22.839802
    round 1701, predicted reward 20.828173,predicted upper bound 20.834864,actual reward 18.167309
    round 1702, predicted reward 27.650114,predicted upper bound 27.655891,actual reward 25.214085
    round 1703, predicted reward 24.168363,predicted upper bound 24.173968,actual reward 27.866684
    round 1704, predicted reward 22.716193,predicted upper bound 22.721897,actual reward 19.613663
    round 1705, predicted reward 24.878822,predicted upper bound 24.884565,actual reward 22.100796
    round 1706, predicted reward 28.562220,predicted upper bound 28.568812,actual reward 28.255748
    round 1707, predicted reward 32.804983,predicted upper bound 32.810192,actual reward 32.801477
    round 1708, predicted reward 38.039200,predicted upper bound 38.045459,actual reward 38.591137
    round 1709, predicted reward 31.521461,predicted upper bound 31.527962,actual reward 32.765802
    round 1710, predicted reward 20.231593,predicted upper bound 20.237827,actual reward 17.782449
    round 1711, predicted reward 26.135833,predicted upper bound 26.141495,actual reward 28.381198
    round 1712, predicted reward 30.020131,predicted upper bound 30.026054,actual reward 32.061507
    round 1713, predicted reward 28.682066,predicted upper bound 28.688741,actual reward 27.466266
    round 1714, predicted reward 27.568008,predicted upper bound 27.574648,actual reward 26.710012
    round 1715, predicted reward 26.304051,predicted upper bound 26.310232,actual reward 25.825225
    round 1716, predicted reward 23.821334,predicted upper bound 23.828328,actual reward 22.681176
    round 1717, predicted reward 30.636391,predicted upper bound 30.642757,actual reward 32.629864
    round 1718, predicted reward 20.690591,predicted upper bound 20.696864,actual reward 16.129806
    round 1719, predicted reward 29.838179,predicted upper bound 29.844113,actual reward 32.345067
    round 1720, predicted reward 28.492334,predicted upper bound 28.498552,actual reward 25.485926
    round 1721, predicted reward 29.838783,predicted upper bound 29.844432,actual reward 31.002716
    round 1722, predicted reward 26.094670,predicted upper bound 26.100979,actual reward 25.536870
    round 1723, predicted reward 31.870220,predicted upper bound 31.876456,actual reward 35.698641
    round 1724, predicted reward 24.246616,predicted upper bound 24.251874,actual reward 22.374535
    round 1725, predicted reward 28.909291,predicted upper bound 28.915707,actual reward 26.474532
    round 1726, predicted reward 27.106296,predicted upper bound 27.112743,actual reward 23.039830
    round 1727, predicted reward 30.319166,predicted upper bound 30.324897,actual reward 29.544600
    round 1728, predicted reward 25.270964,predicted upper bound 25.277754,actual reward 19.823285
    round 1729, predicted reward 22.802064,predicted upper bound 22.807937,actual reward 21.973614
    round 1730, predicted reward 27.273243,predicted upper bound 27.279638,actual reward 23.572925
    round 1731, predicted reward 23.662688,predicted upper bound 23.668839,actual reward 20.292368
    round 1732, predicted reward 39.909641,predicted upper bound 39.914461,actual reward 41.662831
    round 1733, predicted reward 26.380496,predicted upper bound 26.386195,actual reward 26.536286
    round 1734, predicted reward 25.482018,predicted upper bound 25.487657,actual reward 27.698612
    round 1735, predicted reward 25.106164,predicted upper bound 25.111816,actual reward 27.288662
    round 1736, predicted reward 22.729106,predicted upper bound 22.734531,actual reward 19.121002
    round 1737, predicted reward 32.354037,predicted upper bound 32.360186,actual reward 30.756867
    round 1738, predicted reward 27.461023,predicted upper bound 27.467970,actual reward 28.250295
    round 1739, predicted reward 27.659938,predicted upper bound 27.665313,actual reward 29.955945
    round 1740, predicted reward 27.316812,predicted upper bound 27.322858,actual reward 27.554865
    round 1741, predicted reward 30.998766,predicted upper bound 31.004746,actual reward 26.137145
    round 1742, predicted reward 30.716285,predicted upper bound 30.721102,actual reward 27.425463
    round 1743, predicted reward 25.972672,predicted upper bound 25.977619,actual reward 23.503398
    round 1744, predicted reward 27.331352,predicted upper bound 27.336814,actual reward 25.965486
    round 1745, predicted reward 23.623106,predicted upper bound 23.630068,actual reward 21.953902
    round 1746, predicted reward 28.089219,predicted upper bound 28.094578,actual reward 28.880524
    round 1747, predicted reward 22.527326,predicted upper bound 22.533720,actual reward 17.609628
    round 1748, predicted reward 29.899947,predicted upper bound 29.905667,actual reward 36.326332
    round 1749, predicted reward 29.898326,predicted upper bound 29.903571,actual reward 33.182054
    round 1750, predicted reward 25.755672,predicted upper bound 25.761956,actual reward 19.963518
    round 1751, predicted reward 28.069582,predicted upper bound 28.074830,actual reward 26.518521
    round 1752, predicted reward 33.428826,predicted upper bound 33.433700,actual reward 33.461944
    round 1753, predicted reward 30.322243,predicted upper bound 30.327556,actual reward 31.424933
    round 1754, predicted reward 26.068994,predicted upper bound 26.075775,actual reward 28.440516
    round 1755, predicted reward 29.339676,predicted upper bound 29.345232,actual reward 31.334906
    round 1756, predicted reward 31.593687,predicted upper bound 31.599634,actual reward 40.161928
    round 1757, predicted reward 31.252640,predicted upper bound 31.258462,actual reward 33.221951
    round 1758, predicted reward 34.506475,predicted upper bound 34.511468,actual reward 39.668439
    round 1759, predicted reward 21.502542,predicted upper bound 21.508492,actual reward 20.136601
    round 1760, predicted reward 27.985321,predicted upper bound 27.991915,actual reward 27.448100
    round 1761, predicted reward 21.330092,predicted upper bound 21.337282,actual reward 20.283196
    round 1762, predicted reward 23.127459,predicted upper bound 23.133188,actual reward 20.450177
    round 1763, predicted reward 23.293110,predicted upper bound 23.299702,actual reward 20.269584
    round 1764, predicted reward 27.625236,predicted upper bound 27.630826,actual reward 28.867759
    round 1765, predicted reward 31.129500,predicted upper bound 31.135517,actual reward 24.859678
    round 1766, predicted reward 29.478306,predicted upper bound 29.484717,actual reward 25.443222
    round 1767, predicted reward 30.016480,predicted upper bound 30.022305,actual reward 30.670680
    round 1768, predicted reward 25.345170,predicted upper bound 25.350057,actual reward 19.369123
    round 1769, predicted reward 28.906141,predicted upper bound 28.912124,actual reward 26.700690
    round 1770, predicted reward 29.900600,predicted upper bound 29.905446,actual reward 29.770549
    round 1771, predicted reward 26.103110,predicted upper bound 26.108293,actual reward 29.762136
    round 1772, predicted reward 24.519786,predicted upper bound 24.526126,actual reward 19.349060
    round 1773, predicted reward 25.237375,predicted upper bound 25.243369,actual reward 23.534662
    round 1774, predicted reward 29.600981,predicted upper bound 29.606890,actual reward 25.872827
    round 1775, predicted reward 22.929979,predicted upper bound 22.935944,actual reward 21.980727
    round 1776, predicted reward 28.920085,predicted upper bound 28.925131,actual reward 21.651651
    round 1777, predicted reward 28.287308,predicted upper bound 28.292925,actual reward 28.850688
    round 1778, predicted reward 25.728488,predicted upper bound 25.734283,actual reward 22.656801
    round 1779, predicted reward 26.344021,predicted upper bound 26.349862,actual reward 28.091390
    round 1780, predicted reward 28.884812,predicted upper bound 28.890565,actual reward 30.236382
    round 1781, predicted reward 28.404883,predicted upper bound 28.411328,actual reward 27.860066
    round 1782, predicted reward 26.266176,predicted upper bound 26.272739,actual reward 27.112118
    round 1783, predicted reward 23.054456,predicted upper bound 23.061948,actual reward 23.688108
    round 1784, predicted reward 31.668051,predicted upper bound 31.672557,actual reward 33.866673
    round 1785, predicted reward 19.620306,predicted upper bound 19.626564,actual reward 23.899296
    round 1786, predicted reward 32.688769,predicted upper bound 32.694449,actual reward 31.849084
    round 1787, predicted reward 28.482968,predicted upper bound 28.488337,actual reward 29.231576
    round 1788, predicted reward 25.661467,predicted upper bound 25.667087,actual reward 23.287874
    round 1789, predicted reward 24.142532,predicted upper bound 24.148277,actual reward 23.703654
    round 1790, predicted reward 29.046068,predicted upper bound 29.051215,actual reward 26.876546
    round 1791, predicted reward 29.507012,predicted upper bound 29.512571,actual reward 32.398635
    round 1792, predicted reward 29.435892,predicted upper bound 29.440753,actual reward 30.870120
    round 1793, predicted reward 25.743646,predicted upper bound 25.749259,actual reward 24.708199
    round 1794, predicted reward 26.617089,predicted upper bound 26.623076,actual reward 22.051348
    round 1795, predicted reward 27.509880,predicted upper bound 27.516968,actual reward 21.954888
    round 1796, predicted reward 32.903986,predicted upper bound 32.909297,actual reward 36.221464
    round 1797, predicted reward 32.360757,predicted upper bound 32.366550,actual reward 34.827814
    round 1798, predicted reward 23.698248,predicted upper bound 23.704958,actual reward 24.065335
    round 1799, predicted reward 21.740930,predicted upper bound 21.747753,actual reward 23.613614
    round 1800, predicted reward 28.121166,predicted upper bound 28.126895,actual reward 28.382732
    round 1801, predicted reward 30.045854,predicted upper bound 30.050400,actual reward 33.662366
    round 1802, predicted reward 28.416385,predicted upper bound 28.421978,actual reward 25.081252
    round 1803, predicted reward 23.739428,predicted upper bound 23.745199,actual reward 23.056740
    round 1804, predicted reward 32.385946,predicted upper bound 32.392558,actual reward 27.605476
    round 1805, predicted reward 26.776932,predicted upper bound 26.783423,actual reward 28.888464
    round 1806, predicted reward 24.024175,predicted upper bound 24.029781,actual reward 23.324532
    round 1807, predicted reward 23.462314,predicted upper bound 23.467762,actual reward 17.337427
    round 1808, predicted reward 29.654477,predicted upper bound 29.661069,actual reward 35.122262
    round 1809, predicted reward 21.455231,predicted upper bound 21.461690,actual reward 20.522201
    round 1810, predicted reward 22.259544,predicted upper bound 22.265226,actual reward 21.758385
    round 1811, predicted reward 27.597689,predicted upper bound 27.604144,actual reward 29.768623
    round 1812, predicted reward 31.835824,predicted upper bound 31.841142,actual reward 30.149501
    round 1813, predicted reward 30.087020,predicted upper bound 30.092767,actual reward 25.564450
    round 1814, predicted reward 25.035908,predicted upper bound 25.043887,actual reward 25.512466
    round 1815, predicted reward 26.020642,predicted upper bound 26.027138,actual reward 19.420479
    round 1816, predicted reward 25.740608,predicted upper bound 25.748333,actual reward 21.527873
    round 1817, predicted reward 30.670749,predicted upper bound 30.675989,actual reward 33.600442
    round 1818, predicted reward 32.074744,predicted upper bound 32.080277,actual reward 37.910273
    round 1819, predicted reward 25.627384,predicted upper bound 25.634209,actual reward 20.833526
    round 1820, predicted reward 28.087848,predicted upper bound 28.092673,actual reward 27.809748
    round 1821, predicted reward 30.893571,predicted upper bound 30.900653,actual reward 28.306014
    round 1822, predicted reward 24.746344,predicted upper bound 24.752320,actual reward 21.959185
    round 1823, predicted reward 26.145592,predicted upper bound 26.151509,actual reward 29.753861
    round 1824, predicted reward 32.477516,predicted upper bound 32.482423,actual reward 33.861466
    round 1825, predicted reward 29.779861,predicted upper bound 29.786286,actual reward 29.105419
    round 1826, predicted reward 23.548039,predicted upper bound 23.552962,actual reward 19.914301
    round 1827, predicted reward 23.984218,predicted upper bound 23.990826,actual reward 19.723652
    round 1828, predicted reward 32.816810,predicted upper bound 32.822924,actual reward 36.390698
    round 1829, predicted reward 28.610412,predicted upper bound 28.617598,actual reward 29.146592
    round 1830, predicted reward 25.522819,predicted upper bound 25.528042,actual reward 22.779629
    round 1831, predicted reward 21.965360,predicted upper bound 21.970763,actual reward 15.850689
    round 1832, predicted reward 27.567299,predicted upper bound 27.573217,actual reward 23.612303
    round 1833, predicted reward 25.844012,predicted upper bound 25.849961,actual reward 25.677286
    round 1834, predicted reward 28.603212,predicted upper bound 28.608461,actual reward 26.168801
    round 1835, predicted reward 26.712884,predicted upper bound 26.717259,actual reward 23.614569
    round 1836, predicted reward 21.282619,predicted upper bound 21.288327,actual reward 21.017918
    round 1837, predicted reward 28.215131,predicted upper bound 28.221234,actual reward 29.313189
    round 1838, predicted reward 29.504797,predicted upper bound 29.511004,actual reward 29.463540
    round 1839, predicted reward 28.762542,predicted upper bound 28.768072,actual reward 26.116953
    round 1840, predicted reward 30.398245,predicted upper bound 30.403275,actual reward 31.026819
    round 1841, predicted reward 29.604850,predicted upper bound 29.610151,actual reward 31.580337
    round 1842, predicted reward 26.909660,predicted upper bound 26.915802,actual reward 21.290241
    round 1843, predicted reward 25.488477,predicted upper bound 25.494790,actual reward 22.309022
    round 1844, predicted reward 22.600536,predicted upper bound 22.607055,actual reward 20.052080
    round 1845, predicted reward 27.893957,predicted upper bound 27.899457,actual reward 25.657946
    round 1846, predicted reward 25.068972,predicted upper bound 25.074984,actual reward 27.475731
    round 1847, predicted reward 26.552364,predicted upper bound 26.557442,actual reward 28.849048
    round 1848, predicted reward 29.819370,predicted upper bound 29.824848,actual reward 28.930927
    round 1849, predicted reward 21.365671,predicted upper bound 21.372150,actual reward 19.495907
    round 1850, predicted reward 30.449918,predicted upper bound 30.456691,actual reward 35.975202
    round 1851, predicted reward 27.063340,predicted upper bound 27.069335,actual reward 27.176897
    round 1852, predicted reward 24.664185,predicted upper bound 24.669535,actual reward 21.636742
    round 1853, predicted reward 28.982557,predicted upper bound 28.988438,actual reward 30.976731
    round 1854, predicted reward 27.079657,predicted upper bound 27.084784,actual reward 27.972547
    round 1855, predicted reward 28.032507,predicted upper bound 28.038231,actual reward 22.019240
    round 1856, predicted reward 32.010853,predicted upper bound 32.016593,actual reward 33.582880
    round 1857, predicted reward 30.801024,predicted upper bound 30.805730,actual reward 31.067357
    round 1858, predicted reward 28.276883,predicted upper bound 28.282399,actual reward 26.316474
    round 1859, predicted reward 38.446701,predicted upper bound 38.451423,actual reward 40.320064
    round 1860, predicted reward 25.869257,predicted upper bound 25.875305,actual reward 21.982990
    round 1861, predicted reward 29.207508,predicted upper bound 29.212972,actual reward 30.107227
    round 1862, predicted reward 22.357164,predicted upper bound 22.363471,actual reward 17.998464
    round 1863, predicted reward 30.558056,predicted upper bound 30.564070,actual reward 29.124011
    round 1864, predicted reward 24.246373,predicted upper bound 24.251027,actual reward 21.160804
    round 1865, predicted reward 31.618143,predicted upper bound 31.623415,actual reward 21.290525
    round 1866, predicted reward 33.622101,predicted upper bound 33.627427,actual reward 35.071425
    round 1867, predicted reward 22.193585,predicted upper bound 22.200010,actual reward 21.269664
    round 1868, predicted reward 29.305587,predicted upper bound 29.311486,actual reward 28.020526
    round 1869, predicted reward 21.366045,predicted upper bound 21.373734,actual reward 23.620109
    round 1870, predicted reward 25.632097,predicted upper bound 25.637342,actual reward 17.867721
    round 1871, predicted reward 20.556433,predicted upper bound 20.562493,actual reward 18.974091
    round 1872, predicted reward 35.551913,predicted upper bound 35.557861,actual reward 36.136690
    round 1873, predicted reward 25.822697,predicted upper bound 25.828858,actual reward 22.506692
    round 1874, predicted reward 23.583590,predicted upper bound 23.589035,actual reward 20.299083
    round 1875, predicted reward 24.307386,predicted upper bound 24.312961,actual reward 18.676660
    round 1876, predicted reward 23.454998,predicted upper bound 23.460312,actual reward 15.979806
    round 1877, predicted reward 23.823556,predicted upper bound 23.829547,actual reward 25.402122
    round 1878, predicted reward 27.039567,predicted upper bound 27.045036,actual reward 24.078604
    round 1879, predicted reward 24.694369,predicted upper bound 24.700871,actual reward 22.845509
    round 1880, predicted reward 37.706312,predicted upper bound 37.711965,actual reward 40.654658
    round 1881, predicted reward 21.251529,predicted upper bound 21.258019,actual reward 23.298858
    round 1882, predicted reward 22.440032,predicted upper bound 22.445608,actual reward 14.507302
    round 1883, predicted reward 19.620010,predicted upper bound 19.626134,actual reward 21.919864
    round 1884, predicted reward 22.606557,predicted upper bound 22.612995,actual reward 27.830759
    round 1885, predicted reward 23.858294,predicted upper bound 23.863597,actual reward 19.093888
    round 1886, predicted reward 31.047496,predicted upper bound 31.052350,actual reward 30.278392
    round 1887, predicted reward 26.413412,predicted upper bound 26.418402,actual reward 25.138363
    round 1888, predicted reward 33.504398,predicted upper bound 33.510101,actual reward 35.706347
    round 1889, predicted reward 22.366973,predicted upper bound 22.373445,actual reward 20.735823
    round 1890, predicted reward 31.620799,predicted upper bound 31.626619,actual reward 37.727372
    round 1891, predicted reward 24.486216,predicted upper bound 24.493433,actual reward 21.862992
    round 1892, predicted reward 24.951354,predicted upper bound 24.958271,actual reward 21.710784
    round 1893, predicted reward 23.908851,predicted upper bound 23.914835,actual reward 20.130188
    round 1894, predicted reward 26.266431,predicted upper bound 26.271931,actual reward 27.812903
    round 1895, predicted reward 35.169819,predicted upper bound 35.174703,actual reward 36.148636
    round 1896, predicted reward 26.134324,predicted upper bound 26.140410,actual reward 26.056170
    round 1897, predicted reward 25.522426,predicted upper bound 25.529417,actual reward 28.133153
    round 1898, predicted reward 30.760915,predicted upper bound 30.765819,actual reward 31.501943
    round 1899, predicted reward 22.772123,predicted upper bound 22.778205,actual reward 26.972078
    round 1900, predicted reward 36.194933,predicted upper bound 36.199471,actual reward 39.293082
    round 1901, predicted reward 23.458412,predicted upper bound 23.464706,actual reward 24.791248
    round 1902, predicted reward 30.569944,predicted upper bound 30.576511,actual reward 28.790150
    round 1903, predicted reward 23.385660,predicted upper bound 23.392048,actual reward 20.451544
    round 1904, predicted reward 31.166547,predicted upper bound 31.170884,actual reward 36.729511
    round 1905, predicted reward 32.002581,predicted upper bound 32.006843,actual reward 35.160743
    round 1906, predicted reward 29.677222,predicted upper bound 29.681681,actual reward 28.752053
    round 1907, predicted reward 25.408127,predicted upper bound 25.414157,actual reward 22.473592
    round 1908, predicted reward 26.984514,predicted upper bound 26.990574,actual reward 22.595844
    round 1909, predicted reward 27.687880,predicted upper bound 27.693193,actual reward 21.832298
    round 1910, predicted reward 27.689068,predicted upper bound 27.694455,actual reward 30.085007
    round 1911, predicted reward 30.297689,predicted upper bound 30.302828,actual reward 29.283820
    round 1912, predicted reward 28.090830,predicted upper bound 28.096844,actual reward 31.091339
    round 1913, predicted reward 25.218916,predicted upper bound 25.226385,actual reward 21.593612
    round 1914, predicted reward 28.912175,predicted upper bound 28.917459,actual reward 29.614617
    round 1915, predicted reward 23.975830,predicted upper bound 23.982073,actual reward 23.440716
    round 1916, predicted reward 27.754174,predicted upper bound 27.759674,actual reward 25.292146
    round 1917, predicted reward 29.982996,predicted upper bound 29.987769,actual reward 31.801518
    round 1918, predicted reward 30.160290,predicted upper bound 30.166645,actual reward 27.225508
    round 1919, predicted reward 31.840662,predicted upper bound 31.846313,actual reward 32.074728
    round 1920, predicted reward 22.813683,predicted upper bound 22.819736,actual reward 15.140420
    round 1921, predicted reward 27.316550,predicted upper bound 27.322260,actual reward 20.373120
    round 1922, predicted reward 30.471459,predicted upper bound 30.476563,actual reward 31.329693
    round 1923, predicted reward 23.082706,predicted upper bound 23.088333,actual reward 20.824142
    round 1924, predicted reward 28.086671,predicted upper bound 28.092534,actual reward 29.527645
    round 1925, predicted reward 28.830407,predicted upper bound 28.835868,actual reward 27.867434
    round 1926, predicted reward 21.375234,predicted upper bound 21.380982,actual reward 21.290889
    round 1927, predicted reward 26.719336,predicted upper bound 26.726099,actual reward 26.197297
    round 1928, predicted reward 30.071189,predicted upper bound 30.075991,actual reward 29.449115
    round 1929, predicted reward 30.471890,predicted upper bound 30.477305,actual reward 30.349052
    round 1930, predicted reward 34.594351,predicted upper bound 34.599299,actual reward 32.419985
    round 1931, predicted reward 24.797459,predicted upper bound 24.802776,actual reward 22.461176
    round 1932, predicted reward 26.221268,predicted upper bound 26.227731,actual reward 29.038700
    round 1933, predicted reward 28.888113,predicted upper bound 28.893398,actual reward 24.396976
    round 1934, predicted reward 33.287833,predicted upper bound 33.292502,actual reward 33.493156
    round 1935, predicted reward 28.416721,predicted upper bound 28.422029,actual reward 26.569058
    round 1936, predicted reward 30.198486,predicted upper bound 30.204309,actual reward 27.484228
    round 1937, predicted reward 28.662907,predicted upper bound 28.668138,actual reward 29.673027
    round 1938, predicted reward 25.017973,predicted upper bound 25.024343,actual reward 28.884529
    round 1939, predicted reward 29.938330,predicted upper bound 29.944156,actual reward 34.275560
    round 1940, predicted reward 22.642579,predicted upper bound 22.648629,actual reward 23.305016
    round 1941, predicted reward 25.483530,predicted upper bound 25.488902,actual reward 25.699065
    round 1942, predicted reward 34.443875,predicted upper bound 34.449251,actual reward 34.216656
    round 1943, predicted reward 22.581832,predicted upper bound 22.586401,actual reward 18.523297
    round 1944, predicted reward 34.879718,predicted upper bound 34.884876,actual reward 40.638870
    round 1945, predicted reward 27.249251,predicted upper bound 27.254373,actual reward 25.415322
    round 1946, predicted reward 27.301985,predicted upper bound 27.308069,actual reward 24.353097
    round 1947, predicted reward 21.353291,predicted upper bound 21.359366,actual reward 22.204465
    round 1948, predicted reward 26.940633,predicted upper bound 26.945991,actual reward 23.880056
    round 1949, predicted reward 22.914974,predicted upper bound 22.922526,actual reward 21.637987
    round 1950, predicted reward 21.886363,predicted upper bound 21.892249,actual reward 21.944981
    round 1951, predicted reward 24.723800,predicted upper bound 24.729484,actual reward 23.808841
    round 1952, predicted reward 29.868293,predicted upper bound 29.874895,actual reward 35.018094
    round 1953, predicted reward 26.753872,predicted upper bound 26.759917,actual reward 27.220635
    round 1954, predicted reward 35.584729,predicted upper bound 35.590074,actual reward 35.221511
    round 1955, predicted reward 28.856317,predicted upper bound 28.862731,actual reward 26.132581
    round 1956, predicted reward 27.830601,predicted upper bound 27.836394,actual reward 21.253843
    round 1957, predicted reward 24.144906,predicted upper bound 24.150778,actual reward 24.895322
    round 1958, predicted reward 27.832032,predicted upper bound 27.837531,actual reward 22.858300
    round 1959, predicted reward 23.048056,predicted upper bound 23.053369,actual reward 21.280229
    round 1960, predicted reward 25.906686,predicted upper bound 25.911761,actual reward 23.091028
    round 1961, predicted reward 29.183576,predicted upper bound 29.190193,actual reward 26.682329
    round 1962, predicted reward 28.115047,predicted upper bound 28.120694,actual reward 23.166897
    round 1963, predicted reward 28.009540,predicted upper bound 28.016219,actual reward 30.303508
    round 1964, predicted reward 30.552995,predicted upper bound 30.559442,actual reward 40.052616
    round 1965, predicted reward 28.270068,predicted upper bound 28.275156,actual reward 27.463672
    round 1966, predicted reward 25.304272,predicted upper bound 25.309727,actual reward 27.047139
    round 1967, predicted reward 22.990311,predicted upper bound 22.996255,actual reward 25.224405
    round 1968, predicted reward 30.598191,predicted upper bound 30.604134,actual reward 30.971480
    round 1969, predicted reward 27.705213,predicted upper bound 27.711072,actual reward 27.425930
    round 1970, predicted reward 20.754602,predicted upper bound 20.759316,actual reward 16.809658
    round 1971, predicted reward 22.366500,predicted upper bound 22.373058,actual reward 21.054473
    round 1972, predicted reward 33.898098,predicted upper bound 33.904585,actual reward 36.937056
    round 1973, predicted reward 35.307061,predicted upper bound 35.311847,actual reward 35.749582
    round 1974, predicted reward 30.108452,predicted upper bound 30.114582,actual reward 29.680852
    round 1975, predicted reward 24.358921,predicted upper bound 24.364615,actual reward 26.163521
    round 1976, predicted reward 24.314837,predicted upper bound 24.321246,actual reward 26.537058
    round 1977, predicted reward 21.902963,predicted upper bound 21.909696,actual reward 20.327149
    round 1978, predicted reward 33.093913,predicted upper bound 33.099830,actual reward 26.825718
    round 1979, predicted reward 25.700717,predicted upper bound 25.706692,actual reward 26.233695
    round 1980, predicted reward 36.176731,predicted upper bound 36.182028,actual reward 36.786978
    round 1981, predicted reward 36.572882,predicted upper bound 36.578213,actual reward 49.524380
    round 1982, predicted reward 29.697060,predicted upper bound 29.701644,actual reward 33.275539
    round 1983, predicted reward 25.867103,predicted upper bound 25.871780,actual reward 22.886260
    round 1984, predicted reward 31.704988,predicted upper bound 31.710674,actual reward 30.300752
    round 1985, predicted reward 22.585395,predicted upper bound 22.592383,actual reward 22.090444
    round 1986, predicted reward 27.859351,predicted upper bound 27.864915,actual reward 28.834850
    round 1987, predicted reward 29.564525,predicted upper bound 29.569996,actual reward 28.430276
    round 1988, predicted reward 23.553377,predicted upper bound 23.559360,actual reward 25.688265
    round 1989, predicted reward 22.148252,predicted upper bound 22.153739,actual reward 19.095462
    round 1990, predicted reward 20.032460,predicted upper bound 20.039286,actual reward 19.780212
    round 1991, predicted reward 23.325789,predicted upper bound 23.331723,actual reward 23.953185
    round 1992, predicted reward 29.563413,predicted upper bound 29.569559,actual reward 30.353872
    round 1993, predicted reward 29.037299,predicted upper bound 29.042702,actual reward 32.254612
    round 1994, predicted reward 26.706084,predicted upper bound 26.712557,actual reward 26.817471
    round 1995, predicted reward 26.204506,predicted upper bound 26.211070,actual reward 25.851589
    round 1996, predicted reward 32.108943,predicted upper bound 32.114509,actual reward 33.116165
    round 1997, predicted reward 34.308468,predicted upper bound 34.313805,actual reward 40.010628
    round 1998, predicted reward 20.527714,predicted upper bound 20.534440,actual reward 19.805144
    round 1999, predicted reward 20.395682,predicted upper bound 20.401925,actual reward 14.901175
    round 2000, predicted reward 26.353374,predicted upper bound 26.359690,actual reward 25.142495
    round 2001, predicted reward 31.397229,predicted upper bound 31.402956,actual reward 31.269700
    round 2002, predicted reward 23.199853,predicted upper bound 23.206479,actual reward 26.197144
    round 2003, predicted reward 24.470627,predicted upper bound 24.476976,actual reward 22.934094
    round 2004, predicted reward 32.680764,predicted upper bound 32.685953,actual reward 34.705419
    round 2005, predicted reward 20.322780,predicted upper bound 20.328407,actual reward 20.766909
    round 2006, predicted reward 26.842501,predicted upper bound 26.848185,actual reward 27.995495
    round 2007, predicted reward 29.337359,predicted upper bound 29.342725,actual reward 30.209948
    round 2008, predicted reward 18.749650,predicted upper bound 18.755591,actual reward 17.270623
    round 2009, predicted reward 23.300237,predicted upper bound 23.307142,actual reward 25.050790
    round 2010, predicted reward 29.898703,predicted upper bound 29.904474,actual reward 27.283880
    round 2011, predicted reward 32.765831,predicted upper bound 32.770985,actual reward 29.605258
    round 2012, predicted reward 29.371450,predicted upper bound 29.376872,actual reward 27.561393
    round 2013, predicted reward 30.848137,predicted upper bound 30.854299,actual reward 33.585790
    round 2014, predicted reward 21.272647,predicted upper bound 21.278228,actual reward 16.987300
    round 2015, predicted reward 28.189199,predicted upper bound 28.194544,actual reward 26.870393
    round 2016, predicted reward 31.546908,predicted upper bound 31.551733,actual reward 30.964208
    round 2017, predicted reward 26.642639,predicted upper bound 26.647498,actual reward 21.569521
    round 2018, predicted reward 27.816417,predicted upper bound 27.821785,actual reward 22.650646
    round 2019, predicted reward 28.688870,predicted upper bound 28.693360,actual reward 25.498381
    round 2020, predicted reward 32.087017,predicted upper bound 32.092681,actual reward 36.030509
    round 2021, predicted reward 26.781368,predicted upper bound 26.786950,actual reward 23.860498
    round 2022, predicted reward 23.548322,predicted upper bound 23.553397,actual reward 22.368312
    round 2023, predicted reward 23.579318,predicted upper bound 23.583870,actual reward 23.379771
    round 2024, predicted reward 24.447831,predicted upper bound 24.454138,actual reward 24.087848
    round 2025, predicted reward 23.490425,predicted upper bound 23.496235,actual reward 22.922334
    round 2026, predicted reward 25.020976,predicted upper bound 25.027964,actual reward 21.643711
    round 2027, predicted reward 30.176064,predicted upper bound 30.180200,actual reward 26.152175
    round 2028, predicted reward 25.128547,predicted upper bound 25.134724,actual reward 22.578140
    round 2029, predicted reward 31.487530,predicted upper bound 31.493334,actual reward 27.215204
    round 2030, predicted reward 28.347956,predicted upper bound 28.354439,actual reward 32.442320
    round 2031, predicted reward 22.457281,predicted upper bound 22.463038,actual reward 19.723024
    round 2032, predicted reward 23.714434,predicted upper bound 23.720573,actual reward 21.823875
    round 2033, predicted reward 24.513760,predicted upper bound 24.519786,actual reward 23.700567
    round 2034, predicted reward 24.764612,predicted upper bound 24.769467,actual reward 26.428006
    round 2035, predicted reward 24.619085,predicted upper bound 24.624691,actual reward 17.846413
    round 2036, predicted reward 28.460281,predicted upper bound 28.465506,actual reward 26.077179
    round 2037, predicted reward 22.309483,predicted upper bound 22.315534,actual reward 17.943429
    round 2038, predicted reward 28.946275,predicted upper bound 28.951434,actual reward 24.758551
    round 2039, predicted reward 27.063800,predicted upper bound 27.068567,actual reward 25.060183
    round 2040, predicted reward 32.286373,predicted upper bound 32.291219,actual reward 34.242058
    round 2041, predicted reward 35.101480,predicted upper bound 35.106737,actual reward 38.615929
    round 2042, predicted reward 25.974595,predicted upper bound 25.980190,actual reward 22.872003
    round 2043, predicted reward 26.094316,predicted upper bound 26.100132,actual reward 24.716968
    round 2044, predicted reward 27.632086,predicted upper bound 27.637451,actual reward 33.266973
    round 2045, predicted reward 26.083487,predicted upper bound 26.088708,actual reward 22.091182
    round 2046, predicted reward 21.858535,predicted upper bound 21.864089,actual reward 24.526222
    round 2047, predicted reward 23.715575,predicted upper bound 23.721385,actual reward 19.972794
    round 2048, predicted reward 27.996575,predicted upper bound 28.001995,actual reward 24.564684
    round 2049, predicted reward 21.145839,predicted upper bound 21.152192,actual reward 24.747469
    round 2050, predicted reward 25.757870,predicted upper bound 25.762640,actual reward 23.055407
    round 2051, predicted reward 25.032580,predicted upper bound 25.039629,actual reward 20.287597
    round 2052, predicted reward 33.311703,predicted upper bound 33.317041,actual reward 36.997393
    round 2053, predicted reward 22.785153,predicted upper bound 22.791514,actual reward 19.405191
    round 2054, predicted reward 31.229015,predicted upper bound 31.234165,actual reward 32.944711
    round 2055, predicted reward 25.757404,predicted upper bound 25.763744,actual reward 24.843721
    round 2056, predicted reward 28.049752,predicted upper bound 28.055682,actual reward 30.613437
    round 2057, predicted reward 29.442815,predicted upper bound 29.448080,actual reward 19.725383
    round 2058, predicted reward 28.661769,predicted upper bound 28.667472,actual reward 33.481130
    round 2059, predicted reward 30.561839,predicted upper bound 30.566920,actual reward 35.395311
    round 2060, predicted reward 29.140164,predicted upper bound 29.147080,actual reward 28.151468
    round 2061, predicted reward 30.118390,predicted upper bound 30.123100,actual reward 28.791242
    round 2062, predicted reward 25.213912,predicted upper bound 25.218818,actual reward 29.374203
    round 2063, predicted reward 24.608274,predicted upper bound 24.613926,actual reward 27.248164
    round 2064, predicted reward 25.121055,predicted upper bound 25.126336,actual reward 21.500537
    round 2065, predicted reward 27.455112,predicted upper bound 27.460563,actual reward 27.773959
    round 2066, predicted reward 28.509291,predicted upper bound 28.514393,actual reward 27.405514
    round 2067, predicted reward 26.485850,predicted upper bound 26.491893,actual reward 25.766226
    round 2068, predicted reward 26.202468,predicted upper bound 26.207959,actual reward 21.757723
    round 2069, predicted reward 26.553337,predicted upper bound 26.559068,actual reward 27.673164
    round 2070, predicted reward 25.311575,predicted upper bound 25.316834,actual reward 25.559903
    round 2071, predicted reward 22.141307,predicted upper bound 22.147182,actual reward 25.681063
    round 2072, predicted reward 28.962918,predicted upper bound 28.968723,actual reward 22.694268
    round 2073, predicted reward 20.050698,predicted upper bound 20.057244,actual reward 19.849246
    round 2074, predicted reward 23.763550,predicted upper bound 23.768614,actual reward 27.185516
    round 2075, predicted reward 26.304925,predicted upper bound 26.310047,actual reward 25.125781
    round 2076, predicted reward 27.791348,predicted upper bound 27.796116,actual reward 28.841180
    round 2077, predicted reward 25.775147,predicted upper bound 25.780142,actual reward 24.532999
    round 2078, predicted reward 41.703517,predicted upper bound 41.708065,actual reward 47.969544
    round 2079, predicted reward 29.001620,predicted upper bound 29.007431,actual reward 31.110640
    round 2080, predicted reward 32.854734,predicted upper bound 32.859301,actual reward 36.044634
    round 2081, predicted reward 21.817502,predicted upper bound 21.823554,actual reward 18.509769
    round 2082, predicted reward 25.726907,predicted upper bound 25.733066,actual reward 22.690113
    round 2083, predicted reward 27.874541,predicted upper bound 27.879582,actual reward 31.390220
    round 2084, predicted reward 23.928897,predicted upper bound 23.934133,actual reward 22.597277
    round 2085, predicted reward 34.619528,predicted upper bound 34.623712,actual reward 39.312580
    round 2086, predicted reward 28.594368,predicted upper bound 28.600064,actual reward 26.193660
    round 2087, predicted reward 29.758764,predicted upper bound 29.763434,actual reward 28.448187
    round 2088, predicted reward 20.348144,predicted upper bound 20.353299,actual reward 16.854434
    round 2089, predicted reward 24.832705,predicted upper bound 24.838036,actual reward 27.333502
    round 2090, predicted reward 26.433635,predicted upper bound 26.439054,actual reward 22.266376
    round 2091, predicted reward 24.288537,predicted upper bound 24.295181,actual reward 21.037850
    round 2092, predicted reward 28.761939,predicted upper bound 28.767837,actual reward 31.499982
    round 2093, predicted reward 31.341869,predicted upper bound 31.346936,actual reward 33.838608
    round 2094, predicted reward 25.454117,predicted upper bound 25.459063,actual reward 23.251995
    round 2095, predicted reward 26.712660,predicted upper bound 26.717976,actual reward 32.079515
    round 2096, predicted reward 27.747368,predicted upper bound 27.754638,actual reward 28.071651
    round 2097, predicted reward 22.024392,predicted upper bound 22.029186,actual reward 25.999458
    round 2098, predicted reward 31.497020,predicted upper bound 31.501266,actual reward 33.514144
    round 2099, predicted reward 26.346072,predicted upper bound 26.350774,actual reward 25.276522
    round 2100, predicted reward 28.449484,predicted upper bound 28.454408,actual reward 33.153929
    round 2101, predicted reward 22.578065,predicted upper bound 22.584323,actual reward 25.703527
    round 2102, predicted reward 26.414121,predicted upper bound 26.419677,actual reward 26.527799
    round 2103, predicted reward 26.015058,predicted upper bound 26.021187,actual reward 19.026460
    round 2104, predicted reward 22.710101,predicted upper bound 22.716039,actual reward 22.028656
    round 2105, predicted reward 28.853562,predicted upper bound 28.859320,actual reward 31.448019
    round 2106, predicted reward 21.918402,predicted upper bound 21.924025,actual reward 20.109532
    round 2107, predicted reward 36.658222,predicted upper bound 36.662857,actual reward 33.529372
    round 2108, predicted reward 23.379389,predicted upper bound 23.385098,actual reward 25.360612
    round 2109, predicted reward 23.463949,predicted upper bound 23.469999,actual reward 22.502578
    round 2110, predicted reward 26.993330,predicted upper bound 26.999329,actual reward 24.589472
    round 2111, predicted reward 25.473564,predicted upper bound 25.479008,actual reward 26.876679
    round 2112, predicted reward 22.972558,predicted upper bound 22.978966,actual reward 27.067544
    round 2113, predicted reward 23.659905,predicted upper bound 23.665780,actual reward 22.204004
    round 2114, predicted reward 23.522128,predicted upper bound 23.527132,actual reward 19.556443
    round 2115, predicted reward 24.577349,predicted upper bound 24.583678,actual reward 18.619123
    round 2116, predicted reward 30.158330,predicted upper bound 30.163038,actual reward 29.568773
    round 2117, predicted reward 27.822545,predicted upper bound 27.826950,actual reward 30.178911
    round 2118, predicted reward 21.680967,predicted upper bound 21.686473,actual reward 18.518081
    round 2119, predicted reward 29.711463,predicted upper bound 29.716824,actual reward 26.613573
    round 2120, predicted reward 31.060548,predicted upper bound 31.066472,actual reward 34.316017
    round 2121, predicted reward 25.530312,predicted upper bound 25.535947,actual reward 23.683390
    round 2122, predicted reward 22.300033,predicted upper bound 22.306295,actual reward 19.307454
    round 2123, predicted reward 25.924866,predicted upper bound 25.930441,actual reward 24.773075
    round 2124, predicted reward 32.056474,predicted upper bound 32.061788,actual reward 33.062881
    round 2125, predicted reward 33.851561,predicted upper bound 33.856694,actual reward 34.171673
    round 2126, predicted reward 30.335889,predicted upper bound 30.341078,actual reward 27.141407
    round 2127, predicted reward 25.740573,predicted upper bound 25.745699,actual reward 28.559858
    round 2128, predicted reward 26.534494,predicted upper bound 26.539661,actual reward 25.022871
    round 2129, predicted reward 28.034823,predicted upper bound 28.039982,actual reward 23.340342
    round 2130, predicted reward 33.546459,predicted upper bound 33.552092,actual reward 33.728174
    round 2131, predicted reward 24.476596,predicted upper bound 24.481624,actual reward 22.978131
    round 2132, predicted reward 37.325268,predicted upper bound 37.330602,actual reward 39.851067
    round 2133, predicted reward 25.872899,predicted upper bound 25.878333,actual reward 31.901571
    round 2134, predicted reward 36.335968,predicted upper bound 36.341280,actual reward 38.981673
    round 2135, predicted reward 23.659541,predicted upper bound 23.666388,actual reward 20.574436
    round 2136, predicted reward 26.953986,predicted upper bound 26.959399,actual reward 25.343646
    round 2137, predicted reward 31.313251,predicted upper bound 31.317598,actual reward 31.229093
    round 2138, predicted reward 27.917062,predicted upper bound 27.922997,actual reward 23.789932
    round 2139, predicted reward 26.564156,predicted upper bound 26.570459,actual reward 22.087629
    round 2140, predicted reward 27.018269,predicted upper bound 27.022728,actual reward 24.446324
    round 2141, predicted reward 25.689389,predicted upper bound 25.694305,actual reward 22.904457
    round 2142, predicted reward 25.848450,predicted upper bound 25.854704,actual reward 25.894495
    round 2143, predicted reward 27.479488,predicted upper bound 27.484403,actual reward 35.374411
    round 2144, predicted reward 30.570666,predicted upper bound 30.575906,actual reward 28.655959
    round 2145, predicted reward 30.365446,predicted upper bound 30.371120,actual reward 28.176284
    round 2146, predicted reward 29.653880,predicted upper bound 29.659427,actual reward 33.125228
    round 2147, predicted reward 20.908498,predicted upper bound 20.914273,actual reward 19.648635
    round 2148, predicted reward 28.641491,predicted upper bound 28.647450,actual reward 29.980045
    round 2149, predicted reward 33.691980,predicted upper bound 33.696936,actual reward 30.925093
    round 2150, predicted reward 26.156737,predicted upper bound 26.161986,actual reward 24.950506
    round 2151, predicted reward 21.384140,predicted upper bound 21.389695,actual reward 18.051182
    round 2152, predicted reward 31.220651,predicted upper bound 31.225884,actual reward 29.841917
    round 2153, predicted reward 25.553063,predicted upper bound 25.558536,actual reward 27.155592
    round 2154, predicted reward 27.699044,predicted upper bound 27.704422,actual reward 26.101281
    round 2155, predicted reward 20.431182,predicted upper bound 20.437535,actual reward 17.850016
    round 2156, predicted reward 29.811212,predicted upper bound 29.816163,actual reward 33.442587
    round 2157, predicted reward 32.441902,predicted upper bound 32.447331,actual reward 30.556203
    round 2158, predicted reward 29.423531,predicted upper bound 29.429288,actual reward 28.771054
    round 2159, predicted reward 30.154361,predicted upper bound 30.159954,actual reward 25.407549
    round 2160, predicted reward 29.794024,predicted upper bound 29.799185,actual reward 30.008930
    round 2161, predicted reward 30.450847,predicted upper bound 30.456830,actual reward 34.441574
    round 2162, predicted reward 28.138316,predicted upper bound 28.143123,actual reward 21.318268
    round 2163, predicted reward 24.895210,predicted upper bound 24.901177,actual reward 26.153320
    round 2164, predicted reward 22.680104,predicted upper bound 22.686827,actual reward 19.154211
    round 2165, predicted reward 28.463120,predicted upper bound 28.468094,actual reward 26.972987
    round 2166, predicted reward 24.919672,predicted upper bound 24.925777,actual reward 30.732475
    round 2167, predicted reward 20.241957,predicted upper bound 20.248191,actual reward 19.916234
    round 2168, predicted reward 22.214365,predicted upper bound 22.221310,actual reward 23.253040
    round 2169, predicted reward 23.937488,predicted upper bound 23.944017,actual reward 27.533191
    round 2170, predicted reward 30.506212,predicted upper bound 30.511454,actual reward 29.688937
    round 2171, predicted reward 33.136258,predicted upper bound 33.141077,actual reward 32.479181
    round 2172, predicted reward 28.050172,predicted upper bound 28.056216,actual reward 29.878744
    round 2173, predicted reward 28.568045,predicted upper bound 28.573482,actual reward 33.193572
    round 2174, predicted reward 24.833222,predicted upper bound 24.840254,actual reward 18.347687
    round 2175, predicted reward 31.528498,predicted upper bound 31.534196,actual reward 28.045434
    round 2176, predicted reward 29.048230,predicted upper bound 29.053228,actual reward 27.973635
    round 2177, predicted reward 22.512675,predicted upper bound 22.519136,actual reward 19.766507
    round 2178, predicted reward 27.065143,predicted upper bound 27.070317,actual reward 30.741282
    round 2179, predicted reward 28.988213,predicted upper bound 28.993454,actual reward 29.382770
    round 2180, predicted reward 24.206898,predicted upper bound 24.213095,actual reward 22.770425
    round 2181, predicted reward 27.242189,predicted upper bound 27.247591,actual reward 20.124602
    round 2182, predicted reward 40.368619,predicted upper bound 40.372906,actual reward 50.351977
    round 2183, predicted reward 22.968583,predicted upper bound 22.975396,actual reward 25.760789
    round 2184, predicted reward 27.915511,predicted upper bound 27.921281,actual reward 27.985212
    round 2185, predicted reward 27.989707,predicted upper bound 27.995012,actual reward 23.555450
    round 2186, predicted reward 25.090460,predicted upper bound 25.095825,actual reward 19.425431
    round 2187, predicted reward 22.031773,predicted upper bound 22.038160,actual reward 19.472625
    round 2188, predicted reward 30.241228,predicted upper bound 30.245283,actual reward 31.181324
    round 2189, predicted reward 24.527985,predicted upper bound 24.534037,actual reward 20.266593
    round 2190, predicted reward 29.120047,predicted upper bound 29.125656,actual reward 26.919782
    round 2191, predicted reward 26.118739,predicted upper bound 26.124681,actual reward 26.272092
    round 2192, predicted reward 29.004489,predicted upper bound 29.009850,actual reward 27.098606
    round 2193, predicted reward 22.156089,predicted upper bound 22.161532,actual reward 17.595657
    round 2194, predicted reward 25.547813,predicted upper bound 25.553369,actual reward 29.198181
    round 2195, predicted reward 25.673423,predicted upper bound 25.679400,actual reward 22.905733
    round 2196, predicted reward 22.976370,predicted upper bound 22.981632,actual reward 18.466829
    round 2197, predicted reward 28.000681,predicted upper bound 28.005978,actual reward 30.874154
    round 2198, predicted reward 32.479202,predicted upper bound 32.484669,actual reward 35.690185
    round 2199, predicted reward 30.386083,predicted upper bound 30.391130,actual reward 37.241224
    round 2200, predicted reward 28.282554,predicted upper bound 28.287739,actual reward 25.152207
    round 2201, predicted reward 27.994278,predicted upper bound 27.999562,actual reward 30.822462
    round 2202, predicted reward 29.797340,predicted upper bound 29.802506,actual reward 31.814054
    round 2203, predicted reward 21.197620,predicted upper bound 21.203878,actual reward 19.503890
    round 2204, predicted reward 25.709173,predicted upper bound 25.714156,actual reward 23.701270
    round 2205, predicted reward 30.815338,predicted upper bound 30.820434,actual reward 25.887473
    round 2206, predicted reward 26.579643,predicted upper bound 26.585264,actual reward 22.011152
    round 2207, predicted reward 27.586381,predicted upper bound 27.590801,actual reward 27.307102
    round 2208, predicted reward 29.956892,predicted upper bound 29.963497,actual reward 29.717877
    round 2209, predicted reward 30.275772,predicted upper bound 30.281393,actual reward 29.199221
    round 2210, predicted reward 29.077281,predicted upper bound 29.082710,actual reward 28.244324
    round 2211, predicted reward 32.878057,predicted upper bound 32.882359,actual reward 30.915805
    round 2212, predicted reward 21.968678,predicted upper bound 21.974127,actual reward 17.150652
    round 2213, predicted reward 30.322003,predicted upper bound 30.327474,actual reward 30.303616
    round 2214, predicted reward 21.804016,predicted upper bound 21.809589,actual reward 20.037549
    round 2215, predicted reward 35.455664,predicted upper bound 35.461384,actual reward 33.610045
    round 2216, predicted reward 39.965102,predicted upper bound 39.969582,actual reward 44.487671
    round 2217, predicted reward 26.648026,predicted upper bound 26.652914,actual reward 24.684409
    round 2218, predicted reward 30.829723,predicted upper bound 30.834717,actual reward 34.254960
    round 2219, predicted reward 26.797590,predicted upper bound 26.802780,actual reward 18.831428
    round 2220, predicted reward 28.023226,predicted upper bound 28.028468,actual reward 28.346745
    round 2221, predicted reward 24.019957,predicted upper bound 24.025330,actual reward 21.392433
    round 2222, predicted reward 26.627443,predicted upper bound 26.632335,actual reward 25.455602
    round 2223, predicted reward 24.937873,predicted upper bound 24.943294,actual reward 21.797387
    round 2224, predicted reward 28.133649,predicted upper bound 28.138674,actual reward 28.168299
    round 2225, predicted reward 27.929667,predicted upper bound 27.934857,actual reward 26.472722
    round 2226, predicted reward 29.561301,predicted upper bound 29.566994,actual reward 24.177776
    round 2227, predicted reward 21.559189,predicted upper bound 21.564811,actual reward 20.727433
    round 2228, predicted reward 23.000299,predicted upper bound 23.005186,actual reward 20.366058
    round 2229, predicted reward 22.649622,predicted upper bound 22.655359,actual reward 22.486597
    round 2230, predicted reward 31.416325,predicted upper bound 31.421476,actual reward 34.270079
    round 2231, predicted reward 23.807492,predicted upper bound 23.813424,actual reward 21.683380
    round 2232, predicted reward 24.915428,predicted upper bound 24.921096,actual reward 25.245768
    round 2233, predicted reward 28.570370,predicted upper bound 28.575514,actual reward 28.310038
    round 2234, predicted reward 34.063405,predicted upper bound 34.068790,actual reward 33.693833
    round 2235, predicted reward 21.248508,predicted upper bound 21.254290,actual reward 23.701475
    round 2236, predicted reward 24.152856,predicted upper bound 24.158256,actual reward 22.792005
    round 2237, predicted reward 23.623755,predicted upper bound 23.628999,actual reward 22.273234
    round 2238, predicted reward 33.224619,predicted upper bound 33.229773,actual reward 34.806807
    round 2239, predicted reward 22.693504,predicted upper bound 22.698940,actual reward 18.387818
    round 2240, predicted reward 35.646479,predicted upper bound 35.651513,actual reward 35.215462
    round 2241, predicted reward 25.412695,predicted upper bound 25.417859,actual reward 24.241455
    round 2242, predicted reward 26.265536,predicted upper bound 26.271737,actual reward 26.603674
    round 2243, predicted reward 27.676564,predicted upper bound 27.682428,actual reward 26.728910
    round 2244, predicted reward 20.668413,predicted upper bound 20.674276,actual reward 17.855066
    round 2245, predicted reward 28.737139,predicted upper bound 28.742297,actual reward 25.631689
    round 2246, predicted reward 27.962430,predicted upper bound 27.966857,actual reward 27.311438
    round 2247, predicted reward 26.253825,predicted upper bound 26.258240,actual reward 25.712662
    round 2248, predicted reward 27.724263,predicted upper bound 27.729598,actual reward 27.427128
    round 2249, predicted reward 28.683739,predicted upper bound 28.689890,actual reward 27.408651
    round 2250, predicted reward 32.660789,predicted upper bound 32.664656,actual reward 34.148873
    round 2251, predicted reward 27.297240,predicted upper bound 27.303228,actual reward 21.993825
    round 2252, predicted reward 22.169362,predicted upper bound 22.173983,actual reward 20.779544
    round 2253, predicted reward 23.526062,predicted upper bound 23.531698,actual reward 24.500195
    round 2254, predicted reward 24.046320,predicted upper bound 24.052287,actual reward 22.023449
    round 2255, predicted reward 32.995845,predicted upper bound 33.000348,actual reward 32.518221
    round 2256, predicted reward 23.801062,predicted upper bound 23.807105,actual reward 21.484231
    round 2257, predicted reward 27.708857,predicted upper bound 27.713623,actual reward 24.345202
    round 2258, predicted reward 27.137535,predicted upper bound 27.143149,actual reward 30.417102
    round 2259, predicted reward 25.002282,predicted upper bound 25.008285,actual reward 19.325348
    round 2260, predicted reward 32.996846,predicted upper bound 33.001605,actual reward 28.756415
    round 2261, predicted reward 30.087149,predicted upper bound 30.092134,actual reward 32.827948
    round 2262, predicted reward 21.408467,predicted upper bound 21.414279,actual reward 20.152666
    round 2263, predicted reward 23.958415,predicted upper bound 23.963291,actual reward 22.332821
    round 2264, predicted reward 23.876793,predicted upper bound 23.881927,actual reward 23.296156
    round 2265, predicted reward 29.588709,predicted upper bound 29.593722,actual reward 23.668700
    round 2266, predicted reward 25.806641,predicted upper bound 25.812670,actual reward 24.886252
    round 2267, predicted reward 27.842160,predicted upper bound 27.846654,actual reward 26.353198
    round 2268, predicted reward 25.400780,predicted upper bound 25.405897,actual reward 27.251618
    round 2269, predicted reward 24.571294,predicted upper bound 24.576160,actual reward 27.716579
    round 2270, predicted reward 23.900380,predicted upper bound 23.905438,actual reward 19.620607
    round 2271, predicted reward 20.396127,predicted upper bound 20.402608,actual reward 19.354858
    round 2272, predicted reward 25.543584,predicted upper bound 25.549283,actual reward 23.709190
    round 2273, predicted reward 25.662588,predicted upper bound 25.668853,actual reward 20.327350
    round 2274, predicted reward 27.171252,predicted upper bound 27.176272,actual reward 21.871637
    round 2275, predicted reward 27.673192,predicted upper bound 27.678400,actual reward 32.334070
    round 2276, predicted reward 35.109584,predicted upper bound 35.114023,actual reward 37.855383
    round 2277, predicted reward 23.588653,predicted upper bound 23.594046,actual reward 19.088898
    round 2278, predicted reward 27.596265,predicted upper bound 27.601517,actual reward 27.547706
    round 2279, predicted reward 34.908824,predicted upper bound 34.913633,actual reward 37.635575
    round 2280, predicted reward 27.266002,predicted upper bound 27.270840,actual reward 29.371999
    round 2281, predicted reward 32.657229,predicted upper bound 32.661805,actual reward 34.534134
    round 2282, predicted reward 33.833056,predicted upper bound 33.837413,actual reward 35.963901
    round 2283, predicted reward 33.647253,predicted upper bound 33.652353,actual reward 34.698937
    round 2284, predicted reward 29.104949,predicted upper bound 29.109589,actual reward 29.277090
    round 2285, predicted reward 36.485768,predicted upper bound 36.490640,actual reward 38.503271
    round 2286, predicted reward 27.574447,predicted upper bound 27.579773,actual reward 28.843097
    round 2287, predicted reward 21.928903,predicted upper bound 21.934024,actual reward 26.189095
    round 2288, predicted reward 32.135700,predicted upper bound 32.141255,actual reward 28.004429
    round 2289, predicted reward 21.798352,predicted upper bound 21.803967,actual reward 21.698178
    round 2290, predicted reward 31.622063,predicted upper bound 31.627013,actual reward 30.702165
    round 2291, predicted reward 21.179685,predicted upper bound 21.185279,actual reward 19.062517
    round 2292, predicted reward 34.827003,predicted upper bound 34.831751,actual reward 39.717859
    round 2293, predicted reward 27.463589,predicted upper bound 27.468142,actual reward 27.709575
    round 2294, predicted reward 31.559465,predicted upper bound 31.564275,actual reward 33.163105
    round 2295, predicted reward 25.808726,predicted upper bound 25.813848,actual reward 30.434606
    round 2296, predicted reward 22.478784,predicted upper bound 22.484665,actual reward 23.841240
    round 2297, predicted reward 24.632480,predicted upper bound 24.638859,actual reward 26.427468
    round 2298, predicted reward 25.263652,predicted upper bound 25.268903,actual reward 25.257937
    round 2299, predicted reward 31.444551,predicted upper bound 31.449243,actual reward 32.226862
    round 2300, predicted reward 24.102504,predicted upper bound 24.107639,actual reward 23.581415
    round 2301, predicted reward 28.415610,predicted upper bound 28.421233,actual reward 25.871316
    round 2302, predicted reward 30.168722,predicted upper bound 30.174254,actual reward 30.254524
    round 2303, predicted reward 29.232738,predicted upper bound 29.238184,actual reward 28.067769
    round 2304, predicted reward 26.014409,predicted upper bound 26.018864,actual reward 26.266816
    round 2305, predicted reward 28.125013,predicted upper bound 28.129704,actual reward 34.284819
    round 2306, predicted reward 24.367275,predicted upper bound 24.372124,actual reward 21.928616
    round 2307, predicted reward 24.999159,predicted upper bound 25.004599,actual reward 25.191401
    round 2308, predicted reward 27.538920,predicted upper bound 27.544379,actual reward 25.821459
    round 2309, predicted reward 27.994171,predicted upper bound 27.999124,actual reward 30.280317
    round 2310, predicted reward 31.531272,predicted upper bound 31.536746,actual reward 30.550226
    round 2311, predicted reward 31.896890,predicted upper bound 31.901529,actual reward 35.242519
    round 2312, predicted reward 26.289392,predicted upper bound 26.293757,actual reward 28.244926
    round 2313, predicted reward 28.490648,predicted upper bound 28.496136,actual reward 36.111155
    round 2314, predicted reward 30.561663,predicted upper bound 30.566551,actual reward 34.148861
    round 2315, predicted reward 26.990362,predicted upper bound 26.996316,actual reward 26.751144
    round 2316, predicted reward 28.281389,predicted upper bound 28.286563,actual reward 30.705301
    round 2317, predicted reward 26.491878,predicted upper bound 26.496350,actual reward 25.321012
    round 2318, predicted reward 25.920487,predicted upper bound 25.925970,actual reward 21.676670
    round 2319, predicted reward 24.449121,predicted upper bound 24.454050,actual reward 27.248195
    round 2320, predicted reward 24.008175,predicted upper bound 24.013092,actual reward 25.869762
    round 2321, predicted reward 24.187050,predicted upper bound 24.192987,actual reward 24.894467
    round 2322, predicted reward 26.573132,predicted upper bound 26.577602,actual reward 26.981594
    round 2323, predicted reward 26.079124,predicted upper bound 26.085172,actual reward 24.842529
    round 2324, predicted reward 28.536290,predicted upper bound 28.541353,actual reward 29.318443
    round 2325, predicted reward 27.143908,predicted upper bound 27.149022,actual reward 21.676976
    round 2326, predicted reward 27.862268,predicted upper bound 27.867639,actual reward 27.165734
    round 2327, predicted reward 27.785477,predicted upper bound 27.791221,actual reward 25.267799
    round 2328, predicted reward 30.472340,predicted upper bound 30.477927,actual reward 32.806101
    round 2329, predicted reward 28.731482,predicted upper bound 28.736642,actual reward 33.289381
    round 2330, predicted reward 23.372297,predicted upper bound 23.377449,actual reward 22.607912
    round 2331, predicted reward 31.083645,predicted upper bound 31.088603,actual reward 31.196568
    round 2332, predicted reward 25.432627,predicted upper bound 25.437784,actual reward 25.213091
    round 2333, predicted reward 33.884681,predicted upper bound 33.889496,actual reward 34.016772
    round 2334, predicted reward 24.536674,predicted upper bound 24.542630,actual reward 26.140585
    round 2335, predicted reward 20.765788,predicted upper bound 20.771075,actual reward 15.958831
    round 2336, predicted reward 27.525096,predicted upper bound 27.529805,actual reward 24.268194
    round 2337, predicted reward 29.091600,predicted upper bound 29.096958,actual reward 28.159152
    round 2338, predicted reward 25.708195,predicted upper bound 25.713369,actual reward 21.623771
    round 2339, predicted reward 20.672706,predicted upper bound 20.678209,actual reward 20.981657
    round 2340, predicted reward 28.266111,predicted upper bound 28.271313,actual reward 28.201731
    round 2341, predicted reward 27.728090,predicted upper bound 27.732854,actual reward 27.503075
    round 2342, predicted reward 30.650585,predicted upper bound 30.655178,actual reward 32.004544
    round 2343, predicted reward 29.945292,predicted upper bound 29.950251,actual reward 29.881232
    round 2344, predicted reward 27.408940,predicted upper bound 27.414679,actual reward 22.941910
    round 2345, predicted reward 29.846670,predicted upper bound 29.852597,actual reward 32.953079
    round 2346, predicted reward 31.430971,predicted upper bound 31.436110,actual reward 29.282357
    round 2347, predicted reward 27.878308,predicted upper bound 27.883406,actual reward 27.400307
    round 2348, predicted reward 27.051903,predicted upper bound 27.057052,actual reward 26.205096
    round 2349, predicted reward 20.882442,predicted upper bound 20.887760,actual reward 19.208866
    round 2350, predicted reward 30.285619,predicted upper bound 30.290585,actual reward 36.510600
    round 2351, predicted reward 19.077800,predicted upper bound 19.082710,actual reward 18.178139
    round 2352, predicted reward 26.074370,predicted upper bound 26.079033,actual reward 24.898232
    round 2353, predicted reward 30.408744,predicted upper bound 30.413397,actual reward 27.552971
    round 2354, predicted reward 32.096500,predicted upper bound 32.100775,actual reward 36.777233
    round 2355, predicted reward 20.922297,predicted upper bound 20.927350,actual reward 23.779012
    round 2356, predicted reward 29.140125,predicted upper bound 29.144741,actual reward 25.199648
    round 2357, predicted reward 24.139939,predicted upper bound 24.145480,actual reward 20.730014
    round 2358, predicted reward 28.835357,predicted upper bound 28.839647,actual reward 34.632234
    round 2359, predicted reward 27.484020,predicted upper bound 27.488350,actual reward 28.309270
    round 2360, predicted reward 30.813129,predicted upper bound 30.818023,actual reward 31.296234
    round 2361, predicted reward 27.340047,predicted upper bound 27.344246,actual reward 30.243952
    round 2362, predicted reward 28.336293,predicted upper bound 28.341925,actual reward 25.745282
    round 2363, predicted reward 29.359499,predicted upper bound 29.364067,actual reward 30.898251
    round 2364, predicted reward 25.341948,predicted upper bound 25.347443,actual reward 27.633573
    round 2365, predicted reward 23.797233,predicted upper bound 23.802554,actual reward 23.119034
    round 2366, predicted reward 25.537598,predicted upper bound 25.543514,actual reward 14.461822
    round 2367, predicted reward 29.426572,predicted upper bound 29.431173,actual reward 31.936260
    round 2368, predicted reward 25.592221,predicted upper bound 25.597287,actual reward 24.026432
    round 2369, predicted reward 24.872659,predicted upper bound 24.878333,actual reward 20.842100
    round 2370, predicted reward 26.734903,predicted upper bound 26.739590,actual reward 27.216555
    round 2371, predicted reward 28.806629,predicted upper bound 28.811573,actual reward 25.438886
    round 2372, predicted reward 24.833938,predicted upper bound 24.839333,actual reward 19.772468
    round 2373, predicted reward 28.807901,predicted upper bound 28.813244,actual reward 24.341429
    round 2374, predicted reward 23.968354,predicted upper bound 23.973168,actual reward 21.304676
    round 2375, predicted reward 22.566346,predicted upper bound 22.571301,actual reward 17.158877
    round 2376, predicted reward 26.081923,predicted upper bound 26.087528,actual reward 25.440407
    round 2377, predicted reward 25.515008,predicted upper bound 25.520063,actual reward 24.657305
    round 2378, predicted reward 24.729039,predicted upper bound 24.734144,actual reward 22.283388
    round 2379, predicted reward 28.451564,predicted upper bound 28.457601,actual reward 30.526189
    round 2380, predicted reward 21.187386,predicted upper bound 21.192304,actual reward 24.337561
    round 2381, predicted reward 26.802667,predicted upper bound 26.807207,actual reward 25.718763
    round 2382, predicted reward 32.303032,predicted upper bound 32.308131,actual reward 36.380046
    round 2383, predicted reward 36.282473,predicted upper bound 36.286826,actual reward 37.922987
    round 2384, predicted reward 26.709044,predicted upper bound 26.714370,actual reward 23.840195
    round 2385, predicted reward 24.136588,predicted upper bound 24.142049,actual reward 25.377856
    round 2386, predicted reward 24.710658,predicted upper bound 24.715772,actual reward 19.201142
    round 2387, predicted reward 28.023811,predicted upper bound 28.027853,actual reward 28.011980
    round 2388, predicted reward 31.609638,predicted upper bound 31.614271,actual reward 31.914855
    round 2389, predicted reward 29.246378,predicted upper bound 29.251248,actual reward 29.406721
    round 2390, predicted reward 23.742800,predicted upper bound 23.747684,actual reward 26.843282
    round 2391, predicted reward 27.886749,predicted upper bound 27.891362,actual reward 33.843893
    round 2392, predicted reward 31.756546,predicted upper bound 31.760929,actual reward 36.810886
    round 2393, predicted reward 24.964831,predicted upper bound 24.970025,actual reward 27.519584
    round 2394, predicted reward 27.394056,predicted upper bound 27.397906,actual reward 26.875617
    round 2395, predicted reward 30.581373,predicted upper bound 30.586327,actual reward 26.441175
    round 2396, predicted reward 26.663549,predicted upper bound 26.668508,actual reward 22.526665
    round 2397, predicted reward 32.910850,predicted upper bound 32.916265,actual reward 30.567542
    round 2398, predicted reward 24.897677,predicted upper bound 24.902346,actual reward 28.101400
    round 2399, predicted reward 26.664182,predicted upper bound 26.669132,actual reward 20.545341
    round 2400, predicted reward 30.358812,predicted upper bound 30.364175,actual reward 27.907792
    round 2401, predicted reward 29.043024,predicted upper bound 29.047429,actual reward 31.495051
    round 2402, predicted reward 32.073494,predicted upper bound 32.078973,actual reward 33.851957
    round 2403, predicted reward 22.184564,predicted upper bound 22.188974,actual reward 16.391935
    round 2404, predicted reward 31.514978,predicted upper bound 31.519898,actual reward 32.139978
    round 2405, predicted reward 34.926907,predicted upper bound 34.931900,actual reward 36.977831
    round 2406, predicted reward 25.791708,predicted upper bound 25.797144,actual reward 23.966982
    round 2407, predicted reward 29.993268,predicted upper bound 29.997643,actual reward 31.152401
    round 2408, predicted reward 22.175834,predicted upper bound 22.181247,actual reward 22.067148
    round 2409, predicted reward 27.000642,predicted upper bound 27.005333,actual reward 26.750914
    round 2410, predicted reward 22.279600,predicted upper bound 22.284422,actual reward 17.442301
    round 2411, predicted reward 27.797246,predicted upper bound 27.803184,actual reward 29.697218
    round 2412, predicted reward 26.140685,predicted upper bound 26.144702,actual reward 28.084000
    round 2413, predicted reward 31.579992,predicted upper bound 31.584707,actual reward 31.429064
    round 2414, predicted reward 26.460203,predicted upper bound 26.466085,actual reward 26.876914
    round 2415, predicted reward 27.296819,predicted upper bound 27.302735,actual reward 23.746601
    round 2416, predicted reward 22.617859,predicted upper bound 22.623356,actual reward 18.396360
    round 2417, predicted reward 28.953300,predicted upper bound 28.957089,actual reward 32.159530
    round 2418, predicted reward 28.869114,predicted upper bound 28.873413,actual reward 24.129224
    round 2419, predicted reward 26.927039,predicted upper bound 26.931426,actual reward 26.542763
    round 2420, predicted reward 25.776143,predicted upper bound 25.781123,actual reward 23.276508
    round 2421, predicted reward 33.556100,predicted upper bound 33.560615,actual reward 32.833144
    round 2422, predicted reward 25.846870,predicted upper bound 25.851768,actual reward 26.969024
    round 2423, predicted reward 30.340587,predicted upper bound 30.345986,actual reward 26.667743
    round 2424, predicted reward 37.300325,predicted upper bound 37.304658,actual reward 44.416471
    round 2425, predicted reward 36.820114,predicted upper bound 36.824002,actual reward 37.464721
    round 2426, predicted reward 30.262667,predicted upper bound 30.267324,actual reward 30.130428
    round 2427, predicted reward 29.816369,predicted upper bound 29.820577,actual reward 22.596383
    round 2428, predicted reward 21.827112,predicted upper bound 21.832428,actual reward 22.557257
    round 2429, predicted reward 23.761749,predicted upper bound 23.767697,actual reward 26.440309
    round 2430, predicted reward 26.253314,predicted upper bound 26.257690,actual reward 27.530442
    round 2431, predicted reward 33.622289,predicted upper bound 33.626730,actual reward 43.294974
    round 2432, predicted reward 21.002535,predicted upper bound 21.008056,actual reward 17.952133
    round 2433, predicted reward 26.750365,predicted upper bound 26.754968,actual reward 29.096853
    round 2434, predicted reward 31.850006,predicted upper bound 31.854425,actual reward 35.570019
    round 2435, predicted reward 26.962366,predicted upper bound 26.966887,actual reward 23.714121
    round 2436, predicted reward 28.409432,predicted upper bound 28.414207,actual reward 29.689308
    round 2437, predicted reward 28.326609,predicted upper bound 28.331168,actual reward 29.791018
    round 2438, predicted reward 28.062367,predicted upper bound 28.067135,actual reward 25.090330
    round 2439, predicted reward 22.485220,predicted upper bound 22.489657,actual reward 23.552762
    round 2440, predicted reward 25.656775,predicted upper bound 25.662413,actual reward 17.887726
    round 2441, predicted reward 30.234873,predicted upper bound 30.239105,actual reward 27.587210
    round 2442, predicted reward 19.967446,predicted upper bound 19.973767,actual reward 18.826537
    round 2443, predicted reward 31.640440,predicted upper bound 31.644525,actual reward 27.468932
    round 2444, predicted reward 27.451793,predicted upper bound 27.456304,actual reward 27.448633
    round 2445, predicted reward 24.653684,predicted upper bound 24.659755,actual reward 23.633593
    round 2446, predicted reward 30.227168,predicted upper bound 30.231396,actual reward 25.169623
    round 2447, predicted reward 24.173194,predicted upper bound 24.178360,actual reward 24.290342
    round 2448, predicted reward 22.872622,predicted upper bound 22.878385,actual reward 19.786477
    round 2449, predicted reward 34.752436,predicted upper bound 34.756673,actual reward 34.228463
    round 2450, predicted reward 27.068090,predicted upper bound 27.072725,actual reward 30.083812
    round 2451, predicted reward 30.627912,predicted upper bound 30.632300,actual reward 31.210089
    round 2452, predicted reward 26.471972,predicted upper bound 26.477640,actual reward 24.254535
    round 2453, predicted reward 31.309124,predicted upper bound 31.313644,actual reward 26.216860
    round 2454, predicted reward 25.983570,predicted upper bound 25.988603,actual reward 24.696834
    round 2455, predicted reward 18.732092,predicted upper bound 18.737372,actual reward 19.029820
    round 2456, predicted reward 29.607960,predicted upper bound 29.613276,actual reward 23.524955
    round 2457, predicted reward 36.887445,predicted upper bound 36.892080,actual reward 38.900422
    round 2458, predicted reward 24.573404,predicted upper bound 24.578226,actual reward 22.034943
    round 2459, predicted reward 25.018870,predicted upper bound 25.025106,actual reward 25.235994
    round 2460, predicted reward 24.649520,predicted upper bound 24.654762,actual reward 25.194680
    round 2461, predicted reward 23.348723,predicted upper bound 23.354800,actual reward 20.022884
    round 2462, predicted reward 25.493574,predicted upper bound 25.498004,actual reward 27.482186
    round 2463, predicted reward 25.339422,predicted upper bound 25.343528,actual reward 20.970886
    round 2464, predicted reward 25.374052,predicted upper bound 25.379709,actual reward 24.879229
    round 2465, predicted reward 33.093765,predicted upper bound 33.097894,actual reward 31.470030
    round 2466, predicted reward 24.291412,predicted upper bound 24.296510,actual reward 21.082586
    round 2467, predicted reward 26.878900,predicted upper bound 26.883778,actual reward 31.703035
    round 2468, predicted reward 31.543470,predicted upper bound 31.548275,actual reward 37.541953
    round 2469, predicted reward 28.662904,predicted upper bound 28.668190,actual reward 28.209708
    round 2470, predicted reward 26.569512,predicted upper bound 26.573880,actual reward 28.047806
    round 2471, predicted reward 25.824045,predicted upper bound 25.829429,actual reward 21.829914
    round 2472, predicted reward 27.701397,predicted upper bound 27.706480,actual reward 31.659800
    round 2473, predicted reward 25.183392,predicted upper bound 25.188310,actual reward 23.963164
    round 2474, predicted reward 27.606125,predicted upper bound 27.611971,actual reward 27.397266
    round 2475, predicted reward 27.937322,predicted upper bound 27.941895,actual reward 26.495059
    round 2476, predicted reward 25.130948,predicted upper bound 25.136761,actual reward 20.282465
    round 2477, predicted reward 30.520859,predicted upper bound 30.525636,actual reward 34.529036
    round 2478, predicted reward 31.244023,predicted upper bound 31.248902,actual reward 32.245158
    round 2479, predicted reward 32.925251,predicted upper bound 32.929281,actual reward 35.353284
    round 2480, predicted reward 23.859985,predicted upper bound 23.865817,actual reward 24.841040
    round 2481, predicted reward 33.570621,predicted upper bound 33.575603,actual reward 30.898795
    round 2482, predicted reward 32.532971,predicted upper bound 32.537895,actual reward 32.475788
    round 2483, predicted reward 31.055802,predicted upper bound 31.060823,actual reward 33.558426
    round 2484, predicted reward 26.147923,predicted upper bound 26.152112,actual reward 23.200016
    round 2485, predicted reward 36.810759,predicted upper bound 36.814743,actual reward 38.540152
    round 2486, predicted reward 21.604820,predicted upper bound 21.610162,actual reward 24.490223
    round 2487, predicted reward 25.219318,predicted upper bound 25.224911,actual reward 24.620056
    round 2488, predicted reward 27.125697,predicted upper bound 27.130300,actual reward 30.710557
    round 2489, predicted reward 27.084759,predicted upper bound 27.089392,actual reward 30.219493
    round 2490, predicted reward 28.854515,predicted upper bound 28.859928,actual reward 25.445737
    round 2491, predicted reward 28.993356,predicted upper bound 28.998604,actual reward 25.692543
    round 2492, predicted reward 28.091353,predicted upper bound 28.096469,actual reward 26.079483
    round 2493, predicted reward 33.184605,predicted upper bound 33.189300,actual reward 38.868013
    round 2494, predicted reward 25.729288,predicted upper bound 25.733982,actual reward 24.983659
    round 2495, predicted reward 40.800750,predicted upper bound 40.805346,actual reward 47.592909
    round 2496, predicted reward 22.618299,predicted upper bound 22.624208,actual reward 19.843463
    round 2497, predicted reward 28.895762,predicted upper bound 28.901151,actual reward 24.770286
    round 2498, predicted reward 30.595342,predicted upper bound 30.598834,actual reward 32.106047
    round 2499, predicted reward 33.806674,predicted upper bound 33.811466,actual reward 38.578152
    round 2500, predicted reward 24.253882,predicted upper bound 24.259533,actual reward 24.016859
    round 2501, predicted reward 28.078111,predicted upper bound 28.082872,actual reward 27.296606
    round 2502, predicted reward 23.733116,predicted upper bound 23.738806,actual reward 26.594268
    round 2503, predicted reward 24.236417,predicted upper bound 24.241543,actual reward 22.705905
    round 2504, predicted reward 22.528771,predicted upper bound 22.534060,actual reward 23.861394
    round 2505, predicted reward 25.193018,predicted upper bound 25.198019,actual reward 21.234573
    round 2506, predicted reward 26.453088,predicted upper bound 26.458160,actual reward 35.804666
    round 2507, predicted reward 24.073428,predicted upper bound 24.077846,actual reward 20.895739
    round 2508, predicted reward 25.636709,predicted upper bound 25.641371,actual reward 27.883436
    round 2509, predicted reward 21.413791,predicted upper bound 21.419088,actual reward 20.376629
    round 2510, predicted reward 27.351115,predicted upper bound 27.356395,actual reward 24.063230
    round 2511, predicted reward 24.532573,predicted upper bound 24.537438,actual reward 27.036939
    round 2512, predicted reward 17.810765,predicted upper bound 17.815971,actual reward 13.834153
    round 2513, predicted reward 35.211954,predicted upper bound 35.217033,actual reward 38.466517
    round 2514, predicted reward 25.188877,predicted upper bound 25.193981,actual reward 24.290745
    round 2515, predicted reward 24.564759,predicted upper bound 24.569014,actual reward 25.155004
    round 2516, predicted reward 24.603241,predicted upper bound 24.608589,actual reward 22.233530
    round 2517, predicted reward 30.435202,predicted upper bound 30.440651,actual reward 24.824205
    round 2518, predicted reward 22.107986,predicted upper bound 22.112838,actual reward 23.163737
    round 2519, predicted reward 24.529635,predicted upper bound 24.534337,actual reward 27.276526
    round 2520, predicted reward 28.586793,predicted upper bound 28.590908,actual reward 26.766862
    round 2521, predicted reward 30.469109,predicted upper bound 30.474010,actual reward 26.330049
    round 2522, predicted reward 28.036138,predicted upper bound 28.041329,actual reward 29.802105
    round 2523, predicted reward 30.331223,predicted upper bound 30.335408,actual reward 33.464003
    round 2524, predicted reward 27.943702,predicted upper bound 27.949203,actual reward 30.597386
    round 2525, predicted reward 26.085216,predicted upper bound 26.089866,actual reward 28.868242
    round 2526, predicted reward 29.693119,predicted upper bound 29.697618,actual reward 32.339783
    round 2527, predicted reward 29.746808,predicted upper bound 29.751071,actual reward 27.052173
    round 2528, predicted reward 28.793426,predicted upper bound 28.798907,actual reward 25.354358
    round 2529, predicted reward 30.381365,predicted upper bound 30.385891,actual reward 31.394094
    round 2530, predicted reward 25.229183,predicted upper bound 25.234267,actual reward 23.846720
    round 2531, predicted reward 21.701017,predicted upper bound 21.705545,actual reward 21.710962
    round 2532, predicted reward 22.248167,predicted upper bound 22.253095,actual reward 15.612553
    round 2533, predicted reward 23.647428,predicted upper bound 23.651893,actual reward 14.637810
    round 2534, predicted reward 22.902392,predicted upper bound 22.907530,actual reward 18.730173
    round 2535, predicted reward 33.818501,predicted upper bound 33.822856,actual reward 30.650499
    round 2536, predicted reward 24.988749,predicted upper bound 24.993590,actual reward 21.674994
    round 2537, predicted reward 38.465987,predicted upper bound 38.470212,actual reward 44.042618
    round 2538, predicted reward 23.698344,predicted upper bound 23.702911,actual reward 22.250722
    round 2539, predicted reward 28.796229,predicted upper bound 28.800352,actual reward 30.389486
    round 2540, predicted reward 26.614765,predicted upper bound 26.619799,actual reward 22.439718
    round 2541, predicted reward 24.194109,predicted upper bound 24.198726,actual reward 17.699537
    round 2542, predicted reward 30.804752,predicted upper bound 30.809504,actual reward 26.286795
    round 2543, predicted reward 25.160868,predicted upper bound 25.166172,actual reward 24.686466
    round 2544, predicted reward 20.863528,predicted upper bound 20.868865,actual reward 13.355189
    round 2545, predicted reward 28.733493,predicted upper bound 28.738167,actual reward 27.092350
    round 2546, predicted reward 20.334015,predicted upper bound 20.339155,actual reward 17.007320
    round 2547, predicted reward 29.556013,predicted upper bound 29.561145,actual reward 34.225097
    round 2548, predicted reward 30.752878,predicted upper bound 30.757574,actual reward 33.378031
    round 2549, predicted reward 21.601514,predicted upper bound 21.606262,actual reward 23.610175
    round 2550, predicted reward 24.979807,predicted upper bound 24.984361,actual reward 27.149372
    round 2551, predicted reward 30.135727,predicted upper bound 30.140615,actual reward 28.751397
    round 2552, predicted reward 25.727772,predicted upper bound 25.732439,actual reward 24.006265
    round 2553, predicted reward 27.719079,predicted upper bound 27.723193,actual reward 27.636734
    round 2554, predicted reward 24.004731,predicted upper bound 24.009632,actual reward 21.701585
    round 2555, predicted reward 27.947978,predicted upper bound 27.953282,actual reward 26.010064
    round 2556, predicted reward 28.046673,predicted upper bound 28.051483,actual reward 26.436628
    round 2557, predicted reward 23.644250,predicted upper bound 23.649030,actual reward 22.704077
    round 2558, predicted reward 25.612534,predicted upper bound 25.617875,actual reward 24.002118
    round 2559, predicted reward 27.935805,predicted upper bound 27.941876,actual reward 27.903686
    round 2560, predicted reward 30.459351,predicted upper bound 30.463718,actual reward 34.470847
    round 2561, predicted reward 32.199542,predicted upper bound 32.205283,actual reward 35.988244
    round 2562, predicted reward 30.863285,predicted upper bound 30.867118,actual reward 30.724755
    round 2563, predicted reward 22.492382,predicted upper bound 22.496552,actual reward 20.758637
    round 2564, predicted reward 22.959935,predicted upper bound 22.965343,actual reward 19.742711
    round 2565, predicted reward 26.652352,predicted upper bound 26.657234,actual reward 25.407572
    round 2566, predicted reward 30.642105,predicted upper bound 30.646581,actual reward 32.222160
    round 2567, predicted reward 27.298172,predicted upper bound 27.302441,actual reward 27.867636
    round 2568, predicted reward 27.397933,predicted upper bound 27.402408,actual reward 29.020879
    round 2569, predicted reward 24.553434,predicted upper bound 24.558622,actual reward 26.759314
    round 2570, predicted reward 26.994248,predicted upper bound 26.999309,actual reward 23.796600
    round 2571, predicted reward 21.335367,predicted upper bound 21.341250,actual reward 20.149461
    round 2572, predicted reward 24.779654,predicted upper bound 24.784794,actual reward 18.172761
    round 2573, predicted reward 28.508102,predicted upper bound 28.513211,actual reward 26.994282
    round 2574, predicted reward 29.639579,predicted upper bound 29.643641,actual reward 28.571951
    round 2575, predicted reward 25.137425,predicted upper bound 25.142230,actual reward 22.105212
    round 2576, predicted reward 27.965899,predicted upper bound 27.971673,actual reward 26.523364
    round 2577, predicted reward 20.177423,predicted upper bound 20.182932,actual reward 19.781865
    round 2578, predicted reward 30.962676,predicted upper bound 30.967226,actual reward 29.446651
    round 2579, predicted reward 29.754173,predicted upper bound 29.758337,actual reward 35.731790
    round 2580, predicted reward 28.086271,predicted upper bound 28.090937,actual reward 24.734148
    round 2581, predicted reward 31.166754,predicted upper bound 31.171172,actual reward 35.745825
    round 2582, predicted reward 27.215403,predicted upper bound 27.220257,actual reward 22.257462
    round 2583, predicted reward 18.504614,predicted upper bound 18.510672,actual reward 10.489030
    round 2584, predicted reward 27.984870,predicted upper bound 27.989642,actual reward 24.108669
    round 2585, predicted reward 24.413190,predicted upper bound 24.418065,actual reward 26.631261
    round 2586, predicted reward 30.668843,predicted upper bound 30.673658,actual reward 28.755107
    round 2587, predicted reward 24.500987,predicted upper bound 24.505250,actual reward 20.103435
    round 2588, predicted reward 23.789132,predicted upper bound 23.794136,actual reward 22.751700
    round 2589, predicted reward 31.263361,predicted upper bound 31.267523,actual reward 29.974625
    round 2590, predicted reward 31.902309,predicted upper bound 31.907492,actual reward 36.106023
    round 2591, predicted reward 25.859029,predicted upper bound 25.864397,actual reward 28.528148
    round 2592, predicted reward 26.536364,predicted upper bound 26.541836,actual reward 27.160142
    round 2593, predicted reward 26.571913,predicted upper bound 26.577490,actual reward 26.723097
    round 2594, predicted reward 31.689652,predicted upper bound 31.694405,actual reward 33.296341
    round 2595, predicted reward 24.268182,predicted upper bound 24.272289,actual reward 21.627988
    round 2596, predicted reward 22.218693,predicted upper bound 22.223080,actual reward 18.628102
    round 2597, predicted reward 21.944751,predicted upper bound 21.949747,actual reward 18.915978
    round 2598, predicted reward 25.517371,predicted upper bound 25.522137,actual reward 24.700893
    round 2599, predicted reward 20.327520,predicted upper bound 20.332779,actual reward 20.030410
    round 2600, predicted reward 36.545436,predicted upper bound 36.549327,actual reward 36.052319
    round 2601, predicted reward 33.151538,predicted upper bound 33.156582,actual reward 36.289660
    round 2602, predicted reward 23.024568,predicted upper bound 23.029531,actual reward 18.566732
    round 2603, predicted reward 25.507455,predicted upper bound 25.512825,actual reward 24.922886
    round 2604, predicted reward 28.704785,predicted upper bound 28.708773,actual reward 28.908023
    round 2605, predicted reward 30.142979,predicted upper bound 30.148531,actual reward 28.832181
    round 2606, predicted reward 34.593050,predicted upper bound 34.597259,actual reward 39.218401
    round 2607, predicted reward 26.697615,predicted upper bound 26.702582,actual reward 25.487732
    round 2608, predicted reward 35.963037,predicted upper bound 35.966996,actual reward 39.321605
    round 2609, predicted reward 28.997461,predicted upper bound 29.001581,actual reward 26.065699
    round 2610, predicted reward 22.328496,predicted upper bound 22.334099,actual reward 21.913855
    round 2611, predicted reward 27.083213,predicted upper bound 27.088411,actual reward 21.301123
    round 2612, predicted reward 28.293108,predicted upper bound 28.298142,actual reward 27.138608
    round 2613, predicted reward 24.857967,predicted upper bound 24.863553,actual reward 25.926231
    round 2614, predicted reward 20.289094,predicted upper bound 20.295644,actual reward 20.336245
    round 2615, predicted reward 27.012428,predicted upper bound 27.016874,actual reward 25.253953
    round 2616, predicted reward 26.849166,predicted upper bound 26.855026,actual reward 23.700753
    round 2617, predicted reward 26.720730,predicted upper bound 26.726480,actual reward 19.234748
    round 2618, predicted reward 27.677077,predicted upper bound 27.682335,actual reward 22.978272
    round 2619, predicted reward 22.373144,predicted upper bound 22.378881,actual reward 24.263069
    round 2620, predicted reward 25.100208,predicted upper bound 25.105462,actual reward 25.514734
    round 2621, predicted reward 22.925415,predicted upper bound 22.931197,actual reward 21.703179
    round 2622, predicted reward 22.063336,predicted upper bound 22.067957,actual reward 16.173370
    round 2623, predicted reward 29.668328,predicted upper bound 29.672815,actual reward 27.561912
    round 2624, predicted reward 22.067508,predicted upper bound 22.072354,actual reward 18.382751
    round 2625, predicted reward 26.564006,predicted upper bound 26.569095,actual reward 23.840457
    round 2626, predicted reward 23.771059,predicted upper bound 23.776446,actual reward 22.175233
    round 2627, predicted reward 27.882737,predicted upper bound 27.887731,actual reward 25.481096
    round 2628, predicted reward 27.902064,predicted upper bound 27.907290,actual reward 22.849637
    round 2629, predicted reward 27.401342,predicted upper bound 27.406012,actual reward 30.836035
    round 2630, predicted reward 33.303179,predicted upper bound 33.307357,actual reward 40.895819
    round 2631, predicted reward 32.385009,predicted upper bound 32.389368,actual reward 24.991908
    round 2632, predicted reward 27.531441,predicted upper bound 27.536807,actual reward 24.904821
    round 2633, predicted reward 29.539422,predicted upper bound 29.544590,actual reward 29.986919
    round 2634, predicted reward 26.606232,predicted upper bound 26.610593,actual reward 24.233643
    round 2635, predicted reward 23.782571,predicted upper bound 23.787137,actual reward 23.824658
    round 2636, predicted reward 32.431623,predicted upper bound 32.436262,actual reward 28.590216
    round 2637, predicted reward 28.793812,predicted upper bound 28.798232,actual reward 24.537577
    round 2638, predicted reward 23.676847,predicted upper bound 23.681289,actual reward 17.262971
    round 2639, predicted reward 32.054159,predicted upper bound 32.059354,actual reward 25.869101
    round 2640, predicted reward 31.211366,predicted upper bound 31.215964,actual reward 35.062708
    round 2641, predicted reward 29.906581,predicted upper bound 29.911811,actual reward 31.614691
    round 2642, predicted reward 29.874585,predicted upper bound 29.879610,actual reward 26.756074
    round 2643, predicted reward 30.068335,predicted upper bound 30.073568,actual reward 34.312219
    round 2644, predicted reward 30.847336,predicted upper bound 30.851634,actual reward 32.222495
    round 2645, predicted reward 29.152765,predicted upper bound 29.157559,actual reward 26.590320
    round 2646, predicted reward 26.961663,predicted upper bound 26.966023,actual reward 24.650230
    round 2647, predicted reward 23.933759,predicted upper bound 23.938992,actual reward 24.144643
    round 2648, predicted reward 29.932000,predicted upper bound 29.936024,actual reward 30.082479
    round 2649, predicted reward 23.764112,predicted upper bound 23.769771,actual reward 29.072551
    round 2650, predicted reward 30.906616,predicted upper bound 30.912018,actual reward 29.469093
    round 2651, predicted reward 31.855068,predicted upper bound 31.859616,actual reward 31.666546
    round 2652, predicted reward 26.534745,predicted upper bound 26.538958,actual reward 27.447284
    round 2653, predicted reward 31.477772,predicted upper bound 31.482246,actual reward 28.937345
    round 2654, predicted reward 25.602220,predicted upper bound 25.606953,actual reward 26.695887
    round 2655, predicted reward 25.363669,predicted upper bound 25.368626,actual reward 26.422275
    round 2656, predicted reward 24.711147,predicted upper bound 24.715532,actual reward 21.142238
    round 2657, predicted reward 27.417739,predicted upper bound 27.422442,actual reward 28.881441
    round 2658, predicted reward 26.161131,predicted upper bound 26.165379,actual reward 20.738356
    round 2659, predicted reward 24.811447,predicted upper bound 24.816418,actual reward 20.857514
    round 2660, predicted reward 29.222754,predicted upper bound 29.228063,actual reward 36.625052
    round 2661, predicted reward 23.270775,predicted upper bound 23.275495,actual reward 26.312674
    round 2662, predicted reward 28.663850,predicted upper bound 28.668323,actual reward 26.902190
    round 2663, predicted reward 31.993447,predicted upper bound 31.998040,actual reward 28.213099
    round 2664, predicted reward 25.564312,predicted upper bound 25.568698,actual reward 27.188831
    round 2665, predicted reward 28.498435,predicted upper bound 28.502984,actual reward 27.421412
    round 2666, predicted reward 26.895195,predicted upper bound 26.898753,actual reward 30.786628
    round 2667, predicted reward 29.726514,predicted upper bound 29.731183,actual reward 30.421373
    round 2668, predicted reward 28.831716,predicted upper bound 28.836244,actual reward 30.027232
    round 2669, predicted reward 31.423838,predicted upper bound 31.428741,actual reward 33.722877
    round 2670, predicted reward 25.397448,predicted upper bound 25.402224,actual reward 23.773819
    round 2671, predicted reward 28.797668,predicted upper bound 28.802163,actual reward 27.692459
    round 2672, predicted reward 26.240064,predicted upper bound 26.245322,actual reward 30.721947
    round 2673, predicted reward 29.041594,predicted upper bound 29.046151,actual reward 35.886150
    round 2674, predicted reward 27.061162,predicted upper bound 27.066107,actual reward 28.556534
    round 2675, predicted reward 24.458977,predicted upper bound 24.463648,actual reward 21.205852
    round 2676, predicted reward 22.492379,predicted upper bound 22.496572,actual reward 21.142149
    round 2677, predicted reward 23.563446,predicted upper bound 23.569048,actual reward 21.836967
    round 2678, predicted reward 30.087214,predicted upper bound 30.091527,actual reward 35.264549
    round 2679, predicted reward 21.911066,predicted upper bound 21.915839,actual reward 18.915671
    round 2680, predicted reward 18.982189,predicted upper bound 18.986843,actual reward 16.813387
    round 2681, predicted reward 24.565356,predicted upper bound 24.570916,actual reward 22.805098
    round 2682, predicted reward 23.731343,predicted upper bound 23.736436,actual reward 20.567820
    round 2683, predicted reward 38.471647,predicted upper bound 38.476197,actual reward 42.292831
    round 2684, predicted reward 30.540580,predicted upper bound 30.544374,actual reward 32.197983
    round 2685, predicted reward 22.125577,predicted upper bound 22.130324,actual reward 20.029607
    round 2686, predicted reward 35.880341,predicted upper bound 35.884210,actual reward 31.304179
    round 2687, predicted reward 31.133029,predicted upper bound 31.137681,actual reward 32.616525
    round 2688, predicted reward 20.303859,predicted upper bound 20.308385,actual reward 21.842789
    round 2689, predicted reward 23.706394,predicted upper bound 23.711227,actual reward 22.628871
    round 2690, predicted reward 28.239603,predicted upper bound 28.244622,actual reward 27.527144
    round 2691, predicted reward 22.101986,predicted upper bound 22.106740,actual reward 27.437824
    round 2692, predicted reward 31.177207,predicted upper bound 31.180803,actual reward 31.133985
    round 2693, predicted reward 25.078727,predicted upper bound 25.083881,actual reward 22.914725
    round 2694, predicted reward 33.338255,predicted upper bound 33.342787,actual reward 38.619877
    round 2695, predicted reward 28.638644,predicted upper bound 28.642943,actual reward 25.610486
    round 2696, predicted reward 26.337552,predicted upper bound 26.342177,actual reward 25.599230
    round 2697, predicted reward 33.092794,predicted upper bound 33.097474,actual reward 33.533684
    round 2698, predicted reward 30.161536,predicted upper bound 30.166803,actual reward 33.287263
    round 2699, predicted reward 29.325980,predicted upper bound 29.330630,actual reward 25.454793
    round 2700, predicted reward 29.791520,predicted upper bound 29.796871,actual reward 27.957798
    round 2701, predicted reward 32.474189,predicted upper bound 32.479006,actual reward 36.088432
    round 2702, predicted reward 22.997908,predicted upper bound 23.002601,actual reward 24.812218
    round 2703, predicted reward 30.364495,predicted upper bound 30.369147,actual reward 29.585617
    round 2704, predicted reward 28.007042,predicted upper bound 28.012039,actual reward 28.562693
    round 2705, predicted reward 26.908829,predicted upper bound 26.914121,actual reward 19.550858
    round 2706, predicted reward 26.889035,predicted upper bound 26.893165,actual reward 26.177323
    round 2707, predicted reward 29.402538,predicted upper bound 29.407249,actual reward 26.010793
    round 2708, predicted reward 26.291411,predicted upper bound 26.296448,actual reward 23.350149
    round 2709, predicted reward 37.242965,predicted upper bound 37.247340,actual reward 42.337784
    round 2710, predicted reward 23.004023,predicted upper bound 23.008004,actual reward 21.015851
    round 2711, predicted reward 34.055003,predicted upper bound 34.058754,actual reward 31.088121
    round 2712, predicted reward 29.814599,predicted upper bound 29.818762,actual reward 27.659557
    round 2713, predicted reward 28.833045,predicted upper bound 28.837580,actual reward 32.083744
    round 2714, predicted reward 29.813880,predicted upper bound 29.818583,actual reward 25.599467
    round 2715, predicted reward 28.406365,predicted upper bound 28.410244,actual reward 32.203892
    round 2716, predicted reward 27.978600,predicted upper bound 27.983194,actual reward 25.827166
    round 2717, predicted reward 27.997295,predicted upper bound 28.001340,actual reward 24.929524
    round 2718, predicted reward 23.906195,predicted upper bound 23.911544,actual reward 21.160777
    round 2719, predicted reward 26.223303,predicted upper bound 26.227727,actual reward 26.788781
    round 2720, predicted reward 23.569352,predicted upper bound 23.573887,actual reward 15.654229
    round 2721, predicted reward 26.163679,predicted upper bound 26.168390,actual reward 29.781759
    round 2722, predicted reward 21.948186,predicted upper bound 21.953010,actual reward 19.495054
    round 2723, predicted reward 34.050562,predicted upper bound 34.054992,actual reward 32.268413
    round 2724, predicted reward 21.746411,predicted upper bound 21.751851,actual reward 21.595637
    round 2725, predicted reward 28.081946,predicted upper bound 28.085898,actual reward 22.940019
    round 2726, predicted reward 25.900212,predicted upper bound 25.904870,actual reward 25.624477
    round 2727, predicted reward 27.316022,predicted upper bound 27.320632,actual reward 27.715010
    round 2728, predicted reward 26.978596,predicted upper bound 26.984216,actual reward 26.273613
    round 2729, predicted reward 25.101764,predicted upper bound 25.107115,actual reward 24.881641
    round 2730, predicted reward 21.344291,predicted upper bound 21.350081,actual reward 16.826368
    round 2731, predicted reward 20.208504,predicted upper bound 20.213707,actual reward 16.876453
    round 2732, predicted reward 21.554822,predicted upper bound 21.560140,actual reward 17.377447
    round 2733, predicted reward 27.567782,predicted upper bound 27.571945,actual reward 27.239913
    round 2734, predicted reward 25.466877,predicted upper bound 25.472068,actual reward 25.553935
    round 2735, predicted reward 28.111892,predicted upper bound 28.116746,actual reward 25.332294
    round 2736, predicted reward 25.313980,predicted upper bound 25.319566,actual reward 21.785366
    round 2737, predicted reward 24.594262,predicted upper bound 24.598318,actual reward 26.826116
    round 2738, predicted reward 27.131503,predicted upper bound 27.135676,actual reward 29.391143
    round 2739, predicted reward 26.667418,predicted upper bound 26.671866,actual reward 26.887174
    round 2740, predicted reward 28.646322,predicted upper bound 28.651125,actual reward 30.900003
    round 2741, predicted reward 23.176993,predicted upper bound 23.180929,actual reward 20.425008
    round 2742, predicted reward 30.752224,predicted upper bound 30.757041,actual reward 34.329429
    round 2743, predicted reward 28.060368,predicted upper bound 28.065675,actual reward 29.401176
    round 2744, predicted reward 31.276064,predicted upper bound 31.280990,actual reward 30.612738
    round 2745, predicted reward 24.429343,predicted upper bound 24.434181,actual reward 26.811445
    round 2746, predicted reward 25.582382,predicted upper bound 25.587198,actual reward 21.904755
    round 2747, predicted reward 20.209941,predicted upper bound 20.214835,actual reward 14.876561
    round 2748, predicted reward 28.043217,predicted upper bound 28.047926,actual reward 26.554468
    round 2749, predicted reward 25.117576,predicted upper bound 25.122262,actual reward 24.349158
    round 2750, predicted reward 31.044786,predicted upper bound 31.050055,actual reward 28.563233
    round 2751, predicted reward 25.657413,predicted upper bound 25.662271,actual reward 27.758083
    round 2752, predicted reward 34.161850,predicted upper bound 34.166541,actual reward 35.017070
    round 2753, predicted reward 26.791819,predicted upper bound 26.796555,actual reward 32.730373
    round 2754, predicted reward 31.813472,predicted upper bound 31.818423,actual reward 38.627936
    round 2755, predicted reward 27.855560,predicted upper bound 27.860427,actual reward 33.176643
    round 2756, predicted reward 23.643219,predicted upper bound 23.648394,actual reward 21.610852
    round 2757, predicted reward 29.859718,predicted upper bound 29.864515,actual reward 26.832358
    round 2758, predicted reward 25.200219,predicted upper bound 25.205304,actual reward 27.184709
    round 2759, predicted reward 35.927030,predicted upper bound 35.931431,actual reward 39.659427
    round 2760, predicted reward 20.774594,predicted upper bound 20.779482,actual reward 19.352514
    round 2761, predicted reward 24.892919,predicted upper bound 24.897728,actual reward 26.516727
    round 2762, predicted reward 25.351083,predicted upper bound 25.355054,actual reward 22.716795
    round 2763, predicted reward 36.608048,predicted upper bound 36.612316,actual reward 39.830780
    round 2764, predicted reward 29.909566,predicted upper bound 29.914925,actual reward 34.255466
    round 2765, predicted reward 21.255611,predicted upper bound 21.259568,actual reward 24.269989
    round 2766, predicted reward 24.792015,predicted upper bound 24.796143,actual reward 23.152312
    round 2767, predicted reward 32.704216,predicted upper bound 32.708356,actual reward 36.234972
    round 2768, predicted reward 29.637214,predicted upper bound 29.641831,actual reward 30.911317
    round 2769, predicted reward 25.065356,predicted upper bound 25.069375,actual reward 24.561202
    round 2770, predicted reward 26.413025,predicted upper bound 26.418268,actual reward 25.637052
    round 2771, predicted reward 29.012836,predicted upper bound 29.016910,actual reward 28.982490
    round 2772, predicted reward 31.193004,predicted upper bound 31.197399,actual reward 31.284044
    round 2773, predicted reward 26.642241,predicted upper bound 26.646642,actual reward 27.966210
    round 2774, predicted reward 27.027613,predicted upper bound 27.031940,actual reward 23.108832
    round 2775, predicted reward 25.565058,predicted upper bound 25.569507,actual reward 24.905721
    round 2776, predicted reward 31.482365,predicted upper bound 31.486927,actual reward 27.134497
    round 2777, predicted reward 23.109217,predicted upper bound 23.114710,actual reward 21.051438
    round 2778, predicted reward 17.867750,predicted upper bound 17.873438,actual reward 16.122295
    round 2779, predicted reward 26.819270,predicted upper bound 26.824433,actual reward 22.489351
    round 2780, predicted reward 28.109173,predicted upper bound 28.113867,actual reward 24.374329
    round 2781, predicted reward 26.485197,predicted upper bound 26.490072,actual reward 26.012890
    round 2782, predicted reward 30.398360,predicted upper bound 30.402995,actual reward 34.593108
    round 2783, predicted reward 27.599813,predicted upper bound 27.604537,actual reward 28.143747
    round 2784, predicted reward 19.270890,predicted upper bound 19.275734,actual reward 22.125580
    round 2785, predicted reward 24.110168,predicted upper bound 24.114637,actual reward 19.200376
    round 2786, predicted reward 27.733244,predicted upper bound 27.737661,actual reward 30.201300
    round 2787, predicted reward 18.063288,predicted upper bound 18.067671,actual reward 11.990135
    round 2788, predicted reward 30.556977,predicted upper bound 30.561769,actual reward 30.516734
    round 2789, predicted reward 21.705866,predicted upper bound 21.710491,actual reward 20.084216
    round 2790, predicted reward 23.788147,predicted upper bound 23.792534,actual reward 22.709562
    round 2791, predicted reward 27.960363,predicted upper bound 27.964353,actual reward 29.201941
    round 2792, predicted reward 26.017786,predicted upper bound 26.023132,actual reward 24.663331
    round 2793, predicted reward 25.734135,predicted upper bound 25.739216,actual reward 22.596134
    round 2794, predicted reward 23.858928,predicted upper bound 23.863842,actual reward 23.019900
    round 2795, predicted reward 23.180768,predicted upper bound 23.185947,actual reward 20.650919
    round 2796, predicted reward 21.474126,predicted upper bound 21.478824,actual reward 21.754652
    round 2797, predicted reward 28.611148,predicted upper bound 28.616166,actual reward 27.217750
    round 2798, predicted reward 33.458236,predicted upper bound 33.462196,actual reward 31.068243
    round 2799, predicted reward 28.544817,predicted upper bound 28.549551,actual reward 33.986420
    round 2800, predicted reward 31.634926,predicted upper bound 31.638711,actual reward 33.992224
    round 2801, predicted reward 26.886919,predicted upper bound 26.892205,actual reward 31.122447
    round 2802, predicted reward 24.890736,predicted upper bound 24.895596,actual reward 27.950028
    round 2803, predicted reward 29.775705,predicted upper bound 29.780687,actual reward 33.833299
    round 2804, predicted reward 21.175381,predicted upper bound 21.180107,actual reward 14.357170
    round 2805, predicted reward 21.127369,predicted upper bound 21.132126,actual reward 16.683793
    round 2806, predicted reward 33.832222,predicted upper bound 33.836493,actual reward 36.407937
    round 2807, predicted reward 26.934359,predicted upper bound 26.939080,actual reward 25.901916
    round 2808, predicted reward 28.863684,predicted upper bound 28.867122,actual reward 31.697573
    round 2809, predicted reward 32.588228,predicted upper bound 32.591981,actual reward 36.125655
    round 2810, predicted reward 22.977545,predicted upper bound 22.982885,actual reward 21.751474
    round 2811, predicted reward 31.658411,predicted upper bound 31.662717,actual reward 32.915428
    round 2812, predicted reward 27.458898,predicted upper bound 27.463350,actual reward 22.688707
    round 2813, predicted reward 24.574677,predicted upper bound 24.579532,actual reward 23.304822
    round 2814, predicted reward 22.532037,predicted upper bound 22.536921,actual reward 17.933199
    round 2815, predicted reward 27.036564,predicted upper bound 27.041083,actual reward 25.751846
    round 2816, predicted reward 24.994536,predicted upper bound 24.999066,actual reward 22.668013
    round 2817, predicted reward 34.355650,predicted upper bound 34.359299,actual reward 37.227062
    round 2818, predicted reward 27.760582,predicted upper bound 27.764956,actual reward 28.640430
    round 2819, predicted reward 26.712901,predicted upper bound 26.717969,actual reward 25.264147
    round 2820, predicted reward 31.668227,predicted upper bound 31.673298,actual reward 32.257514
    round 2821, predicted reward 24.173725,predicted upper bound 24.178796,actual reward 31.039031
    round 2822, predicted reward 21.476417,predicted upper bound 21.481504,actual reward 22.733572
    round 2823, predicted reward 22.900471,predicted upper bound 22.905257,actual reward 20.140795
    round 2824, predicted reward 33.020242,predicted upper bound 33.024575,actual reward 29.688008
    round 2825, predicted reward 25.237982,predicted upper bound 25.242961,actual reward 21.886607
    round 2826, predicted reward 30.034630,predicted upper bound 30.039131,actual reward 28.834868
    round 2827, predicted reward 26.118280,predicted upper bound 26.123058,actual reward 28.424009
    round 2828, predicted reward 24.760110,predicted upper bound 24.764760,actual reward 25.082563
    round 2829, predicted reward 21.494585,predicted upper bound 21.499346,actual reward 21.397360
    round 2830, predicted reward 32.097848,predicted upper bound 32.102782,actual reward 31.722146
    round 2831, predicted reward 31.315169,predicted upper bound 31.320165,actual reward 36.716752
    round 2832, predicted reward 30.707191,predicted upper bound 30.711493,actual reward 32.679684
    round 2833, predicted reward 31.529326,predicted upper bound 31.534187,actual reward 32.640038
    round 2834, predicted reward 22.146203,predicted upper bound 22.151899,actual reward 17.300943
    round 2835, predicted reward 22.557291,predicted upper bound 22.562359,actual reward 17.544594
    round 2836, predicted reward 26.234024,predicted upper bound 26.238923,actual reward 29.502493
    round 2837, predicted reward 29.055222,predicted upper bound 29.059124,actual reward 30.257000
    round 2838, predicted reward 29.322056,predicted upper bound 29.326999,actual reward 25.555318
    round 2839, predicted reward 26.445022,predicted upper bound 26.450209,actual reward 22.892332
    round 2840, predicted reward 23.722515,predicted upper bound 23.727859,actual reward 24.393460
    round 2841, predicted reward 22.457544,predicted upper bound 22.462010,actual reward 16.771769
    round 2842, predicted reward 27.928836,predicted upper bound 27.933665,actual reward 28.927391
    round 2843, predicted reward 28.111777,predicted upper bound 28.115818,actual reward 26.041538
    round 2844, predicted reward 28.169095,predicted upper bound 28.173418,actual reward 26.457771
    round 2845, predicted reward 23.522184,predicted upper bound 23.527070,actual reward 23.708258
    round 2846, predicted reward 17.804865,predicted upper bound 17.810938,actual reward 15.357513
    round 2847, predicted reward 25.862569,predicted upper bound 25.867522,actual reward 23.060892
    round 2848, predicted reward 24.542198,predicted upper bound 24.546558,actual reward 23.622656
    round 2849, predicted reward 20.114877,predicted upper bound 20.119667,actual reward 13.396448
    round 2850, predicted reward 29.546083,predicted upper bound 29.551618,actual reward 35.078403
    round 2851, predicted reward 24.743757,predicted upper bound 24.749434,actual reward 25.412880
    round 2852, predicted reward 30.122116,predicted upper bound 30.125393,actual reward 30.022071
    round 2853, predicted reward 27.317665,predicted upper bound 27.321975,actual reward 26.042615
    round 2854, predicted reward 28.168856,predicted upper bound 28.173061,actual reward 26.806819
    round 2855, predicted reward 24.975308,predicted upper bound 24.979719,actual reward 22.334079
    round 2856, predicted reward 26.393064,predicted upper bound 26.397354,actual reward 23.913246
    round 2857, predicted reward 26.528247,predicted upper bound 26.533085,actual reward 29.513282
    round 2858, predicted reward 23.843058,predicted upper bound 23.848810,actual reward 16.302448
    round 2859, predicted reward 26.842460,predicted upper bound 26.846156,actual reward 25.445356
    round 2860, predicted reward 28.394344,predicted upper bound 28.398338,actual reward 24.828768
    round 2861, predicted reward 23.450916,predicted upper bound 23.455941,actual reward 17.816336
    round 2862, predicted reward 27.057002,predicted upper bound 27.062371,actual reward 25.325452
    round 2863, predicted reward 25.959391,predicted upper bound 25.964216,actual reward 23.331942
    round 2864, predicted reward 31.562426,predicted upper bound 31.567493,actual reward 24.786517
    round 2865, predicted reward 36.267649,predicted upper bound 36.271406,actual reward 39.345479
    round 2866, predicted reward 28.443783,predicted upper bound 28.447923,actual reward 26.286624
    round 2867, predicted reward 24.476853,predicted upper bound 24.482616,actual reward 20.483394
    round 2868, predicted reward 25.087112,predicted upper bound 25.091220,actual reward 20.835856
    round 2869, predicted reward 23.690482,predicted upper bound 23.695315,actual reward 19.959559
    round 2870, predicted reward 28.211253,predicted upper bound 28.216015,actual reward 22.319389
    round 2871, predicted reward 27.587403,predicted upper bound 27.591721,actual reward 29.494119
    round 2872, predicted reward 35.263110,predicted upper bound 35.266869,actual reward 34.391687
    round 2873, predicted reward 30.310071,predicted upper bound 30.313929,actual reward 30.627737
    round 2874, predicted reward 26.095459,predicted upper bound 26.100654,actual reward 27.769623
    round 2875, predicted reward 29.167345,predicted upper bound 29.171793,actual reward 27.258356
    round 2876, predicted reward 36.593638,predicted upper bound 36.597320,actual reward 36.788237
    round 2877, predicted reward 27.036632,predicted upper bound 27.041781,actual reward 23.178718
    round 2878, predicted reward 21.091891,predicted upper bound 21.097482,actual reward 18.706842
    round 2879, predicted reward 27.844121,predicted upper bound 27.847787,actual reward 31.913172
    round 2880, predicted reward 28.599216,predicted upper bound 28.603419,actual reward 27.976608
    round 2881, predicted reward 34.492682,predicted upper bound 34.497053,actual reward 32.491211
    round 2882, predicted reward 26.382135,predicted upper bound 26.386663,actual reward 23.470136
    round 2883, predicted reward 32.287800,predicted upper bound 32.291832,actual reward 33.635128
    round 2884, predicted reward 28.714958,predicted upper bound 28.719957,actual reward 19.952724
    round 2885, predicted reward 26.685841,predicted upper bound 26.690276,actual reward 24.529257
    round 2886, predicted reward 25.718751,predicted upper bound 25.723456,actual reward 24.118323
    round 2887, predicted reward 24.009607,predicted upper bound 24.014412,actual reward 21.754810
    round 2888, predicted reward 23.973302,predicted upper bound 23.977271,actual reward 19.239937
    round 2889, predicted reward 25.708290,predicted upper bound 25.713032,actual reward 24.407866
    round 2890, predicted reward 31.240410,predicted upper bound 31.244322,actual reward 33.549594
    round 2891, predicted reward 28.994121,predicted upper bound 28.998465,actual reward 29.672874
    round 2892, predicted reward 23.494610,predicted upper bound 23.498735,actual reward 21.122252
    round 2893, predicted reward 23.505653,predicted upper bound 23.511084,actual reward 15.940907
    round 2894, predicted reward 29.749536,predicted upper bound 29.754161,actual reward 34.585025
    round 2895, predicted reward 24.091593,predicted upper bound 24.095385,actual reward 23.344410
    round 2896, predicted reward 27.332085,predicted upper bound 27.335608,actual reward 24.602304
    round 2897, predicted reward 31.225823,predicted upper bound 31.229622,actual reward 35.935521
    round 2898, predicted reward 32.087053,predicted upper bound 32.091515,actual reward 35.848472
    round 2899, predicted reward 30.626606,predicted upper bound 30.631190,actual reward 27.811409
    round 2900, predicted reward 25.073372,predicted upper bound 25.078139,actual reward 23.557636
    round 2901, predicted reward 27.347361,predicted upper bound 27.351277,actual reward 26.230040
    round 2902, predicted reward 30.533934,predicted upper bound 30.538056,actual reward 29.804215
    round 2903, predicted reward 34.013212,predicted upper bound 34.018127,actual reward 37.293216
    round 2904, predicted reward 26.664268,predicted upper bound 26.668712,actual reward 19.844339
    round 2905, predicted reward 31.918356,predicted upper bound 31.923032,actual reward 30.175260
    round 2906, predicted reward 22.752071,predicted upper bound 22.755884,actual reward 22.755584
    round 2907, predicted reward 26.800583,predicted upper bound 26.805617,actual reward 22.179505
    round 2908, predicted reward 20.474572,predicted upper bound 20.479040,actual reward 15.102695
    round 2909, predicted reward 29.321412,predicted upper bound 29.325378,actual reward 24.059393
    round 2910, predicted reward 30.528456,predicted upper bound 30.532656,actual reward 33.402459
    round 2911, predicted reward 29.943094,predicted upper bound 29.947074,actual reward 26.323621
    round 2912, predicted reward 31.407870,predicted upper bound 31.412084,actual reward 29.439116
    round 2913, predicted reward 25.566506,predicted upper bound 25.570846,actual reward 24.597302
    round 2914, predicted reward 39.942104,predicted upper bound 39.945853,actual reward 41.441456
    round 2915, predicted reward 26.782538,predicted upper bound 26.786788,actual reward 27.316246
    round 2916, predicted reward 18.120359,predicted upper bound 18.124840,actual reward 24.037520
    round 2917, predicted reward 24.091535,predicted upper bound 24.096521,actual reward 26.045932
    round 2918, predicted reward 30.254883,predicted upper bound 30.259615,actual reward 33.676840
    round 2919, predicted reward 24.329253,predicted upper bound 24.335016,actual reward 24.851776
    round 2920, predicted reward 19.383848,predicted upper bound 19.389501,actual reward 17.886614
    round 2921, predicted reward 17.072692,predicted upper bound 17.077110,actual reward 16.464426
    round 2922, predicted reward 35.435345,predicted upper bound 35.439676,actual reward 36.796330
    round 2923, predicted reward 30.160344,predicted upper bound 30.164044,actual reward 26.251252
    round 2924, predicted reward 28.912982,predicted upper bound 28.917275,actual reward 34.326467
    round 2925, predicted reward 36.072186,predicted upper bound 36.075896,actual reward 36.866711
    round 2926, predicted reward 27.301518,predicted upper bound 27.305855,actual reward 32.656322
    round 2927, predicted reward 32.509296,predicted upper bound 32.513771,actual reward 32.152662
    round 2928, predicted reward 35.078065,predicted upper bound 35.081792,actual reward 32.450805
    round 2929, predicted reward 24.091769,predicted upper bound 24.095848,actual reward 21.051535
    round 2930, predicted reward 34.726053,predicted upper bound 34.730021,actual reward 37.984468
    round 2931, predicted reward 32.913120,predicted upper bound 32.918247,actual reward 31.566026
    round 2932, predicted reward 24.677516,predicted upper bound 24.681203,actual reward 24.217303
    round 2933, predicted reward 28.054988,predicted upper bound 28.059385,actual reward 29.948051
    round 2934, predicted reward 26.494186,predicted upper bound 26.499412,actual reward 24.587445
    round 2935, predicted reward 28.624946,predicted upper bound 28.629691,actual reward 30.071323
    round 2936, predicted reward 27.454443,predicted upper bound 27.458448,actual reward 23.494553
    round 2937, predicted reward 24.469379,predicted upper bound 24.474731,actual reward 25.677063
    round 2938, predicted reward 31.209632,predicted upper bound 31.214392,actual reward 33.290325
    round 2939, predicted reward 26.264862,predicted upper bound 26.269519,actual reward 20.193382
    round 2940, predicted reward 29.368467,predicted upper bound 29.372575,actual reward 22.532865
    round 2941, predicted reward 28.545799,predicted upper bound 28.549600,actual reward 23.804118
    round 2942, predicted reward 20.902750,predicted upper bound 20.907737,actual reward 15.200150
    round 2943, predicted reward 22.359268,predicted upper bound 22.363715,actual reward 15.880508
    round 2944, predicted reward 31.761630,predicted upper bound 31.765656,actual reward 26.638859
    round 2945, predicted reward 26.306831,predicted upper bound 26.311852,actual reward 25.805303
    round 2946, predicted reward 21.264495,predicted upper bound 21.270210,actual reward 17.917857
    round 2947, predicted reward 24.378660,predicted upper bound 24.383636,actual reward 25.028442
    round 2948, predicted reward 29.020616,predicted upper bound 29.024531,actual reward 22.831482
    round 2949, predicted reward 26.300782,predicted upper bound 26.305201,actual reward 25.807163
    round 2950, predicted reward 29.224024,predicted upper bound 29.227825,actual reward 25.123102
    round 2951, predicted reward 30.953545,predicted upper bound 30.958625,actual reward 37.950904
    round 2952, predicted reward 28.696203,predicted upper bound 28.700880,actual reward 32.348934
    round 2953, predicted reward 20.351161,predicted upper bound 20.355518,actual reward 19.412383
    round 2954, predicted reward 24.776843,predicted upper bound 24.781622,actual reward 22.774817
    round 2955, predicted reward 25.162108,predicted upper bound 25.167583,actual reward 23.318851
    round 2956, predicted reward 28.432217,predicted upper bound 28.436331,actual reward 27.481203
    round 2957, predicted reward 26.329301,predicted upper bound 26.333499,actual reward 27.306210
    round 2958, predicted reward 26.259694,predicted upper bound 26.264269,actual reward 24.565900
    round 2959, predicted reward 27.522133,predicted upper bound 27.526274,actual reward 25.018864
    round 2960, predicted reward 26.637751,predicted upper bound 26.641705,actual reward 23.774601
    round 2961, predicted reward 24.965509,predicted upper bound 24.970689,actual reward 26.493128
    round 2962, predicted reward 31.774004,predicted upper bound 31.778009,actual reward 32.023906
    round 2963, predicted reward 28.240724,predicted upper bound 28.245435,actual reward 24.372166
    round 2964, predicted reward 26.630929,predicted upper bound 26.635504,actual reward 26.479222
    round 2965, predicted reward 29.732206,predicted upper bound 29.737374,actual reward 28.936756
    round 2966, predicted reward 24.826428,predicted upper bound 24.830567,actual reward 26.630429
    round 2967, predicted reward 25.237821,predicted upper bound 25.241692,actual reward 27.440667
    round 2968, predicted reward 29.795278,predicted upper bound 29.799862,actual reward 30.535154
    round 2969, predicted reward 24.008947,predicted upper bound 24.014722,actual reward 22.952686
    round 2970, predicted reward 26.305005,predicted upper bound 26.308543,actual reward 24.234748
    round 2971, predicted reward 34.133735,predicted upper bound 34.138565,actual reward 30.546285
    round 2972, predicted reward 33.846759,predicted upper bound 33.851719,actual reward 30.540607
    round 2973, predicted reward 27.506569,predicted upper bound 27.511042,actual reward 30.860192
    round 2974, predicted reward 28.366929,predicted upper bound 28.370678,actual reward 25.548922
    round 2975, predicted reward 29.247026,predicted upper bound 29.252323,actual reward 27.075758
    round 2976, predicted reward 27.618810,predicted upper bound 27.623358,actual reward 20.789867
    round 2977, predicted reward 25.294949,predicted upper bound 25.299643,actual reward 22.586138
    round 2978, predicted reward 25.906472,predicted upper bound 25.910545,actual reward 21.029298
    round 2979, predicted reward 33.438591,predicted upper bound 33.442222,actual reward 36.697295
    round 2980, predicted reward 28.608740,predicted upper bound 28.613921,actual reward 25.624605
    round 2981, predicted reward 34.166406,predicted upper bound 34.170595,actual reward 33.992449
    round 2982, predicted reward 21.978893,predicted upper bound 21.983526,actual reward 26.094753
    round 2983, predicted reward 21.555774,predicted upper bound 21.560558,actual reward 23.255845
    round 2984, predicted reward 30.276611,predicted upper bound 30.281199,actual reward 32.678933
    round 2985, predicted reward 26.694538,predicted upper bound 26.698515,actual reward 23.137408
    round 2986, predicted reward 31.399216,predicted upper bound 31.403829,actual reward 26.347415
    round 2987, predicted reward 26.490061,predicted upper bound 26.494174,actual reward 25.923404
    round 2988, predicted reward 24.307756,predicted upper bound 24.312881,actual reward 23.459378
    round 2989, predicted reward 23.164376,predicted upper bound 23.168541,actual reward 23.961106
    round 2990, predicted reward 23.457789,predicted upper bound 23.462355,actual reward 23.545262
    round 2991, predicted reward 21.285477,predicted upper bound 21.290960,actual reward 22.307924
    round 2992, predicted reward 26.152724,predicted upper bound 26.157202,actual reward 26.333400
    round 2993, predicted reward 33.925143,predicted upper bound 33.929595,actual reward 35.863799
    round 2994, predicted reward 34.490666,predicted upper bound 34.494505,actual reward 35.778626
    round 2995, predicted reward 26.019727,predicted upper bound 26.023231,actual reward 20.159978
    round 2996, predicted reward 22.331376,predicted upper bound 22.335622,actual reward 23.050716
    round 2997, predicted reward 22.113069,predicted upper bound 22.117848,actual reward 20.528599
    round 2998, predicted reward 26.410667,predicted upper bound 26.414928,actual reward 28.459654
    round 2999, predicted reward 33.059224,predicted upper bound 33.063682,actual reward 35.335862
    round 3000, predicted reward 26.584138,predicted upper bound 26.588738,actual reward 22.877156
    round 3001, predicted reward 27.823665,predicted upper bound 27.828146,actual reward 29.425366
    round 3002, predicted reward 27.581346,predicted upper bound 27.586250,actual reward 23.873646
    round 3003, predicted reward 26.127431,predicted upper bound 26.132413,actual reward 20.632919
    round 3004, predicted reward 22.611606,predicted upper bound 22.616986,actual reward 18.342580
    round 3005, predicted reward 27.568130,predicted upper bound 27.572916,actual reward 25.465469
    round 3006, predicted reward 23.632251,predicted upper bound 23.637103,actual reward 20.880953
    round 3007, predicted reward 22.948651,predicted upper bound 22.953887,actual reward 19.279664
    round 3008, predicted reward 27.489683,predicted upper bound 27.494921,actual reward 31.584824
    round 3009, predicted reward 26.761731,predicted upper bound 26.766891,actual reward 24.752428
    round 3010, predicted reward 22.852365,predicted upper bound 22.857245,actual reward 17.341675
    round 3011, predicted reward 21.767178,predicted upper bound 21.771475,actual reward 19.329089
    round 3012, predicted reward 29.244661,predicted upper bound 29.249314,actual reward 28.487232
    round 3013, predicted reward 30.196619,predicted upper bound 30.200933,actual reward 29.395742
    round 3014, predicted reward 28.894745,predicted upper bound 28.899213,actual reward 26.330995
    round 3015, predicted reward 27.798620,predicted upper bound 27.802697,actual reward 26.908793
    round 3016, predicted reward 24.678873,predicted upper bound 24.684421,actual reward 21.928815
    round 3017, predicted reward 28.192052,predicted upper bound 28.196102,actual reward 27.717078
    round 3018, predicted reward 38.899521,predicted upper bound 38.903526,actual reward 43.114104
    round 3019, predicted reward 18.319535,predicted upper bound 18.324123,actual reward 16.058969
    round 3020, predicted reward 33.122365,predicted upper bound 33.126591,actual reward 36.886621
    round 3021, predicted reward 23.818185,predicted upper bound 23.822753,actual reward 27.787147
    round 3022, predicted reward 33.996929,predicted upper bound 34.001513,actual reward 36.424257
    round 3023, predicted reward 26.702205,predicted upper bound 26.707187,actual reward 29.980900
    round 3024, predicted reward 25.170946,predicted upper bound 25.176088,actual reward 21.736487
    round 3025, predicted reward 23.175565,predicted upper bound 23.180529,actual reward 21.619604
    round 3026, predicted reward 30.028186,predicted upper bound 30.032824,actual reward 31.065413
    round 3027, predicted reward 34.436878,predicted upper bound 34.441495,actual reward 31.855175
    round 3028, predicted reward 31.183472,predicted upper bound 31.188191,actual reward 30.250520
    round 3029, predicted reward 27.613939,predicted upper bound 27.618673,actual reward 27.028803
    round 3030, predicted reward 27.450883,predicted upper bound 27.455450,actual reward 22.145583
    round 3031, predicted reward 24.811291,predicted upper bound 24.815408,actual reward 21.611099
    round 3032, predicted reward 29.219917,predicted upper bound 29.224654,actual reward 29.178853
    round 3033, predicted reward 26.429856,predicted upper bound 26.434322,actual reward 23.317203
    round 3034, predicted reward 25.712547,predicted upper bound 25.717857,actual reward 22.045078
    round 3035, predicted reward 36.966323,predicted upper bound 36.971285,actual reward 36.634119
    round 3036, predicted reward 26.118658,predicted upper bound 26.123340,actual reward 26.220477
    round 3037, predicted reward 25.749308,predicted upper bound 25.753381,actual reward 20.709431
    round 3038, predicted reward 29.935501,predicted upper bound 29.939846,actual reward 30.197246
    round 3039, predicted reward 38.831816,predicted upper bound 38.835534,actual reward 37.844975
    round 3040, predicted reward 27.340903,predicted upper bound 27.345281,actual reward 23.150565
    round 3041, predicted reward 31.187183,predicted upper bound 31.191124,actual reward 26.649688
    round 3042, predicted reward 25.071540,predicted upper bound 25.076374,actual reward 22.596667
    round 3043, predicted reward 29.355025,predicted upper bound 29.360053,actual reward 27.288762
    round 3044, predicted reward 26.027289,predicted upper bound 26.032082,actual reward 25.500269
    round 3045, predicted reward 25.143461,predicted upper bound 25.148406,actual reward 20.204617
    round 3046, predicted reward 25.547905,predicted upper bound 25.552284,actual reward 22.120848
    round 3047, predicted reward 35.101322,predicted upper bound 35.105091,actual reward 32.314279
    round 3048, predicted reward 21.704489,predicted upper bound 21.709631,actual reward 23.073429
    round 3049, predicted reward 30.584753,predicted upper bound 30.589612,actual reward 21.683043
    round 3050, predicted reward 24.500876,predicted upper bound 24.506020,actual reward 20.713206
    round 3051, predicted reward 22.051902,predicted upper bound 22.056648,actual reward 17.981132
    round 3052, predicted reward 22.931717,predicted upper bound 22.935663,actual reward 22.516802
    round 3053, predicted reward 36.296513,predicted upper bound 36.300819,actual reward 38.258994
    round 3054, predicted reward 29.370874,predicted upper bound 29.375197,actual reward 26.894507
    round 3055, predicted reward 29.293281,predicted upper bound 29.297173,actual reward 24.664150
    round 3056, predicted reward 31.085929,predicted upper bound 31.090355,actual reward 31.365390
    round 3057, predicted reward 25.837113,predicted upper bound 25.840603,actual reward 25.900379
    round 3058, predicted reward 28.534224,predicted upper bound 28.539261,actual reward 29.725004
    round 3059, predicted reward 34.768222,predicted upper bound 34.771710,actual reward 38.140418
    round 3060, predicted reward 23.377708,predicted upper bound 23.381908,actual reward 25.887609
    round 3061, predicted reward 26.087404,predicted upper bound 26.091513,actual reward 26.118322
    round 3062, predicted reward 26.002607,predicted upper bound 26.006641,actual reward 22.199678
    round 3063, predicted reward 25.424905,predicted upper bound 25.429360,actual reward 17.974490
    round 3064, predicted reward 28.019030,predicted upper bound 28.023467,actual reward 27.917635
    round 3065, predicted reward 25.749053,predicted upper bound 25.753514,actual reward 23.527320
    round 3066, predicted reward 30.689802,predicted upper bound 30.694530,actual reward 30.699659
    round 3067, predicted reward 35.414709,predicted upper bound 35.418540,actual reward 37.173252
    round 3068, predicted reward 23.602159,predicted upper bound 23.606403,actual reward 24.715882
    round 3069, predicted reward 27.701551,predicted upper bound 27.705700,actual reward 28.842257
    round 3070, predicted reward 22.961074,predicted upper bound 22.964870,actual reward 21.731692
    round 3071, predicted reward 26.466837,predicted upper bound 26.470547,actual reward 26.795425
    round 3072, predicted reward 27.408242,predicted upper bound 27.411676,actual reward 23.087063
    round 3073, predicted reward 28.966651,predicted upper bound 28.970704,actual reward 34.836574
    round 3074, predicted reward 31.994933,predicted upper bound 31.999946,actual reward 28.879590
    round 3075, predicted reward 27.283080,predicted upper bound 27.286492,actual reward 25.292163
    round 3076, predicted reward 23.463709,predicted upper bound 23.467737,actual reward 20.549764
    round 3077, predicted reward 29.820342,predicted upper bound 29.824829,actual reward 25.095972
    round 3078, predicted reward 25.671903,predicted upper bound 25.676620,actual reward 22.576746
    round 3079, predicted reward 22.310513,predicted upper bound 22.315404,actual reward 25.635011
    round 3080, predicted reward 27.167671,predicted upper bound 27.172423,actual reward 26.901029
    round 3081, predicted reward 28.330559,predicted upper bound 28.334134,actual reward 31.978793
    round 3082, predicted reward 23.603543,predicted upper bound 23.607581,actual reward 23.773779
    round 3083, predicted reward 23.502949,predicted upper bound 23.508124,actual reward 23.690657
    round 3084, predicted reward 30.368291,predicted upper bound 30.371902,actual reward 32.292753
    round 3085, predicted reward 26.873077,predicted upper bound 26.876394,actual reward 27.156251
    round 3086, predicted reward 24.062160,predicted upper bound 24.066100,actual reward 19.733419
    round 3087, predicted reward 29.955299,predicted upper bound 29.960063,actual reward 33.947655
    round 3088, predicted reward 27.638396,predicted upper bound 27.643047,actual reward 28.020743
    round 3089, predicted reward 32.669674,predicted upper bound 32.673947,actual reward 29.827524
    round 3090, predicted reward 24.013207,predicted upper bound 24.018528,actual reward 22.381993
    round 3091, predicted reward 18.807787,predicted upper bound 18.812859,actual reward 14.150165
    round 3092, predicted reward 29.505285,predicted upper bound 29.509042,actual reward 30.291814
    round 3093, predicted reward 18.684655,predicted upper bound 18.689966,actual reward 12.439224
    round 3094, predicted reward 34.380556,predicted upper bound 34.384415,actual reward 38.341713
    round 3095, predicted reward 19.747208,predicted upper bound 19.751212,actual reward 14.545345
    round 3096, predicted reward 20.415985,predicted upper bound 20.420834,actual reward 19.843274
    round 3097, predicted reward 31.987334,predicted upper bound 31.992139,actual reward 33.717124
    round 3098, predicted reward 21.990613,predicted upper bound 21.996142,actual reward 23.867944
    round 3099, predicted reward 25.703835,predicted upper bound 25.707766,actual reward 27.238941
    round 3100, predicted reward 26.082023,predicted upper bound 26.086499,actual reward 25.491597
    round 3101, predicted reward 27.493266,predicted upper bound 27.497816,actual reward 29.811483
    round 3102, predicted reward 26.092851,predicted upper bound 26.097910,actual reward 24.477709
    round 3103, predicted reward 33.918516,predicted upper bound 33.923188,actual reward 36.579294
    round 3104, predicted reward 25.267694,predicted upper bound 25.271906,actual reward 24.407710
    round 3105, predicted reward 28.569760,predicted upper bound 28.574083,actual reward 29.966520
    round 3106, predicted reward 29.092066,predicted upper bound 29.095774,actual reward 30.157840
    round 3107, predicted reward 26.200211,predicted upper bound 26.204367,actual reward 17.094366
    round 3108, predicted reward 32.029883,predicted upper bound 32.033775,actual reward 33.564356
    round 3109, predicted reward 31.258276,predicted upper bound 31.262388,actual reward 33.934262
    round 3110, predicted reward 27.147991,predicted upper bound 27.151910,actual reward 23.781610
    round 3111, predicted reward 23.540414,predicted upper bound 23.544367,actual reward 19.120954
    round 3112, predicted reward 28.157845,predicted upper bound 28.162596,actual reward 29.519215
    round 3113, predicted reward 33.722523,predicted upper bound 33.726688,actual reward 31.468197
    round 3114, predicted reward 30.429713,predicted upper bound 30.433858,actual reward 30.684098
    round 3115, predicted reward 26.249127,predicted upper bound 26.253334,actual reward 26.456259
    round 3116, predicted reward 24.800959,predicted upper bound 24.804900,actual reward 22.300443
    round 3117, predicted reward 30.114677,predicted upper bound 30.118408,actual reward 27.333123
    round 3118, predicted reward 23.827753,predicted upper bound 23.832189,actual reward 24.795096
    round 3119, predicted reward 27.029763,predicted upper bound 27.033525,actual reward 23.044764
    round 3120, predicted reward 32.777892,predicted upper bound 32.782026,actual reward 38.291620
    round 3121, predicted reward 23.567154,predicted upper bound 23.571833,actual reward 22.509173
    round 3122, predicted reward 26.547921,predicted upper bound 26.553023,actual reward 25.241557
    round 3123, predicted reward 30.008131,predicted upper bound 30.012768,actual reward 26.902279
    round 3124, predicted reward 23.621221,predicted upper bound 23.626028,actual reward 20.744409
    round 3125, predicted reward 25.176224,predicted upper bound 25.181136,actual reward 27.774379
    round 3126, predicted reward 23.325693,predicted upper bound 23.329843,actual reward 22.413794
    round 3127, predicted reward 23.264187,predicted upper bound 23.268300,actual reward 19.902533
    round 3128, predicted reward 22.609113,predicted upper bound 22.613576,actual reward 25.740806
    round 3129, predicted reward 30.634860,predicted upper bound 30.639797,actual reward 31.814102
    round 3130, predicted reward 29.056656,predicted upper bound 29.061021,actual reward 27.747539
    round 3131, predicted reward 29.534902,predicted upper bound 29.539071,actual reward 32.730852
    round 3132, predicted reward 29.351114,predicted upper bound 29.355240,actual reward 29.527078
    round 3133, predicted reward 27.551626,predicted upper bound 27.555628,actual reward 32.499306
    round 3134, predicted reward 27.512813,predicted upper bound 27.517301,actual reward 29.127066
    round 3135, predicted reward 26.297221,predicted upper bound 26.301571,actual reward 21.591692
    round 3136, predicted reward 23.958168,predicted upper bound 23.962697,actual reward 21.643824
    round 3137, predicted reward 25.025508,predicted upper bound 25.029929,actual reward 20.589293
    round 3138, predicted reward 27.879081,predicted upper bound 27.883548,actual reward 28.014975
    round 3139, predicted reward 22.149466,predicted upper bound 22.153812,actual reward 18.089903
    round 3140, predicted reward 21.307042,predicted upper bound 21.311043,actual reward 17.869329
    round 3141, predicted reward 29.532907,predicted upper bound 29.536746,actual reward 26.047725
    round 3142, predicted reward 32.784781,predicted upper bound 32.789535,actual reward 29.618860
    round 3143, predicted reward 25.532183,predicted upper bound 25.535772,actual reward 26.633662
    round 3144, predicted reward 24.365550,predicted upper bound 24.370273,actual reward 25.681202
    round 3145, predicted reward 27.254016,predicted upper bound 27.258422,actual reward 19.630087
    round 3146, predicted reward 19.810685,predicted upper bound 19.814341,actual reward 14.331284
    round 3147, predicted reward 25.746692,predicted upper bound 25.750731,actual reward 21.853804
    round 3148, predicted reward 24.381274,predicted upper bound 24.385791,actual reward 24.110438
    round 3149, predicted reward 28.383166,predicted upper bound 28.387121,actual reward 26.065705
    round 3150, predicted reward 30.373775,predicted upper bound 30.378228,actual reward 31.372070
    round 3151, predicted reward 28.489441,predicted upper bound 28.494673,actual reward 28.932657
    round 3152, predicted reward 27.503939,predicted upper bound 27.508571,actual reward 25.991721
    round 3153, predicted reward 31.202800,predicted upper bound 31.206896,actual reward 35.166941
    round 3154, predicted reward 24.703537,predicted upper bound 24.707920,actual reward 22.284598
    round 3155, predicted reward 26.978181,predicted upper bound 26.981779,actual reward 28.192964
    round 3156, predicted reward 31.909934,predicted upper bound 31.913655,actual reward 28.148737
    round 3157, predicted reward 31.348515,predicted upper bound 31.352098,actual reward 31.959908
    round 3158, predicted reward 30.985458,predicted upper bound 30.989002,actual reward 29.268130
    round 3159, predicted reward 28.266735,predicted upper bound 28.270621,actual reward 26.932569
    round 3160, predicted reward 27.342604,predicted upper bound 27.346799,actual reward 27.000205
    round 3161, predicted reward 30.063224,predicted upper bound 30.066900,actual reward 33.628811
    round 3162, predicted reward 27.034434,predicted upper bound 27.039517,actual reward 27.017498
    round 3163, predicted reward 27.382997,predicted upper bound 27.387168,actual reward 26.376154
    round 3164, predicted reward 24.324551,predicted upper bound 24.329556,actual reward 24.148449
    round 3165, predicted reward 26.726381,predicted upper bound 26.730751,actual reward 26.786376
    round 3166, predicted reward 20.801655,predicted upper bound 20.805447,actual reward 16.457173
    round 3167, predicted reward 32.969204,predicted upper bound 32.973127,actual reward 34.738640
    round 3168, predicted reward 29.677357,predicted upper bound 29.681187,actual reward 30.525264
    round 3169, predicted reward 23.193833,predicted upper bound 23.197404,actual reward 20.988824
    round 3170, predicted reward 25.677918,predicted upper bound 25.681862,actual reward 28.701455
    round 3171, predicted reward 24.570926,predicted upper bound 24.575186,actual reward 24.871606
    round 3172, predicted reward 24.966505,predicted upper bound 24.970378,actual reward 24.090965
    round 3173, predicted reward 25.723923,predicted upper bound 25.728623,actual reward 24.107870
    round 3174, predicted reward 25.207305,predicted upper bound 25.211772,actual reward 27.276320
    round 3175, predicted reward 35.401621,predicted upper bound 35.405196,actual reward 29.867104
    round 3176, predicted reward 28.566914,predicted upper bound 28.570719,actual reward 32.462477
    round 3177, predicted reward 21.167419,predicted upper bound 21.171043,actual reward 16.282527
    round 3178, predicted reward 26.501293,predicted upper bound 26.505735,actual reward 22.908517
    round 3179, predicted reward 35.750946,predicted upper bound 35.754264,actual reward 36.008157
    round 3180, predicted reward 24.223908,predicted upper bound 24.227882,actual reward 25.623867
    round 3181, predicted reward 25.832172,predicted upper bound 25.836986,actual reward 21.514050
    round 3182, predicted reward 29.602420,predicted upper bound 29.606483,actual reward 27.115965
    round 3183, predicted reward 31.057277,predicted upper bound 31.060813,actual reward 32.772443
    round 3184, predicted reward 23.122542,predicted upper bound 23.127002,actual reward 22.661869
    round 3185, predicted reward 29.186145,predicted upper bound 29.190582,actual reward 28.990613
    round 3186, predicted reward 32.603013,predicted upper bound 32.606606,actual reward 27.940496
    round 3187, predicted reward 22.057679,predicted upper bound 22.061097,actual reward 20.277470
    round 3188, predicted reward 27.592068,predicted upper bound 27.596528,actual reward 30.904286
    round 3189, predicted reward 25.448256,predicted upper bound 25.452732,actual reward 25.970265
    round 3190, predicted reward 24.407172,predicted upper bound 24.410771,actual reward 24.590876
    round 3191, predicted reward 25.832749,predicted upper bound 25.837093,actual reward 23.732585
    round 3192, predicted reward 25.311492,predicted upper bound 25.314944,actual reward 19.760451
    round 3193, predicted reward 29.489504,predicted upper bound 29.494295,actual reward 29.341339
    round 3194, predicted reward 29.778949,predicted upper bound 29.783438,actual reward 30.200642
    round 3195, predicted reward 28.153153,predicted upper bound 28.156834,actual reward 28.285916
    round 3196, predicted reward 31.839285,predicted upper bound 31.843029,actual reward 35.155298
    round 3197, predicted reward 24.649574,predicted upper bound 24.653923,actual reward 26.593601
    round 3198, predicted reward 29.618975,predicted upper bound 29.622500,actual reward 30.449838
    round 3199, predicted reward 37.344404,predicted upper bound 37.347732,actual reward 39.443218
    round 3200, predicted reward 32.128301,predicted upper bound 32.133126,actual reward 31.348397
    round 3201, predicted reward 27.911041,predicted upper bound 27.915341,actual reward 28.612622
    round 3202, predicted reward 30.900193,predicted upper bound 30.904473,actual reward 26.593051
    round 3203, predicted reward 27.347068,predicted upper bound 27.351755,actual reward 26.088795
    round 3204, predicted reward 25.386082,predicted upper bound 25.390239,actual reward 29.641599
    round 3205, predicted reward 23.586531,predicted upper bound 23.590550,actual reward 26.720938
    round 3206, predicted reward 26.581096,predicted upper bound 26.586460,actual reward 26.130912
    round 3207, predicted reward 24.699045,predicted upper bound 24.703961,actual reward 23.473433
    round 3208, predicted reward 25.649750,predicted upper bound 25.654061,actual reward 25.254713
    round 3209, predicted reward 28.733517,predicted upper bound 28.737185,actual reward 28.875098
    round 3210, predicted reward 24.739831,predicted upper bound 24.744389,actual reward 25.750280
    round 3211, predicted reward 34.255814,predicted upper bound 34.259574,actual reward 34.260808
    round 3212, predicted reward 25.837672,predicted upper bound 25.842232,actual reward 21.192640
    round 3213, predicted reward 24.611006,predicted upper bound 24.615385,actual reward 18.815038
    round 3214, predicted reward 26.042595,predicted upper bound 26.046814,actual reward 27.054701
    round 3215, predicted reward 29.901192,predicted upper bound 29.904273,actual reward 35.830879
    round 3216, predicted reward 29.228967,predicted upper bound 29.233381,actual reward 21.899153
    round 3217, predicted reward 25.270714,predicted upper bound 25.274972,actual reward 22.125245
    round 3218, predicted reward 26.999477,predicted upper bound 27.003485,actual reward 26.110321
    round 3219, predicted reward 22.210347,predicted upper bound 22.214947,actual reward 20.311272
    round 3220, predicted reward 26.952985,predicted upper bound 26.956950,actual reward 29.683192
    round 3221, predicted reward 27.921883,predicted upper bound 27.925895,actual reward 19.625030
    round 3222, predicted reward 27.560839,predicted upper bound 27.564764,actual reward 32.434034
    round 3223, predicted reward 25.072623,predicted upper bound 25.076899,actual reward 20.161832
    round 3224, predicted reward 30.023126,predicted upper bound 30.027290,actual reward 27.307517
    round 3225, predicted reward 28.272890,predicted upper bound 28.276810,actual reward 29.235431
    round 3226, predicted reward 25.688956,predicted upper bound 25.693564,actual reward 18.085927
    round 3227, predicted reward 23.607212,predicted upper bound 23.611756,actual reward 25.210702
    round 3228, predicted reward 22.983554,predicted upper bound 22.988220,actual reward 27.595432
    round 3229, predicted reward 24.768760,predicted upper bound 24.773798,actual reward 27.351453
    round 3230, predicted reward 24.407439,predicted upper bound 24.411963,actual reward 26.349497
    round 3231, predicted reward 24.911790,predicted upper bound 24.916189,actual reward 30.004640
    round 3232, predicted reward 31.900222,predicted upper bound 31.903809,actual reward 26.609741
    round 3233, predicted reward 30.757769,predicted upper bound 30.761664,actual reward 35.154916
    round 3234, predicted reward 28.783689,predicted upper bound 28.788047,actual reward 26.862342
    round 3235, predicted reward 22.319106,predicted upper bound 22.323399,actual reward 19.806139
    round 3236, predicted reward 25.034620,predicted upper bound 25.039907,actual reward 21.658590
    round 3237, predicted reward 27.584097,predicted upper bound 27.588973,actual reward 21.778093
    round 3238, predicted reward 26.962122,predicted upper bound 26.966603,actual reward 25.866267
    round 3239, predicted reward 24.790737,predicted upper bound 24.795377,actual reward 28.308852
    round 3240, predicted reward 29.603866,predicted upper bound 29.607462,actual reward 34.441966
    round 3241, predicted reward 30.509221,predicted upper bound 30.513885,actual reward 30.279111
    round 3242, predicted reward 19.271753,predicted upper bound 19.275881,actual reward 24.460131
    round 3243, predicted reward 31.768694,predicted upper bound 31.771686,actual reward 36.455028
    round 3244, predicted reward 24.755526,predicted upper bound 24.759277,actual reward 24.007518
    round 3245, predicted reward 23.573004,predicted upper bound 23.577741,actual reward 16.058815
    round 3246, predicted reward 20.978419,predicted upper bound 20.983003,actual reward 17.130431
    round 3247, predicted reward 24.403923,predicted upper bound 24.408592,actual reward 23.514572
    round 3248, predicted reward 25.891765,predicted upper bound 25.895144,actual reward 23.033142
    round 3249, predicted reward 28.722504,predicted upper bound 28.726685,actual reward 26.658771
    round 3250, predicted reward 26.695226,predicted upper bound 26.699870,actual reward 20.313624
    round 3251, predicted reward 24.777584,predicted upper bound 24.782323,actual reward 22.030140
    round 3252, predicted reward 22.164934,predicted upper bound 22.169638,actual reward 23.637047
    round 3253, predicted reward 23.744223,predicted upper bound 23.748355,actual reward 28.123779
    round 3254, predicted reward 24.684193,predicted upper bound 24.688654,actual reward 23.179281
    round 3255, predicted reward 27.719412,predicted upper bound 27.724066,actual reward 26.502461
    round 3256, predicted reward 38.301812,predicted upper bound 38.305801,actual reward 45.432844
    round 3257, predicted reward 29.384032,predicted upper bound 29.388082,actual reward 26.696699
    round 3258, predicted reward 21.583971,predicted upper bound 21.588098,actual reward 18.496785
    round 3259, predicted reward 27.846642,predicted upper bound 27.850947,actual reward 26.784434
    round 3260, predicted reward 31.113513,predicted upper bound 31.117409,actual reward 35.566743
    round 3261, predicted reward 23.847789,predicted upper bound 23.851883,actual reward 23.842536
    round 3262, predicted reward 30.598055,predicted upper bound 30.601341,actual reward 29.732285
    round 3263, predicted reward 30.708223,predicted upper bound 30.711559,actual reward 24.227740
    round 3264, predicted reward 20.784198,predicted upper bound 20.788615,actual reward 13.726107
    round 3265, predicted reward 28.113157,predicted upper bound 28.117536,actual reward 27.236295
    round 3266, predicted reward 24.126131,predicted upper bound 24.130434,actual reward 21.686221
    round 3267, predicted reward 21.267340,predicted upper bound 21.272196,actual reward 19.592203
    round 3268, predicted reward 28.718166,predicted upper bound 28.721895,actual reward 29.761307
    round 3269, predicted reward 27.356547,predicted upper bound 27.360510,actual reward 24.217388
    round 3270, predicted reward 30.389575,predicted upper bound 30.393594,actual reward 31.186695
    round 3271, predicted reward 28.495057,predicted upper bound 28.499132,actual reward 27.769776
    round 3272, predicted reward 21.238519,predicted upper bound 21.242970,actual reward 19.138130
    round 3273, predicted reward 24.078445,predicted upper bound 24.083061,actual reward 24.949461
    round 3274, predicted reward 29.054532,predicted upper bound 29.058662,actual reward 34.001365
    round 3275, predicted reward 25.905968,predicted upper bound 25.910092,actual reward 26.375434
    round 3276, predicted reward 23.493675,predicted upper bound 23.497823,actual reward 24.401695
    round 3277, predicted reward 29.778137,predicted upper bound 29.782468,actual reward 25.848427
    round 3278, predicted reward 28.923533,predicted upper bound 28.928116,actual reward 23.714464
    round 3279, predicted reward 27.399965,predicted upper bound 27.404306,actual reward 30.704758
    round 3280, predicted reward 23.134932,predicted upper bound 23.139337,actual reward 26.611315
    round 3281, predicted reward 28.564165,predicted upper bound 28.567770,actual reward 29.739763
    round 3282, predicted reward 22.774241,predicted upper bound 22.778239,actual reward 23.461857
    round 3283, predicted reward 24.329178,predicted upper bound 24.333688,actual reward 23.840057
    round 3284, predicted reward 26.660707,predicted upper bound 26.664562,actual reward 22.554042
    round 3285, predicted reward 26.806007,predicted upper bound 26.810627,actual reward 22.211585
    round 3286, predicted reward 20.769139,predicted upper bound 20.773583,actual reward 23.709821
    round 3287, predicted reward 32.075405,predicted upper bound 32.080118,actual reward 32.659285
    round 3288, predicted reward 38.228449,predicted upper bound 38.231663,actual reward 38.150434
    round 3289, predicted reward 33.341988,predicted upper bound 33.345983,actual reward 33.965128
    round 3290, predicted reward 24.094570,predicted upper bound 24.098923,actual reward 21.594139
    round 3291, predicted reward 24.765274,predicted upper bound 24.769641,actual reward 21.505561
    round 3292, predicted reward 27.062476,predicted upper bound 27.066268,actual reward 21.711354
    round 3293, predicted reward 28.673294,predicted upper bound 28.678068,actual reward 29.792772
    round 3294, predicted reward 25.463317,predicted upper bound 25.467118,actual reward 23.604667
    round 3295, predicted reward 32.867360,predicted upper bound 32.870659,actual reward 27.249256
    round 3296, predicted reward 27.324942,predicted upper bound 27.328790,actual reward 21.456365
    round 3297, predicted reward 29.361470,predicted upper bound 29.365639,actual reward 30.749288
    round 3298, predicted reward 34.308951,predicted upper bound 34.312598,actual reward 39.967375
    round 3299, predicted reward 30.867812,predicted upper bound 30.871463,actual reward 32.122893
    round 3300, predicted reward 34.507681,predicted upper bound 34.511547,actual reward 40.541502
    round 3301, predicted reward 27.426257,predicted upper bound 27.429941,actual reward 27.644732
    round 3302, predicted reward 28.506588,predicted upper bound 28.510320,actual reward 29.971791
    round 3303, predicted reward 21.528534,predicted upper bound 21.532783,actual reward 17.355942
    round 3304, predicted reward 25.740067,predicted upper bound 25.743907,actual reward 25.972729
    round 3305, predicted reward 19.118307,predicted upper bound 19.122780,actual reward 18.802193
    round 3306, predicted reward 29.379482,predicted upper bound 29.383745,actual reward 23.472700
    round 3307, predicted reward 23.927336,predicted upper bound 23.932287,actual reward 20.976461
    round 3308, predicted reward 32.097855,predicted upper bound 32.102075,actual reward 29.727834
    round 3309, predicted reward 20.325900,predicted upper bound 20.329959,actual reward 20.460677
    round 3310, predicted reward 32.725706,predicted upper bound 32.730003,actual reward 30.489302
    round 3311, predicted reward 23.605353,predicted upper bound 23.609853,actual reward 19.015114
    round 3312, predicted reward 25.713317,predicted upper bound 25.717212,actual reward 21.013729
    round 3313, predicted reward 23.474288,predicted upper bound 23.478212,actual reward 22.136172
    round 3314, predicted reward 23.248198,predicted upper bound 23.252080,actual reward 24.287367
    round 3315, predicted reward 26.298096,predicted upper bound 26.301857,actual reward 26.099186
    round 3316, predicted reward 21.042928,predicted upper bound 21.047074,actual reward 19.691841
    round 3317, predicted reward 29.414543,predicted upper bound 29.418577,actual reward 29.969137
    round 3318, predicted reward 30.775027,predicted upper bound 30.779991,actual reward 37.065792
    round 3319, predicted reward 26.925099,predicted upper bound 26.929565,actual reward 28.815747
    round 3320, predicted reward 25.497084,predicted upper bound 25.500622,actual reward 22.253019
    round 3321, predicted reward 36.077558,predicted upper bound 36.081620,actual reward 38.918374
    round 3322, predicted reward 34.767409,predicted upper bound 34.770647,actual reward 36.096904
    round 3323, predicted reward 29.270024,predicted upper bound 29.274454,actual reward 32.088025
    round 3324, predicted reward 27.605668,predicted upper bound 27.609516,actual reward 30.548632
    round 3325, predicted reward 29.736512,predicted upper bound 29.740379,actual reward 29.380588
    round 3326, predicted reward 27.207594,predicted upper bound 27.212194,actual reward 27.020713
    round 3327, predicted reward 29.651347,predicted upper bound 29.655870,actual reward 32.551879
    round 3328, predicted reward 25.358344,predicted upper bound 25.361670,actual reward 22.605114
    round 3329, predicted reward 29.126222,predicted upper bound 29.130629,actual reward 25.271676
    round 3330, predicted reward 32.850909,predicted upper bound 32.854727,actual reward 35.978135
    round 3331, predicted reward 31.511605,predicted upper bound 31.515943,actual reward 27.721233
    round 3332, predicted reward 34.477203,predicted upper bound 34.480606,actual reward 35.424496
    round 3333, predicted reward 32.167383,predicted upper bound 32.171733,actual reward 24.880297
    round 3334, predicted reward 30.053642,predicted upper bound 30.056846,actual reward 29.208931
    round 3335, predicted reward 27.132342,predicted upper bound 27.136959,actual reward 32.802161
    round 3336, predicted reward 30.104145,predicted upper bound 30.108123,actual reward 35.739824
    round 3337, predicted reward 27.712071,predicted upper bound 27.716456,actual reward 25.145268
    round 3338, predicted reward 29.278778,predicted upper bound 29.282880,actual reward 33.210280
    round 3339, predicted reward 28.723897,predicted upper bound 28.727949,actual reward 24.411196
    round 3340, predicted reward 24.247011,predicted upper bound 24.251166,actual reward 25.239570
    round 3341, predicted reward 29.928158,predicted upper bound 29.932083,actual reward 29.287188
    round 3342, predicted reward 23.389874,predicted upper bound 23.393752,actual reward 25.058285
    round 3343, predicted reward 30.717718,predicted upper bound 30.721794,actual reward 32.004037
    round 3344, predicted reward 25.047478,predicted upper bound 25.051789,actual reward 21.276228
    round 3345, predicted reward 31.093447,predicted upper bound 31.097643,actual reward 31.665869
    round 3346, predicted reward 23.798947,predicted upper bound 23.802982,actual reward 19.724937
    round 3347, predicted reward 27.299354,predicted upper bound 27.304088,actual reward 24.865523
    round 3348, predicted reward 23.278034,predicted upper bound 23.282560,actual reward 23.426132
    round 3349, predicted reward 31.267009,predicted upper bound 31.270563,actual reward 31.756360
    round 3350, predicted reward 28.988588,predicted upper bound 28.991989,actual reward 31.285344
    round 3351, predicted reward 19.388564,predicted upper bound 19.393326,actual reward 16.004774
    round 3352, predicted reward 28.874591,predicted upper bound 28.878620,actual reward 28.842441
    round 3353, predicted reward 24.251036,predicted upper bound 24.255521,actual reward 21.524705
    round 3354, predicted reward 27.556373,predicted upper bound 27.560227,actual reward 22.324671
    round 3355, predicted reward 28.474498,predicted upper bound 28.478404,actual reward 25.200907
    round 3356, predicted reward 30.355948,predicted upper bound 30.359221,actual reward 31.675481
    round 3357, predicted reward 26.958173,predicted upper bound 26.963499,actual reward 28.975760
    round 3358, predicted reward 31.278255,predicted upper bound 31.282290,actual reward 25.906170
    round 3359, predicted reward 27.413334,predicted upper bound 27.417535,actual reward 23.429543
    round 3360, predicted reward 33.084302,predicted upper bound 33.088781,actual reward 37.674914
    round 3361, predicted reward 30.585595,predicted upper bound 30.590059,actual reward 28.890389
    round 3362, predicted reward 29.010978,predicted upper bound 29.015013,actual reward 30.231324
    round 3363, predicted reward 29.082320,predicted upper bound 29.086755,actual reward 36.716298
    round 3364, predicted reward 26.593903,predicted upper bound 26.597834,actual reward 22.464843
    round 3365, predicted reward 27.621555,predicted upper bound 27.625501,actual reward 35.760173
    round 3366, predicted reward 30.663755,predicted upper bound 30.667802,actual reward 22.091565
    round 3367, predicted reward 28.271731,predicted upper bound 28.275857,actual reward 29.746584
    round 3368, predicted reward 20.321592,predicted upper bound 20.325881,actual reward 24.792588
    round 3369, predicted reward 26.739919,predicted upper bound 26.743086,actual reward 24.087157
    round 3370, predicted reward 28.938096,predicted upper bound 28.941946,actual reward 27.809488
    round 3371, predicted reward 28.863745,predicted upper bound 28.867961,actual reward 28.880495
    round 3372, predicted reward 33.512524,predicted upper bound 33.516571,actual reward 34.737342
    round 3373, predicted reward 24.210537,predicted upper bound 24.215259,actual reward 28.104241
    round 3374, predicted reward 22.760791,predicted upper bound 22.764457,actual reward 23.944086
    round 3375, predicted reward 29.062563,predicted upper bound 29.066786,actual reward 22.139565
    round 3376, predicted reward 24.154150,predicted upper bound 24.158498,actual reward 21.232166
    round 3377, predicted reward 28.146818,predicted upper bound 28.150316,actual reward 30.119965
    round 3378, predicted reward 30.186507,predicted upper bound 30.191120,actual reward 33.643345
    round 3379, predicted reward 24.703361,predicted upper bound 24.707053,actual reward 25.576606
    round 3380, predicted reward 27.582703,predicted upper bound 27.586912,actual reward 28.326620
    round 3381, predicted reward 20.954518,predicted upper bound 20.958501,actual reward 19.286490
    round 3382, predicted reward 25.713860,predicted upper bound 25.717780,actual reward 26.124954
    round 3383, predicted reward 32.195704,predicted upper bound 32.199938,actual reward 30.953180
    round 3384, predicted reward 24.796231,predicted upper bound 24.800808,actual reward 21.205411
    round 3385, predicted reward 34.601399,predicted upper bound 34.605262,actual reward 39.721447
    round 3386, predicted reward 23.541402,predicted upper bound 23.545774,actual reward 18.627979
    round 3387, predicted reward 21.966589,predicted upper bound 21.971184,actual reward 21.034157
    round 3388, predicted reward 26.529024,predicted upper bound 26.533442,actual reward 27.802327
    round 3389, predicted reward 23.402829,predicted upper bound 23.407372,actual reward 15.948164
    round 3390, predicted reward 20.252007,predicted upper bound 20.256099,actual reward 21.480097
    round 3391, predicted reward 34.727315,predicted upper bound 34.731296,actual reward 33.815643
    round 3392, predicted reward 27.178962,predicted upper bound 27.183358,actual reward 23.933178
    round 3393, predicted reward 32.599530,predicted upper bound 32.604057,actual reward 33.989416
    round 3394, predicted reward 29.089438,predicted upper bound 29.093405,actual reward 28.665729
    round 3395, predicted reward 26.156644,predicted upper bound 26.160862,actual reward 23.183931
    round 3396, predicted reward 24.933617,predicted upper bound 24.937625,actual reward 21.512036
    round 3397, predicted reward 27.371966,predicted upper bound 27.376366,actual reward 27.854947
    round 3398, predicted reward 24.318091,predicted upper bound 24.322564,actual reward 24.054206
    round 3399, predicted reward 24.018635,predicted upper bound 24.022822,actual reward 28.965673
    round 3400, predicted reward 18.954069,predicted upper bound 18.958035,actual reward 16.084948
    round 3401, predicted reward 25.889836,predicted upper bound 25.893733,actual reward 26.818505
    round 3402, predicted reward 30.023504,predicted upper bound 30.028036,actual reward 26.762146
    round 3403, predicted reward 25.433635,predicted upper bound 25.437478,actual reward 25.932499
    round 3404, predicted reward 32.019802,predicted upper bound 32.024151,actual reward 29.486968
    round 3405, predicted reward 34.090431,predicted upper bound 34.094401,actual reward 38.768475
    round 3406, predicted reward 27.092851,predicted upper bound 27.096563,actual reward 27.196928
    round 3407, predicted reward 31.044014,predicted upper bound 31.048551,actual reward 31.466428
    round 3408, predicted reward 23.039687,predicted upper bound 23.044148,actual reward 20.501721
    round 3409, predicted reward 30.716835,predicted upper bound 30.720439,actual reward 30.192092
    round 3410, predicted reward 25.572165,predicted upper bound 25.576496,actual reward 25.394135
    round 3411, predicted reward 30.832981,predicted upper bound 30.837276,actual reward 32.000665
    round 3412, predicted reward 25.747969,predicted upper bound 25.752192,actual reward 22.199595
    round 3413, predicted reward 33.900903,predicted upper bound 33.905190,actual reward 35.046580
    round 3414, predicted reward 21.727174,predicted upper bound 21.731673,actual reward 22.209090
    round 3415, predicted reward 24.410553,predicted upper bound 24.415107,actual reward 20.438512
    round 3416, predicted reward 27.950241,predicted upper bound 27.953904,actual reward 26.422204
    round 3417, predicted reward 21.681272,predicted upper bound 21.686041,actual reward 14.753796
    round 3418, predicted reward 32.736386,predicted upper bound 32.739916,actual reward 37.828593
    round 3419, predicted reward 23.187139,predicted upper bound 23.190897,actual reward 17.232236
    round 3420, predicted reward 22.109500,predicted upper bound 22.114009,actual reward 23.388857
    round 3421, predicted reward 33.131795,predicted upper bound 33.135953,actual reward 30.688474
    round 3422, predicted reward 29.373237,predicted upper bound 29.377413,actual reward 26.799564
    round 3423, predicted reward 23.209805,predicted upper bound 23.214548,actual reward 21.527983
    round 3424, predicted reward 26.789691,predicted upper bound 26.793729,actual reward 29.669207
    round 3425, predicted reward 28.665210,predicted upper bound 28.669639,actual reward 26.621831
    round 3426, predicted reward 30.999082,predicted upper bound 31.002663,actual reward 32.626453
    round 3427, predicted reward 26.880319,predicted upper bound 26.884585,actual reward 21.794258
    round 3428, predicted reward 24.529140,predicted upper bound 24.533518,actual reward 21.446856
    round 3429, predicted reward 30.222152,predicted upper bound 30.226159,actual reward 29.749272
    round 3430, predicted reward 30.240281,predicted upper bound 30.244924,actual reward 28.677402
    round 3431, predicted reward 34.753500,predicted upper bound 34.756919,actual reward 35.575037
    round 3432, predicted reward 24.105584,predicted upper bound 24.109895,actual reward 21.700408
    round 3433, predicted reward 21.383540,predicted upper bound 21.387554,actual reward 17.374524
    round 3434, predicted reward 36.124838,predicted upper bound 36.128594,actual reward 38.541360
    round 3435, predicted reward 28.185968,predicted upper bound 28.190201,actual reward 28.095340
    round 3436, predicted reward 34.973751,predicted upper bound 34.977131,actual reward 34.587599
    round 3437, predicted reward 26.757247,predicted upper bound 26.761586,actual reward 27.161880
    round 3438, predicted reward 22.891017,predicted upper bound 22.895540,actual reward 17.586568
    round 3439, predicted reward 30.910149,predicted upper bound 30.914008,actual reward 29.215784
    round 3440, predicted reward 28.816901,predicted upper bound 28.820881,actual reward 29.609293
    round 3441, predicted reward 31.767285,predicted upper bound 31.771448,actual reward 28.238147
    round 3442, predicted reward 19.775159,predicted upper bound 19.779427,actual reward 16.727150
    round 3443, predicted reward 25.283868,predicted upper bound 25.287697,actual reward 27.496225
    round 3444, predicted reward 31.540145,predicted upper bound 31.543558,actual reward 31.274246
    round 3445, predicted reward 22.410747,predicted upper bound 22.414812,actual reward 20.798459
    round 3446, predicted reward 26.297384,predicted upper bound 26.300810,actual reward 20.825872
    round 3447, predicted reward 27.557760,predicted upper bound 27.562037,actual reward 26.726604
    round 3448, predicted reward 31.942232,predicted upper bound 31.946304,actual reward 36.010793
    round 3449, predicted reward 30.950707,predicted upper bound 30.953898,actual reward 31.607292
    round 3450, predicted reward 22.464419,predicted upper bound 22.468929,actual reward 22.346378
    round 3451, predicted reward 23.820443,predicted upper bound 23.824019,actual reward 20.792617
    round 3452, predicted reward 32.118175,predicted upper bound 32.121922,actual reward 31.641684
    round 3453, predicted reward 31.657000,predicted upper bound 31.661128,actual reward 36.207342
    round 3454, predicted reward 34.437749,predicted upper bound 34.441814,actual reward 34.886930
    round 3455, predicted reward 22.608343,predicted upper bound 22.612504,actual reward 21.729156
    round 3456, predicted reward 30.375979,predicted upper bound 30.379983,actual reward 29.370108
    round 3457, predicted reward 24.058821,predicted upper bound 24.062899,actual reward 18.575544
    round 3458, predicted reward 32.691199,predicted upper bound 32.694623,actual reward 34.166719
    round 3459, predicted reward 26.495540,predicted upper bound 26.500328,actual reward 28.700539
    round 3460, predicted reward 28.265273,predicted upper bound 28.269711,actual reward 28.559153
    round 3461, predicted reward 30.410229,predicted upper bound 30.414599,actual reward 27.520657
    round 3462, predicted reward 21.935894,predicted upper bound 21.940308,actual reward 21.412183
    round 3463, predicted reward 35.832263,predicted upper bound 35.836094,actual reward 41.866517
    round 3464, predicted reward 24.999479,predicted upper bound 25.003367,actual reward 23.319010
    round 3465, predicted reward 35.691341,predicted upper bound 35.695488,actual reward 37.646346
    round 3466, predicted reward 30.664612,predicted upper bound 30.668276,actual reward 31.072630
    round 3467, predicted reward 25.345895,predicted upper bound 25.349571,actual reward 26.597663
    round 3468, predicted reward 42.614015,predicted upper bound 42.617258,actual reward 49.549765
    round 3469, predicted reward 24.382807,predicted upper bound 24.386874,actual reward 24.299249
    round 3470, predicted reward 29.750966,predicted upper bound 29.754795,actual reward 30.179418
    round 3471, predicted reward 24.721680,predicted upper bound 24.726342,actual reward 21.383790
    round 3472, predicted reward 32.426422,predicted upper bound 32.430786,actual reward 35.937683
    round 3473, predicted reward 24.310011,predicted upper bound 24.314139,actual reward 24.710260
    round 3474, predicted reward 25.478811,predicted upper bound 25.482558,actual reward 21.838230
    round 3475, predicted reward 27.579426,predicted upper bound 27.583193,actual reward 32.526538
    round 3476, predicted reward 29.652939,predicted upper bound 29.657244,actual reward 26.666216
    round 3477, predicted reward 28.607011,predicted upper bound 28.610625,actual reward 25.550690
    round 3478, predicted reward 23.034766,predicted upper bound 23.039008,actual reward 23.145966
    round 3479, predicted reward 23.263375,predicted upper bound 23.267433,actual reward 24.349528
    round 3480, predicted reward 31.854236,predicted upper bound 31.858576,actual reward 34.487166
    round 3481, predicted reward 29.549458,predicted upper bound 29.553631,actual reward 30.129891
    round 3482, predicted reward 25.060337,predicted upper bound 25.065030,actual reward 27.263553
    round 3483, predicted reward 32.729516,predicted upper bound 32.733790,actual reward 33.791151
    round 3484, predicted reward 21.090064,predicted upper bound 21.094834,actual reward 16.850461
    round 3485, predicted reward 25.379608,predicted upper bound 25.383633,actual reward 19.451080
    round 3486, predicted reward 28.684283,predicted upper bound 28.688664,actual reward 28.132621
    round 3487, predicted reward 24.198437,predicted upper bound 24.201887,actual reward 22.373334
    round 3488, predicted reward 37.607215,predicted upper bound 37.610697,actual reward 37.303514
    round 3489, predicted reward 24.912508,predicted upper bound 24.916681,actual reward 20.460807
    round 3490, predicted reward 27.232689,predicted upper bound 27.236040,actual reward 20.713884
    round 3491, predicted reward 22.230860,predicted upper bound 22.234968,actual reward 18.875790
    round 3492, predicted reward 32.751864,predicted upper bound 32.756030,actual reward 33.741056
    round 3493, predicted reward 33.301117,predicted upper bound 33.304484,actual reward 31.682158
    round 3494, predicted reward 25.101128,predicted upper bound 25.105362,actual reward 23.208183
    round 3495, predicted reward 25.753956,predicted upper bound 25.758276,actual reward 25.441800
    round 3496, predicted reward 24.748640,predicted upper bound 24.752166,actual reward 21.806119
    round 3497, predicted reward 24.440524,predicted upper bound 24.444504,actual reward 25.703949
    round 3498, predicted reward 27.315802,predicted upper bound 27.319600,actual reward 24.161702
    round 3499, predicted reward 28.608986,predicted upper bound 28.612796,actual reward 20.923911
    round 3500, predicted reward 29.201750,predicted upper bound 29.205844,actual reward 28.618469
    round 3501, predicted reward 23.707734,predicted upper bound 23.713004,actual reward 19.566575
    round 3502, predicted reward 30.614629,predicted upper bound 30.617686,actual reward 31.243813
    round 3503, predicted reward 29.421455,predicted upper bound 29.425475,actual reward 29.100907
    round 3504, predicted reward 26.646987,predicted upper bound 26.651334,actual reward 32.257049
    round 3505, predicted reward 25.761418,predicted upper bound 25.765760,actual reward 26.961542
    round 3506, predicted reward 27.581694,predicted upper bound 27.585206,actual reward 24.544575
    round 3507, predicted reward 24.950348,predicted upper bound 24.954468,actual reward 23.980309
    round 3508, predicted reward 23.788290,predicted upper bound 23.792768,actual reward 23.013771
    round 3509, predicted reward 28.287917,predicted upper bound 28.292200,actual reward 20.666989
    round 3510, predicted reward 30.158522,predicted upper bound 30.162381,actual reward 21.601830
    round 3511, predicted reward 27.770836,predicted upper bound 27.774888,actual reward 25.605448
    round 3512, predicted reward 31.586985,predicted upper bound 31.590952,actual reward 31.163843
    round 3513, predicted reward 27.317953,predicted upper bound 27.321638,actual reward 28.481413
    round 3514, predicted reward 30.573313,predicted upper bound 30.577057,actual reward 32.167459
    round 3515, predicted reward 30.146388,predicted upper bound 30.150293,actual reward 29.781229
    round 3516, predicted reward 27.548261,predicted upper bound 27.552561,actual reward 30.207720
    round 3517, predicted reward 27.229861,predicted upper bound 27.234388,actual reward 26.726143
    round 3518, predicted reward 29.132502,predicted upper bound 29.136186,actual reward 26.486909
    round 3519, predicted reward 28.539493,predicted upper bound 28.543940,actual reward 28.875505
    round 3520, predicted reward 28.938371,predicted upper bound 28.942259,actual reward 28.605244
    round 3521, predicted reward 29.738757,predicted upper bound 29.742595,actual reward 32.793363
    round 3522, predicted reward 25.574416,predicted upper bound 25.579063,actual reward 23.930443
    round 3523, predicted reward 27.849653,predicted upper bound 27.852874,actual reward 27.098072
    round 3524, predicted reward 29.898732,predicted upper bound 29.902166,actual reward 32.061956
    round 3525, predicted reward 23.801787,predicted upper bound 23.805891,actual reward 22.820241
    round 3526, predicted reward 27.498330,predicted upper bound 27.502438,actual reward 23.562952
    round 3527, predicted reward 25.659482,predicted upper bound 25.663758,actual reward 24.563925
    round 3528, predicted reward 27.991162,predicted upper bound 27.995504,actual reward 26.826471
    round 3529, predicted reward 30.034699,predicted upper bound 30.038666,actual reward 20.765095
    round 3530, predicted reward 24.916770,predicted upper bound 24.920459,actual reward 21.594323
    round 3531, predicted reward 24.597255,predicted upper bound 24.600750,actual reward 23.081179
    round 3532, predicted reward 23.157611,predicted upper bound 23.161658,actual reward 20.953678
    round 3533, predicted reward 20.664683,predicted upper bound 20.669421,actual reward 23.456715
    round 3534, predicted reward 30.566755,predicted upper bound 30.569977,actual reward 27.748066
    round 3535, predicted reward 27.028796,predicted upper bound 27.032546,actual reward 27.215593
    round 3536, predicted reward 29.799075,predicted upper bound 29.802916,actual reward 30.118844
    round 3537, predicted reward 25.722501,predicted upper bound 25.726544,actual reward 17.667510
    round 3538, predicted reward 26.840113,predicted upper bound 26.844384,actual reward 22.300650
    round 3539, predicted reward 27.651074,predicted upper bound 27.654234,actual reward 28.445294
    round 3540, predicted reward 28.515220,predicted upper bound 28.519000,actual reward 27.371256
    round 3541, predicted reward 22.385819,predicted upper bound 22.390017,actual reward 24.496165
    round 3542, predicted reward 21.594207,predicted upper bound 21.597993,actual reward 21.688751
    round 3543, predicted reward 21.264540,predicted upper bound 21.268754,actual reward 15.972574
    round 3544, predicted reward 31.669354,predicted upper bound 31.673443,actual reward 32.779367
    round 3545, predicted reward 28.488618,predicted upper bound 28.492724,actual reward 31.588650
    round 3546, predicted reward 34.889698,predicted upper bound 34.893745,actual reward 28.764316
    round 3547, predicted reward 29.420095,predicted upper bound 29.424142,actual reward 30.063705
    round 3548, predicted reward 28.381178,predicted upper bound 28.385182,actual reward 25.736300
    round 3549, predicted reward 29.110914,predicted upper bound 29.115063,actual reward 27.083336
    round 3550, predicted reward 23.561681,predicted upper bound 23.565847,actual reward 26.164132
    round 3551, predicted reward 24.265399,predicted upper bound 24.269081,actual reward 22.596285
    round 3552, predicted reward 23.481115,predicted upper bound 23.485591,actual reward 19.760645
    round 3553, predicted reward 28.820028,predicted upper bound 28.823393,actual reward 30.874312
    round 3554, predicted reward 32.326955,predicted upper bound 32.330753,actual reward 29.020710
    round 3555, predicted reward 27.699084,predicted upper bound 27.702568,actual reward 21.208968
    round 3556, predicted reward 27.264894,predicted upper bound 27.268889,actual reward 21.522158
    round 3557, predicted reward 24.540349,predicted upper bound 24.544613,actual reward 24.879467
    round 3558, predicted reward 27.509426,predicted upper bound 27.513487,actual reward 25.547326
    round 3559, predicted reward 27.951286,predicted upper bound 27.955104,actual reward 20.732649
    round 3560, predicted reward 29.037857,predicted upper bound 29.041965,actual reward 26.732289
    round 3561, predicted reward 27.008160,predicted upper bound 27.011845,actual reward 27.713183
    round 3562, predicted reward 24.191057,predicted upper bound 24.195307,actual reward 27.248562
    round 3563, predicted reward 35.658939,predicted upper bound 35.662168,actual reward 36.828725
    round 3564, predicted reward 26.948183,predicted upper bound 26.952929,actual reward 25.734115
    round 3565, predicted reward 27.263399,predicted upper bound 27.267716,actual reward 22.394454
    round 3566, predicted reward 29.438147,predicted upper bound 29.441830,actual reward 32.723584
    round 3567, predicted reward 28.321112,predicted upper bound 28.324960,actual reward 27.678823
    round 3568, predicted reward 27.848512,predicted upper bound 27.852051,actual reward 26.535889
    round 3569, predicted reward 23.906463,predicted upper bound 23.910710,actual reward 25.161398
    round 3570, predicted reward 23.092398,predicted upper bound 23.096647,actual reward 20.269872
    round 3571, predicted reward 29.805272,predicted upper bound 29.809730,actual reward 21.503384
    round 3572, predicted reward 32.943713,predicted upper bound 32.947840,actual reward 32.629076
    round 3573, predicted reward 30.330283,predicted upper bound 30.333552,actual reward 29.566358
    round 3574, predicted reward 28.666205,predicted upper bound 28.669998,actual reward 29.259380
    round 3575, predicted reward 30.314185,predicted upper bound 30.317641,actual reward 26.574172
    round 3576, predicted reward 28.575224,predicted upper bound 28.579231,actual reward 33.184628
    round 3577, predicted reward 28.131188,predicted upper bound 28.134381,actual reward 24.303095
    round 3578, predicted reward 35.781323,predicted upper bound 35.784998,actual reward 34.864978
    round 3579, predicted reward 24.055261,predicted upper bound 24.058816,actual reward 25.520101
    round 3580, predicted reward 27.941876,predicted upper bound 27.945121,actual reward 22.219567
    round 3581, predicted reward 21.968318,predicted upper bound 21.971945,actual reward 20.147757
    round 3582, predicted reward 24.586976,predicted upper bound 24.591073,actual reward 19.934208
    round 3583, predicted reward 24.452556,predicted upper bound 24.456131,actual reward 15.633942
    round 3584, predicted reward 23.209631,predicted upper bound 23.214278,actual reward 21.446437
    round 3585, predicted reward 24.973315,predicted upper bound 24.976833,actual reward 23.361014
    round 3586, predicted reward 26.900383,predicted upper bound 26.903976,actual reward 19.826229
    round 3587, predicted reward 29.702466,predicted upper bound 29.705801,actual reward 29.624355
    round 3588, predicted reward 24.983527,predicted upper bound 24.986780,actual reward 23.448892
    round 3589, predicted reward 26.030654,predicted upper bound 26.034583,actual reward 23.537587
    round 3590, predicted reward 25.010585,predicted upper bound 25.014885,actual reward 20.206762
    round 3591, predicted reward 23.105638,predicted upper bound 23.109244,actual reward 19.445048
    round 3592, predicted reward 23.921800,predicted upper bound 23.925515,actual reward 24.665084
    round 3593, predicted reward 29.803214,predicted upper bound 29.807476,actual reward 35.123640
    round 3594, predicted reward 29.029573,predicted upper bound 29.033540,actual reward 27.726730
    round 3595, predicted reward 33.225725,predicted upper bound 33.229056,actual reward 32.975310
    round 3596, predicted reward 28.564306,predicted upper bound 28.567408,actual reward 25.747632
    round 3597, predicted reward 28.723001,predicted upper bound 28.726437,actual reward 25.907753
    round 3598, predicted reward 27.336711,predicted upper bound 27.339884,actual reward 27.370909
    round 3599, predicted reward 28.908500,predicted upper bound 28.912083,actual reward 27.387111
    round 3600, predicted reward 27.753089,predicted upper bound 27.757064,actual reward 23.694028
    round 3601, predicted reward 28.240104,predicted upper bound 28.244074,actual reward 24.139758
    round 3602, predicted reward 23.322240,predicted upper bound 23.326861,actual reward 16.172254
    round 3603, predicted reward 24.045168,predicted upper bound 24.049742,actual reward 15.681395
    round 3604, predicted reward 28.367372,predicted upper bound 28.371594,actual reward 26.777981
    round 3605, predicted reward 39.767849,predicted upper bound 39.771334,actual reward 46.595520
    round 3606, predicted reward 26.139598,predicted upper bound 26.143935,actual reward 30.958032
    round 3607, predicted reward 27.110274,predicted upper bound 27.113887,actual reward 27.721671
    round 3608, predicted reward 26.194638,predicted upper bound 26.198390,actual reward 27.809620
    round 3609, predicted reward 30.023874,predicted upper bound 30.027484,actual reward 27.480486
    round 3610, predicted reward 26.541261,predicted upper bound 26.544822,actual reward 29.956403
    round 3611, predicted reward 27.910334,predicted upper bound 27.914936,actual reward 24.146737
    round 3612, predicted reward 25.661284,predicted upper bound 25.665324,actual reward 23.271479
    round 3613, predicted reward 22.686231,predicted upper bound 22.690162,actual reward 20.707249
    round 3614, predicted reward 26.150927,predicted upper bound 26.154964,actual reward 29.456420
    round 3615, predicted reward 29.843783,predicted upper bound 29.846835,actual reward 32.105871
    round 3616, predicted reward 34.885128,predicted upper bound 34.888863,actual reward 38.681100
    round 3617, predicted reward 24.645349,predicted upper bound 24.649049,actual reward 22.089030
    round 3618, predicted reward 21.488798,predicted upper bound 21.493480,actual reward 21.656562
    round 3619, predicted reward 26.961694,predicted upper bound 26.965449,actual reward 22.540182
    round 3620, predicted reward 29.095902,predicted upper bound 29.100016,actual reward 27.365077
    round 3621, predicted reward 35.337968,predicted upper bound 35.341542,actual reward 39.155468
    round 3622, predicted reward 25.484082,predicted upper bound 25.488195,actual reward 23.551064
    round 3623, predicted reward 29.318491,predicted upper bound 29.322347,actual reward 29.443412
    round 3624, predicted reward 28.612120,predicted upper bound 28.615584,actual reward 21.554013
    round 3625, predicted reward 33.717389,predicted upper bound 33.721145,actual reward 36.077952
    round 3626, predicted reward 30.448357,predicted upper bound 30.452289,actual reward 22.446262
    round 3627, predicted reward 31.197333,predicted upper bound 31.200876,actual reward 29.627798
    round 3628, predicted reward 30.041275,predicted upper bound 30.045268,actual reward 24.728234
    round 3629, predicted reward 30.450350,predicted upper bound 30.454067,actual reward 31.968321
    round 3630, predicted reward 30.806780,predicted upper bound 30.810645,actual reward 30.242415
    round 3631, predicted reward 22.617195,predicted upper bound 22.621673,actual reward 25.474006
    round 3632, predicted reward 27.965084,predicted upper bound 27.968560,actual reward 31.842040
    round 3633, predicted reward 29.047224,predicted upper bound 29.050735,actual reward 27.730740
    round 3634, predicted reward 22.590015,predicted upper bound 22.594586,actual reward 25.115182
    round 3635, predicted reward 27.529902,predicted upper bound 27.533811,actual reward 32.091817
    round 3636, predicted reward 27.212781,predicted upper bound 27.216330,actual reward 22.126145
    round 3637, predicted reward 31.926023,predicted upper bound 31.929853,actual reward 30.301458
    round 3638, predicted reward 23.269026,predicted upper bound 23.273678,actual reward 21.944871
    round 3639, predicted reward 28.879116,predicted upper bound 28.883477,actual reward 29.246060
    round 3640, predicted reward 25.914686,predicted upper bound 25.919149,actual reward 23.203862
    round 3641, predicted reward 21.229310,predicted upper bound 21.233882,actual reward 21.636549
    round 3642, predicted reward 31.825029,predicted upper bound 31.828877,actual reward 32.249528
    round 3643, predicted reward 28.116091,predicted upper bound 28.120049,actual reward 30.811319
    round 3644, predicted reward 29.146426,predicted upper bound 29.150128,actual reward 24.567246
    round 3645, predicted reward 27.425480,predicted upper bound 27.429247,actual reward 27.581447
    round 3646, predicted reward 22.742390,predicted upper bound 22.746758,actual reward 25.849436
    round 3647, predicted reward 22.537994,predicted upper bound 22.542256,actual reward 23.205454
    round 3648, predicted reward 30.475319,predicted upper bound 30.478919,actual reward 32.430030
    round 3649, predicted reward 29.663523,predicted upper bound 29.667103,actual reward 29.241217
    round 3650, predicted reward 24.224513,predicted upper bound 24.229162,actual reward 22.325925
    round 3651, predicted reward 30.413537,predicted upper bound 30.418323,actual reward 26.303364
    round 3652, predicted reward 27.567053,predicted upper bound 27.570753,actual reward 23.179460
    round 3653, predicted reward 20.322997,predicted upper bound 20.327212,actual reward 15.206259
    round 3654, predicted reward 27.668295,predicted upper bound 27.672512,actual reward 30.942860
    round 3655, predicted reward 24.742874,predicted upper bound 24.747149,actual reward 23.601615
    round 3656, predicted reward 27.440410,predicted upper bound 27.444241,actual reward 24.710494
    round 3657, predicted reward 25.271159,predicted upper bound 25.275399,actual reward 21.693531
    round 3658, predicted reward 31.294272,predicted upper bound 31.297422,actual reward 31.066107
    round 3659, predicted reward 31.232631,predicted upper bound 31.237206,actual reward 31.929295
    round 3660, predicted reward 23.637015,predicted upper bound 23.641588,actual reward 24.244138
    round 3661, predicted reward 30.975806,predicted upper bound 30.979615,actual reward 32.470370
    round 3662, predicted reward 32.325651,predicted upper bound 32.329696,actual reward 38.154785
    round 3663, predicted reward 26.197800,predicted upper bound 26.201017,actual reward 23.476967
    round 3664, predicted reward 29.145241,predicted upper bound 29.149498,actual reward 27.352131
    round 3665, predicted reward 29.290201,predicted upper bound 29.294416,actual reward 28.923662
    round 3666, predicted reward 25.292852,predicted upper bound 25.297158,actual reward 21.290333
    round 3667, predicted reward 35.307923,predicted upper bound 35.311495,actual reward 41.831135
    round 3668, predicted reward 30.651497,predicted upper bound 30.655857,actual reward 28.726221
    round 3669, predicted reward 27.669611,predicted upper bound 27.673375,actual reward 29.349238
    round 3670, predicted reward 24.654632,predicted upper bound 24.658767,actual reward 27.472289
    round 3671, predicted reward 26.182569,predicted upper bound 26.186231,actual reward 24.954806
    round 3672, predicted reward 28.121779,predicted upper bound 28.125411,actual reward 25.971640
    round 3673, predicted reward 22.550143,predicted upper bound 22.554406,actual reward 21.386129
    round 3674, predicted reward 24.748203,predicted upper bound 24.752542,actual reward 22.217627
    round 3675, predicted reward 24.269621,predicted upper bound 24.273434,actual reward 23.811843
    round 3676, predicted reward 25.992119,predicted upper bound 25.995629,actual reward 27.964065
    round 3677, predicted reward 26.749021,predicted upper bound 26.753369,actual reward 24.687849
    round 3678, predicted reward 35.563371,predicted upper bound 35.567517,actual reward 37.891986
    round 3679, predicted reward 28.314815,predicted upper bound 28.318933,actual reward 29.771245
    round 3680, predicted reward 29.160318,predicted upper bound 29.164062,actual reward 25.807462
    round 3681, predicted reward 27.070714,predicted upper bound 27.074765,actual reward 29.587777
    round 3682, predicted reward 31.260733,predicted upper bound 31.264581,actual reward 29.770669
    round 3683, predicted reward 28.922497,predicted upper bound 28.926905,actual reward 26.768508
    round 3684, predicted reward 23.652524,predicted upper bound 23.656083,actual reward 21.637620
    round 3685, predicted reward 34.635565,predicted upper bound 34.638885,actual reward 33.227724
    round 3686, predicted reward 24.235805,predicted upper bound 24.239182,actual reward 19.399515
    round 3687, predicted reward 24.353088,predicted upper bound 24.357623,actual reward 22.823781
    round 3688, predicted reward 36.399201,predicted upper bound 36.402129,actual reward 41.908006
    round 3689, predicted reward 26.781393,predicted upper bound 26.785714,actual reward 24.369055
    round 3690, predicted reward 31.057205,predicted upper bound 31.061422,actual reward 31.258369
    round 3691, predicted reward 23.967435,predicted upper bound 23.971628,actual reward 24.498100
    round 3692, predicted reward 31.912102,predicted upper bound 31.916293,actual reward 33.397500
    round 3693, predicted reward 34.578586,predicted upper bound 34.581567,actual reward 32.485540
    round 3694, predicted reward 25.131777,predicted upper bound 25.135944,actual reward 23.102225
    round 3695, predicted reward 32.920299,predicted upper bound 32.924895,actual reward 31.729885
    round 3696, predicted reward 25.113247,predicted upper bound 25.117446,actual reward 22.236753
    round 3697, predicted reward 24.444483,predicted upper bound 24.448666,actual reward 22.693065
    round 3698, predicted reward 21.846384,predicted upper bound 21.849984,actual reward 18.967543
    round 3699, predicted reward 29.142793,predicted upper bound 29.147414,actual reward 31.671589
    round 3700, predicted reward 39.051760,predicted upper bound 39.055377,actual reward 43.442883
    round 3701, predicted reward 31.970351,predicted upper bound 31.974189,actual reward 35.468036
    round 3702, predicted reward 31.628238,predicted upper bound 31.631593,actual reward 30.634258
    round 3703, predicted reward 23.421570,predicted upper bound 23.425743,actual reward 24.011102
    round 3704, predicted reward 27.088115,predicted upper bound 27.092387,actual reward 24.546939
    round 3705, predicted reward 26.368496,predicted upper bound 26.372597,actual reward 24.722960
    round 3706, predicted reward 22.773500,predicted upper bound 22.777553,actual reward 18.471419
    round 3707, predicted reward 28.917176,predicted upper bound 28.920803,actual reward 23.556262
    round 3708, predicted reward 22.999336,predicted upper bound 23.003816,actual reward 21.275538
    round 3709, predicted reward 24.679011,predicted upper bound 24.683520,actual reward 22.117255
    round 3710, predicted reward 25.794734,predicted upper bound 25.798285,actual reward 27.229366
    round 3711, predicted reward 23.152690,predicted upper bound 23.157220,actual reward 22.279209
    round 3712, predicted reward 28.301751,predicted upper bound 28.305957,actual reward 29.722315
    round 3713, predicted reward 24.526213,predicted upper bound 24.530630,actual reward 19.249043
    round 3714, predicted reward 22.905505,predicted upper bound 22.910153,actual reward 22.499339
    round 3715, predicted reward 37.583307,predicted upper bound 37.586728,actual reward 35.992201
    round 3716, predicted reward 22.867289,predicted upper bound 22.871168,actual reward 22.520386
    round 3717, predicted reward 26.656553,predicted upper bound 26.660254,actual reward 24.713242
    round 3718, predicted reward 26.129820,predicted upper bound 26.134216,actual reward 24.653297
    round 3719, predicted reward 31.269723,predicted upper bound 31.273344,actual reward 33.660680
    round 3720, predicted reward 26.805585,predicted upper bound 26.809725,actual reward 24.736948
    round 3721, predicted reward 23.722345,predicted upper bound 23.726130,actual reward 28.811766
    round 3722, predicted reward 31.744594,predicted upper bound 31.748572,actual reward 35.347442
    round 3723, predicted reward 26.062623,predicted upper bound 26.067018,actual reward 22.156494
    round 3724, predicted reward 34.296861,predicted upper bound 34.300533,actual reward 38.068123
    round 3725, predicted reward 28.538473,predicted upper bound 28.542694,actual reward 32.666835
    round 3726, predicted reward 27.850655,predicted upper bound 27.853670,actual reward 28.971129
    round 3727, predicted reward 21.722048,predicted upper bound 21.725997,actual reward 15.828571
    round 3728, predicted reward 32.599595,predicted upper bound 32.603149,actual reward 33.272134
    round 3729, predicted reward 22.468927,predicted upper bound 22.472633,actual reward 20.042963
    round 3730, predicted reward 24.896421,predicted upper bound 24.900994,actual reward 22.156675
    round 3731, predicted reward 26.400239,predicted upper bound 26.404006,actual reward 31.788005
    round 3732, predicted reward 22.797669,predicted upper bound 22.801818,actual reward 18.928089
    round 3733, predicted reward 23.799852,predicted upper bound 23.803662,actual reward 26.407700
    round 3734, predicted reward 35.217063,predicted upper bound 35.221308,actual reward 32.557969
    round 3735, predicted reward 24.723489,predicted upper bound 24.727771,actual reward 25.884941
    round 3736, predicted reward 23.853428,predicted upper bound 23.856982,actual reward 22.560067
    round 3737, predicted reward 25.349402,predicted upper bound 25.353627,actual reward 28.073481
    round 3738, predicted reward 25.344275,predicted upper bound 25.348159,actual reward 23.479843
    round 3739, predicted reward 32.878636,predicted upper bound 32.882861,actual reward 29.114662
    round 3740, predicted reward 29.793204,predicted upper bound 29.797047,actual reward 33.834330
    round 3741, predicted reward 25.061326,predicted upper bound 25.065494,actual reward 25.888495
    round 3742, predicted reward 23.070667,predicted upper bound 23.074358,actual reward 24.216725
    round 3743, predicted reward 24.949959,predicted upper bound 24.954408,actual reward 22.418907
    round 3744, predicted reward 23.745774,predicted upper bound 23.749433,actual reward 21.097009
    round 3745, predicted reward 28.145131,predicted upper bound 28.148704,actual reward 26.779057
    round 3746, predicted reward 30.686793,predicted upper bound 30.690871,actual reward 32.197678
    round 3747, predicted reward 27.417032,predicted upper bound 27.420757,actual reward 26.401262
    round 3748, predicted reward 22.063056,predicted upper bound 22.067559,actual reward 21.774648
    round 3749, predicted reward 20.038807,predicted upper bound 20.042385,actual reward 18.826002
    round 3750, predicted reward 24.041459,predicted upper bound 24.045883,actual reward 26.056567
    round 3751, predicted reward 25.738434,predicted upper bound 25.743432,actual reward 20.049341
    round 3752, predicted reward 30.429005,predicted upper bound 30.432518,actual reward 25.895618
    round 3753, predicted reward 25.612481,predicted upper bound 25.616352,actual reward 24.525736
    round 3754, predicted reward 29.044158,predicted upper bound 29.047865,actual reward 30.679178
    round 3755, predicted reward 32.005672,predicted upper bound 32.009373,actual reward 30.210265
    round 3756, predicted reward 28.971071,predicted upper bound 28.974393,actual reward 29.611142
    round 3757, predicted reward 20.602762,predicted upper bound 20.606968,actual reward 19.368249
    round 3758, predicted reward 25.710231,predicted upper bound 25.714774,actual reward 22.603578
    round 3759, predicted reward 28.934416,predicted upper bound 28.938603,actual reward 24.286336
    round 3760, predicted reward 26.823376,predicted upper bound 26.826776,actual reward 23.246721
    round 3761, predicted reward 22.157160,predicted upper bound 22.161272,actual reward 25.033980
    round 3762, predicted reward 34.494567,predicted upper bound 34.499044,actual reward 30.924989
    round 3763, predicted reward 26.053126,predicted upper bound 26.056945,actual reward 27.395456
    round 3764, predicted reward 36.316400,predicted upper bound 36.319913,actual reward 37.988007
    round 3765, predicted reward 25.048141,predicted upper bound 25.051415,actual reward 20.724815
    round 3766, predicted reward 27.849306,predicted upper bound 27.852919,actual reward 29.266673
    round 3767, predicted reward 26.630840,predicted upper bound 26.635477,actual reward 19.562836
    round 3768, predicted reward 29.363881,predicted upper bound 29.367860,actual reward 28.061045
    round 3769, predicted reward 23.199471,predicted upper bound 23.204688,actual reward 21.619133
    round 3770, predicted reward 27.187163,predicted upper bound 27.191861,actual reward 31.329967
    round 3771, predicted reward 30.662400,predicted upper bound 30.665932,actual reward 30.207981
    round 3772, predicted reward 25.478736,predicted upper bound 25.482832,actual reward 20.950021
    round 3773, predicted reward 29.800758,predicted upper bound 29.804758,actual reward 27.403215
    round 3774, predicted reward 30.817967,predicted upper bound 30.821362,actual reward 32.517216
    round 3775, predicted reward 31.051892,predicted upper bound 31.055374,actual reward 29.071053
    round 3776, predicted reward 27.224324,predicted upper bound 27.228285,actual reward 24.970060
    round 3777, predicted reward 34.371721,predicted upper bound 34.375068,actual reward 40.427245
    round 3778, predicted reward 30.234451,predicted upper bound 30.238402,actual reward 26.165997
    round 3779, predicted reward 27.121456,predicted upper bound 27.125783,actual reward 26.814761
    round 3780, predicted reward 30.087706,predicted upper bound 30.091512,actual reward 33.224559
    round 3781, predicted reward 27.371849,predicted upper bound 27.375272,actual reward 26.609239
    round 3782, predicted reward 28.986584,predicted upper bound 28.990519,actual reward 25.154245
    round 3783, predicted reward 33.292113,predicted upper bound 33.295039,actual reward 32.923268
    round 3784, predicted reward 23.777930,predicted upper bound 23.781673,actual reward 25.966380
    round 3785, predicted reward 24.149584,predicted upper bound 24.153657,actual reward 28.186216
    round 3786, predicted reward 26.385467,predicted upper bound 26.389064,actual reward 21.360345
    round 3787, predicted reward 24.672284,predicted upper bound 24.676154,actual reward 22.704850
    round 3788, predicted reward 32.289921,predicted upper bound 32.293423,actual reward 27.710123
    round 3789, predicted reward 30.559606,predicted upper bound 30.563030,actual reward 29.499762
    round 3790, predicted reward 28.347414,predicted upper bound 28.351418,actual reward 33.269299
    round 3791, predicted reward 27.457360,predicted upper bound 27.461192,actual reward 25.232326
    round 3792, predicted reward 27.663741,predicted upper bound 27.667112,actual reward 27.656295
    round 3793, predicted reward 32.343226,predicted upper bound 32.346829,actual reward 29.315543
    round 3794, predicted reward 27.841813,predicted upper bound 27.846184,actual reward 25.780002
    round 3795, predicted reward 30.441220,predicted upper bound 30.445219,actual reward 34.692296
    round 3796, predicted reward 31.172763,predicted upper bound 31.176467,actual reward 32.773144
    round 3797, predicted reward 24.700023,predicted upper bound 24.703753,actual reward 23.494621
    round 3798, predicted reward 27.104472,predicted upper bound 27.108229,actual reward 20.238977
    round 3799, predicted reward 25.370113,predicted upper bound 25.373489,actual reward 25.624353
    round 3800, predicted reward 29.323815,predicted upper bound 29.328025,actual reward 29.211721
    round 3801, predicted reward 23.561009,predicted upper bound 23.565539,actual reward 25.735259
    round 3802, predicted reward 32.817575,predicted upper bound 32.821253,actual reward 40.649840
    round 3803, predicted reward 23.479699,predicted upper bound 23.483709,actual reward 24.801653
    round 3804, predicted reward 23.350434,predicted upper bound 23.355215,actual reward 20.732817
    round 3805, predicted reward 29.296176,predicted upper bound 29.299863,actual reward 25.286014
    round 3806, predicted reward 28.510049,predicted upper bound 28.514066,actual reward 20.411058
    round 3807, predicted reward 21.278442,predicted upper bound 21.282761,actual reward 17.654861
    round 3808, predicted reward 22.830626,predicted upper bound 22.835069,actual reward 21.212324
    round 3809, predicted reward 26.442063,predicted upper bound 26.444972,actual reward 20.667374
    round 3810, predicted reward 22.414438,predicted upper bound 22.417900,actual reward 19.960851
    round 3811, predicted reward 37.811079,predicted upper bound 37.814438,actual reward 41.192612
    round 3812, predicted reward 25.892067,predicted upper bound 25.896114,actual reward 23.650162
    round 3813, predicted reward 29.317277,predicted upper bound 29.321085,actual reward 31.976938
    round 3814, predicted reward 35.105844,predicted upper bound 35.109074,actual reward 35.730631
    round 3815, predicted reward 29.459465,predicted upper bound 29.463779,actual reward 28.036970
    round 3816, predicted reward 22.949815,predicted upper bound 22.953645,actual reward 18.721727
    round 3817, predicted reward 23.689760,predicted upper bound 23.693848,actual reward 17.816109
    round 3818, predicted reward 26.873257,predicted upper bound 26.877410,actual reward 27.349659
    round 3819, predicted reward 28.869404,predicted upper bound 28.873353,actual reward 30.352086
    round 3820, predicted reward 21.825414,predicted upper bound 21.829942,actual reward 17.532876
    round 3821, predicted reward 30.118507,predicted upper bound 30.122044,actual reward 28.585973
    round 3822, predicted reward 23.188142,predicted upper bound 23.192449,actual reward 23.427672
    round 3823, predicted reward 22.737853,predicted upper bound 22.741267,actual reward 26.275299
    round 3824, predicted reward 28.141569,predicted upper bound 28.145673,actual reward 23.573595
    round 3825, predicted reward 24.554121,predicted upper bound 24.557656,actual reward 25.146139
    round 3826, predicted reward 29.336740,predicted upper bound 29.340932,actual reward 27.107246
    round 3827, predicted reward 27.608448,predicted upper bound 27.612310,actual reward 29.954945
    round 3828, predicted reward 22.038310,predicted upper bound 22.042699,actual reward 21.656998
    round 3829, predicted reward 22.379800,predicted upper bound 22.383309,actual reward 17.289814
    round 3830, predicted reward 27.779104,predicted upper bound 27.782748,actual reward 29.567062
    round 3831, predicted reward 26.928324,predicted upper bound 26.931788,actual reward 24.814062
    round 3832, predicted reward 25.285390,predicted upper bound 25.289889,actual reward 22.037944
    round 3833, predicted reward 23.618298,predicted upper bound 23.622929,actual reward 24.777435
    round 3834, predicted reward 20.947958,predicted upper bound 20.952605,actual reward 20.236075
    round 3835, predicted reward 30.819985,predicted upper bound 30.823623,actual reward 35.734125
    round 3836, predicted reward 26.972238,predicted upper bound 26.975915,actual reward 24.908242
    round 3837, predicted reward 30.147581,predicted upper bound 30.150735,actual reward 33.283025
    round 3838, predicted reward 27.618728,predicted upper bound 27.621844,actual reward 27.474283
    round 3839, predicted reward 22.860822,predicted upper bound 22.865292,actual reward 25.789679
    round 3840, predicted reward 30.419471,predicted upper bound 30.422780,actual reward 32.170882
    round 3841, predicted reward 29.505467,predicted upper bound 29.509453,actual reward 29.531497
    round 3842, predicted reward 25.874637,predicted upper bound 25.878450,actual reward 21.435367
    round 3843, predicted reward 26.968395,predicted upper bound 26.972055,actual reward 27.679113
    round 3844, predicted reward 28.387012,predicted upper bound 28.390403,actual reward 26.988111
    round 3845, predicted reward 21.101822,predicted upper bound 21.105795,actual reward 16.914895
    round 3846, predicted reward 33.381913,predicted upper bound 33.385603,actual reward 39.334292
    round 3847, predicted reward 27.100184,predicted upper bound 27.104209,actual reward 24.149638
    round 3848, predicted reward 30.119772,predicted upper bound 30.123030,actual reward 28.135578
    round 3849, predicted reward 33.328802,predicted upper bound 33.332219,actual reward 37.359648
    round 3850, predicted reward 31.585382,predicted upper bound 31.588781,actual reward 34.829520
    round 3851, predicted reward 29.932047,predicted upper bound 29.936371,actual reward 31.319225
    round 3852, predicted reward 26.695448,predicted upper bound 26.699864,actual reward 26.178170
    round 3853, predicted reward 30.110500,predicted upper bound 30.113930,actual reward 31.997719
    round 3854, predicted reward 33.657012,predicted upper bound 33.661074,actual reward 37.923010
    round 3855, predicted reward 29.830191,predicted upper bound 29.833847,actual reward 28.932549
    round 3856, predicted reward 31.616428,predicted upper bound 31.620583,actual reward 37.530819
    round 3857, predicted reward 29.963828,predicted upper bound 29.967417,actual reward 28.582865
    round 3858, predicted reward 33.526067,predicted upper bound 33.529745,actual reward 34.072612
    round 3859, predicted reward 25.262788,predicted upper bound 25.266738,actual reward 27.614132
    round 3860, predicted reward 23.160435,predicted upper bound 23.164497,actual reward 19.123802
    round 3861, predicted reward 26.741840,predicted upper bound 26.745479,actual reward 25.615641
    round 3862, predicted reward 34.268464,predicted upper bound 34.271919,actual reward 36.163468
    round 3863, predicted reward 33.633549,predicted upper bound 33.637531,actual reward 33.040724
    round 3864, predicted reward 22.849928,predicted upper bound 22.854520,actual reward 25.437084
    round 3865, predicted reward 25.332430,predicted upper bound 25.336612,actual reward 21.937463
    round 3866, predicted reward 36.877797,predicted upper bound 36.881113,actual reward 41.632656
    round 3867, predicted reward 26.830930,predicted upper bound 26.833956,actual reward 30.413582
    round 3868, predicted reward 32.142415,predicted upper bound 32.145867,actual reward 33.684374
    round 3869, predicted reward 26.643155,predicted upper bound 26.647271,actual reward 19.443475
    round 3870, predicted reward 30.046267,predicted upper bound 30.049423,actual reward 26.731278
    round 3871, predicted reward 29.819755,predicted upper bound 29.823086,actual reward 32.844394
    round 3872, predicted reward 27.948523,predicted upper bound 27.952086,actual reward 31.030660
    round 3873, predicted reward 25.981304,predicted upper bound 25.984293,actual reward 26.430878
    round 3874, predicted reward 28.858694,predicted upper bound 28.861832,actual reward 27.659893
    round 3875, predicted reward 27.021376,predicted upper bound 27.024828,actual reward 33.111020
    round 3876, predicted reward 27.745142,predicted upper bound 27.748764,actual reward 25.973530
    round 3877, predicted reward 30.022817,predicted upper bound 30.026941,actual reward 33.051011
    round 3878, predicted reward 25.950528,predicted upper bound 25.954324,actual reward 26.330939
    round 3879, predicted reward 26.385278,predicted upper bound 26.388792,actual reward 28.971822
    round 3880, predicted reward 24.161908,predicted upper bound 24.165665,actual reward 18.651224
    round 3881, predicted reward 20.715351,predicted upper bound 20.719624,actual reward 23.780070
    round 3882, predicted reward 27.833970,predicted upper bound 27.837626,actual reward 30.446262
    round 3883, predicted reward 23.466975,predicted upper bound 23.470874,actual reward 17.690391
    round 3884, predicted reward 31.324827,predicted upper bound 31.329106,actual reward 34.322429
    round 3885, predicted reward 28.143294,predicted upper bound 28.147039,actual reward 28.601126
    round 3886, predicted reward 24.374096,predicted upper bound 24.377649,actual reward 22.722085
    round 3887, predicted reward 25.084372,predicted upper bound 25.087736,actual reward 25.968776
    round 3888, predicted reward 31.390080,predicted upper bound 31.393619,actual reward 32.258273
    round 3889, predicted reward 23.377180,predicted upper bound 23.381028,actual reward 22.662152
    round 3890, predicted reward 26.408887,predicted upper bound 26.412794,actual reward 29.631938
    round 3891, predicted reward 24.534633,predicted upper bound 24.538474,actual reward 24.649368
    round 3892, predicted reward 20.562747,predicted upper bound 20.566641,actual reward 16.887048
    round 3893, predicted reward 27.227344,predicted upper bound 27.231266,actual reward 25.295754
    round 3894, predicted reward 25.899868,predicted upper bound 25.903481,actual reward 25.736821
    round 3895, predicted reward 27.137642,predicted upper bound 27.141779,actual reward 23.224825
    round 3896, predicted reward 28.333243,predicted upper bound 28.336362,actual reward 28.834734
    round 3897, predicted reward 27.972845,predicted upper bound 27.976274,actual reward 30.461925
    round 3898, predicted reward 23.126405,predicted upper bound 23.129958,actual reward 23.563282
    round 3899, predicted reward 29.377038,predicted upper bound 29.381068,actual reward 26.773988
    round 3900, predicted reward 30.680133,predicted upper bound 30.683299,actual reward 30.144038
    round 3901, predicted reward 29.864692,predicted upper bound 29.868437,actual reward 30.215369
    round 3902, predicted reward 30.680880,predicted upper bound 30.684117,actual reward 28.543598
    round 3903, predicted reward 31.349917,predicted upper bound 31.353638,actual reward 31.381646
    round 3904, predicted reward 23.063237,predicted upper bound 23.067199,actual reward 16.351095
    round 3905, predicted reward 28.568261,predicted upper bound 28.571989,actual reward 23.936005
    round 3906, predicted reward 30.361250,predicted upper bound 30.365194,actual reward 32.375246
    round 3907, predicted reward 32.037578,predicted upper bound 32.041196,actual reward 35.832757
    round 3908, predicted reward 28.510271,predicted upper bound 28.513910,actual reward 35.197857
    round 3909, predicted reward 32.020938,predicted upper bound 32.024598,actual reward 33.569401
    round 3910, predicted reward 31.013806,predicted upper bound 31.017693,actual reward 31.760160
    round 3911, predicted reward 34.024005,predicted upper bound 34.027159,actual reward 35.521670
    round 3912, predicted reward 20.371858,predicted upper bound 20.376370,actual reward 17.005392
    round 3913, predicted reward 27.286440,predicted upper bound 27.289636,actual reward 23.687602
    round 3914, predicted reward 24.987569,predicted upper bound 24.991234,actual reward 20.111421
    round 3915, predicted reward 26.161194,predicted upper bound 26.165081,actual reward 21.002170
    round 3916, predicted reward 34.902883,predicted upper bound 34.906256,actual reward 38.533867
    round 3917, predicted reward 26.478670,predicted upper bound 26.482568,actual reward 30.755253
    round 3918, predicted reward 27.972848,predicted upper bound 27.976568,actual reward 24.352362
    round 3919, predicted reward 31.282756,predicted upper bound 31.286447,actual reward 33.214193
    round 3920, predicted reward 33.635499,predicted upper bound 33.638454,actual reward 37.324235
    round 3921, predicted reward 23.257166,predicted upper bound 23.260143,actual reward 17.658751
    round 3922, predicted reward 21.449077,predicted upper bound 21.452557,actual reward 19.877017
    round 3923, predicted reward 22.989871,predicted upper bound 22.993844,actual reward 21.649304
    round 3924, predicted reward 26.621349,predicted upper bound 26.625624,actual reward 28.213582
    round 3925, predicted reward 23.752578,predicted upper bound 23.755733,actual reward 20.265509
    round 3926, predicted reward 29.293961,predicted upper bound 29.297865,actual reward 25.085507
    round 3927, predicted reward 27.454049,predicted upper bound 27.458335,actual reward 25.371381
    round 3928, predicted reward 25.074784,predicted upper bound 25.078980,actual reward 23.846492
    round 3929, predicted reward 27.896876,predicted upper bound 27.900756,actual reward 30.685948
    round 3930, predicted reward 21.654628,predicted upper bound 21.658738,actual reward 17.219491
    round 3931, predicted reward 26.565124,predicted upper bound 26.569030,actual reward 21.970620
    round 3932, predicted reward 26.019072,predicted upper bound 26.022947,actual reward 30.722955
    round 3933, predicted reward 27.539497,predicted upper bound 27.543706,actual reward 30.814193
    round 3934, predicted reward 24.875755,predicted upper bound 24.879750,actual reward 23.790932
    round 3935, predicted reward 26.047847,predicted upper bound 26.052386,actual reward 27.838446
    round 3936, predicted reward 27.423003,predicted upper bound 27.426541,actual reward 23.157517
    round 3937, predicted reward 27.029422,predicted upper bound 27.033193,actual reward 22.324091
    round 3938, predicted reward 32.436079,predicted upper bound 32.438930,actual reward 31.858492
    round 3939, predicted reward 30.397960,predicted upper bound 30.401540,actual reward 31.572633
    round 3940, predicted reward 27.758164,predicted upper bound 27.761842,actual reward 25.313163
    round 3941, predicted reward 31.392473,predicted upper bound 31.396119,actual reward 35.849083
    round 3942, predicted reward 30.753988,predicted upper bound 30.757687,actual reward 28.267728
    round 3943, predicted reward 32.460215,predicted upper bound 32.463917,actual reward 27.732246
    round 3944, predicted reward 20.566341,predicted upper bound 20.570941,actual reward 15.609420
    round 3945, predicted reward 30.650268,predicted upper bound 30.653601,actual reward 27.352272
    round 3946, predicted reward 24.155513,predicted upper bound 24.159331,actual reward 18.452289
    round 3947, predicted reward 26.560008,predicted upper bound 26.563914,actual reward 24.782136
    round 3948, predicted reward 27.315740,predicted upper bound 27.319524,actual reward 27.626067
    round 3949, predicted reward 33.898771,predicted upper bound 33.902137,actual reward 38.774665
    round 3950, predicted reward 24.230981,predicted upper bound 24.234929,actual reward 18.875440
    round 3951, predicted reward 23.383400,predicted upper bound 23.387150,actual reward 23.212750
    round 3952, predicted reward 24.595467,predicted upper bound 24.599163,actual reward 22.121532
    round 3953, predicted reward 25.214747,predicted upper bound 25.218131,actual reward 20.128142
    round 3954, predicted reward 26.980082,predicted upper bound 26.984041,actual reward 25.045288
    round 3955, predicted reward 24.514819,predicted upper bound 24.518204,actual reward 24.540625
    round 3956, predicted reward 29.073310,predicted upper bound 29.077193,actual reward 29.659902
    round 3957, predicted reward 26.988288,predicted upper bound 26.991862,actual reward 23.514969
    round 3958, predicted reward 22.253006,predicted upper bound 22.257370,actual reward 22.752127
    round 3959, predicted reward 26.275550,predicted upper bound 26.279416,actual reward 25.376078
    round 3960, predicted reward 27.194553,predicted upper bound 27.198271,actual reward 28.765685
    round 3961, predicted reward 21.695107,predicted upper bound 21.699678,actual reward 20.803996
    round 3962, predicted reward 32.331344,predicted upper bound 32.334923,actual reward 30.914940
    round 3963, predicted reward 25.710806,predicted upper bound 25.715180,actual reward 22.125559
    round 3964, predicted reward 26.120604,predicted upper bound 26.124756,actual reward 25.575317
    round 3965, predicted reward 27.375785,predicted upper bound 27.379545,actual reward 23.209575
    round 3966, predicted reward 28.104159,predicted upper bound 28.108112,actual reward 27.191576
    round 3967, predicted reward 26.916968,predicted upper bound 26.920784,actual reward 27.499333
    round 3968, predicted reward 31.226368,predicted upper bound 31.229778,actual reward 33.017518
    round 3969, predicted reward 23.756608,predicted upper bound 23.760263,actual reward 20.427807
    round 3970, predicted reward 28.059681,predicted upper bound 28.063801,actual reward 29.761683
    round 3971, predicted reward 22.414701,predicted upper bound 22.418896,actual reward 18.088186
    round 3972, predicted reward 23.960123,predicted upper bound 23.964175,actual reward 20.081219
    round 3973, predicted reward 24.259429,predicted upper bound 24.263252,actual reward 20.899899
    round 3974, predicted reward 20.093573,predicted upper bound 20.096864,actual reward 15.093915
    round 3975, predicted reward 26.693928,predicted upper bound 26.697449,actual reward 28.330706
    round 3976, predicted reward 29.555978,predicted upper bound 29.559465,actual reward 31.686860
    round 3977, predicted reward 31.684948,predicted upper bound 31.689217,actual reward 32.675689
    round 3978, predicted reward 28.540704,predicted upper bound 28.544910,actual reward 27.135283
    round 3979, predicted reward 21.630628,predicted upper bound 21.634771,actual reward 19.898472
    round 3980, predicted reward 29.279906,predicted upper bound 29.283843,actual reward 30.869694
    round 3981, predicted reward 21.468420,predicted upper bound 21.472418,actual reward 19.679721
    round 3982, predicted reward 23.857343,predicted upper bound 23.861345,actual reward 16.113588
    round 3983, predicted reward 24.586788,predicted upper bound 24.591327,actual reward 21.479645
    round 3984, predicted reward 31.670243,predicted upper bound 31.673392,actual reward 28.880547
    round 3985, predicted reward 22.007598,predicted upper bound 22.011823,actual reward 21.582837
    round 3986, predicted reward 25.826923,predicted upper bound 25.830266,actual reward 22.234141
    round 3987, predicted reward 29.830825,predicted upper bound 29.833613,actual reward 30.350716
    round 3988, predicted reward 23.760156,predicted upper bound 23.764032,actual reward 17.301203
    round 3989, predicted reward 29.501699,predicted upper bound 29.504685,actual reward 27.927078
    round 3990, predicted reward 22.049888,predicted upper bound 22.054260,actual reward 26.334854
    round 3991, predicted reward 30.108201,predicted upper bound 30.112485,actual reward 31.279028
    round 3992, predicted reward 30.053325,predicted upper bound 30.056380,actual reward 29.928584
    round 3993, predicted reward 25.626562,predicted upper bound 25.629974,actual reward 26.865187
    round 3994, predicted reward 21.945896,predicted upper bound 21.949573,actual reward 16.700786
    round 3995, predicted reward 24.600291,predicted upper bound 24.604623,actual reward 26.361102
    round 3996, predicted reward 21.184344,predicted upper bound 21.188817,actual reward 21.308627
    round 3997, predicted reward 26.432439,predicted upper bound 26.436648,actual reward 24.527608
    round 3998, predicted reward 29.726946,predicted upper bound 29.729666,actual reward 34.215108
    round 3999, predicted reward 26.283629,predicted upper bound 26.287363,actual reward 27.317403
    round 4000, predicted reward 24.404166,predicted upper bound 24.407989,actual reward 23.842965
    round 4001, predicted reward 21.811624,predicted upper bound 21.815528,actual reward 15.744375
    round 4002, predicted reward 23.591782,predicted upper bound 23.595657,actual reward 25.737672
    round 4003, predicted reward 27.409045,predicted upper bound 27.412604,actual reward 22.972703
    round 4004, predicted reward 35.225317,predicted upper bound 35.228935,actual reward 36.567045
    round 4005, predicted reward 27.853548,predicted upper bound 27.856615,actual reward 21.832390
    round 4006, predicted reward 27.240831,predicted upper bound 27.244811,actual reward 23.704195
    round 4007, predicted reward 24.368850,predicted upper bound 24.372371,actual reward 28.103332
    round 4008, predicted reward 30.649711,predicted upper bound 30.653279,actual reward 32.810060
    round 4009, predicted reward 25.818266,predicted upper bound 25.821891,actual reward 22.899550
    round 4010, predicted reward 25.132515,predicted upper bound 25.136348,actual reward 23.854832
    round 4011, predicted reward 19.838927,predicted upper bound 19.842969,actual reward 17.396192
    round 4012, predicted reward 28.631070,predicted upper bound 28.634188,actual reward 32.143603
    round 4013, predicted reward 30.005878,predicted upper bound 30.009470,actual reward 30.120697
    round 4014, predicted reward 28.865375,predicted upper bound 28.869273,actual reward 29.280531
    round 4015, predicted reward 26.442848,predicted upper bound 26.446921,actual reward 23.032539
    round 4016, predicted reward 27.310444,predicted upper bound 27.314417,actual reward 27.758315
    round 4017, predicted reward 28.077939,predicted upper bound 28.081073,actual reward 25.900574
    round 4018, predicted reward 27.414170,predicted upper bound 27.417349,actual reward 28.222189
    round 4019, predicted reward 22.603276,predicted upper bound 22.607295,actual reward 22.258085
    round 4020, predicted reward 27.245036,predicted upper bound 27.248595,actual reward 25.122155
    round 4021, predicted reward 32.106654,predicted upper bound 32.109916,actual reward 31.016035
    round 4022, predicted reward 28.446579,predicted upper bound 28.449565,actual reward 27.238744
    round 4023, predicted reward 22.544711,predicted upper bound 22.549077,actual reward 23.342858
    round 4024, predicted reward 23.117183,predicted upper bound 23.121319,actual reward 18.970428
    round 4025, predicted reward 28.401982,predicted upper bound 28.405969,actual reward 23.619005
    round 4026, predicted reward 23.965904,predicted upper bound 23.969963,actual reward 22.034089
    round 4027, predicted reward 26.863029,predicted upper bound 26.867056,actual reward 30.831807
    round 4028, predicted reward 26.345095,predicted upper bound 26.348080,actual reward 25.829920
    round 4029, predicted reward 27.150047,predicted upper bound 27.153860,actual reward 24.890648
    round 4030, predicted reward 22.027537,predicted upper bound 22.031903,actual reward 20.082336
    round 4031, predicted reward 31.983380,predicted upper bound 31.987574,actual reward 29.738782
    round 4032, predicted reward 31.716084,predicted upper bound 31.719839,actual reward 31.261296
    round 4033, predicted reward 27.631185,predicted upper bound 27.635078,actual reward 26.472733
    round 4034, predicted reward 23.576236,predicted upper bound 23.580049,actual reward 20.707464
    round 4035, predicted reward 24.501051,predicted upper bound 24.505164,actual reward 21.206452
    round 4036, predicted reward 29.329754,predicted upper bound 29.333187,actual reward 27.903403
    round 4037, predicted reward 27.841270,predicted upper bound 27.845189,actual reward 28.979087
    round 4038, predicted reward 29.662327,predicted upper bound 29.666059,actual reward 30.068259
    round 4039, predicted reward 27.957691,predicted upper bound 27.960929,actual reward 29.183401
    round 4040, predicted reward 31.566314,predicted upper bound 31.570347,actual reward 28.681730
    round 4041, predicted reward 27.468561,predicted upper bound 27.472705,actual reward 26.404403
    round 4042, predicted reward 30.616433,predicted upper bound 30.620437,actual reward 33.233911
    round 4043, predicted reward 31.002502,predicted upper bound 31.006219,actual reward 34.993012
    round 4044, predicted reward 23.908694,predicted upper bound 23.912318,actual reward 22.842373
    round 4045, predicted reward 26.336620,predicted upper bound 26.340191,actual reward 22.846869
    round 4046, predicted reward 27.963731,predicted upper bound 27.967911,actual reward 22.054619
    round 4047, predicted reward 19.747557,predicted upper bound 19.751049,actual reward 19.469521
    round 4048, predicted reward 27.825458,predicted upper bound 27.828799,actual reward 27.002109
    round 4049, predicted reward 23.505722,predicted upper bound 23.509641,actual reward 24.981102
    round 4050, predicted reward 22.332678,predicted upper bound 22.336863,actual reward 21.458366
    round 4051, predicted reward 29.758717,predicted upper bound 29.762606,actual reward 32.001697
    round 4052, predicted reward 21.132830,predicted upper bound 21.136137,actual reward 22.080277
    round 4053, predicted reward 29.415556,predicted upper bound 29.419080,actual reward 26.830067
    round 4054, predicted reward 25.976669,predicted upper bound 25.980999,actual reward 33.446048
    round 4055, predicted reward 28.545310,predicted upper bound 28.549756,actual reward 32.164887
    round 4056, predicted reward 24.118165,predicted upper bound 24.122559,actual reward 17.485021
    round 4057, predicted reward 26.835229,predicted upper bound 26.839397,actual reward 28.118475
    round 4058, predicted reward 23.772706,predicted upper bound 23.776638,actual reward 24.553004
    round 4059, predicted reward 23.538649,predicted upper bound 23.542264,actual reward 20.186978
    round 4060, predicted reward 24.901487,predicted upper bound 24.905085,actual reward 20.198912
    round 4061, predicted reward 34.975122,predicted upper bound 34.978020,actual reward 38.170903
    round 4062, predicted reward 19.140351,predicted upper bound 19.144228,actual reward 15.667697
    round 4063, predicted reward 32.540699,predicted upper bound 32.544561,actual reward 30.867999
    round 4064, predicted reward 27.041691,predicted upper bound 27.046216,actual reward 22.425035
    round 4065, predicted reward 20.608033,predicted upper bound 20.612614,actual reward 13.738337
    round 4066, predicted reward 27.531291,predicted upper bound 27.535046,actual reward 27.357371
    round 4067, predicted reward 30.784564,predicted upper bound 30.788111,actual reward 32.783947
    round 4068, predicted reward 27.580028,predicted upper bound 27.583984,actual reward 26.430938
    round 4069, predicted reward 23.335331,predicted upper bound 23.339755,actual reward 21.257443
    round 4070, predicted reward 34.450673,predicted upper bound 34.454015,actual reward 31.884196
    round 4071, predicted reward 32.517263,predicted upper bound 32.521101,actual reward 33.622836
    round 4072, predicted reward 29.385363,predicted upper bound 29.388018,actual reward 27.851750
    round 4073, predicted reward 33.128442,predicted upper bound 33.131796,actual reward 24.815220
    round 4074, predicted reward 27.250820,predicted upper bound 27.255009,actual reward 24.489094
    round 4075, predicted reward 28.144341,predicted upper bound 28.147807,actual reward 27.078848
    round 4076, predicted reward 28.592002,predicted upper bound 28.596148,actual reward 23.095879
    round 4077, predicted reward 27.857900,predicted upper bound 27.861782,actual reward 26.491405
    round 4078, predicted reward 32.062850,predicted upper bound 32.066499,actual reward 24.929547
    round 4079, predicted reward 24.991596,predicted upper bound 24.994650,actual reward 19.912197
    round 4080, predicted reward 28.062380,predicted upper bound 28.065176,actual reward 23.827845
    round 4081, predicted reward 26.566107,predicted upper bound 26.569804,actual reward 30.222876
    round 4082, predicted reward 26.190318,predicted upper bound 26.194120,actual reward 27.347061
    round 4083, predicted reward 27.529903,predicted upper bound 27.533684,actual reward 25.544983
    round 4084, predicted reward 31.211693,predicted upper bound 31.215339,actual reward 32.555349
    round 4085, predicted reward 27.938310,predicted upper bound 27.941791,actual reward 30.698154
    round 4086, predicted reward 30.635816,predicted upper bound 30.639091,actual reward 35.365746
    round 4087, predicted reward 25.869873,predicted upper bound 25.873623,actual reward 30.238455
    round 4088, predicted reward 21.779750,predicted upper bound 21.783265,actual reward 20.693308
    round 4089, predicted reward 29.576563,predicted upper bound 29.580332,actual reward 27.830139
    round 4090, predicted reward 22.452993,predicted upper bound 22.456823,actual reward 20.163935
    round 4091, predicted reward 28.523257,predicted upper bound 28.527540,actual reward 26.951336
    round 4092, predicted reward 25.812632,predicted upper bound 25.816656,actual reward 26.296358
    round 4093, predicted reward 26.226491,predicted upper bound 26.229893,actual reward 21.189480
    round 4094, predicted reward 27.972849,predicted upper bound 27.976713,actual reward 30.659962
    round 4095, predicted reward 26.214430,predicted upper bound 26.218169,actual reward 19.964471
    round 4096, predicted reward 27.407367,predicted upper bound 27.411212,actual reward 25.107843
    round 4097, predicted reward 27.719735,predicted upper bound 27.722976,actual reward 23.812575
    round 4098, predicted reward 34.161260,predicted upper bound 34.164531,actual reward 29.667531
    round 4099, predicted reward 25.162211,predicted upper bound 25.166443,actual reward 20.662068
    round 4100, predicted reward 24.054665,predicted upper bound 24.059224,actual reward 25.598712
    round 4101, predicted reward 22.784463,predicted upper bound 22.788503,actual reward 22.571448
    round 4102, predicted reward 27.879943,predicted upper bound 27.882974,actual reward 28.812858
    round 4103, predicted reward 26.268292,predicted upper bound 26.271982,actual reward 27.562912
    round 4104, predicted reward 33.670710,predicted upper bound 33.674259,actual reward 32.607938
    round 4105, predicted reward 26.436181,predicted upper bound 26.439949,actual reward 26.973366
    round 4106, predicted reward 28.794576,predicted upper bound 28.798315,actual reward 27.744147
    round 4107, predicted reward 31.049320,predicted upper bound 31.053099,actual reward 27.692507
    round 4108, predicted reward 23.345423,predicted upper bound 23.349621,actual reward 22.535591
    round 4109, predicted reward 25.886319,predicted upper bound 25.889836,actual reward 27.902489
    round 4110, predicted reward 27.200400,predicted upper bound 27.203687,actual reward 26.411091
    round 4111, predicted reward 26.502250,predicted upper bound 26.505461,actual reward 27.444668
    round 4112, predicted reward 25.074735,predicted upper bound 25.078570,actual reward 21.109601
    round 4113, predicted reward 28.890314,predicted upper bound 28.893807,actual reward 27.544900
    round 4114, predicted reward 29.929633,predicted upper bound 29.933796,actual reward 26.815844
    round 4115, predicted reward 27.857604,predicted upper bound 27.861112,actual reward 25.574087
    round 4116, predicted reward 29.545968,predicted upper bound 29.549824,actual reward 28.216873
    round 4117, predicted reward 26.031194,predicted upper bound 26.035615,actual reward 24.817212
    round 4118, predicted reward 23.705263,predicted upper bound 23.709144,actual reward 22.135841
    round 4119, predicted reward 25.860826,predicted upper bound 25.864561,actual reward 18.874196
    round 4120, predicted reward 34.583465,predicted upper bound 34.587262,actual reward 38.589381
    round 4121, predicted reward 25.785027,predicted upper bound 25.788776,actual reward 26.950484
    round 4122, predicted reward 25.054115,predicted upper bound 25.058393,actual reward 19.488839
    round 4123, predicted reward 28.474278,predicted upper bound 28.478251,actual reward 27.157125
    round 4124, predicted reward 32.538755,predicted upper bound 32.542254,actual reward 27.696814
    round 4125, predicted reward 29.748814,predicted upper bound 29.752600,actual reward 23.313296
    round 4126, predicted reward 23.290497,predicted upper bound 23.294854,actual reward 21.897658
    round 4127, predicted reward 22.384245,predicted upper bound 22.388035,actual reward 20.515820
    round 4128, predicted reward 29.874162,predicted upper bound 29.878484,actual reward 25.838851
    round 4129, predicted reward 27.417382,predicted upper bound 27.421105,actual reward 23.343695
    round 4130, predicted reward 20.916575,predicted upper bound 20.920783,actual reward 13.605539
    round 4131, predicted reward 32.566563,predicted upper bound 32.570249,actual reward 33.030055
    round 4132, predicted reward 26.059780,predicted upper bound 26.063631,actual reward 22.762501
    round 4133, predicted reward 20.543495,predicted upper bound 20.547294,actual reward 16.613413
    round 4134, predicted reward 29.457380,predicted upper bound 29.461615,actual reward 31.911966
    round 4135, predicted reward 26.843687,predicted upper bound 26.847588,actual reward 26.200127
    round 4136, predicted reward 27.545351,predicted upper bound 27.549175,actual reward 28.295695
    round 4137, predicted reward 21.298681,predicted upper bound 21.302543,actual reward 16.839681
    round 4138, predicted reward 24.502593,predicted upper bound 24.506299,actual reward 25.611673
    round 4139, predicted reward 31.237399,predicted upper bound 31.240801,actual reward 25.955028
    round 4140, predicted reward 22.263114,predicted upper bound 22.266991,actual reward 25.372708
    round 4141, predicted reward 25.069088,predicted upper bound 25.072901,actual reward 24.450432
    round 4142, predicted reward 18.249763,predicted upper bound 18.253858,actual reward 20.157297
    round 4143, predicted reward 32.122968,predicted upper bound 32.126962,actual reward 31.649589
    round 4144, predicted reward 22.918570,predicted upper bound 22.922423,actual reward 19.586578
    round 4145, predicted reward 25.415028,predicted upper bound 25.419067,actual reward 25.282302
    round 4146, predicted reward 23.584348,predicted upper bound 23.587721,actual reward 25.709355
    round 4147, predicted reward 32.773561,predicted upper bound 32.776959,actual reward 35.600237
    round 4148, predicted reward 23.669197,predicted upper bound 23.672172,actual reward 23.408726
    round 4149, predicted reward 22.192854,predicted upper bound 22.196828,actual reward 16.235990
    round 4150, predicted reward 38.229140,predicted upper bound 38.232526,actual reward 42.910523
    round 4151, predicted reward 33.466472,predicted upper bound 33.470188,actual reward 33.300373
    round 4152, predicted reward 25.298093,predicted upper bound 25.301267,actual reward 27.240387
    round 4153, predicted reward 23.825691,predicted upper bound 23.829155,actual reward 24.732281
    round 4154, predicted reward 29.911752,predicted upper bound 29.915182,actual reward 33.709769
    round 4155, predicted reward 20.327116,predicted upper bound 20.331421,actual reward 20.607644
    round 4156, predicted reward 25.172475,predicted upper bound 25.176686,actual reward 25.307983
    round 4157, predicted reward 23.992395,predicted upper bound 23.996212,actual reward 19.469525
    round 4158, predicted reward 24.957433,predicted upper bound 24.961473,actual reward 19.087255
    round 4159, predicted reward 26.434489,predicted upper bound 26.437691,actual reward 27.518280
    round 4160, predicted reward 26.356530,predicted upper bound 26.360818,actual reward 22.160006
    round 4161, predicted reward 30.897197,predicted upper bound 30.900086,actual reward 28.190807
    round 4162, predicted reward 29.593354,predicted upper bound 29.597134,actual reward 30.220973
    round 4163, predicted reward 29.770721,predicted upper bound 29.774064,actual reward 27.503788
    round 4164, predicted reward 22.539395,predicted upper bound 22.543303,actual reward 24.615519
    round 4165, predicted reward 29.456120,predicted upper bound 29.459749,actual reward 27.708277
    round 4166, predicted reward 21.211228,predicted upper bound 21.214485,actual reward 21.467685
    round 4167, predicted reward 27.169038,predicted upper bound 27.172638,actual reward 26.732894
    round 4168, predicted reward 30.167205,predicted upper bound 30.170815,actual reward 29.777804
    round 4169, predicted reward 24.611090,predicted upper bound 24.614277,actual reward 20.735241
    round 4170, predicted reward 34.228606,predicted upper bound 34.231438,actual reward 38.006277
    round 4171, predicted reward 24.427379,predicted upper bound 24.431457,actual reward 25.269506
    round 4172, predicted reward 33.519757,predicted upper bound 33.522657,actual reward 32.637661
    round 4173, predicted reward 30.690685,predicted upper bound 30.694453,actual reward 29.636540
    round 4174, predicted reward 26.604158,predicted upper bound 26.607936,actual reward 22.119672
    round 4175, predicted reward 33.006059,predicted upper bound 33.009107,actual reward 35.078992
    round 4176, predicted reward 29.411661,predicted upper bound 29.415502,actual reward 27.256203
    round 4177, predicted reward 28.746225,predicted upper bound 28.749450,actual reward 26.933629
    round 4178, predicted reward 21.633483,predicted upper bound 21.637615,actual reward 19.463213
    round 4179, predicted reward 25.416615,predicted upper bound 25.419731,actual reward 21.641685
    round 4180, predicted reward 22.066208,predicted upper bound 22.069918,actual reward 26.087449
    round 4181, predicted reward 20.034116,predicted upper bound 20.038613,actual reward 21.746012
    round 4182, predicted reward 34.083593,predicted upper bound 34.086935,actual reward 37.661718
    round 4183, predicted reward 29.023607,predicted upper bound 29.026814,actual reward 27.305337
    round 4184, predicted reward 33.131075,predicted upper bound 33.134403,actual reward 30.375094
    round 4185, predicted reward 25.743019,predicted upper bound 25.746916,actual reward 27.602969
    round 4186, predicted reward 24.712634,predicted upper bound 24.715977,actual reward 17.579960
    round 4187, predicted reward 31.476172,predicted upper bound 31.479627,actual reward 31.079832
    round 4188, predicted reward 27.501042,predicted upper bound 27.505115,actual reward 22.661048
    round 4189, predicted reward 25.037753,predicted upper bound 25.041527,actual reward 21.301385
    round 4190, predicted reward 24.969908,predicted upper bound 24.973603,actual reward 24.305878
    round 4191, predicted reward 31.569572,predicted upper bound 31.573211,actual reward 34.781789
    round 4192, predicted reward 29.282620,predicted upper bound 29.286363,actual reward 28.224130
    round 4193, predicted reward 27.965146,predicted upper bound 27.968944,actual reward 27.009698
    round 4194, predicted reward 26.603904,predicted upper bound 26.607565,actual reward 21.166158
    round 4195, predicted reward 20.881638,predicted upper bound 20.885089,actual reward 23.598829
    round 4196, predicted reward 23.095435,predicted upper bound 23.099579,actual reward 20.129931
    round 4197, predicted reward 25.705172,predicted upper bound 25.708367,actual reward 25.531199
    round 4198, predicted reward 36.976776,predicted upper bound 36.980319,actual reward 38.810325
    round 4199, predicted reward 32.231226,predicted upper bound 32.234613,actual reward 31.199277
    round 4200, predicted reward 25.459960,predicted upper bound 25.463919,actual reward 18.054663
    round 4201, predicted reward 22.808465,predicted upper bound 22.812274,actual reward 22.023880
    round 4202, predicted reward 24.304334,predicted upper bound 24.308377,actual reward 21.812052
    round 4203, predicted reward 25.358032,predicted upper bound 25.361756,actual reward 25.209357
    round 4204, predicted reward 32.091739,predicted upper bound 32.094777,actual reward 29.703635
    round 4205, predicted reward 22.498060,predicted upper bound 22.501596,actual reward 19.764836
    round 4206, predicted reward 27.865876,predicted upper bound 27.869564,actual reward 23.553934
    round 4207, predicted reward 28.912079,predicted upper bound 28.915111,actual reward 29.284406
    round 4208, predicted reward 26.437340,predicted upper bound 26.440521,actual reward 27.974798
    round 4209, predicted reward 22.122436,predicted upper bound 22.126269,actual reward 23.467258
    round 4210, predicted reward 25.748623,predicted upper bound 25.752472,actual reward 22.683767
    round 4211, predicted reward 29.866054,predicted upper bound 29.869502,actual reward 35.663608
    round 4212, predicted reward 28.652696,predicted upper bound 28.656208,actual reward 21.829953
    round 4213, predicted reward 25.455048,predicted upper bound 25.458508,actual reward 26.706271
    round 4214, predicted reward 32.681072,predicted upper bound 32.684361,actual reward 25.677414
    round 4215, predicted reward 28.105789,predicted upper bound 28.109295,actual reward 31.345131
    round 4216, predicted reward 22.948316,predicted upper bound 22.952003,actual reward 25.269322
    round 4217, predicted reward 25.365791,predicted upper bound 25.369810,actual reward 22.343162
    round 4218, predicted reward 26.365288,predicted upper bound 26.368934,actual reward 32.057530
    round 4219, predicted reward 35.657646,predicted upper bound 35.661170,actual reward 38.807818
    round 4220, predicted reward 29.011487,predicted upper bound 29.015366,actual reward 30.253299
    round 4221, predicted reward 29.956961,predicted upper bound 29.960566,actual reward 31.048988
    round 4222, predicted reward 27.865269,predicted upper bound 27.868788,actual reward 21.363134
    round 4223, predicted reward 30.158340,predicted upper bound 30.161877,actual reward 30.004868
    round 4224, predicted reward 33.918925,predicted upper bound 33.922202,actual reward 33.840283
    round 4225, predicted reward 33.798395,predicted upper bound 33.801686,actual reward 34.253005
    round 4226, predicted reward 22.585880,predicted upper bound 22.589062,actual reward 25.166850
    round 4227, predicted reward 31.358060,predicted upper bound 31.360927,actual reward 32.241841
    round 4228, predicted reward 30.888682,predicted upper bound 30.892115,actual reward 31.153447
    round 4229, predicted reward 22.937168,predicted upper bound 22.940650,actual reward 20.840714
    round 4230, predicted reward 28.468293,predicted upper bound 28.471887,actual reward 28.987186
    round 4231, predicted reward 25.535295,predicted upper bound 25.539002,actual reward 26.377685
    round 4232, predicted reward 24.762618,predicted upper bound 24.765923,actual reward 25.712017
    round 4233, predicted reward 26.774524,predicted upper bound 26.778332,actual reward 23.196035
    round 4234, predicted reward 30.554292,predicted upper bound 30.557196,actual reward 26.172262
    round 4235, predicted reward 26.871713,predicted upper bound 26.875054,actual reward 26.316277
    round 4236, predicted reward 21.880157,predicted upper bound 21.884099,actual reward 17.287151
    round 4237, predicted reward 33.173778,predicted upper bound 33.177412,actual reward 33.228938
    round 4238, predicted reward 23.480922,predicted upper bound 23.484856,actual reward 16.732272
    round 4239, predicted reward 25.007993,predicted upper bound 25.011974,actual reward 23.791882
    round 4240, predicted reward 23.621065,predicted upper bound 23.624688,actual reward 16.036211
    round 4241, predicted reward 30.580928,predicted upper bound 30.584787,actual reward 28.940222
    round 4242, predicted reward 38.097346,predicted upper bound 38.100556,actual reward 40.862769
    round 4243, predicted reward 26.238249,predicted upper bound 26.241784,actual reward 31.153717
    round 4244, predicted reward 27.527518,predicted upper bound 27.530937,actual reward 23.501853
    round 4245, predicted reward 31.468016,predicted upper bound 31.471071,actual reward 28.940204
    round 4246, predicted reward 30.478404,predicted upper bound 30.482062,actual reward 30.503788
    round 4247, predicted reward 23.340367,predicted upper bound 23.344353,actual reward 22.847790
    round 4248, predicted reward 24.811133,predicted upper bound 24.814191,actual reward 23.654711
    round 4249, predicted reward 25.500318,predicted upper bound 25.503894,actual reward 27.217921
    round 4250, predicted reward 26.439794,predicted upper bound 26.443007,actual reward 32.875294
    round 4251, predicted reward 22.783803,predicted upper bound 22.787488,actual reward 21.060035
    round 4252, predicted reward 27.539662,predicted upper bound 27.543831,actual reward 25.222575
    round 4253, predicted reward 29.003570,predicted upper bound 29.007222,actual reward 31.996806
    round 4254, predicted reward 24.443862,predicted upper bound 24.447376,actual reward 26.081449
    round 4255, predicted reward 23.338242,predicted upper bound 23.342222,actual reward 24.688098
    round 4256, predicted reward 21.281480,predicted upper bound 21.285302,actual reward 24.434398
    round 4257, predicted reward 29.396634,predicted upper bound 29.400084,actual reward 30.313514
    round 4258, predicted reward 21.695565,predicted upper bound 21.699584,actual reward 17.654550
    round 4259, predicted reward 30.673715,predicted upper bound 30.676986,actual reward 31.178297
    round 4260, predicted reward 26.252858,predicted upper bound 26.256356,actual reward 20.948581
    round 4261, predicted reward 26.954324,predicted upper bound 26.958179,actual reward 27.579226
    round 4262, predicted reward 25.175036,predicted upper bound 25.179222,actual reward 25.428007
    round 4263, predicted reward 29.752123,predicted upper bound 29.755845,actual reward 28.679507
    round 4264, predicted reward 32.789752,predicted upper bound 32.792739,actual reward 33.811997
    round 4265, predicted reward 25.163215,predicted upper bound 25.166781,actual reward 22.543767
    round 4266, predicted reward 26.647384,predicted upper bound 26.651186,actual reward 24.171335
    round 4267, predicted reward 37.026444,predicted upper bound 37.029730,actual reward 43.431882
    round 4268, predicted reward 32.715251,predicted upper bound 32.718474,actual reward 35.896545
    round 4269, predicted reward 24.107484,predicted upper bound 24.111374,actual reward 22.539611
    round 4270, predicted reward 25.262910,predicted upper bound 25.266928,actual reward 26.618246
    round 4271, predicted reward 30.452915,predicted upper bound 30.456687,actual reward 24.573597
    round 4272, predicted reward 28.492660,predicted upper bound 28.495809,actual reward 27.735523
    round 4273, predicted reward 27.807766,predicted upper bound 27.811633,actual reward 27.727185
    round 4274, predicted reward 30.923891,predicted upper bound 30.927337,actual reward 30.635221
    round 4275, predicted reward 26.448218,predicted upper bound 26.452507,actual reward 23.272667
    round 4276, predicted reward 28.821896,predicted upper bound 28.825125,actual reward 32.708097
    round 4277, predicted reward 33.863782,predicted upper bound 33.866909,actual reward 35.046746
    round 4278, predicted reward 32.799970,predicted upper bound 32.803427,actual reward 28.987894
    round 4279, predicted reward 33.869823,predicted upper bound 33.873259,actual reward 33.043093
    round 4280, predicted reward 22.955245,predicted upper bound 22.958890,actual reward 19.915658
    round 4281, predicted reward 24.166759,predicted upper bound 24.170601,actual reward 25.710075
    round 4282, predicted reward 35.267776,predicted upper bound 35.270756,actual reward 36.709702
    round 4283, predicted reward 31.323234,predicted upper bound 31.326607,actual reward 32.716248
    round 4284, predicted reward 25.699105,predicted upper bound 25.702314,actual reward 26.428292
    round 4285, predicted reward 29.213707,predicted upper bound 29.217418,actual reward 26.502633
    round 4286, predicted reward 30.834355,predicted upper bound 30.837873,actual reward 31.799185
    round 4287, predicted reward 28.597326,predicted upper bound 28.600341,actual reward 32.320855
    round 4288, predicted reward 29.147824,predicted upper bound 29.151170,actual reward 23.049482
    round 4289, predicted reward 27.178902,predicted upper bound 27.182335,actual reward 24.645534
    round 4290, predicted reward 29.400528,predicted upper bound 29.403203,actual reward 26.590761
    round 4291, predicted reward 31.872632,predicted upper bound 31.876411,actual reward 27.690865
    round 4292, predicted reward 29.687760,predicted upper bound 29.691225,actual reward 30.001876
    round 4293, predicted reward 37.364077,predicted upper bound 37.366834,actual reward 35.704292
    round 4294, predicted reward 23.775763,predicted upper bound 23.779315,actual reward 21.180635
    round 4295, predicted reward 27.730442,predicted upper bound 27.733654,actual reward 27.997237
    round 4296, predicted reward 28.175232,predicted upper bound 28.178476,actual reward 22.677880
    round 4297, predicted reward 31.096556,predicted upper bound 31.099822,actual reward 32.033760
    round 4298, predicted reward 27.049845,predicted upper bound 27.053550,actual reward 26.666823
    round 4299, predicted reward 22.846914,predicted upper bound 22.850511,actual reward 24.563997
    round 4300, predicted reward 24.169543,predicted upper bound 24.172722,actual reward 24.480964
    round 4301, predicted reward 29.631337,predicted upper bound 29.634843,actual reward 27.722964
    round 4302, predicted reward 38.306929,predicted upper bound 38.309519,actual reward 41.683449
    round 4303, predicted reward 32.071754,predicted upper bound 32.075092,actual reward 33.412717
    round 4304, predicted reward 32.088212,predicted upper bound 32.091515,actual reward 36.583170
    round 4305, predicted reward 28.292982,predicted upper bound 28.296267,actual reward 25.321019
    round 4306, predicted reward 28.762956,predicted upper bound 28.766671,actual reward 24.850862
    round 4307, predicted reward 35.733549,predicted upper bound 35.736221,actual reward 38.551546
    round 4308, predicted reward 29.524855,predicted upper bound 29.528198,actual reward 32.268242
    round 4309, predicted reward 18.524664,predicted upper bound 18.528481,actual reward 13.392241
    round 4310, predicted reward 32.452252,predicted upper bound 32.455705,actual reward 31.807383
    round 4311, predicted reward 24.117512,predicted upper bound 24.121280,actual reward 23.268977
    round 4312, predicted reward 32.213321,predicted upper bound 32.216587,actual reward 30.988964
    round 4313, predicted reward 30.289121,predicted upper bound 30.292550,actual reward 30.484980
    round 4314, predicted reward 23.969606,predicted upper bound 23.973336,actual reward 20.555611
    round 4315, predicted reward 33.443033,predicted upper bound 33.446485,actual reward 32.167015
    round 4316, predicted reward 27.836792,predicted upper bound 27.840353,actual reward 24.081057
    round 4317, predicted reward 28.973951,predicted upper bound 28.977587,actual reward 31.316388
    round 4318, predicted reward 19.534256,predicted upper bound 19.537752,actual reward 21.249221
    round 4319, predicted reward 32.307603,predicted upper bound 32.310640,actual reward 34.074510
    round 4320, predicted reward 24.910904,predicted upper bound 24.915281,actual reward 20.746779
    round 4321, predicted reward 26.485046,predicted upper bound 26.488810,actual reward 20.864327
    round 4322, predicted reward 30.718787,predicted upper bound 30.721764,actual reward 31.681343
    round 4323, predicted reward 28.381035,predicted upper bound 28.384836,actual reward 33.450398
    round 4324, predicted reward 28.665688,predicted upper bound 28.669161,actual reward 26.255734
    round 4325, predicted reward 24.847073,predicted upper bound 24.850842,actual reward 20.457775
    round 4326, predicted reward 24.448368,predicted upper bound 24.451386,actual reward 20.498541
    round 4327, predicted reward 24.557444,predicted upper bound 24.561065,actual reward 20.908419
    round 4328, predicted reward 20.602984,predicted upper bound 20.606428,actual reward 18.933597
    round 4329, predicted reward 23.284514,predicted upper bound 23.288176,actual reward 18.034898
    round 4330, predicted reward 21.769202,predicted upper bound 21.773013,actual reward 21.609492
    round 4331, predicted reward 31.985763,predicted upper bound 31.988816,actual reward 36.794521
    round 4332, predicted reward 27.583330,predicted upper bound 27.586993,actual reward 24.207345
    round 4333, predicted reward 29.761422,predicted upper bound 29.765681,actual reward 30.087038
    round 4334, predicted reward 32.459689,predicted upper bound 32.462756,actual reward 32.151693
    round 4335, predicted reward 24.821507,predicted upper bound 24.825271,actual reward 21.253743
    round 4336, predicted reward 26.150128,predicted upper bound 26.153463,actual reward 25.242710
    round 4337, predicted reward 24.660616,predicted upper bound 24.664066,actual reward 20.830679
    round 4338, predicted reward 17.190608,predicted upper bound 17.193916,actual reward 14.791194
    round 4339, predicted reward 28.424216,predicted upper bound 28.427631,actual reward 27.827864
    round 4340, predicted reward 30.397056,predicted upper bound 30.400578,actual reward 25.691915
    round 4341, predicted reward 35.701729,predicted upper bound 35.705036,actual reward 38.544474
    round 4342, predicted reward 24.838342,predicted upper bound 24.841968,actual reward 22.056556
    round 4343, predicted reward 32.522000,predicted upper bound 32.525264,actual reward 32.165799
    round 4344, predicted reward 21.943464,predicted upper bound 21.946598,actual reward 20.636805
    round 4345, predicted reward 31.323125,predicted upper bound 31.326119,actual reward 32.685124
    round 4346, predicted reward 27.525646,predicted upper bound 27.528893,actual reward 21.867177
    round 4347, predicted reward 22.732218,predicted upper bound 22.736349,actual reward 23.412721
    round 4348, predicted reward 22.489737,predicted upper bound 22.493305,actual reward 22.324217
    round 4349, predicted reward 23.841942,predicted upper bound 23.846085,actual reward 22.259793
    round 4350, predicted reward 29.172152,predicted upper bound 29.175575,actual reward 29.537142
    round 4351, predicted reward 35.352898,predicted upper bound 35.355988,actual reward 39.098729
    round 4352, predicted reward 23.615396,predicted upper bound 23.619126,actual reward 21.192701
    round 4353, predicted reward 33.948979,predicted upper bound 33.951958,actual reward 33.775383
    round 4354, predicted reward 29.183722,predicted upper bound 29.187333,actual reward 31.088966
    round 4355, predicted reward 31.809999,predicted upper bound 31.813510,actual reward 30.054197
    round 4356, predicted reward 26.775185,predicted upper bound 26.777915,actual reward 20.426290
    round 4357, predicted reward 28.526029,predicted upper bound 28.530049,actual reward 27.854854
    round 4358, predicted reward 25.761270,predicted upper bound 25.764324,actual reward 25.324120
    round 4359, predicted reward 29.143351,predicted upper bound 29.146663,actual reward 28.596902
    round 4360, predicted reward 29.141219,predicted upper bound 29.144344,actual reward 29.946857
    round 4361, predicted reward 23.395514,predicted upper bound 23.398870,actual reward 25.005917
    round 4362, predicted reward 28.994364,predicted upper bound 28.996927,actual reward 24.422204
    round 4363, predicted reward 27.133168,predicted upper bound 27.136847,actual reward 26.573686
    round 4364, predicted reward 27.398894,predicted upper bound 27.402539,actual reward 27.062943
    round 4365, predicted reward 27.674108,predicted upper bound 27.677594,actual reward 25.038362
    round 4366, predicted reward 24.362640,predicted upper bound 24.366514,actual reward 21.095590
    round 4367, predicted reward 27.073764,predicted upper bound 27.077175,actual reward 26.683888
    round 4368, predicted reward 28.506105,predicted upper bound 28.509661,actual reward 26.135772
    round 4369, predicted reward 22.483277,predicted upper bound 22.486295,actual reward 20.861106
    round 4370, predicted reward 30.956009,predicted upper bound 30.958788,actual reward 32.935334
    round 4371, predicted reward 22.805322,predicted upper bound 22.809192,actual reward 19.694217
    round 4372, predicted reward 27.950272,predicted upper bound 27.953944,actual reward 28.000998
    round 4373, predicted reward 24.617106,predicted upper bound 24.620729,actual reward 28.189766
    round 4374, predicted reward 23.244477,predicted upper bound 23.248324,actual reward 20.408439
    round 4375, predicted reward 21.889993,predicted upper bound 21.893680,actual reward 15.548684
    round 4376, predicted reward 24.002617,predicted upper bound 24.006250,actual reward 20.874668
    round 4377, predicted reward 27.137255,predicted upper bound 27.140613,actual reward 28.282844
    round 4378, predicted reward 28.259817,predicted upper bound 28.263005,actual reward 27.164811
    round 4379, predicted reward 22.981759,predicted upper bound 22.985014,actual reward 22.278325
    round 4380, predicted reward 24.129119,predicted upper bound 24.132640,actual reward 26.247055
    round 4381, predicted reward 30.898166,predicted upper bound 30.901785,actual reward 32.148167
    round 4382, predicted reward 27.201422,predicted upper bound 27.205342,actual reward 24.722498
    round 4383, predicted reward 28.144592,predicted upper bound 28.147440,actual reward 23.864656
    round 4384, predicted reward 26.858475,predicted upper bound 26.861867,actual reward 25.670208
    round 4385, predicted reward 22.989734,predicted upper bound 22.993343,actual reward 21.550701
    round 4386, predicted reward 31.873828,predicted upper bound 31.876792,actual reward 24.832094
    round 4387, predicted reward 26.658953,predicted upper bound 26.662224,actual reward 23.424899
    round 4388, predicted reward 24.828238,predicted upper bound 24.831417,actual reward 26.816601
    round 4389, predicted reward 24.866497,predicted upper bound 24.870181,actual reward 26.100135
    round 4390, predicted reward 25.408346,predicted upper bound 25.412101,actual reward 22.496031
    round 4391, predicted reward 28.963044,predicted upper bound 28.965786,actual reward 33.459879
    round 4392, predicted reward 17.935182,predicted upper bound 17.938076,actual reward 12.519734
    round 4393, predicted reward 22.968412,predicted upper bound 22.971994,actual reward 22.067510
    round 4394, predicted reward 30.216214,predicted upper bound 30.219569,actual reward 26.466805
    round 4395, predicted reward 23.185236,predicted upper bound 23.188678,actual reward 21.687406
    round 4396, predicted reward 29.242219,predicted upper bound 29.245038,actual reward 32.019823
    round 4397, predicted reward 29.990593,predicted upper bound 29.993605,actual reward 33.521698
    round 4398, predicted reward 27.251732,predicted upper bound 27.255227,actual reward 26.023737
    round 4399, predicted reward 24.445106,predicted upper bound 24.448612,actual reward 25.306865
    round 4400, predicted reward 28.603303,predicted upper bound 28.606818,actual reward 26.762494
    round 4401, predicted reward 31.835492,predicted upper bound 31.838942,actual reward 30.562290
    round 4402, predicted reward 25.743186,predicted upper bound 25.747119,actual reward 21.677759
    round 4403, predicted reward 29.746885,predicted upper bound 29.750573,actual reward 27.192679
    round 4404, predicted reward 23.138053,predicted upper bound 23.142176,actual reward 20.347741
    round 4405, predicted reward 31.458904,predicted upper bound 31.461962,actual reward 31.130166
    round 4406, predicted reward 28.138074,predicted upper bound 28.142024,actual reward 28.147105
    round 4407, predicted reward 27.125388,predicted upper bound 27.129027,actual reward 28.474864
    round 4408, predicted reward 27.392295,predicted upper bound 27.395580,actual reward 24.220613
    round 4409, predicted reward 28.271083,predicted upper bound 28.273977,actual reward 31.335511
    round 4410, predicted reward 24.078760,predicted upper bound 24.082111,actual reward 27.031009
    round 4411, predicted reward 20.834415,predicted upper bound 20.838003,actual reward 18.787090
    round 4412, predicted reward 25.579075,predicted upper bound 25.582773,actual reward 26.052977
    round 4413, predicted reward 24.069622,predicted upper bound 24.073014,actual reward 27.049133
    round 4414, predicted reward 29.300726,predicted upper bound 29.303648,actual reward 37.163735
    round 4415, predicted reward 24.361027,predicted upper bound 24.364449,actual reward 19.480107
    round 4416, predicted reward 31.306898,predicted upper bound 31.310289,actual reward 30.853156
    round 4417, predicted reward 29.054021,predicted upper bound 29.057764,actual reward 28.936648
    round 4418, predicted reward 34.052055,predicted upper bound 34.055424,actual reward 36.211272
    round 4419, predicted reward 20.740753,predicted upper bound 20.744287,actual reward 20.985176
    round 4420, predicted reward 28.193985,predicted upper bound 28.197547,actual reward 18.703342
    round 4421, predicted reward 25.193302,predicted upper bound 25.197041,actual reward 25.275366
    round 4422, predicted reward 28.832328,predicted upper bound 28.835282,actual reward 25.358456
    round 4423, predicted reward 26.876230,predicted upper bound 26.879454,actual reward 29.952952
    round 4424, predicted reward 30.444911,predicted upper bound 30.448239,actual reward 29.521826
    round 4425, predicted reward 27.602828,predicted upper bound 27.605772,actual reward 26.508056
    round 4426, predicted reward 29.032568,predicted upper bound 29.035980,actual reward 28.502170
    round 4427, predicted reward 24.058443,predicted upper bound 24.062106,actual reward 23.709675
    round 4428, predicted reward 28.419038,predicted upper bound 28.422929,actual reward 27.178643
    round 4429, predicted reward 30.648867,predicted upper bound 30.652163,actual reward 32.377503
    round 4430, predicted reward 27.127720,predicted upper bound 27.131228,actual reward 20.035827
    round 4431, predicted reward 29.953391,predicted upper bound 29.956667,actual reward 28.657231
    round 4432, predicted reward 20.582005,predicted upper bound 20.585357,actual reward 18.920893
    round 4433, predicted reward 31.615233,predicted upper bound 31.618340,actual reward 28.862119
    round 4434, predicted reward 20.806436,predicted upper bound 20.809870,actual reward 24.010958
    round 4435, predicted reward 30.358772,predicted upper bound 30.361888,actual reward 34.742132
    round 4436, predicted reward 25.319901,predicted upper bound 25.323268,actual reward 18.621060
    round 4437, predicted reward 22.621650,predicted upper bound 22.625103,actual reward 24.772104
    round 4438, predicted reward 31.445599,predicted upper bound 31.448959,actual reward 29.067914
    round 4439, predicted reward 21.308226,predicted upper bound 21.311878,actual reward 14.789910
    round 4440, predicted reward 27.059996,predicted upper bound 27.063019,actual reward 27.322316
    round 4441, predicted reward 23.562403,predicted upper bound 23.566111,actual reward 20.386308
    round 4442, predicted reward 23.657218,predicted upper bound 23.660758,actual reward 27.477774
    round 4443, predicted reward 23.178952,predicted upper bound 23.182408,actual reward 22.422102
    round 4444, predicted reward 23.473877,predicted upper bound 23.477602,actual reward 18.947500
    round 4445, predicted reward 31.357641,predicted upper bound 31.360892,actual reward 28.526537
    round 4446, predicted reward 35.801101,predicted upper bound 35.804076,actual reward 40.228375
    round 4447, predicted reward 25.977584,predicted upper bound 25.981057,actual reward 24.286751
    round 4448, predicted reward 22.866375,predicted upper bound 22.870091,actual reward 24.491049
    round 4449, predicted reward 26.875507,predicted upper bound 26.878694,actual reward 24.734727
    round 4450, predicted reward 31.709868,predicted upper bound 31.713183,actual reward 31.486309
    round 4451, predicted reward 23.416102,predicted upper bound 23.419579,actual reward 21.212096
    round 4452, predicted reward 27.678884,predicted upper bound 27.681623,actual reward 27.334256
    round 4453, predicted reward 26.155542,predicted upper bound 26.159069,actual reward 22.561817
    round 4454, predicted reward 25.167141,predicted upper bound 25.170809,actual reward 28.922418
    round 4455, predicted reward 30.767726,predicted upper bound 30.771599,actual reward 30.003221
    round 4456, predicted reward 34.399857,predicted upper bound 34.403145,actual reward 32.474493
    round 4457, predicted reward 28.250484,predicted upper bound 28.254048,actual reward 26.775211
    round 4458, predicted reward 28.048499,predicted upper bound 28.052043,actual reward 26.690597
    round 4459, predicted reward 34.013686,predicted upper bound 34.016513,actual reward 34.995396
    round 4460, predicted reward 28.101449,predicted upper bound 28.104705,actual reward 26.063509
    round 4461, predicted reward 26.686300,predicted upper bound 26.689704,actual reward 24.169319
    round 4462, predicted reward 30.678537,predicted upper bound 30.682352,actual reward 28.381278
    round 4463, predicted reward 32.137355,predicted upper bound 32.140225,actual reward 31.887074
    round 4464, predicted reward 23.989859,predicted upper bound 23.993497,actual reward 21.585167
    round 4465, predicted reward 31.614447,predicted upper bound 31.618028,actual reward 31.257636
    round 4466, predicted reward 23.166301,predicted upper bound 23.169955,actual reward 21.603810
    round 4467, predicted reward 25.689221,predicted upper bound 25.693323,actual reward 22.624528
    round 4468, predicted reward 24.225218,predicted upper bound 24.228906,actual reward 22.286128
    round 4469, predicted reward 34.771703,predicted upper bound 34.774824,actual reward 44.008341
    round 4470, predicted reward 26.333681,predicted upper bound 26.337201,actual reward 21.968175
    round 4471, predicted reward 26.734139,predicted upper bound 26.738018,actual reward 25.919888
    round 4472, predicted reward 23.618252,predicted upper bound 23.621786,actual reward 30.255255
    round 4473, predicted reward 32.150992,predicted upper bound 32.155005,actual reward 37.581049
    round 4474, predicted reward 21.498109,predicted upper bound 21.501621,actual reward 15.480236
    round 4475, predicted reward 27.864962,predicted upper bound 27.868467,actual reward 20.139127
    round 4476, predicted reward 29.301633,predicted upper bound 29.305447,actual reward 30.938349
    round 4477, predicted reward 29.904698,predicted upper bound 29.907879,actual reward 23.997229
    round 4478, predicted reward 22.669260,predicted upper bound 22.673113,actual reward 18.650481
    round 4479, predicted reward 24.419845,predicted upper bound 24.423427,actual reward 24.260686
    round 4480, predicted reward 23.858611,predicted upper bound 23.862587,actual reward 20.819213
    round 4481, predicted reward 21.657773,predicted upper bound 21.661850,actual reward 18.517345
    round 4482, predicted reward 23.180223,predicted upper bound 23.184187,actual reward 20.099029
    round 4483, predicted reward 27.356565,predicted upper bound 27.360196,actual reward 27.387200
    round 4484, predicted reward 27.572012,predicted upper bound 27.575687,actual reward 28.317783
    round 4485, predicted reward 27.244309,predicted upper bound 27.247904,actual reward 27.144034
    round 4486, predicted reward 30.378812,predicted upper bound 30.382871,actual reward 34.253658
    round 4487, predicted reward 21.151142,predicted upper bound 21.154814,actual reward 20.670885
    round 4488, predicted reward 27.245963,predicted upper bound 27.249721,actual reward 23.432681
    round 4489, predicted reward 25.982932,predicted upper bound 25.986486,actual reward 23.964042
    round 4490, predicted reward 27.575337,predicted upper bound 27.578577,actual reward 27.532119
    round 4491, predicted reward 25.294358,predicted upper bound 25.298054,actual reward 23.522545
    round 4492, predicted reward 30.717986,predicted upper bound 30.720908,actual reward 34.677595
    round 4493, predicted reward 27.785879,predicted upper bound 27.789519,actual reward 29.091830
    round 4494, predicted reward 33.051885,predicted upper bound 33.055048,actual reward 37.127270
    round 4495, predicted reward 25.737806,predicted upper bound 25.741484,actual reward 24.165855
    round 4496, predicted reward 32.844152,predicted upper bound 32.847561,actual reward 33.291214
    round 4497, predicted reward 24.776867,predicted upper bound 24.780365,actual reward 22.104075
    round 4498, predicted reward 28.240325,predicted upper bound 28.243340,actual reward 25.708338
    round 4499, predicted reward 25.425131,predicted upper bound 25.428756,actual reward 21.180114
    round 4500, predicted reward 30.808093,predicted upper bound 30.811633,actual reward 29.257390
    round 4501, predicted reward 21.682726,predicted upper bound 21.686655,actual reward 22.931669
    round 4502, predicted reward 26.008636,predicted upper bound 26.012018,actual reward 21.308557
    round 4503, predicted reward 23.814751,predicted upper bound 23.817994,actual reward 22.462229
    round 4504, predicted reward 21.364391,predicted upper bound 21.368532,actual reward 23.864197
    round 4505, predicted reward 26.365336,predicted upper bound 26.368981,actual reward 29.300436
    round 4506, predicted reward 28.016387,predicted upper bound 28.019734,actual reward 26.496799
    round 4507, predicted reward 25.333380,predicted upper bound 25.336692,actual reward 23.700059
    round 4508, predicted reward 26.003332,predicted upper bound 26.006414,actual reward 26.455004
    round 4509, predicted reward 29.268927,predicted upper bound 29.272446,actual reward 33.190776
    round 4510, predicted reward 24.738070,predicted upper bound 24.741325,actual reward 23.012811
    round 4511, predicted reward 29.502509,predicted upper bound 29.506124,actual reward 34.062491
    round 4512, predicted reward 26.468994,predicted upper bound 26.473091,actual reward 20.464013
    round 4513, predicted reward 29.569343,predicted upper bound 29.572567,actual reward 31.105318
    round 4514, predicted reward 25.438395,predicted upper bound 25.442266,actual reward 23.316779
    round 4515, predicted reward 24.889915,predicted upper bound 24.893912,actual reward 18.815422
    round 4516, predicted reward 26.649901,predicted upper bound 26.653494,actual reward 27.448341
    round 4517, predicted reward 25.925982,predicted upper bound 25.929585,actual reward 23.420279
    round 4518, predicted reward 30.809662,predicted upper bound 30.813509,actual reward 29.248715
    round 4519, predicted reward 30.116955,predicted upper bound 30.119983,actual reward 29.057586
    round 4520, predicted reward 30.999800,predicted upper bound 31.002625,actual reward 29.478303
    round 4521, predicted reward 22.170741,predicted upper bound 22.173994,actual reward 26.165079
    round 4522, predicted reward 21.228942,predicted upper bound 21.233041,actual reward 18.399793
    round 4523, predicted reward 30.190265,predicted upper bound 30.193706,actual reward 30.244349
    round 4524, predicted reward 26.307583,predicted upper bound 26.311465,actual reward 27.508984
    round 4525, predicted reward 27.015767,predicted upper bound 27.019550,actual reward 29.236369
    round 4526, predicted reward 19.765627,predicted upper bound 19.769128,actual reward 18.127179
    round 4527, predicted reward 32.346408,predicted upper bound 32.349881,actual reward 28.264110
    round 4528, predicted reward 30.811016,predicted upper bound 30.814490,actual reward 31.078597
    round 4529, predicted reward 24.734178,predicted upper bound 24.737852,actual reward 25.205728
    round 4530, predicted reward 27.184593,predicted upper bound 27.188513,actual reward 27.536310
    round 4531, predicted reward 23.167424,predicted upper bound 23.171259,actual reward 21.586232
    round 4532, predicted reward 24.338668,predicted upper bound 24.342157,actual reward 24.936505
    round 4533, predicted reward 24.124664,predicted upper bound 24.128069,actual reward 24.749341
    round 4534, predicted reward 26.541057,predicted upper bound 26.544822,actual reward 27.877137
    round 4535, predicted reward 30.526562,predicted upper bound 30.530080,actual reward 31.568407
    round 4536, predicted reward 23.562759,predicted upper bound 23.566327,actual reward 20.658324
    round 4537, predicted reward 28.513726,predicted upper bound 28.517093,actual reward 23.396604
    round 4538, predicted reward 30.197427,predicted upper bound 30.200638,actual reward 33.393233
    round 4539, predicted reward 30.134736,predicted upper bound 30.138259,actual reward 29.424156
    round 4540, predicted reward 29.475997,predicted upper bound 29.479436,actual reward 35.838313
    round 4541, predicted reward 24.648458,predicted upper bound 24.652026,actual reward 23.604852
    round 4542, predicted reward 26.575338,predicted upper bound 26.578923,actual reward 25.819097
    round 4543, predicted reward 24.253140,predicted upper bound 24.255851,actual reward 20.905679
    round 4544, predicted reward 22.218081,predicted upper bound 22.221906,actual reward 23.173791
    round 4545, predicted reward 26.869362,predicted upper bound 26.872863,actual reward 24.557518
    round 4546, predicted reward 29.006316,predicted upper bound 29.009373,actual reward 33.317574
    round 4547, predicted reward 27.350390,predicted upper bound 27.354006,actual reward 25.887707
    round 4548, predicted reward 29.793272,predicted upper bound 29.796834,actual reward 24.541750
    round 4549, predicted reward 21.997712,predicted upper bound 22.001667,actual reward 20.097558
    round 4550, predicted reward 28.565580,predicted upper bound 28.568971,actual reward 30.984033
    round 4551, predicted reward 27.743810,predicted upper bound 27.747478,actual reward 28.189332
    round 4552, predicted reward 24.929287,predicted upper bound 24.933060,actual reward 25.367232
    round 4553, predicted reward 31.759950,predicted upper bound 31.763636,actual reward 29.375456
    round 4554, predicted reward 25.467689,predicted upper bound 25.471556,actual reward 22.674169
    round 4555, predicted reward 27.597166,predicted upper bound 27.600813,actual reward 29.671828
    round 4556, predicted reward 21.556092,predicted upper bound 21.559662,actual reward 17.345576
    round 4557, predicted reward 23.803019,predicted upper bound 23.806673,actual reward 21.295239
    round 4558, predicted reward 25.129230,predicted upper bound 25.132791,actual reward 22.346853
    round 4559, predicted reward 24.707262,predicted upper bound 24.710854,actual reward 22.466534
    round 4560, predicted reward 26.025365,predicted upper bound 26.029173,actual reward 24.455027
    round 4561, predicted reward 29.574580,predicted upper bound 29.577838,actual reward 26.242856
    round 4562, predicted reward 32.985691,predicted upper bound 32.989143,actual reward 33.283519
    round 4563, predicted reward 22.777361,predicted upper bound 22.781075,actual reward 18.841326
    round 4564, predicted reward 25.363530,predicted upper bound 25.367507,actual reward 20.148271
    round 4565, predicted reward 24.532830,predicted upper bound 24.536287,actual reward 25.313280
    round 4566, predicted reward 25.989233,predicted upper bound 25.992889,actual reward 23.990671
    round 4567, predicted reward 26.317651,predicted upper bound 26.321200,actual reward 25.325618
    round 4568, predicted reward 25.465690,predicted upper bound 25.468644,actual reward 24.716037
    round 4569, predicted reward 22.562709,predicted upper bound 22.566342,actual reward 18.837479
    round 4570, predicted reward 33.338545,predicted upper bound 33.342027,actual reward 33.026128
    round 4571, predicted reward 25.500532,predicted upper bound 25.503562,actual reward 25.437039
    round 4572, predicted reward 30.059473,predicted upper bound 30.062847,actual reward 31.737788
    round 4573, predicted reward 28.925413,predicted upper bound 28.928299,actual reward 28.138544
    round 4574, predicted reward 24.999779,predicted upper bound 25.002831,actual reward 25.313049
    round 4575, predicted reward 29.299969,predicted upper bound 29.303141,actual reward 27.981612
    round 4576, predicted reward 28.474145,predicted upper bound 28.477950,actual reward 30.556545
    round 4577, predicted reward 31.079894,predicted upper bound 31.082857,actual reward 33.939893
    round 4578, predicted reward 31.173285,predicted upper bound 31.176723,actual reward 27.911211
    round 4579, predicted reward 29.252386,predicted upper bound 29.256180,actual reward 33.308183
    round 4580, predicted reward 23.329594,predicted upper bound 23.332771,actual reward 22.705625
    round 4581, predicted reward 28.875173,predicted upper bound 28.879154,actual reward 27.022500
    round 4582, predicted reward 29.883830,predicted upper bound 29.887005,actual reward 32.926829
    round 4583, predicted reward 21.233684,predicted upper bound 21.236797,actual reward 17.952447
    round 4584, predicted reward 25.094455,predicted upper bound 25.097910,actual reward 20.087962
    round 4585, predicted reward 31.619295,predicted upper bound 31.622632,actual reward 27.147481
    round 4586, predicted reward 26.169954,predicted upper bound 26.173488,actual reward 26.942836
    round 4587, predicted reward 31.000528,predicted upper bound 31.003904,actual reward 31.069222
    round 4588, predicted reward 31.882808,predicted upper bound 31.886157,actual reward 31.045937
    round 4589, predicted reward 38.092636,predicted upper bound 38.095834,actual reward 35.455965
    round 4590, predicted reward 32.138829,predicted upper bound 32.142196,actual reward 25.460557
    round 4591, predicted reward 32.993861,predicted upper bound 32.997765,actual reward 35.227382
    round 4592, predicted reward 33.586299,predicted upper bound 33.589662,actual reward 33.960315
    round 4593, predicted reward 33.722840,predicted upper bound 33.726280,actual reward 32.805195
    round 4594, predicted reward 36.773472,predicted upper bound 36.776408,actual reward 41.518442
    round 4595, predicted reward 27.380020,predicted upper bound 27.383111,actual reward 25.768104
    round 4596, predicted reward 24.649845,predicted upper bound 24.653477,actual reward 27.032386
    round 4597, predicted reward 24.373935,predicted upper bound 24.378225,actual reward 17.363761
    round 4598, predicted reward 32.921261,predicted upper bound 32.925032,actual reward 34.184054
    round 4599, predicted reward 28.919339,predicted upper bound 28.923024,actual reward 26.026081
    round 4600, predicted reward 26.915287,predicted upper bound 26.918918,actual reward 24.419790
    round 4601, predicted reward 24.649719,predicted upper bound 24.653538,actual reward 22.816197
    round 4602, predicted reward 34.299958,predicted upper bound 34.303197,actual reward 36.434533
    round 4603, predicted reward 31.377740,predicted upper bound 31.381675,actual reward 32.510543
    round 4604, predicted reward 28.736626,predicted upper bound 28.740265,actual reward 28.055747
    round 4605, predicted reward 29.797894,predicted upper bound 29.801217,actual reward 26.103129
    round 4606, predicted reward 32.070331,predicted upper bound 32.073433,actual reward 27.300698
    round 4607, predicted reward 21.905069,predicted upper bound 21.908975,actual reward 25.128232
    round 4608, predicted reward 22.717301,predicted upper bound 22.720705,actual reward 21.947101
    round 4609, predicted reward 24.162561,predicted upper bound 24.166277,actual reward 25.868112
    round 4610, predicted reward 20.651280,predicted upper bound 20.654782,actual reward 14.299900
    round 4611, predicted reward 24.525102,predicted upper bound 24.528363,actual reward 22.529086
    round 4612, predicted reward 24.166445,predicted upper bound 24.170394,actual reward 21.957012
    round 4613, predicted reward 31.933187,predicted upper bound 31.936388,actual reward 32.090765
    round 4614, predicted reward 30.502860,predicted upper bound 30.506252,actual reward 29.810148
    round 4615, predicted reward 27.713200,predicted upper bound 27.716755,actual reward 32.326798
    round 4616, predicted reward 24.674864,predicted upper bound 24.678441,actual reward 19.761976
    round 4617, predicted reward 23.185125,predicted upper bound 23.189770,actual reward 22.083070
    round 4618, predicted reward 22.961885,predicted upper bound 22.965684,actual reward 24.538423
    round 4619, predicted reward 37.119572,predicted upper bound 37.122776,actual reward 41.101428
    round 4620, predicted reward 22.799449,predicted upper bound 22.802949,actual reward 25.622522
    round 4621, predicted reward 33.325545,predicted upper bound 33.328753,actual reward 31.630079
    round 4622, predicted reward 30.990194,predicted upper bound 30.993176,actual reward 33.922057
    round 4623, predicted reward 32.148859,predicted upper bound 32.152303,actual reward 35.722543
    round 4624, predicted reward 27.525525,predicted upper bound 27.528635,actual reward 28.074109
    round 4625, predicted reward 28.753093,predicted upper bound 28.756706,actual reward 26.198979
    round 4626, predicted reward 23.099737,predicted upper bound 23.103478,actual reward 18.869264
    round 4627, predicted reward 23.637088,predicted upper bound 23.640528,actual reward 19.502664
    round 4628, predicted reward 29.488587,predicted upper bound 29.491672,actual reward 29.532351
    round 4629, predicted reward 27.948076,predicted upper bound 27.951422,actual reward 28.135902
    round 4630, predicted reward 29.701335,predicted upper bound 29.704778,actual reward 28.501818
    round 4631, predicted reward 26.934122,predicted upper bound 26.937673,actual reward 24.929000
    round 4632, predicted reward 26.259964,predicted upper bound 26.263527,actual reward 26.775698
    round 4633, predicted reward 32.653935,predicted upper bound 32.657195,actual reward 33.752372
    round 4634, predicted reward 33.026591,predicted upper bound 33.029733,actual reward 36.402873
    round 4635, predicted reward 29.427102,predicted upper bound 29.429871,actual reward 29.955044
    round 4636, predicted reward 27.464052,predicted upper bound 27.467967,actual reward 32.316614
    round 4637, predicted reward 31.348252,predicted upper bound 31.351665,actual reward 35.308956
    round 4638, predicted reward 30.426090,predicted upper bound 30.429818,actual reward 24.809821
    round 4639, predicted reward 21.672519,predicted upper bound 21.675563,actual reward 20.308486
    round 4640, predicted reward 23.000547,predicted upper bound 23.004312,actual reward 19.090554
    round 4641, predicted reward 24.391350,predicted upper bound 24.394587,actual reward 25.613147
    round 4642, predicted reward 27.612248,predicted upper bound 27.615367,actual reward 29.323917
    round 4643, predicted reward 23.473722,predicted upper bound 23.477010,actual reward 23.241176
    round 4644, predicted reward 22.505500,predicted upper bound 22.508948,actual reward 19.539737
    round 4645, predicted reward 26.754775,predicted upper bound 26.758078,actual reward 26.549357
    round 4646, predicted reward 26.348103,predicted upper bound 26.351304,actual reward 20.942391
    round 4647, predicted reward 28.892214,predicted upper bound 28.895500,actual reward 28.200071
    round 4648, predicted reward 28.993165,predicted upper bound 28.996317,actual reward 27.672606
    round 4649, predicted reward 28.545729,predicted upper bound 28.548936,actual reward 31.940863
    round 4650, predicted reward 31.097049,predicted upper bound 31.100120,actual reward 31.251707
    round 4651, predicted reward 26.592610,predicted upper bound 26.595829,actual reward 24.681921
    round 4652, predicted reward 29.734484,predicted upper bound 29.737734,actual reward 29.943489
    round 4653, predicted reward 24.023787,predicted upper bound 24.027478,actual reward 24.671378
    round 4654, predicted reward 30.894765,predicted upper bound 30.897967,actual reward 35.385665
    round 4655, predicted reward 24.205255,predicted upper bound 24.208586,actual reward 15.273508
    round 4656, predicted reward 26.437390,predicted upper bound 26.440392,actual reward 20.679843
    round 4657, predicted reward 25.299087,predicted upper bound 25.302641,actual reward 20.693658
    round 4658, predicted reward 29.550933,predicted upper bound 29.554093,actual reward 34.560256
    round 4659, predicted reward 29.570810,predicted upper bound 29.574186,actual reward 32.731747
    round 4660, predicted reward 25.127065,predicted upper bound 25.130195,actual reward 23.372362
    round 4661, predicted reward 28.779362,predicted upper bound 28.782663,actual reward 21.905052
    round 4662, predicted reward 23.923565,predicted upper bound 23.926981,actual reward 22.991836
    round 4663, predicted reward 28.684118,predicted upper bound 28.687752,actual reward 34.447126
    round 4664, predicted reward 31.296851,predicted upper bound 31.299919,actual reward 34.756893
    round 4665, predicted reward 27.672139,predicted upper bound 27.675623,actual reward 21.628302
    round 4666, predicted reward 29.629203,predicted upper bound 29.632678,actual reward 33.945934
    round 4667, predicted reward 28.897317,predicted upper bound 28.900964,actual reward 28.878571
    round 4668, predicted reward 22.667361,predicted upper bound 22.670995,actual reward 20.533508
    round 4669, predicted reward 24.157281,predicted upper bound 24.160681,actual reward 23.624403
    round 4670, predicted reward 28.426523,predicted upper bound 28.430123,actual reward 25.123005
    round 4671, predicted reward 17.076083,predicted upper bound 17.079713,actual reward 11.840772
    round 4672, predicted reward 26.359733,predicted upper bound 26.363414,actual reward 23.008907
    round 4673, predicted reward 23.148059,predicted upper bound 23.151367,actual reward 19.081134
    round 4674, predicted reward 28.339942,predicted upper bound 28.343658,actual reward 28.422939
    round 4675, predicted reward 30.192884,predicted upper bound 30.196353,actual reward 30.958689
    round 4676, predicted reward 27.424911,predicted upper bound 27.428563,actual reward 24.879106
    round 4677, predicted reward 25.916891,predicted upper bound 25.920217,actual reward 24.844061
    round 4678, predicted reward 25.773355,predicted upper bound 25.776686,actual reward 18.780720
    round 4679, predicted reward 26.462864,predicted upper bound 26.466142,actual reward 24.536207
    round 4680, predicted reward 27.097302,predicted upper bound 27.100262,actual reward 24.418865
    round 4681, predicted reward 24.914312,predicted upper bound 24.917954,actual reward 26.276225
    round 4682, predicted reward 28.365275,predicted upper bound 28.368215,actual reward 26.566143
    round 4683, predicted reward 30.053071,predicted upper bound 30.057070,actual reward 24.696925
    round 4684, predicted reward 29.194684,predicted upper bound 29.197602,actual reward 28.250647
    round 4685, predicted reward 28.541188,predicted upper bound 28.544157,actual reward 27.222029
    round 4686, predicted reward 28.913231,predicted upper bound 28.917309,actual reward 23.882023
    round 4687, predicted reward 29.118120,predicted upper bound 29.121737,actual reward 36.039094
    round 4688, predicted reward 26.113390,predicted upper bound 26.116741,actual reward 21.710948
    round 4689, predicted reward 29.428578,predicted upper bound 29.431982,actual reward 29.887827
    round 4690, predicted reward 30.639652,predicted upper bound 30.643148,actual reward 27.326436
    round 4691, predicted reward 22.614615,predicted upper bound 22.617623,actual reward 19.066887
    round 4692, predicted reward 27.521603,predicted upper bound 27.524883,actual reward 24.176567
    round 4693, predicted reward 39.804140,predicted upper bound 39.807179,actual reward 45.120052
    round 4694, predicted reward 23.840003,predicted upper bound 23.843342,actual reward 22.830235
    round 4695, predicted reward 24.608855,predicted upper bound 24.612100,actual reward 20.682197
    round 4696, predicted reward 22.415858,predicted upper bound 22.419763,actual reward 15.150650
    round 4697, predicted reward 22.902662,predicted upper bound 22.905833,actual reward 23.314437
    round 4698, predicted reward 26.091232,predicted upper bound 26.094941,actual reward 22.280177
    round 4699, predicted reward 30.540105,predicted upper bound 30.543349,actual reward 35.246530
    round 4700, predicted reward 29.429533,predicted upper bound 29.432877,actual reward 28.404502
    round 4701, predicted reward 28.875297,predicted upper bound 28.878898,actual reward 29.164339
    round 4702, predicted reward 26.744402,predicted upper bound 26.747474,actual reward 29.343856
    round 4703, predicted reward 26.276498,predicted upper bound 26.279846,actual reward 25.317382
    round 4704, predicted reward 32.018699,predicted upper bound 32.021746,actual reward 33.902434
    round 4705, predicted reward 24.895265,predicted upper bound 24.899024,actual reward 21.334678
    round 4706, predicted reward 32.196219,predicted upper bound 32.199639,actual reward 31.371204
    round 4707, predicted reward 33.157908,predicted upper bound 33.161766,actual reward 33.318446
    round 4708, predicted reward 22.741811,predicted upper bound 22.745766,actual reward 26.508055
    round 4709, predicted reward 24.233353,predicted upper bound 24.237092,actual reward 23.577799
    round 4710, predicted reward 32.706158,predicted upper bound 32.710454,actual reward 38.582651
    round 4711, predicted reward 37.788364,predicted upper bound 37.791463,actual reward 40.663999
    round 4712, predicted reward 25.236208,predicted upper bound 25.240164,actual reward 28.017335
    round 4713, predicted reward 26.293871,predicted upper bound 26.297002,actual reward 26.871914
    round 4714, predicted reward 25.263965,predicted upper bound 25.267531,actual reward 27.179632
    round 4715, predicted reward 36.580687,predicted upper bound 36.583947,actual reward 40.703900
    round 4716, predicted reward 25.807185,predicted upper bound 25.810591,actual reward 25.925835
    round 4717, predicted reward 34.305211,predicted upper bound 34.308120,actual reward 33.704493
    round 4718, predicted reward 35.832170,predicted upper bound 35.835014,actual reward 44.380927
    round 4719, predicted reward 21.694513,predicted upper bound 21.697843,actual reward 17.967630
    round 4720, predicted reward 28.444134,predicted upper bound 28.447609,actual reward 29.039484
    round 4721, predicted reward 24.778367,predicted upper bound 24.781698,actual reward 21.000563
    round 4722, predicted reward 26.970889,predicted upper bound 26.974179,actual reward 31.559928
    round 4723, predicted reward 27.918790,predicted upper bound 27.922322,actual reward 27.545783
    round 4724, predicted reward 31.053971,predicted upper bound 31.057692,actual reward 29.388877
    round 4725, predicted reward 30.687290,predicted upper bound 30.690500,actual reward 31.744290
    round 4726, predicted reward 21.329448,predicted upper bound 21.333762,actual reward 25.657260
    round 4727, predicted reward 30.412294,predicted upper bound 30.415512,actual reward 27.823821
    round 4728, predicted reward 27.918858,predicted upper bound 27.922612,actual reward 27.725837
    round 4729, predicted reward 28.867364,predicted upper bound 28.870131,actual reward 30.565378
    round 4730, predicted reward 28.038346,predicted upper bound 28.041738,actual reward 24.983078
    round 4731, predicted reward 26.195802,predicted upper bound 26.198839,actual reward 29.422389
    round 4732, predicted reward 26.227513,predicted upper bound 26.230894,actual reward 30.061477
    round 4733, predicted reward 24.550497,predicted upper bound 24.554241,actual reward 21.537512
    round 4734, predicted reward 29.252077,predicted upper bound 29.255890,actual reward 25.481927
    round 4735, predicted reward 23.872675,predicted upper bound 23.876472,actual reward 17.514838
    round 4736, predicted reward 26.690567,predicted upper bound 26.694026,actual reward 25.818096
    round 4737, predicted reward 23.764316,predicted upper bound 23.767685,actual reward 18.656587
    round 4738, predicted reward 38.445973,predicted upper bound 38.448584,actual reward 44.682762
    round 4739, predicted reward 34.374019,predicted upper bound 34.377478,actual reward 36.913564
    round 4740, predicted reward 28.609588,predicted upper bound 28.612699,actual reward 22.653548
    round 4741, predicted reward 27.853174,predicted upper bound 27.856812,actual reward 30.578976
    round 4742, predicted reward 37.418469,predicted upper bound 37.422275,actual reward 48.805774
    round 4743, predicted reward 27.065124,predicted upper bound 27.068870,actual reward 24.367006
    round 4744, predicted reward 21.992751,predicted upper bound 21.996030,actual reward 19.277272
    round 4745, predicted reward 34.009705,predicted upper bound 34.013093,actual reward 33.586379
    round 4746, predicted reward 25.873845,predicted upper bound 25.877214,actual reward 21.473513
    round 4747, predicted reward 26.653538,predicted upper bound 26.657022,actual reward 23.259507
    round 4748, predicted reward 29.757917,predicted upper bound 29.761803,actual reward 27.536396
    round 4749, predicted reward 24.239620,predicted upper bound 24.243144,actual reward 24.080038
    round 4750, predicted reward 25.195917,predicted upper bound 25.198982,actual reward 20.186183
    round 4751, predicted reward 20.187263,predicted upper bound 20.190818,actual reward 13.151231
    round 4752, predicted reward 28.933775,predicted upper bound 28.937139,actual reward 31.870568
    round 4753, predicted reward 36.121515,predicted upper bound 36.125378,actual reward 31.838981
    round 4754, predicted reward 24.355502,predicted upper bound 24.359171,actual reward 20.878878
    round 4755, predicted reward 29.485773,predicted upper bound 29.489291,actual reward 29.037989
    round 4756, predicted reward 34.567093,predicted upper bound 34.570252,actual reward 43.285928
    round 4757, predicted reward 28.310110,predicted upper bound 28.313477,actual reward 29.528042
    round 4758, predicted reward 25.125569,predicted upper bound 25.128644,actual reward 20.223758
    round 4759, predicted reward 20.184978,predicted upper bound 20.189084,actual reward 22.681415
    round 4760, predicted reward 27.538126,predicted upper bound 27.541392,actual reward 30.350921
    round 4761, predicted reward 30.803025,predicted upper bound 30.806935,actual reward 29.994290
    round 4762, predicted reward 26.576417,predicted upper bound 26.580045,actual reward 26.353034
    round 4763, predicted reward 30.529457,predicted upper bound 30.532839,actual reward 26.360610
    round 4764, predicted reward 27.468383,predicted upper bound 27.472206,actual reward 26.515274
    round 4765, predicted reward 29.733399,predicted upper bound 29.736810,actual reward 27.322458
    round 4766, predicted reward 28.183814,predicted upper bound 28.187354,actual reward 28.845312
    round 4767, predicted reward 28.148216,predicted upper bound 28.151569,actual reward 32.052380
    round 4768, predicted reward 24.622170,predicted upper bound 24.625103,actual reward 26.392781
    round 4769, predicted reward 22.524936,predicted upper bound 22.528872,actual reward 22.768598
    round 4770, predicted reward 28.694595,predicted upper bound 28.698027,actual reward 27.975027
    round 4771, predicted reward 30.256128,predicted upper bound 30.259463,actual reward 29.776621
    round 4772, predicted reward 26.253838,predicted upper bound 26.257057,actual reward 27.742672
    round 4773, predicted reward 32.925746,predicted upper bound 32.928808,actual reward 31.121802
    round 4774, predicted reward 22.824653,predicted upper bound 22.827994,actual reward 17.884980
    round 4775, predicted reward 25.452413,predicted upper bound 25.456208,actual reward 24.746655
    round 4776, predicted reward 23.743561,predicted upper bound 23.747210,actual reward 21.075529
    round 4777, predicted reward 25.630799,predicted upper bound 25.634742,actual reward 26.549087
    round 4778, predicted reward 23.293325,predicted upper bound 23.296787,actual reward 21.882965
    round 4779, predicted reward 29.751512,predicted upper bound 29.755175,actual reward 26.819782
    round 4780, predicted reward 28.631073,predicted upper bound 28.634613,actual reward 27.160041
    round 4781, predicted reward 23.986652,predicted upper bound 23.990103,actual reward 17.037033
    round 4782, predicted reward 22.295199,predicted upper bound 22.298034,actual reward 19.645209
    round 4783, predicted reward 32.046044,predicted upper bound 32.049275,actual reward 26.583118
    round 4784, predicted reward 28.769344,predicted upper bound 28.772398,actual reward 25.158028
    round 4785, predicted reward 31.753833,predicted upper bound 31.757304,actual reward 27.781382
    round 4786, predicted reward 30.460890,predicted upper bound 30.464433,actual reward 34.685771
    round 4787, predicted reward 29.351388,predicted upper bound 29.354419,actual reward 28.339847
    round 4788, predicted reward 28.788539,predicted upper bound 28.792278,actual reward 29.602120
    round 4789, predicted reward 21.116072,predicted upper bound 21.119411,actual reward 19.594981
    round 4790, predicted reward 28.198575,predicted upper bound 28.202276,actual reward 28.136968
    round 4791, predicted reward 30.649780,predicted upper bound 30.653194,actual reward 27.166065
    round 4792, predicted reward 26.726637,predicted upper bound 26.729913,actual reward 33.728311
    round 4793, predicted reward 34.809670,predicted upper bound 34.812925,actual reward 34.938378
    round 4794, predicted reward 36.335894,predicted upper bound 36.339398,actual reward 39.471992
    round 4795, predicted reward 21.008935,predicted upper bound 21.012875,actual reward 23.804491
    round 4796, predicted reward 20.913325,predicted upper bound 20.917164,actual reward 16.800638
    round 4797, predicted reward 31.315995,predicted upper bound 31.319116,actual reward 26.862399
    round 4798, predicted reward 31.039721,predicted upper bound 31.043181,actual reward 33.377599
    round 4799, predicted reward 30.816576,predicted upper bound 30.819634,actual reward 34.501417
    round 4800, predicted reward 25.194962,predicted upper bound 25.198111,actual reward 22.974697
    round 4801, predicted reward 35.924800,predicted upper bound 35.927772,actual reward 34.812956
    round 4802, predicted reward 24.016001,predicted upper bound 24.019269,actual reward 26.118643
    round 4803, predicted reward 27.331467,predicted upper bound 27.334730,actual reward 29.372592
    round 4804, predicted reward 40.066721,predicted upper bound 40.069940,actual reward 45.182434
    round 4805, predicted reward 26.780587,predicted upper bound 26.784587,actual reward 20.510571
    round 4806, predicted reward 25.147330,predicted upper bound 25.150859,actual reward 28.510107
    round 4807, predicted reward 23.845531,predicted upper bound 23.849055,actual reward 21.970557
    round 4808, predicted reward 23.966245,predicted upper bound 23.969728,actual reward 24.733990
    round 4809, predicted reward 31.611911,predicted upper bound 31.614993,actual reward 28.919209
    round 4810, predicted reward 28.230297,predicted upper bound 28.233352,actual reward 29.720609
    round 4811, predicted reward 24.524742,predicted upper bound 24.528350,actual reward 22.071102
    round 4812, predicted reward 30.966268,predicted upper bound 30.969665,actual reward 29.505445
    round 4813, predicted reward 29.210180,predicted upper bound 29.213730,actual reward 30.988534
    round 4814, predicted reward 30.655468,predicted upper bound 30.658704,actual reward 30.181341
    round 4815, predicted reward 25.743802,predicted upper bound 25.747349,actual reward 21.308576
    round 4816, predicted reward 39.673156,predicted upper bound 39.676135,actual reward 42.827665
    round 4817, predicted reward 25.766412,predicted upper bound 25.770077,actual reward 19.723741
    round 4818, predicted reward 30.782536,predicted upper bound 30.785342,actual reward 29.388993
    round 4819, predicted reward 31.903594,predicted upper bound 31.907421,actual reward 28.994485
    round 4820, predicted reward 30.472265,predicted upper bound 30.475738,actual reward 33.383600
    round 4821, predicted reward 24.804448,predicted upper bound 24.808202,actual reward 22.815195
    round 4822, predicted reward 34.306272,predicted upper bound 34.309293,actual reward 29.470347
    round 4823, predicted reward 24.783057,predicted upper bound 24.786860,actual reward 25.219861
    round 4824, predicted reward 28.192042,predicted upper bound 28.194992,actual reward 29.439173
    round 4825, predicted reward 27.079078,predicted upper bound 27.082367,actual reward 28.078904
    round 4826, predicted reward 28.258568,predicted upper bound 28.261910,actual reward 28.298723
    round 4827, predicted reward 28.363100,predicted upper bound 28.366989,actual reward 25.112717
    round 4828, predicted reward 28.464766,predicted upper bound 28.467093,actual reward 26.644157
    round 4829, predicted reward 31.195737,predicted upper bound 31.199704,actual reward 35.106215
    round 4830, predicted reward 20.794839,predicted upper bound 20.798442,actual reward 20.330769
    round 4831, predicted reward 26.828463,predicted upper bound 26.831407,actual reward 23.745113
    round 4832, predicted reward 24.159445,predicted upper bound 24.163307,actual reward 26.947905
    round 4833, predicted reward 19.913157,predicted upper bound 19.916710,actual reward 15.484870
    round 4834, predicted reward 26.606107,predicted upper bound 26.609744,actual reward 22.273762
    round 4835, predicted reward 28.066682,predicted upper bound 28.069794,actual reward 22.828869
    round 4836, predicted reward 28.638896,predicted upper bound 28.642320,actual reward 26.678383
    round 4837, predicted reward 25.694919,predicted upper bound 25.698372,actual reward 21.116343
    round 4838, predicted reward 24.373417,predicted upper bound 24.376528,actual reward 24.223229
    round 4839, predicted reward 36.485077,predicted upper bound 36.487730,actual reward 42.392471
    round 4840, predicted reward 27.586100,predicted upper bound 27.589730,actual reward 23.939564
    round 4841, predicted reward 24.721735,predicted upper bound 24.725443,actual reward 24.940629
    round 4842, predicted reward 23.987479,predicted upper bound 23.990591,actual reward 22.119993
    round 4843, predicted reward 22.225131,predicted upper bound 22.229297,actual reward 17.858237
    round 4844, predicted reward 24.797751,predicted upper bound 24.801181,actual reward 22.587767
    round 4845, predicted reward 29.705731,predicted upper bound 29.709070,actual reward 30.297358
    round 4846, predicted reward 31.231309,predicted upper bound 31.235233,actual reward 32.345902
    round 4847, predicted reward 28.298769,predicted upper bound 28.302316,actual reward 27.736735
    round 4848, predicted reward 31.224575,predicted upper bound 31.228232,actual reward 32.481657
    round 4849, predicted reward 25.631121,predicted upper bound 25.634718,actual reward 23.687329
    round 4850, predicted reward 22.997097,predicted upper bound 23.000526,actual reward 24.804317
    round 4851, predicted reward 31.309802,predicted upper bound 31.313027,actual reward 25.624747
    round 4852, predicted reward 26.574077,predicted upper bound 26.577867,actual reward 28.699243
    round 4853, predicted reward 29.816550,predicted upper bound 29.819903,actual reward 34.447912
    round 4854, predicted reward 29.717040,predicted upper bound 29.720708,actual reward 29.634608
    round 4855, predicted reward 27.388787,predicted upper bound 27.392764,actual reward 26.801237
    round 4856, predicted reward 27.030030,predicted upper bound 27.032955,actual reward 30.942786
    round 4857, predicted reward 22.404275,predicted upper bound 22.408101,actual reward 23.290679
    round 4858, predicted reward 28.217848,predicted upper bound 28.220917,actual reward 27.008672
    round 4859, predicted reward 19.082923,predicted upper bound 19.086520,actual reward 16.305560
    round 4860, predicted reward 25.212899,predicted upper bound 25.216096,actual reward 26.421202
    round 4861, predicted reward 25.432197,predicted upper bound 25.435622,actual reward 20.261297
    round 4862, predicted reward 23.538644,predicted upper bound 23.542098,actual reward 17.590117
    round 4863, predicted reward 26.053141,predicted upper bound 26.056250,actual reward 26.298715
    round 4864, predicted reward 24.692521,predicted upper bound 24.696080,actual reward 22.305614
    round 4865, predicted reward 29.720352,predicted upper bound 29.723650,actual reward 32.403549
    round 4866, predicted reward 22.957925,predicted upper bound 22.961133,actual reward 19.918541
    round 4867, predicted reward 29.224451,predicted upper bound 29.227719,actual reward 34.600656
    round 4868, predicted reward 35.461847,predicted upper bound 35.465166,actual reward 37.080134
    round 4869, predicted reward 27.503291,predicted upper bound 27.506787,actual reward 27.505565
    round 4870, predicted reward 22.393780,predicted upper bound 22.397250,actual reward 16.438495
    round 4871, predicted reward 23.075272,predicted upper bound 23.078642,actual reward 17.206229
    round 4872, predicted reward 33.962754,predicted upper bound 33.965943,actual reward 32.735572
    round 4873, predicted reward 23.766670,predicted upper bound 23.770391,actual reward 26.764693
    round 4874, predicted reward 30.864504,predicted upper bound 30.867383,actual reward 26.249856
    round 4875, predicted reward 31.155608,predicted upper bound 31.159079,actual reward 30.308677
    round 4876, predicted reward 34.651025,predicted upper bound 34.654338,actual reward 34.480221
    round 4877, predicted reward 28.467803,predicted upper bound 28.471103,actual reward 30.942015
    round 4878, predicted reward 24.636333,predicted upper bound 24.639783,actual reward 25.254537
    round 4879, predicted reward 26.073606,predicted upper bound 26.077252,actual reward 26.903106
    round 4880, predicted reward 26.408258,predicted upper bound 26.412083,actual reward 22.825524
    round 4881, predicted reward 29.041894,predicted upper bound 29.045024,actual reward 32.389857
    round 4882, predicted reward 29.984961,predicted upper bound 29.988451,actual reward 28.826253
    round 4883, predicted reward 22.250108,predicted upper bound 22.253630,actual reward 20.179341
    round 4884, predicted reward 20.308008,predicted upper bound 20.311882,actual reward 22.074262
    round 4885, predicted reward 24.780694,predicted upper bound 24.784230,actual reward 20.886133
    round 4886, predicted reward 26.366907,predicted upper bound 26.370757,actual reward 20.849453
    round 4887, predicted reward 23.491065,predicted upper bound 23.494332,actual reward 24.185776
    round 4888, predicted reward 26.185157,predicted upper bound 26.188685,actual reward 25.755905
    round 4889, predicted reward 27.146256,predicted upper bound 27.149593,actual reward 25.512882
    round 4890, predicted reward 27.712942,predicted upper bound 27.716970,actual reward 33.344131
    round 4891, predicted reward 27.568282,predicted upper bound 27.571130,actual reward 25.390663
    round 4892, predicted reward 23.759204,predicted upper bound 23.763000,actual reward 23.615929
    round 4893, predicted reward 29.729813,predicted upper bound 29.733411,actual reward 28.314728
    round 4894, predicted reward 25.972023,predicted upper bound 25.975601,actual reward 27.750776
    round 4895, predicted reward 28.578300,predicted upper bound 28.581343,actual reward 30.319038
    round 4896, predicted reward 27.608422,predicted upper bound 27.612038,actual reward 23.747391
    round 4897, predicted reward 29.815515,predicted upper bound 29.818325,actual reward 28.899612
    round 4898, predicted reward 25.557603,predicted upper bound 25.560981,actual reward 22.686880
    round 4899, predicted reward 30.276344,predicted upper bound 30.279751,actual reward 29.701461
    round 4900, predicted reward 25.748641,predicted upper bound 25.751876,actual reward 26.503240
    round 4901, predicted reward 18.389963,predicted upper bound 18.393097,actual reward 16.288632
    round 4902, predicted reward 28.417145,predicted upper bound 28.420761,actual reward 28.242651
    round 4903, predicted reward 19.398351,predicted upper bound 19.402039,actual reward 20.246939
    round 4904, predicted reward 23.980184,predicted upper bound 23.983731,actual reward 26.665233
    round 4905, predicted reward 25.723650,predicted upper bound 25.727359,actual reward 26.416105
    round 4906, predicted reward 25.272159,predicted upper bound 25.275623,actual reward 26.214151
    round 4907, predicted reward 28.299474,predicted upper bound 28.302653,actual reward 22.792790
    round 4908, predicted reward 27.461581,predicted upper bound 27.464459,actual reward 23.504429
    round 4909, predicted reward 32.940043,predicted upper bound 32.943565,actual reward 35.378684
    round 4910, predicted reward 21.391114,predicted upper bound 21.395116,actual reward 21.378244
    round 4911, predicted reward 25.492710,predicted upper bound 25.496115,actual reward 25.663124
    round 4912, predicted reward 26.143684,predicted upper bound 26.146691,actual reward 28.359063
    round 4913, predicted reward 28.560704,predicted upper bound 28.564136,actual reward 29.625494
    round 4914, predicted reward 30.501521,predicted upper bound 30.504691,actual reward 29.151988
    round 4915, predicted reward 26.680913,predicted upper bound 26.684212,actual reward 29.530570
    round 4916, predicted reward 21.557803,predicted upper bound 21.561575,actual reward 17.937623
    round 4917, predicted reward 32.238227,predicted upper bound 32.241447,actual reward 31.614717
    round 4918, predicted reward 20.343105,predicted upper bound 20.346275,actual reward 22.604958
    round 4919, predicted reward 31.854839,predicted upper bound 31.857703,actual reward 36.073819
    round 4920, predicted reward 25.889202,predicted upper bound 25.892871,actual reward 26.980248
    round 4921, predicted reward 32.566478,predicted upper bound 32.569943,actual reward 33.489174
    round 4922, predicted reward 25.565149,predicted upper bound 25.568703,actual reward 21.211837
    round 4923, predicted reward 22.186491,predicted upper bound 22.190236,actual reward 23.118246
    round 4924, predicted reward 28.554177,predicted upper bound 28.557981,actual reward 27.019093
    round 4925, predicted reward 21.322372,predicted upper bound 21.325918,actual reward 17.498192
    round 4926, predicted reward 35.560187,predicted upper bound 35.563177,actual reward 37.987018
    round 4927, predicted reward 36.151319,predicted upper bound 36.154367,actual reward 37.725499
    round 4928, predicted reward 22.368936,predicted upper bound 22.372279,actual reward 19.556997
    round 4929, predicted reward 25.402711,predicted upper bound 25.406062,actual reward 27.312758
    round 4930, predicted reward 25.611790,predicted upper bound 25.615066,actual reward 25.396517
    round 4931, predicted reward 32.242663,predicted upper bound 32.245773,actual reward 29.605116
    round 4932, predicted reward 37.707897,predicted upper bound 37.711117,actual reward 39.750510
    round 4933, predicted reward 28.268564,predicted upper bound 28.271992,actual reward 30.139276
    round 4934, predicted reward 23.723007,predicted upper bound 23.725817,actual reward 26.723093
    round 4935, predicted reward 27.571978,predicted upper bound 27.575485,actual reward 28.207607
    round 4936, predicted reward 28.049276,predicted upper bound 28.052434,actual reward 28.220342
    round 4937, predicted reward 36.413401,predicted upper bound 36.416621,actual reward 41.283683
    round 4938, predicted reward 24.565735,predicted upper bound 24.569391,actual reward 22.854746
    round 4939, predicted reward 35.455397,predicted upper bound 35.458583,actual reward 38.717145
    round 4940, predicted reward 32.833635,predicted upper bound 32.836751,actual reward 37.790202
    round 4941, predicted reward 27.658706,predicted upper bound 27.662121,actual reward 27.390829
    round 4942, predicted reward 32.915593,predicted upper bound 32.918723,actual reward 35.402596
    round 4943, predicted reward 23.740846,predicted upper bound 23.744569,actual reward 25.620008
    round 4944, predicted reward 28.160204,predicted upper bound 28.163059,actual reward 24.352726
    round 4945, predicted reward 32.065714,predicted upper bound 32.068694,actual reward 34.080185
    round 4946, predicted reward 30.802859,predicted upper bound 30.806618,actual reward 27.205220
    round 4947, predicted reward 22.805345,predicted upper bound 22.809251,actual reward 24.782072
    round 4948, predicted reward 30.825497,predicted upper bound 30.828585,actual reward 31.440848
    round 4949, predicted reward 28.222930,predicted upper bound 28.226324,actual reward 28.130807
    round 4950, predicted reward 33.221769,predicted upper bound 33.224954,actual reward 42.295054
    round 4951, predicted reward 25.935678,predicted upper bound 25.939059,actual reward 20.777133
    round 4952, predicted reward 27.831880,predicted upper bound 27.835105,actual reward 24.104806
    round 4953, predicted reward 26.459733,predicted upper bound 26.463351,actual reward 24.577644
    round 4954, predicted reward 27.655663,predicted upper bound 27.659032,actual reward 27.364949
    round 4955, predicted reward 27.171939,predicted upper bound 27.175039,actual reward 29.125195
    round 4956, predicted reward 31.111947,predicted upper bound 31.115485,actual reward 26.945953
    round 4957, predicted reward 24.015302,predicted upper bound 24.019459,actual reward 20.008714
    round 4958, predicted reward 26.475048,predicted upper bound 26.478016,actual reward 21.967803
    round 4959, predicted reward 23.359900,predicted upper bound 23.363590,actual reward 26.021774
    round 4960, predicted reward 20.198063,predicted upper bound 20.201335,actual reward 21.079236
    round 4961, predicted reward 27.944726,predicted upper bound 27.947649,actual reward 26.839641
    round 4962, predicted reward 26.470087,predicted upper bound 26.473485,actual reward 26.062996
    round 4963, predicted reward 25.164131,predicted upper bound 25.167479,actual reward 24.430589
    round 4964, predicted reward 25.014773,predicted upper bound 25.018657,actual reward 23.153182
    round 4965, predicted reward 28.186964,predicted upper bound 28.190753,actual reward 27.108823
    round 4966, predicted reward 24.611970,predicted upper bound 24.614927,actual reward 27.623583
    round 4967, predicted reward 23.980570,predicted upper bound 23.983862,actual reward 23.257094
    round 4968, predicted reward 30.705533,predicted upper bound 30.708994,actual reward 31.272304
    round 4969, predicted reward 25.998794,predicted upper bound 26.002105,actual reward 24.794907
    round 4970, predicted reward 26.056527,predicted upper bound 26.059782,actual reward 26.441597
    round 4971, predicted reward 31.204698,predicted upper bound 31.207723,actual reward 28.128856
    round 4972, predicted reward 23.106125,predicted upper bound 23.109638,actual reward 22.898104
    round 4973, predicted reward 27.188411,predicted upper bound 27.191672,actual reward 25.990747
    round 4974, predicted reward 29.255329,predicted upper bound 29.259073,actual reward 25.954251
    round 4975, predicted reward 21.911887,predicted upper bound 21.915304,actual reward 22.536785
    round 4976, predicted reward 32.403013,predicted upper bound 32.405914,actual reward 32.773971
    round 4977, predicted reward 28.538373,predicted upper bound 28.541957,actual reward 29.091667
    round 4978, predicted reward 23.005786,predicted upper bound 23.009583,actual reward 24.661568
    round 4979, predicted reward 29.063198,predicted upper bound 29.066606,actual reward 28.411031
    round 4980, predicted reward 24.448313,predicted upper bound 24.451607,actual reward 27.048027
    round 4981, predicted reward 24.637468,predicted upper bound 24.640975,actual reward 21.055092
    round 4982, predicted reward 24.000526,predicted upper bound 24.003748,actual reward 22.158658
    round 4983, predicted reward 30.650624,predicted upper bound 30.653494,actual reward 36.843869
    round 4984, predicted reward 25.968667,predicted upper bound 25.971890,actual reward 29.164415
    round 4985, predicted reward 24.518542,predicted upper bound 24.522503,actual reward 22.530659
    round 4986, predicted reward 29.773040,predicted upper bound 29.776358,actual reward 27.114129
    round 4987, predicted reward 28.664226,predicted upper bound 28.667007,actual reward 25.248989
    round 4988, predicted reward 21.920604,predicted upper bound 21.924243,actual reward 15.328395
    round 4989, predicted reward 26.704728,predicted upper bound 26.707877,actual reward 29.511249
    round 4990, predicted reward 33.348367,predicted upper bound 33.350810,actual reward 40.177833
    round 4991, predicted reward 27.510555,predicted upper bound 27.513675,actual reward 27.426223
    round 4992, predicted reward 29.572991,predicted upper bound 29.576145,actual reward 34.939596
    round 4993, predicted reward 23.657027,predicted upper bound 23.660841,actual reward 19.723196
    round 4994, predicted reward 29.905052,predicted upper bound 29.908328,actual reward 28.999252
    round 4995, predicted reward 26.173200,predicted upper bound 26.176490,actual reward 25.089410
    round 4996, predicted reward 24.956617,predicted upper bound 24.960538,actual reward 26.215060
    round 4997, predicted reward 26.046181,predicted upper bound 26.048869,actual reward 22.337862
    round 4998, predicted reward 32.957670,predicted upper bound 32.961174,actual reward 33.827069
    round 4999, predicted reward 29.024906,predicted upper bound 29.028452,actual reward 29.235928

```python
# Best agent benchmark
np.random.seed(12345)

# reward = np.zeros(T)
# X_history = np.zeros((d, T))
# params_history = {}
# grad_history = {}

bestagent = BestAgent(K, T, d)
for tt in range(1, T + 1):
    # observe \{x_{t,a}\}_{a=1}^{k=1}
    context_list = SampleContext(d, K)

    # compute the upper bound of reward
    ind = bestagent.Action(context_list)

    # play ind and observe reward
    reward = GetRealReward(context_list[:, ind], A)
    bestagent.Update(reward)
```

```python
# Uniform agent benchmark
np.random.seed(12345)

uniformagent = UniformAgent(K, T, d)
for tt in range(1, T + 1):
    # observe \{x_{t,a}\}_{a=1}^{k=1}
    context_list = SampleContext(d, K)

    # compute the upper bound of reward
    ind = uniformagent.Action(context_list)

    # play ind and observe reward
    reward = GetRealReward(context_list[:, ind], A)
    uniformagent.Update(reward)
```

```python
import matplotlib.pyplot as plt
h_r_b = bestagent.GetHistoryReward()
plt.plot(range(0, T), np.cumsum(h_r_b))

h_r_u = uniformagent.GetHistoryReward()
plt.plot(range(0, T), np.cumsum(h_r_u))

h_r_n = neuralagent.GetHistoryReward()
plt.plot(range(0, T), np.cumsum(h_r_n))

plt.legend(["Best", "Uniform", "Neural"])
plt.xlabel("Round Index")
plt.ylabel("Total Reward")
```

```python
plt.plot(range(100, T), np.cumsum(h_r_n)[100:T] / np.cumsum(h_r_b)[100:T])
plt.plot(range(100, T), np.cumsum(h_r_u)[100:T] / np.cumsum(h_r_n)[100:T])
plt.legend(["Neural / Best", "Uniform / Neural"])
plt.plot(range(100, T), np.ones(T)[100:T])
plt.plot(range(100, T), 0.9 * np.ones(T)[100:T])
```

```python

```