# Bayesian Temporal Matrix Factorization

**Published**: October 8, 2019

**Author**: Xinyu Chen [[**GitHub homepage**](https://github.com/xinychen)]

**Download**: This Jupyter notebook is at our GitHub repository. If you want to evaluate the code, please download the notebook from the repository of [**tensor-learning**](https://github.com/xinychen/tensor-learning/blob/master/content/BTMF.ipynb).

## Abstract

Large-scale and multidimensional spatiotemporal data sets are becoming ubiquitous in many real-world applications such as monitoring traffic and air quality. Making predictions on these time series has become a critical challenge due to not only the large-scale and high-dimensional nature but also the considerable amount of missing data. In this work, we propose a Bayesian Temporal Matrix Factorization (BTMF) model for modeling multidimensional time series - and in particular spatiotemporal data - in the presence of missing data. By integrating low-rank matrix factorization and vector autoregressive (VAR) process into a single probabilistic graphical model, our model can effectively perform predictions without imputing those missing values. We develop efficient Gibbs sampling algorithms for model inference and test the proposed BTMF on several real-world spatiotemporal data sets for both missing data imputation and short-term rolling prediction tasks. This post is mainly about BTMF models and their **`Python`** implementation with an application of spatiotemporal data imputation.

## 1 Motivation

## 2 Problem Description

We assume a spatiotemporal setting for multidimensional time series data throughout this work. In general, modern spatiotemporal data sets collected from sensor networks can be organized as matrix time series. For example, we can denote by matrix $Y\in\mathbb{R}^{N\times T}$ a multivariate time series collected from $N$ locations/sensors on $T$ time stamps, with each row $$\boldsymbol{y}_{i}=\left(y_{i,1},y_{i,2},...,y_{i,t-1},y_{i,t},y_{i,t+1},...,y_{i,T}\right)$$
corresponding to the time series collected at location $i$.

As mentioned, making accurate predictions on incomplete time series is very challenging, while missing data problem is almost inevitable in real-world applications. Figure 1 illustrates the prediction problem for incomplete time series data. Here we use $(i,t)\in\Omega$ to index the observed entries in matrix $Y$.

> **Figure 1**: Illustration of multivariate time series and the prediction problem in the presence of missing values (green: observed data; white: missing data; red: prediction).

## 3 Model Description

Given a partially observed spatiotemporal matrix $Y\in\mathbb{R}^{N \times T}$, one can factorize it into a spatial factor matrix $W\in\mathbb{R}^{R \times N}$ and a temporal factor matrix $X\in\mathbb{R}^{R \times T}$ following general matrix factorization model:
\begin{equation}
Y\approx W^{\top}X,
\label{btmf_equation1}
\end{equation}
and element-wise, we have
\begin{equation}
y_{it}\approx \boldsymbol{w}_{i}^\top\boldsymbol{x}_{t}, \quad \forall (i,t),
\label{btmf_equation2}
\end{equation}
where vectors $\boldsymbol{w}_{i}$ and $\boldsymbol{x}_{t}$ refer to the $i$-th column of $W$ and the $t$-th column of $X$, respectively.

The standard matrix factorization model is a good approach to deal with the missing data problem; however, it cannot capture the dependencies among different columns in $X$, which are critical in modeling time series data. To better characterize the temporal dependencies and impose temporal smoothness, a novel AR regularizer is introduced on $X$ in TRMF (i.e., Temporal Regularizer Matrix Factorization proposed by [Yu et al., 2016](https://www.cs.utexas.edu/~rofuyu/papers/tr-mf-nips.pdf)):
\begin{equation} \label{equ:VAR}
\begin{aligned}
    \boldsymbol{x}_{t+1}&=\sum\nolimits_{k=1}^{d}A_{k}\boldsymbol{x}_{t+1-h_k}+\boldsymbol{\epsilon}_t, \\
    &=A^\top \boldsymbol{v}_{t+1}+\boldsymbol{\epsilon}_{t}, \\
\end{aligned}
\end{equation}
where $\mathcal{L}=\left\{h_1,\ldots,h_k,\ldots,h_d\right\}$ is a lag set ($d$ is the order of this AR model), each $A_k$ ($k\in\left\{1,...,d\right\}$) is a $R\times R$ coefficient matrix, and $\boldsymbol{\epsilon}_t$ is a zero mean Gaussian noise vector. For brevity, matrix $A\in \mathbb{R}^{(R d) \times R}$ and vector $\boldsymbol{v}_{t+1}\in \mathbb{R}^{(R d) \times 1}$ are defined as
\begin{equation*}
A=\left[A_{1}, \ldots, A_{d}\right]^{\top} ,\quad \boldsymbol{v}_{t+1}=\left[\begin{array}{c}{\boldsymbol{x}_{t+1-h_1}} \\ {\vdots} \\ {\boldsymbol{x}_{t+1-h_d}}\end{array}\right] .
\end{equation*}

> **Figure 2**: A graphical illustration of the rolling prediction scheme using BTMF (with VAR process) (green: observed data; white: missing data; red: prediction).

In [Yu et al., 2016](https://www.cs.utexas.edu/~rofuyu/papers/tr-mf-nips.pdf), to avoid overfitting and reduce the number of parameters, the coefficient matrix in TRMF is further assumed to be a diagonal $A_k=\text{diag}(\boldsymbol{\theta}_{k})$. Therefore, they have
\begin{equation} \label{equ:AR}
\boldsymbol{x}_{t+1}=\boldsymbol{\theta}_{1}\circledast\boldsymbol{x}_{t+1-h_1}+\cdots+\boldsymbol{\theta}_{d}\circledast\boldsymbol{x}_{t+1-h_d}+\boldsymbol{\epsilon}_t,
\end{equation}
where the symbol $\circledast$ denotes the element-wise Hadamard product. However, unlike Equation (4), a vector autoregressive (VAR) model in Equation (3) is actually more powerful for capturing multivariate time series patterns.

> **Figure 3**: A graphical illustration of the rolling prediction scheme using BTMF (with AR process) (green: observed data; white: missing data; red: prediction).

In the following, we first introduce a Bayesian temporal matrix factorization model with an autoregressive model given in Equation (4), and then discuss another model with a vector autoregressive (VAR) model shown in Equation (3).

## 4 Bayesian Sequential Matrix Factorization (BSMF)

## 5 Bayesian Temporal Matrix Factorization with Vector Autoregressive Model

### 5.1 Model Specification

Following the general Bayesian probabilistic matrix factorization models (e.g., BPMF proposed by [Salakhutdinov & Mnih, 2008](https://www.cs.toronto.edu/~amnih/papers/bpmf.pdf)), we assume that each observed entry in $Y$ follows a Gaussian distribution with precision $\tau$:
\begin{equation}
y_{i,t}\sim\mathcal{N}\left(\boldsymbol{w}_i^\top\boldsymbol{x}_t,\tau^{-1}\right),\quad \left(i,t\right)\in\Omega.
\label{btmf_equation3}
\end{equation}

On the spatial dimension, we use a simple Gaussian factor matrix without imposing any dependencies explicitly:
\begin{equation}
\boldsymbol{w}_i\sim\mathcal{N}\left(\boldsymbol{\mu}_{w},\Lambda_w^{-1}\right),
\end{equation}
and we place a conjugate Gaussian-Wishart prior on the mean vector and the precision matrix:
\begin{equation}
\boldsymbol{\mu}_w | \Lambda_w \sim\mathcal{N}\left(\boldsymbol{\mu}_0,(\beta_0\Lambda_w)^{-1}\right),\Lambda_w\sim\mathcal{W}\left(W_0,\nu_0\right),
\end{equation}
where $\boldsymbol{\mu}_0\in \mathbb{R}^{R}$ is a mean vector, $\mathcal{W}\left(W_0,\nu_0\right)$ is a Wishart distribution with a $R\times R$ scale matrix $W_0$ and $\nu_0$ degrees of freedom.

In modeling the temporal factor matrix $X$, we re-write the VAR process as:
\begin{equation}
\begin{aligned}
\boldsymbol{x}_{t}&\sim\begin{cases}
\mathcal{N}\left(\boldsymbol{0},I_R\right),&\text{if $t\in\left\{1,2,...,h_d\right\}$}, \\
\mathcal{N}\left(A^\top \boldsymbol{v}_{t},\Sigma\right),&\text{otherwise},\\
\end{cases}\\
\end{aligned}
\label{btmf_equation5}
\end{equation}

Since the mean vector is defined by VAR, we need to place the conjugate matrix normal inverse Wishart (MNIW) prior on the coefficient matrix $A$ and the covariance matrix $\Sigma$ as follows,
\begin{equation}
\begin{aligned}
A\sim\mathcal{MN}_{(Rd)\times R}\left(M_0,\Psi_0,\Sigma\right),\quad
\Sigma \sim\mathcal{IW}\left(S_0,\nu_0\right), \\
\end{aligned}
\end{equation}
where the probability density function for the $Rd$-by-$R$ random matrix $A$ has the form:
\begin{equation}
\begin{aligned}
&p\left(A\mid M_0,\Psi_0,\Sigma\right) \\
=&\left(2\pi\right)^{-R^2d/2}\left|\Psi_0\right|^{-R/2}\left|\Sigma\right|^{-Rd/2} \\
&\times \exp\left(-\frac{1}{2}\text{tr}\left[\Sigma^{-1}\left(A-M_0\right)^{\top}\Psi_{0}^{-1}\left(A-M_0\right)\right]\right), \\
\end{aligned}
\label{mnpdf}
\end{equation}
where $\Psi_0\in\mathbb{R}^{(Rd)\times (Rd)}$ and $\Sigma\in\mathbb{R}^{R\times R}$ are played as covariance matrices.

For the only remaining parameter $\tau$, we place a Gamma prior  $\tau\sim\text{Gamma}\left(\alpha,\beta\right)$ where $\alpha$ and $\beta$ are the shape and rate parameters, respectively.

The above specifies the full generative process of BTMF, and we could also see the Bayesian graphical model shown in Figure 4. Several parameters are introduced to define the prior distributions for hyperparameters, including $\boldsymbol{\mu}_{0}$, $W_0$, $\nu_0$, $\beta_0$, $\alpha$, $\beta$, $M_0$, $\Psi_0$, and $S_0$. These parameters need to provided in advance when training the model. However, it should be noted that the specification of these parameters has little impact on the final results, as the training data will play a much more important role in defining the posteriors of the hyperparameters.

> **Figure 4**: An overview graphical model of BTMF (time lag set: $\left\{1,2,...,d\right\}$). The shaded nodes ($y_{i,t}$) are the observed data in $\Omega$.

### 5.2 Model Inference

Given the complex structure of BTMF, it is intractable to write down the posterior distribution. Here we rely on the MCMC technique for Bayesian learning. In detail, we introduce a Gibbs sampling algorithm by deriving the full conditional distributions for all parameters and hyperparameters. Thanks to the use of conjugate priors in Figure 4, we can actually write down all the conditional distributions analytically. Below we summarize the Gibbs sampling procedure.

#### 1) Sampling Factor Matrix $W$ and Its Hyperparameters

> For programming convenience, we use $W\in\mathbb{R}^{N\times R}$ to replace $W\in\mathbb{R}^{R\times N}$.

```python
import numpy as np
from numpy.linalg import inv as inv
from numpy.random import multivariate_normal as mvnrnd
from scipy.stats import wishart

def cov_mat(mat):
    new_mat = mat - np.mean(mat, axis = 0)
    return np.einsum('ti, tj -> ij', new_mat, new_mat)

def sample_factor_w(sparse_mat, binary_mat, W, X, tau):
    """Sampling N-by-R factor matrix W and its hyperparameters (mu_w, Lambda_w)."""
    dim1, rank = W.shape
    beta0 = 1
    W_bar = np.mean(W, axis = 0)
    var_mu_hyper = (dim1 * W_bar) / (dim1 + beta0)
    var_W_hyper = inv(np.eye(rank) + cov_mat(W) + dim1 * beta0 / (dim1 + beta0) * np.outer(W_bar, W_bar))
    var_Lambda_hyper = wishart(df = dim1 + rank, scale = var_W_hyper, seed = None).rvs()
    var_mu_hyper = mvnrnd(var_mu_hyper, inv((dim1 + beta0) * var_Lambda_hyper))
    for i in range(dim1):
        pos0 = np.where(sparse_mat[i, :] != 0)
        Xt = X[pos0[0], :]
        var_mu = tau * np.matmul(Xt.T, sparse_mat[i, pos0[0]]) + np.matmul(var_Lambda_hyper, var_mu_hyper)
        inv_var_Lambda = inv(tau * np.matmul(Xt.T, Xt) + var_Lambda_hyper)
        W[i, :] = mvnrnd(np.matmul(inv_var_Lambda, var_mu), inv_var_Lambda)
    return W
```

#### 2) Sampling VAR Coefficients $A$ and Its Hyperparameters

**Foundations of VAR**

Vector autoregression (VAR) is a multivariate extension of autoregression (AR). Formally, VAR for $R$-dimensional vectors $\boldsymbol{x}_{t}$ can be written as follows,
\begin{equation}
\begin{aligned}
\boldsymbol{x}_{t}&=A_{1} \boldsymbol{x}_{t-h_1}+\cdots+A_{d} \boldsymbol{x}_{t-h_d}+\boldsymbol{\epsilon}_{t}, \\
&= A^\top \boldsymbol{v}_{t}+\boldsymbol{\epsilon}_{t},~t=h_d+1, \ldots, T, \\
\end{aligned}
\end{equation}
where
\begin{equation}
A=\left[A_{1}, \ldots, A_{d}\right]^{\top} \in \mathbb{R}^{(R d) \times R},\quad \boldsymbol{v}_{t}=\left[\begin{array}{c}{\boldsymbol{x}_{t-h_1}} \\ {\vdots} \\ {\boldsymbol{x}_{t-h_d}}\end{array}\right] \in \mathbb{R}^{(R d) \times 1}.
\end{equation}

In the following, if we define
\begin{equation}
Z=\left[\begin{array}{c}{\boldsymbol{x}_{h_d+1}^{\top}} \\ {\vdots} \\ {\boldsymbol{x}_{T}^{\top}}\end{array}\right] \in \mathbb{R}^{(T-h_d) \times R},\quad Q=\left[\begin{array}{c}{\boldsymbol{v}_{h_d+1}^{\top}} \\ {\vdots} \\ {\boldsymbol{v}_{T}^{\top}}\end{array}\right] \in \mathbb{R}^{(T-h_d) \times(R d)},
\end{equation}
then, we could write the above mentioned VAR as
\begin{equation}
\underbrace{Z}_{(T-h_d)\times R}\approx \underbrace{Q}_{(T-h_d)\times (Rd)}\times \underbrace{A}_{(Rd)\times R}.
\end{equation}

> To include temporal factors $\boldsymbol{x}_{t},t=1,...,h_d$, we also define $$Z_0=\left[\begin{array}{c}{\boldsymbol{x}_{1}^{\top}} \\ {\vdots} \\ {\boldsymbol{x}_{h_d}^{\top}}\end{array}\right] \in \mathbb{R}^{h_d \times R}.$$

**Build a Bayesian VAR on temporal factors $\boldsymbol{x}_{t}$**
\begin{equation}
\begin{aligned}
\boldsymbol{x}_{t}&\sim\begin{cases}\mathcal{N}\left(A^\top \boldsymbol{v}_{t},\Sigma\right),~\text{if $t\in\left\{h_d+1,...,T\right\}$},\\{\mathcal{N}\left(\boldsymbol{0},I_R\right),~\text{otherwise}}.\end{cases}\\
A&\sim\mathcal{MN}_{(Rd)\times R}\left(M_0,\Psi_0,\Sigma\right), \\
\Sigma &\sim\mathcal{IW}\left(S_0,\nu_0\right), \\
\end{aligned}
\end{equation}
where
\begin{equation}
\begin{aligned}
&\mathcal{M N}_{(R d) \times R}\left(A | M_{0}, \Psi_{0}, \Sigma\right)\\
\propto|&\Sigma|^{-R d / 2} \exp \left(-\frac{1}{2} \operatorname{tr}\left[\Sigma^{-1}\left(A-M_{0}\right)^{\top} \Psi_{0}^{-1}\left(A-M_{0}\right)\right]\right), \\
\end{aligned}
\end{equation}
and
\begin{equation}
\mathcal{I} \mathcal{W}\left(\Sigma | S_{0}, \nu_{0}\right) \propto|\Sigma|^{-\left(\nu_{0}+R+1\right) / 2} \exp \left(-\frac{1}{2} \operatorname{tr}\left(\Sigma^{-1}S_{0}\right)\right).
\end{equation}

**Likelihood from temporal factors $\boldsymbol{x}_{t}$**
\begin{equation}
\begin{aligned}
&\mathcal{L}\left(X\mid A,\Sigma\right) \\
\propto &\prod_{t=1}^{h_d}p\left(\boldsymbol{x}_{t}\mid \Sigma\right)\times \prod_{t=h_d+1}^{T}p\left(\boldsymbol{x}_{t}\mid A,\Sigma\right) \\
\propto &\left|\Sigma\right|^{-T/2}\exp\left\{-\frac{1}{2}\sum_{t=h_d+1}^{T}\left(\boldsymbol{x}_{t}-A^\top \boldsymbol{v}_{t}\right)^\top\Sigma^{-1}\left(\boldsymbol{x}_{t}-A^\top \boldsymbol{v}_{t}\right)\right\} \\
\propto &\left|\Sigma\right|^{-T/2}\exp\left\{-\frac{1}{2}\text{tr}\left[\Sigma^{-1}\left(Z_0^\top Z_0+\left(Z-QA\right)^\top \left(Z-QA\right)\right)\right]\right\}
\end{aligned}
\end{equation}

**Posterior distribution**

Consider
\begin{equation}
\begin{aligned}
&\left(A-M_{0}\right)^{\top} \Psi_{0}^{-1}\left(A-M_{0}\right)+S_0+Z_0^\top Z_0+\left(Z-QA\right)^\top \left(Z-QA\right) \\
=&A^\top\left(\Psi_0^{-1}+Q^\top Q\right)A-A^\top\left(\Psi_0^{-1}M_0+Q^\top Z\right) \\
&-\left(\Psi_0^{-1}M_0+Q^\top Z\right)^\top A \\
&+\left(\Psi_0^{-1}M_0+Q^\top Z\right)^\top\left(\Psi_0^{-1}+Q^\top Q\right)\left(\Psi_0^{-1}M_0+Q^\top Z\right) \\
&-\left(\Psi_0^{-1}M_0+Q^\top Z\right)^\top\left(\Psi_0^{-1}+Q^\top Q\right)\left(\Psi_0^{-1}M_0+Q^\top Z\right) \\
&+M_0^\top\Psi_0^{-1}M_0+S_0+Z_0^\top Z_0+Z^\top Z \\
=&\left(A-M^{*}\right)^\top\left(\Psi^{*}\right)^{-1}\left(A-M^{*}\right)+S^{*}, \\
\end{aligned}
\end{equation}
which is in the form of $\mathcal{MN}\left(\cdot\right)$ and $\mathcal{IW}\left(\cdot\right)$.

The $Rd$-by-$R$ matrix $A$ has a matrix normal distribution, and $R$-by-$R$ covariance matrix $\Sigma$ has an inverse Wishart distribution, that is,
\begin{equation}
A \sim \mathcal{M N}_{(R d) \times R}\left(M^{*}, \Psi^{*}, \Sigma\right), \quad \Sigma \sim \mathcal{I} \mathcal{W}\left(S^{*}, \nu^{*}\right),
\end{equation}
with
\begin{equation}
\begin{cases}
{\Psi^{*}=\left(\Psi_{0}^{-1}+Q^{\top} Q\right)^{-1}}, \\ {M^{*}=\Psi^{*}\left(\Psi_{0}^{-1} M_{0}+Q^{\top} Z\right)}, \\ {S^{*}=S_{0}+Z^\top Z+M_0^\top\Psi_0^{-1}M_0-\left(M^{*}\right)^\top\left(\Psi^{*}\right)^{-1}M^{*}}, \\
{\nu^{*}=\nu_{0}+T-h_d}.
\end{cases}
\end{equation}

```python
from scipy.stats import invwishart

def mnrnd(M, U, V):
    """
    Generate matrix normal distributed random matrix.
    M is a m-by-n matrix, U is a m-by-m matrix, and V is a n-by-n matrix.
    """
    dim1, dim2 = M.shape
    X0 = np.random.rand(dim1, dim2)
    P = np.linalg.cholesky(U)
    Q = np.linalg.cholesky(V)
    return M + np.matmul(np.matmul(P, X0), Q.T)

def sample_var_coefficient(X, time_lags):
    dim2, rank = X.shape
    d = time_lags.shape[0]
    Z_mat = X[np.max(time_lags) : dim2, :]
    Q_mat = X[np.max(time_lags) - time_lags[0] : dim2 - time_lags[0], :]
    for k in range(1, d):
        Q_mat = np.append(Q_mat, X[np.max(time_lags) - time_lags[k] : dim2 - time_lags[k], :], axis = 1)
    var_Psi = inv(np.eye(rank * d) + np.matmul(Q_mat.T, Q_mat))
    var_M = np.matmul(var_Psi, np.matmul(Q_mat.T, Z_mat))
    var_S = (np.eye(rank) + np.matmul(Z_mat.T, Z_mat) - np.matmul(np.matmul(var_M.T, inv(var_Psi)), var_M))
    Sigma = invwishart(df = rank + dim2 - np.max(time_lags), scale = var_S, seed = None).rvs()
    return mnrnd(var_M, var_Psi, Sigma), Sigma
```

#### 3) Sampling Factor Matrix $X$

**Posterior distribution**
\begin{equation}
\begin{aligned}
y_{it}&\sim\mathcal{N}\left(\boldsymbol{w}_{i}^\top\boldsymbol{x}_{t},\tau^{-1}\right),~\left(i,t\right)\in\Omega, \\
\boldsymbol{x}_{t}&\sim\begin{cases}\mathcal{N}\left(\sum_{k=1}^{d}A_{k} \boldsymbol{x}_{t-h_k},\Sigma\right),~\text{if $t\in\left\{h_d+1,...,T\right\}$},\\{\mathcal{N}\left(\boldsymbol{0},I\right),~\text{otherwise}}.\end{cases}\\
\end{aligned}
\end{equation}

If $t\in\left\{1,...,h_d\right\}$, parameters of the posterior distribution $\mathcal{N}\left(\boldsymbol{x}_{t}\mid \boldsymbol{\mu}_{t}^{*},\Sigma_{t}^{*}\right)$ are
\footnotesize{
\begin{equation}
\begin{aligned}
\Sigma_{t}^{*}&=\left(\sum_{k=1, h_{d}<t+h_{k} \leq T}^{d} {A}_{k}^{\top} \Sigma^{-1} A_{k}+\tau\sum_{i:(i,t)\in\Omega}\boldsymbol{w}_{i}\boldsymbol{w}_{i}^\top+I\right)^{-1}, \\
\boldsymbol{\mu}_{t}^{*}&=\Sigma_{t}^{*}\left(\sum_{k=1, h_{d}<t+h_{k} \leq T}^{d} A_{k}^{\top} \Sigma^{-1} \boldsymbol{\psi}_{t+h_{k}}+\tau\sum_{i:(i,t)\in\Omega}\boldsymbol{w}_{i}y_{it}\right). \\
\end{aligned}
\end{equation}

If $t\in\left\{h_d+1,...,T\right\}$, then parameters of the posterior distribution $\mathcal{N}\left(\boldsymbol{x}_{t}\mid \boldsymbol{\mu}_{t}^{*},\Sigma_{t}^{*}\right)$ are
\begin{equation}
\begin{aligned}
\Sigma_{t}^{*}&=\left(\sum_{k=1, h_{d}<t+h_{k} \leq T}^{d} {A}_{k}^{\top} \Sigma^{-1} A_{k}+\tau\sum_{i:(i,t)\in\Omega}\boldsymbol{w}_{i}\boldsymbol{w}_{i}^\top+\Sigma^{-1}\right)^{-1}, \\
\boldsymbol{\mu}_{t}^{*}&=\Sigma_{t}^{*}\left(\sum_{k=1, h_{d}<t+h_{k} \leq T}^{d} A_{k}^{\top} \Sigma^{-1} \boldsymbol{\psi}_{t+h_{k}}+\tau\sum_{i:(i,t)\in\Omega}\boldsymbol{w}_{i}y_{it}+\Sigma^{-1}\sum_{k=1}^{d}A_{k}\boldsymbol{x}_{t-h_k}\right), \\
\end{aligned}
\end{equation}
where
$$\boldsymbol{\psi}_{t+h_k}=\boldsymbol{x}_{t+h_k}-\sum_{l=1,l\neq k}^{d}A_{l}\boldsymbol{x}_{t+h_k-h_l}.$$

```python
def sample_factor_x(sparse_mat, binary_mat, time_lags, W, X, tau, A, Lambda_x):
    dim2, rank = X.shape
    d = time_lags.shape[0]
    mat0 = np.matmul(Lambda_x, A.T)
    mat1 = np.zeros((rank, rank, d))
    mat2 = np.zeros((rank, rank))
    for k in range(d):
        Ak = A[k * rank : (k + 1) * rank, :]
        mat1[:, :, k] = np.matmul(Ak, Lambda_x)
        mat2 += np.matmul(mat1[:, :, k], Ak.T)
    for t in range(dim2):
        pos0 = np.where(sparse_mat[:, t] != 0)
        Wt = W[pos0[0], :]
        Nt = np.zeros(rank)
        if t >= np.max(time_lags):
            Qt = np.matmul(mat0, X[t - time_lags, :].reshape([rank * d]))
            if t < dim2 - np.max(time_lags):
                Mt = mat2.copy()
                for k in range(d):
                    A0 = A.copy()
                    A0[k * rank : (k + 1) * rank, :] = 0
                    var5 = (X[t + time_lags[k], :]
                            - np.matmul(A0.T, X[t + time_lags[k]
                                                - time_lags, :].reshape([rank * d])))
                    Nt += np.matmul(mat1[:, :, k], var5)
            elif t >= dim2 - np.max(time_lags) and t < dim2 - np.min(time_lags):
                index = list(np.where(t + time_lags < dim2))[0]
                Mt = np.zeros((rank, rank))
                for k in index:
                    Ak = A[k * rank : (k + 1) * rank, :]
                    Mt += np.matmul(np.matmul(Ak, Lambda_x), Ak.T)
                    A0 = A.copy()
                    A0[k * rank : (k + 1) * rank, :] = 0
                    var5 = (X[t + time_lags[k], :]
                            - np.matmul(A0.T, X[t + time_lags[k]
                                                - time_lags, :].reshape([rank * d])))
                    Nt += np.matmul(np.matmul(Ak, Lambda_x), var5)
            inv_var_Lambda = inv(tau * np.matmul(Wt.T, Wt) + Mt + Lambda_x)
        elif t < np.max(time_lags):
            Qt = np.zeros(rank)
            index = list(np.where(t + time_lags >= np.max(time_lags)))[0]
            Mt = np.zeros((rank, rank))
            for k in index:
                Ak = A[k * rank : (k + 1) * rank, :]
                Mt += np.matmul(np.matmul(Ak, Lambda_x), Ak.T)
                A0 = A.copy()
                A0[k * rank : (k + 1) * rank, :] = 0
                var5 = (X[t + time_lags[k], :]
                        - np.matmul(A0.T, X[t + time_lags[k]
                                            - time_lags, :].reshape([rank * d])))
                Nt += np.matmul(np.matmul(Ak, Lambda_x), var5)
            inv_var_Lambda = inv(tau * np.matmul(Wt.T, Wt) + Mt + np.eye(rank))
        var_mu = tau * np.matmul(Wt.T, sparse_mat[pos0[0], t]) + Nt + Qt
        X[t, :] = mvnrnd(np.matmul(inv_var_Lambda, var_mu), inv_var_Lambda)
    return X
```

```python
def sample_factor_x(sparse_mat, binary_mat, time_lags, W, X, tau, A, Lambda_x):
    dim2, rank = X.shape
    d = time_lags.shape[0]
    for t in range(dim2):
        pos0 = np.where(sparse_mat[:, t] != 0)
        Wt = W[pos0[0], :]
        Mt = np.zeros((rank, rank))
        Nt = np.zeros(rank)
        if t >= np.max(time_lags):
            Qt = np.matmul(Lambda_x, np.matmul(A.T, X[t - time_lags, :].reshape([rank * d])))
        if t >= np.max(time_lags) and t < dim2 - np.max(time_lags):
            index = list(range(0, d))
        elif t >= dim2 - np.max(time_lags) and t < dim2 - np.min(time_lags):
            index = list(np.where(t + time_lags < dim2))[0]
        elif t < np.max(time_lags):
            Qt = np.zeros(rank)
            index = list(np.where(t + time_lags >= np.max(time_lags)))[0]
        if t < dim2 - np.min(time_lags):
            for k in index:
                Ak = A[k * rank : (k + 1) * rank, :]
                Mt += np.matmul(np.matmul(Ak, Lambda_x), Ak.T)
                A0 = A.copy()
                A0[k * rank : (k + 1) * rank, :] = 0
                var5 = (X[t + time_lags[k], :]
                        - np.matmul(A0.T, X[t + time_lags[k] - time_lags, :].reshape([rank * d])))
                Nt += np.matmul(np.matmul(Ak, Lambda_x), var5)
        var_mu = tau * np.matmul(Wt.T, sparse_mat[pos0[0], t]) + Nt + Qt
        if t < np.max(time_lags):
            inv_var_Lambda = inv(tau * np.matmul(Wt.T, Wt) + Mt + np.eye(rank))
        else:
            inv_var_Lambda = inv(tau * np.matmul(Wt.T, Wt) + Mt + Lambda_x)
        X[t, :] = mvnrnd(np.matmul(inv_var_Lambda, var_mu), inv_var_Lambda)
    return X
```

#### 4) Sampling Precision $\tau$

```python
def sample_precision_tau(sparse_mat, mat_hat, position):
    var_alpha = 1e-6 + 0.5 * sparse_mat[position].shape[0]
    var_beta = 1e-6 + 0.5 * np.sum((sparse_mat - mat_hat)[position] ** 2)
    return np.random.gamma(var_alpha, 1 / var_beta)
```

#### 5) BTMF Implementation

- **Gibbs sampling**

    - Burn-in process
    - Sampling process

- **Imputation**

- **Prediction**

```python
def BTMF(dense_mat, sparse_mat, init, rank, time_lags, burn_iter, gibbs_iter):
    """Bayesian Temporal Matrix Factorization, BTMF."""

    W = init["W"]
    X = init["X"]
    dim1, dim2 = sparse_mat.shape
    d = time_lags.shape[0]
    pos = np.where((dense_mat != 0) & (sparse_mat == 0))
    position = np.where(sparse_mat != 0)
    binary_mat = np.zeros((dim1, dim2))
    binary_mat[position] = 1
    tau = 1
    mat_hat_plus = np.zeros((dim1, dim2))
    for it in range(burn_iter + gibbs_iter):
        W = sample_factor_w(sparse_mat, binary_mat, W, X, tau)
        A, Sigma = sample_var_coefficient(X, time_lags)
        X = sample_factor_x(sparse_mat, binary_mat, time_lags, W, X, tau, A, inv(Sigma))
        mat_hat = np.matmul(W, X.T)
        tau = sample_precision_tau(sparse_mat, mat_hat, position)
        rmse = np.sqrt(np.sum((dense_mat[pos] - mat_hat[pos]) ** 2) / dense_mat[pos].shape[0])
        if (it + 1) % 1 == 0 and it < burn_iter:
            print('Iteration: {}'.format(it + 1))
            print('RMSE: {:.6}'.format(rmse))
            print()
        if it + 1 > burn_iter:
            mat_hat_plus += mat_hat
    mat_hat = mat_hat_plus / gibbs_iter
    final_mape = np.sum(np.abs(dense_mat[pos] - mat_hat[pos]) / dense_mat[pos]) / dense_mat[pos].shape[0]
    final_rmse = np.sqrt(np.sum((dense_mat[pos] - mat_hat[pos]) ** 2) / dense_mat[pos].shape[0])
    print('Imputation MAPE: {:.6}'.format(final_mape))
    print('Imputation RMSE: {:.6}'.format(final_rmse))
    print()
    return mat_hat
```

## 6 Spatiotemporal Missing Data Imputation

```python
import scipy.io

tensor = scipy.io.loadmat('../datasets/Guangzhou-data-set/tensor.mat')
tensor = tensor['tensor']
random_matrix = scipy.io.loadmat('../datasets/Guangzhou-data-set/random_matrix.mat')
random_matrix = random_matrix['random_matrix']
random_tensor = scipy.io.loadmat('../datasets/Guangzhou-data-set/random_tensor.mat')
random_tensor = random_tensor['random_tensor']

dense_mat = tensor.reshape([tensor.shape[0], tensor.shape[1] * tensor.shape[2]])
missing_rate = 0.4

# =============================================================================
### Random missing (RM) scenario
### Set the RM scenario by:
binary_mat = (np.round(random_tensor + 0.5 - missing_rate)
              .reshape([random_tensor.shape[0], random_tensor.shape[1] * random_tensor.shape[2]]))
# =============================================================================

# binary_tensor = np.zeros(tensor.shape)
# for i1 in range(tensor.shape[0]):
#     for i2 in range(tensor.shape[1]):
#         binary_tensor[i1, i2, :] = np.round(random_matrix[i1, i2] + 0.5 - missing_rate)
# binary_mat = binary_tensor.reshape([binary_tensor.shape[0], binary_tensor.shape[1] * binary_tensor.shape[2]])

sparse_mat = np.multiply(dense_mat, binary_mat)
```

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 10
time_lags = np.array([1, 2, 144])
init = {"W": 0.1 * np.random.rand(dim1, rank), "X": 0.1 * np.random.rand(dim2, rank)}
burn_iter = 1000
gibbs_iter = 100
BTMF(dense_mat, sparse_mat, init, rank, time_lags, burn_iter, gibbs_iter)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

    Iteration: 1
    RMSE: 6.54248

    Iteration: 2
    RMSE: 5.76746

    Iteration: 3
    RMSE: 5.7331

    Iteration: 4
    RMSE: 5.74967

    Iteration: 5
    RMSE: 5.48408

    Iteration: 6
    RMSE: 5.36829

    Iteration: 7
    RMSE: 4.93222

    Iteration: 8
    RMSE: 4.70856

    Iteration: 9
    RMSE: 4.66498

    Iteration: 10
    RMSE: 4.74202

    Iteration: 11
    RMSE: 4.69944

    Iteration: 12
    RMSE: 4.64699

    Iteration: 13
    RMSE: 4.58376

    Iteration: 14
    RMSE: 4.56541

    Iteration: 15
    RMSE: 4.57934

    Iteration: 16
    RMSE: 4.53475

    Iteration: 17
    RMSE: 4.54741

    Iteration: 18
    RMSE: 4.60075

    Iteration: 19
    RMSE: 4.63094

    Iteration: 20
    RMSE: 4.59308

    Iteration: 21
    RMSE: 4.5828

    Iteration: 22
    RMSE: 4.54219

    Iteration: 23
    RMSE: 4.51025

    Iteration: 24
    RMSE: 4.482

    Iteration: 25
    RMSE: 4.46875

    Iteration: 26
    RMSE: 4.46078

    Iteration: 27
    RMSE: 4.45712

    Iteration: 28
    RMSE: 4.45436

    Iteration: 29
    RMSE: 4.4563

    Iteration: 30
    RMSE: 4.45248

    Iteration: 31
    RMSE: 4.45449

    Iteration: 32
    RMSE: 4.45683

    Iteration: 33
    RMSE: 4.4544

    Iteration: 34
    RMSE: 4.45835

    Iteration: 35
    RMSE: 4.46065

    Iteration: 36
    RMSE: 4.45809

    Iteration: 37
    RMSE: 4.46387

    Iteration: 38
    RMSE: 4.4565

    Iteration: 39
    RMSE: 4.46289

    Iteration: 40
    RMSE: 4.46492

    Iteration: 41
    RMSE: 4.46722

    Iteration: 42
    RMSE: 4.46884

    Iteration: 43
    RMSE: 4.46624

    Iteration: 44
    RMSE: 4.4711

    Iteration: 45
    RMSE: 4.47314

    Iteration: 46
    RMSE: 4.47297

    Iteration: 47
    RMSE: 4.47793

    Iteration: 48
    RMSE: 4.47917

    Iteration: 49
    RMSE: 4.48087

    Iteration: 50
    RMSE: 4.48217

    Iteration: 51
    RMSE: 4.48399

    Iteration: 52
    RMSE: 4.48549

    Iteration: 53
    RMSE: 4.48188

    Iteration: 54
    RMSE: 4.48285

    Iteration: 55
    RMSE: 4.48434

    Iteration: 56
    RMSE: 4.48593

    Iteration: 57
    RMSE: 4.49323

    Iteration: 58
    RMSE: 4.48787

    Iteration: 59
    RMSE: 4.49444

    Iteration: 60
    RMSE: 4.49477

    Iteration: 61
    RMSE: 4.49183

    Iteration: 62
    RMSE: 4.48968

    Iteration: 63
    RMSE: 4.49057

    Iteration: 64
    RMSE: 4.49384

    Iteration: 65
    RMSE: 4.49177

    Iteration: 66
    RMSE: 4.49185

    Iteration: 67
    RMSE: 4.49192

    Iteration: 68
    RMSE: 4.48892

    Iteration: 69
    RMSE: 4.48786

    Iteration: 70
    RMSE: 4.49479

    Iteration: 71
    RMSE: 4.49446

    Iteration: 72
    RMSE: 4.49888

    Iteration: 73
    RMSE: 4.49949

    Iteration: 74
    RMSE: 4.49848

    Iteration: 75
    RMSE: 4.49856

    Iteration: 76
    RMSE: 4.49747

    Iteration: 77
    RMSE: 4.50231

    Iteration: 78
    RMSE: 4.49866

    Iteration: 79
    RMSE: 4.49953

    Iteration: 80
    RMSE: 4.49757

    Iteration: 81
    RMSE: 4.49981

    Iteration: 82
    RMSE: 4.49921

    Iteration: 83
    RMSE: 4.49948

    Iteration: 84
    RMSE: 4.49625

    Iteration: 85
    RMSE: 4.50165

    Iteration: 86
    RMSE: 4.50148

    Iteration: 87
    RMSE: 4.50663

    Iteration: 88
    RMSE: 4.50327

    Iteration: 89
    RMSE: 4.50887

    Iteration: 90
    RMSE: 4.50576

    Iteration: 91
    RMSE: 4.5023

    Iteration: 92
    RMSE: 4.50629

    Iteration: 93
    RMSE: 4.50923

    Iteration: 94
    RMSE: 4.51307

    Iteration: 95
    RMSE: 4.50818

    Iteration: 96
    RMSE: 4.50953

    Iteration: 97
    RMSE: 4.50612

    Iteration: 98
    RMSE: 4.50728

    Iteration: 99
    RMSE: 4.50728

    Iteration: 100
    RMSE: 4.50946

    Iteration: 101
    RMSE: 4.50996

    Iteration: 102
    RMSE: 4.51108

    Iteration: 103
    RMSE: 4.5158

    Iteration: 104
    RMSE: 4.51864

    Iteration: 105
    RMSE: 4.51461

    Iteration: 106
    RMSE: 4.51631

    Iteration: 107
    RMSE: 4.509

    Iteration: 108
    RMSE: 4.50882

    Iteration: 109
    RMSE: 4.51614

    Iteration: 110
    RMSE: 4.51157

    Iteration: 111
    RMSE: 4.51044

    Iteration: 112
    RMSE: 4.50869

    Iteration: 113
    RMSE: 4.50927

    Iteration: 114
    RMSE: 4.50959

    Iteration: 115
    RMSE: 4.51389

    Iteration: 116
    RMSE: 4.51256

    Iteration: 117
    RMSE: 4.51325

    Iteration: 118
    RMSE: 4.50974

    Iteration: 119
    RMSE: 4.51655

    Iteration: 120
    RMSE: 4.5115

    Iteration: 121
    RMSE: 4.51645

    Iteration: 122
    RMSE: 4.51753

    Iteration: 123
    RMSE: 4.51597

    Iteration: 124
    RMSE: 4.51971

    Iteration: 125
    RMSE: 4.52052

    Iteration: 126
    RMSE: 4.51515

    Iteration: 127
    RMSE: 4.51936

    Iteration: 128
    RMSE: 4.52494

    Iteration: 129
    RMSE: 4.52475

    Iteration: 130
    RMSE: 4.52563

    Iteration: 131
    RMSE: 4.5213

    Iteration: 132
    RMSE: 4.51497

    Iteration: 133
    RMSE: 4.50955

    Iteration: 134
    RMSE: 4.51106

    Iteration: 135
    RMSE: 4.51367

    Iteration: 136
    RMSE: 4.51811

    Iteration: 137
    RMSE: 4.51519

    Iteration: 138
    RMSE: 4.5184

    Iteration: 139
    RMSE: 4.51618

    Iteration: 140
    RMSE: 4.51715

    Iteration: 141
    RMSE: 4.5148

    Iteration: 142
    RMSE: 4.51163

    Iteration: 143
    RMSE: 4.51101

    Iteration: 144
    RMSE: 4.50948

    Iteration: 145
    RMSE: 4.50372

    Iteration: 146
    RMSE: 4.50677

    Iteration: 147
    RMSE: 4.51051

    Iteration: 148
    RMSE: 4.51091

    Iteration: 149
    RMSE: 4.512

    Iteration: 150
    RMSE: 4.51022

    Iteration: 151
    RMSE: 4.51022

    Iteration: 152
    RMSE: 4.50981

    Iteration: 153
    RMSE: 4.51303

    Iteration: 154
    RMSE: 4.51394

    Iteration: 155
    RMSE: 4.51371

    Iteration: 156
    RMSE: 4.50995

    Iteration: 157
    RMSE: 4.51253

    Iteration: 158
    RMSE: 4.51617

    Iteration: 159
    RMSE: 4.51278

    Iteration: 160
    RMSE: 4.52169

    Iteration: 161
    RMSE: 4.52016

    Iteration: 162
    RMSE: 4.51821

    Iteration: 163
    RMSE: 4.51732

    Iteration: 164
    RMSE: 4.51899

    Iteration: 165
    RMSE: 4.52507

    Iteration: 166
    RMSE: 4.5249

    Iteration: 167
    RMSE: 4.52547

    Iteration: 168
    RMSE: 4.52445

    Iteration: 169
    RMSE: 4.52202

    Iteration: 170
    RMSE: 4.52317

    Iteration: 171
    RMSE: 4.52458

    Iteration: 172
    RMSE: 4.52301

    Iteration: 173
    RMSE: 4.52225

    Iteration: 174
    RMSE: 4.5229

    Iteration: 175
    RMSE: 4.51735

    Iteration: 176
    RMSE: 4.5254

    Iteration: 177
    RMSE: 4.51979

    Iteration: 178
    RMSE: 4.52697

    Iteration: 179
    RMSE: 4.51904

    Iteration: 180
    RMSE: 4.51799

    Iteration: 181
    RMSE: 4.52044

    Iteration: 182
    RMSE: 4.52175

    Iteration: 183
    RMSE: 4.52076

    Iteration: 184
    RMSE: 4.5163

    Iteration: 185
    RMSE: 4.51807

    Iteration: 186
    RMSE: 4.51643

    Iteration: 187
    RMSE: 4.521

    Iteration: 188
    RMSE: 4.52175

    Iteration: 189
    RMSE: 4.51775

    Iteration: 190
    RMSE: 4.51784

    Iteration: 191
    RMSE: 4.51418

    Iteration: 192
    RMSE: 4.51654

    Iteration: 193
    RMSE: 4.51179

    Iteration: 194
    RMSE: 4.51213

    Iteration: 195
    RMSE: 4.50986

    Iteration: 196
    RMSE: 4.51197

    Iteration: 197
    RMSE: 4.51769

    Iteration: 198
    RMSE: 4.51498

    Iteration: 199
    RMSE: 4.51622

    Iteration: 200
    RMSE: 4.51443

    Iteration: 201
    RMSE: 4.51696

    Iteration: 202
    RMSE: 4.51898

    Iteration: 203
    RMSE: 4.51956

    Iteration: 204
    RMSE: 4.52316

    Iteration: 205
    RMSE: 4.51832

    Iteration: 206
    RMSE: 4.51886

    Iteration: 207
    RMSE: 4.52576

    Iteration: 208
    RMSE: 4.5217

    Iteration: 209
    RMSE: 4.5252

    Iteration: 210
    RMSE: 4.5261

    Iteration: 211
    RMSE: 4.52519

    Iteration: 212
    RMSE: 4.52557

    Iteration: 213
    RMSE: 4.52475

    Iteration: 214
    RMSE: 4.52106

    Iteration: 215
    RMSE: 4.52829

    Iteration: 216
    RMSE: 4.53196

    Iteration: 217
    RMSE: 4.52872

    Iteration: 218
    RMSE: 4.53156

    Iteration: 219
    RMSE: 4.52768

    Iteration: 220
    RMSE: 4.52834

    Iteration: 221
    RMSE: 4.53278

    Iteration: 222
    RMSE: 4.53164

    Iteration: 223
    RMSE: 4.52732

    Iteration: 224
    RMSE: 4.53258

    Iteration: 225
    RMSE: 4.52714

    Iteration: 226
    RMSE: 4.52354

    Iteration: 227
    RMSE: 4.52532

    Iteration: 228
    RMSE: 4.52041

    Iteration: 229
    RMSE: 4.52318

    Iteration: 230
    RMSE: 4.51986

    Iteration: 231
    RMSE: 4.52249

    Iteration: 232
    RMSE: 4.52028

    Iteration: 233
    RMSE: 4.52147

    Iteration: 234
    RMSE: 4.51878

    Iteration: 235
    RMSE: 4.52191

    Iteration: 236
    RMSE: 4.52596

    Iteration: 237
    RMSE: 4.52439

    Iteration: 238
    RMSE: 4.52472

    Iteration: 239
    RMSE: 4.52445

    Iteration: 240
    RMSE: 4.525

    Iteration: 241
    RMSE: 4.52026

    Iteration: 242
    RMSE: 4.52282

    Iteration: 243
    RMSE: 4.5235

    Iteration: 244
    RMSE: 4.52011

    Iteration: 245
    RMSE: 4.52712

    Iteration: 246
    RMSE: 4.52684

    Iteration: 247
    RMSE: 4.52991

    Iteration: 248
    RMSE: 4.52969

    Iteration: 249
    RMSE: 4.53143

    Iteration: 250
    RMSE: 4.51897

    Iteration: 251
    RMSE: 4.52219

    Iteration: 252
    RMSE: 4.52754

    Iteration: 253
    RMSE: 4.52623

    Iteration: 254
    RMSE: 4.52428

    Iteration: 255
    RMSE: 4.53503

    Iteration: 256
    RMSE: 4.53656

    Iteration: 257
    RMSE: 4.53299

    Iteration: 258
    RMSE: 4.52972

    Iteration: 259
    RMSE: 4.5281

    Iteration: 260
    RMSE: 4.52628

    Iteration: 261
    RMSE: 4.52702

    Iteration: 262
    RMSE: 4.52688

    Iteration: 263
    RMSE: 4.52302

    Iteration: 264
    RMSE: 4.52649

    Iteration: 265
    RMSE: 4.52815

    Iteration: 266
    RMSE: 4.52705

    Iteration: 267
    RMSE: 4.52549

    Iteration: 268
    RMSE: 4.52887

    Iteration: 269
    RMSE: 4.52597

    Iteration: 270
    RMSE: 4.52783

    Iteration: 271
    RMSE: 4.52954

    Iteration: 272
    RMSE: 4.52559

    Iteration: 273
    RMSE: 4.5254

    Iteration: 274
    RMSE: 4.53485

    Iteration: 275
    RMSE: 4.53453

    Iteration: 276
    RMSE: 4.53197

    Iteration: 277
    RMSE: 4.53227

    Iteration: 278
    RMSE: 4.53732

    Iteration: 279
    RMSE: 4.5352

    Iteration: 280
    RMSE: 4.53879

    Iteration: 281
    RMSE: 4.53062

    Iteration: 282
    RMSE: 4.53645

    Iteration: 283
    RMSE: 4.53852

    Iteration: 284
    RMSE: 4.53585

    Iteration: 285
    RMSE: 4.52965

    Iteration: 286
    RMSE: 4.5337

    Iteration: 287
    RMSE: 4.52432

    Iteration: 288
    RMSE: 4.5294

    Iteration: 289
    RMSE: 4.53376

    Iteration: 290
    RMSE: 4.52812

    Iteration: 291
    RMSE: 4.5261

    Iteration: 292
    RMSE: 4.5263

    Iteration: 293
    RMSE: 4.52704

    Iteration: 294
    RMSE: 4.52451

    Iteration: 295
    RMSE: 4.52665

    Iteration: 296
    RMSE: 4.52461

    Iteration: 297
    RMSE: 4.52631

    Iteration: 298
    RMSE: 4.51936

    Iteration: 299
    RMSE: 4.52083

    Iteration: 300
    RMSE: 4.51877

    Iteration: 301
    RMSE: 4.52033

    Iteration: 302
    RMSE: 4.52117

    Iteration: 303
    RMSE: 4.52341

    Iteration: 304
    RMSE: 4.52289

    Iteration: 305
    RMSE: 4.52642

    Iteration: 306
    RMSE: 4.52659

    Iteration: 307
    RMSE: 4.52489

    Iteration: 308
    RMSE: 4.51955

    Iteration: 309
    RMSE: 4.52345

    Iteration: 310
    RMSE: 4.52159

    Iteration: 311
    RMSE: 4.52437

    Iteration: 312
    RMSE: 4.52354

    Iteration: 313
    RMSE: 4.52731

    Iteration: 314
    RMSE: 4.52163

    Iteration: 315
    RMSE: 4.52916

    Iteration: 316
    RMSE: 4.53132

    Iteration: 317
    RMSE: 4.5321

    Iteration: 318
    RMSE: 4.5296

    Iteration: 319
    RMSE: 4.53563

    Iteration: 320
    RMSE: 4.53643

    Iteration: 321
    RMSE: 4.52883

    Iteration: 322
    RMSE: 4.53063

    Iteration: 323
    RMSE: 4.53079

    Iteration: 324
    RMSE: 4.52998

    Iteration: 325
    RMSE: 4.53215

    Iteration: 326
    RMSE: 4.52885

    Iteration: 327
    RMSE: 4.52988

    Iteration: 328
    RMSE: 4.52789

    Iteration: 329
    RMSE: 4.52612

    Iteration: 330
    RMSE: 4.53023

    Iteration: 331
    RMSE: 4.52489

    Iteration: 332
    RMSE: 4.53017

    Iteration: 333
    RMSE: 4.52457

    Iteration: 334
    RMSE: 4.51781

    Iteration: 335
    RMSE: 4.5236

    Iteration: 336
    RMSE: 4.52071

    Iteration: 337
    RMSE: 4.52232

    Iteration: 338
    RMSE: 4.52527

    Iteration: 339
    RMSE: 4.51647

    Iteration: 340
    RMSE: 4.5226

    Iteration: 341
    RMSE: 4.52087

    Iteration: 342
    RMSE: 4.51478

    Iteration: 343
    RMSE: 4.51801

    Iteration: 344
    RMSE: 4.51371

    Iteration: 345
    RMSE: 4.51591

    Iteration: 346
    RMSE: 4.52391

    Iteration: 347
    RMSE: 4.52513

    Iteration: 348
    RMSE: 4.5226

    Iteration: 349
    RMSE: 4.52642

    Iteration: 350
    RMSE: 4.52443

    Iteration: 351
    RMSE: 4.52426

    Iteration: 352
    RMSE: 4.52348

    Iteration: 353
    RMSE: 4.52185

    Iteration: 354
    RMSE: 4.5259

    Iteration: 355
    RMSE: 4.52545

    Iteration: 356
    RMSE: 4.52195

    Iteration: 357
    RMSE: 4.51736

    Iteration: 358
    RMSE: 4.52086

    Iteration: 359
    RMSE: 4.51767

    Iteration: 360
    RMSE: 4.52084

    Iteration: 361
    RMSE: 4.52417

    Iteration: 362
    RMSE: 4.52255

    Iteration: 363
    RMSE: 4.51977

    Iteration: 364
    RMSE: 4.5217

    Iteration: 365
    RMSE: 4.5242

    Iteration: 366
    RMSE: 4.52695

    Iteration: 367
    RMSE: 4.52257

    Iteration: 368
    RMSE: 4.5255

    Iteration: 369
    RMSE: 4.52872

    Iteration: 370
    RMSE: 4.52941

    Iteration: 371
    RMSE: 4.52717

    Iteration: 372
    RMSE: 4.52902

    Iteration: 373
    RMSE: 4.52998

    Iteration: 374
    RMSE: 4.52738

    Iteration: 375
    RMSE: 4.52985

    Iteration: 376
    RMSE: 4.52792

    Iteration: 377
    RMSE: 4.53206

    Iteration: 378
    RMSE: 4.53315

    Iteration: 379
    RMSE: 4.53134

    Iteration: 380
    RMSE: 4.53562

    Iteration: 381
    RMSE: 4.53507

    Iteration: 382
    RMSE: 4.53692

    Iteration: 383
    RMSE: 4.53146

    Iteration: 384
    RMSE: 4.53446

    Iteration: 385
    RMSE: 4.53091

    Iteration: 386
    RMSE: 4.5357

    Iteration: 387
    RMSE: 4.53309

    Iteration: 388
    RMSE: 4.53531

    Iteration: 389
    RMSE: 4.54061

    Iteration: 390
    RMSE: 4.53918

    Iteration: 391
    RMSE: 4.53562

    Iteration: 392
    RMSE: 4.53603

    Iteration: 393
    RMSE: 4.52891

    Iteration: 394
    RMSE: 4.532

    Iteration: 395
    RMSE: 4.53347

    Iteration: 396
    RMSE: 4.53356

    Iteration: 397
    RMSE: 4.53255

    Iteration: 398
    RMSE: 4.53476

    Iteration: 399
    RMSE: 4.53475

    Iteration: 400
    RMSE: 4.52886

    Iteration: 401
    RMSE: 4.52994

    Iteration: 402
    RMSE: 4.52655

    Iteration: 403
    RMSE: 4.53032

    Iteration: 404
    RMSE: 4.52539

    Iteration: 405
    RMSE: 4.52453

    Iteration: 406
    RMSE: 4.52824

    Iteration: 407
    RMSE: 4.53087

    Iteration: 408
    RMSE: 4.52986

    Iteration: 409
    RMSE: 4.52777

    Iteration: 410
    RMSE: 4.53172

    Iteration: 411
    RMSE: 4.53029

    Iteration: 412
    RMSE: 4.53078

    Iteration: 413
    RMSE: 4.52824

    Iteration: 414
    RMSE: 4.52678

    Iteration: 415
    RMSE: 4.52826

    Iteration: 416
    RMSE: 4.52866

    Iteration: 417
    RMSE: 4.52841

    Iteration: 418
    RMSE: 4.52684

    Iteration: 419
    RMSE: 4.52831

    Iteration: 420
    RMSE: 4.5276

    Iteration: 421
    RMSE: 4.52809

    Iteration: 422
    RMSE: 4.52785

    Iteration: 423
    RMSE: 4.52665

    Iteration: 424
    RMSE: 4.52958

    Iteration: 425
    RMSE: 4.52598

    Iteration: 426
    RMSE: 4.52429

    Iteration: 427
    RMSE: 4.52721

    Iteration: 428
    RMSE: 4.52759

    Iteration: 429
    RMSE: 4.52677

    Iteration: 430
    RMSE: 4.52952

    Iteration: 431
    RMSE: 4.52845

    Iteration: 432
    RMSE: 4.53172

    Iteration: 433
    RMSE: 4.52989

    Iteration: 434
    RMSE: 4.5319

    Iteration: 435
    RMSE: 4.53024

    Iteration: 436
    RMSE: 4.53294

    Iteration: 437
    RMSE: 4.52975

    Iteration: 438
    RMSE: 4.53034

    Iteration: 439
    RMSE: 4.53403

    Iteration: 440
    RMSE: 4.52657

    Iteration: 441
    RMSE: 4.52537

    Iteration: 442
    RMSE: 4.52632

    Iteration: 443
    RMSE: 4.525

    Iteration: 444
    RMSE: 4.52483

    Iteration: 445
    RMSE: 4.52809

    Iteration: 446
    RMSE: 4.52304

    Iteration: 447
    RMSE: 4.51666

    Iteration: 448
    RMSE: 4.51913

    Iteration: 449
    RMSE: 4.52086

    Iteration: 450
    RMSE: 4.52373

    Iteration: 451
    RMSE: 4.52259

    Iteration: 452
    RMSE: 4.52086

    Iteration: 453
    RMSE: 4.52423

    Iteration: 454
    RMSE: 4.51958

    Iteration: 455
    RMSE: 4.52224

    Iteration: 456
    RMSE: 4.523

    Iteration: 457
    RMSE: 4.52758

    Iteration: 458
    RMSE: 4.52867

    Iteration: 459
    RMSE: 4.52593

    Iteration: 460
    RMSE: 4.52626

    Iteration: 461
    RMSE: 4.52714

    Iteration: 462
    RMSE: 4.52334

    Iteration: 463
    RMSE: 4.5271

    Iteration: 464
    RMSE: 4.53112

    Iteration: 465
    RMSE: 4.52943

    Iteration: 466
    RMSE: 4.5279

    Iteration: 467
    RMSE: 4.53

    Iteration: 468
    RMSE: 4.52898

    Iteration: 469
    RMSE: 4.53234

    Iteration: 470
    RMSE: 4.53738

    Iteration: 471
    RMSE: 4.53088

    Iteration: 472
    RMSE: 4.52802

    Iteration: 473
    RMSE: 4.52704

    Iteration: 474
    RMSE: 4.52213

    Iteration: 475
    RMSE: 4.5274

    Iteration: 476
    RMSE: 4.52198

    Iteration: 477
    RMSE: 4.52191

    Iteration: 478
    RMSE: 4.52346

    Iteration: 479
    RMSE: 4.52429

    Iteration: 480
    RMSE: 4.52095

    Iteration: 481
    RMSE: 4.52704

    Iteration: 482
    RMSE: 4.53436

    Iteration: 483
    RMSE: 4.52934

    Iteration: 484
    RMSE: 4.52894

    Iteration: 485
    RMSE: 4.52883

    Iteration: 486
    RMSE: 4.53024

    Iteration: 487
    RMSE: 4.53104

    Iteration: 488
    RMSE: 4.52468

    Iteration: 489
    RMSE: 4.52624

    Iteration: 490
    RMSE: 4.52493

    Iteration: 491
    RMSE: 4.52657

    Iteration: 492
    RMSE: 4.52444

    Iteration: 493
    RMSE: 4.52802

    Iteration: 494
    RMSE: 4.52425

    Iteration: 495
    RMSE: 4.52896

    Iteration: 496
    RMSE: 4.52695

    Iteration: 497
    RMSE: 4.53165

    Iteration: 498
    RMSE: 4.52803

    Iteration: 499
    RMSE: 4.52833

    Iteration: 500
    RMSE: 4.52886

    Iteration: 501
    RMSE: 4.5265

    Iteration: 502
    RMSE: 4.53043

    Iteration: 503
    RMSE: 4.53015

    Iteration: 504
    RMSE: 4.52619

    Iteration: 505
    RMSE: 4.52874

    Iteration: 506
    RMSE: 4.53217

    Iteration: 507
    RMSE: 4.52727

    Iteration: 508
    RMSE: 4.52778

    Iteration: 509
    RMSE: 4.52443

    Iteration: 510
    RMSE: 4.5266

    Iteration: 511
    RMSE: 4.53239

    Iteration: 512
    RMSE: 4.53004

    Iteration: 513
    RMSE: 4.53317

    Iteration: 514
    RMSE: 4.53468

    Iteration: 515
    RMSE: 4.53953

    Iteration: 516
    RMSE: 4.53206

    Iteration: 517
    RMSE: 4.53579

    Iteration: 518
    RMSE: 4.5304

    Iteration: 519
    RMSE: 4.52855

    Iteration: 520
    RMSE: 4.53703

    Iteration: 521
    RMSE: 4.53452

    Iteration: 522
    RMSE: 4.53407

    Iteration: 523
    RMSE: 4.53334

    Iteration: 524
    RMSE: 4.53168

    Iteration: 525
    RMSE: 4.53437

    Iteration: 526
    RMSE: 4.53955

    Iteration: 527
    RMSE: 4.53421

    Iteration: 528
    RMSE: 4.53459

    Iteration: 529
    RMSE: 4.53562

    Iteration: 530
    RMSE: 4.54047

    Iteration: 531
    RMSE: 4.54247

    Iteration: 532
    RMSE: 4.53641

    Iteration: 533
    RMSE: 4.53905

    Iteration: 534
    RMSE: 4.54087

    Iteration: 535
    RMSE: 4.53636

    Iteration: 536
    RMSE: 4.53788

    Iteration: 537
    RMSE: 4.53504

    Iteration: 538
    RMSE: 4.53872

    Iteration: 539
    RMSE: 4.53922

    Iteration: 540
    RMSE: 4.53894

    Iteration: 541
    RMSE: 4.53858

    Iteration: 542
    RMSE: 4.53996

    Iteration: 543
    RMSE: 4.54151

    Iteration: 544
    RMSE: 4.54145

    Iteration: 545
    RMSE: 4.54106

    Iteration: 546
    RMSE: 4.53761

    Iteration: 547
    RMSE: 4.53532

    Iteration: 548
    RMSE: 4.53868

    Iteration: 549
    RMSE: 4.5374

    Iteration: 550
    RMSE: 4.53311

    Iteration: 551
    RMSE: 4.53784

    Iteration: 552
    RMSE: 4.53647

    Iteration: 553
    RMSE: 4.53308

    Iteration: 554
    RMSE: 4.53142

    Iteration: 555
    RMSE: 4.52967

    Iteration: 556
    RMSE: 4.52901

    Iteration: 557
    RMSE: 4.52616

    Iteration: 558
    RMSE: 4.5256

    Iteration: 559
    RMSE: 4.52306

    Iteration: 560
    RMSE: 4.52195

    Iteration: 561
    RMSE: 4.52115

    Iteration: 562
    RMSE: 4.53097

    Iteration: 563
    RMSE: 4.52992

    Iteration: 564
    RMSE: 4.52793

    Iteration: 565
    RMSE: 4.53062

    Iteration: 566
    RMSE: 4.52713

    Iteration: 567
    RMSE: 4.53011

    Iteration: 568
    RMSE: 4.52876

    Iteration: 569
    RMSE: 4.53496

    Iteration: 570
    RMSE: 4.53019

    Iteration: 571
    RMSE: 4.52878

    Iteration: 572
    RMSE: 4.5281

    Iteration: 573
    RMSE: 4.53244

    Iteration: 574
    RMSE: 4.52934

    Iteration: 575
    RMSE: 4.52825

    Iteration: 576
    RMSE: 4.53346

    Iteration: 577
    RMSE: 4.53242

    Iteration: 578
    RMSE: 4.53382

    Iteration: 579
    RMSE: 4.53286

    Iteration: 580
    RMSE: 4.53213

    Iteration: 581
    RMSE: 4.53003

    Iteration: 582
    RMSE: 4.52839

    Iteration: 583
    RMSE: 4.52916

    Iteration: 584
    RMSE: 4.53061

    Iteration: 585
    RMSE: 4.52901

    Iteration: 586
    RMSE: 4.53405

    Iteration: 587
    RMSE: 4.53424

    Iteration: 588
    RMSE: 4.53169

    Iteration: 589
    RMSE: 4.52615

    Iteration: 590
    RMSE: 4.52935

    Iteration: 591
    RMSE: 4.52739

    Iteration: 592
    RMSE: 4.53195

    Iteration: 593
    RMSE: 4.53024

    Iteration: 594
    RMSE: 4.53021

    Iteration: 595
    RMSE: 4.53175

    Iteration: 596
    RMSE: 4.52985

    Iteration: 597
    RMSE: 4.53102

    Iteration: 598
    RMSE: 4.52636

    Iteration: 599
    RMSE: 4.53202

    Iteration: 600
    RMSE: 4.5258

    Iteration: 601
    RMSE: 4.52745

    Iteration: 602
    RMSE: 4.51972

    Iteration: 603
    RMSE: 4.52381

    Iteration: 604
    RMSE: 4.52075

    Iteration: 605
    RMSE: 4.52824

    Iteration: 606
    RMSE: 4.52638

    Iteration: 607
    RMSE: 4.51942

    Iteration: 608
    RMSE: 4.52171

    Iteration: 609
    RMSE: 4.52829

    Iteration: 610
    RMSE: 4.52577

    Iteration: 611
    RMSE: 4.52418

    Iteration: 612
    RMSE: 4.52322

    Iteration: 613
    RMSE: 4.52642

    Iteration: 614
    RMSE: 4.52789

    Iteration: 615
    RMSE: 4.52859

    Iteration: 616
    RMSE: 4.52633

    Iteration: 617
    RMSE: 4.52544

    Iteration: 618
    RMSE: 4.52514

    Iteration: 619
    RMSE: 4.52644

    Iteration: 620
    RMSE: 4.53211

    Iteration: 621
    RMSE: 4.52624

    Iteration: 622
    RMSE: 4.53716

    Iteration: 623
    RMSE: 4.52543

    Iteration: 624
    RMSE: 4.52536

    Iteration: 625
    RMSE: 4.5277

    Iteration: 626
    RMSE: 4.52221

    Iteration: 627
    RMSE: 4.52674

    Iteration: 628
    RMSE: 4.52772

    Iteration: 629
    RMSE: 4.52888

    Iteration: 630
    RMSE: 4.52968

    Iteration: 631
    RMSE: 4.52842

    Iteration: 632
    RMSE: 4.53178

    Iteration: 633
    RMSE: 4.53182

    Iteration: 634
    RMSE: 4.53385

    Iteration: 635
    RMSE: 4.52912

    Iteration: 636
    RMSE: 4.53271

    Iteration: 637
    RMSE: 4.5326

    Iteration: 638
    RMSE: 4.5278

    Iteration: 639
    RMSE: 4.53181

    Iteration: 640
    RMSE: 4.52993

    Iteration: 641
    RMSE: 4.53449

    Iteration: 642
    RMSE: 4.52944

    Iteration: 643
    RMSE: 4.53202

    Iteration: 644
    RMSE: 4.52921

    Iteration: 645
    RMSE: 4.53377

    Iteration: 646
    RMSE: 4.52777

    Iteration: 647
    RMSE: 4.53272

    Iteration: 648
    RMSE: 4.52852

    Iteration: 649
    RMSE: 4.52956

    Iteration: 650
    RMSE: 4.52949

    Iteration: 651
    RMSE: 4.53218

    Iteration: 652
    RMSE: 4.53422

    Iteration: 653
    RMSE: 4.52896

    Iteration: 654
    RMSE: 4.52982

    Iteration: 655
    RMSE: 4.53125

    Iteration: 656
    RMSE: 4.52995

    Iteration: 657
    RMSE: 4.53204

    Iteration: 658
    RMSE: 4.52827

    Iteration: 659
    RMSE: 4.52993

    Iteration: 660
    RMSE: 4.52843

    Iteration: 661
    RMSE: 4.52466

    Iteration: 662
    RMSE: 4.53145

    Iteration: 663
    RMSE: 4.52921

    Iteration: 664
    RMSE: 4.53189

    Iteration: 665
    RMSE: 4.538

    Iteration: 666
    RMSE: 4.53556

    Iteration: 667
    RMSE: 4.53759

    Iteration: 668
    RMSE: 4.53156

    Iteration: 669
    RMSE: 4.52688

    Iteration: 670
    RMSE: 4.52716

    Iteration: 671
    RMSE: 4.52928

    Iteration: 672
    RMSE: 4.52819

    Iteration: 673
    RMSE: 4.53209

    Iteration: 674
    RMSE: 4.529

    Iteration: 675
    RMSE: 4.53476

    Iteration: 676
    RMSE: 4.53239

    Iteration: 677
    RMSE: 4.52864

    Iteration: 678
    RMSE: 4.53138

    Iteration: 679
    RMSE: 4.53058

    Iteration: 680
    RMSE: 4.53075

    Iteration: 681
    RMSE: 4.53052

    Iteration: 682
    RMSE: 4.52817

    Iteration: 683
    RMSE: 4.5321

    Iteration: 684
    RMSE: 4.52772

    Iteration: 685
    RMSE: 4.52689

    Iteration: 686
    RMSE: 4.53069

    Iteration: 687
    RMSE: 4.5321

    Iteration: 688
    RMSE: 4.53238

    Iteration: 689
    RMSE: 4.52893

    Iteration: 690
    RMSE: 4.53086

    Iteration: 691
    RMSE: 4.5339

    Iteration: 692
    RMSE: 4.52961

    Iteration: 693
    RMSE: 4.53223

    Iteration: 694
    RMSE: 4.53309

    Iteration: 695
    RMSE: 4.53002

    Iteration: 696
    RMSE: 4.53247

    Iteration: 697
    RMSE: 4.53161

    Iteration: 698
    RMSE: 4.52681

    Iteration: 699
    RMSE: 4.53073

    Iteration: 700
    RMSE: 4.53233

    Iteration: 701
    RMSE: 4.532

    Iteration: 702
    RMSE: 4.53403

    Iteration: 703
    RMSE: 4.53078

    Iteration: 704
    RMSE: 4.53706

    Iteration: 705
    RMSE: 4.53681

    Iteration: 706
    RMSE: 4.5351

    Iteration: 707
    RMSE: 4.53331

    Iteration: 708
    RMSE: 4.53511

    Iteration: 709
    RMSE: 4.54066

    Iteration: 710
    RMSE: 4.54084

    Iteration: 711
    RMSE: 4.53453

    Iteration: 712
    RMSE: 4.53941

    Iteration: 713
    RMSE: 4.54032

    Iteration: 714
    RMSE: 4.53667

    Iteration: 715
    RMSE: 4.53503

    Iteration: 716
    RMSE: 4.53776

    Iteration: 717
    RMSE: 4.53762

    Iteration: 718
    RMSE: 4.54009

    Iteration: 719
    RMSE: 4.53594

    Iteration: 720
    RMSE: 4.53677

    Iteration: 721
    RMSE: 4.53586

    Iteration: 722
    RMSE: 4.52967

    Iteration: 723
    RMSE: 4.53158

    Iteration: 724
    RMSE: 4.52934

    Iteration: 725
    RMSE: 4.53479

    Iteration: 726
    RMSE: 4.53863

    Iteration: 727
    RMSE: 4.53261

    Iteration: 728
    RMSE: 4.53745

    Iteration: 729
    RMSE: 4.53554

    Iteration: 730
    RMSE: 4.53565

    Iteration: 731
    RMSE: 4.53635

    Iteration: 732
    RMSE: 4.53439

    Iteration: 733
    RMSE: 4.53859

    Iteration: 734
    RMSE: 4.53381

    Iteration: 735
    RMSE: 4.53835

    Iteration: 736
    RMSE: 4.53386

    Iteration: 737
    RMSE: 4.52928

    Iteration: 738
    RMSE: 4.53142

    Iteration: 739
    RMSE: 4.53259

    Iteration: 740
    RMSE: 4.53116

    Iteration: 741
    RMSE: 4.53143

    Iteration: 742
    RMSE: 4.53251

    Iteration: 743
    RMSE: 4.52778

    Iteration: 744
    RMSE: 4.5266

    Iteration: 745
    RMSE: 4.53205

    Iteration: 746
    RMSE: 4.52854

    Iteration: 747
    RMSE: 4.52901

    Iteration: 748
    RMSE: 4.52818

    Iteration: 749
    RMSE: 4.52808

    Iteration: 750
    RMSE: 4.52765

    Iteration: 751
    RMSE: 4.52938

    Iteration: 752
    RMSE: 4.52935

    Iteration: 753
    RMSE: 4.52982

    Iteration: 754
    RMSE: 4.52737

    Iteration: 755
    RMSE: 4.52888

    Iteration: 756
    RMSE: 4.5248

    Iteration: 757
    RMSE: 4.5276

    Iteration: 758
    RMSE: 4.53045

    Iteration: 759
    RMSE: 4.52649

    Iteration: 760
    RMSE: 4.52516

    Iteration: 761
    RMSE: 4.5279

    Iteration: 762
    RMSE: 4.52506

    Iteration: 763
    RMSE: 4.52742

    Iteration: 764
    RMSE: 4.5259

    Iteration: 765
    RMSE: 4.52556

    Iteration: 766
    RMSE: 4.52728

    Iteration: 767
    RMSE: 4.52895

    Iteration: 768
    RMSE: 4.52796

    Iteration: 769
    RMSE: 4.52635

    Iteration: 770
    RMSE: 4.53237

    Iteration: 771
    RMSE: 4.53229

    Iteration: 772
    RMSE: 4.53328

    Iteration: 773
    RMSE: 4.53033

    Iteration: 774
    RMSE: 4.53227

    Iteration: 775
    RMSE: 4.52984

    Iteration: 776
    RMSE: 4.53046

    Iteration: 777
    RMSE: 4.53181

    Iteration: 778
    RMSE: 4.52834

    Iteration: 779
    RMSE: 4.5283

    Iteration: 780
    RMSE: 4.52956

    Iteration: 781
    RMSE: 4.53006

    Iteration: 782
    RMSE: 4.52947

    Iteration: 783
    RMSE: 4.53159

    Iteration: 784
    RMSE: 4.53054

    Iteration: 785
    RMSE: 4.5322

    Iteration: 786
    RMSE: 4.53251

    Iteration: 787
    RMSE: 4.53305

    Iteration: 788
    RMSE: 4.53242

    Iteration: 789
    RMSE: 4.53519

    Iteration: 790
    RMSE: 4.53574

    Iteration: 791
    RMSE: 4.53697

    Iteration: 792
    RMSE: 4.53377

    Iteration: 793
    RMSE: 4.53626

    Iteration: 794
    RMSE: 4.5319

    Iteration: 795
    RMSE: 4.52958

    Iteration: 796
    RMSE: 4.53134

    Iteration: 797
    RMSE: 4.52882

    Iteration: 798
    RMSE: 4.52978

    Iteration: 799
    RMSE: 4.53047

    Iteration: 800
    RMSE: 4.53358

    Iteration: 801
    RMSE: 4.52785

    Iteration: 802
    RMSE: 4.52874

    Iteration: 803
    RMSE: 4.52888

    Iteration: 804
    RMSE: 4.53159

    Iteration: 805
    RMSE: 4.5317

    Iteration: 806
    RMSE: 4.52803

    Iteration: 807
    RMSE: 4.53252

    Iteration: 808
    RMSE: 4.53031

    Iteration: 809
    RMSE: 4.5247

    Iteration: 810
    RMSE: 4.52764

    Iteration: 811
    RMSE: 4.52238

    Iteration: 812
    RMSE: 4.5245

    Iteration: 813
    RMSE: 4.52781

    Iteration: 814
    RMSE: 4.52699

    Iteration: 815
    RMSE: 4.5261

    Iteration: 816
    RMSE: 4.52934

    Iteration: 817
    RMSE: 4.52881

    Iteration: 818
    RMSE: 4.53071

    Iteration: 819
    RMSE: 4.52363

    Iteration: 820
    RMSE: 4.52889

    Iteration: 821
    RMSE: 4.52984

    Iteration: 822
    RMSE: 4.53265

    Iteration: 823
    RMSE: 4.53176

    Iteration: 824
    RMSE: 4.53325

    Iteration: 825
    RMSE: 4.52734

    Iteration: 826
    RMSE: 4.532

    Iteration: 827
    RMSE: 4.53267

    Iteration: 828
    RMSE: 4.52872

    Iteration: 829
    RMSE: 4.52942

    Iteration: 830
    RMSE: 4.53005

    Iteration: 831
    RMSE: 4.53429

    Iteration: 832
    RMSE: 4.53

    Iteration: 833
    RMSE: 4.52923

    Iteration: 834
    RMSE: 4.52564

    Iteration: 835
    RMSE: 4.52425

    Iteration: 836
    RMSE: 4.52404

    Iteration: 837
    RMSE: 4.52049

    Iteration: 838
    RMSE: 4.52268

    Iteration: 839
    RMSE: 4.52179

    Iteration: 840
    RMSE: 4.52469

    Iteration: 841
    RMSE: 4.52052

    Iteration: 842
    RMSE: 4.52798

    Iteration: 843
    RMSE: 4.52719

    Iteration: 844
    RMSE: 4.52822

    Iteration: 845
    RMSE: 4.52859

    Iteration: 846
    RMSE: 4.52124

    Iteration: 847
    RMSE: 4.52414

    Iteration: 848
    RMSE: 4.52514

    Iteration: 849
    RMSE: 4.52461

    Iteration: 850
    RMSE: 4.5199

    Iteration: 851
    RMSE: 4.51987

    Iteration: 852
    RMSE: 4.5177

    Iteration: 853
    RMSE: 4.52069

    Iteration: 854
    RMSE: 4.52308

    Iteration: 855
    RMSE: 4.52469

    Iteration: 856
    RMSE: 4.52429

    Iteration: 857
    RMSE: 4.52416

    Iteration: 858
    RMSE: 4.52467

    Iteration: 859
    RMSE: 4.52621

    Iteration: 860
    RMSE: 4.52895

    Iteration: 861
    RMSE: 4.52379

    Iteration: 862
    RMSE: 4.52622

    Iteration: 863
    RMSE: 4.52564

    Iteration: 864
    RMSE: 4.5248

    Iteration: 865
    RMSE: 4.52622

    Iteration: 866
    RMSE: 4.52746

    Iteration: 867
    RMSE: 4.52238

    Iteration: 868
    RMSE: 4.51981

    Iteration: 869
    RMSE: 4.52142

    Iteration: 870
    RMSE: 4.52218

    Iteration: 871
    RMSE: 4.52303

    Iteration: 872
    RMSE: 4.51897

    Iteration: 873
    RMSE: 4.52221

    Iteration: 874
    RMSE: 4.52117

    Iteration: 875
    RMSE: 4.52593

    Iteration: 876
    RMSE: 4.5243

    Iteration: 877
    RMSE: 4.52011

    Iteration: 878
    RMSE: 4.52336

    Iteration: 879
    RMSE: 4.52029

    Iteration: 880
    RMSE: 4.52199

    Iteration: 881
    RMSE: 4.52244

    Iteration: 882
    RMSE: 4.5225

    Iteration: 883
    RMSE: 4.52427

    Iteration: 884
    RMSE: 4.52516

    Iteration: 885
    RMSE: 4.5242

    Iteration: 886
    RMSE: 4.52271

    Iteration: 887
    RMSE: 4.52822

    Iteration: 888
    RMSE: 4.5245

    Iteration: 889
    RMSE: 4.52579

    Iteration: 890
    RMSE: 4.52192

    Iteration: 891
    RMSE: 4.52051

    Iteration: 892
    RMSE: 4.52079

    Iteration: 893
    RMSE: 4.5188

    Iteration: 894
    RMSE: 4.51464

    Iteration: 895
    RMSE: 4.51549

    Iteration: 896
    RMSE: 4.51603

    Iteration: 897
    RMSE: 4.51803

    Iteration: 898
    RMSE: 4.51942

    Iteration: 899
    RMSE: 4.52185

    Iteration: 900
    RMSE: 4.51905

    Iteration: 901
    RMSE: 4.52046

    Iteration: 902
    RMSE: 4.51986

    Iteration: 903
    RMSE: 4.52328

    Iteration: 904
    RMSE: 4.52418

    Iteration: 905
    RMSE: 4.52608

    Iteration: 906
    RMSE: 4.52332

    Iteration: 907
    RMSE: 4.51948

    Iteration: 908
    RMSE: 4.52113

    Iteration: 909
    RMSE: 4.5165

    Iteration: 910
    RMSE: 4.5155

    Iteration: 911
    RMSE: 4.5213

    Iteration: 912
    RMSE: 4.51997

    Iteration: 913
    RMSE: 4.52115

    Iteration: 914
    RMSE: 4.51878

    Iteration: 915
    RMSE: 4.51856

    Iteration: 916
    RMSE: 4.52534

    Iteration: 917
    RMSE: 4.52514

    Iteration: 918
    RMSE: 4.52971

    Iteration: 919
    RMSE: 4.52668

    Iteration: 920
    RMSE: 4.52256

    Iteration: 921
    RMSE: 4.53017

    Iteration: 922
    RMSE: 4.52728

    Iteration: 923
    RMSE: 4.52537

    Iteration: 924
    RMSE: 4.52452

    Iteration: 925
    RMSE: 4.52477

    Iteration: 926
    RMSE: 4.52484

    Iteration: 927
    RMSE: 4.52344

    Iteration: 928
    RMSE: 4.52644

    Iteration: 929
    RMSE: 4.52175

    Iteration: 930
    RMSE: 4.52767

    Iteration: 931
    RMSE: 4.53032

    Iteration: 932
    RMSE: 4.52528

    Iteration: 933
    RMSE: 4.52911

    Iteration: 934
    RMSE: 4.52504

    Iteration: 935
    RMSE: 4.52358

    Iteration: 936
    RMSE: 4.52341

    Iteration: 937
    RMSE: 4.52468

    Iteration: 938
    RMSE: 4.52237

    Iteration: 939
    RMSE: 4.52759

    Iteration: 940
    RMSE: 4.5322

    Iteration: 941
    RMSE: 4.53629

    Iteration: 942
    RMSE: 4.53121

    Iteration: 943
    RMSE: 4.53599

    Iteration: 944
    RMSE: 4.53496

    Iteration: 945
    RMSE: 4.53192

    Iteration: 946
    RMSE: 4.53273

    Iteration: 947
    RMSE: 4.52694

    Iteration: 948
    RMSE: 4.53442

    Iteration: 949
    RMSE: 4.52953

    Iteration: 950
    RMSE: 4.52779

    Iteration: 951
    RMSE: 4.5288

    Iteration: 952
    RMSE: 4.52918

    Iteration: 953
    RMSE: 4.5308

    Iteration: 954
    RMSE: 4.5365

    Iteration: 955
    RMSE: 4.5374

    Iteration: 956
    RMSE: 4.53578

    Iteration: 957
    RMSE: 4.53429

    Iteration: 958
    RMSE: 4.53475

    Iteration: 959
    RMSE: 4.53505

    Iteration: 960
    RMSE: 4.53629

    Iteration: 961
    RMSE: 4.5373

    Iteration: 962
    RMSE: 4.53279

    Iteration: 963
    RMSE: 4.5353

    Iteration: 964
    RMSE: 4.53679

    Iteration: 965
    RMSE: 4.53489

    Iteration: 966
    RMSE: 4.53818

    Iteration: 967
    RMSE: 4.53242

    Iteration: 968
    RMSE: 4.53528

    Iteration: 969
    RMSE: 4.53466

    Iteration: 970
    RMSE: 4.52915

    Iteration: 971
    RMSE: 4.53365

    Iteration: 972
    RMSE: 4.53147

    Iteration: 973
    RMSE: 4.53764

    Iteration: 974
    RMSE: 4.53687

    Iteration: 975
    RMSE: 4.52839

    Iteration: 976
    RMSE: 4.53139

    Iteration: 977
    RMSE: 4.53105

    Iteration: 978
    RMSE: 4.53345

    Iteration: 979
    RMSE: 4.53612

    Iteration: 980
    RMSE: 4.53581

    Iteration: 981
    RMSE: 4.53507

    Iteration: 982
    RMSE: 4.53987

    Iteration: 983
    RMSE: 4.5382

    Iteration: 984
    RMSE: 4.53801

    Iteration: 985
    RMSE: 4.53654

    Iteration: 986
    RMSE: 4.53308

    Iteration: 987
    RMSE: 4.53264

    Iteration: 988
    RMSE: 4.53146

    Iteration: 989
    RMSE: 4.52973

    Iteration: 990
    RMSE: 4.53141

    Iteration: 991
    RMSE: 4.53123

    Iteration: 992
    RMSE: 4.53162

    Iteration: 993
    RMSE: 4.53167

    Iteration: 994
    RMSE: 4.53414

    Iteration: 995
    RMSE: 4.53266

    Iteration: 996
    RMSE: 4.53523

    Iteration: 997
    RMSE: 4.5357

    Iteration: 998
    RMSE: 4.53572

    Iteration: 999
    RMSE: 4.53471

    Iteration: 1000
    RMSE: 4.52807

    Imputation MAPE: 0.1038
    Imputation RMSE: 4.48337

    Running time: 3183 seconds

```python
import pandas as pd

dense_mat = pd.read_csv('../datasets/Seattle-data-set/mat.csv', index_col = 0)
RM_mat = pd.read_csv('../datasets/Seattle-data-set/RM_mat.csv', index_col = 0)
dense_mat = dense_mat.values
RM_mat = RM_mat.values

missing_rate = 0.4

# =============================================================================
### Random missing (RM) scenario
### Set the RM scenario by:
binary_mat = np.round(RM_mat + 0.5 - missing_rate)
# =============================================================================

sparse_mat = np.multiply(dense_mat, binary_mat)
```

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 50
time_lags = np.array([1, 2, 288])
init = {"W": 0.1 * np.random.rand(dim1, rank), "X": 0.1 * np.random.rand(dim2, rank)}
maxiter1 = 1100
maxiter2 = 100
BTMF(dense_mat, sparse_mat, init, rank, time_lags, maxiter1, maxiter2)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

## 7 Multivariate Time Series Prediction

```python
def BTMF_burn(dense_mat, sparse_mat, init, time_lags, burn_iter):

    W = init["W"]
    X = init["X"]
    dim1, dim2 = sparse_mat.shape
    d = time_lags.shape[0]
    pos = np.where((dense_mat != 0) & (sparse_mat == 0))
    position = np.where(sparse_mat != 0)
    binary_mat = np.zeros((dim1, dim2))
    binary_mat[position] = 1
    tau = 1
    for it in range(burn_iter):
        W = sample_factor_w(sparse_mat, binary_mat, W, X, tau)
        A, Sigma = sample_var_coefficient(X, time_lags)
        X = sample_factor_x(sparse_mat, binary_mat, time_lags, W, X, tau, A, inv(Sigma))
        mat_hat = np.matmul(W, X.T)
        tau = sample_precision_tau(sparse_mat, mat_hat, position)
        rmse = np.sqrt(np.sum((dense_mat[pos] - mat_hat[pos]) ** 2) / dense_mat[pos].shape[0])
        if (it + 1) % 1 == 0 and it < burn_iter:
            print('Iteration: {}'.format(it + 1))
            print('RMSE: {:.6}'.format(rmse))
            print()
    return W, X, tau, A
```

```python
def BTMF_4cast(mat, binary_mat, num_step, time_lags, init, gibbs_iter):
    """Forecast (`4cast`) time series with Bayesian Temporal Matrix Factorization (BTMF)."""

    W = init["W"]
    X = init["X"]
    tau = init["tau"]
    A = init["A"]
    rank = W.shape[1]
    d = time_lags.shape[0]
    mat_hat = np.zeros((W.shape[0], num_step, gibbs_iter))
    for it in range(gibbs_iter):
        W = sample_factor_w(mat, binary_mat, W, X, tau)
        A, Sigma = sample_var_coefficient(X, time_lags)
        X = sample_factor_x(sparse_mat, binary_mat, time_lags, W, X, tau, A, inv(Sigma))
        X_new = X.copy()
        for t in range(num_step):
            var = X_new[X.shape[0] + t - 1 - time_lags, :].reshape([rank * d])
            X_new = np.append(X_new, np.matmul(A.T, var).reshape([1, rank]), axis = 0)
#         mat_hat[:, :, it] = np.random.normal(np.matmul(W, X_new[-1 - num_step : -1, :].T), 1 / tau) # dim1 * num_step
        mat_hat[:, :, it] = np.matmul(W, X_new[-1 - num_step : -1, :].T) # dim1 * num_step
    return mat_hat, W, X_new, tau, A
```

```python
def forecastor(dense_mat, sparse_mat, init, time_lags,
               num_roll, start_time, num_step, burn_iter, gibbs_iter):

    W, X, tau, A = BTMF_burn(dense_mat[:, : start_time], sparse_mat[:, : start_time],
                             init, time_lags, burn_iter)
    result = np.zeros((W.shape[0], num_roll * num_step, gibbs_iter))
    for t in range(num_roll):
        mat = sparse_mat[:, : start_time + t * num_step]
        print(mat.shape[1])
        position = np.where(mat != 0)
        binary_mat = mat.copy()
        binary_mat[position] = 1
        init = {"W": W, "X": X, "tau": tau, "A": A}
        mat_hat, W, X, tau, A = BTMF_4cast(mat, binary_mat,
                                           num_step, time_lags, init, gibbs_iter)
        result[:, t * num_step : (t + 1) * num_step, :] = mat_hat

    mat_hat0 = np.mean(result, axis = 2)
    small_dense_mat = dense_mat[:, start_time : dense_mat.shape[1]]
    pos = np.where(small_dense_mat != 0)
    final_mape = np.sum(np.abs(small_dense_mat[pos] -
                               mat_hat0[pos]) / small_dense_mat[pos]) / small_dense_mat[pos].shape[0]
    final_rmse = np.sqrt(np.sum((small_dense_mat[pos] -
                                 mat_hat0[pos]) ** 2) / small_dense_mat[pos].shape[0])
    print('Final MAPE: {:.6}'.format(final_mape))
    print('Final RMSE: {:.6}'.format(final_rmse))
    print()
    return result
```

```python
import scipy.io

tensor = scipy.io.loadmat('../datasets/Guangzhou-data-set/tensor.mat')
tensor = tensor['tensor']
random_matrix = scipy.io.loadmat('../datasets/Guangzhou-data-set/random_matrix.mat')
random_matrix = random_matrix['random_matrix']
random_tensor = scipy.io.loadmat('../datasets/Guangzhou-data-set/random_tensor.mat')
random_tensor = random_tensor['random_tensor']

dense_mat = tensor.reshape([tensor.shape[0], tensor.shape[1] * tensor.shape[2]])
missing_rate = 0.4

# =============================================================================
### Random missing (RM) scenario
### Set the RM scenario by:
binary_mat = (np.round(random_tensor + 0.5 - missing_rate)
              .reshape([random_tensor.shape[0], random_tensor.shape[1] * random_tensor.shape[2]]))
# =============================================================================

# binary_tensor = np.zeros(tensor.shape)
# for i1 in range(tensor.shape[0]):
#     for i2 in range(tensor.shape[1]):
#         binary_tensor[i1, i2, :] = np.round(random_matrix[i1, i2] + 0.5 - missing_rate)
# binary_mat = binary_tensor.reshape([binary_tensor.shape[0], binary_tensor.shape[1] * binary_tensor.shape[2]])

sparse_mat = np.multiply(dense_mat, binary_mat)
```

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 10
time_lags = np.array([1, 2, 144])
num_step = 6
num_roll = int(144 * 5 / num_step)
start_time = dim2 - num_roll * num_step
init = {"W": 0.1 * np.random.rand(dim1, rank),
        "X": 0.1 * np.random.rand(start_time, rank)}
burn_iter = 10
gibbs_iter = 2
result = forecastor(dense_mat, sparse_mat, init, time_lags,
                    num_roll, start_time, num_step, burn_iter, gibbs_iter)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

    Iteration: 1
    RMSE: 5.94597

    Iteration: 2
    RMSE: 5.74879

    Iteration: 3
    RMSE: 5.70634

    Iteration: 4
    RMSE: 5.47132

    Iteration: 5
    RMSE: 4.99418

    Iteration: 6
    RMSE: 4.76256

    Iteration: 7
    RMSE: 4.68645

    Iteration: 8
    RMSE: 4.5744

    Iteration: 9
    RMSE: 4.55261

    Iteration: 10
    RMSE: 4.54463

    8064
    8070
    8076
    8082
    8088
    8094
    8100
    8106
    8112
    8118
    8124
    8130
    8136
    8142
    8148
    8154
    8160
    8166
    8172
    8178
    8184
    8190
    8196
    8202
    8208
    8214
    8220
    8226
    8232
    8238
    8244
    8250
    8256
    8262
    8268
    8274
    8280
    8286
    8292
    8298
    8304
    8310
    8316
    8322
    8328
    8334
    8340
    8346
    8352
    8358
    8364
    8370
    8376
    8382
    8388
    8394
    8400
    8406
    8412
    8418
    8424
    8430
    8436
    8442
    8448
    8454
    8460
    8466
    8472
    8478
    8484
    8490
    8496
    8502
    8508
    8514
    8520
    8526
    8532
    8538
    8544
    8550
    8556
    8562
    8568
    8574
    8580
    8586
    8592
    8598
    8604
    8610
    8616
    8622
    8628
    8634
    8640
    8646
    8652
    8658
    8664
    8670
    8676
    8682
    8688
    8694
    8700
    8706
    8712
    8718
    8724
    8730
    8736
    8742
    8748
    8754
    8760
    8766
    8772
    8778
    Final MAPE: 0.372138
    Final RMSE: 15.0209

    Running time: 762 seconds

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 10
time_lags = np.array([1, 2, 144])
num_step = 6
num_roll = int(144 * 5 / num_step)
start_time = dim2 - num_roll * num_step
init = {"W": 0.1 * np.random.rand(dim1, rank),
        "X": 0.1 * np.random.rand(start_time, rank)}
burn_iter = 100
gibbs_iter = 20
result = forecastor(dense_mat, sparse_mat, init, time_lags,
                    num_roll, start_time, num_step, burn_iter, gibbs_iter)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

    Iteration: 1
    RMSE: 5.94449

    Iteration: 2
    RMSE: 5.75295

    Iteration: 3
    RMSE: 5.71641

    Iteration: 4
    RMSE: 5.53581

    Iteration: 5
    RMSE: 5.0634

    Iteration: 6
    RMSE: 4.88969

    Iteration: 7
    RMSE: 4.74457

    Iteration: 8
    RMSE: 4.68477

    Iteration: 9
    RMSE: 4.56635

    Iteration: 10
    RMSE: 4.54667

    Iteration: 11
    RMSE: 4.53713

    Iteration: 12
    RMSE: 4.50478

    Iteration: 13
    RMSE: 4.44558

    Iteration: 14
    RMSE: 4.41896

    Iteration: 15
    RMSE: 4.40447

    Iteration: 16
    RMSE: 4.39169

    Iteration: 17
    RMSE: 4.3801

    Iteration: 18
    RMSE: 4.36146

    Iteration: 19
    RMSE: 4.34012

    Iteration: 20
    RMSE: 4.323

    Iteration: 21
    RMSE: 4.2991

    Iteration: 22
    RMSE: 4.27874

    Iteration: 23
    RMSE: 4.26798

    Iteration: 24
    RMSE: 4.26194

    Iteration: 25
    RMSE: 4.25591

    Iteration: 26
    RMSE: 4.25072

    Iteration: 27
    RMSE: 4.24422

    Iteration: 28
    RMSE: 4.24307

    Iteration: 29
    RMSE: 4.23493

    Iteration: 30
    RMSE: 4.22458

    Iteration: 31
    RMSE: 4.21628

    Iteration: 32
    RMSE: 4.21207

    Iteration: 33
    RMSE: 4.21048

    Iteration: 34
    RMSE: 4.20769

    Iteration: 35
    RMSE: 4.20458

    Iteration: 36
    RMSE: 4.20501

    Iteration: 37
    RMSE: 4.20187

    Iteration: 38
    RMSE: 4.20378

    Iteration: 39
    RMSE: 4.20147

    Iteration: 40
    RMSE: 4.2014

    Iteration: 41
    RMSE: 4.20172

    Iteration: 42
    RMSE: 4.20174

    Iteration: 43
    RMSE: 4.20179

    Iteration: 44
    RMSE: 4.19926

    Iteration: 45
    RMSE: 4.20019

    Iteration: 46
    RMSE: 4.1975

    Iteration: 47
    RMSE: 4.19601

    Iteration: 48
    RMSE: 4.19907

    Iteration: 49
    RMSE: 4.19479

    Iteration: 50
    RMSE: 4.19641

    Iteration: 51
    RMSE: 4.19439

    Iteration: 52
    RMSE: 4.19538

    Iteration: 53
    RMSE: 4.19394

    Iteration: 54
    RMSE: 4.19151

    Iteration: 55
    RMSE: 4.19388

    Iteration: 56
    RMSE: 4.18899

    Iteration: 57
    RMSE: 4.1936

    Iteration: 58
    RMSE: 4.19189

    Iteration: 59
    RMSE: 4.19198

    Iteration: 60
    RMSE: 4.1927

    Iteration: 61
    RMSE: 4.19205

    Iteration: 62
    RMSE: 4.19358

    Iteration: 63
    RMSE: 4.19379

    Iteration: 64
    RMSE: 4.19318

    Iteration: 65
    RMSE: 4.19543

    Iteration: 66
    RMSE: 4.19417

    Iteration: 67
    RMSE: 4.19487

    Iteration: 68
    RMSE: 4.19499

    Iteration: 69
    RMSE: 4.19354

    Iteration: 70
    RMSE: 4.19311

    Iteration: 71
    RMSE: 4.19313

    Iteration: 72
    RMSE: 4.18999

    Iteration: 73
    RMSE: 4.18869

    Iteration: 74
    RMSE: 4.19073

    Iteration: 75
    RMSE: 4.18876

    Iteration: 76
    RMSE: 4.19048

    Iteration: 77
    RMSE: 4.19235

    Iteration: 78
    RMSE: 4.19325

    Iteration: 79
    RMSE: 4.18861

    Iteration: 80
    RMSE: 4.19046

    Iteration: 81
    RMSE: 4.19224

    Iteration: 82
    RMSE: 4.18991

    Iteration: 83
    RMSE: 4.19028

    Iteration: 84
    RMSE: 4.19048

    Iteration: 85
    RMSE: 4.18996

    Iteration: 86
    RMSE: 4.19008

    Iteration: 87
    RMSE: 4.18816

    Iteration: 88
    RMSE: 4.18801

    Iteration: 89
    RMSE: 4.1857

    Iteration: 90
    RMSE: 4.18845

    Iteration: 91
    RMSE: 4.18824

    Iteration: 92
    RMSE: 4.18866

    Iteration: 93
    RMSE: 4.1874

    Iteration: 94
    RMSE: 4.19071

    Iteration: 95
    RMSE: 4.18984

    Iteration: 96
    RMSE: 4.18867

    Iteration: 97
    RMSE: 4.18954

    Iteration: 98
    RMSE: 4.18876

    Iteration: 99
    RMSE: 4.18756

    Iteration: 100
    RMSE: 4.18824

    8064
    8070
    8076
    8082
    8088
    8094
    8100
    8106
    8112
    8118
    8124
    8130
    8136
    8142
    8148
    8154
    8160
    8166
    8172
    8178
    8184
    8190
    8196
    8202
    8208
    8214
    8220
    8226
    8232
    8238
    8244
    8250
    8256
    8262
    8268
    8274
    8280
    8286
    8292
    8298
    8304
    8310
    8316
    8322
    8328
    8334
    8340
    8346
    8352
    8358
    8364
    8370
    8376
    8382
    8388
    8394
    8400
    8406
    8412
    8418
    8424
    8430
    8436
    8442
    8448
    8454
    8460
    8466
    8472
    8478
    8484
    8490
    8496
    8502
    8508
    8514
    8520
    8526
    8532
    8538
    8544
    8550
    8556
    8562
    8568
    8574
    8580
    8586
    8592
    8598
    8604
    8610
    8616
    8622
    8628
    8634
    8640
    8646
    8652
    8658
    8664
    8670
    8676
    8682
    8688
    8694
    8700
    8706
    8712
    8718
    8724
    8730
    8736
    8742
    8748
    8754
    8760
    8766
    8772
    8778
    Final MAPE: 0.178763
    Final RMSE: 6.72478

    Running time: 6631 seconds

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 10
time_lags = np.array([1, 2, 144])
num_step = 6
num_roll = int(144 * 5 / num_step)
start_time = dim2 - num_roll * num_step
init = {"W": 0.1 * np.random.rand(dim1, rank),
        "X": 0.1 * np.random.rand(start_time, rank)}
burn_iter = 1000
gibbs_iter = 100
result = forecastor(dense_mat, sparse_mat, init, time_lags,
                    num_roll, start_time, num_step, burn_iter, gibbs_iter)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

    Iteration: 1
    RMSE: 5.93895

    Iteration: 2
    RMSE: 5.75626

    Iteration: 3
    RMSE: 5.70908

    Iteration: 4
    RMSE: 5.47771

    Iteration: 5
    RMSE: 5.05707

    Iteration: 6
    RMSE: 4.87126

    Iteration: 7
    RMSE: 4.72067

    Iteration: 8
    RMSE: 4.6074

    Iteration: 9
    RMSE: 4.55761

    Iteration: 10
    RMSE: 4.54906

    Iteration: 11
    RMSE: 4.54095

    Iteration: 12
    RMSE: 4.53278

    Iteration: 13
    RMSE: 4.51274

    Iteration: 14
    RMSE: 4.48254

    Iteration: 15
    RMSE: 4.43496

    Iteration: 16
    RMSE: 4.40604

    Iteration: 17
    RMSE: 4.39511

    Iteration: 18
    RMSE: 4.37981

    Iteration: 19
    RMSE: 4.35406

    Iteration: 20
    RMSE: 4.32049

    Iteration: 21
    RMSE: 4.29999

    Iteration: 22
    RMSE: 4.29119

    Iteration: 23
    RMSE: 4.28679

    Iteration: 24
    RMSE: 4.27596

    Iteration: 25
    RMSE: 4.268

    Iteration: 26
    RMSE: 4.25774

    Iteration: 27
    RMSE: 4.25484

    Iteration: 28
    RMSE: 4.24948

    Iteration: 29
    RMSE: 4.24814

    Iteration: 30
    RMSE: 4.24573

    Iteration: 31
    RMSE: 4.24307

    Iteration: 32
    RMSE: 4.23946

    Iteration: 33
    RMSE: 4.23698

    Iteration: 34
    RMSE: 4.22852

    Iteration: 35
    RMSE: 4.22466

    Iteration: 36
    RMSE: 4.22338

    Iteration: 37
    RMSE: 4.21678

    Iteration: 38
    RMSE: 4.21427

    Iteration: 39
    RMSE: 4.20808

    Iteration: 40
    RMSE: 4.20989

    Iteration: 41
    RMSE: 4.20762

    Iteration: 42
    RMSE: 4.20557

    Iteration: 43
    RMSE: 4.20634

    Iteration: 44
    RMSE: 4.20334

    Iteration: 45
    RMSE: 4.20354

    Iteration: 46
    RMSE: 4.19892

    Iteration: 47
    RMSE: 4.20183

    Iteration: 48
    RMSE: 4.19973

    Iteration: 49
    RMSE: 4.19988

    Iteration: 50
    RMSE: 4.19659

    Iteration: 51
    RMSE: 4.19682

    Iteration: 52
    RMSE: 4.19734

    Iteration: 53
    RMSE: 4.19564

    Iteration: 54
    RMSE: 4.19198

    Iteration: 55
    RMSE: 4.19448

    Iteration: 56
    RMSE: 4.19818

    Iteration: 57
    RMSE: 4.19714

    Iteration: 58
    RMSE: 4.19567

    Iteration: 59
    RMSE: 4.19736

    Iteration: 60
    RMSE: 4.19704

    Iteration: 61
    RMSE: 4.19558

    Iteration: 62
    RMSE: 4.19394

    Iteration: 63
    RMSE: 4.19251

    Iteration: 64
    RMSE: 4.19598

    Iteration: 65
    RMSE: 4.19175

    Iteration: 66
    RMSE: 4.19533

    Iteration: 67
    RMSE: 4.1947

    Iteration: 68
    RMSE: 4.19227

    Iteration: 69
    RMSE: 4.1952

    Iteration: 70
    RMSE: 4.19366

    Iteration: 71
    RMSE: 4.19342

    Iteration: 72
    RMSE: 4.19299

    Iteration: 73
    RMSE: 4.19461

    Iteration: 74
    RMSE: 4.19233

    Iteration: 75
    RMSE: 4.19217

    Iteration: 76
    RMSE: 4.19459

    Iteration: 77
    RMSE: 4.19308

    Iteration: 78
    RMSE: 4.19372

    Iteration: 79
    RMSE: 4.1936

    Iteration: 80
    RMSE: 4.19267

    Iteration: 81
    RMSE: 4.19254

    Iteration: 82
    RMSE: 4.1938

    Iteration: 83
    RMSE: 4.19193

    Iteration: 84
    RMSE: 4.19022

    Iteration: 85
    RMSE: 4.19177

    Iteration: 86
    RMSE: 4.19154

    Iteration: 87
    RMSE: 4.19062

    Iteration: 88
    RMSE: 4.19284

    Iteration: 89
    RMSE: 4.18914

    Iteration: 90
    RMSE: 4.18927

    Iteration: 91
    RMSE: 4.19005

    Iteration: 92
    RMSE: 4.18961

    Iteration: 93
    RMSE: 4.18975

    Iteration: 94
    RMSE: 4.19173

    Iteration: 95
    RMSE: 4.19157

    Iteration: 96
    RMSE: 4.18828

    Iteration: 97
    RMSE: 4.18815

    Iteration: 98
    RMSE: 4.18974

    Iteration: 99
    RMSE: 4.18887

    Iteration: 100
    RMSE: 4.18804

    Iteration: 101
    RMSE: 4.18748

    Iteration: 102
    RMSE: 4.18849

    Iteration: 103
    RMSE: 4.188

    Iteration: 104
    RMSE: 4.19015

    Iteration: 105
    RMSE: 4.18699

    Iteration: 106
    RMSE: 4.18692

    Iteration: 107
    RMSE: 4.1884

    Iteration: 108
    RMSE: 4.18753

    Iteration: 109
    RMSE: 4.18557

    Iteration: 110
    RMSE: 4.18822

    Iteration: 111
    RMSE: 4.189

    Iteration: 112
    RMSE: 4.1869

    Iteration: 113
    RMSE: 4.18694

    Iteration: 114
    RMSE: 4.18876

    Iteration: 115
    RMSE: 4.1872

    Iteration: 116
    RMSE: 4.18639

    Iteration: 117
    RMSE: 4.18803

    Iteration: 118
    RMSE: 4.18574

    Iteration: 119
    RMSE: 4.18825

    Iteration: 120
    RMSE: 4.18677

    Iteration: 121
    RMSE: 4.18598

    Iteration: 122
    RMSE: 4.18498

    Iteration: 123
    RMSE: 4.18506

    Iteration: 124
    RMSE: 4.18846

    Iteration: 125
    RMSE: 4.18845

    Iteration: 126
    RMSE: 4.18425

    Iteration: 127
    RMSE: 4.18349

    Iteration: 128
    RMSE: 4.18544

    Iteration: 129
    RMSE: 4.18539

    Iteration: 130
    RMSE: 4.18532

    Iteration: 131
    RMSE: 4.18514

    Iteration: 132
    RMSE: 4.18433

    Iteration: 133
    RMSE: 4.18594

    Iteration: 134
    RMSE: 4.18414

    Iteration: 135
    RMSE: 4.18297

    Iteration: 136
    RMSE: 4.18576

    Iteration: 137
    RMSE: 4.18116

    Iteration: 138
    RMSE: 4.1837

    Iteration: 139
    RMSE: 4.18314

    Iteration: 140
    RMSE: 4.18137

    Iteration: 141
    RMSE: 4.18295

    Iteration: 142
    RMSE: 4.18098

    Iteration: 143
    RMSE: 4.18215

    Iteration: 144
    RMSE: 4.18241

    Iteration: 145
    RMSE: 4.18162

    Iteration: 146
    RMSE: 4.18243

    Iteration: 147
    RMSE: 4.18267

    Iteration: 148
    RMSE: 4.18263

    Iteration: 149
    RMSE: 4.18137

    Iteration: 150
    RMSE: 4.18372

    Iteration: 151
    RMSE: 4.18208

    Iteration: 152
    RMSE: 4.17983

    Iteration: 153
    RMSE: 4.18229

    Iteration: 154
    RMSE: 4.18344

    Iteration: 155
    RMSE: 4.18001

    Iteration: 156
    RMSE: 4.17874

    Iteration: 157
    RMSE: 4.18139

    Iteration: 158
    RMSE: 4.18192

    Iteration: 159
    RMSE: 4.18291

    Iteration: 160
    RMSE: 4.18102

    Iteration: 161
    RMSE: 4.18168

    Iteration: 162
    RMSE: 4.18134

    Iteration: 163
    RMSE: 4.18262

    Iteration: 164
    RMSE: 4.18136

    Iteration: 165
    RMSE: 4.18115

    Iteration: 166
    RMSE: 4.18201

    Iteration: 167
    RMSE: 4.17968

    Iteration: 168
    RMSE: 4.18225

    Iteration: 169
    RMSE: 4.18097

    Iteration: 170
    RMSE: 4.1841

    Iteration: 171
    RMSE: 4.17984

    Iteration: 172
    RMSE: 4.1809

    Iteration: 173
    RMSE: 4.17986

    Iteration: 174
    RMSE: 4.17861

    Iteration: 175
    RMSE: 4.18045

    Iteration: 176
    RMSE: 4.17714

    Iteration: 177
    RMSE: 4.17921

    Iteration: 178
    RMSE: 4.18006

    Iteration: 179
    RMSE: 4.17881

    Iteration: 180
    RMSE: 4.17908

    Iteration: 181
    RMSE: 4.18011

    Iteration: 182
    RMSE: 4.18031

    Iteration: 183
    RMSE: 4.17814

    Iteration: 184
    RMSE: 4.17795

    Iteration: 185
    RMSE: 4.18009

    Iteration: 186
    RMSE: 4.18009

    Iteration: 187
    RMSE: 4.17787

    Iteration: 188
    RMSE: 4.17836

    Iteration: 189
    RMSE: 4.1789

    Iteration: 190
    RMSE: 4.17868

    Iteration: 191
    RMSE: 4.17927

    Iteration: 192
    RMSE: 4.17811

    Iteration: 193
    RMSE: 4.17838

    Iteration: 194
    RMSE: 4.17664

    Iteration: 195
    RMSE: 4.17891

    Iteration: 196
    RMSE: 4.17975

    Iteration: 197
    RMSE: 4.18044

    Iteration: 198
    RMSE: 4.18138

    Iteration: 199
    RMSE: 4.17869

    Iteration: 200
    RMSE: 4.17578

    Iteration: 201
    RMSE: 4.17624

    Iteration: 202
    RMSE: 4.17747

    Iteration: 203
    RMSE: 4.17532

    Iteration: 204
    RMSE: 4.17876

    Iteration: 205
    RMSE: 4.17715

    Iteration: 206
    RMSE: 4.17796

    Iteration: 207
    RMSE: 4.17551

    Iteration: 208
    RMSE: 4.17704

    Iteration: 209
    RMSE: 4.17478

    Iteration: 210
    RMSE: 4.17757

    Iteration: 211
    RMSE: 4.17695

    Iteration: 212
    RMSE: 4.17369

    Iteration: 213
    RMSE: 4.17359

    Iteration: 214
    RMSE: 4.17426

    Iteration: 215
    RMSE: 4.17247

    Iteration: 216
    RMSE: 4.17126

    Iteration: 217
    RMSE: 4.177

    Iteration: 218
    RMSE: 4.17701

    Iteration: 219
    RMSE: 4.17552

    Iteration: 220
    RMSE: 4.17676

    Iteration: 221
    RMSE: 4.17863

    Iteration: 222
    RMSE: 4.17941

    Iteration: 223
    RMSE: 4.17908

    Iteration: 224
    RMSE: 4.17827

    Iteration: 225
    RMSE: 4.17631

    Iteration: 226
    RMSE: 4.17672

    Iteration: 227
    RMSE: 4.1752

    Iteration: 228
    RMSE: 4.17723

    Iteration: 229
    RMSE: 4.17695

    Iteration: 230
    RMSE: 4.17613

    Iteration: 231
    RMSE: 4.17632

    Iteration: 232
    RMSE: 4.17808

    Iteration: 233
    RMSE: 4.17542

    Iteration: 234
    RMSE: 4.17551

    Iteration: 235
    RMSE: 4.17575

    Iteration: 236
    RMSE: 4.17817

    Iteration: 237
    RMSE: 4.17549

    Iteration: 238
    RMSE: 4.17589

    Iteration: 239
    RMSE: 4.1765

    Iteration: 240
    RMSE: 4.17623

    Iteration: 241
    RMSE: 4.17397

    Iteration: 242
    RMSE: 4.17509

    Iteration: 243
    RMSE: 4.17429

    Iteration: 244
    RMSE: 4.17529

    Iteration: 245
    RMSE: 4.17607

    Iteration: 246
    RMSE: 4.17392

    Iteration: 247
    RMSE: 4.17289

    Iteration: 248
    RMSE: 4.17157

    Iteration: 249
    RMSE: 4.17634

    Iteration: 250
    RMSE: 4.17411

    Iteration: 251
    RMSE: 4.17242

    Iteration: 252
    RMSE: 4.17225

    Iteration: 253
    RMSE: 4.17385

    Iteration: 254
    RMSE: 4.17513

    Iteration: 255
    RMSE: 4.17334

    Iteration: 256
    RMSE: 4.17315

    Iteration: 257
    RMSE: 4.17348

    Iteration: 258
    RMSE: 4.17558

    Iteration: 259
    RMSE: 4.17489

    Iteration: 260
    RMSE: 4.17453

    Iteration: 261
    RMSE: 4.17606

    Iteration: 262
    RMSE: 4.17288

    Iteration: 263
    RMSE: 4.17344

    Iteration: 264
    RMSE: 4.17338

    Iteration: 265
    RMSE: 4.17289

    Iteration: 266
    RMSE: 4.17387

    Iteration: 267
    RMSE: 4.17399

    Iteration: 268
    RMSE: 4.17473

    Iteration: 269
    RMSE: 4.17627

    Iteration: 270
    RMSE: 4.1739

    Iteration: 271
    RMSE: 4.17374

    Iteration: 272
    RMSE: 4.17678

    Iteration: 273
    RMSE: 4.17561

    Iteration: 274
    RMSE: 4.17334

    Iteration: 275
    RMSE: 4.17215

    Iteration: 276
    RMSE: 4.1725

    Iteration: 277
    RMSE: 4.17401

    Iteration: 278
    RMSE: 4.17075

    Iteration: 279
    RMSE: 4.17269

    Iteration: 280
    RMSE: 4.17289

    Iteration: 281
    RMSE: 4.17093

    Iteration: 282
    RMSE: 4.16826

    Iteration: 283
    RMSE: 4.17359

    Iteration: 284
    RMSE: 4.17055

    Iteration: 285
    RMSE: 4.17155

    Iteration: 286
    RMSE: 4.17145

    Iteration: 287
    RMSE: 4.16986

    Iteration: 288
    RMSE: 4.16994

    Iteration: 289
    RMSE: 4.17305

    Iteration: 290
    RMSE: 4.1739

    Iteration: 291
    RMSE: 4.17216

    Iteration: 292
    RMSE: 4.1747

    Iteration: 293
    RMSE: 4.17331

    Iteration: 294
    RMSE: 4.17382

    Iteration: 295
    RMSE: 4.17009

    Iteration: 296
    RMSE: 4.172

    Iteration: 297
    RMSE: 4.174

    Iteration: 298
    RMSE: 4.17279

    Iteration: 299
    RMSE: 4.17181

    Iteration: 300
    RMSE: 4.17332

    Iteration: 301
    RMSE: 4.17236

    Iteration: 302
    RMSE: 4.17359

    Iteration: 303
    RMSE: 4.17273

    Iteration: 304
    RMSE: 4.17165

    Iteration: 305
    RMSE: 4.17166

    Iteration: 306
    RMSE: 4.17236

    Iteration: 307
    RMSE: 4.17234

    Iteration: 308
    RMSE: 4.17236

    Iteration: 309
    RMSE: 4.17428

    Iteration: 310
    RMSE: 4.17245

    Iteration: 311
    RMSE: 4.17205

    Iteration: 312
    RMSE: 4.171

    Iteration: 313
    RMSE: 4.17249

    Iteration: 314
    RMSE: 4.17232

    Iteration: 315
    RMSE: 4.17093

    Iteration: 316
    RMSE: 4.17175

    Iteration: 317
    RMSE: 4.16898

    Iteration: 318
    RMSE: 4.17128

    Iteration: 319
    RMSE: 4.17105

    Iteration: 320
    RMSE: 4.17003

    Iteration: 321
    RMSE: 4.16914

    Iteration: 322
    RMSE: 4.17083

    Iteration: 323
    RMSE: 4.16978

    Iteration: 324
    RMSE: 4.17017

    Iteration: 325
    RMSE: 4.17097

    Iteration: 326
    RMSE: 4.16986

    Iteration: 327
    RMSE: 4.1682

    Iteration: 328
    RMSE: 4.16749

    Iteration: 329
    RMSE: 4.16929

    Iteration: 330
    RMSE: 4.16873

    Iteration: 331
    RMSE: 4.17118

    Iteration: 332
    RMSE: 4.17154

    Iteration: 333
    RMSE: 4.17001

    Iteration: 334
    RMSE: 4.1708

    Iteration: 335
    RMSE: 4.17264

    Iteration: 336
    RMSE: 4.17023

    Iteration: 337
    RMSE: 4.16931

    Iteration: 338
    RMSE: 4.16813

    Iteration: 339
    RMSE: 4.16789

    Iteration: 340
    RMSE: 4.16668

    Iteration: 341
    RMSE: 4.17025

    Iteration: 342
    RMSE: 4.16777

    Iteration: 343
    RMSE: 4.1675

    Iteration: 344
    RMSE: 4.1687

    Iteration: 345
    RMSE: 4.16835

    Iteration: 346
    RMSE: 4.16751

    Iteration: 347
    RMSE: 4.16978

    Iteration: 348
    RMSE: 4.16877

    Iteration: 349
    RMSE: 4.1689

    Iteration: 350
    RMSE: 4.16963

    Iteration: 351
    RMSE: 4.16777

    Iteration: 352
    RMSE: 4.16809

    Iteration: 353
    RMSE: 4.16797

    Iteration: 354
    RMSE: 4.16736

    Iteration: 355
    RMSE: 4.1671

    Iteration: 356
    RMSE: 4.16618

    Iteration: 357
    RMSE: 4.1656

    Iteration: 358
    RMSE: 4.1659

    Iteration: 359
    RMSE: 4.16696

    Iteration: 360
    RMSE: 4.16637

    Iteration: 361
    RMSE: 4.16601

    Iteration: 362
    RMSE: 4.16693

    Iteration: 363
    RMSE: 4.16618

    Iteration: 364
    RMSE: 4.16718

    Iteration: 365
    RMSE: 4.16688

    Iteration: 366
    RMSE: 4.16519

    Iteration: 367
    RMSE: 4.16792

    Iteration: 368
    RMSE: 4.16583

    Iteration: 369
    RMSE: 4.16547

    Iteration: 370
    RMSE: 4.16399

    Iteration: 371
    RMSE: 4.16468

    Iteration: 372
    RMSE: 4.16694

    Iteration: 373
    RMSE: 4.16725

    Iteration: 374
    RMSE: 4.16577

    Iteration: 375
    RMSE: 4.16579

    Iteration: 376
    RMSE: 4.16584

    Iteration: 377
    RMSE: 4.16751

    Iteration: 378
    RMSE: 4.16708

    Iteration: 379
    RMSE: 4.16637

    Iteration: 380
    RMSE: 4.1665

    Iteration: 381
    RMSE: 4.16789

    Iteration: 382
    RMSE: 4.16743

    Iteration: 383
    RMSE: 4.16706

    Iteration: 384
    RMSE: 4.16624

    Iteration: 385
    RMSE: 4.16684

    Iteration: 386
    RMSE: 4.16697

    Iteration: 387
    RMSE: 4.16894

    Iteration: 388
    RMSE: 4.16874

    Iteration: 389
    RMSE: 4.16899

    Iteration: 390
    RMSE: 4.16905

    Iteration: 391
    RMSE: 4.16961

    Iteration: 392
    RMSE: 4.16781

    Iteration: 393
    RMSE: 4.16878

    Iteration: 394
    RMSE: 4.16832

    Iteration: 395
    RMSE: 4.16762

    Iteration: 396
    RMSE: 4.16623

    Iteration: 397
    RMSE: 4.16665

    Iteration: 398
    RMSE: 4.16558

    Iteration: 399
    RMSE: 4.16722

    Iteration: 400
    RMSE: 4.16444

    Iteration: 401
    RMSE: 4.16517

    Iteration: 402
    RMSE: 4.16674

    Iteration: 403
    RMSE: 4.16591

    Iteration: 404
    RMSE: 4.16632

    Iteration: 405
    RMSE: 4.16617

    Iteration: 406
    RMSE: 4.16708

    Iteration: 407
    RMSE: 4.16571

    Iteration: 408
    RMSE: 4.16768

    Iteration: 409
    RMSE: 4.16844

    Iteration: 410
    RMSE: 4.16772

    Iteration: 411
    RMSE: 4.16555

    Iteration: 412
    RMSE: 4.16743

    Iteration: 413
    RMSE: 4.16563

    Iteration: 414
    RMSE: 4.1675

    Iteration: 415
    RMSE: 4.16743

    Iteration: 416
    RMSE: 4.16605

    Iteration: 417
    RMSE: 4.16326

    Iteration: 418
    RMSE: 4.16383

    Iteration: 419
    RMSE: 4.16552

    Iteration: 420
    RMSE: 4.16279

    Iteration: 421
    RMSE: 4.16415

    Iteration: 422
    RMSE: 4.16367

    Iteration: 423
    RMSE: 4.16418

    Iteration: 424
    RMSE: 4.16417

    Iteration: 425
    RMSE: 4.16372

    Iteration: 426
    RMSE: 4.16442

    Iteration: 427
    RMSE: 4.16545

    Iteration: 428
    RMSE: 4.16462

    Iteration: 429
    RMSE: 4.16555

    Iteration: 430
    RMSE: 4.16425

    Iteration: 431
    RMSE: 4.16431

    Iteration: 432
    RMSE: 4.16681

    Iteration: 433
    RMSE: 4.16657

    Iteration: 434
    RMSE: 4.16499

    Iteration: 435
    RMSE: 4.16477

    Iteration: 436
    RMSE: 4.16458

    Iteration: 437
    RMSE: 4.16168

    Iteration: 438
    RMSE: 4.16246

    Iteration: 439
    RMSE: 4.16225

    Iteration: 440
    RMSE: 4.16225

    Iteration: 441
    RMSE: 4.16407

    Iteration: 442
    RMSE: 4.16112

    Iteration: 443
    RMSE: 4.16249

    Iteration: 444
    RMSE: 4.16291

    Iteration: 445
    RMSE: 4.16253

    Iteration: 446
    RMSE: 4.16451

    Iteration: 447
    RMSE: 4.16368

    Iteration: 448
    RMSE: 4.16489

    Iteration: 449
    RMSE: 4.16381

    Iteration: 450
    RMSE: 4.16302

    Iteration: 451
    RMSE: 4.163

    Iteration: 452
    RMSE: 4.16421

    Iteration: 453
    RMSE: 4.16465

    Iteration: 454
    RMSE: 4.1636

    Iteration: 455
    RMSE: 4.1642

    Iteration: 456
    RMSE: 4.16564

    Iteration: 457
    RMSE: 4.16436

    Iteration: 458
    RMSE: 4.16503

    Iteration: 459
    RMSE: 4.16319

    Iteration: 460
    RMSE: 4.16312

    Iteration: 461
    RMSE: 4.16486

    Iteration: 462
    RMSE: 4.16428

    Iteration: 463
    RMSE: 4.16379

    Iteration: 464
    RMSE: 4.16337

    Iteration: 465
    RMSE: 4.16406

    Iteration: 466
    RMSE: 4.16658

    Iteration: 467
    RMSE: 4.16517

    Iteration: 468
    RMSE: 4.16659

    Iteration: 469
    RMSE: 4.16484

    Iteration: 470
    RMSE: 4.16298

    Iteration: 471
    RMSE: 4.1631

    Iteration: 472
    RMSE: 4.1661

    Iteration: 473
    RMSE: 4.16564

    Iteration: 474
    RMSE: 4.16446

    Iteration: 475
    RMSE: 4.16275

    Iteration: 476
    RMSE: 4.16292

    Iteration: 477
    RMSE: 4.16035

    Iteration: 478
    RMSE: 4.16126

    Iteration: 479
    RMSE: 4.16133

    Iteration: 480
    RMSE: 4.16026

    Iteration: 481
    RMSE: 4.16129

    Iteration: 482
    RMSE: 4.16133

    Iteration: 483
    RMSE: 4.16274

    Iteration: 484
    RMSE: 4.16192

    Iteration: 485
    RMSE: 4.16262

    Iteration: 486
    RMSE: 4.16308

    Iteration: 487
    RMSE: 4.16263

    Iteration: 488
    RMSE: 4.16222

    Iteration: 489
    RMSE: 4.16207

    Iteration: 490
    RMSE: 4.16218

    Iteration: 491
    RMSE: 4.16179

    Iteration: 492
    RMSE: 4.1642

    Iteration: 493
    RMSE: 4.16363

    Iteration: 494
    RMSE: 4.16102

    Iteration: 495
    RMSE: 4.16284

    Iteration: 496
    RMSE: 4.16171

    Iteration: 497
    RMSE: 4.1652

    Iteration: 498
    RMSE: 4.16433

    Iteration: 499
    RMSE: 4.16273

    Iteration: 500
    RMSE: 4.16107

    Iteration: 501
    RMSE: 4.16232

    Iteration: 502
    RMSE: 4.1618

    Iteration: 503
    RMSE: 4.16223

    Iteration: 504
    RMSE: 4.16168

    Iteration: 505
    RMSE: 4.16284

    Iteration: 506
    RMSE: 4.16378

    Iteration: 507
    RMSE: 4.16212

    Iteration: 508
    RMSE: 4.16422

    Iteration: 509
    RMSE: 4.16234

    Iteration: 510
    RMSE: 4.16324

    Iteration: 511
    RMSE: 4.16225

    Iteration: 512
    RMSE: 4.16082

    Iteration: 513
    RMSE: 4.163

    Iteration: 514
    RMSE: 4.16321

    Iteration: 515
    RMSE: 4.16434

    Iteration: 516
    RMSE: 4.16235

    Iteration: 517
    RMSE: 4.1636

    Iteration: 518
    RMSE: 4.16216

    Iteration: 519
    RMSE: 4.16235

    Iteration: 520
    RMSE: 4.16234

    Iteration: 521
    RMSE: 4.16288

    Iteration: 522
    RMSE: 4.16282

    Iteration: 523
    RMSE: 4.16184

    Iteration: 524
    RMSE: 4.16182

    Iteration: 525
    RMSE: 4.16098

    Iteration: 526
    RMSE: 4.16173

    Iteration: 527
    RMSE: 4.15926

    Iteration: 528
    RMSE: 4.16168

    Iteration: 529
    RMSE: 4.16106

    Iteration: 530
    RMSE: 4.15992

    Iteration: 531
    RMSE: 4.16317

    Iteration: 532
    RMSE: 4.16298

    Iteration: 533
    RMSE: 4.16369

    Iteration: 534
    RMSE: 4.16314

    Iteration: 535
    RMSE: 4.16304

    Iteration: 536
    RMSE: 4.16304

    Iteration: 537
    RMSE: 4.16133

    Iteration: 538
    RMSE: 4.16195

    Iteration: 539
    RMSE: 4.16294

    Iteration: 540
    RMSE: 4.16286

    Iteration: 541
    RMSE: 4.16243

    Iteration: 542
    RMSE: 4.16176

    Iteration: 543
    RMSE: 4.16272

    Iteration: 544
    RMSE: 4.15979

    Iteration: 545
    RMSE: 4.16191

    Iteration: 546
    RMSE: 4.16243

    Iteration: 547
    RMSE: 4.16039

    Iteration: 548
    RMSE: 4.15978

    Iteration: 549
    RMSE: 4.16092

    Iteration: 550
    RMSE: 4.15984

    Iteration: 551
    RMSE: 4.1623

    Iteration: 552
    RMSE: 4.16409

    Iteration: 553
    RMSE: 4.16033

    Iteration: 554
    RMSE: 4.16119

    Iteration: 555
    RMSE: 4.15966

    Iteration: 556
    RMSE: 4.15945

    Iteration: 557
    RMSE: 4.16213

    Iteration: 558
    RMSE: 4.15928

    Iteration: 559
    RMSE: 4.16085

    Iteration: 560
    RMSE: 4.16203

    Iteration: 561
    RMSE: 4.16011

    Iteration: 562
    RMSE: 4.15991

    Iteration: 563
    RMSE: 4.15951

    Iteration: 564
    RMSE: 4.1596

    Iteration: 565
    RMSE: 4.16355

    Iteration: 566
    RMSE: 4.15974

    Iteration: 567
    RMSE: 4.15856

    Iteration: 568
    RMSE: 4.15861

    Iteration: 569
    RMSE: 4.15716

    Iteration: 570
    RMSE: 4.15849

    Iteration: 571
    RMSE: 4.16046

    Iteration: 572
    RMSE: 4.16011

    Iteration: 573
    RMSE: 4.15921

    Iteration: 574
    RMSE: 4.15938

    Iteration: 575
    RMSE: 4.16154

    Iteration: 576
    RMSE: 4.16154

    Iteration: 577
    RMSE: 4.15887

    Iteration: 578
    RMSE: 4.1588

    Iteration: 579
    RMSE: 4.15796

    Iteration: 580
    RMSE: 4.15974

    Iteration: 581
    RMSE: 4.15819

    Iteration: 582
    RMSE: 4.15807

    Iteration: 583
    RMSE: 4.15667

    Iteration: 584
    RMSE: 4.15733

    Iteration: 585
    RMSE: 4.15993

    Iteration: 586
    RMSE: 4.16053

    Iteration: 587
    RMSE: 4.16118

    Iteration: 588
    RMSE: 4.16026

    Iteration: 589
    RMSE: 4.15987

    Iteration: 590
    RMSE: 4.15952

    Iteration: 591
    RMSE: 4.15682

    Iteration: 592
    RMSE: 4.15832

    Iteration: 593
    RMSE: 4.15833

    Iteration: 594
    RMSE: 4.1577

    Iteration: 595
    RMSE: 4.15775

    Iteration: 596
    RMSE: 4.15642

    Iteration: 597
    RMSE: 4.15571

    Iteration: 598
    RMSE: 4.1577

    Iteration: 599
    RMSE: 4.15653

    Iteration: 600
    RMSE: 4.15757

    Iteration: 601
    RMSE: 4.15906

    Iteration: 602
    RMSE: 4.16079

    Iteration: 603
    RMSE: 4.15945

    Iteration: 604
    RMSE: 4.15811

    Iteration: 605
    RMSE: 4.1588

    Iteration: 606
    RMSE: 4.15881

    Iteration: 607
    RMSE: 4.15717

    Iteration: 608
    RMSE: 4.15933

    Iteration: 609
    RMSE: 4.15987

    Iteration: 610
    RMSE: 4.15728

    Iteration: 611
    RMSE: 4.15994

    Iteration: 612
    RMSE: 4.15902

    Iteration: 613
    RMSE: 4.15852

    Iteration: 614
    RMSE: 4.15839

    Iteration: 615
    RMSE: 4.1578

    Iteration: 616
    RMSE: 4.15797

    Iteration: 617
    RMSE: 4.15774

    Iteration: 618
    RMSE: 4.15835

    Iteration: 619
    RMSE: 4.15824

    Iteration: 620
    RMSE: 4.15705

    Iteration: 621
    RMSE: 4.15726

    Iteration: 622
    RMSE: 4.15782

    Iteration: 623
    RMSE: 4.15857

    Iteration: 624
    RMSE: 4.15799

    Iteration: 625
    RMSE: 4.15884

    Iteration: 626
    RMSE: 4.16054

    Iteration: 627
    RMSE: 4.15902

    Iteration: 628
    RMSE: 4.15699

    Iteration: 629
    RMSE: 4.16109

    Iteration: 630
    RMSE: 4.15861

    Iteration: 631
    RMSE: 4.15917

    Iteration: 632
    RMSE: 4.15887

    Iteration: 633
    RMSE: 4.15647

    Iteration: 634
    RMSE: 4.15823

    Iteration: 635
    RMSE: 4.16028

    Iteration: 636
    RMSE: 4.1601

    Iteration: 637
    RMSE: 4.15742

    Iteration: 638
    RMSE: 4.15817

    Iteration: 639
    RMSE: 4.15769

    Iteration: 640
    RMSE: 4.15625

    Iteration: 641
    RMSE: 4.15763

    Iteration: 642
    RMSE: 4.15804

    Iteration: 643
    RMSE: 4.15785

    Iteration: 644
    RMSE: 4.15737

    Iteration: 645
    RMSE: 4.15898

    Iteration: 646
    RMSE: 4.15838

    Iteration: 647
    RMSE: 4.15853

    Iteration: 648
    RMSE: 4.15881

    Iteration: 649
    RMSE: 4.15838

    Iteration: 650
    RMSE: 4.16069

    Iteration: 651
    RMSE: 4.15955

    Iteration: 652
    RMSE: 4.15669

    Iteration: 653
    RMSE: 4.15708

    Iteration: 654
    RMSE: 4.15769

    Iteration: 655
    RMSE: 4.15694

    Iteration: 656
    RMSE: 4.157

    Iteration: 657
    RMSE: 4.15634

    Iteration: 658
    RMSE: 4.15761

    Iteration: 659
    RMSE: 4.15692

    Iteration: 660
    RMSE: 4.15807

    Iteration: 661
    RMSE: 4.16184

    Iteration: 662
    RMSE: 4.15858

    Iteration: 663
    RMSE: 4.16103

    Iteration: 664
    RMSE: 4.15995

    Iteration: 665
    RMSE: 4.15981

    Iteration: 666
    RMSE: 4.1576

    Iteration: 667
    RMSE: 4.15777

    Iteration: 668
    RMSE: 4.15947

    Iteration: 669
    RMSE: 4.1592

    Iteration: 670
    RMSE: 4.15904

    Iteration: 671
    RMSE: 4.1585

    Iteration: 672
    RMSE: 4.1585

    Iteration: 673
    RMSE: 4.15837

    Iteration: 674
    RMSE: 4.15709

    Iteration: 675
    RMSE: 4.15779

    Iteration: 676
    RMSE: 4.15935

    Iteration: 677
    RMSE: 4.15877

    Iteration: 678
    RMSE: 4.15883

    Iteration: 679
    RMSE: 4.15922

    Iteration: 680
    RMSE: 4.15867

    Iteration: 681
    RMSE: 4.15815

    Iteration: 682
    RMSE: 4.15678

    Iteration: 683
    RMSE: 4.1566

    Iteration: 684
    RMSE: 4.15598

    Iteration: 685
    RMSE: 4.15618

    Iteration: 686
    RMSE: 4.15449

    Iteration: 687
    RMSE: 4.15927

    Iteration: 688
    RMSE: 4.15882

    Iteration: 689
    RMSE: 4.15921

    Iteration: 690
    RMSE: 4.15909

    Iteration: 691
    RMSE: 4.15761

    Iteration: 692
    RMSE: 4.15742

    Iteration: 693
    RMSE: 4.15599

    Iteration: 694
    RMSE: 4.15684

    Iteration: 695
    RMSE: 4.15769

    Iteration: 696
    RMSE: 4.15657

    Iteration: 697
    RMSE: 4.15524

    Iteration: 698
    RMSE: 4.15687

    Iteration: 699
    RMSE: 4.1581

    Iteration: 700
    RMSE: 4.15827

    Iteration: 701
    RMSE: 4.15746

    Iteration: 702
    RMSE: 4.15712

    Iteration: 703
    RMSE: 4.15504

    Iteration: 704
    RMSE: 4.15773

    Iteration: 705
    RMSE: 4.159

    Iteration: 706
    RMSE: 4.15919

    Iteration: 707
    RMSE: 4.15864

    Iteration: 708
    RMSE: 4.15756

    Iteration: 709
    RMSE: 4.15506

    Iteration: 710
    RMSE: 4.15699

    Iteration: 711
    RMSE: 4.15752

    Iteration: 712
    RMSE: 4.15788

    Iteration: 713
    RMSE: 4.15727

    Iteration: 714
    RMSE: 4.15913

    Iteration: 715
    RMSE: 4.15857

    Iteration: 716
    RMSE: 4.15899

    Iteration: 717
    RMSE: 4.15628

    Iteration: 718
    RMSE: 4.15863

    Iteration: 719
    RMSE: 4.15747

    Iteration: 720
    RMSE: 4.15825

    Iteration: 721
    RMSE: 4.15687

    Iteration: 722
    RMSE: 4.1563

    Iteration: 723
    RMSE: 4.15624

    Iteration: 724
    RMSE: 4.15781

    Iteration: 725
    RMSE: 4.15611

    Iteration: 726
    RMSE: 4.1586

    Iteration: 727
    RMSE: 4.15703

    Iteration: 728
    RMSE: 4.15698

    Iteration: 729
    RMSE: 4.15683

    Iteration: 730
    RMSE: 4.15752

    Iteration: 731
    RMSE: 4.15657

    Iteration: 732
    RMSE: 4.15914

    Iteration: 733
    RMSE: 4.16022

    Iteration: 734
    RMSE: 4.16061

    Iteration: 735
    RMSE: 4.16015

    Iteration: 736
    RMSE: 4.15716

    Iteration: 737
    RMSE: 4.16056

    Iteration: 738
    RMSE: 4.15791

    Iteration: 739
    RMSE: 4.15853

    Iteration: 740
    RMSE: 4.15696

    Iteration: 741
    RMSE: 4.15783

    Iteration: 742
    RMSE: 4.15702

    Iteration: 743
    RMSE: 4.15734

    Iteration: 744
    RMSE: 4.1568

    Iteration: 745
    RMSE: 4.15814

    Iteration: 746
    RMSE: 4.15754

    Iteration: 747
    RMSE: 4.15596

    Iteration: 748
    RMSE: 4.15875

    Iteration: 749
    RMSE: 4.15772

    Iteration: 750
    RMSE: 4.15802

    Iteration: 751
    RMSE: 4.15765

    Iteration: 752
    RMSE: 4.15695

    Iteration: 753
    RMSE: 4.15986

    Iteration: 754
    RMSE: 4.15844

    Iteration: 755
    RMSE: 4.15867

    Iteration: 756
    RMSE: 4.15851

    Iteration: 757
    RMSE: 4.15696

    Iteration: 758
    RMSE: 4.15604

    Iteration: 759
    RMSE: 4.15601

    Iteration: 760
    RMSE: 4.15818

    Iteration: 761
    RMSE: 4.15847

    Iteration: 762
    RMSE: 4.15796

    Iteration: 763
    RMSE: 4.15887

    Iteration: 764
    RMSE: 4.15793

    Iteration: 765
    RMSE: 4.15877

    Iteration: 766
    RMSE: 4.15808

    Iteration: 767
    RMSE: 4.15952

    Iteration: 768
    RMSE: 4.15749

    Iteration: 769
    RMSE: 4.15667

    Iteration: 770
    RMSE: 4.15812

    Iteration: 771
    RMSE: 4.15867

    Iteration: 772
    RMSE: 4.15655

    Iteration: 773
    RMSE: 4.15763

    Iteration: 774
    RMSE: 4.15908

    Iteration: 775
    RMSE: 4.15774

    Iteration: 776
    RMSE: 4.15607

    Iteration: 777
    RMSE: 4.1559

    Iteration: 778
    RMSE: 4.15557

    Iteration: 779
    RMSE: 4.15565

    Iteration: 780
    RMSE: 4.15667

    Iteration: 781
    RMSE: 4.15631

    Iteration: 782
    RMSE: 4.1573

    Iteration: 783
    RMSE: 4.15513

    Iteration: 784
    RMSE: 4.15728

    Iteration: 785
    RMSE: 4.15477

    Iteration: 786
    RMSE: 4.15596

    Iteration: 787
    RMSE: 4.15689

    Iteration: 788
    RMSE: 4.15679

    Iteration: 789
    RMSE: 4.15816

    Iteration: 790
    RMSE: 4.15667

    Iteration: 791
    RMSE: 4.15663

    Iteration: 792
    RMSE: 4.15647

    Iteration: 793
    RMSE: 4.15827

    Iteration: 794
    RMSE: 4.15685

    Iteration: 795
    RMSE: 4.15613

    Iteration: 796
    RMSE: 4.15649

    Iteration: 797
    RMSE: 4.1563

    Iteration: 798
    RMSE: 4.15712

    Iteration: 799
    RMSE: 4.15812

    Iteration: 800
    RMSE: 4.1572

    Iteration: 801
    RMSE: 4.15575

    Iteration: 802
    RMSE: 4.15595

    Iteration: 803
    RMSE: 4.15698

    Iteration: 804
    RMSE: 4.15932

    Iteration: 805
    RMSE: 4.15982

    Iteration: 806
    RMSE: 4.1565

    Iteration: 807
    RMSE: 4.15554

    Iteration: 808
    RMSE: 4.15535

    Iteration: 809
    RMSE: 4.15631

    Iteration: 810
    RMSE: 4.15458

    Iteration: 811
    RMSE: 4.15689

    Iteration: 812
    RMSE: 4.15488

    Iteration: 813
    RMSE: 4.15619

    Iteration: 814
    RMSE: 4.15715

    Iteration: 815
    RMSE: 4.15415

    Iteration: 816
    RMSE: 4.1535

    Iteration: 817
    RMSE: 4.15597

    Iteration: 818
    RMSE: 4.15546

    Iteration: 819
    RMSE: 4.15563

    Iteration: 820
    RMSE: 4.15337

    Iteration: 821
    RMSE: 4.15443

    Iteration: 822
    RMSE: 4.15354

    Iteration: 823
    RMSE: 4.15319

    Iteration: 824
    RMSE: 4.15386

    Iteration: 825
    RMSE: 4.15305

    Iteration: 826
    RMSE: 4.1542

    Iteration: 827
    RMSE: 4.15446

    Iteration: 828
    RMSE: 4.15537

    Iteration: 829
    RMSE: 4.15485

    Iteration: 830
    RMSE: 4.15455

    Iteration: 831
    RMSE: 4.15381

    Iteration: 832
    RMSE: 4.15293

    Iteration: 833
    RMSE: 4.15339

    Iteration: 834
    RMSE: 4.15438

    Iteration: 835
    RMSE: 4.15388

    Iteration: 836
    RMSE: 4.15402

    Iteration: 837
    RMSE: 4.15334

    Iteration: 838
    RMSE: 4.1554

    Iteration: 839
    RMSE: 4.15568

    Iteration: 840
    RMSE: 4.15788

    Iteration: 841
    RMSE: 4.15639

    Iteration: 842
    RMSE: 4.15776

    Iteration: 843
    RMSE: 4.15681

    Iteration: 844
    RMSE: 4.15712

    Iteration: 845
    RMSE: 4.15679

    Iteration: 846
    RMSE: 4.15642

    Iteration: 847
    RMSE: 4.15971

    Iteration: 848
    RMSE: 4.15703

    Iteration: 849
    RMSE: 4.15466

    Iteration: 850
    RMSE: 4.15552

    Iteration: 851
    RMSE: 4.15658

    Iteration: 852
    RMSE: 4.15462

    Iteration: 853
    RMSE: 4.1566

    Iteration: 854
    RMSE: 4.15421

    Iteration: 855
    RMSE: 4.15498

    Iteration: 856
    RMSE: 4.15379

    Iteration: 857
    RMSE: 4.15441

    Iteration: 858
    RMSE: 4.15279

    Iteration: 859
    RMSE: 4.15556

    Iteration: 860
    RMSE: 4.15473

    Iteration: 861
    RMSE: 4.15602

    Iteration: 862
    RMSE: 4.15379

    Iteration: 863
    RMSE: 4.15428

    Iteration: 864
    RMSE: 4.15354

    Iteration: 865
    RMSE: 4.1558

    Iteration: 866
    RMSE: 4.15519

    Iteration: 867
    RMSE: 4.15477

    Iteration: 868
    RMSE: 4.15636

    Iteration: 869
    RMSE: 4.15373

    Iteration: 870
    RMSE: 4.15472

    Iteration: 871
    RMSE: 4.15732

    Iteration: 872
    RMSE: 4.15654

    Iteration: 873
    RMSE: 4.15646

    Iteration: 874
    RMSE: 4.15549

    Iteration: 875
    RMSE: 4.15704

    Iteration: 876
    RMSE: 4.15463

    Iteration: 877
    RMSE: 4.15368

    Iteration: 878
    RMSE: 4.15772

    Iteration: 879
    RMSE: 4.15647

    Iteration: 880
    RMSE: 4.15665

    Iteration: 881
    RMSE: 4.15569

    Iteration: 882
    RMSE: 4.15439

    Iteration: 883
    RMSE: 4.1527

    Iteration: 884
    RMSE: 4.1554

    Iteration: 885
    RMSE: 4.15438

    Iteration: 886
    RMSE: 4.15238

    Iteration: 887
    RMSE: 4.15265

    Iteration: 888
    RMSE: 4.1552

    Iteration: 889
    RMSE: 4.15408

    Iteration: 890
    RMSE: 4.15447

    Iteration: 891
    RMSE: 4.15393

    Iteration: 892
    RMSE: 4.15473

    Iteration: 893
    RMSE: 4.15443

    Iteration: 894
    RMSE: 4.15489

    Iteration: 895
    RMSE: 4.15547

    Iteration: 896
    RMSE: 4.15547

    Iteration: 897
    RMSE: 4.15386

    Iteration: 898
    RMSE: 4.15405

    Iteration: 899
    RMSE: 4.15272

    Iteration: 900
    RMSE: 4.15296

    Iteration: 901
    RMSE: 4.15311

    Iteration: 902
    RMSE: 4.15507

    Iteration: 903
    RMSE: 4.15346

    Iteration: 904
    RMSE: 4.15461

    Iteration: 905
    RMSE: 4.15206

    Iteration: 906
    RMSE: 4.15446

    Iteration: 907
    RMSE: 4.15504

    Iteration: 908
    RMSE: 4.15574

    Iteration: 909
    RMSE: 4.15402

    Iteration: 910
    RMSE: 4.15308

    Iteration: 911
    RMSE: 4.155

    Iteration: 912
    RMSE: 4.15647

    Iteration: 913
    RMSE: 4.15674

    Iteration: 914
    RMSE: 4.15459

    Iteration: 915
    RMSE: 4.15744

    Iteration: 916
    RMSE: 4.15384

    Iteration: 917
    RMSE: 4.15472

    Iteration: 918
    RMSE: 4.15455

    Iteration: 919
    RMSE: 4.15229

    Iteration: 920
    RMSE: 4.15317

    Iteration: 921
    RMSE: 4.15333

    Iteration: 922
    RMSE: 4.15408

    Iteration: 923
    RMSE: 4.15353

    Iteration: 924
    RMSE: 4.15217

    Iteration: 925
    RMSE: 4.15339

    Iteration: 926
    RMSE: 4.15074

    Iteration: 927
    RMSE: 4.15465

    Iteration: 928
    RMSE: 4.15373

    Iteration: 929
    RMSE: 4.15498

    Iteration: 930
    RMSE: 4.1556

    Iteration: 931
    RMSE: 4.15304

    Iteration: 932
    RMSE: 4.15465

    Iteration: 933
    RMSE: 4.15367

    Iteration: 934
    RMSE: 4.15473

    Iteration: 935
    RMSE: 4.15471

    Iteration: 936
    RMSE: 4.15567

    Iteration: 937
    RMSE: 4.1553

    Iteration: 938
    RMSE: 4.15612

    Iteration: 939
    RMSE: 4.15595

    Iteration: 940
    RMSE: 4.15551

    Iteration: 941
    RMSE: 4.15551

    Iteration: 942
    RMSE: 4.15364

    Iteration: 943
    RMSE: 4.1547

    Iteration: 944
    RMSE: 4.1545

    Iteration: 945
    RMSE: 4.15482

    Iteration: 946
    RMSE: 4.15417

    Iteration: 947
    RMSE: 4.15322

    Iteration: 948
    RMSE: 4.15297

    Iteration: 949
    RMSE: 4.154

    Iteration: 950
    RMSE: 4.15177

    Iteration: 951
    RMSE: 4.15395

    Iteration: 952
    RMSE: 4.15337

    Iteration: 953
    RMSE: 4.15431

    Iteration: 954
    RMSE: 4.15226

    Iteration: 955
    RMSE: 4.15305

    Iteration: 956
    RMSE: 4.15527

    Iteration: 957
    RMSE: 4.15138

    Iteration: 958
    RMSE: 4.15376

    Iteration: 959
    RMSE: 4.15315

    Iteration: 960
    RMSE: 4.15528

    Iteration: 961
    RMSE: 4.15594

    Iteration: 962
    RMSE: 4.15606

    Iteration: 963
    RMSE: 4.15597

    Iteration: 964
    RMSE: 4.15667

    Iteration: 965
    RMSE: 4.15627

    Iteration: 966
    RMSE: 4.15389

    Iteration: 967
    RMSE: 4.15359

    Iteration: 968
    RMSE: 4.15297

    Iteration: 969
    RMSE: 4.15342

    Iteration: 970
    RMSE: 4.15471

    Iteration: 971
    RMSE: 4.15533

    Iteration: 972
    RMSE: 4.15454

    Iteration: 973
    RMSE: 4.15643

    Iteration: 974
    RMSE: 4.15336

    Iteration: 975
    RMSE: 4.15231

    Iteration: 976
    RMSE: 4.15104

    Iteration: 977
    RMSE: 4.154

    Iteration: 978
    RMSE: 4.15376

    Iteration: 979
    RMSE: 4.15413

    Iteration: 980
    RMSE: 4.15162

    Iteration: 981
    RMSE: 4.15275

    Iteration: 982
    RMSE: 4.15388

    Iteration: 983
    RMSE: 4.15513

    Iteration: 984
    RMSE: 4.15584

    Iteration: 985
    RMSE: 4.15474

    Iteration: 986
    RMSE: 4.15343

    Iteration: 987
    RMSE: 4.15481

    Iteration: 988
    RMSE: 4.15436

    Iteration: 989
    RMSE: 4.15307

    Iteration: 990
    RMSE: 4.15283

    Iteration: 991
    RMSE: 4.15265

    Iteration: 992
    RMSE: 4.1551

    Iteration: 993
    RMSE: 4.15512

    Iteration: 994
    RMSE: 4.15345

    Iteration: 995
    RMSE: 4.15506

    Iteration: 996
    RMSE: 4.15246

    Iteration: 997
    RMSE: 4.15427

    Iteration: 998
    RMSE: 4.15387

    Iteration: 999
    RMSE: 4.15278

    Iteration: 1000
    RMSE: 4.15219

    8064
    8070
    8076
    8082
    8088
    8094
    8100
    8106
    8112
    8118
    8124
    8130
    8136
    8142
    8148
    8154
    8160
    8166
    8172
    8178
    8184
    8190
    8196
    8202
    8208
    8214
    8220
    8226
    8232
    8238
    8244
    8250
    8256
    8262
    8268
    8274
    8280
    8286
    8292
    8298
    8304
    8310
    8316
    8322
    8328
    8334
    8340
    8346
    8352
    8358
    8364
    8370
    8376
    8382
    8388
    8394
    8400
    8406
    8412
    8418
    8424
    8430
    8436
    8442
    8448
    8454
    8460
    8466
    8472
    8478
    8484
    8490
    8496
    8502
    8508
    8514
    8520
    8526
    8532
    8538
    8544
    8550
    8556
    8562
    8568
    8574
    8580
    8586
    8592
    8598
    8604
    8610
    8616
    8622
    8628
    8634
    8640
    8646
    8652
    8658
    8664
    8670
    8676
    8682
    8688
    8694
    8700
    8706
    8712
    8718
    8724
    8730
    8736
    8742
    8748
    8754
    8760
    8766
    8772
    8778
    Final MAPE: 0.164031
    Final RMSE: 6.08458

    Running time: 32810 seconds

```python
mat_hat10 = np.percentile(result, 5, axis = 2)
mat_hat90 = np.percentile(result, 95, axis = 2)
mat_hat = np.mean(result, axis = 2)
```

```python
X = dense_mat.copy()
pred_steps = int(num_roll * num_step)
tv = 144
```

```python
import matplotlib.pyplot as plt
plt.style.use('ggplot')
figsize = 2
for i in range(3):
    fig = plt.figure(figsize = (4 * figsize, 1 * figsize))
    ax = fig.add_axes([0.13, 0.28, 0.85, 0.68])
    plt.plot(X[i, 54 * tv :], color = "black", linewidth = 0.5)
    plt.plot(list(range(X.shape[1] - pred_steps - 54 * tv, X.shape[1] - 54 * tv)),
             mat_hat[i, :], color = "#e3120b", linewidth = 2.0)
    plt.plot(list(range(X.shape[1] - pred_steps - 54 * tv, X.shape[1] - 54 * tv)),
             mat_hat10[i, :], color = "blue", linewidth = 0.5)
    plt.plot(list(range(X.shape[1] - pred_steps - 54 * tv, X.shape[1] - 54 * tv)),
             mat_hat90[i, :], color = "green", linewidth = 0.5)
```

```python
import scipy.io

tensor = scipy.io.loadmat('../datasets/Hangzhou-data-set/tensor.mat')
tensor = tensor['tensor']
random_matrix = scipy.io.loadmat('../datasets/Hangzhou-data-set/random_matrix.mat')
random_matrix = random_matrix['random_matrix']
random_tensor = scipy.io.loadmat('../datasets/Hangzhou-data-set/random_tensor.mat')
random_tensor = random_tensor['random_tensor']

dense_mat = tensor.reshape([tensor.shape[0], tensor.shape[1] * tensor.shape[2]])
missing_rate = 0.4

# =============================================================================
### Random missing (RM) scenario
### Set the RM scenario by:
binary_mat = np.round(random_tensor + 0.5 - missing_rate).reshape([random_tensor.shape[0],
                                                                   random_tensor.shape[1]
                                                                   * random_tensor.shape[2]])
# =============================================================================

sparse_mat = np.multiply(dense_mat, binary_mat)
```

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 10
time_lags = np.array([1, 2, 108])
num_step = 5
num_roll = int(108 * 5 / num_step)
start_time = dim2 - num_roll * num_step
init = {"W": 0.1 * np.random.rand(dim1, rank),
        "X": 0.1 * np.random.rand(start_time, rank)}
burn_iter = 100
gibbs_iter = 10
result = forecastor(dense_mat, sparse_mat, init, time_lags,
                    num_roll, start_time, num_step, burn_iter, gibbs_iter)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

    Iteration: 1
    RMSE: 101.287

    Iteration: 2
    RMSE: 70.3381

    Iteration: 3
    RMSE: 53.6979

    Iteration: 4
    RMSE: 49.4189

    Iteration: 5
    RMSE: 47.1461

    Iteration: 6
    RMSE: 45.86

    Iteration: 7
    RMSE: 45.7183

    Iteration: 8
    RMSE: 45.8896

    Iteration: 9
    RMSE: 46.2484

    Iteration: 10
    RMSE: 46.7664

    Iteration: 11
    RMSE: 47.4777

    Iteration: 12
    RMSE: 47.3682

    Iteration: 13
    RMSE: 47.5301

    Iteration: 14
    RMSE: 46.6131

    Iteration: 15
    RMSE: 45.4952

    Iteration: 16
    RMSE: 44.8379

    Iteration: 17
    RMSE: 44.3899

    Iteration: 18
    RMSE: 44.7274

    Iteration: 19
    RMSE: 45.0761

    Iteration: 20
    RMSE: 45.8206

    Iteration: 21
    RMSE: 45.0064

    Iteration: 22
    RMSE: 44.4442

    Iteration: 23
    RMSE: 45.485

    Iteration: 24
    RMSE: 45.1824

    Iteration: 25
    RMSE: 44.531

    Iteration: 26
    RMSE: 44.591

    Iteration: 27
    RMSE: 44.6757

    Iteration: 28
    RMSE: 44.9708

    Iteration: 29
    RMSE: 45.0923

    Iteration: 30
    RMSE: 45.0159

    Iteration: 31
    RMSE: 44.904

    Iteration: 32
    RMSE: 45.3887

    Iteration: 33
    RMSE: 46.0251

    Iteration: 34
    RMSE: 45.6158

    Iteration: 35
    RMSE: 45.502

    Iteration: 36
    RMSE: 44.7124

    Iteration: 37
    RMSE: 44.7735

    Iteration: 38
    RMSE: 43.9153

    Iteration: 39
    RMSE: 43.7807

    Iteration: 40
    RMSE: 44.1097

    Iteration: 41
    RMSE: 43.6194

    Iteration: 42
    RMSE: 44.0903

    Iteration: 43
    RMSE: 44.4178

    Iteration: 44
    RMSE: 43.6577

    Iteration: 45
    RMSE: 43.8949

    Iteration: 46
    RMSE: 44.4267

    Iteration: 47
    RMSE: 43.694

    Iteration: 48
    RMSE: 43.6536

    Iteration: 49
    RMSE: 44.7691

    Iteration: 50
    RMSE: 43.8228

    Iteration: 51
    RMSE: 44.2445

    Iteration: 52
    RMSE: 43.9087

    Iteration: 53
    RMSE: 43.3626

    Iteration: 54
    RMSE: 43.486

    Iteration: 55
    RMSE: 43.7106

    Iteration: 56
    RMSE: 44.1793

    Iteration: 57
    RMSE: 43.8947

    Iteration: 58
    RMSE: 44.2573

    Iteration: 59
    RMSE: 44.0629

    Iteration: 60
    RMSE: 43.4897

    Iteration: 61
    RMSE: 42.5123

    Iteration: 62
    RMSE: 42.6549

    Iteration: 63
    RMSE: 43.0575

    Iteration: 64
    RMSE: 43.3994

    Iteration: 65
    RMSE: 42.6277

    Iteration: 66
    RMSE: 43.9714

    Iteration: 67
    RMSE: 43.942

    Iteration: 68
    RMSE: 43.5723

    Iteration: 69
    RMSE: 44.0702

    Iteration: 70
    RMSE: 43.9521

    Iteration: 71
    RMSE: 44.324

    Iteration: 72
    RMSE: 44.3996

    Iteration: 73
    RMSE: 44.9953

    Iteration: 74
    RMSE: 43.1493

    Iteration: 75
    RMSE: 43.4973

    Iteration: 76
    RMSE: 43.2437

    Iteration: 77
    RMSE: 43.317

    Iteration: 78
    RMSE: 42.9942

    Iteration: 79
    RMSE: 42.1611

    Iteration: 80
    RMSE: 42.6557

    Iteration: 81
    RMSE: 42.1078

    Iteration: 82
    RMSE: 42.2983

    Iteration: 83
    RMSE: 43.1372

    Iteration: 84
    RMSE: 42.7722

    Iteration: 85
    RMSE: 43.1752

    Iteration: 86
    RMSE: 42.8701

    Iteration: 87
    RMSE: 43.3126

    Iteration: 88
    RMSE: 42.5251

    Iteration: 89
    RMSE: 42.6926

    Iteration: 90
    RMSE: 42.7488

    Iteration: 91
    RMSE: 42.4678

    Iteration: 92
    RMSE: 42.4275

    Iteration: 93
    RMSE: 43.4612

    Iteration: 94
    RMSE: 43.1562

    Iteration: 95
    RMSE: 42.5489

    Iteration: 96
    RMSE: 42.4343

    Iteration: 97
    RMSE: 41.5781

    Iteration: 98
    RMSE: 42.5625

    Iteration: 99
    RMSE: 43.1908

    Iteration: 100
    RMSE: 43.0127

    2160
    2165
    2170
    2175
    2180
    2185
    2190
    2195
    2200
    2205
    2210
    2215
    2220
    2225
    2230
    2235
    2240
    2245
    2250
    2255
    2260
    2265
    2270
    2275
    2280
    2285
    2290
    2295
    2300
    2305
    2310
    2315
    2320
    2325
    2330
    2335
    2340
    2345
    2350
    2355
    2360
    2365
    2370
    2375
    2380
    2385
    2390
    2395
    2400
    2405
    2410
    2415
    2420
    2425
    2430
    2435
    2440
    2445
    2450
    2455
    2460
    2465
    2470
    2475
    2480
    2485
    2490
    2495
    2500
    2505
    2510
    2515
    2520
    2525
    2530
    2535
    2540
    2545
    2550
    2555
    2560
    2565
    2570
    2575
    2580
    2585
    2590
    2595
    2600
    2605
    2610
    2615
    2620
    2625
    2630
    2635
    2640
    2645
    2650
    2655
    2660
    2665
    2670
    2675
    2680
    2685
    2690
    2695
    Final MAPE: 0.585583
    Final RMSE: 80.7796

    Running time: 875 seconds

```python
mat_hat10 = np.percentile(result, 10, axis = 2)
mat_hat90 = np.percentile(result, 90, axis = 2)
mat_hat = np.mean(result, axis = 2)

X = dense_mat.copy()
pred_steps = int(num_roll * num_step)
tv = 108

import matplotlib.pyplot as plt
plt.style.use('ggplot')
figsize = 2
for i in range(3):
    fig = plt.figure(figsize = (8 * figsize, 2 * figsize))
    ax = fig.add_axes([0.13, 0.28, 0.85, 0.68])
    plt.plot(X[i, 18 * tv :], color = "black", linewidth = 0.5)
    plt.plot(list(range(X.shape[1] - pred_steps - 18 * tv, X.shape[1] - 18 * tv)),
             mat_hat[i, :], color = "#e3120b", linewidth = 2.0)
    plt.plot(list(range(X.shape[1] - pred_steps - 18 * tv, X.shape[1] - 18 * tv)),
             mat_hat10[i, :], color = "blue", linewidth = 0.5)
    plt.plot(list(range(X.shape[1] - pred_steps - 18 * tv, X.shape[1] - 18 * tv)),
             mat_hat90[i, :], color = "green", linewidth = 0.5)
```

```python
import scipy.io

tensor = scipy.io.loadmat('../datasets/Hangzhou-data-set/tensor.mat')
tensor = tensor['tensor']
random_matrix = scipy.io.loadmat('../datasets/Hangzhou-data-set/random_matrix.mat')
random_matrix = random_matrix['random_matrix']
random_tensor = scipy.io.loadmat('../datasets/Hangzhou-data-set/random_tensor.mat')
random_tensor = random_tensor['random_tensor']

dense_mat = tensor.reshape([tensor.shape[0], tensor.shape[1] * tensor.shape[2]])
missing_rate = 0.0

# =============================================================================
### Random missing (RM) scenario
### Set the RM scenario by:
binary_mat = np.round(random_tensor + 0.5 - missing_rate).reshape([random_tensor.shape[0],
                                                                   random_tensor.shape[1]
                                                                   * random_tensor.shape[2]])
# =============================================================================

sparse_mat = np.multiply(dense_mat, binary_mat)
```

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 10
time_lags = np.array([1, 2, 108])
num_step = 5
num_roll = int(108 * 5 / num_step)
start_time = dim2 - num_roll * num_step
init = {"W": 0.1 * np.random.rand(dim1, rank),
        "X": 0.1 * np.random.rand(start_time, rank)}
burn_iter = 100
gibbs_iter = 10
result = forecastor(dense_mat, sparse_mat, init, time_lags,
                    num_roll, start_time, num_step, burn_iter, gibbs_iter)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

    /Users/xinyuchen/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:18: RuntimeWarning: invalid value encountered in double_scalars

    Iteration: 1
    RMSE: nan

    Iteration: 2
    RMSE: nan

    Iteration: 3
    RMSE: nan

    Iteration: 4
    RMSE: nan

    Iteration: 5
    RMSE: nan

    Iteration: 6
    RMSE: nan

    Iteration: 7
    RMSE: nan

    Iteration: 8
    RMSE: nan

    Iteration: 9
    RMSE: nan

    Iteration: 10
    RMSE: nan

    Iteration: 11
    RMSE: nan

    Iteration: 12
    RMSE: nan

    Iteration: 13
    RMSE: nan

    Iteration: 14
    RMSE: nan

    Iteration: 15
    RMSE: nan

    Iteration: 16
    RMSE: nan

    Iteration: 17
    RMSE: nan

    Iteration: 18
    RMSE: nan

    Iteration: 19
    RMSE: nan

    Iteration: 20
    RMSE: nan

    Iteration: 21
    RMSE: nan

    Iteration: 22
    RMSE: nan

    Iteration: 23
    RMSE: nan

    Iteration: 24
    RMSE: nan

    Iteration: 25
    RMSE: nan

    Iteration: 26
    RMSE: nan

    Iteration: 27
    RMSE: nan

    Iteration: 28
    RMSE: nan

    Iteration: 29
    RMSE: nan

    Iteration: 30
    RMSE: nan

    Iteration: 31
    RMSE: nan

    Iteration: 32
    RMSE: nan

    Iteration: 33
    RMSE: nan

    Iteration: 34
    RMSE: nan

    Iteration: 35
    RMSE: nan

    Iteration: 36
    RMSE: nan

    Iteration: 37
    RMSE: nan

    Iteration: 38
    RMSE: nan

    Iteration: 39
    RMSE: nan

    Iteration: 40
    RMSE: nan

    Iteration: 41
    RMSE: nan

    Iteration: 42
    RMSE: nan

    Iteration: 43
    RMSE: nan

    Iteration: 44
    RMSE: nan

    Iteration: 45
    RMSE: nan

    Iteration: 46
    RMSE: nan

    Iteration: 47
    RMSE: nan

    Iteration: 48
    RMSE: nan

    Iteration: 49
    RMSE: nan

    Iteration: 50
    RMSE: nan

    Iteration: 51
    RMSE: nan

    Iteration: 52
    RMSE: nan

    Iteration: 53
    RMSE: nan

    Iteration: 54
    RMSE: nan

    Iteration: 55
    RMSE: nan

    Iteration: 56
    RMSE: nan

    Iteration: 57
    RMSE: nan

    Iteration: 58
    RMSE: nan

    Iteration: 59
    RMSE: nan

    Iteration: 60
    RMSE: nan

    Iteration: 61
    RMSE: nan

    Iteration: 62
    RMSE: nan

    Iteration: 63
    RMSE: nan

    Iteration: 64
    RMSE: nan

    Iteration: 65
    RMSE: nan

    Iteration: 66
    RMSE: nan

    Iteration: 67
    RMSE: nan

    Iteration: 68
    RMSE: nan

    Iteration: 69
    RMSE: nan

    Iteration: 70
    RMSE: nan

    Iteration: 71
    RMSE: nan

    Iteration: 72
    RMSE: nan

    Iteration: 73
    RMSE: nan

    Iteration: 74
    RMSE: nan

    Iteration: 75
    RMSE: nan

    Iteration: 76
    RMSE: nan

    Iteration: 77
    RMSE: nan

    Iteration: 78
    RMSE: nan

    Iteration: 79
    RMSE: nan

    Iteration: 80
    RMSE: nan

    Iteration: 81
    RMSE: nan

    Iteration: 82
    RMSE: nan

    Iteration: 83
    RMSE: nan

    Iteration: 84
    RMSE: nan

    Iteration: 85
    RMSE: nan

    Iteration: 86
    RMSE: nan

    Iteration: 87
    RMSE: nan

    Iteration: 88
    RMSE: nan

    Iteration: 89
    RMSE: nan

    Iteration: 90
    RMSE: nan

    Iteration: 91
    RMSE: nan

    Iteration: 92
    RMSE: nan

    Iteration: 93
    RMSE: nan

    Iteration: 94
    RMSE: nan

    Iteration: 95
    RMSE: nan

    Iteration: 96
    RMSE: nan

    Iteration: 97
    RMSE: nan

    Iteration: 98
    RMSE: nan

    Iteration: 99
    RMSE: nan

    Iteration: 100
    RMSE: nan

    2160
    2165
    2170
    2175
    2180
    2185
    2190
    2195
    2200
    2205
    2210
    2215
    2220
    2225
    2230
    2235
    2240
    2245
    2250
    2255
    2260
    2265
    2270
    2275
    2280
    2285
    2290
    2295
    2300
    2305
    2310
    2315
    2320
    2325
    2330
    2335
    2340
    2345
    2350
    2355
    2360
    2365
    2370
    2375
    2380
    2385
    2390
    2395
    2400
    2405
    2410
    2415
    2420
    2425
    2430
    2435
    2440
    2445
    2450
    2455
    2460
    2465
    2470
    2475
    2480
    2485
    2490
    2495
    2500
    2505
    2510
    2515
    2520
    2525
    2530
    2535
    2540
    2545
    2550
    2555
    2560
    2565
    2570
    2575
    2580
    2585
    2590
    2595
    2600
    2605
    2610
    2615
    2620
    2625
    2630
    2635
    2640
    2645
    2650
    2655
    2660
    2665
    2670
    2675
    2680
    2685
    2690
    2695
    Final MAPE: 0.582281
    Final RMSE: 75.4304

    Running time: 866 seconds

```python
import time
start = time.time()
dim1, dim2 = sparse_mat.shape
rank = 10
time_lags = np.array([1, 2, 108])
num_step = 5
num_roll = int(108 * 5 / num_step)
start_time = dim2 - num_roll * num_step
init = {"W": 0.1 * np.random.rand(dim1, rank),
        "X": 0.1 * np.random.rand(start_time, rank)}
burn_iter = 500
gibbs_iter = 50
result = forecastor(dense_mat, sparse_mat, init, time_lags,
                    num_roll, start_time, num_step, burn_iter, gibbs_iter)
end = time.time()
print('Running time: %d seconds'%(end - start))
```

    /Users/xinyuchen/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:18: RuntimeWarning: invalid value encountered in double_scalars

    Iteration: 1
    RMSE: nan

    Iteration: 2
    RMSE: nan

    Iteration: 3
    RMSE: nan

    Iteration: 4
    RMSE: nan

    Iteration: 5
    RMSE: nan

    Iteration: 6
    RMSE: nan

    Iteration: 7
    RMSE: nan

    Iteration: 8
    RMSE: nan

    Iteration: 9
    RMSE: nan

    Iteration: 10
    RMSE: nan

    Iteration: 11
    RMSE: nan

    Iteration: 12
    RMSE: nan

    Iteration: 13
    RMSE: nan

    Iteration: 14
    RMSE: nan

    Iteration: 15
    RMSE: nan

    Iteration: 16
    RMSE: nan

    Iteration: 17
    RMSE: nan

    Iteration: 18
    RMSE: nan

    Iteration: 19
    RMSE: nan

    Iteration: 20
    RMSE: nan

    Iteration: 21
    RMSE: nan

    Iteration: 22
    RMSE: nan

    Iteration: 23
    RMSE: nan

    Iteration: 24
    RMSE: nan

    Iteration: 25
    RMSE: nan

    Iteration: 26
    RMSE: nan

    Iteration: 27
    RMSE: nan

    Iteration: 28
    RMSE: nan

    Iteration: 29
    RMSE: nan

    Iteration: 30
    RMSE: nan

    Iteration: 31
    RMSE: nan

    Iteration: 32
    RMSE: nan

    Iteration: 33
    RMSE: nan

    Iteration: 34
    RMSE: nan

    Iteration: 35
    RMSE: nan

    Iteration: 36
    RMSE: nan

    Iteration: 37
    RMSE: nan

    Iteration: 38
    RMSE: nan

    Iteration: 39
    RMSE: nan

    Iteration: 40
    RMSE: nan

    Iteration: 41
    RMSE: nan

    Iteration: 42
    RMSE: nan

    Iteration: 43
    RMSE: nan

    Iteration: 44
    RMSE: nan

    Iteration: 45
    RMSE: nan

    Iteration: 46
    RMSE: nan

    Iteration: 47
    RMSE: nan

    Iteration: 48
    RMSE: nan

    Iteration: 49
    RMSE: nan

    Iteration: 50
    RMSE: nan

    Iteration: 51
    RMSE: nan

    Iteration: 52
    RMSE: nan

    Iteration: 53
    RMSE: nan

    Iteration: 54
    RMSE: nan

    Iteration: 55
    RMSE: nan

    Iteration: 56
    RMSE: nan

    Iteration: 57
    RMSE: nan

    Iteration: 58
    RMSE: nan

    Iteration: 59
    RMSE: nan

    Iteration: 60
    RMSE: nan

    Iteration: 61
    RMSE: nan

    Iteration: 62
    RMSE: nan

    Iteration: 63
    RMSE: nan

    Iteration: 64
    RMSE: nan

    Iteration: 65
    RMSE: nan

    Iteration: 66
    RMSE: nan

    Iteration: 67
    RMSE: nan

    Iteration: 68
    RMSE: nan

    Iteration: 69
    RMSE: nan

    Iteration: 70
    RMSE: nan

    Iteration: 71
    RMSE: nan

    Iteration: 72
    RMSE: nan

    Iteration: 73
    RMSE: nan

    Iteration: 74
    RMSE: nan

    Iteration: 75
    RMSE: nan

    Iteration: 76
    RMSE: nan

    Iteration: 77
    RMSE: nan

    Iteration: 78
    RMSE: nan

    Iteration: 79
    RMSE: nan

    Iteration: 80
    RMSE: nan

    Iteration: 81
    RMSE: nan

    Iteration: 82
    RMSE: nan

    Iteration: 83
    RMSE: nan

    Iteration: 84
    RMSE: nan

    Iteration: 85
    RMSE: nan

    Iteration: 86
    RMSE: nan

    Iteration: 87
    RMSE: nan

    Iteration: 88
    RMSE: nan

    Iteration: 89
    RMSE: nan

    Iteration: 90
    RMSE: nan

    Iteration: 91
    RMSE: nan

    Iteration: 92
    RMSE: nan

    Iteration: 93
    RMSE: nan

    Iteration: 94
    RMSE: nan

    Iteration: 95
    RMSE: nan

    Iteration: 96
    RMSE: nan

    Iteration: 97
    RMSE: nan

    Iteration: 98
    RMSE: nan

    Iteration: 99
    RMSE: nan

    Iteration: 100
    RMSE: nan

    Iteration: 101
    RMSE: nan

    Iteration: 102
    RMSE: nan

    Iteration: 103
    RMSE: nan

    Iteration: 104
    RMSE: nan

    Iteration: 105
    RMSE: nan

    Iteration: 106
    RMSE: nan

    Iteration: 107
    RMSE: nan

    Iteration: 108
    RMSE: nan

    Iteration: 109
    RMSE: nan

    Iteration: 110
    RMSE: nan

    Iteration: 111
    RMSE: nan

    Iteration: 112
    RMSE: nan

    Iteration: 113
    RMSE: nan

    Iteration: 114
    RMSE: nan

    Iteration: 115
    RMSE: nan

    Iteration: 116
    RMSE: nan

    Iteration: 117
    RMSE: nan

    Iteration: 118
    RMSE: nan

    Iteration: 119
    RMSE: nan

    Iteration: 120
    RMSE: nan

    Iteration: 121
    RMSE: nan

    Iteration: 122
    RMSE: nan

    Iteration: 123
    RMSE: nan

    Iteration: 124
    RMSE: nan

    Iteration: 125
    RMSE: nan

    Iteration: 126
    RMSE: nan

    Iteration: 127
    RMSE: nan

    Iteration: 128
    RMSE: nan

    Iteration: 129
    RMSE: nan

    Iteration: 130
    RMSE: nan

    Iteration: 131
    RMSE: nan

    Iteration: 132
    RMSE: nan

    Iteration: 133
    RMSE: nan

    Iteration: 134
    RMSE: nan

    Iteration: 135
    RMSE: nan

    Iteration: 136
    RMSE: nan

    Iteration: 137
    RMSE: nan

    Iteration: 138
    RMSE: nan

    Iteration: 139
    RMSE: nan

    Iteration: 140
    RMSE: nan

    Iteration: 141
    RMSE: nan

    Iteration: 142
    RMSE: nan

    Iteration: 143
    RMSE: nan

    Iteration: 144
    RMSE: nan

    Iteration: 145
    RMSE: nan

    Iteration: 146
    RMSE: nan

    Iteration: 147
    RMSE: nan

    Iteration: 148
    RMSE: nan

    Iteration: 149
    RMSE: nan

    Iteration: 150
    RMSE: nan

    Iteration: 151
    RMSE: nan

    Iteration: 152
    RMSE: nan

    Iteration: 153
    RMSE: nan

    Iteration: 154
    RMSE: nan

    Iteration: 155
    RMSE: nan

    Iteration: 156
    RMSE: nan

    Iteration: 157
    RMSE: nan

    Iteration: 158
    RMSE: nan

    Iteration: 159
    RMSE: nan

    Iteration: 160
    RMSE: nan

    Iteration: 161
    RMSE: nan

    Iteration: 162
    RMSE: nan

    Iteration: 163
    RMSE: nan

    Iteration: 164
    RMSE: nan

    Iteration: 165
    RMSE: nan

    Iteration: 166
    RMSE: nan

    Iteration: 167
    RMSE: nan

    Iteration: 168
    RMSE: nan

    Iteration: 169
    RMSE: nan

    Iteration: 170
    RMSE: nan

    Iteration: 171
    RMSE: nan

    Iteration: 172
    RMSE: nan

    Iteration: 173
    RMSE: nan

    Iteration: 174
    RMSE: nan

    Iteration: 175
    RMSE: nan

    Iteration: 176
    RMSE: nan

    Iteration: 177
    RMSE: nan

    Iteration: 178
    RMSE: nan

    Iteration: 179
    RMSE: nan

    Iteration: 180
    RMSE: nan

    Iteration: 181
    RMSE: nan

    Iteration: 182
    RMSE: nan

    Iteration: 183
    RMSE: nan

    Iteration: 184
    RMSE: nan

    Iteration: 185
    RMSE: nan

    Iteration: 186
    RMSE: nan

    Iteration: 187
    RMSE: nan

    Iteration: 188
    RMSE: nan

    Iteration: 189
    RMSE: nan

    Iteration: 190
    RMSE: nan

    Iteration: 191
    RMSE: nan

    Iteration: 192
    RMSE: nan

    Iteration: 193
    RMSE: nan

    Iteration: 194
    RMSE: nan

    Iteration: 195
    RMSE: nan

    Iteration: 196
    RMSE: nan

    Iteration: 197
    RMSE: nan

    Iteration: 198
    RMSE: nan

    Iteration: 199
    RMSE: nan

    Iteration: 200
    RMSE: nan

    Iteration: 201
    RMSE: nan

    Iteration: 202
    RMSE: nan

    Iteration: 203
    RMSE: nan

    Iteration: 204
    RMSE: nan

    Iteration: 205
    RMSE: nan

    Iteration: 206
    RMSE: nan

    Iteration: 207
    RMSE: nan

    Iteration: 208
    RMSE: nan

    Iteration: 209
    RMSE: nan

    Iteration: 210
    RMSE: nan

    Iteration: 211
    RMSE: nan

    Iteration: 212
    RMSE: nan

    Iteration: 213
    RMSE: nan

    Iteration: 214
    RMSE: nan

    Iteration: 215
    RMSE: nan

    Iteration: 216
    RMSE: nan

    Iteration: 217
    RMSE: nan

    Iteration: 218
    RMSE: nan

    Iteration: 219
    RMSE: nan

    Iteration: 220
    RMSE: nan

    Iteration: 221
    RMSE: nan

    Iteration: 222
    RMSE: nan

    Iteration: 223
    RMSE: nan

    Iteration: 224
    RMSE: nan

    Iteration: 225
    RMSE: nan

    Iteration: 226
    RMSE: nan

    Iteration: 227
    RMSE: nan

    Iteration: 228
    RMSE: nan

    Iteration: 229
    RMSE: nan

    Iteration: 230
    RMSE: nan

    Iteration: 231
    RMSE: nan

    Iteration: 232
    RMSE: nan

    Iteration: 233
    RMSE: nan

    Iteration: 234
    RMSE: nan

    Iteration: 235
    RMSE: nan

    Iteration: 236
    RMSE: nan

    Iteration: 237
    RMSE: nan

    Iteration: 238
    RMSE: nan

    Iteration: 239
    RMSE: nan

    Iteration: 240
    RMSE: nan

    Iteration: 241
    RMSE: nan

    Iteration: 242
    RMSE: nan

    Iteration: 243
    RMSE: nan

    Iteration: 244
    RMSE: nan

    Iteration: 245
    RMSE: nan

    Iteration: 246
    RMSE: nan

    Iteration: 247
    RMSE: nan

    Iteration: 248
    RMSE: nan

    Iteration: 249
    RMSE: nan

    Iteration: 250
    RMSE: nan

    Iteration: 251
    RMSE: nan

    Iteration: 252
    RMSE: nan

    Iteration: 253
    RMSE: nan

    Iteration: 254
    RMSE: nan

    Iteration: 255
    RMSE: nan

    Iteration: 256
    RMSE: nan

    Iteration: 257
    RMSE: nan

    Iteration: 258
    RMSE: nan

    Iteration: 259
    RMSE: nan

    Iteration: 260
    RMSE: nan

    Iteration: 261
    RMSE: nan

    Iteration: 262
    RMSE: nan

    Iteration: 263
    RMSE: nan

    Iteration: 264
    RMSE: nan

    Iteration: 265
    RMSE: nan

    Iteration: 266
    RMSE: nan

    Iteration: 267
    RMSE: nan

    Iteration: 268
    RMSE: nan

    Iteration: 269
    RMSE: nan

    Iteration: 270
    RMSE: nan

    Iteration: 271
    RMSE: nan

    Iteration: 272
    RMSE: nan

    Iteration: 273
    RMSE: nan

    Iteration: 274
    RMSE: nan

    Iteration: 275
    RMSE: nan

    Iteration: 276
    RMSE: nan

    Iteration: 277
    RMSE: nan

    Iteration: 278
    RMSE: nan

    Iteration: 279
    RMSE: nan

    Iteration: 280
    RMSE: nan

    Iteration: 281
    RMSE: nan

    Iteration: 282
    RMSE: nan

    Iteration: 283
    RMSE: nan

    Iteration: 284
    RMSE: nan

    Iteration: 285
    RMSE: nan

    Iteration: 286
    RMSE: nan

    Iteration: 287
    RMSE: nan

    Iteration: 288
    RMSE: nan

    Iteration: 289
    RMSE: nan

    Iteration: 290
    RMSE: nan

    Iteration: 291
    RMSE: nan

    Iteration: 292
    RMSE: nan

    Iteration: 293
    RMSE: nan

    Iteration: 294
    RMSE: nan

    Iteration: 295
    RMSE: nan

    Iteration: 296
    RMSE: nan

    Iteration: 297
    RMSE: nan

    Iteration: 298
    RMSE: nan

    Iteration: 299
    RMSE: nan

    Iteration: 300
    RMSE: nan

    Iteration: 301
    RMSE: nan

    Iteration: 302
    RMSE: nan

    Iteration: 303
    RMSE: nan

    Iteration: 304
    RMSE: nan

    Iteration: 305
    RMSE: nan

    Iteration: 306
    RMSE: nan

    Iteration: 307
    RMSE: nan

    Iteration: 308
    RMSE: nan

    Iteration: 309
    RMSE: nan

    Iteration: 310
    RMSE: nan

    Iteration: 311
    RMSE: nan

    Iteration: 312
    RMSE: nan

    Iteration: 313
    RMSE: nan

    Iteration: 314
    RMSE: nan

    Iteration: 315
    RMSE: nan

    Iteration: 316
    RMSE: nan

    Iteration: 317
    RMSE: nan

    Iteration: 318
    RMSE: nan

    Iteration: 319
    RMSE: nan

    Iteration: 320
    RMSE: nan

    Iteration: 321
    RMSE: nan

    Iteration: 322
    RMSE: nan

    Iteration: 323
    RMSE: nan

    Iteration: 324
    RMSE: nan

    Iteration: 325
    RMSE: nan

    Iteration: 326
    RMSE: nan

    Iteration: 327
    RMSE: nan

    Iteration: 328
    RMSE: nan

    Iteration: 329
    RMSE: nan

    Iteration: 330
    RMSE: nan

    Iteration: 331
    RMSE: nan

    Iteration: 332
    RMSE: nan

    Iteration: 333
    RMSE: nan

    Iteration: 334
    RMSE: nan

    Iteration: 335
    RMSE: nan

    Iteration: 336
    RMSE: nan

    Iteration: 337
    RMSE: nan

    Iteration: 338
    RMSE: nan

    Iteration: 339
    RMSE: nan

    Iteration: 340
    RMSE: nan

    Iteration: 341
    RMSE: nan

    Iteration: 342
    RMSE: nan

    Iteration: 343
    RMSE: nan

    Iteration: 344
    RMSE: nan

    Iteration: 345
    RMSE: nan

    Iteration: 346
    RMSE: nan

    Iteration: 347
    RMSE: nan

    Iteration: 348
    RMSE: nan

    Iteration: 349
    RMSE: nan

    Iteration: 350
    RMSE: nan

    Iteration: 351
    RMSE: nan

    Iteration: 352
    RMSE: nan

    Iteration: 353
    RMSE: nan

    Iteration: 354
    RMSE: nan

    Iteration: 355
    RMSE: nan

    Iteration: 356
    RMSE: nan

    Iteration: 357
    RMSE: nan

    Iteration: 358
    RMSE: nan

    Iteration: 359
    RMSE: nan

    Iteration: 360
    RMSE: nan

    Iteration: 361
    RMSE: nan

    Iteration: 362
    RMSE: nan

    Iteration: 363
    RMSE: nan

    Iteration: 364
    RMSE: nan

    Iteration: 365
    RMSE: nan

    Iteration: 366
    RMSE: nan

    Iteration: 367
    RMSE: nan

    Iteration: 368
    RMSE: nan

    Iteration: 369
    RMSE: nan

    Iteration: 370
    RMSE: nan

    Iteration: 371
    RMSE: nan

    Iteration: 372
    RMSE: nan

    Iteration: 373
    RMSE: nan

    Iteration: 374
    RMSE: nan

    Iteration: 375
    RMSE: nan

    Iteration: 376
    RMSE: nan

    Iteration: 377
    RMSE: nan

    Iteration: 378
    RMSE: nan

    Iteration: 379
    RMSE: nan

    Iteration: 380
    RMSE: nan

    Iteration: 381
    RMSE: nan

    Iteration: 382
    RMSE: nan

    Iteration: 383
    RMSE: nan

    Iteration: 384
    RMSE: nan

    Iteration: 385
    RMSE: nan

    Iteration: 386
    RMSE: nan

    Iteration: 387
    RMSE: nan

    Iteration: 388
    RMSE: nan

    Iteration: 389
    RMSE: nan

    Iteration: 390
    RMSE: nan

    Iteration: 391
    RMSE: nan

    Iteration: 392
    RMSE: nan

    Iteration: 393
    RMSE: nan

    Iteration: 394
    RMSE: nan

    Iteration: 395
    RMSE: nan

    Iteration: 396
    RMSE: nan

    Iteration: 397
    RMSE: nan

    Iteration: 398
    RMSE: nan

    Iteration: 399
    RMSE: nan

    Iteration: 400
    RMSE: nan

    Iteration: 401
    RMSE: nan

    Iteration: 402
    RMSE: nan

    Iteration: 403
    RMSE: nan

    Iteration: 404
    RMSE: nan

    Iteration: 405
    RMSE: nan

    Iteration: 406
    RMSE: nan

    Iteration: 407
    RMSE: nan

    Iteration: 408
    RMSE: nan

    Iteration: 409
    RMSE: nan

    Iteration: 410
    RMSE: nan

    Iteration: 411
    RMSE: nan

    Iteration: 412
    RMSE: nan

    Iteration: 413
    RMSE: nan

    Iteration: 414
    RMSE: nan

    Iteration: 415
    RMSE: nan

    Iteration: 416
    RMSE: nan

    Iteration: 417
    RMSE: nan

    Iteration: 418
    RMSE: nan

    Iteration: 419
    RMSE: nan

    Iteration: 420
    RMSE: nan

    Iteration: 421
    RMSE: nan

    Iteration: 422
    RMSE: nan

    Iteration: 423
    RMSE: nan

    Iteration: 424
    RMSE: nan

    Iteration: 425
    RMSE: nan

    Iteration: 426
    RMSE: nan

    Iteration: 427
    RMSE: nan

    Iteration: 428
    RMSE: nan

    Iteration: 429
    RMSE: nan

    Iteration: 430
    RMSE: nan

    Iteration: 431
    RMSE: nan

    Iteration: 432
    RMSE: nan

    Iteration: 433
    RMSE: nan

    Iteration: 434
    RMSE: nan

    Iteration: 435
    RMSE: nan

    Iteration: 436
    RMSE: nan

    Iteration: 437
    RMSE: nan

    Iteration: 438
    RMSE: nan

    Iteration: 439
    RMSE: nan

    Iteration: 440
    RMSE: nan

    Iteration: 441
    RMSE: nan

    Iteration: 442
    RMSE: nan

    Iteration: 443
    RMSE: nan

    Iteration: 444
    RMSE: nan

    Iteration: 445
    RMSE: nan

    Iteration: 446
    RMSE: nan

    Iteration: 447
    RMSE: nan

    Iteration: 448
    RMSE: nan

    Iteration: 449
    RMSE: nan

    Iteration: 450
    RMSE: nan

    Iteration: 451
    RMSE: nan

    Iteration: 452
    RMSE: nan

    Iteration: 453
    RMSE: nan

    Iteration: 454
    RMSE: nan

    Iteration: 455
    RMSE: nan

    Iteration: 456
    RMSE: nan

    Iteration: 457
    RMSE: nan

    Iteration: 458
    RMSE: nan

    Iteration: 459
    RMSE: nan

    Iteration: 460
    RMSE: nan

    Iteration: 461
    RMSE: nan

    Iteration: 462
    RMSE: nan

    Iteration: 463
    RMSE: nan

    Iteration: 464
    RMSE: nan

    Iteration: 465
    RMSE: nan

    Iteration: 466
    RMSE: nan

    Iteration: 467
    RMSE: nan

    Iteration: 468
    RMSE: nan

    Iteration: 469
    RMSE: nan

    Iteration: 470
    RMSE: nan

    Iteration: 471
    RMSE: nan

    Iteration: 472
    RMSE: nan

    Iteration: 473
    RMSE: nan

    Iteration: 474
    RMSE: nan

    Iteration: 475
    RMSE: nan

    Iteration: 476
    RMSE: nan

    Iteration: 477
    RMSE: nan

    Iteration: 478
    RMSE: nan

    Iteration: 479
    RMSE: nan

    Iteration: 480
    RMSE: nan

    Iteration: 481
    RMSE: nan

    Iteration: 482
    RMSE: nan

    Iteration: 483
    RMSE: nan

    Iteration: 484
    RMSE: nan

    Iteration: 485
    RMSE: nan

    Iteration: 486
    RMSE: nan

    Iteration: 487
    RMSE: nan

    Iteration: 488
    RMSE: nan

    Iteration: 489
    RMSE: nan

    Iteration: 490
    RMSE: nan

    Iteration: 491
    RMSE: nan

    Iteration: 492
    RMSE: nan

    Iteration: 493
    RMSE: nan

    Iteration: 494
    RMSE: nan

    Iteration: 495
    RMSE: nan

    Iteration: 496
    RMSE: nan

    Iteration: 497
    RMSE: nan

    Iteration: 498
    RMSE: nan

    Iteration: 499
    RMSE: nan

    Iteration: 500
    RMSE: nan

    2160
    2165
    2170
    2175
    2180
    2185
    2190
    2195
    2200
    2205
    2210
    2215
    2220
    2225
    2230
    2235
    2240
    2245
    2250
    2255
    2260
    2265
    2270
    2275
    2280
    2285
    2290
    2295
    2300
    2305
    2310
    2315
    2320
    2325
    2330
    2335
    2340
    2345
    2350
    2355
    2360
    2365
    2370
    2375
    2380
    2385
    2390
    2395
    2400
    2405
    2410
    2415
    2420
    2425
    2430
    2435
    2440
    2445
    2450
    2455
    2460
    2465
    2470
    2475
    2480
    2485
    2490
    2495
    2500
    2505
    2510
    2515
    2520
    2525
    2530
    2535
    2540
    2545
    2550
    2555
    2560
    2565
    2570
    2575
    2580
    2585
    2590
    2595
    2600
    2605
    2610
    2615
    2620
    2625
    2630
    2635
    2640
    2645
    2650
    2655
    2660
    2665
    2670
    2675
    2680
    2685
    2690
    2695
    Final MAPE: 0.514535
    Final RMSE: 74.9771

    Running time: 4369 seconds