TIMENET TIME SERIES CLASSIFICATION SERIES

However, if they are trained with stochastic gradient descent, they will have difficulty in learning long-term dependencies that are encoded in the input sequences due to the vanishing gradient problem 8, 9. In such modelling, the RNN maintains a vector of activation parameters for each time step, especially when short-term dependencies are included in the input data.

TIMENET TIME SERIES CLASSIFICATION SERIES

The reason for such attention is that RNNs are a class of ANN models that possess an internal state or short-term memory due to the recurrent feedback connections, which makes RNNs suitable for modelling sequential or time series data. Among these models, recurrent neural networks (RNNs) have received much attention 4, 5. There are many neural network models that are widely used to solve several types of time series forecasting problems.

The most important advantage of deep learning is that it does not need any hand-crafted features to learn and can easily learn a hierarchical feature representation from the raw data directly 7.

Recently, deep learning has gained substantial popularity in the machine learning community since it is considered a general framework that facilitates the training of deep neural networks with many hidden layers 6. In contrast to shallow ANN architectures, it is widely demonstrated that the deep ANN architecture, which is called a deep neural network (DNN), outperforms the conventional shallow ANN architecture in several applications 5. conventional ANNs that have shallow architectures are difficult to train if they become too complex, e.g., when the network includes many layers and, consequently, many parameters. The artificial neural network (ANN) is a widely used model employed for the time series forecasting problem in the context of its universal approximation capabilities 4. Thus, exploring the optimum parameters of a regression model with multivariate sequential data is still in demand. However, it is demonstrated that regression models based on non-parametric methods are highly sensitive to the model parameters 3. In related research, sensor integrity has been analysed by non-parametric methods, such as Bayesian methods 2. In such complex applications, one of the key requirements is to maintain the integrity of the sensory data so that it can be monitored and analysed in a trusted manner. They are continuously produced in a wide spectrum of industrial, environmental, social, and healthcare applications, such as health monitoring, volatility analysis in financial markets, control systems in automobiles and avionics, and monitoring in data centres, to mention just a few. Such complex datasets may include several correlated variables therefore, they are denoted as multivariate time series (MTS) datasets 1. Recent advances in sensors and measurement technology have led to the collection of high-dimensional datasets from multiple sources recorded over time. Overall, the experimental results clearly show that the unsupervised pre-training approach improves the performance of deep LSTM and leads to better and faster convergence than other models. In addition, the proposed approach outperforms several reference models investigating the same case studies. For evaluation purposes, two different case studies that include real-world datasets are investigated, where the performance of the proposed approach compares favourably with the deep LSTM approach. In this paper, we propose a pre-trained LSTM-based stacked autoencoder (LSTM-SAE) approach in an unsupervised learning fashion to replace the random weight initialization strategy adopted in deep LSTM recurrent networks. The reason is that the supervised learning approach initializes the neurons randomly in such recurrent networks, disabling the neurons that ultimately must properly learn the latent features of the correlated variables included in the MTS dataset. Despite the reported advantages of the deep LSTM model, its performance in modelling multivariate time series (MTS) data has not been satisfactory, particularly when attempting to process highly non-linear and long-interval MTS datasets. Recently, the deep architecture of the recurrent neural network (RNN) and its variant long short-term memory (LSTM) have been proven to be more accurate than traditional statistical methods in modelling time series data. Such datasets are attracting much attention therefore, the need for accurate modelling of such high-dimensional datasets is increasing. Currently, most real-world time series datasets are multivariate and are rich in dynamical information of the underlying system.

Timenet time series classification