Exploring LSTM-based Attention Mechanisms with PSO and Grid Search under Different Normalization Techniques for Energy demands Time Series Forecasting

Pranolo, Andri and Zhou, Xiaofeng and Mao, Yingchi and Pratolo, Bambang Widi and Wibawa, Aji Prasetya and Utama, Agung Bella Putra and Ba, Abdoul Fatakhou and Muhammad, Abdullahi Uwaisu (2024) Exploring LSTM-based Attention Mechanisms with PSO and Grid Search under Different Normalization Techniques for Energy demands Time Series Forecasting. Knowledge Engineering and Data Science, 7 (1). pp. 1-12.

Text 51081-167600-3-PB.pdf Download (1MB)

Abstract

Advanced analytical approaches are required to accurately forecast the energy sector's rising complexity and volume of time series data. This research aims to forecast the energy demand utilizing sophisticated Long Short-Term Memory (LSTM) configurations with Attention mechanisms (Att), Grid search, and Particle Swarm Optimization (PSO). In addition, the study also examines the influence of Min-Max and Z-Score normalization approaches in the preprocessing stage on the accuracy performances of the baselines and the proposed models. PSO and Grid Search techniques are used to select the best hyperparameters for LSTM models, while the attention mechanism selects the important input for the LSTM. The research compares the performance of baselines (LSTM, Grid-search-LSTM, and PSO-LSTM) and proposes models (Att-LSTM, Att-Grid-search-LSTM, and Att-PSO-LSTM) based on MAPE, RMSE, and R2 metrics into two scenarios normalization: Min-Max, and ZScore. The results show that all models with Min-Max normalization have better MAPE, RMSE, and R2 than those with Z-Score. The best model performance is shown in Att-PSO-LSTM MAPE 3.1135, RMSE 0.0551, and R2 0.9233, followed by Att-Grid-search-LSTM, Att-LSTM, PSO-LSTM, Grid-search-LSTM, and LSTM. These findings emphasize the effectiveness of attention mechanisms in improving model predictions and the influence of normalization methods on model performance. This study's novel approach provides valuable insights into time series forecasting in energy demands.

Item Type:	Artikel Umum
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering
Divisi / Prodi:	Faculty of Industrial Technology (Fakultas Teknologi Industri) > S1-Electrical Engineering (S1-Teknik Elektro)
Depositing User:	M.Eng. Alfian Ma'arif
Date Deposited:	21 Apr 2025 01:36
Last Modified:	21 Apr 2025 01:36
URI:	http://eprints.uad.ac.id/id/eprint/83066

Actions (login required)

View Item