VISTA: A Variable Length Genetic Algorithm and LSTM-Based Surrogate Assisted Ensemble Selection algorithm in Multiple Layers Ensemble System

Han, Kate; Thanh Nguyen, Truong; Anh Vu, Viet; Wee-Chung Liew, Alan; Dang, Truong; Thanh Nguyen, Tien

Abstract

We proposed a novel ensemble selection method called VISTA for multiple layers ensemble systems (MLES). Our ensemble model consists of multiple layers of ensemble of classifiers (EoC) in which the EoC in each layer is trained on the data generated by a concatenation of the original training data and the predictions by classifiers of the previous layer. The predictions of the EoC in the final layer are aggregated to obtain the final prediction. To enhance the accuracy of the MLES, we used the Variable-Length Genetic Algorithm (VLGA) to search for the optimal configuration of EoC in each layer. Since the optimisation process is computationally intensive, we use Surrogate-Assisted Evolutionary Algorithms (SAEA) to reduce the training time. Most surrogate models developed in the literature require a fixed-length input, which limits their applications when the encoding is of variable length. In this paper, we proposed to use a Long Short-Term Memory (LSTM)-based surrogate model, in which the LSTM transforms the variable-length encoding to a fixed-size representation which will then be used by the surrogate model to predict the fitness values in VLGA. For the surrogate model, we adopted Radial Basis Function (RBF) for surrogation. We first conducted experiments in comparing two types of LSTM converters, and the results suggest that the proposed chunk-based LSTM converter provides better results compared to the normal LSTM converter. Our experiments on 15 datasets show that VISTA outperforms several benchmark algorithms.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2024 IEEE Congress on Evolutionary Computation (CEC)
• Start Date: Jun 30, 2024
• End Date: Jul 5, 2024
• Acceptance Date: Jun 28, 2024
• Publication Date: Jun 30, 2024
• Deposit Date: Jan 7, 2025
• Publisher: Institute of Electrical and Electronics Engineers
• Volume: 15
• Pages: 1-9
• Book Title: Congress on Evolutionary Computation
• ISBN: 9798350308372
• DOI: https://doi.org/10.1109/cec60901.2024.10612029