Automated machine learning hyperparameters tuning through meta-guided Bayesian optimization

The selection of one or more optimized Machine Learning (ML) algorithms and the configuration of significant hyperparameters are among the crucial but challenging tasks for the advanced data analytics using ML methodologies. However, it is one of the essential tasks in order to apply the ML-based solutions to deal with the real-world problems. In this regard, Bayesian Optimization (BO) is a popular method for optimizing black-box functions. But, yet it is deficient for large-scale problems because it fails to leverage the knowledge from historical applications. The major challenge in this aspect is due to the waste of function evaluations on bad design choices of ML hyperparameters. To address this issue, we propose to integrate Bayesian Optimization via Meta-Guidance. Consequently, Meta-Guided Bayesian Optimization provides means to use the knowledge from previous optimization cycles on similar tasks. This capability takes the form of pre-requisite to decide the specific parts of the input space to be evaluated next. In this regard, we intend to guide the BO with a functional ANOVA of configurations as suggested by a meta-learning process. In this paper, we demonstrate, with the help of a large collection of hyperparameters optimization benchmark, that the proposed Meta-Guided Optimization approach is about 3 times faster than the vanilla BO. Thence, it achieves a new state-of-the-art performance as proved by the experiments on 09 classification datasets.