Multi-Objective Hyperparameter Search via Damped Gauss--Newton Optimization

תקציר מקורי באנגליתarXiv:2401.03580v2 Announce Type: replace-cross Abstract: We study hyperparameter optimization (HPO) from a numerical-optimization perspective and propose a multi-objective, damped Gauss--Newton search method. Rather than treating model evaluations as independent trials, the method estimates a finite-difference Jacobian that captures the local sensitivity of multiple validation metrics to hyperparameter perturbations. A Tikhonov-regularized Gauss--Newton system then produces a directed joint update, addressing the underdetermined setting in which the number of hyperparameters exceeds the number of performance objectives. We evaluate the method on three public classification datasets by tuning four XGBoost hyperparameters and compare it with exhaustive grid search, random search, and tree-s