Back to the list

Accelerating the Nelder-Mead Method with Predictive Parallel Evaluation

Workshop Paper
Peer Reviewed

6th ICML Workshop on Automated Machine Learning (AutoML2019)

Download PDF Experimental Codes

@inproceedings{ozaki2019accelerating,

title={Accelerating the Nelder-Mead Method with Predictive Parallel Evaluation},

author={Yoshihiko, Ozaki and Shuhei, Watanabe and Masaki, Onishi},

booktitle={6th ICML Workshop on Automated Machine Learning (AutoML2019)},

year={2019}

}

Authors

  • Yoshihiko Ozaki
  • Shuhei Watanabe
  • Masaki Onishi
  • Abstract

    The Nelder–Mead (NM) method has been recently proposed for application in hyperparameter optimization (HPO) of deep neural networks. However, the NM method is not suitable for parallelization, which is a serious drawback for its practical application in HPO. In this study, we propose a novel approach to accelerate the NM method with respect to the parallel computing resources. The numerical results indicate that the proposed method is significantly faster and more efficient when compared with the previous naive approaches with respect to the HPO tabular benchmarks.