Subject: activation, regularization, deep neural networks, atomic simulations
Year: 2022
Type: Proceedings
Title: Activation functions’ impact on regularization of deep neural networks application in atomic simulations
Author: Sandjakoska, Ljubinka
Author: Madevska Bogdanova, Ana
Abstract: When it comes to atomic simulations, the regularization of the deep neural networks is key to its successful application. The generalization capability of deep network depends on some factors. This paper aims to show that activation function is one of the most important factors that influence to decreasing the generalization error. For that purpose, several experiments were performed. Moreover, new approach for choosing the activation function is proposed. The purpose of the activation mechanism is not to find a universal, new activation function, although this is not excluded, but the most appropriate for the given task and for the given data set. The obtained results show that using the proposed activation approach, a decreasing of the mean absolute error compared to the benchmark set is achieved.
Publisher:
Relation: The 19th International Conference on Informatics and Information Technologies – CIIT 2022
Identifier: oai:repository.ukim.mk:20.500.12188/25693
Identifier: http://hdl.handle.net/20.500.12188/25693
