Guided Stochastic Gradient Descent Algorithm for inconsistent datasets

Sharma, Anuraganand (2018) Guided Stochastic Gradient Descent Algorithm for inconsistent datasets. Applied Soft Computing, 73 . 1068 - 1080. ISSN 1568-4946

Preview

PDF Download (2MB) | Preview

Abstract

Stochastic Gradient Descent (SGD) Algorithm, despite its simplicity, is considered an effective and default standard optimization algorithm for machine learning classification models such as neural networks and logistic regression. However, SGD's gradient descent is biased towards the random selection of a data instance. In this paper, it has been termed as data inconsistency. The proposed variation of SGD, Guided Stochastic Gradient Descent (GSGD) Algorithm, tries to overcome this inconsistency in a given dataset through greedy selection of consistent data instances for gradient descent. The empirical test results show the efficacy of the method. Moreover, GSGD has also been incorporated and tested with other popular variations of SGD, such as Adam, Adagrad and Momentum. The guided search with GSGD achieves better convergence and classification accuracy in a limited time budget than its original counterpart of canonical and other variation of SGD. Additionally, it maintains the same efficiency when experimented on medical benchmark datasets with logistic regression for classification.

Item Type:	Journal Article
Uncontrolled Keywords:	Stochastic Gradient Descent Algorithm, Machine learning, Classification, Logistic regression, Neural networks, Greedy selection, Guided Stochastic Gradient Descent Algorithm
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Divisions:	Faculty of Science, Technology and Environment (FSTE) > School of Computing, Information and Mathematical Sciences
Depositing User:	Anuraganand Sharma
Date Deposited:	17 Jul 2019 22:36
Last Modified:	17 Jul 2019 22:36
URI:	http://repository.usp.ac.fj/id/eprint/11610

UNSPECIFIED

Actions (login required)

View Item