University of Kashan Mathematics Interdisciplinary Research 2538-3639 9 3 2024 09 01 Modified‎ ‎Step‎ ‎Size‎ ‎for‎ ‎Enhanced‎ ‎Stochastic Gradient Descent‎: ‎Convergence and Experiments 237 253 114474 10.22052/mir.2023.253279.1426 EN Mahsa Soheil Shamaee ‎Department of Computer Science, ‎Faculty of Mathematical Science‎, ‎University of Kashan‎, ‎Kashan‎, ‎Iran Sajad Fathi Hafshejani ‎Department of Applied Mathematics, ‎Shiraz University of Technology‎,‎ ‎Shiraz‎, ‎I‎. ‎R‎. ‎Iran‎ Journal Article 2023 07 18 ‎This paper introduces a novel approach to enhance the performance of the stochastic gradient descent (SGD) algorithm by incorporating a modified decay step size based on $\frac{1}{\sqrt{t}}$‎. ‎The proposed step size integrates a logarithmic term‎, ‎leading to the selection of smaller values in the final iterations‎. ‎Our analysis establishes a convergence rate of $O(\frac{\ln T}{\sqrt{T}})$ for smooth non-convex functions without the Polyak-Łojasiewicz condition‎. ‎To evaluate the effectiveness of our approach‎, ‎we conducted numerical experiments on image classification tasks using the Fashion-MNIST and CIFAR10 datasets‎, ‎and the results demonstrate significant improvements in accuracy‎, ‎with enhancements of $0.5\%$ and $1.4\%$ observed‎, ‎respectively‎, ‎compared to the traditional $\frac{1}{\sqrt{t}}$ step size‎. ‎The source code can be found at  https://github.com/Shamaeem/LNSQRTStepSize. https://mir.kashanu.ac.ir/article_114474_dde1b8da8db0067c294448415069d8b8.pdf