Graduation Term
Spring 2025
Degree Name
Master of Science (MS)
Department
Department of Mathematics
Committee Chair
Gaywalee Yamskulna
Committee Co-Chair
Mehdi Karimi
Abstract
Gradient descent is a popular optimization method that utilizes a model’s prediction error to iteratively improve its parameters for a given task. The functions that measure this error can be defined to align with the user’s goals and sometimes satisfy metric or norm properties. It is common for these functions to measure over Rn, but any differentiable space allows for gradient descent to occur. There has been some research investigating the influence of topological spaces on optimization methods, but it is a limited field of study. This thesis further explores this phenomenon by applying a transformation prediction model to multiple standard benchmark image datasets. The transformations applied are ordinary rotations, which form well-studied groups, namely S1 and S1 × S1. For each dataset, three structurally identical models with the same task are trained using distinct loss functions, which are squared Euclidean distance over the embedding space, squared Frobenius norm over the group’s topological manifold, and squared Euclidean distance in the tangent vector space of the group. After running the experiments, it was found that incorporating a topological manifold structure on the target values improves predictive accuracy. Generalizing this result into higher dimensions requires further examination, possibly utilizing more complicated data.
Access Type
Thesis-Open Access
Recommended Citation
Skudnig, Robert B. Jr., "The Impact of Loss Function Topology on Gradient Descent" (2025). Theses and Dissertations. 2087.
https://ir.library.illinoisstate.edu/etd/2087
Included in
Algebra Commons, Artificial Intelligence and Robotics Commons, Geometry and Topology Commons
