Georgia Institute of Technology College of Sciences

In this presentation, I will discuss my research in the field of data science, specifically in two areas: improving autoencoder interpolations and accelerating federated learning algorithms. My work combines advanced mathematical concepts with practical machine learning applications, contributing to both the theoretical and applied aspects of data science. The first part of my talk focuses on image sequence interpolation using autoencoders, which are essential tools in generative modeling. The focus is when there is only limited training data. By introducing a novel regularization term based on dynamic optimal transport to the loss function of autoencoder, my method can generate more robust and semantically coherent interpolation results. Additionally, the trained autoencoder can be used to generate barycenters. However, computation efficiency is a bottleneck of our method, and we are working on improving it. The second part of my presentation focuses on accelerating federated learning (FL) through the application of Anderson Acceleration. Our method achieves the same level of convergence performance as state-of-the-art second-order methods like GIANT by reweighting the local points and their gradients. However, our method only requires first-order information, making it a more practical and efficient choice for large-scale and complex training problems. Furthermore, our method is theoretically guaranteed to converge to the global minimizer with a linear rate.

We consider a numerical method to approximate the solution operator for evolutional partial differential equations (PDEs). By employing a general reduced-order model, such as a deep neural network, we connect the evolution of a model's parameters with trajectories in a corresponding function space. Using the Neural Ordinary Differential Equations (NODE) technique we learn a vector field over the parameter space such that from any initial starting point, the resulting trajectory solves the evolutional PDE. Numerical results are presented for a number of high-dimensional problems where traditional methods fail due to the curse of dimensionality.

We'll talk about problems of optimizing a quadratic function subject to quadratic constraints, in addition to a sparsity constraint that requires that solutions have only a few nonzero entries. Such problems include sparse versions of linear regression and principal components analysis. We'll see that this problem can be formulated as a convex conical optimization problem over a sparse version of the positive semidefinite cone, and then see how we can approximate such problems using ideas arising from the study of hyperbolic polynomials. We'll also describe a fast algorithm for such problems, which performs well in practical situations.

We review some theoretical and computational results on locating eigenvalues coalescence for matrices smoothly depending on parameters. Focus is on the symmetric 2 parameter case, and Hermitian 3 parameter case. Full and banded matrices are of interest.