Many aspects of modern applied research rely on a crucial algorithm called gradient descent. This is a procedure generally used for finding the largest or smallest values of a particular ...

The most widely used technique for finding the largest or smallest values of a math function turns out to be a fundamentally difficult computational problem. Many aspects of modern applied research ...

Neel, Seth, Aaron Leon Roth, and Saeed Sharifi-Malvajerdi. "Descent-to-Delete: Gradient-Based Methods for Machine Unlearning." Paper presented at the 32nd Algorithmic Learning Theory Conference, March ...

However, the gradient descent algorithms need to update variables one by one to calculate the loss function with each iteration, which leads to a large amount of computation and a long training time.

Unlike the metaphorical mountaineer, optimization researchers can program their gradient descent algorithms to take steps of any size. Giant leaps are tempting but also risky, as they could overshoot ...

The EM algorithm is a very popular and widely applicable algorithm for the computation of maximum likelihood estimates. Although its implementation is generally simple, the EM algorithm often exhibits ...