BEGIN:VCALENDAR
VERSION:2.0
CALSCALE:GREGORIAN
PRODID:UW-Madison-Physics-Events
BEGIN:VEVENT
SEQUENCE:1
UID:UW-Physics-Event-5933
DTSTART:20200826T160000Z
DTEND:20200826T170000Z
DTSTAMP:20260415T010425Z
LAST-MODIFIED:20200714T135608Z
LOCATION:Please register for this online event: http://physicsmeetsml.
 org
SUMMARY:The large learning rate phase of deep learning \, Physics ∩ 
 ML Seminar\, Yasaman Bahri\, Google Brain
DESCRIPTION:Recent investigations of infinitely-wide deep neural netwo
 rks have given rise to foundational connections between deep nets\, ke
 rnels\, and Gaussian processes. Nonetheless\, there is still a gap to 
 characterizing the dynamics of finite-width neural networks in common 
 optimization settings. I’ll discuss how the choice of learning rate 
 is a crucial factor to be considered and naturally classifies gradient
  descent dynamics of deep nets into two classes (a ‘lazy’ regime a
 nd a ‘catapult’ regime) which are separated by a sharp transition 
 as networks become wider. I’ll discuss the distinct phenomenological
  signatures of the two phases and how they are elucidated in a class o
 f solvable simple models we analyze.
URL:https://www.physics.wisc.edu/events/?id=5933
END:VEVENT
END:VCALENDAR