Abstract: I will show how crystalline inversion symmetry can be dynamically broken by optical phonons with generic, hardening Kerr-like non-linearities. The symmetry-broken state is reached through a dynamical phase transition that can be accessed by driving close to half the phonon resonance. After going through an instability, the system settles to a steady state with inversion-symmetry breaking phonon trajectories and strong second harmonic generation. The time averaged positions of the atoms are displaced relative to equilibrium, resulting in a rectification of the driving signal. This leads to a dynamical dipolar, ferroelectric response that shows features like hysteresis and memory effects, known from equilibrium ferroelectrics.