We establish quantitative stability results for the entropy power inequality (EPI) in arbitrary dimension. Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic Wasserstein distance. Further, if one of the densities is log-concave and the other is Gaussian, then the deficit in the EPI can be controlled in terms of the L1-Wasserstein distance. As a counterpoint, an example shows that the EPI can be unstable with respect to the quadratic Wasserstein distance even if densities are uniformly log-concave on sets of measure arbitrarily close to one. The proofs are based on optimal transportation.