We investigate, within the framework of linear elasticity theory, edge Rayleigh waves of a two-dimensional elastic solid with broken time-reversal and parity symmetries due to a Berry term. As our prime example, we study the elastic edge wave traveling along the boundary of a two-dimensional skyrmion lattice hosted inside a thin-film chiral magnet. We find that the direction of propagation of the Rayleigh modes is determined not only by the chirality of the thin film, but also by the Poisson ratio of the crystal. We discover three qualitatively different regions distinguished by the chirality of the low-frequency edge waves, and study their properties. To illustrate the Rayleigh edge waves in real time, we have carried out finite-difference simulations of the model. Apart from skyrmion crystals, our results are also applicable to edge waves of gyroelastic media and screened Wigner crystals in magnetic fields. Our work opens a pathway towards controlled manipulation of elastic signals along boundaries of crystals with broken time-reversal symmetry.