Lens-based water-window X-ray microscopy allows two- and three-dimensional (2D and 3D) imaging of intact unstained cells in their near-native state with unprecedented contrast and resolution. Cryofixation is essential to avoid radiation damage to the sample. Present cryo X-ray microscopes rely on synchrotron radiation sources, thereby limiting the accessibility for a wider community of biologists. In the present paper we demonstrate water-window cryo X-ray microscopy with a laboratory-source-based arrangement. The microscope relies on a lambda = 2.48-nm liquid-jet high-brightness laser-plasma source, normal-incidence multilayer condenser optics, 30-nm zone-plate optics, and a cryo sample chamber. We demonstrate 2D imaging of test patterns, and intact unstained yeast, protozoan parasites and mammalian cells. Overview 3D information is obtained by stereo imaging while complete 3D microscopy is provided by full tomographic reconstruction. The laboratory microscope image quality approaches that of the synchrotron microscopes, but with longer exposure times. The experimental image quality is analyzed from a numerical wave-propagation model of the imaging system and a path to reach synchrotron-like exposure times in laboratory microscopy is outlined.