The development of robust generative models for highly varied non-stationary time-series data is a complex and important problem. Traditional models for time-series data prediction, such as Long Short-Term Memory (LSTM), are inefficient and generalize poorly as they cannot capture complex temporal relationships. In this paper, we present a probabilistic generative model that can be trained to capture complex temporal information, and that is robust to data errors. We call it Time Deep Latent Gaussian Model (tDLGM). Its novel architecture is an extension of the popular Deep Latent Gaussian Model (DLGM). Our model is trained to minimize a novel regularized version of the free energy loss function (an upper bound for the negative log loss). Our regularizer, which accounts for data trends, facilitates robustness to data errors that arise from additive noise. Experiments conducted show that tDLGM is able to reconstruct and generate complex time-series data. Further, the prediction error does not increase in the presence of additive Gaussian noise.