Objectives: Subjective memory complaints (SMCs) are among the key concerns in the elderly, but their role in detecting objective cognitive problems is unclear. The aim of this study was to clarify the association between SMCs (both prospective and retrospective memory complaints) and neuropsychological test performance in older adults at risk of cognitive decline. Methods: This investigation is part of the FINGER project, a multicenter randomized controlled trial aiming at preventing cognitive decline in high-risk individuals. The cognitive assessment of participants was conducted at baseline using a modified neuropsychological test battery (NTB). SMCs were evaluated with the Prospective and Retrospective Memory Questionnaire (PRMQ) in a sub-sample of 560 participants (mean age, 69.9 years). Results: Having more prospective SMCs was associated with slower processing speed, but not with other NTB domains. Retrospective SMCs were linked to poorer function on NTB total score, processing speed, and memory. Executive function domain was not associated with any PRMQ ratings. Depressive symptoms and poor quality of life diluted the observed associations for NTB total score and memory. However, the association between PRMQ and processing speed remained even after full adjustments. Conclusions: Our results indicate that self-reported memory problems, measured with PRMQ, are associated with objectively measured cognitive performance. Such complaints in healthy elderly people also seem to reflect reduced mental tempo, rather than memory deficits. Slowing of processing speed may thus be negatively related to memory self-efficacy. It is also important to consider affective factors among those who report memory problems.