Background: Cigarette smoking is a known risk factor for Alzheimer’s disease (AD). However, the association between neurodegeneration and other substances has not been fully determined. It is of vital importance to evaluate this relationship in populations at high risk of dementia. Since substance use possibly modifies the progression rate of cognitive decline, we studied this association in a unique and well-phenotyped cohort from the University of Antioquia: carriers of the PSEN1-E280A genetic variant. Objective: To determine the association between substance use and cognitive decline in carriers of the PSEN1-E280A genetic variant. Methods: A retrospective cohort study was conducted with 94 carriers and 69 noncarriers recruited between January 2019 and April 2020. A psychiatrist interviewed the participants using the Consumption of Alcohol, Cigarettes and other Substances questionnaire. The participants were also submitted to cognitive evaluation. The relationship between cognitive decline and substance use was explored through a mixed effects regression model. Results: There was an association between cigarettes and better performance on tasks related to perceptual organization, verbal fluency, and memory in carriers. Alcohol had a positive or negative effect on memory according to the type of alcoholic beverage. Results on marijuana use were no conclusive. Coffee was associated with progressive improvements in executive function and verbal fluency. Conclusion: Cigarette and alcohol were associated with an improvement of some cognitive assessments, possibly by a survival bias. In addition, coffee was related to improvements in executive function and language; therefore, its short-term neuroprotective potential should be studied.