Growing evidence suggests that rhythmic physical activity (PA) improves cognitive function in both persons with normal brain aging and with cognitive impairment. This study aims to conduct a systematic review of randomized controlled trials assessing the effects of rhythmic PA over global cognition in older adults with and without mild cognitive impairment. Different keywords related to the topic and Boolean operators were used in the Web of Science, PubMed, and Scopus databases. A total of 11 articles that met the inclusion criteria were analyzed; all of them assessed global cognition using either the Mini-Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA) or the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). Five studies showed beneficial effects over global cognition. All studies had at least one experimental group with rhythmic training, and the interventions evidenced a great diversity of rhythmic stimuli, as well as a varied frequency, duration and type of activities. The heterogeneity of the protocols could be the reason for the mixed findings. Future studies with more precise exercise prescriptions are needed to establish whether rhythmic PA has beneficial effects on global cognition.