Cadmium (Cd) is a heavy metal that poses a threat to food safety via the ingestion of food products with Cd. The uptake of Cd by the cacao tree (Theobroma cacao) has gained attention after the European Union set limits for Cd in chocolate products, the main commodity produced from cacao beans. In this study, we analyzed levels of Cd in soils and plant tissues across five cacao farms in the Piura region of Peru to identify the origins of Cd accumulation, and the natural and human factors controlling its concentration. Our results show that Cd levels varied in the order: leaves (1.25 mg kg−1) > beans (0.78 mg kg−1) > soils at 5 cm (0.68 mg kg−1) > soils at 20cm (0.6 mg kg−1). Our findings suggest that the higher concentration of Cd in plant tissues and surface soils can be explained by readily available Cd from fertilizers and a litter layer being absorbed, cycled, and accumulated by the plant. Moreover, even when fertilizers are within regulatory limits, their continuous application, combined with the biocycling of Cd, may lead to high Cd concentrations in beans and leaves. Conversely, farms on alluvial soils and more stable topographic positions display higher soil and plant Cd concentrations. Likewise, farms located at lower altitudes, with higher contributing areas are more likely to receive Cd transported through sediments and water in the river network. Our results also suggest that variations in the underlying geology and soil mineralogy may be a source of potentially Cd bearing sediments. Overall, this study indicates that the high levels of Cd in plants in the study area are the result of a combined mechanism involving plant bioaccumulation and high Cd in fertilizers for the most part, with a minor contribution from potential Cd bearing minerals in sediments of alluvial soils.