Abstract Up to 40% of genetic and rare disorders (RD) present facial dysmorphologies, and visual assessment is commonly used for clinical diagnosis. Although quantitative approaches are more objective and accurate, most current methods based on European descent populations disregard population ancestry. Here we assessed the facial phenotypes associated to Down (DS), Morquio (MS), Noonan (NS) and Neurofibromatosis type 1 (NF1) syndromes in a Latino-American population from Colombia. We recorded the coordinates of 18 landmarks in 2D images from 79 controls and 51 pediatric patients. We quantified facial differences using Euclidean Distance Matrix Analysis, and assessed the diagnostic accuracy of Face2gene, an automatic deep-learning algorithm. Individuals diagnosed with DS and MS presented severe phenotypes, with 58.2% and 65.4% of significantly different facial traits. The percentage decreased to 47.7% in NS and 11.4% in NF1. Each syndrome presented characteristic dysmorphology patterns, supporting the diagnostic potential of facial biomarkers. However, population-specific traits were detected, and the diagnostic accuracy of Face2Gene was affected by ancestry. Accuracy was high in DS, moderate in NS and NF1, but low in MS, with low facial gestalt similarity in admixed individuals. Our study underscores that facial quantitative analysis in populations with diverse Amerindian, African and European ancestry are crucial to improve diagnostic methods.