Table tennis performance depends on multiple factors such as technique, tactics and fitness. Several studies have focused on investigating different technical-tactical variables. However, research analysing the specific physical qualities of this sport is scarce, particularly in the female sex. The aim of the present study was to assess the physical fitness variables associated with individual performance in elite table tennis players according to sex. Forty-eight elite players divided into males (n = 24; 25.38 ± 4.01 years) and females (n = 24; 22.33 ± 3.83 years) participated in the study. To determine physical fitness, participants performed vertical jump, hand grip strength, ergospirometry and lateral displacement tests (reaction time, displacement time and lateral acceleration). Male players showed higher values in vertical jump, hand grip strength and maximum oxygen consumption (p < 0.001). Likewise, male players moved laterally faster (p < 0.001). On the other hand, female players had a better reaction time towards the dominant side (p < 0.01). Elite male table tennis players showed better physical fitness compared to female players. Due to the scarcity of data on elite table tennis players, these results can serve as reference values for different table tennis practitioners.