Abstract While the General Health Questionnaire, 12‐item version (GHQ‐12) has been widely used in cross‐cultural comparisons, rigorous tests of the measurement equivalence of different language versions are still lacking. Thus, our study aims at investigating configural, metric and scalar invariance across the German and the Spanish version of the GHQ‐12 in two population samples. The GHQ‐12 was applied in two large‐scale population‐based samples in Germany ( N = 1,977) and Colombia ( N = 1,500). To investigate measurement equivalence, confirmatory factor analyses were conducted in both samples. In the German sample mean GHQ‐12 total scores were higher than in the Colombian sample. A one‐factor model including response bias on the negatively worded items showed superior fit in the German and the Colombian sample; thus both versions of the GHQ‐12 showed configural invariance. Factor loadings and intercepts were not equal across both samples; thus GHQ‐12 showed no metric and scalar invariance. As both versions of the GHQ‐12 did not show measurement equivalence, it is not recommendable to compare both measures and to conclude that mental distress is higher in the German sample, although we do not know if the differences are attributable to measurement problems or represent a real difference in mental distress. The study underlines the importance of measurement equivalence in cross‐cultural comparisons.