Summary Soil fertility influences plant community structure, yet few studies have focused on how this influence is affected by the type of mycorrhizal association formed by tree species within local communities. We examined the relationship of aboveground biomass ( AGB ) and diversity of adult trees with soil fertility (nitrogen, phosphorus, organic matter, etc.) in the context of different spatial distributions of arbuscular mycorrhizal ( AM ) and ectomycorrhizal ( EM ) trees in a temperate forest in Northeast China. Diversity showed a positive trend along the soil fertility gradient driven mostly by a positive relationship between AM tree abundance and soil fertility. By contrast, the AGB showed a negative trend along the soil fertility gradient driven mostly by a negative relationship between EM tree AGB and soil fertility. Furthermore, the opposite trend in the AGB and tree species diversity along the soil fertility gradient led to an overall negative diversity–biomass relationship at the 50‐m scale but not the 20‐m scale. These results suggest that tree mycorrhizal associations play a critical role in driving forest community structure along soil fertility gradients and highlight the importance of tree mycorrhizal associations in influencing how the diversity–ecosystem function (e.g. biomass) relationships change with soil fertility.