The determination of the primary energy of extensive air showers using the fluorescence detection technique requires an estimation of the energy carried away by particles that do not deposit all their energy in the atmosphere. This estimation is typically made using Monte Carlo simulations and thus depends on the assumed primary particle mass and on model predictions for neutrino and muon production. In this work we present a new method to obtain the invisible energy from events detected by the Pierre Auger Observatory. The method uses measurements of the muon number at ground level, and it allows us to significantly reduce the systematic uncertainties related to the mass composition and the high energy hadronic interaction models, and consequently to improve the estimation of the energy scale of the Pierre Auger Observatory.