We present the first hybrid measurement of the average muon number in air showers at ultra-high energies, initiated by cosmic rays with zenith angles between $62^\circ$ and $80^\circ$. The measurement is based on 174 hybrid events recorded simultaneously with the Surface Detector array and the Fluorescence Detector of the Pierre Auger Observatory. The muon number for each shower is derived by scaling a simulated reference profile of the lateral muon density distribution at the ground until it fits the data. A $10^{19}$~eV shower with a zenith angle of $67^\circ$, which arrives at the Surface Detector array at an altitude of 1450 m above sea level, contains on average $(2.68 \pm 0.04 \pm 0.48\,\mathrm{(sys.)}) \times 10^{7}$ muons with energies larger than 0.3 GeV. The logarithmic gain $\mathrm{d}\ln{N_\mu} / \mathrm{d}\ln{E}$ of muons with increasing energy between $4\times 10^{18}$ eV and $5 \times 10^{19}$ eV is measured to be $(1.029\, \pm\, 0.024\, \pm 0.030\,\mathrm{(sys.)})$.