A measurement of the forward-backward asymmetry of pairs of oppositely charged leptons (dimuons and dielectrons) produced by the Drell-Yan process in proton-proton collisions is presented. The data sample corresponds to an integrated luminosity of 138 fb$^{-1}$ collected with the CMS detector at the LHC at a center-of-mass energy of 13 TeV. The asymmetry is measured as a function of lepton pair mass for masses larger than 170 GeV and compared with standard model predictions. An inclusive measurement across both channels and the full mass range yields an asymmetry of 0.612 $\pm$ 0.005 (stat) $\pm$ 0.007 (syst). As a test of lepton flavor universality, the difference between the dimuon and dielectron asymmetries is measured as well. No statistically significant deviations from standard model predictions are observed. The measurements are used to set limits on the presence of additional gauge bosons. For a Z' boson in the sequential standard model the observed (expected) 95% confidence level lower limit on the Z' mass is 4.4 (3.7) TeV.