Exclusive dimuon production in ultraperipheral collisions (UPC), resulting from photon-photon interactions in the strong electromagnetic fields of colliding high-energy lead nuclei, $\mathrm{PbPb}(\gamma\gamma) \rightarrow \mu^+\mu^- (\mathrm{Pb}^{(\star)}\mathrm{Pb}^{(\star)} )$, is studied using $\mathcal{L}_{\mathrm{int}} = 0.48$ nb$^{-1}$ of $\sqrt{s_\mathrm{NN}}=5.02$ TeV lead-lead collision data at the LHC with the ATLAS detector. Dimuon pairs are measured in the fiducial region $p_{\mathrm{T}\mu} > 4$ GeV, $|\eta_{\mu}| < 2.4$, invariant mass $m_{\mu\mu} > 10$ GeV, and $p_{\mathrm{T,\mu\mu}} < 2$ GeV. The primary background from single-dissociative processes is extracted from the data using a template fitting technique. Differential cross sections are presented as a function of $m_{\mu\mu}$, absolute pair rapidity ($|y_{\mu\mu}|$), scattering angle in the dimuon rest frame ($|\cos \vartheta^{\star}_{\mu\mu}|$) and the colliding photon energies. The total cross section of the UPC $\gamma \gamma \rightarrow \mu^{+}\mu^{-}$ process in the fiducial volume is measured to be $\sigma_{\mathrm{fid}}^{\mu\mu} = 34.1 \! \pm \! 0.3 \mathrm{(stat.)} \! \pm \! 0.7 \mathrm{(syst.)}$ $\mu\mathrm{b}$. Generally good agreement is found with calculations from STARlight, which incorporate the leading-order Breit-Wheeler process with no final-state effects, albeit differences between the measurements and theoretical expectations are observed. In particular, the measured cross sections at larger $|y_{\mu\mu}|$ are found to be about 10-20% larger in data than in the calculations, suggesting the presence of larger fluxes of photons in the initial state. Modification of the dimuon cross sections in the presence of forward and/or backward neutron production is also studied and is found to be associated with a harder incoming photon spectrum, consistent with expectations.