Charge-dependent anisotropy Fourier coefficients (${v}_{n}$) of particle azimuthal distributions are measured in $p\mathrm{Pb}$ and $\mathrm{PbPb}$ collisions at $\sqrt{{s}_{{}_{\mathrm{NN}}}}=5.02\phantom{\rule{0.16em}{0ex}}\mathrm{TeV}$ with the CMS detector at the LHC. The normalized difference in the second-order anisotropy coefficients (${v}_{2}$) between positively and negatively charged particles is found to depend linearly on the observed event charge asymmetry with comparable slopes for both $p\mathrm{Pb}$ and $\mathrm{PbPb}$ collisions over a wide range of charged particle multiplicity. In $\mathrm{PbPb}$, the third-order anisotropy coefficient ${v}_{3}$ shows a similar linear dependence with the same slope as seen for ${v}_{2}$. The observed similarities between the ${v}_{2}$ slopes for $p\mathrm{Pb}$ and $\mathrm{PbPb}$, as well as the similar slopes for ${v}_{2}$ and ${v}_{3}$ in $\mathrm{PbPb}$, are compatible with expectations based on local charge conservation in the decay of clusters or resonances, and constitute a challenge to the hypothesis that, at LHC energies, the observed charge asymmetry dependence of ${v}_{2}$ in heavy ion collisions arises from a chiral magnetic wave.