The first measurements of the Fourier coefficients ($V_{n\Delta}$) of the azimuthal distributions of charged hadrons emitted from photon-proton ($\gamma$p) interactions at the LHC are presented. The data are extracted from 68.8 nb$^{-1}$ of ultra-peripheral proton-lead (pPb) collisions at $\sqrt{s_\mathrm{NN}}$ = 8.16 TeV using the CMS detector. The high energy lead ions produce a flux of photons that can interact with the oncoming proton. This $\gamma$p system provides a set of unique initial conditions with multiplicity lower than in photon-lead collisions but comparable to recent electron-positron and electron-proton data. The $V_{n\Delta}$ coefficients are presented in ranges of event multiplicity and transverse momentum ($p_\mathrm{T}$) and are compared to corresponding hadronic minimum bias pPb results. For a given multiplicity range, the mean $p_\mathrm{T}$ of charged particles is smaller in $\gamma$p than in pPb collisions. For both the $\gamma$p and pPb samples, $V_{1\Delta}$ is negative, $V_{2\Delta}$ is positive, and $V_{3\Delta}$ consistent with 0. For each multiplicity and $p_\mathrm{T}$ range, $V_{2\Delta}$ is larger for $\gamma$p events. The $\gamma$p data are consistent with model predictions that have no collective effects.