Spherical harmonic moments are well-suited for capturing anisotropy at any scale in the flux of cosmic rays. An unambiguous measurement of the full set of spherical harmonic coefficients requires full-sky coverage. This can be achieved by combining data from observatories located in both the northern and southern hemispheres. To this end, a joint analysis using data recorded at the Telescope Array and the Pierre Auger Observatory above $10^{19}$ eV is presented in this work. The resulting multipolar expansion of the flux of cosmic rays allows us to perform a series of anisotropy searches, and in particular to report on the angular power spectrum of cosmic rays above $10^{19}$ eV. No significant deviation from isotropic expectations is found throughout the analyses performed. Upper limits on the amplitudes of the dipole and quadrupole moments are derived as a function of the direction in the sky, varying between 7% and 13% for the dipole and between 7% and 10% for a symmetric quadrupole.