A search for high-mass charged and neutral bosons decaying to $W\gamma$ and $Z\gamma$ final states is presented in this paper. The analysis uses a data sample of $\sqrt{s} = 13$ TeV proton-proton collisions with an integrated luminosity of 139 fb$^{-1}$ collected by the ATLAS detector during LHC Run 2 operation. The sensitivity of the search is determined using models of the production and decay of spin-1 charged bosons and spin-0/2 neutral bosons. The range of resonance masses explored extends from 1.0 TeV to 6.8 TeV. At these high resonance masses, it is beneficial to target the hadronic decays of the $W$ and $Z$ bosons because of their large branching fractions. The decay products of the high-momentum $W/Z$ bosons are strongly collimated and boosted-boson tagging techniques are employed to improve the sensitivity. No evidence of a signal above the Standard Model backgrounds is observed, and upper limits on the production cross-sections of these bosons times their branching fractions to $W\gamma$ and $Z\gamma$ are derived for various boson production models.