This paper reports a detailed study of techniques for identifying boosted, hadronically decaying $W$ bosons using 20fb$^{-1}$ of proton-proton collision data collected by the ATLAS detector at the LHC at a centre-of-mass energy $\sqrt{s} =$ 8 TeV. A range of techniques for optimising the signal jet mass resolution are combined with various jet substructure variables. The results of these studies in Monte Carlo simulations show that a simple pairwise combination of groomed jet mass and one substructure variable can provide a 50% efficiency for identifying $W$ bosons with transverse momenta larger than 200 GeV while maintaining multijet background efficiencies of 2-4% for jets with the same transverse momentum. These signal and background efficiencies are confirmed in data for a selection of tagging techniques.