A $6.8 \ {\rm nb^{-1}}$ sample of $pp$ collision data collected under low-luminosity conditions at $\sqrt{s} = 7$ TeV by the ATLAS detector at the Large Hadron Collider is used to study diffractive dijet production. Events containing at least two jets with $p_\mathrm{T} > 20$ GeV are selected and analysed in terms of variables which discriminate between diffractive and non-diffractive processes. Cross sections are measured differentially in $\Delta\eta^F$, the size of the observable forward region of pseudorapidity which is devoid of hadronic activity, and in an estimator, $\tilde{\xi}$, of the fractional momentum loss of the proton assuming single diffractive dissociation ($pp \rightarrow pX$). Model comparisons indicate a dominant non-diffractive contribution up to moderately large $\Delta\eta^F$ and small $\tilde{\xi}$, with a diffractive contribution which is significant at the highest $\Delta\eta^F$ and the lowest $\tilde{\xi}$. The rapidity-gap survival probability is estimated from comparisons of the data in this latter region with predictions based on diffractive parton distribution functions.