A search for a heavy, CP-odd Higgs boson, A, decaying into a Z boson and a 125 GeV Higgs boson, h, with the ATLAS detector at the LHC is presented. The search uses proton–proton collision data at a centre-of-mass energy of 8 TeV corresponding to an integrated luminosity of 20.3 fb−1. Decays of CP-even h bosons to ττ or bb pairs with the Z boson decaying to electron or muon pairs are considered, as well as h→bb decays with the Z boson decaying to neutrinos. No evidence for the production of an A boson in these channels is found and the 95% confidence level upper limits derived for σ(gg→A)×BR(A→Zh)×BR(h→ff¯) are 0.098–0.013 pb for f=τ and 0.57–0.014 pb for f=b in a range of mA=220–1000GeV. The results are combined and interpreted in the context of two-Higgs-doublet models.