Many extensions of the Standard Model posit the existence of heavy particles with long lifetimes. In this Letter, results are presented of a search for such particles, which decay at a significant distance from their production point, using a final state containing charged hadrons and an associated muon. This analysis uses a data sample of proton-proton collisions at sqrt(s)= 7 TeV corresponding to an integrated luminosity of 4.4 fb-1 collected in 2011 by the ATLAS detector operating at the Large Hadron Collider. Results are interpreted in the context of R-parity violating supersymmetric scenarios. No events in the signal region are observed and limits are set on the production cross section for supersymmetric particles, multiplied by the square of the branching ratio for a neutralino to decay to charged hadrons and a muon, as a function of the neutralino lifetime. To allow these limits to be used in a variety of models, they are presented for a range of squark and neutralino masses.