A search for pair production of third-generation scalar leptoquarks and supersymmetric top quark partners, top squarks, in final states involving tau leptons and bottom quarks is presented. The search uses events from a data sample of proton-proton collisions corresponding to an integrated luminosity of 19.7 inverse femtobarns, collected with the CMS detector at the LHC with sqrt(s) = 8 TeV. The number of observed events is found to be in agreement with the expected standard model background. Third-generation scalar leptoquarks with masses below 740 GeV are excluded at 95% confidence level, assuming a 100% branching fraction for the leptoquark decay to a tau lepton and a bottom quark. In addition, this mass limit applies directly to top squarks decaying via an R-parity violating coupling lambda'[333]. The search also considers a similar signature from top squarks undergoing a chargino-mediated decay involving the R-parity violating coupling lambda'[3jk]. Each top squark decays to a tau lepton, a bottom quark, and two light quarks. Top squarks in this model with masses below 580 GeV are excluded at 95% confidence level. The constraint on the leptoquark mass is the most stringent to date, and this is the first search for top squarks decaying via lambda'[3jk].