A search is conducted for both resonant and non-resonant high-mass new phenomena in dielectron and dimuon final states. The search uses 3.2 fb$^{-1}$ of proton-proton collision data, collected at $\sqrt{s} = 13$ TeV by the ATLAS experiment at the LHC in 2015. The dilepton invariant mass is used as the discriminating variable. No significant deviation from the Standard Model prediction is observed; therefore limits are set on the signal model parameters of interest at 95% credibility level. Upper limits are set on the cross-section times branching ratio for resonances decaying to dileptons, and the limits are converted into lower limits on the resonance mass, ranging between 2.74 TeV and 3.36 TeV, depending on the model. Lower limits on the $\ell\ell qq$ contact interaction scale are set between 16.7 TeV and 25.2 TeV, also depending on the model.