We present a study of μμμμ, eeee, and μμee events using 1 fb−1 of data collected with the D0 detector at the Fermilab Tevatron pp¯ Collider at s=1.96 TeV. Requiring the lepton pair masses to be greater than 30 GeV, we observe one event, consistent with the expected background of 0.13±0.03 events and with the predicted standard model ZZ and Zγ* production of 1.71±0.15 events. We set an upper limit on the ZZ and Zγ* cross section of 4.4 pb at the 95% C.L. We also derive limits on anomalous neutral trilinear ZZZ and ZZγ* gauge couplings. The one-parameter 95% C.L. coupling limits with a form-factor scale Λ=1.2 TeV are −0.28<f40Z<0.28, −0.31<f50Z<0.29, −0.26<f40γ<0.26, and −0.30<f50γ<0.28.Received 5 December 2007DOI:https://doi.org/10.1103/PhysRevLett.100.131801©2008 American Physical Society