Quantum frequency combs from chip-scale integrated sources are promising candidates for scalable and robust quantum information processing (QIP).However, to use these quantum combs for frequency domain QIP, demonstration of entanglement in the frequency basis, showing that the entangled photons are in a coherent superposition of multiple frequency bins, is required.We present a verification of qubit and qutrit frequency-bin entanglement using an on-chip quantum frequency comb with 40 mode pairs, through a two-photon interference measurement that is based on electro-optic phase modulation.Our demonstrations provide an important contribution in establishing integrated optical microresonators as a source for high-dimensional frequency-bin encoded quantum computing, as well as dense quantum key distribution.