Based on the connection between the spectral form factor and the probability to return, the origin of the $1/f^\alpha$-noise in fully chaotic and fully integrable systems is traced to the quantum interference between invariant manifolds of the classical dynamics and the dimensionality of those invariant manifolds. This connection and the order-to-chaos transition are analyzed in terms of the statistics of Floquet's quasienergies of a classically chaotic driving non-linear system. An immediate prediction of the connection established here is that in the presence of decoherence, the spectral exponent $\alpha$ takes the same value, $\alpha=2$, for both, fully chaotic and fully integrable systems.