This paper evaluates the mechanical properties and performance of the fine aggregate matrix (FAM) existing within full reclaimed asphalt pavement (RAP) asphalt mixtures, in terms of their rheological and fatigue deterioration properties. The RAP material was produced in the laboratory to control its properties and to reduce the effects of variability associated with these materials. Four FAM mixtures were analysed, including a virgin hot mix asphalt, a 100%RAP, and two mixtures containing 50% RAP in combination with virgin materials using different penetration grade virgin binders. The analysis of the deterioration properties was based on the application of a fracture model that incorporates the viscoelastic properties of the material, the quality of the adhesive bonds developed between the aggregates and the corresponding bitumen present in each mixture, and the rate at which the material dissipates energy when subjected to cycling loading. The input parameters for this model include the results obtained from dynamic mechanical analysis and surface free energy tests. In general, the results showed that the incorporation of 50% RAP content increased the stiffness of the final mixtures, as expected. However, this hardening effect did not result in mixtures with inferior fatigue performance of the FAM present in the full mixtures, at the applied strain level.