Current hot-mix asphalt (HMA) mixture design includes general criteria to define the amount and type of natural filler (or filler) leading to proper mixture performance and response. In this context, this research evaluated the feasibility of optimising the effect of natural fillers on the response of asphalt-aggregate interfaces of HMA mixtures by improving the selection of both the type and amount of filler to be used. The assessment, based on energy parameters derived from surface free energy measurements of the constituent materials, included evaluation of materials combinations from seven natural fillers, three asphalts, and six aggregates. Results suggest that optimisation of both the amount and type of filler is possible based on the energy parameters assessed. Namely, the material combinations analysed led to recommend an optimum range of filler volumetric concentrations between 0.13 and 0.24 to simultaneously optimise all the energy parameters for resistance to fracture and moisture damage, as well as the mastic wettability over aggregate. Additional research is suggested to relate the responses reported here and HMA mixture-performance and -response as well as to analyse the effect of asphalt aging on both the energy parameters quantified and recommended optimum range of filler volumetric concentration.