Abstract Experimenters should be aware of the possibility that some of their observations may be unavailable for analysis. This article considers different criteria to assess the impact that missing data can have when running three-level designs to estimate a full second-order polynomial model. Designs for three to seven factors were studied and included Box–Behnken designs, face-centered composite designs, and designs due to Morris, Mee, Block–Mee, Draper–Lin, Hoke, Katasaounis, and Notz. These designs were studied under two existing robustness criteria: (i) the maximum number of runs that can be missing and still allow the remaining runs to estimate a given model; and (ii) the loss of D-efficiency in the remaining design compared with the original design. The robustness of three-level designs was studied using a third, new criterion: the maximum number of observations that can be missing from a design and still allow the estimation of the given model with a high probability. This criterion represents a useful generalization of the first criterion, which determines the maximum number of runs that make the probability of estimating the model equal to one. The new criterion provides a better assessment of the robustness of each design than previous criteria. Keywords: Response Surface Methodology (RSM)robustness criteriamissing datathree-level designs Acknowledgement The authors wish to thank an anonymous referee for his/her careful review and suggestions, in particular those regarding the Monte Carlo estimation of ρ(t) and t max(1 − α), which led to major changes in the article.