There is not a consensus about the benefits of implementing Real‐Time Optimization (RTO) technologies to increase the profit of process plants. A lack of experimental and theoretical works which evaluate the scope and limitations of different RTO approaches makes it more difficult to have a sensible opinion about this topic. Most works available in the open literature that study different RTO approaches use few (often one) operation conditions to draw general conclusions about the virtues of a particular methodology. In the present work, we compare the performance of the classical two‐step method with more recently proposed derivative‐based methods (modifier adaptation, Integrated System Optimization Parameter Estimation (ISOPE), and an algorithm based on the Sufficient Conditions of Feasibility and Optimality (SCFO)) under different measurement noise, model mismatch, and disturbance using a Monte Carlo methodology. The results show that the classical RTO method can be reasonably reliable if provided with a model flexible enough to mimic the local process topology, a parameter estimation method suitable for handling measurement noise characteristics, and a method to improve the sample information quality. Implementing a derivative‐based RTO method, in cases of evident model mismatch, should be considered only if the gap between the predicted and the real optimum is large enough and the level of measurement noise is low.