Abstract This paper proposes to evaluate learning efficiency by implementing the flipped classroom and automatic source code evaluation based on the Kirkpatrick evaluation model in students of CS1 programming course. The experimentation was conducted with 82 students from two CS1 courses; an experimental group (EG = 56) and a control group (CG = 26). Each student in each group completed 15 programming tasks. The level of knowledge of the participants acquired between the two groups is measured using the Kirkpatrick model, taking as a source a pre-test of previous knowledge, the grade assessment, the time of the activities, and a post-test of learning achieved. When comparing the submits time between the experimental and control group, it is observed that the value of the means is similar for the EG and CG; in this case, time is not a factor for comparison between the groups. However, in the grading, the value of the means is different for the EG and CG; EG students scored better than CG students. The evaluation of the Kirkpatrick model shows that the strategy implemented, on the one hand, does not improve the time of the activities; on the other hand, it improves the grades in the CS1 course.