Transcription factor Runx2 controls bone development and osteoblast differentiation by regulating expression of a significant number of bone-related target genes. Here, we report that transcriptional activation and repression of the Runx2 gene via its osteoblast-specific P1 promoter (encoding mRNA for the Runx2/p57 isoform) is accompanied by selective deposition and elimination of histone marks during differentiation of mesenchymal cells to the osteogenic and myoblastic lineages. These epigenetic profiles are mediated by key components of the Trithorax/COMPASS-like and Polycomb group complexes together with histone arginine methylases like PRMT5 and lysine demethylases like JARID1B/KDM5B. Importantly, knockdown of the H3K4me2/3 demethylase JARID1B, but not of the demethylases UTX and NO66, prevents repression of the Runx2 P1 promoter during myogenic differentiation of mesenchymal cells. The epigenetically forced expression of Runx2/p57 and osteocalcin, a classical bone-related target gene, under myoblastic-differentiation is accompanied by enrichment of the H3K4me3 and H3K27ac marks at the Runx2 P1 promoter region. Our results identify JARID1B as a key component of a potent epigenetic switch that controls mesenchymal cell fate into myogenic and osteogenic lineages.