A key challenge for theories of language evolution is to explain why language is the way it is and how it came to be that way. It is clear that how we learn and use language is governed by genetic constraints. However, the nature of these innate constraints has been the subject of much debate. Although many accounts of language evolution have emphasized the importance of biological adaptations specific to language, we discuss evidence from computer simulations pointing to strong restrictions on such adaptations. Instead, we argue that processes of cultural evolution have been the primary factor affecting the evolution of linguistic structure, suggesting that the genetic constraints on language largely predate the emergence of language itself.