I love this question because I have always wondered the same thing myself! Yawning is by far the most common behavior I have observed while scanning babies.
This activity, and the frequency with which it occurs, has always fascinated me as I have always associated the act of yawning with a lack of oxygen and a means for the body to rid itself of excess carbon dioxide. Clearly this is not the case since babies in utero do not breathe air. So why then, do babies yawn?
Surprisingly, very little research has been done regarding fetal yawning in utero. Several studies have been conducted and published on why people yawn and the common consensus seems to be that yawning occurs when brain temperature rises and therefore the act of yawning helps to cool the brain and help it to work more efficiently. Since sleep deprivation and exhaustion increase deep brain temperatures that explains why new parents yawn a lot!!!
Since there are no clear cut answers as to why a fetus yawns, it is my thought that perhaps like us, they may do it to regulate their body temperature. It could therefore be considered an indication of healthy brain development and that would explain the frequency with which we see this behavior. We already know babies practice and develop other life skills in the womb (swallowing, stretching, etc.) so why not the yawn?