I took mommy-baby yoga classes when my daughter was 15 months old...and I saw in class other mommies with younger babies. So I think it doesn't matter..there are different yoga exercises you both can do togheter depending of age.
I recomend this book:
Itsy Bitsy Yoga: Poses to Help Your Baby Sleep Longer, Digest Better, and Grow Stronger
You can find it here:
http://www.amazon.com/Itsy-Bitsy-Yoga-Longer-Stronger/dp/0743243552I read it and showed me a lot of exercises we can do with our babies. It was really cool!