Yes, I don't think babies have so much pressure (if any!) that they NEED to meditate.
They definitely don't NEED to meditate. The question, however, would be whether learning to meditate would be good for them, and on this, I believe it would be a good habit to foster, especially from young.
Having said that, I too agree that it's very difficult to get them to meditate meaningfully at such young ages, as we all know they have an ultra-short attention span and it's difficult to get them to concentrate and sit still for even 1 minute, let alone more 5 minutes or 15 minutes or more!
So yes, yoga IS the best activity to do with your child at early ages. What I would suggest would be some very short 'meditation' (if you can even call it that) sessions at the end of these yoga sessions, whether it's chanting a few OM/AUMs, focusing on the breath, or whatever method you prefer.