As soon as you start talking to your child, which is newborn, start using the basic math terminology.
I am a front believer that you need to treat your baby like an adult. Remember that they are tabula rasa at this gentle time and absorb everything like sponges, literally. So all those "goo-goos" and "moo-moos" will not enrich your baby's vocabulary or assist in adjustment to the new and unfamiliar environment. Use the real words. Don't introduce your baby to a useless baby-talk. I know it's kind of cute but here we are not trying to be. When it comes to parenting we are all business. Or almost always...;)
So, Moms and Dads, why don't we use simple numerals along with simple names?
When you play with your baby(special note for new moms: I mean, after you got over the shock of having a baby and figured out how to take care of the basic baby's needs ) instead of saying "This is a bunny. This is a teddy-bear" say that and add to it "This is one bunny. And this is one teddy-bear". A little later you can add to all of the above "They are friends. There are two of them". And that was the first math concept introduction: "one/two(more than one)". You got the idea.
After you child grasped this math concept you can introduce other more complicated ones.
I watched this video on YouTube today. It is very basic but that's what you need for babies and toddlers: simple, clear and preferably in the form of a game not to bore them too much.
I like the early concepts of sorting and matching by color. Those are the groundwork for the following success in mathematics. Children love bright colors! Use it to your advantage( and their too, they just don't know it yet). Stock up on multiple toys in different colors and go through the sorting and matching games with your little one. Any kinds of blocks are great. There is not enough good things I can tell you about blocks. Ultimate math-skill builders!
In the video DVDs are introduced as well. I didn't let my child watch them till she turned 2 just like pediatricians recommend and after that they had no educational value to her. By the time she turned two she already knew her ABC's, 123s, shapes and colors.They are well-made though and my daughter did get a kick out of crawling around babies. She also enjoyed the parts with puppets. So it's completely your call to use DVDs or not.
Make your children love math early! They will thank you later.