You do remember things from being a day old, you still remember to breathe, you learnt that when you took your first breath of air the moment you were born.
Memories are eletrical signals imprinted in your brain, the more you visit a memory the more you re-use this "path" the more it becomes inprinted in your brain. So things you think about often, like AGUEROOO!!, will never fade, it is why you can walk, talk, breathe etc. without even thinking about it because you've been doing it every few seconds for a very very long time.
Before you were born there is no brain for you to make memories, your heart keeps you alive but it is your brain that makes you, you. This is why I believe death to be like pre-existance.
Please don't think I'm trying to disagree with anyone else's point, it is purely personal belief, there is no way of knowing for sure, and I don't want to find out for a long time yet!