Hello, happy_atheist (although, according to some you are not really an atheist, nor really happy.)
The usual analogy is a globe, and the north-south direction representing time, and the north pole being "time zero". Since there is no earth north of the north pole, nor south of the south pole, then all existing latitudes represent a part of the earth, just like all possible time coordinates represent a time in the universe, ie, the universe has always existed. Kind of a tautology, if one properly defines "always existing", but one that I like.
But what I think Crashfrog was referring to was that our present understanding of the laws of physics allow us to only get within some minute fraction of a second after the supposed Big Bang event, and that it is an assumption that we can extrapolate the rate of expansion backwards to the initial singularity. But it may be possible that a newer understanding of the elementary principles of physics will show that this is an unwarranted assumption -- perhaps the univers was never a singularity, or maybe our portion of the universe was a singularity in a larger univers that has always existed.
™ Version 4.2