No, an atheist would not say that there was not a coin in your hand. We know that coins exist and that they can be held in the hand. SO the statement that you have a coin in your hand is too plausible to be dismissed just because I do not know whether there is or not.
Now we don't know that any Gods exist, and in fact we can be pretty sure that many Gods that people beleive or used to believe in don't exist (even if we can't be sure which ones). And come to that Gods are pretty unlikely sorts of beings - highly ordered as well as radically different from anything that has been shown to exist.
So the lack of evidence as well as the admittedly weak negative evidence is sufficient to at least justify taking the view that Gods don't exist even if we can't say that it is more than a tentative opinion.
Speaking personally I am an atheist in that I hold the opinion that there are no Gods (unless or until evidence arrives to make me reassess that view) and an agnostic in the sense that I do not claim to know that there are no Gods (and that is at least closer to the original sense of the word than the way it is often used).
I don't propose to get into the difference between agnosticism and atheism beyond that other to say that there are different definitions of agnosticism and atheism and some include agnosticism as a subset of atheism while others don't. The definition I use above can include atheists and theists rather than being a distinct position somehow inbetween.