You seem not to be distinguishing between evidence and conclusive proof. Somebody's fingerprints on a murder weapon is evidence for the prosecution, but it doesn't allow us to definitvely conclude they're guilty. A runny nose is evidence for the flu, but not proof. Evidence allows us to increase our confidence, or narrow our uncertainty - as Perdition pointed out, you can have evidence for things that are incorrect. In the case of the pen on the desk, it's always possible we overlooked the novelty pen shaped that looks a bit like a paperweight, but the fact that we can't see the pen is still evidence for its absence. 'Desks with pens on' represent a smaller proportion of the set 'desks on which we can't see pens' than they do of the set of all desks. We have narrowed our uncertainty, so we have evidence that the pen isn't there - even if we're wrong.
Also, there are different kinds of absence of evidence. In your example of being blindfolded and not looking at the desk, you haven't got evidence of anything, certainly. But people don't throw the phrase about only in such circumstances. Usually, people have looked. We may not have the means at our disposal to rule out all possibilities, but if we can think of the evidence an entity or occurence would leave, look for it, and don't find it, we have narrowed our uncertainy and provided evidence for the non-existence of non-occurence of whatever's under discussion. Again - evidence doesn't mean that we've proven it doesn't exist.