I agree with your assessment of the probability of life on other planets. As the highly missed Carl Sagan said (paraphrased), "If there's no one else out there, it's an awful waste of space."
Now, while I agree there are probably other life forms out there, the probability of
intelligent life is probably less than any life whatsoever. I think there's probably some out there, but the odds of intelligence occurring close to other intelligence seems to be very low, and until we show that Einstein's speed limit is not really a limit, distance and time become extremely limiting factors. As such, it seems highly unlikely that we have been visited by anyone.
I think the preponderance of "alien abduction" stories comes from the same place as "demon" and "witchcraft" stories. A healthy imagination, fear, and a liberal sprinkling of sleep paralysis. I've experienced it once, and I can definitely see how people would believe the things they imagined had actually happened. Luckily, I had heard of the phenomenon, and upon waking, I was able to realize what had happened. But let me tell you, it was one of the freakiest things I've ever experienced.