Thanks for relating your experiences in such depth. It was very interesting.
With regard to your final question about failing to understand why generations of intelligent men could continue to pretend to uphold a belief in something they surely couldn't buy into, I know Daniel Dennet and Linda Lascola have been doing some work on trying to understand this phenomenon by studying priests who are in this position. I haven't had time to check out their work recently, but here is a link via Richard Dawkins' website:
Page not found | Richard Dawkins Foundation for Reason and Science