I don't necessarily disagree with what you are saying. When I read about quantum physics I realize I don't have the education to really understand what the experts are saying to the degree that the experts do. In those cases, I form my
opinion about which school of thought is correct in a particular disagreement, and if I was placing bets I'd probably go with the view that the majority of experts hold.
But, if one doesn't understand the arguments I don't think they have a right to defend one position over another based on how many experts favor this or that position. That seems too close, if not exactly, to the infamous "argument from authority" which I think should never be introduced in a debate.