Everything that comes to be has a cause.
So you don't like the idea that it possibly wouln't have a cause. But let's say that it turns out that it has indeed a cause. Say, "God". "God" caused the beginning of the Universe. I'm sure you would be delighted!
But what puzzles people like us the most, is why you suddenly would stop asking for further causes. What caused God? If you aren't interested in knowing, WHY aren't you? If God doesn't NEED a cause, then WHY doesn't he (after all you just argued that EVERYTHING has a cause?)? WHY would a Universe absolutely need a cause, while "God" wouldn't? Why the double standard? HOW does it help you to assume that "God" caused the Universe? Does it give you special insight? Do you now know more than the rest of us? What would it tell you about "God" himself? Are you sure it's not just another word for "Universe"? Does it even make sense to make a distinction between the words when you have ZERO additional information beyond just putting a label on whatever supposedly happened at the beginning of the Universe?
Did you ever sit back for a second and consider these things?