The agreement between the different radiometric dating methods is the issue. They are based on very different physical processes and yet they all agree. And are also not in disagreement with any other ways of estimating the time that has passed.
If I had an atomic clock, a pendulum clock with hanging weights, a modern quartz watch and a wind up wrist watch which all agree on the amount of time that has passed then it would be a bit crazy to decide that the time interval is not correct and that somehow they are all running slow (or fast) to the same degree.
However, I also understand that there is very little radiometric evidence on earth of a 4+ billion year formation. It is also the case that material on earth arrived from outside of the solar system and thus at least conceivable might predate the formation of the earth by a large margin. No changes in decay rates would be required.
I understand better now (or should have read more carefully before). You are correct that we haven't found anything on the planet that is original, unaltered material from it's initial formation.
However, everything we have that is clearly younger than the earth points back to an origin of 4+ billion years. In addition there is good reason to expect the moon to be close to the same age as the earth but possibly younger and it dates to the same time.
In addition, because of the nature of the materials in the meteorites, there is good reason to expect them or at least most of them to originate at the time of the formation of the solar system too and they date in the 4+ billion year time as well.
You are right that there is some mixing in of the idea of how the solar system formed mixed into the thinking here but if that is ignored that there is still have another form of consillience to explain-- that of the agreement of measurements made of the different samples -- earth based material, meteorites and the moon samples.