The way I would briefly explain it myself would be to say that we know how fast time is proceeding at great distances from Earth by observing the rate of the passage of time through observing the natural processes taking place there, such as the frequencies at which various gases emit or absorb light.
Another method is by observation of the revolution of binary stars around their common centre of gravity. By measuring the distance between the components (the ratio of their angular separation and their parallax) and their orbital period, one can obtain the total mass of the binary from Newton's theory of gravitation. It turns out that not only are the measured masses of binary systems of the same order of magnitude as the mass of the Sun, but the masses increase with the luminosities of the stars, as one would expect.
None of this would be true if space and time in the stellar realm were different from space and time as we experience them on Earth.