Yes, we're switching arbitrary values. Prior to 1961, physicists defined unit atomic weight as 1/16th of the mass of an oxygen-16 atom. Chemists defined it as 1/16th of the abundance-weighted average of the three oxygen isotopes present in the Earth's atmosphere. Since then it has been, by definition, 1/12 of the mass of a carbon-12 atom. (Before 1925, it was something different, but my chemistry books aren't
that old.
Avagadro's number is just the number of atoms or molecules in the mass in grams of a substance that is numerically equal to the molecular weight, so it's really quite arbitrary.
And by the way, Avagadro's number of avocados does, in fact, have the same mass as Earth's Moon for avocados of the unusually large mass of 122 grams each. (or 61 grams each if avocados are naturally diatomic....)