That is, we treat the foundations of arithmetic as axioms even though they're not.
Strangely enough they are axioms. In line with what Percy said there are several different meanings of the word axiom. In mathematics an axiom is simply a definition. It explains the nature of the mathematical object you are talking about.
One can formulate arithmetic independent of
formal set theory with Peano's axioms. These axioms describe an object called
the naturals obeying certain conditions, elements of which are called numbers. It can easily be proven that these numbers produce all of the results of school mathematics.
However if one takes Zermelo-Fraenkel-Choice set theory, then these axioms describe a universe of sets obeying certain conditions. It is an easy matter to prove that there is a set inside this universe which obeys Peano's axioms and hence all of school mathematics is produced by ZFC set theory.
However both ZFC or Peano's axioms are axiomatic systems. Peano's axioms really are axioms. ZFC is more powerful in the sense that it contains a system which obeys Peano's axioms, as well as other systems (such as the real numbers and other mathematical objects).