But the question is whether those structural differences really define one's gender.
I understand that it's generally society that defines your gender, based in part on whatever gender role you decide (or are made) to internalize. But I'd have to talk more with my gender studies friends.
There's the physical plumbing, sure. But that doesn't define your gender. It just provides a pretty powerful stimulus to adefine yourself as one gender rather than another.