You seemed to be assuming he was with her when the serpent was talking to her.
Because it's the most obvious choice, mainly since it doesn't say anything like "then adam comes walking over and sees eve talking to a snake". Instead, the very next sentence says "who (adam) was with her", which implies he was with her when the event in the preceding sentence took place.
This is the problem with the bible, and something that has reared it's head on EvC recently. It's far too open to interpretation, yet people take it as a literal truth. So open to interpretation is it that you cannot even tell people they are wrong. Every reading is just as valid as the next.
"Why don't you call upon your God to strike me? Oh, I forgot it's because he's fake like Thor, so bite me" -Greydon Square