If Jesus is a historical figure, and was not considered a mamzer, then the likelyhood that he was married would be very high. If Jesus was for what ever reason considered to be a mamzer (for example, if rumors about his father not being Joseph was around during his life time), then he would be 'ritualistically impure' , and would only allowed to marry another bastard.. which would reduce his chance of being married. On the other hand, if that was true, he would not have been allowed in the temple at all, and other stories about Jesus had him going to the temple.
Isaiah 53, if you read about it in context, is talking about the nation of Israel, not Jesus.
Let us look at some of the preceding lines to Isaiah 53. In these preceding lines
Chapter 41:8 But you, Israel My servant, Jacob whom I have chosen, the seed of Abraham, who loved Me, 9 Whom I grasped from the ends of the earth, and from its nobles I called you, and I said to you, "You are My servant"
Chapter 44:1 And now, hearken, Jacob My servant, and Israel whom I have chosen. 2 So said the Lord your Maker, and He Who formed you from the womb shall aid you. Fear not, My servant Jacob, and Jeshurun whom I have chosen. ... 21 Remember these, O Jacob; and Israel, for you are My servant; I formed you that you be a servant to Me, Israel, do not forget Me.
Chapter 45:4 For the sake of My servant Jacob, and Israel My chosen one, and I called to you (i.e. Cyrus) by your name;
Chapter 48:20 Leave Babylon, flee from the Chaldeans; with a voice of singing declare, tell this, publicize it to the end of the earth; say, "The Lord has redeemed His servant Jacob."
Chapter 49:3 And He said to me, "You are My servant, Israel, about whom I will boast."
So, in line and line after line, before Isaiah 53, the servant of isreal is repeatedly mentioned to be Israel. It just doesn't make sense for a writer to consistantly refer to the servant of Israel, change gears to try to make a prediction of someone 700 years in the future, and then try to shift back. That is all without discussing the translation problems the KJV version of the bible has with this passage.