I think that the Nephilim mentioned in the Torah are kind of a Canaanitic/Israelitish equivalint to the Titans of the Greek traditions; Men of strength and stature, born of the gods, and were heroes of the ancient world before "the flood."
It is known that the name, 'Rpum', has been found in the Ras Shamra tablets of Ugarit which dated to the 1200's bc. These were warrior gods who traveled by chariot to attend feasts of El in Syria.
When the earliest traditions of the Torah were first written down between 900's and 700's bc in the Israelite and Judean courts, the Nephilim and Rephaim were obviously names which envoked the idea of superhumans or gods/demi-gods. The name Rephaim was used to describe ancient heroes and tribes whose mysterious origins and legendary attributes were unrivaled.
Were the Nephilim giants? It seems the Hebrew chroniclers felt that they were, but vaguely so. They used the name "Nephilim", similarly how a Greek would use the word "Titan" to convey an idea of strength and power, and this would coincide with great stature aswell.
How big were these giants? Within some of the ranks of a tribal army, there would undoubtedly have been some big and highly respected warriors who may have even formed special fighting units. If you had several families who had a genetic average stature of 6'to 6'6 for men, and they all served in the same division these men would almost certainly have been feared by a regular platoon of soldiers.
The average stature of the ancient Israelites has been reliably estimated at around 5'5 for men and 5' for women. Since Goliath was 2 meters tall, or about 6 feet 7 (according to DSS)and may have weighed 250 or 300 lbs, this fits quite well with the height range I described.
In 2 samuel 23:21 there is also mention of an Egyptian 5 cubits tall, who was slain by one of David's mighty men. 5 cubits is about seven feet six.
The WWE has atleast six men who could really be considered gigantic even by modern standards: The Great Khali, 7'3 420 lbs, Big Show, 7' 500 lbs, Kane 6'9 326lbs, Undertaker 6'9 320 lbs, and Mark Henry 6'1 400 lbs.
If these modern wrestlers were introduced to the ancient Israelites 3,000 yeears ago, I could almost bet you that legends would be born not unlike those of Goliath.
Acromegaly is a rather rare disease affecting 1 in several thousand people, but Gigantism is far rarer, perhaps affecting 1 in half a million or more. The earth's population at the time of King David was about 70 to 100 million people, and maybe only 300,000 living in Palestine at his time. Yet this does make it statisticaly possible for "pathological" giants to exist in Palestine at the time.
However, the likliest explanation is that these giants described in the bible refer to men who were constitutionally taller than average men, and perhaps a few were affected by gigantism or Acromegaly.