Some vestigial organs still have a purpose but this isn't the one it originally evolved to fill. Flightless birds which evolved from birds that could fly are a prime example of this as they can use them for stability or threat displays but cannot fly. Other vestigial organs serve little or no apparant purpose (As far as we can tell) but still develop. The appendix for example while not used to store cellulose digesting bacteria could act as a haven for our current digestive bacteria. A possible reason for the appendix remaining is that further reduction in its size may result in increased incidents of appencitis which can result from blockages in the appendix which are obviously more likely if it becomes smaller.
Be careful not to confuse vestigial with atavistic features. Atavistic being the reappearance of old features such as tails on humans and hind legs in cetaceans (
http://www.talkorigins.org/...omdesc/images/hindflippers.jpg). Even though its quite rare for this to happen the faetus of dolphins still begin developing the hind legs before they are reabsorbed (
http://www.talkorigins.org/...desc/images/dolphin_embryo.jpg) so its understandable why an error in embryo development could result in this.