You know, I'm not sure this answer is all that helpful, and some of it seems a little disingenuous.
It is important to explain to someone the nature of nested hierarchies, and that descendants of verterbrates will always be a part of Vertebrata (at least, it is if that someone is actually interested in learning, otherwise it might just be a waste of time). However, I think we have to be clear about what we mean by this.
To be a member of Veterbrata simply means to be a descendant of an organism arbitrarily defined as the last common ancestor of Vertebrata, But there's a bit of equivocation here with the traditional meaning of vertebrate, which predates evolutionary classifcations, that of 'an animal with a backbone'. If, in the distant future, there's eome species knocking about that descends from verterbrates, but which does not possess a backbone, it would still be a member of Vertebrata, still a vertebrate phylogenetically, but it wouldn't fit the original defining concept of vertebrate any more. This is the sort of thing creationists are looking for when they protest 'but it's still a moth' (though of course they wouldn't define the key transition explicitly, and they seem to expect it to happen in an afternoon).
While it might be difficult to imagine vertebrates evolving into something without a backbone, for an example of the kind of thing I mean, look at the tunicates below. These aren't vertebrates, but they are chordates, and so closely related. Chordata is those animals with a notochord - a cartilaginous rod which, in vertebrates, is replaced by the vertebrae. Tunicates have a notochord in their larval form, but metamorphose into an adult form which doesn't have one, and in fact looks nothing like we'd expect from a close relative of vertebrates. Were a tunicate to evolve that did away with it's larval stage and skipped straight to the adult, repodcutive form, it would still be a chordate phylogenetically, but it would no longer look anything like most chordates and would not fit any description you usually see of what a chordate is, except one which only used phylogenetic criteria.
The names we have for types of organisms are based on the organisms we see around us at present. Saying the descendants of moths will always be moths is not really true. They'll always be Heterocera, the moth clade, but if they evolve into a form that doesn't look like a moth they're not really moths any more. Similarly, cows aren't fish, even though they're nested within them phylogenetically.
------------
A less fundamental criticism - shrimp do not have a ventral notochord, they have no notochord at all. Only chordates have notochords, that's where the term 'chordate' comes from.
Edited by caffeine, : Just wanted to point out that the picture is taken from wikipedia's page on tunicates.