Holmes, we have 2 definitions of "secularists". I don't use the word as you suggest because I think most people understand it to mean something akin to the government being secular, which is wrong. The government is not suppossed to be "secular" or non-secular. If anything, it should to being pro-religion in general, but non-coercive and non-sectarian.
I think people use the claim of "secular" to propose secularism as the official religion/ideology, and so a secular hero, for example, can have a statue or something secular in a courthouse or public grounds, but anything connected to religion is suspect. Imo, that's wrong, and that's not what the Anabaptists, Baptists and others were talking about when they advocated separation of Church and State.