I've just installed a new feature: a message rating system.
Each message can now be rated on a scale from 1 to 5. Little rating buttons now appear in the left hand column next to each message. Messages that have been rated display their rating. Members whose messages have been rated are given a rating which is displayed with their profile info next to each message, and with their member profiles.
This is just the bare bones. In the future the feature will be made graphically attractive (something like the Netflix approach), rankings of the best posts by week, month, year, and all-time will be provided, and members will be able to comment on rated messages.
It may not be obvious that you have to be logged in to get the buttons.
The software checks to make sure you're logged in before it will display the buttons. This is because each member's rating for a message is stored in the database by his member ID. The database is updated directly from the mouse-click using Ajax.
But what a wonderful coincidence that you're on-line! As I mentioned, I want to make the feature more graphically attractive. Can I get an icon that looks like this, only icon-size, of course:
My experience on other webforums is that post rating rapidly becomes little more than a way of saying "ya! boo! you suck!". Given the predominance of resident evolutionists here I imagine it will simply emphasise that weakness.
I have to say I agree with Mr Jack. I am dubious as to the value of this. Creationists are under under enough pressure as it is, this feature is just going to rub salt in the wound. I see that poor old Peg has already been stuck with a 1 rating.
People are just going to vote low for messages they disagree with and high for those that they approve of, regardless of writing style or message quality. It will discourage creationist participation and encourage divisiveness.
Mutate and Survive
"A curious aspect of the theory of evolution is that everybody thinks he understands it." - Jacques Monod
I third that. I think the ratings could potentially distract people from the actual discussion. And I have experience of being an unpopular poster on a creationist forum; the more science I posted, the fewer stars I got. I have a feeling that the unpopular things I say here could go the same way, though I think most of the regular posters aren't petty enough to bother.
I wonder if the system could be tweaked so that the ratings are used to flag up exemplary posts? So one star could mean "this post is quite good" and 5 stars could mean "this is bloody marvellous and everyone here should take a look." Though I'm a bit hazy, too, on what's meant by quality. Quality to me is RAZD's recent posts about homo erectus but it's also Dr. Adequate's razor-sharp posts which don't always contain a lot of information but are just damn funny.
I agree that a negative rating is not worth having. If we have to have a message rating system, I'd like even the lowest rating to express approval. Not that I think that even that is a great idea.
I'd rather have a system that flagged problem posts for moderator attention. The "Post of the Month" feature is great for reporting good posts, the "report problem posts" thread is not so good.
Responding to the last few posts, I don't have any experience with message rating systems at other boards, but the downside risks mentioned above do seem real to me. The intent is consistent with what LindaLou mentioned at one point, and is the same as the Post of the Month threads. Members should rate messages as to their quality, not as to whether they agree or disagree, or whether they like or dislike the author.
In case it helps, I'll describe the algorithm for calculating member ratings that I'll be installing soon. The current algorithm is extremely simplistic, just the average rating across all of the ratings other member's have assigned (people might have noticed that they cannot rate their own posts).
The new algorithm will first calculate a single rating from each of the other members by averaging all their ratings for the member whose rating is being calculated. Then the member's rating will be the average of all these ratings from all the other members. This prevents any single member from overly influencing a member's rating by visiting all that member's old messages and giving every single one a low or high rating. When someone does this, all those ratings still only count once when calculating a member's rating.
One advantage of the rating system here is that your feedback goes directly to the implementer. This isn't a case of somebody buying bulletin board software and then one day deciding to enable the message rating feature. This is a case of somebody implementing a bulletin board software system from scratch, and then one day finally getting around to implementing a message rating system. Since this person would like to one day market this software, features like message rating (and many other features, too) are essential in order to be competitive. EvC Forum is actually the testbed for dBoard bulletin board software. The current membership are the guinea pigs for this development process, and the big advantage is that I'm not only listening to all feedback, I can actually go in and modify the software in reaction to that feedback.
I hope people have noticed that this board has features that no other bulletin board system has, and this will be the case with the message rating system, too, it will just take a little more time. Sometime soon you'll be able to list best messages of the day, week, month, year and all-time, not just for the current, but for any. And every message with one or more ratings will be able to collect comments about the message and why it is good or bad or whatever, similar to the Post of the Month threads but much more flexible since any and all can add comments and discussions can develop. It will be as if every rated message had its own thread.
I think there's gonna be a lot of 1's and 5's with very little 2-4's.
Another thing I've seen that might work good is just a thumbs up or thumbs down. A post starts at a "0" and then each thumbs up gives it a +1 and each thumbs down gives is -1. You can tell how much people like a post by how far positive or negetive it goes.
I don't think you should scrap the idea but I don't think a 1-5 rating system yields the best results.
The intent is consistent with what LindaLou mentioned at one point, and is the same as the Post of the Month threads. Members should rate messages as to their quality, not as to whether they agree or disagree, or whether they like or dislike the author.
That will be difficult to control.
I've been on boards where ratings were used to grade participants from the lurkers, who did not have to justify their ratings -- it could be that they honestly rated people OR it could be that they used the ratings as part of their campaign to prove their side.
Certainly it is difficult to objectively rate a post that you are debating against, and this will be the average result.
You might want fewer buttons with some guide as to their meaning
For the 1 to 5 system, I find the 3 to be pretty useless, and the 1 and 2 to be near totally useless. Why bother trying to rate if a message is average, poor or profoundly bad.
Good Point/s
Above Average Post
Post of the Month Material
I like! A 1, 2, or a 3 system where you only flag messages you find to have some special good quality (something better than "OK, but so what"). Keep track of them separately for the individual messages (with maybe also an average) and do a number of ratings and average and for the member values. Maybe you could even offer a message filter that only shows messages over a chosen threshold.
Of course, maybe there could be a forth button to cover what had been the old 1, 2 and 3's. The "apathetic" rating button. I don't think we need an "atheistic" button, for messages one might think don't actually exist.
™ Version 4.2
(1)
(1)