New Feature: Message Rating System

I've just installed a new feature: a message rating system.Each message can now be rated on a scale from 1 to 5. Little rating buttons now appear in the left hand column next to each message. Messages that have been rated display their rating. Members whose messages have been rated are given a rating which is displayed with their profile info next to each message, and with their member profiles.This is just the bare bones. In the future the feature will be made graphically attractive (something like the Netflix approach), rankings of the best posts by week, month, year, and all-time will be provided, and members will be able to comment on rated messages.

---

--	Percy
	EvC Forum Director

The software checks to make sure you're logged in before it will display the buttons. This is because each member's rating for a message is stored in the database by his member ID. The database is updated directly from the mouse-click using Ajax.But what a wonderful coincidence that you're on-line! As I mentioned, I want to make the feature more graphically attractive. Can I get an icon that looks like this, only icon-size, of course:

---

--	Percy
	EvC Forum Director

1 is lowest or worst, 5 is highest or best.

---

--	Percy
	EvC Forum Director

Responding to the last few posts, I don't have any experience with message rating systems at other boards, but the downside risks mentioned above do seem real to me. The intent is consistent with what LindaLou mentioned at one point, and is the same as the Post of the Month threads. Members should rate messages as to their quality, not as to whether they agree or disagree, or whether they like or dislike the author.In case it helps, I'll describe the algorithm for calculating member ratings that I'll be installing soon. The current algorithm is extremely simplistic, just the average rating across all of the ratings other member's have assigned (people might have noticed that they cannot rate their own posts).The new algorithm will first calculate a single rating from each of the other members by averaging all their ratings for the member whose rating is being calculated. Then the member's rating will be the average of all these ratings from all the other members. This prevents any single member from overly influencing a member's rating by visiting all that member's old messages and giving every single one a low or high rating. When someone does this, all those ratings still only count once when calculating a member's rating.One advantage of the rating system here is that your feedback goes directly to the implementer. This isn't a case of somebody buying bulletin board software and then one day deciding to enable the message rating feature. This is a case of somebody implementing a bulletin board software system from scratch, and then one day finally getting around to implementing a message rating system. Since this person would like to one day market this software, features like message rating (and many other features, too) are essential in order to be competitive. EvC Forum is actually the testbed for dBoard bulletin board software. The current membership are the guinea pigs for this development process, and the big advantage is that I'm not only listening to all feedback, I can actually go in and modify the software in reaction to that feedback.I hope people have noticed that this board has features that no other bulletin board system has, and this will be the case with the message rating system, too, it will just take a little more time. Sometime soon you'll be able to list best messages of the day, week, month, year and all-time, not just for the current, but for any. And every message with one or more ratings will be able to collect comments about the message and why it is good or bad or whatever, similar to the Post of the Month threads but much more flexible since any and all can add comments and discussions can develop. It will be as if every rated message had its own thread.

---

--	Percy
	EvC Forum Director

I'm actually responding to the last few posts...A "moderator only" demerit type system seems a good idea, I like it. For anyone familiar with soccer where the accumulation of a certain number of yellow cards (I think 7 here in the US) brings an automatic one game suspension (a red card is ejection from the current game and suspension from the next), the software could issue an automatic suspension after the accumulation of a certain number of points.The number of levels in the current rating system is not hardcoded, and I will be adding a couple lines in the board's control panel to make it settable. Directors will be able to set it to as many or as few levels as they like.The rating system is a feature that I have to keep turned on in order to hone it to production quality and to wring out the bugs. Some with experience with them elsewhere haven't had much good to say about them, so I'm not sure how seriously to take it yet, not that I won't be making frequent mention of my lofty rating in the moderator lounge.

---

--	Percy
	EvC Forum Director

All such information is retained in the database. At some point I'll be making it available.Netflix adjusts a movie's ratings according to how you've rated other movies relative to how everyone else has rated them, and I plan to do something similar, including detection of when someone is rating in inverted fashion.But I hope we don't agonize over the rating system too much. If it happens to be EvC Forum that manages to develop the most effective rating system yet devised, more power to us, but if not, it's just a board feature I'm testing.

---

--	Percy
	EvC Forum Director

Anyone ever heard of a member rating system called "reputation points". I think vBulletin might use it.

---

--	Percy
	EvC Forum Director

Moderators and so forth couldn't be ignored, and in turn they couldn't use the ignore feature themselves.About making moderators exempt from ratings, I don't know. It seems like a good option to be able to make them exempt, but I actually like the idea that membership can give their opinion about moderation, and just let me apologize profusely to anyone I've recently suspended.

---

--	Percy
	EvC Forum Director

Can you point me at a discussion board that has implemented this feature in a way you like?

---

--	Percy
	EvC Forum Director

Just to save you some time, if you're going to give every moderator post the score of 1 then you only need to vote for one message from each moderator. It's the average of all your ratings for a moderator's messages that matters. That average rating is combined with the average rating for that moderator from all the other members, then an average is computed. You could rate every single message from a moderator a 1 and it would be the same as just rating one of his messages a 1.Hope this doesn't spoil your fun. I know how enjoyable it must be watching those little radio buttons blink while the database is udpated.

---

--	Percy
	EvC Forum Director

Thanks, Iano. That's not hard to do at all. I was worried you were requesting WYSIWYG.But let me qualify "not hard to do at all." There's still only one of me. It won't be soon. Sorry.

---

--	Percy
	EvC Forum Director

Good point. I'll give this some careful thought. Maintaining the advantages of the current approach while adding automation feels tricky.

---

--	Percy
	EvC Forum Director

Good point. What if I made the rating only pay attention to ratings from active members? Active members are anyone who has visited the board while logged in during the past 90 days (the time period is settable through the control panel and can range from 1 day up to infinity).

---

--	Percy
	EvC Forum Director

But the last 20 ratings could be from long ago, or someone could have rated messages posted long ago. With moving averages for stocks you're guaranteed that the last x prices are for the last x days, but for messages the last x ratings could be all over the map timewise. I was thinking that the member active period could act as a reliable time horizon. The average would "move" just like a stock average, but instead of averaging a fixed number of values it would average all the values over a moving period of fixed length.Looking at the example chart on the webpage you linked to, wouldn't it be neat to see member's ratings plotted over time?

Let me try again. I suddenly realized you were talking about the message ratings, not the member ratings. You're thinking of placing a time stamp on each rating for a message and only averaging the most recent x ratings. Let me think on that.

---

--	Percy
	EvC Forum Director