Register | Sign In


Understanding through Discussion


EvC Forum active members: 65 (9164 total)
2 online now:
Newest Member: ChatGPT
Post Volume: Total: 916,913 Year: 4,170/9,624 Month: 1,041/974 Week: 368/286 Day: 11/13 Hour: 0/0


Thread  Details

Email This Thread
Newer Topic | Older Topic
  
Author Topic:   Why Ratings Are Not Objective.
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(1)
Message 37 of 88 (536012)
11-19-2009 9:23 AM


How Member Ratings Are Calculated
Just to reduce uncertainty, what Granny Magda described is how it works. In case it helps make it more clear, here's a simple example.
Let's say I rate five of Buzsaw's messages, giving them these ratings: 5, 2, 3, 4, 3. The average of these five message ratings is 3.4, so I contribute a rating to 3.4 to Buzsaw's member rating. If I'm the only person who has rated any of Buzsaw's messages then 3.4 would be his member rating.
Now let's say someone else rates five of Buzsaw's messages as follows: 5, 4, 5, 5, 3. The average of these five message ratings is 4.4. If you average 3.4 and 4.4 you get 3.9, which would be Buzsaw's new member rating.
As quickly became apparent when I added message ratings, the current approach is inherently both unfair and ambiguous. The ratings are based upon opinion, which is highly subjective, and every person will judge according to their own individually selected criteria.
While I wouldn't consider the current rating system a very reliable parameter upon which to judge the quality of a member's contributions, there does seem to be a correlation of higher ratings with those who discuss things they know something about or who take the time to investigate. Those who tend to put fingers in gear without taking the time to develop an appropriate level of understanding of the topic seem in general to have the lower ratings.

--Percy
EvC Forum Director

Replies to this message:
 Message 38 by PaulK, posted 11-19-2009 9:33 AM Admin has replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(1)
Message 39 of 88 (536017)
11-19-2009 9:57 AM
Reply to: Message 38 by PaulK
11-19-2009 9:33 AM


Re: How Member Ratings Are Calculated
Leaving aside the question of why you think the developer described his own algorithm incorrectly, you could test your theory fairly easily. Find an old thread and rate a single message of a member who hasn't been active in some time and see what it does to his rating (click the rating number you want, then click on "refresh" or "reload"). I'm betting his rating will become whatever you rate that message. You can change the rating you've assigned this message, enabling you to check the effect of any rating from 1 to 5 when there's only the single message rated for the member. Please let me know if you find a bug.

--Percy
EvC Forum Director

This message is a reply to:
 Message 38 by PaulK, posted 11-19-2009 9:33 AM PaulK has replied

Replies to this message:
 Message 40 by PaulK, posted 11-19-2009 10:21 AM Admin has seen this message but not replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(1)
Message 46 of 88 (536203)
11-20-2009 11:44 AM
Reply to: Message 45 by petrophysics1
11-20-2009 11:33 AM


The thread for reporting discussion problems is Report discussion problems here: No.2.

--Percy
EvC Forum Director

This message is a reply to:
 Message 45 by petrophysics1, posted 11-20-2009 11:33 AM petrophysics1 has not replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(1)
Message 65 of 88 (536594)
11-24-2009 7:05 AM
Reply to: Message 61 by Stile
11-23-2009 1:47 PM


Re: Possible New Rating System?
Hi Stile!
I want a system with the flexibility to implement a number of approaches, including ones along the lines you describe. It should also be possible using correlations to divide members into camps and adjust the impact of their ratings according to how strongly they and who they're rating correlate with the different camps. Properly implemented, a low rating from some members could have the effect of increasing someone's rating.
But this will take time to implement. For now I advise that no one pay much attention to the current ratings. Whether high or low, they're both imposters a la Kipling's If.

--Percy
EvC Forum Director

This message is a reply to:
 Message 61 by Stile, posted 11-23-2009 1:47 PM Stile has replied

Replies to this message:
 Message 66 by Stile, posted 11-24-2009 8:24 AM Admin has seen this message but not replied
 Message 67 by Straggler, posted 12-04-2009 3:13 PM Admin has replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(1)
Message 68 of 88 (538223)
12-04-2009 3:29 PM
Reply to: Message 67 by Straggler
12-04-2009 3:13 PM


Re: Possible New Rating System?
Straggler writes:
I fail to see how this even could work given the widespread nature of debate. Two members from opposing camps of the deistic/theistic Vs atheistic debate might wholly agree with each other on social threads relating to gun laws or anti-discrimination laws. For example.
Oh ye of little faith.
I like the new member stats on forums participated in with percentages etc. Is there a way to see when a member last logged in? Regardless of having posted. If not is this a stat that could be published in a members stats profile?
At some point the member's profile page will include whether he's currently active, and if so what he's doing, like reading a thread or posting a message. And if he's not active then it will include when he was last active and what he was doing at the time.
And if we want to get controversial is there a way to show not just the stats pertaining to the threads a member participates in and the number of posts per day - but their number of words per post?
How about a readability index. Your first paragraph scores at 11th grade with an ease of reading of 44, about mid-range of the Flesch-Kincaid reading ease score.
I'm currently working on private messaging.

--Percy
EvC Forum Director

This message is a reply to:
 Message 67 by Straggler, posted 12-04-2009 3:13 PM Straggler has replied

Replies to this message:
 Message 69 by Straggler, posted 12-04-2009 3:37 PM Admin has replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(2)
Message 70 of 88 (538242)
12-04-2009 6:16 PM
Reply to: Message 69 by Straggler
12-04-2009 3:37 PM


Re: Possible New Rating System?
Straggler writes:
I don't know if that is a subtle compliment, a subtle dig or just a statement of pure fact. But it sounds like a good idea.
Viv Pope puts us all to shame. Cut-n-paste some stuff from him and some of the rest of us into this Readabilty Index Calculator. I bet you won't find anyone who beats him.

--Percy
EvC Forum Director

This message is a reply to:
 Message 69 by Straggler, posted 12-04-2009 3:37 PM Straggler has not replied

Replies to this message:
 Message 71 by lyx2no, posted 12-04-2009 7:01 PM Admin has seen this message but not replied
 Message 75 by RAZD, posted 12-05-2009 4:43 AM Admin has seen this message but not replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(2)
Message 82 of 88 (564099)
06-08-2010 8:45 AM
Reply to: Message 81 by Straggler
06-08-2010 7:32 AM


Re: Finding messages having ratings
Straggler writes:
I don't think I had ever rated Moose until my mini campaign of experimental 1 ratings. And each 1 rating I gave him had a significant effect.
Thanks for finding this. I must have broken it in the last release. Should be a simple fix.

--Percy
EvC Forum Director

This message is a reply to:
 Message 81 by Straggler, posted 06-08-2010 7:32 AM Straggler has not replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(2)
Message 83 of 88 (564103)
06-08-2010 9:21 AM
Reply to: Message 81 by Straggler
06-08-2010 7:32 AM


Re: Finding messages having ratings
Member ratings are updated only when someone rates one of their messages. Could you try rating a few more of Moose's messages? The first one should assign him the correct rating, subsequent ones shouldn't change that rating at all, assuming you continue assigning him 1's.
--Percy
Edited by Admin, : No reason given.

This message is a reply to:
 Message 81 by Straggler, posted 06-08-2010 7:32 AM Straggler has replied

Replies to this message:
 Message 84 by Straggler, posted 06-08-2010 12:22 PM Admin has replied

  
Admin
Director
Posts: 13046
From: EvC Forum
Joined: 06-14-2002
Member Rating: 2.6


(1)
Message 85 of 88 (564186)
06-08-2010 8:58 PM
Reply to: Message 84 by Straggler
06-08-2010 12:22 PM


Re: Finding messages having ratings
Yep, it's working as it should!
Edited by Admin, : No reason given.

--Percy
EvC Forum Director

This message is a reply to:
 Message 84 by Straggler, posted 06-08-2010 12:22 PM Straggler has not replied

  
Newer Topic | Older Topic
Jump to:


Copyright 2001-2023 by EvC Forum, All Rights Reserved

™ Version 4.2
Innovative software from Qwixotic © 2024