Since implementing the scoring system on the site in 1999, it has been the only controversial part of the whole site. Many users found it useful, some didn’t trust it and some ignored it.
To authors it’s even more troublesome. Many authors expect the score to tell them how they did in their writing, they wanted it to reflect the effort that they put into the story regardless of the story’s content and its subject’s appeal to the readers.
Those concerns and expectations are not something that I can really address. Authors need to simply realize that the score simply reflects how much a reader liked the story and whether they recommend that others read it too. It’s like a thumbs up signal.
However, there is a problem with the scoring system that I can address: Score compression.
Score compression is when votes, like they are now on the site, tend to be mostly on one end of the scale. Last check revealed that the median for all scores on the site is 8.62!
A median of 8.62 means that half the stories on the site have a score of 8.62 and more. That means about 8000 stories have about 1.2 points spread. That means anything below 9 didn’t get a good score. 8.62 is so close to the top, it’s making scores meaningless.
The reasons for this compression are multiple.
- Some readers never vote anything but 10; they’re nice people, they don’t want to hurt the author’s feelings.
- Some readers vote only for stories they like. For stories they don’t like, they abstain from voting.
- The psychological effect of high scores. The higher the scores the higher the readers will tend to vote.
So, I’m introducing two changes to the system to be rolled out gradually.
The first change is the wording accompanying the number scores in the vote form and I’m removing the numbers. I’m proposing the following as the new list:
Amazing; Impossible to Improve
Excellent Story
Great Story
Good Story
Not Bad
Some Good, Some Bad
Not Good
Pretty Bad
Hated it
You Call this a Story!?
This way it’s not mixed signals. The old list was a bit misleading to authors as it implied that the score may represent the readers’ judgement on how well the story is written. Words like ‘Needs Work’, imply that the reader noticed the errors in the story and commenting on them.
This list is not final. I’m open to suggestions of a better wording that improves the distinction in your minds about the meaning of the score you’re casting.
The second change is the more drastic one. I’m replacing the current scores with weighed scores.
The new scores shown on the site will reflect the story’s score relative to the median of all scores on the site. This will have the effect of lowering all scores. I’ve implemented the formulas that calculate the weighed scores and here is a sample of scores and their new values:
Old Score -> New Score
(average) -> (weighed)
10 -> 10
9.85 -> 9.56
9.5 -> 8.55
9 -> 7.10
8.62 -> 6
7 -> 4.93
6 -> 4.28
5 -> 3.62
4 -> 2.96
3 -> 2.31
1 -> 1
One thing to remember, the weighed score is relative to the current median. So a story’s score may change even if it received no new votes. If the median changes, then the story’s score will change.
Hopefully, the wording change will make the votes that readers cast more reasonable, so that automatic 10s change to something more meaningful.
One problem I don’t know how to address is the fact that the more recent the story is on the site, the higher the average score is; this is related to the psychological effect of higher and higher scores. So if you have a reasonable solution, I’m all ears.
I know that scoring is a controversial subject but let’s all try to be as objective and reasonable as possible. I’m trying to make the system work for everybody the best possible way. I appreciate everybody’s contributions.
And before you fire off your reply, one thing I will not do, I won’t ever scrap the voting system. So don’t even suggest it.