After following many discussions on the forum about the scoring system, it became evident that everybody notices that serial stories tend to score higher than single shot stories for various reasons. The most obvious reason for this trend is that those who do follow serial stories tend to be those who like them to begin with, so they will definitely score them higher.
Due to this fundamental difference in reader behaviour, it makes it unfair to compare single stories’ scores to serial stories’ scores when weighing each story’s score.
The change that I’ve decided to make to the weighing formula is to make it aware of the story’s serial status.
So the system will have two separate medians for the each scoring period. One for single shot stories and one for serial stories last updated in this time period.
I’ve tested this change already in the development environment and one thing became clear: serial stories’ scores will drop after this change and single shot stories’ scores will rise after the change.
I believe this change will make for a weighing forumla that is more fair for single shot stories that don’t benefit from inherent reader bias.
Although my mind is kind of set on making this change, I thought I would get your opinion about it first.
Please use the form below to voice your opinion.
Update: Before you post telling me that longer stories are better, this algorithm change isn’t about Long vs. Short. This is Serially posted stories vs stories (long and short) posted in a single shot. This isn’t about giving advantage to short stories, this is about removing formula bias against stories (Long and Short) posted in one shot. The change in algorithm does not look at the story’s length, it only looks at whether the story is posted in one shot or over time.
Update 2: Since most replies were negative, I cancelled the change.
29 responses to “Coming change to the scoring system”
To what degree will series scores drop, might I ask?
while I’m at it, why not show chapter by chapter scores when a writer is checking his or her statistis?
The drop will vary by posting period and by serial’s score.
We don’t keep chapter scores.
It seems this changes is designed to penalize writers of long stories. As a writer of both, I’ve never been discontent to see scores of the shorter stories a point or two below the serials. Why not separate the categories and weigh them as serials and short stories. (I think you’ve established a length in contests of 25,000 words which would be a good cut-off without worrying about whether it has chapters.) Frankly, I believe the weighting of scores has outlived its purpose. People see a score of 8.60 and are pleased that they agreed with everyone and voted it a 9.00. In reality, that lowers their average as only a 10.00 can raise the raw score.
I would consider turning scoring off with this new system as it seems to remove the scores further from reality.
Also, you have often mentioned the ‘posting period’ but I have never seen it clearly defined. Is that 30 days? 3 months? A year?
I love this site and it is the only place I post my free stories. Thank you for maintaining it and for always looking for better ways to serve both readers and writers. I do appreciate it.
This isn’t long vs. short. This is single shot post vs serial post.
Long stories that are posted in a single shot suffer the same bias as short stories posted in one shot.
Those who come back to the story and keep following it over weeks, months or years will tend to view the story more favourably and give it a higher score.
The new algorithm doesn’t look at the length of the story, it only looks if it has an update date that is different from the posting date.
I’ve posted the posting multiple times. But here is the latest version:
+------------+------------+
| Start | End |
+------------+------------+
| 1998-01-01 | 2001-07-01 |
| 2001-06-30 | 2003-01-01 |
| 2003-01-01 | 2003-12-31 |
| 2004-01-01 | 2004-12-31 |
| 2005-01-01 | 2005-12-31 |
| 2006-01-01 | 2006-12-18 |
| 2006-12-19 | 2008-12-31 |
| 2009-01-01 | 2011-12-31 |
| 2012-01-01 | 2013-12-31 |
| 2014-01-01 | 2021-12-31 |
+------------+------------+
This is an excellent idea. I was concerned about some run on stories that were getting out of control. While shorter single focus, stories would have been better. The stories may have been in the same universe or even in a series if indicated.
The long story format was benefiting from the rating bias. This may have been the cause for the long story in the first place. An idea well past its time.
It would make a great deal more sense just to prevent scoring on serial stories until they are concluded.
Nope.
Some authors need the scores to keep writing.
And it wouldn’t change the fact that those who keep reading until the story is concluded will tend to score it higher as soon as they can.
How about going back to not being able to score a story every time you opened it?
When I first signed on to SOL you could only vote on a story about every six months.
The current system allows you to change your vote. Not cast a new vote. Each time you score the same story, you simply update your old vote and the system updates the story’s score accordingly.
I think thqat, since the data shows that analysis to be accurate, that it’s a reasonable and valid way to attack the problem.
My only concern is that long stories that get finished tend to be the ones which get much support, which mean this may bring long stories which actually deserve to be ranked higher down.
However, I guess no matter what you do, someone will be biased for and someone against, so if this ends up with a more equitable score distribution, then go for it.
–Ben
I think serial stories score higher because there is more character development. Why penalize them for writing a better story? By your logic, flash stories should score the highest.
See girl. Fuck girl. The end. —- it’s a 10!
If that were true, then long stories with great character development, posted in a single posting should score as high or higher than serial stories and higher than short stories. Analysis shows that they don’t. It’s not about the length of the story. It’s about the time that the reader invests in the story, the longer it is, the higher the vote tends to be.
You make it sound like a bad thing that people get invested in a story over time.
Another question I have is will this be applied site wide, meaning on all stories already on SOL. I notice many of the older stories seem to score a lot higher than current ones. I haven’t been here long, so I assume that is because there was a change in scoring at some point. This is why I assume the Top 50 stories are all older. Just an observation.
My two cents worth is scores shouldn’t be manipulated.
G Younger
Laz, as I have messaged you several times, your scoring system is very biased, not just on short Vs. long but other things. Your responses have been that the reader does not know enough to score things so you do it no matter what the votes show.
May be why so many leave or don’t post. Why not have the real score as voted by readers and the SOL score showing the 2 scores side by side.
As I have always said it is your site, you do what you want, but isn’t reader input as important as what SOL thinks.
You put a lot of work in the site, and it is one of the best free sites, but over time it has been so obvious to authors to just delete stories or to turn off voting not to have your adjusted scores.
Dear Lazeez,
Having written both short an d long stories, I have a few comments on your plans.
Firstly, diverging scores of shorts and epic tomes may also have to do with a larger percentage of short stories written without much plot, character development and often quality. Many newbies start with shorts and it often shows.
Secondly, from the feedback I have got, I can tell that readers like longer stories better. The score is simply a reflection of readers’ preferences, not a quality assessment.
Thirdly, it is mostly the serials that draw readers to your site. Why on earth punish those stories that draw readers to the site?
Further, any defined limit of what constitutes “short” or “long” will favour or punish stories close to that boundary and cause evasive mechanisms, such as chopping a long story into short sequels. E.g., I could divide In The Navy into about thirty short installments, each depicting a defined period in the life of the protagonist. So may other writers with their material. You may end up with a best list consisting entirely of one Jay Cantrell story :o)
Another point in case: introducing a bias towards short stories will be a starting point only. Gay stories are also scored low, as are stories with squick content. Giving those a boost would be only fair. Military type stories show above average scores. That could be corrected too. I could go on and on.
In summary, let the score be what the readers think, not what they should think. You cannot change the fact that readers appreciate the efforts going into the development of a long story. There is a reason why JK Rowling sells better than any collection of short stories.
Cheers
Argon
This isn’t about long vs. short. It’s about serial vs. single shot stories.
A serial is a story that gets posted in multiple submissions over time, whether it’s a million words or two thousand words. A non-serial story is a story posted in one shot, whether it has a hundred chapter or two paragraphs.
This algorithm change doesn’t look at story length, it only looks at update dates for the stories and whether they differ from the initial posting date. This algorithm change deals with the mechanics of posting and voting, and not story contents.
Why are authors who write long stories being penalized and why are writers of one off stories being given an advantage? Are there more writers submitting one of stories or are you trying to encourage more one off stories and dissaude authors who like pumping out longer stories. Personally I’ve found that the quality of writing improves in stories that are more than one chapter long. Readers usually pick up typos and grammatical errors when their reading something longer than 5,000 words and the authors usually try to fix those mistakes thus improving the reading enjoyment of subsequent readers.
By the way what constitutes a serial story. Is it a story with more than one chapter or is it a story in multiple book format?
My view is to leave the scoring alone. At the moment I don’t like how it’s set up but I can live with it. What you’re proposing is just a slap in the face to any author who writes more than a one off feature.
This isn’t about long vs. short. It’s about serial vs. single shot stories.
A serial is a story that gets posted in multiple submissions over time, whether it’s a million words or two thousand words. A non-serial story is a story posted in one shot, whether it has a hundred chapter or two paragraphs.
Okay I see what you are saying. You want to encourage only complete stories to be posted on the site and stop authors from posting as they write… or did I get that wrong?
G Younger
No, you don’t see what I’m saying. The new algorithm doesn’t give advantage to anybody. It simply removes an advantage that serial stories have over single shot stories. So after the change, single shot posted stories won’t look like they’re lesser stories.
My analysis shows that potentially, the same stories, if posted over time will score better than if posted in a single shot simply because those who don’t like it, drop it early and never score it negatively, thus eliminating lower votes. But a story posted in a single shot, has a better chance of being read right through and getting the lower vote.
I’m starting to question the wisdom of this change.
As a reader, I’ve sensed that some writers who have completed their longer stories stretch out the posting, in part, to garner more readers and higher scores. To the extent that the higher scores are the goal, this seems like a good change to encourage posting full stories. While it is slightly off topic, I would be interested on your thoughts on the issue of slow posting to, effectively, bump the story. As a reader, I hate it. On the other hand, I see the point. Any other ways to achieve the same goal and still let me finish reading a finished story in less than a year?
No doubt, this will lead to:
– Friction with authors.
– Adaptive behavior (such as converting chapters to a series or a universe) which will impede the usefulness of the site for the reader.
– Remove an incentive to write longer stories.
You should get rid of the scoring altogether and switch to a recommendation system. At least this will get rid of scoring complaints and allow like minded reader to find something to their taste.
This seems to discourage posting stories as serials. If I have a 60 chapter book, and post it all in one shot, I’ll get a higher score than if I post it over a period of weeks/months. What this means is, that I have incentive (score-wise) to withhold the story until it is 100% written, edited, proofread and finalized, rather than post in batches as they are finalized.
This seems to me to be counter to the goals of having new stories/chapters posted regularly. I think my avid readers would rather have something to read each week rather than have to wait one to two months between posts.
I use the scoring system to decide if I want to read a story or not. If a story cannot achieve a rating of 7 or better, it usually isn’t of interest to me. I would like to see serials posted faster, but only because I want to read them sooner. I don’t want to see short stories achieve a higher rating, only to find the content/writing style isn’t of interest to me. Perhaps other readers/writers may have similar views. Many of the poorly rated short stories tend to be unsubtle, fuck the bitch in description, which isn’t necessarily bad, but aren’t of interest to me. As far as gay to squick content, as long as the description makes it clear, I’m not concerned, as I can just skip over them.
Thanks,
Craig Talbot
Lazeez,
How will this affect the authors who do not allow voting on their stories? The obvious answer I’m sure of, but I wanted your thought on that subject.
Regards,
Roger
As my highest rating has been for a brief work, I have little to argue about. However, I wonder about single- vs multiple-postings. It seems to me that Refusenik and gwresearch have posted whole novels as a single shot. I’m with Argon here. Should I be posting University as a hundred short stories?
I’ve always found the rating system impenetrable. I have read the explanations fruitlessly.
My personal opinion is that the scoring should be straight with no adjustments whatsoever.
Is it not possible to switch to a system the equivalent to the “reviews” as used by lots of sites selling things. Amazon, Tripadvisor et al. Adjust it to show 1 – 10 with no comments. If need be, or possible, not show it on the listing page but only after clicking on a button in the score column. This would also help where a story has a lot of 7s / 8s and 9s and then some 1s. This would let the reader know that it has a low score because there may be a squick factor, or other content. I was looking for a cheap mobile phone. A particular one had a good write up but there were 3 one stars, no 2 or 3 stars. One comment with 1 star was because the phone did not have a camera. It was sold with a pay-as-you-go sim card, charger, the works at the expensive price of £9.95, approx $15.00 U.S. At least with that system, even without any comments, the reader would see that it was a high scoring story with someone or something bringing it down.
Whatever scoring system is used, I’d really be happy if it showed both the raw and adjusted scores. In other words, 7.85/8.35.
That would give everyone an idea of the actual effect of the system. And would help clarify if a story was slightly lower rated simply because there were a lot of other good stories by popular authors posted at the same time.
An author with a big following of dedicated readers is going to get higher scores for a story than one that is posted by a lesser-known author, and is going to get read more by people who want to give it higher scores.
Fundamentally, we can pick apart any scoring system and find flaws. This is why I’d like to see ‘raw’ scores posted as well.
Thanks for your consideration.