Scoring and Voting System’s Upcoming Changes

As I announced back in 2012, changes are coming to the site’s voting system. Since work on the updates has progressed enough to make things clear, I guess it’s time to update you all on the coming changes.

1 – The TPA system is going away. The system never lived up to its potential. The members that could use it effectively and correctly are very few and those who used it tended to ruin the results. So it became a waste of resources that delivered very little value for everyone. The existing votes will be converted to normal votes (what used to be basic).

2 – The updated system now keeps individual votes and that means new useful stuff and less drawbacks.

3 – Due to the way the updated system works, there will be no display of the old bScore. Only the weighed score (the bScore was already hidden from new authors).

So what useful stuff will result from #2? Well, now we can calculate a story’s score while dropping the top 5% and the bottom 5% of votes; thus eliminating outliers and minimizing the effects of vindictive/manipulative/destructive votes. This top and bottom drop will kick in once the story reaches 20 votes. So, I don’t want to hear anything anymore about votes of ‘1’. Those are taken care of with this change.

I’ve been keeping individual basic votes since 2009, so those, with the individual votes from the TPA, will make up the source for the scores in the updated system (millions of votes already). I’ll be dropping the old vote results completely. Theoretically, all the scores displayed for all the stories on the site will now reflect the same voting conditions. What that means is that the old stories’ scores most likely will change drastically. I haven’t completed enough work to make a comparison between the old scores and the new scores for the top performing classic stories, so I don’t really know what will happen yet.

The updated system will allow readers to change their votes. So no more “where is the voting form” except when authors turn off voting. And readers will be able to update their votes on serials as chapters get posted.

Update 2013-05-21: All the changes have been implemented.

Popup Ads on our sites!

I’ve been receiving many questions and complaints lately about popup ads littering the text on storiesonline and finestories. Even a couple of reports about outright graphical ads on the site.

Let me make something clear: storiesonline and finestories don’t serve, host or link to ads.

These ads are typically served by a malware-style browser extensions. Firefox, Chrome and IE have such extensions.

The easiest way to verify this is to turn off all the browser extension that you have and see the ads vanish.

To identify the offending extension, turn extensions back on one at a time. When the ads come back, the last extension activated is the culprit.

iOS 6 and the non-working random story display

Anybody who upgraded to iOS 6 on the iPhone soon found the random story display on the home page to be broken. It took two days to track the problem to excessively aggressive page caching on the part of Safari.

I finally worked around the problem by varying the URL where the random story is fetched for each request. It’s now working on iPhones with iOS6.

You may need to force reload the site’s home page and javascripts by clearing all sites’ data on your iPhone.

To clear all sites’ data, go to: Settings -> Safari -> Advanced -> Website Data and clear the data.

Recent move followup

As you all know, the sites moved to a new location and even onto a new platform. During the move the platform changed and that led to some unexpected issues.

  1. Temporary SSL Certificate issue: Some people, namely those on IE7 and IE8 and a few on Android phones were getting an error about untrusted security certificate when they try to log into the site or use any secure pages. It turned out to be a configuration problem and it’s fixed (needed to configure and specify unique IP addresses for each secure domain).
  2. Kindle eBooks didn’t work: Due to the platform change new updated binaries for the kindlegen software from amazon were needed.
  3. Zip files issue: Some members reported inability to expand zipped archives generated on the new servers. I couldn’t duplicate the issue, but nonetheless, changed the zip archive generation algorithm to use my own hand-rolled zipping solution. Hopefully that solves it. I haven’t heard any complaints since the change.
  4. Slow site during peak time: The new set up doesn’t handle text search well. I’m not sure if it’s a hardware problem (not enough processor bandwidth) or a software configuration problem. Whatever it is, I couldn’t find a solution quickly. I’ve moved text search to a small server that I had lying around. It’s slow but for now it does the job and now text search works without affecting the main database server.
    I’m still working on this one. I will be installing a different indexing software (something more efficient than MySQL’s own built in full text index) and rewriting the text search scripts to use the new software. Hopefully that would solve it.

If there are any outstanding issues that you’ve encountered, let me know. You can leave a comment here or send me a message directly through one of the sites’ webmaster forms.

We’ve moved!

Alright, Storiesonline is now at home on it’s new server farm.

The whole move between stopping the site on the old servers, backing up the files, dumping the database, uploading the files to the new farm, build the database, create the content index, Changing DNS entries and seeing hits come to the new servers, took roughly an hour and a half. More than 45 minutes were simply spent waiting for DNS change to propagate.

Let me know of any issues with the new site, especially performance.

Time to go to bed for me. It’s been a long day.

Slowdowns, Offline and slowdowns follow up

Since the Sunday right before Labor day monday, the sites have been slow. And today, Sunday the 9th or September, the whole server farm went offline from 1:00am EDT ’til 9:40am EDT. Even now that the sites are online, things are flaky with lots of slowdowns and dropped connections.

After much investigation, we found that our oldest two servers (circa 2003) have given up the ghost and the rest are not keeping up with demand. It seems that the electric power in the facilities have something to do with the issue.

Here is what we will do to remedy the situation:

This week sometime, the servers will be moved to a different facility and the dead servers will be replaced. We don’t know the exact time of the move, but it will be in the coming few days. We will make an announcement few hours before the actual move and as soon as we have a firm time.

During the few hours before the move, the site will be in limited functionality. No new postings and no payments accepted.

During the move, the site will be completely offline, and it will last for a couple of hours at least. After that, everything should be back to normal.

Meanwhile, until that happens, the server farm has decreased in capacity, so expect lower performance from all the site that we have, including SOL and Finestories.

We apologize for any inconvenience that this will have. Hopefully, we’ll restore the service to a better than ever shape.

Voting system. Again!

Although I swore I would never discuss scores again because I’ve done it literally hundreds of times since the end of 2006, I will attempt to clarify things here again since there is a lot of misinformation floating around.

So the purpose of this post is to keep this info available for the future.

First point of many: All scores are adjusted. All of them. And it’s scores, not votes. Individual votes are never touched. A story’s general score gets weighed by the system.

What makes it look like the above is not true is the fact that the older stories’ scores, the ones posted before December 19, 2006 (that’s when I brought the new system online), had ridiculously high scores, so high that even with the insane amount of weighing that these stories’ scores are subjected to, some of those scores are still unbelievably high compared to new scores.

The one reason that mandated the need for the new system is Human Nature!

One would think that a rating system for stories would be fairly simple; Get votes, average them out and be done with it. Sadly, that turns out to be very naïve. So how does human nature make this naïve?

Consider the following:

– Not all readers vote. So voters are self selective and not necessarily a real representation of the general readership.

– People are nice in general (unless they have an agenda). They don’t like to punish or discourage. The general tendency is to only vote when they like a story.

– People are lazy. They don’t vote on stories they don’t finish. To finish a story, they have to like it. So the general tendency is to get votes from those who liked the story in general.

– People are easily influenced. If most scores are higher than 8, then voting less than 8 becomes psychologically impossible for most people. How can you give a story a vote lower than the average when it’s fairly readable?

All the above would drive any system’s scores ever higher. It’s an unescapable trend.

Voting on SOL started way back in 1998 and went through few changes. By the end of 2006, the above described trend had pushed the scores on the site to a ridiculous level.

Consider this: the ‘average’ story, which on a scale of 1 – 10 should get theoretically a 6, was getting no less than 9. Most readers said that they never read anything below 9.25. Most readers started to think of the voting system as a silly joke. When I first checked the median of scores for the year 2006, I was shocked to find it around 9.15 (I don’t remember the exact number).

To have a median of 9.15 means that half of the stories are on a point spread of less than.6 points and the other half is on a spread of more than 8 points. That can’t be right, not logically anyway.

Something needed to be done. Fix the system or scrap it. No other choices.

So, after brainstorming for a while, I came up with the new system and after having it on the site for more than five years now, I’m convinced that it was the right choice, and considering all factors, the best system that could be used. Yes, it has flaws, I know them all. Some can be fixed (and will be), and some can’t (and must be lived with). But even with these few flaws, it’s still the best.

There is one negative to the system, and yes, only one negative. It pisses authors off.

Nobody likes anything that lowers their impression of their own work. So to have a system that lowers their scores is a major turn off. So, I allow authors to opt out of the scoring system selectively. They can opt out completely or partially or not at all.

The system’s calculations are actually quite simple. All it does is it shifts the median of the stories’ scores to 6. That sounds complicated, but it is actually quite simple. If you sort stories by their weighed score and by their average score you get the exact same lineup of stories.

So it gives no advantage to any story.

Why do I mention ‘stories posted in the same period’ often? it’s because such a system must consider the factors that affect how and why people vote.

Changing the voting form’s wording affects votes. Even the placement of the voting form affects votes. Before 2007, the voting form had nothing more than number choices from 1 – 10. After the end of 2006, the choices changed. So it wouldn’t be fair to have stories posted before and stories posted after the change be treated the same. Something needs to compensate for the change.

So, now whenever I change something, I start a new period. Later this year, when I scrap the TPA and start keeping individual votes, I’ll need to start a new period.

The system takes all the stories posted in each period and finds the median of their scores (the basic scores calculated by averaging all their votes). That median becomes the focal point of the calculations and the system recalculates the average scores to create weighed scores whose median is 6.

The brilliant thing about the current system is that it kills the psychological effect higher and higher scores and compensates for shifts in voting trends regardless of their source.

If votes trend higher for some reason, the system keeps the scores in check and doesn’t allow the psychological effect of bigger numbers to influence future votes.

And if votes trends lower, the same thing happens the other way around.

Again, the system works on the whole set of stories being posted. So high quality stories will still get high scores and low quality stories still get low scores and the readers’ collective opinion is still represented exactly the same. Which story is more liked by readers is preserved by keeping the same story lineup whether the listing is sorted by average score or by weighed score. Just the representation of numbers is affected.

Love them or hate them, weighed scores are here to stay.

A hint of things to come:

The TPA is going away. I may, I MAY, keep it for authors to use to give other authors feedback, but it will be separated from the score and will have no influence on it.

I will be keeping individual scores, so removing outlier votes from the results will become possible.

Few things to keep in mind:

SOL has competitors that send minions to do some sabotage. It’s often in the form of bombs of 1s.

Some people are irrational about certain subjects and will either give a lot of 10s or a lot of 1s to those stories that touch on those subjects.

If you need clarification about some points, please don’t hesitate to post your questions and comments here. I’ll do my best to respond and I’ll update the article with anything that may be missing.


I hate Murphy and all his laws!

If you look closely, you’ll find that the last previous article is all the way from 2007. That’s because the server that this blog was hosted on previously got seriously zapped by a lightning strike and its hard drive got fried, and all the data about this site, since its move to wordpress got lost.

All that got lost despite having a UPS with supposedly serious surge protection and nightly backups!

Well, the UPS didn’t do its job and none of the backups that we relied on were retrievable.

Also, I’ve been quite busy, so I didn’t have the time to set this up again until now. So here we are, starting from scratch.

Well, not completely from scratch, but since we have a gap of four and half years, might as well be from scratch.

Needless to say, we had to replace several UPSs because we don’t trust the older ones anymore and a new backup policy.

Live and Learn!