Sunday, March 22, 2009

Statistics can lie

Today's topic is the Alter Aeon "asshole" detector. We've had code in the game for years that tries to do a rough estimation of how big an asshole people are, and after multiple years of tweaks and updates, we have something that produces pretty valid results that correlate spectacularly with bad behavior.

The problem is that it correlates with the wrong bad behavior.

Take a look at the following asshole numbers:

10.249 Demon
-1.848 Cow
-6.941 Woem

In this example, Woem is a very well-liked socialite, and is known for being a nice, helpful person. At the other extreme is Demon, who currently holds the top seven slots on the all-time asshole list. As of this writing, he has no competitors for top slot.

However, Cow is the real problem here. This guy is a real asshole, and is extremely discriminatory toward blind players (and other groups.) I have considered several times explicitly flagging him as an asshole. Both fortunately and unfortunately, most of his bad behaviour happens on private or clan channels that I don't or cannot monitor - fortunately it stays away from the general population, unfortunately it festers and breeds hatred in his social circles.

After thinking about it for a while, the explanation appears to be related to spam. Demon, and his predecessor Gamlin before him, are both prolific and irritating spammers. They would both constantly pester people and spam channels with inane and generally low-brow conversation. Even I have ignored them for periods of time simply to stop the stupidity and prevent them from bothering me. On the other hand, I cannot recall ever having ignored Cow.

The overall point is that people feel free to ignore spam because it's meaningless, irritating, and usually undirected. On the other hand, ignoring the purveyor of a pointed, directed and possibly malicious attack seems like a bad idea. If I wouldn't do it, why would I expect anyone else to do so? Cow remains off the asshole list because people don't dare ignore him. He's too dangerous.

Somehow, I need to find a way to either counteract this, or allow people to mark or flag assholes. This is of course the age-old problem of automated board moderation, and I don't expect there to be an easy answer.

On the plus side, the current algorithm could simply be renamed the 'spammer factor' instead of the 'asshole factor'. At least then people wouldn't have misconceptions about what it meant.

No comments: