Raph Koster wrote an interesting post talking about the numbers on MMOGChart.com. He points out how different companies will use different metrics, the weaknesses of the different metrics, and what he thinks would be the ideal metric (unique connections per week).
Now I’m assuming that the idea of these metrics is to rate the ‘health’ of the game. If all you’re doing is trying to rate the fiscal strength of a game there’s already a bunch of numbers to use to do that (gross revenue, net profit, etc.), but it seems to me that if you’re trying to determine how ‘healthy’ a game is what you probably want to look at is how much people are playing.
A person who logs on once a week to check their mail obviously isn’t getting the same enjoyment out of the game as someone who plays twelve hours a day and shouldn’t be weighed the same (in my opinion). They aren’t contributing to the community and all things being equal they are probably much more likely to quit should personal finances get tight.
It would seem to me to be much more logical to track man hours played per week. A person who logs in five nights a week for an hour would seem to me to be a much stronger indicator of game health than someone who logs in one night a week for an hour. Likewise, someone who plays three hours every night would seem to me to be a much stronger indicator than someone who spends one hour a night.
Of course one argument that can be made against this is that games which have longer play sessions benefit more from this metric, but is that such a bad thing? Yes, a round of Bejeweled is shorter than a typical session of Everquest, but if Bejeweled was actually holding someone’s interest better than Everquest then wouldn’t they just play round after round until their Bejeweled sessions were longer than their Everquest sessions?
Ideally what you would want to track is player enjoyment. A player who enjoys every minute of a four hour session probably has a much greater value than someone who plays for six hours but only enjoys half of it. However, short of some sort of device wired up to the user to measure heart-rate, respiration, etc. and then interpret that into enjoyment I can’t see any good way to quantitatively track such a number, so I think we are stuck with the idea that if people don’t get enjoyment from a recreation they will stop doing it. Yes, sometimes people will do something they don’t enjoy for a while in the hopes of getting to a payoff, but if they payoff isn’t great enough to offset the tedium then they will eventually stop.
Now when I talk about ‘played’ I am referring to a person actually being actively involved in some aspect with the game, not just being logged in. Time in which a person is not at the keyboard shouldn’t be counted since that would favor games that force people to be logged in for long periods of time while conducting activities (such as selling) that they don’t enjoy enough to stay around for.
Since we can’t really track when players are at their keyboards we have to settle for the next best thing; tracking when they are active. Probably the best way to do this is to take a short amount of time, say five minutes, and as long as there is some activity during that time (position or orientation change, keystrokes, etc.) that is sent to the game than the person is counted as being active for those five minutes. Five minutes is, of course, just an arbitrary number but if you make it shorter you run the risk of missing activity because a short distraction takes them away from the game (phone, a knock at the door, etc.) and if you make it longer than you run the problem that someone could be idle but occasionally checks in to see if anything has happened.
The only thing that can really throw this out of whack is if you have a lot of people running third party macros, but since most games frown on that I would think that they would only cause a slight distortion in the data (and if third party macros are causing a major distortion then there are probably bigger things to worry about than metrics).