Are weekly and monthly hiscores a good proxy for concurrent player counts?

Yes, but only for monthly hiscores, and even then with restrictions.

Player Count Definitions

Let's start by defining what player counts are.

Concurrent player counts

Most people think of player counts as the number of people online right now, and that is true for a live count, but a single live count can be close to the daily maximum of players, close to the daily minimum of players, or the daily average of players.

A single sample from the live ticker count of players tells us very little contextually about what that single count means.

Average daily player counts are samples taken across an entire day and averaged together. A daily max is the highest sampled count during that day. A daily min is the lowest sampled count during that day.

The daily average is the better estimate of what the population looked like across all hours of the day, and it is a truer estimate of what the player counts actually are across any window.

If RS3 briefly peaks at 30K players but averages around 20K across the day, is it fairer to say the player count for that day was 30K or around 20K? Both can be useful depending on what you are trying to describe, but the average is a better data model for contextualizing variation across an entire day than a max or min would be.

Weekly and monthly hiscores

Weekly and monthly hiscore counts are different in that they are totals, not averages, for a period. These hiscores capture the accounts that gained at least 1K XP in the given period.

Weekly hiscores and monthly hiscores are more about XP gains rather than persistent activity online. In that way, they are more like a max rather than an average.

Confounders

A confounder is something that distorts the data and makes interpretation less clean.

Both measures have confounders.

Bots inflate both concurrent player counts and hiscores, but in different ways. In RS3, where there are fewer visible bots, the bigger distortion is likely on the weekly and monthly hiscores because a bot only needs a small amount of XP to appear, and yet might be banned before contributing to persistent online activity.

On OSRS, bot farms can be so large and so resilient that long stretches online can inflate player counts. We cannot see the weekly and monthly hiscores for OSRS, so for our purposes, where bots show on hiscores is inconsequential.

For probably the last decade or so, the number of players using alternate accounts has increased dramatically. These alt accounts, especially during holidays and events, can dramatically increase both concurrent and hiscores counts as more people reap rewards from playing more than one account at the same time.

Concurrent player counts can also be affected by server outages or any reason why player counts might falsely show zero players online, which is why I remove rows under 1K from the population data. Concurrent player counts also capture people logged in but not actively gaining XP.

Weekly and monthly hiscores also capture non-members who gain XP in a period. If we are trying to understand the health of the P2P game, these numbers can overinflate the count we care about. A members/non-members split would be ideal, but that split is not available for weekly and monthly hiscores yet.

In summary, I think of concurrent player counts as player engagement and hiscores as total account activity in a period of time.

There are other mechanical issues with hiscores also. Concurrent player counts update every three minutes here, so we can start analysis right away. Hiscores require waiting either a week or a month before the period tells us what happened.

What's the correlation between hiscores and concurrent player counts?

I correlated RS3 hiscores counts with RS3 concurrent player counts.

I only included periods where all seven days and all days of the month had data.

This is a construct validity question: how much of the same general concept, in this case player counts, is measurable in both hiscores and average player counts? If they are both generally measuring the same idea, they should move in roughly the same direction as each other statistically.

If we average player counts to the monthly grain and weekly grain, then correlate those with the monthly and weekly hiscores, we should be able to determine the level of relationship between the metrics.

Weekly hiscores were lower overall, ranging from a correlation of 0.43 to 0.96.

The last six months of weeks show a stronger weekly relationship, and there are several reasons why that might be the case: fewer people are playing on alts, fewer people are playing irregularly at different times throughout the month, and fewer people are playing in lower activity playthrough cycles, such as doing dailies.

We see a consistent stronger relationship with monthly hiscores, starting at around a correlation of 0.79 through 0.95 over the last six months. The last six months, similar to weekly hiscores, probably shows the stronger relationship for the same reasons.

Overall, with fewer people playing the game, but more of those people playing on the same playthrough rhythm, I expect that as hiscores continue to show fewer accounts per month, the correlation will also strengthen between average concurrent players and hiscores, simply because more people are playing the same playthrough rhythm.

The OSRS concurrent player counts are there for contrast.

Across all 123 weekly periods, there is almost no relationship with OSRS, which would be expected normally. However, over the last year and over the last six months, the weekly and monthly OSRS correlations are positive. That tells us both games have recently been moving in the same direction, whether from seasonality, botting, or people playing both games at the same time.

The most interesting finding here is the all-time monthly relationship across the last 142 months (roughly 12 years). As RS3 player counts have gone down, OSRS player counts have gone up, suggesting RS3 players have been leaving in droves for OSRS.

Conclusion

Are hiscores a good proxy for concurrent player counts? Yes, but with the understanding of what each is telling us.

I think the two measures are gradually becoming closer indicators over time. Concurrent player counts tell me how many people were online in a given hour or across a given day. Hiscores tell me the total number of accounts that gained XP during a period.

Hiscores may be useful when looking at how rare a rare is during a holiday period. However, concurrent player counts are better when explaining trends over specific hours, days, events, or specific updates. But I think it is safe to say at this point, over the last six months, they have both converged into a measure of player engagement.