Studying Game Design Through Empiricism

There are armchair video game design critics everywhere. Not only is video game design discussed regularly on social media through platforms like YouTube and Twitch, it's also now widely taught as a subject in universities around the world. You might be tempted to think that video game discourse has reached a level of maturity where game design analysis is a settled or exact science. It might be that I'm not actively searching for them, but I have yet to see someone discuss game design critically without basing their examination on their personal point of view. Importantly, I'm not suggesting that subjective critique of art is a problem; I'm pointing out that discussing games critically - especially when analyzing game design - is about far more than simply concluding a game is "good or bad".

Does the past hold up?

If you went on social media in 2024, it was common to hear people - especially content creators - suggest that there weren't any good games released or that they "miss the old days". This rear view mirror perspective, where we continuously anchor analysis to the past, can be a problem in my view. One of the common complaints about my own game design books is that I don't spend enough time analyzing games like Half-Life or Super Mario World.

While there are certainly tentpole games that have large impacts on the industry, I think many of us are too wedded to the notion that game design progress is almost entirely centered on these releases. The more accurate analogy is that of throwing a rock into a pond; every ripple, large or small, leads to new designs and new subgenres.

A common trap for game designers is to think of the past as being a gospel for the future; that older games should be copied as close to 100% as possible in an effort to replicate that earlier success (or with the thought that developers totally got it right back then).

But people tend to misunderstand or forget that these "classic" games were the future and present at some point in time. Super Mario World was to platformers what FTL, Darkest Dungeon, Balatro, and many more are to their respective genres today.

One way to think about this differently is to consider that classic games are more like benchmarks to be cleared rather than goalposts.

Wise developers who want to make games that echo the past will often go beyond those designs in some important respects (maybe the save system is fairer, or there is controller mapping, better onboarding, and so on). When I interview developers who have made "modern retro" titles, one of my favorite takeaways is that they often wanted to recreate a childhood experience but they also wanted to iterate on that original design so that someone who wasn't around in the late '80s or early '90s could appreciate the philosophy of the time but still have a highly enjoyable and playable experience with modern conveniences.

Many classic games were - to be blunt - complete failures from the point of view of user experience design. I want to be clear that this isn't the fault of the original designers; it's just the nature of experiences created at a much earlier time in the industry when it hadn't yet matured to the extent it has now. Considering design decisions within this context is important, especially if we want to understand those decisions and their relative success/failure.

Studying game design as a discipline

I've been discussing this point for almost a decade now: there needs to be a formal appreciation and methodology for game design as its own unique discipline. Game development is different from game design; learning programming won't prepare you for designing a game's systems.

The challenge from my point of view is that nobody has figured out a standard way of studying game design. Some might say you need to read books by experts, others might say you simply have to keep playing video games (or only play the 'best' ones).

But both approaches have their downsides.

Books are inherently bound to the time they are written and published; my first two game design books (on platformers and horror) already feel dated thanks to newer perspectives that were released afterwards. You can't just study what video games were like in a particular year or era, then go and replicate that experience expecting it to perform well today. I recall a conversation I had with Rami Ismail where he explained how hard it is to give people advice about being successful in the industry, partly because it's impossible to predict which games will be successful in the future. If you try to simply remake a game that did well in the past - if you recreate all of the pieces that worked and release it today - it will likely fail. People have already played that game, why would they spend the money to play it again?

And on that second point about simply playing more games...well, playing games day-in and day-out isn't the same as examining them. As I've mentioned many times in the past, I didn't really start to study games until 2012. All the years playing games prior to that really meant nothing. This is why, when I play classic games today, I have newfound insight and appreciation for them that I really wouldn't have had when I played them originally.

More broadly, your ability to analyze game design is somewhat bound by the scope of your experience. In other words, you can't just play the games that everyone else is playing. Doing so will naturally give you a highly curated experience that will limit your game design vocabulary. I played over 1,000 games in 2024, including more than 200 that were released that same year.

So, if you want to study game design - and study it effectively - you need to both understand the methodology and philosophy of a game genre and then examine how a game works within the definitions of said genre. Being able to see why a particular design did or didn't work in one example will help you to understand how those lessons can be applied to other games. With that said, let's shift focus slightly and discuss the difference between a good game and a successful game.

What is a good game?

When I examine a game, my goal is to figure out the developer's intent and whether or not the game's mechanics meet, exceed, or fail to meet that intent. This is why I sometimes express polarizing opinions when I study famous games; I can find something amazing in a game that everyone has written off, and I can find a huge problem in a game that everyone has declared as their own personal best of the year.

This provides an advantage because I can get away from supplanting objective analysis of an experience for my own personal taste (i.e. whether I personally liked or disliked a game). This means I can sometimes wholeheartedly recommend a game that I personally hated and I can argue against games I loved.

The ability to look at a game critically and objectively is something that many people claim to be able to do with some degree of expertise. But importantly, a critical analysis of games isn't just about looking back on an experience and assessing it. Rather, it enables you to start thinking about the next - and even more important - point.

What is a successful game?

As we've established, anyone can at least explain why they liked or didn't like a game. A good reviewer/critic can highlight why a game worked or didn't work in terms of designer intent. But one question that is often not discussed is whether or not a game actually succeeded. You might initially be inclined to doubt this, especially because sales numbers are discussed frequently. But let me drop a bombshell here: a universally-loved game that sold millions of copies may still be considered a failure. In other words, copies sold does not measure the success of a game. This may sound very counter-intuitive. After all, if a game sold millions of copies and was universally loved by hardcore fans...in what sense could it have failed?

The key metric, in my view, is simple: churn rate. If a game sold 1 million copies but only 100,000 people played for more than 30 minutes, I can assure you that the next game by the same studio is unlikely to be another million-copy seller (or anywhere close). By contrast, if a game sold 10,000 copies but 9,000 of those players saw it through to the end, the developer is well-positioned to achieve greater success with their next project.

Churn rate is a critical high-level metric, but how can we dig deeper from here? As a developer, it's important to try to understand the underlying causes of the churn rate. What is keeping players engaged? What's stopping them? Was there a major point in the experience where the game fell apart for a large proportion of players? Digging into these root cause questions in an impartial way is crucial for truly understanding which lessons to learn and how to apply them in your next game.

The difference between a "good game" and a successful game is critical because only a successful game ensures continued growth for the studio (whether through expanding revenue, new fans, or ideally both). You might be producing good games that meet your own metrics for success (or that are critically acclaimed), but you'll be living on borrowed time if this doesn't translate into the kind of success that ensures sustainability and longevity.

I've lost count of the number of studios, just from 2010 to 2020, that had one amazing game, but within a few years had been shuttered.

Sustainability

In the 2010s, I repeatedly discussed variance and RNG in terms of gameplay and replayability. In this decade, the new industry buzzword is "sustainability". Given the horrid year of layoffs and studio closures in 2024, sustainability is a more important factor for developers than ever before.

From the outside, it's easy to view game design and development under the same high-pressure lens of other types of business: that is, it's all about that one product that will make millions for your company, so you do anything necessary to achieve that. But game design and studio health isn't about a single game, but rather, a portfolio of quality games that get people excited about what you're doing next. The failure of big business investors like Embracer Group was due, in part, to them treating game developers like any other business. Almost no studios alive today can create high-quality, hugely-profitable games in a guaranteed way year on year. Likewise, studios are not interchangeable; telling an RPG studio they must develop a reflex-driven, high-octane action game simply won't work.

There's another element to this too: understanding how much investment should go into a single game. The 2010s was a period when many developers who entered the indie space focused on "the dream game". No matter how long it would take, and how much it would cost, the goal was to make that one game. Part of being sustainable in the industry is understanding that, as a solo dev or a small team, one game shouldn't consume the next 5-10 years of your life. Sustainability is therefore, at least in part, about knowing how much to invest in each single idea.

You can theoretically throw an infinite amount of cash at a game concept, but at some point, you'll hit diminishing returns (and ensure that the game is unlikely to ever be profitable). Players generally want polished, memorable experiences they haven't seen before (this doesn't necessarily equate to a 500+ hour slog with every single mechanic and game system one could imagine).

Looking ahead

2024 was a great year for finding new and exciting games to play, but it was a terrible year for game development as a business. I'm a broken record at this point, but I'm hoping we'll continue to see a growing movement around game dev unions, as well as raising the bar in relation to how we discuss and study game design as a discipline.

Game development continues to be a mystery for many, and I think it's important to try to educate developers about the mistakes and issues that can arise before they start working on a project. Small game developers are often putting their lives on the line for these projects, and while it's valuable to "learn by doing", I think it's prudent to anticipate major risks and mitigate them as early as possible to improve chances of success.