Managed Databases for Awesome Games

Games? Databases? How do they go together? Aren’t databases what insurance companies use to keep their actuarial tables? Are you asking me to become an enterprise developer?

Honestly, many of you already have a general idea of what databases are, though you might be wondering where they fit in to your game or game development process.

Do I need a database?

Basically anywhere you have data that your game reads or writes lends to the potential to use a database. When you have a central database, your players can access that data from any device they play on. This means players don’t have to worry about transferring save files or remembering crazy unlock codes. It also means your game client could access data from other players to make your game seem more vibrant and “alive” by using real world information to influence game play or allow social interaction, or even allow players to trade in game items. Not to mention it makes it easier to deliver content updates without needing players to download a new executable.

A database is a central, online place where you can store data. Yes, you could just store your data on the disk of your game server, assuming you even have one. And in fact, there are some games (even some major commercial releases) that just store everything in a big flat file on their disk. But, it’s usually a better idea to leave your game server to run the game logic and have a database to actually store the data. And for good reason.

When there are thousands, or even millions, of simultaneous reads against a flat file, it becomes very difficult to make it perform well. Write contention and disk failures increase the potential for unintended corruption. A flat file may not have data redundancy for when things do go wrong, and there is no good strategy to recover from problems. As data grows it becomes slower to access, and it can even grow beyond the capacity of your hardware and code. Flat files are also not inherently searchable, so trying to find data or patterns of data takes a lot of work and messy code.

All of the above (and more) are the reasons why databases were developed in the first place. This kind of tech would be very difficult to duplicate on your own. And besides, you want to write a game, not a database.

Even if you don’t have a game server, a database is still very useful. In this case, the game client would typically communicate with the database more or less directly. In fact, it’s possible to make a turn based multi-player game that is driven only by a central database. If your game isn’t multiplayer, the database can be used to store character data, deliver new content, keep metrics, store settings, player profiles, save games, item inventory, etc.

The list goes on, and this is just a small sampling of how central databases are used in games. They really have a lot of potential to make interesting, new game experiences. And I haven’t even talked about how databases can be used in your production pipeline to make your development life easier! In fact, if you already use source control or a wiki, you’re already using a database. There are numerous benefits to adding a database to your custom tool chain, which are very similar to the points above for your game, including data sharing, consistency, speed and reliability.

SQL vs. NoSQL

You’ve probably heard of NoSQL and might be wondering how it’s different from SQL and what problems it solves. And, more importantly, how is it relevant to games?

SQL is an “older” database technology. Actually, SQL is not really a type of database, it’s a language used to query (ask questions about) a database, and the acronym stands for “structured query language.”

The name SQL often gets conflated with a particular kind of database. SQL is typically used to query a relational database which stores everything in specifically defined tables consisting of rows and columns. The columns represent the attributes of an item, and the items are the rows. Doubtless you’ve encountered tables in your life. A spreadsheet is a good analog to a relational database, and in fact, is usually just a UI on top of a relational database. This design provides some benefits and some drawbacks.

The drawback I’m going to focus on is the inflexibility of this table model. First, the structure of the data must be defined up front (this structure is also known as the schema.) So, you have to specify what each column, or attribute, represents ahead of time.

It’s possible to add columns later, however for every item there must be a value for each column. So, for example, you might want to add an “item” table which would store various in game items like weapons, armor, potions, etc. You now need to make a column for each attribute, like damage, defense and potency. However, those don’t all make sense, for example, why does armor have a damage value?

There are several solutions to this problem. You could create separate tables for each item type, and refer to them from the main item, and try to figure out the shared attributes (cost for example) to try to save space. Or you could use NULL values for columns that don’t make sense. Either way, it’s an extra layer of complexity or bloat.

Now, let’s say your game has been running for a while, and lots of items have been created. You decide you want to add a feature to shields where they can bash other players. Now you actually do want armor to have a damage attribute. What to do? Do you promote damage to be in your item table, even though potions don’t need damage attributes? Do you make a new type of table, a “damage” table that can be referenced from items? Once you’ve done that, how do you update all the existing items that use the old schema? And do this all without breaking anything in the process!

This is where a NoSQL database comes in. Again, this is a bit of a conflation of terms. What most people mean when they say NoSQL is actually a key-value database, document database, non-relational database, object database or some variant thereof. Some of them can even be queried by SQL, making it a misnomer to call them NoSQL databases in the first place! And don’t even get us started on NewSQL.

Terminology aside, the focus here is that these databases don’t require an upfront schema. This means the data can match how your game object attributes are stored, and can easily be changed as attributes are added. So, in our example of representing in game items, every item could just have a list of whatever attributes go with that item, and leave off any that don’t belong. Need to add a new attribute, like damage to our shields? No problem, just update the shield items to now include the damage attribute. Easy peasy!

Of course TANSTAAFL. There are some things that non-relational databases don’t do so well. In particular, ranged queries. Let’s say you are storing players and their score in your database. You want to find out who the 10 players who have the closest score to another player. In a relational database, this is a very fast query as it would have the levels stored in a separate column and it would quickly sort the players accordingly. In a non-relational database you typically have to scan every single player record and keep track of the nearest 10 as you go along. Alas, this is not very efficient or scalable

There is another gotcha for many non-relational databases you may have heard of, though it’s only an issue that happens when you need to have multiple copies of your database, called eventual consistency. This is an important architecture trade off if you want to have databases in different regions to make access faster for various parts of the world. Eventual consistency means that eventually all the data in the separate regions will match, but there’s no guarantees that the data you query from one region won’t be different from another. This is usually solved by having “fast access” where you don’t care if the data is exactly up to date, you just want it fast, and “slower but consistent” access where you’re guaranteed the data is the newest, but all regions need to be checked first. Relational databases on the other hand feature something called ACID which guarantees that operations return consistent data. There are some non-relational databases that offer this via “transactions“, but that’s another topic.

What it boils down to is that for things like player and item attributes, non-relational databases are often a great choice! However for high score tables or other data you want to sort, or inventory where you want no question as to what items a player owns, a relational database would be a good choice.

And finally

I hope you have fun figuring out new and unique uses of databases in your game. There’s a lot of value to be gained from having data centralized online and shareable for your players. Have any questions? Want to tell us what tutorials you’d like to see next? Drop by the AWS subreddit and chat with us!

AWS for Games Blog

Managed Databases for Awesome Games

Do I need a database?

SQL vs. NoSQL

And finally

More reading and next steps

Resources

Follow