Fortnite's backend took a year to scale to 15 million users

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:26:52

   Can you actually speak to the technical challenge of going from mostly and not online, large-scale gaming platform to being able to support with Battle Royale a huge number of people playing with each other at the same exact time? Like, what's the technical? I mean, four weeks, what's the technical challenges there that had to be overcome? Since 2012, we've been building online backend systems to support player accounts and login, and all of the different systems there needed to make a multiplayer game, and we've been building them to be scalable. And by some miracle, we built them stably enough that they were able to scale up. And so the online team was responsible for patching that code, spent a year of intense work getting it to scale from like 40 000 concurrent users to 15 million concurrent users

2. 2:27:44

   yeah i mean there's scaling there's scaling that's a lot that's it's immense but they'd done such an awesome job of building the foundations that uh it was tractable it was doable if it if they hadn't done that then the company would have died that you know fortnite just wouldn't have been playable and the whole thing would have failed i mean there's just so much detail there that makes all the difference because i mean uh that's what spotify has talked about that like the latency it's like how quickly you can deliver the song changes the product from being this shitty thing that i'd rather pirate the songs to like this is good enough to where i really enjoy the experience i want to use it So like, yeah, that's really important to create an experience for 15 million concurrent users to where they're not, where it's not lagging, where it actually works, right?

3. 2:28:41

   Is there something you could say to the server, like how difficult that is to pull off? You know, the trend nowadays for building online services is microservices. There's not one big server that handles all of the interactions with Fortnite. There's game servers running 100-player game instances for each Battle Royale session, and then there's an account server and many instances of it all talking to a shared database, and there's hundreds of different microservices talking to each other.

4. 2:29:11

   And so scaling is a matter of identifying what are the bottlenecks in that system and making sure that each one can scale and has enough redundancy to be able to handle the load. Thank God for Amazon Web Services and cloud hosting, because Epic went to 15 million concurrent users without buying any server hardware. We were able to just call up Amazon and say, we need more. And there was a period of time there where Fortnite was undergoing this exponential growth.

5. 2:29:39

   And we'd find like one week we ran out of servers in Brazil during a heavy weekend of play. And next week we had an even heavier weekend of play and there were servers to handle it. Like somebody at Amazon had drop shipped, you know, millions of dollars of server hardware into brazil and turned it on just in time for fortnite to need it um and you know there are a lot of unsung heroes uh in that story many of them we've never heard of yeah i mean behind aws many unsung heroes there's like so much so much of those folks who run the modern the modern internet all the incredible services the games the services that we take for granted are currently being run on aws or were originally and google cloud and so on yeah uh can you speak to how much