Arrowhead's Deputy Technical Director discusses the install size of Helldivers 2
Hey, Helldivers. This is the first of what we intend to become a regular series of posts where the engineering team talks about the technical health of the game and some of the technical challenges we’re working through.
Installation Size
The installation size of HELLDIVERS 2 on PC seems to be a hot topic right now so let’s start with that. The current install size on PCs is around 150 GB. This is roughly three times larger than the same game installed on consoles! Given the amount of content in the game, the size on consoles seems quite reasonable so the obvious question is - why is it so large on PC?
Data Duplication
Much of the data in the PC version of HELLDIVERS 2 is duplicated. The practice of duplicating data to reduce loading times is a game development technique that is primarily used to optimize games for older storage media, particularly mechanical Hard Disk Drives (HDDs) and optical discs like DVDs.
This practice is largely unnecessary for games deployed on Solid State Drives (SSDs) which is why the console versions of HELLDIVERS 2 do not do this.
The Problem with Mechanical Hard Drives
The main issue with a mechanical HDD is seek time. An HDD stores data on a spinning platter, and a physical arm with a read head has to move across the platter to find and retrieve data. The time it takes for this arm to "seek" or move to the correct location is a significant performance bottleneck.
Imagine a large game level with various objects - trees, rocks, buildings, props. If the data for these objects is scattered all over the hard drive, the read head has to physically jump around the disk, which adds a lot of time to the loading process.
The Solution: Duplication
To solve this problem, we deliberately duplicate certain data files (like a common tree texture or a sound effect) and place copies of them in physically close proximity to where they would be needed in the game.
For example, our build system will ensure that a copy of a tree texture is stored on the same part of the disk as the level geometry data. When the game loads the level, the read head can access all the necessary information in a single, continuous sweep, without having to "seek" to a different location. This dramatically speeds up loading times.
The Modern Era: SSDs
The need for this technique has largely disappeared with the widespread adoption of SSDs. An SSD stores data on flash memory chips and has no moving parts. This means that "seek time" is virtually nonexistent. An SSD can access any piece of data on the drive almost instantly, regardless of where it is physically stored.
Increasingly, modern games are optimized for the sequential read speeds of SSDs and do not need to rely on the older method of duplicating assets. This is one of the key reasons why new games often explicitly require an SSD in their minimum system specifications.
Should HELLDIVERS 2 continue to optimize for mechanical HDDs?
This is the six-million-dollar question. On the one hand, they are a part of our minimum spec PC requirements. On the other hand - how many HELLDIVERS 2 players are still using mechanical HDDs? The truth is that we don’t currently know. Even the Steam user surveys are unable to give us data on mechanical HDD use in the overall gamer population. Our best estimates put it at around 12% of all PC gamers but the data is very unreliable and relies on a lot of extrapolations. Until we can more accurately determine the number of mechanical HDDs that HELLDIVERS 2 is installed on, it is difficult to know how many players will be impacted by reducing the amount of data duplication. Even if that number is small, keep in mind that the load time for each player dropping into a mission is determined by the slowest member of the squad.
Solutions
While we take steps to gain more clarity on the number of impacted players, we are actively exploring several different solutions in parallel and will begin rolling them out in future updates as soon as they are ready. We cannot eliminate all duplication without making loading times for mechanical HDDs 10 times slower and we do not feel that this is acceptable. There are however some compromises that we can make which will improve the installation size without blowing out the loading times too much.
Short term
We’ve made some small gains in the next update by sweeping our systems for unused assets and obvious problems, but you will not likely notice them because the new stuff we’ve added will eat those gains. It’s not a game-changer but at least the install size will stop growing.
Medium term
Beyond the next update, we’re exploring taking some of the worst offenders in terms of duplication and de-duplicating them by putting them in “very common assets” bundles which will always be loaded under set conditions (eg- specific faction/biome). This does mean that loading times will get a bit worse for players using mechanical HDDs - it is unfortunate but unavoidable. Our early testing shows we should be able to keep this in the range of “less than 30 seconds” rather than “several minutes”. It can also increase the amount of RAM used by the game by loading “common” data that is not always needed. If we don’t make “worst case” RAM usage worse, this should not be noticeable, but it is a risk we are monitoring. By doing some careful measurements and analysis, we should be able to deliver acceptable tradeoffs between RAM usage, loading times and install size.
Long term
Looking further into the future, we plan to make improvements to the engine which will ensure we never waste RAM loading common data that isn’t needed - eliminating one of the drawbacks of the above technique.
Beyond that, the remaining work is a bigger, riskier, more speculative project where we apply some kind of compression to the game data and potentially replicate some of the de-duplication we do on consoles. We don’t yet know if the impacts to load-times could make these approaches infeasible.
Optional 4k Textures?
Could we create a solution where the highest resolution textures are an optional download? Technically yes - anything is possible. It is not something that is natively supported in our engine though. It would be a substantial project to add this capability. Due to the scope and complexity of the changes we would have to make, this is not our first preference and is honestly something we would only consider if we’re unable to make a big enough impact with our other solutions. Nothing comes for free - time spent making these changes is time not spent optimizing the performance of the game or fixing stability issues.
Summary
So in summary - we’re taking your concerns very seriously but there are no easy solutions. Until we live in a world where we know that most of our PC players are using SSD drives, sacrificing some extra hard drive space is necessary to ensure we’re all able to load into missions in a reasonable amount of time. We’ve clearly reached the limits of how much duplicated data is acceptable so smarter solutions and compromises are now required. We are very carefully weighing up the costs and tradeoffs of the options we have, and we’ll be sure to find a better balance between loading times and installation size soon.
I hope you enjoyed this deep dive into our tech. If you have any questions about this topic or suggestions for future topics, please reply to this thread.
Thanks for reading, we’re always eager to hear from great engineers and gameplay coders: https://jobs.arrowheadgamestudios.com/
