Storing trillions of your Discord messages is waaaaay harder than you thought
On big public Discord servers, most of the the displayed messages are recent and cached which is good for performance. But smaller servers used by just a few friends can often involve old messages that aren't cached being displayed or read from the database every time a user logs in.
According to numbers from 2019, Discord handled 25 billion messages per month.
The specifics of the company's struggles have been outlined in forensic detail in a blog by Discord's Bo Ingram(opens in new tab), a senior software engineer for the platform.
On big public Discord servers, most of the the displayed messages are recent and cached, which is good for performance. But smaller servers used by just a few friends can often involve old messages that aren't cached being displayed or read from the database every time a user logs in.
According to numbers from 2019, Discord handled something like 25 billion messages per month(opens in new tab), 850 million every day, and 600,000 per minute. You can expect those numbers to be even bigger now.
Discord started off using the MongoDB database to support this activity, but then moved to Cassandra. Its appeal was as a NoSQL database that had support for clustering, which means you can have multiple instances of a given database working as one to improve performance.
Both Apple and Netflix, among other big tech companies, both reportedly use Cassandra. So, it's got proper blue chip cred. Surely it's good enough for Discord? Initially, yes. Once Discord had ironed out a few issues, the Cassandra databases were writing new messages within one millisecond and read back old messages to users in five milliseconds. Quick enough for your typical text chat, that's for sure.
But the good times didn't last.
Read more on pcgamer.com
