This 11-Byte TLS Request Could Lock Up Your OpenSSL Server's Memory

ยท
Listen to this article~4 min
This 11-Byte TLS Request Could Lock Up Your OpenSSL Server's Memory

An 11-byte TLS request can lock up 131 KB of memory on unpatched OpenSSL servers, causing a denial-of-service vulnerability named HollowByte. Okta's Red Team discovered the flaw, and OpenSSL fixed it silently in June without a CVE or advisory. Here's what you need to know.

Imagine sending a tiny 11-byte request and having your server lock up 131 KB of memory for a message that never arrives. That's exactly what the HollowByte flaw does to unpatched OpenSSL servers. And here's the kicker: on systems using glibc (which is most Linux systems), that memory stays gone until you restart the entire process. Okta's Red Team discovered this denial-of-service vulnerability and gave it the name HollowByte. They found that by sending a specially crafted TLS handshake message, an attacker could trick OpenSSL into reserving a huge chunk of memory for data that would never come. The server just sits there, holding onto that memory like a friend who won't let go of an inside joke. ### What Exactly Is HollowByte? HollowByte isn't your typical buffer overflow or remote code execution bug. It's a memory leak that turns into a denial-of-service weapon. Here's how it works: - An attacker sends an 11-byte TLS request to an unpatched OpenSSL server. - The server allocates up to 131 KB of memory, expecting more data to follow. - That additional data never arrives, but the memory is never freed. - On glibc systems, that memory is effectively lost until the process restarts. Over time, repeated attacks can exhaust the server's available memory, causing crashes or making it unresponsive. It's like slowly draining a water tank one cup at a time. ### The Silent Fix OpenSSL actually shipped the fix for HollowByte back in June, but they did it without any fanfare. No CVE was assigned. No security advisory was issued. No changelog entry pointed directly at the vulnerability. This has raised some eyebrows in the security community. Why the silence? It's hard to say. Maybe they thought it wasn't critical enough. Maybe they wanted to avoid panic. But for system administrators, this is a problem. If you don't know a fix exists, you won't apply it. ### What You Need to Do If you're running OpenSSL, here's what matters: - **Check your version**: The fix was included in OpenSSL 3.0.x and 1.1.1x releases from June onward. Make sure you're on the latest patch level. - **Don't rely on CVEs**: This is a reminder that not all security fixes get publicized. Keep your software updated regularly, not just when a CVE pops up. - **Monitor memory usage**: If you see unexplained memory growth on your TLS servers, HollowByte could be the cause. ### The Bigger Picture HollowByte is a good example of why security isn't just about preventing data breaches. Sometimes, the goal is to keep the lights on. A denial-of-service attack can be just as damaging as a data leak, especially if it takes down critical infrastructure. Okta's Red Team did the right thing by reporting the bug responsibly. But the lack of transparency from OpenSSL around the fix is concerning. Security relies on trust, and that trust is built through clear communication. ### Final Thoughts An 11-byte request shouldn't be able to bring down a server. But with HollowByte, it can. The fix is out there, but you have to know to look for it. Update your OpenSSL installations, monitor your systems, and stay curious. Because sometimes, the quietest bugs cause the biggest headaches.