How Do You Break Into a Program? Give It More Than It Asked For
By the BrainSnail editorial team. How these articles are written and checked, and how to tell us when one is wrong.
Writing more data into a space than that space was sized for overwrites whatever sits next to it in memory, and for decades that was the commonest way to take over a computer.
What goes wrong
A program sets aside a region of memory of a fixed size to hold something, such as a name typed by a user, and then copies incoming data into it. If the amount copied exceeds the size reserved, and nothing checks, the extra is written past the end of the region into whatever memory lies immediately beyond, overwriting it. That neighbouring memory holds other variables and, crucially, bookkeeping information the program relies on. The language matters here, since some languages check every access and make the fault impossible, while others do not check and trust the programmer entirely.
Why it becomes a takeover
Overwriting one particular value turns a crash into control:
- •A program records where to resume after a function finishes
- •That record sits in memory near the space being overflowed
- •Overwriting it changes where the program goes next
- •An attacker sets it to point at data they supplied
- •That data is then executed as instructions
- •So arbitrary code runs with the program's own privileges
What was done about it
A sequence of defences was added over two decades and each raised the difficulty without closing the hole. Marking memory that holds data as non-executable stopped supplied data being run directly, so attackers responded by stitching together fragments of the program's own existing code instead. Randomising where things sit in memory each time a program runs made it hard to know what address to jump to, so attackers responded by finding ways to leak that information first. Adding a known value before the critical record and checking it before use detects an overwrite. Compilers warn about unsafe functions. None of that is complete, and combining them is what works.
The variants of the same fault
The basic mistake appears in several forms and the names distinguish where the overwritten memory sits. Overflowing space reserved on the working area used by functions is the classic form and the one that reaches the record of where to resume. Overflowing space allocated dynamically corrupts the bookkeeping structures that track allocations instead, which can be turned into an arbitrary write. Reading past the end rather than writing leaks memory contents, which is how a well known flaw exposed private keys from servers. Miscalculating a size so that it wraps around to a small number produces the same result by a different route.
Why it persists
The class of fault has been understood since at least 1988, when a program exploiting one spread across a large part of the early internet, and it still appears in software written today, which needs explaining. Enormous quantities of existing code are written in languages that do not check, and rewriting them is expensive and risky. The mistake is easy to make and hard to see in review, since a bound that is correct in every tested case can fail on an input nobody thought of. And the pressure to ship keeps producing it. Languages that check by design are being adopted for new work specifically to remove the category, which is the only approach that eliminates rather than mitigates it.
The takeaway
Copying more data than a reserved region holds writes into whatever sits beyond it, and one of the things sitting there records where the program resumes next, so overwriting it directs execution to data the attacker supplied. Non-executable memory, randomised layouts and guard values each raise the difficulty without closing it. Languages that check every access remove the category rather than mitigating it.