Co-authored by Matthias Neugschwandtner.

Recently, collaborative research efforts between Northeastern University and IBM devised a means to reduce a Windows application’s attack surface by removing unused functions from libraries that are loaded dynamically, known as Dynamic Link Libraries (DLLs). The researchers will present their findings at Black Hat USA 2015.

What Are Dynamic Link Libraries?

Most Windows applications perform common functions, such as displaying a message box. Instead of each developer writing this function independently — which would result in incompatibility and an inconsistent user experience — Microsoft delivers multiple predefined libraries with common functions. The same goes for software development platform/compiler manufacturers and application developers. These are packaged as DLLs.

DLLs are loaded by applications so that they can use the functions. Even simple applications such as Notepad will load a DLL like User32.dll. Notepad then has access to not just the MessageBox function, but to all other functions contained in User32.dll — even though it will not use many of these capabilities.

Attackers Count on Consistency

When an attacker writes an exploit, he or she counts on certain code to exist in memory. For example, the actor may rely on code from the connect() function of the Windows Socket API to establish a communication channel. Therefore, removing all DLL functions from memory that an application will not use reduces the chances that an exploit will work because it decreases the amount of potentially exploitable code.

The Solution

Collin Mulliner from Northeastern University and Matthias Neugschwandtner from IBM devised a tool that iterates through the DLL function dependencies in an application and determines which functions in loaded DLLs are not needed. The tool then overwrites the unused functions’ code in memory with machine code instructions to stop the program. Since DLLs themselves can call other DLLs, it iterates through the entire load chain.

After the tool has stripped unused code, it freezes the code base of the program, i.e., it denies adding or modifying executable memory pages. As a result, attackers can no longer inject additional code into the process.

Application Function Complexities

It is not simple to determine the recursive library function chain, so in its initial incarnation, the solution requires a training period for each application. The application is executed and the solution monitors its activity for a period to determine which functions are called and which are not. Once an application profile has been created, the solution can be configured to strip unused functions, effectively moving from training to protection mode.

Side Effects

There is no runtime overhead after program startup, when stripping and freezing actually take place. Stripping and freezing is also a memory-only operation and does not affect the DLLs on disk. It only affects memory consumption since DLL code pages are stripped individually per application and can thus no longer be shared among multiple processes.

But if you ask us, this is a fair trade-off for security.

To learn more, attend the Black Hat USA presentation Aug. 6 at 3:50pm in South Seas IJ.