Heng Yin Awarded NSF Grant to Fight Against Malicious Code

October 14, 2010

LCS’s Heng Yin Awarded NSF Grant to Fight Against Malicious Code

September 7, 2010

It is a notable achievement to receive a National Science Foundation (NSF) grant. It is a rare and laudable achievement to receive an NSF grant on the first attempt and as a first-year professor who completed a doctorate degree less than a year ago. Yet, that is exactly what Heng Yin, assistant professor of computer science in the L.C. Smith College of Engineering and Computer Science has accomplished.

After completing his Ph.D. in 2009, Yin went straight to work preparing his grant proposal, “Mining Operating System Semantics: Techniques and Applications.” For this proposal, Yin has received a $427,000 grant from the NSF to fight against malicious code.

“Previously I have conducted considerable research on understanding and detecting malicious code,” says Yin. “In this proposal, I switched the analysis target, which is the operating system to be protected against malicious code.”

Operating systems manage hardware resources and provide a higher-level environment for user applications. Operating systems play a central role in computer systems, especially with respect to security and trustworthiness. The growing focus around security makes it crucial to have in-depth knowledge about inner workings of an operating system.

Researchers look to track and analyze events such as: what processes are active in the system, which process is currently running, what modules are loaded into a specific process, which files are opened by a process and which network connections have been established. The knowledge about operating system semantics is the foundation for many computer security applications, such as virtual machine introspection, malware detection and analysis and computer forensics.

However, the existing techniques for obtaining the operating system semantics fall short. They perform static analysis on the operating system source code, and thus cannot be applied to the closed-source operating systems (e.g., MS Windows). The source code analysis also suffers from the WYSINWYX (i.e., What You See Is Not What You eXecute) problem. Furthermore, the obtained semantics knowledge can be easily compromised by various attacks. With such an unsound foundation, the functionality and trustworthiness of these security applications become questionable.

Yin will work over the next three years to build a novel analysis framework to fortify this base knowledge of code analysis. This analysis framework aims to automatically extract the operating system semantics simply from the binary distribution of an operating system and capture invariants, areas of constancy, among these semantics.

The benefit of this framework is that it is binary-centric, and therefore can deal with closed-source operating systems. The identified invariants can also help derive trustworthy semantics knowledge, so various forgery attacks can be detected and thwarted. With this proposed analysis framework, Yin will further investigate how to strengthen the functionality and robustness of several key security applications, including virtual machine introspection, rootkit defense and live memory forensics.

“By analyzing the operating system instead of individual malware instances, we may come up with better defense mechanisms that can defeat entire classes of malware attacks even before new malware attacks are launched,” says Yin.