Rootkits

An Emerging Threat YouCan’t Afford to Ignore

How Trend Micro Can ProtectYour Small/Mid-size Business

A Trend Micro White Paper I November 2006

Small and Medium Business

CONTENTI. THE DANGERS OF ROOTKITS......................................................................................................

II. GROWTH IN USE OF ROOTKITS TIED TO PROFIT-BASED THREATS...............................

III. HOW ROOTKITS WORK...................................................................................................................

IV. DEFENDING AGAINST ROOTKITS................................................................................................

• Bypass operating system functions

• Monitor system behavior

V. TREND MICRO TECHNOLOGY DEFENDS AGAINST ROOTKITS..........................................

VI. APPENDIX A: A BRIEF HISTORY OF ROOTKITS......................................................................

VII. APPENDIX B: A ROOTKIT IN ACTION..........................................................................................

VIII. APPENDIX C: ROOTKITS IN THE OPERATING SYSTEM........................................................

2 White Paper I Rootkits

3

4

4

5

5

5

6

8

ROOTKITS

3 White Paper I Rootkits

Just when you thought your network was safe:

“Microsoft security researchers are warning about a new generation of powerful system-monitoring programs, or ‘rootkits,’ that are almost impossible to detect using current security products and could pose a serious risk to corporationsand individuals.”

“Be afraid, be very afraid”Computerworld, February 17, 2005

In this paper, we’ll give you an overview of the danger that rootkits pose to you and your business, and propose some simple steps you can take to address this emerging threat.

WHAT IS A ROOTKIT?

In simplest terms, a rootkit is a softwareprogram that can hide computer malwareon a PC. Virtually every SMB deploys asecurity program to protect against viruses,worms, Trojans, and spyware—all the nasty threats that are roaming the Internet,looking for a way into your network. Thisdefense strategy is mature and reliable,and—assuming you keep your signaturefiles updated—provides a high level ofprotection against incoming threats.

You may have heard of the flap overSony’s attempt to enforce copy protectionon CDs in 2005 1. That was a case of awell-meaning enterprise underestimatingthe impact of using rootkits, a stealthtechnology that has been around since theearly days of the UNIX operating system.But far more malevolent organizations areusing rootkits with malice, intending toprofit from you and your business.

But what if the threats are cleverly hiddenfrom the security program? If the securityprogram can’t locate them, it can’t defendagainst them. That’s why today’s hackersare using rootkits as one of their primarytools to defeat your security defenses. And once your defenses are breached,your business is an open book to hackers.

II. THE DANGERS OF ROOTKITSSuppose you left your company’s front door wide open, wrote your computerpasswords on sticky notes and put them on every keyboard—and went onvacation for a month.

Whatever you can imagine happening in that scenario, that’s the potentialdanger that a rootkit poses. An intruder could steal your entire customerdatabase and send malware to every email address. Or thieves could usebank account numbers to siphon funds from customers, suppliers, employeesand even your own business. Competitors could steal your intellectualproperty—engineering drawings, source code, process formulas—that tookyears and millions of dollars to develop.

As you can imagine, a successful rootkit has the potential to cost yourbusiness dearly. Suppose an attacker uses spyware cloaked by rootkits tosteal personal information for your best customers. In an attempt to mitigatethe damage, you may be forced to compensate your customers out of yourown pocket, an unexpected and unwelcome expense. And the financialconsequences can be far worse if one of those customers launches a civillawsuit, or your business is subject to fines from a regulatory agency.

Beyond money, the damage to your company’s reputation can be irreparable.You know how hard it is to gain and keep customers by earning their trust. Amajor security breach can undo all that work—in an instant.

1 “As a copy protection measure, Sony BMG included the Extended Copy Protection (XCP) and MediaMax CD-3 software on music CDs. This software was automatically installed on desktop computers when customers tried to play the CDs. The software interferes with the normal way in which the Microsoft Windows or Mac OS X operatingsystems play CDs, opens security holes that allow viruses to break in, and causes other problems.” From Wikipedia entry “rootkit”.

ROOTKITS

4 White Paper I Rootkits

II. GROWTH IN USE OF ROOTKITS TIED TO PROFIT-BASED THREATSTo understand the motivation behind rootkits, let’s look at the way threats have evolved over the last decadeor so. At first, attacks were intended to disrupt—a network, a business, a personal computer. The agents ofthese malicious activities, usually viruses, infected as many machines as possible in the shortest period oftime. The payoff? Notoriety, even if only a handful of the virus writer’s closest friends knew who was behindthe attack. It’s similar to vandalism: One member of a gang breaks car windows to gain the admiration andapproval of the others.

But a funny thing happened to hacking: It went commercial. Hackers now are looking for ways to makemoney, not fame, by stealing credit card and bank account numbers, intellectual property, customer namesand addresses—anything that has commercial value or can be exploited for financial gain. Others havechanged from disrupting e-commerce for sheer fun or malice to denying service on Web sites as a form of extortion or blackmail. Using the earlier analogy, it’s as though the gang switched from vandalism tostealing cars.

Profit motives are behind the increased use of spyware and adware, malware programs that can detectsensitive information and relay it to unknown third parties outside your company. But to do so, they need to remain undetected.

If your company takes orders by phone, your service representatives probably enter customer credit cardnumbers into your order management system. Say your computer is infected with a keylogger—a lessknown form of spyware—that is designed to surreptitiously record those credit card numbers and relay them to a thief on the outside. The longer the keylogger remains undetected, the more credit card numbersit can steal, and the more money the thieves make. It’s simple economics.

That’s where rootkits help the attackers. Instead of just installing the keylogger, the intruder “wraps” thekeylogger inside a rootkit. By compromising the computer’s operating system files so that the user—andmany security programs—can’t see the malware, the rootkit ensures that the keylogger remains on yourcomputer. Meanwhile, your customers are getting calls from their credit card companies asking aboutcharges from Sao Paolo or Bucharest—and tracing their problems back to your company.

The rootkit didn’t steal the credit card numbers. It just hid—or “cloaked”—the spyware that did!

III. HOW ROOTKITS WORKRootkits compromise the operating system in ways that hide the spyware program. The rootkit may alterWindows Explorer, the part of the operating system that presents the user interface on the monitor andenables the user to control the computer. When the rootkit alters Windows Explorer the spyware programdoesn’t appear in a directory listing. Or the rootkit could intercept a security program that’s trying to examinethe hard disk sectors where the spyware resides, and direct it instead to another, noninfected, part of thedisk. The security program doesn’t detect anything unusual—and the spyware remains intact.

While the technology of rootkits is complex, there are resources to help even inexperienced hackers usethem. The source code for working rootkits is readily available on the Internet. While these “starter” rootkitsare not fully optimized for stealth, they can still be highly effective against unprotected networks.

ROOTKITS

5 White Paper I Rootkits

IV. DEFENDING AGAINST ROOTKITSRootkits spread by the same techniques as viruses and other threats, through email, instant messaging, Websites—anything that comes from outside your organization. Fortunately, Trend Micro is one of the few securityvendors that offer gateway and endpoint solutions that are highly effective in intercepting rootkits before they can attack operating systems. The best security solutions include two advanced strategies for defeating rootkitcloaking techniques:

Bypass operating system functionsAs described earlier, rootkits often modify operating system functions to hide themselves. A security solution designed to stop rootkits bypasses vulnerable operating system functions to generate its owndirectory listings revealing the rootkits and associated spyware.

Monitor system behaviorAs more rootkits are deployed and detected, security vendors learn more about the common behaviorsthat rootkits use. They can use this behavioral knowledge to improve the rootkit detection capabilities of their security solutions.

V. TREND MICRO TECHNOLOGY DEFENDS AGAINST ROOTKITSTrend Micro has studied rootkits extensively and developed sophisticated detection and removal techniques to protect your data. Trend Micro’s technology can see past the effects of corrupted operating system layers or compromised Windows APIs. This “x-ray vision” provides an accurate way to defeat the defenses used byrootkits. Once a security program can see the threats that the rootkit is trying to cloak, it can remove them.

To protect your business against this growing threat, contact your Trend Micro reseller, or visit our Web site at: www.trendmicro.com

VI. APPENDIX A: A BRIEF HISTORY OF ROOTKITSHiding and concealing an infection has a long history. In the 1980s, Brain and Frodo were the first viruses to modify the IBM DOS operating system to hide the infection from the suspecting user. Brain would simplyredirect any attempt to read the boot-sector of an infected disc to another area on the disc, Frodo, a file-infector virus modified the dir-command in a way that it would display the original file-size of infected files, thus concealing the infection.

In the following years, malware such as Trojans used stealth techniques to hide their presence on the affectedsystems and remain installed a longer time. Also, instead of modifying single tools or built-in commands of theoperating system, they modified the operating system function calls. This is a much more efficient approach as it fools all applications using this function call, including some antivirus solutions. This process, known as“hooking a function,” can either completely divert the function to a replacement or–more commonly–add a call to the malicious code before or after the original function code is being called. That way, all traces of themalware file’s presence in a directory could be removed from the function's return data, making it virtuallyinvisible to any application using the operating system’s directory function.

ROOTKITS

6 White Paper I Rootkits

VII. APPENDIX B: A ROOTKIT IN ACTIONThis section shows an example of how a rootkit can cloak the presence of malware. In this example,BKDR_HACDEF.A-73 is a virus that is wrapped inside a rootkit. Figure 1 shows the Windows directory listing before the rootkit executes.

Figure 1. Directory listing before execution of the rootkitNote that the malware file is clearly visible to the user

When the malware is executed, it modifies the Windows function call that lists directory contents, hidingitself as shown in Figure 2.

Figure 2. Directory listing after rootkit executionNeither Windows Explorer nor the “DIR”-command show the virus now

ROOTKITS

7 White Paper I Rootkits

However, in this case the file is not completely hidden. Since we have prior knowledge of the filename,we can open it in Notepad, a Windows function not compromised by the rootkit, as shown in Figure 3.

Figure 3. Notepad listing of the contents of the malware file.The user would have to know of the file’s existence beforehand and the exact file name to use Notepad in this way

In addition, if this folder or drive is being shared, its content is displayed on any remote machine accessing it as shown in Figure 4. This occurs because the rootkit’s actions are local and do not affect the Windowsfunctions on the remote machine.

Figure 4. Listing on remote computer sharing this directoryBecause the rootkit did not affect the Windows functions on the remote system, the malware file is still visible there

ROOTKITS

TREND MICRO INC.

10101 N. De Anza Blvd.Cupertino, CA 95014USA toll free: 1+800-228-5651phone: 1+408-257-1500fax: 1+408-257-2003www.trendmicro.com

©2006 by Trend Micro Incorporated. All rights reserved. Trend Micro, the Trend Micro t-ball logo, InterScan, OfficeScan, ScanMail, ServerProtect,Trend Micro Control Manager, TrendLabs, and VirusWall are trademarks or registered trademarks of Trend Micro Incorporated. All other companyand/or product names may be trademarks or registered trademarks of their owners. Information contained in this document is subject to changewithout notice. [WP03Rootkits_070104US]

8 White Paper I Rootkits

TREND MICRO™

Trend Micro Incorporated is a pioneer in secure content and threatmanagement. Founded in 1988, Trend Micro provides individualsand organizations of all sizes with award-winning security software,hardware and services. With headquarters in Tokyo and operationsin more than 30 countries, Trend Micro solutions are sold throughcorporate and value-added resellers and service providers world-wide. For additional information and evaluation copies of TrendMicro products, visit our Web site at www.trendmicro.com.

VIII. APPENDIX C: ROOTKITS IN THE OPERATING SYSTEMModern operating systems provide a more secure environment to applications by setting them apart andrestricting their access to critical resources. Every application is contained within a “memory-cage” thatprevents it from accessing or modifying the memory of another application. Such applications are called“user-mode” processes, as shown in Figure 5.

The core of the operating system and modules granting access to the hardware (also known as drivers) run in kernel mode. Kernel mode processes can access and modify all of the memory on the system, including the operating system itself.

Figure 5. Operating System Security Layers

Rootkits can be either user mode or kernel mode. The ultimate goal of any rootkit-writer is to create a kernelmode rootkit, since these have the best access to the system. However, kernel programming requires sophis-ticated knowledge and experience that is out of the reach of many hackers. Also, installation and distribution of a kernel mode driver or module is difficult. Therefore, most rootkits target the user mode, more specificallythe Windows API. Hooking Windows API functions is easier to do and requires fewer programming skills.

Trend Micro anti-rootkit technology defeats rootkits by bypassing operating system functions, allowing securityprograms to find and remove malware.

ROOTKITS

KERNEL-MODE

USER-MODE

Ring 1

Ring 2

Ring 3

Ring 0

Gate