Technical Notes: Data Execution Prevention (DEP)

Data Execution Prevention (DEP) is a relatively new feature of both Intel and AMD hardware that is supported beginning with Windows Server 2003 Service Pack 1 and Windows XP Service Pack 2. DEP is a set of related hardware and software features designed to make it more difficult for malicious programs to execute sensitive code in the operating system. DEP is activated automatically when machines with the DEP hardware protection installed are booted in PAE (Physical Address Extension) mode.

Version 2.4.7 of the Performance Sentry (NTSMF) data collector will not execute if DEP is active. This does not apply to versions after 2.4.7

Data Execution Prevention (DEP) is enabled automatically if

• The OS supports DEP
• A 32-bit OS is booted with PAE-mode enabled

Note that DEP is enabled automatically whether or not DEP-enabled hardware is installed if you boot the OS in PAE-mode.

This Technical Note documents the steps you can take for version 2.4.7 and earlier to exclude the NTSMF data collector from DEP enforcement so that it can execute. It also describes the problem symptoms so that you readily identify machines where you need to use this procedure. Newer versions of the NTSMF data collector are able to run without special considerations on DEP hardware.

Background

Hardware-enforced DEP is designed to defeat a class of malicious attacks that attempt to insert and execute code from what should be non-executable memory locations. DEP prevents these attacks by intercepting them and raising a hardware exception.

Hardware-enforced DEP marks pages in memory with an attribute that prevents code on that page from being executed. A new bit in the Page Table entries (PTEs) is used to mark the page in memory as nonexecutable. (In the Intel x64 architecture, this new PTE flag bit is called Execute Disable.) The nonexecutable flag is only supported in 32-bit operating systems when PAE (Physical Address Extension) mode is enabled. PAE is a boot option generally used only for machines that have more than 4 GB of RAM installed.

You can verify that PAE is enabled using the System applet from the Control Panel, as illustrated below:

When PAE is enabled using the /PAE boot.ini switch, the OS builds 64-bit PTEs. The PTE Disable Execute flag bit only exists in the 64-bit PTEs.

The Problem

Processors running Windows Server 2003 Service Pack 1 or Windows XP Service Pack 2 with PAE-enabled that support hardware-enforced DEP raise an exception when the Performance Sentry collection agent first attempts to execute code from a Performance Library DLL.

Error Event ID 209 messages similar to the following are written to the NTSMF log file during initialization for each Perflib DLL that the collector attempts to load:

The unhandled 0xC0000005 Win32 exception is caused by DEP. The Data Execution Prevention feature prevents the collection services from executing code in the Performance Library modules that are loaded. (This example of the error messages that are written is abbreviated.)

These error messages will be followed by a series of Warning messages that will indicate a failure to gather the performance data Objects associated with these Performance Libraries.

08/27/05-14:06:16 - Event ID: 2200, Category: Collection, Severity: Warning
Object, Memory (L4, G4), failed to return data this interval

 

08/27/05-14:06:16 - Event ID: 2200, Category: Collection, Severity: Warning
Object, Cache (L86, G86), failed to return data this interval

 

08/27/05-14:06:16 - Event ID: 2200, Category: Collection, Severity: Warning
Object, Thread (L232, G232), failed to return data this interval

 

08/27/05-14:06:16 - Event ID: 2200, Category: Collection, Severity: Warning
Object, PhysicalDisk (L234, G234), failed to return data this interval

 

08/27/05-14:06:16 - Event ID: 2200, Category: Collection, Severity: Warning
Object, LogicalDisk (L236, G236), failed to return data this interval

 

08/27/05-14:06:16 - Event ID: 2200, Category: Collection, Severity: Warning
Object, Processor (L238, G238), failed to return data this interval

08/27/05-14:08:16 - Event ID: 2200, Category: Collection, Severity: Error
The cycle cannot continue because the following objects are required for cycle start but failed to

 

return data.
System (L2, G2)
Memory (L4, G4)
Cache (L86, G86)
Process (L230, G230)
Thread (L232, G232)
Processor (L238, G238)

Finally, an Event ID 107 Error message will indicate the collection service is suspended.

08/27/05-14:08:17 - Event ID: 107, Category: Initialization, Severity: Warning
The Performance Sentry service is suspended with error code, 0xC00200D1

 

Procedure to override your DEP settings to exclude the Performance Sentry collection service.

To configure an OptOut DEP policy on the computer that will allow the Performance Sentry collection agent to run, follow this procedure:

1. Click Start, click Control Panel, and then double-click System.

2. Click the Advanced tab. Then, under Performance, click Settings.

3. Click the Data Execution Prevention tab.

4. Click Turn on DEP for all programs and services except those I select to select the OptOut policy.

5. Click the Add button and add the applications that you do not want to use DEP with. You can use the Browse button to locate the dmperfss.exe module in the \NTSMF24 root folder that should be excluded from DEP.

Get the Demand Tech Advantage for Your Windows Environment

Find out for yourself why systems professionals who are serious about Windows performance turn to Performance Sentry.