[转]通过 SetUnhandledExceptionFilter 来禁止调试的小技巧

src：http://evilcodecave.wordpress.com/2008/07/24/setunhandledexception-filter-anti-debug-trick/

SetUnhandledExceptionFilter() Anti Debug Trick is frequently used, especially in Malware Applications. Around here there are various plugins for Olly that allows the Reverser to trasparently debug this kind of protection, so there is not a real necessity add other words about the mere practical part.

Due to the fact that today, too many young reversers uses a ton of plugins anti – anti – xxx without knowing how internally they works, I decided to expose here SetUnhandledExceptionFilter() Anti Debug Trick from Internals.

First of all, what is SetUnhandledExceptionFilter() ? according to MSDN documentation:

Enables an application to supersede the top-level exception handler of each thread of a process.

After calling this function, if an exception occurs in a process that is not being debugged, and the exception makes it to the unhandled exception filter, that filter will call the exception filter function specified by the lpTopLevelExceptionFilter parameter.

And this is the Syntax:

LPTOP_LEVEL_EXCEPTION_FILTER WINAPI SetUnhandledExceptionFilter(
__in  LPTOP_LEVEL_EXCEPTION_FILTER lpTopLevelExceptionFilter
);

lpTopLevelExceptionFilter is a pointer to top-level exception filter function that will be called whenever the UnhandledExceptionFilterfunction gets control, and the process is not being debugged. A value of NULL for this parameter specifies default handling withinUnhandledExceptionFilter.

Usually, in absence of an UnhandledExceptionFilter the topmost handler called when an uncatched exception occours, is the default one provided by Windows Itself, the classical MessageBox that advices the user that an Unhandled Exception has occured.

But Windows allow programs to use custom Handlers for UnhandledException. The core of the trick is here, if the application isNOT debugged, the application is able to call the Custom Handler, but if the application IS debugged the Custom Handler will be never called.

The possibility of cognitive differentiation make obviously able the target application to apply a series of countemeasures against debugging, from detection to code hidding.

Just remember that due to the architecture of Windows Exception Handling, in every case is called UnhlandledExceptionFilter() function, and this will our point of attack (for anti – anti dbg trick).

This is the general inner meccanism of SetUnhandledExceptionFilter(), going more deep we observe the call stack of the first thread of any Win32 application, we can see that execution in every case is reported to BaseProcess, here the pseudo definition:

VOID BaseProcessStart( PPROCESS_START_ROUTINE pfnStartAddr )
{
    __try
    {
        ExitThread( (pfnStartAddr)() );
    }
    __except( UnhandledExceptionFilter( GetExceptionInformation()) )
    {
        ExitProcess( GetExceptionCode() );
    }
}

The same thing happens for threads, by referencing toBaseThreadStart:

VOID BaseThreadStart( PTHREAD_START_ROUTINE pfnStartAddr, PVOID pParam )
{
    __try
    {
        ExitThread( (pfnStartAddr)(pParam) );
    }
    __except( UnhandledExceptionFilter(GetExceptionInformation()) )
    {
        ExitProcess( GetExceptionCode() );
    }
}

All that happens inside BaseProcessStart() and BaseThreadStart() for what previously said, will be passed to the UnhandledExceptionFilter().

It’s now time to see what really is UnhandledExceptionFilter(), according to MSDN:

An application-defined function that passes unhandled exceptions to the debugger, if the process is being debugged. Otherwise, it optionally displays an Application Error message box and causes the exception handler to be executed. This function can be called only from within the filter expression of an exception handler.

Syntax

LONG WINAPI UnhandledExceptionFilter(
  __in  struct _EXCEPTION_POINTERS *ExceptionInfo
);

Became clear that UnhandledExceptionFilter represents the last choise for processing unhandled exceptions, so the Debugger Presence surely is located inside this function, let’s see a simplified version of this function:

LONG UnhandledExceptionFilter( EXCEPTION_POINTERS* pep )
{
    DWORD rv;

    EXCEPTION_RECORD* per = pep->ExceptionRecord;

    if( ( per->ExceptionCode == EXCEPTION_ACCESS_VIOLATION ) &&
         ( per->ExceptionInformation[0] != 0 ) )
    {
        rv = BasepCheckForReadOnlyResource( per->ExceptionInformation[1] );

        if( rv == EXCEPTION_CONTINUE_EXECUTION )
            return EXCEPTION_CONTINUE_EXECUTION;
    }

    DWORD DebugPort = 0;

    rv = NtQueryInformationProcess( GetCurrentProcess(), ProcessDebugPort,
                                    &DebugPort, sizeof( DebugPort ), 0 );

    if( ( rv >= 0 ) && ( DebugPort != 0 ) )
    {
        // Yes, it is -> Pass exception to the debugger
        return EXCEPTION_CONTINUE_SEARCH;
    }

    // Is custom filter for unhandled exceptions registered ?

    if( BasepCurrentTopLevelFilter != 0 )
    {
        // Yes, it is -> Call the custom filter

        rv = (BasepCurrentTopLevelFilter)(pep);

        if( rv == EXCEPTION_EXECUTE_HANDLER )
            return EXCEPTION_EXECUTE_HANDLER;

        if( rv == EXCEPTION_CONTINUE_EXECUTION )
            return EXCEPTION_CONTINUE_EXECUTION;
    }   

}

As you can see, inside UnhandledExceptionFilter() is calledNtQueryInformationProcess() that has as first parameter our process and next DebugPort, this is done to know if the process is debugged.

All that we have to do to obtain an apparently undebugged process is to modify the first parameter (last pushed at debugging time), in other words we have to change the retur value of GetCurrentProcess()from 0xFFFFFFFF to 0×00000000.

So remember, when you have to overcome a SetUnhandledExceptionFilter() just put a Breakpoint for UnhandledExceptionFilter() and go inside this function to modify the previously exposed parameter 

Thanks to Oleg Starodumov for pseudocodes 

See you to the next blog post..