.Net CLR assembly and stack dump and SQL Crash…   2 comments


This post is a result of another thread on MSDN forums I answered the other day. I thought to blog about it from a SQL perspective. SQL Server 2005 SP3 x64 crashes generating a stack dump.

<Date Time> spidxx       Using ‘dbghelp.dll’ version ‘4.0.5’

<Date Time> spidxx       **Dump thread – spid = xx, PSS = 0x0000000080A23F10, EC = 0x0000000080A23F20

<Date Time> spidxx       ***Stack Dump being sent to C:\Program Files\Microsoft SQLServer\MSSQL.1\MSSQL\LOG\SQLDump0023.txt

<Date Time> spidxx       * *******************************************************************************

<Date Time> spidxx       *

<Date Time> spidxx       * BEGIN STACK DUMP:

<Date Time> spidxx       *   08/14/12 14:16:11 spid xx

<Date Time> spidxx       *

<Date Time> spidxx       * A fatal error occurred in .NET Framework runtime.

<Date Time> spidxx       *

<Date Time> spidxx       * Input Buffer 86 bytes –

<Date Time> spidxx       *             [dbo].[My_sproc]’P1′

From the stack dump, I see the below call stack:

                Child-SP          RetAddr           Call Site

00000000`18a6b5f0 00000000`018c783c kernel32!RaiseException+0x73

00000000`18a6b6c0 00000000`01f6dd03 sqlservr!CDmpDump::Dump+0x7c

00000000`18a6b710 00000000`01f70fb7 sqlservr!CImageHelper::DoMiniDump+0x413

00000000`18a6b870 00000000`02085e93 sqlservr!stackTrace+0x6e7

00000000`18a6cda0 00000000`02085fe1 sqlservr!CHostPolicyManager::OnFailure+0x63

00000000`18a6cdf0 00000642`7f87fb75 sqlservr!CHostPolicyManagerWrapper::OnFailure+0x31

00000000`18a6ce40 00000642`7f9fa4bb mscorwks!EEPolicy::GetActionOnFailure+0x95

00000000`18a6ceb0 00000642`7f44b681 mscorwks!EEPolicy::HandleFatalError+0x2b

00000000`18a6cf00 00000642`7f47b7a0 mscorwks!CLRVectoredExceptionHandlerPhase3+0xcd

00000000`18a6cf40 00000642`7f47b9a7 mscorwks!CLRVectoredExceptionHandlerPhase2+0x30

00000000`18a6cfb0 00000642`7f44b586 mscorwks!CLRVectoredExceptionHandler+0xff

00000000`18a6d030 00000000`77f2583c mscorwks!CLRVectoredExceptionHandlerShim+0x42

00000000`18a6d070 00000000`77ee58a6 ntdll!RtlpCallVectoredHandlers+0x26f

00000000`18a6d100 00000000`77ef2c2d ntdll!RtlDispatchException+0x46

00000000`18a6d7b0 00000642`7f9f7459 ntdll!KiUserExceptionDispatch+0x2d

00000000`18a6dd50 00000642`78acb793 mscorwks!COMCryptography::_DecryptData+0x329

00000000`18a6df90 00000642`802da474 mscorlib_ni!System.Security.Cryptography.CryptoAPITransform.TransformFinalBlock(Byte[], Int32, Int32)+0x123

00000000`18a6e020 00000642`80620282 0x642`802da474

00000000`18a6e0e0 00000642`802d9ab9 0x642`80620282

00000000`18a6e140 00000642`7f5ff9ea DomainBoundILStubClass.IL_STUB(Int64, Int64)+0x19

00000000`18a6e170 00000000`01d30fb1 mscorwks!UMThunkStubAMD64+0x7a

00000000`18a6e200 00000000`020834bf sqlservr!CallProtectorImpl::CallWithSEH<AppDomainUserCallTraits,void,FunctionCallBinder_2<void,void

00000000`18a6e240 00000000`0229488c sqlservr!CallProtectorImpl::CallExternalFull<AppDomainUserCallTraits,void,FunctionCallBinder_2<void,void

00000000`18a6e350 00000000`027a98af sqlservr!CAppDomain::InvokeClrFn+0x9c

00000000`18a6e400 00000000`016d5cf4 sqlservr!UDFInvokeExternal+0x6cf

00000000`18a6e6f0 00000000`016d2149 sqlservr!CQScanProjectNew::GetRow+0x3d4

00000000`18a6e8e0 00000000`016d1a21 sqlservr!CQueryScan::GetRow+0x69

00000000`18a6e920 00000000`016c5a93 sqlservr!CXStmtQuery::ErsqExecuteQuery+0x281

00000000`18a6eab0 00000000`02775e57 sqlservr!CXStmtSelect::XretDoExecute+0x103

00000000`18a6eb80 00000000`01be3133 sqlservr!UM_LoopbackForStatementExecution+0xf7

00000000`18a6ec70 00000642`7fa01914 sqlservr!AppDomainCallback<FunctionCallBinder_5<void,void

00000000`18a6ecb0 00000642`7fa85776 mscorwks!ExecuteInAppDomainHelper+0x94

00000000`18a6ed30 00000000`01d31038 mscorwks!CorHost2::ExecuteInAppDomain+0x226

00000000`18a6ef20 00000000`01ea5907 sqlservr!CallProtectorImpl::CallWithSEH<AppDomainCallTraits,long,MethodCallBinder_3<long,ICLRRuntimeHost,long

00000000`18a6ef60 00000000`0228b87b sqlservr!CallProtectorImpl::CallExternalFull<AppDomainCallTraits,long,MethodCallBinder_3<long,ICLRRuntimeHost,long

00000000`18a6f000 00000000`0229e9c0 sqlservr!CAppDomain::LoopbackForStatementExecution+0x16b

00000000`18a6f0d0 00000000`01adf9f2 sqlservr!CXStmtQuery::XretCLRExecute+0xc0

00000000`18a6f140 00000000`01708f59 sqlservr!CMsqlExecContext::ExecuteStmts<1,1>+0x3d89c2

00000000`18a6f310 00000000`0170b9c5 sqlservr!CMsqlExecContext::FExecute+0x439

00000000`18a6f460 00000000`017053e2 sqlservr!CSQLSource::Execute+0x355

00000000`18a6f5a0 00000000`0170a361 sqlservr!process_request+0x312

00000000`18a6f830 00000000`016bc32e sqlservr!process_commands+0x1c1

00000000`18a6faf0 00000000`016bc5c9 sqlservr!SOS_Task::Param::Execute+0xee

00000000`18a6fc00 00000000`016b5cc4 sqlservr!SOS_Scheduler::RunTask+0xc9

00000000`18a6fc90 00000000`0155e837 sqlservr!SOS_Scheduler::ProcessTasks+0xb4

00000000`18a6fd00 00000000`01522f59 sqlservr!SchedulerManager::WorkerEntryPoint+0xe7

00000000`18a6fda0 00000000`015372b0 sqlservr!SystemThread::RunWorker+0x59

00000000`18a6fde0 00000000`014f72f8 sqlservr!SystemThreadDispatcher::ProcessWorker+0x130

00000000`18a6fe80 00000000`781337d7 sqlservr!SchedulerManager::ThreadEntryPoint+0x128

00000000`18a6ff20 00000000`78133894 msvcr80!_callthreadstartex+0x17

00000000`18a6ff50 00000000`77d6b71a msvcr80!_threadstartex+0x84

00000000`18a6ff80 00000000`00000000 kernel32!BaseThreadStart+0x3a

The exception is raised at mscorwks!COMCryptography::_DecryptData . The function call at 0x642`802da474 doesn’t appear since that is part of managed call. But it does look like this function triggered the issue. If I closely look at the Errorlog again, I see the following lines just before the stack dump was generated:


<Date Time> spidxx       Common language runtime (CLR) functionality initialized using CLR version v2.0.50727 from C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\
<Date Time> spidxx       AppDomain 2 (EES.dbo[runtime].1) created.
<Date Time> spidxx       Unsafe assembly 'Faulty_Assembly, version=0.0.0.0, culture=neutral, publickeytoken=null,processorarchitecture=msil' loaded into appdomain 2 (EES.dbo[runtime].1)

I see CLR assembly named Faulty_Assembly was loaded here.Sounds like a good candidate to be blamed 😉

Searching over internet on function name mscorwks!COMCryptography::_DecryptData takes me to Carlos’s post where he seems to be describing almost the similar stack. However, in this case it turned out that this assembly was compiled for 32-bit code and recently SQL server was migrated to a 64-bit OS. Due to this incompatibility, System.Convert.FromBase64String was returning NULL hence the issue. Once this assembly was compiled for ANY PLATOFRM (or 64-bit) it ran just fine.

Conclusion: Make sure you care about CLR assemblies while migrating SQL Server across different CPU architecture.

For managed debugging, refer to Tess’s blog

Stay tuned for the next debugging tips!

2 responses to “.Net CLR assembly and stack dump and SQL Crash…

Subscribe to comments with RSS.

  1. Thanks for posting this nice blog on SQL CLR assemblies.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: