Generating and analysing Post Mortem memory dump files

Hi,

This week, a client of ours experienced their IIS Application pool crashing, and this occurred several times during the week. For months they could not figure out what was causing the problem.

We could not reproduce it either.

So the first step was to attach ADPLUS to the IIS process that was crashing, and then configure ADPlus to catch the specific exception we were having.

The only clue we had in the event log was this exception code, without any additional information, except that the Kernel.dll was involved.

Exception Code: 0xe053534f

Armed with our new found knowledge, I downloaded the debugging tools for windows and installed them on the server, I made sure I only selected the debugging tools in the common section and deselected everything else, due to it being a production server (64 bit version):

http://www.microsoft.com/whdc/devtools/debugging/install64bit.mspx

This will install tools to:

%systemdrive%\Program Files\Debugging Tools for Windows (x64)

I then configured ADPlus to catch this type of exception, this is crucial, else it would catch other exception types and the dumps generated will not be the ones we need, so a filter with a config file is used! Note I am using AdPlus verson that requires version 2 attribute in the header and the schema has changed since the previosu version.

——————————————ExceptionFilter.cfg———————————————

<ADPlus  Version="2">
     <Settings> 
      <RunMode>Crash</RunMode>
      <Option>NoDumpOnFirst</Option>
      <Sympath>c:\symbols\</Sympath>
      <OutputDir>F:\TEMP\Debug\Dump</OutputDir>
     </Settings> 
     <Exceptions>
       <!–
                av = AccessViolation
                ch = InvalidHandle
                ii = IllegalInstruction
                dz =  IntegerDivide
                c000008e = FloatingDivide
                iov = IntegerOverflow
                lsq = InvalidLockSequence
                sov = StackOverflowException
                eh = CPlusPlusEH
                * = UnknownException
                clr = NET_CLR
                bpe = CONTRL_C_OR_Debug_Break
                ld = DLL_Load
                ud = DLL_UnLoad
                epr = Process_Shut_Down
                sbo = Stack_buffer_overflow
                –>
         <NewException Code="0xe053534f" Name="Unknown_Exception "> 
         </NewException> 
    <Exception Code="0xe053534f">   
       <Actions1>FullDump;Log;EventLog</Actions1>
       <Actions2>MiniDump</Actions2>
       <ReturnAction1>gn</ReturnAction1>
    </Exception>
    <Exception Code="sov">   
       <Actions1>FullDump;Log;EventLog</Actions1>
       <Actions2>MiniDump</Actions2>
     <ReturnAction1>gn</ReturnAction1>
    </Exception>
    <Exception Code="*">   
       <Actions1>FullDump;Log;EventLog</Actions1>
       <Actions2>MiniDump</Actions2>
     <ReturnAction1>gn</ReturnAction1>
    </Exception>   
     </Exceptions> 
</ADPlus>

————————————————————————————————————————-

We then open a command prompt and then run the Adplus.exe command:

I also downloaded the symbol files for the NT system and the private symbol files (pdb) for the custom .net web application and put them in the c:\symbols folder

adplus.exe –pi 4567 –y c:\\symbols  /c ExceptionFilter.cfg

This was left to run and after 2 days, we got this dump file Smile

FULLDUMP_FirstChance_0xe053534f_Unknown_Exception_w3wp.exe__106c_2010-12-08_16-08-27-451_1608

and then this one

FULLDUMP_FirstChance_epr_Process_Shut_Down_w3wp.exe__106c_2010-12-08_16-08-38-169_1608

The one we want to check out is the first one, since it managed to find the 0xe053534f exception that I was concerned about, and I was sure this code is a stack overflow, but lets go find out!

Before I get started with WinDBG, I created this file template, since I copied the symbol files and private pdb files to my local laptop, I also needed the DAC for CLR (mscordacwks.dll) on my machine, which is the version from the server.

So here is my template file:

———————————cmdtree.txt————————————-

windbg ANSI Command Tree 1.0
title {"Common Commands"}
body
{"Common Commands"}
{"Information"}
  {"Time of dump"} {".time"}
  {"Process being debugged"} {"|"}
  {"Dump Location"} {"||"}
  {"Create server on port 9999"} {".server tcp:port=9999"}
  {"Show remote connections"} {".clients"}
  {"Process Environment Block"} {"!peb"}
{"Logging"}
  {"Open Log"} {".logopen /t /u /d"}
  {"Close Log"} {".logclose"}
{"Modules"}
  {"All Modules"} {"lm D sm"}
  {"Loaded Modules"} {"lmo D sm"}
  {"Loaded Modules (verbose)"} {"lmvo D sm"}
  {"Modules w/o symbols"} {"lme D sm"}
{"Stacks"}
  {"Set frame length to 2000"} {".kframes 2000"}
  {"Dump current stack w/ DML"} {"kpM 1000"}
  {"Dump stacks without private info"} {"knL 1000"}
  {"Dump stacks with all parameters"} {"kPn 1000"}
  {"Dump stacks (distance from last frame)"} {"kf 1000"}
  {"Dump stacks with Frame Pointer Omission"} {"kvn 1000"}
  {"Dump all stack"} {"~*kbn 1000"}
  {"Dump unique stacks"} {"!uniqstack -pn"}
  {"Thread environment block"} {"!teb"}
  {"Move to next frame"} {".f+"}
  {"Move to previous frame"} {".f-"}
{"Memory"}
  {"Dump heaps"} {"!heap -a"}
{"Automated Task"}
  {"!analyze"} {"!analyze -v"}
  {"Locks"} {"!ntsdexts.locks"}
  {"CPU time for User and Kernel Mode"} {"!runaway 7"}
{"Managed"}
  {"Load sos"} {".loadby sos mscorwks"}
  {"clrstack"} {"!clrstack"}
  {"Threads"} {"!threads"}
  {"Stack Objects"} {"!dso"}
  {"Exceptions"} {"!dae"}
{"CustomerTravelXYZ"}
{"Fix Symbols"} {".symfix"}
{"Add PrivateSymbols"} {".sympath+ H:\Projects\Travel\Symbols"}
{"Load Correct CLR DAC"} {".cordll -lp H:\Projects\Travel\mscordacwks.dll"}
{"RELOAD"} {".reload"}
{"Load SOS"} {".loadby sos mscorwks"}
{"Get Stack"} {"!clrstack"}

————————————————————————————-

Now we we open windbg and load the dump file, we can issue this command:

windbg> .cmdtree H:\Projects\Debug\cmdtree.txt

Now I get this window popup to execute my command to initialise the debugging environment:

image

 

I can now double click Fix Symbols all the way up to Get Stack. I hate typing this in all the time when debugging multiple DMP files! This generates these commands

0:030> .symfix
0:030> .sympath+ H:\Projects\Travel\Symbols
Symbol search path is: srv*;H:\Projects\Travel\Symbols
Expanded Symbol search path is: cache*;SRV*http://msdl.microsoft.com/download/symbols;h:\projects\travel\symbols
0:030> .cordll -lp H:\Projects\Travel\mscordacwks.dll
CLR DLL status: No load attempts
0:030> .reload
……………………………………………………….
……………………………………………………….
……………………………………………………….
………….
Loading unloaded module list
..
0:030> .loadby sos mscorwks

0:030> !clrstack

0:030> !clrstack
OS Thread Id: 0x1748 (30)
*** WARNING: Unable to verify checksum for mscorlib.ni.dll
*** WARNING: Unable to verify checksum for System.Web.ni.dll
Child-SP         RetAddr          Call Site
0000000004ce2500 000007fef8564cba System.Diagnostics.StackTrace.ToString(TraceFormat)
0000000004ce25e0 000007fef850396c System.Exception.get_StackTrace()
0000000004ce2620 000007ff001a69ad System.Exception.ToString()
0000000004ce2660 000007ff001a2153 Company.XYZ._EmptyErrorObject(System.Type, Company.XYZ.DBError)
0000000004ce26c0 000007fef92642e1 Company.XYZ._FirstWhere(System.Type, Company.XYZ.SqlStatement)
0000000004ce45e0 000007ff001a20be Company.XYZ.Broker.FindFirst(System.Type, Company.XYZ.SqlStatement)
0000000004ce47e0 000007ff0019fa63 Company.XYZ._FirstWhere(System.Type, Company.XYZ.SqlStatement)
0000000004ce4890 000007ff0019eec9 ConcordeLib.STSessionDataObjects.LoginMessage.Latest()
0000000004ce49a0 000007ff001a97af Company.XYZControllers.AccountController.LogOn(System.String)
0000000004ce4bd0 000007fef92642e1 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004ce6af0 000007ff001ad7cb Company.XYZ.Login(System.String, System.String, System.String)
0000000004ce6bd0 000007ff001ad717 Company.XYZ.Login(Int32, System.String, System.String, System.String, System.String)
0000000004ce6cd0 000007ff001a92e7 Company.XYZModels.StandardTicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)
0000000004ce6d20 000007ff0019f517 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004ce6f50 000007ff001a97af Company.XYZControllers.AccountController.LogOn(System.String)
0000000004ce7180 000007fef92642e1 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004ce90a0 000007ff001ad7cb Company.XYZ.Login(System.String, System.String, System.String)
0000000004ce9180 000007ff001ad717 Company.XYZ.Login(Int32, System.String, System.String, System.String, System.String)
0000000004ce9280 000007ff001a92e7 Company.XYZModels.StandardTicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)
0000000004ce92d0 000007ff0019f517 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004ce9500 000007ff001a97af Company.XYZControllers.AccountController.LogOn(System.String)
0000000004ce9730 000007fef92642e1 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004ceb650 000007ff001ad7cb Company.XYZ.Login(System.String, System.String, System.String)
0000000004ceb730 000007ff001ad717 Company.XYZ.Login(Int32, System.String, System.String, System.String, System.String)
System.String)
0000000004d2b1c0 000007ff001a92e7 Company.XYZModels.StandardTicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)
0000000004d2b210 000007ff0019f517 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004d2b440 000007ff001a97af Company.XYZControllers.AccountController.LogOn(System.String)
0000000004d2b670 000007fef92642e1 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004d2d590 000007ff001ad7cb Company.XYZ.Login(System.String, System.String, System.String)
0000000004d2d670 000007ff001ad717 Company.XYZ.Login(Int32, System.String, System.String, System.String, System.String)
0000000004d2d770 000007ff001a92e7 Company.XYZModels.StandardTicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)
0000000004d2d7c0 000007ff0019f517 Company.XYZControllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
0000000004d2d9f0 000007ff00241035 Company.XYZControllers.AccountController.LogOn(System.String)
0000000004d2dc20 000007ff0019e3bb DynamicClass.lambda_method(System.Runtime.CompilerServices.ExecutionScope, System.Web.Mvc.ControllerBase, System.Object[])
0000000004d2dc60 000007ff0019e27f System.Web.Mvc.ReflectedActionDescriptor.Execute(System.Web.Mvc.ControllerContext, System.Collections.Generic.IDictionary`2<System.String,System.Object>)
0000000004d2dcc0 000007ff0019e0c8 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(System.Web.Mvc.ControllerContext, System.Web.Mvc.ActionDescriptor, System.Collections.Generic.IDictionary`2<System.String,System.Object>)
0000000004d2dd00 000007ff0019de55 System.Web.Mvc.ControllerActionInvoker+<>c__DisplayClassd.<InvokeActionMethodWithFilters>b__a()
0000000004d2dd60 000007ff0019dac8 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(System.Web.Mvc.IActionFilter, System.Web.Mvc.ActionExecutingContext, System.Func`1<System.Web.Mvc.ActionExecutedContext>)
0000000004d2ddf0 000007ff00196906 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(System.Web.Mvc.ControllerContext, System.Collections.Generic.IList`1<System.Web.Mvc.IActionFilter>, System.Web.Mvc.ActionDescriptor, System.Collections.Generic.IDictionary`2<System.String,System.Object>)
0000000004d2de50 000007ff00195db8 System.Web.Mvc.ControllerActionInvoker.InvokeAction(System.Web.Mvc.ControllerContext, System.String)
0000000004d2def0 000007ff001957e3 System.Web.Mvc.Controller.ExecuteCore()
0000000004d2df50 000007ff00195775 System.Web.Mvc.MvcHandler+<>c__DisplayClass8.<BeginProcessRequest>b__4()
0000000004d2df90 000007ff0019572f System.Web.Mvc.Async.AsyncResultWrapper+<>c__DisplayClass1.<MakeVoidDelegate>b__0()
0000000004d2dfd0 000007ff001955bd System.Web.Mvc.Async.AsyncResultWrapper+<>c__DisplayClass8`1[[System.Web.Mvc.Async.AsyncVoid, System.Web.Mvc]].<BeginSynchronous>b__7(System.IAsyncResult)
0000000004d2e000 000007ff00195417 System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResult`1[[System.Web.Mvc.Async.AsyncVoid, System.Web.Mvc]].End()
0000000004d2e040 000007fef29714c6 System.Web.Mvc.MvcHandler.EndProcessRequest(System.IAsyncResult)
0000000004d2e070 000007fef29356d7 System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
0000000004d2e120 000007fef301dfa1 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
0000000004d2e1c0 000007fef300f1e2 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
0000000004d2e350 000007fef2ff1409 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
0000000004d2e3a0 000007fef3117fb4 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
0000000004d2e4c0 000007fef3117bcb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
0000000004d2e630 000007fef3116c34 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
0000000004d2e690 000007fef93cb07a DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
0000000004d2eec0 000007fef3118094 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
0000000004d2efa0 000007fef3117bcb System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
0000000004d2f110 000007fef3116c34 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
0000000004d2f170 000007fef93cb2cb DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)

 

This is clearly a repetition of method calls on the stack, and is overflowing it. We can see a bug in the MVC Account Logon controller, so the developers will need to fix logon retries etc.

We can then issue a:

0:030> !clrstack -p
OS Thread Id: 0x1748 (30)
Child-SP         RetAddr          Call Site
0000000004ce2500 000007fef8564cba System.Diagnostics.StackTrace.ToString(TraceFormat)
    PARAMETERS:
        this = 0x00000001000fd088
        traceFormat = 0x000007ff00000000

0000000004ce25e0 000007fef850396c System.Exception.get_StackTrace()
    PARAMETERS:
        this = <no data>

0000000004ce2620 000007ff001a69ad System.Exception.ToString()
    PARAMETERS:
        this = <no data>

0000000004ce2660 000007ff001a2153Comapny.XYZ._EmptyErrorObject(System.Type, Comapny.XYZ.DBError)
    PARAMETERS:
        source = 0x00000000ffba39b0
        ex = 0x00000001000fc398

0000000004ce26c0 000007fef92642e1Comapny.XYZ._FirstWhere(System.Type, Comapny.XYZ.SqlStatement)
    PARAMETERS:
        source = <no data>
        stmt = <no data>

0000000004ce45e0 000007ff001a20be Comapny.XYZ.Broker.FindFirst(System.Type, Comapny.XYZ.SqlStatement)
    PARAMETERS:
        source = 0x00000000ffba39b0
        stmt = 0x00000001000ccef0

0000000004ce47e0 000007ff0019fa63Comapny.XYZ._FirstWhere(System.Type, Comapny.XYZ.SqlStatement)
    PARAMETERS:
        source = 0x00000000ffba39b0
        stmt = 0x00000001000ccef0

0000000004ce4890 000007ff0019eec9 Comapny.XYZ.LoginMessage.Latest()
0000000004ce49a0 000007ff001a97af Comapny.XYZ.Controllers.AccountController.LogOn(System.String)
    PARAMETERS:
        this = 0x00000000fff856a0
        returnUrl = 0x00000000fff86458

0000000004ce4bd0 000007fef92642e1 Comapny.XYZ.Controllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)
    PARAMETERS:
        this = <no data>
        BLAHNo = <no data>
        uid = <no data>
        pwd = <no data>
        rememberMe = <no data>
        returnUrl = <no data>

0000000004ce6af0 000007ff001ad7cb Comapny.XYZ.Login(System.String, System.String, System.String)
    PARAMETERS:
        BLAHOrDebtorId = 0x0000000100098e90
        userId = 0x0000000100098cd0
        password =Omitted

0000000004ce6bd0 000007ff001ad717 Comapny.XYZ.TicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)
    PARAMETERS:
        BLAHNo = 0x0000000000018c8c
        userName = 0x0000000100098cd0
        password = Omitted
        sessionId = 0x00000000ffbe00d8
        shopCode = 0x00000000ffbe0390

 

from here I can Dump Objects related to the username that caused the problem.

0:030> !do 0x0000000100098cd0
Name: System.String
MethodTable: 000007fef8657a80
EEClass: 000007fef825e530
Size: 40(0x28) bytes
(C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
String: RomikoDerbynewNaughty
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fef865ecf0  4000096        8         System.Int32  1 instance                8 m_arrayLength
000007fef865ecf0  4000097        c         System.Int32  1 instance                7 m_stringLength
000007fef86594c8  4000098       10          System.Char  1 instance               77 m_firstChar
000007fef8657a80  4000099       20        System.String  0   shared           static Empty
                                 >> Domain:Value  000000000133cc40:000000013fb00370 00000000030d2cc0:000000013fb00370 <<
000007fef8659378  400009a       28        System.Char[]  0   shared           static WhitespaceChars
                                 >> Domain:Value  000000000133cc40:000000013fb00ac0 00000000030d2cc0:00000000ffb03758 <<

I can see this user is always in the stackoverlfow, this method call with these parameters are repeated. Why???

Lets dump the exceptions

0:030> !pe
Exception object: 00000001000fc398
Exception type: Company.XYZ.DBAccessLayer.DBError
Message: Couldn’t find any results in LoginMessages for the statement
set transaction isolation level read uncommitted; select top 1 * from mytable where DateFrom <= @DateFrom and DateTo >= @DateTo and IsActive = @IsActive  order by Id desc
Parameters:
@DateFrom: 2002-12-08
@DateTo: 2002-12-08
@IsActive: True

InnerException: <none>
StackTrace (generated):
    SP               IP               Function
    0000000004CE45E0 000007FF001A2660 GentleCompany.XYZ!Company.XYZ.DBAccessLayer.Broker.FindFirst(System.Type, Company.XYZ.DBAccessLayer.SqlStatement)+0x3e0
    0000000004CE47E0 000007FF001A20BF GentleCompany.XYZ!Company.XYZ.DataAccessObject._FirstWhere(System.Type, Company.XYZ.DBAccessLayer.SqlStatement)+0x6f

StackTraceString: <none>
HResult: 80131500
There are nested exceptions on this thread. Run with -nested for details

This exception does not seem to be the real thing, has nothing to do with the user, lets get this exceptions nested exceptions.

Why, since the above SQL would not cause an IIS crash, it is just a SQLReader getting zero results and the code is not dealing with a null.

 

So moving into the nested exceptions. There were 100’s of them, so I kept it short here:

0:030> !pe -nested 00000001000fc398
Exception object: 00000001000fc398
Exception type: CompanyXYZLib.DBAccessLayer.DBError
Message: Couldn’t find any results in LoginMessages for the statement
set transaction isolation level read uncommitted; select top 1 * from mytable where DateFrom <= @DateFrom and DateTo >= @DateTo and IsActive = @IsActive  order by Id desc

Parameters:
@DateFrom: 2010-12-08
@DateTo: 2010-12-08
@IsActive: True

InnerException: <none>
StackTrace (generated):
    SP               IP               Function
    0000000004CE45E0 000007FF001A2660 CompanyXYZLib!CompanyXYZLib.DBAccessLayer.Broker.FindFirst(System.Type, CompanyXYZLib.DBAccessLayer.SqlStatement)+0x3e0
    0000000004CE47E0 000007FF001A20BF CompanyXYZLib!CompanyXYZLib.DataAccessObject._FirstWhere(System.Type, CompanyXYZLib.DBAccessLayer.SqlStatement)+0x6f

StackTraceString: <none>
HResult: 80131500

Nested exception ————————————————————-
Exception object: 00000001000cccb8
Exception type: CompanyXYZLib.AgentDataObjects.TicketsWebUser+LoginError

Message: Your BLAHNo is not currently active with Attribute2.

InnerException: <none>
StackTrace (generated):
    SP               IP               Function
    0000000004CE6AF0 000007FF001ADDDE CompanyXYZLib!CompanyXYZLib.AgentDataObjects.TicketsWebUser.Login(System.String, System.String, System.String)+0x2de
    0000000004CE6BD0 000007FF001AD7CC CompanyXYZ!CompanyXYZ.TicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)+0x7c
    0000000004CE6CD0 000007FF001AD718 CompanyXYZ!CompanyXYZ.Models.StandardTicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)+0x58
    0000000004CE6D20 000007FF001A92E8 CompanyXYZ!CompanyXYZ.Controllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)+0x238

StackTraceString: <none>
HResult: 80131500

Nested exception ————————————————————-
Exception object: 00000001000977d0
Exception type: CompanyXYZLib.AgentDataObjects.TicketsWebUser+LoginError
Message: Your BLAHNo is not currently active with Attribute2.
InnerException: <none>
StackTrace (generated):
    SP               IP               Function
    0000000004CE90A0 000007FF001ADDDE CompanyXYZLib!CompanyXYZLib.AgentDataObjects.TicketsWebUser.Login(System.String, System.String, System.String)+0x2de
    0000000004CE9180 000007FF001AD7CC CompanyXYZ!CompanyXYZ.TicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)+0x7c
    0000000004CE9280 000007FF001AD718 CompanyXYZ!CompanyXYZ.Models.StandardTicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)+0x58
    0000000004CE92D0 000007FF001A92E8 CompanyXYZ!CompanyXYZ.Controllers.AccountController.LogOn(Int32, System.String, System.String, Boolean, System.String)+0x238

StackTraceString: <none>
HResult: 80131500

Perfect, we found the problem Smile BLAH attributes are causing issues when not active with BLAH2 and user RomikODebrynewNaughty has this unique condition which caused the software code to infinitely retry his login! So we isolated the problem in the logon business rules and fixed the infinite loop and the web server was back to normal Smile

 

The BLAHNo  from the !clrstack –p and then the !do on the parameter confirms this e.g.

0000000004ce6bd0 000007ff001ad717 Comapny.XYZ.TicketProcessor.Login(Int32, System.String, System.String, System.String, System.String)
    PARAMETERS:
        BLAHNo = 0x0000000000018c8c
        userName = 0x0000000100098cd0
        password = Omitted
        sessionId = 0x00000000ffbe00d8
        shopCode = 0x00000000ffbe0390

 

Hoe you find this useful. I omitted allot of the output as there were about 130 of the same method call patterns on the stack.

I am loving this sort of debugging and will be playing around with SOSEX soon and see what we can talk about Smile

Advertisement

One thought on “Generating and analysing Post Mortem memory dump files”

  1. Pingback: WinDbg .cmdtree file format reverse engineered | Debugging

Leave a Reply

Please log in using one of these methods to post your comment:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s