Author: Romiko Derbynew

Memory Dump Analysis–W3WP IIS Process

At a customer I had prepared some Visual Studio 2010 WebTests which were calling their Java based website hosted on IBM Websphere, there is an IFrame on the Java page that points to an IIS hosted Asp.net 2.0 web site.

When running load tests I noticed that the webtests were taking a very long time to execute at one point the load test grinded the servers to a halt and a colleague of mine Tatham Oddie decided it was time we dump the memory from the IIS Process hosting the site.

So, right click the w3wp.exe process and click Create Dump File.

We also have windbg installed, which you can download from

http://www.windbg.org/

Ensure you click debugging tools for windows under redistributable, so you get 32/64 bit versions

So, the big picture is for us to look at what is on the stack, ideally in software we should have items living on the stack for a very short period of time, so lets analyze the stack and see what clues it might have about the web site behaving badly.

Open Windbg, it is located in

C:\Program Files\Debugging Tools for Windows (x64) or you might have the 32 bit version.

I also copied the following files

w3wp.dmp (the generated dump

mscordacwks.dll (From Server hosting the IIS service)

mscordacwks_64.dll (From Server hosting the IIS service)

Start windbg and open the dump file.

excellent, we are presented with

SOS will need the Data Access Layer to make calls into the CLR, so we need the correct version of the mscordacwks.dll file from the Server where the software was running. This can be either 32 bit or 64 bit.

They are located at:

C:\Windows\Microsoft.NET\Framework\v2.0.50727

C:\Windows\Microsoft.NET\Framework64\v2.0.50727

Now, we need to tell the debugger where this file is, I copied these files and the dmp file from the server and put it into my E:\projects\debug folder

first, lets get the correct mscordacwks loaded, I renamed my 64bit version to this name:

mscordacwks_AMD64_AMD64_2.0.50727.5420.dll you can find the file version in the property page for mscordacwks_64

I then point my debugger to the renamed version of mscordackwks.

.sympath+ e:\projects\debug

then I load the DAC

.cordll –ve –u –l

I can now load the sos commands

.loadby sos mscorwks

if you have issues load the mscordacwks, you can also try

!sym noisy
.symfix e:\projects\debug
.cordll -ve -u -l

this will then try load the symbols from the microsoft server (http://msdl.microsoft.com/download/symbols), you must have an internet connection.

Lets see what threads are in the process:

0:000> !threads
ThreadCount: 37
UnstartedThread: 0
BackgroundThread: 10
PendingThread: 0
DeadThread: 27
Hosted Runtime: no
                                              PreEmptive                                                Lock
       ID OSID        ThreadOBJ     State   GC     GC Alloc Context                  Domain           Count APT Exception
   4    1 1378 00000000017fb730      8220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
12    2 970 00000000017dd1d0      b220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Finalizer)
13    4 107c 000000001a9c8cf0    80a220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Threadpool Completion Port)
14    5 12d0 000000001a9d5fc0      1220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
XXXX    6    0 000000001aa36200   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Threadpool Worker)
XXXX    8    0 000000001aaac0b0      9820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
XXXX    9    0 000000001aaab850   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX    a    0 000000001ca2b840      9820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
XXXX    b    0 000000001caa3830   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX    c    0 000000001ab937c0   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX    d    0 000000001cab4680   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX    e    0 000000001ca52940   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX    f    0 000000001caaf450   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   10    0 000000001ca57290   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   11    0 000000001cab5c60   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Threadpool Worker)
XXXX   12    0 000000001aa52ca0   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   13    0 000000001aa53270   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
   2   14 f48 000000001aa53840       220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
XXXX   15    0 000000001aa543e0   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
17   16 834 000000001aa549b0   180b220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Threadpool Worker)
XXXX   17    0 000000001aa54f80      9820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
XXXX   18    0 000000001aa55550   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Threadpool Worker)
XXXX   19    0 000000001aa55b20   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   1a    0 000000001aa560f0   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   1b    0 000000001aa566c0   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   1c    0 000000001aa56c90   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   1d    0 000000001aa57260   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   1e    0 000000001aa57830   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   1f    0 000000001aa57e00   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   20    0 000000001aa583d0   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
18   21 121c 000000001aa589a0   380b220 Enabled 0000000000000000:0000000000000000 000000001ab979c0     1 MTA (Threadpool Worker)
XXXX   22    0 000000001aa58f70   1801820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn (Threadpool Worker)
XXXX   24    0 000000001aa59b10      9820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
XXXX   26    0 000000001ef10680      9820 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 Ukn
19   25 11bc 000000001ef100b0   200b220 Enabled 0000000000000000:0000000000000000 000000001ab979c0     1 MTA
21    3 13c8 000000001aa526d0   880b220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Threadpool Completion Port)
22    7 12f4 000000001aa53e10   880b220 Enabled 0000000000000000:0000000000000000 00000000017ea4f0     0 MTA (Threadpool Completion Port)

Ok, from this we will need to try find the thread that is running all the IIS web pages etc. SO lets execute

0:000> !eestack
———————————————
Thread   4
Child-SP         RetAddr          Call Site
0000000001a2f7a8 000007fefd47169d ntdll!NtRemoveIoCompletion+0xa
0000000001a2f7b0 000000007731a4e1 KERNELBASE!GetQueuedCompletionStatus+0x3d
0000000001a2f810 000007fef9e81f7b kernel32!GetQueuedCompletionStatusStub+0x11
0000000001a2f850 000007fef9e82024 w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x3b
0000000001a2f8a0 000007fef9e820a1 w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x34
0000000001a2f8d0 000000007732652d w3tp!THREAD_MANAGER::ThreadManagerThread+0x61
0000000001a2f900 000000007745c521 kernel32!BaseThreadInitThunk+0xd
0000000001a2f930 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 12
Child-SP         RetAddr          Call Site
000000001a9af5e8 000007fefd471420 ntdll!ZwWaitForMultipleObjects+0xa
000000001a9af5f0 0000000077321220 KERNELBASE!GetCurrentProcess+0x40
000000001a9af6f0 000007fef82a0353 kernel32!WaitForMultipleObjects+0xb0
000000001a9af780 000007fef8297b9a mscorwks!WKS::WaitForFinalizerEvent+0x93
000000001a9af7b0 000007fef8295a3c mscorwks!WKS::GCHeap::FinalizerThreadWorker+0x4a
000000001a9af7f0 000007fef827ba45 mscorwks!CClosedHashEx<CCustAttrHashKey,CCustAttrHash>::Hash+0x30
000000001a9af840 000007fef81735b5 mscorwks!SVR::gc_heap::make_heap_segment+0x155
000000001a9af910 000007fef829837a mscorwks!EEToProfilerExceptionInterfaceWrapper::ExceptionCatcherEnter+0x4d
000000001a9af950 000007fef82979b4 mscorwks!ManagedThreadBase_NoADTransition+0x42
000000001a9af9b0 000007fef8298164 mscorwks!WKS::GCHeap::FinalizerThreadStart+0x74
000000001a9af9f0 000000007732652d mscorwks!Thread::intermediateThreadProc+0x78
000000001a9afac0 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001a9afaf0 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 13
Child-SP         RetAddr          Call Site
000000001a8afd28 000007fefd471203 ntdll!NtDelayExecution+0xa
000000001a8afd30 000007fef814812d KERNELBASE!SleepEx+0xb3
000000001a8afdd0 000007fef82a1d67 mscorwks!ThreadpoolMgr::TimerThreadFire+0x51
000000001a8afe70 000000007732652d mscorwks!ThreadpoolMgr::TimerThreadStart+0x6b
000000001a8afeb0 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001a8afee0 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 14
Child-SP         RetAddr          Call Site
000000001adcf368 000007fefd4710ac ntdll!NtWaitForSingleObject+0xa
000000001adcf370 000007fef81cb540 KERNELBASE!WaitForSingleObjectEx+0x9c
000000001adcf410 000007fef81cb42f mscorwks!CLREvent::WaitEx+0x174
000000001adcf460 000007fef82b1a9f mscorwks!CLREvent::WaitEx+0x63
000000001adcf510 000007fef8298164 mscorwks!AppDomain::ADUnloadThreadStart+0x153
000000001adcf630 000000007732652d mscorwks!Thread::intermediateThreadProc+0x78
000000001adcf780 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001adcf7b0 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread   2
Child-SP         RetAddr          Call Site
0000000000b3f578 000000007744fe0b ntdll!ZwWaitForWorkViaWorkerFactory+0xa
0000000000b3f580 000000007732652d ntdll!RtlValidateHeap+0x3bb
0000000000b3f880 000000007745c521 kernel32!BaseThreadInitThunk+0xd
0000000000b3f8b0 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 17
Child-SP         RetAddr          Call Site
000000001ea0ed78 000007fefd4710ac ntdll!NtWaitForSingleObject+0xa
000000001ea0ed80 000007fef81cb540 KERNELBASE!WaitForSingleObjectEx+0x9c
000000001ea0ee20 000007fef81cb42f mscorwks!CLREvent::WaitEx+0x174
000000001ea0ee70 000007fef81cb18b mscorwks!CLREvent::WaitEx+0x63
000000001ea0ef20 000007fef81d841a mscorwks!ThreadpoolMgr::SafeWait+0x7b
000000001ea0efe0 000007fef8298164 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x11a
000000001ea0f080 000000007732652d mscorwks!Thread::intermediateThreadProc+0x78
000000001ea0fad0 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001ea0fb00 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 18
Child-SP         RetAddr          Call Site
000000001e88d578 000007fefd471420 ntdll!ZwWaitForMultipleObjects+0xa
000000001e88d580 0000000077332d53 KERNELBASE!GetCurrentProcess+0x40
000000001e88d680 000007fef817212d kernel32!WaitForMultipleObjectsExImplementation+0xb3
000000001e88d710 000007fef81769b9 mscorwks!WaitForMultipleObjectsEx_SO_TOLERANT+0xc1
000000001e88d7b0 000007fef8277cf9 mscorwks!Thread::DoAppropriateAptStateWait+0x41
000000001e88d810 000007fef8192b5c mscorwks!Thread::DoAppropriateWaitWorker+0x191
000000001e88d910 000007fef87133c1 mscorwks!Thread::DoAppropriateWait+0x5c
000000001e88d980 000007fef36c6653 mscorwks!WaitHandleNative::CorWaitOneNative+0x221
000000001e88dbc0 000007fef2b3e817 mscorlib_ni!System.Threading.WaitHandle.WaitOne(Int64, Boolean)+0x23
000000001e88dc00 000007fef2b48641 System_ni!System.Net.LazyAsyncResult.WaitForCompletion(Boolean)+0xc7
000000001e88dc60 000007fef2b47f34 System_ni!System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest)+0x2d1
000000001e88dcf0 000007fef2b474f8 System_ni!System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest, System.String)+0x84
000000001e88dd60 000007fef2b4b031 System_ni!System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint)+0x268
000000001e88ddc0 000007ff00705bbb System_ni!System.Net.HttpWebRequest.GetResponse()+0x291
000000001e88de50 000007fef09da30a RomikoCommon_72a00000!RomikoCommon.PageBase.OnPreInit(System.EventArgs)+0x36b
000000001e88df00 000007fef09dadb8 System_Web_ni!System.Web.UI.Page.PerformPreInit()+0x2a
000000001e88df30 000007fef09da750 System_Web_ni!System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)+0x4e8
000000001e88e000 000007fef09da67b System_Web_ni!System.Web.UI.Page.ProcessRequest(Boolean, Boolean)+0xa0
000000001e88e060 000007fef09da610 System_Web_ni!System.Web.UI.Page.ProcessRequest()+0x5b
000000001e88e0c0 000007ff007049f4 System_Web_ni!System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)+0xf0
000000001e88e120 000007fef09e1ab7 App_Web_d_5hdouw!ASP.policysearch_aspx.ProcessRequest(System.Web.HttpContext)+0x34
000000001e88e150 000007fef09a571b System_Web_ni!System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()+0x257
000000001e88e200 000007fef1090561 System_Web_ni!System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)+0xab
000000001e88e2a0 000007fef10811c2 System_Web_ni!System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)+0x511
000000001e88e430 000007fef1062df9 System_Web_ni!System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)+0x72
000000001e88e480 000007fef1189931 System_Web_ni!System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)+0x269
000000001e88e5a0 000007fef1189d8b System_Web_ni!System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)+0x411
000000001e88e720 000007fef1189294 System_Web_ni!System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)+0x2b
000000001e88e780 000007fef831f18a System_Web_ni!DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)+0x24
000000001e88e7c0 000007fef7dd51a7 mscorwks!UMThunkStubAMD64+0x7a
000000001e88e850 000007fef7dd6262 webengine!RegisterModule+0x1e87
000000001e88e8a0 000007fef7dd1ade webengine!GetEcb+0x3c2
000000001e88e910 000007fef7dd2114 webengine!CSpinLock::ConvertSharedToExclusive+0x129e
000000001e88e9a0 000007fef8012de7 webengine!CSpinLock::ConvertSharedToExclusive+0x18d4
000000001e88e9e0 000007fef80146a4 iiscore!NOTIFICATION_CONTEXT::RequestDoWork+0x233
000000001e88ea20 000007fef801a775 iiscore!NOTIFICATION_CONTEXT::CallModulesInternal+0x174
000000001e88eb10 000007fef8015a03 iiscore!NOTIFICATION_CONTEXT::CallModules+0x25
000000001e88eb60 000007fef801a81c iiscore!W3_CONTEXT::DoWork+0x34d
000000001e88ee80 000007fef7dcfc41 iiscore!W3_CONTEXT::IndicateCompletion+0x8c
000000001e88eee0 000007fef8320e37 webengine!MgdIndicateCompletion+0x61
000000001e88ef10 000007fef108b25b mscorwks!DoNDirectCall__PatchGetThreadCall+0x7b
000000001e88efb0 000007fef1189a60 System_Web_ni!DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)+0x4b
000000001e88f090 000007fef1189d8b System_Web_ni!System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)+0x540
000000001e88f210 000007fef1189294 System_Web_ni!System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)+0x2b
000000001e88f270 000007fef831f3db System_Web_ni!DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)+0x24
000000001e88f2b0 000007fef830d079 mscorwks!UM2MThunk_WrapperHelper+0x4b
000000001e88f2f0 000007fef830d165 mscorwks!UM2MThunk_WrapperWorker+0x99
000000001e88f380 000007fef8295bc9 mscorwks!UM2MThunk_Wrapper+0x19
000000001e88f3b0 000007fef82fe398 mscorwks!Thread::DoADCallBack+0x145
000000001e88f520 000007fef831f386 mscorwks!UM2MDoADCallBack+0x90
000000001e88f5b0 000007fef7dd51a7 mscorwks!UMThunkStubAMD64+0x276
000000001e88f640 000007fef7dd1066 webengine!RegisterModule+0x1e87
000000001e88f690 000007fef81c56f7 webengine!CSpinLock::ConvertSharedToExclusive+0x826
000000001e88f6c0 000007fef81d84ba mscorwks!UnManagedPerAppDomainTPCount::DispatchWorkItem+0x157
000000001e88f760 000007fef8298164 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x1ba
000000001e88f800 000000007732652d mscorwks!Thread::intermediateThreadProc+0x78
000000001e88fdd0 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001e88fe00 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 19
Child-SP         RetAddr          Call Site
000000001b5ed648 000007fefd471420 ntdll!ZwWaitForMultipleObjects+0xa
000000001b5ed650 0000000077332d53 KERNELBASE!GetCurrentProcess+0x40
000000001b5ed750 000007fef817212d kernel32!WaitForMultipleObjectsExImplementation+0xb3
000000001b5ed7e0 000007fef81769b9 mscorwks!WaitForMultipleObjectsEx_SO_TOLERANT+0xc1
000000001b5ed880 000007fef8277cf9 mscorwks!Thread::DoAppropriateAptStateWait+0x41
000000001b5ed8e0 000007fef8192b5c mscorwks!Thread::DoAppropriateWaitWorker+0x191
000000001b5ed9e0 000007fef8712f21 mscorwks!Thread::DoAppropriateWait+0x5c
000000001b5eda50 000007fef3687e24 mscorwks!WaitHandleNative::CorWaitMultipleNative+0x2d1
000000001b5edcd0 000007fef2b11a67 mscorlib_ni!System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)+0x64
000000001b5edd30 000007fef366dd38 System_ni!System.Net.TimerThread.ThreadProc()+0x327
000000001b5ede00 000007fef8321612 mscorlib_ni!System.Threading.ExecutionContext.runTryCode(System.Object)+0x178
000000001b5edec0 000007fef824ee13 mscorwks!CallDescrWorker+0x82
000000001b5edf10 000007fef86fbc51 mscorwks!CallDescrWorkerWithHandler+0xd3
000000001b5edfb0 000007fef82456f2 mscorwks!MethodDesc::CallDescr+0x2b1
000000001b5ee200 000007fef877b182 mscorwks!ExecuteCodeWithGuaranteedCleanupHelper+0x12a
000000001b5ee490 000007fef3652b82 mscorwks!ReflectionInvocation::ExecuteCodeWithGuaranteedCleanup+0x172
000000001b5ee6a0 000007fef36ea91d mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)+0x62
000000001b5ee6f0 000007fef8321612 mscorlib_ni!System.Threading.ThreadHelper.ThreadStart()+0x4d
000000001b5ee740 000007fef824ee13 mscorwks!CallDescrWorker+0x82
000000001b5ee790 000007fef86fbc51 mscorwks!CallDescrWorkerWithHandler+0xd3
000000001b5ee830 000007fef8137a7d mscorwks!MethodDesc::CallDescr+0x2b1
000000001b5eea70 000007fef8295a3c mscorwks!ThreadNative::KickOffThread_Worker+0x191
000000001b5eed90 000007fef827ba45 mscorwks!CClosedHashEx<CCustAttrHashKey,CCustAttrHash>::Hash+0x30
000000001b5eede0 000007fef81735b5 mscorwks!SVR::gc_heap::make_heap_segment+0x155
000000001b5eeeb0 000007fef82a277d mscorwks!EEToProfilerExceptionInterfaceWrapper::ExceptionCatcherEnter+0x4d
000000001b5eeef0 000007fef8295bc9 mscorwks!SharedFileLockHolderBase::DoRelease+0x4d
000000001b5eef20 000007fef8295a61 mscorwks!Thread::DoADCallBack+0x145
000000001b5ef090 000007fef827ba45 mscorwks!CClosedHashEx<CCustAttrHashKey,CCustAttrHash>::Hash+0x55
000000001b5ef0e0 000007fef81735b5 mscorwks!SVR::gc_heap::make_heap_segment+0x155
000000001b5ef1b0 000007fef82e05e9 mscorwks!EEToProfilerExceptionInterfaceWrapper::ExceptionCatcherEnter+0x4d
000000001b5ef1f0 000007fef829ffbb mscorwks!ZapNibbleMap::GetSize+0x55
000000001b5ef250 000007fef8298164 mscorwks!ThreadNative::KickOffThread+0xd3
000000001b5ef330 000000007732652d mscorwks!Thread::intermediateThreadProc+0x78
000000001b5ef880 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001b5ef8b0 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 21
Child-SP         RetAddr          Call Site
000000001de2eed8 000007fefd47169d ntdll!NtRemoveIoCompletion+0xa
000000001de2eee0 000000007731a4e1 KERNELBASE!GetQueuedCompletionStatus+0x3d
000000001de2ef40 000007fef82b347a kernel32!GetQueuedCompletionStatusStub+0x11
000000001de2ef80 000007fef8298164 mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0xee
000000001de2f020 000000007732652d mscorwks!Thread::intermediateThreadProc+0x78
000000001de2f7f0 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001de2f820 0000000000000000 ntdll!RtlUserThreadStart+0x21
———————————————
Thread 22
Child-SP         RetAddr          Call Site
000000001e07eed8 000007fefd47169d ntdll!NtRemoveIoCompletion+0xa
000000001e07eee0 000000007731a4e1 KERNELBASE!GetQueuedCompletionStatus+0x3d
000000001e07ef40 000007fef82b347a kernel32!GetQueuedCompletionStatusStub+0x11
000000001e07ef80 000007fef8298164 mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0xee
000000001e07f020 000000007732652d mscorwks!Thread::intermediateThreadProc+0x78
000000001e07f870 000000007745c521 kernel32!BaseThreadInitThunk+0xd
000000001e07f8a0 0000000000000000 ntdll!RtlUserThreadStart+0x21

Thread 18 looks like a good candidate, we dealing with ASP>NET and we can see page loads etc, in fact a Page_Init is on the stack in that thread and a web request, also, the page I was testing was indeed a policysearch which had crashed!

000000001e88e120 000007fef09e1ab7 App_Web_d_5hdouw!ASP.policysearch_aspx.ProcessRequest

So, we on the right track, lets activate this as the active thread

we will then switch threads by typing ~18s, now we can load the stack as it is managed code.

0:000> ~18s
ntdll!ZwWaitForMultipleObjects+0xa:
00000000`774818ca c3 ret
0:018> !clrstack
OS Thread Id: 0x121c (18)
Child-SP RetAddr Call Site
000000001e88dbc0 000007fef2b3e817 System.Threading.WaitHandle.WaitOne(Int64, Boolean)
000000001e88dc00 000007fef2b48641 System.Net.LazyAsyncResult.WaitForCompletion(Boolean)
000000001e88dc60 000007fef2b47f34 System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest)
000000001e88dcf0 000007fef2b474f8 System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest, System.String)
000000001e88dd60 000007fef2b4b031 System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint)
000000001e88ddc0 000007ff00705bbb System.Net.HttpWebRequest.GetResponse()
000000001e88de50 000007fef09da30a RomikoCommon.PageBase.OnPreInit(System.EventArgs)
000000001e88df00 000007fef09dadb8 System.Web.UI.Page.PerformPreInit()
000000001e88df30 000007fef09da750 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
000000001e88e000 000007fef09da67b System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
000000001e88e060 000007fef09da610 System.Web.UI.Page.ProcessRequest()
000000001e88e0c0 000007ff007049f4 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
000000001e88e120 000007fef09e1ab7 ASP.policysearch_aspx.ProcessRequest(System.Web.HttpContext)
000000001e88e150 000007fef09a571b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
000000001e88e200 000007fef1090561 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
000000001e88e2a0 000007fef10811c2 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
000000001e88e430 000007fef1062df9 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
000000001e88e480 000007fef1189931 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
000000001e88e5a0 000007fef1189d8b System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
000000001e88e720 000007fef1189294 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
000000001e88e780 000007fef831f18a DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
000000001e88efb0 000007fef1189a60 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
000000001e88f090 000007fef1189d8b System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
000000001e88f210 000007fef1189294 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
000000001e88f270 000007fef831f3db DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)

Interesting, we can see a SibmitRequest on the stack which is a synchronous call to a web page, so this web page is called another web page, and is waiting for a response. Lets get more info about this item:

000000001e88dc60 000007fef2b47f34 System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest)

lets get the paramters etc with the –p option for the clrstack

0:018> !clrstack -p
OS Thread Id: 0x121c (18)
Child-SP         RetAddr          Call Site
000000001e88dbc0 000007fef2b3e817 System.Threading.WaitHandle.WaitOne(Int64, Boolean)
    PARAMETERS:
        this = <no data>
        timeout = <no data>
        exitContext = <no data>

000000001e88dc00 000007fef2b48641 System.Net.LazyAsyncResult.WaitForCompletion(Boolean)
    PARAMETERS:
        this = 0x00000000033d4778
        snap = <no data>

000000001e88dc60 000007fef2b47f34 System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest)
    PARAMETERS:
        this = <no data>
        request = <no data>

000000001e88dcf0 000007fef2b474f8 System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest, System.String)
    PARAMETERS:
        this = <no data>
        request = <no data>
        connName = <no data>

000000001e88dd60 000007fef2b4b031 System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint)
    PARAMETERS:
        this = 0x00000000033d3b48
        servicePoint = <no data>

000000001e88ddc0 000007ff00705bbb System.Net.HttpWebRequest.GetResponse()
PARAMETERS:
this = <no data>

000000001e88de50 000007fef09da30a RomikoCommon.PageBase.OnPreInit(System.EventArgs)
    PARAMETERS:
        this = <no data>
        e = <no data>

000000001e88df00 000007fef09dadb8 System.Web.UI.Page.PerformPreInit()
PARAMETERS:
this = <no data>

000000001e88df30 000007fef09da750 System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
    PARAMETERS:
        this = 0x00000000033adfc8
        includeStagesBeforeAsyncPoint = 0x0000000000000001
        includeStagesAfterAsyncPoint = 0x0000000000000001

000000001e88e000 000007fef09da67b System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
    PARAMETERS:
        this = 0x00000000033adfc8
        includeStagesBeforeAsyncPoint = <no data>
        includeStagesAfterAsyncPoint = 0x0000000000000001

000000001e88e060 000007fef09da610 System.Web.UI.Page.ProcessRequest()
PARAMETERS:
this = 0x00000000033adfc8

000000001e88e0c0 000007ff007049f4 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
    PARAMETERS:
        this = <no data>
        context = <no data>

000000001e88e120 000007fef09e1ab7 ASP.policysearch_aspx.ProcessRequest(System.Web.HttpContext)
    PARAMETERS:
        this = 0x00000000033adfc8
        context = 0x00000000033acc28

000000001e88e150 000007fef09a571b System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
PARAMETERS:
this = <no data>

000000001e88e200 000007fef1090561 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
    PARAMETERS:
        this = 0x000000000322d088
        step = 0x0000000003230990
        completedSynchronously = 0x000000001e88e359

000000001e88e2a0 000007fef10811c2 System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
    PARAMETERS:
        this = 0x00000000032307c8
        error = <no data>

000000001e88e430 000007fef1062df9 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)
    PARAMETERS:
        this = <no data>
        context = <no data>
        cb = <no data>

000000001e88e480 000007fef1189931 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)
    PARAMETERS:
        this = 0x0000000002579188
        wr = 0x00000000033ac938
        context = 0x00000000033acc28

000000001e88e5a0 000007fef1189d8b System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        managedHttpContext = <no data>
        nativeRequestContext = 0x000000001cd7fc58
        moduleData = <no data>
        flags = <no data>

000000001e88e720 000007fef1189294 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        managedHttpContext = <no data>
        nativeRequestContext = <no data>
        moduleData = <no data>
        flags = <no data>

000000001e88e780 000007fef831f18a DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
    PARAMETERS:
        0x000000001cd7fc58
        <no data>
        <no data>
        <no data>

000000001e88efb0 000007fef1189a60 DomainNeutralILStubClass.IL_STUB(IntPtr, System.Web.RequestNotificationStatus ByRef)
PARAMETERS:
0x0000000000b51500

000000001e88f090 000007fef1189d8b System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        managedHttpContext = <no data>
        nativeRequestContext = <no data>
        moduleData = <no data>
        flags = <no data>

000000001e88f210 000007fef1189294 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        managedHttpContext = <no data>
        nativeRequestContext = <no data>
        moduleData = <no data>
        flags = <no data>

000000001e88f270 000007fef831f3db DomainNeutralILStubClass.IL_STUB(Int64, Int64, Int64, Int32)
    PARAMETERS:
        0x000007fef830d14c
        <no data>
        <no data>
        <no data>

This has an object we can look at:

000000001e88dd60 000007fef2b4b031 System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint)
    PARAMETERS:
        this = 0x00000000033d3b48
        servicePoint = <no data>

so lets use the !do command.

0:018> !do 0x00000000033d3b48
Name: System.Net.HttpWebRequest
MethodTable: 000007fef2c22f48
EEClass: 000007fef29997d8
Size: 400(0x190) bytes
(C:\Windows\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fef37973f8 400018a        8        System.Object 0 instance 0000000000000000 __identity
000007fef31529b0 4001d76       28         System.Int32 1 instance                1 m_AuthenticationLevel
000007fef3f306a0 4001d77       2c         System.Int32 1 instance                4 m_ImpersonationLevel
000007fef2c22c70 4001d78       10 …equestCachePolicy 0 instance 00000000026b2930 m_CachePolicy
000007fef316c8a8 4001d79       18 …uestCacheProtocol 0 instance 0000000000000000 m_CacheProtocol
000007fef2c0bd50 4001d7a       20 …questCacheBinding 0 instance 00000000026b35d0 m_CacheBinding
000007fef379e9f0 4001d73      de0 …ections.ArrayList 0   shared           static s_PrefixList
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bee98 000000001ab979c0:00000000026b00a0 <<
000007fef37973f8 4001d74      de8        System.Object 0   shared           static s_InternalSyncObject
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bce20 000000001ab979c0:00000000026ae428 <<
000007fef3172938 4001d75      df0 …TimerThread+Queue 0   shared           static s_DefaultTimerQueue
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bc428 000000001ab979c0:00000000026ae368 <<
000007fef316af60 4001d7b      df8 System.Net.IWebProxy 0   shared           static s_DefaultWebProxy
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022c3740 000000001ab979c0:00000000026b39b0 <<
000007fef3796cd8 4001d7c      990       System.Boolean 1   shared           static s_DefaultWebProxyInitialized
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:1 000000001ab979c0:1 <<
000007fef3796cd8 4001f37      164       System.Boolean 1 instance                0 m_Saw100Continue
000007fef3796cd8 4001f38      165       System.Boolean 1 instance                1 m_KeepAlive
000007fef3796cd8 4001f39      166       System.Boolean 1 instance                0 m_LockConnection
000007fef3796cd8 4001f3a      167       System.Boolean 1 instance                0 m_NtlmKeepAlive
000007fef3796cd8 4001f3b      168       System.Boolean 1 instance                0 m_PreAuthenticate
000007fef3144c48 4001f3c      130         System.Int32 1 instance                0 m_AutomaticDecompression
000007fef379ed78 4001f3d      134         System.Int32 1 instance                0 m_Aborted
000007fef3796cd8 4001f3e      169       System.Boolean 1 instance                0 m_OnceFailed
000007fef3796cd8 4001f3f      16a       System.Boolean 1 instance                1 m_Pipelined
000007fef3796cd8 4001f40      16b       System.Boolean 1 instance                0 m_Retry
000007fef3796cd8 4001f41      16c       System.Boolean 1 instance                0 m_HeadersCompleted
000007fef3796cd8 4001f42      16d       System.Boolean 1 instance                0 m_IsCurrentAuthenticationStateProxy
000007fef3796cd8 4001f43      16e       System.Boolean 1 instance                0 m_SawInitialResponse
000007fef3796cd8 4001f44      16f       System.Boolean 1 instance                0 m_BodyStarted
000007fef3796cd8 4001f45      170       System.Boolean 1 instance                1 m_RequestSubmitted
000007fef3796cd8 4001f46      171       System.Boolean 1 instance                0 m_OriginallyBuffered
000007fef3796cd8 4001f47      172       System.Boolean 1 instance                0 m_Extra401Retry
000007fef3144dd8 4001f4f      138        System.UInt32 1 instance                7 _Booleans
000007fef37d8040 4001f50      178      System.DateTime 1 instance 00000000033d3cc0 _CachedIfModifedSince
000007fef2c0db18 4001f51       30 …TimerThread+Timer 0 instance 0000000000000000 m_ContinueTimer
000007fef2c0f0c0 4001f52      180 …t.InterlockedGate 1 instance 00000000033d3cc8 m_ContinueGate
000007fef37973f8 4001f53       38        System.Object 0 instance 0000000002573160 m_PendingReturnResult
000007fef2c00018 4001f54       40 …t.LazyAsyncResult 0 instance 0000000000000000 _WriteAResult
000007fef2c00018 4001f55       48 …t.LazyAsyncResult 0 instance 00000000033d4500 _ReadAResult
000007fef2c00018 4001f56       50 …t.LazyAsyncResult 0 instance 00000000033d4778 _ConnectionAResult
000007fef2c00018 4001f57       58 …t.LazyAsyncResult 0 instance 00000000033d47c0 _ConnectionReaderAResult
000007fef3146c10 4001f58      13c         System.Int32 1 instance                0 _RequestIsAsync
000007fef316ea50 4001f59       60 …pContinueDelegate 0 instance 0000000000000000 _ContinueDelegate
000007fef2c0c9e8 4001f5a       68 ….Net.ServicePoint 0 instance 00000000026c34d0 _ServicePoint
000007fef2c23c08 4001f5b       70 …t.HttpWebResponse 0 instance 0000000000000000 _HttpResponse
000007fef37973f8 4001f5c       78        System.Object 0 instance 0000000002573160 _CoreResponse
000007fef379ed78 4001f5d      140         System.Int32 1 instance                0 _NestedWriteSideCheck
000007fef2c0c968 4001f5e       80 …Net.KnownHttpVerb 0 instance 00000000026b3b28 _Verb
000007fef2c0c968 4001f5f       88 …Net.KnownHttpVerb 0 instance 00000000026b3b28 _OriginVerb
000007fef2bffea8 4001f60       90 …bHeaderCollection 0 instance 00000000033d3e58 _HttpRequestHeaders
000007fef379fb48 4001f61       98        System.Byte[] 0 instance 0000000000000000 _WriteBuffer
000007fef3146718 4001f62      144         System.Int32 1 instance                4 _HttpWriteMode
000007fef2c28bd8 4001f63       a0           System.Uri 0 instance 00000000033d3a50 _Uri
000007fef2c28bd8 4001f64       a8           System.Uri 0 instance 00000000033d3a50 _OriginUri
000007fef3797b08 4001f65       b0        System.String 0 instance 0000000000000000 _MediaType
000007fef379b028 4001f66      128         System.Int64 1 instance -1 _ContentLength
000007fef316af60 4001f67       b8 System.Net.IWebProxy 0 instance 00000000026b39b0 _Proxy
000007fef3171a00 4001f68       c0 …em.Net.ProxyChain 0 instance 00000000033d45f0 _ProxyChain
000007fef3797b08 4001f69       c8        System.String 0 instance 0000000000000000 _ConnectionGroupName
000007fef3796cd8 4001f6a      173       System.Boolean 1 instance                0 m_InternalConnectionGroup
000007fef2c0caf8 4001f6b       d0 …thenticationState 0 instance 00000000033d4b68 _ProxyAuthenticationState
000007fef2c0caf8 4001f6c       d8 …thenticationState 0 instance 00000000033d4bb8 _ServerAuthenticationState
000007fef316ac68 4001f6d       e0 ….Net.ICredentials 0 instance 0000000000000000 _AuthInfo
000007fef2c0cbf8 4001f6e       e8 …HttpAbortDelegate 0 instance 00000000027f3c00 _AbortDelegate
000007fef2c0ea00 4001f6f       f0 …Net.ConnectStream 0 instance 0000000000000000 _SubmitWriteStream
000007fef2c0ea00 4001f70       f8 …Net.ConnectStream 0 instance 0000000000000000 _OldSubmitWriteStream
000007fef379ed78 4001f71      148         System.Int32 1 instance               50 _MaximumAllowedRedirections
000007fef379ed78 4001f72      14c         System.Int32 1 instance                0 _AutoRedirects
000007fef379ed78 4001f73      150         System.Int32 1 instance                0 _RerequestCount
000007fef379ed78 4001f74      154         System.Int32 1 instance           100000 _Timeout
000007fef2c0db18 4001f75      100 …TimerThread+Timer 0 instance 00000000033d4548 _Timer
000007fef3172938 4001f76      108 …TimerThread+Queue 0 instance 00000000026ae368 _TimerQueue
000007fef379ed78 4001f77      158         System.Int32 1 instance                0 _RequestContinueCount
000007fef379ed78 4001f78      15c         System.Int32 1 instance           300000 _ReadWriteTimeout
000007fef3169e88 4001f79      110 …t.CookieContainer 0 instance 0000000000000000 _CookieContainer
000007fef379ed78 4001f7a      160         System.Int32 1 instance               64 _MaximumResponseHeadersLength
000007fef2c0d8b0 4001f7b      118 …onnectionDelegate 0 instance 0000000000000000 _UnlockDelegate
000007fef2c04da0 4001f7c      120 …ificateCollection 0 instance 0000000000000000 _ClientCertificates
000007fef379fb48 4001f48      e80        System.Byte[] 0   shared           static HttpBytes
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022befe0 000000001ab979c0:00000000026b6928 <<
000007fef3786130 4001f49      e88 …ding.WaitCallback 0   shared           static s_EndWriteHeaders_Part2Callback
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bf000 000000001ab979c0:00000000026b6948 <<
000007fef2c1d668 4001f4a      e90 …erThread+Callback 0   shared           static s_ContinueTimeoutCallback
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bf040 000000001ab979c0:00000000026b6988 <<
000007fef3172938 4001f4b      e98 …TimerThread+Queue 0   shared           static s_ContinueTimerQueue
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bf080 000000001ab979c0:00000000026b69c8 <<
000007fef2c1d668 4001f4c      ea0 …erThread+Callback 0   shared           static s_TimeoutCallback
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bf158 000000001ab979c0:00000000026b6aa0 <<
000007fef3786130 4001f4d      ea8 …ding.WaitCallback 0   shared           static s_AbortWrapper
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022bf198 000000001ab979c0:00000000026b6ae0 <<
000007fef379ed78 4001f4e      9a8         System.Int32 1   shared           static s_UniqueGroupId
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0 000000001ab979c0:0 <<

now we getting somewhere, I see this as an HttpWebRequest Object, and in here I can access the URI:

000007fef2c28bd8 4001f63 a0 System.Uri 0 instance 00000000033d3a50 _Uri

ok, so lets dump this out

0:018> !DumpObj 00000000033d3a50
Name: System.Uri
MethodTable: 000007fef2c28bd8
EEClass: 000007fef29543b8
Size: 72(0x48) bytes
(C:\Windows\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fef3797b08 4001b8a        8        System.String 0 instance 00000000025ee6a0 m_String
000007fef3797b08 4001b8b       10        System.String 0 instance 0000000000000000 m_originalUnicodeString
000007fef2c28e60 4001b8c       18     System.UriParser 0 instance 000000000257e0c0 m_Syntax
000007fef3797b08 4001b8d       20        System.String 0 instance 0000000000000000 m_DnsSafeHost
000007fef313f598 4001b8e       30        System.UInt64 1 instance 37615763456 m_Flags
000007fef2c28fd0 4001b8f       28   System.Uri+UriInfo 0 instance 00000000033d3ac0 m_Info
000007fef3796cd8 4001b90       38       System.Boolean 1 instance                0 m_iriParsing
000007fef3797b08 4001b7f      c88        System.String 0   shared           static UriSchemeFile
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002122830 000000001ab979c0:0000000002122830 <<
000007fef3797b08 4001b80      c90        System.String 0   shared           static UriSchemeFtp
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002122810 000000001ab979c0:0000000002122810 <<
000007fef3797b08 4001b81      c98        System.String 0   shared           static UriSchemeGopher
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002122858 000000001ab979c0:0000000002122858 <<
000007fef3797b08 4001b82      ca0        System.String 0   shared           static UriSchemeHttp
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000021227c0 000000001ab979c0:00000000021227c0 <<
000007fef3797b08 4001b83      ca8        System.String 0   shared           static UriSchemeHttps
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000021227e8 000000001ab979c0:00000000021227e8 <<
000007fef3797b08 4001b84      cb0        System.String 0   shared           static UriSchemeMailto
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000021228d0 000000001ab979c0:00000000021228d0 <<
000007fef3797b08 4001b85      cb8        System.String 0   shared           static UriSchemeNews
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000021228a8 000000001ab979c0:00000000021228a8 <<
000007fef3797b08 4001b86      cc0        System.String 0   shared           static UriSchemeNntp
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002122880 000000001ab979c0:0000000002122880 <<
000007fef3797b08 4001b87      cc8        System.String 0   shared           static UriSchemeNetTcp
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002122970 000000001ab979c0:0000000002122970 <<
000007fef3797b08 4001b88      cd0        System.String 0   shared           static UriSchemeNetPipe
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002122998 000000001ab979c0:0000000002122998 <<
000007fef3797b08 4001b89      cd8        System.String 0   shared           static SchemeDelimiter
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000021227a0 000000001ab979c0:00000000021227a0 <<
000007fef31691a8 4001b91      ce0 …etSecurityManager 0   shared           static s_ManagerRef
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000000000000 000000001ab979c0:0000000000000000 <<
000007fef37973f8 4001b92      ce8        System.Object 0   shared           static s_IntranetLock
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002123388 000000001ab979c0:000000000257e280 <<
000007fef3796cd8 4001b93      970       System.Boolean 1   shared           static s_ConfigInitialized
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:1 000000001ab979c0:0 <<
000007fef31403c8 4001b94      974         System.Int32 1   shared           static s_IdnScope
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0 000000001ab979c0:0 <<
000007fef3796cd8 4001b95      978       System.Boolean 1   shared           static s_IriParsing
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0 000000001ab979c0:0 <<
000007fef37973f8 4001b96      cf0        System.Object 0   shared           static s_initLock
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000022ec898 000000001ab979c0:0000000000000000 <<
000007fef3799400 4001b97      cf8        System.Char[] 0   shared           static HexUpperChars
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000021233a0 000000001ab979c0:000000000257e298 <<
000007fef3799400 4001b98      d00        System.Char[] 0   shared           static HexLowerChars
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:00000000021233d8 000000001ab979c0:000000000257e2d0 <<
000007fef3799400 4001b99      d08        System.Char[] 0   shared           static _WSchars
                                 >> Domain:Value 00000000017ea4f0:NotInit 000000001a9d6a70:0000000002123410 000000001ab979c0:000000000257e308 <<

excellent, now lets get the string value

0:018> !DumpObj 00000000025ee6a0
Name: System.String
MethodTable: 000007fef3797b08
EEClass: 000007fef339e550
Size: 126(0x7e) bytes
(C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
String: http://Romiko.com.au/test.jsp
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fef379ed78 4000096        8         System.Int32 1 instance               51 m_arrayLength
000007fef379ed78 4000097        c         System.Int32 1 instance               50 m_stringLength
000007fef3799550 4000098       10          System.Char 1 instance               68 m_firstChar
000007fef3797b08 4000099       20        System.String 0   shared           static Empty
                                 >> Domain:Value 00000000017ea4f0:0000000002101308 000000001a9d6a70:0000000002101308 000000001ab979c0:0000000002101308 <<
000007fef3799400 400009a       28        System.Char[] 0   shared           static WhitespaceChars
                                 >> Domain:Value 00000000017ea4f0:0000000002101af8 000000001a9d6a70:0000000002109b68 000000001ab979c0:0000000002572798 <<

YAY ! We can see from here that the policy search is calling the test.jsp page, also we notice this happening in the stack from within the Page_Init. Lets go and get this source code my disassembling the module Smile

We need to tell the customer the exact code causing this problem, the way to do that is to get the source code and look at the Page_Init for the policysearch class.

Lets go back to the stack and find out which dll this code is coming from

0:018> !clrstack
OS Thread Id: 0x121c (18)
Child-SP RetAddr Call Site
000000001e88dbc0 000007fef2b3e817 System.Threading.WaitHandle.WaitOne(Int64, Boolean)
000000001e88dc00 000007fef2b48641 System.Net.LazyAsyncResult.WaitForCompletion(Boolean)
000000001e88dc60 000007fef2b47f34 System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest)
000000001e88dcf0 000007fef2b474f8 System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest, System.String)
000000001e88dd60 000007fef2b4b031 System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint)
000000001e88ddc0 000007ff00705bbb System.Net.HttpWebRequest.GetResponse()
000000001e88de50 000007fef09da30a RomikoCommon.PageBase.OnPreInit(System.EventArgs)

FOUND IT:

000000001e88de50 000007fef09da30a RomikoCommon.PageBase.OnPreInit(System.EventArgs)

what we need to do now is get this dll file, lets look at what is in the domain

0:018> !DumpDomain
————————————–
System Domain: 000007fef8933f20
LowFrequencyHeap: 000007fef8933f68
HighFrequencyHeap: 000007fef8933ff8
StubHeap: 000007fef8934088
Stage: OPEN
Name: None
————————————–
Shared Domain: 000007fef8934820
LowFrequencyHeap: 000007fef8934868
HighFrequencyHeap: 000007fef89348f8
StubHeap: 000007fef8934988
Stage: OPEN
Name: None
Assembly: 000000000172e0c0
Assembly: 000000001aa8f210
Assembly: 000000001ab529f0
Assembly: 000000001aa247e0
Assembly: 000000001aa9b220
Assembly: 000000001aac5970
Assembly: 000000000172e3c0
Assembly: 000000001aa8ef10
Assembly: 000000001aac5730
Assembly: 000000001ab52c30
Assembly: 000000001ca16590
Assembly: 00000000017e4f30
Assembly: 000000000172e540
Assembly: 000000001aa8f5d0
Assembly: 000000001ca03fd0
Assembly: 000000001aa8df50
Assembly: 00000000017e5230
Assembly: 000000001aa24ba0
Assembly: 00000000017e59b0
Assembly: 000000001aa25560
Assembly: 000000001ca16290
Assembly: 000000001aa253e0
Assembly: 000000001aa9a9e0
Assembly: 000000001aa8e0d0
Assembly: 000000001aa8f450
————————————–
Domain 1: 00000000017ea4f0
LowFrequencyHeap: 00000000017ea538
HighFrequencyHeap: 00000000017ea5c8
StubHeap: 00000000017ea658
Stage: OPEN
SecurityDescriptor: 00000000017c9d60
Name: DefaultDomain
Assembly: 000000000172e0c0 [C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll]
ClassLoader: 000000000172e180
SecurityDescriptor: 000000000172df40
Module Name
000007fef3361000 C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll
000007ff000e2568 C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\sortkey.nlp
000007ff000e2020 C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\sorttbls.nlp

I cut allot out, but lets look at this, we have our romikocommon.dll further down

Assembly: 00000000017e6c70 [C:\Windows\Microsoft.NET\Framework64\v2.0.50727\Temporary ASP.NET Files\purchasepath\2d20f8c3\9af2cafc\assembly\dl3\455d0c2c\ff027527_32ffcb01\RomikoCommon.DLL]
ClassLoader: 00000000017e6d30
SecurityDescriptor: 00000000017e6bb0
Module Name
000007ff00176880 C:\Windows\Microsoft.NET\Framework64\v2.0.50727\Temporary ASP.NET Files\purchasepath\2d20f8c3\9af2cafc\assembly\dl3\455d0c2c\ff027527_32ffcb01\CommonCommon.DLL

Nice, now, we going to dump this to a file Smile

We will use the savemodule command, but we need the base address, so issue the LM command

0:018> LM
start             end                 module name
00000000`1c170000 00000000`1c197000   System_EnterpriseServices_Wrapper_1c170000   (no symbols)
00000000`1c290000 00000000`1c2b6000   ICSharpCode_SharpZipLib   (no symbols)
00000000`1cdf0000 00000000`1d0ef000   System_Data   (export symbols)       System.Data.dll
00000000`6b570000 00000000`6bb04000   AjaxControlToolkit   (no symbols)
00000000`72a00000 00000000`72a18000   RomikoCommon_72a00000 C (no symbols)
00000000`72ee0000 00000000`72ef8000   App_Web_jweapfn4 C (no symbols)
00000000`72f00000 00000000`72f0a000   App_Web_xg2ybqom C (no symbols)
00000000`72f10000 00000000`72f18000   App_Web_1elshlii C (no symbols)
00000000`72f20000 00000000`72f38000   RomikoCommon C (no symbols)

perfect now we export the dll for .net reflector to use or ildasm, grabbing the base address above and dumping the contents as a dll.

0:018> !SaveModule 00000000`72a00000 e:\projects\debug\romikocommom.dll

remember in the stack we had this

RomikoCommon.PageBase.OnPreInit(System.EventArgs)

The savemodule has saved/dumped the dll, in reflector we can see the code causing the problem!

Here is the code

 protected override void OnPreInit(EventArgs e)
    {
        string str = (this._deployOrDev.ToUpper() == "DEV") ? this._sessionId : base.Request.QueryString["sid"];
        if (!string.IsNullOrEmpty(str))
        {
            string str2 = this.Session["sid"] as string;
            if (!string.IsNullOrEmpty(str2) && (string.Compare(str2, str, true) != 0))
            {
                this.Session.Clear();
            }
            this.Session["sid"] = str;
        }
        else
        {
            str = this.Session["sid"] as string;
        }
        if (string.IsNullOrEmpty(str))
        {
            throw new SecurityException("Session is empty.");
        }
        try
        {
            using (SecurityServicesProxy proxy = new SecurityServicesProxy())
            {
                UserCredential = proxy.GetUserCredential(str, Encrypt.EncryptData(str));
                if (UserCredential == null)
                {
                    throw new SessionExpiredException("The session is invalid. Please login again.");
                }
            }
        }
        catch (Exception exception)
        {
            HandleException(base.GetType().Name, MethodBase.GetCurrentMethod().Name, "Session Indentifier:" + str, exception, new Action<string>(this.DisplayError));
        }
        this.SetUserCredentialsForSession();
        try
        {
            WebRequest request = WebRequest.Create(this._keepAliveUrl);
            request.Headers.Add(HttpRequestHeader.Cookie, "JSESSIONID=" + str);
            request.GetResponse();
        }
        catch (Exception exception2)
        {
            _log.Info(exception2);
            throw new SessionExpiredException("Connecting Java system failed.", exception2);
        }
    }

WOW, look, every PAGE in their system is inheriting a custom pagebase, and this is called a external web page (remember test.jsp. It is doing this to keep the java session alive since the .net session is in an Iframe.

My load test always showed errors Connecting to Java system failed and here we can see why. The code should be changed, instead of all page loads calling java, rather have a background thread run and do it every 30 seconds whatever and we improve the performance of EVERY page.

we can also check that the request was not working and was waiting for the javasystem which crashed by checking cpu time.
0:018> kerneltime

Couldn’t resolve error at ‘erneltime’

0:018> .time

Debug session time: Fri May 6 15:31:14.000 2011 (UTC + 10:00)

System Uptime: 58 days 6:25:30.968

Process Uptime: 0 days 0:28:42.000

Kernel time: 0 days 0:00:01.000

User time: 0 days 0:00:05.000

it only using a fraction of a second, so it was definitely handing and waiting for a response, which of course would timeout at some point.

I hope you enjoyed this debugging session as much as I did, and thank you Tatham Oddie for geeking it up with me on this Smile

Romiko Derbynew

BizTalk– Extracting xs:Base64Binary

Hi,

Sometimes you will need to extract binary data from an XML file. It may be in a form of a string or Base64Binary, either way, you can extract it by using a helper class.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using Microsoft.XLANGs.BaseTypes;
using System.IO;


namespace Romiko.Common.Helpers
{
    [Serializable]
    public  class CustomStreamFactory : IStreamFactory
    {

        byte[] _data;

        public CustomStreamFactory(byte[] b)
        {
            _data = b;
        }

        public Stream CreateStream()
        {
            return new MemoryStream(_data);
        }

    }
}

Then all we need is to create a helper method:

using System;
using Microsoft.XLANGs.BaseTypes;

namespace Romiko.Common.Helpers
{
    [Serializable]
    public  static class Utilities
    {

        public static void GetBinary(XLANGMessage message, string base64data)
        {
            byte[] b = System.Convert.FromBase64String(base64data);
            message[0].LoadFrom(new CustomStreamFactory(b));

        }

        public static void GetBinary(XLANGMessage message, byte[] base64data)
        {
            message[0].LoadFrom(new CustomStreamFactory(base64data));

        }
    }
}

I created two overloads here, in case you have it in a byte array in XML or as a string.

In an orchestration you can then create a new message in a message construct like so:

System.Diagnostics.Trace.WriteLine(">>> PersistCompletedApplication.odx - Sending Pdf");

GeneratedPdf = new System.Xml.XmlDocument();

//Helper to Extract Binary data into a new binary message.
Romiko.Common.Helpers.Utilities.GetBinary(GeneratedPdf, MyXmlDocument.GeneratedPdf);
GeneratedPdf(FILE.ReceivedFileName) = nFileName + ".pdf";

MyXmlDocument.GeneratedPdf is a distinguished field here, so it is easier to pass the data over, not the most memory efficient, but in my case the documents (PDF) are tiny, so no need to over engineer the solution.

Thanks to Richard Seroter’s tips Smile

Cheers

BizTalk Mapper–Inline XSLT Templates–Tracking Profile Editor Limits

Hi Guys,

I hit a limit with the Tracking Profile Editor where I needed to tracking aggregate results from repeating fields in the incoming message within an orchestration. So I decided to try edit the .btt file directly and see if I can fool the BTTDeploy command. However it did not work, it detected my XPath was too fancy and rejected it. Since you can edit .btt files and get away with some minor fancy xpathing.

So the solution was to create an intermediate message to serve as the profiling message source. Hence an orchestration that has an additional construct shape that BAM profiling can consume.

As usual, the BizTalk mapper makes a mess of things when you want to loop through records and frankly I was not in the mood to use the Table Extractor/Looping “functoid” for simple sums. So Inline XSLT here we come.

In fact 90% of my time I will opt for xslt than the biztalk mapper, purely due to the mapper is reserved for very simple mappings. XSLT’s are by far easier to read than the BizTalk Mapper when it comes to complex conditions.

So, below is a message constructed that BAM TPE can consume easily without hacking the .btt file.

It is a bit odd to create a message that does nothing in the orchestration at first look, but it serves as an invaluable source of information for BAM profiling.

I opted for a BizTalk Map to create the BAM message, to keep maintenance of schema’s visible and no dirty XmlDocument.Load statements in the orchestration Smile Yes you know who you are!

Here is the sample MAP that takes input message and creates a BAM message for TPE to consume:

The Scripting Functoids will then have the following for calculating Annual Sums.

Notice that Param1 is the Input element relative to the Script Functoid, this is usefull, so I can grab a subset of the xml data to pass into the scripting functoid, in this case Benefits.

<xsl:template name="AnnualTotalPremium">
<xsl:param name="param1" />
    <xsl:variable name="Premium" select="sum(//*[local-name()='Premium' and namespace-uri()='urn:Romiko.Submission']) * 12" />
 <xsl:element name="AnnualTotalPremium" >
          <xsl:value-of select="$Premium"/>
 </xsl:element>
  </xsl:template>

Below is a more interesting one that takes record counts as a second parameter for average benefits per policy premium.

<xsl:template name="AveragePolicyAnnualPremium">
<xsl:param name="param1" />
<xsl:param name="param2" />
    <xsl:variable name="Premium" select="(sum(//*[local-name()='Premium' and namespace-uri()='urn:Romiko.Submission']) * 12) div $param2 " />
 <xsl:element name="AveragePolicyAnnualPremium" >
          <xsl:value-of select="$Premium"/>
 </xsl:element>
  </xsl:template>

So, I hope this inspires anyone needed to collect data for BAM that is a bit too more for TPE to handle and you do not want to implement custom BAM Interceptors.

BizTalk–WCF Oracle Composite Transactions and a small think on ESB’s

When using the BizTalk Adapter Pack 2.0/WCF LOB on BizTalk 2009.

Remember with Oracle WCF Adapter to install these before the oracle client :

WCF LOB SDK SP2
BizTalk Adapter Pack 2.0

When configuring the operations in the WCF adapter ensure you fully qualify the operation name that matches the logical orchestration port in the WCF ADAPTER Bindings.

If it does not have this, then you will get errors where an InvalidOperationException will occur.

<Operation Name=”LogicalPortOperationName” Action=”http://Microsoft.LobServices.OracleDB/2007/03/CompositeOperation” />

You can read more here.

http://msdn.microsoft.com/en-us/library/dd788415(v=bts.10).aspx

Since we use ambient Transactions and Nested transaction for the batch, also ensure the Oracle Microsoft Transaction Services is installed. I use the 64 bit version which runs fine with Biztalk in 32 bit mode. However I have BOTH oracle 32bit and 64 bit client (Oracle Dara Access Components 11.2.0.7.20 or higher, since the 32 bit is needed for Developer Design time work with the ConsumeAdapterService Wizards on the developer box. Use different install paths for 32 bit and 64 bit as well.

HINT: Generate all insert, update, select, delete operations in one go on the ConsumeAdapterService wizard, then you never have namespace clashes if you need the operations later!

Another point is to Update the machine.config if you use a later version of the oracle client on the Development box, else the LOB wizard will not work.

 <!-- Add in Machine.Config -->
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="Oracle.DataAccess" publicKeyToken="89b483f429c47342" culture="neutral" />
<bindingRedirect oldVersion="2.111.7.0" newVersion="2.112.1.2"/>
</dependentAssembly></assemblyBinding>

Here is some notes on it:

1. Ensure you have the Adapters downloaded.

2. Install WCF LOB Adapter SDK SP2 x64.

Ensure you include the TOOLS on the development server.

If you do not install the tool, then Visual Studio dev environment will not have the consume adapter feature!

3. Install BizTalk adapter Pack 2.0 for 2009, both for x86 and x64. Get from MSDN Subscription

Oracle Client Install

4. Install Oracle Data Access Components (ODAC) 11.1.0.7.20 bundled with Oracle Instant Client making sure no other client versions are present before installation. 32 First and then 64 Bit.

5. Use the follow name for BASE: Oracle and software location fro 64 bit is \11, 32 bit is 11-32

Keep the above install path as short as possible

7. The Visual Studio system used an older version of oracle at DESIGN TIME on a dev box, the following int he machine config will fix this error:

Fix:

Add to Machine.config in C:\Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG

<!– Add in Machine.Config –>

</dependentAssembly>

</assemblyBinding>

8. Install the Transaction Oracle Service for Microsoft Transaction Services Support 64 bit

Ensure you choose the correct Software location corresponding to the 64 bit folders. 64 bit is d:\Oracle\11

9. Restart the Machine.

Microsoft ESB Toolkit for BizTalk

I am in disagreement with how Microsoft has bundled and named this toolkit. For one BizTalk is not an ESB it is a message broker. So the toolkit should be renamed. An ESB is something that is 100% distributed with no centralised publisher. An ESB is similar to a PCI Bus, Ethernet or CPU Microprocessor Bus Architecture, where all endpoints participate. If you wanted a pure ESB with BizTalk then all endpoints would be BizTalk, which would never be feasible from a financial perspective. A good example of an ESB would be NServiceBus (http://nservicebus.net/).

I love working with BizTalk, but it should never be abused or sold as an ESB. Perhaps a blog about this in depth at some point when I get a chance will be nice.

It is very dangerous selling BizTalk as an ESB to clients, you putting them down a path that is not going to look pretty in 5 years time. .Net to .Net communication does not need a BizTalk Broker in-between it. BizTalk is ideal for integration services and that’s it.

BizTalk can act as a Integration layer as ONE OF the endpoints in an ESB. If Ethernet or PCI was used a central server to publish/subscribe messages, I can tell you that it would not be a very scalable and reliable system.

It is not only BizTalk that is a culprit for misunderstanding ESB’s all major vendors are in the same boat trying to sell ESB servers like IBM Websphere and Oracle SOA. And I can understand why, centralised = licensing made easy.

Remember a true ESB cannot be touched and pointed at, it is a DISTRIBUTED ARCHITECTURE with each endpoint managing it’s own persistence, any endpoint can just go down when it wants and be turned on when it wants to.

Data Recovery of Formatted Disk Drive

Hi,

Today I was a bit silly and accidentally formatted the wrong drive with all my work on it. After hours of playing with different disk recovery tools, I finally found one that I highly recommend, as it keeps the folder structure in tact. The free version will restore up to 1GB, so it is nice to play with to see what you can recover before buying it, worth it!!

It is called, EASEUS Data Recovery Wizard 5.0.1.

So if you need to recover any files from deleted partitions, then I highly recommend this.

Cheers

NHibernate/NHProf Dependency Injection and Sessions

Hi,

I recently been working with the Castle.Components.Scheduler and Jobs. The job runs in a NServiceBus host on Version: 2.0.0.1219

Basically, I created a Job that implemented IJob and had constructor injection for the NHibernate Session.

<!–Castle.Components.Scheduler–>
<component id=”TheJob”
service=”Castle.Components.Scheduler.IJob”
type=”Romiko.TheJob, Romiko”
lifestyle=”transient”/>

  public class TheJob : TheJobBase, IJob
    {
        public TheJob(Idependency1 dep1, Idependency2 dep2, ISession myNHSession)
            : base(dep1, dep2,myNHSession) )
        {
        }

        public override bool Execute(JobExecutionContext context)
        {
            Process(context);
            return true;
        }

I colleague of mine, Matthew Wills, from Readify has been telling me how much NHProf can save you time, and indeed today it was proved Smile

The process method above runs a series of job steps which execute 3 SQL select staments.

NHProf was moaning about the Session being shared:

Alert: Using a single session in multiple threads is likely a bug

Alert: Use of implicit transactions is discouraged

We noticed that subsequent job runs were using the same session!

See above, that 1 session is being used by the job, which runs all the time, this is not good at all for performance. The above shows the job running 4 to 5 times, each time executing 3 statements.

So, what we did, was remove the Session from the constructor injection, as we found a bug where a session was being created on one thread (in constructor) and then used in another thread (in the execute method).

To solve the problem above, we used Windsor to resolve the dependency in the execute method instead.

 public override bool Execute(JobExecutionContext context)
        {
            _session = WindsorAccessor.Container.Resolve<ISession>();
            Process(context);
            return true;
        }

Now, when we run NHProf, we get a sessoin per job and no longer hold a session for along time on a long running process within NServiceBus.

Lets rerun the code now and see how it works when we run 4 to 5 jobs Smile

From the above, we can now see different sessions per JOB CYCLE. This will consume less resources and we solved the red nasty warnings in NHProf.

Mathew pointed out that we can go further, and optimised the 3 transactions above to be called in BATCH. So we can optimise the QueryOver methods to use Future operations, which will then batch the 3 transactions, which I can then use from within the job process, thus reducing each job cycle from 3 database calls to 1.

What i did was use something along the line of Future in my code and then delay looping through my result set until the last minute, so some code shuffling around Smile

 return Session.QueryOver<RomikoTable1>()
                .Where(x => x.CreatedDate >= RegDate
                    && x.CreatedDate <= dateUnreg
                    && x.Count == pollCount
                    && x.BobTheBuilder == null)
                .JoinQueryOver(y => y.TableTwo)
                .JoinQueryOver(z => z.PostManPat).Where(a => a.Id == PostManPatId)
                .Future<RomikoTable1>();

Now check the BATCHING, hence less connections, notice duration is for all of them, where before you see a duration for each query.

I would like to thank Matthew Wills for show me some new cool tools that will definitely help me along the road in the future!

I would say that NHProf is definitely worth buying!

Install NHibernate, FluentNHibernate viaNuGet

PM> install-package NHibernate
‘Iesi.Collections (≥ 1.0.1)’ not installed. Attempting to retrieve dependency from source…
Done
‘Antlr (≥ 3.1.3.42154)’ not installed. Attempting to retrieve dependency from source…
Done
‘Castle.Core (≥ 2.5.1)’ not installed. Attempting to retrieve dependency from source…
Done
Successfully installed ‘Iesi.Collections 1.0.1’
Successfully installed ‘Antlr 3.1.3.42154’
Successfully installed ‘Castle.Core 2.5.1’
Successfully installed ‘NHibernate 3.0.0.4000’
Successfully added ‘Iesi.Collections 1.0.1’ to Common
Successfully added ‘Antlr 3.1.3.42154’ to .Common
Successfully added ‘Castle.Core 2.5.1’ to Common
Successfully added ‘NHibernate 3.0.0.4000’ to .Common

PM> install-package FluentNhibernate
‘Iesi.Collections 1.0.1’ already installed
‘Antlr 3.1.3.42154’ already installed
‘Castle.Core 2.5.1’ already installed
‘NHibernate 3.0.0.4000’ already installed
Successfully installed ‘FluentNHibernate 1.1.1.694’
Common already has a reference to ‘Iesi.Collections 1.0.1’
Common already has a reference to ‘Antlr 3.1.3.42154’
Common already has a reference to ‘Castle.Core 2.5.1’
Common already has a reference to ‘NHibernate 3.0.0.4000’
Successfully added ‘FluentNHibernate 1.1.1.694’ to Common

PM> Add-BindingRedirects

Name                  : NHibernate
Culture               : neutral
PublicKeyToken        : aa95f207798dfdb4
ProcessorArchitecture :
NewVersion            : 3.0.0.4000
OldVersion            : 0.0.0.0-3.0.0.4000
CodeBaseHref          :
CodeBaseVersion       :
PublisherPolicy       :

PM>

If you want, you can manually install NHIbernate and then NuGet FluentNHibernate if you want the Trunk version of the assemblies. The binding direct will redirect minor version bindings for FluentNhibernate.

Musings on Biological Sensors

AFM cantilever probe modes

In stress sensing mode, the reaction from the molecules or sample occur on one side of the cantilever, when there is a change in the total surface free energy, this will the equate to a change in surface stress, which will cause the cantilever to bend. The magnitude and direction of the bending can then be measures using optical means or electrical means by use of lasers and piezo-resistors. The spring constant is inversely proportional to the sensitivity of the device.

In mass sensing mode, the cantilever is excited mechanically, this causes it to vibrate at it’s resonant frequency. The change is mass can be detected by the shift in resonant frequency once a biological molecule attached to the cantilever. Optical or electrical means can be used to measure these shifts.

The above two modes is similar to the contact and non-contact mode on an AFM probe, all methods cause the cantilever to bend either by deflection (Contact Mode), therefore contact mode is similar to stress sensing mode.

Mass sensing mode is similar to Non-contact mode, since here the resonant frequency of the cantilever is used to measure the deflection of the cantilever, where oscillation in the vertical direction are excited and a piezoelectric actuator can be used to pick up the oscillations.

(b) Mass of virus particle

k=0.006 N m^-1

Therefore the mass will be the difference in mass changes over the two frequencies.

f₁≈1.35 x 10⁶ Hz

f₂≈1.42 x 10⁶ Hz

m₁= 8.34×10^-17 kg

m₂= 7.54×10^-17 kg

∆m≈8.02×10^-18 kg which is the mass of the virus particle.

So a the virus is approximately 8.02 fg (femto grams).

(c)Cantilever modifications

Biding sites may be attached to the cantilever to attract a certain ligand to bind to the cantilever. Gold is a excellent coating to apply to a silicon based cantilever, since this provides a good surface for capture probes to bind to. The cantilever can then be used in stress/mass-sensing mode.

So in summary:

· Create a silicon based cantilever

· Coat with gold

· Attach capture probes to cantilever surface which will match the target ligand

(d)Feasibility and detection (Tuning In)

A cantilever will need to have a high resonant frequency as possible to minimise vibration from external noise. Therefore the lowest resonant frequency of the cantilever should be at least 10 times greater than the highest frequency present in the room. Therefore this specific cantilever’s resonant frequency should be greater than 200kHz.

The feasibility will be to see if the cantilever can vibrate with the range of the resonant frequency required to detect this mass size, which is:

k=0.006 N m^-1

m=0.20 x 10^-18 kg

f=2.76 * 10⁷ Hz

f≈27.6 MHz
This is well out of the range my one order of magnitude, so the cantilever will not be able to detect a mass of 0.2 fg.

The maximum frequency detected is 1.42 x 10⁶ Hz. It is like trying to tune a radio to listen to microwaves, well close!

However, I wonder if a cantilever has ever been designed to work in amplitude mode, like AM radios?

Musings on CMOS basics

Where did CMOS come from and how did old CMOS chips get made, with large gate lengths, I mean, they now at 30nm in production with Intel

The reason is a p-n junction is made of a metal for the contact and conduction of electricity, and oxide for insulation to establish a potential difference and a semi-conductor that is doped p-type and another n-type, hence complimentary to each other’s type of doping, to establish drift, diffusion and a depletion region.
They are wide spread in portable devices such as camera’s due to the use of photodiodes which are sensitive to light and can generate current, thus acting as sensitive detectors/ switches of photons, depending on how they are arranges and connected. They are relatively inexpensive to manufacture on a large scale due to the maturity of such devices in the computing industry, where established lithography and fabrication techniques have been perfected over the years and feature sizes have been following Moore’s law.
With slight modifications these transistors can be addressed very easily with an electrical grid system, which is sufficient for transporting and processing the information carried across the wires and establishing an addressing system.

(a) A switch is a simple MOS p-n junction, when there is a bias voltage the switch is on as there is a conduction path between the terminals.
An inverter is two switches connected in series.
A latch is an arrangement of switches that can be set into two different states and provides an electrical output, can be set to either state electrically and is also stable.
Two switches will create an inverter, for CMOS memory this is not practical as it will store the opposite state desired, so a 1 would be a 0 and vice versa and also the output is indeterminate. The solution is 4 switches, so that there are two inverters, one feeding the other, this is what is used to create a memory latch, So sending a 1 or 0 will result in a 1 or 0 being stored on the output and it is determinate, so loss of power will keep the output the same.2
No drain will occur when a switch is open, as the conducting path through the gate terminal has no applied bias applied.

(b) Most of the process steps in the fabrication of CSMOS require photolithography to define areas that are etched or deposited. The photolithography uses a photoresist polymer which is then exposed to UV light and the weaker parts of the resists are washed off, producing a positive or negative depending on the resist used. Resolution of features are limited by these photolithography.
To create CMOS we will need an adjacent p-well and n-well, kitted out with their respective components:

· Gate oxide

· Gate

· Source

· Drain

· P⁺Substrate and p^– layer

· Electrical contacts

One of the most important factors is the gate length and the doping requirements used for these gates.

Some of the challenges involved are:

v Isolating switches from each other

o The p-Type and N-Type switches need to be isolated from each other, this is achieved by Shallow Trench Isolation. The isolation trenches are etched around 400nm deep into the silicon using LPCVD for the sacrificial layer.

o Then a plasma etch is used to form the trenches. This trench is the barrier between switches. There must be no sharp points so the STI etch agent used is HBrO₂.

o The trenches must be filled with an insulator, which uses CVD process and the chemical TEOS. The capacitance between each switch has an impact on the speed and power of the chip, so the dielectric constant of the insulator must be as low as possible.

v Defining wells and the role of photolithography

o Wells are the doped silicon which will form the junction with the source and drain contacts. Alternating p/n type wells are masked off by photolithography. Ion bombardment implants donor or acceptor atoms in the silicon surface. The doping stage defines the polarity of the final channel and the sign of the gate voltage used to activate the switch. So the doping concentrations must be correctly applied.

o Width on old chips is around 200nm, new ones will be much smaller now.

v Formation of the Gate Oxide (Insulator)

o The integrity of the gate oxide to act as a insulator is crucial, as electrical breakdown must not occur here. The trick is to control the thickness of the gate oxide to be as thin as possible and use a high quality oxide with high-k dielectrics, due to the capacitance affecting the performance of the device. This oxide is deposited using conventional lithography techniques and uniformity is crucial to a quality gate oxide. The gate length will be in the region of 250nm, which will sit on top of this oxide. The thickness is around 1.2 +- 0.5 nm . With new techniques we looking at 30nm production quality as of 2010.

v Making electrical connections to the switches

o Metals can be deposited using sputter deposition such as PVD. Advanced PVD is needed here to ensure a high degree of directionality, where control of sidewall angle is need for criss-cross grid system, CL₂/HBr plasma sis appropriate for this.

This is due to the fact that reducing the gate length will reduce the power consumption of the chip and also allows creating faster gate architectures by increasing the number of transistors in the chip, so smaller feature sizes means more processing power. We need oxides with a High-K dielectrics, hafnium oxide is a promising material since it has a dielectric constant which is 5 times more than silica, this means we can have 5 times the capacitance at the price of the same thickness!

Reverse Bias in Solar Cells

Solar Cell Basics

A photo cell (solar cell) is a p-n junction. A photo cell convert’s light energy into electrical energy and the photons is the current source.

When photons incident the silicon, it either travels through the material if its energy is lower than the band gap energy of the silicon semiconductor (transmission), or is absorbed by the silicon if its energy is higher than the band gap energy , or reflected. If the energy is high enough then an electron-hole pair is produced, and the electron and hole are separated by depletion region of the p-n junction and a current is generated through a connected circuit , since the positive carriers prefer the p-type material and the negative carriers prefer the n-type material, this increases the conductivity of the material.

Diagram showing electron-hole pairs being affected by incident photons.

When no light or photons are incident on the solar cell, then there is a balance between drift and diffusion and the number of electrons and holes are equal between the two regions, and no net current flow is present.

The light detector or solar cell is in fact a photodiode where is will readily conduct current in one direction and hardly at all in the other. With photocells, we need to apply a reverse bias in order to increase the effect of an internal electric field in the junction, thus causing an imbalance of drift and diffusion across the depletion region.

For the photocell, the holes tend to enjoy staying in the p region and the electrons in the n region, reverse bias enhances this tendency. When a stream of photons are absorbed into the silicon and are within the transition region, then electron-hole pairs are formed, then photon generated charge carriers will prefer their n/p type material respectively.

This will result in an additional reverse current through the junction.

In the above diagram, you will notice electron-hole pairs outside of the depletion region, still being able to make the transition, this will be due if they live long enough to slip into the depletion region.

In practise it is preferred to have one of the p-type or n-type materials to be much less than the other to ensure the depletion region extends further on one side only, this ensures. Assuming the p-type is much thinner, then the absorption length of the silicon which is the reciprocal of the absorption co-efficient will need to be tuned with some light doping to ensure the transition region is wider which will increase the probability that incident photons will occur around the transition region where electron-hole pairs are close to the depletion region, else the pairs will recombine due to lower lifetime constants. Therefore the doping requirements will be that one of the material types will have a weaker doping than the other. Also the p-Type material is in contact with the substrate and the initial layers close to the substrate where the electrical contacts are located will have a high doping population P⁺the p layer closer to the junction will have a lower doping population p^–.

Diffusion is responsible for the leakage of charge across the p-n junction when it is experiencing reverse bias due to thermal energy, charge carriers move from high concentration to low concentration. Diffusion pushes minority charge towards the edge of the junction within the depletion region where it is swept across to the other side and becomes a majority charge carrier. See the diagram above, where charge carries are swept across (charge separation). Diffusion is negligible in reverse bias

Drift is where the positive charge will drift in the direction of the positive electric field (negative end of battery) and negative charge will go the opposite way. The total current flowing through the depletion region under reverse biasing is made up of mostly of minority carrier drift.

There are various manufacture processes that can create drift or diffusion dominant diodes depending on the dopant used (Science Direct, 2009)

We will look at the materials used in the next section, that and a battery, where the battery will be used to store the current generated by the solar cell.

The materials required to fabricate a p-n junction are:

· Metal or similar material that is electrically conducting

· Insulator to isolate the voltage applied

· A Semiconductor where the conductivity can be altered by doping and bias voltages

The device to build is in fact a complementary metal oxide semiconductor field effect transistor (CMOSFET). This is where n-type and p-type devices are situated side by side. We will focus on a n-channel silicon MOSFET.

The junction is the region where the n-type and the p-type silicon are in contact with each other.

Diagram representing a p-n junction

As we can see from the above, when in equilibrium the electrons “diffuse” across and combine with the holes; this area is known as the depletion region.

When a current is applied by making the p-type more positive and the n-type more negative, it will flow readily in one direction (Forward Bias) but not in the other (Reverse Bias).

To form a pn junction different conductive regions must be adjacent to each other and a rapid spatial change in the dopant species from donor to acceptor must be created.

The reverse bias enhances the large field potential in the device and thus charge carriers are rapidly accelerated to the respective anode and cathode electrodes, thus contributing to the current.

What about forward bias?

In forward bias, the internal field would essentially be destroyed and the charge carriers would move very slowly and hence your solar cell would be less effective.

Furthermore applying a forward bias would temporarily reduce or destroy the junction as it would induce carriers to move in the same direction as the diffusive flow, thus smoothing out the carrier differences.