Hello, I'm Feng Hui, a researcher of Microsoft MVP laboratory in this issue. This article mainly introduces how to use Windbg to analyze the memory problems in the application process, from managed heap to unmanaged heap and to memory allocation. Let's explore it together next.
Recently, several friends have used our Magicodes.IE Feedback: the memory soared during export. Next, let's see what caused it through Windbg.
Let's take a look at the current application memory usage through address -summary.
0:000> !address -summary --- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal Free 581 7df8`ef0c9000 ( 125.972 TB) 98.42% <unknown> 1678 206`ffb9e000 ( 2.027 TB) 99.99% 1.58% Image 950 0`064fd000 ( 100.988 MB) 0.00% 0.00% Heap 58 0`050f6000 ( 80.961 MB) 0.00% 0.00% Stack 156 0`04380000 ( 67.500 MB) 0.00% 0.00% Other 11 0`019ad000 ( 25.676 MB) 0.00% 0.00% TEB 52 0`00068000 ( 416.000 kB) 0.00% 0.00% PEB 1 0`00001000 ( 4.000 kB) 0.00% 0.00% --- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_MAPPED 282 200`038a6000 ( 2.000 TB) 98.64% 1.56% MEM_PRIVATE 1674 7`07184000 ( 28.111 GB) 1.35% 0.02% MEM_IMAGE 950 0`064fd000 ( 100.988 MB) 0.00% 0.00% --- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal MEM_FREE 581 7df8`ef0c9000 ( 125.972 TB) 98.42% MEM_RESERVE 295 205`f8659000 ( 2.023 TB) 99.79% 1.58% MEM_COMMIT 2611 1`188ce000 ( 4.384 GB) 0.21% 0.00% --- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal PAGE_READWRITE 1595 1`0dc6c000 ( 4.215 GB) 0.20% 0.00% PAGE_EXECUTE_READ 156 0`04d66000 ( 77.398 MB) 0.00% 0.00% PAGE_READONLY 600 0`03851000 ( 56.316 MB) 0.00% 0.00% PAGE_NOACCESS 99 0`021f2000 ( 33.945 MB) 0.00% 0.00% PAGE_EXECUTE_READWRITE 19 0`0027b000 ( 2.480 MB) 0.00% 0.00% PAGE_WRITECOPY 90 0`001a0000 ( 1.625 MB) 0.00% 0.00% PAGE_READWRITE | PAGE_GUARD 52 0`0009e000 ( 632.000 kB) 0.00% 0.00% --- Largest Region by Usage ----------- Base Address -------- Region Size ---------- Free 189`0413c000 7c6b`01ed4000 ( 124.418 TB) <unknown> 7dfb`2a153000 1f9`bd2ef000 ( 1.976 TB) Image 7ffc`883c1000 0`009ba000 ( 9.727 MB) Heap 183`0e9a1000 0`00f01000 ( 15.004 MB) Stack 37`62980000 0`0017b000 ( 1.480 MB) Other 183`77707000 0`01775000 ( 23.457 MB) TEB 37`62600000 0`00002000 ( 8.000 kB) PEB 37`627dd000 0`00001000 ( 4.000 kB)
MEM_COMMIT occupies 4.384G. Next, we use eeheap -gc to check the managed heap.
0:000> !eeheap -gc GC Allocated Heap Size: Size: 0x11ac2568 (296494440) bytes. GC Committed Heap Size: Size: 0x120e7000 (302936064) bytes.
According to these memories, it seems that the problem is not here. A large amount of memory still appears in unmanaged memory. Let's take a look at the Windows NT heap. In fact, most of the user heap allocators in windows are in ntdll NT heap manager API in DLL (RtlAllocateHeap/RtlFreeHeap), such as malloc/free and new/delete in C, SysAllocString in COM framework and LocalAlloc, GlobalAlloc and heaprealloc in Win32. Although these allocators will create different heaps to store their memory, they will eventually call the NT heap in ntdll.dll.
0:000> !heap -s
************************************************************************************************************************
NT HEAP STATS BELOW
************************************************************************************************************************
NtGlobalFlag enables following debugging aids for new heaps:
stack back traces
LFH Key : 0x7cfd4cc2db4ddb4d
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-------------------------------------------------------------------------------------
0000018378fd0000 08000002 65128 15296 64928 1720 177 17 2 c LFH
External fragmentation 11 % (177 free blocks)
00000183775c0000 08008000 64 4 64 2 1 1 0 0
000001837aa90000 08001002 1280 108 1080 26 3 2 0 0 LFH
000001837ad20000 08001002 60 8 60 2 1 1 0 0
000001837aca0000 08041002 60 8 60 5 1 1 0 0
000001887bfd0000 08001002 60 20 60 1 2 1 0 0
000001830cf30000 08001002 3324 1364 3124 19 10 3 0 0 LFH
000001830ce30000 08001002 60 8 60 5 1 1 0 0
-------------------------------------------------------------------------------------The output result is as shown above. The NT heap has a small content What's the reason? Well, according to maoni It seems that there is something wrong with the verification.
On Windows, all user mode allocations ultimately obtain memory through VirtualAlloc, bitmaps or GC heap.
The difference is whether you call VirtualAlloc directly or call it in other ways. If it is unmanaged memory, GC does not care about it, and of course, it does not know their existence.
GC does not govern these memories, so there is still a problem with the code we write. Let's go back and consider another thing, "when the export is in progress, the memory will increase greatly, and the memory will decrease after the export is completed". Let's take a look at the code, as shown below. In fact, what we now understand is that these memories must have been "held" and not released during our execution.
app.MapGet("/excel", async content =>
{
string path = Path.Combine(Directory.GetCurrentDirectory(), "test.xlsx");
List<TestDto> list = new();
for (int i = 0; i < 400; i++)
{
list.Add(new TestDto
{
ImageUrl = "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fup.enterdesk.com%2Fedpic_source%2F53%2F0a%2Fda%2F530adad966630fce548cd408237ff200.jpg&refer=http%3A%2F%2Fup.enterdesk.com&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=jpeg?sec=1641193100&t=417a589da8c9ba3103ed74c33fbd6c70"
});
}
Stopwatch stopwatch = Stopwatch.StartNew();
ExcelExporter exporter = new ExcelExporter();
await exporter.Export(path, list);
stopwatch.Stop();
await content.Response.WriteAsync(stopwatch.Elapsed.TotalSeconds.ToString());
});According to the memory performance and our theory, we continue to use Windbg to check. Now we can find that these objects are finally recovered by GC. With the theory, we continue to conceive that GC knows which objects can be terminated, right? And they call their finalizers when they become unreachable, and these finalization objects will be recorded in the GC using the finalization queue. So can we check? As shown below, let's take a look.
0:000> !finalizequeue
----------------------------------
Statistics for all finalizable objects (including all objects ready for finalization):
MT Count TotalSize Class Name
00007ffc2dc23818 1 24 System.Net.Security.SafeCredentialReference
00007ffc2dac4238 1 24 System.WeakReference
00007ffc2d6eb908 1 24 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerOptions, Microsoft.AspNetCore.Server.Kestrel.Core]]
00007ffc2d6e4120 1 24 System.WeakReference`1[[System.Runtime.Loader.AssemblyLoadContext, System.Private.CoreLib]]
00007ffc2d572b68 1 24 System.WeakReference`1[[Microsoft.Extensions.DependencyInjection.ServiceProvider, Microsoft.Extensions.DependencyInjection]]
00007ffc2d429258 1 24 System.WeakReference`1[[System.IO.FileSystemWatcher, System.IO.FileSystem.Watcher]]
00007ffc2dd15c20 1 32 Microsoft.Win32.SafeHandles.SafeBCryptAlgorithmHandle
00007ffc2d6de4d8 1 32 Internal.Cryptography.Pal.Native.SafeLocalAllocHandle
00007ffc2d68fa00 1 32 Internal.Cryptography.Pal.Native.SafeCertStoreHandle
00007ffc2d3a5cc0 1 32 System.Net.Quic.Implementations.MsQuic.Internal.SafeMsQuicRegistrationHandle
00007ffc2db390c8 1 40 Interop+WinHttp+SafeWinHttpHandle
00007ffc2d69a420 1 40 Internal.Cryptography.Pal.Native.SafeCertContextHandle
00007ffc2d5bea18 1 40 System.Diagnostics.EventLog
00007ffc2dc29a38 1 48 System.Net.Security.SafeFreeCredential_SECURITY
00007ffc2d963f80 2 48 System.WeakReference`1[[System.Text.RegularExpressions.RegexReplacement, System.Text.RegularExpressions]]
00007ffc2d7a3750 2 48 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection, Microsoft.AspNetCore.Server.Kestrel.Core]]
00007ffc2d685e10 1 56 System.Runtime.CompilerServices.ConditionalWeakTable`2+Container[[System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1+ThreadLocalArray[[System.Char, System.Private.CoreLib]][], System.Private.CoreLib],[System.Object, System.Private.CoreLib]]
00007ffc2d44c4d0 1 56 System.Runtime.CompilerServices.ConditionalWeakTable`2+Container[[System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1+ThreadLocalArray[[System.Byte, System.Private.CoreLib]][], System.Private.CoreLib],[System.Object, System.Private.CoreLib]]
00007ffc2d96be68 1 64 CellStore`1[[System.Uri, System.Private.Uri]]
00007ffc2d96b780 1 64 FlagCellStore
00007ffc2d96af48 1 64 CellStore`1[[System.Object, System.Private.CoreLib]]
00007ffc2d96a5b8 1 64 CellStore`1[[OfficeOpenXml.ExcelCoreValue, Magicodes.IE.EPPlus]]
00007ffc2d6ddab8 2 64 Internal.Cryptography.Pal.Native.SafeChainEngineHandle
00007ffc2d69d528 2 64 Internal.Win32.SafeHandles.SafeRegistryHandle
00007ffc2d685bc8 2 64 Microsoft.Win32.SafeHandles.SafeWaitHandle
00007ffc2d685280 3 72 System.Threading.ThreadInt64PersistentCounter+ThreadLocalNodeFinalizationHelper
00007ffc2d5f5f50 3 72 System.Runtime.InteropServices.PosixSignalRegistration
00007ffc2d4299d0 1 72 Microsoft.Win32.SafeHandles.SafeFileHandle
00007ffc2d6e40b8 1 80 System.Runtime.Loader.DefaultAssemblyLoadContext
00007ffc2dac9ed0 2 96 PageIndex
00007ffc2d96d0c8 2 96 ColumnIndex
00007ffc2d464470 3 120 System.Gen2GcCallback
00007ffc2d40a620 1 120 System.IO.FileSystemWatcher
00007ffc2d96bc18 2 128 CellStore`1[[System.Int32, System.Private.CoreLib]]
00007ffc2dac20c8 2 144 System.Reflection.Emit.DynamicResolver
00007ffc2d680f10 3 144 System.Threading.LowLevelLock
00007ffc2d683c48 3 168 System.Threading.ThreadPoolWorkQueueThreadLocals
00007ffc2d681e80 1 176 System.Threading.LowLevelLifoSemaphore
00007ffc2dc25ef0 1 184 System.Collections.Concurrent.CDSCollectionETWBCLProvider
00007ffc2db8e658 1 184 System.Net.NetEventSource
00007ffc2db8c378 1 184 System.Net.NetEventSource
00007ffc2db38f90 1 184 System.Net.NetEventSource
00007ffc2d90c658 1 184 Microsoft.IO.RecyclableMemoryStreamManager+Events
00007ffc2d689b48 1 184 Microsoft.AspNetCore.Certificates.Generation.CertificateManager+CertificateManagerEventSource
00007ffc2d66f9f8 1 184 System.Diagnostics.Tracing.FrameworkEventSource
00007ffc2d66b720 1 184 System.Net.NetEventSource
00007ffc2d44d128 1 184 System.Buffers.ArrayPoolEventSource
00007ffc2d2e2ec8 1 184 System.Diagnostics.Tracing.NativeRuntimeEventSource
00007ffc2d694e10 1 192 System.Threading.Tasks.TplEventSource
00007ffc2d572ab0 1 192 Microsoft.Extensions.DependencyInjection.DependencyInjectionEventSource
00007ffc2d505f00 1 200 Microsoft.Extensions.Logging.EventSource.LoggingEventSource
00007ffc2db8ade8 1 224 System.Net.NameResolutionTelemetry
00007ffc2d428b08 7 224 System.Threading.PreAllocatedOverlapped
00007ffc2d563c78 1 232 System.Diagnostics.DiagnosticSourceEventSource
00007ffc2d61fe88 1 240 Microsoft.AspNetCore.Hosting.HostingEventSource
00007ffc2db6b788 8 256 System.Threading.TimerQueue+AppDomainTimerSafeHandle
00007ffc2d690270 1 280 System.Net.Sockets.SocketsTelemetry
00007ffc2db6bc80 1 296 System.Net.Http.HttpTelemetry
00007ffc2d68b998 1 336 Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelEventSource
00007ffc2dc21998 1 360 System.Net.Security.NetSecurityTelemetry
00007ffc2d2dae28 1 384 System.Diagnostics.Tracing.RuntimeEventSource
00007ffc2d66ad60 10 480 System.Net.Sockets.SafeSocketHandle
00007ffc2d2e0240 21 504 System.WeakReference`1[[System.Diagnostics.Tracing.EventSource, System.Private.CoreLib]]
00007ffc2d2b0538 9 648 System.Threading.Thread
00007ffc2d77a188 2 704 Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketReceiver
00007ffc2d90cec0 6 960 Microsoft.IO.RecyclableMemoryStream
00007ffc2d5fc658 10 1280 System.Net.Sockets.Socket
00007ffc2d68d898 4 1536 System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs
00007ffc2d2dc778 42 4704 System.Diagnostics.Tracing.EventSource+OverrideEventProvider
00007ffc2daec058 356 14240 System.Drawing.Bitmap
Total 553 objectsWOW!!!， Look at the 356 systems above Drawing. Bitmap is waiting for recycling. It seems that this is our influencing factor. Let's check the code.
try
{
cell.Value = string.Empty;
Bitmap bitmap;
if (url.IsBase64StringValid())
{
bitmap = url.Base64StringToBitmap();
}
else
{
bitmap = Extension.GetBitmapByUrl(url);
}
if (bitmap == null)
{
cell.Value = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Alt;
}
else
{
ExcelPicture pic = CurrentExcelWorksheet.Drawings.AddPicture(Guid.NewGuid().ToString(), bitmap);
AddImage((rowIndex + (ExcelExporterSettings.HeaderRowIndex > 1 ? ExcelExporterSettings.HeaderRowIndex : 0)),
colIndex - ignoreCount, pic, ExporterHeaderList[colIndex].ExportImageFieldAttribute.YOffset, ExporterHeaderList[colIndex].ExportImageFieldAttribute.XOffset);
CurrentExcelWorksheet.Row(rowIndex + 1).Height = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height;
pic.SetSize(ExporterHeaderList[colIndex].ExportImageFieldAttribute.Width * 7, ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height);
}
}
catch (Exception)
{
cell.Value = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Alt;
}Use the Bitmap object in the ExcelPicture object. For the online picture source, we will read and store it in the Bitmap, but we find that the object has not been released, so a large number of bitmaps have not been released. Let's deal with it through using.
using (ExcelPicture pic = CurrentExcelWorksheet.Drawings.AddPicture(Guid.NewGuid().ToString(), bitmap))
{
AddImage((rowIndex + (ExcelExporterSettings.HeaderRowIndex > 1 ? ExcelExporterSettings.HeaderRowIndex : 0)),
colIndex - ignoreCount, pic, ExporterHeaderList[colIndex].ExportImageFieldAttribute.YOffset, ExporterHeaderList[colIndex].ExportImageFieldAttribute.XOffset);
CurrentExcelWorksheet.Row(rowIndex + 1).Height = ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height;
pic.SetSize(ExporterHeaderList[colIndex].ExportImageFieldAttribute.Width * 7, ExporterHeaderList[colIndex].ExportImageFieldAttribute.Height);
}
A new object with a finalizer must be added to the finalization queue. This behavior is also called "registering for finalization". Of course, I also suggest you choose to use the SOSEX extension. It provides similar content to finalization, which seems more intuitive, as shown below.
-
Download address: http://www.stevestechspot.com/default.aspx
:000> .load D:\sosex_64\sosex.dll
This dump has no SOSEX heap index.
The heap index makes searching for references and roots much faster.
To create a heap index, run !bhi
0:000> !finq -stat
Generation 0:
Count Total Size Type
---------------------------------------------------------
54 2160 System.Drawing.Bitmap
54 objects, 2,160 bytes
Generation 1:
Count Total Size Type
---------------------------------------------------------
1 184 Microsoft.AspNetCore.Certificates.Generation.CertificateManager+CertificateManagerEventSource
1 336 Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelEventSource
4 1536 System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs
1 32 Internal.Cryptography.Pal.Native.SafeCertStoreHandle
1 280 System.Net.Sockets.SocketsTelemetry
1 192 System.Threading.Tasks.TplEventSource
1 40 Internal.Cryptography.Pal.Native.SafeCertContextHandle
2 64 Internal.Win32.SafeHandles.SafeRegistryHandle
2 64 Internal.Cryptography.Pal.Native.SafeChainEngineHandle
1 32 Internal.Cryptography.Pal.Native.SafeLocalAllocHandle
1 80 System.Runtime.Loader.DefaultAssemblyLoadContext
1 24 System.WeakReference`1[[System.Runtime.Loader.AssemblyLoadContext, System.Private.CoreLib]]
1 24 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerOptions, Microsoft.AspNetCore.Server.Kestrel.Core]]
2 704 Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketReceiver
2 48 System.WeakReference`1[[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.KestrelConnection, Microsoft.AspNetCore.Server.Kestrel.Core]]
1 184 Microsoft.IO.RecyclableMemoryStreamManager+Events
6 960 Microsoft.IO.RecyclableMemoryStream
2 48 System.WeakReference`1[[System.Text.RegularExpressions.RegexReplacement, System.Text.RegularExpressions]]
1 64 CellStore`1[[OfficeOpenXml.ExcelCoreValue, Magicodes.IE.EPPlus]]
1 64 CellStore`1[[System.Object, System.Private.CoreLib]]
1 64 FlagCellStore
2 128 CellStore`1[[System.Int32, System.Private.CoreLib]]
1 64 CellStore`1[[System.Uri, System.Private.Uri]]
2 96 ColumnIndex
2 144 System.Reflection.Emit.DynamicResolver
1 24 System.WeakReference
2 96 PageIndex
302 12080 System.Drawing.Bitmap
1 184 System.Net.NetEventSource
1 40 Interop+WinHttp+SafeWinHttpHandle
8 256 System.Threading.TimerQueue+AppDomainTimerSafeHandle
1 296 System.Net.Http.HttpTelemetry
1 224 System.Net.NameResolutionTelemetry
1 184 System.Net.NetEventSource
1 184 System.Net.NetEventSource
1 360 System.Net.Security.NetSecurityTelemetry
1 24 System.Net.Security.SafeCredentialReference
1 184 System.Collections.Concurrent.CDSCollectionETWBCLProvider
1 48 System.Net.Security.SafeFreeCredential_SECURITY
1 32 Microsoft.Win32.SafeHandles.SafeBCryptAlgorithmHandle
499 objects, 30,736 bytes
Generation 2:
0 objects, 0 bytes
TOTAL: 553 objects, 32,896 bytesMaybe everyone will have a question like me at the beginning. I saw your picture It's not that big, and it doesn't show size in Windbg. First, let's look at the quality of this picture. The pixel of the picture is 2560x1440 and the bit depth is 24. At present, we know this information. Let's calculate the size of the uncompressed picture.
2560x1440x24/8
For a picture of about 10M, the number of known pictures x10M=3G. In fact, for this problem, it does not belong to memory leakage.
summary
This article mainly introduces how to use Windbg to analyze the memory problems in the application process, explore from managed heap to unmanaged heap and memory allocation, and finally confirm the memory problems according to the memory performance and theory. Of course, for memory analysis, we don't have to love a tool. Of course, we can do it together with PerfView, which may have a better effect.
Microsoft MVP
Microsoft's most valuable expert is a global award awarded by Microsoft to third-party technology professionals. Over the past 28 years, technology community leaders around the world have won this award for sharing expertise and experience in their online and offline technology communities.
MVP is a strictly selected expert team. They represent the most skilled and intelligent people. They are experts who are enthusiastic and helpful to the community. MVP is committed to helping others through lectures, forum Q & A, creating websites, writing blogs, sharing videos, open source projects, organizing meetings, and helping users in the Microsoft technology community use Microsoft technology to the greatest extent.
For more details, please visit the official website:
https://mvp.microsoft.com/zh-cn