-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Make HttpClientFactory implement IDisposable to clean up resources #115826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: CarnaViire <[email protected]>
Co-authored-by: CarnaViire <[email protected]>
Tagging subscribers to this area: @dotnet/ncl |
src/libraries/Microsoft.Extensions.Http/src/DefaultHttpClientFactory.cs
Outdated
Show resolved
Hide resolved
…actory.cs Co-authored-by: CarnaViire <[email protected]>
src/libraries/Microsoft.Extensions.Http/src/DefaultHttpClientFactory.cs
Outdated
Show resolved
Hide resolved
src/libraries/Microsoft.Extensions.Http/src/DefaultHttpClientFactory.cs
Outdated
Show resolved
Hide resolved
src/libraries/Microsoft.Extensions.Http/src/DefaultHttpClientFactory.cs
Outdated
Show resolved
Hide resolved
src/libraries/Microsoft.Extensions.Http/src/DefaultHttpClientFactory.cs
Outdated
Show resolved
Hide resolved
src/libraries/Microsoft.Extensions.Http/src/DefaultHttpClientFactory.cs
Outdated
Show resolved
Hide resolved
...osoft.Extensions.Http/tests/Microsoft.Extensions.Http.Tests/HttpClientFactoryDisposeTests.cs
Outdated
Show resolved
Hide resolved
...osoft.Extensions.Http/tests/Microsoft.Extensions.Http.Tests/HttpClientFactoryDisposeTests.cs
Outdated
Show resolved
Hide resolved
...osoft.Extensions.Http/tests/Microsoft.Extensions.Http.Tests/HttpClientFactoryDisposeTests.cs
Outdated
Show resolved
Hide resolved
...osoft.Extensions.Http/tests/Microsoft.Extensions.Http.Tests/HttpClientFactoryDisposeTests.cs
Outdated
Show resolved
Hide resolved
...osoft.Extensions.Http/tests/Microsoft.Extensions.Http.Tests/HttpClientFactoryDisposeTests.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: CarnaViire <[email protected]>
src/libraries/Microsoft.Extensions.Http/src/ActiveHandlerTrackingEntry.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: CarnaViire <[email protected]>
src/libraries/Microsoft.Extensions.Http/src/ActiveHandlerTrackingEntry.cs
Outdated
Show resolved
Hide resolved
...osoft.Extensions.Http/tests/Microsoft.Extensions.Http.Tests/HttpClientFactoryDisposeTests.cs
Show resolved
Hide resolved
…expired handler testing Co-authored-by: CarnaViire <[email protected]>
@copilot can you pls fix the races mentioned below:
|
Co-authored-by: CarnaViire <[email protected]>
... Fixed the race conditions in commit 62c0678. The changes address both identified issues:
The implementation now ensures proper coordination between disposal and all timer callbacks, preventing the scenarios where handlers could be leaked or timers recreated after disposal. |
// If factory is disposed, don't process expiry | ||
if (_disposed) | ||
{ | ||
return; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at ActiveHandlerTrackingEntry.Timer_Tick()
-- the callback (= this method) is called only if it won the race with ActiveHandlerTrackingEntry.Dispose()
. That means, if we are already within this method:
- we've acquired ownership of the object
ActiveHandlerTrackingEntry.Dispose()
is now a no-op
Thus, even if factory's Dispose()
has already started (= _disposed is true), it is unfortunately in situation where it won't be able to do anything about this entry. We are the ones responsible.
The more I look at it, the more I think it is somehow more complicated than it should be
if we could guarantee that Dispose is only ever called when the entry is not within the collection anymore, we could even just null out the handler reference (via compare exchange obviously), so technically it shouldn't be touched by the CreateClient anymore? But what if it got retrieved there just before it was removed and about to be disposed?
thinking more, while it was guaranteed previously that the handler will remain alive while it's still being used - it is now broken, but only at the moment of factory disposal. and factory's (and transitively active entry's) dispose is only ever called during the full container dispose. meaning, this is a time when it becomes erroneous to use anything created by this container anymore. so this race - as any other errors during - is smth that can be expected. but I think just in case we should still play nice and throw ODE in the property getter instead of leaving it for a subsequent NRE.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you know what? I definitely was overthinking this.
let's remove IDisposable interface from both active and expired entries, it just makes things more complicated.
then, from active entry's perspective, it immutable with exception of the timer. from expired entry perspective, it's completely immutable.
the most straightforward approach to Factory's dispose would be:
- mark factory as disposed. after that, no public methods should be able to reach the active handler collection. meaning, no new timers should be started after it.
- we brush the active handlers so all the timers are stopped, AND all the conversions are completed => everything is in the expired queue now. not the best thing perf-wise, but it's full container disposal, so we might want to favor simplicity more here.
- add parameter "force" to the expired cleanup. checks are only done with force=false (ordinary cleanup), for force=true (dispose) everything is cleaned by kipping the check.
from thread-safety perspective:
- external approach stopped -> nothing new can be added to active entries
- all remaining active entries are converted -> nothing new can be added to expired entries
- all expired handlers are disposed
all these steps will be sequential.
if (!_disposed) | ||
{ | ||
var expired = new ExpiredHandlerTrackingEntry(active); | ||
_expiredHandlers.Enqueue(expired); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here as well. what's more, if the _disposed
got changed after the if
check above but before _expiredHandlers.Enqueue
, we can end up in the situation where Dispose
believes it has cleared the queue, but this callback will add it just after.
one of the possible solutions would be to null out the _expiredHandlers reference completely on dispose, same as with active entry
} | ||
finally | ||
{ | ||
if (!_livenessTracker.IsAlive) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is wrong. this either should be already checked by the caller, or the caller wants everything to be disposed regardless
disposables.Add(entry); | ||
|
||
// Force dispose the scope even if the handler is still alive | ||
if (entry.Scope != null) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
when we fix entry's Dispose to remove the redundant CanDispose check, it should be enough to just dispose the entry here.
Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it. |
Issue Description
When creating and disposing service providers that use HttpClientFactory, there's a memory leak because timers aren't properly cleaned up. This particularly affects scenarios where service providers are frequently created and disposed, such as in test environments.
The problem occurs because:
Changes Made
DefaultHttpClientFactory
implementIDisposable
interfaceStopTimer()
method toActiveHandlerTrackingEntry
to stop and dispose timersDispose()
method inDefaultHttpClientFactory
that:Benefits
Fixes #113494.
Warning
Firewall rules blocked me from connecting to one or more addresses
I tried to connect to the following addresses, but was blocked by firewall rules:
pkgs.dev.azure.com
/home/REDACTED/work/runtime/runtime/.dotnet/dotnet msbuild /m /nologo /clp:Summary /v:minimal /nr:true /warnaserror /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=false /home/REDACTED/work/runtime/runtime/artifacts/toolset/restore.proj /t:__WriteToolsetLocation /clp:ErrorsOnly;NoSummary /p:__ToolsetLocationOutputFile=/home/REDACTED/work/runtime/runtime/artifacts/toolset/10.0.0-beta.25260.104.txt
(dns block)./.dotnet/dotnet build src/libraries/Microsoft.Extensions.Http/src/Microsoft.Extensions.Http.csproj
(dns block)If you need me to access, download, or install something from one of these locations, you can either:
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.