Virtually all medium to large applications uses some Dependency Injection technology. I assume my readers know about it; I won’t repeat the basics here.
The full code for the article is on GitHub.
An application or service can easily have several thousand components configured in a Dependency Injection container. Each component is configured as one of the three main types of lifetimes: SingleInstance, InstancePerLifetimeScope, and InstancePerDependency. These are names from Autofac. (Read this primer if you are not familiar with Autofac.) Others might call them Singleton, PerRequest, and Transient. We can deduce from their names that SingleInstance has the most extended lifetime, generally equal to the containing process lifetime. InstancePerLifetimeScope is usually tied to a service request, so such an object lives some seconds typically. InstancePerDependencyis similar to calling the constructor on a class; a new instance will be created on each resolve request.
Web applications and services are multi-threaded. A SingleInstance component should be thread-safe because, by definition, all requests will be dispatched to a single instance. Hopefully, the developer who configured a SingleInstance lifetime for a component is aware of this fact. If (s)he does not, then we lose at step 0.
A SingleInstance component can depend on other SingleInstance components. Both are (assumed to be) thread-safe, no problem at all.
InstancePerLifetimeScope and InstancePerDependency objects are generally not thread-safe. For example, they call some data access layer code, which is not thread-safe. (I already wrote an article about the problems when a non-thread-safe code is called from multiple threads.)
Now comes the problem. What happens if a SingleInstance component depends on an InstancePerDependency or InstancePerDependency component? The called component will experience a multi-threaded workload and probably crash badly. Essentially, the shorter lifetime component becomes a SingleInstance component! This configuration is called Captive Dependency.
“Captive Dependency is a dependency with an incorrectly configured lifetime. A service should never depend on a service with a shorter lifetime than its own.”
Captive Dependency is a very severe and hard-to-diagnose problem. Most Dependency Injection containers do not detect this kind of misconfiguration.
I experienced this problem during a consultancy job. To demonstrate the issue, I’ve created a demo project to serve as an educational source.
The scenario consists of an image loader service. It can load images stored in an SQL Server database. The database access is based on EF Core. The component which encapsulates the data access details (LINQ, potential conversations, etc.) is named ImageRepository.
Here is the code of the ImageRepository:
internal sealed class ApplicationImageRepository : IApplicationImageRepository
{
private readonly IImageDbContext _dbContext;
public ApplicationImageRepository(IImageDbContext dbContext)
{
_dbContext = dbContext;
Console.WriteLine($"{GetType()} instance #{GetHashCode()} created from thread: {Thread.CurrentThread.ManagedThreadId}");
}
public byte[] GetImage(ImageType imageType)
{
Console.WriteLine($"{GetType()}.{nameof(GetImage)} #{GetHashCode()}({imageType}) call start from thread: {Thread.CurrentThread.ManagedThreadId}");
ApplicationImage? image = _dbContext.ApplicationImages.SingleOrDefault(i => i.ImageType == imageType);
if (image == null)
{
throw new InvalidOperationException($"Missing image for {imageType}");
}
Console.WriteLine($"{GetType()}.{nameof(GetImage)} #{GetHashCode()}({imageType}) call end from thread: {Thread.CurrentThread.ManagedThreadId}");
return image.ImageContent;
}
}
This is a straightforward code. It gets EF Core DbContext as a dependency. The Console.WriteLine lines track the timing of the calls from the threads. You won’t include such traces in the production code.
ImageRepository uses EF DbContext, so it is not thread-safe. Hence, we register it in Autofac as InstancePerLifetimeScope because we know it will be accessed from a short-lived child lifetime scope. It could have been InstancePerDependency too.
Because these images rarely change, we want to cache them in memory to conserve database resources. So, we want to create a cache layer that will use this repository to load the requested image and store it in memory for later access. Because this component should keep the images for the application’s lifetime, it probably should be configured as SingleInstance, so it will logically belong to the root scope. (Technically, you could create a long-life child scope for this purpose.)
Here is the cache layer implementation:
public sealed class ApplicationImageCacheBogus1 : IApplicationImageCacheBogus1
{
private readonly Func<IApplicationImageRepository> _applicationImageRepository;
private readonly ConcurrentDictionary<ImageType, byte[]> _applicationImages = new();
public ApplicationImageCacheBogus1(Func<IApplicationImageRepository> applicationImageRepository)
{
_applicationImageRepository = applicationImageRepository ?? throw new ArgumentNullException(nameof(applicationImageRepository));
Console.WriteLine($"{GetType()} instance #{GetHashCode()} created from thread: {Thread.CurrentThread.ManagedThreadId}");
}
public byte[] GetApplicationImage(ImageType imageType)
{
Console.WriteLine($"{GetType()}.{nameof(GetApplicationImage)} #{GetHashCode()}({imageType}) " +
$"call from thread: {Thread.CurrentThread.ManagedThreadId}");
return _applicationImages.GetOrAdd(imageType, it => _applicationImageRepository().GetImage(it));
}
}
The code attempts to be thread-safe, but it does not, despite it using ConcurrentDictionary and using it correctly. I’ve seen solutions that check the existence of an item by calling Contains and if it is missing, then calls TryAdd to add the item to the collection. And they used ConcurrentDictionary, so they thought the code is thread-safe. This would be a lame error. I won’t publish such obviously wrong code.
What is the problem with this implementation? First, it contains a captive dependency. The caller, ApplicationImageCacheBogus1, is registered as SingleInstance; it is a singleton object. It has reference to an IApplicationImageRepository implementation that is registered as InstancePerLifetimeScope. This is a captive dependency. You might think that wrapping the dependency injected into the constructor in Func solve this problem, but it does not.
I have to stress the last sentence. Even if a component consumes a dependency via the delegate factory (Func), the lifetime of the dependency will be longer if the caller has a longer lifetime. So, repeated calls to the delegate factory will return the same single instance of the repository. This is a very important statement; it should sink into your mind!
So, many threads start to attack ApplicationImageCacheBogus1.GetApplicationImage(). ConcurrentDictionary.GetOrAdd will call back the factory delegate from all threads simultaneously! This might not be intuitive. ConcurrentDictionary makes the factories compete, but only 1, the fastest one, will be stored and returned from GetOrAdd. It means our poor IApplicationImageRepository sole instance returned from _applicationImageRepository() delegate factory will experience calls from many threads. And it will die badly.
Let’s test it! Here is the initialization code of the test class:
private IContainer _container = null!;
[TestInitialize]
public void Init()
{
var imageDbContext = new ImageDbContext();
imageDbContext.Database.EnsureCreated();
SeedData.Initialize(imageDbContext);
ContainerBuilder builder = new ContainerBuilder();
builder.RegisterModule<SampleDataAccessModule>();
_container = builder.Build();
}
We create a DbContext instance. Then call EnsureCreated() to create the sample database if needed. SeedData.Initialize will insert 3 sample pictures into the test table.
Then we create an Autofac ContainerBuilder, which calls back SampleDataAccessModule for the registrations I’ll show you in a moment. Then it builds the root container.
SampleDataAccessModule contains the registrations of the types used in this sample:
public class SampleDataAccessModule: Module
{
protected override void Load(ContainerBuilder builder)
{
builder.RegisterType<ApplicationImageCacheBogus1>().As<IApplicationImageCacheBogus1>().SingleInstance();
builder.RegisterType<ImageDbContext>().As<IImageDbContext>().InstancePerLifetimeScope();
builder.RegisterType<ApplicationImageRepository>().As<IApplicationImageRepository>().InstancePerLifetimeScope();
}
}
You see ApplicationImageCacheBogus1 is SingleInstance, ImageDbContext, and ApplicationImageRepository are InstancePerLifetimeScope.
Here is the actual test:
[TestMethod]
public void ApplicationImageCacheBogus1_GetApplicationImage_WhenCalledFromManyThread_ShouldNotThrow()
{
//Arrange
Action a = () => Parallel.For(0, 100, i =>
{
using var scope = _container.BeginLifetimeScope();
//Act
scope.Resolve<IApplicationImageCacheBogus1>().GetApplicationImage((ImageType)(i % 3));
});
//Assert
a.Should().NotThrow();
}
Parallel.For spawns many threads (from ThreadPool). In each thread, we create a new child scope and resolve the cache from the child scope. This simulates what a service DI integration does. Then, we call GetApplicationImage() with all three image types it supports. Running the test, we see the failure:
EF Core became angry. And it was correct. The log reveals the race situation nicely:
ApplicationImageCacheBogus1 instance #57506335 created from thread: 20
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 5
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 20
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 21
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 23
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 18
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 8
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 24
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 14
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 12
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 22
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 16
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 15
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 7
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 13
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 19
ApplicationImageCacheBogus1.GetApplicationImage #57506335(HeaderImage) call from thread: 17
ApplicationImageRepository instance #12295835 created from thread: 20
ApplicationImageRepository.GetImage #12295835(HeaderImage) call start from thread: 14
ApplicationImageRepository.GetImage #12295835(HeaderImage) call start from thread: 19
ApplicationImageRepository.GetImage #12295835(HeaderImage) call start from thread: 20
ApplicationImageRepository.GetImage #12295835(HeaderImage) call start from thread: 22
In line 1, the single instance of the cache class is created. Then many threads call into the GetApplicationImage() method of this instance.
At line 18, ApplicationImageRepository is created by thread 20. Only 1 instance is created, despite it being configured as InstancePerLifetimeScope. It became a single instance component — a captured one. And from line 19, it is tortured by many threads.
I used (ImageType)(i % 3) in the test code with the intention of testing all 3 types simultaneously. What is strange, however, is that all calls requested HeaderImage only (lines 19-22) and did not ask for the 2 other types. Hm.
Parallel.For is not ideal for multi-threaded testing when the race condition happens at the first few requests and not later when the system warmed up. Parallel (more precisely, the ThreadPool) has a slow thread ramp-up because it wants to allocate threads conservatively. It slowly injects more and more threads trying to saturate all CPU cores. This means that too few threads will call the code at the first moments of the test, so the race condition is harder to provoke.
So, I created a more aggressive version of the test that tried to start the threads as fast as possible. Here is it:
[TestMethod]
public async Task ApplicationImageCacheBogus1_GetApplicationImage_WhenCalledFromManyThreadMoreAggressively_ShouldNotThrow()
{
//Arrange
List<Task> tasks = Enumerable.Range(0, 100).Select(i => new Task(() =>
{
using var scope = _container.BeginLifetimeScope();
scope.Resolve<IApplicationImageCacheBogus1>().GetApplicationImage((ImageType)(i % 3));
})).ToList();
//Act
foreach (var task in tasks)
{
task.Start();
}
Func<Task> action = async () => await Task.WhenAll(tasks);
//Assert
await action.Should().NotThrowAsync();
}
You can observe that I did not use Task.Run to create and start the tasks rather I create new Task instances without starting them. Then I started them in the second loop. In this way, the degree of parallelism is more at the start when the critical congestion happens so that this test is more likely provoke the bogus behavior. Let’s see how the log changes:
ApplicationImageCacheBogus1 instance #50883745 created from thread: 17
ApplicationImageCacheBogus1.GetApplicationImage #50883745(FooterImage) call from thread: 14
ApplicationImageCacheBogus1.GetApplicationImage #50883745(FooterImage) call from thread: 17
ApplicationImageCacheBogus1.GetApplicationImage #50883745(BackgroundImage) call from thread: 21
ApplicationImageCacheBogus1.GetApplicationImage #50883745(FooterImage) call from thread: 20
ApplicationImageCacheBogus1.GetApplicationImage #50883745(HeaderImage) call from thread: 16
ApplicationImageCacheBogus1.GetApplicationImage #50883745(BackgroundImage) call from thread: 15
ApplicationImageCacheBogus1.GetApplicationImage #50883745(HeaderImage) call from thread: 13
ApplicationImageCacheBogus1.GetApplicationImage #50883745(HeaderImage) call from thread: 25
ApplicationImageCacheBogus1.GetApplicationImage #50883745(BackgroundImage) call from thread: 24
ApplicationImageCacheBogus1.GetApplicationImage #50883745(HeaderImage) call from thread: 19
ApplicationImageCacheBogus1.GetApplicationImage #50883745(HeaderImage) call from thread: 12
ApplicationImageCacheBogus1.GetApplicationImage #50883745(FooterImage) call from thread: 23
ApplicationImageCacheBogus1.GetApplicationImage #50883745(HeaderImage) call from thread: 22
ApplicationImageCacheBogus1.GetApplicationImage #50883745(BackgroundImage) call from thread: 18
ApplicationImageCacheBogus1.GetApplicationImage #50883745(FooterImage) call from thread: 8
ApplicationImageCacheBogus1.GetApplicationImage #50883745(BackgroundImage) call from thread: 5
SampleDataAccess.Repository.ApplicationImageRepository instance #38583594 created from thread: 13
ApplicationImageRepository.GetImage #38583594(FooterImage) call start from thread: 14
ApplicationImageRepository.GetImage #38583594(HeaderImage) call start from thread: 13
ApplicationImageRepository.GetImage #38583594(HeaderImage) call start from thread: 16
ApplicationImageRepository.GetImage #38583594(BackgroundImage) call start from thread: 24
ApplicationImageRepository.GetImage #38583594(FooterImage) call start from thread: 8
Now we see all kinds of image-type requests inside the poor, captured repository.
Ok, we proved by the test that this code is clearly bogus; the singleton cache component captured the repository that did not want to be a singleton, but it became so; and thus, the threads crushed it.
How can we solve this problem? If you google about ConcurrentDictionary.GetOrAdd racing behavior, you might see solutions to the issue at the mighty StackOverflow site.
They say combining ConcurrentDictionary and Lazy can circumvent the concurrent calls to the factory delegate of GetOrAdd. Here is the second attempt to repair the cached image service based on StackOverflow suggestion:
public sealed class ApplicationImageCacheBogus2 : IApplicationImageCacheBogus2
{
private readonly Func<IApplicationImageRepository> _applicationImageRepository;
private readonly ConcurrentDictionary<ImageType, Lazy<byte[]>> _applicationImages = new();
public ApplicationImageCacheBogus2(Func<IApplicationImageRepository> applicationImageRepository)
{
_applicationImageRepository = applicationImageRepository ?? throw new ArgumentNullException(nameof(applicationImageRepository));
Console.WriteLine($"{GetType()} instance #{GetHashCode()} created from thread: {Thread.CurrentThread.ManagedThreadId}");
}
public byte[] GetApplicationImage(ImageType imageType)
{
Console.WriteLine($"{GetType()}.{nameof(GetApplicationImage)} #{GetHashCode()}({imageType}) " +
$"call from thread: {Thread.CurrentThread.ManagedThreadId}");
return _applicationImages.GetOrAdd(imageType, it => new Lazy<byte[]>(() =>
_applicationImageRepository().GetImage(it)))
.Value;
}
}
The hack is clever. ConcurrentDictionary will create many instances of the Lazy class for each thread, but only 1 will be visible outside GetOrAdd. And that 1 gets the opportunity to call the factory delegate when we call the Value property, create the repository, and load the image. But this might be just some fancy bullshit explanation. We have to test the proposed solution. The test is almost the same as the previous one, but at this time, we get an instance from the IApplicationImageCacheBogus2 interface implementation:
[TestMethod]
public async Task ApplicationImageCacheBogus2_GetApplicationImage_WhenCalledFromManyThread_ShouldNotThrow()
{
//Arrange
List<Task> tasks = Enumerable.Range(0, 100).Select(i => new Task(() =>
{
using var scope = _container.BeginLifetimeScope();
scope.Resolve<IApplicationImageCacheBogus2>().GetApplicationImage((ImageType)(i % 3));
})).ToList();
//Act
foreach (var task in tasks)
{
task.Start();
}
Func<Task> action = async () => await Task.WhenAll(tasks);
//Assert
await action.Should().NotThrowAsync();
}
Running the test reveals that our reasoning was flawed. It crashes precisely like the first one. Here is the log:
ApplicationImageCacheBogus2 instance #31423778 created from thread: 15
ApplicationImageCacheBogus2.GetApplicationImage #31423778(FooterImage) call from thread: 20
ApplicationImageCacheBogus2.GetApplicationImage #31423778(FooterImage) call from thread: 23
ApplicationImageCacheBogus2.GetApplicationImage #31423778(BackgroundImage) call from thread: 18
ApplicationImageCacheBogus2.GetApplicationImage #31423778(HeaderImage) call from thread: 13
ApplicationImageCacheBogus2.GetApplicationImage #31423778(HeaderImage) call from thread: 7
ApplicationImageCacheBogus2.GetApplicationImage #31423778(BackgroundImage) call from thread: 15
ApplicationImageCacheBogus2.GetApplicationImage #31423778(FooterImage) call from thread: 17
ApplicationImageCacheBogus2.GetApplicationImage #31423778(BackgroundImage) call from thread: 8
ApplicationImageCacheBogus2.GetApplicationImage #31423778(FooterImage) call from thread: 12
ApplicationImageCacheBogus2.GetApplicationImage #31423778(HeaderImage) call from thread: 22
ApplicationImageCacheBogus2.GetApplicationImage #31423778(HeaderImage) call from thread: 25
ApplicationImageCacheBogus2.GetApplicationImage #31423778(HeaderImage) call from thread: 16
ApplicationImageCacheBogus2.GetApplicationImage #31423778(FooterImage) call from thread: 14
ApplicationImageCacheBogus2.GetApplicationImage #31423778(BackgroundImage) call from thread: 21
ApplicationImageCacheBogus2.GetApplicationImage #31423778(BackgroundImage) call from thread: 24
ApplicationImageCacheBogus2.GetApplicationImage #31423778(HeaderImage) call from thread: 19
ApplicationImageRepository instance #63449985 created from thread: 23
ApplicationImageRepository.GetImage #63449985(FooterImage) call start from thread: 23
ApplicationImageRepository.GetImage #63449985(BackgroundImage) call start from thread: 15
ApplicationImageRepository.GetImage #63449985(HeaderImage) call start from thread: 22
Why did the clever Lazy trick not help? Think about it!
ConcurrentDictionary will create many Lazy instances for each 3 image type. For each type, 1 image will win and return from GetOrAdd. Then the 3 threads happily call into the sole repository (63449985) simultaneously! 3 threads access the repo, and not many, but this is more than 1 yet.
The fundamental problem is that we have 1 captured repository that experienced a multi-threaded load again.
To make sure you understand the issue, I repeat the test, but I ask for just 1 image type only:
List<Task> tasks = Enumerable.Range(0, 100).Select(i => new Task(() =>
{
using var scope = _container.BeginLifetimeScope();
scope.Resolve<IApplicationImageCacheBogus2>().GetApplicationImage(ImageType.BackgroundImage);
})).ToList();
I don’t use the modulo (%) trick here; I am asking for ImageType.BackgroundImage only. And the test at this time succeeds:
You see, Lazy ensured that only thread 14 had the chance to create the repo and get the image from the database. The remaining threads waited till thread 14, loading the image. Then the remainder threads got the images from ConcurrentDictionary, and they did not touch the repo; hence the code did not throw. It’s interesting.
It is essential to realize that this simplified test gave us false positive results. We thought that the implementation was thread-safe when in reality, it does not! Proper and thoughtful testing is crucial, especially for multi-threaded code.
Ok, we hacked enough that now it’s time to use our brains and give a proper solution to this seemingly simple problem.
First, the main issue here is that the cache has to be a single instance and therefore multi-threaded, but the repo is not multi-threaded, so it cannot be a single instance, but the cache captures it, and the repo becomes a single instance too.
But the cache must call the repo to access the data! It looks like there are self-contradicting requirements here. When there seems to be no solution for a problem in object-oriented design, it is helpful to think about the SOLID principles. Think about S, Single Responsibility.
What are the responsibilities of the cache class?
- Knowing which component retrieves the data from the database
- Storing the data for later access
- Giving an interface for the callers to access the images
If there is more than one responsibility of a module (method, class, anything), then it might be advantageous to break it apart into more modules. In this problem, the data store is the one that needs the singleton lifetime; the others do not. So, we might separate the storage to a new class which will be a single instance, but the cache itself, which fulfills items 1 and 3, remains an instance per lifetime scope. This way, the repo won’t be captured, so we save the world.
Here is the refactored code:
public sealed class ApplicationImageCache : IApplicationImageCache
{
private readonly Func<IApplicationImageRepository> _applicationImageRepository;
private readonly Func<IApplicationImageCacheStorage> _storage;
public ApplicationImageCache(Func<IApplicationImageRepository> applicationImageRepository, Func<IApplicationImageCacheStorage> storage)
{
_applicationImageRepository = applicationImageRepository ?? throw new ArgumentNullException(nameof(applicationImageRepository));
_storage = storage ?? throw new ArgumentNullException(nameof(storage));
Console.WriteLine($"{GetType()} instance #{GetHashCode()} created");
}
public byte[] GetApplicationImage(ImageType imageType)
{
Console.WriteLine($"{GetType()}.{nameof(GetApplicationImage)} #{GetHashCode()}({imageType}) " +
$"call from thread: {Thread.CurrentThread.ManagedThreadId}");
return _storage().GetApplicationImage(imageType, kt => _applicationImageRepository().GetImage(kt));
}
}
ApplicationImageCache has 2 dependencies. The repository we already knew and a new component, IApplicationImageCacheStorage, store the images in memory.
This is the new component called ApplicationImageCacheStorage:
internal sealed class ApplicationImageCacheStorage : IApplicationImageCacheStorage
{
private readonly ConcurrentDictionary<ImageType, byte[]> _applicationImages = new();
public ApplicationImageCacheStorage()
{
Console.WriteLine($"{GetType()} instance #{GetHashCode()} created from thread: {Thread.CurrentThread.ManagedThreadId}");
}
public byte[] GetApplicationImage(ImageType imageType, Func<ImageType, byte[]> loaderFunc)
{
if (loaderFunc == null) throw new ArgumentNullException(nameof(loaderFunc));
Console.WriteLine($"{GetType()}.{nameof(GetApplicationImage)} #{GetHashCode()}({imageType}) " +
$"call from thread: {Thread.CurrentThread.ManagedThreadId}");
if (loaderFunc == null) throw new ArgumentNullException(nameof(loaderFunc));
return _applicationImages.GetOrAdd(imageType, loaderFunc);
}
}
The interaction between the three components is a bit more complex now. Remember, ApplicationImageCacheStorage is a single instance. However, ApplicationImageCache and ApplicationImageRepository are InstancePerDependency.
When many threads ask Autofac to resolve an instance of ApplicationImageCache, each thread gets a separate instance. ApplicationImageCache.GetApplicationImage() calls into ApplicationImageCacheStorage.GetApplicationImage(). Because ApplicationImageCacheStorage is a singleton, all threads arrive at the same instance. ConcurrentDictionary will call the factories using many threads. However, the callback Func will dispatch them to separate instances of repositories! This is the key point. Why? Because the delegate will call back to ApplicationImageCache, and ApplicationImageCache has the ApplicationImageRepository dependency. Both ApplicationImageCache and ApplicationImageRepository are registered as InstancePerDependency, so the injected ApplicationImageRepository remains InstancePerDependency, so each call to the Func will create a new instance. No capturing at all.
All repository instances can work parallel; each instance handles 1 thread only. So, ConcurrentDictionary initiates many database access via the repo instances, and only 1 will win for each image type. The work of the other repositories will be dropped.
The log demonstrates this nicely:
ApplicationImageCache instance #21647132 created from thread: 17
ApplicationImageCache instance #32901400 created from thread: 5
ApplicationImageCache instance #42931033 created from thread: 15
ApplicationImageCache instance #6451435 created from thread: 18
ApplicationImageCache instance #34717384 created from thread: 19
ApplicationImageCache instance #2256335 created from thread: 25
ApplicationImageCache instance #35743531 created from thread: 23
ApplicationImageCache instance #49538252 created from thread: 16
ApplicationImageCache instance #22460983 created from thread: 20
ApplicationImageCache instance #25960677 created from thread: 22
ApplicationImageCache instance #58366981 created from thread: 13
ApplicationImageCache instance #24388906 created from thread: 21
ApplicationImageCache instance #20974680 created from thread: 12
ApplicationImageCache instance #11865849 created from thread: 24
ApplicationImageCache instance #56140151 created from thread: 14
ApplicationImageCache instance #16754362 created from thread: 7
ApplicationImageCache.GetApplicationImage #11865849(BackgroundImage) call from thread: 24
ApplicationImageCache.GetApplicationImage #2256335(HeaderImage) call from thread: 25
ApplicationImageCache.GetApplicationImage #20974680(BackgroundImage) call from thread: 12
ApplicationImageCache.GetApplicationImage #58366981(HeaderImage) call from thread: 13
ApplicationImageCache.GetApplicationImage #42931033(BackgroundImage) call from thread: 15
ApplicationImageCache.GetApplicationImage #49538252(HeaderImage) call from thread: 16
ApplicationImageCache.GetApplicationImage #32901400(HeaderImage) call from thread: 5
ApplicationImageCache.GetApplicationImage #25960677(HeaderImage) call from thread: 22
ApplicationImageCache.GetApplicationImage #21647132(FooterImage) call from thread: 17
ApplicationImageCache.GetApplicationImage #35743531(FooterImage) call from thread: 23
ApplicationImageCache.GetApplicationImage #24388906(BackgroundImage) call from thread: 21
ApplicationImageCache.GetApplicationImage #6451435(BackgroundImage) call from thread: 18
ApplicationImageCache.GetApplicationImage #22460983(FooterImage) call from thread: 20
ApplicationImageCache.GetApplicationImage #34717384(HeaderImage) call from thread: 19
ApplicationImageCache.GetApplicationImage #16754362(FooterImage) call from thread: 7
ApplicationImageCache.GetApplicationImage #56140151(FooterImage) call from thread: 14
ApplicationImageCacheStorage instance #13273752 created from thread: 19
ApplicationImageCacheStorage.GetApplicationImage #13273752(HeaderImage) call from thread: 25
ApplicationImageCacheStorage.GetApplicationImage #13273752(FooterImage) call from thread: 14
ApplicationImageCacheStorage.GetApplicationImage #13273752(FooterImage) call from thread: 23
ApplicationImageCacheStorage.GetApplicationImage #13273752(HeaderImage) call from thread: 19
ApplicationImageCacheStorage.GetApplicationImage #13273752(HeaderImage) call from thread: 22
ApplicationImageCacheStorage.GetApplicationImage #13273752(BackgroundImage) call from thread: 24
ApplicationImageCacheStorage.GetApplicationImage #13273752(HeaderImage) call from thread: 16
ApplicationImageCacheStorage.GetApplicationImage #13273752(BackgroundImage) call from thread: 15
ApplicationImageCacheStorage.GetApplicationImage #13273752(BackgroundImage) call from thread: 18
ApplicationImageCacheStorage.GetApplicationImage #13273752(HeaderImage) call from thread: 5
ApplicationImageCacheStorage.GetApplicationImage #13273752(HeaderImage) call from thread: 13
ApplicationImageCacheStorage.GetApplicationImage #13273752(FooterImage) call from thread: 7
ApplicationImageCacheStorage.GetApplicationImage #13273752(FooterImage) call from thread: 17
ApplicationImageCacheStorage.GetApplicationImage #13273752(BackgroundImage) call from thread: 21
ApplicationImageCacheStorage.GetApplicationImage #13273752(FooterImage) call from thread: 20
ApplicationImageCacheStorage.GetApplicationImage #13273752(BackgroundImage) call from thread: 12
ApplicationImageRepository instance #64538993 created from thread: 17
ApplicationImageRepository instance #1432091 created from thread: 19
ApplicationImageRepository instance #46251828 created from thread: 24
ApplicationImageRepository instance #54718731 created from thread: 14
ApplicationImageRepository instance #58998806 created from thread: 7
ApplicationImageRepository instance #51781231 created from thread: 13
ApplicationImageRepository instance #45214701 created from thread: 21
ApplicationImageRepository instance #63403007 created from thread: 12
ApplicationImageRepository instance #55467396 created from thread: 15
ApplicationImageRepository instance #11373314 created from thread: 20
ApplicationImageRepository instance #28366852 created from thread: 23
ApplicationImageRepository instance #54752321 created from thread: 22
ApplicationImageRepository instance #28062782 created from thread: 5
ApplicationImageRepository instance #65615241 created from thread: 18
ApplicationImageRepository instance #31071611 created from thread: 16
ApplicationImageRepository instance #45814213 created from thread: 25
ApplicationImageRepository.GetImage #54718731(FooterImage) call start from thread: 14
ApplicationImageRepository.GetImage #64538993(FooterImage) call start from thread: 17
ApplicationImageRepository.GetImage #1432091(HeaderImage) call start from thread: 19
ApplicationImageRepository.GetImage #46251828(BackgroundImage) call start from thread: 24
ApplicationImageRepository.GetImage #58998806(FooterImage) call start from thread: 7
ApplicationImageRepository.GetImage #45214701(BackgroundImage) call start from thread: 21
ApplicationImageRepository.GetImage #51781231(HeaderImage) call start from thread: 13
ApplicationImageRepository.GetImage #63403007(BackgroundImage) call start from thread: 12
ApplicationImageRepository.GetImage #55467396(BackgroundImage) call start from thread: 15
ApplicationImageRepository.GetImage #28366852(FooterImage) call start from thread: 23
ApplicationImageRepository.GetImage #11373314(FooterImage) call start from thread: 20
ApplicationImageRepository.GetImage #28062782(HeaderImage) call start from thread: 5
ApplicationImageRepository.GetImage #54752321(HeaderImage) call start from thread: 22
ApplicationImageRepository.GetImage #65615241(BackgroundImage) call start from thread: 18
ApplicationImageRepository.GetImage #31071611(HeaderImage) call start from thread: 16
ApplicationImageRepository.GetImage #45814213(HeaderImage) call start from thread: 25
ApplicationImageRepository.GetImage #64538993(FooterImage) call end from thread: 17
ApplicationImageRepository.GetImage #28366852(FooterImage) call end from thread: 23
ApplicationImageRepository.GetImage #45814213(HeaderImage) call end from thread: 25
ApplicationImageRepository.GetImage #31071611(HeaderImage) call end from thread: 16
ApplicationImageRepository.GetImage #54752321(HeaderImage) call end from thread: 22
ApplicationImageRepository.GetImage #46251828(BackgroundImage) call end from thread: 24
ApplicationImageRepository.GetImage #45214701(BackgroundImage) call end from thread: 21
ApplicationImageRepository.GetImage #65615241(BackgroundImage) call end from thread: 18
ApplicationImageRepository.GetImage #54718731(FooterImage) call end from thread: 14
ApplicationImageRepository.GetImage #11373314(FooterImage) call end from thread: 20
ApplicationImageRepository.GetImage #1432091(HeaderImage) call end from thread: 19
ApplicationImageRepository.GetImage #51781231(HeaderImage) call end from thread: 13
ApplicationImageRepository.GetImage #55467396(BackgroundImage) call end from thread: 15
ApplicationImageRepository.GetImage #58998806(FooterImage) call end from thread: 7
ApplicationImageRepository.GetImage #63403007(BackgroundImage) call end from thread: 12
ApplicationImageRepository.GetImage #28062782(HeaderImage) call end from thread: 5
ApplicationImageCache instance #21822500 created from thread: 19
ApplicationImageCache instance #15580004 created from thread: 23
ApplicationImageCache instance #16866778 created from thread: 5
ApplicationImageCache instance #6987146 created from thread: 16
ApplicationImageCache instance #50644886 created from thread: 17
ApplicationImageCache instance #17785081 created from thread: 18
ApplicationImageCache instance #1177678 created from thread: 12
ApplicationImageCache instance #3543679 created from thread: 21
ApplicationImageCache instance #30187349 created from thread: 13
ApplicationImageCache instance #64340990 created from thread: 14
ApplicationImageCache.GetApplicationImage #21822500(FooterImage) call from thread: 19
ApplicationImageCache.GetApplicationImage #15580004(HeaderImage) call from thread: 23
ApplicationImageCache.GetApplicationImage #6987146(FooterImage) call from thread: 16
ApplicationImageCache.GetApplicationImage #1177678(BackgroundImage) call from thread: 12
ApplicationImageCache.GetApplicationImage #3543679(HeaderImage) call from thread: 21
ApplicationImageCache.GetApplicationImage #16866778(HeaderImage) call from thread: 5
ApplicationImageCache.GetApplicationImage #17785081(BackgroundImage) call from thread: 18
ApplicationImageCache.GetApplicationImage #30187349(FooterImage) call from thread: 13
ApplicationImageCache.GetApplicationImage #64340990(FooterImage) call from thread: 14
ApplicationImageCache.GetApplicationImage #50644886(BackgroundImage) call from thread: 17
From lines 1 to 32, many instances of ApplicationImageCache were created, and the threads are called into them. At line 33, the single instance of ApplicationImageCacheStorage is created. Then many threads entered GetApplicationImage() in this instance (lines 34-49). From line 50, many instances of the repository were created. From line 66, the threads start to call into the ApplicationImageRepository.GetImage() methods. Observe that the for each instance, the thread that created the instance is eventually called GetImage() on that exact same instance. For example, ApplicationImageRepository instance #64538993 is created from thread 17 in line 50. GetImage() on this instance is called in line 67 from the same thread.
So, many repository instances are created and called parallel, but this usage is ok. All but 3 results are dropped, and the fastest ones are kept.
108 is the first line showing an image served from the cache. The later ones are done from memory forever.
This solution does not contain captured dependency and is thread-safe. If the first threads arrive very aggressively, as in this contrived test, we initially waste the database resources. Realistically, like in a web service, the workload is not arriving so hard at startup, so this won’t be an issue.
However, we are very close to the ultimate solution, which combines everything we learned in this article. We can combine Lazy and the latest solution to arrive at a code that is thread-safe AND does not query the database more than once for an image.
Here is the cache class implementation:
public sealed class ApplicationImageCache2 : IApplicationImageCache2
{
private readonly Func<IApplicationImageRepository> _applicationImageRepository;
private readonly Func<IApplicationImageCacheStorage2> _storage;
public ApplicationImageCache2(Func<IApplicationImageRepository> applicationImageRepository, Func<IApplicationImageCacheStorage2> storage)
{
_applicationImageRepository = applicationImageRepository ?? throw new ArgumentNullException(nameof(applicationImageRepository));
_storage = storage ?? throw new ArgumentNullException(nameof(storage));
Console.WriteLine($"{GetType()} instance #{GetHashCode()} created from thread: {Thread.CurrentThread.ManagedThreadId}");
}
public byte[] GetApplicationImage(ImageType imageType)
{
Console.WriteLine($"{GetType()}.{nameof(GetApplicationImage)} #{GetHashCode()}({imageType}) " +
$"call from thread: {Thread.CurrentThread.ManagedThreadId}");
return _storage().GetApplicationImage(imageType,
kt => new Lazy<byte[]>(() => _applicationImageRepository().GetImage(kt)));
}
}
The change from the previous code is that the loader delegate wraps the database result in a Lazy, as described in the second solution.
The storage class is adapted similarly:
internal sealed class ApplicationImageCacheStorage2 : IApplicationImageCacheStorage2
{
private readonly ConcurrentDictionary<ImageType, Lazy<byte[]>> _applicationImages = new();
public ApplicationImageCacheStorage2()
{
Console.WriteLine($"{GetType()} instance #{GetHashCode()} created from thread: {Thread.CurrentThread.ManagedThreadId}");
}
public byte[] GetApplicationImage(ImageType imageType, Func<ImageType, Lazy<byte[]>> loaderFunc)
{
if (loaderFunc == null) throw new ArgumentNullException(nameof(loaderFunc));
Console.WriteLine($"{GetType()}.{nameof(GetApplicationImage)} #{GetHashCode()}({imageType}) " +
$"call from thread: {Thread.CurrentThread.ManagedThreadId}");
if (loaderFunc == null) throw new ArgumentNullException(nameof(loaderFunc));
return _applicationImages.GetOrAdd(imageType, loaderFunc).Value;
}
}
This test has the most interesting log:
ApplicationImageCache2 instance #21647132 created from thread: 17
ApplicationImageCache2 instance #34717384 created from thread: 19
ApplicationImageCache2 instance #49538252 created from thread: 16
ApplicationImageCache2 instance #22460983 created from thread: 20
ApplicationImageCache2 instance #6451435 created from thread: 18
ApplicationImageCache2 instance #31787101 created from thread: 23
ApplicationImageCache2 instance #56140151 created from thread: 14
ApplicationImageCache2 instance #20974680 created from thread: 12
ApplicationImageCache2 instance #32332666 created from thread: 24
ApplicationImageCache2 instance #24388906 created from thread: 21
ApplicationImageCache2 instance #15225125 created from thread: 8
ApplicationImageCache2 instance #5896758 created from thread: 7
ApplicationImageCache2 instance #2256335 created from thread: 25
ApplicationImageCache2 instance #58366981 created from thread: 13
ApplicationImageCache2 instance #42931033 created from thread: 15
ApplicationImageCache2 instance #25960677 created from thread: 22
ApplicationImageCache2.GetApplicationImage #56140151(FooterImage) call from thread: 14
ApplicationImageCache2.GetApplicationImage #58366981(HeaderImage) call from thread: 13
ApplicationImageCache2.GetApplicationImage #6451435(BackgroundImage) call from thread: 18
ApplicationImageCache2.GetApplicationImage #2256335(HeaderImage) call from thread: 25
ApplicationImageCache2.GetApplicationImage #24388906(BackgroundImage) call from thread: 21
ApplicationImageCache2.GetApplicationImage #20974680(FooterImage) call from thread: 12
ApplicationImageCache2.GetApplicationImage #22460983(FooterImage) call from thread: 20
ApplicationImageCache2.GetApplicationImage #31787101(FooterImage) call from thread: 23
ApplicationImageCache2.GetApplicationImage #25960677(HeaderImage) call from thread: 22
ApplicationImageCache2.GetApplicationImage #15225125(HeaderImage) call from thread: 8
ApplicationImageCache2.GetApplicationImage #5896758(BackgroundImage) call from thread: 7
ApplicationImageCache2.GetApplicationImage #32332666(BackgroundImage) call from thread: 24
ApplicationImageCache2.GetApplicationImage #49538252(HeaderImage) call from thread: 16
ApplicationImageCache2.GetApplicationImage #42931033(BackgroundImage) call from thread: 15
ApplicationImageCache2.GetApplicationImage #34717384(HeaderImage) call from thread: 19
ApplicationImageCache2.GetApplicationImage #21647132(FooterImage) call from thread: 17
ApplicationImageCacheStorage2 instance #45214701 created from thread: 21
ApplicationImageCacheStorage2.GetApplicationImage #45214701(HeaderImage) call from thread: 25
ApplicationImageCacheStorage2.GetApplicationImage #45214701(BackgroundImage) call from thread: 18
ApplicationImageCacheStorage2.GetApplicationImage #45214701(BackgroundImage) call from thread: 21
ApplicationImageCacheStorage2.GetApplicationImage #45214701(HeaderImage) call from thread: 16
ApplicationImageCacheStorage2.GetApplicationImage #45214701(FooterImage) call from thread: 12
ApplicationImageCacheStorage2.GetApplicationImage #45214701(FooterImage) call from thread: 20
ApplicationImageCacheStorage2.GetApplicationImage #45214701(HeaderImage) call from thread: 8
ApplicationImageCacheStorage2.GetApplicationImage #45214701(HeaderImage) call from thread: 13
ApplicationImageCacheStorage2.GetApplicationImage #45214701(FooterImage) call from thread: 14
ApplicationImageCacheStorage2.GetApplicationImage #45214701(HeaderImage) call from thread: 22
ApplicationImageCacheStorage2.GetApplicationImage #45214701(BackgroundImage) call from thread: 7
ApplicationImageCacheStorage2.GetApplicationImage #45214701(BackgroundImage) call from thread: 24
ApplicationImageCacheStorage2.GetApplicationImage #45214701(FooterImage) call from thread: 23
ApplicationImageCacheStorage2.GetApplicationImage #45214701(FooterImage) call from thread: 17
ApplicationImageCacheStorage2.GetApplicationImage #45214701(BackgroundImage) call from thread: 15
ApplicationImageCacheStorage2.GetApplicationImage #45214701(HeaderImage) call from thread: 19
ApplicationImageRepository instance #54718731 created from thread: 14
ApplicationImageRepository instance #39449526 created from thread: 8
ApplicationImageRepository instance #64685481 created from thread: 21
ApplicationImageRepository.GetImage #39449526(HeaderImage) call start from thread: 8
ApplicationImageRepository.GetImage #54718731(FooterImage) call start from thread: 14
ApplicationImageRepository.GetImage #64685481(BackgroundImage) call start from thread: 21
ApplicationImageRepository.GetImage #54718731(FooterImage) call end from thread: 14
ApplicationImageRepository.GetImage #64685481(BackgroundImage) call end from thread: 21
ApplicationImageRepository.GetImage #39449526(HeaderImage) call end from thread: 8
ApplicationImageCache2 instance #45814213 created from thread: 25
ApplicationImageCache2 instance #13273752 created from thread: 19
ApplicationImageCache2 instance #47741942 created from thread: 23
ApplicationImageCache2 instance #11373314 created from thread: 20
ApplicationImageCache2 instance #54752321 created from thread: 22
ApplicationImageCache2 instance #46228029 created from thread: 7
ApplicationImageCache2 instance #31071611 created from thread: 16
ApplicationImageCache2.GetApplicationImage #46228029(HeaderImage) call from thread: 7
ApplicationImageCache2.GetApplicationImage #13273752(BackgroundImage) call from thread: 19
ApplicationImageCacheStorage2.GetApplicationImage #45214701(HeaderImage) call from thread: 7
ApplicationImageCacheStorage2.GetApplicationImage #45214701(BackgroundImage) call from thread: 19
ApplicationImageCache2.GetApplicationImage #31071611(FooterImage) call from thread: 16
ApplicationImageCacheStorage2.GetApplicationImage #45214701(FooterImage) call from thread: 16
ApplicationImageCache2.GetApplicationImage #45814213(BackgroundImage) call from thread: 25
ApplicationImageCacheStorage2.GetApplicationImage #45214701(BackgroundImage) call from thread: 25
Lines 50-52 show the creation of exactly 3 repository instances on 3 threads. Line 53-58 is the trace of the database calls. It is excellently visible that they overlap each other. I.e. they worked in parallel! 1 thread was allowed to load the image for each image type, and not more.
Thread-safe code? Yes. Efficient? Yes.
We lock the initial requests by not a large-scale lock, but we use locks per image type! You can consider it as we have achieved a fined graned locking. Where are those locks implemented I’m talking about? Inside the Lazy class.
I like this final solution. Do you like it too? Comment your opinion below.
Could you hire me? Contact me if you like what I’ve done in this article and think I can create value for your company with my skills.