Hangfire job creation throughput performance with Redis RDB - redis

Question

In official documentation, there is a chart, which tells, that creation job throughput with Redis RDB could be around 6 000 jobs per second. I have tried different Hangfire, Redis and HW configurations, but I always get max around 200 jobs per second. I even created simple example that reproduces it (Hangfire configuration, job creation).
Am I doing something wrong? What job creation throughput performance are you getting?
I am using latest versions: Hangfire 1.7.24, Hangfire.Pro 2.3.0, Hangfire.Pro.Redis 2.8.10 and Redis 6.2.1.

Answer 1

The point is that in the referenced sample application, background jobs are being created sequentially, one after another. In this case background jobs aren't created fast enough due to I/O delays (round-trips to the storage), to result in better throughput. And since there's also a call to Hangfire.Console that requires even more I/O, creation process is performed even slower.
Try to create background jobs in a Parallel.For loop to create background job in parallel and amortize the latency. And try to create all the background jobs before starting the server to make a clear distinction between created/sec and performed/sec metrics as shown below, otherwise everything will be mixed up.
var sw = Stopwatch.StartNew();
Parallel.For(0, 100000, new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount }, i =>
{
BackgroundJob.Enqueue(() => Empty());
});
Console.WriteLine(sw.Elapsed);
using (new BackgroundJobServer())
{
Console.ReadLine();
}
On my development machine I've got 7.7 sec to create 100,000 background jobs (~13,000 jobs/sec) and Dashboard UI told me that perform rate is ~3,500 jobs/sec that's a bit lower than displayed on the chart, but that's because there are more extension filters now in Hangfire than 6 years ago when that chart was created. And if we clear them with GlobalJobFilters.Filters.Clear(), we'll get about 4,000 jobs/sec.
To avoid the confusion, I've removed the absolute numbers from those charts today. Absolute numbers are different for different environments, e.g. on-premise (can be faster) and cloud (will be slower). That chart was created to show the relative difference between SQL Server and Redis in different modes, which is approximately the same in different env, not to show the precise numbers that depend on a lot of factors, especially when network is involved.