High lock contention in JTF

[sharwell]

During a profiling pass on a solution load of Roslyn.sln, I observed over 30 seconds of lock contention within this library.

The key offenders were the following:

Method	Exclusive time in lock contention	Fix
JoinableTask.get_ApplicableJobSyncContext()	7310ms	#409
JoinableTask.Complete()	5340ms
JoinableTaskContext.get_Factory()	3587ms	#409
JoinableTask.SetWrappedTask(Task)	2505ms
JoinableTask.get_IsCompleted()	2431ms
JoinableTaskCollection.Disjoin(JoinableTask)	1727ms
JoinableTask.get_Task()	1595ms	#411
JoinableTask.CompleteOnCurrentThread()	1232ms
JoinableTaskCollection.Join()	1155ms

[AArnott]

Is this a dupe of #132? JTF normally runs without much contention I think. But in #132 it's awful because a lock is held while performing an n^3 algorithm.

[sharwell]

@AArnott I will try to determine if this is a duplicate. Keep in mind my test was run on a system with more cores than average (14C + HT), so based on what we know about end users today this information is more of a "this situation is inevitable but yet to get here". Either way, lock-free fast paths in these methods would provide immediate measurable wins.

[sharwell]

@AArnott I see a total of 1863ms spent in Disjoin, so it's possible this is fallout from #132. I'm hesitant to treat it as a full duplicate since investigation suggests there are several cases above where lock-free fast paths are not unreasonable to incorporate.

[sharwell]

I just collected a 30 second trace where 99.5% of the UI thread activity was vs-threading contention in three methods:

• JoinableTaskContext.get_Factory
• JoinableTask.get_ApplicableJobSyncContext
• JoinableTask.get_Task

With 28 logical processors, it was a fairly frequent occurrence to have problems with the global locking policy. However, a recent move to 64 is quickly making this unsustainable.

[AArnott]

All of those property getters are trivially quick, I wonder if something else was holding the lock while those three were contending over it. It should be straightforward to allow those properties to be lock free (or hold private locks, perhaps).

I was talking to @stephentoub about this earlier this week. He suggested that you share the exact CPU and .NET Framework runtime version you're using. There was a lock contention regression with certain combinations that later versions of netfx fixed.

[stephentoub]

cc: @kouvel

[AArnott]

@sharwell Can you reproduce the problem, such that I could give you a patched vs-threading dll to see if it resolves the issue?

[sharwell]

@AArnott I'm running a 2990WX with .NET 4.7.2 (32-bit). I can check to see if the issue is reproducible later this week.

[kouvel]

The only regression in .NET's lock contention I'm aware of that was fixed is a CPU time regression in spin-waiting in Monitor and a few other types due to increase pause instruction delay in Skylake and more recent processors. The fix was ported to 4.7.2 but it doesn't yet seem relevant here.