Skip to content

Fix concurrent type translation minting duplicate target types (#521) - #522

Merged
dsyme merged 2 commits into
fsprojects:masterfrom
Thorium:fix-concurrent-type-translation
Jul 29, 2026
Merged

Fix concurrent type translation minting duplicate target types (#521)#522
dsyme merged 2 commits into
fsprojects:masterfrom
Thorium:fix-concurrent-type-translation

Conversation

@Thorium

@Thorium Thorium commented Jul 9, 2026

Copy link
Copy Markdown
Member

A possible solution, fixes #521

Problem

The F# compiler may drive a type provider from multiple threads — ParallelCompilation is on by default in recent .NET SDKs — and two code paths in ProvidedTypes.fs are not thread-safe under that.

1. Cross-targeting type translation (the cause of the intermittent FS0193/FS0001 failures). convProvidedTypeDefToTgt does an unsynchronized check-then-create over the plain Dictionary typeTableFwd:

match typeTableFwd.TryGetValue(x) with
| true, newT -> (newT :?> ProvidedTypeDefinition)
| false, _ ->
    ...
    let rec xT : ProvidedTypeDefinition = ProvidedTypeDefinition(true, ...)
    Debug.Assert(not (typeTableFwd.ContainsKey(x)))   // the invariant the race violates
    typeTableFwd.[x] <- xT

Two concurrent conversions of the same source ProvidedTypeDefinition both miss the lookup and each mint a distinct target type. The compiler then holds two reference-distinct copies of the same erased provided type and fails intermittently with FS0193: The type 'X' is not compatible with the type 'X' (identical names on both sides). Provider-side caching cannot prevent this — it deduplicates the source type, while the duplication happens here on the target side.

2. Member realization (hardening). evalMembers/evalInterfaces/evalMethodOverrides drain their delayed queues and append to plain ResizeArrays without synchronization, so concurrent GetMembers calls can run the delayed factories twice and corrupt the backing lists. The bindings lookup cache in save has the same problem.

Fix

  • A re-entrant lock (typeTablesLock) makes lookup+create+store in convProvidedTypeDefToTgt one atomic step, shared with convTypeRef/convType whose unlocked Dictionary reads could otherwise race the locked write.
  • A per-instance re-entrant lock (realizationLock) around member/interface/override realization and the bindings cache.

Both use the rename-and-wrap pattern (xUnsafe + locked wrapper) to keep the diff reviewable.

Lock ordering is acyclic: translation acquires source-type realization locks (via the member cursors); target-type realization acquires the translation lock (via convMemberDefToTgt in the fresh-member closures); translation never forces target member realization (PatchDeclaringTypeOfMember only repoints the declaring type). No deadlock observed across repeated full builds of a heavy consumer.

Validation

SQLProvider's test suite (17 SqlDataProvider<...> instantiations of identical static arguments, .NET SDK 10, clean-rebuild loop, single target framework):

Configuration Result
Unpatched fails ~1 in 10 iterations with FS0193/FS0001
Provider-side permanent memoization still fails — source side was never the problem
Realization lock only still fails, duplicating a type added eagerly (not via delayed members)
Realization + translation locks (this PR) 12/12 clean-rebuild iterations green (every failing configuration above died by iteration 14)

Thorium added 2 commits July 10, 2026 00:16
…jects#521)

The F# compiler may drive a type provider from multiple threads
(ParallelCompilation is on by default in recent .NET SDKs). Two
unsynchronized code paths in ProvidedTypes.fs then race:

1. ProvidedTypesContext type translation (the fix that matters):
   convProvidedTypeDefToTgt performs check-then-create on the plain
   Dictionary typeTableFwd. Two concurrent conversions of the same
   source ProvidedTypeDefinition both miss the lookup and each mint a
   distinct target ProvidedTypeDefinition, so the compiler sees two
   unrelated erased types and fails intermittently with
   FS0193/FS0001 "type X is not compatible with type X". The existing
   Debug.Assert(not (typeTableFwd.ContainsKey(x))) documents exactly
   the invariant the race violates. The lookup+create+store is now one
   atomic step under a re-entrant lock shared by convTypeRef/convType,
   whose unlocked table reads could otherwise race the locked write.

2. ProvidedTypeDefinition member realization (hardening): evalMembers/
   evalInterfaces/evalMethodOverrides drain their delayed queues and
   append to plain ResizeArrays with no synchronization, so concurrent
   GetMembers calls can run delayed factories twice and corrupt the
   backing lists. Realization now runs under a per-instance re-entrant
   lock, which also guards the bindings lookup cache.
@dsyme

dsyme commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Great work. Following up with some additional things code review found in a separate issue

@dsyme
dsyme merged commit 4f73f60 into fsprojects:master Jul 29, 2026
2 checks passed
@dsyme

dsyme commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Thanks for this fix! During review I noted that varTableFwd/varTableBwd and assemblyTableFwd have the same unguarded check-then-create pattern and are not covered by typeTablesLock. Filed #525 and opened #526 to extend the lock to those tables as well.

dsyme added a commit that referenced this pull request Jul 29, 2026
Three new tests exercise the translation tables under parallel access:

1. Concurrent ConvertSourceExprToTarget on shared Vars - exercises
   convVarToTgt/convVarToSrc under contention (8 threads, shared Var
   instances, barrier-synchronized start).

2. Concurrent ConvertSourceTypeToTarget - exercises convType/convTypeRef
   with various type kinds (generics, arrays, primitives) from 8 threads.

3. Concurrent type translation through ProvidedAssembly - exercises
   convProvidedAssembly by translating 16 types sharing one
   ProvidedAssembly from 8 threads simultaneously.

Without the locks from #522 and this PR, these tests would fail with
ArgumentException (duplicate Dictionary key) or state corruption.
dsyme added a commit that referenced this pull request Jul 29, 2026
…ety (#525) (#526)

## Problem

varTableFwd/varTableBwd and assemblyTableFwd in ProvidedTypesContext
have the same unguarded check-then-create race condition that
typeTableFwd/typeTableBwd had before #522.

Under ParallelCompilation=true (default in recent .NET SDKs):
- Var tables: Two threads realizing different target types concurrently
can both translate quotations referencing the same Var, racing on
Dictionary.Add (ArgumentException on duplicate key) or corrupting state
via concurrent read/write.
- Assembly table: The lazy namespace closure in convProvidedTypeDefToTgt
can call convProvidedAssembly from multiple threads, duplicating the
assembly translation.

## Fix

Wrap convVarToSrc, convVarToTgt, and convProvidedAssembly in the
existing typeTablesLock (introduced in #522). The lock is already
re-entrant (Monitor), so the recursive calls to
convTypeToTgt/convTypeToSrc inside these functions are safe.

The lock comment is updated to reflect the broader scope.

## Validation

- All 160 existing tests pass.
- No deadlock risk: same lock ordering as #522 (typeTablesLock then
source realizationLock; target realizationLock then typeTablesLock - no
cycle).

Fixes #525
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NET10: ProvidedTypeDefinition member realization is not thread-safe → intermittent FS0193/FS0001 under ParallelCompilation=true

2 participants