[Repo Assist] Add more stress benchmarks and perf improvements (transTypeRef + transMethRef caching) - #457
Merged
dsyme merged 2 commits intoFeb 26, 2026
Conversation
…sMethRef caching)
- Add transTypeRefCache in AssemblyCompiler.Compile: caches ILTypeRef per
System.Type (using generic type definition as key). Avoids allocating
duplicate ILTypeRef objects when the same BCL type appears repeatedly as
field/parameter/declaring type across many schema members.
- Add transMethRefCache in AssemblyCompiler.Compile: caches ILMethodRef
per MethodInfo (using GetDefinition() as key). Eliminates redundant
ILMethodRef allocations when the same BCL method is called from many
generated method bodies (e.g. String.IsNullOrEmpty in data validation).
- New benchmark helpers:
* buildProviderWithProperties: types with ProvidedField backing +
getter (FieldGet) and setter (FieldSet) bodies — simulates REST API
data-transfer-object schemas
* buildProviderWithComplexBodies: methods whose bodies call real BCL
methods (String.IsNullOrEmpty / Object.ToString) — exercises the
transMeth/transMethRef path
* buildProviderWithNestedTypes: outer types with configurable nesting
depth — exercises the recursive type-definition walk in Compile()
- New BenchmarkDotNet classes: PropertyHeavyBenchmark,
ComplexBodyBenchmark, NestedTypesBenchmark, XLargeStressBenchmark
(1000 types × 10 methods)
- Updated runQuickMeasurement to cover all new scenarios with clear
labelled output
- Updated README with new benchmark table and optimisation notes
All 102 existing tests pass unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dsyme
approved these changes
Feb 26, 2026
dsyme
marked this pull request as ready for review
February 26, 2026 11:26
This was referenced Feb 26, 2026
Closed
dsyme
pushed a commit
that referenced
this pull request
Mar 7, 2026
🤖 *This PR was created by Repo Assist, an automated AI assistant.* Prepares the RELEASE_NOTES.md for version **8.3.0** — a minor release capturing significant improvements merged since 8.2.0 (February 24). ## Changes since 8.2.0 | Type | PR | Description | |------|----|-------------| | 🐛 Bug fix | #432 | Fix custom attributes on nested erased types | | 🐛 Bug fix | #458 | Fix `GetNestedType` on `TypeSymbol`/`ProvidedTypeSymbol` for generic provided types | | 🐛 Bug fix | #459 | Fix mutable variable captures in `QuotationSimplifier` — promote to ref cells | | ⚡ Performance | #443 | Memoize `transType` in `AssemblyCompiler` to reduce redundant type translation | | ⚡ Performance | #457 | Cache `transTypeRef` and `transMethRef` in assembly compiler | | ✨ Feature | #428 | New warning when all static parameters in a type provider are optional | | 📚 Docs | #455 | Documentation guide overhaul | | 🧪 Tests | #442 | Add coverage tests and Coverage build target | | 🔧 Toolchain | #431 | Update to .NET 8 SDK and toolchain | This is a minor version bump (8.2.0 → 8.3.0) due to the new feature (#428) and significant improvements. If preferred, a patch release (8.2.1) is also reasonable given the emphasis on bug fixes. ## Test Status This PR only modifies RELEASE_NOTES.md. The build/test status for the underlying changes is tracked in the individual PRs listed above (all passed CI before merging). --- *To release: merge this PR, then tag `v8.3.0` and publish the NuGet package via the existing build pipeline.* > Generated by [Repo Assist](https://github.com/fsprojects/FSharp.TypeProviders.SDK/actions/runs/22448247879) > > To install this [agentic workflow](https://github.com/githubnext/agentics/tree/afb00b92a9514fee9a14c583f059a03d05738f70/workflows/repo-assist.md), run > ``` > gh aw add githubnext/agentics@afb00b9 > ``` <!-- gh-aw-agentic-workflow: Repo Assist, engine: copilot, id: 22448247879, workflow_id: repo-assist, run: https://github.com/fsprojects/FSharp.TypeProviders.SDK/actions/runs/22448247879 --> <!-- gh-aw-workflow-id: repo-assist --> --------- Co-authored-by: Repo Assist <github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dsyme
added a commit
that referenced
this pull request
Jul 29, 2026
…er body (issue #341) (#520) ## Problem Issue #341 reported that generative compilation (`AssemblyCompiler.Compile`) is still very slow for large providers (SwaggerProvider + Stripe schema), even after the memoization work in #443/#457/#471/#493/#497/#502/#509/#513. Profiling the benchmark suite with dotnet-trace showed that with those caches in place, **~80% of `Compile` time was FSharp.Core quotation deserialization** (`PatternsModule.deserialize` → `instMeth` → `GetGenericArguments`/`MakeGenericType`), attributed to the `CodeGenerator` constructor. Root cause: `CodeGenerator` defines ~63 active patterns from quotation literals — `let (|Addition|_|) = (|SpecificCall|_|) <@ (+) @>`, `<@ max @>`, `<@ sqrt @>`, etc. Evaluating a quotation literal deserializes it (reflection-binding every method it mentions), and a `CodeGenerator` is constructed **once per emitted member body**. A provider with 10,000 members therefore deserialized those quotations ~630,000 times. `lessThan` was worse still: it deserialized `<@@ (<) @@>` on every call. ## Fix * New `CodeGenPatterns(convTypeToTgt)` type holding all these patterns as `member val` properties, so they are computed once at construction (a plain property getter would re-evaluate per access). * `AssemblyCompiler.Compile` constructs one instance and passes it to every `CodeGenerator`; the constructor now just re-binds the local active patterns from it, so the large `emitExpr` match code is untouched. * `lessThan` uses a precomputed generic method definition. ## Results Quick benchmark (`benchmarks -- --quick`, ms/iter, before → after): | Scenario | Before | After | Speedup | |---|---|---|---| | Methods 50×20 | 880 | 86 | 10× | | Methods 200×20 | 2893 | 180 | 16× | | Methods 1000×10 | 6892 | 535 | 13× | | Props 500×10 | 8074 | 1259 | 6.4× | | ComplexBody 300×10 | 2528 | 158 | 16× | | Nested 100×4×5 | 1668 | 127 | 13× | End-to-end with the exact repro from #341 (SwaggerProvider built from source against this branch, Stripe `spec3.sdk.json`, `dotnet fsi`): **62–72 s → 19–20 s** wall clock; the remainder is dominated by schema download, OpenApi parsing and fsi startup rather than the TPSDK. Re-profiling after the change confirms the SDK-side deserialization is gone; the remaining `deserialize` samples come from the provider author's own `<@@ ... @@>` literals in `invokeCode` (once per member, inherent to user code). ## Testing * Full test suite passes: 160/160. * Benchmark suite builds and runs (numbers above). * SwaggerProvider builds unmodified against the patched `ProvidedTypes.fs` and the generated Stripe client works (`Stripe.Client()` constructed successfully). Fixes #341 (the remaining slowness reported there after the earlier rounds of caching). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Don Syme <dsyme@github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 This PR was created by Repo Assist, an automated AI assistant. Closes / contributes to #341.
What this PR does
Continues the perf-improvement work started in PR #443 by adding two new memoization caches in
AssemblyCompiler.Compileand significantly expanding the benchmark suite with realistic stress scenarios.Performance improvements in
ProvidedTypes.fs1.
transTypeRefCache— cacheILTypeRefperSystem.TypePreviously
transTypeRefallocated a newILTypeRefobject on every call even for the same type.transTypecaches the fullILType, buttransTypeRefis also called independently fromtransFieldSpec,transMethRef, etc. — so the sameILTypeRefwas being reconstructed many times per compilation. The cache eliminates these redundant allocations.2.
transMethRefCache— cacheILMethodRefperMethodInfo(Numbers from a GitHub Actions runner — useful as relative comparison baselines.)
Test Status
✅ All 102 existing tests pass (
dotnet test tests/ -c Release), 1 pre-existing skip unchanged.✅ Benchmarks build cleanly (
dotnet build benchmarks/... -c Release).✅
--quickmode runs to completion with correct output for all scenarios.