Use FUSE for bundle commands on serverless#4450
Merged
shreyas-goenka merged 4 commits intomainfrom Feb 6, 2026
Merged
Conversation
Collaborator
|
Commit: 8e13d0f
19 interesting tests: 7 KNOWN, 5 SKIP, 5 RECOVERED, 2 flaky
Top 50 slowest tests (at least 2 minutes):
|
denik
approved these changes
Feb 6, 2026
Contributor
denik
left a comment
There was a problem hiding this comment.
Much faster bundle deployments / summaries etc for DABs in the workspace.
Could you share the measurements?
| // On serverless (client version 2.5+), use the native sync root directly via FUSE. | ||
| // The FUSE provides capabilities for both reading and writing notebooks. It also | ||
| // is much faster and enables running cloud tests on DBR, since otherwise the tests | ||
| // fail with an AsyncFlushError because of the conflict between writing to FUSE |
Contributor
There was a problem hiding this comment.
q - does this also impact Dabs-in-ws?
Contributor
Author
There was a problem hiding this comment.
This is necessary to make the bundle generate --bind command work. Right now DABs in ws mixes the API and FUSE. This PR makes that consistent.
Contributor
Author
In theory it should have better speed, because it uses the same API underneath the root, but has better caching and async writes that do not wait for network requests to go through. My setup got deleted as part of the nightly script unfortunately. We'll get some concrete numbers soon via telemetry though (in ~2-3 weeks) |
Collaborator
|
Commit: 1bdf869
27 interesting tests: 10 flaky, 7 KNOWN, 5 SKIP, 5 RECOVERED
Top 50 slowest tests (at least 2 minutes):
|
pietern
reviewed
Feb 6, 2026
| // GetVersion returns the parsed runtime version from the context. | ||
| // It expects a context returned by [DetectRuntime] or [MockRuntime]. | ||
| func GetVersion(ctx context.Context) Version { | ||
| return ParseVersion(RuntimeVersion(ctx)) |
Contributor
There was a problem hiding this comment.
Shouldn't RuntimeVersion() also return a Version type?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
This PR updates the CLI to use the FUSE to read and write notebooks instead of the workspace APIs, when the DBR version is serverless 2.5+. This has the following benefits:
bundle generate --bindwhich were previously failing on DABs in the workspace due to both the workspace APIs and FUSE being used simultaneously.Tests
Manually verified that both bundle deploy / bundle generate continue to work correctly on DBR. Also verified that
bundle generate --bindworks on DBR.Acceptance tests will be added once #4416 lands.
bundle generate --bind works, i.e. all notebooks are written successfully via FUSE:
bundle deploy works, for all sorts of notebooks, i.e. they are read correctly.
This was tested and run on serverless
client.2.5(standard v2). The UI does not allow me to select a minor version so I could not test other client.2 versions like 2.4. That's why we only use FUSE from 2.5 onwards.In practice this is fine because customers wil also likely be using
2.5or above it they are on serverless.