Skip to content

Conversation

@andrewnester
Copy link
Contributor

Changes

Separate test output for direct and terraform for interactive_single_user test

Why

TF test behaviour for wheel jobs on single-user interactive clusters is technically incorrect because TF in such a setup produces permanent drift (incorrect), which leads to cluster restart on each bundle deploy, and therefore wheel changes are picked up

  # databricks_cluster.shared_autoscaling will be updated in-place
  ~ resource "databricks_cluster" "shared_autoscaling" {
        id                             = "1015-102652-xxxxxxx"
      - single_user_name               = "aaaaaaaabbbb-bbbb-cccc-dddd-bb6ce382668a" -> null
        # (19 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Direct does not update/restart the cluster on subsequent deploys (correct), and changes for the wheel are not picked, which is expected.

I separated the test output into 2 separate files for now, to unlock the test.

Tests

Existing tests pass

Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment about this test:

We have the dynamic_version property for artifacts that forces the upgrade. If this is not used, and we observe a code change (perhaps mtime), should we start warning the user? Not sure if this is easy or not.

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Oct 15, 2025

Run: 18527270214

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip
💚​ aws linux 1 321 545
💚​ aws windows 1 322 544
🔄​ aws-ucws linux 10 1 427 441
🔄​ aws-ucws windows 9 1 429 440
💚​ azure linux 1 321 544
💚​ azure windows 1 322 543
🔄​ azure-ucws linux 5 1 432 440
🔄​ azure-ucws windows 1 1 437 439
🔄​ gcp linux 3 318 546
❌​ gcp windows 18 1 304 544
30 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f 🟨​K
TestAccept/bundle/deployment/bind/dashboard ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
TestAccept/bundle/deployment/bind/dashboard/recreation ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
TestAccept/bundle/resources/dashboards/detect-change ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
TestAccept/bundle/resources/dashboards/generate_inplace ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/resources/dashboards/nested-folders ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
TestAccept/bundle/resources/dashboards/simple ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
TestAccept/bundle/resources/dashboards/simple_outside_bundle_root ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
TestAccept/bundle/resources/dashboards/simple_syncroot ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
TestAccept/bundle/resources/synced_database_tables/basic 🙈​s 🙈​s 🔄​f ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s
TestAccept/bundle/templates/default-python/integration_classic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestDashboardAssumptions_WorkspaceImport ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p
TestFsCatOnADir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCatOnADir/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCatOnNonExistentFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCatOnNonExistentFile/dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpDirToDirFileNotOverwritten ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpDirToDirFileNotOverwritten/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpDirToDirFileNotOverwritten/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpDirToDirFileNotOverwritten/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToDir/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToDir/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToDir/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToFile ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToFile/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToFile/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestFsCpFileToFile/dbfs_to_uc-volumes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s ❌​F
TestFsCpFileToFile/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F

@andrewnester
Copy link
Contributor Author

@pietern I'd actually make it easier and warn customers if they use the wheels with interactive clusters and don't use the dynamic version option, and recommend that they use it. But will make it separate from this PR anyway

@andrewnester andrewnester requested a review from pietern October 15, 2025 12:28
@andrewnester andrewnester merged commit de9567a into main Oct 15, 2025
12 of 13 checks passed
@andrewnester andrewnester deleted the fix/cluster-test branch October 15, 2025 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants