Skip to content

Add interactive mode telemetry to track terminal capabilities#4449

Open
simonfaltum wants to merge 1 commit intomainfrom
simonfaltum/telemetry-interactive-mode
Open

Add interactive mode telemetry to track terminal capabilities#4449
simonfaltum wants to merge 1 commit intomainfrom
simonfaltum/telemetry-interactive-mode

Conversation

@simonfaltum
Copy link
Member

@simonfaltum simonfaltum commented Feb 5, 2026

This change adds a new interactive_mode field to telemetry to measure what proportion of CLI invocations can support interactive features like selection prompts.

The field captures one of three values:

  • "full": Both interactive output (spinners, colors) and user prompts are supported. This requires stderr to be a TTY, color enabled, stdin to be a TTY, and not running in Git Bash.
  • "output_only": Interactive output is supported but prompts are not. This occurs when stdin is not a TTY (e.g., input piped) or when running in Git Bash (which has broken readline/ANSI support).
  • "none": Non-interactive environment where neither spinners nor prompts work. This includes CI/CD pipelines, cron jobs, and cases where stderr is redirected.

Changes

  • Add InteractiveMode field to ExecutionContext in telemetry protos
  • Add InteractiveMode() method to Capabilities struct in cmdio
  • Add GetInteractiveMode(ctx) public function for context-safe access
  • Populate the field during telemetry upload in root.Execute()

Why

This telemetry helps inform decisions about when the CLI can prompt users for missing information versus requiring explicit flags. By measuring the proportion of invocations in each mode, we can understand how often interactive prompts would actually be shown to users.
This will also help give us some numbers when we want to track impact of a given change

Tests

Added unit tests in capabilities_test.go covering all three interactive modes:

  • Full interactive (all TTYs, color enabled, no Git Bash)
  • Output only (stdin not TTY, or Git Bash)
  • None (stderr not TTY, or color disabled)

Comment on lines +80 to +83
if c.SupportsPrompt() {
return InteractiveModeFull
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SupportsPrompt requires SupportsInteractive as well, so thats why we can do the logic this way

This change adds a new `interactive_mode` field to telemetry to measure
what proportion of CLI invocations can support interactive features like
selection prompts.

The field captures one of three values:
- "full": Both interactive output (spinners, colors) and user prompts
  are supported. This requires stderr to be a TTY, color enabled, stdin
  to be a TTY, and not running in Git Bash.
- "output_only": Interactive output is supported but prompts are not.
  This occurs when stdin is not a TTY (e.g., input piped) or when
  running in Git Bash (which has broken readline/ANSI support).
- "none": Non-interactive environment where neither spinners nor prompts
  work. This includes CI/CD pipelines, cron jobs, and cases where stderr
  is redirected.

This telemetry helps inform decisions about when the CLI can prompt users
for missing information versus requiring explicit flags.

Changes:
- Add InteractiveMode field to ExecutionContext in telemetry protos
- Add InteractiveMode() method to Capabilities struct in cmdio
- Add GetInteractiveMode(ctx) public function for context-safe access
- Populate the field during telemetry upload in root.Execute()
- Add comprehensive tests for all three interactive modes
@simonfaltum simonfaltum force-pushed the simonfaltum/telemetry-interactive-mode branch from 5df80af to cbf3069 Compare February 5, 2026 11:19
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Feb 5, 2026

Commit: cbf3069

Run: 21709486854

Env 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 1 7 448 731 26:48
🟨​ aws windows 7 2 1 7 412 741 23:15
🟨​ aws-ucws linux 3 9 5 701 584 80:32
🟨​ aws-ucws windows 3 2 9 5 662 595 75:44
💚​ azure linux 2 9 448 730 24:43
💚​ azure windows 2 9 414 740 21:19
💚​ azure-ucws linux 6 7 663 595 72:16
💚​ azure-ucws windows 6 7 626 606 57:40
💚​ gcp linux 2 9 437 736 26:46
💚​ gcp windows 2 9 403 746 24:12
21 interesting tests: 7 KNOWN, 5 SKIP, 5 RECOVERED, 4 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/deployment/bind/alert 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/generate/alert 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/alerts/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/alerts/with_file 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/bundle/resources/apps/inline_config ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
💚​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/templates/default-python/integration_classic ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12 ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
💚​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
8:17 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
8:09 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:39 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
7:35 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
7:31 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
7:19 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:13 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
7:04 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:01 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
6:52 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
6:50 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
6:37 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
6:12 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
6:12 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
6:08 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
6:06 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
5:42 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
5:39 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:25 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:20 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:15 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:11 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:06 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:21 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
4:13 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
4:12 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
4:03 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
4:02 azure-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
4:02 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
4:01 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
3:56 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
3:55 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
3:44 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
3:42 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:41 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:41 azure-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
3:39 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:37 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
3:29 azure-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
3:22 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:13 azure-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
3:12 azure-ucws linux TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=yes/READPLAN=
3:12 azure-ucws linux TestAccept/bundle/resources/models/basic/DATABRICKS_BUNDLE_ENGINE=terraform
3:07 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:07 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:07 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:04 azure-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
3:02 azure-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
2:58 aws-ucws linux TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no/READPLAN=
2:57 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants