Investigate test case association feature and its support with MTP

[Youssef1313]

https://learn.microsoft.com/en-us/azure/devops/test/associate-automated-test-with-test-case?view=azure-devops

It looks like this feature may be working by setting a VSTest property on TestCase, which means it may not work with MTP.

We need to investigate further.

[JoaoAntunes8210474]

@Youssef1313 Hello sir, hope you are doing well!

Would you be able to give any information or any plans/dates for this feature being supported by MTP?

[Youssef1313]

Hi @JoaoAntunes8210474

Unfortunately we didn't yet investigate what's needed to support this in MTP.

[JoaoAntunes8210474]

Hey @Youssef1313, hope you are doing well!

Would you be able to let me know if investigating how to support this feature in MTP is something that's planned for the near future?

It's a somewhat important feature for integrating tests with Azure DevOps' test cases.

[nohwnd]

I don't have the technical details, I've reached out to ADO team to see if they can point me to the code that does it.

[JoaoAntunes8210474]

@nohwnd Hello sir, hope you are doing well!

Would you happen to have any updates regarding contacting the ADO team for information over how this feature was previously supported on VSTest?

[nohwnd]

I got the supposed locations of the code, but did not get to investigate it yet, I will at best get to it next week.

[Youssef1313]

Related to #5951

I assume the general flow that users do is:

• Associate test cases in Test Explorer to AzDO test case, and an AzDO test case is part of a test plan.
• Use VSTest AzDO task to run a given test plan by specifying testSelector: 'testPlan' and then specify the test plan they want to run.

What we need to do for MTP to support this:

1. Update Visual Studio to support associating test cases to AzDO test cases for MTP. We have a whitelisted set of adapters that we should include testingplatform-bridge in.
2. The NuGet package Microsoft.TeamFoundationServer.Client has APIs related to test plans. We need to figure out the exact API we need that exposes test cases in a given test plan (hopefully there will be something there in that package?).
3. We don't implement the support in an AzDO task. Instead, we create an AzDO test plan extension.
   • The test plan AzDO extension package will have a dependency on MS.TFS.Client
   • When the user specificies --azdo-test-plan, we will query the test plan, and use AddTestExecutionFilterFactory to construct a filter. But here, we will face an issue, as filters are not very well standardized, except for test node uids. I'm guessing that on VSTest, there was a special casing for MSTest/NUnit/xUnit when the filter is constructed.

[JoaoAntunes8210474]

@Youssef1313 The general flow you described is precisely the most common way we use the feature in our company.

We also have a dozen or so Azure Pipelines running tasks to request from the Azure DevOps API information on which test cases have the Automated status. Which is why this feature is very important to us (although we are probably not the only ones using the feature in this way).

[JoaoAntunes8210474]

@nohwnd Hello sir, hope you are doing well!

Would you be able to tell us if you've had the opportunity to investigate this issue?

[Youssef1313]

Unfortunately, we didn't yet prioritize an implementation for this feature.

[Youssef1313]

FYI @mckennabarlow