VMware: VM ingestion broken in case of dynamically handled hypervisor/guest OS versions.

[andrijapanicsb]

ACS 4.19 RC1

VMware ingestion is broken when testing with VMware 8.0.0.2as the hypervisor and Rocky Linux (8 or 9) - this is a consequence of changes implemented in #7380:

• if no record for 8.0.0.2 in the hypervisor_capabilities table, then use "capabilities" of the "parent" version which is VMware 8.0;
• if no records in the guest_os_hypervisor table, then use the records/mappings of the "parent" version - from 8.0
• this all works fine during VM deployment
• this does not work during VMware ingestion, where we do reverse-mapping, and code seems to look for a DB record for "rockylinux_64Guest" for version 8.0.0.2 in the guest_os_hypervisor table - and this record does not exist

The logic of "mapping minor version guest OS mappings, if missing, to a parent version mappings" - sounds fine, as long as:

• It's supported by all ACS features/code where we try to use guest OS hypervisor mappings (this now includes VMware VM ingestion that needs to be fixed, but there might be other places)
• there is detailed logging similar to "The guest OS mappings for XXX was not found for the hypervisor version YYYYY (8.0.0.2), looking for parent version mappings (8.0), found, using that mapping etc" - something similar to this, needs to be very verbose, otherwise troubleshooting will be impossible.

For the hypervisor capabilities, I would propose the following:

• Revert the logic of dynamically "assuming" that the minor version has the same capabilities as the major version (in fugure, VMware e.g. 8.5 version will certainly NOT have the same capabilities as the 8.0 version, it would have more capabilities) - so revert the code and recreated the needed DB records in the hypervisor_capabilities table
• Worst case, even though it's a bad solution - leave it as it is, but add a loooot of logging, similar to the guest OS mappings logging (not found this, using that version capabilities)

[shwstppr]

@andrijapanicsb @weizhouapache @nvazquez @DaanHoogland @harikrishna-patnala @rohityadavcloud this seems like a blocker. I guess I'll have to recall the RC1 and wait for a fix. Any other suggestions?

[shwstppr]

Did some investigation for this. Guest OS and VM import issue should be solved with #8426 (currently it needs #8429 to be merged). The hypervisor capabilities issue is more complex and below are my findings,

• Currently, the code will use the capabilities of the parent version if the capabilities of the minor/patch hypervisor version are missing.
• In the above case there is no option to update the capabilities manually using updateHypervisorCapabilities API as there is no entry for the given hypervisor version, eg: VMware 8.0.0.2

Based on the above findings and the original report do we need to consider the hypervisor capabilities issue as a blocker?

To solve this hypervisor capabilities behaviour I feel the following are our options:

1. Use the mechanism of retrieving hypervisor capabilities from the parent hypervisor version only for patch or lower versions. Add logs in the code to highlight parent version capabilities are used. Document the behaviour.
2. Use the mechanism of retrieving hypervisor capabilities from the parent hypervisor version as it is with additional logs. Support manually updating capabilities for inexisting minor or lower hypervisor versions by adding hypervisorversion param in the updateHypervisorCapabilities API.
3. Revert this mechanism of retrieving hypervisor capabilities from the parent hypervisor version and add capabilities for 8.0.0.2 in the traditional manner.

@andrijapanicsb @weizhouapache cc @harikrishna-patnala @DaanHoogland @nvazquez feedback, suggestions?

[DaanHoogland]

@shwstppr at this stage I think you can make the call you do above. In general and as a matter of design principle I feel we should always fall back even over major versions (ad 1). In this case, woking around a backwards incompatible vmware feature we will have to be explicit about what we support. making an extra API is good if that means the operator has more control, but we must make sure regular upgrades can not conflict with that (ad 2). I think we should fix forward, not backwards (ad 3). Of course only if feasible given we want to release.

[shwstppr]

Based on the discussion with some of the participants here, I've created #8444 to explicitly add support for VMware 8.0.0.2 and also add logs when capabilities retrieval falls back to the parent version

[DaanHoogland]

Based on the discussion with some of the participants here, I've created #8444 to explicitly add support for VMware 8.0.0.2 and also add logs when capabilities retrieval falls back to the parent version

will you include that now? or is it for 4.19.1?

[shwstppr]

Based on the discussion with some of the participants here, I've created #8444 to explicitly add support for VMware 8.0.0.2 and also add logs when capabilities retrieval falls back to the parent version

will you include that now? or is it for 4.19.1?

@DaanHoogland already created the PR. We can include it in 4.19.0