SOF FW 1.9 ABI vs kernel 5.14.17 ABI mismatch

[joakim-tjernlund]

dmesg reports:

[    4.217351] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware info: version 1:9:0-fa857
[    4.217353] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware: ABI 3:19:0 Kernel ABI 3:18:0
[    4.217354] sof-audio-pci-intel-tgl 0000:00:1f.3: warn: FW ABI is more recent than kernel
[    4.217356] sof-audio-pci-intel-tgl 0000:00:1f.3: unknown sof_ext_man header type 3 size 0x30
[    4.312194] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware info: version 1:9:0-fa857
[    4.312197] sof-audio-pci-intel-tgl 0000:00:1f.3: Firmware: ABI 3:19:0 Kernel ABI 3:18:0
[    4.312198] sof-audio-pci-intel-tgl 0000:00:1f.3: warn: FW ABI is more recent than kernel
[    4.314783] sof-audio-pci-intel-tgl 0000:00:1f.3: Topology: ABI 3:19:0 Kernel ABI 3:18:0
[    4.314785] sof-audio-pci-intel-tgl 0000:00:1f.3: warn: topology ABI is more recent than kernel

Should not kernel 5.14.x have matching ABI with SOF 1.9 ?

[lgirdwood]

This is allowed as the FW has to work with multiple kernel versions, but its possible this warn level should be an info as it's not fatal. @plbossart @marc-hb IIRC, was there a recent discussion on updating some of the kernel messages to different levels ?

[joakim-tjernlund]

Right, it "works" but maybe there will be some artifacs ?
Anyhow, a recent kernel should have matching ABI, what is the point of have a new ABI if no one is using it?

[plbossart]

@joakim-tjernlund 5.14.x is already "old" compared to SOF 1.9. Due to the pipeline effect of Linux kernel integration, we upstreamed the patches for 5.14 before 5.13 was released by Linus Torvalds, so June 2021. That means that by construction some of the newer features of the SOF 1.9 release will not be enabled, but the SOF firmware operates in backwards-compatible mode.

We could argue and repaint the bikeshed on whether this is really a warning. Maybe info is more relevant indeed.

[joakim-tjernlund]

looks like even 5.15 is still on the old ABI ?

include/uapi/sound/sof/abi.h:#define SOF_ABI_MAJOR 3
include/uapi/sound/sof/abi.h:#define SOF_ABI_MINOR 18

[marc-hb]

@plbossart @marc-hb IIRC, was there a recent discussion on updating some of the kernel messages to different levels ?

I can think of these two:

• DOC: what version we're supposed to use (kernel-compat) sof-bin#9
• Ignoring the PATCH level on the user side: Warning emitted if FW revision is newer than what kernel knows about linux#2647

looks like even 5.15 is still on the old (3.18) ABI ?

Even the latest topic/sov-dev branch is still on 3.18, AFAICT there hasn't been any 3.19 bump on the ABI user side anywhere yet: https://github.com/thesofproject/linux/blob/66bd4e2aef93dd/include/uapi/sound/sof/abi.h

We could argue and repaint the bikeshed on whether this is really a warning. Maybe info is more relevant indeed.

If it's officially supported (in a semver-defined way) and totally harmless then there should be no warning.

[joakim-tjernlund]

looks like even 5.15 is still on the old (3.18) ABI ?

Even the latest topic/sov-dev branch is still on 3.18, AFAICT there hasn't been any 3.19 bump on the ABI user side anywhere yet: https://github.com/thesofproject/linux/blob/66bd4e2aef93dd/include/uapi/sound/sof/abi.h

Why not? Seems strange to define a new ABI if no one is using it.

[marc-hb]

On the SOF side, 3.19 added SOF_IPC_INFO_D3_PERSISTENT (commit f79bbb2) while 3.20 added SOF_DAI_CONFIG_FLAGS_PAUSE and related (commit f6c4853). Github links commits to the corresponding code reviews.

[plbossart]

looks like even 5.15 is still on the old (3.18) ABI ?

Even the latest topic/sov-dev branch is still on 3.18, AFAICT there hasn't been any 3.19 bump on the ABI user side anywhere yet: https://github.com/thesofproject/linux/blob/66bd4e2aef93dd/include/uapi/sound/sof/abi.h

Why not? Seems strange to define a new ABI if no one is using it.

it's not a 'new' ABI, only minor variations to enable new capabilities in backwards-compatible ways. There's no way we can synchronize release of kernels and firmware, and it's even worse when you take into account downstream, so these new capabilities will be used when the kernel is updated.

[marc-hb]

@plbossart are you basically saying that SOF 3.18 is the minimum firmware required by the current kernel, however recent kernel versions will dynamically and already use 3.19 and 3.20 SOF features when they find them available?

The docs don't seem to mention this https://thesofproject.github.io/latest/contribute/process/abiprocess.html

[plbossart]

no, we've tested that our development kernel also works with SOF releases 1.6 and 1.7. We haven't tested earlier SOF versions but they should also work when the kernel is updated ("we don't break userspace" (tm))

I don't know what 'current' kernel means, it depends on whom you ask.

[marc-hb]

We haven't tested earlier SOF versions but they should also work when the kernel is updated ("we don't break userspace" (tm))

So what do the numbers 3.18 really mean when used on the kernel side?

I don't know what 'current' kernel means, it depends on whom you ask.

I meant any kernel new enough to use 3.19 and 3.20 features while still being marked 3.18

[plbossart]

I have no idea what you are asking @marc-hb, please see https://thesofproject.github.io/latest/contribute/linux/development_tree.html#id3

[marc-hb]

Unlike what our ABI versioning may sometimes look like, semver.org is asymmetric: it defines upgrade rules for one side only and one-way backward compatibility only. Example: semver guarantees that upgrading libfoo 3.14 to libfoo 3.15 works fine when the user needs libfoo 3.12 features. Semver says nothing about upgrading the other, "user" side and does not even care whether there is a version number on the user side.

Semver has a very clear asymmetry because it assumes one side offers a (growing) number of features while the user side is merely using these features. For simplicity I'm calling these two sides the semver library side and the semver user side because that's the most common case but don't take these terms too literally. If/when there is a MINOR number on the user side, then it's the minimum library version required (what else?) and a combination is valid when the MINOR is lower on the user side. Otherwise it does not work because the user side misses some features.

For semver, backwards compatibility refers to a newer library looking "backwards" at a non-moving user. It does not say anything about the other way round. It would not make sense to re-use the same MINOR numbers for backwards compatibility in the other direction because you can't have MINOR_A > MINOR_B and MINOR_B > MINOR_A at the same time.

Yet: thesofproject/sof-bin#9 (comment)

... and MINOR changes allow for compatibility which means you can run a newer firmware under an older kernel or viceversa just fine

... sounds like a "symmetrical abuse" of semver and I'm afraid it's not the only example of symmetrical abuse.

Thanks for the link to https://thesofproject.github.io/latest/contribute/linux/development_tree.html#id3. There:

users may update their kernels at any time while keeping the SOF firmware binary and topology files stored in the root filesystem unchanged. [...] conversely, [...] a new firmware does not need to include any backwards-compatibility code to interface with an older kernel.

So from a semver perspective, this all sounds like the "easily upgraded library" side is the kernel side while the semver user side is the firmware side, correct? This seems confirmed by the MINOR warning check in snd_sof_ipc_valid() which is asymmetrical, example:

Firmware: ABI 3:19:0 Kernel ABI 3:18:0
warn: FW ABI is more recent than kernel

... but no warning the other way round.

SOF release notes also frequently include a "minimum Linux kernel X.Y or newer" requirement.

But ! Most recent SOF release notes require a minimum kernel version with a LOWER ABI version?! In other words, most recent SOF release notes recommend an ABI warning.

So if it's not actually used for semver backwards compatibility, then what is the actual purpose of the ABI MINOR?

cc: @kv2019i

[kv2019i]

I agree @marc-hb with most of the above and we should drop the warning. A few specific points

So if it's not actually used for semver backwards compatibility, then what is the actual purpose of the ABI MINOR?

We have many cases where kernel conditionally uses a new ABI if and only if FW is new enough. These checks can be done because we have ABI MINOR bumps. With oldkernel+newfw and newkernel+oldfw, the new ABI is not used and things still work.

But ! Most recent SOF release notes require a minimum kernel version with a LOWER ABI version?! In other words, most recent SOF release notes recommend an ABI warning.

The release notes document the version the FW release has been tested with. Maybe we should not use "required" there.