EFI stub booting fails with ld.lld: __efistub__end miscalculated

[kees]

Building latest LLVM and v5.3-rc1, I'm seeing:

ld.lld: error: /home/kees/src/linux/drivers/firmware/efi/libstub/arm64-stub.c:134:(.init.text+0x70): relocation R_AARCH64_ADR_PREL_PG_HI21 out of range: -70505092558848 is not in [-4294967296, 4294967295]

line 134 is:
*reserve_size = round_up(kernel_memsize, EFI_ALLOC_ALIGN);

This is only visible if I disable CONFIG_ARM64_ERRATUM_843419. If I leave this enabled, the EFI stub fails to boot the kernel, with handle_kernel_image() failing...

[kees]

Just for fun, if I try to remove references to _text, _edata, and _end:

    kernel_size = 1; //_edata - _text;
    kernel_memsize = 2; //kernel_size + (_end - _edata);

I get a different error:

kallsyms failure: relative symbol value 0xffff000001c36000 out of range in relative mode

[kees]

I think this is all due to a negative calculation about kernel_memsize (and likely the real bug). With ld.bfd:

ffff0000110f91e4: f0ff7c39 adrp x25, ffff000010080000 <__efistub__text>
ffff0000110f91e8: f90002f8 str x24, [x23]
ffff0000110f91ec: f90002b8 str x24, [x21]
ffff0000110f91f0: 91000339 add x25, x25, #0x0
ffff0000110f91f4: 90007138 adrp x24, ffff000011f1d000 <__efistub__end>

with ld.lld __efistub__end is very wrong (off by 0x10000000):

ffff0000111191f4: b0f86878 adrp x24, ffff000001e26000 <__efistub__end>

Literally adding 0x10000000 to kernel_memsize appears to fix it (?!)

[nickdesaulniers]

cc @smithp35

[kees]

FYI, to reproduce this under qemu, just add the following drive arguments, after copying AAVMF_VARS.fd to the local directory from /usr/share/AAVMF/:

        -drive if=pflash,format=raw,readonly,file=/usr/share/AAVMF/AAVMF_CODE.fd \
        -drive if=pflash,format=raw,file=AAVMF_VARS.fd \

If I leave these off, the kernels boot fine, so something with the EFI stub's symbol aliases and objcopy pass break the _end location.

[kees]

Trouble appears to be with processing the linker script, perhaps due to _end being present in math leading up to its definition:

 . = ALIGN((1 << 12));
 init_pg_dir = .;
 . += ((1 << 12) * ( 1 + (((((_end)) >> (((12 - 3) * (4 - (4 - 4)) + 3))) - (((((((((((0xffffffffffffffff)) - (((1)) << (48)) + 1) + (0)) + (0x08000000))) + (0x08000000))) + 0x00080000)) >> (((12 - 3) * (4 - (4 - 4)) + 3))) + 1 + (1))) + (0) + (((((_end)) >> (((12 - 3) * (4 - (1)) + 3))) - (((((((((((0xffffffffffffffff)) - (((1)) << (48)) + 1) + (0)) + (0x08000000))) + (0x08000000))) + 0x00080000)) >> (((12 - 3) * (4 - (1)) + 3))) + 1 + (1)))));
 init_pg_end = .;

 __pecoff_data_size = ABSOLUTE(. - __initdata_begin);
 _end = .;

This math works out to 0x5000 in both bfd and lld (measured by adding __init_pg_size = ABSOLUTE(. - init_pg_dir);) However, lld for some reason breaks __efistub__end afterwards:

$ ld.lld -EL -maarch64elf --no-undefined -X -shared -Bsymbolic -z notext -z norelro --no-apply-dynamic-relocs --fix-cortex-a53-843419 --build-id -o vmlinux.lld -T poc.lds --whole-archive vmlinux.o && readelf -Ws vmlinux.lld | egrep '\b(__init_pg_size|(__efistub_|)_end)\b'
368272: ffff000002218000     0 NOTYPE  LOCAL  HIDDEN    38 __efistub__end
368322: ffff000012318000     0 NOTYPE  GLOBAL DEFAULT   38 _end
368822: 0000000000005000     0 NOTYPE  GLOBAL DEFAULT  ABS __init_pg_size

bfd doesn't:

$ aarch64-linux-gnu-ld.bfd -EL -maarch64elf --no-undefined -X -shared -Bsymbolic -z notext -z norelro --no-apply-dynamic-relocs --fix-cortex-a53-843419 --build-id -o vmlinux.bfd -T poc.lds --whole-archive vmlinux.o && readelf -Ws vmlinux.bfd | egrep '\b(__init_pg_size|(__efistub_|)_end)\b'
338124: ffff000012318000     0 NOTYPE  LOCAL  DEFAULT  ABS __efistub__end
383812: ffff000012318000     0 NOTYPE  GLOBAL DEFAULT 15325 _end
383825: 0000000000005000     0 NOTYPE  GLOBAL DEFAULT  ABS __init_pg_size

[kees]

Ugh. It's evaluation ordering. ld.lld treats an assignment as an immediate assignment instead of as an alias:

poc.lds:

__efistub__end = _end;

SECTIONS
{
 . = 0xffff;
 . += ((_end >> 8) - 0xff );

 _end = .;
}

$ ld.lld -maarch64elf -shared -o vmlinux.lld -T poc.lds  /tmp/test.o && readelf -Ws vmlinux.lld | egrep '\b(__init_pg_size|(__efistub_|)_end)\b'
     2: 000000000000ffff     0 NOTYPE  GLOBAL DEFAULT    1 _end
     3: 000000000000ff00     0 NOTYPE  GLOBAL DEFAULT    1 __efistub__end

but if __efistub__end = _end; is moved after SECTIONS:

$ ld.lld -maarch64elf -shared -o vmlinux.lld -T poc.lds  /tmp/test.o && readelf -Ws vmlinux.lld | egrep '\b(__init_pg_size|(__efistub_|)_end)\b'
     2: 000000000000ffff     0 NOTYPE  GLOBAL DEFAULT    1 _end
     3: 000000000000ffff     0 NOTYPE  GLOBAL DEFAULT    1 __efistub__end

[nathanchance]

Is this potentially related to #561?

[nickdesaulniers]

cc @MaskRay @ardbiesheuvel @GeorgiiR @rui314 @smithp35

Thanks for the reproducer, @kees !

[kees]

https://llvm.org/pr42990

[kees]

Is this potentially related to #561?

When I revert the kernel work-around for #561 but move all the variables assignments to after the SECTIONS, I appear to get the correct value for __efistub_stext_offset! I thought I tested this when checking for bug dups, but I guess I did it wrong initially. :)