The current J-Link upload code always uploads the .bin file at offset 0 (if no offset is specified in upload.offset_address). The .elf file contains the correct program offset and JLink flashes the code to the correct location when .elf file is passed in instead of the current default .bin file.
I noticed a similar fix in platform-stm32. This should probably be the default anytime JLink is the upload method