DMG creation fails on macOS Big Sur 11.0.1 on M1 hardware (aarch64)

[AstroPixelProcessor]

I'm submitting a…

• [ X ] bug report

Short description of the issue/suggestion:

DMG creation is not possible on aarch64 hardware. It fails on the line

execute("hdiutil", "create", "-srcfolder", appFolder, "-volname", volumeName, "-ov", "-fs", "HFS+", "-format", "UDRW", tempDmgFile);

in GenerateDMG.java.

I have confirmed that this does work on x86_64 hardware Big Sur 11.0.1, so it seems it is really specific to the aarch64 version of hdiutil.

The error message is :

[INFO] Executing command: /bin/sh -c cd '/Users/mabula/IdeaProjects/astropixelprocessor-jdk17/.' && 'hdiutil' create -srcfolder /Users/mabula/IdeaProjects/astropixelprocessor-jdk17/target/astropixelprocessor -volname astropixelprocessor-1.1.0 -ov -fs HFS+ -format UDRW /Users/mabula/IdeaProjects/astropixelprocessor-jdk17/target/assets/astropixelprocessor_1.1.0.dmg
[INFO] could not access /Volumes/astropixelprocessor-1.1.0/astropixelprocessor.app/Contents/PlugIns/jre/Contents/Home/.BC.T_xQRv4X - Operation not permitted
[ERROR] hdiutil: create failed - Operation not permitted
[ERROR] DMG image generation failed due to: Command execution failed: hdiutil create -srcfolder /Users/mabula/IdeaProjects/astropixelprocessor-jdk17/target/astropixelprocessor -volname astropixelprocessor-1.1.0 -ov -fs HFS+ -format UDRW /Users/mabula/IdeaProjects/astropixelprocessor-jdk17/target/assets/astropixelprocessor_1.1.0.dmg
[ERROR]
org.codehaus.plexus.util.cli.CommandLineException: Command execution failed: hdiutil create -srcfolder /Users/mabula/IdeaProjects/astropixelprocessor-jdk17/target/astropixelprocessor -volname astropixelprocessor-1.1.0 -ov -fs HFS+ -format UDRW /Users/mabula/IdeaProjects/astropixelprocessor-jdk17/target/assets/astropixelprocessor_1.1.0.dmg
at io.github.fvarrui.javapackager.utils.CommandUtils.execute (CommandUtils.java:19)
at io.github.fvarrui.javapackager.utils.CommandUtils.execute (CommandUtils.java:29)
at io.github.fvarrui.javapackager.packagers.GenerateDmg.doApply (GenerateDmg.java:88)
at io.github.fvarrui.javapackager.packagers.GenerateDmg.doApply (GenerateDmg.java:22)

What is the expected behavior?

DMG should be created

What is the current behavior?

DMG creation fails

Suggestion with tested solution:

Do a check on the hardware architecture and if aarch64/arm then build the DMG with the newer APFS filesystem.

execute("hdiutil", "create", "-srcfolder", appFolder, "-volname", volumeName, "-ov", "-fs", "APFS", "-format", "UDRW", tempDmgFile);

The bless command fails then, but you can simply disable the bless command, since the DMG is properly created with APFS filesystem in that case.

// no longer needed
//Logger.info("Blessing ...");
//execute("bless", "--folder", mountFolder, "--openfolder", mountFolder);

Google searches seem to indicate that hdiutil on aarch64 actually is broken for HFS+ somehow... or perhaps Apple is forcing us to no longer use it since APFS is better ? I have no clue at the moment what is happening, I only know that hdiutil with HFS+ is a problem right now on aarch64, many problems are encountered with google search on this.)

Please tell us about your environment:

• JavaPackager version current 1.6.6-devel branch:
• OS version Big Sur 11.0.1:
• JDK version openJDK 17:
• Build tool:
  • [ X ] Maven
  • Gradle

[AstroPixelProcessor]

I will make a merge request so you can see my suggestion to fix this issue on macOS aarch64

[fvarrui]

Hi @AstroPixelProcessor!
Thank you for reporting this issue and for proposing a solution. Please, could you make your PR to devel branch? And is your proposal backward compatible? or would it work from a certain version of Mac OS?

[AstroPixelProcessor]

Hi @fvarrui , my proposal will not work on quite a bit older MacOS versions, I think older than 10.13. Therefore I think it is safe to only do this on aarch64 architecture where macOS that old does not exist.

I will make a push request ;-)

[AstroPixelProcessor]

Having issues to push somehow, this is how it will work @fvarrui 👍

[AstroPixelProcessor]

// creates image
Logger.info("Creating image: " + tempDmgFile.getAbsolutePath());

	String osArchitecture = System.getProperty("os.arch");
	if (osArchitecture.toLowerCase().equals("aarch64")) {

		execute("hdiutil", "create", "-srcfolder", appFolder, "-volname", volumeName, "-ov", "-fs", "APFS", "-format", "UDRW", tempDmgFile);

	} else {

		execute("hdiutil", "create", "-srcfolder", appFolder, "-volname", volumeName, "-ov", "-fs", "HFS+", "-format", "UDRW", tempDmgFile);

	}

	if (mountFolder.exists()) {
		Logger.info("Unmounting volume: " + mountFolder);
		execute("hdiutil", "detach", mountFolder);
	}
	
	// mounts image
	Logger.info("Mounting image: " + tempDmgFile.getAbsolutePath());
	String result = execute("hdiutil", "attach", "-readwrite", "-noverify", "-noautoopen", tempDmgFile);
	String deviceName = Arrays.asList(result.split("\n"))
								.stream()
								.filter(s -> s.contains(mountFolder.getAbsolutePath()))
								.map(s -> StringUtils.normalizeSpace(s))
								.map(s -> s.split(" ")[0])
								.findFirst().get();
	Logger.info("- Device name: " + deviceName);
	
	// pause to prevent occasional "Can't get disk" (-1728) issues 
	// https://github.com/seltzered/create-dmg/commit/5fe7802917bb85b40c0630b026d33e421db914ea
	ThreadUtils.sleep(2000L);

	// creates a symlink to Applications folder
	Logger.info("Creating Applications link");
	File targetFolder = new File("/Applications");
	File linkFile = new File(mountFolder, "Applications");
	FileUtils.createSymlink(linkFile, targetFolder);

	// renders applescript 
	Logger.info("Rendering DMG customization applescript ... ");
	File applescriptFile = new File(assetsFolder, "customize-dmg.applescript");
	VelocityUtils.render("/mac/customize-dmg.applescript.vtl", applescriptFile, packager);
	Logger.info("Applescript rendered in " + applescriptFile.getAbsolutePath() + "!");
	
	// runs applescript 
	Logger.info("Running applescript");
	execute("/usr/bin/osascript", applescriptFile, volumeName);

	// makes sure it's not world writeable and user readable
	Logger.info("Fixing permissions...");
	execute("chmod", "-Rf", "u+r,go-w", mountFolder);

	if (!osArchitecture.toLowerCase().equals("aarch64")) {
		// makes the top window open itself on mount:
		Logger.info("Blessing ...");
		execute("bless", "--folder", mountFolder, "--openfolder", mountFolder);
	}

	// tells the volume that it has a special file attribute
	execute("SetFile", "-a", "C", mountFolder);

[fvarrui]

Ah, ok, great!! nice solution!!! ... so, it depends on the architecture instead on the OS version. Thanks! I think this will keep backward compatibility

[AstroPixelProcessor]

Yes it should ;-)

[fvarrui]

Don't worry about where you push your PR ... I can rebase it, sorry for being so picky 😅

[AstroPixelProcessor]

@fvarrui Pull Request is created :-)

[fvarrui]

Branch issue-184 merged into devel.

[fvarrui]

JavaPackager v1.6.7 released to Maven Central. See changes here.