When macOS High Sierra was released around 25th September, I tried to install it in Parallels Desktop Lite. However, both a clean install in a Parallels Virtual Machine (VM), nor an in-place update did not work (the High Sierra VM would refuse to boot). I discovered the High Sierra installer converts the VM file system to Apple File System (APFS). However, Parallels does not support APFS, so here's how to disable the conversion and get High Sierra running in a Parallels.

I couldn't find any info on the Parallel's documentation except that one has to upgrade to Parallels Desktop 13 (paid) instead of Lite (free).

Method 1: Upgrade macOS Sierra VM to High Sierra

Parallels has an automated wizard that creates a macOS Sierra VM with absolutely no hassle. So, the easiest method to get a High Sierra VM is to upgrade from Sierra in-place.

  1. First, you need a running macOS Sierra VM running in Parallels Desktop Lite. See this post for guidance to setup a virtual macOS.

  2. In the VM, download High Sierra from the Mac Store, or follow this URI macappstores://itunes.apple.com/app/id1246284741 The installer will be in Applications (or copy a previously downloaded Install macOS High Sierra.app to Applications in the VM).

  3. Still in the VM, run terminal and enter:

/Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/startosinstall --converttoapfs YES 
  1. This will start the installation process, without converting to APFS!

  2. The VM will reboot and if all goes well, congratulations! You are running High Sierra!

As always, don't blindly follow anything on the Internet - verify everything yourself!

In this case, check what the command does with /Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/startosinstall --usage

Usage: startosinstall

Arguments
--applicationpath, a path to copy of the OS installer application to start the install with.
--license, prints the user license agreement only.
--agreetolicense, agree to license the license you printed with --license.
--rebootdelay, how long to delay the reboot at the end of preparing. This delay is in seconds and has a maximum of 300 (5 minutes).
--pidtosignal, Specify a PID to which to send SIGUSR1 upon completion of the prepare phase. To bypass "rebootdelay" send SIGUSR1 back to startosinstall.
--converttoapfs, specify either YES or NO on if you wish to convert to APFS.
--installpackage, the path of a package to install after the OS installation is complete; this option can be specified multiple times.
--usage, prints this message.

Example: startosinstall --converttoapfs YES 

Method 2: Clean Install of macOS High Sierra VM

If you'd prefer not to upgrade in-place, then here's how to perform a clean install since the Parallels Wizard does not work.

To create a bootable virtual disk with the macOS High Sierra installer, I use the (slightly modified) script from How to create a Bootable ISO image of macOS 10.13 High Sierra installer by Tyler Woods. Without the script, you'll need to copy the installer to a USB or another disk to create the High Sierra boot disk.

  1. As above, download High Sierra from the Mac Store, or follow this URI macappstores://itunes.apple.com/app/id1246284741. The installer will be in Applications.

  2. Now, in terminal, run this, which finally creates a file on the Desktop called "HighSierra.iso". Note that you'll be prompted "If you wish to continue type (Y) then press return" - just do what it says.

cd /tmp
hdiutil create -o HighSierra.cdr -size 5130m -layout SPUD -fs HFS+J
hdiutil attach HighSierra.cdr.dmg -noverify -mountpoint /Volumes/install_build
sudo /Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/createinstallmedia --volume /Volumes/install_build
hdiutil detach /Volumes/Install\ macOS\ High\ Sierra
hdiutil convert HighSierra.cdr.dmg -format UDTO -o HighSierra.iso
rm HighSierra.cdr.dmg
mv HighSierra.iso.cdr ~/Desktop/HighSierra.iso

As above, verify! Note the use of sudo (superuser) and rm (delete a file), so double verify!

Here, /Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/createinstallmedia --usage gives the output below, and note the last line that explains the use of sudo i.e. "This tool must be run as root.":

Usage: createinstallmedia --volume <path to volume to convert>

Arguments
--volume, A path to a volume that can be unmounted and erased to create the install media.
--applicationpath, A path to copy of the OS installer application to create the bootable media from.
--nointeraction, Erase the disk pointed to by volume without prompting for confirmation.

Example: createinstallmedia --volume /Volumes/Untitled

This tool must be run as root.

The kind of output to expect is below:

Terminal.app
~$ cd /tmp
/tmp$ hdiutil create -o HighSierra.cdr -size 5130m -layout SPUD -fs HFS+J
.....................................................................................................
created: /private/tmp/HighSierra.cdr.dmg

/tmp$ hdiutil attach HighSierra.cdr.dmg -noverify -mountpoint /Volumes/install_build
/dev/disk2              Apple_partition_scheme
/dev/disk2s1            Apple_partition_map
/dev/disk2s2            Apple_HFS                       /Volumes/install_build

/tmp$ sudo /Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/createinstallmedia --volume /Volumes/install_build
Password:
Ready to start.
To continue we need to erase the volume at /Volumes/install_build.
If you wish to continue type (Y) then press return: y
Erasing Disk: 0%... 10%... 20%... 30%...100%...
Copying installer files to disk...
Copy complete.
Making disk bootable...
Copying boot files...
Copy complete.
Done.

/tmp$ hdiutil detach /Volumes/Install\ macOS\ High\ Sierra
"disk2" unmounted.
"disk2" ejected.

/tmp$ hdiutil convert HighSierra.cdr.dmg -format UDTO -o HighSierra.iso
Reading Driver Descriptor Map (DDM : 0)…
Reading Apple (Apple_partition_map : 1)…
Reading  (Apple_Free : 2)…
Reading disk image (Apple_HFS : 3)…
..................................................................................................
Elapsed Time: 39.131s
Speed: 131.1Mbytes/sec
Savings: 0.0%
created: /private/tmp/HighSierra.iso.cdr

/tmp$ rm HighSierra.cdr.dmg

/tmp$ mv HighSierra.iso.cdr ~/Desktop/HighSierra.iso
  1. From Parallels, File > New, go ahead and create a new VM via "Install Windows or another OS from a DVD or image file".

Parallels Wizard Install OS from image file

  1. Hit Continue, then "Locate Manually" and when prompted, drag and drop the ISO created in the previous step into the Wizard. Parallels will say "Unable to detect Operating System," since Parallels does not support High Sierra. That's ok, hit Continue and manually specify macOS. I left all the settings as default.

Parallels Wizard Select Operating System

  1. Before starting the VM, Configure it, and most importantly, under Boot Order, check "Select boot device on start up".

Parallels VM Configuration Select Boot Device on Startup

  1. Start the VM and you'll see a prompt "Press any key to enter boot device menu". So, press any key, and in the Boot Manager, boot from image by selecting EFI DVD/CDROM.

Parallels Boot Manager

  1. Now, the installer will run, and you should be greeted with a window to select language. Keep going, and when prompted in the macOS Utilities menu, select Install macOS, then choose to install to the single Macintosh HD (don't worry, this is a virtual hard disk and not your main macOS).

macOS High Sierra Installer Language

  1. Now, the installer will copy a few files and reboot. Again, press any key to get to the Boot Manager, and select EFI DVD/CDROM. This time, from the macOS Utilities screen, select Utilities > Terminal from the menu instead.

macOS Utilities Terminal

  1. In the terminal, edit the file minstallconfig.xml, to change <key>ConvertToAPFS</key><true/> to <key>ConvertToAPFS</key><false/> Here I'm using vi:
vi /Volumes/Macintosh\ HD/macOS\ Install\ Data/minstallconfig.xml

vi is something you'll have to figure out separately. In short, press the arrow keys till the cursor is on the "t" in "true" just under the line <key>ConvertToAPFS</key>. Press c, w, type false, press Esc, then type :wq to save and quit.

Edit minstallconfig.xml in vi

  1. Then from the menu Terminal > Quit Terminal. And you'll be back to the macOS Utilities menu. Now, hit the Apple menu, and Restart.

  2. Again, press any key, but this time, select to boot from Mac OS X. The installer will install, and at some point, state "Installing: About x minutes remaining".

If you've gotten this far, then it's likely you'll get High Sierra running soon! But if you didn't, e.g. because step 9 couldn't boot from Mac OS, then it's you've missed a step, and the installer has gone ahead and converted the file system to APFS...

If this is the case, the VM will not boot to macOS. Instead, after a wait, you'll get a screen like this:

Parallels EFI Shell Boot Error

Conclusion

For a simple method to get High Sierra in Parallels, go with Method 1. However, it takes longer to upgrade, and requires more disk space.

For a clean install, try Method 2 which assumes a higher level of expertise. Good luck!