I had exactly the same problem - I managed to fix it by downgrading the BIOS from A06 to A04...
Is your Mini running BIOS version A06? (You can find out what version in the BIOS setup).
I received mine 3 days ago and it had BIOS version A06 installed, which kept freezing with the OS X install.
The only other thread on these forums suggest using a USB DVD Drive, which was not viable for me.
Here's what I did:
*** I don't take any responsibility with your BIOS... This worked for me. YMMV.
*** When updating the BIOS, the mini makes a VERY LOUD BEEPING NOISE!
While normal, it can be annoying for people trying to sleep
1) Download the BIOS A04 update from Dell Support (the DOS version):
Drivers and Downloads
2) Create a DOS Environment on a USB Stick using the instructions here:
Creating A DOS Bootable USB Key (All-in-One Package) - Dashken's I-Blog
(the link is to Google's Cache of the webpage as the website appears to be having problems at the moment)
I used the Windows95 DOS files.
3) Copy the BIOS update .exe file to the USB stick
4) Rename the update .exe file to something easier to remember. I suggest "biosupd.exe".
5) Connect the Mini to AC Power, as you don't want to interrupt the BIOS process...
6) Plug in the USB Drive to the Mini, booting from the USB Stick. This will boot you into the DOS environment (looks like Command Prompt)
7) Type (without the quotes) "biosupd.exe", then hit enter. It should say something about not being able to continue because the version you are trying to install is lower than the version currently installed.
8) To force the installation of the A04 BIOS, type (without the quotes) "biosupd.exe /forceit /forcetype", then hit enter.
9) Beeping commences (brings up blue screen showing progress). This took about 2 - 3 mins for me.
10) Once the downgrade of the BIOS has finished, computer will reboot, probably back into the DOS environment if it automatically booted from USB.
11) Remove the DOS USB Drive, reboot the Mini, and check the BIOS to see if it installed the correct BIOS update - it should read A04.
After downgrading the BIOS, my OS X installed perfectly, without any freezes or Kernel Panics.
Please note that THIS WORKED FOR ME.... Your mileage may vary.
I believe that the A06 BIOS doesn't like the OS X install as after downgrading the BIOS to A04, the install worked great.
If you try this, I hope that it all goes smoothly and I'll try to answer any questions you have!