No luck on either Ubuntu or Tiny XP when trying to boot from the USB. I'm somewhat convinced this is a weird software issue, though, even if my little experiment with different USB keys and software failed.
The weird thing is that when in Windows XP, my old phone can connect through the USB ports. It's the first thing I even connected to the mini, and after I did that, it updated itself (XP did), and nothing USB based worked again. I have read somewhere, though I have no idea where, that someone else had a similar problem, where only devices that were connected before a certain time would still work, and anything else would never be recognized. Is there any way I can get rid of this little quirk?
In the meantime, I'll try installing Ubuntu from inside Windows and see how that goes. Maybe my problems will end up solved after all.