Problems After Installing Ubuntu
If your newly installed system will not boot without the usb plugged
in, then grub has probably been installed to the usb. From your booted
Ubuntu system, open a terminal, then use the following commands to
install grub to the internal drive :
sudo grub-install /dev/sdX
Replacing sdX with the actual drive, which will probably be sda, but use disk utility to double check if you are unsure.
Problems After Installing Ubuntu, Windows or other Distro's
If you install windows after you have installed Ubuntu or any other Linux distro, windows will overwrite the master boot record.
means grub will be lost and you will not be able to access or
boot Ubuntu or any other Linux distribution you have installed.
You will now have to reinstall grub to the master boot record, the simplest way of doing this is to use boot repair.
Boot repair is an automatic boot repair tool with a simple graphical interface.
We have not written a guide yet, but the following links should explain everything.
Boot Repair - community documentation.
Recovering Ubuntu after installing windows - community documentation.