Why do I get feedback when I use headphones in my computer?

I just bought a new computer from iBuyPower.com and it is an amazing computer besides the fact that I get this awful high pitched noise and frequent boops that I get from my headphones when plug them in to the CPU. I used multiple headphones but I get feedback when using any of them. Is there a simple way of fixing this?