It's a little more complicated than that, and it is a penalty. You may also owe interest, and they are two separate things (and interest is statutory, so you're really not negotiating there). There are multiple penalties for underpayment and nonpayment of federal taxes. If you're an employee and not withholding enough (because you have other income or AMT for example), you should be making estimated payments to cover the difference. Your employer is also subject to penalties if it fails to withhold and pay the gov't employment taxes (social security and medicare), and it's usually 100%. If you are self-employed, you need to make estimated payments.

tl;dr: the gov't has cashflow problems, too, and doesn't like to issue t-bonds to finance its receivables.