In the relatively short run, of course, one can square the babies' circle by just piling up debt. That's the Reagan/Bush strategy: cut taxes while expanding government, run huge deficits, and watch the debt pile up. That strategy can't work forever. Eventually things have to be brought back into balance, and some mixture of shrinking the core/popular state (i.e., entitlements+defense) and raising taxes must occur. Whining about debt/deficits can be useful insofar as it calls attention to this necessary end-game. But most deficit whining (shading into sanctimony) is more ambitious: (i) it claims to be in favor of low taxes but without saying in any level of detail how core/popular govt functions are to be seriously shrunk to make that possible; (ii) it claims to be opposed to any additional/emergency government debt even when private aggregate demand in the economy has collapsed (perhaps with a 'balanced budget amendment' to cement the point). Point (i) reprises 'free lunch' childishness, and point (ii) is the sort of lunacy that turns recessions into great depressions.
Let's expand on point (ii). It's one thing to be a deficit hawk over the whole business cycle, it's quite another to be a year-by-year deficit hawk. A 'whole cycler' gathers more taxes than she needs/runs surpluses/reduces debt during the good times (thereby reducing aggregate demand at peaks and helping to deflate economic bubbles), and has the option of using (a mixture of) deficit-creating public spending and deficit-creating tax cuts to boost aggregate demand in the economy during bad times. A year-by-year (YbY) deficit hawk, by way of contrast foregoes all automatic, public sector stabilization (whether through spending or tax cuts) of the private sector's flux, thereby increasing the risks of bubble booms and depression busts like those that plagued the US economy from the Civil War until WW2. YbY deficit hawks - true 'know nothings' really, who are prepared to endorse society-shredding, revolution-promoting capitalist phenomena that would warm the hearts of Marx and Lenin as 'the system working' - are, of course, completely ascendant in current US debates. And that's to say, in the real world, sanctimony about deficits tends to be a continuation of rather than an antidote to ultra-irritating 'anti-tax' or 'low tax' posturing and remonstrance.
And see this story for a reminder of how internally divided and difficult to govern the US really is: Republicans captured both the Federal House and Senate in 1994, in part because of backlash in rural and southern areas against a ban on assault weapons (which Bill Clinton had campaigned on, with the vigorous support of police associations across the US) that was part of a big 'new cops on the street' Crime Bill that Democrats passed that year. One might have thought that conservatives would have found plenty to cheer in that Bill even if they themselves didn't see much of an issue with assault weapons. This was a centrist, right-ish, 'law and order' Bill both in spirit and in many, though not all of its details. But no: God forbid that you do exactly what you said you were going to do, and end private ownership of machine-guns. For that, it's 'throw the bums out'/'now you've gone too far'/'final straw that broke the camel's back' time. Of course, lots else was going on in 1994, including a general 'time for a change' pall over Congress, and Gingrich's well-played nationalization of (traditionally highly local) House and Senate races, i.e., under the 'Contract for America' banner. But the story I just sketched is an essential and depressing part of what transpired.