This is an edited version of an article that first appeared in the Times Literary Supplement in the 4th July issue. I’ve cut it a little to focus on the question in the headline. Many thanks to the TLS for allowing me to reproduce the article for subscribers.

The HS2 building site at Old Oak Common

Despite our polarized times, there is one thing everyone agrees on: government isn’t working very well.

For the modern right, this is axiomatic. They believe that government is incapable of working as effectively as the private sector because it lacks the right incentives and has been captured by the bureaucratic class of metropolitan graduates that is responsible for all our woes.

This certainty that the state is a force for bad is impermeable to evidence or reason. One can note that much of the modern technology that supplies the profits of our biggest companies – from the internet to touchscreens – was developed thanks to government-funded research. Or that state health initiatives have saved billions of lives over the past century. Or one can point to the litany of private sector failures when services have been outsourced to them.

It won’t make any difference. Even personal experience to the contrary can be subject to cognitive dissonance – recall the famous sign held aloft by a Tea Party protestor during the Barack Obama years, “Keep your government hands off our Medicare”. Indeed, the modern radical right is happy to use state apparatus in a highly authoritarian fashion when it suits it to do so, while maintaining that it is incapable of improving society more broadly.

Liberals tend to be more divided on the subject. They agree government should be a force for good and that it has worked in the past, and that it even works now from time to time – the Covid death toll would have been far higher without the rapid funding and distribution of vaccines. But there is a growing lack of confidence, on both sides of the Atlantic, in government’s ability to provide effective and efficient public services and to support economic growth. It is one reason why the Democrats, Labour and other centre-left parties around the world are struggling to find a story to tell the electorate. They are the parties of big government, and if they don’t believe in it either, what’s left?

For many liberals, all the blame lies with the right and their media cheerleaders, who have attacked helpless public servants relentlessly as lazy pen-pushers while deploying policies that undermine the state. Until recently, this conservative degradation of state capacity was achieved incrementally via funding cuts, privatization and targeted deregulation. While there was a desire for the state to do less, backed by a political narrative typified by Ronald Reagan’s “nine scariest words in the English language”, there was an acceptance among this cohort that there were some things government had to do, and an intention that it should do them effectively.

As the radical right has become more prominent, however, overwhelming the more cautious centre right, a new philosophy has emerged. This is about destroying government, not shrinking it by reducing state capacity, often in a deliberately chaotic way, rather than targeted reform of rules and regulations or cuts to spending seen as unncessary. I’ve written elsewhere about this determination to eliminate the bureaucratic layer of society seen by the new right as responsible for the world’s problems, as is most obviously demonstrated by the Trump regime.

But two new books, Abundance by Ezra Klein and Derek Thompson, and Why Nothing Works by Marc J. Dunkelman, argue that liberals are at fault too. The authors are themselves liberals and concur entirely with the idea that the right in the US has no interest in good governance, and has become a destructive force under Trump. But these books are targeted at their own side, arguing that the centre left has become addicted to procedure and regulation, which has kludged up the machinery of the state and created an easy target for the destructive forces of Musk and co. If government could free itself and return to its New Deal muscularity, then a new age of cheap housing, energy and technology could be ushered in.

Abundance is written for a wide audience, presenting its argument with simple, and sometimes simplistic, clarity. In Why Nothing Works, Dunkelman spends more time on the history of the American progressive movement, setting out how it has always been animated by the tension between two competing drives. He uses the well-worn binary of the statist Alexander Hamilton, who wanted to use government to transform society, and Thomas Jefferson, who wanted to protect the people from the misuse of state power, as the avatars for this tension. Since the arrival of the new left in the 1960s, he argues, the Jeffersonians have taken control, becoming adept at taking regulatory tools passed by Congress and building them up through judicial review to induce stasis.

Both books use environmental laws as an example of how regulatory powers can spiral well beyond the intention of legislatures. Dunkelman explains how Richard Nixon’s National Environmental Policy Act was passed through Congress with few objections, with the expectation that it would simply require agencies to consider the environmental impact of projects via an impact statement. It was supposed to be a cheap nudge. But advocacy organizations quickly realized that they could challenge these impact assessments in court, creating a powerful new lever to stop developments. Within a few years, agencies responsible for building national infrastructure, such as the Federal Highway Administration, were overwhelmed with costly legal challenges.

A grim irony of this shift is that it has made it harder to build the renewable energy sources required to limit the damage of climate change. Dunkelman gives the example of a transmission line that would allow Massachusetts to use clean hydropower from Canada, and the years-long battle that has ensued to get permission to build it through neighbouring states. An odd alliance of preservation societies, fossil fuel companies and opportunistic politicians has used every trick available, including the courts and a referendum, to stop development. It remains unfinished. Thompson and Klein use the example of California’s ill-fated high-speed rail project as a comparable case study.

A similar story can be told about the UK, where the rapid growth of the regulatory state has led to an increasing burden on developers to produce reams of impact assessments, along with more widespread use of judicial review to delay or block projects. Commonly cited examples include the Lower Thames Crossing, which was designated a national priority fourteen years ago, but still hasn’t started, and for which 360,000 pages of planning documentation have now been produced, at a cost of almost £270 million. There is also the fact that the UK hasn’t managed to build any reservoirs in the past thirty-three years.

On both sides of the Atlantic, housing has become the most bitter battleground in the internecine liberal wars between self-proclaimed YIMBYs, who favour deregulation to improve the lives of younger people who are struggling to afford somewhere to live, and those who see such a move as a capitalist plot to enrich developers smuggled in under progressive clothing.

Klein and Thompson make a good case for the moral high ground, contrasting California, which has made it steadily harder to build housing, despite booming populations – especially around Silicon Valley – and Texas. They note that California is home to 12 per cent of the US population and 50 per cent of the country’s unsheltered homeless people, which correlates directly with housing shortages. That this most Democratic of states has refused to pass legislation to alleviate the problem does show up a certain moral hypocrisy.

That said, there are two glaring problems with the “abundance” argument that neither book even attempts to resolve. The first concerns trade-offs. The whole thesis of both books is that progressives have refused to accept the negative costs of a regulatory regime that protects them from bad actors. Yet neither one sets out where the trade-offs should lie. Both accept that some of this regulation was necessary to protect the quality of air and water, and to improve safety in all sorts of industries. Dunkelman notes the regulatory state was created in part as a response to the destructive behaviour of high-handed administrators such as Robert Moses, who bulldozed through communities without consultation while building a series of gargantuan highways and bridges across New York.

So what is the right amount of regulation, and why? It is easy to find examples of infrastructure projects held up by seemingly ridiculous concerns about newts or bats, or to identify particularly selfish groups of NIMBYs, determined to preserve their property values by blocking a children’s home or affordable housing. At the same time, we don’t want a world in which local residents have no say at all, or developers can ignore any habitat or historic building. YIMBYs often point to the huge amount of infrastructure being built in China, contrasting this with the UK’s or California’s meagre efforts, but would they really want to live in China?

Klein and Thompson make some fair criticisms of “everything bagel” policy-making: the idea that liberals try to achieve wider social policy goals around equality and redistribution via procurement and contracting. Insisting that certain proportions of jobs go to women or to minorities, or that only small businesses can win certain contracts, can undermine the core purpose of procurement. But where should one draw the line? Not taking wider social purpose into account can also be distorting if systems are designed in the interests of big corporations that know how to play the contracts game.

There is also little engagement with the failures of deregulation over the past few decades. At a national level, the American right has been at war with the regulatory state for some time, and has successfully undermined many of the tools created during the 1960s and 1970s. Increasingly conservative and “originalist” Supreme Courts have rolled back many of the powers created by a previous generation of judges. In some cases, this may have helped infrastructure to get built, but at the cost of rising inequality and health scandals. In the UK, the Grenfell scandal was partly the result of inadequate building regulation, following lobbying from industry not to tighten restrictions. The “abundance” authors would no doubt counter that their version of deregulation would be more thoughtful and discriminating. But how?

Many on the left have attacked the abundance agenda as little more than neoliberalism in fancy packaging. Some of this is just tribal infighting that ignores the genuine iisues authors such as Klein and Thompson raise. But they do have a point when they observe that there is little attempt to reckon with the legacy of previous deregulatory efforts. What the “abundance” thesis needs desperately is a theory of regulation that explains how you avoid the daft rigidities of the current system, which stop obviously beneficial and widely supported projects from going ahead, while maintaining the ability to stop bad actors.

Dunkelman does acknowledge this by arguing that deregulation is a matter of creating a more powerful state, capable of acting in a Hamiltonian way, rather than of simply neutering government. For example, in a brief discussion on US welfare regulation, he notes that the old model of highly autonomous officials involving themselves in the lives of welfare recipients could lead to imperious behaviour based on prejudice. But this model also made the service far more responsive than it is now, when everything is fixed in rules and regulations, and officials are powerless to use common sense to identify the right support. Again, the author doesn’t really attempt to square the circle by explaining how to create a balance between the two models.

Bringing autonomy into regulatory decision-making is incredibly tricky in large, complex states with judicial review because it requires giving regulators powers, then allowing them to use those powers selectively, which in turn creates grounds for challenge due to subjectivity. It is also the case that, in a world filled increasingly with capricious demagogues seeking to take over liberal democracies, stronger rules seem like an important defence. Yet the quest to create tightly defined and objective rules that make sense in all circumstances is doomed to failure because legislators cannot anticipate all eventualities in advance.

While this is a problem, to some degree, in every rich democracy, it seems to be particularly marked in Anglo-Saxon countries, which have historically had more adversarial relationships between capital and labour, and have emphasized individual freedom over social responsibilities. Klein and Thompson note that European countries seem to do a better job (on average) with infrastructure and housing, but do not stop to ask why. Presumably this is because to do so would require opening up a much bigger set of questions about the political economy of the US. It would also require questioning what exactly “abundance” means. Europeans are less litigious and more able to co-operate when it comes to compromising between different interests, but their national economies have grown far less quickly – both in general and certainly since the 2008 financial crisis – than that of the US. These things are not unconnected. Likewise, Japan is much better at building infrastructure quickly, but that is due in part to a culture of reciprocity that is deeply embedded in politics and employment practices as well (and may have contributed to weaker growth there). You can’t have one without the other.

Without considering these wider questions, it is hard to see how the politics of abundance can work. Even if one accepts that liberals have been complicit in the undermining of government, it doesn’t make it any easier to undo the damage. Trust in politicians and officials is exceptionally low, and with good reason. Klein and Thompson argue that liberal politicians would benefit from taking more risks and being able to show results, and they cite the example of the Pennsylvania’s governor, Josh Shapiro, rushing through the rebuilding of the I-95 bridge in Philadelphia after a disastrous accident. As they acknowledge, however, his scrapping of procurement rules, and the autonomy given to project leads to bypass normal safety procedures, could have gone horribly wrong. If it had, voters would have been unlikely to give Shapiro the benefit of the doubt. In the UK, during Covid, the vaccine programme bypassed normal procurement rules to great acclaim, yet attempts to do the same for protective equipment led to widespread fraud and a big scandal. Much of the worst regulation is created reactively – in response to scandals, to assuage an angry public – and it is not hard to see how taking more risks could backfire in such an adversarial political system.

One of the best chapters in Abundance looks at the way in which scientific funding in the US is granted ever more cautiously to academics who – and projects that – already have a track record, thus failing to support riskier ideas that could lead to paradigm-shifting breakthroughs. The authors also highlight the absurdly bureaucratic grant processes that lead to scientists spending a vast amount of time filling out forms and managing obsessive audits of their spending. The same is true in the UK. But there is a reason that this happens – this is public money, and it is easy to embarrass politicians by highlighting even small amounts of fraud or by “outing” a silly-sounding project in the media. The political environment is what decreases risk appetite.

Ultimately, “abundance” is a highly optimistic political philosophy – even utopian in Ezra Klein and Derek Thompson’s version – and we do not live in an optimistic age. It is hard to get people excited about nuclear fusion and driverless cars when even the most basic services don’t seem to work, and public spaces look ever shabbier. These authors are making an important point about the rigidities of regulation and the cost of procedural sludge. But to make progress, we need a much more coherent and broader framing that tackles the deeper defects of our economic and political systems. Assuming that the problem is primarily about over-regulation mistakes a symptom for the cause.