If Conservative MPs hadn’t sacked two prime ministers in the past year, some of them would surely be plotting against Rishi Sunak by now. He has plenty of enemies and his friends can’t say things are going well.

But even the surliest malcontents recognise that a challenge at this point would look absurd. Where a stronger leader might call on personal loyalty, Sunak has to make do with embarrassment. Sticking with the current leader is the least ridiculous option.

That doesn’t prevent regular sabotage of Sunak’s authority. This week’s unhelpful intervention was a demand for drastic cuts in immigration and a cap on refugee numbers from a group calling themselves “New Conservatives”. The name refers to the recency of their arrival in parliament, as members of the 2019 intake, not the originality of their big idea.

It is a bit late for a change in strategy. Sunak has set the five pledges against which his performance will be judged. He can’t junk them now, even though he is failing on most counts. Public argument for a new direction is mostly rehearsal for the blame game that will follow defeat.

The more revealing measure of the Tory mood is the number of MPs standing down. About 40 have so far said they won’t run again. More are expected to follow suit. Pre-emptive exodus from the ruling party is usually a sign that a political tide has turned.

That feeling of being swept along by elemental forces instils nervous buoyancy in Labour MPs. They are glad to be heading for victory but don’t feel in control of the motion. Keir Starmer is not the most dynamic helmsman but he earns credit for getting the party seaworthy enough to go with the flow, and for navigating patches of turbulence without capsizing.

'Focus on the future': Rishi Sunak sidesteps question on Boris Johnson – video

Starmer lacks clarity about the destination but it is somewhere on the shores of government, and a double-digit lead in opinion polls is a compelling enough argument against mutiny.

Politicians do not have to inspire mass movements or have doctrines named after them to be effective. But it is remarkable, given the scale of recent volatility, that neither candidate for prime minister at the next election will bring ideological definition or any depth of grassroots enthusiasm to the contest. There will be no Sunakites and no Starmerites, except in the shallowest sense of candidates who rely on their leaders’ patronage networks.

That void is palpable on the ground. I visited Uxbridge with a colleague last week and we struggled to find much trace of engagement with the coming byelection to choose a replacement for Boris Johnson. There is a residue of fondness for the outgoing MP as a roguish local mascot, but that doesn’t extend to support for his would-be Tory successor.

Labour benefits from discontent with the abject state of everything – soaring prices, crippling mortgage payments, broken public services – but only by default. No one is able to say what the opposition proposes to do differently. Sunak is viewed as well-meaning but hapless, at best. Eyes glaze over at the mention of Starmer’s name.

Opposition MPs tell the same story from every contested constituency. There is a common determination to be rid of the Tories, for which voting Labour is usually the handiest tool.

The defence of Starmer is that getting this far is a remarkable feat within four years of a catastrophic defeat. If the only thing disgruntled Tories know about Labour under its current leader is that it is no longer the party of Jeremy Corbyn, some of the steepest terrain on the road to power has been covered.

Here is another thing Starmer and Sunak have in common and another reason why the competition between them is anaemic. Both leaders are more clearly defined by distance sought from predecessors on their own side than by rivalry with each other. Both are compromised in that endeavour by records of loyal service to the men they repudiate, although Starmer’s disavowal of Corbynism has been more ruthless than Sunak’s cavilling repudiations of Johnson.

There is a related awkwardness tinged with denial in both men’s relationships with Brexit. Sunak doesn’t champion the deal that Johnson struck because it is making Britain poorer by the day. Starmer doesn’t challenge Sunak on that point for fear of alienating those voters, disproportionately clustered in Labour target constituencies, with no regrets about voting to leave the EU, or none that they want to hear repeated by a man who once campaigned to have the referendum result overturned.

It is reasonable for party leaders to steer away from arguments that would mean reopening and salting wounds to national pride before any practical remedy could be made to work. Even those who think that debate is necessary and inevitable would want it conducted with nuance, in a spirit of rational compromise, which is to say nowhere near an election campaign.

Starmer and Sunak act as if Britain has achieved post-Brexit politics, which is almost true. It is true in the technical sense that Britain has left the EU, much as it is technically true that Labour is a post-Corbyn party, and Sunak is a post-Johnson prime minister. The peak of conflict and polarisation associated with all of those things has passed. The appetite to move on is real. But the legacies are not settled. Current leaders reveal as much about themselves with awkward silence about the past as with platitudes and pledges for the future.

Their situations are not exactly alike but there is a symmetry. The parameters of the next election are being drawn as part correction, part erasure of the last one. There is tacit acceptance that Brexit was not really done, and that neither candidate to be prime minister was fit for the job. But no acknowledgment and no apology.

This queasy, furtive realignment creates a diffidence at the top that feeds into public despondency. It is an awkward excuse mumbled in the background of British politics. It is the failure of Labour and Tory leaders to look the electorate in the eye.

Rafael Behr is a Guardian columnist