The Prime Minister had signed up to a plan to take on the Labour leader on Sunday night.
But Mr Corbyn was pushing for a rival ITV proposal, and said he would only take part in the corporation’s version if the format was changed to suit him.
Mrs May has so far been declining to take part in the ITV version.
A BBC statement this evening said: ‘We are disappointed that we could not reach an agreement on the BBC’s proposals for a debate on Brexit.’
It added: ‘We believe ours was a fair and appropriate format for those taking part, and crucially for our audiences around the country, and it is a shame we will not be able to bring them this programme.’
Prime Minister Theresa May (pictured at No10 today) had signed up to a plan to take on the Labour leader on Sunday night
The debate on December 9 could have gone into the 8pm slot on BBC1, or the 7pm slot on ITV
The BBC had been under pressure to broaden out its debate, with leading Tory Eurosceptics warning a straight head to head would ‘breach the concept of impartiality’.
Boris Johnson, Dominic Raab and other former Cabinet ministers wrote to BBC chairman Sir David Clementi to complain the views of the 17.4 million people who voted for Brexit would be ‘nowhere represented’ in the discussion.
They noted that both Mrs May and Mr Corbyn voted Remain in the referendum, adding: ‘They are both wedded to slightly different models of staying in the customs union.’
There had been speculation the BBC would try to press ahead with a programme even if Mr Corbyn declined to join in – but it appears to have decided that would not be sustainable.
Earlier, Labour made clear Mr Corbyn was still insisting on any TV debate being a straight head-to-head with Mrs May.
A party spokesman said: ‘When Number 10 told the media she wanted a head-to-head debate on her botched Brexit deal, Jeremy Corbyn immediately agreed. Jeremy Corbyn then swiftly accepted ITV’s proposal for a straightforward head-to-head debate with Theresa May. But the Prime Minister has rejected it.
‘Since then, the Prime Minister’s team and their preferred broadcaster, the BBC, have put together a confused format which would limit head-to-head debating time, with a built-in advantage for the Government.’
But the BBC statement said: ‘We have been clear throughout the whole of this process that, as well as a substantive head-to-head debate, any programme we broadcast would need to include other voices, including other political parties, to reflect the wide range of views the public and parliamentarians hold about Brexit.
‘The final proposal we put to both of the main parties was for a head-to-head debate between the Prime Minister and the leader of the Opposition, followed by a discussion between eight panellists, including politicians, with a wide range of views on Brexit, and ending with further head-to-head debate and closing statements.
Mr Corbyn was pushing for a rival ITV proposal, and said he would only take part in the corporation’s version if the format was changed to suit him
‘We believe ours was a fair and appropriate format for those taking part and, crucially, for our audiences around the country, and it is a shame we will not be able to bring them this programme.
‘However, we will keep our audiences informed with extensive news coverage and analysis around the vote, and with other programmes including a Brexitcast ‘takeover’ of The One Show tomorrow and a special half-hour programme on Monday December 10.’
Channel 4 is planning to steal a march by staging its own debate with other high-profile Brexiteers and Remainers.
Mr Johnson last week condemned the idea of a head-to-head showdown between Mr Corbyn and Mrs May, saying they were both Remainers.
However, friends of the former foreign secretary have made clear he is not interested in the ‘second string’ clash – and the Remain camp would be unwilling to field Mr Blair.