Afghan officials said the Taliban recaptured the district centre of Now Zad over two days last week.
Provincial police chief spokesman Obaidullah Obaid said: “Our security forces are still on the outskirts of the district and fighting with the Taliban.”
Now Zad in Helmand province was the scene of heavy fighting between British troops and the Taliban from 2006 to 2009 before it was handed over to the US Marines.
At one point in 2006 a small unit of Fusiliers was cut off in the town.
Three British troops were killed in Now Zad including Marine Richard Watson, 23, from Caterham in Surrey, killed in December 2006 and Private Chris Gray, 19, from Leicestershire, of the 1st Battalion The Royal Anglian regiment, who was shot by a sniper.
Britain lost 454 personnel in Afghanistan between October 2001 and the withdrawal in December last year.
Coalition forces handed security over to Afghan troops and police whom they trained over several years.
Our withdrawal was inevitably going to lead to the reoccupation of many areas by the Taliban. We are going to see this more and more in the future
Small numbers of British troops are still there, mainly in a training role.
Colonel Richard Kemp, who once commanded British forces in Afghanistan, said: “Many British soldiers lost their lives fighting to keep the Taliban out of Now Zad and other places like it.
“But our withdrawal was inevitably going to lead to the reoccupation of many areas by the Taliban. We are going to see this more and more in the future.
“Since we withdrew we have seen a descent into further violence and we will see more fighting.
“But this does not undermine what we did in Afghanistan. We succeeded in denying Afghanistan as a safe haven for Al Qaeda.
“We still have military capability in Afghanistan, predominantly American.
"It cannot hold swathes of land but it can carry out strikes against terrorist groups that pose a threat to Kabul or the West.”
The Taliban confirmed the capture of Now Zad, saying it had also seized weapons and ammunition.
Its official website said check posts were overrun on the first day and it killed 25 security personnel.
Intelligence sources in Kabul say the Taliban are "highly active" across Helmand and have also taken control of Kajaki and parts of central Helmand including Sangin where British forces suffered heavy losses.
They included Para Cpl Bryan Budd, 29, from Ripon, who won a Victoria Cross.