The statue of the Duke of Sutherland on the summit of Ben Bhraggie has been badly damaged for the second time in the space of just five months
Police investigating the latest incident said this week they believe an ongoing effort is being made to topple it from its foundations.
And a local community councillor has labelled those responsible as "political fanatics."
Six months ago two large sandstone bricks from either side of the massive plinth on which the 100ft statue stands, were wrenched free and left lying on the grass.
The repair work carried out has since been undone and, in the latest incident, stones have been removed from the northwest corner of the plinth.
A Northern Constabulary spokesman said: "It's suspected that the damage is being caused deliberately, bit by bit, in an effort to topple the statue and that some person with a grudge and a knowledge of history is behind it."
The latest attack is thought to have happened at some point between Friday, 28th October and the following Monday.
The controversial monument was erected around 1837 in memory of George Granville Leveson-Gower, the 1st Duke of Sutherland, who is notorious for the role he played in the Highland Clearances, forcing many of his tenants to leave Sutherland's glens to make way for sheep.
In 1995 the late Sandy Lindsay, of Newtonmore, launched a campaign to demolish the statue - known locally as the "Mannie" - and replace it with a Celtic Cross or a plaque dedicated to the Clearances.
He was unsuccessful but the move sparked a huge debate amongst the Scottish diaspora.
Graffiti has been sprayed across the plinth on more than one occasion and it was defaced in May last year with the word "monster" sprayed across it in green paint.
Golspie community councillor Allan Barclay, a retired teacher, said the latest damage had been reported to him by his neighbour, Mary Argyll, who regularly walks her dog to the top of the 1302ft Ben Bhraggie.
He said the massive plinth was made up of one cubic metre blocks of sandstone, but he was unaware if it was hollow or solid all the way through.
"Whoever is doing it, is hell bent on taking it down," he said.
"The damage caused in April was patched up, but the repair didn't last more than a few weeks before the stones were pulled out again.
"Three out of the four corners have now been vandalised. There is a big hole in both the back and front of the east corner and recent incident saw the biggest stone yet taken out of the back west corner.
"I don't know how solid the inside of the plinth is but it could come down - it's as simple as that."
Mr Barclay is convinced that the vandalism could not have been carried out by one person alone and that tools would have been required.
"I can't see how one person could do what has been done on their own. There must be two to three people involved at the very least with someone acting as a look-out," he said.
"They've planned it well. These are not stones that can be lifted out easily. They would have had to be knocked out with a crow bar or power tools.
"There is no way anyone could walk up the Ben with tools of that sort and not be seen. It wouldn't surprise me if they've hidden the equipment somewhere up there. There is a vast hillside in which to secrete stuff away."
Asked if he thought the statue was now unsafe, Mr Barclay replied: "I would say that probably it is not yet unsafe but it cannot be far off it."
He added: "It is a shame. I have always maintained they should never take it down because as long as it is there, you can tell children what happened in the past but if it is taken away it will not be long before the past is forgotten. "I think the people who are doing this are fanatics and I mean by that political fanatics."
Sutherland Estates factor Peter Voy said he had been disappointed to learn of the latest incident.
"I have seen photographs of it and it looks as if they have been chipping away at all three corners," he said.
"It is disappointing as it seems to keep happening and it looks as if the repairs that were carried out have been undone. It is something that we will no doubt have to continue to put right."
Police are asking anyone with any information about the damage to contact them at Brora Police Station on 01408 621222 or Crimestoppers on 0800 555111.
What is your view - is The Mannie a reminder of history and should remain intact? Comment on our website.