Residents raised concern over redevelopment plans for Linlithgow’s Vennel at the community council meeting on Tuesday.
Locals were worried that any development in the area could include the flats – some of which are privately owned.
However, council election candidates moved to allay fears, insisting the flats were not included in the planning brief.
West Lothian Council announced plans at the start of April to completely redevelop the site on the north side of the High Street after the library and public toilets move to the new Partnership Centre at The Cross.
The site includes the library, shops, toilets, parkland, car park and landscaping and green space.
An eight-week public consultation on the redevelopment is due to take place.
At a busy Linlithgow and Linlithgow Bridge community council meeting in the Burgh Halls residents made their views heard.
One man said: “I live in the Vennel and I am a council tenant as well. I’ve had a long-held suspicion that should a developer dangle some money in front of the council then we would be out of a house.
“When I heard about this proposal it started ringing alarm bells.”
Candidates said the planning brief did not affect the residential area, only the site where the council fully owned the properties and encouraged interested parties to put their views forward in the forthcoming consultation.
Conservative candidate Tom Kerr said: “The whole thing is open to comment, put in whatever ideas you have.
“I would recommend that you constrain yourself to the area that is there.
“These guidelines are only for that area which is 100 per cent owned by the council. This is nothing to do with demolishing the flats.”
Labour candidate Tom Conn said: “The council is doing this the right way, we are asking the public what they want it is a once in a generation opportunity.”
Mike Vickers of the community council added: “We should be developing this area in relationship to the total picture of Linlithgow.”