The centre was closed for 20 months due to the coronavirus pandemic and only opened its doors again on November 1.
However from mid-October to the start of the year, vandals smashed a number of windows - including three over the Christmas period.
Dave Baker, 65, is one of the volunteers on the committee and he was called by police on Christmas Day and told that one of the windows was smashed.
“It’s getting very frustrating now as they’ve smashed the three main ones in the hall.
“I got a phone call on Christmas Day from the police about the window being smashed - it made a mess of Christmas.
“It’’s getting ridiculous now.”
There is a youth club held on Fridays at the centre and older kids ‘gatecrashed’ the club last year and threatened members.
Some of the youngsters at the group identified those who were being threatening and ever since even more windows have started to be broken.
“There’s a lot of speculation about who might be doing it,” he added.
“There seems to be some kind of vendetta against the centre and it doesn’t look like they’re going to stop until they get caught.
“When the windows do get fixed you just ask ‘when will this be smashed again?’.
“It’s getting really bad but we don’t want to give in - we’ve got to keep fighting to remain open so they don’t win.”