The attackers burned tents and destroyed other equipment after tying up a guard. They also left a letter threatening the head of the UN Relief and Works Agency (Unrwa), John Ging.
No group has claimed the overnight raid, but observers suspect it is the work of Islamic extremists opposed to such camps for boys and girls.
"The armed men torched the camp, which contained recreational equipment and (plastic) swimming pools, and completely destroyed it," Unrwa spokesman Abu Hasna told journalists.
The camp, due to open next month, is one of dozens of beach facilities set up by the UN offering a summer programme of arts, sport and other activities for some quarter of a milllion children in the Gaza Strip.
Representatives of the Hamas government condemned the attack and said those responsible would be found.
Unrwa chief John Ging said the agency "would not be intimidated by such attacks".
He pledged the summer camp would take place as planned.
Mr Ging said there was "no doubt in my mind that it is vandalism linked to a certain degree of extremism. It is an attack on the happiness of children."