A total of 21 bodies were found near the Guangfu River, Jining City officials said in a statement Tuesday.
"The investigation has found that the Jining Hospital Affiliated Medical College mortuary staff violated hospital management policies/regulations," the statement said.
No foul play was found in the deaths of the children. "It was determined that the babies died in normal circumstances," Li Wanmei, section chief for the Jining City Health Bureau, told the television network.
The hospital's deputy director and the deputy director of its logistics department were removed, the statement said. In addition, two staffers working in the "repose room" of the hospital were fired "and have been detained according to the law."
The dead children and fetuses were believed to have died from illness, and the two staffers, Zhu Zhenyu and Wang Zhijun, verbally agreed with relatives to dispose of the bodies and took money from them, but instead secretly transported the bodies to the river, the statement said.
The bodies, which officials said "were not fully buried," were found after media reports prompted officials to investigate, the statement said.
Shanghai, China-based Oriental Cable TV reported the initial discovery was made by area residents. "I found the bodies while I was herding the geese," a resident said. "How could I possibly sleep? I haven't eaten for two days."
Eight of the bodies had tabs with clinic code numbers attached to their feet, China's state-run Xinhua news agency reported. The tabs showed the bodies came from the Jining hospital.
The incident highlighted "severe flaws" in the hospital's management, city officials said in the statement.
"The Jining City Health Bureau recognizes its inadequate supervision and bears responsibility for this incident," the statement said, adding the bureau "issues their sincere apology to the community."
Environmental officials said the most recent tests on the river's water showed it was not contaminated, Xinhua reported. The Guangfu is not a source of the city's drinking water, according to the news agency.
Jining is in Shandong province, about 300 miles south of Beijing.