The prime minister delivered the opening speech Monday at a two-day conference on anti-Semitism, taking place on Parliament Hill.
He noted Canada's unsuccessful bid for a UN security council seat this fall, saying he believes the country is morally obligated to stand up against those who would attack Israel.
"And I know, by the way, because I have the bruises to show for it, that whether it is at the United Nations or any other international forum, the easy thing to do is simply to just get along and go along with this anti-Israel rhetoric, to pretend it is just being even-handed, and to excuse oneself with the label of honest broker.
"There are, after all, a lot more votes — a lot more — in being anti-Israel than in taking a stand."
The reference to "honest broker" was a thinly veiled swipe at Liberal Leader Michael Ignatieff.
Ignatieff attacked the Conservative government in a speech this month for sacrificing Canada's ability to command respect on the issue of peace in the Middle East. Ignatieff said Harper has used Israel as a wedge issue in Canadian politics, and called for a return to a time when Canada was seen as an honest broker in the world.
Harper said Israel may be subjected to fair criticism, and noted that Israel subjects itself to such criticism as part of a healthy, democratic debate. But he spoke of a "solemn duty" to defend the vulnerable and challenge the aggressor at home and abroad.
"Not just because it is the right thing to do, but because history shows us, and the ideology of the anti-Israel mob tell us all too well, that those who threaten the existence of the Jewish people are in the longer term a threat to all of us."