I can’t see having a meal with Harry Pettit, who seems a bit morally depraved, but his political speech is probably lawful even in a jurisdiction with typical EU hate speech laws (see, e.g.). In any case, barring a legal judgment against him, it seems to me the VUB has clearly violated his rights, including his right to academic freedom understood as including a right to extramural speech. You can sign a petition here in protest of the university’s decision, as I have.