Yeah but he love the empire through and through. He was raised by the king and probably understood that if a sudden change happend it will have negative impact to the king and the nation. It's either the noble's clash for the next queen or the church's backlash.
A man that yearns, is a man that earns