The king/emperor and queen both have dark hair. The concubine and the baby are blond. The king/emperor does not visit the baby. The concubine always visits the baby. The queen disagreeing that her son will be the heir. The queen's remark about the concubine's face. Therefore, I conclude that the prince is the concubine's baby.
It’s much darker than that, go look through the spoiler in the comments. The kid is definitely the concubines, we can all c that, but that’s the least of our worries.
The king/emperor and queen both have dark hair. The concubine and the baby are blond.
The king/emperor does not visit the baby. The concubine always visits the baby.
The queen disagreeing that her son will be the heir.
The queen's remark about the concubine's face.
Therefore, I conclude that the prince is the concubine's baby.