Responses
http://www.mangago.zone/read-manga/aka_to_kuro_katsura_komachi/mf/c001/26/
The terms, "I want to hold you" is usually equivalent to "I want to have sex with you" and said from the guy or seme's point of view. Since this story has been from Yahagi's POV we assume he's talking. And plus, it just makes sense. Yamato is oozing uke vibes.
And in http://www.mangago.zone/read-manga/aka_to_kuro_katsura_komachi/mf/c002/34/ Yamato is basically inviting him.

Wait... How.do u guys know yamato is uke? Was there a implication or scene I missed???