I get exactly what you mean, we all need some "me time". It's good that you do at least sometimes hang out with them rather than none at all. You're not a terrible friend, if you were they wouldn't be so eager to hang out with you :D. Have a fun vacation!
