You’re out of town.
I didn’t give you permission to leave. I mean, of course I would have given you permission, but it would have been nice if you had asked. I expect you to ask.
As I understand it, I have a right to your “free time” outside of work. That’s what you offered and I accepted. But now it seems like your “free time” is “free time” as long as you define it as such? So I have the pleasure of having you if and when you want me to have you?