For a lot of people, that will surely count as a difficult situation. However, you didn't allow the fact that he's family and also your boss to get in the way.
Despite being in all that position, everyone deserves respect and no man wants to be treated like a junk. Your timing was perfect and I like how you didn't get into exchanging too personal words with him.