Do you and your wife possibly have an "understanding" where certain things can happen within the context of your relationship? Agreed that you don't tell her every single time that you masturbate. But I agree with Oldfart. Part of the thrill seems to be doing it with this guy. Think about your long term comfort (both yours and your wife's). No judgement here...do what is right for you.

__________________
I always wanted to be somebody, but now I realize I should have been more specific. --Lily Tomlin