This is more of question for a gay chat room than for a survivor's chat room. However, to answer your question, it depends on your age and his age. You indicate that he is twice your age, and if you are say, for example 18, and he is 48, that is a big age difference. The way you ask the question to the forum is more like an adolescent asking if he can stay out an hour later than his imposed curfew.
If you are comfortable with having a sexual encounter with someone, and your partner is also comfortable, if it is consentual by both parties, and you're both above the legal age, then the decision lies between you and your partner. If, however, you are asking if it is morally acceptable, then only you can answer that question, no one else, since such a question is subjective to and for each individual.
The strongest hearts are sometimes broken,
As the deepest thoughts arent always soken,
and the greatest wounds we hide inside ourselves where they never show