!rant - seeking advice

So I found a new job and will start at the beginning of July.
I will have holidays (approved) 3 weeks in June.
My resignation can be handed in after midst of May (1 month notice period).
The main reason I'm leaving is my boss/the company structure/the way we are forced to work. Therefore I fear having a bad time when telling my boss early that I will resign.
But I also want to leave the company with a good feeling for everybody, especially my colleagues who already know I leave.

So, the question which is torturing me right now: should I tell my boss in the next days already that I will leave or should I tell him the day I resign.
The latter would mean that I work 2 weeks after resigning, then take my holidays I have approved and actually leave the company by taking the holidays because after those June is over.

I fear that he might give me a hard time when I tell him now. On the other hand, when I tell him so close to my holidays, he might be angry (I am sure he will be angry anyway) and try to cancel my holidays...

For me it's really a tricky situation, because I think my boss has already a problem with me (although he says no when I asked).