Miss IRC so much

For web devs here, do we really still need to support browsers of the evil (yeah I'm talking about MS browsers, Edge included) ?

I mean, building a css ui library here in 2017, without the benefits of custom properties, grid and so many other cool things, is so fucking frustrating.
A practical example : color theming with custom properties = Fuck Yeah / color theming without custom properties = so verbose and painfull, sucks.

The library is mostly for private usage at the moment so... I'm about to drop IE and Edge in the deepest shithole of the darkest cavern of my memory, and move on coding my lib with modern CSS, with almost no regret for the ghosts of the past who are still using these shitware today.

Should I ? Or should I... maintain compatibility as we traditionnally do ?
What's you guys opinion about this ? Can we finally kickban these browsers from our lives ?