If you're just making it back on to your side a few seconds before you pass, i would guess that the person flashing you has had to stab their brakes on as they see you coming towards them, so although you're making it back in how do you know you've not caused mad panic and them to slam the brakes on to avoid smashing into you head on????
You say they're not there as you over take but then appear as you're nearly past - that means pure and simple you're over taking blind!! You need to be waiting until you can see a long stretch (as in a stretch long enough for you to pass and get back in way before a car is coming)!
I hate people that risk others lives over-taking..... I don't think you realise the impact of hitting a car head on - even at 30 it's literally like stopping dead at 60mph so think about that before you kill someone.
__________________
|