Do even 5mins naps destroy anyone else?

If I'm working late and a have a 5 min nap anytime after like 4pm, it makes me have insomnia that night by more than 5 mins. Happens quite consistently.