How can you let your best friend know that you need space?

My friend, Stacy, is a cockblocker. I mean,she is always around my other friends boyfriend and her. She interferes with their relationship. How can, J, let her know that she needs her to back off sometimes?