There is bound to have rumours and gossip in the work. I encountered such circumstances during the full-time employment. I adopted the non-chalencent approach and maintain a low profile on such situation. Life is short and there is no point in been engaged in dispute or conflicts with the people. I have left the full-time employment and such circumstance is unlikely to be prevalent in my situation.
You may disagree with my views.To each of our own.