The only time I look at who someone is being followed by is when they are a well known bigot such as JKRowling. If I see a mutual supporting them I will assume they either share their beliefs or at least find them acceptable.