There is a reason why Punk Rock and Rap music are so full of violent lyrics likewise R&B is mostly lewd and immoral. Different styles of music bring out different emotions.
When moishe came down from har Hamoria the first time he said ??? ???? ????? ??? the rishonim explain that he heard the sound of their singing and dancing, the music of avoda zora,the music of hefkeirus.
Such kind of music comes from the base human desires and carries a negative influence even without the lyrics.
Jewish music can only be influenced by uplifting music which comes from the heichal hangeina.
There is also an argument that the composer’s and feelings and emotions are reflected in their compositions so music composed by an immoral person will inadvertently have a negative effect on the listener.