There is nothing wrong with speaking Yiddish if you speak it well and want to reach only a Yiddish-speaking audience. Likewise, there is nothing wrong with speaking English if you speak it well and want to reach a major portion of the world. There is plenty wrong with speaking any language badly, because the speaker fails to convey his/her ideas clearly.
In my very limited experience, people who try to speak both languages do not speak English well, but I have no idea how clearly they speak Yiddish. Given the importance of English throughout the world in the 57th Century, it is important for the Jewish people to have a good command of English.