The appalling attitude of anti-smokers is much more obvious and speaks to a much deeper problem. If you don’t like the smell of something that’s okay. Just recognize that you are not the only person in the world. If you didn’t like the smell of hot dogs and someone was eating one near you what would you do? Nothing. Therefore the smell issue is a non-issue.
Being near a smoker outdoors occasionally is not significantly detrimental to your health. Period.
The question is, does your definition of taste, plus your extreme definition of right or wrong give you a right to judge, condemn, vilify, demonize, and control others? I say no.