I have pretty serious asthma, and, yes, the smoking really bothers me. At work also I can barely walk outside, because people are always smoking right at the door and all around the plaza.
And I barely go to shul to learn ever, because in just about every shul there are usually people smoking. I already abandoned one shul completely because of this. Once I had to daven Mincha alone outside, because at the only shul there where I was, someone inside was smoking while others were davening mincha. (And even if he would have gone outside, it would have taken at least an hour with all doors and windows open to get the smoke out.)
I also really, really don’t understand how people can smoke on Yom Tov.
There are some translations of interesting letters from the gedolim here: http://chareidi.shemayisrael.com/chareidiview.htm under “Smoking”. (I assume YWN will allow this external link…)