I don’t do rashi.
Because your brother had kids then–your wife. But maybe your wife is dead and now the brother has no kids left?
Ok, pretty good answer. I suppose you can say it was obvious to tosfos that yom kippur is not domeh to achos isha, but nidda was less obvious.