Joseph,
I am not sure what you mean by drawing the line. Yichud d’oraisa is only one man and one woman where there is an isur erva d’oraisa between them. Everything else is d’rabbanan.
So, for example, Yichud with a non-Jewish woman is d’rabbanan. Many Rabbonim if asked a shaila about Yichud with a non-Jewish woman will look the reality of the situation and the danger of greater isurim before deciding whether to be rely on certain kulos. If the case is a 13-year old boy staying home with a 65-year old cleaning lady, they will be more meikel (and for sure if the boy is under 13). If the case is a 19-year old boy with a 25-year old cleaning lady, they will be more machmir.