Its not a problem if you wash with it, If someone in your house eats Kitniyot (Like a Baby) it doesnt traif anything. You can clean the babies stuff in the sink
It might only be a problem if you soak the oil or something, but dont listen to me, Ask your Rov if its a Chumra or Halacha