There are many Jewish couples who would love to adopt, but there are very few Jewish healthy babies available for adoption.
There are also people who are concerned about Yichud. After a certain age a son cannot remain alone with his mother, and a daughter cannot remain alone with her father.
How did your husband feel about this?