Since the marriage is not an illicit relationship, they are not Mamzerim. The case of Ba’a Memedinas Hayam they are Mamzerim, because even though she is allowed to marry, sof davar, the relationship is illicit (i.e., we only allowed her to marry due to our own ignorance, not due to it actually being muttar).
Think of it like the Torah allowing the woman to marry a second husband. The child would not be a Mamzer in that case.
WADR, I believe you are confusing two different cases.
An agunah is allowed to marry based on the testimony of one witness. However, if she does so, she is required to perform an investigation and if her first husband shows up, she’s out of luck in that she loses both husbands.
If she marries on the basis of two witnesses and with the permission of Bais Din, however, she is afforded the protection that if her husband returns, she does not have to leave him (the first husband). In this case, the marriage is sanctioned by Bais Din and is perfectly permitted. And yet, her kids (with the second husband) end up as mamzeirim anyway.
The Wolf