If you allow them to go to the library, you are essentially being mechanech them that the library is a kosher place to be, and if not, that it’s not a kosher place to be.
If you take your child to Jewel, are you being mechanech them that everything there is Kosher as well? If you take them to the zoo, are you being mechanech them the concession stands are kosher?
Once again, the key is education. Restrictions can also work in limited scenarios, after explanation. I don’t see the public library (even with Harry Potter books, of all horrors) as the place to draw the line, but admit it is subjective to the parent and the child.