it depends what section you’re in. libraries have plenty of innocuous (and very educational) books about history, politics, biographies, geography, sports, money management, language learning, etc. Red flags go up when you get to novels, modern poetry, dating advice, etc.
So if you have a good relationship with your kid, you’ll just talk about the books they got (ask for the receipt in case they’re hiding some) and see if anything is objectionable. If it is, explain why and return. Personally, I think the problem with libraries isn’t shmutz or apikorsus, but rather bitul zman. For smart kids who like to read but need more motivation, it’s easy to get lazy and read library books instead of learning Torah. And that’s one reason why yeshiva is so great and important, as Yekke said earlier.