I quoted the shu”a in OC, 307 16 which forbids secular novels that do not contain any inappropriate content, because of al tifnu snd moshav leitzim. He speaks out in the same seif that if the content has divrei chesek, what we would call romance, then it is even more assur. He also says that novels or literature that has value for yiras shomayim are permitted, quoting Josephus as an example. Rav Hirsch purportedly ecommended reading the german writer schiller, saying that he has Jewish values.
Why the yeshivos emphasize bitul torah in these discussions is mind boggling