I don’t think it’s so crazy that Harry never noticed Sirius or Peter. First of all, he didn’t exactly use the map very often. And when he was using the map, Sirius may have been out of the maps boundaries. Peter was probably either with Ron or chilling in the dormitory, which are not exactly places that Harry would be wont to look at, and during the periods when he was missing, he may have also been outside the boundaries.
It is interesting though, because Peter and Sirius both knew about the map and Peter for sure knew that Lupin was in Hogwarts, so they may have had some fear of being seen. This in turn may have led them to be careful to whatever extent possible to not be seen on the map.