A good chunk of those who perished were not relgious. Proably a majority of jews in Europe were not religious even in 1939.
I was reading a raw account (many accounts of the holocaust are filtered because the survivors have alot of guilt for suriving and their own actions (like stealing) in order to surive) of the warsaw ghetto. The writer says that during the deportations in the summer of 1942 peoples greatest fear was not only dying, but disappearing forever meaning not even a trace of their existence. In fact Yad Vashem who has tried to at least give a name to all the victims only has about 1.5-2 million of them