I don’t think it’s at all crazy to start school at your age. Your maturity and life experience will probably be a big asset in a field like social work.
However, I think the bigger question is what stage of life are you at. Some questions to think about: You mention you are a mother of 4 — how old is your youngest child? Will it be logistically possible for you to work out child care, etc., not just for the hours that you are actually at school, but also for the time you will need to put into school work? Will this fit in to your other roles at this point, as a mother, wife, etc.? So you currently work, and if so can you manage to give up your salary at this point until you finish school, or do you know of any programs that you can go to that would allow you to keep your job (and is it feasible to be in school, have a job, and run your home)?
All of these types of questions can be dealt with, but it’s important to keep in mind that going back to school at a point in your life when you have more responsibilities then the typical student may require some more preparation on your part. Sometimes it is easier to wait until your children are a little older, even if it means that you will be a bit older as well.
Whatever your decision, Hatzlacha Rabbah!