“Rambam says in his introduction to Perek Chelek that the aggadot are all metaphoric.”
I have always learned that some are “metaphoric” (not sure if that’s the right word or not) and some are literal. I just looked at the Rambam and it is not clear if he is saying that they are all “metaphoric” or not. If he is saying that they are all metaphoric, he probably means that they all have a deeper meaning (which is probably the main point in any case) but some of them could still have a literal meaning as well. There difinitely are midrashim that are meant literally.
In terms of knowing which are which, I think that I have heard that the mefarshim on the midrashim discuss it.
There is something that is very important to be aware of that I think many people misunderstand:
When we say that the Midrashim are meant “metaphorically”, it doesn’t really mean “metaphorically” the way the terms is usually used. Every word is Emes. Everything really happened the way it is told, it just happened on a deeper metaphysical sense and not in a physical sense. For example, Hashem really does have a “hand” but it’s not a physical “hand” – it’s a spiritual concept and what we refer to as a hand is really a “mashal” and a physical representation of a spiritual concept called hand which is something that Hashem has.
Vashti may or may not have had a physical tail, but she had some spiritual malady which can be called a “tail” and which a physical tail is a physical representation of.
I think this last point should answer your question to some extent, Flatbusher. Remember, we are referring to the Aggadic part of the Gemara and not the Halachic part. The halachic part is obviously meant to be learned literally (although there are deeper, conceptual meanings as well which no one except maybe mekubalim are probably even aware of), and the Aggadic part is the deeper more conceptual part of the Torah She’beal peh (and the part that almost everyone says that girls can learn).