I have a 90 5.7 with the factory exhaust. I can confirm that with enough fiddling, the starter will come out without removing the exhaust or taking the solenoid off. It's a royal PITA, but it can be done by bringing it out snout first. Be careful when it's ready to come out, it's coming out with a vengeance. Don't crush your head.

Ahh.....no more embarassing click...click...click from the bad solenoid the factory installed for me.