I have a 350 trans that is not allowing part throttle downshifts 3-2 or 2-1. It will do a forced kickdown downshift from 3-2 and from 2-1. The trans has approx 4k miles on it. The fluid is fresh as well as the modulator (red stripe). Could this be a problem with the valve body or the fact that the modulator is hooked up with a rubber hose with no metal pipe between the intake and modulator.

I would say the vacuum hose is the problem. It collapses and prevents the modulator from releasing its vacuum hence no downshift. Go find the metal tube and use as little vacuum hose as possible. I bet that will cure your problem.