Stock speaker cups Use them or not

[n2sandcars]

I'm currently using the factory speaker cups. Has anyone had problems with the sound of new speakers in these cups?
The driver side sounds much better than the others an does not have a speaker cup. All the others sound muffled and kinda sound blown. Any advise.

Running a Rockford Fosgate 400.4 and using Diamond Audio Hex nets 6.0 a with crossovers.