99 XJ Infinity Speaker Question

[96XJCLASSIC]

I just bought a 99 XJ sport with the infinity system only to discover the front door speakers didn't work. The passenger side will work occasionally but the drivers side won't work at all.
I removed the drivers side door panel and found a el-cheapo speaker instead of the stock "infinity" speaker. This speaker was blown, so I replaced it with an phoenix gold speaker. There was a + and - wiring in the door to hook up the speaker and also a jumper wire with a ring connector from the factory negative wiring that was grounded behind one of the speaker mounting screws. I hooked everything up and everything worked great. Turned the jeep off and back on and the front stopped working again. I removed the ground and reinstalled it and it works again.
However, everytime I shut off the jeep and turn it back on I need to unhook the ground and rehook it up to get the speakers to work again.
What gives? Suggestions?

 

[UNCC_99XJ]

Which front door speaker is it? It's fairly common for the drivers side to work when it wants to. Mine will occasionally stop working on the drivers side, and it's because there's something wrong with the wiring that goes between the body of the jeep and the door. I can usually move that rubber piece around a little bit and make it work again.

 

[96XJCLASSIC]

Which front door speaker is it? It's fairly common for the drivers side to work when it wants to. Mine will occasionally stop working on the drivers side, and it's because there's something wrong with the wiring that goes between the body of the jeep and the door. I can usually move that rubber piece around a little bit and make it work again.

The problem speaker is in the drivers door, I honestly don't think the problem is a cracked wire in between the door and body. It seems to be something with the "extra" speaker ground...