You only repeated what you already described before.Normally the PWR LED is not controlled by any protocol or any firmware.
So check first the power of the module under the different conditions.
General picture from the web don’t show your real connections and work.
Also measure the voltage on the GPS module. All information I have at the moment show a power problem
It doesn’t look like the connector is properly seated at the module in any of those photos. Don’t make another reply until you have measured the Vcc voltage as asked.
As far as wiring it’s this:
Were you always only connected to USB when you were using the 5V pin or did you try with a battery connected? The 4V5 pin will definitely be powered thru USB, the 5V pin probably not. Connect a battery and test it again.