It does sound like a duff connector on the charger.
I had similar issues because local cats like to get into my shed and jump on the warm bike and would often knock the connector slightly out of true.
Best thing, though, is to go to a local garage and put the battery on a Load Tester (Drop Test) just to make sure it's ok.