Looks like it was the front engine mount / isolator. It was looking a bit perished and when I fiddled with it a bit some of the rubber came away which meant I got vibrations on decel. Changed the part and the bike feels great now, no wobble and feels better in the corners too.

Thanks for all those that read and suggested stuff. On to my next issues now