I'm 5'6" 200#, riding an XT and have the XL, I tried the XXL but it was too big with the liner out. With the XL and the liner its too snug to put a heated liner on too, so I figured a heated liner under the jacket and rain jacket over the top, not using the jackets liner will work the best.
The armor fits good and stays put, the snap adjuster on the forearm is in the 1st(loosest) with the liner, and with just a t-shirt I put in the 3rd(tightest) position.
The only complaint is I think it could be longer in the torso, I'm using it with a set of First Gear perf pants so I need to get the zippers to match and see if that helps, but the problem is in the front not the back where the zipper is.
You should try the L