I'd take it to a tailor. One of the best things you get out of a tailor is not the alteration, but an informed second opinion as to what needs doing. My tailors so honest he quite often says "it's your imagination, the fit is fine as it is", rather than charge me for an alteration.
Ask what he thinks, and whether it's worth upsizing. For what it's worth, you should feel a bit of a tug around the shoulders the first time you put it on. It's a good sign the jacket fits.