I partially agree with your regarding different browsers and devices. I have Lumia 435 for tests and it acts not like any other mobile device on the marker, thus with different browsers (default IE, UC or Opera Mini) it acts even more differently regarding images. I think the current version of the File Upload add-on's pop-ups already does the best it could do - also, I personally like the feature when the opened image could be closed, clicking it - it's very comfortable and it works on large amount of browsers.
I doubt that installing solutions like Lightbox could improve those behaviors.
I'm currently working out the newer version of File Upload add-on, and came to quite simple CSS solution which works mostly in every modern browser in terms of scaling images which do not fit the screen. Whatever you'd need to add to the CSS file, is something like
where 'fPopUp' may be substituted by any body/table/td class which is supposed to have images in it. I mostly checked this for images which were embedded using BB code from external websites, and I think I'm going to include this in default miniBB CSS - also, this could work in some term for the File Upload add-on's code, you may try...