Is there a way to force all photo file attachments to display vertically?

Adjust your formula to produce this HTML for the image:

<img src='...' height='400' width='400' style='transform: rotate(90deg);'>

FWIW, all those "backslash double quote"s (ie \") are just making your formula hard to read. You can just use a single quote instead (ie ').

Or if there are no spaces in the attribute value you don't even need any quotes at all:

<img src=https://www.gravatar.com/avatar/30d923ee56cbae00df6ad2921abac67b?d=identicon&s=100 height=400 width=400 style=transform:rotate(90deg)>

BTW, you are better off rotating the image before importing it to QuickBase as some images will need rotation and other don't. This can be done using script.

Worked like a charm. Thanks for the advice.

Edit: For those curious, to make the button, you would need to create two fields, 

1 for Degree of Rotation (Numeric) and a Formula URL field. Youd put the Degree of Rotation field right before the "deg" innto the formula above. 

Then the button would just add 90 to the Degree of Rotation field (this would be a URL formula field)