Create a new rich text formula field. Then use this formula:
//Where it says e50, change the 50 to your attachment field ID
var text url = URLRoot() & "up/" & Dbid() & "/a/r" & [Record ID#] & "/e50/v0";
//Change to your file attachment field
var text img = If(
[File Attachement] <> "",
"<img src='" & $url & "' />\n",
""
);
$img
Make the minor changes per the comments in the formula. Now use the formula in the document template instead of the file attachment field directly. If you need to customize height/width of image, it can be done too by editing the HTML in the formula.