If you're using the formula URL method to generate the document, you can set the filename there. Below is the example code for the button:
var number recordId = [Record ID#]; //record ID# you want you are generating the document from
var number templateId = 1; //ID of the template you'd like to generate
var text tableId = Dbid(); //tableID of the table the template was built in
var text fileName = "myqbdoc"; //name of the generated file
var text format = "pdf"; //output formats: html, pdf, docx
var text qbRealm = NotRight(NotLeft(URLRoot(), 8), "/") ; //gets the current realm, required for API requests
"https://api.quickbase.com/v1/docTemplates/" & $templateId &"/generate?&tableId=" &
$tableId & "&realm=" & $qbRealm & "&filename=" & $fileName & "&format=" & $format
&"&recordId=" & ToText($recordId)