Create the report formula and once it is created it will be at the bottom of your field list in the regular report filters and you can then use it there. You just need to filter it based on your formula checkbox being equal to 1.
You want to go with a formula checkbox and filter your report based on this being checked to show those records. I am fairly sure this formula would work for you but let me know how it goes. It will show them once the duration of hours between the submission or the date modified is over 3. Hopefully I understood your request but let me know if I did not.
if(
toHours(Now() - [Date Created]) > 3 or toHours(Now() - [Date Modified]) > 3, True, False
)