Next/Return/Previous exist because you've entered that record from a report or report link, whether implicitly or otherwise. So those relationships are based on whatever you were looking at before you clicked into the record.
Certain types of actions, though, don't retain that contextual info.
Examine the url for proof. From my dashboard, I click into a record from a dashboard report, and I have something like this:
https://[mydomain].quickbase.com/db/[dbid]?a=dr&rid=[rid]&rl=qy2the last bit after the rid indicates how I got there and includes enough implicit information for qb to understand what records come 'before' and 'after' the one I'm looking at, in the context of how I got to it.
Without seeing your formula URL, I'm guessing it includes a redirect to a version of that url that doesn't include that additional data.
There are some options - if you think you know how the user is always going to be accessing that record, ie the next/previous records will always be in reference to the same relationships - you could hard code the [rl] characteristic into your formula URL redirect. You might be able to use javascript to get the [rl] from your url and pass that to the formula URL button. That kind of thing is a bit over my head but I'm fairly sure it's possible. This thread might be a step in that direction:
https://community.quickbase.com/quickbase/topics/how-to-get-javascript-to-return-current-url-with-re...
